GitLab

           locks on the table

Fixing the partitioning specifics after TRUNCATE TABLE in
bug-42643 was fixed.

Reorganize of code to decrease the size of the giant switch
in mysql_execute_command, and to prepare for future parser
reengineering. Moved code into Sql_statement objects.

Updated patch according to davi's review comments.

libmysqld/CMakeLists.txt:
  Added new files.
libmysqld/Makefile.am:
  Added new files.
mysql-test/r/not_partition.result:
  now returning error on partitioning commands
  if partitioning is not enabled.
mysql-test/r/partition_disabled.result:
  There is no partition handlerton, so it cannot
  find the specified engine in the .frm file.
mysql-test/r/partition_truncate.result:
  Updated test results.
mysql-test/suite/parts/inc/partition_mgm.inc:
  Added check that TRUNCATE PARTITION does not delete on failure.
mysql-test/suite/parts/r/partition_debug_sync_innodb.result:
  updated results.
mysql-test/suite/parts/r/partition_mgm_lc0_archive.result:
  updated results.
mysql-test/suite/parts/r/partition_mgm_lc1_archive.result:
  updated results.
mysql-test/suite/parts/r/partition_mgm_lc2_archive.result:
  updated results.
mysql-test/suite/parts/t/partition_debug_sync_innodb.test:
  Test case for this bug.
mysql-test/t/not_partition.test:
  Added check for TRUNCATE PARTITION without partitioning.
mysql-test/t/partition_truncate.test:
  Added test of TRUNCATE PARTITION on non partitioned table.
sql/CMakeLists.txt:
  Added new files.
sql/Makefile.am:
  Added new files.
sql/datadict.cc:
  Moved out the storage engine check into an own
  function, including assert for lock.
sql/datadict.h:
  added dd_frm_storage_engine.
sql/sql_alter_table.cc:
  moved the code for SQLCOM_ALTER_TABLE in mysql_execute_command
  into its own file, and using the Sql_statement object to
  prepare for future parser reengineering.
sql/sql_alter_table.h:
  Created Sql_statement object for ALTER TABLE.
sql/sql_lex.cc:
  resetting m_stmt.
sql/sql_lex.h:
  Temporary hack for forward declaration of enum_alter_table_change_level.
sql/sql_parse.cc:
  Moved out ALTER/ANALYZE/CHECK/OPTIMIZE/REPAIR TABLE
  from the giant switch into their own Sql_statement
  objects.
sql/sql_parse.h:
  Exporting check_merge_table_access.
sql/sql_partition_admin.cc:
  created Sql_statement for
  ALTER TABLE t ANALYZE/CHECK/OPTIMIZE/REPAIR/TRUNCATE
  PARTITION. To be able to reuse the TABLE equivalents.
sql/sql_partition_admin.h:
  Added Sql_statement of partition admin statements.
sql/sql_table.cc:
  Moved table maintenance code into sql_table_maintenance.cc
sql/sql_table.h:
  Moved table maintenance code into sql_table_maintenance.h
  exporting functions used by sql_table_maintenance.
sql/sql_table_maintenance.cc:
  Moved table maintenance code from sql_table.cc
sql/sql_table_maintenance.h:
  Sql_statement objects for ANALYZE/CHECK/OPTIMIZE/REPAIR TABLE.
  Also declaring the keycache functions.
sql/sql_truncate.cc:
  Moved code from SQLCOM_TRUNCATE in mysql_execute_command into
  Truncate_statement::execute.
  Added check for partitioned table on TRUNCATE PARTITION.
  Moved locking fix for partitioned table into
  Alter_table_truncate_partition::execute.
sql/sql_truncate.h:
  Truncate_statement declaration (sub class of Sql_statement).
sql/sql_yacc.yy:
  Using the new Sql_statment objects.

pushdown.
      
NDB supports only a limited set of item nodes for use in engine condition
pushdown. Because of this adding cache for const expression effectively
disabled this optimization.
      
The ndb_serialize_cond function is extended to support Item_cache and treat
it as a constant values.
A helper function called ndb_serialize_const is added. It is used to create
Ndb_cond value node from given const item.


mysql-test/suite/ndb/t/disabled.def:
  Bug#49746: Const expression caching led to NDB not using engine condition
  pushdown.
  Enabled ndb_condition_pushdown test after fixing appropriate bug.
sql/ha_ndbcluster_cond.cc:
  Bug#49746: Const expression caching led to NDB not using engine condition
  pushdown.
  The ndb_serialize_cond function is extended to support Item_cache and treat
  it as a constant values.
  A helper function called ndb_serialize_const is added. It is used to create
  Ndb_cond value node from given const item.
sql/item.cc:
  Bug#49746: Const expression caching led to NDB not using engine condition
  pushdown.
  The Item::cache_const_expr_analyzer function is adjusted to not create
  cache for Item_int_with_ref objects.
sql/item.h:
  Bug#49746: Const expression caching led to NDB not using engine condition
  pushdown.
  The result_type() method is added to Item_cache class.
  The Item_cache_str now initializes its collation.

           corruption on ADD PARTITION and LOCK TABLE
Bug#53770: Server crash at handler.cc:2076 on
           LOAD DATA after timed out COALESCE PARTITION

5.5 fix for:
Bug#51042: REORGANIZE PARTITION can leave table in an
           inconsistent state in case of crash
Needs to be back-ported to 5.1

5.5 fix for:
Bug#50418: DROP PARTITION does not interact with
           transactions

Main problem was non-persistent operations done
before meta-data lock was taken (53770+53676).
And 53676 needed to keep the table/partitions opened and locked
while copying the data to the new partitions.

Also added thorough tests to spot some additional bugs
in the ddl_log code, which could result in bad state
between the .frm and partitions.

Collapsed patch, includes all fixes required from the reviewers.

mysql-test/r/partition_innodb.result:
  updated result with new test
mysql-test/suite/parts/inc/partition_crash.inc:
  crash test include file
mysql-test/suite/parts/inc/partition_crash_add.inc:
  test all states in fast_alter_partition_table
  ADD PARTITION branch
mysql-test/suite/parts/inc/partition_crash_change.inc:
  test all states in fast_alter_partition_table
  CHANGE PARTITION branch
mysql-test/suite/parts/inc/partition_crash_drop.inc:
  test all states in fast_alter_partition_table
  DROP PARTITION branch
mysql-test/suite/parts/inc/partition_fail.inc:
  recovery test including injecting errors
mysql-test/suite/parts/inc/partition_fail_add.inc:
  test all states in fast_alter_partition_table
  ADD PARTITION branch
mysql-test/suite/parts/inc/partition_fail_change.inc:
  test all states in fast_alter_partition_table
  CHANGE PARTITION branch
mysql-test/suite/parts/inc/partition_fail_drop.inc:
  test all states in fast_alter_partition_table
  DROP PARTITION branch
mysql-test/suite/parts/inc/partition_mgm_crash.inc:
  include file that runs all crash and failure injection tests.
mysql-test/suite/parts/r/partition_debug_innodb.result:
  new test result file
mysql-test/suite/parts/r/partition_debug_myisam.result:
  new test result file
mysql-test/suite/parts/r/partition_special_innodb.result:
  updated result
mysql-test/suite/parts/r/partition_special_myisam.result:
  updated result
mysql-test/suite/parts/t/partition_debug_innodb-master.opt:
  opt file for using with crashing tests of partitioned innodb
mysql-test/suite/parts/t/partition_debug_innodb.test:
  partitioned innodb test that require debug builds
mysql-test/suite/parts/t/partition_debug_myisam-master.opt:
  opt file for using with crashing tests of partitioned myisam
mysql-test/suite/parts/t/partition_debug_myisam.test:
  partitioned myisam test that require debug builds
mysql-test/suite/parts/t/partition_special_innodb-master.opt:
  added innodb-file-per-table to easier verify partition status on disk
mysql-test/suite/parts/t/partition_special_innodb.test:
  added test case
mysql-test/suite/parts/t/partition_special_myisam.test:
  added test case
mysql-test/t/partition_innodb.test:
  added test case
sql/sql_base.cc:
  Moved alter_close_tables to sql_partition.cc
sql/sql_base.h:
  removed some non existing and duplicated functions.
sql/sql_partition.cc:
  fast_alter_partition_table:
  Spletted abort_and_upgrad_lock_and_close_table
  to its parts (wait_while_table_is_used and
  alter_close_tables) and always have
  wait_while_table_is_used before any persistent
  operations (including logs, which will be executed
  on failure) and alter_close_tables after
  create/read/write operations and before
  drop operations.
  
  moved alter_close_tables here from sql_base.cc
  
  Added error injections for better test coverage.
  
  write_log_final_change_partition:
  fixed a log_entry linking bug (delete_frm was not
  linked to change/drop partition)
  and drop partition must be executed before
  change partition (change partition can rename a
  partition to an old name, like REORG p1 INTO (p1,p2).
  
  write_log_add_change_partition:
  need to use drop_frm first, and relinking that entry
  and reusing its execute entry.
sql/sql_table.cc:
  added initialization of next_active_log_entry.
sql/table.h:
  removed a duplicate declaration.

Fixing copyright text.

                        
After BUG#36649, warnings for sub-statements are cleared when a 
new sub-statement is started. This is problematic since it suppresses
warnings for unsafe statements in some cases. It is important that we
always give a warning to the client, because the user needs to know
when there is a risk that the slave goes out of sync.
                        
We fixed the problem by generating warning messages for unsafe statements
while returning from a stored procedure, function, trigger or while
executing a top level statement.
                        
We also started checking unsafeness when both performance and log tables are
used. This is necessary after the performance schema which does a distinction
between performance and log tables.

mysql-test/extra/rpl_tests/create_recursive_construct.inc:
  Changed the order of the calls in the procedure because the code
  that checks if a warning message is printed out expects that the
  first statement gives an warning what is not the case for INSERT
  INTO ta$CRC_ARG_level VALUES (47);
mysql-test/suite/binlog/r/binlog_stm_unsafe_warning.result:
  Updated the result file.
mysql-test/suite/binlog/r/binlog_unsafe.result:
  There are several changes here:
          
  (1) - Changed the CREATE PROCEDURE $CRC.
                                  
  (2) - The procedure $CRC was failing and the content of the binlog
        was being printed out, after fix (1) the failure disappeared.
                                  
  (3) - The warning message for unsafeness due to auto-increment collumns was
        changed.
                                  
  (4) - The warning message for unsafeness due to VERSION(), RAND() was changed.
mysql-test/suite/binlog/t/binlog_stm_unsafe_warning.test:
  Tested filters.
mysql-test/suite/binlog/t/binlog_unsafe.test:
  Reenabled the test case binlog_unsafe.
mysql-test/suite/binlog/t/disabled.def:
  Reenabled the test case binlog_unsafe.
mysql-test/suite/rpl/r/rpl_begin_commit_rollback.result:
  Updated the result file.
mysql-test/suite/rpl/r/rpl_non_direct_stm_mixing_engines.result:
  Updated the result file.
mysql-test/suite/rpl/r/rpl_stm_auto_increment_bug33029.result:
  Updated the result file.
sql/sql_class.cc:
  Moved the stmt_accessed_table_flag variable and related information to the
  LEX as we need the variable reset after each statement even inside a stored
  procedure, what did not happen if the information was in the THD.
          
  Changed the routine in the THD::binlog_query that prints the warning
  messages to avoid trying to print them when inside a stored procedure,
  function or trigger.
                          
  Checked for unsafeness when both performance and log tables where used.
  After the introduction of the performance schema, we need to check both.

Fix by adding "DEPENDS gen_lex_hash" to ADD_CUSTOM_COMMAND(),
so it regenerates lex_hash.h whenever gen_lex_lash changes.

Also, make sql dependent on GenServerSource to avoid concurrent 
generation of lex_hash.h (once for sql , and in parallel for embedded)

- Simplify sql-bench installation, 
do not try to resolve names : this leads to probem
if builddir is symlink located on different filesystem.
(reported by alik)

- Make WITHOUT_XXX (disabling plugin) work for DEFAULT
plugins. Prior to the patch is behaved just like 
MANDATORY.

- LINK_LIBRARIES in MYSQL_ADD_PLUGIN had no effect for
statically linked plugins.

- Fix constant rebuild of initdb target on Windows.

A typo in the Item_cache_datetime::val_str caused an assertion to fail on the
maximum time value.


mysql-test/r/func_group.result:
  A test case for the bug#55648.
mysql-test/t/func_group.test:
  A test case for the bug#55648.
sql/item.cc:
  Bug#55648: Server crash on MIX/MAX on maximum time value
  Corrected assertion.

******
This patch fixes the following bugs:
  - Bug#5889: Exit handler for a warning doesn't hide the warning in
    trigger
  - Bug#9857: Stored procedures: handler for sqlwarning ignored
  - Bug#23032: Handlers declared in a SP do not handle warnings generated
    in sub-SP
  - Bug#36185: Incorrect precedence for warning and exception handlers

The problem was in the way warnings/errors during stored routine execution
were handled. Prior to this patch the logic was as follows:

  - when a warning/an error happens: if we're executing a stored routine,
    and there is a handler for that warning/error, remember the handler,
    ignore the warning/error and continue execution.

  - after a stored routine instruction is executed: check for a remembered
    handler and activate one (if any).

This logic caused several problems:

  - if one instruction generates several warnings (errors) it's impossible
    to choose the right handler -- a handler for the first generated
    condition was chosen and remembered for activation.

  - mess with handling conditions in scopes different from the current one.

  - not putting generated warnings/errors into Warning Info (Diagnostic
    Area) is against The Standard.

The patch changes the logic as follows:

  - Diagnostic Area is cleared on the beginning of each statement that
    either is able to generate warnings, or is able to work with tables.

  - at the end of a stored routine instruction, Diagnostic Area is left
    intact.

  - Diagnostic Area is checked after each stored routine instruction. If
    an instruction generates several condition, it's now possible to take a
    look at all of them and determine an appropriate handler.

mysql-test/r/signal.result:
  Update result file:
    1. handled conditions are not cleared any more;
    2. reflect changes in signal.test
mysql-test/r/signal_demo3.result:
  Update result file: handled conditions are not cleared any more.
  Due to playing with max_error_count, resulting warning lists
  have changed.
mysql-test/r/sp-big.result:
  Update result file: handled conditions are not cleared any more.
mysql-test/r/sp-bugs.result:
  Update result file: handled conditions are not cleared any more.
mysql-test/r/sp-code.result:
  Update result file:
    1. handled conditions are not cleared any more.
    2. add result for a new test case in sp-code.test.
mysql-test/r/sp-error.result:
  Update result file:
    1. handled conditions are not cleared any more.
    2. add result for a new test case in sp-error.test.
mysql-test/r/sp.result:
  Update result file: handled conditions are not cleared any more.
mysql-test/r/sp_trans.result:
  Update result file: handled conditions are not cleared any more.
mysql-test/r/strict.result:
  Update result file: handled conditions are not cleared any more.
mysql-test/r/view.result:
  Update result file: handled conditions are not cleared any more.
mysql-test/suite/funcs_1/r/innodb_storedproc_02.result:
  Update result file: handled conditions are not cleared any more.
mysql-test/suite/funcs_1/r/memory_storedproc_02.result:
  Update result file: handled conditions are not cleared any more.
mysql-test/suite/funcs_1/r/myisam_storedproc_02.result:
  Update result file: handled conditions are not cleared any more.
mysql-test/suite/funcs_1/r/storedproc.result:
  Update result file: handled conditions are not cleared any more.
mysql-test/suite/rpl/r/rpl_row_sp005.result:
  Update result file: handled conditions are not cleared any more.
mysql-test/suite/rpl/r/rpl_row_sp006_InnoDB.result:
  Update result file: handled conditions are not cleared any more.
mysql-test/suite/rpl/r/rpl_row_trig003.result:
  Update result file: handled conditions are not cleared any more.
mysql-test/t/signal.test:
  Make a test case more readable in the result file.
mysql-test/t/sp-code.test:
  Add a test case for Bug#23032 checking that
  No Data takes precedence on Warning.
mysql-test/t/sp-error.test:
  Adding test cases for:
    - Bug#23032
    - Bug#36185
    - Bug#5889
    - Bug#9857
mysql-test/t/sp.test:
  Fixing test case to reflect behavioral changes made by the patch.
sql/sp_head.cc:
  Reset the per-statement warning count before executing
  a stored procedure instruction.
  
  Move to a separate function code which checks the
  completion status of the executed statement and searches
  for a handler.
  
  Remove redundant code now that search for a handler is
  done after execution, errors are always pushed.
sql/sp_pcontext.h:
  Remove unused code.
sql/sp_rcontext.cc:
  - Polish sp_rcontext::find_handler(): use sp_rcontext::m_hfound instead
    of an extra local variable;
  
  - Remove sp_rcontext::handle_condition();
  
  - Introduce sp_rcontext::activate_handler(), which prepares
    previously found handler for execution.
  
  - Move sp_rcontext::enter_handler() code into activate_handler(),
    because enter_handler() is used only from there;
  
  - Cleanups;
  
  - Introduce DBUG_EXECUTE_IF() for a test case in sp-code.test
sql/sp_rcontext.h:
  - Remove unused code
  - Cleanups
sql/sql_class.cc:
  Merge THD::raise_condition_no_handler() into THD::raise_condition().
  After the patch raise_condition_no_handler() was called
  in raise_condition() only.
sql/sql_class.h:
  Remove raise_condition_no_handler().
sql/sql_error.cc:
  Remove Warning_info::reserve_space() -- handled conditions are not
  cleared any more, so there is no need for RESIGNAL to re-push them.
sql/sql_error.h:
  Remove Warning_info::reserve_space().
sql/sql_signal.cc:
  Handled conditions are not cleared any more,
  so there is no need for RESIGNAL to re-push them.

sql/ha_ndbcluster_binlog.cc:
  Fix a compilation failure when built with 
  autotools.

the precursor patch for Bug#52044.
When passing the TABLE instance for invalidation to the
query cache, we didn't always have a valid share
(in case of error).
Make sure we invalidate the table using TABLE_LIST, not
TABLE, object.

sql/sql_table.cc:
  Make sure we invalidate the table using TABLE_LIST, not
  TABLE, object.

/*![:version:] Query Code */, where [:version:] is a sequence of 5 
digits representing the mysql server version(e.g /*!50200 ... */),
is a special comment that the query in it can be executed on those 
servers whose versions are larger than the version appearing in the 
comment. It leads to a security issue when slave's version is larger 
than master's. A malicious user can improve his privileges on slaves. 
Because slave SQL thread is running with SUPER privileges, so it can
execute queries that he/she does not have privileges on master.

This bug is fixed with the logic below: 
- To replace '!' with ' ' in the magic comments which are not applied on
  master. So they become common comments and will not be applied on slave.

- Example:
  'INSERT INTO t1 VALUES (1) /*!10000, (2)*/ /*!99999 ,(3)*/
  will be binlogged as
  'INSERT INTO t1 VALUES (1) /*!10000, (2)*/ /* 99999 ,(3)*/

mysql-test/suite/rpl/t/rpl_conditional_comments.test:
  Test the patch for this bug.
sql/mysql_priv.h:
  Rename inBuf as rawBuf and remove the const limitation.
sql/sql_lex.cc:
  To replace '!' with ' ' in the magic comments which are not applied on
  master.
sql/sql_lex.h:
  Remove the const limitation on parameter buff, as it can be modified in the function since
  this patch.
  Add member function yyUnput for Lex_input_stream. It set a character back the query buff.
sql/sql_parse.cc:
  Rename inBuf as rawBuf and remove the const limitation.
sql/sql_partition.cc:
  Remove the const limitation on parameter part_buff, as it can be modified in the function since
  this patch.
sql/sql_partition.h:
  Remove the const limitation on parameter part_buff, as it can be modified in the function since
  this patch.
sql/table.h:
  Remove the const limitation on variable partition_info, as it can be modified since
  this patch.

            DELETE statement

Single-table delete ordered by a field that has a hash-type index
may cause an assertion failure or a crash.

An optimization added by the fix for the bug 36569 forced the
optimizer to use ORDER BY-compatible indices when applicable.

However, the existence of unsorted indices (HASH index algorithm
for some engines such as MEMORY/HEAP, NDB) was ignored.

The test_if_order_by_key function has been modified to skip
unsorted indices.


mysql-test/r/heap_hash.result:
  Test case for bug #55472.
mysql-test/t/heap_hash.test:
  Test case for bug #55472.
sql/sql_select.cc:
  Bug #55472: Assertion failed in heap_rfirst function of hp_rfirst.c on
              DELETE statement
  
  The test_if_order_by_key function has been modified to skip
  unsorted indices.

The failure was introduced by a precursor patch for the
fix for Bug#52044.

When opening tables for GRANT statement
to check that subject columns exist,
mysql_table_grant() would try to lock the
tables, and thus start a transaction.
This was unnecessary and lead to an assert.


sql/sql_acl.cc:
  Use open_normal_and_derived_tables() rather than
  open_and_lock_tables() to avoid an assert
  that no transaction is started in GRANT statement.

to be more descriptive.

Remove dead and unused code.
Update to reflect the code review requests.

include/thr_lock.h:
  Remove declarations for THR_LOCK_OWNER,
  added along with the patch for sensitive cursors.
mysys/thr_lock.c:
  Remove support for multiple thr_lock requestors
  per THD.
sql/lock.cc:
  Revert the patch that added support for sensitive cursors.
sql/sp_rcontext.cc:
  Updated the use of mysql_open_cursor().
sql/sql_class.cc:
  Move the instance of Server_side_cursor
  from class Prepared_statement to class Statement.
sql/sql_class.h:
  Move the isntance of Server_side_cursor
  from class Prepared_statement to class
  Statement.
  Remove multiple lock_ids of thr_lock.
sql/sql_cursor.cc:
  Remove Sensitive_cursor implementation.
sql/sql_cursor.h:
  Remove declarations for sensitive cursors.
sql/sql_prepare.cc:
  Move the declaration of instance of Server_side_cursor
  from class Statement to class Prepared_statement, 
  where it's used.
sql/sql_select.cc:
  Remove sensitive cursor support.
sql/sql_select.h:
  Remove sensitive cursor support.
sql/sql_union.cc:
  Remove sensitive cursor support.

This patch also fixes Bug#55452 "SET PASSWORD is
replicated twice in RBR mode".

The goal of this patch is to remove the release of 
metadata locks from close_thread_tables().
This is necessary to not mistakenly release
the locks in the course of a multi-step
operation that involves multiple close_thread_tables()
or close_tables_for_reopen().

On the same token, move statement commit outside 
close_thread_tables().

Other cleanups:
Cleanup COM_FIELD_LIST.
Don't call close_thread_tables() in COM_SHUTDOWN -- there
are no open tables there that can be closed (we leave
the locked tables mode in THD destructor, and this
close_thread_tables() won't leave it anyway).

Make open_and_lock_tables() and open_and_lock_tables_derived()
call close_thread_tables() upon failure.
Remove the calls to close_thread_tables() that are now
unnecessary.

Simplify the back off condition in Open_table_context.

Streamline metadata lock handling in LOCK TABLES 
implementation.

Add asserts to ensure correct life cycle of 
statement transaction in a session.

Remove a piece of dead code that has also become redundant
after the fix for Bug 37521.

mysql-test/r/variables.result:
  Update results: set @@autocommit and statement transaction/
  prelocked mode.
mysql-test/r/view.result:
  A harmless change in CHECK TABLE <view> status for a broken view.
  If previously a failure to prelock all functions used in a view 
  would leave the connection in LTM_PRELOCKED mode, now we call
  close_thread_tables() from open_and_lock_tables()
  and leave prelocked mode, thus some check in mysql_admin_table() that
  works only in prelocked/locked tables mode is no longer activated.
mysql-test/suite/rpl/r/rpl_row_implicit_commit_binlog.result:
  Fixed Bug#55452 "SET PASSWORD is replicated twice in
  RBR mode": extra binlog events are gone from the
  binary log.
mysql-test/t/variables.test:
  Add a test case: set autocommit and statement transaction/prelocked
  mode.
sql/event_data_objects.cc:
  Simplify code in Event_job_data::execute().
  Move sp_head memory management to lex_end().
sql/event_db_repository.cc:
  Move the release of metadata locks outside
  close_thread_tables().
  Make sure we call close_thread_tables() when
  open_and_lock_tables() fails and remove extra
  code from the events data dictionary.
  Use close_mysql_tables(), a new internal
  function to properly close mysql.* tables
  in the data dictionary.
  Contract Event_db_repository::drop_events_by_field,
  drop_schema_events into one function.
  When dropping all events in a schema,
  make sure we don't mistakenly release all
  locks acquired by DROP DATABASE. These
  include locks on the database name
  and the global intention exclusive
  metadata lock.
sql/event_db_repository.h:
  Function open_event_table() does not require an instance 
  of Event_db_repository.
sql/events.cc:
  Use close_mysql_tables() instead of close_thread_tables()
  to bootstrap events, since the latter no longer
  releases metadata locks.
sql/ha_ndbcluster.cc:
  - mysql_rm_table_part2 no longer releases
  acquired metadata locks. Do it in the caller.
sql/ha_ndbcluster_binlog.cc:
  Deploy the new protocol for closing thread
  tables in run_query() and ndb_binlog_index
  code.
sql/handler.cc:
  Assert that we never call ha_commit_trans/
  ha_rollback_trans in sub-statement, which
  is now the case.
sql/handler.h:
  Add an accessor to check whether THD_TRANS object
  is empty (has no transaction started).
sql/log.cc:
  Update a comment.
sql/log_event.cc:
  Since now we commit/rollback statement transaction in 
  mysql_execute_command(), we need a mechanism to communicate
  from Query_log_event::do_apply_event() to mysql_execute_command()
  that the statement transaction should be rolled back, not committed.
  Ideally it would be a virtual method of THD. I hesitate
  to make THD a virtual base class in this already large patch.
  Use a thd->variables.option_bits for now.
  
  Remove a call to close_thread_tables() from the slave IO
  thread. It doesn't open any tables, and the protocol
  for closing thread tables is more complicated now.
  
  Make sure we properly close thread tables, however, 
  in Load_data_log_event, which doesn't
  follow the standard server execution procedure
  with mysql_execute_command().
  @todo: this piece should use Server_runnable
  framework instead.
  Remove an unnecessary call to mysql_unlock_tables().
sql/rpl_rli.cc:
  Update Relay_log_info::slave_close_thread_tables()
  to follow the new close protocol.
sql/set_var.cc:
  Remove an unused header.
sql/slave.cc:
  Remove an unnecessary call to
  close_thread_tables().
sql/sp.cc:
  Remove unnecessary calls to close_thread_tables()
  from SP DDL implementation. The tables will
  be closed by the caller, in mysql_execute_command().
  When dropping all routines in a database, make sure
  to not mistakenly drop all metadata locks acquired
  so far, they include the scoped lock on the schema.
sql/sp_head.cc:
  Correct the protocol that closes thread tables
  in an SP instruction.
  Clear lex->sphead before cleaning up lex
  with lex_end to make sure that we don't
  delete the sphead twice. It's considered
  to be "cleaner" and more in line with
  future changes than calling delete lex->sphead
  in other places that cleanup the lex.
sql/sp_head.h:
  When destroying m_lex_keeper of an instruction,
  don't delete the sphead that all lex objects
  share. 
  @todo: don't store a reference to routine's sp_head
  instance in instruction's lex.
sql/sql_acl.cc:
  Don't call close_thread_tables() where the caller will
  do that for us.
  Fix Bug#55452 "SET PASSWORD is replicated twice in RBR 
  mode" by disabling RBR replication in change_password()
  function.
  Use close_mysql_tables() in bootstrap and ACL reload
  code to make sure we release all metadata locks.
sql/sql_base.cc:
  This is the main part of the patch:
  - remove manipulation with thd->transaction
  and thd->mdl_context from close_thread_tables().
  Now this function is only responsible for closing
  tables, nothing else.
  This is necessary to be able to easily use
  close_thread_tables() in procedures, that
  involve multiple open/close tables, which all
  need to be protected continuously by metadata
  locks.
  Add asserts ensuring that TABLE object
  is only used when is protected by a metadata lock.
  Simplify the back off condition of Open_table_context,
  we no longer need to look at the autocommit mode.
  Make open_and_lock_tables() and open_normal_and_derived_tables()
  close thread tables and release metadata locks acquired so-far 
  upon failure. This simplifies their usage.
  Implement close_mysql_tables().
sql/sql_base.h:
  Add declaration for close_mysql_tables().
sql/sql_class.cc:
  Remove a piece of dead code that has also become redundant
  after the fix for Bug 37521.
  The code became dead when my_eof() was made a non-protocol method,
  but a method that merely modifies the diagnostics area.
  The code became redundant with the fix for Bug#37521, when 
  we started to cal close_thread_tables() before
  Protocol::end_statement().
sql/sql_do.cc:
  Do nothing in DO if inside a substatement
  (the assert moved out of trans_rollback_stmt).
sql/sql_handler.cc:
  Add comments.
sql/sql_insert.cc:
  Remove dead code. 
  Release metadata locks explicitly at the
  end of the delayed insert thread.
sql/sql_lex.cc:
  Add destruction of lex->sphead to lex_end(),
  lex "reset" method called at the end of each statement.
sql/sql_parse.cc:
  Move close_thread_tables() and other related
  cleanups to mysql_execute_command()
  from dispatch_command(). This has become
  possible after the fix for Bug#37521.
  Mark federated SERVER statements as DDL.
  
  Next step: make sure that we don't store
  eof packet in the query cache, and move
  the query cache code outside mysql_parse.
  
  Brush up the code of COM_FIELD_LIST.
  Remove unnecessary calls to close_thread_tables().
  
  When killing a query, don't report "OK"
  if it was a suicide.
sql/sql_parse.h:
  Remove declaration of a function that is now static.
sql/sql_partition.cc:
  Remove an unnecessary call to close_thread_tables().
sql/sql_plugin.cc:
  open_and_lock_tables() will clean up
  after itself after a failure.
  Move close_thread_tables() above
  end: label, and replace with close_mysql_tables(),
  which will also release the metadata lock
  on mysql.plugin.
sql/sql_prepare.cc:
  Now that we no longer release locks in close_thread_tables()
  statement prepare code has become more straightforward.
  Remove the now redundant check for thd->killed() (used
  only by the backup project) from Execute_server_runnable.
  Reorder code to take into account that now mysql_execute_command()
  performs lex->unit.cleanup() and close_thread_tables().
sql/sql_priv.h:
  Add a new option to server options to interact
  between the slave SQL thread and execution
  framework (hack). @todo: use a virtual
  method of class THD instead.
sql/sql_servers.cc:
  Due to Bug 25705 replication of 
  DROP/CREATE/ALTER SERVER is broken.
  Make sure at least we do not attempt to 
  replicate these statements using RBR,
  as this violates the assert in close_mysql_tables().
sql/sql_table.cc:
  Do not release metadata locks in mysql_rm_table_part2,
  this is done by the caller.
  Do not call close_thread_tables() in mysql_create_table(),
  this is done by the caller. 
  Fix a bug in DROP TABLE under LOCK TABLES when,
  upon error in wait_while_table_is_used() we would mistakenly
  release the metadata lock on a non-dropped table.
  Explicitly release metadata locks when doing an implicit
  commit.
sql/sql_trigger.cc:
  Now that we delete lex->sphead in lex_end(),
  zero the trigger's sphead in lex after loading
  the trigger, to avoid double deletion.
sql/sql_udf.cc:
  Use close_mysql_tables() instead of close_thread_tables().
sql/sys_vars.cc:
  Remove code added in scope of WL#4284 which would
  break when we perform set @@session.autocommit along
  with setting other variables and using tables or functions.
  A test case added to variables.test.
sql/transaction.cc:
  Add asserts.
sql/tztime.cc:
  Use close_mysql_tables() rather than close_thread_tables().

* Fixed obvious errors (HAVE_BROKEN_PREAD is not true for on any
of systems we use, definitely not on HPUX)

* Remove other junk flags for OSX and HPUX

* Avoid checking type sizes in universal builds on OSX, again 
(CMake2.8.0 fails is different architectures return different results)

* Do not compile template instantiation stuff unless 
EXPLICIT_TEMPLATE_INSTANTIATION is used.

* Some cleanup (make gen_lex_hash simpler, avoid dependencies)

* Exclude some unused files from compilation (strtol.c etc)

Fix assorted warnings in order for the warning-mode to be effective.

Post-merge fix: remove remaining casts which are now
unnecessary and are actually causing warnings.

Remove workarounds for ancient systems.

mysys/default.c:
  Make cast more explicit to ensure a correct offset.

Remove unused macros or macro which are always defined.

include/my_global.h:
  Remove unused macros and move macros which aren't used globally.

Remove unused source code and associated paraphernalia.

Remove unused variables.

Remove code that has been disabled for a long time.

The first problem was that SHOW CREATE TRIGGER took a stronger metadata
lock than required. This caused the statement to be blocked when it was
not needed. For example, LOCK TABLE WRITE in one connection would block
SHOW CREATE TRIGGER in another connection.

Another problem was that a SHOW CREATE TRIGGER statement issued inside
a transaction did not release its metadata locks at the end of the
statement execution. This happened even if SHOW CREATE TRIGGER is an
information statement. The consequence was that SHOW CREATE TRIGGER
was able to block other connections from accessing the table
(e.g. using ALTER TABLE).

This patch fixes the problem by changing SHOW CREATE TRIGGER to take
a MDL_SHARED_HIGH_PRIO metadata lock similar to what is already done
for SHOW CREATE TABLE. The patch also changes SHOW CREATE TRIGGER to
explicitly release any metadata locks taken by the statement after
it completes.

Test case added to show_check.test.

Bug#49542 - Do as the comment suggests and downgrade directory errors from find_file() to a warning unless they happen during a SHOW command.

           sporadically

There are two problems:

1. When closing temporary tables, during the THD clean up - and
   after the session connection was already closed, there is a
   chance we can push an error into the THD diagnostics area, if
   the writing of the implicit DROP event to the binary log fails
   for some reason. As a consequence an assertion can be
   triggered, because at that point the diagnostics area is
   already set.

2. Using push_warning with MYSQL_ERROR::WARN_LEVEL_ERROR is a 
   bug.

Given that close_temporary_tables is mostly called from
THD::cleanup - ie, with the session already closed, we fix
problem #1 by allowing the diagnostics area to be
overwritten. There is one other place in the code that calls
close_temporary_tables - while applying Start_log_event_v3. To
cover that case, we make close_temporary_tables to return the
error, thus, propagating upwards in the stack.

To fix problem #2, we replace push_warning with sql_print_error.

sql/log_event.cc:
  Added handling of error returned by close_temporary_tables to
  Start_log_event_v3::do_apply_event.
sql/sql_base.cc:
  Three changes to close_temporary_tables:
  1. it returns a boolean now (instead of void)
  2. it uses sql_print_error instead of push_warning when writing to
     binary log fails
  3. we set can_overwrite_status before writing to the binary log,
     thence not risking triggering an assertion by any other push
     into diagnostics area happening inside mysql_bin_log.write.
sql/sql_base.h:
  Changed the interface of close_temporary_tables so that it returns
  bool instead of void.

           concurrent SHOW CREATE

The problem was that a SHOW CREATE TABLE statement issued inside
a transaction did not release its metadata locks at the end of the
statement execution. This happened even if SHOW CREATE TABLE is an
information statement. 

The consequence was that SHOW CREATE TABLE was able to block other
connections from accessing the table (e.g. using ALTER TABLE).

This patch fixes the problem by explicitly releasing any metadata
locks taken by SHOW CREATE TABLE after the statement completes.

Test case added to show_check.test.

The problem was that a statement could cause an assert if it was aborted by
KILL QUERY while it waited on a metadata lock. This assert checks that a
statement either sends OK or an error to the client. If the bug was triggered
on release builds, it caused OK to be sent to the client instead of
ER_QUERY_INTERRUPTED.

The root cause of the problem was that there are two separate ways to tell if a
statement is killed: thd->killed and mysys_var->abort. KILL QUERY causes both
to be set, thd->killed before mysys_var->abort. Also, both values are reset
at the end of statement execution. This means that it is possible for
KILL QUERY to first set thd->killed, then have the killed statement reset
both thd->killed and mysys_var->abort and finally have KILL QUERY set
mysys_var->abort. This means that the connection with the killed statement
will start executing the next statement with the two values out of sync - i.e.
thd->killed not set but mysys_var->abort set.

Since mysys_var->abort is used to check if a wait for a metadata lock should
be aborted, the next statement would immediately abort any such waiting.
When waiting is aborted, no OK message is sent and thd->killed is checked to
see if ER_QUERY_INTERRUPTED should be sent to the client. But since
the->killed had been reset, neither OK nor an error message was sent to the
client. This then triggered the assert.

This patch fixes the problem by changing the metadata lock waiting code to
check thd->killed.

No test case added as reproducing the assert is dependent on very exact timing
of two (or more) threads. The patch has been checked using RQG and the grammar
posted on the bug report.

SHOW DATABASES LIKE ... was not converting to lowercase on comparison as the
documentation is suggesting. 
Fixed it to behave similarly to SHOW TABLES LIKE ... and updated the failing
on MacOSX lowercase_table2 test case.

to write into a closed socket

sql/protocol.cc:
  Protocol::flush modified: set thd->main_da.can_overwrite_status= TRUE
  before call to net_flush() in order to prevent crash on assert in case
  of socket write failure, reset it to FALSE when net_flush() returned;
  Protocol::send_fields modified: return from method with error if call to
  my_net_write(), proto.write() or write_eof_packet() failed.
sql/sql_cache.cc:
  Query_cache::send_result_to_client modified: call to
  thd->main_da.disable_status() only if write to socket
  was successful.
sql/sql_cursor.cc:
  Materialized_cursor::fetch modified: leave method if call to
  result->send_data() failed.
sql/sql_prepare.cc:
  send_prep_stmt() modified: call to thd->main_da.disable_status()
  only if thd->protocol_text.send_fields() completed successfully.

Fix warnings flagged by the new warning option -Wunused-but-set-variable
that was added to GCC 4.6 and that is enabled by -Wunused and -Wall. The
option causes a warning whenever a local variable is assigned to but is
later unused. It also warns about meaningless pointer dereferences.

client/mysql.cc:
  Meaningless pointer dereferences.
client/mysql_upgrade.c:
  Check whether reading from the file succeeded.
extra/comp_err.c:
  Unused.
extra/yassl/src/yassl_imp.cpp:
  Skip instead of reading data that is discarded.
include/my_pthread.h:
  Variable is only used in debug builds.
include/mysys_err.h:
  Add new error messages.
mysys/errors.c:
  Add new error message for permission related functions.
mysys/mf_iocache.c:
  Variable is only checked under THREAD.
mysys/my_copy.c:
  Raise a error if chmod or chown fails.
mysys/my_redel.c:
  Raise a error if chmod or chown fails.
regex/engine.c:
  Use a equivalent variable for the assert.
server-tools/instance-manager/instance_options.cc:
  Unused.
sql/field.cc:
  Unused.
sql/item.cc:
  Unused.
sql/log.cc:
  Do not ignore the return value of freopen: only set buffer if
  reopening succeeds.
  
  Adjust doxygen comment to the right function.
  
  Pass message lenght to log function.
sql/mysqld.cc:
  Do not ignore the return value of freopen: only set buffer if
  reopening succeeds.
sql/partition_info.cc:
  Unused.
sql/slave.cc:
  No need to set pointer to the address of '\0'.
sql/spatial.cc:
  Unused. Left for historical purposes.
sql/sql_acl.cc:
  Unused.
sql/sql_base.cc:
  Pointers are always set to the same variables.
sql/sql_parse.cc:
  End statement if reading fails.
  
  Store the buffer after it has actually been updated.
sql/sql_repl.cc:
  No need to set pointer to the address of '\0'.
sql/sql_show.cc:
  Put variable under the same ifdef block.
sql/udf_example.c:
  Set null pointer flag appropriately.
storage/csv/ha_tina.cc:
  Meaningless dereferences.
storage/example/ha_example.cc:
  Return the error since it's available.
storage/myisam/mi_locking.c:
  Remove unused and dead code.

           table with active trx

Essentially, the problem is that InnoDB does a implicit commit
when a cursor (table handler) is unlocked/closed, creating
a dissonance between the transaction state within the server
layer and the storage engine layer. Theoretically, a statement
transaction can encompass several table instances in a similar
manner to a multiple statement transaction, hence it does not
make sense to limit a statement transaction to the lifetime of
the table instances (cursors) used within it.

Since this particular instance of the problem is only triggerable
on 5.1 and is masked on 5.5 due 2PC being skipped (assertion is in
the prepare phase of a 2PC), the solution (which is less risky) is
to explicitly end the transaction before the cached table is unlock
on rename table.

The patch is to be null merged into trunk.

mysql-test/include/commit.inc:
  Fix counters, the binlog engine does not get involved anymore.
mysql-test/suite/innodb_plugin/r/innodb_bug54453.result:
  Add test case result for Bug#54453
mysql-test/suite/innodb_plugin/t/innodb_bug54453.test:
  Add test case for Bug#54453
sql/sql_table.cc:
  End transaction as otherwise InnoDB will end it behind our backs.

Problem: when SHOW BINLOG EVENTS was issued, it increased the value of
@@session.max_allowed_packet. This allowed a non-root user to increase
the amount of memory used by her thread arbitrarily. Thus, it removes
the bound on the amount of system resources used by a client, so it
presents a security risk (DoS attack).

Fix: it is correct to increase the value of @@session.max_allowed_packet
while executing SHOW BINLOG EVENTS (see BUG 30435). However, the
increase should only be temporary. Thus, the fix is to restore the value
when SHOW BINLOG EVENTS ends.
The value of @@session.max_allowed_packet is also increased in
mysql_binlog_send (i.e., the binlog dump thread). It is not clear if this
can cause any trouble, since normally the client that issues
COM_BINLOG_DUMP will not issue any other commands that would be affected
by the increased value of @@session.max_allowed_packet. However, we
restore the value just in case.


mysql-test/suite/rpl/r/rpl_packet.result:
  update result file
mysql-test/suite/rpl/t/rpl_packet.test:
  Add test that verifies that @@session.max_allowed_packet does not change
  when issuing SHOW BINLOG EVENTS.
  Make previous sub-test clean up.
  Add comments listing the bugs in this test case.
sql/sql_repl.cc:
  Restore the old value of thd->variables.max_allowed_packet at the
  end of mysql_binlog_send and mysql_show_binlog_events.

This bug is a design flaw of the fix for the bug#33546. It assumed that an
item can be used only in one comparison context, but actually it isn't the
case. Item_cache_datetime is used to store result for MIX/MAX aggregate
functions. Because Arg_comparator always compares datetime values as INTs when
possible the Item_cache_datetime most time caches only INT value. But
since all datetime values has STRING result type MIN/MAX functions are asked
for a STRING value when the result is being sent to a client. The
Item_cache_datetime was designed to avoid conversions and get INT/STRING
values from an underlying item, but at the moment the values is asked
underlying item doesn't hold it anymore thus wrong result is returned.
Beside that MIN/MAX aggregate functions was wrongly initializing cached result
and this led to a wrong result.

The Item::has_compatible_context helper function is added. It checks whether
this and given items has the same comparison context or can be compared as
DATETIME values by Arg_comparator. The equality propagation optimization is
adjusted to take into account that items which being compared as DATETIME
can have different comparison contexts.
The Item_cache_datetime now converts cached INT value to a correct STRING
DATETIME value by means of number_to_datetime & my_TIME_to_str functions.
The Arg_comparator::set_cmp_context_for_datetime helper function is added. 
It sets comparison context of items being compared as DATETIMEs to INT if
items will be compared as longlong.
The Item_sum_hybrid::setup function now correctly initializes its result
value.
In order to avoid unnecessary conversions Item_sum_hybrid now states that it
can provide correct longlong value if the item being aggregated can do it
too.

mysql-test/r/group_by.result:
  Added a test case for the bug#49771.
sql/item.cc:
  Bug#49771: Incorrect MIN/MAX for date/time values.
  The equality propagation mechanism is adjusted to take into account that
  items which being compared as DATETIME can have different comparison
  contexts.
  The Item_cache_datetime now converts cached INT value to a correct STRING
  DATETIME/TIME value.
sql/item.h:
  Bug#49771: Incorrect MIN/MAX for date/time values.
  The Item::has_compatible_context helper function is added. It checks whether
  this and given items has the same comparison context or can be compared as
  DATETIME values by Arg_comparator.
  Added Item_cache::clear helper function.
sql/item_cmpfunc.cc:
  Bug#49771: Incorrect MIN/MAX for date/time values.
  The Arg_comparator::set_cmp_func now sets the correct comparison context
  for items being compared as DATETIME values.
sql/item_cmpfunc.h:
  Bug#49771: Incorrect MIN/MAX for date/time values.
  The Arg_comparator::set_cmp_context_for_datetime helper function is added. 
  It sets comparison context of items being compared as DATETIMEs to INT if
  items will be compared as longlong.
sql/item_sum.cc:
  Bug#49771: Incorrect MIN/MAX for date/time values.
  The Item_sum_hybrid::setup function now correctly initializes its result
  value.
sql/item_sum.h:
  Bug#49771: Incorrect MIN/MAX for date/time values.
  In order to avoid unnecessary conversions Item_sum_hybrid now states that it
  can provide correct longlong value if the item being aggregated can do it
  too.