GitLab

When using Unique Keys with nullable parts in RBR, the slave can
choose the wrong row to update. This happens because a table with
an unique key containing nullable parts cannot strictly guarantee 
uniqueness. As stated in the manual, for all engines, a UNIQUE 
index allows multiple NULL values for columns that can contain 
NULL.

We fix this at the slave by extending the checks before assuming
that the row found through an unique index is is the correct
one. This means that when a record (R) is fetched from the storage
engine and a key that is not primary (K) is used, the server does 
the following: 

 - If K is unique and has no nullable parts, it returns R;
 - Otherwise, if any field in the before image that is part of K
   is null do an index scan;
 - If there is no NULL field in the BI part of K, then return R.

A side change: renamed the existing test case file and added a
test case covering the changes in this patch.

multiquery packet).

Background:

  - a query can contain multiple SQL statements;

  - the server frees resources allocated to process a query when the
    whole query is handled. In other words, resources allocated to process
    one SQL statement from a multi-statement query are freed when all SQL
    statements are handled.

The problem was that the parser allocated a buffer of size of the whole
query for each SQL statement in a multi-statement query. Thus, if a query
had many SQL-statements (so, the query was long), but each SQL statement
was short, ther parser tried to allocate huge amount of memory (number of
small SQL statements * length of the whole query).

The memory was allocated for a so-called "cpp buffer", which is intended to
store pre-processed SQL statement -- SQL text without version specific
comments.

The fix is to allocate memory for the "cpp buffer" once for all SQL
statements (once for a query).

Clarified error messages related to unsafe statements:
 - avoid the internal technical term "row injection"
 - use 'binary log' instead of 'binlog'
 - avoid the word 'unsafeness'


mysql-test/extra/binlog_tests/blackhole.test:
  updated suppression pattern
mysql-test/extra/binlog_tests/mix_innodb_myisam_binlog.test:
  updated suppression pattern
mysql-test/extra/rpl_tests/rpl_binlog_max_cache_size.test:
  updated suppression pattern
mysql-test/extra/rpl_tests/rpl_insert_id.test:
  updated suppression pattern
mysql-test/extra/rpl_tests/rpl_loaddata.test:
  updated suppression pattern
mysql-test/extra/rpl_tests/rpl_mixing_engines.test:
  updated suppression pattern
mysql-test/extra/rpl_tests/rpl_start_stop_slave.test:
  updated suppression pattern
mysql-test/extra/rpl_tests/rpl_stm_000001.test:
  updated suppression pattern
mysql-test/extra/rpl_tests/rpl_stop_middle_group.test:
  updated suppression pattern
mysql-test/r/archive.result:
  updated result file
mysql-test/r/commit_1innodb.result:
  updated result file
mysql-test/r/ctype_cp932_binlog_stm.result:
  updated result file
mysql-test/r/partition_innodb_stmt.result:
  updated result file
mysql-test/suite/binlog/r/binlog_innodb.result:
  updated result file
mysql-test/suite/binlog/r/binlog_killed.result:
  updated result file
mysql-test/suite/binlog/r/binlog_multi_engine.result:
  updated result file
mysql-test/suite/binlog/r/binlog_row_mix_innodb_myisam.result:
  updated result file
mysql-test/suite/binlog/r/binlog_statement_insert_delayed.result:
  updated result file
mysql-test/suite/binlog/r/binlog_stm_blackhole.result:
  updated result file
mysql-test/suite/binlog/r/binlog_stm_do_db.result:
  updated result file
mysql-test/suite/binlog/r/binlog_stm_mix_innodb_myisam.result:
  updated result file
mysql-test/suite/binlog/r/binlog_stm_ps.result:
  updated result file
mysql-test/suite/binlog/r/binlog_stm_row.result:
  updated result file
mysql-test/suite/binlog/r/binlog_stm_unsafe_warning.result:
  updated result file
mysql-test/suite/binlog/r/binlog_unsafe.result:
  updated result file
mysql-test/suite/binlog/t/binlog_killed.test:
  updated suppression pattern
mysql-test/suite/binlog/t/binlog_multi_engine.test:
  updated suppression pattern
mysql-test/suite/binlog/t/binlog_statement_insert_delayed.test:
  updated suppression pattern
mysql-test/suite/binlog/t/binlog_stm_ps.test:
  updated suppression pattern
mysql-test/suite/binlog/t/binlog_stm_row.test:
  updated suppression pattern
mysql-test/suite/binlog/t/binlog_stm_unsafe_warning.test:
  updated suppression pattern
mysql-test/suite/binlog/t/binlog_tmp_table.test:
  updated suppression pattern
mysql-test/suite/binlog/t/binlog_unsafe.test:
  updated suppression pattern
mysql-test/suite/ndb/r/ndb_binlog_format.result:
  updated result file
mysql-test/suite/ndb/t/ndb_binlog_format.test:
  updated suppression pattern
mysql-test/suite/perfschema/r/binlog_stmt.result:
  updated result file
mysql-test/suite/perfschema/t/binlog_stmt.test:
  updated suppression pattern
mysql-test/suite/rpl/r/rpl_begin_commit_rollback.result:
  updated result file
mysql-test/suite/rpl/r/rpl_blackhole.result:
  updated result file
mysql-test/suite/rpl/r/rpl_concurrency_error.result:
  updated result file
mysql-test/suite/rpl/r/rpl_get_lock.result:
  updated result file
mysql-test/suite/rpl/r/rpl_insert_id.result:
  updated result file
mysql-test/suite/rpl/r/rpl_insert_ignore.result:
  updated result file
mysql-test/suite/rpl/r/rpl_misc_functions.result:
  updated result file
mysql-test/suite/rpl/r/rpl_mixed_binlog_max_cache_size.result:
  updated result file
mysql-test/suite/rpl/r/rpl_mixed_mixing_engines.result:
  updated result file
mysql-test/suite/rpl/r/rpl_mysql_upgrade.result:
  updated result file
mysql-test/suite/rpl/r/rpl_non_direct_mixed_mixing_engines.result:
  updated result file
mysql-test/suite/rpl/r/rpl_non_direct_row_mixing_engines.result:
  updated result file
mysql-test/suite/rpl/r/rpl_non_direct_stm_mixing_engines.result:
  updated result file
mysql-test/suite/rpl/r/rpl_nondeterministic_functions.result:
  updated result file
mysql-test/suite/rpl/r/rpl_read_only.result:
  updated result file
mysql-test/suite/rpl/r/rpl_row_binlog_max_cache_size.result:
  updated result file
mysql-test/suite/rpl/r/rpl_row_inexist_tbl.result:
  updated result file
mysql-test/suite/rpl/r/rpl_row_mixing_engines.result:
  updated result file
mysql-test/suite/rpl/r/rpl_slow_query_log.result:
  updated result file
mysql-test/suite/rpl/r/rpl_sp.result:
  updated result file
mysql-test/suite/rpl/r/rpl_stm_000001.result:
  updated result file
mysql-test/suite/rpl/r/rpl_stm_auto_increment_bug33029.result:
  updated result file
mysql-test/suite/rpl/r/rpl_stm_binlog_max_cache_size.result:
  updated result file
mysql-test/suite/rpl/r/rpl_stm_loadfile.result:
  updated result file
mysql-test/suite/rpl/r/rpl_stm_mixing_engines.result:
  updated result file
mysql-test/suite/rpl/r/rpl_stm_start_stop_slave.result:
  updated result file
mysql-test/suite/rpl/r/rpl_stm_stop_middle_group.result:
  updated result file
mysql-test/suite/rpl/r/rpl_temp_temporary.result:
  updated result file
mysql-test/suite/rpl/r/rpl_variables_stm.result:
  updated result file
mysql-test/suite/rpl/t/rpl000013.test:
  updated suppression pattern
mysql-test/suite/rpl/t/rpl_begin_commit_rollback.test:
  updated suppression pattern
mysql-test/suite/rpl/t/rpl_blackhole.test:
  updated suppression pattern
mysql-test/suite/rpl/t/rpl_concurrency_error.test:
  updated suppression pattern
mysql-test/suite/rpl/t/rpl_err_ignoredtable.test:
  updated suppression pattern
mysql-test/suite/rpl/t/rpl_get_lock.test:
  updated suppression pattern
mysql-test/suite/rpl/t/rpl_insert.test:
  updated suppression pattern
mysql-test/suite/rpl/t/rpl_insert_id.test:
  updated suppression pattern
mysql-test/suite/rpl/t/rpl_insert_ignore.test:
  updated suppression pattern
mysql-test/suite/rpl/t/rpl_invoked_features.test:
  updated suppression pattern
mysql-test/suite/rpl/t/rpl_misc_functions.test:
  updated suppression pattern
mysql-test/suite/rpl/t/rpl_mixed_bit_pk.test:
  updated suppression pattern
mysql-test/suite/rpl/t/rpl_mysql_upgrade.test:
  updated suppression pattern
mysql-test/suite/rpl/t/rpl_nondeterministic_functions.test:
  updated suppression pattern
mysql-test/suite/rpl/t/rpl_optimize.test:
  updated suppression pattern
mysql-test/suite/rpl/t/rpl_read_only.test:
  updated suppression pattern
mysql-test/suite/rpl/t/rpl_semi_sync.test:
  updated suppression pattern
mysql-test/suite/rpl/t/rpl_semi_sync_event.test:
  updated suppression pattern
mysql-test/suite/rpl/t/rpl_session_var.test:
  updated suppression pattern
mysql-test/suite/rpl/t/rpl_slow_query_log.test:
  updated suppression pattern
mysql-test/suite/rpl/t/rpl_sp.test:
  updated suppression pattern
mysql-test/suite/rpl/t/rpl_stm_auto_increment_bug33029.test:
  updated suppression pattern
mysql-test/suite/rpl/t/rpl_stm_found_rows.test:
  updated suppression pattern
mysql-test/suite/rpl/t/rpl_stm_insert_delayed.test:
  updated suppression pattern
mysql-test/suite/rpl/t/rpl_stm_loadfile.test:
  updated suppression pattern
mysql-test/suite/rpl/t/rpl_temp_table.test:
  updated suppression pattern
mysql-test/suite/rpl/t/rpl_temp_temporary.test:
  updated suppression pattern
mysql-test/suite/rpl/t/rpl_temporary.test:
  updated suppression pattern
mysql-test/suite/rpl/t/rpl_timezone.test:
  updated suppression pattern
mysql-test/suite/rpl/t/rpl_trigger.test:
  updated suppression pattern
mysql-test/suite/rpl/t/rpl_udf.test:
  updated suppression pattern
mysql-test/suite/rpl/t/rpl_user_variables.test:
  updated suppression pattern
mysql-test/suite/rpl/t/rpl_variables_stm.test:
  updated suppression pattern
mysql-test/suite/rpl/t/rpl_view_multi.test:
  updated suppression pattern
mysql-test/suite/rpl_ndb/r/rpl_ndb_binlog_format_errors.result:
  updated result file
mysql-test/suite/rpl_ndb/t/rpl_ndb_binlog_format_errors.test:
  updated suppression pattern
mysql-test/suite/rpl_ndb/t/rpl_ndb_stm_innodb.test:
  updated suppression pattern
mysql-test/suite/sys_vars/t/rpl_init_slave_func.test:
  updated suppression pattern
mysql-test/t/archive.test:
  updated suppression pattern
mysql-test/t/commit_1innodb.test:
  updated suppression pattern
mysql-test/t/create_select_tmp.test:
  updated suppression pattern
mysql-test/t/ctype_cp932_binlog_stm.test:
  updated suppression pattern
mysql-test/t/lock_sync.test:
  updated suppression pattern
mysql-test/t/mysqlbinlog.test:
  updated suppression pattern
mysql-test/t/mysqldump.test:
  updated suppression pattern
mysql-test/t/sp_trans.test:
  updated suppression pattern
sql/log_event.cc:
  Clarified error message.
sql/share/errmsg-utf8.txt:
  Clarified error messages.

Statements with CONNECTION_ID were forced to be kept in the transactional
cache and by consequence non-transactional changes that were supposed to
be flushed ahead of the transaction were kept in the transactional cache.

This happened because after BUG#51894 any statement whose thd's
thread_specific_used was set was kept in the transactional cache. The idea
was to keep changes on temporary tables in the transactional cache. However,
the thread_specific_used was set not only for statements that accessed
temporary tables but also when the CONNECTION_ID was used.

To fix the problem, we created a new variable to keep track of updates
to temporary tables.


mysql-test/suite/rpl/r/rpl_temp_temporary.result:
  Added a test case.
mysql-test/suite/rpl/t/rpl_temp_temporary.test:
  Added a test case.
sql/log_event.cc:
  Uses the thread_temporary_used to decide if a statement should
  be kept in the transactional cache or not.
sql/sql_class.cc:
  Sets the thread_temporary_used while calling the decide_logging_format.
sql/sql_class.h:
  Defines the thread_temporary_used.
sql/sql_parse.cc:
  Resets the thread_temporary_used.

           of sync

In RBR, sometimes the table->s->last_null_bit_pos can be zero. This
has impact at the slave when it compares records fetched from the
storage engine against records in the binary log event. If
last_null_bit_pos is zero the slave, while comparing in
log_event.cc:record_compare function, would set all bits in the last
null_byte to 1 (assumed all 8 were unused) . Thence it would loose the
ability to distinguish records that were similar in contents except
for the fact that some field was null in one record, but not in the
other. Ultimately this would cause wrong matches, and in the specific
case depicted in the bug report the same record would be updated
twice, resulting in a lost update.

Additionally, in the record_compare function the slave was setting the
X bit unconditionally. There are cases that the X bit does not exist
in the record header. This could also lead to wrong matches between
records.

We fix both by conditionally resetting the bits: (i) unused null_bits
are set if last_null_bit_pos > 0; (ii) X bit is set if
HA_OPTION_PACK_RECORD is in use.

mysql-test/extra/rpl_tests/rpl_record_compare.test:
  Shared part of the test case for MyISAM and InnoDB.
mysql-test/suite/rpl/t/rpl_row_rec_comp_innodb.test:
  InnoDB test case.
mysql-test/suite/rpl/t/rpl_row_rec_comp_myisam.test:
  MyISAM test case. Added also coverage for Field_bits case.
sql/log_event.cc:
  Deployed conditional setting of unused bits at record_compare.
sql/log_event_old.cc:
  Same change as in log_event.cc.

          transaction
BUG#52616 Temp table prevents switch binlog format from STATEMENT to ROW

Before the WL#2687 and BUG#46364, every non-transactional change that happened
after a transactional change was written to trx-cache and flushed upon
committing the transaction. WL#2687 and BUG#46364 changed this behavior and
non-transactional changes are now written to the binary log upon committing
the statement.

A binary log event is identified as transactional or non-transactional through
a flag in the Log_event which is set taking into account the underlie storage
engine on what it is stems from. In the current bug, this flag was not being
set properly when the DROP TEMPORARY TABLE was executed.

However, while fixing this bug we figured out that changes to temporary tables
should be always written to the trx-cache if there is an on-going transaction.
Otherwise, binlog events in the reversed order would be produced.

Regarding concurrency, keeping changes to temporary tables in the trx-cache is
also safe as temporary tables are only visible to the owner connection.

In this patch, we classify the following statements as unsafe:
   1 - INSERT INTO t_myisam SELECT * FROM t_myisam_temp

   2 - INSERT INTO t_myisam_temp SELECT * FROM t_myisam

   3 - CREATE TEMPORARY TABLE t_myisam_temp SELECT * FROM t_myisam

On the other hand, the following statements are classified as safe:

   1 - INSERT INTO t_innodb SELECT * FROM t_myisam_temp

   2 - INSERT INTO t_myisam_temp SELECT * FROM t_innodb

The patch also guarantees that transactions that have a DROP TEMPORARY are
always written to the binary log regardless of the mode and the outcome:
commit or rollback. In particular, the DROP TEMPORARY is extended with the
IF EXISTS clause when the current statement logging format is set to row.

Finally, the patch allows to switch from STATEMENT to MIXED/ROW when there
are temporary tables but the contrary is not possible.

mysql-test/extra/rpl_tests/rpl_binlog_max_cache_size.test:
  Updated the test case because 
     CREATE TEMPORARY TABLE t_innodb_temp SELECT * FROM t_myisam is not unsafe.
mysql-test/extra/rpl_tests/rpl_implicit_commit_binlog.test:
  Updated the test case due to the new rules: changes to
  temporary tables are written to the binary log in the
  boundaries of a transaction if there is any.
mysql-test/extra/rpl_tests/rpl_innodb.test:
  Removed comments from the test case that became false after the patch.
mysql-test/extra/rpl_tests/rpl_loaddata.test:
  Suppressed warning messages due to the following cases:
  
     1 - INSERT INTO t_myisam SELECT * FROM t_myisam_temp
  
     2 - INSERT INTO t_myisam_temp SELECT * FROM t_myisam
mysql-test/include/ctype_utf8_table.inc:
  Suppressed warning messages due to the following cases:
  
     1 - INSERT INTO t_myisam SELECT * FROM t_myisam_temp
  
     2 - INSERT INTO t_myisam_temp SELECT * FROM t_myisam
mysql-test/r/ctype_cp932_binlog_stm.result:
  Updated the test case due to the new rules: changes to
  temporary tables are written to the binary log in the
  boundaries of a transaction if there is any.
mysql-test/suite/binlog/r/binlog_database.result:
  Updated the test case due to the new rules: changes to
  temporary tables are written to the binary log in the
  boundaries of a transaction if there is any.
mysql-test/suite/binlog/r/binlog_innodb_row.result:
  Updated the result file.
mysql-test/suite/binlog/r/binlog_multi_engine.result:
  Updated the unsafe message.
mysql-test/suite/binlog/r/binlog_row_binlog.result:
  Updated the test case due to the new rules: changes to
  temporary tables are written to the binary log in the
  boundaries of a transaction if there is any.
mysql-test/suite/binlog/r/binlog_row_drop_tmp_tbl.result:
  Updated the test case due to the new rules: changes to
  temporary tables are written to the binary log in the
  boundaries of a transaction if there is any.
mysql-test/suite/binlog/r/binlog_row_mix_innodb_myisam.result:
  Updated the test case due to the new rules: changes to
  temporary tables are written to the binary log in the
  boundaries of a transaction if there is any.
mysql-test/suite/binlog/r/binlog_stm_binlog.result:
  Updated the result file.
mysql-test/suite/binlog/r/binlog_stm_mix_innodb_myisam.result:
  Updated the test case due to the new rules: changes to
  temporary tables are written to the binary log in the
  boundaries of a transaction if there is any.
mysql-test/suite/binlog/t/binlog_tmp_table.test:
  Suppressed warning messages due to the following cases:
  
     1 - INSERT INTO t_myisam SELECT * FROM t_myisam_temp
  
     2 - INSERT INTO t_myisam_temp SELECT * FROM t_myisam
mysql-test/suite/ndb/r/ndb_binlog_format.result:
  Updated the unsafe message.
mysql-test/suite/rpl/r/rpl_concurrency_error.result:
  Updated the unsafe message.
mysql-test/suite/rpl/r/rpl_mixed_binlog_max_cache_size.result:
  Updated the result file because 
     CREATE TEMPORARY TABLE t_innodb_temp SELECT * FROM t_myisam is not unsafe.
mysql-test/suite/rpl/r/rpl_mixed_implicit_commit_binlog.result:
  Updated the test case due to the new rules: changes to
  temporary tables are written to the binary log in the
  boundaries of a transaction if there is any.
mysql-test/suite/rpl/r/rpl_mixed_mixing_engines.result:
  Updated the test case due to the new rules: changes to
  temporary tables are written to the binary log in the
  boundaries of a transaction if there is any.
mysql-test/suite/rpl/r/rpl_mixed_row_innodb.result:
  Added some comments to ease the understanding of the result file.
mysql-test/suite/rpl/r/rpl_non_direct_mixed_mixing_engines.result:
  Updated the test case due to the new rules: changes to
  temporary tables are written to the binary log in the
  boundaries of a transaction if there is any.
mysql-test/suite/rpl/r/rpl_non_direct_row_mixing_engines.result:
  Updated the test case due to the new rules: changes to
  temporary tables are written to the binary log in the
  boundaries of a transaction if there is any.
mysql-test/suite/rpl/r/rpl_non_direct_stm_mixing_engines.result:
  Updated the test case due to the new rules: changes to
  temporary tables are written to the binary log in the
  boundaries of a transaction if there is any.
mysql-test/suite/rpl/r/rpl_row_drop.result:
  Updated the test case due to the new rules: changes to
  temporary tables are written to the binary log in the
  boundaries of a transaction if there is any.
mysql-test/suite/rpl/r/rpl_row_implicit_commit_binlog.result:
  Updated the test case due to the new rules: changes to
  temporary tables are written to the binary log in the
  boundaries of a transaction if there is any.
mysql-test/suite/rpl/r/rpl_row_mixing_engines.result:
  Updated the test case due to the new rules: changes to
  temporary tables are written to the binary log in the
  boundaries of a transaction if there is any.
mysql-test/suite/rpl/r/rpl_stm_binlog_max_cache_size.result:
  Updated the result file because 
     CREATE TEMPORARY TABLE t_innodb_temp SELECT * FROM t_myisam is not unsafe.
mysql-test/suite/rpl/r/rpl_stm_implicit_commit_binlog.result:
  Updated the test case due to the new rules: changes to
  temporary tables are written to the binary log in the
  boundaries of a transaction if there is any.
mysql-test/suite/rpl/r/rpl_stm_innodb.result:
  Added some comments to ease the understanding of the result file.
mysql-test/suite/rpl/r/rpl_stm_mixing_engines.result:
  Updated the test case due to the new rules: changes to
  temporary tables are written to the binary log in the
  boundaries of a transaction if there is any.
mysql-test/suite/rpl/r/rpl_stm_stop_middle_group.result:
  Updated the unsafe message.
mysql-test/suite/rpl/r/rpl_temp_temporary.result:
  Added a test case.
mysql-test/suite/rpl/t/rpl000013.test:
  Suppressed warning messages due to the following cases:
  
     1 - INSERT INTO t_myisam SELECT * FROM t_myisam_temp
  
     2 - INSERT INTO t_myisam_temp SELECT * FROM t_myisam
  
     3 - CREATE TEMPORARY TABLE t_myisam_temp SELECT * FROM t_myisam
mysql-test/suite/rpl/t/rpl_misc_functions.test:
  Suppressed warning messages.
mysql-test/suite/rpl/t/rpl_temp_table.test:
  Suppressed warning messages due to the following cases:
  
     1 - INSERT INTO t_myisam SELECT * FROM t_myisam_temp
  
     2 - INSERT INTO t_myisam_temp SELECT * FROM t_myisam
  
     3 - CREATE TEMPORARY TABLE t_myisam_temp SELECT * FROM t_myisam
mysql-test/suite/rpl/t/rpl_temp_temporary.test:
  Added a test case.
mysql-test/suite/rpl/t/rpl_temporary.test:
  Suppressed warning messages due to the following cases:
  
     1 - INSERT INTO t_myisam SELECT * FROM t_myisam_temp
  
     2 - INSERT INTO t_myisam_temp SELECT * FROM t_myisam
  
     3 - CREATE TEMPORARY TABLE t_myisam_temp SELECT * FROM t_myisam
mysql-test/suite/rpl_ndb/r/rpl_ndb_row_implicit_commit_binlog.result:
  Updated the test case due to the new rules: changes to
  temporary tables are written to the binary log in the
  boundaries of a transaction if there is any.
mysql-test/suite/rpl_ndb/r/rpl_truncate_7ndb.result:
  Updated the test case to remove references to positions
  in the binary log.
mysql-test/suite/rpl_ndb/t/rpl_truncate_7ndb.test:
  Updated the test case to remove references to positions
  in the binary log.
mysql-test/t/create_select_tmp.test:
  Suppressed warning messages due to the following cases:
  
     1 - INSERT INTO t_myisam SELECT * FROM t_myisam_temp
  
     2 - INSERT INTO t_myisam_temp SELECT * FROM t_myisam
  
     3 - CREATE TEMPORARY TABLE t_myisam_temp SELECT * FROM t_myisam
mysql-test/t/ctype_cp932_binlog_stm.test:
  Suppressed warning messages due to the following cases:
  
     1 - INSERT INTO t_myisam SELECT * FROM t_myisam_temp
  
     2 - INSERT INTO t_myisam_temp SELECT * FROM t_myisam
  
     3 - CREATE TEMPORARY TABLE t_myisam_temp SELECT * FROM t_myisam
mysql-test/t/mysqlbinlog.test:
  Suppressed warning messages due to the following cases:
  
     1 - INSERT INTO t_myisam SELECT * FROM t_myisam_temp
  
     2 - INSERT INTO t_myisam_temp SELECT * FROM t_myisam
  
     3 - CREATE TEMPORARY TABLE t_myisam_temp SELECT * FROM t_myisam
sql/log.cc:
  Improved the code by creating several functions to hide decision
  on type of engine changed, commit/abort, etc:  
  
  . stmt_has_updated_non_trans_table
  
  . trans_has_updated_non_trans_table
  
  . ending_trans
  
  Updated the binlog_rollback function and the use of the 
  OPTION_KEEP_LOG which indincates when a temporary table was
  either created or dropped and as such the command must be 
  logged if not in MIXED mode and even while rolling back the
  transaction.
sql/log.h:
  Improved the code by creating several functions to hide decision
  on type of engine changed, commit/abort, etc.
sql/log_event.cc:
  Removed the setting of the OPTION_KEEP_LOG as it is related to CREATE
  TEMPORARY and DROP TEMPORARY and not to the type of engine (i.e.
  transactional or non-transactional).
sql/log_event_old.cc:
  Removed the setting of the OPTION_KEEP_LOG as it is related to CREATE
  TEMPORARY and DROP TEMPORARY and not to the type of engine (i.e.
  transactional or non-transactional).
sql/share/errmsg-utf8.txt:
  Updated the unsafe message.
sql/sql_class.cc:
  Classifies the following statements as unsafe:
     1 - INSERT INTO t_myisam SELECT * FROM t_myisam_temp
  
     2 - INSERT INTO t_myisam_temp SELECT * FROM t_myisam
  
  On the other hand, the following statements are classified as safe:
  
     1 - INSERT INTO t_innodb SELECT * FROM t_myisam_temp
  
     2 - INSERT INTO t_myisam_temp SELECT * FROM t_innodb
sql/sql_class.h:
  It allows to switch from STATEMENT to MIXED/ROW when there are temporary
  tables but the contrary is not possible.
sql/sql_table.cc:
  Fixed the case that a DROP/DROP TEMPORARY that affects a temporary table in MIXED
  mode is written as a DROP TEMPORARY TABLE IF EXISTS because the table may not exist in the slave and due to the IF EXISTS token an error will never happen
  while processing the statement in the slave.
  
  Removed a function that was not being used.

This patch:

- Moves all definitions from the mysql_priv.h file into
  header files for the component where the variable is
  defined
- Creates header files if the component lacks one
- Eliminates all include directives from mysql_priv.h
- Eliminates all circular include cycles
- Rename time.cc to sql_time.cc
- Rename mysql_priv.h to sql_priv.h

When mysqlbinlog was given the --database=X flag, it always printed
'ROLLBACK TO', but the corresponding 'SAVEPOINT' statement was not
printed. The replicated filter(replicated-do/ignore-db) and binlog
filter (binlog-do/ignore-db) has the same problem. They are solved
in this patch together.

After this patch, We always check whether the query is 'SAVEPOINT'
statement or not. Because this is a literal check, 'SAVEPOINT' and
'ROLLBACK TO' statements are also binlogged in uppercase with no
any comments.

The binlog before this patch can be handled correctly except one case
that any comments are in front of the keywords. for example:
 /* bla bla */ SAVEPOINT a;
 /* bla bla */ ROLLBACK TO a;

           debug_max

There was a buffer overrun when unpacking the date
field. Incidentaly, this seems to affect only solaris x86_64
debug builds, but others platforms may be vulnerable as well.

In particular, the buffer size used was not taking into
consideration that the '\0' character would be written into
it.

We fix this by increasing the size of the buffer used to
accommodate one extra byte (the one for the '\0').

           for InnoDB
            
The class Field_bit_as_char stores the metadata for the
field incorrecly because bytes_in_rec and bit_len are set
to (field_length + 7 ) / 8 and 0 respectively, while
Field_bit has the correct values field_length / 8 and
field_length % 8.
            
Solved the problem by re-computing the values for the
metadata based on the field_length instead of using the
bytes_in_rec and bit_len variables.
            
To handle compatibility with old server, a table map
flag was added to indicate that the bit computation is
exact. If the flag is clear, the slave computes the
number of bytes required to store the bit field and
compares that instead, effectively allowing replication
*without conversion* from any field length that require
the same number of bytes to store.


mysql-test/suite/rpl/t/rpl_typeconv_innodb.test:
  Adding test to check compatibility for bit field
  replication when using InnoDB
sql/field.cc:
  Extending compatible_field_size() with flags from
  table map to allow fields to check master info.
sql/field.h:
  Extending compatible_field_size() with flags from
  table map to allow fields to check master info.
sql/log.cc:
  Removing table map flags since they are not used
  outside table map class.
sql/log_event.cc:
  Removing flags parameter from table map constructor
  since it is not used and does not have to be exposed.
sql/log_event.h:
  Adding flag to denote that bit length for bit field type
  is exact and not potentially rounded to even bytes.
sql/rpl_utility.cc:
  Adding fields to table_def to store table map flags.
sql/rpl_utility.h:
  Removing obsolete comment and adding flags to store
  table map flags from master.

Attempts to execute RESET statements within a transaction that
had acquired metadata locks, led to an assertion failure on 
debug servers. This bug didn't cause any problems on release
builds.

The triggered assert is designed to check that caches are not
flushed or reset while having active transactions. It is triggered
if acquired metadata locks exist that are not from LOCK TABLE or
HANDLER statements.

In this case it was triggered by RESET QUERY CACHE while having
an active transaction that had acquired locks. The reason the
assertion was triggered, was that RESET statements, unlike the
similar FLUSH statements, was not causing an implicit commit.

This patch fixes the problem by making sure RESET statements
commit the current transaction before executing. The commit
causes acquired metadata locks to be released, preventing the
assertion from being triggered.

Incompatible change: This patch changes RESET statements so
that they cause an implicit commit.

Test case added to query_cache.test.

This patch prevents system threads and system table accesses from
using user-specified values for "lock_wait_timeout". Instead all
such accesses are done using the default value (1 year).

This prevents background tasks (such as replication, events, 
accessing stored function definitions, logging, reading time-zone
information, etc.) from failing in cases where the global value
of "lock_wait_timeout" is set very low.

The patch also simplifies the open tables API. Rather than adding
another convenience function for opening and locking system tables,
this patch removes most of the existing convenience functions for
open_and_lock_tables_derived(). Before, open_and_lock_tables() was
a convenience function that enforced derived tables handling, while
open_and_lock_tables_derived() was the main function where derived
tables handling was optional. Now, this convencience function is
gone and the main function is renamed to open_and_lock_tables(). 

No test case added as it would have required the use of --sleep to
check that system threads and system tables have a different timeout
value from the user-specified "lock_wait_timeout" system variable.

MySQL with gcc 4.3.2
      
This is the final patch in the context of this bug. 

cmd-line-utils/readline/rlmbutil.h:
  Changed in a previous patch, reverted by a backport.
cmd-line-utils/readline/text.c:
  Static var initialization.
extra/yassl/include/yassl_error.hpp:
  SetErrorString handles errors outside of the YasslError
  enum.
extra/yassl/src/ssl.cpp:
  SetErrorString handles errors outside of the YasslError
  enum.
extra/yassl/src/yassl_error.cpp:
  SetErrorString handles errors outside of the YasslError
  enum.

I found three issues during the analysis:
 1. Memory leak caused by temp_buf not being freed;
 2. Memory leak caused when handling argv;
 3. Conditional jump that depended on unitialized values.

Issue #1
--------

  DESCRIPTION: when mysqlbinlog is reading from a remote location
  the event temp_buf references the incoming stream (in NET
  object), which is not freed by mysqlbinlog explicitly. On the
  other hand, when it is reading local binary log, it points to a
  temporary buffer that needs to be explicitly freed. For both
  cases, the temp_buf was not freed by mysqlbinlog, instead was
  set to 0.  This clearly disregards the free required in the
  second case, thence creating a memory leak.

  FIX: we make temp_buf to be conditionally freed depending on
  the value of remote_opt. Found out that similar fix is already
  in most recent codebases.

Issue #2 
--------

  DESCRIPTION: load_defaults is called by parse_args, and it
  reads default options from configuration files and put them
  BEFORE the arguments that are already in argc and argv. This is
  done resorting to MEM_ROOT. However, parse_args calls
  handle_options immediately after which changes argv. Later when
  freeing the defaults, pointers to MEM_ROOT won't match, causing
  the memory not to be freed:

  void free_defaults(char **argv)
  {
    MEM_ROOT ptr
    memcpy_fixed((char*) &ptr,(char *) argv - sizeof(ptr), sizeof(ptr));
    free_root(&ptr,MYF(0));
  }

  FIX: we remove load_defaults from parse_args and call it
  before. Then we save argv with defaults in defaults_argv BEFORE
  calling parse_args (which inside can then call handle_options
  at will). Actually, found out that this is in fact kind of a
  backport for BUG#38468 into 5.1, so I merged in the test case
  as well and added error check for load_defaults call.

  Fix based on:
  revid:zhenxing.he@sun.com-20091002081840-uv26f0flw4uvo33y


Issue #3 
--------

  DESCRIPTION: the structure st_print_event_info constructor
  would not initialize the sql_mode member, although it did for
  sql_mode_inited (set to false). This would later raise the
  warning in valgrind when printing the sql_mode in the event
  header, as this print out is protected by a check against
  sql_mode_inited and sql_mode variables. Given that sql_mode was
  not initialized valgrind would output the warning.

  FIX: we add initialization of sql_mode to the
  st_print_event_info constructor.
 

client/mysqlbinlog.cc:
  - Conditionally free ev->temp_buf.
  - save defaults_argv before handle_options is called.
mysql-test/t/mysqlbinlog.test:
  Added test case from BUG#38468.
sql/log_event.cc:
  Added initialization of sql_mode for st_print_event_info.

--slave-load-tm
      
The MDL_SHARED lock was introduced for an object in 5.4, but the 'TABLE_LIST' 
object was not initialized with the MDL_SHARED lock when applying event with 
LOAD DATA INFILE into table. So the failure is caused when checking the 
MDL_SHARED lock for the object.
To fix the problem, the 'TABLE_LIST' object was initialized with the MDL_SHARED 
lock when applying event with LOAD DATA INFILE into table.


mysql-test/suite/rpl/t/disabled.def:
  Got rid of the line for enabling 'rpl_cross_version' test.

into slow log
      
While processing a statement, down the mysql_parse execution
stack, the thd->enable_slow_log can be assigned to
opt_log_slow_admin_statements, depending whether one is executing
administrative statements, such as ALTER TABLE, OPTIMIZE,
ANALYZE, etc, or not. This can have an impact on slow logging for
statements that are executed after an administrative statement
execution is completed.
      
When executing statements directly from the user this is fine
because, the thd->enable_slow_log is reset right at the beginning
of the dispatch_command function, ie, everytime a new statement
is set is set to execute.
      
On the other hand, for slave SQL thread (sql_thd) the story is a
bit different. When in SBR the sql_thd applies statements by
calling mysql_parse. Right after, it calls log_slow_statement
function to log them if they take too long. Calling mysql_parse
directly is fine, but also means that dispatch_command function
is bypassed. As a consequence, thd->enable_slow_log does not get
a chance to be reset before the next statement to be executed by
the sql_thd. If the statement just executed by the sql_thd was an
administrative statement and logging of admin statements was
disabled, this means that sql_thd->enable_slow_log will be set to
0 (disabled) from that moment on. End result: sql_thd stops
logging slow statements.
      
We fix this by resetting the value of sql_thd->enable_slow_log to
the value of opt_log_slow_slave_statements right after
log_slow_stement is called by the sql_thd.

Cherry-pick a fix Bug#37148 from next-mr, to preserve
file ids of the added files, and ensure that all the necessary
changes have been pulled.

Since initially Bug#37148 was null-merged into 6.0,
the changeset that is now being cherry-picked was likewise
null merged into next-4284.

Now that Bug#37148 has been reapplied to 6.0, try to make
it work with next-4284. This is also necessary to be able
to pull other changes from 5.1-rep into next-4284.

To resolve the merge issues use this changeset applied
to 6.0:
revid:jperkin@sun.com-20091216103628-ylhqf7s6yegui2t9
revno: 3776.1.1
committer: He Zhenxing <zhenxing.he@sun.com>
branch nick: 6.0-codebase-bugfixing
timestamp: Thu 2009-12-17 17:02:50 +0800
message:
  Fix merge problem with Bug#37148

of the TABLE_LIST instance, which did not initialize the 
MDL request properly.
Use init_one_table() to initialize the MDL request.

Rename method as to not hide a base.
Reorder attributes initialization.
Remove unused variable.
Rework code to silence a warning due to assignment used as truth value.


sql/item_strfunc.cc:
  Rename method as to not hide a base.
sql/item_strfunc.h:
  Rename method as to not hide a base.
sql/log_event.cc:
  Reorder attributes initialization.
sql/rpl_injector.cc:
  Rework code to silence a warning due to assignment used as truth value.
sql/rpl_record.cc:
  Remove unused variable.
sql/sql_db.cc:
  Rework code to silence a warning due to assignment used as truth value.
sql/sql_parse.cc:
  Rework code to silence a warning due to assignment used as truth value.
sql/sql_table.cc:
  Rework code to silence a warning due to assignment used as truth value.

When replicating from 4.1 master to 5.0 slave START SLAVE UNTIL can stop too late.
The necessary in calculating of the beginning of an event the event's length
did not correspond to the master's genuine information at the event's execution time.
That piece of info was changed at the event's relay-logging due to binlog_version<4 event
conversion by IO thread.

Fixed with storing the master genuine Query_log_event size into a new status
variable at relay-logging of the event. The stored info is extacted at the event
execution and participate further to caclulate the correct start position of the event
in the until-pos stopping routine.
The new status variable's algorithm will be only active when the event comes
from the master of version < 5.0 (binlog_version < 4).


mysql-test/r/rpl_until.result:
  results changed.
mysql-test/std_data/bug47142_master-bin.000001:
  a binlog from 4.1 master to replace one of the running 5.x master is added as 
  part of Bug #47142 regression test.
mysql-test/t/rpl_until.test:
  Regression test for Bug #47142 is added.
sql/log_event.cc:
  Storing and extracting the master's genuine size of the event from the status
  var of the event packet header.
  The binlog_version<4 query-log-event is 
  a. converted into the modern binlog_version==4 to store the original size of the event
     into a new status var; the converted representation goes into the relay log.
  b. the converted event is read out and the stored size is engaged in the start pos calculation.
  
  The new status is active only for events that IO thread instantiates for the sake of the conversion.
sql/log_event.h:
  Incrementing the max szie of MAX_SIZE_LOG_EVENT_STATUS because of the new status var;
  Defining the new status variable to hold the master's genuine event size;
  Augmenting the Query_log_event with a new member to hold a value to store/extact from the status
  var of the event packet header.

enabled binary

The test case injects an error in the server by deleting the
temporary file that it uses during the load data statement
execution. The error consisted of closing, deleting and setting
the file descriptor to -1 right before calling mysql_file_write.

Although, this error injection seems to work OK in Unix like
environments, in Windows, this would cause the server to hit an
assertion in 'my_get_open_flags':

  DBUG_ASSERT(fd >= MY_FILE_MIN && fd < (int)my_file_limit)

We fix this by changing the error injection to just call the
macro my_delete_allow_opened, instead of the close + delete + set
fd=-1. The macro deletes the file and is platform
independent. Additionally, this required some changes to how the
assertion is handled in the test case to make it cope with this
change.

PB2 run uncovered issue that needs further analysis.

variable

Incremental commit: 
  1. Moved part of the test case to binlog suite.
  2. Removed cast set when writing the flags field.

variable

The User_var_log_event was not serializing the unsigned
flag. This would cause the slave to always assume signed values.

We fix this by extending the User_var_log_event to also contain
information on the unsigned_flag, meaning that it gets into the
binlog as well, therefore the slave will get this information as
well. Events without information on unsigned flag (old events)
are treated as they were before (always signed: unsigned_flag=
FALSE).

The information on the unsigned_flag, is shipped in an extra byte
appended to the end of the User_var_log_event and added by this
patch. This extra byte holds values for general purpose
User_var_log_event flags which are now packed in the binlog as
well. One of these flags contains information about whether the
value is signed or unsigned (currently this extra byte is only
used to hold data on the unsigned flag, in the future we can use
it to pack extra flags if there is the need to).

BUG#49481: RBR: MyISAM and bit fields may cause slave to stop on delete: 
cant find record
      
BUG#49482: RBR: Replication may break on deletes when MyISAM tables + 
char field are used

When using MyISAM tables, despite the fact that the null bit is
set for some fields, their old value is still in the row. This
can cause the comparison of records to fail when the slave is
doing an index or range scan.

We fix this by avoiding memcmp for MyISAM tables when comparing
records. Additionally, when comparing field by field, we first
check if both fields are not null and if so, then we compare
them. If just one field is null we return failure immediately. If
both fields are null, we move on to the next field.

Part IV: sql instrumentation

Fixed valgrind failure on PB2.

sql/log_event.cc:
  Added code to fix valgrind failure on PB2.

Fixed valgrind failure on PB2.

sql/log_event.cc:
  Added code to fix valgrind failure on PB2.

For tables with metadata sizes ranging from 251 to 255 the size
of the event data (m_data_size) was being improperly calculated
in the Table_map_log_event constructor. This was due to the fact
that when writing the Table_map_log_event body (in
Table_map_log_event::write_data_body) a call to net_store_length
is made for packing the m_field_metadata_size. It happens that
net_store_length uses *one* byte for storing
m_field_metadata_size when it is smaller than 251 but *three*
bytes when it exceeds that value. BUG 42749 had already
pinpointed and fix this fact, but the fix was incomplete, as the
calculation in the Table_map_log_event constructor considers 255
instead of 251 as the threshold to increment m_data_size by
three. Thence, the window for having a mismatch between the
number of bytes written and the number of bytes accounted in the
event length (m_data_size) was left open for
m_field_metadata_size values between 251 and 255.

We fix this by changing the condition in the Table_map_log_event
constructor to match the one in the net_store_length, ie,
increment one byte if m_field_metadata_size < 251 and three if it
exceeds this value.

mysql-test/suite/rpl/r/rpl_row_tbl_metadata.result:
  Updated result file.
mysql-test/suite/rpl/t/rpl_row_tbl_metadata.test:
  Changes to the original test case: added slave and moved
  file into the rpl suite.
  
  New test case: replicates two tables one with 250 and 
  another with 252 metadata sizes. This exercises the usage
  of 1 or 3 bytes while packing the m_field_metadata_size.
sql/log_event.cc:
  Made the m_data_size calculation for the table map log event
  to match the number of bytes used while packing the 
  m_field_metadata_size value (according to net_store_length
  function in pack.c).

      
In statement-based or mixed-mode replication, use DROP TEMPORARY TABLE
to drop multiple tables causes different errors on master and slave, 
when one or more of these tables do not exist. Because when executed
on slave, it would automatically add IF EXISTS to the query to ignore
all ER_BAD_TABLE_ERROR errors.

To fix the problem, do not add IF EXISTS when executing DROP TEMPORARY
TABLE on the slave, and clear the ER_BAD_TABLE_ERROR error after
execution if the query does not expect any errors.


mysql-test/r/rpl_drop_temp.result:
  Updated for the patch of bug#49137.
mysql-test/t/rpl_drop_temp.test:
  Added the test file to verify if DROP MULTI TEMPORARY TABLE
  will cause different errors on master and slave, when one or
  more of these tables do not exist.
sql/log_event.cc:
  Added code to handle above cases which are
  removed from sql_parse.cc
sql/sql_parse.cc:
  Remove the code to issue the 'Unknown table' error,
  if the temporary table does not exist when dropping
  it on slave. The above cases decribed in comments
  will be handled later in log_event.cc.

                      to string conversions and vice versa" 
Initial import of the dtoa.c code and custom wrappers around it 
to allow its usage from the server code. 
 
Conversion of FLOAT/DOUBLE values to DECIMAL ones or strings 
and vice versa has been significantly reworked. As the new 
algoritms are more precise than the older ones, results of such 
conversions may not always match those obtained from older 
server versions. This in turn may break compatibility for some 
applications. 
 
This patch also fixes the following bugs: 
- bug #12860 "Difference in zero padding of exponent between 
Unix and Windows" 
- bug #21497 "DOUBLE truncated to unusable value" 
- bug #26788 "mysqld (debug) aborts when inserting specific 
numbers into char fields" 
- bug #24541 "Data truncated..." on decimal type columns 
without any good reason" 

"HANDLER statements within a transaction might lead to deadlocks".
Introduce a notion of a sentinel to MDL_context. A sentinel
is a ticket that separates all tickets in the context into two
groups: before and after it. Currently we can have (and need) only
one designated sentinel -- it separates all locks taken by LOCK
TABLE or HANDLER statement, which must survive COMMIT and ROLLBACK
and all other locks, which must be released at COMMIT or ROLLBACK.
The tricky part is maintaining the sentinel up to date when
someone release its corresponding ticket. This can happen, e.g.
if someone issues DROP TABLE under LOCK TABLES (generally,
see all calls to release_all_locks_for_name()).
MDL_context::release_ticket() is modified to take care of it.

******
A fix and a test case for Bug#46224 "HANDLER statements within a
transaction might lead to deadlocks".

An attempt to mix HANDLER SQL statements, which are transaction-
agnostic, an open multi-statement transaction,
and DDL against the involved tables (in a concurrent connection) 
could lead to a deadlock. The deadlock would occur when
HANDLER OPEN or HANDLER READ would have to wait on a conflicting
metadata lock. If the connection that issued HANDLER statement
also had other metadata locks (say, acquired in scope of a 
transaction), a classical deadlock situation of mutual wait
could occur.

Incompatible change: entering LOCK TABLES mode automatically
closes all open HANDLERs in the current connection.

Incompatible change: previously an attempt to wait on a lock
in a connection that has an open HANDLER statement could wait
indefinitely/deadlock. After this patch, an error ER_LOCK_DEADLOCK
is produced.

The idea of the fix is to merge thd->handler_mdl_context
with the main mdl_context of the connection, used for transactional
locks. This makes deadlock detection possible, since all waits
with locks are "visible" and available to analysis in a single
MDL context of the connection.

Since HANDLER locks and transactional locks have a different life
cycle -- HANDLERs are explicitly open and closed, and so
are HANDLER locks, explicitly acquired and released, whereas
transactional locks "accumulate" till the end of a transaction
and are released only with COMMIT, ROLLBACK and ROLLBACK TO SAVEPOINT,
a concept of "sentinel" was introduced to MDL_context.
All locks, HANDLER and others, reside in the same linked list.
However, a selected element of the list separates locks with
different life cycle. HANDLER locks always reside at the
end of the list, after the sentinel. Transactional locks are
prepended to the beginning of the list, before the sentinel.
Thus, ROLLBACK, COMMIT or ROLLBACK TO SAVEPOINT, only
release those locks that reside before the sentinel. HANDLER locks
must be released explicitly as part of HANDLER CLOSE statement,
or an implicit close. 
The same approach with sentinel
is also employed for LOCK TABLES locks. Since HANDLER and LOCK TABLES
statement has never worked together, the implementation is
made simple and only maintains one sentinel, which is used either
for HANDLER locks, or for LOCK TABLES locks.


mysql-test/include/handler.inc:
  Add test coverage for Bug#46224 "HANDLER statements within a
  transaction might lead to deadlocks".
  Extended HANDLER coverage to cover a mix of HANDLER, transactions
  and DDL statements.
mysql-test/r/handler_innodb.result:
  Update results (Bug#46224).
mysql-test/r/handler_myisam.result:
  Update results (Bug#46224).
sql/lock.cc:
  Remove thd->some_tables_deleted, it's never used.
sql/log_event.cc:
  No need to check for thd->locked_tables_mode, 
  it's done inside release_transactional_locks().
sql/mdl.cc:
  Implement the concept of HANDLER and LOCK TABLES "sentinel".
  Implement a method to clone an acquired ticket.
  Do not return tickets beyond the sentinel when acquiring
  locks, create a copy.
  Remove methods to merge and backup MDL_context, they are now
  not used (Hurra!). This opens a path to a proper constructor
  and destructor of class MDL_context (to be done in a separate
  patch).
  Modify find_ticket() to provide information about where
  the ticket position is with regard to the sentinel.
sql/mdl.h:
  Add declarations necessary for the implementation of the concept
  of "sentinel", a dedicated ticket separating transactional and
  non-transactional locks.
sql/mysql_priv.h:
  Add mark_tmp_table_for_reuse() declaration, 
  a function to "close" a single session (temporary) table.
sql/sql_base.cc:
  Remove thd->some_tables_deleted.
  Modify deadlock-prevention asserts and deadlock detection
  heuristics to take into account that from now on HANDLER locks
  reside in the same locking context.
  Add broadcast_refresh() to mysql_notify_thread_having_shared_lock():
  this is necessary for the case when a thread having a shared lock
  is asleep in tdc_wait_for_old_versions(). This situation is only
  possible with HANDLER t1 OPEN; FLUSH TABLE (since all over code paths
  that lead to tdc_wait_for_old_versions() always have an
  empty MDL_context). Previously the server would simply deadlock
  in this situation.
sql/sql_class.cc:
  Remove now unused member "THD::some_tables_deleted". 
  Move mysql_ha_cleanup() a few lines above in THD::cleanup() 
  to make sure that all handlers are closed when it's time to 
  destroy the MDL_context of this connection.
  Remove handler_mdl_context and handler_tables.
sql/sql_class.h:
  Remove THD::handler_tables, THD::handler_mdl_context,
  THD::some_tables_deleted.
sql/sql_handler.cc:
  Remove thd->handler_tables.
  Remove thd->handler_mdl_context.
  Rewrite mysql_ha_open() to have no special provision for MERGE
  tables, now that we don't have to manipulate with thd->handler_tables
  it's easy to do.
  Remove dead code.
  Fix a bug in mysql_ha_flush() when we would always flush
  a temporary HANDLER when mysql_ha_flush() is called (actually
  mysql_ha_flush() never needs to flush temporary tables).
sql/sql_insert.cc:
  Update a comment, no more thd->some_tables_deleted.
sql/sql_parse.cc:
  Implement an incompatible change: entering LOCK TABLES closes
  active HANDLERs, if any.
  Now that we have a sentinel, we don't need to check
  for thd->locked_tables_mode when releasing metadata locks in
  COMMIT/ROLLBACK.
sql/sql_plist.h:
  Add new (now necessary) methods to the list class.
sql/sql_prepare.cc:
  Make sure we don't release HANDLER locks when rollback to a
  savepoint, set to not keep locks taken at PREPARE.
sql/sql_servers.cc:
  Update to a new signature of MDL_context::release_all_locks().
sql/sql_table.cc:
  Remove thd->some_tables_deleted.
sql/transaction.cc:
  Add comments. 
  Make sure rollback to (MDL) savepoint works under LOCK TABLES and
  with HANDLER tables.

Bug#16565 mysqld --help --verbose does not order variablesBug#20413 sql_slave_skip_counter is not shown in show variables
Bug#20415 Output of mysqld --help --verbose is incomplete
Bug#25430 variable not found in SELECT @@global.ft_max_word_len;
Bug#32902 plugin variables don't know their names
Bug#34599 MySQLD Option and Variable Reference need to be consistent in formatting!
Bug#34829 No default value for variable and setting default does not raise error
Bug#34834 ? Is accepted as a valid sql mode
Bug#34878 Few variables have default value according to documentation but error occurs  
Bug#34883 ft_boolean_syntax cant be assigned from user variable to global var.
Bug#37187 `INFORMATION_SCHEMA`.`GLOBAL_VARIABLES`: inconsistent status
Bug#40988 log_output_basic.test succeeded though syntactically false.
Bug#41010 enum-style command-line options are not honoured (maria.maria-recover fails)
Bug#42103 Setting key_buffer_size to a negative value may lead t...

         replicating

Fixes to make mysqlbinlog work correctly and updates
to rpl_ndb result files.

sql/log_event.cc:
  The processing of the MYSQL_STRING_TYPE is now done
  in table_def::type(), so the support functions for
  mysqlbinlog are changed to work correctly.

           for InnoDB

The class Field_bit_as_char stores the metadata for the
field incorrecly because bytes_in_rec and bit_len are set
to (field_length + 7 ) / 8 and 0 respectively, while
Field_bit has the correct values field_length / 8 and
field_length % 8.

Solved the problem by re-computing the values for the
metadata based on the field_length instead of using the
bytes_in_rec and bit_len variables.

To handle compatibility with old server, a table map
flag was added to indicate that the bit computation is
exact. If the flag is clear, the slave computes the
number of bytes required to store the bit field and
compares that instead, effectively allowing replication
*without conversion* from any field length that require
the same number of bytes to store.

mysql-test/suite/rpl/t/rpl_typeconv_innodb.test:
  Adding test to check compatibility for bit field
  replication when using InnoDB.
sql/field.cc:
  Extending compatible_field_size() with flags from
  table map to allow fields to check master info.
sql/field.h:
  Extending compatible_field_size() with flags from
  table map to allow fields to check master info.
sql/log.cc:
  Removing table map flags since they are not used
  outside table map class.
sql/log_event.cc:
  Removing flags parameter from table map constructor
  since it is not used and does not have to be exposed.
sql/log_event.h:
  Adding flag to denote that bit length for bit field type
  is exact and not potentially rounded to even bytes.
sql/rpl_utility.cc:
  Adding fields to table_def to store table map flags.
sql/rpl_utility.h:
  Removing obsolete comment and adding flags to store
  table map flags from master.

'LOAD DATA CONCURRENT [LOCAL] INFILE ...' statment only is binlogged as
'LOAD DATA [LOCAL] INFILE ...' in SBR and MBR.  As a result, if replication is on, 
queries on slaves will be blocked by the replication SQL thread.

This patch write code to write 'CONCURRENT' into the log event if 'CONCURRENT' option
is in the original statement in SBR and MBR. 

Row-based replication requires the types of columns on the
master and slave to be approximately the same (some safe
conversions between strings are allowed), but does not
allow safe conversions between fields of similar types such
as TINYINT and INT.

This patch implement type conversions between similar fields
on the master and slave.

The conversions are controlled using a new variable
SLAVE_TYPE_CONVERSIONS of type SET('ALL_LOSSY','ALL_NON_LOSSY').

Non-lossy conversions are any conversions that do not run the
risk of losing any information, while lossy conversions can
potentially truncate the value. The column definitions are
checked to decide if the conversion is acceptable.

If neither conversion is enabled, it is required that the
definitions of the columns are identical on master and slave.

Conversion is done by creating an internal conversion table,
unpacking the master data into it, and then copy the data to
the real table on the slav...

----------------------------------------------------------
revno: 2617.69.20
committer: Konstantin Osipov <kostja@sun.com>
branch nick: 5.4-4284-1-assert
timestamp: Thu 2009-08-13 18:29:55 +0400
message:
  WL#4284 "Transactional DDL locking"
  A review fix.
  Since WL#4284 implementation separated MDL_request and MDL_ticket,
  MDL_request becamse a utility object necessary only to get a ticket.
  Store it by-value in TABLE_LIST with the intent to merge
  MDL_request::key with table_list->table_name and table_list->db
  in future.
  Change the MDL subsystem to not require MDL_requests to
  stay around till close_thread_tables().
  Remove the list of requests from the MDL context.
  Requests for shared metadata locks acquired in open_tables()
  are only used as a list in recover_from_failed_open_table_attempt(),
  which calls mdl_context.wait_for_locks() for this list.
  To keep such list for recover_from_failed_open_table_attempt(),
  introduce a context class (Open_table_context), that collects
  all requests.
  A lot of minor cleanups and simplications that became possible
  with this change.


sql/event_db_repository.cc:
  Remove alloc_mdl_requests(). Now MDL_request instance is a member
  of TABLE_LIST, and init_one_table() initializes it.
sql/ha_ndbcluster_binlog.cc:
  Remove now unnecessary declaration and initialization
  of binlog_mdl_request.
sql/lock.cc:
  No need to allocate MDL requests in lock_table_names() now.
sql/log.cc:
  Use init_one_table() method, remove alloc_mdl_requests(),
  which is now unnecessary.
sql/log_event.cc:
  No need to allocate mdl_request separately now.
  Use init_one_table() method.
sql/log_event_old.cc:
  Update to the new signature of close_tables_for_reopen().
sql/mdl.cc:
  Update try_acquire_exclusive_lock() to be more easy to use.
  Function lock_table_name_if_not_cached() has been removed.
  Make acquire_shared_lock() signature consistent with
  try_acquire_exclusive_lock() signature.
  Remove methods that are no longer used.
  Update comments.
sql/mdl.h:
  Implement an assignment operator that doesn't
  copy MDL_key (MDL_key::operator= is private and
  should remain private).
  This is a hack to work-around assignment of TABLE_LIST
  by value in several places. Such assignments violate
  encapsulation, since only perform a shallow copy.
  In most cases these assignments are a hack on their own.
sql/mysql_priv.h:
  Update signatures of close_thread_tables() and close_tables_for_reopen().
sql/sp.cc:
  Allocate TABLE_LIST in thd->mem_root.
  Use init_one_table().
sql/sp_head.cc:
  Use init_one_table().
  Remove thd->locked_tables_root, it's no longer needed.
sql/sql_acl.cc:
  Use init_mdl_requests() and init_one_table().
sql/sql_base.cc:
  Update to new signatures of try_acquire_shared_lock() and
  try_acquire_exclusive_lock().
  Remove lock_table_name_if_not_cached().
  Fix a bug in open_ltable() that would not return ER_LOCK_DEADLOCK
  in case of a failed lock_tables() and a multi-statement
  transaction.
  Fix a bug in open_and_lock_tables_derived() that would
  not return ER_LOCK_DEADLOCK in case of a multi-statement
  transaction and a failure of lock_tables().
  Move assignment of enum_open_table_action to a method of Open_table_context, a new class that maintains information
  for backoff actions.
  Minor rearrangements of the code.
  Remove alloc_mdl_requests() in functions that work with system
  tables: instead the patch ensures that callers always initialize
  TABLE_LIST argument.
sql/sql_class.cc:
  THD::locked_tables_root is no more.
sql/sql_class.h:
  THD::locked_tables_root is no more.
  Add a declaration for Open_table_context class.
sql/sql_delete.cc:
  Update to use the simplified MDL API.
sql/sql_handler.cc:
  TABLE_LIST::mdl_request is stored by-value now.
  Ensure that mdl_request.ticket is NULL for every request
  that is passed into MDL, to satisfy MDL asserts.
  @ sql/sql_help.cc
  Function open_system_tables_for_read() no longer initializes
  mdl_requests.
  Move TABLE_LIST::mdl_request initialization closer to
  TABLE_LIST initialization.
sql/sql_help.cc:
  Function open_system_tables_for_read() no longer initializes
  mdl_requests.
  Move TABLE_LIST::mdl_request initialization closer to
  TABLE_LIST initialization.
sql/sql_insert.cc:
  Remove assignment by-value of TABLE_LIST in
  TABLEOP_HOOKS. We can't carry over a granted
  MDL ticket from one table list to another.
sql/sql_parse.cc:
  Change alloc_mdl_requests() -> init_mdl_requests().
  @todo We can remove init_mdl_requests() altogether
  in some places: all places that call add_table_to_list()
  already have mdl requests initialized.
sql/sql_plugin.cc:
  Use init_one_table().
  THD::locked_tables_root is no more.
sql/sql_servers.cc:
  Use init_one_table().
sql/sql_show.cc:
  Update acquire_high_priority_shared_lock() to use
  TABLE_LIST::mdl_request rather than allocate an own.
  Fix get_trigger_table_impl() to use init_one_table(),
  check for out of memory, follow the coding style.
sql/sql_table.cc:
  Update to work with TABLE_LIST::mdl_request by-value.
  Remove lock_table_name_if_not_cached().
  The code that used to delegate to it is quite simple and
  concise without it now.
sql/sql_udf.cc:
  Use init_one_table().
sql/sql_update.cc:
  Update to use the new signature of close_tables_for_reopen().
sql/table.cc:
  Move re-setting of mdl_requests for prepared statements
  and stored procedures from close_thread_tables() to
  reinit_stmt_before_use().
  Change alloc_mdl_requests() to init_mdl_requests().
  init_mdl_requests() is a hack that can't be deleted
  until we don't have a list-aware TABLE_LIST constructor.
  Hopefully its use will be minimal
sql/table.h:
  Change alloc_mdl_requests() to init_mdl_requests()
  TABLE_LIST::mdl_request is stored by value.
sql/tztime.cc:
  We no longer initialize mdl requests in open_system_tables_for*()
  functions. Move this initialization closer to initialization
  of the rest of TABLE_LIST members.
storage/myisammrg/ha_myisammrg.cc:
  Simplify mdl_request initialization.

2617.31.12, 2617.31.15, 2617.31.15, 2617.31.16, 2617.43.1
- initial changeset that introduced the fix for 
Bug#989 and follow up fixes for all test suite failures
introduced in the initial changeset. 
------------------------------------------------------------
revno: 2617.31.1
committer: Davi Arnaut <Davi.Arnaut@Sun.COM>
branch nick: 4284-6.0
timestamp: Fri 2009-03-06 19:17:00 -0300
message:
Bug#989: If DROP TABLE while there's an active transaction, wrong binlog order
WL#4284: Transactional DDL locking

Currently the MySQL server does not keep metadata locks on
schema objects for the duration of a transaction, thus failing
to guarantee the integrity of the schema objects being used
during the transaction and to protect then from concurrent
DDL operations. This also poses a problem for replication as
a DDL operation might be replicated even thought there are
active transactions using the object being modified.

The solution is to defer the release of metadata locks until
a active transaction is either committed or rolled back. This
prevents other statements from modifying the table for the
entire duration of the transaction. This provides commitment
ordering for guaranteeing serializability across multiple
transactions.

- Incompatible change:

If MySQL's metadata locking system encounters a lock conflict,
the usual schema is to use the try and back-off technique to
avoid deadlocks -- this schema consists in releasing all locks
and trying to acquire them all in one go.

But in a transactional context this algorithm can't be utilized
as its not possible to release locks acquired during the course
of the transaction without breaking the transaction commitments.
To avoid deadlocks in this case, the ER_LOCK_DEADLOCK will be
returned if a lock conflict is encountered during a transaction.

Let's consider an example:

A transaction has two statements that modify table t1, then table
t2, and then commits. The first statement of the transaction will
acquire a shared metadata lock on table t1, and it will be kept
utill COMMIT to ensure serializability.

At the moment when the second statement attempts to acquire a
shared metadata lock on t2, a concurrent ALTER or DROP statement
might have locked t2 exclusively. The prescription of the current
locking protocol is that the acquirer of the shared lock backs off
-- gives up all his current locks and retries. This implies that
the entire multi-statement transaction has to be rolled back.

- Incompatible change:

FLUSH commands such as FLUSH PRIVILEGES and FLUSH TABLES WITH READ
LOCK won't cause locked tables to be implicitly unlocked anymore.


mysql-test/extra/binlog_tests/drop_table.test:
  Add test case for Bug#989.
mysql-test/extra/binlog_tests/mix_innodb_myisam_binlog.test:
  Fix test case to reflect the fact that transactions now hold
  metadata locks for the duration of a transaction.
mysql-test/include/mix1.inc:
  Fix test case to reflect the fact that transactions now hold
  metadata locks for the duration of a transaction.
mysql-test/include/mix2.inc:
  Fix test case to reflect the fact that transactions now hold
  metadata locks for the duration of a transaction.
mysql-test/r/flush_block_commit.result:
  Update test case result (WL#4284).
mysql-test/r/flush_block_commit_notembedded.result:
  Update test case result (WL#4284).
mysql-test/r/innodb.result:
  Update test case result (WL#4284).
mysql-test/r/innodb_mysql.result:
  Update test case result (WL#4284).
mysql-test/r/lock.result:
  Add test case result for an effect of WL#4284/Bug#989
  (all locks should be released when a connection terminates).
mysql-test/r/mix2_myisam.result:
  Update test case result (effects of WL#4284/Bug#989).
mysql-test/r/not_embedded_server.result:
  Update test case result (effects of WL#4284/Bug#989).
  Add a test case for interaction of WL#4284 and FLUSH PRIVILEGES.
mysql-test/r/partition_innodb_semi_consistent.result:
  Update test case result (effects of WL#4284/Bug#989).
mysql-test/r/partition_sync.result:
  Temporarily disable the test case for Bug#43867,
  which will be fixed by a subsequent backport.
mysql-test/r/ps.result:
  Add a test case for effect of PREPARE on transactional
  locks: we take a savepoint at beginning of PREAPRE
  and release it at the end. Thus PREPARE does not 
  accumulate metadata locks (Bug#989/WL#4284).
mysql-test/r/read_only_innodb.result:
  Update test case result (effects of WL#4284/Bug#989).
mysql-test/suite/binlog/r/binlog_row_drop_tbl.result:
  Add a test case result (WL#4284/Bug#989).
mysql-test/suite/binlog/r/binlog_row_mix_innodb_myisam.result:
  Update test case result (effects of WL#4284/Bug#989).
mysql-test/suite/binlog/r/binlog_stm_drop_tbl.result:
  Add a test case result (WL#4284/Bug#989).
mysql-test/suite/binlog/r/binlog_stm_mix_innodb_myisam.result:
  Update test case result (effects of WL#4284/Bug#989).
mysql-test/suite/binlog/r/binlog_unsafe.result:
  A side effect of Bug#989 -- slightly different table map ids.
mysql-test/suite/binlog/t/binlog_row_drop_tbl.test:
  Add a test case for WL#4284/Bug#989.
mysql-test/suite/binlog/t/binlog_stm_drop_tbl.test:
  Add a test case for WL#4284/Bug#989.
mysql-test/suite/binlog/t/binlog_stm_row.test:
  Update to the new state name. This
  is actually a follow up to another patch for WL#4284, 
  that changes Locked thread state to Table lock.
mysql-test/suite/ndb/r/ndb_index_ordered.result:
  Remove result for disabled part of the test case.
mysql-test/suite/ndb/t/disabled.def:
  Temporarily disable a test case (Bug#45621).
mysql-test/suite/ndb/t/ndb_index_ordered.test:
  Disable a part of a test case (needs update to
  reflect semantics of Bug#989).
mysql-test/suite/rpl/t/disabled.def:
  Disable tests made meaningless by transactional metadata
  locking.
mysql-test/suite/sys_vars/r/autocommit_func.result:
  Add a commit (Bug#989).
mysql-test/suite/sys_vars/t/autocommit_func.test:
  Add a commit (Bug#989).
mysql-test/t/flush_block_commit.test:
  Fix test case to reflect the fact that transactions now hold
  metadata locks for the duration of a transaction.
mysql-test/t/flush_block_commit_notembedded.test:
  Fix test case to reflect the fact that transactions now hold
  metadata locks for the duration of a transaction.
  Add a test case for transaction-scope locks and the global
  read lock (Bug#989/WL#4284).
mysql-test/t/innodb.test:
  Fix test case to reflect the fact that transactions now hold
  metadata locks for the duration of a transaction
  (effects of Bug#989/WL#4284).
mysql-test/t/lock.test:
  Add a test case for Bug#989/WL#4284.
mysql-test/t/not_embedded_server.test:
  Add a test case for Bug#989/WL#4284.
mysql-test/t/partition_innodb_semi_consistent.test:
  Replace TRUNCATE with DELETE, to not issue
  an implicit commit of a transaction, and not depend
  on metadata locks.
mysql-test/t/partition_sync.test:
  Temporarily disable the test case for Bug#43867,
  which needs a fix to be backported from 6.0.
mysql-test/t/ps.test:
  Add a test case for semantics of PREPARE and transaction-scope
  locks: metadata locks on tables used in PREPARE are enclosed into a
  temporary savepoint, taken at the beginning of PREPARE,
  and released at the end. Thus PREPARE does not effect
  what locks a transaction owns.
mysql-test/t/read_only_innodb.test:
  Fix test case to reflect the fact that transactions now hold
  metadata locks for the duration of a transaction 
  (Bug#989/WL#4284).
  
  Wait for the read_only statement to actually flush tables before
  sending other concurrent statements that depend on its state.
mysql-test/t/xa.test:
  Fix test case to reflect the fact that transactions now hold
  metadata locks for the duration of a transaction 
  (Bug#989/WL#4284).
sql/ha_ndbcluster_binlog.cc:
  Backport bits of changes of ha_ndbcluster_binlog.cc
  from 6.0, to fix the failing binlog test suite with
  WL#4284. WL#4284 implementation does not work
  with 5.1 implementation of ndbcluster binlog index.
sql/log_event.cc:
  Release metadata locks after issuing a commit.
sql/mdl.cc:
  Style changes (WL#4284).
sql/mysql_priv.h:
  Rename parameter to match the name used in the definition (WL#4284).
sql/rpl_injector.cc:
  Release metadata locks on commit (WL#4284).
sql/rpl_rli.cc:
  Remove assert made meaningless, metadata locks are released
  at the end of the transaction.
sql/set_var.cc:
  Close tables and release locks if autocommit mode is set.
sql/slave.cc:
  Release metadata locks after a rollback.
sql/sql_acl.cc:
  Don't implicitly unlock locked tables. Issue a implicit commit
  at the end and unlock tables.
sql/sql_base.cc:
  Defer the release of metadata locks when closing tables
  if not required to.
  Issue a deadlock error if the locking protocol requires
  that a transaction re-acquire its locks.
  
  Release metadata locks when closing tables for reopen.
sql/sql_class.cc:
  Release metadata locks if the thread is killed.
sql/sql_parse.cc:
  Release metadata locks after implicitly committing a active
  transaction, or after explicit commits or rollbacks.
sql/sql_plugin.cc:
  
  Allocate MDL request on the stack as the use of the table
  is contained within the function. It will be removed from
  the context once close_thread_tables is called at the end
  of the function.
sql/sql_prepare.cc:
  The problem is that the prepare phase of the CREATE TABLE
  statement takes a exclusive metadata lock lock and this can
  cause a self-deadlock the thread already holds a shared lock
  on the table being that should be created.
  
  The solution is to make the prepare phase take a shared
  metadata lock when preparing a CREATE TABLE statement. The
  execution of the statement will still acquire a exclusive
  lock, but won't cause any problem as it issues a implicit
  commit.
  
  After some discussions with stakeholders it has been decided that
  metadata locks acquired during a PREPARE statement must be released
  once the statement is prepared even if it is prepared within a multi
  statement transaction.
sql/sql_servers.cc:
  Don't implicitly unlock locked tables. Issue a implicit commit
  at the end and unlock tables.
sql/sql_table.cc:
  Close table and release metadata locks after a admin operation.
sql/table.h:
  The problem is that the prepare phase of the CREATE TABLE
  statement takes a exclusive metadata lock lock and this can
  cause a self-deadlock the thread already holds a shared lock
  on the table being that should be created.
  
  The solution is to make the prepare phase take a shared
  metadata lock when preparing a CREATE TABLE statement. The
  execution of the statement will still acquire a exclusive
  lock, but won't cause any problem as it issues a implicit
  commit.
sql/transaction.cc:
  Release metadata locks after the implicitly committed due
  to a new transaction being started. Also, release metadata
  locks acquired after a savepoint if the transaction is rolled
  back to the save point.
  
  The problem is that in some cases transaction-long metadata locks
  could be released before the transaction was committed. This could
  happen when a active transaction was ended by a "START TRANSACTION"
  or "BEGIN" statement, in which case the metadata locks would be
  released before the actual commit of the active transaction.
  
  The solution is to defer the release of metadata locks to after the
  transaction has been implicitly committed. No test case is provided
  as the effort to provide one is too disproportional to the size of
  the fix.