transaction
BUG#52616 Temp table prevents switch binlog format from STATEMENT to ROW

Before the WL#2687 and BUG#46364, every non-transactional change that happened
after a transactional change was written to trx-cache and flushed upon
committing the transaction. WL#2687 and BUG#46364 changed this behavior and
non-transactional changes are now written to the binary log upon committing
the statement.

A binary log event is identified as transactional or non-transactional through
a flag in the Log_event which is set taking into account the underlie storage
engine on what it is stems from. In the current bug, this flag was not being
set properly when the DROP TEMPORARY TABLE was executed.

However, while fixing this bug we figured out that changes to temporary tables
should be always written to the trx-cache if there is an on-going transaction.
Otherwise, binlog events in the reversed order would be produced.

Regarding concurrency, keeping changes to temporary tables in the trx-cache is
also safe as temporary tables are only visible to the owner connection.

In this patch, we classify the following statements as unsafe:
   1 - INSERT INTO t_myisam SELECT * FROM t_myisam_temp

   2 - INSERT INTO t_myisam_temp SELECT * FROM t_myisam

   3 - CREATE TEMPORARY TABLE t_myisam_temp SELECT * FROM t_myisam

On the other hand, the following statements are classified as safe:

   1 - INSERT INTO t_innodb SELECT * FROM t_myisam_temp

   2 - INSERT INTO t_myisam_temp SELECT * FROM t_innodb

The patch also guarantees that transactions that have a DROP TEMPORARY are
always written to the binary log regardless of the mode and the outcome:
commit or rollback. In particular, the DROP TEMPORARY is extended with the
IF EXISTS clause when the current statement logging format is set to row.

Finally, the patch allows to switch from STATEMENT to MIXED/ROW when there
are temporary tables but the contrary is not possible.

mysql-test/extra/rpl_tests/rpl_binlog_max_cache_size.test:
  Updated the test case because 
     CREATE TEMPORARY TABLE t_innodb_temp SELECT * FROM t_myisam is not unsafe.
mysql-test/extra/rpl_tests/rpl_implicit_commit_binlog.test:
  Updated the test case due to the new rules: changes to
  temporary tables are written to the binary log in the
  boundaries of a transaction if there is any.
mysql-test/extra/rpl_tests/rpl_innodb.test:
  Removed comments from the test case that became false after the patch.
mysql-test/extra/rpl_tests/rpl_loaddata.test:
  Suppressed warning messages due to the following cases:
  
     1 - INSERT INTO t_myisam SELECT * FROM t_myisam_temp
  
     2 - INSERT INTO t_myisam_temp SELECT * FROM t_myisam
mysql-test/include/ctype_utf8_table.inc:
  Suppressed warning messages due to the following cases:
  
     1 - INSERT INTO t_myisam SELECT * FROM t_myisam_temp
  
     2 - INSERT INTO t_myisam_temp SELECT * FROM t_myisam
mysql-test/r/ctype_cp932_binlog_stm.result:
  Updated the test case due to the new rules: changes to
  temporary tables are written to the binary log in the
  boundaries of a transaction if there is any.
mysql-test/suite/binlog/r/binlog_database.result:
  Updated the test case due to the new rules: changes to
  temporary tables are written to the binary log in the
  boundaries of a transaction if there is any.
mysql-test/suite/binlog/r/binlog_innodb_row.result:
  Updated the result file.
mysql-test/suite/binlog/r/binlog_multi_engine.result:
  Updated the unsafe message.
mysql-test/suite/binlog/r/binlog_row_binlog.result:
  Updated the test case due to the new rules: changes to
  temporary tables are written to the binary log in the
  boundaries of a transaction if there is any.
mysql-test/suite/binlog/r/binlog_row_drop_tmp_tbl.result:
  Updated the test case due to the new rules: changes to
  temporary tables are written to the binary log in the
  boundaries of a transaction if there is any.
mysql-test/suite/binlog/r/binlog_row_mix_innodb_myisam.result:
  Updated the test case due to the new rules: changes to
  temporary tables are written to the binary log in the
  boundaries of a transaction if there is any.
mysql-test/suite/binlog/r/binlog_stm_binlog.result:
  Updated the result file.
mysql-test/suite/binlog/r/binlog_stm_mix_innodb_myisam.result:
  Updated the test case due to the new rules: changes to
  temporary tables are written to the binary log in the
  boundaries of a transaction if there is any.
mysql-test/suite/binlog/t/binlog_tmp_table.test:
  Suppressed warning messages due to the following cases:
  
     1 - INSERT INTO t_myisam SELECT * FROM t_myisam_temp
  
     2 - INSERT INTO t_myisam_temp SELECT * FROM t_myisam
mysql-test/suite/ndb/r/ndb_binlog_format.result:
  Updated the unsafe message.
mysql-test/suite/rpl/r/rpl_concurrency_error.result:
  Updated the unsafe message.
mysql-test/suite/rpl/r/rpl_mixed_binlog_max_cache_size.result:
  Updated the result file because 
     CREATE TEMPORARY TABLE t_innodb_temp SELECT * FROM t_myisam is not unsafe.
mysql-test/suite/rpl/r/rpl_mixed_implicit_commit_binlog.result:
  Updated the test case due to the new rules: changes to
  temporary tables are written to the binary log in the
  boundaries of a transaction if there is any.
mysql-test/suite/rpl/r/rpl_mixed_mixing_engines.result:
  Updated the test case due to the new rules: changes to
  temporary tables are written to the binary log in the
  boundaries of a transaction if there is any.
mysql-test/suite/rpl/r/rpl_mixed_row_innodb.result:
  Added some comments to ease the understanding of the result file.
mysql-test/suite/rpl/r/rpl_non_direct_mixed_mixing_engines.result:
  Updated the test case due to the new rules: changes to
  temporary tables are written to the binary log in the
  boundaries of a transaction if there is any.
mysql-test/suite/rpl/r/rpl_non_direct_row_mixing_engines.result:
  Updated the test case due to the new rules: changes to
  temporary tables are written to the binary log in the
  boundaries of a transaction if there is any.
mysql-test/suite/rpl/r/rpl_non_direct_stm_mixing_engines.result:
  Updated the test case due to the new rules: changes to
  temporary tables are written to the binary log in the
  boundaries of a transaction if there is any.
mysql-test/suite/rpl/r/rpl_row_drop.result:
  Updated the test case due to the new rules: changes to
  temporary tables are written to the binary log in the
  boundaries of a transaction if there is any.
mysql-test/suite/rpl/r/rpl_row_implicit_commit_binlog.result:
  Updated the test case due to the new rules: changes to
  temporary tables are written to the binary log in the
  boundaries of a transaction if there is any.
mysql-test/suite/rpl/r/rpl_row_mixing_engines.result:
  Updated the test case due to the new rules: changes to
  temporary tables are written to the binary log in the
  boundaries of a transaction if there is any.
mysql-test/suite/rpl/r/rpl_stm_binlog_max_cache_size.result:
  Updated the result file because 
     CREATE TEMPORARY TABLE t_innodb_temp SELECT * FROM t_myisam is not unsafe.
mysql-test/suite/rpl/r/rpl_stm_implicit_commit_binlog.result:
  Updated the test case due to the new rules: changes to
  temporary tables are written to the binary log in the
  boundaries of a transaction if there is any.
mysql-test/suite/rpl/r/rpl_stm_innodb.result:
  Added some comments to ease the understanding of the result file.
mysql-test/suite/rpl/r/rpl_stm_mixing_engines.result:
  Updated the test case due to the new rules: changes to
  temporary tables are written to the binary log in the
  boundaries of a transaction if there is any.
mysql-test/suite/rpl/r/rpl_stm_stop_middle_group.result:
  Updated the unsafe message.
mysql-test/suite/rpl/r/rpl_temp_temporary.result:
  Added a test case.
mysql-test/suite/rpl/t/rpl000013.test:
  Suppressed warning messages due to the following cases:
  
     1 - INSERT INTO t_myisam SELECT * FROM t_myisam_temp
  
     2 - INSERT INTO t_myisam_temp SELECT * FROM t_myisam
  
     3 - CREATE TEMPORARY TABLE t_myisam_temp SELECT * FROM t_myisam
mysql-test/suite/rpl/t/rpl_misc_functions.test:
  Suppressed warning messages.
mysql-test/suite/rpl/t/rpl_temp_table.test:
  Suppressed warning messages due to the following cases:
  
     1 - INSERT INTO t_myisam SELECT * FROM t_myisam_temp
  
     2 - INSERT INTO t_myisam_temp SELECT * FROM t_myisam
  
     3 - CREATE TEMPORARY TABLE t_myisam_temp SELECT * FROM t_myisam
mysql-test/suite/rpl/t/rpl_temp_temporary.test:
  Added a test case.
mysql-test/suite/rpl/t/rpl_temporary.test:
  Suppressed warning messages due to the following cases:
  
     1 - INSERT INTO t_myisam SELECT * FROM t_myisam_temp
  
     2 - INSERT INTO t_myisam_temp SELECT * FROM t_myisam
  
     3 - CREATE TEMPORARY TABLE t_myisam_temp SELECT * FROM t_myisam
mysql-test/suite/rpl_ndb/r/rpl_ndb_row_implicit_commit_binlog.result:
  Updated the test case due to the new rules: changes to
  temporary tables are written to the binary log in the
  boundaries of a transaction if there is any.
mysql-test/suite/rpl_ndb/r/rpl_truncate_7ndb.result:
  Updated the test case to remove references to positions
  in the binary log.
mysql-test/suite/rpl_ndb/t/rpl_truncate_7ndb.test:
  Updated the test case to remove references to positions
  in the binary log.
mysql-test/t/create_select_tmp.test:
  Suppressed warning messages due to the following cases:
  
     1 - INSERT INTO t_myisam SELECT * FROM t_myisam_temp
  
     2 - INSERT INTO t_myisam_temp SELECT * FROM t_myisam
  
     3 - CREATE TEMPORARY TABLE t_myisam_temp SELECT * FROM t_myisam
mysql-test/t/ctype_cp932_binlog_stm.test:
  Suppressed warning messages due to the following cases:
  
     1 - INSERT INTO t_myisam SELECT * FROM t_myisam_temp
  
     2 - INSERT INTO t_myisam_temp SELECT * FROM t_myisam
  
     3 - CREATE TEMPORARY TABLE t_myisam_temp SELECT * FROM t_myisam
mysql-test/t/mysqlbinlog.test:
  Suppressed warning messages due to the following cases:
  
     1 - INSERT INTO t_myisam SELECT * FROM t_myisam_temp
  
     2 - INSERT INTO t_myisam_temp SELECT * FROM t_myisam
  
     3 - CREATE TEMPORARY TABLE t_myisam_temp SELECT * FROM t_myisam
sql/log.cc:
  Improved the code by creating several functions to hide decision
  on type of engine changed, commit/abort, etc:  
  
  . stmt_has_updated_non_trans_table
  
  . trans_has_updated_non_trans_table
  
  . ending_trans
  
  Updated the binlog_rollback function and the use of the 
  OPTION_KEEP_LOG which indincates when a temporary table was
  either created or dropped and as such the command must be 
  logged if not in MIXED mode and even while rolling back the
  transaction.
sql/log.h:
  Improved the code by creating several functions to hide decision
  on type of engine changed, commit/abort, etc.
sql/log_event.cc:
  Removed the setting of the OPTION_KEEP_LOG as it is related to CREATE
  TEMPORARY and DROP TEMPORARY and not to the type of engine (i.e.
  transactional or non-transactional).
sql/log_event_old.cc:
  Removed the setting of the OPTION_KEEP_LOG as it is related to CREATE
  TEMPORARY and DROP TEMPORARY and not to the type of engine (i.e.
  transactional or non-transactional).
sql/share/errmsg-utf8.txt:
  Updated the unsafe message.
sql/sql_class.cc:
  Classifies the following statements as unsafe:
     1 - INSERT INTO t_myisam SELECT * FROM t_myisam_temp
  
     2 - INSERT INTO t_myisam_temp SELECT * FROM t_myisam
  
  On the other hand, the following statements are classified as safe:
  
     1 - INSERT INTO t_innodb SELECT * FROM t_myisam_temp
  
     2 - INSERT INTO t_myisam_temp SELECT * FROM t_innodb
sql/sql_class.h:
  It allows to switch from STATEMENT to MIXED/ROW when there are temporary
  tables but the contrary is not possible.
sql/sql_table.cc:
  Fixed the case that a DROP/DROP TEMPORARY that affects a temporary table in MIXED
  mode is written as a DROP TEMPORARY TABLE IF EXISTS because the table may not exist in the slave and due to the IF EXISTS token an error will never happen
  while processing the statement in the slave.
  
  Removed a function that was not being used.

back into the development branches.

mysql-trunk-bugfixing (bug 52341).

mysql-trunk-bugfixing.

            my_global.h first

We may end up with a compilation failure on certain platforms
because zlib.h is included before my_global.h.

Fixed by moving zlib.h inclusion down after my_global.h.

storage/archive/azlib.h:
  zlib.h must be included after my_global.h.

            e.g.MYSQL_AUDIT_GENERAL_ERROR

General audit API (MYSQL_AUDIT_GENERAL_CLASS) didn't expose event
subclass to plugins.

This patch exposes event subclass to plugins via
struct mysql_event_general::event_subclass.

This change is not compatible with existing general audit plugins.
Audit interface major version has been incremented.

include/mysql/plugin_audit.h:
  Expose event subclass to audit general plugins.
plugin/audit_null/audit_null.c:
  Added distinct counters for general event sub-classes.
  
  Removed printf() from deinit(). One can easily see number of
  calls via status variables.
  
  To make code nicer, modified class mask to use macro instead
  of hardcoded number.
  
  Incremented audit plugin minor version.
sql/sql_audit.cc:
  Expose event subclass to audit general plugins.

The problem was that the code that works with stored routines was present
in mysql_system_tables.sql. This is wrong because until upgrade is finished,
stored routines may be (and were) not available.

The fix is to move that code to the end of mysql_system_tables_fix.sql.

Allow stored procedure variables in LIMIT clause.
Only allow variables of INTEGER types. 
Handle negative values by means of an implicit cast to UNSIGNED 
(similarly to prepared statement placeholders).
Add tests.
Make sure replication works by not doing NAME_CONST substitution
for variables in LIMIT clause.
Add replication tests.

mysql-test/r/sp.result:
  Update results (Bug#11918).
mysql-test/suite/rpl/r/rpl_sp.result:
  Update results (Bug#11918).
mysql-test/suite/rpl/t/rpl_sp.test:
  Add a test case for Bug#11918.
mysql-test/t/sp.test:
  Add a test case for Bug#11918.
sql/item.cc:
  Mark sp variables in LIMIT clause (a hack for replication).
sql/item.h:
  Mark sp variables in LIMIT clause (a hack for replication).
sql/share/errmsg-utf8.txt:
  Add a new error message (a type mismatch for LIMIT
  clause parameter).
sql/sp_head.cc:
  Binlog rewrite sp variables in LIMIT clause without NAME_CONST
  substitution.
sql/sql_string.cc:
  Implement append_ulonglong method.
sql/sql_string.h:
  Declare append_ulonglong().
sql/sql_yacc.yy:
  Support stored procedure variables in LIMIT.

ChangeSet@1.2703, 2007-12-07 09:35:28-05:00, cmiller@zippy.cornsilk.net +40 -0
Bug#13174: SHA2 function
Patch contributed from Bill Karwin, paper unnumbered CLA in Seattle

Implement SHA2 functions.

Chad added code to make it work with YaSSL.  Also, he removed the 
(probable) bug of embedded server never using SSL-dependent 
functions.  (libmysqld/Makefile.am didn't read ANY autoconf defs.)

Function specification:
  SHA2( string cleartext, integer hash_length ) 
    -> string hash, or NULL
where hash_length is one of 224, 256, 384, or 512.  If either is 
NULL or a length is unsupported, then the result is NULL.  The 
resulting string is always the length of the hash_length parameter
or is NULL.

Include the canonical hash examples from the NIST in the test
results.
---
Polish and address concerns of reviewers.


.bzrignore:
  Added libmysqld/sha2.cc to the ignore list.
client/mysql.cc:
  Add condition to remove code for embedded server.
client/mysqltest.cc:
  Add condition to remove code for embedded server.
include/Makefile.am:
  New header file to header list.
include/mysql_embed.h:
  Embedded servers can use SSL-library functions too!
include/sha2.h:
  Compatibility layer to make YaSSL behave like OpenSSL.
include/sslopt-case.h:
  Remove SSL-communication parameters from command lines.
include/sslopt-longopts.h:
  Remove SSL-communication parameters from command lines.
include/sslopt-vars.h:
  Don't declare variables that are only used in SSL communication, if
  we are compiling the embedded server.
include/violite.h:
  Don't even compile the SSL-communication function if we're in the
  embedded server.
  ---
  Remove CPP condition indentation.
libmysqld/CMakeLists.txt:
  Add new file to source list.
libmysqld/Makefile.am:
  Include standard DEFS in embedded compilation.  It's an undiscovered
  but that it's not there.
  
  Add new file to source list.
libmysqld/examples/Makefile.am:
  Include autoconf DEFS.
libmysqld/lib_sql.cc:
  Initialize SSL-related variables in embedded server.
mysql-test/include/have_ssl_crypto_functs.inc:
  Distinguish between communication and crypto.
  Use the tristate value of "have_ssl" variable to know whether to
  test or not for SSL-provided crypto functions.
mysql-test/r/func_digest.result:
  
  Test against the sample test vectors in the NIST Secure
  Hash Standard (http://csrc.nist.gov/cryptval/shs.htm)
mysql-test/r/func_encrypt_nossl.result:
  Update results to the new error message text.
mysql-test/r/have_ssl_is_yes_or_disabled_only.require:
  Distinguish between communication and crypto.
  
  Use the tristate value of "have_ssl" variable to know whether to
  test or not for SSL-provided crypto functions.
mysql-test/suite/rpl/t/rpl_ssl.test:
  Distinguish between communication and crypto.
mysql-test/suite/rpl/t/rpl_ssl1.test:
  Distinguish between communication and crypto.
mysql-test/t/func_des_encrypt.test:
  Distinguish between communication and crypto.
mysql-test/t/func_digest.test:
  Test against the sample test vectors in the NIST Secure
  Hash Standard (http://csrc.nist.gov/cryptval/shs.htm)
  
  Also, test that various parameters (legal and illegal)
  do what we expect.
  ---
  Distinguish between communication and crypto.
mysql-test/t/func_encrypt.test:
  Distinguish between communication and crypto.
mysql-test/t/openssl_1.test:
  Don't test SSL communication if we're in the embedded server.
  ---
  Distinguish between communication and crypto.
mysql-test/t/ssl-big.test:
  Don't test SSL communication if we're in the embedded server.
  ---
  Distinguish between communication and crypto.
mysql-test/t/ssl.test:
  Don't test SSL communication if we're in the embedded server.
  ---
  Distinguish between communication and crypto.
mysql-test/t/ssl_8k_key.test:
  Don't test SSL communication if we're in the embedded server.
  ---
  Distinguish between communication and crypto.
mysql-test/t/ssl_compress.test:
  Don't test SSL communication if we're in the embedded server.
  ---
  Distinguish between communication and crypto.
mysql-test/t/ssl_connect.test:
  Don't test SSL communication if we're in the embedded server.
  ---
  Distinguish between communication and crypto.
sql-common/client.c:
  SSL is useful for more functionality than just connecting.  Test
  for whether we are not embedded server also.
sql/CMakeLists.txt:
  Add new source file to source list so that we have access to SHA2
  functions.
sql/Makefile.am:
  Add new source file to source list so that we have access to SHA2
  functions.
sql/item_create.cc:
  Bootstrap the SHA2 function into the server.
sql/item_strfunc.cc:
  Add new SHA2 Item class methods.
  
  Clean up two minor problems.
  ---
  Remove extraneous debugging.
  ---
  We must check nullness of a parameter only /after/ computing its
  value.
sql/item_strfunc.h:
  Declare new SHA2 Item class.
sql/mysqld.cc:
  For embedded server, don't refer to SSL-communications variables
  or values.
  ---
  Remove CPP condition indentation.
sql/sha2.cc:
  Compatibility layer to make YaSSL behave like OpenSSL.
  ---
  Add comment for generated functions.
sql/sql_acl.cc:
  For embedded server, don't refer to SSL-communications variables
  or values.
sql/sql_connect.cc:
  SSL is useful for more functionality than just connecting.  Test
  for whether we are not embedded server also.
sql/sys_vars.cc:
  For embedded server, don't refer to SSL-communications variables
  or values.

pthread_t is not defined unless my_pthread.h is included.

Problem: Segmentation fault in add_group_and_distinct_keys() when accessing
field of what is assumed to be an Item_field object.

Cause: In case of views, the item added to list by is_indexed_agg_distinct() 
was not of type Item_field, but Item_ref.

Resolution:  Add the real Item_field object, the one referred to by 
Item_ref object, to the list, instead.

mysql-test/r/count_distinct.result:
  Results for test case for Bug#51980.
mysql-test/t/count_distinct.test:
  Test case for Bug#51980.
  Table needs to contain at least two rows to avoid const table optimization.
sql/sql_select.cc:
  Make sure it is the actual Item_field object that is pushed to the out_args
  list of is_indexed_agg_distinct(), and not Item_ref objects.

when trying to build innodb as plugin.

The reason for the error is mismatch in mysql_temp_dir_list
declaration between mysqld.h and usage in ha_innodb.cc

Add missing MYSQL_PLUGIN_IMPORT to mysql_tmpdir_list
(variables exported by the server and used by plugin need it).

Fix tons of warnings about mismatch struct and class usage.

Tree cleaup after the last major merges in mysql-trunk:

The files sql/lex_hash.h and sql/sql_yacc.h are automatically
generated, and should not be checked in the configuration management system.
These files are now removed.
No changes are required for .bzrignore, which already listed these files
(and similar files in libmysqld/).

The file storage/perfschema/unittest/pfs_timer-t.cc did not build
after the header files refactoring affecting mysql_priv.h
The file now builds properly using sql_priv.h

Adding my_global.h first in all files using
NO_EMBEDDED_ACCESS_CHECKS.

Correcting a merge problem resulting from a changed definition
of check_some_access compared to the original patches.

The failing assertion was written with the assumption that a NULL
string can never be passed to my_strtod(). However, an empty string
may be passed under some circumstances by passing str == NULL and
*end == NULL.

Fixed the assertion to take the above case into account.

mysql-test/r/func_misc.result:
  Added a test case for bug #52165.
mysql-test/t/func_misc.test:
  Added a test case for bug #52165.
strings/dtoa.c:
  Fixed the assertion in my_strtod() to take the case of 'str == NULL
  && *end == NULL' into account.

Adding include to build on Windows.

Adding includes of my_global.h that seems to be necessary on Windows.

Moving an extern "C" to not cover #includes.

- Adding include/mysql/innodb_priv.h which was missing before.
- Updated .bzrignore to ignore CMake generated cruft.

Minor fixes required because of changes since patches were developed.

Fixing merge problem that causes CSV engine to misbehave.