replication):

Patch to add binlog format capabilities to the InnoDB storage engine.
The engine will not allow statement format logging when in READ COMMITTED
or READ UNCOMMITTED transaction isolation level.

In addition, an error is generated when trying to use READ COMMITTED
or READ UNCOMMITTED transaction isolation level in STATEMENT binlog
mode.


sql/handler.h:
  Adding declaration of already global arrays.
sql/share/errmsg.txt:
  Adding error messages for invalid changes of transaction isolation level
  and binlog mode switch.  Removing messages that are not needed any more
  (this cset it pushed together with the cset that introduced these
  messages, so it is safe to remove the messages).
sql/sql_base.cc:
  Some changes to error reporting code to get more informative messages.
storage/innobase/handler/ha_innodb.cc:
  Adding capabilities to storage engine.
  
  Ha_innobase:table_flags() now compute flags on a per-statement basis
  and the statement capabilities flag is just set if the transaction
  isolation level is below READ COMMITTED.
  
  An informative message is printed in the event that the transaction
  isolation level is below READ COMMITTED and the binlog mode is STATEMENT.
storage/innobase/handler/ha_innodb.h:
  Accomodating to changes in the server code that switched from ulong
  to Table_flags as type for the table flags.
mysql-test/r/binlog_innodb.result:
  New BitKeeper file ``mysql-test/r/binlog_innodb.result''
mysql-test/t/binlog_innodb.test:
  New BitKeeper file ``mysql-test/t/binlog_innodb.test''

Moving code to check storage engine capabilities to after tables
are locked.  Moving code to cache table flags so that table flags
are read from the storage engine at the beginning of the statement
in addition to when the storage engine is opened.

To handle CREATE-SELECT, the decision function is called after the
table is created and it is called with all tables that are in the select
part of the statement as well as the newly created table.


sql/handler.cc:
  Changing code to cache table flags on a per-statement basis. The table
  flags are now retrieved inside ha_external_lock().
sql/handler.h:
  Extending TABLEOP_HOOKS with postlock() member.
sql/mysql_priv.h:
  Adding prototype declaration of decide_logging_format() function.
sql/sql_base.cc:
  Factoring out code to check capabilities into decide_logging_format().
  Moving code to check engine capabilities to after the tables are locked.
  Correcting a bug causing row-based to not be set when the engines
  were not statement-logging capable.
sql/sql_class.h:
  Adding selected tables as select_create::select_tables member variable.
sql/sql_insert.cc:
  Introducing logic to handle post-locking hook.
  select_create::prepare now uses post-lock hook instead of pre-lock hook.
  Deciding on logging format especially for CREATE-SELECT by calling
  decide_logging_format() in the post-lock hook.
sql/sql_parse.cc:
  Adding selected tables as argument to select_create constructor.

into  kindahl-laptop.dnsalias.net:/home/bk/w3303-mysql-5.1-rpl

Adding support to allow engines to tell what formats they can handle.
The server will generate an error if it is not possible to log the
statement according to the logging mode in effect.

Adding flags to several storage engines to state what they can handle.

Changes to NDB handler removing code that forces row-based mode and
adding flag saying that NDB can only handle row format.

Adding check that binlog flags are only used for real tables that are
opened for writing.


BitKeeper/deleted/.del-binlog_row_blackhole.result:
  Rename: mysql-test/r/binlog_row_blackhole.result -> BitKeeper/deleted/.del-binlog_row_blackhole.result
BitKeeper/deleted/.del-binlog_row_blackhole.test:
  Rename: mysql-test/t/binlog_row_blackhole.test -> BitKeeper/deleted/.del-binlog_row_blackhole.test
mysql-test/t/partition_hash.test:
  Adding error check for statement that might fail.
sql/ha_ndbcluster.cc:
  Removing statements that switch to row-based format.
  Adding row capabilities.
sql/handler.h:
  Adding handler/table flags to indicate that the engine is row- and/or
  statement-logging capable.
  
  Adding typedef for table_flags type.
sql/set_var.cc:
  Removing code that prevents changing binlog format when NDB is active.
sql/share/errmsg.txt:
  Adding error messages for when row- and/or statement-based logging
  formats cannot be used.
sql/sql_base.cc:
  Adding business logic in lock_tables() to decide when an error should
  be thrown because logging is not possible.
  Add logic to switch to row format when that is allowed and needed.
  ---
  Binlog flags should only be checked for real tables that are opened for
  writing. Adding code to check that.
storage/archive/ha_archive.h:
  Adding row- and statement-logging capabilities to engine.
storage/blackhole/ha_blackhole.h:
  Blackhole can handle statement-format only.
storage/csv/ha_tina.h:
  Adding row- and statement-logging capabilities to engine.
storage/example/ha_example.h:
  For the example engine, we arbitrarily decided that it only can handle
  row format.
storage/federated/ha_federated.h:
  Adding row- and statement-logging capabilities to engine.
storage/heap/ha_heap.h:
  Heap can handle both row- and statement-based logging format.
storage/myisam/ha_myisam.cc:
  MyISAM can handle both row- and statement-based logging format.
storage/myisammrg/ha_myisammrg.h:
  MyISAM can handle both row- and statement-based logging format.
mysql-test/r/binlog_multi_engine.result:
  New BitKeeper file ``mysql-test/r/binlog_multi_engine.result''
mysql-test/t/binlog_multi_engine.test:
  New BitKeeper file ``mysql-test/t/binlog_multi_engine.test''

into  kindahl-laptop.dnsalias.net:/home/bk/w3339-mysql-5.1-rpl

Fixing case where a false warning could be printed in mixed mode.
Also fixing some test cases that generated different result files as
a consequence of the patch.


mysql-test/r/check.result:
  Result change.
mysql-test/t/check.test:
  Dropping all views created in the test before trying to execute test.
mysql-test/t/func_misc.test:
  Disabling warning for unsafe statement since test is not concerned with
  this and there are tests that test this.
sql/sql_class.cc:
  Moving code to print warning for unsafe statement to look at logging
  *mode* instead of logging *format*. The latter one can print a false
  warning when executing in mixed mode and part of the statement is
  written in statement format (i.e., CREATE-SELECT).

into  kindahl-laptop.dnsalias.net:/home/bkroot/mysql-5.1-rpl

This patch corrects a problem encountered when reading the binlog from a remote
host. The application was crashing because the buffer variable (temp_buf) in 
log_event was not pointing to the incoming data. For a normal file read, this 
buffer is allocated by a previous call of read_log_event. However, when reading
from a remote host, the first call to read_log_event is not executed therefore
no buffer is allocated. Furthermore, there is no need to allocate a new buffer 
because the incoming stream is what needs to be read. 

This patch adds the call to initialize the temp_buf variable if reading from a 
remote host. It also adds a check at destroy time to ensure the temp_buf is not
freed if reading from a remote host.


client/mysqlbinlog.cc:
  BUG#17654 : --read-from-remote-server causes core
  
  This patch corrects a problem when reading from a remote host. The temp_buf 
  variable of the log_event class is undefined. This patch assigns the temp_buf
  variable to the address of the incoming stream. This allows the print functions
  to print the binlog events correctly.
mysql-test/r/rpl_row_mysqlbinlog.result:
  BUG#17654 : --read-from-remote-server causes core
  
  This patch adds the results for the test that were disabled when the bug report
  was investigated. The patch also adds an additional test was added to ensure 
  the output of reading from a remote host is the same as reading from a local file.
mysql-test/t/rpl_row_mysqlbinlog.test:
  BUG#17654 : --read-from-remote-server causes core
  
  This patch enables the portions of the test that were disabled when the bug report
  was investigated. The patch also adds an additional test was added to ensure 
  the output of reading from a remote host is the same as reading from a local file.

Replacing binlog_row_based_if_mixed with variable binlog_stmt_flags
holding several flags and adding member functions to manipulate the
flags.

Added code to generate a warning when an attempt to log an unsafe
statement to the binary log was made. The warning is both pushed to the
SHOW WARNINGS table and written to the error log. The prevent flooding
the error log, the warning is just written to the error log once per
open session.


sql/item_create.cc:
  Using {is,set,clear}_stmt_unsafe() member functions instead of
  binlog_row_based_if_mixed member variable.
sql/sp_head.cc:
  Using {is,set,clear}_stmt_unsafe() member functions instead of
  binlog_row_based_if_mixed member variable.
sql/sp_head.h:
  Using {is,set,clear}_stmt_unsafe() member functions instead of
  binlog_row_based_if_mixed member variable.
sql/sql_base.cc:
  Using {is,set,clear}_stmt_unsafe() member functions instead of
  binlog_row_based_if_mixed member variable.
sql/sql_class.cc:
  Adding THD::binlog_flags to store thread-specific binary log state.
  Adding code to push a warning and write an entry into the error log
  when an attempt is made to log an unsafe statement.
sql/sql_class.h:
  Adding THD::binlog_flags to store thread-specific binary log state.
  Adding BINLOG_FLAG_UNSAFE_STMT_PRINTED to denote that a warning for
  an unsafe statement has already been generated for this thread.
sql/sql_insert.cc:
  Using {is,set,clear}_stmt_unsafe() member functions instead of
  binlog_row_based_if_mixed member variable.
sql/sql_lex.cc:
  Replacing binlog_row_based_if_mixed with a variable binlog_stmt_flags
  holding several flags.
sql/sql_lex.h:
  Replacing binlog_row_based_if_mixed with a variable binlog_stmt_flags
  holding several flags.
sql/share/errmsg.txt:
  Adding error message to indicate that an attempt to log an unsafe
  statement was made.
sql/sql_view.cc:
  Using {is,set,clear}_stmt_unsafe() member functions instead of
  binlog_row_based_if_mixed member variable.
mysql-test/r/binlog_unsafe.result:
  New BitKeeper file ``mysql-test/r/binlog_unsafe.result''
mysql-test/t/binlog_unsafe.test:
  New BitKeeper file ``mysql-test/t/binlog_unsafe.test''

into  mysql_cab.:C:/source/c++/mysql-5.1_BUG_17233

This patch corrects a bug involving a LOAD DATA INFILE operation on a 
transactional table. It corrects a problem in the error handler by moving
the transactional table check and autocommit_or_rollback operation to the 
end of the error handler. 

The problem was an assert was thrown after the operation completed. The 
assert found a non-sunk event in the transaction cache. The events in the 
transaction cache were added after commit_or_rollack and thereafter nothing 
removed them.

An additional test case was added to detect this
condition.


mysql-test/extra/rpl_tests/rpl_loaddata.test:
  BUG#17233 : LOAD DATA INFILE: failure causes mysqld dbug_assert, binlog not flushed 
      
  This patch adds an additional test to rpl_loaddata for handling the
  duplicate key error on LOAD DATA INFILE.
mysql-test/r/rpl_loaddata.result:
  BUG#17233 : LOAD DATA INFILE: failure causes mysqld dbug_assert, binlog not flushed 
      
  This patch adds the results for the additional test for properly handling the
  duplicate key error on LOAD DATA INFILE.
sql/sql_load.cc:
  BUG#17233 : LOAD DATA INFILE: failure causes mysqld dbug_assert, binlog not flushed 
  
  This patch moves the check for a transactional table and rollback in the
  error handler for mysql_load(). The patch moves the transactional table
  check to the end of the error handler matching the implementation for other
  similar operations (see sql_insert).

into  mysql.com:/home/bar/mysql-5.1.b26518


mysql-test/r/xml.result:
  Auto merged
mysql-test/t/xml.test:
  Auto merged
sql/item_xmlfunc.cc:
  Auto merged

Problem: XPath variables didn't work.
Fix: adding variables support,
both user-defined and sp local variables are now supported by XPath.


mysql-test/r/xml.result:
  Adding test case
mysql-test/t/xml.test:
  Adding test case
sql/item_xmlfunc.cc:
  Adding variables support:
  - SP variables with standard XPath syntax:  $i
  - User variables with non-standard syntax:  $@i

into  mysql_cab_desk.:C:/source/c++/mysql-5.1_BUG_18980

Failure was not reproduced under testing with the latest clone.
This patch re-enables the test.


mysql-test/t/disabled.def:
  BUG#18980 : Test 'rpl_row_blob_innodb' fails randomly
  
  This patch enables the test.

Problem: when replacing the root element, UpdateXML
erroneously tried to mix old XML content with the
replacement string, which led to crash.
Fix: don't use the old XML content in these cases,
just return the replacement string.


mysql-test/r/xml.result:
  Adding test case
mysql-test/t/xml.test:
  Adding test case
sql/item_xmlfunc.cc:
  Adding special code to handle replacements of the root element -
  the replacing content is just copied to the result,
  the previous content of the XML value is removed.

Reason:
   This test executes DML statements on a NDB table to detect if some SQL statements of special interest commits the ongoing transaction.
   When running in MIXED mode, automatic switching from statement-based to row-based replication takes place when a DML statement
   updates an NDB table.
   That means running this test on NDB with binlog-format=mixed and binlog-format=row mostly checks the same routines twice.
   Therefore we skip the variant with binlog-format=mixed.


mysql-test/t/rpl_ndb_ddl.test:
  Prevent the execution of this test if replication format is statement or mixed.

The combination NDB and statement based replication is extreme slow and this fails usually because of testing timeout.


mysql-test/t/rpl_ndb_ddl.test:
  Avoid execution of this test with statement based replication because it is far way too slow.

The fix for   Bug#18946: Test case rpl_ndb_ddl disabled   pushed around end of March 2007 enabled this testcase.
It was later disabled because the test failed with timeout on one testing box.
The reason for this failing test could not be found because we do not have informations about the conditions on the box during this test.
Jeb and I tried this test on other boxes and it passed.
My experience is that
- tests using NDB need in general often significant more runtime
  than comparable tests of other storage engines
- the actual load of the box where the test is running and the
  filesystem (nfs could be extreme slow) where the tests are
  executed might have a huge impact on the test performance 
  (runtime * 2 till 3)
- there are sometimes problems with the ports most probably
  caused by OS properties (NDB+RPL need many ports) or
  parallel tests accidently running with the same ports.
AFAIK these are the reasons why the NDB tests fail sometimes with timeout.
Conclusion: We enable rpl_ndb_ddl again because the failure happens in rare cases
and seems not to be caused by errors within the server or test code. 


mysql-test/t/disabled.def:
  Enable t/rpl_ndb_ddl.test again

mysql-test/r/rpl_critical_errors.result:
  New BitKeeper file ``mysql-test/r/rpl_critical_errors.result''

into  romeo.(none):/home/bk/merge-mysql-5.1


client/mysql.cc:
  Auto merged
sql/ha_ndbcluster.cc:
  Auto merged
sql/item.cc:
  Auto merged
sql/item_cmpfunc.cc:
  Auto merged
sql/mysql_priv.h:
  Auto merged
sql/mysqld.cc:
  Auto merged
sql/set_var.cc:
  Auto merged
sql/sql_class.cc:
  Auto merged
sql/sql_delete.cc:
  Auto merged
sql/sql_select.cc:
  Auto merged
sql/sql_table.cc:
  Auto merged
configure.in:
  Merging 5.0 into 5.1
sql/slave.cc:
  Merging 5.0 into 5.1

into  romeo.(none):/home/bk/merge-mysql-5.1


BitKeeper/etc/ignore:
  auto-union
client/mysql.cc:
  Auto merged
client/mysqlbinlog.cc:
  Auto merged
configure.in:
  Auto merged
client/mysqltest.c:
  Auto merged
mysql-test/r/rpl_ndb_basic.result:
  Auto merged
mysql-test/t/disabled.def:
  Auto merged
sql/CMakeLists.txt:
  Auto merged
sql/ha_ndbcluster.cc:
  Auto merged
sql/ha_ndbcluster.h:
  Auto merged
sql/ha_ndbcluster_binlog.cc:
  Auto merged
sql/log.cc:
  Auto merged
sql/log_event.cc:
  Auto merged
sql/mysql_priv.h:
  Auto merged
sql/mysqld.cc:
  Auto merged
sql/set_var.cc:
  Auto merged
sql/slave.cc:
  Auto merged
sql/sql_class.cc:
  Auto merged
sql/sql_class.h:
  Auto merged
sql/sql_insert.cc:
  Auto merged
sql/sql_repl.cc:
  Auto merged

into  romeo.(none):/home/bk/merge-mysql-5.0


client/mysql.cc:
  Auto merged
sql/mysql_priv.h:
  Auto merged
sql/mysqld.cc:
  Auto merged
sql/set_var.cc:
  Auto merged
sql/slave.cc:
  Auto merged
sql/sql_class.cc:
  Auto merged

into  romeo.(none):/home/bkroot/mysql-5.0-rpl

into  romeo.(none):/home/bk/b27779-mysql-5.1-rpl

mysql-test/r/ndb_binlog_ddl_multi.result:
  corrrection of test case
  (this was actually a bug that had not been spotted, that was fixed by previous patch)

revert back to old show_binlog_events asto many test failures, and create a show_binlog_events2 instead


BitKeeper/etc/ignore:
  Added client/log_event_old.cc client/log_event_old.h client/rpl_record_old.cc client/rpl_record_old.h libmysqld/log_event_old.cc libmysqld/rpl_record.cc libmysqld/rpl_record_old.cc to the ignore list
mysql-test/include/show_binlog_events2.inc:
  New BitKeeper file ``mysql-test/include/show_binlog_events2.inc''

 - test case


sql/ha_ndbcluster_binlog.cc:
  drop table not logged when it should
mysql-test/r/ndb_binlog_log_bin.result:
  New BitKeeper file ``mysql-test/r/ndb_binlog_log_bin.result''
mysql-test/t/ndb_binlog_log_bin.test:
  New BitKeeper file ``mysql-test/t/ndb_binlog_log_bin.test''

- both for data schema operations
- also make sure schema events vet the right server id when injected into the binlog
- use same mechanism to signal server_id in bug#17095, and reserve some "id's" for flagging special conditions on the event, in this case do not log it
- enable printing of server ids in the testcases to show that we cot it right

- make sure any value is set for linked operations as well, e.g. blob tables

into  bk-internal.mysql.com:/data0/bk/mysql-5.0-marvel


sql/ha_ndbcluster.cc:
  Auto merged
sql/item.cc:
  Auto merged
sql/item_cmpfunc.cc:
  Auto merged
sql/sql_delete.cc:
  Auto merged
sql/sql_select.cc:
  Auto merged
sql/sql_table.cc:
  Auto merged

into  bk-internal.mysql.com:/data0/bk/mysql-5.1-marvel


mysql-test/r/strict.result:
  Auto merged
sql/ha_ndbcluster.cc:
  Auto merged
sql/item.cc:
  Auto merged
sql/item.h:
  Auto merged
sql/item_cmpfunc.cc:
  Auto merged
sql/sql_delete.cc:
  Auto merged
sql/sql_select.cc:
  Auto merged
sql/sql_table.cc:
  Auto merged

into  mysql.com:/home/my/mysql-5.1

(Removes some warnings about UNIX_TIMESTAMP from the slave.err logs)
Marked federated_server as a '--big-test'
Change error in net_clear to 'Note', as it interfered with mysql-test-run.


client/mysqltest.c:
  More DBUG messages
  Adding missing DBUG_RETURN
mysql-test/extra/rpl_tests/rpl_max_relay_size.test:
  Added missing sync_slave_with_master
mysql-test/extra/rpl_tests/rpl_relayrotate.test:
  Added missing sync_slave_with_master
mysql-test/include/federated.inc:
  Don't do stop slave before the slave has started properly
  (Removes some warnings about UNIX_TIMESTAMP from the slave.err logs)
mysql-test/include/federated_cleanup.inc:
  Trivial cleanup
mysql-test/t/federated_server.test:
  Don't run this unless under --big-test
  Made test-loop smaller. (We will find out errors in code over time, as the test is run under a lot of difference machines which will compensate for the smaller loop)
mysql-test/t/rpl_flushlog_loop.test:
  Added missing sync_slave_with_master
sql/net_serv.cc:
  Change error to Note (low level warning), as it interfered with mysql-test-run.
  This is probably fine as we in some context on sever shutdown can get information about connection shutdown on the connection while we are doing a query at the same time.
  Still, in normal context one should get this, so it's good to have it in the log as it enables one to find errors easier.
sql/slave.cc:
  Added reason to why things failed to error message

compile warnings on windows machines.

mysql-test/r/rpl_ndb_circular_simplex.result:
  Result change
mysql-test/t/rpl_ndb_circular_simplex.test:
  Using master port on slave side instead of slave port (which should be
  used on master side).

refining the test with active innodb


mysql-test/r/sp_trans_log.result:
  changed
mysql-test/t/sp_trans_log.test:
  innodb activated