BUG#11758262 - 50439: MARK INSERT...SEL...ON DUP KEY UPD,REPLACE...SEL,CREATE...[IGN|REPL] SEL

Problem: The following statements can cause the slave to go out of sync 
if logged in statement format: 
INSERT IGNORE...SELECT 
INSERT ... SELECT ... ON DUPLICATE KEY UPDATE 
REPLACE ... SELECT 
UPDATE IGNORE :
CREATE ... IGNORE SELECT 
CREATE ... REPLACE SELECT  
           
Background: Since the order of the rows returned by the SELECT 
statement or otherwise may differ on master and slave, therefore
the above statements may cuase the salve to go out of sync with
the master. 
      
Fix:
Issue a warning when statements like the above are exectued and 
the bin-logging format is statement. If the logging format is mixed,
use row based logging. Marking a statement as unsafe has been 
done in the sql/sql_parse.cc instead of sql/sql_yacc.cc, because while
parsing for a token has been done we cannot be sure if the parsing
of the other tokens has been done as well.
      
Six new warning  messages has been added for each unsafe statement. 
      
binlog.binlog_unsafe.test has been updated to incoporate these additional unsafe statments.


******
BUG#11758262 - 50439: MARK INSERT...SEL...ON DUP KEY UPD,REPLACE...SEL,CREATE...[IGN|REPL] SEL 
Problem: The following statements can cause the slave to go out of sync 
if logged in statement format: 
INSERT IGNORE...SELECT 
INSERT ... SELECT ... ON DUPLICATE KEY UPDATE 
REPLACE ... SELECT 
UPDATE IGNORE :
CREATE ... IGNORE SELECT 
CREATE ... REPLACE SELECT  
           
Background: Since the order of the rows returned by the SELECT 
statement or otherwise may differ on master and slave, therefore
the above statements may cuase the salve to go out of sync with
the master. 
      
Fix:
Issue a warning when statements like the above are exectued and 
the bin-logging format is statement. If the logging format is mixed,
use row based logging. Marking a statement as unsafe has been 
done in the sql/sql_parse.cc instead of sql/sql_yacc.cc, because while
parsing for a token has been done we cannot be sure if the parsing
of the other tokens has been done as well.
      
Six new warning  messages has been added for each unsafe statement. 
      
binlog.binlog_unsafe.test has been updated to incoporate these additional unsafe statments.



mysql-test/extra/rpl_tests/rpl_insert_duplicate.test:
  Test removed: Added the test to rpl.rpl_insert_ignore.test
  
  
  
  ******
  Test removed: the test is redundant as the same is being tested in rpl.rpl_insert_ignore.
mysql-test/extra/rpl_tests/rpl_insert_id.test:
  Warnings disabled for the unsafe statements.
mysql-test/extra/rpl_tests/rpl_insert_ignore.test:
  1. Disabled warnings while  for unsafe statements
  2. As INSERT...IGNORE is an unsafe statement, an insert ignore not changing any rows, 
  will not be logged in the binary log, in the ROW and MIXED modes. It will however be logged
  in STATEMENT mode.
mysql-test/r/commit_1innodb.result:
  updated result file
  
  
  
  ******
  updated result file
mysql-test/suite/binlog/r/binlog_stm_blackhole.result:
  Updated result file.
mysql-test/suite/binlog/r/binlog_unsafe.result:
  updated result file
mysql-test/suite/binlog/t/binlog_unsafe.test:
   added tests for the statements marked as unsafe.
mysql-test/suite/rpl/r/rpl_insert_duplicate.result:
  File Removed :Result file of rpl_insert_duplicate, which has been removed.
mysql-test/suite/rpl/r/rpl_insert_ignore.result:
  Added the content of rpl.rpl_insert_duplicate here.
mysql-test/suite/rpl/r/rpl_insert_select.result:
  Result file removed as the corresponding test has beenn removed.
mysql-test/suite/rpl/r/rpl_known_bugs_detection.result:
  Updated result file.
mysql-test/suite/rpl/t/rpl_insert_duplicate.test:
  File Removed: this was a wrapper for rpl.rpl_insert_duplicate.test, which has been removed.
mysql-test/suite/rpl/t/rpl_insert_select.test:
  File Removed: This test became redundant after this fix, This test showed how INSERT IGNORE...SELECT break replication, which has been handled in this fix.
mysql-test/suite/rpl/t/rpl_known_bugs_detection.test:
  Since all the tests are statement based bugs are being tested, having mixed format
  forces the event to be written in row format. When the statement and causes the
  test to fail as certain known bugs do not occur when the even is logged in row format.
sql/share/errmsg-utf8.txt:
  added 6 new Warning messages.
  
  
  
  ******
  added 6 new Warning messages.
sql/sql_lex.cc:
  Added 6 new error Identifier [ER_BINLOG_STMT_UNSAFEE_*]
sql/sql_lex.h:
  Added 6 new BINLOG_STMT_UNSAFE_* enums to identify the type of unsafe statement dealt with in this bug.
  
  
  ******
  Added 6 new BINLOG_STMT_UNSAFE_* enums to identify the type of unsafe statement dealt with in this bug.
sql/sql_parse.cc:
  added check for specific queries and marked them as unsafe.
  
  
  ******
  added check for specific queries and marked them as unsafe.

mysql-test/extra/rpl_tests/rpl_insert_duplicate.test

-# BUG#59338 Inconsistency in binlog for statements that don't change any rows STATEMENT SBR
-# In SBR, if a statement does not fail, it is always written to the binary log,
-# regardless if rows are changed or not. If there is a failure, a statement is
-# only written to the binary log if a non-transactional (.e.g. MyIsam) engine
-# is updated. INSERT ON DUPLICATE KEY UPDATE was not following the rule above
-# and was not written to the binary log, if then engine was Innodb.
-#
-# In this test case, we check if INSERT ON DUPLICATE KEY UPDATE that does not
-# change anything is still written to the binary log.
-
-# Prepare environment
---connection master
-
-eval CREATE TABLE t1 (
- a INT UNSIGNED NOT NULL PRIMARY KEY
-) ENGINE=$engine_type;
-
-eval CREATE TABLE t2 (
- a INT UNSIGNED
-) ENGINE=$engine_type;
-
-INSERT INTO t1 VALUES (1);
-INSERT INTO t2 VALUES (1);
-
-# An insert duplicate that does not update anything must be written to the binary
-# log in SBR and MIXED modes. We check this property by summing a before and after
-# the update and comparing the binlog positions. The sum should be the same at both
-# points and the statement should be in the binary log.
---let $binlog_file= query_get_value("SHOW MASTER STATUS", File, 1)
---let $binlog_start= query_get_value("SHOW MASTER STATUS", Position, 1)
---let $statement_file=INSERT INTO t1 SELECT t2.a FROM t2 ORDER BY t2.a ON DUPLICATE KEY UPDATE t1.a= t1.a
---eval $statement_file
-
---let $assert_cond= SUM(a) = 1 FROM t1
---let $assert_text= Sum of elements in t1 should be 1.
---source include/assert.inc
-
-if (`SELECT @@BINLOG_FORMAT = 'ROW'`)
-{
-  --let $binlog_position_cmp= =
-  --let $assert_cond= [SHOW MASTER STATUS, Position, 1] $binlog_position_cmp $binlog_start
-  --let $assert_text= In SBR or MIXED modes, the event in the binlog should be the same that was executed. In RBR mode, binlog position should stay unchanged.
-}
-if (`SELECT @@BINLOG_FORMAT != 'ROW'`)
-{
-  --let $assert_cond= \'[\'SHOW BINLOG EVENTS IN "$binlog_file" FROM $binlog_start LIMIT 1, 1\', Info, 1]\' LIKE \'%$statement_file\'
-  --let $assert_text= In SBR or MIXED modes, the event in the binlog should be the same that was executed. In RBR mode, binlog position should stay unchanged.
-}
---source include/assert.inc
-
-# Compare master and slave
---sync_slave_with_master
---let $diff_tables= master:test.t1 , slave:test.t1
---source include/diff_tables.inc
-
-# Clean up
---connection master
-drop table t1, t2;
---sync_slave_with_master

mysql-test/extra/rpl_tests/rpl_insert_id.test

@@ -77,6 +77,7 @@ eval create table t2(b int auto_increment, c int, key(b)) engine=$engine_type;
 insert into t1 values (10);
 insert into t1 values (null),(null),(null);
 insert into t2 values (5,0);
+--disable_warnings ONCE
 insert into t2 (c) select * from t1 ORDER BY a;
 select * from t2 ORDER BY b;
 sync_slave_with_master;
@@ -113,8 +114,10 @@ set @@session.sql_auto_is_null=1;
 eval create table t1(a int auto_increment, key(a)) engine=$engine_type;
 eval create table t2(a int) engine=$engine_type;
 insert into t1 (a) values (null);
+--disable_warnings
 insert into t2 (a) select a from t1 where a is null;
 insert into t2 (a) select a from t1 where a is null;
+--enable_warnings
 select * from t2;
 sync_slave_with_master;
 connection slave;
@@ -172,17 +175,15 @@ begin
 end|
 delimiter ;|
 
---disable_warnings
+--disable_warnings ONCE
 insert into t1 (last_id) values (0);
---enable_warnings
 
 drop trigger t1_bi;
 
 # Check that nested call doesn't affect outer context.
 select last_insert_id();
---disable_warnings
+--disable_warnings ONCE
 select bug15728_insert();
---enable_warnings
 select last_insert_id();
 insert into t1 (last_id) values (bug15728());
 # This should be exactly one greater than in the previous call.
@@ -190,9 +191,8 @@ select last_insert_id();
 
 # BUG#20339 - stored procedure using LAST_INSERT_ID() does not
 # replicate statement-based
---disable_warnings
+--disable_warnings ONCE
 drop procedure if exists foo;
---enable_warnings
 delimiter |;
 create procedure foo()
 begin
@@ -252,6 +252,7 @@ select * from t1 order by n;
 # table's counter, the counter for next row is bigger than the
 # after-value of the updated row.
 connection master;
+--disable_warnings ONCE
 insert into t1 values (NULL,400),(3,500),(NULL,600) on duplicate key UPDATE n=1000;
 select * from t1 order by n;
 sync_slave_with_master;
@@ -270,6 +271,7 @@ delete from t1 where b <> 100;
 select * from t1 order by n;
 
 connection master;
+--disable_warnings ONCE
 insert into t1 values(null,100),(null,350) on duplicate key update n=2;
 select * from t1 order by n;
 sync_slave_with_master;
@@ -287,6 +289,7 @@ connection master;
 # testcase with INSERT VALUES
 eval CREATE TABLE t1 (a INT NOT NULL PRIMARY KEY AUTO_INCREMENT, b INT,
                       UNIQUE(b)) ENGINE=$engine_type;
+--disable_warnings ONCE
 INSERT INTO t1(b) VALUES(1),(1),(2) ON DUPLICATE KEY UPDATE t1.b=10;
 SELECT * FROM t1 ORDER BY a;
 sync_slave_with_master;
@@ -314,19 +317,23 @@ INSERT INTO t2 (field_a, field_b, field_c) VALUES (3, 'c', '3c');
 INSERT INTO t2 (field_a, field_b, field_c) VALUES (4, 'd', '4d');
 INSERT INTO t2 (field_a, field_b, field_c) VALUES (5, 'e', '5e');
 # Updating table t1 based on values from table t2
+--disable_warnings
 INSERT INTO t1 (field_1, field_2, field_3)
 SELECT t2.field_a, t2.field_b, t2.field_c
 FROM t2
 ON DUPLICATE KEY UPDATE
 t1.field_3 = t2.field_c;
+--enable_warnings
 # Inserting new record into t2
 INSERT INTO t2 (field_a, field_b, field_c) VALUES (6, 'f', '6f');
 # Updating t1 again
+--disable_warnings
 INSERT INTO t1 (field_1, field_2, field_3)
 SELECT t2.field_a, t2.field_b, t2.field_c
 FROM t2
 ON DUPLICATE KEY UPDATE
 t1.field_3 = t2.field_c;
+--enable_warnings
 SELECT * FROM t1 ORDER BY id;
 sync_slave_with_master;
 SELECT * FROM t1 ORDER BY id;
@@ -433,9 +440,8 @@ delimiter ;|
 
 INSERT INTO t1 VALUES (NULL, -1);
 CALL p1();
---disable_warnings
+--disable_warnings ONCE
 SELECT f1();
---enable_warnings
 INSERT INTO t1 VALUES (NULL, f2()), (NULL, LAST_INSERT_ID()),
                       (NULL, LAST_INSERT_ID()), (NULL, f2()), (NULL, f2());
 INSERT INTO t1 VALUES (NULL, f2());
@@ -504,16 +510,14 @@ insert into t2 (id) values(1),(2),(3);
 delete from t2;
 set sql_log_bin=1;
 #inside SELECT, then inside INSERT
---disable_warnings
+--disable_warnings ONCE
 select insid();
---enable_warnings
 set sql_log_bin=0;
 insert into t2 (id) values(5),(6),(7);
 delete from t2 where id>=5;
 set sql_log_bin=1;
---disable_warnings
+--disable_warnings ONCE
 insert into t1 select insid();
---enable_warnings
 select * from t1 order by id;
 select * from t2 order by id;
 
@@ -537,6 +541,7 @@ begin
   insert into t2 values(null,3);
 end|
 delimiter ;|
+--disable_warnings ONCE
 call foo();
 select * from t1 order by n;
 select * from t2 order by id;

mysql-test/extra/rpl_tests/rpl_insert_ignore.test

@@ -31,7 +31,7 @@ INSERT INTO t2 VALUES (3, 5);
 INSERT INTO t2 VALUES (4, 3);
 INSERT INTO t2 VALUES (5, 4);
 INSERT INTO t2 VALUES (6, 6);
-
+--disable_warnings ONCE
 INSERT IGNORE INTO t1 SELECT NULL, t2.b FROM t2 ORDER BY t2.a;
 --let $assert_cond= COUNT(*) = 6 FROM t1
 --let $assert_text= Count of elements in t1 should be 6.
@@ -51,25 +51,70 @@ INSERT IGNORE INTO t1 SELECT NULL, t2.b FROM t2 ORDER BY t2.a;
 --let $binlog_file= query_get_value("SHOW MASTER STATUS", File, 1)
 --let $binlog_start= query_get_value("SHOW MASTER STATUS", Position, 1)
 --let $statement_file=INSERT IGNORE INTO t1 SELECT NULL, t2.b FROM t2 ORDER BY t2.a
+--disable_warnings ONCE
 --eval $statement_file
 
 --let $assert_cond= COUNT(*) = 6 FROM t1
 --let $assert_text= Count of elements in t1 should be 6.
 --source include/assert.inc
 
-if (`SELECT @@BINLOG_FORMAT = 'ROW'`)
+if (`SELECT @@BINLOG_FORMAT != 'STATEMENT'`)
 {
   --let $binlog_position_cmp= =
   --let $assert_cond= [SHOW MASTER STATUS, Position, 1] $binlog_position_cmp $binlog_start
   --let $assert_text= In SBR or MIXED modes, the event in the binlog should be the same that was executed. In RBR mode, binlog position should stay unchanged.
 }
-if (`SELECT @@BINLOG_FORMAT != 'ROW'`)
+if (`SELECT @@BINLOG_FORMAT = 'STATEMENT'`)
 {
   --let $assert_cond= \'[\'SHOW BINLOG EVENTS IN "$binlog_file" FROM $binlog_start LIMIT 2, 1\', Info, 1]\' LIKE \'%$statement_file\'
   --let $assert_text= In SBR or MIXED modes, the event in the binlog should be the same that was executed. In RBR mode, binlog position should stay unchanged.
 }
+
+--source include/assert.inc
+
+# An insert duplicate that does not update anything must be written to the binary
+# log in SBR and MIXED modes. We check this property by summing a before and after
+# the update and comparing the binlog positions. The sum should be the same at both
+# points and the statement should be in the binary log.
+--disable_warnings
+DROP TABLE t1;
+DROP TABLE t2;
+--enable_warnings
+eval CREATE TABLE t1 (
+ a INT UNSIGNED NOT NULL PRIMARY KEY
+) ENGINE=$engine_type;
+
+eval CREATE TABLE t2 (
+ a INT UNSIGNED
+) ENGINE=$engine_type;
+
+INSERT INTO t1 VALUES (1);
+INSERT INTO t2 VALUES (1);
+
+--let $binlog_file= query_get_value("SHOW MASTER STATUS", File, 1)
+--let $binlog_start= query_get_value("SHOW MASTER STATUS", Position, 1)
+--let $statement_file=INSERT INTO t1 SELECT t2.a FROM t2 ORDER BY t2.a ON DUPLICATE KEY UPDATE t1.a= t1.a
+--disable_warnings ONCE
+--eval $statement_file
+
+--let $assert_cond= SUM(a) = 1 FROM t1
+--let $assert_text= Sum of elements in t1 should be 1.
+--source include/assert.inc
+
+if (`SELECT @@BINLOG_FORMAT != 'STATEMENT'`)
+{
+  --let $binlog_position_cmp= =
+  --let $assert_cond= [SHOW MASTER STATUS, Position, 1] $binlog_position_cmp $binlog_start
+  --let $assert_text= In SBR or MIXED modes, the event in the binlog should be the same that was executed. In RBR mode, binlog position should stay unchanged.
+}
+if (`SELECT @@BINLOG_FORMAT = 'STATEMENT'`)
+{
+  --let $assert_cond= \'[\'SHOW BINLOG EVENTS IN "$binlog_file" FROM $binlog_start LIMIT 1, 1\', Info, 1]\' LIKE \'%$statement_file\'
+  --let $assert_text= In SBR or MIXED modes, the event in the binlog should be the same that was executed. In RBR mode, binlog position should stay unchanged.
+}
 --source include/assert.inc
 
+
 # Clean up
 --connection master
 drop table t1, t2;

mysql-test/r/commit_1innodb.result

@@ -549,6 +549,8 @@ SUCCESS
 # 15. Read-write statement: UPDATE IGNORE, change 0 rows. 
 #
 update ignore t1 set a=2 where a=1;
+Warnings:
+Note	1592	Unsafe statement written to the binary log using statement format since BINLOG_FORMAT = STATEMENT. UPDATE IGNORE is unsafe because the order in which rows are updated determines which (if any) rows are ignored. This order cannot be predicted and may differ on master and the slave.
 call p_verify_status_increment(2, 2, 1, 0);
 SUCCESS
 
@@ -814,6 +816,8 @@ SUCCESS
 insert into t2 select a from t1;
 commit;
 replace into t2 select a from t1;
+Warnings:
+Note	1592	Unsafe statement written to the binary log using statement format since BINLOG_FORMAT = STATEMENT. REPLACE... SELECT is unsafe because the order in which rows are retrieved by the SELECT determines which (if any) rows are replaced. This order cannot be predicted and may differ on master and the slave.
 commit;
 call p_verify_status_increment(8, 8, 8, 8);
 SUCCESS

mysql-test/suite/binlog/r/binlog_stm_blackhole.result

@@ -99,6 +99,8 @@ alter table t1 drop b;
 create table t3 like t1;
 insert into t1 select * from t3;
 replace into t1 select * from t3;
+Warnings:
+Note	1592	Unsafe statement written to the binary log using statement format since BINLOG_FORMAT = STATEMENT. REPLACE... SELECT is unsafe because the order in which rows are retrieved by the SELECT determines which (if any) rows are replaced. This order cannot be predicted and may differ on master and the slave.
 select * from t1;
 a
 select * from t2;

mysql-test/suite/binlog/r/binlog_unsafe.result

@@ -2352,6 +2352,7 @@ Note	1592	Unsafe statement written to the binary log using statement format sinc
 REPLACE INTO t1 SELECT * FROM t1 LIMIT 1;
 Warnings:
 Note	1592	Unsafe statement written to the binary log using statement format since BINLOG_FORMAT = STATEMENT. The statement is unsafe because it uses a LIMIT clause. This is unsafe because the set of rows included cannot be predicted.
+Note	1592	Unsafe statement written to the binary log using statement format since BINLOG_FORMAT = STATEMENT. REPLACE... SELECT is unsafe because the order in which rows are retrieved by the SELECT determines which (if any) rows are replaced. This order cannot be predicted and may differ on master and the slave.
 UPDATE t1 SET a=1 LIMIT 1;
 Warnings:
 Note	1592	Unsafe statement written to the binary log using statement format since BINLOG_FORMAT = STATEMENT. The statement is unsafe because it uses a LIMIT clause. This is unsafe because the set of rows included cannot be predicted.
@@ -2368,6 +2369,7 @@ END|
 CALL p1();
 Warnings:
 Note	1592	Unsafe statement written to the binary log using statement format since BINLOG_FORMAT = STATEMENT. The statement is unsafe because it uses a LIMIT clause. This is unsafe because the set of rows included cannot be predicted.
+Note	1592	Unsafe statement written to the binary log using statement format since BINLOG_FORMAT = STATEMENT. REPLACE... SELECT is unsafe because the order in which rows are retrieved by the SELECT determines which (if any) rows are replaced. This order cannot be predicted and may differ on master and the slave.
 DROP PROCEDURE p1;
 DROP TABLE t1;
 DROP TABLE IF EXISTS t1;
@@ -2651,4 +2653,40 @@ a
 13:46:40
 1970-01-12 13:46:40
 DROP TABLE t1;
+CREATE TABLE filler_table (a INT, b INT);
+INSERT INTO filler_table values (1,1),(1,2);
+CREATE TABLE insert_table (a INT, b INT, PRIMARY KEY(a));
+CREATE TABLE replace_table (a INT, b INT, PRIMARY KEY(a));
+INSERT INTO replace_table values (1,1),(2,2);
+CREATE TABLE update_table (a INT, b INT, PRIMARY KEY(a));
+INSERT INTO update_table values (1,1),(2,2);
+INSERT IGNORE INTO insert_table SELECT * FROM filler_table;
+Warnings:
+Note	1592	Unsafe statement written to the binary log using statement format since BINLOG_FORMAT = STATEMENT. INSERT IGNORE... SELECT is unsafe because the order in which rows are retrieved by the SELECT determines which (if any) rows are ignored. This order cannot be predicted and may differ on master and the slave.
+TRUNCATE TABLE insert_table;
+INSERT INTO insert_table SELECT * FROM filler_table ON DUPLICATE KEY UPDATE a = 1;
+Warnings:
+Note	1592	Unsafe statement written to the binary log using statement format since BINLOG_FORMAT = STATEMENT. INSERT... SELECT... ON DUPLICATE KEY UPDATE is unsafe because the order in which rows are retrieved by the SELECT determines which (if any) rows are updated. This order cannot be predicted and may differ on master and the slave.
+TRUNCATE TABLE insert_table;
+REPLACE INTO replace_table SELECT * FROM filler_table;
+Warnings:
+Note	1592	Unsafe statement written to the binary log using statement format since BINLOG_FORMAT = STATEMENT. REPLACE... SELECT is unsafe because the order in which rows are retrieved by the SELECT determines which (if any) rows are replaced. This order cannot be predicted and may differ on master and the slave.
+UPDATE IGNORE update_table SET a=2;
+Warnings:
+Note	1592	Unsafe statement written to the binary log using statement format since BINLOG_FORMAT = STATEMENT. UPDATE IGNORE is unsafe because the order in which rows are updated determines which (if any) rows are ignored. This order cannot be predicted and may differ on master and the slave.
+CREATE TABLE create_ignore_test (a INT, b INT, PRIMARY KEY(b)) IGNORE SELECT * FROM filler_table;
+Warnings:
+Note	1592	Unsafe statement written to the binary log using statement format since BINLOG_FORMAT = STATEMENT. CREATE... IGNORE SELECT is unsafe because the order in which rows are retrieved by the SELECT determines which (if any) rows are ignored. This order cannot be predicted and may differ on master and the slave.
+CREATE TABLE create_replace_test (a INT, b INT, PRIMARY KEY(b)) REPLACE SELECT * FROM filler_table;
+Warnings:
+Note	1592	Unsafe statement written to the binary log using statement format since BINLOG_FORMAT = STATEMENT. CREATE... REPLACE SELECT is unsafe because the order in which rows are retrieved by the SELECT determines which (if any) rows are replaced. This order cannot be predicted and may differ on master and the slave.
+CREATE TEMPORARY TABLE temp1 (a INT, b INT, PRIMARY KEY(b)) REPLACE SELECT * FROM filler_table;
+Warnings:
+Note	1592	Unsafe statement written to the binary log using statement format since BINLOG_FORMAT = STATEMENT. CREATE... REPLACE SELECT is unsafe because the order in which rows are retrieved by the SELECT determines which (if any) rows are replaced. This order cannot be predicted and may differ on master and the slave.
+DROP TABLE filler_table;
+DROP TABLE insert_table;
+DROP TABLE update_table;
+DROP TABLE replace_table;
+DROP TABLE create_ignore_test;
+DROP TABLE create_replace_test;
 "End of tests"

mysql-test/suite/binlog/t/binlog_unsafe.test

@@ -12,6 +12,11 @@
 #  - insert into two autoinc columns;
 #  - statements using UDF's.
 #  - statements reading from log tables in the mysql database.
+#  - INSERT ... SELECT ... ON DUPLICATE KEY UPDATE
+#  - REPLACE ... SELECT
+#  - CREATE TABLE [IGNORE/REPLACE] SELECT
+#  - INSERT IGNORE...SELECT
+#  - UPDATE IGNORE
 #
 # Note that statements that use stored functions, stored procedures,
 # triggers, views, or prepared statements that invoke unsafe
@@ -79,6 +84,7 @@
 # BUG#45785: LIMIT in SP does not cause RBL if binlog_format=MIXED
 # BUG#47995: Mark user functions as unsafe
 # BUG#49222: Mark RAND() unsafe
+# BUG#11758262: MARK INSERT...SEL...ON DUP KEY UPD,REPLACE...SEL,CREATE...[IGN|REPL] SEL
 #
 # ==== Related test cases ====
 #
@@ -699,5 +705,47 @@ INSERT INTO t1 VALUES
 SELECT * FROM t1;
 
 DROP TABLE t1;
+#
+#BUG#11758262-50439: MARK INSERT...SEL...ON DUP KEY UPD,REPLACE..
+#The following statement may be unsafe when logged in statement format.
+#INSERT IGNORE...SELECT 
+#INSERT ... SELECT ... ON DUPLICATE KEY UPDATE 
+#REPLACE ... SELECT 
+#UPDATE IGNORE 
+#CREATE TABLE... IGNORE SELECT 
+#CREATE TABLE... REPLACE SELECT
+
+#setup tables
+CREATE TABLE filler_table (a INT, b INT);
+INSERT INTO filler_table values (1,1),(1,2);
+CREATE TABLE insert_table (a INT, b INT, PRIMARY KEY(a));
+CREATE TABLE replace_table (a INT, b INT, PRIMARY KEY(a));
+INSERT INTO replace_table values (1,1),(2,2);
+CREATE TABLE update_table (a INT, b INT, PRIMARY KEY(a));
+INSERT INTO update_table values (1,1),(2,2);
+
+#INSERT IGNORE... SELECT
+INSERT IGNORE INTO insert_table SELECT * FROM filler_table;
+TRUNCATE TABLE insert_table;
+#INSERT ... SELECT ... ON DUPLICATE KEY UPDATE 
+INSERT INTO insert_table SELECT * FROM filler_table ON DUPLICATE KEY UPDATE a = 1;
+TRUNCATE TABLE insert_table;
+#REPLACE...SELECT
+REPLACE INTO replace_table SELECT * FROM filler_table;
+#UPDATE IGNORE
+UPDATE IGNORE update_table SET a=2;
+#CREATE TABLE [IGNORE/REPLACE] SELECT
+CREATE TABLE create_ignore_test (a INT, b INT, PRIMARY KEY(b)) IGNORE SELECT * FROM filler_table;
+CREATE TABLE create_replace_test (a INT, b INT, PRIMARY KEY(b)) REPLACE SELECT * FROM filler_table;
+#temporary tables should not throw the warning.
+CREATE TEMPORARY TABLE temp1 (a INT, b INT, PRIMARY KEY(b)) REPLACE SELECT * FROM filler_table;
+
+###clean up
+DROP TABLE filler_table;
+DROP TABLE insert_table;
+DROP TABLE update_table;
+DROP TABLE replace_table;
+DROP TABLE create_ignore_test;
+DROP TABLE create_replace_test;
 
 --echo "End of tests"

mysql-test/suite/rpl/r/rpl_insert_duplicate.result

-include/master-slave.inc
-[connection master]
-CREATE TABLE t1 (
-a INT UNSIGNED NOT NULL PRIMARY KEY
-) ENGINE=innodb;
-CREATE TABLE t2 (
-a INT UNSIGNED
-) ENGINE=innodb;
-INSERT INTO t1 VALUES (1);
-INSERT INTO t2 VALUES (1);
-INSERT INTO t1 SELECT t2.a FROM t2 ORDER BY t2.a ON DUPLICATE KEY UPDATE t1.a= t1.a;
-include/assert.inc [Sum of elements in t1 should be 1.]
-include/assert.inc [In SBR or MIXED modes, the event in the binlog should be the same that was executed. In RBR mode, binlog position should stay unchanged.]
-include/diff_tables.inc [master:test.t1 , slave:test.t1]
-drop table t1, t2;
-CREATE TABLE t1 (
-a INT UNSIGNED NOT NULL PRIMARY KEY
-) ENGINE=myisam;
-CREATE TABLE t2 (
-a INT UNSIGNED
-) ENGINE=myisam;
-INSERT INTO t1 VALUES (1);
-INSERT INTO t2 VALUES (1);
-INSERT INTO t1 SELECT t2.a FROM t2 ORDER BY t2.a ON DUPLICATE KEY UPDATE t1.a= t1.a;
-include/assert.inc [Sum of elements in t1 should be 1.]
-include/assert.inc [In SBR or MIXED modes, the event in the binlog should be the same that was executed. In RBR mode, binlog position should stay unchanged.]
-include/diff_tables.inc [master:test.t1 , slave:test.t1]
-drop table t1, t2;
-include/rpl_end.inc

mysql-test/suite/rpl/r/rpl_insert_ignore.result

@@ -26,6 +26,19 @@ include/diff_tables.inc [master:test.t1 , slave:test.t1]
 INSERT IGNORE INTO t1 SELECT NULL, t2.b FROM t2 ORDER BY t2.a;
 include/assert.inc [Count of elements in t1 should be 6.]
 include/assert.inc [In SBR or MIXED modes, the event in the binlog should be the same that was executed. In RBR mode, binlog position should stay unchanged.]
+DROP TABLE t1;
+DROP TABLE t2;
+CREATE TABLE t1 (
+a INT UNSIGNED NOT NULL PRIMARY KEY
+) ENGINE=innodb;
+CREATE TABLE t2 (
+a INT UNSIGNED
+) ENGINE=innodb;
+INSERT INTO t1 VALUES (1);
+INSERT INTO t2 VALUES (1);
+INSERT INTO t1 SELECT t2.a FROM t2 ORDER BY t2.a ON DUPLICATE KEY UPDATE t1.a= t1.a;
+include/assert.inc [Sum of elements in t1 should be 1.]
+include/assert.inc [In SBR or MIXED modes, the event in the binlog should be the same that was executed. In RBR mode, binlog position should stay unchanged.]
 drop table t1, t2;
 CREATE TABLE t1 (
 a int unsigned not null auto_increment primary key,
@@ -52,5 +65,18 @@ include/diff_tables.inc [master:test.t1 , slave:test.t1]
 INSERT IGNORE INTO t1 SELECT NULL, t2.b FROM t2 ORDER BY t2.a;
 include/assert.inc [Count of elements in t1 should be 6.]
 include/assert.inc [In SBR or MIXED modes, the event in the binlog should be the same that was executed. In RBR mode, binlog position should stay unchanged.]
+DROP TABLE t1;
+DROP TABLE t2;
+CREATE TABLE t1 (
+a INT UNSIGNED NOT NULL PRIMARY KEY
+) ENGINE=myisam;
+CREATE TABLE t2 (
+a INT UNSIGNED
+) ENGINE=myisam;
+INSERT INTO t1 VALUES (1);
+INSERT INTO t2 VALUES (1);
+INSERT INTO t1 SELECT t2.a FROM t2 ORDER BY t2.a ON DUPLICATE KEY UPDATE t1.a= t1.a;
+include/assert.inc [Sum of elements in t1 should be 1.]
+include/assert.inc [In SBR or MIXED modes, the event in the binlog should be the same that was executed. In RBR mode, binlog position should stay unchanged.]
 drop table t1, t2;
 include/rpl_end.inc

mysql-test/suite/rpl/r/rpl_insert_select.result

-include/master-slave.inc
-[connection master]
-create table t1 (n int not null primary key);
-insert into t1 values (1);
-create table t2 (n int);
-insert into t2 values (1);
-insert ignore into t1 select * from t2;
-insert into t1 values (2);
-select * from t1;
-n
-1
-2
-drop table t1,t2;
-include/rpl_end.inc

mysql-test/suite/rpl/r/rpl_known_bugs_detection.result

@@ -42,12 +42,16 @@ SELECT t2.field_a, t2.field_b, t2.field_c
 FROM t2
 ON DUPLICATE KEY UPDATE
 t1.field_3 = t2.field_c;
+Warnings:
+Note	1592	Unsafe statement written to the binary log using statement format since BINLOG_FORMAT = STATEMENT. INSERT... SELECT... ON DUPLICATE KEY UPDATE is unsafe because the order in which rows are retrieved by the SELECT determines which (if any) rows are updated. This order cannot be predicted and may differ on master and the slave.
 INSERT INTO t2 (field_a, field_b, field_c) VALUES (6, 'f', '6f');
 INSERT INTO t1 (field_1, field_2, field_3)
 SELECT t2.field_a, t2.field_b, t2.field_c
 FROM t2
 ON DUPLICATE KEY UPDATE
 t1.field_3 = t2.field_c;
+Warnings:
+Note	1592	Unsafe statement written to the binary log using statement format since BINLOG_FORMAT = STATEMENT. INSERT... SELECT... ON DUPLICATE KEY UPDATE is unsafe because the order in which rows are retrieved by the SELECT determines which (if any) rows are updated. This order cannot be predicted and may differ on master and the slave.
 SELECT * FROM t1;
 id	field_1	field_2	field_3
 1	1	a	1a

mysql-test/suite/rpl/t/rpl_insert_duplicate.test

-#########################################
-# Wrapper for rpl_insert_duplicate.test #
-#########################################
--- source include/master-slave.inc
--- source include/have_innodb.inc
-#-- source include/have_binlog_format_mixed_or_statement.inc
-
-let $engine_type=innodb;
--- source extra/rpl_tests/rpl_insert_duplicate.test
-
-let $engine_type=myisam;
--- source extra/rpl_tests/rpl_insert_duplicate.test
-
---source include/rpl_end.inc

mysql-test/suite/rpl/t/rpl_insert_select.test

-# Testcase for BUG#10456 - INSERT INTO ... SELECT violating a primary key
-# breaks replication
-
--- source include/master-slave.inc
-connection master;
-
-create table t1 (n int not null primary key);
-insert into t1 values (1);
-create table t2 (n int);
-insert into t2 values (1);
-insert ignore into t1 select * from t2;
-insert into t1 values (2);
-sync_slave_with_master;
-connection slave;
-select * from t1;
-
-connection master;
-drop table t1,t2;
-sync_slave_with_master;
---source include/rpl_end.inc

mysql-test/suite/rpl/t/rpl_known_bugs_detection.test

@@ -7,7 +7,7 @@ source include/have_debug.inc;
 source include/master-slave.inc;
 
 # Currently only statement-based-specific bugs are here
--- source include/have_binlog_format_mixed_or_statement.inc
+-- source include/have_binlog_format_statement.inc
 
 #
 # This is to test that slave properly detects if

sql/share/errmsg-utf8.txt

@@ -6421,6 +6421,24 @@ ER_INDEX_CORRUPT
 ER_UNDO_RECORD_TOO_BIG
   eng "Undo log record is too big."
 
+ER_BINLOG_UNSAFE_INSERT_IGNORE_SELECT
+  eng "INSERT IGNORE... SELECT is unsafe because the order in which rows are retrieved by the SELECT determines which (if any) rows are ignored. This order cannot be predicted and may differ on master and the slave."
+
+ER_BINLOG_UNSAFE_INSERT_SELECT_UPDATE
+  eng "INSERT... SELECT... ON DUPLICATE KEY UPDATE is unsafe because the order in which rows are retrieved by the SELECT determines which (if any) rows are updated. This order cannot be predicted and may differ on master and the slave."
+
+ER_BINLOG_UNSAFE_REPLACE_SELECT
+ eng "REPLACE... SELECT is unsafe because the order in which rows are retrieved by the SELECT determines which (if any) rows are replaced. This order cannot be predicted and may differ on master and the slave."
+
+ER_BINLOG_UNSAFE_CREATE_IGNORE_SELECT
+  eng "CREATE... IGNORE SELECT is unsafe because the order in which rows are retrieved by the SELECT determines which (if any) rows are ignored. This order cannot be predicted and may differ on master and the slave."
+
+ER_BINLOG_UNSAFE_CREATE_REPLACE_SELECT
+  eng "CREATE... REPLACE SELECT is unsafe because the order in which rows are retrieved by the SELECT determines which (if any) rows are replaced. This order cannot be predicted and may differ on master and the slave."
+
+ER_BINLOG_UNSAFE_UPDATE_IGNORE
+  eng "UPDATE IGNORE is unsafe because the order in which rows are updated determines which (if any) rows are ignored. This order cannot be predicted and may differ on master and the slave."
+
 ER_PLUGIN_NO_UNINSTALL
   eng "Plugin '%s' is marked as not dynamically uninstallable. You have to stop the server to uninstall it."
 

sql/sql_lex.cc

@@ -59,6 +59,12 @@ Query_tables_list::binlog_stmt_unsafe_errcode[BINLOG_STMT_UNSAFE_COUNT] =
   ER_BINLOG_UNSAFE_NONTRANS_AFTER_TRANS,
   ER_BINLOG_UNSAFE_MULTIPLE_ENGINES_AND_SELF_LOGGING_ENGINE,
   ER_BINLOG_UNSAFE_MIXED_STATEMENT,
+  ER_BINLOG_UNSAFE_INSERT_IGNORE_SELECT,
+  ER_BINLOG_UNSAFE_INSERT_SELECT_UPDATE,
+  ER_BINLOG_UNSAFE_REPLACE_SELECT,
+  ER_BINLOG_UNSAFE_CREATE_IGNORE_SELECT,
+  ER_BINLOG_UNSAFE_CREATE_REPLACE_SELECT,
+  ER_BINLOG_UNSAFE_UPDATE_IGNORE
 };
 
 

sql/sql_lex.h

@@ -1253,6 +1253,48 @@ public:
     */
     BINLOG_STMT_UNSAFE_MIXED_STATEMENT,
 
+    /**
+      INSERT...IGNORE SELECT is unsafe because which rows are ignored depends
+      on the order that rows are retrieved by SELECT. This order cannot be
+      predicted and may differ on master and the slave.
+    */
+    BINLOG_STMT_UNSAFE_INSERT_IGNORE_SELECT,
+
+    /**
+      INSERT...SELECT...UPDATE is unsafe because which rows are updated depends
+      on the order that rows are retrieved by SELECT. This order cannot be
+      predicted and may differ on master and the slave.
+    */
+    BINLOG_STMT_UNSAFE_INSERT_SELECT_UPDATE,
+
+    /**
+      INSERT...REPLACE SELECT is unsafe because which rows are replaced depends
+      on the order that rows are retrieved by SELECT. This order cannot be
+      predicted and may differ on master and the slave.
+    */
+    BINLOG_STMT_UNSAFE_REPLACE_SELECT,
+
+    /**
+      CREATE TABLE... IGNORE... SELECT is unsafe because which rows are ignored
+      depends on the order that rows are retrieved by SELECT. This order cannot
+      be predicted and may differ on master and the slave.
+    */
+    BINLOG_STMT_UNSAFE_CREATE_IGNORE_SELECT,
+
+    /**
+      CREATE TABLE...REPLACE... SELECT is unsafe because which rows are replaced
+      depends on the order that rows are retrieved from SELECT. This order
+      cannot be predicted and may differ on master and the slave
+    */
+    BINLOG_STMT_UNSAFE_CREATE_REPLACE_SELECT,
+
+    /**
+      UPDATE...IGNORE is unsafe because which rows are ignored depends on the
+      order that rows are updated. This order cannot be predicted and may differ
+      on master and the slave.
+    */
+    BINLOG_STMT_UNSAFE_UPDATE_IGNORE,
+
     /* The last element of this enumeration type. */
     BINLOG_STMT_UNSAFE_COUNT
   };

sql/sql_parse.cc

@@ -2386,6 +2386,19 @@ case SQLCOM_PREPARE:
     {
       select_result *result;
 
+      /*
+        CREATE TABLE...IGNORE/REPLACE SELECT... can be unsafe, unless
+        ORDER BY PRIMARY KEY clause is used in SELECT statement. We therefore
+        use row based logging if mixed or row based logging is available.
+        TODO: Check if the order of the output of the select statement is
+        deterministic. Waiting for BUG#42415
+      */
+      if(lex->ignore)
+        lex->set_stmt_unsafe(LEX::BINLOG_STMT_UNSAFE_CREATE_IGNORE_SELECT);
+      
+      if(lex->duplicates == DUP_REPLACE)
+        lex->set_stmt_unsafe(LEX::BINLOG_STMT_UNSAFE_CREATE_REPLACE_SELECT);
+
       /*
         If:
         a) we inside an SP and there was NAME_CONST substitution,
@@ -2720,6 +2733,16 @@ end_with_restore_list:
     DBUG_ASSERT(first_table == all_tables && first_table != 0);
     if (update_precheck(thd, all_tables))
       break;
+
+    /*
+      UPDATE IGNORE can be unsafe. We therefore use row based
+      logging if mixed or row based logging is available.
+      TODO: Check if the order of the output of the select statement is
+      deterministic. Waiting for BUG#42415
+    */
+    if (lex->ignore)
+      lex->set_stmt_unsafe(LEX::BINLOG_STMT_UNSAFE_UPDATE_IGNORE);
+
     DBUG_ASSERT(select_lex->offset_limit == 0);
     unit->set_limit(select_lex);
     MYSQL_UPDATE_START(thd->query());
@@ -2886,6 +2909,23 @@ end_with_restore_list:
     DBUG_ASSERT(first_table == all_tables && first_table != 0);
     if ((res= insert_precheck(thd, all_tables)))
       break;
+    /*
+      INSERT...SELECT...ON DUPLICATE KEY UPDATE/REPLACE SELECT/
+      INSERT...IGNORE...SELECT can be unsafe, unless ORDER BY PRIMARY KEY
+      clause is used in SELECT statement. We therefore use row based
+      logging if mixed or row based logging is available.
+      TODO: Check if the order of the output of the select statement is
+      deterministic. Waiting for BUG#42415
+    */
+    if (lex->sql_command == SQLCOM_INSERT_SELECT &&
+        lex->duplicates == DUP_UPDATE)
+      lex->set_stmt_unsafe(LEX::BINLOG_STMT_UNSAFE_INSERT_SELECT_UPDATE);
+
+    if (lex->sql_command == SQLCOM_INSERT_SELECT && lex->ignore)
+      lex->set_stmt_unsafe(LEX::BINLOG_STMT_UNSAFE_INSERT_IGNORE_SELECT);
+
+    if (lex->sql_command == SQLCOM_REPLACE_SELECT)
+      lex->set_stmt_unsafe(LEX::BINLOG_STMT_UNSAFE_REPLACE_SELECT);
 
     /* Fix lock for first table */
     if (first_table->lock_type == TL_WRITE_DELAYED)