FULL
Bug#21753696: MAKE SHOW SLAVE STATUS NON BLOCKING IF IO
THREAD WAITS FOR DISK SPACE

Problem:
========
Currently SHOW SLAVE STATUS blocks if IO thread waits for
disk space. This makes automation tools verifying
server health block on taking relevant action. Finally this
will create SHOW SLAVE STATUS piles.

Analysis:
=========
SHOW SLAVE STATUS hangs on mi->data_lock if relay log write
is waiting for free disk space while holding mi->data_lock.
mi->data_lock is needed to protect the format description
event (mi->format_description_event) which is accessed by
the clients running FLUSH LOGS and slave IO thread. Note
relay log writes don't need to be protected by
mi->data_lock, LOCK_log is used to protect relay log between
IO and SQL thread (see MYSQL_BIN_LOG::append_event). The
code takes mi->data_lock to protect
mi->format_description_event during relay log rotate which
might get triggered right after relay log write.

Fix:
====
Release the data_lock just for the duration of writing into
relay log.

Made change to ensure the following lock order is maintained
to avoid deadlocks.

data_lock, LOCK_log

data_lock is held during relay log rotations to protect
the description event.

REPLICATION

Problem: In RBR mode, merge table updates are not successfully applied on a cascading
replication.

Analysis & Fix: Every type of row event is preceded by one or more table_map_log_events
that gives the information about all the tables that are involved in the row
event. Server maintains the list in RPL_TABLE_LIST and it goes through all the
tables and checks for the compatibility between master and slave. Before
checking for the compatibility, it calls 'open_tables()' which takes the list
of all tables that needs to be locked and opened. In RBR, because of the
Table_map_log_event , we already have all the tables including base tables in
the list. But the open_tables() which is generic call takes care of appending
base tables if the list contains merge tables. There is an assumption in the
current replication layer logic that these tables (TABLE_LIST type objects) are always
added in the end of the list. Replication layer maintains the count of
tables(tables_to_lock_count) that needs to be verified for compatibility check
and runs through only those many tables from the list and rest of the objects
in linked list can be skipped. But this assumption is wrong.
open_tables()->..->add_children_to_list() adds base tables to the list immediately
after seeing the merge table in the list.

For eg: If the list passed to open_tables() is t1->t2->t3 where t3 is merge
table (and t1 and t2 are base tables), it adds t1'->t2' to the list after t3.
New table list looks like t1->t2->t3->t1'->t2'. It looks like it added at the
end of the list but that is not correct. If the list passed to open_tables()
is t3->t1->t2 where t3 is merge table (and t1 and t2 are base tables), the new
prepared list will be t3->t1'->t2'->t1->t2. Where t1' and t2' are of
TABLE_LIST objects which were added by add_children_to_list() call and replication
layer should not look into them. Here tables_to_lock_count  will not help as the
objects are added in between the list.

Fix: After investigating add_children_list() logic (which is called from open_tables()),
there is no flag/logic in it to skip adding the children to the list even if the
children are already included in the table list. Hence to fix the issue, a
logic should be added in the replication layer to skip children in the list by
checking whether  'parent_l' is non-null or not. If it is children, we will skip 'compatibility'
check for that table.

Also this patch is not removing 'tables_to_lock_count' logic for the performance issues
if there are any children at the end of the list, those can be easily skipped directly by
stopping the loop with tables_to_lock_count check.

              FOUND

Description:- Failure during the validation of CA
certificate path which is provided as an option for 'ssl-ca'
returns two different errors for YaSSL and OPENSSL.

Analysis:- 'ssl-ca', option used for specifying the ssl ca
certificate path. Failing to validate this certificate with
OPENSSL returns an error, "ERROR 2026 (HY000): SSL
connection error: SSL_CTX_set_default_verify_paths failed".
While YASSL returns "ERROR 2026 (HY000): SSL connection
error: ASN: bad other signature confirmation". Error
returned by the OPENSSL is correct since
"SSL_CTX_load_verify_locations()" returns 0 (in case of
OPENSSL) for the failure and sets error as
"SSL_INITERR_BAD_PATHS". In case of YASSL,
"SSL_CTX_load_verify_locations()" returns an error number
which is less than or equal to 0 in case of error. Error
numbers for YASSL is mentioned in the file,
'extra/yassl/include/openssl/ssl.h'(line no : 292). Also
'ssl-ca' does not accept tilde home directory path
substitution.

Fix:- The condition which checks for the error in the
"SSL_CTX_load_verify_locations()" is changed in order to
accommodate YASSL as well. A logic is written in
"mysql_ssl_set()" in order accept the tilde home directory
path substitution for all ssl options.

The main.merge test case was failing when tested using row based
binlog format.

While analyzing the issue it was found the following issues:

a) The server is calling binlog related code even when a statement will
   not be binlogged;
b) The child table list was not present into table structure by the time
   to generate the create table statement;
c) The tables in the child table list will not be opened yet when
   generating table create info using row based replication;
d) CREATE TABLE LIKE TEMP_TABLE does not preserve original table storage
   engine when using row based replication;

This patch addressed all above issues.

@ sql/sql_class.h

Added a function to determine if the binary log is disabled to
  the current session. This is related with issue (a) above.

@ sql/sql_table.cc

Added code to skip binary logging related code if the statement
  will not be binlogged. This is related with issue (a) above.

Added code to add the children to the query list of the table that
  will have its CREATE TABLE generated. This is related with issue (b)
  above.

Added code to force the storage engine to be generated into the
  CREATE TABLE. This is related with issue (d) above.

@ storage/myisammrg/ha_myisammrg.cc

Added a test to skip a table getting info about a child table if the
  child table is not opened. This is related to issue (c) above.

No commit message

No commit message

No commit message

              CONSTRAINT.

Analysis
=======

INSERT and UPDATE operations using the IGNORE keyword which
causes FOREIGN KEY constraint violations reports an error
despite using the IGNORE keyword.

Foreign key violation errors were not ignored and reported
as errors instead of warnings even when IGNORE was set.

Fix
===
Added code to ignore the foreign key violation errors and
report them as warnings when the IGNORE keyword is used.

Fix the following two build warnings so that 5.5 can be compiled
with GCC5.

storage/innobase/dict/dict0crea.c:1143:21: error: logical not is only applied
to the left hand side of comparison [-Werror=logical-not-parentheses]
   ut_a(!node->index == (err != DB_SUCCESS));
                     ^
storage/innobase/log/log0recv.c:1770:20: error: logical not is only applied
to the left hand side of comparison [-Werror=logical-not-parentheses]
  ut_ad(!allow_ibuf == mutex_own(&log_sys->mutex));
                    ^

               ITEM_PARAM::SAFE_CHARSET_CONVERTER

ISSUE:
------
Charset conversion on a null parameter is not handled
correctly.

SOLUTION:
---------
Item_param's charset converter does not handle the case
where it might have to deal with a null value. This is
fine for other charset converters since the value is not
supplied to them at runtime.

The fix is to check if the parameter is now set to null and
return an Item_null object. Also, there is no need to
initialize Item_param's cnvitem in the constructor to a
string. This can be done in
ITEM_PARAM::SAFE_CHARSET_CONVERTER itself.

Members of Item_param, cnvbuf and cnvstr, have been removed
and cnvitem has been made a local variable in
ITEM_PARAM::SAFE_CHARSET_CONVERTER.

'SYSTEM LOCK' IN PROCESSLIST

Analysis
=========
Show processlist shows 'System Lock' in 'State' field while
LOAD DATA INFILE is running.

thd->proc_info update is missing in LOAD DATA INFILE path.
Thus any request will get last unpdated status from lock_table()
during open_table().

Fix:
=======
Update state information from LOAD DATA INFILE path.

    Enable mysqld service by default in sysv initscrips

Go back to __sync_* operations in TAS on x86.

Patch is reviewed by Sunny over IM.

Post-push fix: The problem was that condition variable
timeouts could in some cases (slow machines and/or short
timeouts) be infinite.

When the number of milliseconds to wait is computed, the
end time is computed before the now() time. This can result
in the now() time being later than the end time, leading to
negative timeout. Which after conversion to unsigned becomes
~infinite.

This patch fixes the problem by explicitly checking if we
get negative timeout and then using 0 if this is the case.

Post push fix for 5.5 and 5.6.Disabled the test code due to Bug#22587377

Problem:
mysql-test/suite/rpl/t/rpl_killed_ddl.test

This test contains code which was disabled because of certain bugs.
BUG#44041 declared to be a duplicate of Bug#45516 which was fixed 2010
BUG#43353 fixed 2012
BUG#44171 fixed 2010

Fix:
Enabled the test code related to the above mentioned bugs.

              GENERATED BY THE EXP() FUNCTION

When generating the error message for numeric overflow, pass a flag to
Item::print() that prevents it from expanding constant expressions and
parameters to the values they evaluate to.

For consistency, also pass the flag to Item::print() when
Item_func_spatial_collection::fix_length_and_dec() generates an error
message. It doesn't make any difference at the moment, since constant
expressions haven't been evaluated yet when this function is called.

we can see from the hang stacktrace, srv_monitor_thread is blocked
when getting log_sys::mutex, so that sync_arr_wake_threads_if_sema_free
cannot get a change to break the mutex deadlock.

The fix is simply removing any mutex wait in srv_monitor_thread.

Patch is reviewed by Sunny over IM.

we can see from the hang stacktrace, srv_monitor_thread is blocked
when getting log_sys::mutex, so that sync_arr_wake_threads_if_sema_free
cannot get a change to break the mutex deadlock.

The fix is simply removing any mutex wait in srv_monitor_thread.

Patch is reviewed by Sunny over IM.

Test fix for 5.5 and 5.6

Test Fix

               PROCEDURE RESULTS IN GARBAGE BYTES

Issue:
-----
This problem occurs under the following conditions:

a) Stored procedure has a variable is declared as TEXT/BLOB.
b) Data is copied into the the variable using the
   SELECT...INTO syntax from a TEXT/BLOB column.

Data corruption can occur in such cases.

SOLUTION:
---------
The blob type does not allocate space for the string to be
stored. Instead it contains a pointer to the source string.
Since the source is deallocated immediately after the
select statement, this can cause data corruption.

As part of the fix for Bug #21143080, when the source was
part of the table's write-set, blob would allocate the
neccessary space. But this fix missed the possibility that,
as in the above case, the target might be a variable.

The fix will add the copy_blobs check that was removed by
the earlier fix.

Problem Statement
=========
Fix various issues when building MySQL with Visual Studio 2015.

Fix:
=======
- Visual Studio 2015 adds support for timespec. Add check and
  related code to use this and only use our replacement if
  timespec is not defined.
- Rename lfind/lsearch to my* to avoid redefinition problems.
- Set default value for TMPDIR to "" on Windows as P_tmpdir
  no longer exists.
- using VS definition of snprintf if available
- tzname are now renamed to _tzname.

one leading zeros for the key agreement instead of
potentially any number.
There is about 1 in 50,000 connections to fail
when using DHE cipher suites.  The second problem was the
case where a server would send a public value shorter than
the prime value, causing about 1 in 128 client connections
to fail, and also caused the yaSSL client to read off the
end of memory.
All client side DHE cipher suite users should update.
Note: The patch is received from YaSSL people

               UNIX_TIMESTAMP(STR_TO_DATE('201506', "%Y%M"

Issue:
-----
When an invalid date is supplied to the UNIX_TIMESTAMP
function from STR_TO_DATE, no check is performed before
converting it to a timestamp value.

SOLUTION:
---------
Add the check_date function and only if it succeeds,
proceed to the timestamp conversion.

No warning will be returned for dates having zero in
month/date, since partial dates are allowed. UNIX_TIMESTAMP
will return only a zero for such values.

The problem has been handled in 5.6+ with WL#946.

               COLUMNS

ANALYSIS:
=========
A valgrind error is reported when CREATE TABLE .. SELECT
involving BIT columns triggers a column type redefinition.

In general the pack_flag is set for BIT columns in
'mysql_prepare_create_table()'. However, during the above
operation, redefined column types was handled after the
special handling for BIT columns and thus pack_flag ended
up not being set correctly triggering the valgrind error.

FIX:
====
The patch fixes this problem by setting pack_flag correctly
for BIT columns in the case of column type redefinition.

If the configuration files contains multiple datadir lines, use the last datadir
entry in the RPM installation scripts

Problem:
=======
rpl_binlog_index.test fails with following valgrind error.

line
Conditional jump or move depends on uninitialised value(s)
at 0x4C2F842: __memcmp_sse4_1 (in /usr/lib64/valgrind/
vgpreload_memcheck-amd64-linux.so)
0x739E39: find_uniq_filename(char*) (log.cc:2212)
0x73A11B: MYSQL_LOG::generate_new_name(char*, char const*)
(log.cc:2492)
0x73A1ED: MYSQL_LOG::init_and_set_log_file_name(char const*,
char const*, enum_log_type, cache_type) (log.cc:2289)
0x73B6F5: MYSQL_BIN_LOG::open(char const*, enum_log_type,


Analysis and fix:
=================
This issue was fixed as part of Bug#20459363 fix in 5.6 and
above. Hence backporting the fix to MySQL-5.5.

    Problem:
    ========
    1) Drop table queries are re-generated by server
    before writing the events(queries) into binlog
    for various reasons. If table name/db name contains
    a non regular characters (like latin characters),
    the generated query is wrong. Hence it breaks the
    replication.
    2) In the edge case, when table name/db name contains
    64 characters, server is throwing an assert
    assert(M_TBLLEN < 128)
    3) In the edge case, when db name contains 64 latin
    characters, binlog content is interpreted badly
    which is leading replication failure.

    Analysis & Fix :
    ================
    1) Parser reads the table name from the query and converts
    it to standard charset(utf8) and stores it in table_name variable.
    When drop table query is regenerated with the same table_name
    variable, it should be converted back to the original charset
    from standard charset(utf8).

    2) Latin character takes two bytes for each character. Limit
    of the identifier is 64. SYSTEM_CHARSET_MBMAXLEN is set to '3'.
    So there is a possiblity that tablename/dbname contains 3 * 64.
    Hence assert is changed to
    (M_TBLLEN <= NAME_CHAR_LEN*SYSTEM_CHARSET_MBMAXLEN)

    3) db_len in the binlog event header is taking 1 byte.
       db_len is ranged from 0 to 192 bytes (3 * 64).
       While reading the db_len from the event, server
       is casting to uint instead of uchar which is leading
       to bad db_len. This problem is fixed by changing the
       cast type to uchar.

Fixing pb2 valgrind failure
Missed a 'if condition' check while moving the logic
from one place to another place.

Problem:
At the end of first execution select_lex->prep_where is pointing to
a runtime created object (temporary table field). As a result
server exits trying to access a invalid pointer during second
execution.

Analysis:
While optimizing the join conditions for the query, after the
permanent transformation, optimizer makes a copy of the new
where conditions in select_lex->prep_where. "prep_where" is what
is used as the "where condition" for the query at the start of execution.
W.r.t the query in question, "where" condition is actually pointing
to a field in the temporary table. As a result, for the  second
execution the pointer is no more valid resulting in server exit.

Fix:
At the end of the first execution, select_lex->where will have the
original item of the where condition.
Make prep_where the new place where the original item of select->where
has to be rolled back.
Fixed in 5.7 with the wl#7082 - Move permanent transformations from
JOIN::optimize to JOIN::prepare

Patch for 5.5 includes the following backports from 5.6:

Bugfix for Bug12603141 - This makes the first execute statement in the testcase
pass in 5.5

However it was noted later in in Bug16163596 that the above bugfix needed to
be modified. Although Bug16163596 is reproducible only with changes done for
Bug12582849, we have decided include the fix.

Considering that Bug12582849 is related to Bug12603141, the fix is
also included here. However this results in Bug16317817, Bug16317685,
Bug16739050. So fix for the above three bugs is also part of this patch.

               WITH NEW VALGRIND

Issue:
------
Function signature in valgrind.supp requires a change with
valgrind 3.11. Static function is replaced with wild card.

    Problem & Analysis: If DML invokes a trigger or a
    stored function that inserts into an AUTO_INCREMENT column,
    that DML has to be marked as 'unsafe' statement. If the
    tables are locked in the transaction prior to DML statement
    (using LOCK TABLES), then the same statement is not marked as
    'unsafe' statement. The logic of checking whether unsafeness
    is protected with if (!thd->locked_tables_mode). Hence if
    we lock the tables prior to DML statement, it is *not* entering
    into this if condition. Hence the statement is not marked
    as unsafe statement.

    Fix: Irrespective of locked_tables_mode value, the unsafeness
    check should be done. Now with this patch, the code is moved
    out to 'decide_logging_format()' function where all these checks
    are happening and also with out 'if(!thd->locked_tables_mode)'.
    Along with the specified test case in the bug scenario
    (BINLOG_STMT_UNSAFE_AUTOINC_COLUMNS), we also identified that
    other cases BINLOG_STMT_UNSAFE_AUTOINC_NOT_FIRST,
    BINLOG_STMT_UNSAFE_WRITE_AUTOINC_SELECT, BINLOG_STMT_UNSAFE_INSERT_TWO_KEYS
    are also protected with thd->locked_tables_mode which is not right. All
    of those checks also moved to 'decide_logging_format()' function.