BUG#45749 - Race condition in SET GLOBAL innodb_commit_concurrency=DEFAULT

Detailed revision comments:

r5419 | marko | 2009-06-25 16:11:57 +0300 (Thu, 25 Jun 2009) | 18 lines
branches/5.1: Merge r5418 from branches/zip:

  ------------------------------------------------------------------------
  r5418 | marko | 2009-06-25 15:55:52 +0300 (Thu, 25 Jun 2009) | 5 lines
  Changed paths:
     M /branches/zip/ChangeLog
     M /branches/zip/handler/ha_innodb.cc
     M /branches/zip/mysql-test/innodb_bug42101-nonzero.result
     M /branches/zip/mysql-test/innodb_bug42101-nonzero.test
     M /branches/zip/mysql-test/innodb_bug42101.result
     M /branches/zip/mysql-test/innodb_bug42101.test
  
  branches/zip: Fix a race condition caused by
  SET GLOBAL innodb_commit_concurrency=DEFAULT. (Bug #45749)
  When innodb_commit_concurrency is initially set nonzero,
  DEFAULT would change it back to 0, triggering Bug #42101.
  rb://139 approved by Heikki Tuuri.
  ------------------------------------------------------------------------

1. Fixes build warnings caused by applying snapshot 5.1-ss5282

2. Fix the Makefile.am in storage/innobase to remove the header file
   'fsp0types.h' which was added twice to fix build warning generated 
   after applying the 5.1-ss5282 snapshot

Detailed revision comments:

r5410 | marko | 2009-06-24 22:26:34 +0300 (Wed, 24 Jun 2009) | 2 lines
branches/5.1: Add missing #include "mtr0log.h" to avoid warnings
when compiling with -DUNIV_MUST_NOT_INLINE.

use partial primary key if another index can prevent filesort

The fix for bug #28404 causes the covering ordering indexes to be 
preferred unconditionally over non-covering and ref indexes.

Fixed by comparing the cost of using a covering index to the cost of
using a ref index even for covering ordering indexes.
Added an assertion to clarify the condition the local variables should
be in.

mysql-test/include/mix1.inc:
  Bug #36259: fixed a non-stable test case
mysql-test/r/innodb_mysql.result:
  Bug #36259 and #45828 : test case
mysql-test/t/innodb_mysql.test:
  Bug #36259 and #45828 : test case
sql/sql_select.cc:
  Bug #36259 and #45828 : don't consider covering indexes supperior to
  ref keys.

purge_relay_logs() did not propagate an error happend in count_relay_log_space().

Fixed with the suggestesd setting the error= true.
Note, propagation generally out of purge_relay_logs() was fixed for Bug #44179,
and the issue does not exist in 6.0 thanks to a patch for WL#2775.



sql/rpl_rli.cc:
  the error is set on if count_relay_log_space(rli) fails.

Based upon patch contributed by Stewart Smith

mysql-test/lib/My/SafeProcess/safe_process.cc:
  Fix style -- remove unneeded spaces.
  Specify C linkage for the signal handling functions.
  Check return value from read()/write().

the auto_increment value
      
This is an alternative patch that instead of allowing RECREATE TABLE
on TRUNCATE TABLE it implements reset_auto_increment that is called
after delete_all_rows.

Note: this bug was fixed by Mattias Jonsson:
Pusing this patch: http://lists.mysql.com/commits/70370


mysql-test/suite/parts/r/partition_auto_increment_memory.result:
  Bug#35111: Truncate a MyISAM partitioned table does not reset
  the auto_increment value
mysql-test/suite/parts/r/partition_auto_increment_myisam.result:
  Bug#35111: Truncate a MyISAM partitioned table does not reset
  the auto_increment value
sql/ha_partition.cc:
  Bug#35111: Truncate a MyISAM partitioned table does not reset
  the auto_increment value
  
  Added reset_auto_increment, to be used after delete_all_rows
  to simulate truncate.
storage/heap/ha_heap.cc:
  Bug#35111: Truncate a MyISAM partitioned table does not reset
  the auto_increment value
  
  Added reset_auto_increment, to be used after delete_all_rows
  to simulate truncate
storage/heap/ha_heap.h:
  Bug#35111: Truncate a MyISAM partitioned table does not reset
  the auto_increment value
  
  Added reset_auto_increment, to be used after delete_all_rows
  to simulate truncate
storage/myisam/ha_myisam.cc:
  Bug#35111: Truncate a MyISAM partitioned table does not reset
  the auto_increment value
  
  Added reset_auto_increment, to be used after delete_all_rows
  to simulate truncate.
storage/myisam/ha_myisam.h:
  Bug#35111: Truncate a MyISAM partitioned table does not reset
  the auto_increment value
  
  Added reset_auto_increment, to be used after delete_all_rows
  to simulate truncate.

Item_field::fix_fields()

With ibmdb2i_create_index_option set to 1, creating an IBMDB2I table
with a primary key should produce an additional index that uses EBCDIC
hexadecimal sorting. However, this does not work. Adding indexes that
are not primary keys does work. The ibmdb2i_create_index_option should
be honoured when creating a table with a primary key.

This patch adds code to the create() function to check for the value
of the ibmdb2i_create_index_option variable and, when appropriate, to 
generate a *HEX-based shadow index in DB2 for the primary key. Previously 
this behavior was limited to secondary indexes.

Additionally, this patch restricts the creation of shadow indexes to
cases in which a non-*HEX sort sequence is used, as the documentation
for ibmdb2i_create_index_option describes. Previously, the shadow index
would in some cases be created even when the MySQL-specific index used
*HEX sorting, leading to redundant indexes.

Finally, the code used to generate the list of fields for indexes 
and the code used to generate the SQL statement for the shadow
indexes has been refactored into individual functions.

mysql-test/suite/ibmdb2i/r/ibmdb2i_bug_45983.result:
  Bug#45983 ibmdb2i_create_index_option=1 not working for primary key
  
  Result file for the test case.
mysql-test/suite/ibmdb2i/t/ibmdb2i_bug_45983.test:
  Bug#45983 ibmdb2i_create_index_option=1 not working for primary key
  
  Add tests to verify that the ibmdb2i_create_index_option is being honoured
  when creating a table with a primary key.
storage/ibmdb2i/ha_ibmdb2i.cc:
  Bug#45983 ibmdb2i_create_index_option=1 not working for primary key
  
  - Add code to the create() function to check for the value of the
    ibmdb2i_create_index_option variable and, when appropriate, to 
    generate a *HEX-based shadow index in DB2 for the primary key.
  
  - Restrict the creation of shadow indexes to cases in which a
    non-*HEX sort sequence is used.
  
  - Refractor code used to generate the list of fields for indexes
    and the code used to generate the SQL statement for the shadow
    indexes into individual functions.
storage/ibmdb2i/ha_ibmdb2i.h:
  Bug#45983 ibmdb2i_create_index_option=1 not working for primary key
  
  Add function prototypes for the functions that.
  
  - Generate the list of fields for indexes
  - Generate the SQL statement for the shadow
    indexes

Added code to the .test file to skip this test on Win64 for PB2 stability.
Please remove this code when the bug is fixed.

Had attempted to disable this test on Windows only, but the nature of this bug
does not allow for this.  The master.opt file is processed before anything in 
in the actual test.  As a result, we must use disabled.def files to ensure
these tests are skipped on the problematic platforms.

Removed Windows-only code and updated the proper disabled.def files accordingly.

GHz.

Some collations were causing IBMDB2I to report
inaccurate key range estimations to the optimizer
for LIKE clauses that select substrings. This can
be seen by running EXPLAIN. This problem primarily
affects multi-byte and unicode character sets.

This patch involves substantial changes to several
modules. There are a number of problems with the
character set and collation handling. These problems
have been or are being fixed,  and a comprehensive
test has been included which should provide much
better coverage than there was before. This test
is enabled only for IBM i 6.1, because that version
has support for the greatest number of collations.

mysql-test/suite/ibmdb2i/r/ibmdb2i_collations.result:
  Bug#45803 Inaccurate estimates for partial key values with IBMDB2I
  
  result file for test case.
mysql-test/suite/ibmdb2i/t/ibmdb2i_collations.test:
  Bug#45803 Inaccurate estimates for partial key values with IBMDB2I
  
  Tests for character sets and collations. This test
  is enabled only for IBM i 6.1, because that version
  has support for the greatest number of collations.
storage/ibmdb2i/db2i_conversion.cc:
  Bug#45803 Inaccurate estimates for partial key values with IBMDB2I
  
  - Added support in convertFieldChars to enable records_in_range
    to determine how many substitute characters were inserted and
    to suppress conversion warnings.
  
  - Fixed bug which was causing all multi-byte and Unicode fields
    to be created as UTF16 (CCSID 1200) fields in DB2. The corrected
    code will now create UCS2 fields as UCS2 (CCSID 13488), UTF8
    fields (except for utf8_general_ci) as UTF8 (CCSID 1208), and
    all other multi-byte or Unicode fields as UTF16.  This will only
    affect tables that are newly created through the IBMDB2I storage
    engine. Existing IBMDB2I tables will retain the original CCSID
    until recreated. The existing behavior is believed to be
    functionally correct, but it may negatively impact performance
    by causing unnecessary character conversion. Additionally, users
    accessing IBMDB2I tables through DB2 should be aware that mixing 
    tables created before and after this change may require extra type
    casts or other workarounds.  For this reason, users who have
    existing IBMDB2I tables using a Unicode collation other than
    utf8_general_ci are encouraged to recreate their tables (e.g.
    ALTER TABLE t1 ENGINE=IBMDB2I) in order to get the updated CCSIDs
    associated with their DB2 tables.
  
  - Improved error reporting for unsupported character sets by forcing
    a check for the iconv conversion table at table creation time,
    rather than at data access time.
storage/ibmdb2i/db2i_myconv.h:
  Bug#45803 Inaccurate estimates for partial key values with IBMDB2I
  
  Fix to set errno when iconv fails.
storage/ibmdb2i/db2i_rir.cc:
  Bug#45803 Inaccurate estimates for partial key values with IBMDB2I
  
  Significant improvements were made to the records_in_range code
  that handles partial length string data in keys for optimizer plan
  estimation. Previously, to obtain an estimate for a partial key
  value, the implementation would perform any necessary character
  conversion and then attempt to determine the unpadded length of
  the partial key by searching for the minimum or maximum sort
  character. While this algorithm was sufficient for most single-byte
  character sets, it did not treat Unicode and multi-byte strings
  correctly. Furthermore, due to an operating system limitation,
  partial keys having UTF8 collations (ICU sort sequences in DB2)
  could not be estimated with this method.
  
  With this patch, the code no longer attempts to explicitly determine
  the unpadded length of the key. Instead, the entire key is converted
  (if necessary), including padding, and then passed to the operating
  system for estimation. Depending on the source and target character
  sets and collations, additional logic is required to correctly
  handle cases in which MySQL uses unconvertible or differently
  -weighted values to pad the key. The bulk of the patch exists
  to implement this additional logic.
storage/ibmdb2i/ha_ibmdb2i.h:
  Bug#45803 Inaccurate estimates for partial key values with IBMDB2I
  
  The convertFieldChars declaration was updated to support additional 
  optional behaviors.

timeout
            
In STMT and MIXED modes, a statement that changes both non-transactional and
transactional tables must be written to the binary log whenever there are
changes to non-transactional tables. This means that the statement gets into the
binary log even when the changes to the transactional tables fail. In particular
, in the presence of a failure such statement is annotated with the error number
and wrapped in a begin/rollback. On the slave, while applying the statement, it
is expected the same failure and the rollback prevents the transactional changes
to be persisted.
            
Unfortunately, statements that fail due to concurrency issues (e.g. deadlocks,
timeouts) are logged in the same way causing the slave to stop as the statements
are applied sequentially by the SQL Thread. To fix this bug, we automatically
ignore concurrency failures on the slave. Specifically, the following failures
are ignored: ER_LOCK_WAIT_TIMEOUT, ER_LOCK_DEADLOCK and ER_XA_RBDEADLOCK.