thd->open_tables"

thd->open_tables list is not normally accessed concurrently
except for one case: when the connection has open SQL
HANDLER tables, and we want to perform a DDL on the table,
we want to abort waits on MyISAM thr_lock of those connections
that prevent the DDL from proceeding, and iterate
over thd->open_tables list to find out the tables on which
the thread is waiting.

In 5.5 we mostly use deadlock detection and soft deadlock 
prevention, as opposed to "hard" deadlock prevention
of 5.1, which would abort any transaction that
may cause a deadlock. The only remaining case when
neither deadlock detection nor deadlock prevention
is implemented in 5.5 is HANDLER SQL, where we use
old good thr_lock_abort() technique form 5.1. 

Thus, replace use of LOCK_open to protect thd->open_tables
with thd->LOCK_ha_data (a lock protecting various session
private data).

This is a port of the work done for 5.5.4 for review
and inclusion into 5.5.5.

    
1) No mutex and no function call if we're not using
plugins.
2) If we're above the table definition cache limit,
delete the oldest unused share, not the share on our hands.

3413 Davi Arnaut	2010-06-03
     Don't enable safemalloc for valgrind builds, it's too slow.
to trunk-runtime.

for MyISAM in sysbench OLTP_RW test".

Fixes compilation warnings about local variable
shadowing I_P_List_fast_push_back::last member.

Renamed I_P_List_fast_push_back::last member
to m_last. Also to keep member naming consistent
renamed I_P_List::first member to m_first.

information_schema query.
Porting a fix done for a feature tree by Ingo Struewing.
No test case since the crash only occurs under a stress
load.

locks for DML statements and changes the way MDL locks
are acquired/granted in contended case.

Instead of backing-off when a lock conflict is encountered
and waiting for it to go away before restarting open_tables()
process we now wait for lock to be released without releasing
any previously acquired locks. If conflicting lock goes away
we resume opening tables. If waiting leads to a deadlock we
try to resolve it by backing-off and restarting open_tables()
immediately.

As result both waiting for possibility to acquire and
acquiring of a metadata lock now always happen within the
same MDL API call. This has allowed to make release of a lock
and granting it to the most appropriate pending request an
atomic operation.
Thanks to this it became possible to wake up during release
of lock only those waiters which requests can be satisfied
at the moment as well as wake up only one waiter in case
when granting its request would prevent all other requests
from being satisfied. This solves thundering herd problem
which occured in cases when we were releasing some lock and
woke up many waiters for SNRW or X locks (this was the issue
in bug#52289 "performance regression for MyISAM in sysbench
OLTP_RW test".
This also allowed to implement more fair (FIFO) scheduling
among waiters with the same priority.
It also opens the door for introducing new types of requests
for metadata locks such as low-prio SNRW lock which is
necessary in order to support LOCK TABLES LOW_PRIORITY WRITE.

Notice that after this sometimes can report ER_LOCK_DEADLOCK
error in cases in which it has not happened before.
Particularly we will always report this error if waiting for
conflicting lock has happened in the middle of transaction
and resulted in a deadlock. Before this patch the error was
not reported if deadlock could have been resolved by backing
off all metadata locks acquired by the current statement.

Conflicts:
Text conflict in mysql-test/r/archive.result
Contents conflict in mysql-test/r/innodb_bug38231.result
Text conflict in mysql-test/r/mdl_sync.result
Text conflict in mysql-test/suite/binlog/t/disabled.def
Text conflict in mysql-test/suite/rpl_ndb/r/rpl_ndb_binlog_format_errors.result
Text conflict in mysql-test/t/archive.test
Contents conflict in mysql-test/t/innodb_bug38231.test
Text conflict in mysql-test/t/mdl_sync.test
Text conflict in sql/sp_head.cc
Text conflict in sql/sql_show.cc
Text conflict in sql/table.cc
Text conflict in sql/table.h

Also, update a few cases missed by the initial patch.

A code review comment for Bug#52289.
Encapsulate the deadlock detection functionality into
a visitor class...

Remove a race introduced by omission: 
initialize iterators under a read lock on the object.

Encapsulate the deadlock detection functionality into 
a visitor class, and separate it from the wait-for graph
traversal code.

Use "Internal iterator" and "Visitor" patterns to 
achieve the desired separation of responsibilities.

Add comments.

and methods.

transactional SELECT and ALTER TABLE ...  REBUILD PARTITION".

Make open flags part of Open_table_context.
This allows to simplify some code and (in future)
enforce the invariant that we don't, say, request a back 
off on the table when there is MYSQL_OPEN_IGNORE_FLUSH 
flag.

 transactional SELECT and ALTER TABLE ...  REBUILD PARTITION".

Move declarations of sql_base.cc classes to sql_base.h
(previously declared in sql_class.h).
Became possible after a header file split.

transactional SELECT and ALTER TABLE ...  REBUILD PARTITION".

Remove unused code - TL_WRITE_ALLOW_READ thr_lock.c lock.

Fix various mismatches between function's language linkage. Any
particular function that is declared in C++ but should be callable
from C must have C linkage. Note that function types with different
linkages are also distinct. Thus, if a function type is declared in
C code, it will have C linkage (same if declared in a extern "C"
block).

merge of patch for bug#52913 from 5.1-bugteam to trunk-bugfixing. Changed $basedir to $bindir in mysql-test-run.pl.

      Problems:
      - regression (compating to version 5.1) in metadata for BLOB types
      - inconsistency between length metadata in server and embedded for BLOB types
      - wrong max_length calculation in items derived from BLOB columns
     @ libmysqld/lib_sql.cc
        Calculating length metadata in embedded similary to server version,
        using new function char_to_byte_length_safe().
     @ mysql-test/r/ctype_utf16.result
        Adding tests
     @ mysql-test/r/ctype_utf32.result
        Adding tests
     @ mysql-test/r/ctype_utf8.result
        Adding tests
     @ mysql-test/r/ctype_utf8mb4.result
        Adding tests
     @ mysql-test/t/ctype_utf16.test
        Adding tests
     @ mysql-test/t/ctype_utf32.test
        Adding tests
     @ mysql-test/t/ctype_utf8.test
        Adding tests
     @ mysql-test/t/ctype_utf8mb4.test
        Adding tests
     @ sql/field.cc
        Overriding char_length() for Field_blob:
        unlike in generic Item::char_length() we don't
        divide to mbmaxlen for BLOBs.
     @ sql/field.h
        - Making Field::char_length() virtual
        - Adding prototype for Field_blob::char_length()
     @ sql/item.h
        - Adding new helper function char_to_byte_length_safe()
        - Using new function
     @ sql/protocol.cc
        Using new function char_to_byte_length_safe().

    modified:
      libmysqld/lib_sql.cc
      mysql-test/r/ctype_utf16.result
      mysql-test/r/ctype_utf32.result
      mysql-test/r/ctype_utf8.result
      mysql-test/r/ctype_utf8mb4.result
      mysql-test/t/ctype_utf16.test
      mysql-test/t/ctype_utf32.test
      mysql-test/t/ctype_utf8.test
      mysql-test/t/ctype_utf8mb4.test
      sql/field.cc
      sql/field.h
      sql/item.h
      sql/protocol.cc

  innodb.innodb                            [ fail ]
          Test ended at 2010-06-02 15:04:06
  
  CURRENT_TEST: innodb.innodb
  --- /usr/w/mysql-trunk-innodb/mysql-test/suite/innodb/r/innodb.result	2010-05-23 23:10:26.576407000 +0300
  +++ /usr/w/mysql-trunk-innodb/mysql-test/suite/innodb/r/innodb.reject	2010-06-02 15:04:05.000000000 +0300
  @@ -2648,7 +2648,7 @@
   create table t4 (s1 char(2) binary,primary key (s1)) engine=innodb;
   insert into t1 values (0x41),(0x4120),(0x4100);
   insert into t2 values (0x41),(0x4120),(0x4100);
  -ERROR 23000: Duplicate entry 'A\x00' for key 'PRIMARY'
  +ERROR 23000: Duplicate entry 'A' for key 'PRIMARY'
   insert into t2 values (0x41),(0x4120);

The change in the printout was introduced in:

  ------------------------------------------------------------
  revno: 3008.6.2
  revision-id: sergey.glukhov@sun.com-20100527160143-57nas8nplzpj26dz
  parent: sergey.glukhov@sun.com-20100527155443-24vqi9o8rpnkyci7
  committer: Sergey Glukhov <Sergey.Glukhov@sun.com>
  branch nick: mysql-trunk-bugfixing
  timestamp: Thu 2010-05-27 20:01:43 +0400
  message:
    Bug#52430 Incorrect key in the error message for duplicate key error involving BINARY type
    For BINARY(N) strip trailing zeroes to make the error message nice-looking
  
   @ mysql-test/r/errors.result
      test case
  
   @ mysql-test/r/type_binary.result
      result fix
  
   @ mysql-test/t/errors.test
      test case
  
   @ sql/key.cc
      For BINARY(N) strip trailing zeroes to make the error message nice-looking

and its author (Sergey) did not notice the test failure because that test
has been disabled in his tree.

------------------------------------------------------------
revno: 3495
committer: Marko Mäkelä <marko.makela@oracle.com>
branch nick: 5.1-innodb
timestamp: Wed 2010-06-02 13:37:14 +0300
message:
  Bug#53674: InnoDB: Error: unlock row could not find a 4 mode lock on the record

  In semi-consistent read, only unlock freshly locked non-matching records.

  lock_rec_lock_fast(): Return LOCK_REC_SUCCESS,
  LOCK_REC_SUCCESS_CREATED, or LOCK_REC_FAIL instead of TRUE/FALSE.

  enum db_err: Add DB_SUCCESS_LOCKED_REC for indicating a successful
  operation where a record lock was created.

  lock_sec_rec_read_check_and_lock(),
  lock_clust_rec_read_check_and_lock(), lock_rec_enqueue_waiting(),
  lock_rec_lock_slow(), lock_rec_lock(), row_ins_set_shared_rec_lock(),
  row_ins_set_exclusive_rec_lock(), sel_set_rec_lock(),
  row_sel_get_clust_rec_for_mysql(): Return DB_SUCCESS_LOCKED_REC if a
  new record lock was created. Adjust callers.

  row_unlock_for_mysql(): Correct the function documentation.

  row_prebuilt_t::new_rec_locks: Correct the documentation.

------------------------------------------------------------
revno: 3493
revision-id: marko.makela@oracle.com-20100602101940-60x32xiivtqj9va1
parent: marko.makela@oracle.com-20100601135802-hgplcpr8089ura8g
committer: Marko Mäkelä <marko.makela@oracle.com>
branch nick: 5.1-innodb
timestamp: Wed 2010-06-02 13:19:40 +0300
message:
  fil_print_orphaned_tablespaces(): Unused function, remove.

 - Reserve line 17 in master.info for master_bind which has been
   added in MySQL Cluster 6.3
 - move the line for "list of server id for ignorable servers" to line 18

parameter to virtual function store() for longlong data type.

rb://371 approved by Sunny.

errors
            
In the fix of BUG#39934 in 5.1-rep+3, errors are generated when
binlog_format=row and a statement modifies a table restricted to
statement-logging (ER_BINLOG_ROW_MODE_AND_STMT_ENGINE); or if
binlog_format=statement and a statement modifies a table restricted to
row-logging (ER_BINLOG_STMT_MODE_AND_ROW_ENGINE).
            
However, some DDL statements that lock tables (e.g. ALTER TABLE,
CREATE INDEX and CREATE TRIGGER) were causing spurious errors,
although no row might be inserted into the binary log.
            
To fix the problem, we tagged statements that may generate
rows into the binary log and thence the warning messages are
only printed out when the appropriate conditions hold and rows
might be changed.

- Ported relevant changes from the upstream version to not
break strict-aliasing rules and to fix compiler warnings and
and infinite loops caused by that issue.

- Fixed compilation with Honor_FLT_ROUNDS defined.

- Fixed an unused variable warning.