mutex protecting thd->open_tables".

We should not manipulate with table->s->version outside the 
table definition cache code, but use the TDC API
to achieve the desired result.

Fix one violation: close_all_tables_for_name().

Bug#46527 COMMIT AND CHAIN RELEASE does not make sense
Bug#53343 completion_type=1, COMMIT/ROLLBACK AND CHAIN don't 
          preserve the isolation level
Bug#53346 completion_type has strange effect in a stored 
          procedure/prepared statement

Added test cases to verify the expected behaviour of :
 SET SESSION TRANSACTION ISOLATION LEVEL, 
 SET TRANSACTION ISOLATION LEVEL,
 @@completion_type,
 COMMIT AND CHAIN,
 ROLLBACK AND CHAIN
 ..and some combinations of the above

thd->open_tables"

thd->open_tables list is not normally accessed concurrently
except for one case: when the connection has open SQL
HANDLER tables, and we want to perform a DDL on the table,
we want to abort waits on MyISAM thr_lock of those connections
that prevent the DDL from proceeding, and iterate
over thd->open_tables list to find out the tables on which
the thread is waiting.

In 5.5 we mostly use deadlock detection and soft deadlock 
prevention, as opposed to "hard" deadlock prevention
of 5.1, which would abort any transaction that
may cause a deadlock. The only remaining case when
neither deadlock detection nor deadlock prevention
is implemented in 5.5 is HANDLER SQL, where we use
old good thr_lock_abort() technique form 5.1. 

Thus, replace use of LOCK_open to protect thd->open_tables
with thd->LOCK_ha_data (a lock protecting various session
private data).

This is a port of the work done for 5.5.4 for review
and inclusion into 5.5.5.

This crash happened if a table was listed twice in a DROP TABLE statement,
and the statement was executed while in LOCK TABLES mode. Since the two
elements of table list were identical, they were assigned the same TABLE object. 
During processing of the first table element, the TABLE instance was destroyed
and the second table list element was left with a dangling reference.
When this reference was later accessed, the server crashed.

Listing the same table twice in DROP TABLES should give an ER_NONUNIQ_TABLE
error. However, this did not happen as the check for unique table names was 
skipped due to the lock type for table list elements being set to TL_IGNORE.
Previously TL_UNLOCK was used and the unique check was performed.

This bug was a regression introduced by a pre-requisite patch for
Bug#51263 "Deadlock between transactional SELECT and ALTER TABLE ...
REBUILD PARTITION". The regression only existed in an internal team
tree and never in any released code.

This patch reverts DROP TABLE (and DROP VIEW) to the old behavior of
using TL_UNLOCK locks. Test case added to drop.test.

    
1) No mutex and no function call if we're not using
plugins.
2) If we're above the table definition cache limit,
delete the oldest unused share, not the share on our hands.

3413 Davi Arnaut	2010-06-03
     Don't enable safemalloc for valgrind builds, it's too slow.
to trunk-runtime.

for MyISAM in sysbench OLTP_RW test".

Fixes compilation warnings about local variable
shadowing I_P_List_fast_push_back::last member.

Renamed I_P_List_fast_push_back::last member
to m_last. Also to keep member naming consistent
renamed I_P_List::first member to m_first.

information_schema query.
Porting a fix done for a feature tree by Ingo Struewing.
No test case since the crash only occurs under a stress
load.

locks for DML statements and changes the way MDL locks
are acquired/granted in contended case.

Instead of backing-off when a lock conflict is encountered
and waiting for it to go away before restarting open_tables()
process we now wait for lock to be released without releasing
any previously acquired locks. If conflicting lock goes away
we resume opening tables. If waiting leads to a deadlock we
try to resolve it by backing-off and restarting open_tables()
immediately.

As result both waiting for possibility to acquire and
acquiring of a metadata lock now always happen within the
same MDL API call. This has allowed to make release of a lock
and granting it to the most appropriate pending request an
atomic operation.
Thanks to this it became possible to wake up during release
of lock only those waiters which requests can be satisfied
at the moment as well as wake up only one waiter in case
when granting its request would prevent all other requests
from being satisfied. This solves thundering herd problem
which occured in cases when we were releasing some lock and
woke up many waiters for SNRW or X locks (this was the issue
in bug#52289 "performance regression for MyISAM in sysbench
OLTP_RW test".
This also allowed to implement more fair (FIFO) scheduling
among waiters with the same priority.
It also opens the door for introducing new types of requests
for metadata locks such as low-prio SNRW lock which is
necessary in order to support LOCK TABLES LOW_PRIORITY WRITE.

Notice that after this sometimes can report ER_LOCK_DEADLOCK
error in cases in which it has not happened before.
Particularly we will always report this error if waiting for
conflicting lock has happened in the middle of transaction
and resulted in a deadlock. Before this patch the error was
not reported if deadlock could have been resolved by backing
off all metadata locks acquired by the current statement.

Conflicts:
Text conflict in mysql-test/r/archive.result
Contents conflict in mysql-test/r/innodb_bug38231.result
Text conflict in mysql-test/r/mdl_sync.result
Text conflict in mysql-test/suite/binlog/t/disabled.def
Text conflict in mysql-test/suite/rpl_ndb/r/rpl_ndb_binlog_format_errors.result
Text conflict in mysql-test/t/archive.test
Contents conflict in mysql-test/t/innodb_bug38231.test
Text conflict in mysql-test/t/mdl_sync.test
Text conflict in sql/sp_head.cc
Text conflict in sql/sql_show.cc
Text conflict in sql/table.cc
Text conflict in sql/table.h

Also, update a few cases missed by the initial patch.

A code review comment for Bug#52289.
Encapsulate the deadlock detection functionality into
a visitor class...

Remove a race introduced by omission: 
initialize iterators under a read lock on the object.

Encapsulate the deadlock detection functionality into 
a visitor class, and separate it from the wait-for graph
traversal code.

Use "Internal iterator" and "Visitor" patterns to 
achieve the desired separation of responsibilities.

Add comments.

and methods.

transactional SELECT and ALTER TABLE ...  REBUILD PARTITION".

Make open flags part of Open_table_context.
This allows to simplify some code and (in future)
enforce the invariant that we don't, say, request a back 
off on the table when there is MYSQL_OPEN_IGNORE_FLUSH 
flag.

 transactional SELECT and ALTER TABLE ...  REBUILD PARTITION".

Move declarations of sql_base.cc classes to sql_base.h
(previously declared in sql_class.h).
Became possible after a header file split.

transactional SELECT and ALTER TABLE ...  REBUILD PARTITION".

Remove unused code - TL_WRITE_ALLOW_READ thr_lock.c lock.

Fix various mismatches between function's language linkage. Any
particular function that is declared in C++ but should be callable
from C must have C linkage. Note that function types with different
linkages are also distinct. Thus, if a function type is declared in
C code, it will have C linkage (same if declared in a extern "C"
block).

merge of patch for bug#52913 from 5.1-bugteam to trunk-bugfixing. Changed $basedir to $bindir in mysql-test-run.pl.

      Problems:
      - regression (compating to version 5.1) in metadata for BLOB types
      - inconsistency between length metadata in server and embedded for BLOB types
      - wrong max_length calculation in items derived from BLOB columns
     @ libmysqld/lib_sql.cc
        Calculating length metadata in embedded similary to server version,
        using new function char_to_byte_length_safe().
     @ mysql-test/r/ctype_utf16.result
        Adding tests
     @ mysql-test/r/ctype_utf32.result
        Adding tests
     @ mysql-test/r/ctype_utf8.result
        Adding tests
     @ mysql-test/r/ctype_utf8mb4.result
        Adding tests
     @ mysql-test/t/ctype_utf16.test
        Adding tests
     @ mysql-test/t/ctype_utf32.test
        Adding tests
     @ mysql-test/t/ctype_utf8.test
        Adding tests
     @ mysql-test/t/ctype_utf8mb4.test
        Adding tests
     @ sql/field.cc
        Overriding char_length() for Field_blob:
        unlike in generic Item::char_length() we don't
        divide to mbmaxlen for BLOBs.
     @ sql/field.h
        - Making Field::char_length() virtual
        - Adding prototype for Field_blob::char_length()
     @ sql/item.h
        - Adding new helper function char_to_byte_length_safe()
        - Using new function
     @ sql/protocol.cc
        Using new function char_to_byte_length_safe().

    modified:
      libmysqld/lib_sql.cc
      mysql-test/r/ctype_utf16.result
      mysql-test/r/ctype_utf32.result
      mysql-test/r/ctype_utf8.result
      mysql-test/r/ctype_utf8mb4.result
      mysql-test/t/ctype_utf16.test
      mysql-test/t/ctype_utf32.test
      mysql-test/t/ctype_utf8.test
      mysql-test/t/ctype_utf8mb4.test
      sql/field.cc
      sql/field.h
      sql/item.h
      sql/protocol.cc

  innodb.innodb                            [ fail ]
          Test ended at 2010-06-02 15:04:06
  
  CURRENT_TEST: innodb.innodb
  --- /usr/w/mysql-trunk-innodb/mysql-test/suite/innodb/r/innodb.result	2010-05-23 23:10:26.576407000 +0300
  +++ /usr/w/mysql-trunk-innodb/mysql-test/suite/innodb/r/innodb.reject	2010-06-02 15:04:05.000000000 +0300
  @@ -2648,7 +2648,7 @@
   create table t4 (s1 char(2) binary,primary key (s1)) engine=innodb;
   insert into t1 values (0x41),(0x4120),(0x4100);
   insert into t2 values (0x41),(0x4120),(0x4100);
  -ERROR 23000: Duplicate entry 'A\x00' for key 'PRIMARY'
  +ERROR 23000: Duplicate entry 'A' for key 'PRIMARY'
   insert into t2 values (0x41),(0x4120);

The change in the printout was introduced in:

  ------------------------------------------------------------
  revno: 3008.6.2
  revision-id: sergey.glukhov@sun.com-20100527160143-57nas8nplzpj26dz
  parent: sergey.glukhov@sun.com-20100527155443-24vqi9o8rpnkyci7
  committer: Sergey Glukhov <Sergey.Glukhov@sun.com>
  branch nick: mysql-trunk-bugfixing
  timestamp: Thu 2010-05-27 20:01:43 +0400
  message:
    Bug#52430 Incorrect key in the error message for duplicate key error involving BINARY type
    For BINARY(N) strip trailing zeroes to make the error message nice-looking
  
   @ mysql-test/r/errors.result
      test case
  
   @ mysql-test/r/type_binary.result
      result fix
  
   @ mysql-test/t/errors.test
      test case
  
   @ sql/key.cc
      For BINARY(N) strip trailing zeroes to make the error message nice-looking

and its author (Sergey) did not notice the test failure because that test
has been disabled in his tree.

------------------------------------------------------------
revno: 3495
committer: Marko Mäkelä <marko.makela@oracle.com>
branch nick: 5.1-innodb
timestamp: Wed 2010-06-02 13:37:14 +0300
message:
  Bug#53674: InnoDB: Error: unlock row could not find a 4 mode lock on the record

  In semi-consistent read, only unlock freshly locked non-matching records.

  lock_rec_lock_fast(): Return LOCK_REC_SUCCESS,
  LOCK_REC_SUCCESS_CREATED, or LOCK_REC_FAIL instead of TRUE/FALSE.

  enum db_err: Add DB_SUCCESS_LOCKED_REC for indicating a successful
  operation where a record lock was created.

  lock_sec_rec_read_check_and_lock(),
  lock_clust_rec_read_check_and_lock(), lock_rec_enqueue_waiting(),
  lock_rec_lock_slow(), lock_rec_lock(), row_ins_set_shared_rec_lock(),
  row_ins_set_exclusive_rec_lock(), sel_set_rec_lock(),
  row_sel_get_clust_rec_for_mysql(): Return DB_SUCCESS_LOCKED_REC if a
  new record lock was created. Adjust callers.

  row_unlock_for_mysql(): Correct the function documentation.

  row_prebuilt_t::new_rec_locks: Correct the documentation.

------------------------------------------------------------
revno: 3493
revision-id: marko.makela@oracle.com-20100602101940-60x32xiivtqj9va1
parent: marko.makela@oracle.com-20100601135802-hgplcpr8089ura8g
committer: Marko Mäkelä <marko.makela@oracle.com>
branch nick: 5.1-innodb
timestamp: Wed 2010-06-02 13:19:40 +0300
message:
  fil_print_orphaned_tablespaces(): Unused function, remove.

 - Reserve line 17 in master.info for master_bind which has been
   added in MySQL Cluster 6.3
 - move the line for "list of server id for ignorable servers" to line 18

parameter to virtual function store() for longlong data type.

rb://371 approved by Sunny.