An error occurred fetching the project authors.
  1. 11 Feb, 2010 1 commit
    • Jon Olav Hauglid's avatar
      Bug #45225 Locking: hang if drop table with no timeout · 3d6a89e7
      Jon Olav Hauglid authored
      This patch introduces timeouts for metadata locks. 
      
      The timeout is specified in seconds using the new dynamic system 
      variable  "lock_wait_timeout" which has both GLOBAL and SESSION
      scopes. Allowed values range from 1 to 31536000 seconds (= 1 year). 
      The default value is 1 year.
      
      The new server parameter "lock-wait-timeout" can be used to set
      the default value parameter upon server startup.
      
      "lock_wait_timeout" applies to all statements that use metadata locks.
      These include DML and DDL operations on tables, views, stored procedures
      and stored functions. They also include LOCK TABLES, FLUSH TABLES WITH
      READ LOCK and HANDLER statements.
      
      The patch also changes thr_lock.c code (table data locks used by MyISAM
      and other simplistic engines) to use the same system variable.
      InnoDB row locks are unaffected.
      
      One exception to the handling of the "lock_wait_timeout" variable
      is delayed inserts. All delayed inserts are executed with a timeout
      of 1 year regardless of the setting for the global variable. As the
      connection issuing the delayed insert gets no notification of 
      delayed insert timeouts, we want to avoid unnecessary timeouts.
      
      It's important to note that the timeout value is used for each lock
      acquired and that one statement can take more than one lock.
      A statement can therefore block for longer than the lock_wait_timeout 
      value before reporting a timeout error. When lock timeout occurs, 
      ER_LOCK_WAIT_TIMEOUT is reported.
      
      Test case added to lock_multi.test.
      
      
      include/my_pthread.h:
        Added macros for comparing two timespec structs.
      include/thr_lock.h:
        Introduced timeouts for thr_lock.c locks.
      mysql-test/r/mysqld--help-notwin.result:
        Updated result file with the new server variable.
      mysql-test/r/mysqld--help-win.result:
        Updated result file with the new server variable.
      mysql-test/suite/sys_vars/r/lock_wait_timeout_basic.result:
        Added basic test for the new server variable.
      mysql-test/suite/sys_vars/t/lock_wait_timeout_basic.test:
        Added basic test for the new server variable.
      mysys/thr_lock.c:
        Introduced timeouts for thr_lock.c locks.
      sql/mdl.cc:
        Introduced timeouts for metadata locks.
      sql/mdl.h:
        Introduced timeouts for metadata locks.
      sql/sql_base.cc:
        Introduced timeouts in tdc_wait_for_old_versions().
      sql/sql_class.h:
        Added new server variable lock_wait_timeout.
      sql/sys_vars.cc:
        Added new server variable lock_wait_timeout.
      3d6a89e7
  2. 08 Feb, 2010 1 commit
    • Dmitry Lenev's avatar
      Fix for bug #50913 "Deadlock between open_and_lock_tables_derived · 8018ec5a
      Dmitry Lenev authored
      and MDL".
      
      Concurrent execution of a multi-DELETE statement and ALTER
      TABLE statement which affected one of the tables used in
      the multi-DELETE sometimes led to deadlock.
      Similar deadlocks might have occured when one performed
      INSERT/UPDATE/DELETE on a view and concurrently executed
      ALTER TABLE for the view's underlying table, or when one
      concurrently executed TRUNCATE TABLE for InnoDB table and
      ALTER TABLE for the same table.
      
      These deadlocks were caused by a discrepancy between types of
      metadata and thr_lock.cc locks acquired by those statements.
      
      What happened was that multi-DELETE/TRUNCATE/DML-through-the-
      view statement in the first connection acquired SR lock on a
      table, then ALTER TABLE would come in in the second connection
      and acquire SNW metadata lock and TL_WRITE_ALLOW_READ
      thr_lock.c lock and then would start waiting for the first
      connection during lock upgrade. After that the statement in
      the first connection would try to acquire TL_WRITE lock on
      table and would start waiting for the second connection,
      creating a deadlock.
      
      This patch solves this problem by ensuring that we acquire
      SW metadata lock in all cases in which we acquiring write
      thr_lock.c lock. This guarantees that deadlocks like the
      one described above won't occur since all lock conflicts
      in such situation are resolved within MDL subsystem.
      
      This patch also adds assert which should guarantee that
      such situations won't arise in future.
      
      mysql-test/r/lock_multi.result:
        Added main test for bug #50913 "Deadlock between
        open_and_lock_tables_derived and MDL".
      mysql-test/r/mdl_sync.result:
        Added additional coverage for bug #50913 "Deadlock
        between open_and_lock_tables_derived and MDL".
      mysql-test/t/lock_multi.test:
        Added main test for bug #50913 "Deadlock between
        open_and_lock_tables_derived and MDL".
      mysql-test/t/mdl_sync.test:
        Added additional coverage for bug #50913 "Deadlock
        between open_and_lock_tables_derived and MDL".
      sql/lock.cc:
        Added assert that enforces that when we are locking
        a non-temporary table we have an appropriate type of
        metadata lock on this table.
      sql/mysql_priv.h:
        Added separate flag for open_tables() to be able specify that
        SH metadata locks on table to be open should be acquired.
        We can no longer use MYSQL_LOCK_IGNORE_FLUSH flag for this
        as in addition to use in I_S implementation it is also used
        for opening system tables. Since in the latter case we also
        acquire thr_lock.c locks using SH metadata lock in it instead
        of SR or SW locks may lead to deadlock.
      sql/sql_base.cc:
        When opening tables don't interpret MYSQL_LOCK_IGNORE_FLUSH
        flag as request to acquire SH metadata locks. This flag is
        also used for opening system tables for which we also take
        thr_lock.c locks and thus proper metadata lock to take in
        this case is SR or SW lock (otherwise deadlocks can occur).
        In cases when SH lock is really required (e.g. when tables
        are open by I_S implementation) we rely on that newly
        introduced MYSQL_OPEN_FORCE_SHARED_HIGH_PRIO_MDL flag is
        used.
      sql/sql_delete.cc:
        mysql_truncate_by_delete():
          Adjust type of metadata lock to be requested after changing
          type of thr_lock.c lock for table list element from one
          which was set in parser to TL_WRITE.
          This removes discrepancy between types of these locks which
          allowed deadlocks to creep in.
      sql/sql_handler.cc:
        When closing table which was open by HANDLER statement clear
        TABLE::open_by_handler flag. This allows to use this flag as
        a reliable indication that TABLE instance was open by HANDLER
        statement in assert which was added to mysql_lock_tables().
      sql/sql_parse.cc:
        multi_delete_set_locks_and_link_aux_tables():
          Adjust type of metadata lock to be requested after changing
          type of thr_lock.c lock for table list element from one
          which was set in parser to TL_WRITE.
          This removes discrepancy between types of these locks which
          allowed deadlocks to creep in.
      sql/sql_show.cc:
        Use newly introduced MYSQL_OPEN_FORCE_SHARED_HIGH_PRIO_MDL
        flag in order to acquire SH metadata locks when opening tables
        in I_S implementation.
      sql/sql_update.cc:
        Added comment explaining why in multi-update after deciding
        that we need weaker thr_lock.c lock on a table we don't
        downgrade metadata lock on it.
      sql/sql_view.cc:
        When merging view into main statement adjust type of metadata
        lock to be requested after changing type of thr_lock.c lock
        for table. This removes discrepancy between types of these
        locks which allowed deadlocks to creep in.
      8018ec5a
  3. 01 Feb, 2010 1 commit
    • Dmitry Lenev's avatar
      Implement new type-of-operation-aware metadata locks. · eba5d30e
      Dmitry Lenev authored
      Add a wait-for graph based deadlock detector to the
      MDL subsystem.
      
      Fixes bug #46272 "MySQL 5.4.4, new MDL: unnecessary deadlock" and
      bug #37346 "innodb does not detect deadlock between update and
      alter table".
      
      The first bug manifested itself as an unwarranted abort of a
      transaction with ER_LOCK_DEADLOCK error by a concurrent ALTER
      statement, when this transaction tried to repeat use of a
      table, which it has already used in a similar fashion before
      ALTER started.
      
      The second bug showed up as a deadlock between table-level
      locks and InnoDB row locks, which was "detected" only after
      innodb_lock_wait_timeout timeout.
      
      A transaction would start using the table and modify a few
      rows.
      Then ALTER TABLE would come in, and start copying rows
      into a temporary table. Eventually it would stumble on
      the modified records and get blocked on a row lock.
      The first transaction would try to do more updates, and get
      blocked on thr_lock.c lock.
      This situation of circular wait would only get resolved
      by a timeout.
      
      Both these bugs stemmed from inadequate solutions to the
      problem of deadlocks occurring between different
      locking subsystems.
      
      In the first case we tried to avoid deadlocks between metadata
      locking and table-level locking subsystems, when upgrading shared
      metadata lock to exclusive one.
      Transactions holding the shared lock on the table and waiting for
      some table-level lock used to be aborted too aggressively.
      
      We also allowed ALTER TABLE to start in presence of transactions
      that modify the subject table. ALTER TABLE acquires
      TL_WRITE_ALLOW_READ lock at start, and that block all writes
      against the table (naturally, we don't want any writes to be lost
      when switching the old and the new table). TL_WRITE_ALLOW_READ
      lock, in turn, would block the started transaction on thr_lock.c
      lock, should they do more updates. This, again, lead to the need
      to abort such transactions.
      
      The second bug occurred simply because we didn't have any
      mechanism to detect deadlocks between the table-level locks
      in thr_lock.c and row-level locks in InnoDB, other than
      innodb_lock_wait_timeout.
      
      This patch solves both these problems by moving lock conflicts
      which are causing these deadlocks into the metadata locking
      subsystem, thus making it possible to avoid or detect such
      deadlocks inside MDL.
      
      To do this we introduce new type-of-operation-aware metadata
      locks, which allow MDL subsystem to know not only the fact that
      transaction has used or is going to use some object but also what
      kind of operation it has carried out or going to carry out on the
      object.
      
      This, along with the addition of a special kind of upgradable
      metadata lock, allows ALTER TABLE to wait until all
      transactions which has updated the table to go away.
      This solves the second issue.
      Another special type of upgradable metadata lock is acquired
      by LOCK TABLE WRITE. This second lock type allows to solve the
      first issue, since abortion of table-level locks in event of
      DDL under LOCK TABLES becomes also unnecessary.
      
      Below follows the list of incompatible changes introduced by
      this patch:
      
      - From now on, ALTER TABLE and CREATE/DROP TRIGGER SQL (i.e. those
        statements that acquire TL_WRITE_ALLOW_READ lock)
        wait for all transactions which has *updated* the table to
        complete.
      
      - From now on, LOCK TABLES ... WRITE, REPAIR/OPTIMIZE TABLE
        (i.e. all statements which acquire TL_WRITE table-level lock) wait
        for all transaction which *updated or read* from the table
        to complete.
        As a consequence, innodb_table_locks=0 option no longer applies
        to LOCK TABLES ... WRITE.
      
      - DROP DATABASE, DROP TABLE, RENAME TABLE no longer abort
        statements or transactions which use tables being dropped or
        renamed, and instead wait for these transactions to complete.
      
      - Since LOCK TABLES WRITE now takes a special metadata lock,
        not compatible with with reads or writes against the subject table
        and transaction-wide, thr_lock.c deadlock avoidance algorithm
        that used to ensure absence of deadlocks between LOCK TABLES
        WRITE and other statements is no longer sufficient, even for
        MyISAM. The wait-for graph based deadlock detector of MDL
        subsystem may sometimes be necessary and is involved. This may
        lead to ER_LOCK_DEADLOCK error produced for multi-statement
        transactions even if these only use MyISAM:
      
        session 1:         session 2:
        begin;
      
        update t1 ...      lock table t2 write, t1 write;
                           -- gets a lock on t2, blocks on t1
      
        update t2 ...
        (ER_LOCK_DEADLOCK)
      
      - Finally,  support of LOW_PRIORITY option for LOCK TABLES ... WRITE
        was abandoned.
        LOCK TABLE ... LOW_PRIORITY WRITE from now on has the same
        priority as the usual LOCK TABLE ... WRITE.
        SELECT HIGH PRIORITY no longer trumps LOCK TABLE ... WRITE  in
        the wait queue.
      
      - We do not take upgradable metadata locks on implicitly
        locked tables. So if one has, say, a view v1 that uses
        table t1, and issues:
        LOCK TABLE v1 WRITE;
        FLUSH TABLE t1; -- (or just 'FLUSH TABLES'),
        an error is produced.
        In order to be able to perform DDL on a table under LOCK TABLES,
        the table must be locked explicitly in the LOCK TABLES list.
      
      mysql-test/include/handler.inc:
        Adjusted test case to trigger an execution path on which bug 41110
        "crash with handler command when used concurrently with alter
        table" and bug 41112 "crash in mysql_ha_close_table/get_lock_data
        with alter table" were originally discovered. Left old test case
        which no longer triggers this execution path for the sake of
        coverage.
        Added test coverage for HANDLER SQL statements and type-aware
        metadata locks.
        Added a test for the global shared lock and HANDLER SQL.
        Updated tests to take into account that the old simple deadlock
        detection heuristics was replaced with a graph-based deadlock
        detector.
      mysql-test/r/debug_sync.result:
        Updated results (see debug_sync.test).
      mysql-test/r/handler_innodb.result:
        Updated results (see handler.inc test).
      mysql-test/r/handler_myisam.result:
        Updated results (see handler.inc test).
      mysql-test/r/innodb-lock.result:
        Updated results (see innodb-lock.test).
      mysql-test/r/innodb_mysql_lock.result:
        Updated results (see innodb_mysql_lock.test).
      mysql-test/r/lock.result:
        Updated results (see lock.test).
      mysql-test/r/lock_multi.result:
        Updated results (see lock_multi.test).
      mysql-test/r/lock_sync.result:
        Updated results (see lock_sync.test).
      mysql-test/r/mdl_sync.result:
        Updated results (see mdl_sync.test).
      mysql-test/r/sp-threads.result:
        SHOW PROCESSLIST output has changed due to the fact that waiting
        for LOCK TABLES WRITE now happens within metadata locking
        subsystem.
      mysql-test/r/truncate_coverage.result:
        Updated results (see truncate_coverage.test).
      mysql-test/suite/funcs_1/datadict/processlist_val.inc:
        SELECT FROM I_S.PROCESSLIST output has changed due to fact that
        waiting for LOCK TABLES WRITE now happens within metadata locking
        subsystem.
      mysql-test/suite/funcs_1/r/processlist_val_no_prot.result:
        SELECT FROM I_S.PROCESSLIST output has changed due to fact that
        waiting for LOCK TABLES WRITE now happens within metadata locking
        subsystem.
      mysql-test/suite/rpl/t/rpl_sp.test:
        Updated to a new SHOW PROCESSLIST state name.
      mysql-test/t/debug_sync.test:
        Use LOCK TABLES READ instead of LOCK TABLES WRITE as the latter
        no longer allows to trigger execution path involving waiting on
        thr_lock.c lock and therefore reaching debug sync-point covered
        by this test.
      mysql-test/t/innodb-lock.test:
        Adjusted test case to the fact that innodb_table_locks=0 option is
        no longer supported, since LOCK TABLES WRITE handles all its
        conflicts within MDL subsystem.
      mysql-test/t/innodb_mysql_lock.test:
        Added test for bug #37346 "innodb does not detect deadlock between
        update and alter table".
      mysql-test/t/lock.test:
        Added test coverage which checks the fact that we no longer support
        DDL under LOCK TABLES on tables which were locked implicitly.
        Adjusted existing test cases accordingly.
      mysql-test/t/lock_multi.test:
        Added test for bug #46272 "MySQL 5.4.4, new MDL: unnecessary
        deadlock".  Adjusted other test cases to take into account the
        fact that waiting for LOCK TABLES ... WRITE now happens within MDL
        subsystem.
      mysql-test/t/lock_sync.test:
        Since LOCK TABLES ... WRITE now takes SNRW metadata lock for
        tables locked explicitly we have to implicitly lock InnoDB tables
        (through view) to trigger the table-level lock conflict between
        TL_WRITE and TL_WRITE_ALLOW_WRITE.
      mysql-test/t/mdl_sync.test:
        Added basic test coverage for type-of-operation-aware metadata
        locks. Also covered with tests some use cases involving HANDLER
        statements in which a deadlock could arise.
        Adjusted existing tests to take type-of-operation-aware MDL into
        account.
      mysql-test/t/multi_update.test:
        Update to a new SHOW PROCESSLIST state name.
      mysql-test/t/truncate_coverage.test:
        Adjusted test case after making LOCK TABLES WRITE to wait until
        transactions that use the table to be locked are completed.
        Updated to the changed name of DEBUG_SYNC point.
      sql/handler.cc:
        Global read lock functionality has been
        moved into a class.
      sql/lock.cc:
        Global read lock functionality has been
        moved into a class.
        Updated code to use the new MDL API.
      sql/mdl.cc:
        Introduced new type-of-operation aware metadata locks.
        To do this:
        - Changed MDL_lock to use one list for waiting requests and one
          list for granted requests. For each list, added a bitmap
          that holds information what lock types a list contains.
          Added a helper class MDL_lock::List to manipulate with granted
          and waited lists while keeping the bitmaps in sync
          with list contents.
        - Changed lock-compatibility functions to use bitmaps that
          define compatibility.
        - Introduced a graph based deadlock detector inspired by
          waiting_threads.c from Maria implementation.
        - Now that we have a deadlock detector, and no longer have
          a global lock to protect individual lock objects, but rather
          use an rw lock per object, removed redundant code for upgrade,
          and the global read lock. Changed the MDL API to
          no longer require the caller to acquire the global
          intention exclusive lock by means of a separate method.
          Removed a few more methods that became redundant.
        - Removed deadlock detection heuristic, it has been made
          obsolete by the deadlock detector.
        - With operation-type-aware metadata locks, MDL subsystem has
          become aware of potential conflicts between DDL and open
          transactions. This made it possible to remove calls to
          mysql_abort_transactions_with_shared_lock() from acquisition
          paths for exclusive lock and lock upgrade. Now we can simply
          wait for these transactions to complete without fear of
          deadlock. Function mysql_lock_abort() has also become
          unnecessary for all conflicting cases except when a DDL
          conflicts with a connection that has an open HANDLER.
      sql/mdl.h:
        Introduced new type-of-operation aware metadata locks.
        Introduced a graph based deadlock detector and supporting
        methods.
        Added comments.
        God rid of redundant API calls.
        Renamed m_lt_or_ha_sentinel to m_trans_sentinel,
        since now it guards the global read lock as well as
        LOCK TABLES and HANDLER locks.
      sql/mysql_priv.h:
        Moved the global read lock functionality into a
        class.
        Added MYSQL_OPEN_FORCE_SHARED_MDL flag which forces
        open_tables() to take MDL_SHARED on tables instead of
        metadata locks specified in the parser. We use this to
        allow PREPARE run concurrently in presence of
        LOCK TABLES ... WRITE.
        Added signature for find_table_for_mdl_ugprade().
      sql/set_var.cc:
        Global read lock functionality has been
        moved into a class.
      sql/sp_head.cc:
        When creating TABLE_LIST elements for prelocking or
        system tables set the type of request for metadata
        lock according to the operation that will be performed
        on the table.
      sql/sql_base.cc:
        - Updated code to use the new MDL API.
        - In order to avoid locks starvation we take upgradable
          locks all at once. As result implicitly locked tables no
          longer get an upgradable lock. Consequently DDL and FLUSH
          TABLES for such tables is prohibited.
          find_write_locked_table() was replaced by
          find_table_for_mdl_upgrade() function.
          open_table() was adjusted to return TABLE instance with
          upgradable ticket when necessary.
        - We no longer wait for all locks on OT_WAIT back off
          action -- only on the lock that caused the wait
          conflict. Moreover, now we distinguish cases when we
          have to wait due to conflict in MDL and old version
          of table in TDC.
        - Upate mysql_notify_threads_having_share_locks()
          to only abort thr_lock.c waits of threads that
          have open HANDLERs, since lock conflicts with only
          these threads now can lead to deadlocks not detectable
          by the MDL deadlock detector.
        - Remove mysql_abort_transactions_with_shared_locks()
          which is no longer needed.
      sql/sql_class.cc:
        Global read lock functionality has been moved into a class.
        Re-arranged code in THD::cleanup() to simplify assert.
      sql/sql_class.h:
        Introduced class to incapsulate global read lock
        functionality.
        Now sentinel in MDL subsystem guards the global read lock
        as well as LOCK TABLES and HANDLER locks. Adjusted code
        accordingly.
      sql/sql_db.cc:
        Global read lock functionality has been moved into a class.
      sql/sql_delete.cc:
        We no longer acquire upgradable metadata locks on tables
        which are locked by LOCK TABLES implicitly. As result
        TRUNCATE TABLE is no longer allowed for such tables.
        Updated code to use the new MDL API.
      sql/sql_handler.cc:
        Inform MDL_context about presence of open HANDLERs.
        Since HANLDERs break MDL protocol by acquiring table-level
        lock while holding only S metadata lock on a table MDL
        subsystem should take special care about such contexts (Now
        this is the only case when mysql_lock_abort() is used).
      sql/sql_parse.cc:
        Global read lock functionality has been moved into a class.
        Do not take upgradable metadata locks when opening tables
        for CREATE TABLE SELECT as it is not necessary and limits
        concurrency.
        When initializing TABLE_LIST objects before adding them
        to the table list set the type of request for metadata lock
        according to the operation that will be performed on the
        table.
        We no longer acquire upgradable metadata locks on tables
        which are locked by LOCK TABLES implicitly. As result FLUSH
        TABLES is no longer allowed for such tables.
      sql/sql_prepare.cc:
        Use MYSQL_OPEN_FORCE_SHARED_MDL flag when opening
        tables during PREPARE. This allows PREPARE to run
        concurrently in presence of LOCK TABLES ... WRITE.
      sql/sql_rename.cc:
        Global read lock functionality has been moved into a class.
      sql/sql_show.cc:
        Updated code to use the new MDL API.
      sql/sql_table.cc:
        Global read lock functionality has been moved into a class.
        We no longer acquire upgradable metadata locks on tables
        which are locked by LOCK TABLES implicitly. As result DROP
        TABLE is no longer allowed for such tables.
        Updated code to use the new MDL API.
      sql/sql_trigger.cc:
        Global read lock functionality has been moved into a class.
        We no longer acquire upgradable metadata locks on tables
        which are locked by LOCK TABLES implicitly. As result
        CREATE/DROP TRIGGER is no longer allowed for such tables.
        Updated code to use the new MDL API.
      sql/sql_view.cc:
        Global read lock functionality has been moved into a class.
        Fixed results of wrong merge that led to misuse of GLR API.
        CREATE VIEW statement is not a commit statement.
      sql/table.cc:
        When resetting TABLE_LIST objects for PS or SP re-execution
        set the type of request for metadata lock according to the
        operation that will be performed on the table. Do the same
        in auxiliary function initializing metadata lock requests
        in a table list.
      sql/table.h:
        When initializing TABLE_LIST objects set the type of request
        for metadata lock according to the operation that will be
        performed on the table.
      sql/transaction.cc:
        Global read lock functionality has been moved into a class.
      eba5d30e
  4. 21 Jan, 2010 1 commit
    • Dmitry Lenev's avatar
      Patch that changes metadata locking subsystem to use mutex per lock and · 6ddd01c2
      Dmitry Lenev authored
      condition variable per context instead of one mutex and one conditional
      variable for the whole subsystem.
      
      This should increase concurrency in this subsystem.
      
      It also opens the way for further changes which are necessary to solve
      such bugs as bug #46272 "MySQL 5.4.4, new MDL: unnecessary deadlock"
      and bug #37346 "innodb does not detect deadlock between update and alter
      table".
      
      Two other notable changes done by this patch:
      
      - MDL subsystem no longer implicitly acquires global intention exclusive
        metadata lock when per-object metadata lock is acquired. Now this has
        to be done by explicit calls outside of MDL subsystem.
      - Instead of using separate MDL_context for opening system tables/tables
        for purposes of I_S we now create MDL savepoint in the main context
        before opening tables and rollback to this savepoint after closing
        them. This means that it is now possible to get ER_LOCK_DEADLOCK error
        even not inside a transaction. This might happen in unlikely case when
        one runs DDL on one of system tables while also running DDL on some
        other tables. Cases when this ER_LOCK_DEADLOCK error is not justified
        will be addressed by advanced deadlock detector for MDL subsystem which
        we plan to implement.
      
      mysql-test/include/handler.inc:
        Adjusted handler_myisam.test and handler_innodb.test to the fact that
        exclusive metadata locks on tables are now acquired according to
        alphabetical order of fully qualified table names instead of order
        in which tables are mentioned in statement.
      mysql-test/r/handler_innodb.result:
        Adjusted handler_myisam.test and handler_innodb.test to the fact that
        exclusive metadata locks on tables are now acquired according to
        alphabetical order of fully qualified table names instead of order
        in which tables are mentioned in statement.
      mysql-test/r/handler_myisam.result:
        Adjusted handler_myisam.test and handler_innodb.test to the fact that
        exclusive metadata locks on tables are now acquired according to
        alphabetical order of fully qualified table names instead of order
        in which tables are mentioned in statement.
      mysql-test/r/mdl_sync.result:
        Adjusted mdl_sync.test to the fact that exclusive metadata locks on
        tables are now acquired according to alphabetical order of fully
        qualified table names instead of order in which tables are mentioned
        in statement.
      mysql-test/t/mdl_sync.test:
        Adjusted mdl_sync.test to the fact that exclusive metadata locks on
        tables are now acquired according to alphabetical order of fully
        qualified table names instead of order in which tables are mentioned
        in statement.
      sql/events.cc:
        Instead of using separate MDL_context for opening system tables we now
        create MDL savepoint in the main context before opening such tables
        and rollback to this savepoint after closing them. To support this
        change methods of THD responsible for saving/restoring open table
        state were changed to use Open_tables_backup class which in addition
        to Open_table_state has a member for this savepoint. As result code
        opening/closing system tables was changed to use Open_tables_backup
        instead of Open_table_state class as well.
      sql/ha_ndbcluster.cc:
        Since manipulations with open table state no longer install proxy
        MDL_context it does not make sense to perform them in order to
        satisfy assert in mysql_rm_tables_part2(). Removed them per agreement
        with Cluster team. This has not broken test suite since scenario in
        which deadlock can occur and assertion fails is not covered by tests.
      sql/lock.cc:
        MDL subsystem no longer implicitly acquires global intention exclusive
        metadata lock when per-object exclusive metadata lock is acquired.
        Now this has to be done by explicit calls outside of MDL subsystem.
      sql/log.cc:
        Instead of using separate MDL_context for opening system tables we now
        create MDL savepoint in the main context before opening such tables
        and rollback to this savepoint after closing them. To support this
        change methods of THD responsible for saving/restoring open table
        state were changed to use Open_tables_backup class which in addition
        to Open_table_state has a member for this savepoint. As result code
        opening/closing system tables was changed to use Open_tables_backup
        instead of Open_table_state class as well.
      sql/mdl.cc:
        Changed metadata locking subsystem to use mutex per lock and condition
        variable per context instead of one mutex and one conditional variable
        for the whole subsystem.
        Changed approach to handling of global metadata locks. Instead of
        implicitly acquiring intention exclusive locks when user requests
        per-object upgradeable or exclusive locks now we require them to be
        acquired explicitly in the same way as ordinary metadata locks.
        In fact global lock are now ordinary metadata locks in new GLOBAL
        namespace.
        
        To implement these changes:
        - Removed LOCK_mdl mutex and COND_mdl condition variable.
        - Introduced MDL_lock::m_mutex mutexes which protect individual lock
          objects.
        - Replaced mdl_locks hash with MDL_map class, which has hash for
          MDL_lock objects as a member and separate mutex which protects this
          hash. Methods of this class allow to find(), find_or_create() or
          remove() MDL_lock objects in concurrency-friendly fashion (i.e.
          for most common operation, find_or_create(), we don't acquire
          MDL_lock::m_mutex while holding MDL_map::m_mutex. Thanks to MikaelR
          for this idea and benchmarks!). Added three auxiliary members to
          MDL_lock class (m_is_destroyed, m_ref_usage, m_ref_release) to
          support this concurrency-friendly behavior.
        - Introduced MDL_context::m_ctx_wakeup_cond condition variable to be
          used for waiting until this context's pending request can be
          satisfied or its thread has to perform actions to resolve potential
          deadlock. Context which want to wait add ticket corresponding to the
          request to an appropriate queue of waiters in MDL_lock object so
          they can be noticed when other contexts change state of lock and be
          awaken by them by signalling on MDL_context::m_ctx_wakeup_cond.
          As consequence MDL_ticket objects has to be used for any waiting
          in metadata locking subsystem including one which happens in
          MDL_context::wait_for_locks() method.
          Another consequence is that MDL_context is no longer copyable and
          can't be saved/restored when working with system tables.
        - Made MDL_lock an abstract class, which delegates specifying exact
          compatibility matrix to its descendants. Added MDL_global_lock child
          class for global lock (The old is_lock_type_compatible() method
          became can_grant_lock() method of this class). Added MDL_object_lock
          class to represent per-object lock (The old MDL_lock::can_grant_lock()
          became its method). Choice between two classes happens based on MDL
          namespace in MDL_lock::create() method.
        - Got rid of MDL_lock::type member as its meaning became ambigous for
          global locks.
        - To simplify waking up of contexts waiting for lock split waiting queue
          in MDL_lock class in two queues. One for pending requests for exclusive
          (including intention exclusive) locks and another for requests for
          shared locks.
        - Added virtual wake_up_waiters() method to MDL_lock, MDL_global_lock and
          MDL_object_lock classes which allows to wake up waiting contexts after
          state of lock changes. Replaced old duplicated code with calls to this
          method.
        - Adjusted MDL_context::try_acquire_shared_lock()/exclusive_lock()/
          global_shared_lock(), MDL_ticket::upgrade_shared_lock_to_exclusive_lock()
          and MDL_context::release_ticket() methods to use MDL_map and
          MDL_lock::m_mutex instead of single LOCK_mdl mutex and wake up
          waiters according to the approach described above. The latter method
          also was renamed to MDL_context::release_lock().
        - Changed MDL_context::try_acquire_shared_lock()/exclusive_lock() and
          release_lock() not to handle global locks. They are now supposed to
          be taken explicitly like ordinary metadata locks.
        - Added helper MDL_context::try_acquire_global_intention_exclusive_lock()
          and acquire_global_intention_exclusive_lock() methods.
        - Moved common code from MDL_context::acquire_global_shared_lock() and
          acquire_global_intention_exclusive_lock() to new method -
          MDL_context::acquire_lock_impl().
        - Moved common code from MDL_context::try_acquire_shared_lock(),
          try_acquire_global_intention_exclusive_lock()/exclusive_lock()
          to MDL_context::try_acquire_lock_impl().
        - Since acquiring of several exclusive locks can no longer happen under
          single LOCK_mdl mutex the approach to it had to be changed. Now we do
          it in one by one fashion. This is done in alphabetical order to avoid
          deadlocks. Changed MDL_context::acquire_exclusive_locks() accordingly
          (as part of this change moved code responsible for acquiring single
          exclusive lock to new MDL_context::acquire_exclusive_lock_impl()
          method).
        - Since we no longer have single LOCK_mdl mutex which protects all
          MDL_context::m_is_waiting_in_mdl members using these members to
          determine if we have really awaken context holding conflicting
          shared lock became inconvinient. Got rid of this member and changed
          notify_shared_lock() helper function and process of acquiring
          of/upgrading to exclusive lock not to rely on such information.
          Now in MDL_context::acquire_exclusive_lock_impl() and
          MDL_ticket::upgrade_shared_lock_to_exclusive_lock() we simply
          re-try to wake up threads holding conflicting shared locks after
          small time out.
        - Adjusted MDL_context::can_wait_lead_to_deadlock() and
          MDL_ticket::has_pending_conflicting_lock() to use per-lock
          mutexes instead of LOCK_mdl. To do this introduced
          MDL_lock::has_pending_exclusive_lock() method.
      sql/mdl.h:
        Changed metadata locking subsystem to use mutex per lock and condition
        variable per context instead of one mutex and one conditional variable
        for the whole subsystem. In order to implement this change:
        
        - Added MDL_key::cmp() method to be able to sort MDL_key objects
          alphabetically. Changed length fields in MDL_key class to uint16
          as 16-bit is enough for length of any key.
        - Changed MDL_ticket::get_ctx() to return pointer to non-const
          object in order to be able to use MDL_context::awake() method
          for such contexts.
        - Got rid of unlocked versions of can_wait_lead_to_deadlock()/
          has_pending_conflicting_lock() methods in MDL_context and
          MDL_ticket. We no longer has single mutex which protects all
          locks. Thus one always has to use versions of these methods
          which acquire per-lock mutexes.
        - MDL_request_list type of list now counts its elements.
        - Added MDL_context::m_ctx_wakeup_cond condition variable to be used
          for waiting until this context's pending request can be satisfied
          or its thread has to perform actions to resolve potential deadlock.
          Added awake() method to wake up context from such wait.
          Addition of condition variable made MDL_context uncopyable.
          As result we no longer can save/restore MDL_context when working
          with system tables. Instead we create MDL savepoint before opening
          those tables and rollback to it once they are closed.
        - MDL_context::release_ticket() became release_lock() method.
        - Added auxiliary MDL_context::acquire_exclusive_lock_impl() method
          which does all necessary work to acquire exclusive lock on one object
          but should not be used directly as it does not enforce any asserts
          ensuring that no deadlocks are possible.
        - Since we no longer need to know if thread trying to acquire exclusive
          lock managed to wake up any threads having conflicting shared locks
          (as, anyway, we will try to wake up such threads again shortly)
        - MDL_context::m_is_waiting_in_mdl member became unnecessary and
          notify_shared_lock() no longer needs to be friend of MDL_context.
        
        Changed approach to handling of global metadata locks. Instead of
        implicitly acquiring intention exclusive locks when user requests
        per-object upgradeable or exclusive locks now we require them to be
        acquired explicitly in the same way as ordinary metadata locks.
        
        - Added new GLOBAL namespace for such locks.
        - Added new type of lock to be requested MDL_INTENTION_EXCLISIVE.
        - Added MDL_context::try_acquire_global_intention_exclusive_lock()
          and acquire_global_intention_exclusive_lock() methods.
        - Moved common code from MDL_context::acquire_global_shared_lock()
          and acquire_global_intention_exclusive_lock() to new method -
          MDL_context::acquire_lock_impl().
        - Moved common code from MDL_context::try_acquire_shared_lock(),
          try_acquire_global_intention_exclusive_lock()/exclusive_lock()
          to MDL_context::try_acquire_lock_impl().
        - Added helper MDL_context::is_global_lock_owner() method to be
          able easily to find what kind of global lock this context holds.
        - MDL_context::m_has_global_shared_lock became unnecessary as
          global read lock is now represented by ordinary ticket.
        - Removed assert in MDL_context::set_lt_or_ha_sentinel() which became
          false for cases when we execute LOCK TABLES under global read lock
          mode.
      sql/mysql_priv.h:
        Instead of using separate MDL_context for opening system tables we now
        create MDL savepoint in the main context before opening such tables
        and rollback to this savepoint after closing them. To support this
        change methods of THD responsible for saving/restoring open table
        state were changed to use Open_tables_backup class which in addition
        to Open_table_state has a member for this savepoint. As result calls
        opening/closing system tables were changed to use Open_tables_backup
        instead of Open_table_state class as well.
      sql/sp.cc:
        Instead of using separate MDL_context for opening system tables we now
        create MDL savepoint in the main context before opening such tables
        and rollback to this savepoint after closing them. To support this
        change methods of THD responsible for saving/restoring open table
        state were changed to use Open_tables_backup class which in addition
        to Open_table_state has a member for this savepoint. As result code
        opening/closing system tables was changed to use Open_tables_backup
        instead of Open_table_state class as well.
      sql/sp.h:
        Instead of using separate MDL_context for opening system tables we now
        create MDL savepoint in the main context before opening such tables
        and rollback to this savepoint after closing them. To support this
        change methods of THD responsible for saving/restoring open table
        state were changed to use Open_tables_backup class which in addition
        to Open_table_state has a member for this savepoint. As result code
        opening/closing system tables was changed to use Open_tables_backup
        instead of Open_table_state class as well.
      sql/sql_base.cc:
        close_thread_tables():
          Since we no longer use separate MDL_context for opening system
          tables we need to avoid releasing all transaction locks when
          closing system table. Releasing metadata lock on system table
          is now responsibility of THD::restore_backup_open_tables_state().
        open_table_get_mdl_lock(),
        Open_table_context::recover_from_failed_open():
          MDL subsystem no longer implicitly acquires global intention exclusive
          metadata lock when per-object upgradable or exclusive metadata lock is
          acquired. So this have to be done explicitly from these calls.
          Changed Open_table_context class to store MDL_request object for
          global intention exclusive lock acquired when opening tables.
        open_table():
          Do not release metadata lock if we have failed to open table as
          this lock might have been acquired by one of previous statements
          in transaction, and therefore should not be released.
        open_system_tables_for_read()/close_system_tables()/
        open_performance_schema_table():
          Instead of using separate MDL_context for opening system tables we now
          create MDL savepoint in the main context before opening such tables
          and rollback to this savepoint after closing them. To support this
          change methods of THD responsible for saving/restoring open table
          state were changed to use Open_tables_backup class which in addition
          to Open_table_state has a member for this savepoint. As result code
          opening/closing system tables was changed to use Open_tables_backup
          instead of Open_table_state class as well.
        close_performance_schema_table():
          Got rid of duplicated code.
      sql/sql_class.cc:
        Instead of using separate MDL_context for opening system tables we now
        create MDL savepoint in the main context before opening such tables
        and rollback to this savepoint after closing them. To support this
        change methods of THD responsible for saving/restoring open table
        state were changed to use Open_tables_backup class which in addition
        to Open_table_state has a member for this savepoint. Also releasing
        metadata lock on system table is now responsibility of
        THD::restore_backup_open_tables_state().
        Adjusted assert in THD::cleanup() to take into account fact that now
        we also use MDL sentinel for global read lock.
      sql/sql_class.h:
        Instead of using separate MDL_context for opening system tables we now
        create MDL savepoint in the main context before opening such tables
        and rollback to this savepoint after closing them. As result:
        - 'mdl_context' member was moved out of Open_tables_state to THD class.
          enter_locked_tables_mode()/leave_locked_tables_mode() had to follow.
        - Methods of THD responsible for saving/restoring open table state were
          changed to use Open_tables_backup class which in addition to
          Open_table_state has a member for this savepoint.
        Changed Open_table_context class to store MDL_request object for
        global intention exclusive lock acquired when opening tables.
      sql/sql_delete.cc:
        MDL subsystem no longer implicitly acquires global intention exclusive
        metadata lock when per-object exclusive metadata lock is acquired.
        Now this has to be done by explicit calls outside of MDL subsystem.
      sql/sql_help.cc:
        Instead of using separate MDL_context for opening system tables we now
        create MDL savepoint in the main context before opening such tables
        and rollback to this savepoint after closing them. To support this
        change methods of THD responsible for saving/restoring open table
        state were changed to use Open_tables_backup class which in addition
        to Open_table_state has a member for this savepoint. As result code
        opening/closing system tables was changed to use Open_tables_backup
        instead of Open_table_state class as well.
      sql/sql_parse.cc:
        Adjusted assert reload_acl_and_cache() to the fact that global read
        lock now takes full-blown metadata lock.
      sql/sql_plist.h:
        Added support for element counting to I_P_List list template.
        One can use policy classes to specify if such counting is needed
        or not needed for particular list.
      sql/sql_show.cc:
        Instead of using separate MDL_context for opening tables for I_S
        purposes we now create MDL savepoint in the main context before
        opening tables and rollback to this savepoint after closing them.
        To support this and similar change for system tables methods of
        THD responsible for saving/restoring open table state were changed
        to use Open_tables_backup class which in addition to Open_table_state
        has a member for this savepoint. As result code opening/closing tables
        for I_S purposes was changed to use Open_tables_backup instead of
        Open_table_state class as well.
      sql/sql_table.cc:
        mysql_rm_tables_part2():
          Since now global intention exclusive metadata lock is ordinary
          metadata lock we no longer can rely that by releasing MDL locks
          on all tables we will release all locks acquired by this routine.
          So in non-LOCK-TABLES mode we have to release all locks acquired
          explicitly.
        prepare_for_repair(), mysql_alter_table():
          MDL subsystem no longer implicitly acquires global intention
          exclusive metadata lock when per-object exclusive metadata lock
          is acquired. Now this has to be done by explicit calls outside of
          MDL subsystem.
      sql/tztime.cc:
        Instead of using separate MDL_context for opening system tables we now
        create MDL savepoint in the main context before opening such tables
        and rollback to this savepoint after closing them. To support this
        change methods of THD responsible for saving/restoring open table
        state were changed to use Open_tables_backup class which in addition
        to Open_table_state has a member for this savepoint. As result code
        opening/closing system tables was changed to use Open_tables_backup
        instead of Open_table_state class as well.
        Also changed code not to use special mechanism for open system tables
        when it is not really necessary.
      6ddd01c2
  5. 12 Jan, 2010 1 commit
  6. 07 Jan, 2010 1 commit
  7. 29 Dec, 2009 1 commit
    • Konstantin Osipov's avatar
      Apply and review: · bf9c1b73
      Konstantin Osipov authored
      3655 Jon Olav Hauglid   2009-10-19
      Bug #30977 Concurrent statement using stored function and DROP FUNCTION 
                 breaks SBR
      Bug #48246 assert in close_thread_table
      
      Implement a fix for:
      Bug #41804 purge stored procedure cache causes mysterious hang for many
                 minutes
      Bug #49972 Crash in prepared statements
      
      The problem was that concurrent execution of DML statements that
      use stored functions and DDL statements that drop/modify the same
      function might result in incorrect binary log in statement (and
      mixed) mode and therefore break replication.
      
      This patch fixes the problem by introducing metadata locking for
      stored procedures and functions. This is similar to what is done
      in Bug#25144 for views. Procedures and functions now are
      locked using metadata locks until the transaction is either
      committed or rolled back. This prevents other statements from
      modifying the procedure/function while it is being executed. This
      provides commit ordering - guaranteeing serializability across
      multiple transactions and thus fixes the reported binlog problem.
      
      Note that we do not take locks for top-level CALLs. This means
      that procedures called directly are not protected from changes by
      simultaneous DDL operations so they are executed at the state they
      had at the time of the CALL. By not taking locks for top-level
      CALLs, we still allow transactions to be started inside
      procedures.
      
      This patch also changes stored procedure cache invalidation.
      Upon a change of cache version, we no longer invalidate the entire
      cache, but only those routines which we use, only when a statement
      is executed that uses them.
      
      This patch also changes the logic of prepared statement validation.
      A stored procedure used by a prepared statement is now validated
      only once a metadata lock has been acquired. A version mismatch
      causes a flush of the obsolete routine from the cache and
      statement reprepare.
      Incompatible changes:
      1) ER_LOCK_DEADLOCK is reported for a transaction trying to access
         a procedure/function that is locked by a DDL operation in
         another connection.
      
      2) Procedure/function DDL operations are now prohibited in LOCK
         TABLES mode as exclusive locks must be taken all at once and
         LOCK TABLES provides no way to specifiy procedures/functions to
         be locked.
      
      Test cases have been added to sp-lock.test and rpl_sp.test.
      
      Work on this bug has very much been a team effort and this patch
      includes and is based on contributions from Davi Arnaut, Dmitry
      Lenev, Magne Mæhre and Konstantin Osipov.
      
      
      mysql-test/r/ps_ddl.result:
        Update results (Bug#30977).
      mysql-test/r/ps_ddl1.result:
        Update results (Bug#30977).
      mysql-test/r/sp-error.result:
        Update results (Bug#30977).
      mysql-test/r/sp-lock.result:
        Update results (Bug#30977).
      mysql-test/suite/rpl/r/rpl_sp.result:
        Update results (Bug#30977).
      mysql-test/suite/rpl/t/rpl_sp.test:
        Add a test case for Bug#30977.
      mysql-test/t/ps_ddl.test:
        Update comments. We no longer re-prepare a prepared statement
        when a stored procedure used in top-level CALL is changed.
      mysql-test/t/ps_ddl1.test:
        Modifying stored procedure p1 no longer invalidates prepared
        statement "call p1" -- we can re-use the prepared statement
        without invalidation.
      mysql-test/t/sp-error.test:
        Use a constant for an error value.
      mysql-test/t/sp-lock.test:
        Add test coverage for Bug#30977.
      sql/lock.cc:
        Implement lock_routine_name() - a way to acquire an 
        exclusive metadata lock (ex- name-lock) on 
        stored procedure/function.
      sql/sp.cc:
        Change semantics of sp_cache_routine() -- now it has an option
        to make sure that the routine that is cached is up to date (has
        the latest sp cache version).
        
        Add sp_cache_invalidate() to sp_drop_routine(), where it was
        missing (a bug!).
        
        Acquire metadata locks for SP DDL (ALTER/CREATE/DROP). This is
        the core of the fix for Bug#30977.
        
        Since caching and cache invalidation scheme was changed, make 
        sure we don't invalidate the SP cache in the middle of a stored
        routine execution. At the same time, make sure we don't access
        stale data due to lack of invalidation. 
        For that, change ALTER FUNCTION/PROCEDURE to not use the cache,
        and SHOW PROCEDURE CODE/SHOW CREATE PROCEDURE/FUNCTION to always
        read an up to date version of the routine from the cache.
      sql/sp.h:
        Add a helper wrapper around sp_cache_routine().
      sql/sp_cache.cc:
        Implement new sp_cache_version() and sp_cache_flush_obsolete().
        Now we flush stale routines individually, rather than all at once.
      sql/sp_cache.h:
        Update signatures of sp_cache_version() and sp_cache_flush_obsolete().
      sql/sp_head.cc:
        Add a default initialization of sp_head::m_sp_cache_version.
        Remove a redundant sp_head::create().
      sql/sp_head.h:
        Add m_sp_cache_version to sp_head class - we now 
        keep track of every routine in the stored procedure cache, rather than
        of the entire cache.
      sql/sql_base.cc:
        Implement prelocking for stored routines. Validate stored
        routines after they were locked.
        Flush obsolete routines upon next access, one by one, not all at once
        (Bug#41804).
        Style fixes.
      sql/sql_class.h:
        Rename a Open_table_context method.
      sql/sql_parse.cc:
        Make sure stored procedures DDL commits the active transaction 
        (issues an implicit commit before and after).
        Remove sp_head::create(), a pure redundancy.
        Move the semantical check during alter routine inside sp_update_routine() code in order to:
        - avoid using SP cache during update, it may be obsolete.
        - speed up and simplify the update procedure.
        
        Remove sp_cache_flush_obsolete() calls, we no longer flush the entire
        cache, ever, stale routines are flushed before next use, one at a time.
      sql/sql_prepare.cc:
        Move routine metadata validation to open_and_process_routine().
        Fix Bug#49972 (don't swap flags at reprepare).
        Reset Sroutine_hash_entries in reinit_stmt_before_use().
        Remove SP cache invalidation, it's now done by open_tables().
      sql/sql_show.cc:
        Fix a warning: remove an unused label.
      sql/sql_table.cc:
        Reset mdl_request.ticket for tickets acquired for routines inlined
        through a view, in CHECK TABLE statement, to satisfy an MDL assert.
      sql/sql_update.cc:
        Move the cleanup of "translation items" to close_tables_for_reopen(),
        since it's needed in all cases when we back off, not just
        the back-off in multi-update. This fixes a bug when the server
        would crash on attempt to back off when opening tables
        for a statement that uses information_schema tables.
      bf9c1b73
  8. 22 Dec, 2009 1 commit
    • Konstantin Osipov's avatar
      A prerequisite patch for the fix for Bug#46224 · dfdbc845
      Konstantin Osipov authored
      "HANDLER statements within a transaction might lead to deadlocks".
      Introduce a notion of a sentinel to MDL_context. A sentinel
      is a ticket that separates all tickets in the context into two
      groups: before and after it. Currently we can have (and need) only
      one designated sentinel -- it separates all locks taken by LOCK
      TABLE or HANDLER statement, which must survive COMMIT and ROLLBACK
      and all other locks, which must be released at COMMIT or ROLLBACK.
      The tricky part is maintaining the sentinel up to date when
      someone release its corresponding ticket. This can happen, e.g.
      if someone issues DROP TABLE under LOCK TABLES (generally,
      see all calls to release_all_locks_for_name()).
      MDL_context::release_ticket() is modified to take care of it.
      
      ******
      A fix and a test case for Bug#46224 "HANDLER statements within a
      transaction might lead to deadlocks".
      
      An attempt to mix HANDLER SQL statements, which are transaction-
      agnostic, an open multi-statement transaction,
      and DDL against the involved tables (in a concurrent connection) 
      could lead to a deadlock. The deadlock would occur when
      HANDLER OPEN or HANDLER READ would have to wait on a conflicting
      metadata lock. If the connection that issued HANDLER statement
      also had other metadata locks (say, acquired in scope of a 
      transaction), a classical deadlock situation of mutual wait
      could occur.
      
      Incompatible change: entering LOCK TABLES mode automatically
      closes all open HANDLERs in the current connection.
      
      Incompatible change: previously an attempt to wait on a lock
      in a connection that has an open HANDLER statement could wait
      indefinitely/deadlock. After this patch, an error ER_LOCK_DEADLOCK
      is produced.
      
      The idea of the fix is to merge thd->handler_mdl_context
      with the main mdl_context of the connection, used for transactional
      locks. This makes deadlock detection possible, since all waits
      with locks are "visible" and available to analysis in a single
      MDL context of the connection.
      
      Since HANDLER locks and transactional locks have a different life
      cycle -- HANDLERs are explicitly open and closed, and so
      are HANDLER locks, explicitly acquired and released, whereas
      transactional locks "accumulate" till the end of a transaction
      and are released only with COMMIT, ROLLBACK and ROLLBACK TO SAVEPOINT,
      a concept of "sentinel" was introduced to MDL_context.
      All locks, HANDLER and others, reside in the same linked list.
      However, a selected element of the list separates locks with
      different life cycle. HANDLER locks always reside at the
      end of the list, after the sentinel. Transactional locks are
      prepended to the beginning of the list, before the sentinel.
      Thus, ROLLBACK, COMMIT or ROLLBACK TO SAVEPOINT, only
      release those locks that reside before the sentinel. HANDLER locks
      must be released explicitly as part of HANDLER CLOSE statement,
      or an implicit close. 
      The same approach with sentinel
      is also employed for LOCK TABLES locks. Since HANDLER and LOCK TABLES
      statement has never worked together, the implementation is
      made simple and only maintains one sentinel, which is used either
      for HANDLER locks, or for LOCK TABLES locks.
      
      
      mysql-test/include/handler.inc:
        Add test coverage for Bug#46224 "HANDLER statements within a
        transaction might lead to deadlocks".
        Extended HANDLER coverage to cover a mix of HANDLER, transactions
        and DDL statements.
      mysql-test/r/handler_innodb.result:
        Update results (Bug#46224).
      mysql-test/r/handler_myisam.result:
        Update results (Bug#46224).
      sql/lock.cc:
        Remove thd->some_tables_deleted, it's never used.
      sql/log_event.cc:
        No need to check for thd->locked_tables_mode, 
        it's done inside release_transactional_locks().
      sql/mdl.cc:
        Implement the concept of HANDLER and LOCK TABLES "sentinel".
        Implement a method to clone an acquired ticket.
        Do not return tickets beyond the sentinel when acquiring
        locks, create a copy.
        Remove methods to merge and backup MDL_context, they are now
        not used (Hurra!). This opens a path to a proper constructor
        and destructor of class MDL_context (to be done in a separate
        patch).
        Modify find_ticket() to provide information about where
        the ticket position is with regard to the sentinel.
      sql/mdl.h:
        Add declarations necessary for the implementation of the concept
        of "sentinel", a dedicated ticket separating transactional and
        non-transactional locks.
      sql/mysql_priv.h:
        Add mark_tmp_table_for_reuse() declaration, 
        a function to "close" a single session (temporary) table.
      sql/sql_base.cc:
        Remove thd->some_tables_deleted.
        Modify deadlock-prevention asserts and deadlock detection
        heuristics to take into account that from now on HANDLER locks
        reside in the same locking context.
        Add broadcast_refresh() to mysql_notify_thread_having_shared_lock():
        this is necessary for the case when a thread having a shared lock
        is asleep in tdc_wait_for_old_versions(). This situation is only
        possible with HANDLER t1 OPEN; FLUSH TABLE (since all over code paths
        that lead to tdc_wait_for_old_versions() always have an
        empty MDL_context). Previously the server would simply deadlock
        in this situation.
      sql/sql_class.cc:
        Remove now unused member "THD::some_tables_deleted". 
        Move mysql_ha_cleanup() a few lines above in THD::cleanup() 
        to make sure that all handlers are closed when it's time to 
        destroy the MDL_context of this connection.
        Remove handler_mdl_context and handler_tables.
      sql/sql_class.h:
        Remove THD::handler_tables, THD::handler_mdl_context,
        THD::some_tables_deleted.
      sql/sql_handler.cc:
        Remove thd->handler_tables.
        Remove thd->handler_mdl_context.
        Rewrite mysql_ha_open() to have no special provision for MERGE
        tables, now that we don't have to manipulate with thd->handler_tables
        it's easy to do.
        Remove dead code.
        Fix a bug in mysql_ha_flush() when we would always flush
        a temporary HANDLER when mysql_ha_flush() is called (actually
        mysql_ha_flush() never needs to flush temporary tables).
      sql/sql_insert.cc:
        Update a comment, no more thd->some_tables_deleted.
      sql/sql_parse.cc:
        Implement an incompatible change: entering LOCK TABLES closes
        active HANDLERs, if any.
        Now that we have a sentinel, we don't need to check
        for thd->locked_tables_mode when releasing metadata locks in
        COMMIT/ROLLBACK.
      sql/sql_plist.h:
        Add new (now necessary) methods to the list class.
      sql/sql_prepare.cc:
        Make sure we don't release HANDLER locks when rollback to a
        savepoint, set to not keep locks taken at PREPARE.
      sql/sql_servers.cc:
        Update to a new signature of MDL_context::release_all_locks().
      sql/sql_table.cc:
        Remove thd->some_tables_deleted.
      sql/transaction.cc:
        Add comments. 
        Make sure rollback to (MDL) savepoint works under LOCK TABLES and
        with HANDLER tables.
      dfdbc845
  9. 11 Dec, 2009 1 commit
  10. 10 Dec, 2009 4 commits
    • Jon Olav Hauglid's avatar
      Backport of revno: 3685 · 3d062adf
      Jon Olav Hauglid authored
      Bug #48210 FLUSH TABLES WITH READ LOCK deadlocks
                 against concurrent CREATE PROCEDURE
      
      This deadlock occured between
      a) CREATE PROCEDURE (or other commands listed below)
      b) FLUSH TABLES WITH READ LOCK
      
      If the execution of them happened in the following order:
      - a) opens a table (e.g. mysql.proc)
      - b) locks the global read lock (or GRL)
      - a) sleeps inside wait_if_global_read_lock()
      - b) increases refresh_version and sleeps waiting 
           for old tables to go away
      
      Note that a) must start waiting on the GRL before FLUSH increases
      refresh_version. Otherwise a) won't wait on the GRL and instead
      close its tables for reopen, allowing FLUSH to complete and thus
      avoid the deadlock.
      
      With this patch the deadlock is avoided by making CREATE PROCEDURE
      acquire a protection against global read locks before it starts
      executing. This means that FLUSH TABLES WITH READ LOCK will have
      to wait until CREATE PROCEDURE completes before acquiring the global
      read lock, thereby avoiding the deadlock.
      
      This is implemented by introducing a new SQL command flag called
      CF_PROTECT_AGAINST_GRL. Commands marked with this flag will
      acquire a GRL protection in the beginning of mysql_execute_command().
      This patch adds the flag to CREATE, ALTER and DROP for PROCEDURE
      and FUNCTION, as well as CREATE USER, DROP USER, RENAME USER and 
      REVOKE ALL. All these commands either call open_grant_tables() or
      open_system_table_for_updated() which make them susceptible for
      this deadlock.
      
      The patch also adds the CF_PROTECT_AGAINST_GRL flag to a number
      of commands that previously acquired GRL protection in their
      respective SQLCOM case in mysql_execute_command().
      
      Test case that checks for GRL protection for CREATE PROCEDURE
      and CREATE USER added to mdl_sync.test.
      3d062adf
    • Konstantin Osipov's avatar
      Backport a part of Monty's fix for Bug#39396, rev. 2736.2.11 · f8055175
      Konstantin Osipov authored
      "ha_maria.cc:2415: assertion in ha_maria::store_lock()".
      
      sql/lock.cc:
        Fixed wrong cleanup of mysql_lock_tables()
        - We must call read_lock_data() BEFORE we set 
        lock_count to 0. Added DBUG statements.
      f8055175
    • Konstantin Osipov's avatar
      Backport of: · f26f632b
      Konstantin Osipov authored
      ------------------------------------------------------------
      revno: 2617.68.25
      committer: Dmitry Lenev <dlenev@mysql.com>
      branch nick: mysql-next-bg-pre2-2
      timestamp: Wed 2009-09-16 18:26:50 +0400
      message:
        Follow-up for one of pre-requisite patches for fixing bug #30977
        "Concurrent statement using stored function and DROP FUNCTION
        breaks SBR".
      
        Made enum_mdl_namespace enum part of MDL_key class and removed MDL_
        prefix from the names of enum members. In order to do the latter
        changed name of PROCEDURE symbol to PROCEDURE_SYM (otherwise macro
        which was automatically generated for this symbol conflicted with
        MDL_key::PROCEDURE enum member).
      f26f632b
    • Marc Alff's avatar
      WL#2360 Performance schema · c082955f
      Marc Alff authored
      Part III: mysys instrumentation
      c082955f
  11. 09 Dec, 2009 2 commits
    • Jon Olav Hauglid's avatar
      Backport of revno: 2617.68.13 · f3e9b392
      Jon Olav Hauglid authored
      Introduce a counter for protection against global read lock on thread level.
      
      The functions for protection against global read lock sometimes need a local
      variable to signal when the protection is set, and hence need to be released.
      It would be better to control this behaviour via a counter on the THD struct,
      telling how many times the protection has been claimed by the current thread.
      A side-effect of the fix is that if protection is claimed twice for a thread,
      only a simple increment is required for the second claim, instead of a
      mutex-protected increment of the global variable protect_against_global_read_lock.
      
      
      sql/lock.cc:
        Count how many times that we have claimed protection against global read lock.
        Assert that we really have the protection when releasing it.
        Added comments to all functions operating on global_read_lock.
      sql/sql_class.cc:
        Added the counter variable global_read_lock_protection.
      sql/sql_class.h:
        Added the counter variable global_read_lock_protection.
      sql/sql_parse.cc:
        Replaced test on local variable need_start_waiting with test on
        thd->global_read_lock_protection.
      sql/sql_table.cc:
        Replaced test on local variable need_start_waiting with test on
        thd->global_read_lock_protection.
      sql/sql_trigger.cc:
        Inserted test on thd->global_read_lock_protection.
      f3e9b392
    • Jon Olav Hauglid's avatar
      Backport of revno: 2617.69.40 · 546e16eb
      Jon Olav Hauglid authored
      A pre-requisite patch for Bug#30977 "Concurrent statement using 
      stored function and DROP FUNCTION breaks SBR".
      
      This patch changes the MDL API by introducing a namespace for
      lock keys: MDL_TABLE for tables and views and MDL_PROCEDURE
      for stored procedures and functions. The latter is needed for
      the fix for Bug#30977.
      546e16eb
  12. 08 Dec, 2009 1 commit
    • Konstantin Osipov's avatar
      Backport of: · ce5c87a3
      Konstantin Osipov authored
      ----------------------------------------------------------
      revno: 2617.69.20
      committer: Konstantin Osipov <kostja@sun.com>
      branch nick: 5.4-4284-1-assert
      timestamp: Thu 2009-08-13 18:29:55 +0400
      message:
        WL#4284 "Transactional DDL locking"
        A review fix.
        Since WL#4284 implementation separated MDL_request and MDL_ticket,
        MDL_request becamse a utility object necessary only to get a ticket.
        Store it by-value in TABLE_LIST with the intent to merge
        MDL_request::key with table_list->table_name and table_list->db
        in future.
        Change the MDL subsystem to not require MDL_requests to
        stay around till close_thread_tables().
        Remove the list of requests from the MDL context.
        Requests for shared metadata locks acquired in open_tables()
        are only used as a list in recover_from_failed_open_table_attempt(),
        which calls mdl_context.wait_for_locks() for this list.
        To keep such list for recover_from_failed_open_table_attempt(),
        introduce a context class (Open_table_context), that collects
        all requests.
        A lot of minor cleanups and simplications that became possible
        with this change.
      
      
      sql/event_db_repository.cc:
        Remove alloc_mdl_requests(). Now MDL_request instance is a member
        of TABLE_LIST, and init_one_table() initializes it.
      sql/ha_ndbcluster_binlog.cc:
        Remove now unnecessary declaration and initialization
        of binlog_mdl_request.
      sql/lock.cc:
        No need to allocate MDL requests in lock_table_names() now.
      sql/log.cc:
        Use init_one_table() method, remove alloc_mdl_requests(),
        which is now unnecessary.
      sql/log_event.cc:
        No need to allocate mdl_request separately now.
        Use init_one_table() method.
      sql/log_event_old.cc:
        Update to the new signature of close_tables_for_reopen().
      sql/mdl.cc:
        Update try_acquire_exclusive_lock() to be more easy to use.
        Function lock_table_name_if_not_cached() has been removed.
        Make acquire_shared_lock() signature consistent with
        try_acquire_exclusive_lock() signature.
        Remove methods that are no longer used.
        Update comments.
      sql/mdl.h:
        Implement an assignment operator that doesn't
        copy MDL_key (MDL_key::operator= is private and
        should remain private).
        This is a hack to work-around assignment of TABLE_LIST
        by value in several places. Such assignments violate
        encapsulation, since only perform a shallow copy.
        In most cases these assignments are a hack on their own.
      sql/mysql_priv.h:
        Update signatures of close_thread_tables() and close_tables_for_reopen().
      sql/sp.cc:
        Allocate TABLE_LIST in thd->mem_root.
        Use init_one_table().
      sql/sp_head.cc:
        Use init_one_table().
        Remove thd->locked_tables_root, it's no longer needed.
      sql/sql_acl.cc:
        Use init_mdl_requests() and init_one_table().
      sql/sql_base.cc:
        Update to new signatures of try_acquire_shared_lock() and
        try_acquire_exclusive_lock().
        Remove lock_table_name_if_not_cached().
        Fix a bug in open_ltable() that would not return ER_LOCK_DEADLOCK
        in case of a failed lock_tables() and a multi-statement
        transaction.
        Fix a bug in open_and_lock_tables_derived() that would
        not return ER_LOCK_DEADLOCK in case of a multi-statement
        transaction and a failure of lock_tables().
        Move assignment of enum_open_table_action to a method of Open_table_context, a new class that maintains information
        for backoff actions.
        Minor rearrangements of the code.
        Remove alloc_mdl_requests() in functions that work with system
        tables: instead the patch ensures that callers always initialize
        TABLE_LIST argument.
      sql/sql_class.cc:
        THD::locked_tables_root is no more.
      sql/sql_class.h:
        THD::locked_tables_root is no more.
        Add a declaration for Open_table_context class.
      sql/sql_delete.cc:
        Update to use the simplified MDL API.
      sql/sql_handler.cc:
        TABLE_LIST::mdl_request is stored by-value now.
        Ensure that mdl_request.ticket is NULL for every request
        that is passed into MDL, to satisfy MDL asserts.
        @ sql/sql_help.cc
        Function open_system_tables_for_read() no longer initializes
        mdl_requests.
        Move TABLE_LIST::mdl_request initialization closer to
        TABLE_LIST initialization.
      sql/sql_help.cc:
        Function open_system_tables_for_read() no longer initializes
        mdl_requests.
        Move TABLE_LIST::mdl_request initialization closer to
        TABLE_LIST initialization.
      sql/sql_insert.cc:
        Remove assignment by-value of TABLE_LIST in
        TABLEOP_HOOKS. We can't carry over a granted
        MDL ticket from one table list to another.
      sql/sql_parse.cc:
        Change alloc_mdl_requests() -> init_mdl_requests().
        @todo We can remove init_mdl_requests() altogether
        in some places: all places that call add_table_to_list()
        already have mdl requests initialized.
      sql/sql_plugin.cc:
        Use init_one_table().
        THD::locked_tables_root is no more.
      sql/sql_servers.cc:
        Use init_one_table().
      sql/sql_show.cc:
        Update acquire_high_priority_shared_lock() to use
        TABLE_LIST::mdl_request rather than allocate an own.
        Fix get_trigger_table_impl() to use init_one_table(),
        check for out of memory, follow the coding style.
      sql/sql_table.cc:
        Update to work with TABLE_LIST::mdl_request by-value.
        Remove lock_table_name_if_not_cached().
        The code that used to delegate to it is quite simple and
        concise without it now.
      sql/sql_udf.cc:
        Use init_one_table().
      sql/sql_update.cc:
        Update to use the new signature of close_tables_for_reopen().
      sql/table.cc:
        Move re-setting of mdl_requests for prepared statements
        and stored procedures from close_thread_tables() to
        reinit_stmt_before_use().
        Change alloc_mdl_requests() to init_mdl_requests().
        init_mdl_requests() is a hack that can't be deleted
        until we don't have a list-aware TABLE_LIST constructor.
        Hopefully its use will be minimal
      sql/table.h:
        Change alloc_mdl_requests() to init_mdl_requests()
        TABLE_LIST::mdl_request is stored by value.
      sql/tztime.cc:
        We no longer initialize mdl requests in open_system_tables_for*()
        functions. Move this initialization closer to initialization
        of the rest of TABLE_LIST members.
      storage/myisammrg/ha_myisammrg.cc:
        Simplify mdl_request initialization.
      ce5c87a3
  13. 03 Dec, 2009 3 commits
    • Konstantin Osipov's avatar
      Backport of: · a9013f8f
      Konstantin Osipov authored
      ----------------------------------------------------------
      revno: 2617.23.20
      committer: Konstantin Osipov <kostja@sun.com>
      branch nick: mysql-6.0-runtime
      timestamp: Wed 2009-03-04 16:31:31 +0300
      message:
        WL#4284 "Transactional DDL locking"
        Review comments: "Objectify" the MDL API.
        MDL_request and MDL_context still need manual construction and
        destruction, since they are used in environment that is averse
        to constructors/destructors.
      
      
      sql/mdl.cc:
        Improve comments.
        Add asserts to backup()/restore_from_backup()/merge() methods.
        Fix an order bug in the error path of mdl_acquire_exclusive_locks():
        we used to first free a ticket object, and only then exclude
        it from the list of tickets.
      a9013f8f
    • Konstantin Osipov's avatar
      Backport of: · f477e66e
      Konstantin Osipov authored
      ------------------------------------------------------------
      revno: 2617.23.18
      committer: Davi Arnaut <Davi.Arnaut@Sun.COM>
      branch nick: 4284-6.0
      timestamp: Mon 2009-03-02 18:18:26 -0300
      message:
      Bug#989: If DROP TABLE while there's an active transaction, wrong binlog order
      WL#4284: Transactional DDL locking
      
      This is a prerequisite patch:
      
      These changes are intended to split lock requests from granted
      locks and to allow the memory and lifetime of granted locks to
      be managed within the MDL subsystem. Furthermore, tickets can
      now be shared and therefore are used to satisfy multiple lock
      requests, but only shared locks can be recursive.
      
      The problem is that the MDL subsystem morphs lock requests into
      granted locks locks but does not manage the memory and lifetime
      of lock requests, and hence, does not manage the memory of
      granted locks either. This can be problematic because it puts the
      burden of tracking references on the users of the subsystem and
      it can't be easily done in transactional contexts where the locks
      have to be kept around for the duration of a transaction.
      
      Another issue is that recursive locks (when the context trying to
      acquire a lock already holds a lock on the same object) requires
      that each time the lock is granted, a unique lock request/granted
      lock structure structure must be kept around until the lock is
      released. This can lead to memory leaks in transactional contexts
      as locks taken during the transaction should only be released at
      the end of the transaction. This also leads to unnecessary wake
      ups (broadcasts) in the MDL subsystem if the context still holds
      a equivalent of the lock being released.
      
      These issues are exacerbated due to the fact that WL#4284 low-level
      design says that the implementation should "2) Store metadata locks
      in transaction memory root, rather than statement memory root" but
      this is not possible because a memory root, as implemented in mysys,
      requires all objects allocated from it to be freed all at once.
      
      This patch combines review input and significant code contributions
      from Konstantin Osipov (kostja) and Dmitri Lenev (dlenev).
      
      
      mysql-test/r/mdl_sync.result:
        Add test case result.
      mysql-test/t/mdl_sync.test:
        Add test case for shared lock upgrade case.
      sql/event_db_repository.cc:
        Rename mdl_alloc_lock to mdl_request_alloc.
      sql/ha_ndbcluster_binlog.cc:
        Use new function names to initialize MDL lock requests.
      sql/lock.cc:
        Rename MDL functions.
      sql/log_event.cc:
        The MDL request now holds the table and database name data (MDL_KEY).
      sql/mdl.cc:
        Move the MDL key to the MDL_LOCK structure in order to make the
        object suitable for allocation from a fixed-size allocator. This
        allows the simplification of the lists in the MDL_LOCK object,
        which now are just two, one for granted tickets and other for
        waiting (upgraders) tickets.
        
        Recursive requests for a shared lock on the same object can now
        be granted using the same lock ticket. This schema is only used
        for shared locks because that the only case that matters. This
        is used to avoid waste of resources in case a context (connection)
        already holds a shared lock on a object.
      sql/mdl.h:
        Introduce a metadata lock object key which is used  to uniquely
        identify lock objects.
        
        Separate the structure used to represent pending lock requests
        from the structure used to represent granted metadata locks.
        
        Rename functions used to manipulate locks requests in order to
        have a more consistent function naming schema.
      sql/sp_head.cc:
        Rename mdl_alloc_lock to mdl_request_alloc.
      sql/sql_acl.cc:
        Rename alloc_mdl_locks to alloc_mdl_requests.
      sql/sql_base.cc:
        Various changes to accommodate that lock requests are separated
        from lock tickets (granted locks).
      sql/sql_class.h:
        Last acquired lock before the savepoint was set.
      sql/sql_delete.cc:
        Various changes to accommodate that lock requests are separated
        from lock tickets (granted locks).
      sql/sql_handler.cc:
        Various changes to accommodate that lock requests are separated
        from lock tickets (granted locks).
      sql/sql_insert.cc:
        Rename alloc_mdl_locks to alloc_mdl_requests.
      sql/sql_parse.cc:
        Rename alloc_mdl_locks to alloc_mdl_requests.
      sql/sql_plist.h:
        Typedef for iterator type.
      sql/sql_plugin.cc:
        Rename alloc_mdl_locks to alloc_mdl_requests.
      sql/sql_servers.cc:
        Rename alloc_mdl_locks to alloc_mdl_requests.
      sql/sql_show.cc:
        Various changes to accommodate that lock requests are separated
        from lock tickets (granted locks).
      sql/sql_table.cc:
        Various changes to accommodate that lock requests are separated
        from lock tickets (granted locks).
      sql/sql_trigger.cc:
        Save reference to the lock ticket so it can be downgraded later.
      sql/sql_udf.cc:
        Rename alloc_mdl_locks to alloc_mdl_requests.
      sql/table.cc:
        Rename mdl_alloc_lock to mdl_request_alloc.
      sql/table.h:
        Separate MDL lock requests from lock tickets (granted locks).
      storage/myisammrg/ha_myisammrg.cc:
        Rename alloc_mdl_locks to alloc_mdl_requests.
      f477e66e
    • Konstantin Osipov's avatar
      Backport of: · cd155be5
      Konstantin Osipov authored
      ------------------------------------------------------------
      revno: 3035.4.1
      committer: Davi Arnaut <Davi.Arnaut@Sun.COM>
      branch nick: 39897-6.0
      timestamp: Thu 2009-01-15 12:17:57 -0200
      message:
      Bug#39897: lock_multi fails in pushbuild: timeout waiting for processlist
      
      The problem is that relying on the "Table lock" thread state in
      its current position to detect that a thread is waiting on a lock
      is race prone. The "Table lock" state change happens before the
      thread actually tries to grab a lock on a table.
      
      The solution is to move the "Table lock" state so that its set
      only when a thread is actually going to wait for a lock. The state
      change happens after the thread fails to grab the lock (because it
      is owned by other thread) and proceeds to wait on a condition.
      
      This is considered part of work related to WL#4284 "Transactional
      DDL locking"
      Warning: this patch contains an incompatible change. 
      When waiting on a lock in thr_lock.c, the server used to display "Locked"
      processlist state. After this patch, the state is "Table lock".
      The new state was actually intended to be display since year 2002,
      when Monty added it. But up until removal of thd->locked boolean
      member, this state was ignored by SHOW PROCESSLIST code.  
      
      mysql-test/r/lock_multi.result:
        A style fix.
      mysql-test/r/sp-threads.result:
        Changed output of SHOW PROCESSLIST (new wait state).
      mysql-test/t/lock_multi.test:
        Use a more accurate state description when waiting inside thr_lock.c.
      mysql-test/t/lock_sync.test:
        Use a more accurate state description when waiting inside thr_lock.c.
      mysql-test/t/multi_update.test:
        Use a more accurate state description when waiting inside thr_lock.c.
      mysql-test/t/query_cache_28249.test:
        Use a more accurate state description when waiting inside thr_lock.c.
      mysql-test/t/sp_notembedded.test:
        Use a more accurate state description when waiting inside thr_lock.c.
      mysql-test/t/status.test:
        Use a more accurate state description when waiting inside thr_lock.c.
      mysys/thr_lock.c:
        Update thread state while waiting for a table lock.
      sql/lock.cc:
        State change was moved inside thr_lock.c.
      cd155be5
  14. 02 Dec, 2009 3 commits
    • Konstantin Osipov's avatar
      Backport of: · 77cbfd85
      Konstantin Osipov authored
      ----------------------------------------------------------
      revno: 2630.4.35
      committer: Konstantin Osipov <konstantin@mysql.com>
      branch nick: mysql-6.0-3726
      timestamp: Wed 2008-06-25 16:44:00 +0400
      message:
        Fix a MyISAM-specific bug in the new implementation of
        LOCK TABLES (WL#3726).
        If more than one instance of a MyISAM table are open in the
        same connection, all of them must share the same status_param.
        Otherwise, unlock of a table may lead to lost records.
        See also comments in thr_lock.c.
      
      include/thr_lock.h:
         Declare thr_lock_merge_status().
      mysql-test/r/lock.result:
        Update test results (WL#3726).
      mysql-test/t/lock.test:
        Add a test case for the situation when the same table is locked
        twice by LOCK TABLES, and only one instance is updated.
      mysys/thr_lock.c:
        Move the code that makes sure all status_params of the same
        table are shared into a separate function.
      sql/lock.cc:
        Make sure that status_param is shared when a table is reopened.
      77cbfd85
    • Konstantin Osipov's avatar
      Backport of: · e3b3907c
      Konstantin Osipov authored
      ------------------------------------------------------------
      revno: 2630.4.32
      committer: Dmitry Lenev <dlenev@mysql.com>
      branch nick: mysql-6.0-3726-w2
      timestamp: Thu 2008-06-19 16:39:58 +0400
      message:
        WL#3726 "DDL locking for all metadata objects".
      
        After-review fixes in progress.
      
        Ensure that metadata locking subsystem properly handles
        out-of-memory conditions. Clarified MDL interface by
        separating release of locks and removal of lock requests
        from the context.
      
      sql/lock.cc:
        mdl_release_lock(), mdl_acquire_exclusive_locks() and 
        mdl_try_acquire_exclusive_lock() are no longer responsible
        for removal of metadata lock requests from the context.
        One should explicitly call mdl_remove_all_locks() and
        mdl_remove_lock() to do this.
      sql/mdl.cc:
        Ensured that metadata locking subsystem properly handles
        out-of-memory conditions.
        Introduced new MDL_INITIALIZED state for metadata lock
        request which is used in all cases when lock is not acquired 
        and we have not associated request with object respesenting 
        lock.
        
        MDL_PENDING is now only used for requests for exclusive locks
        which are added to the MDL_LOCK::waiting_exclusive queue.
        mdl_release_lock(), mdl_acquire_exclusive_locks() and 
        mdl_try_acquire_exclusive_lock() are no longer responsible
        for removal of metadata lock requests from the context.
        One should explicitly call mdl_remove_all_locks() and
        newly introduced mdl_remove_lock() to do this.
        Also renamed mdl_release_all_locks_for_name() to 
        emphasize that it also actually removes lock requests
        from the context.
        
        Finally mdl_try_acquire_exclusive_lock() is now returs
        information about encountered lock conflict in separate
        out parameter since its return value is used for distinguishing
        between error (e.g. due to OOM) and success.
      sql/mdl.h:
        Introduced new MDL_INITIALIZED state for metadata lock
        request which is used in all cases when lock is not acquired 
        and we have not associated request with object respesenting 
        lock.
        
        MDL_PENDING is now only used for requests for exclusive locks
        which are added to the MDL_LOCK::waiting_exclusive queue.
        mdl_release_lock(), mdl_acquire_exclusive_locks() and 
        mdl_try_acquire_exclusive_lock() are no longer responsible
        for removal of metadata lock requests from the context.
        One should explicitly call mdl_remove_all_locks() and
        newly introduced mdl_remove_lock() to do this.
        Also renamed mdl_release_all_locks_for_name() to 
        emphasize that it also actually removes lock requests
        from the context.
        
        Finally mdl_try_acquire_exclusive_lock() is now returs
        information about encountered lock conflict in separate
        out parameter since its return value is used for distinguishing
        between error (e.g. due to OOM) and success.
      sql/sql_base.cc:
        mdl_release_lock(), mdl_acquire_exclusive_locks() and mdl_try_acquire_exclusive_lock() are no longer responsible
        for removal of metadata lock requests from the context.
        One should explicitly call mdl_remove_all_locks() and
        mdl_remove_lock() to do this.
        Also adjusted open_table() to ensure that it 
        releases/removes metadata locks in case of error 
        after adding/acquiring them (unless keeping these
        lock requests is required for recovering action).
      sql/sql_delete.cc:
        mdl_release_lock(), mdl_acquire_exclusive_locks() and mdl_try_acquire_exclusive_lock() are no longer responsible
        for removal of metadata lock requests from the context.
        One should explicitly call mdl_remove_all_locks() and
        mdl_remove_lock() to do this.
      sql/sql_handler.cc:
        mdl_release_lock(), mdl_acquire_exclusive_locks() and mdl_try_acquire_exclusive_lock() are no longer responsible
        for removal of metadata lock requests from the context.
        One should explicitly call mdl_remove_all_locks() and
        mdl_remove_lock() to do this.
      sql/sql_show.cc:
        mdl_release_lock(), mdl_acquire_exclusive_locks() and mdl_try_acquire_exclusive_lock() are no longer responsible
        for removal of metadata lock requests from the context.
        One should explicitly call mdl_remove_all_locks() and
        mdl_remove_lock() to do this.
      sql/sql_table.cc:
        Renamed mdl_release_all_locks_for_name() to emphasize
        that it also actually removes lock requests from the context.
        mdl_release_lock(), mdl_acquire_exclusive_locks() and mdl_try_acquire_exclusive_lock() are no longer responsible
        for removal of metadata lock requests from the context.
        One should explicitly call mdl_remove_all_locks() and
        mdl_remove_lock() to do this.
        Finally mdl_try_acquire_exclusive_lock() is now returs
        information about encountered lock conflict in separate
        out parameter since its return value is used for distinguishing
        between error (e.g. due to OOM) and success.
      e3b3907c
    • Konstantin Osipov's avatar
      Backport of: · 3104af49
      Konstantin Osipov authored
      ----------------------------------------------------------
      revno: 2630.10.1
      committer: Konstantin Osipov <konstantin@mysql.com>
      branch nick: mysql-6.0-lock-tables-tidyup
      timestamp: Wed 2008-06-11 15:49:58 +0400
      message:
        WL#3726, review fixes.
        Now that we have metadata locks, we don't need to keep a crippled
        TABLE instance in the table cache to indicate that a table is locked.
        Remove all code that used this technique. Instead, rely on metadata
        locks and use the standard open_table() and close_thread_table()
        to manipulate with the table cache tables.
        Removes a list of functions that have become unused (see the comment
        for sql_base.cc for details).
        Under LOCK TABLES, keep a TABLE_LIST instance for each table
        that may be temporarily closed. For that, implement an own class for
        LOCK TABLES mode, Locked_tables_list.
      
      This is a pre-requisite patch for WL#4144.
      This is not exactly a backport: there is no new 
      online ALTER table in Celosia, so the old alter table
      code was changed to work with the new table cache API.
      
      
      mysql-test/r/lock.result:
        Update results (WL#3726 post-review patch).
      mysql-test/r/trigger-compat.result:
        We take the table from the table cache now, thus no warning.
      mysql-test/suite/rpl/r/rpl_trigger.result:
        We take the table from the table cache now, thus no warning.
      mysql-test/t/lock.test:
        Additional tests for LOCK TABLES mode (previously not covered by
        the test suite (WL#3726).
      sql/field.h:
        Remove reopen_table().
      sql/lock.cc:
        Remove an obsolete parameter of mysql_lock_remove().
        It's not used anywhere now either.
      sql/mysql_priv.h:
        Add 4 new open_table() flags.
        Remove declarations of removed functions.
      sql/sp_head.cc:
        Rename thd->mdl_el_root to thd->locked_tables_root.
      sql/sql_acl.cc:
        Use the new implementation of unlock_locked_tables().
      sql/sql_base.cc:
        Implement class Locked_tables_list.
        Implement close_all_tables_for_name().
        Rewrite close_cached_tables() to use the new reopen_tables().
        Remove reopen_table(), reopen_tables(), reopen_table_entry() 
        (ex. open_unireg_entry()), close_data_files_and_leave_as_placeholders(),
        close_handle_and_leave_table_as_placeholder(), close_cached_table(),
        table_def_change_share(), reattach_merge(), reopen_name_locked_table(),
        unlink_open_table().
        
        Move acquisition of a metadata lock into an own function
        - open_table_get_mdl_lock().
      sql/sql_class.cc:
        Deploy class Locked_tables_list.
      sql/sql_class.h:
        Declare class Locked_tables_list.
        Keep one instance of this class in class THD.
        Rename mdl_el_root to locked_tables_root.
      sql/sql_db.cc:
        Update a comment.
      sql/sql_insert.cc:
        Use the plain open_table() to open a just created table in
        CREATE TABLE .. SELECT.
      sql/sql_parse.cc:
        Use thd->locked_tables_list to enter and leave LTM_LOCK_TABLES mode.
      sql/sql_partition.cc:
        Deploy the new method of working with partitioned table locks.
      sql/sql_servers.cc:
        Update to the new signature of unlock_locked_tables().
      sql/sql_table.cc:
        In mysql_rm_table_part2(), the branch that removes a table under
        LOCK TABLES, make sure that the table being dropped
        is also removed from THD::locked_tables_list.
        Update ALTER TABLE and CREATE TABLE LIKE implementation to use
        open_table() and close_all_tables_for_name() instead of 
        reopen_tables().
      sql/sql_trigger.cc:
        Use the new locking way.
      sql/table.h:
        Add TABLE::pos_in_locked_tables, which is used only under
        LOCK TABLES.
      3104af49
  15. 01 Dec, 2009 1 commit
    • Konstantin Osipov's avatar
      Backport of: · 5969dcda
      Konstantin Osipov authored
      ----------------------------------------------------------
      revno: 2630.4.26
      committer: Konstantin Osipov <konstantin@mysql.com>
      branch nick: mysql-6.0-prelocked_mode-to-push
      timestamp: Fri 2008-06-06 23:19:04 +0400
      message:
        WL#3726: work on review comments.
        Remove thd->locked_tables. Always store MYSQL_LOCK instances in
        thd->lock.
        Rename thd->prelocked_mode to thd->locked_tables_mode.
        Use thd->locked_tables_mode to determine if we
        are under LOCK TABLES. Update the code to not assume that
        if thd->lock is set, LOCK TABLES mode is off.
        Review comments.
      
      
      sql/ha_ndbcluster_binlog.cc:
        Don't unlock the lock under LOCK TABLES (safety).
      sql/handler.cc:
        There is no thd->locked_tables any more.
        Update comments.
      sql/lock.cc:
        There is no thd->locked_tables any more.
      sql/log.cc:
        Rename thd->prelocked_mode to thd->locked_tables_mode.
      sql/set_var.cc:
        Use thd->locked_tables_mode to determine if we are under LOCK TABLES.
      sql/sp_head.cc:
        Rename thd->prelocked_mode to thd->locked_tables_mode.
      sql/sql_base.cc:
        Use thd->locked_tables_mode to determine if we are under LOCK TABLES.
        Remove thd->locked_tables.
      sql/sql_cache.cc:
        Use thd->locked_tables_mode to determine if we are under LOCK TABLES.
      sql/sql_class.cc:
        Avoid code duplication.
        Do not release the table locks prematurely if we're under LOCK TABLES.
        Use thd->locked_tables_mode instead of thd->locked_tables.
      sql/sql_class.h:
        Remove thd->locked_tables.
        Make prelocked mode a kind of LOCK TABLES mode.
        Update comments.
      sql/sql_cursor.cc:
        Update comments.
      sql/sql_insert.cc:
        Use thd->locked_tables_mode to determine if we are under LOCK TABLES.
        Rename thd->prelocked_mode to thd->locked_tables_mode.
      sql/sql_load.cc:
        Rename thd->prelocked_mode to thd->locked_tables_mode.
      sql/sql_parse.cc:
        Use thd->locked_tables_mode to determine if we are under LOCK TABLES.
        Remove thd->locked_tables.
      sql/sql_partition.cc:
        Use thd->locked_tables_mode to determine if we are under LOCK TABLES.
      sql/sql_rename.cc:
        Use thd->locked_tables_mode to determine if we are under LOCK TABLES.
      sql/sql_select.cc:
        Use thd->locked_tables_mode to determine if we are under LOCK TABLES.
      sql/sql_table.cc:
        Use thd->locked_tables_mode to determine if we are under LOCK TABLES.
      sql/sql_trigger.cc:
        Use thd->locked_tables_mode to determine if we are under LOCK TABLES.
      sql/sql_update.cc:
        Use thd->locked_tables_mode to determine if we are under LOCK TABLES.
      sql/sql_view.cc:
        Use thd->locked_tables_mode to determine if we are under LOCK TABLES.
      storage/myisam/ha_myisam.cc:
        Use thd->locked_tables_mode to determine if we are under LOCK TABLES.
      5969dcda
  16. 30 Nov, 2009 4 commits
    • Konstantin Osipov's avatar
      Backport of: · b05303c1
      Konstantin Osipov authored
      ------------------------------------------------------------
      revno: 2630.4.18
      committer: Dmitry Lenev <dlenev@mysql.com>
      branch nick: mysql-6.0-3726-w2
      timestamp: Tue 2008-06-03 21:07:58 +0400
      message:
        WL#3726 "DDL locking for all metadata objects".
      
        After review fixes in progress.
      
        Now during upgrading/downgrading metadata locks we deal with
        individual metadata lock requests rather than with all requests
        for this object in the context. This makes API a bit more clear
        and makes adjust_mdl_locks_upgradability() much nicer.
      
      
      sql/lock.cc:
        lock_table_names():
          Set TABLE_LIST::mdl_lock_data when allocating new metadata
          lock request object for table list element.
      sql/mdl.cc:
        Now during upgrading/downgrading metadata locks we deal with
        individual metadata lock requests rather than with all
        requests for this object in the context. Adjusted upgrade/
        downgrade functions accordingly.
        We also got rid of mdl_release_exclusive_locks() and
        now release locks individually. To simplify this process
        mdl_release_all_locks_for_name() was introduced.
      sql/mdl.h:
        Now during upgrading/downgrading metadata locks we deal with
        individual metadata lock requests rather than with all
        requests for this object in the context. Adjusted upgrade/
        downgrade functions accordingly.
        We also got rid of mdl_release_exclusive_locks() and
        now release locks individually. To simplify this process
        mdl_release_all_locks_for_name() was introduced.
      sql/sql_base.cc:
        Now during upgrading/downgrading metadata locks we deal with
        individual metadata lock requests rather than with all
        requests for this object in the context.
        We also got rid of mdl_release_exclusive_locks() and
        now release locks individually.
      sql/sql_parse.cc:
        adjust_mdl_locks_upgradability() is much simplier now due to the
        fact that now during upgrading/downgrading metadata locks we
        deal with individual metadata lock requests rather than with
        all requests for this object in the context.
      sql/sql_table.cc:
        Now during upgrading/downgrading metadata locks we deal with
        individual metadata lock requests rather than with all
        requests for this object in the context. Adjusted upgrade/
        downgrade functions accordingly.
        We also got rid of mdl_release_exclusive_locks() and
        now release locks individually. To simplify this process
        mdl_release_all_locks_for_name() was introduced.
      sql/sql_trigger.cc:
        ow during upgrading/downgrading metadata locks we deal with
        individual metadata lock requests rather than with all
        requests for this object in the context.
      b05303c1
    • Konstantin Osipov's avatar
      Backport of: · e23046d1
      Konstantin Osipov authored
      ------------------------------------------------------------
      revno: 2630.4.17
      committer: Dmitry Lenev <dlenev@mysql.com>
      branch nick: mysql-6.0-3726-w2
      timestamp: Thu 2008-05-29 16:52:56 +0400
      message:
        WL#3726 "DDL locking for all metadata objects".
      
        After review fixes in progress.
      
        "The great correction of names".
      
        Renamed MDL_LOCK and MDL_LOCK_DATA classes to make usage of
        these names in metadata locking subsystem consistent with
        other parts of server (i.e. thr_lock.cc). Now we MDL_LOCK_DATA
        corresponds to request for a lock and MDL_LOCK to the lock
        itself. Adjusted code in MDL subsystem and other places
        using these classes accordingly.
        Did similar thing for GLOBAL_MDL_LOCK_DATA class and also
        changed name of its members to correspond to names of
        MDL_LOCK_DATA members.
        Finally got rid of usage of one letter variables in MDL
        code since it makes code harder to search in (according
        to reviewer).
      e23046d1
    • Konstantin Osipov's avatar
      Backport of: · aeebede1
      Konstantin Osipov authored
      ------------------------------------------------------------
      revno: 2630.4.11
      committer: Dmitry Lenev <dlenev@mysql.com>
      branch nick: mysql-6.0-3726-w
      timestamp: Tue 2008-05-27 21:31:53 +0400
      message:
        WL#3726 "DDL locking for all metadata objects".
      
        After review fixes in progress.
      
        Changed mysql_lock_tables() to be no longer responsible for
        reopening table if waiting for the lock on it was aborted.
        This allows to get rid of several annoying functions.
      
      
      sql/ha_ndbcluster_binlog.cc:
        lock_tables() now also accepts set of options to be passed to
        mysql_lock_tables().
      sql/lock.cc:
        Changed mysql_lock_tables() always requests caller to reopen
        table instead doing this on its own when waiting for lock was
        aborted. This allows us to get rid of several functions which
        were used in rare cases and significantly complicated our life.
      sql/log_event_old.cc:
        lock_tables() now also accepts set of options to be passed
        to mysql_lock_tables().
      sql/mysql_priv.h:
        Now mysql_lock_tables() always requests caller to reopen
        table instead doing this on its own when waiting for lock was
        aborted. So we no longer need wait_for_tables() and
        table_is_used() functions and MYSQL_LOCK_NOTIFY_IF_NEED_REOPEN
        flag.
        open_and_lock_table_derived() and lock_tables() now accept
        options to be passed to open_tables() and mysql_lock_tables()
        calls.
      sql/sql_base.cc:
        Since now mysql_lock_tables() always requests caller to
        reopen table instead doing this on its own when waiting for
        lock was aborted we no longer need wait_for_tables(),
        table_is_used() and close_old_data_files() functions.
        open_and_lock_table_derived() and lock_tables() now accept
        options to be passed to open_tables() and mysql_lock_tables()
        calls. This was needed in order to get rid of redundant code
        in open_system_tables_for_read() function.
      sql/sql_handler.cc:
        mysql_lock_tables() is now always requests reopen if waiting
        for lock is aborted. MYSQL_LOCK_NOTIFY_IF_NEED_REOPEN flag
        was removed.
      sql/sql_insert.cc:
        handle_delayed_insert():
        Since mysql_lock_tables() is no longer responsible for
        reopening tables when waiting for lock was aborted we have
        to handle such situation outside of this function. To simplify
        this extracted code opening table for delayed insert thread to
        separate method of Delayed_insert class.
      sql/sql_update.cc:
        lock_tables() now also accepts set of options to be passed
        to mysql_lock_tables().
      aeebede1
    • Konstantin Osipov's avatar
      Initial import of WL#3726 "DDL locking for all metadata objects". · eff3780d
      Konstantin Osipov authored
      Backport of:
      ------------------------------------------------------------
      revno: 2630.4.1
      committer: Dmitry Lenev <dlenev@mysql.com>
      branch nick: mysql-6.0-3726-w
      timestamp: Fri 2008-05-23 17:54:03 +0400
      message:
        WL#3726 "DDL locking for all metadata objects".
      
        After review fixes in progress.
      ------------------------------------------------------------
      
      This is the first patch in series. It transforms the metadata 
      locking subsystem to use a dedicated module (mdl.h,cc). No 
      significant changes in the locking protocol. 
      The import passes the test suite with the exception of 
      deprecated/removed 6.0 features, and MERGE tables. The latter
      are subject to a fix by WL#4144.
      Unfortunately, the original changeset comments got lost in a merge,
      thus this import has its own (largely insufficient) comments.
      
      This patch fixes Bug#25144 "replication / binlog with view breaks".
      Warning: this patch introduces an incompatible change:
      Under LOCK TABLES, it's no longer possible to FLUSH a table that 
      was not locked for WRITE.
      Under LOCK TABLES, it's no longer possible to DROP a table or
      VIEW that was not locked for WRITE.
      
      ******
      Backport of:
      ------------------------------------------------------------
      revno: 2630.4.2
      committer: Dmitry Lenev <dlenev@mysql.com>
      branch nick: mysql-6.0-3726-w
      timestamp: Sat 2008-05-24 14:03:45 +0400
      message:
        WL#3726 "DDL locking for all metadata objects".
      
        After review fixes in progress.
      
      ******
      Backport of:
      ------------------------------------------------------------
      revno: 2630.4.3
      committer: Dmitry Lenev <dlenev@mysql.com>
      branch nick: mysql-6.0-3726-w
      timestamp: Sat 2008-05-24 14:08:51 +0400
      message:
        WL#3726 "DDL locking for all metadata objects"
      
        Fixed failing Windows builds by adding mdl.cc to the lists
        of files needed to build server/libmysqld on Windows.
      
      ******
      Backport of:
      ------------------------------------------------------------
      revno: 2630.4.4
      committer: Dmitry Lenev <dlenev@mysql.com>
      branch nick: mysql-6.0-3726-w
      timestamp: Sat 2008-05-24 21:57:58 +0400
      message:
        WL#3726 "DDL locking for all metadata objects".
      
        Fix for assert failures in kill.test which occured when one
        tried to kill ALTER TABLE statement on merge table while it
        was waiting in wait_while_table_is_used() for other connections
        to close this table.
      
        These assert failures stemmed from the fact that cleanup code
        in this case assumed that temporary table representing new
        version of table was open with adding to THD::temporary_tables
        list while code which were opening this temporary table wasn't
        always fulfilling this.
      
        This patch changes code that opens new version of table to
        always do this linking in. It also streamlines cleanup process
        for cases when error occurs while we have new version of table
        open.
      
      ******
      WL#3726 "DDL locking for all metadata objects"
      Add libmysqld/mdl.cc to .bzrignore.
      ******
      Backport of:
      ------------------------------------------------------------
      revno: 2630.4.6
      committer: Dmitry Lenev <dlenev@mysql.com>
      branch nick: mysql-6.0-3726-w
      timestamp: Sun 2008-05-25 00:33:22 +0400
      message:
        WL#3726 "DDL locking for all metadata objects".
      
        Addition to the fix of assert failures in kill.test caused by
        changes for this worklog.
      
      
      Make sure we close the new table only once.
      
      .bzrignore:
        Add libmysqld/mdl.cc
      libmysqld/CMakeLists.txt:
        Added mdl.cc to the list of files needed for building of libmysqld.
      libmysqld/Makefile.am:
        Added files implementing new meta-data locking subsystem to the server.
      mysql-test/include/handler.inc:
        Use separate connection for waiting while threads performing DDL
        operations conflicting with open HANDLER tables reach blocked
        state. This is required because now we check and close tables open
        by HANDLER statements in this connection conflicting with DDL in
        another each time open_tables() is called and thus select from I_S
        which is used for waiting will unblock DDL operations if issued
        from connection with open HANDLERs.
      mysql-test/r/create.result:
        Adjusted test case after change in implementation of CREATE TABLE
        ... SELECT.  We no longer have special check in open_table() which
        catches the case when we select from the table created. Instead we
        rely on unique_table() call which happens after opening and
        locking all tables.
      mysql-test/r/flush.result:
        FLUSH TABLES WITH READ LOCK can no longer happen under LOCK
        TABLES.  Updated test accordingly.
      mysql-test/r/flush_table.result:
        Under LOCK TABLES we no longer allow to do FLUSH TABLES for tables
        locked for read. Updated test accordingly.
      mysql-test/r/handler_innodb.result:
        Use separate connection for waiting while threads performing DDL
        operations conflicting with open HANDLER tables reach blocked
        state. This is required because now we check and close tables open
        by HANDLER statements in this connection conflicting with DDL in
        another each time open_tables() is called and thus select from I_S
        which is used for waiting will unblock DDL operations if issued
        from connection with open HANDLERs.
      mysql-test/r/handler_myisam.result:
        Use separate connection for waiting while threads performing DDL
        operations conflicting with open HANDLER tables reach blocked
        state. This is required because now we check and close tables open
        by HANDLER statements in this connection conflicting with DDL in
        another each time open_tables() is called and thus select from I_S
        which is used for waiting will unblock DDL operations if issued
        from connection with open HANDLERs.
      mysql-test/r/information_schema.result:
        Additional test for WL#3726 "DDL locking for all metadata
        objects".  Check that we use high-priority metadata lock requests
        when filling I_S tables.
        
        Rearrange tests to match 6.0 better (fewer merge conflicts).
      mysql-test/r/kill.result:
        Added tests checking that DDL and DML statements waiting for
        metadata locks can be interrupted by KILL command.
      mysql-test/r/lock.result:
        One no longer is allowed to do DROP VIEW under LOCK TABLES even if
        this view is locked by LOCK TABLES. The problem is that in such
        situation write locks on view are not mutually exclusive so
        upgrading metadata lock which is required for dropping of view
        will lead to deadlock.
      mysql-test/r/partition_column_prune.result:
        Update results (same results in 6.0), WL#3726
      mysql-test/r/partition_pruning.result:
        Update results (same results in 6.0), WL#3726
      mysql-test/r/ps_ddl.result:
        We no longer invalidate prepared CREATE TABLE ... SELECT statement
        if target table changes. This is OK since it is not strictly
        necessary.
        
        
        The first change is wrong, is caused by FLUSH TABLE
        now flushing all unused tables. This is a regression that
        Dmitri fixed in 6.0 in a follow up patch.
      mysql-test/r/sp.result:
        Under LOCK TABLES we no longer allow accessing views which were
        not explicitly locked. To access view we need to obtain metadata
        lock on it and doing this under LOCK TABLES may lead to deadlocks.
      mysql-test/r/view.result:
        One no longer is allowed to do DROP VIEW under LOCK TABLES even if
        this view is locked by LOCK TABLES. The problem is that in such
        situation even "write locks" on view are not mutually exclusive so
        upgrading metadata lock which is required for dropping of view
        will lead to deadlock
      mysql-test/r/view_grant.result:
        ALTER VIEW implementation was changed to open a view only after
        checking that user which does alter has appropriate privileges on
        it. This means that in case when user's privileges are
        insufficient for this we won't check that new view definer is the
        same as original one or user performing alter has SUPER privilege.
        Adjusted test case accordingly.
      mysql-test/r/view_multi.result:
        Added test case for bug#25144 "replication / binlog with view
        breaks".
      mysql-test/suite/rpl/t/disabled.def:
        Disable test for deprecated features (they don't work with new MDL).
      mysql-test/t/create.test:
        Adjusted test case after change in implementation of CREATE TABLE
        ... SELECT.  We no longer have special check in open_table() which
        catches the case when we select from the table created. Instead we
        rely on unique_table() call which happens after opening and
        locking all tables.
      mysql-test/t/disabled.def:
        Disable merge.test, subject of WL#4144
      mysql-test/t/flush.test:
        
        FLUSH TABLES WITH READ LOCK can no longer happen under LOCK
        TABLES.  Updated test accordingly.
      mysql-test/t/flush_table.test:
        Under LOCK TABLES we no longer allow to do FLUSH TABLES for tables
        locked for read. Updated test accordingly.
      mysql-test/t/information_schema.test:
        Additional test for WL#3726 "DDL locking for all metadata
        objects".  Check that we use high-priority metadata lock requests
        when filling I_S tables.
        
        Rearrange the results for easier merges with 6.0.
      mysql-test/t/kill.test:
        Added tests checking that DDL and DML statements waiting for
        metadata locks can be interrupted by KILL command.
      mysql-test/t/lock.test:
        One no longer is allowed to do DROP VIEW under LOCK TABLES even if
        this view is locked by LOCK TABLES. The problem is that in such
        situation write locks on view are not mutually exclusive so
        upgrading metadata lock which is required for dropping of view
        will lead to deadlock.
      mysql-test/t/lock_multi.test:
        Adjusted test case to the changes of status in various places
        caused by change in implementation FLUSH TABLES WITH READ LOCK,
        which is now takes global metadata lock before flushing tables and
        therefore waits on at these places.
      mysql-test/t/ps_ddl.test:
        We no longer invalidate prepared CREATE TABLE ... SELECT statement
        if target table changes. This is OK since it is not strictly
        necessary.
        
        
        The first change is wrong, is caused by FLUSH TABLE
        now flushing all unused tables. This is a regression that
        Dmitri fixed in 6.0 in a follow up patch.
      mysql-test/t/sp.test:
        Under LOCK TABLES we no longer allow accessing views which were
        not explicitly locked. To access view we need to obtain metadata
        lock on it and doing this under LOCK TABLES may lead to deadlocks.
      mysql-test/t/trigger_notembedded.test:
        Adjusted test case to the changes of status in various places
        caused by change in implementation FLUSH TABLES WITH READ LOCK,
        which is now takes global metadata lock before flushing tables and
        therefore waits on at these places.
      mysql-test/t/view.test:
        One no longer is allowed to do DROP VIEW under LOCK TABLES even if
        this view is locked by LOCK TABLES. The problem is that in such
        situation even "write locks" on view are not mutually exclusive so
        upgrading metadata lock which is required for dropping of view
        will lead to deadlock.
      mysql-test/t/view_grant.test:
        ALTER VIEW implementation was changed to open a view only after
        checking that user which does alter has appropriate privileges on
        it. This means that in case when user's privileges are
        insufficient for this we won't check that new view definer is the
        same as original one or user performing alter has SUPER privilege.
        Adjusted test case accordingly.
      mysql-test/t/view_multi.test:
        Added test case for bug#25144 "replication / binlog with view
        breaks".
      sql/CMakeLists.txt:
        Added mdl.cc to the list of files needed for building of server.
      sql/Makefile.am:
        Added files implementing new meta-data locking subsystem to the
        server.
      sql/event_db_repository.cc:
        
        Allocate metadata lock requests objects (MDL_LOCK) on execution
        memory root in cases when TABLE_LIST objects is also allocated
        there or on stack.
      sql/ha_ndbcluster.cc:
        Adjusted code to work nicely with new metadata locking subsystem.
        close_cached_tables() no longer has wait_for_placeholder argument.
        Instead of relying on this parameter and related behavior FLUSH
        TABLES WITH READ LOCK now takes global shared metadata lock.
      sql/ha_ndbcluster_binlog.cc:
        Adjusted code to work with new metadata locking subsystem.
        close_cached_tables() no longer has wait_for_placeholder argument.
        Instead of relying on this parameter and related behavior FLUSH
        TABLES WITH READ LOCK now takes global shared metadata lock.
      sql/handler.cc:
        update_frm_version():
          Directly update TABLE_SHARE::mysql_version member instead of
          going through all TABLE instances for this table (old code was a
          legacy from pre-table-definition-cache days).
      sql/lock.cc:
        Use new metadata locking subsystem. Threw away most of functions
        related to name locking as now one is supposed to use metadata
        locking API instead.  In lock_global_read_lock() and
        unlock_global_read_lock() in order to avoid problems with global
        read lock sneaking in at the moment when we perform FLUSH TABLES
        or ALTER TABLE under LOCK TABLES and when tables being reopened
        are protected only by metadata locks we also have to take global
        shared meta data lock.
      sql/log_event.cc:
        Adjusted code to work with new metadata locking subsystem.  For
        tables open by slave thread for applying RBR events allocate
        memory for lock request object in the same chunk of memory as
        TABLE_LIST objects for them. In order to ensure that we keep these
        objects around until tables are open always close tables before
        calling Relay_log_info::clear_tables_to_lock(). Use new auxiliary
        Relay_log_info::slave_close_thread_tables() method to enforce
        this.
      sql/log_event_old.cc:
        Adjusted code to work with new metadata locking subsystem.  Since
        for tables open by slave thread for applying RBR events memory for
        lock request object is allocated in the same chunk of memory as
        TABLE_LIST objects for them we have to ensure that we keep these
        objects around until tables are open. To ensure this we always
        close tables before calling
        Relay_log_info::clear_tables_to_lock(). To enfore this we use
        new auxiliary Relay_log_info::slave_close_thread_tables()
        method.
      sql/mdl.cc:
        Implemented new metadata locking subsystem and API described in
        WL3726 "DDL locking for all metadata objects".
      sql/mdl.h:
        Implemented new metadata locking subsystem and API described in
        WL3726 "DDL locking for all metadata objects".
      sql/mysql_priv.h:
        - close_thread_tables()/close_tables_for_reopen() now has one more
          argument which indicates that metadata locks should be released
          but not removed from the context in order to be used later in
          mdl_wait_for_locks() and tdc_wait_for_old_version().
        - close_cached_table() routine is no longer public.
        - Thread waiting in wait_while_table_is_used() can be now killed
          so this function returns boolean to make caller aware of such
          situation.
        - We no longer have  table cache as separate entity instead used
          and unused TABLE instances are linked to TABLE_SHARE objects in
          table definition cache.
        - Now third argument of open_table() is also used for requesting
          table repair or auto-discovery of table's new definition. So its
          type was changed from bool to enum.
        - Added tdc_open_view() function for opening view by getting its
          definition from disk (and table cache in future).
        - reopen_name_locked_table() no longer needs "link_in" argument as
          now we have exclusive metadata locks instead of dummy TABLE
          instances when this function is called.
        - find_locked_table() now takes head of list of TABLE instances
          instead of always scanning through THD::open_tables list. Also
          added find_write_locked_table() auxiliary.
        - reopen_tables(), close_cached_tables() no longer have
          mark_share_as_old and wait_for_placeholder arguments. Instead of
          relying on this parameters and related behavior FLUSH TABLES
          WITH READ LOCK now takes global shared metadata lock.
        - We no longer need drop_locked_tables() and
          abort_locked_tables().
        - mysql_ha_rm_tables() now always assume that LOCK_open is not
          acquired by caller.
        - Added notify_thread_having_shared_lock() callback invoked by
          metadata locking subsystem when acquiring an exclusive lock, for
          each thread that has a conflicting shared metadata lock.
        - Introduced expel_table_from_cache() as replacement for
          remove_table_from_cache() (the main difference is that this new
          function assumes that caller follows metadata locking protocol
          and never waits).
        - Threw away most of functions related to name locking. One should
          use new metadata locking subsystem and API instead.
      sql/mysqld.cc:
        Got rid of call initializing/deinitializing table cache since now
        it is embedded into table definition cache. Added calls for
        initializing/ deinitializing metadata locking subsystem.
      sql/rpl_rli.cc:
        Introduced auxiliary Relay_log_info::slave_close_thread_tables()
        method which is used for enforcing that we always close tables
        open for RBR before deallocating TABLE_LIST elements and MDL_LOCK
        objects for them.
      sql/rpl_rli.h:
        Introduced auxiliary Relay_log_info::slave_close_thread_tables()
        method which is used for enforcing that we always close tables
        open for RBR before deallocating TABLE_LIST elements and MDL_LOCK
        objects for them.
      sql/set_var.cc:
        close_cached_tables() no longer has wait_for_placeholder argument.
        Instead of relying on this parameter and related behavior FLUSH
        TABLES WITH READ LOCK now takes global shared metadata lock.
      sql/sp_head.cc:
        For tables added to the statement's table list by prelocking
        algorithm we allocate these objects either on the same memory as
        corresponding table list elements or on THD::locked_tables_root
        (if we are building table list for LOCK TABLES).
      sql/sql_acl.cc:
        Allocate metadata lock requests objects (MDL_LOCK) on execution
        memory root in cases when we use stack TABLE_LIST objects to open
        tables.  Got rid of redundant code by using unlock_locked_tables()
        function.
      sql/sql_base.cc:
        Changed code to use new MDL subsystem. Got rid of separate table
        cache.  Now used and unused TABLE instances are linked to the
        TABLE_SHAREs in table definition cache.
        
        check_unused():
          Adjusted code to the fact that we no longer have separate table
          cache.  Removed dead code.
        table_def_free():
          Free TABLE instances referenced from TABLE_SHARE objects before
          destroying table definition cache.
        get_table_share():
          Added assert which ensures that noone will be able to access
          table (and its share) without acquiring some kind of metadata
          lock first.
        close_handle_and_leave_table_as_lock():
          Adjusted code to the fact that TABLE instances now are linked to
          list in TABLE_SHARE.
        list_open_tables():
          Changed this function to use table definition cache instead of
          table cache.
        free_cache_entry():
          Unlink freed TABLE elements from the list of all TABLE instances
          for the table in TABLE_SHARE.
        kill_delayed_thread_for_table():
          Added auxiliary for killing delayed insert threads for
          particular table.
        close_cached_tables():
          Got rid of wait_for_refresh argument as we now rely on global
          shared metadata lock to prevent FLUSH WITH READ LOCK sneaking in
          when we are reopening tables. Heavily reworked this function to
          use new MDL code and not to rely on separate table cache entity.
        close_open_tables():
          We no longer have separate table cache.
        close_thread_tables():
          Release metadata locks after closing all tables. Added skip_mdl
          argument which allows us not to remove metadata lock requests
          from the context in case when we are going to use this requests
          later in mdl_wait_for_locks() and tdc_wait_for_old_versions().
        close_thread_table()/close_table_for_reopen():
          Since we no longer have separate table cache and all TABLE
          instances are linked to TABLE_SHARE objects in table definition
          cache we have to link/unlink TABLE object to/from appropriate
          lists in the share.
        name_lock_locked_table():
         Moved redundant code to find_write_locked_table() function and
          adjusted code to the fact that wait_while_table_is_used() can
          now return with an error if our thread is killed.
        reopen_table_entry():
          We no longer need "link_in" argument as with MDL we no longer
          call this function with dummy TABLE object pre-allocated and
          added to the THD::open_tables. Also now we add newly-open TABLE
          instance to the list of share's used TABLE instances.
        table_cache_insert_placeholder():
          Got rid of name-locking legacy.
        lock_table_name_if_not_cached():
          Moved to sql_table.cc the only place where it is used. It was
          also reimplemented using new MDL API.
        open_table():
          - Reworked this function to use new MDL subsystem.
          - Changed code to deal with table definition cache directly
            instead of going through separate table cache.
          - Now third argument is also used for requesting table repair
            or auto-discovery of table's new definition. So its type was
            changed from bool to enum.
        find_locked_table()/find_write_locked_table():
          Accept head of list of TABLE objects as first argument and use
          this list instead of always searching in THD::open_tables list.
          Also added auxiliary for finding write-locked locked tables.
        reopen_table():
          Adjusted function to work with new MDL subsystem and to properly
          manuipulate with lists of used/unused TABLE instaces in
          TABLE_SHARE.
        reopen_tables():
          Removed mark_share_as_old parameter. Instead of relying on it
          and related behavior FLUSH TABLES WITH READ LOCK now takes
          global shared metadata lock. Changed code after removing
          separate table cache.
        drop_locked_tables()/abort_locked_tables():
          Got rid of functions which are no longer needed.
          unlock_locked_tables():
          Moved this function from sql_parse.cc and changed it to release
          memory which was used for allocating metadata lock requests for
          tables open and locked by LOCK TABLES.
        tdc_open_view():
          Intoduced function for opening a view by getting its definition
          from disk (and table cache in future).
        reopen_table_entry():
          Introduced function for opening table definitions while holding
          exclusive metatadata lock on it.
        open_unireg_entry():
         Got rid of this function. Most of its functionality is relocated
          to open_table() and open_table_fini() functions, and some of it
          to reopen_table_entry() and tdc_open_view(). Also code
          resposible for auto-repair and auto-discovery of tables was
          moved to separate function.
        open_table_entry_fini():
          Introduced function which contains common actions which finalize
          process of TABLE object creation.
        auto_repair_table():
          Moved code responsible for auto-repair of table being opened
          here.
        handle_failed_open_table_attempt()
          Moved code responsible for handling failing attempt to open
          table to one place (retry due to lock conflict/old version,
          auto-discovery and repair).
        open_tables():
          - Flush open HANDLER tables if they have old version of if there
            is conflicting metadata lock against them (before this moment
            we had this code in open_table()).
          - When we open view which should be processed via derived table
            on the second execution of prepared statement or stored
            routine we still should call open_table() for it in order to
            obtain metadata lock on it and prepare its security context.
          - In cases when we discover that some special handling of
            failure to open table is needed call
            handle_failed_open_table_attempt() which handles all such
            scenarios.
        open_ltable():
          Handling of various special scenarios of failure to open a table
          was moved to separate handle_failed_open_table_attempt()
          function.
        remove_db_from_cache():
          Removed this function as it is no longer used.
        notify_thread_having_shared_lock():
          Added callback which is invoked by MDL subsystem when acquiring
          an exclusive lock, for each thread that has a conflicting shared
          metadata lock.
        expel_table_from_cache():
          Introduced function for removing unused TABLE instances. Unlike
          remove_table_from_cache() it relies on caller following MDL
          protocol and having appropriate locks when calling it and thus
          does not do any waiting if table is still in use.
        tdc_wait_for_old_version():
          Added function which allows open_tables() to wait in cases when
          we discover that we should back-off due to presence of old
          version of table.
        abort_and_upgrade_lock():
          Use new MDL calls.
        mysql_wait_completed_table():
          Got rid of unused function.
        open_system_tables_for_read/for_update()/performance_schema_table():
          Allocate MDL_LOCK objects on execution memory root in cases when
          TABLE_LIST objects for corresponding tables is allocated on
          stack.
        close_performance_schema_table():
          Release metadata locks after closing tables.
        ******
        Use I_P_List for free/used tables list in the table share.
      sql/sql_binlog.cc:
        Use Relay_log_info::slave_close_thread_tables() method to enforce
        that we always close tables open for RBR before deallocating
        TABLE_LIST elements and MDL_LOCK objects for them.
      sql/sql_class.cc:
        Added meta-data locking contexts as part of Open_tables_state
        context.  Also introduced THD::locked_tables_root memory root
        which is to be used for allocating MDL_LOCK objects for tables in
        LOCK TABLES statement (end of lifetime for such objects is UNLOCK
        TABLES so we can't use statement or execution root for them).
      sql/sql_class.h:
        Added meta-data locking contexts as part of Open_tables_state
        context.  Also introduced THD::locked_tables_root memory root
        which is to be used for allocating MDL_LOCK objects for tables in
        LOCK TABLES statement (end of lifetime for such objects is UNLOCK
        TABLES so we can't use statement or execution root for them).
        
        Note: handler_mdl_context and locked_tables_root and
        mdl_el_root will be removed by subsequent patches.
      sql/sql_db.cc:
        mysql_rm_db() does not really need to call remove_db_from_cache()
        as it drops each table in the database using
        mysql_rm_table_part2(), which performs all necessary operations on
        table (definition) cache.
      sql/sql_delete.cc:
        Use the new metadata locking API for TRUNCATE.
      sql/sql_handler.cc:
        Changed HANDLER implementation to use new metadata locking
        subsystem.  Note that MDL_LOCK objects for HANDLER tables are
        allocated in the same chunk of heap memory as TABLE_LIST object
        for those tables.
      sql/sql_insert.cc:
        mysql_insert():
          find_locked_table() now takes head of list of TABLE object as
          its argument instead of always scanning through THD::open_tables
          list.
        handle_delayed_insert():
          Allocate metadata lock request object for table open by delayed
          insert thread on execution memroot.  create_table_from_items():
          We no longer allocate dummy TABLE objects for tables being
          created if they don't exist. As consequence
          reopen_name_locked_table() no longer has link_in argument.
          open_table() now has one more argument which is not relevant for
          temporary tables.
      sql/sql_parse.cc:
        - Moved unlock_locked_tables() routine to sql_base.cc and made
          available it in other files. Got rid of some redundant code by
          using this function.
        - Replaced boolean TABLE_LIST::create member with enum
          open_table_type member.
        - Use special memory root for allocating MDL_LOCK objects for
          tables open and locked by LOCK TABLES (these object should live
          till UNLOCK TABLES so we can't allocate them on statement nor
          execution memory root). Also properly set metadata lock
          upgradability attribure for those tables.
        - Under LOCK TABLES it is no longer allowed to flush tables which
          are not write-locked as this breaks metadata locking protocol
          and thus potentially might lead to deadlock.
        - Added auxiliary adjust_mdl_locks_upgradability() function.
      sql/sql_partition.cc:
        Adjusted code to the fact that reopen_tables() no longer has
        "mark_share_as_old" argument. Got rid of comments which are no
        longer true.
      sql/sql_plist.h:
        Added I_P_List template class for parametrized intrusive doubly
        linked lists and I_P_List_iterator for corresponding iterator.
        Unlike for I_List<> list elements of such list can participate in
        several lists. Unlike List<> such lists are doubly-linked and
        intrusive.
      sql/sql_plugin.cc:
        Allocate metadata lock requests objects (MDL_LOCK) on execution
        memory root in cases when we use stack TABLE_LIST objects to open
        tables.
      sql/sql_prepare.cc:
        Replaced boolean TABLE_LIST::create member with enum
        open_table_type member.  This allows easily handle situation in
        which instead of opening the table we want only to take exclusive
        metadata lock on it.
      sql/sql_rename.cc:
        Use new metadata locking subsystem in implementation of RENAME
        TABLE.
      sql/sql_servers.cc:
        Allocate metadata lock requests objects (MDL_LOCK) on execution
        memory root in cases when we use stack TABLE_LIST objects to open
        tables. Got rid of redundant code by using unlock_locked_tables()
        function.
      sql/sql_show.cc:
        Acquire shared metadata lock when we are getting information for
        I_S table directly from TABLE_SHARE without doing full-blown table
        open.  We use high priority lock request in this situation in
        order to avoid deadlocks.
        Also allocate metadata lock requests objects (MDL_LOCK) on
        execution memory root in cases when TABLE_LIST objects are also
        allocated there
      sql/sql_table.cc:
        mysql_rm_table():
          Removed comment which is no longer relevant.
        mysql_rm_table_part2():
          Now caller of mysql_ha_rm_tables() should not own LOCK_open.
          Adjusted code to use new metadata locking subsystem instead of
          name-locks.
        lock_table_name_if_not_cached():
          Moved this function from sql_base.cc to this file and
          reimplemented it using metadata locking API.
        mysql_create_table():
          Adjusted code to use new MDL API.
        wait_while_table_is_used():
          Changed function to use new MDL subsystem. Made thread waiting
          in it killable (this also led to introduction of return value so
          caller can distinguish successful executions from situations
          when waiting was aborted).
        close_cached_tables():
          Thread waiting in this function is killable now. As result it
          has return value for distinguishing between succes and failure.
          Got rid of redundant boradcast_refresh() call.
        prepare_for_repair():
          Use MDL subsystem instead of name-locks.
        mysql_admin_table():
          mysql_ha_rm_tables() now always assumes that caller doesn't own
          LOCK_open.
        mysql_repair_table():
          We should mark all elements of table list as requiring
          upgradable metadata locks.
        mysql_create_table_like():
          Use new MDL subsystem instead of name-locks.
        create_temporary_tables():
          We don't need to obtain metadata locks when creating temporary
          table.
        mysql_fast_or_online_alter_table():
          Thread waiting in wait_while_table_is_used() is now killable.
        mysql_alter_table():
          Adjusted code to work with new MDL subsystem and to the fact
          that threads waiting in what_while_table_is_used() and
          close_cached_table() are now killable.
      sql/sql_test.cc:
        We no longer have separate table cache. TABLE instances are now
        associated with/linked to TABLE_SHARE objects in table definition
        cache.
      sql/sql_trigger.cc:
        Adjusted code to work with new metadata locking subsystem.  Also
        reopen_tables() no longer has mark_share_as_old argument (Instead
        of relying on this parameter and related behavior FLUSH TABLES
        WITH READ LOCK now takes global shared metadata lock).
      sql/sql_udf.cc:
        Allocate metadata lock requests objects (MDL_LOCK) on execution
        memory root in cases when we use stack TABLE_LIST objects to open
        tables.
      sql/sql_update.cc:
        Adjusted code to work with new meta-data locking subsystem.
      sql/sql_view.cc:
        Added proper meta-data locking to implementations of
        CREATE/ALTER/DROP VIEW statements. Now we obtain exclusive
        meta-data lock on a view before creating/ changing/dropping it.
        This ensures that all concurrent statements that use this view
        will finish before our statement will proceed and therefore we
        will get correct order of statements in the binary log.
        Also ensure that TABLE_LIST::mdl_upgradable attribute is properly
        propagated for underlying tables of view.
      sql/table.cc:
        Added auxiliary alloc_mdl_locks() function for allocating metadata
        lock request objects for all elements of table list.
      sql/table.h:
        TABLE_SHARE:
          Got rid of unused members. Introduced members for storing lists
          of used and unused TABLE objects for this share.
        TABLE:
          Added members for linking TABLE objects into per-share lists of
          used and unused TABLE instances. Added member for holding
          pointer to metadata lock for this table.
        TABLE_LIST:
          Replaced boolean TABLE_LIST::create member with enum
          open_table_type member.  This allows easily handle situation in
          which instead of opening the table we want only to take
          exclusive meta-data lock on it (we need this in order to handle
          ALTER VIEW and CREATE VIEW statements).
          Introduced new mdl_upgradable member for marking elements of
          table list for which we need to take upgradable shared metadata
          lock instead of plain shared metadata lock.  Added pointer for
          holding pointer to MDL_LOCK for the table.
        Added auxiliary alloc_mdl_locks() function for allocating metadata
        lock requests objects for all elements of table list.  Added
        auxiliary set_all_mdl_upgradable() function for marking all
        elements in table list as requiring upgradable metadata locks.
      storage/myisammrg/ha_myisammrg.cc:
        Allocate MDL_LOCK objects for underlying tables of MERGE table.
        To be reworked once Ingo pushes his patch for WL4144.
      eff3780d
  17. 24 Nov, 2009 1 commit
    • Konstantin Osipov's avatar
      Backport of: · 9a9e8d23
      Konstantin Osipov authored
      ----------------------------------------------------------------------
      ChangeSet@1.2571, 2008-04-08 12:30:06+02:00, vvaintroub@wva. +122 -0
        Bug#32082 : definition of VOID in my_global.h conflicts with Windows
        SDK headers
        
        VOID macro is now removed. Its usage is replaced with void cast.
        In some cases, where cast does not make much sense (pthread_*, printf, 
        hash_delete, my_seek), cast is ommited.
      
      
      client/mysqladmin.cc:
        Bug#32082 : remove VOID macro
      client/mysqldump.c:
        Bug#32082 : remove VOID macro
      client/mysqlimport.c:
        Bug#32082 : remove VOID macro
      client/mysqlslap.c:
        Bug#32082 : remove VOID macro
      client/mysqltest.cc:
        Bug#32082 : remove VOID macro
      client/sql_string.cc:
        Bug#32082 : remove VOID macro
      extra/comp_err.c:
        Bug#32082 : remove VOID macro
      extra/replace.c:
        Bug#32082 : remove VOID macro
      include/my_alarm.h:
        Bug#32082 : remove VOID macro
      include/my_global.h:
        Bug#32082 : remove VOID macro
      libmysql/libmysql.c:
        Bug#32082 : remove VOID macro
      mysys/errors.c:
        Bug#32082 : remove VOID macro
      mysys/hash.c:
        Bug#32082 : remove VOID macro
      mysys/mf_iocache2.c:
        Bug#32082 : remove VOID macro
      mysys/mf_loadpath.c:
        Bug#32082 : remove VOID macro
      mysys/mf_path.c:
        Bug#32082 : remove VOID macro
      mysys/my_append.c:
        Bug#32082 : remove VOID macro
      mysys/my_clock.c:
        Bug#32082 : remove VOID macro
      mysys/my_copy.c:
        Bug#32082 : remove VOID macro
      mysys/my_fstream.c:
        Bug#32082 : remove VOID macro
      mysys/my_getwd.c:
        Bug#32082 : remove VOID macro
      mysys/my_lib.c:
        Bug#32082 : remove VOID macro
      mysys/my_lockmem.c:
        Bug#32082 : remove VOID macro
      mysys/my_pthread.c:
        Bug#32082 : remove VOID macro
      mysys/my_redel.c:
        Bug#32082 : remove VOID macro
      mysys/stacktrace.c:
        Bug#32082 : remove VOID macro
      mysys/thr_alarm.c:
        Bug#32082 : remove VOID macro
      mysys/thr_lock.c:
        Bug#32082 : remove VOID macro
      sql/derror.cc:
        Bug#32082 : remove VOID macro
      sql/des_key_file.cc:
        Bug#32082 : remove VOID macro
      sql/discover.cc:
        Bug#32082 : remove VOID macro
      sql/field.cc:
        Bug#32082 : remove VOID macro
      sql/filesort.cc:
        Bug#32082 : remove VOID macro
      sql/ha_ndbcluster.cc:
        Bug#32082 : remove VOID macro
      sql/ha_partition.cc:
        Bug#32082 : remove VOID macro
      sql/handler.cc:
        Bug#32082 : remove VOID macro
      sql/hostname.cc:
        Bug#32082 : remove VOID macro
      sql/init.cc:
        Bug#32082 : remove VOID macro
      sql/item.cc:
        Bug#32082 : remove VOID macro
      sql/item_cmpfunc.cc:
        Bug#32082 : remove VOID macro
      sql/item_strfunc.cc:
        Bug#32082 : remove VOID macro
      sql/lock.cc:
        Bug#32082 : remove VOID macro
      sql/log.cc:
        Bug#32082 : remove VOID macro
      sql/log_event.cc:
        Bug#32082 : remove VOID macro
      sql/mysqld.cc:
        Bug#32082 : remove VOID macro
      sql/opt_range.h:
        Bug#32082 : remove VOID macro
      sql/protocol.cc:
        Bug#32082 : remove VOID macro
      sql/records.cc:
        Bug#32082 : remove VOID macro
      sql/sp_head.cc:
        Bug#32082 : remove VOID macro
      sql/sp_pcontext.cc:
        Bug#32082 : remove VOID macro
      sql/sql_acl.cc:
        Bug#32082 : remove VOID macro
      sql/sql_base.cc:
        Bug#32082 : remove VOID macro
      sql/sql_cache.cc:
        Bug#32082 : remove VOID macro
      sql/sql_connect.cc:
        Bug#32082 : remove VOID macro
      sql/sql_db.cc:
        Bug#32082 : remove VOID macro
      sql/sql_delete.cc:
        Bug#32082 : remove VOID macro
      sql/sql_handler.cc:
        Bug#32082 : remove VOID macro
      sql/sql_insert.cc:
        Bug#32082 : remove VOID macro
      sql/sql_map.cc:
        Bug#32082 : remove VOID macro
      sql/sql_parse.cc:
        Bug#32082 : remove VOID macro
      sql/sql_select.cc:
        Bug#32082 : remove VOID macro
      sql/sql_servers.cc:
        Bug#32082 : remove VOID macro
      sql/sql_show.cc:
        Bug#32082 : remove VOID macro
      sql/sql_string.cc:
        Bug#32082 : remove VOID macro
      sql/sql_table.cc:
        Bug#32082 : remove VOID macro
      sql/sql_test.cc:
        Bug#32082 : remove VOID macro
      sql/sql_trigger.cc:
        Bug#32082 : remove VOID macro
      sql/sql_update.cc:
        Bug#32082 : remove VOID macro
      sql/sql_view.cc:
        Bug#32082 : remove VOID macro
      sql/table.cc:
        Bug#32082 : remove VOID macro
      sql/tztime.cc:
        Bug#32082 : remove VOID macro
      sql/udf_example.c:
        Bug#32082 : remove VOID macro
      sql/uniques.cc:
        Bug#32082 : remove VOID macro
      sql/unireg.cc:
        Bug#32082 : remove VOID macro
      storage/archive/ha_archive.cc:
        Bug#32082 : remove VOID macro
      storage/blackhole/ha_blackhole.cc:
        Bug#32082 : remove VOID macro
      storage/csv/ha_tina.cc:
        Bug#32082 : remove VOID macro
      storage/csv/transparent_file.cc:
        Bug#32082 : remove VOID macro
      storage/example/ha_example.cc:
        Bug#32082 : remove VOID macro
      storage/federated/ha_federated.cc:
        Bug#32082 : remove VOID macro
      storage/heap/hp_clear.c:
        Bug#32082 : remove VOID macro
      storage/heap/hp_create.c:
        Bug#32082 : remove VOID macro
      storage/heap/hp_test1.c:
        Bug#32082 : remove VOID macro
      storage/heap/hp_test2.c:
        Bug#32082 : remove VOID macro
      storage/innobase/handler/ha_innodb.cc:
        Bug#32082 : remove VOID macro
      storage/myisam/ft_eval.c:
        Bug#32082 : remove VOID macro
      storage/myisam/ha_myisam.cc:
        Bug#32082 : remove VOID macro
      storage/myisam/mi_changed.c:
        Bug#32082 : remove VOID macro
      storage/myisam/mi_check.c:
        Bug#32082 : remove VOID macro
      storage/myisam/mi_close.c:
        Bug#32082 : remove VOID macro
      storage/myisam/mi_create.c:
        Bug#32082 : remove VOID macro
      storage/myisam/mi_dbug.c:
        Bug#32082 : remove VOID macro
      storage/myisam/mi_delete.c:
        Bug#32082 : remove VOID macro
      storage/myisam/mi_delete_all.c:
        Bug#32082 : remove VOID macro
      storage/myisam/mi_dynrec.c:
        Bug#32082 : remove VOID macro
      storage/myisam/mi_info.c:
        Bug#32082 : remove VOID macro
      storage/myisam/mi_locking.c:
        Bug#32082 : remove VOID macro
      storage/myisam/mi_log.c:
        Bug#32082 : remove VOID macro
      storage/myisam/mi_open.c:
        Bug#32082 : remove VOID macro
      storage/myisam/mi_packrec.c:
        Bug#32082 : remove VOID macro
      storage/myisam/mi_panic.c:
        Bug#32082 : remove VOID macro
      storage/myisam/mi_rsame.c:
        Bug#32082 : remove VOID macro
      storage/myisam/mi_statrec.c:
        Bug#32082 : remove VOID macro
      storage/myisam/mi_test1.c:
        Bug#32082 : remove VOID macro
      storage/myisam/mi_test2.c:
        Bug#32082 : remove VOID macro
      storage/myisam/mi_test3.c:
        Bug#32082 : remove VOID macro
      storage/myisam/mi_update.c:
        Bug#32082 : remove VOID macro
      storage/myisam/mi_write.c:
        Bug#32082 : remove VOID macro
      storage/myisam/myisamchk.c:
        Bug#32082 : remove VOID macro
      storage/myisam/myisamlog.c:
        Bug#32082 : remove VOID macro
      storage/myisam/myisampack.c:
        Bug#32082 : remove VOID macro
      storage/myisam/sort.c:
        Bug#32082 : remove VOID macro
      storage/myisammrg/myrg_close.c:
        Bug#32082 : remove VOID macro
      storage/myisammrg/myrg_create.c:
        Bug#32082 : remove VOID macro
      storage/myisammrg/myrg_open.c:
        Bug#32082 : remove VOID macro
      strings/str_test.c:
        Bug#32082 : remove VOID macro
      tests/thread_test.c:
        Bug#32082 : remove VOID macro
      9a9e8d23
  18. 20 Nov, 2009 2 commits
    • Konstantin Osipov's avatar
      Backport of: · 1b34d593
      Konstantin Osipov authored
      ------------------------------------------------------------
      revno: 2476.784.3
      committer: davi@moksha.local
      timestamp: Tue 2007-10-02 21:27:31 -0300
      message:
      Bug#25858 Some DROP TABLE under LOCK TABLES can cause deadlocks
              
      When a client (connection) holds a lock on a table and attempts to
      drop (obtain a exclusive lock) on a second table that is already
      held by a second client and the second client then attempts to
      drop the table that is held by the first client, leads to a
      circular wait deadlock. This scenario is very similar to trying to
      drop (or rename) a table while holding read locks and are
      correctly forbidden.
              
      The solution is to allow a drop table operation to continue only
      if the table being dropped is write (exclusively) locked, or if
      the table is temporary, or if the client is not holding any
      locks. Using this scheme prevents the creation of a circular
      chain in which each client is waiting for one table that the
      next client in the chain is holding.
                  
      This is incompatible change, as can be seen by number of tests
      cases that needed to be fixed, but is consistent with respect to
      behavior of the different scenarios in which the circular wait
      might happen.
      
      mysql-test/r/drop.result:
        Test case result for Bug#25858
      mysql-test/r/group_by.result:
        Fix test case result wrt drop table under lock tables -- unlock tables
        before dropping table.
      mysql-test/r/insert_notembedded.result:
        Fix test case result wrt drop table under lock tables -- unlock tables
        before dropping table
      mysql-test/r/lock.result:
        Fix test case result wrt drop table under lock tables -- unlock tables
        before dropping table.
      mysql-test/r/lock_multi.result:
        Fix test case result wrt drop table under lock tables -- unlock tables
        before dropping table.
      mysql-test/r/myisam.result:
        Fix test case result wrt drop table under lock tables -- unlock tables
        before dropping table.
      mysql-test/r/view.result:
        Fix test case result wrt drop table under lock tables -- unlock tables
        before dropping table.
      mysql-test/t/drop.test:
        Add test case for Bug#25858
      mysql-test/t/group_by.test:
        Fix test case: unlock tables in preparation for a drop table. In some
        circumstances, dropping tables while holding locks leads to a deadlock.
      mysql-test/t/insert_notembedded.test:
        Fix test case: unlock tables in preparation for a drop table. In some
        circumstances, dropping tables while holding locks leads to a deadlock.
      mysql-test/t/lock.test:
        Fix test case: unlock tables in preparation for a drop table. In some
        circumstances, dropping tables while holding locks leads to a deadlock.
      mysql-test/t/lock_multi.test:
        Fix test case: unlock tables in preparation for a drop table. In some
        circumstances, dropping tables while holding locks leads to a deadlock.
      mysql-test/t/myisam.test:
        Fix test case: unlock tables in preparation for a drop table. In some
        circumstances, dropping tables while holding locks leads to a deadlock.
      mysql-test/t/query_cache_notembedded.test:
        Fix test case: unlock tables in preparation for a drop table. In some
        circumstances, dropping tables while holding locks leads to a deadlock.
      mysql-test/t/view.test:
        Fix test case: unlock tables in preparation for a drop table. In some
        circumstances, dropping tables while holding locks leads to a deadlock.
      sql/lock.cc:
        When trying to obtain a name lock under lock tables, ensure that the table is properly exclusively locked and fail otherwise.
      1b34d593
    • Konstantin Osipov's avatar
      Backport of: · fcf6c154
      Konstantin Osipov authored
      revno: 2476.784.2
      committer: davi@moksha.local
      timestamp: Thu 2007-09-27 16:56:27 -0300 
      message:
      Bug#28870 check that table locks are released/reset
          
      The problem is that some mysql_lock_tables error paths are not
      resetting the tables lock type back to TL_UNLOCK. If the lock
      types are not reset properly, a table might be returned to the
      table cache with wrong lock_type.
            
      The proposed fix is to ensure that the tables lock type is always
      properly reset when mysql_lock_tables fails. This is a
      incompatible change with respect to the process state information.
      
      
      sql/lock.cc:
        Merge mysql_lock_tables cleanup sequence and the reset_lock_data
        function into a single function and take steps to ensure it is
        always called for each error exit path. Also remove references
        to the redundant THD::locked variable which was almost exclusively
        used by this function and the same information is already on proc_info.
      sql/sql_class.cc:
        Remove references to the THD::locked variable.
      sql/sql_class.h:
        Remove the THD::locked variable.
      sql/sql_show.cc:
        Remove references to THD:locked, state_info will now default to proc_info.
      fcf6c154
  19. 14 Oct, 2009 1 commit
    • Konstantin Osipov's avatar
      Backport of: · 9b41c753
      Konstantin Osipov authored
      ----------------------------------------------------------
      revno: 2617.22.5
      committer: Konstantin Osipov <kostja@sun.com>
      branch nick: mysql-6.0-runtime
      timestamp: Tue 2009-01-27 05:08:48 +0300
      message:
        Remove non-prefixed use of HASH.
        Always use my_hash_init(), my_hash_inited(), my_hash_search(),
        my_hash_element(), my_hash_delete(), my_hash_free() rather
        than non-prefixed counterparts (hash_init(), etc).
        Remove the backward-compatible defines.
      9b41c753
  20. 28 Aug, 2009 1 commit
    • Staale Smedseng's avatar
      Bug #43414 Parenthesis (and other) warnings compiling MySQL · 1ba25ae4
      Staale Smedseng authored
      with gcc 4.3.2
            
      This patch fixes a number of GCC warnings about variables used
      before initialized. A new macro UNINIT_VAR() is introduced for
      use in the variable declaration, and LINT_INIT() usage will be
      gradually deprecated. (A workaround is used for g++, pending a
      patch for a g++ bug.)
            
      GCC warnings for unused results (attribute warn_unused_result)
      for a number of system calls (present at least in later
      Ubuntus, where the usual void cast trick doesn't work) are
      also fixed.
      
      
      client/mysqlmanager-pwgen.c:
        A fix for warn_unused_result, adding fallback to use of
        srand()/rand() if /dev/random cannot be used. Also actually
        adds calls to rand() in the second branch so that it actually
        creates a random password.
      1ba25ae4
  21. 10 Feb, 2009 1 commit
  22. 10 Nov, 2008 1 commit
  23. 29 Sep, 2008 1 commit
    • Davi Arnaut's avatar
      Bug#34306: Can't make copy of log tables when server binary log is enabled · 0406d409
      Davi Arnaut authored
      The problem is that when statement-based replication was enabled,
      statements such as INSERT INTO .. SELECT FROM .. and CREATE TABLE
      .. SELECT FROM need to grab a read lock on the source table that
      does not permit concurrent inserts, which would in turn be denied
      if the source table is a log table because log tables can't be
      locked exclusively.
      
      The solution is to not take such a lock when the source table is
      a log table as it is unsafe to replicate log tables under statement
      based replication. Furthermore, the read lock that does not permits
      concurrent inserts is now only taken if statement-based replication
      is enabled and if the source table is not a log table.
      
      include/thr_lock.h:
        Introduce yet another lock type that my get upgraded depending
        on the binary log format. This is not a optimal solution but
        can be easily improved later.
      mysql-test/r/log_tables.result:
        Add test case result for Bug#34306
      mysql-test/suite/binlog/r/binlog_stm_row.result:
        Add test case result for Bug#34306
      mysql-test/suite/binlog/t/binlog_stm_row.test:
        Add test case for Bug#34306
      mysql-test/t/log_tables.test:
        Add test case for Bug#34306
      sql/lock.cc:
        Assert that TL_READ_DEFAULT is not a real lock type.
      sql/mysql_priv.h:
        Export new function.
      sql/mysqld.cc:
        Remove using_update_log.
      sql/sql_base.cc:
        Introduce function that returns the appropriate read lock type
        depending on how the statement is going to be replicated. It will
        only take a TL_READ_NO_INSERT log if the binary is enabled and the
        binary log format is statement-based and the table is not a log table.
      sql/sql_parse.cc:
        Remove using_update_log.
      sql/sql_update.cc:
        Use new function to choose read lock type.
      sql/sql_yacc.yy:
        The lock type is now decided at open_tables time. This old behavior was
        actually misleading as the binary log format can be dynamically switched
        and this would not change for statements that have already been parsed
        when the binary log format is changed (ie: prepared statements).
      0406d409
  24. 08 Apr, 2008 1 commit
    • unknown's avatar
      Fix for bug #35732: read-only blocks SELECT statements in InnoDB · 9fd89afc
      unknown authored
      Problem: SELECTs prohibited for a transactional SE in autocommit mode
      if read_only is set.
      
      Fix: allow them.
      
      
      mysql-test/r/read_only_innodb.result:
        Fix for bug #35732: read-only blocks SELECT statements in InnoDB
          - test result.
      mysql-test/t/read_only_innodb.test:
        Fix for bug #35732: read-only blocks SELECT statements in InnoDB
          - test case.
      sql/handler.cc:
        Fix for bug #35732: read-only blocks SELECT statements in InnoDB
          - in autocommit mode thd->transaction.all list is empty thus 
            is_real_trans set to TRUE for any SELECTs, so using it in the
            "read_only" check is insufficient.
            ha_check_and_coalesce_trx_read_only() changed to return number
            of engines with read-write changes. This value is used in the
            "read-only" check and checks for GLOBAL READ LOCK.
      sql/lock.cc:
        Fix for bug #35732: read-only blocks SELECT statements in InnoDB
          - added assert(protect_against_global_read_lock) before decreasing,
            in order to catch (uint) 0 - 1 situation due to wrong 
            wait_if_global_read_lock()/start_waiting_global_read_lock() call
            sequence.
      9fd89afc
  25. 29 Mar, 2008 1 commit
  26. 11 Dec, 2007 1 commit
    • unknown's avatar
      Bug#30273 - merge tables: Can't lock file (errno: 155) · e223c320
      unknown authored
      The patch for Bug 26379 (Combination of FLUSH TABLE and
      REPAIR TABLE corrupts a MERGE table) fixed this bug too.
      However it revealed a new bug that crashed the server.
      
      Flushing a merge table at the moment when it is between open
      and attach of children crashed the server.
      
      The flushing thread wants to abort locks on the flushed table.
      It calls ha_myisammrg::lock_count() and ha_myisammrg::store_lock()
      on the TABLE object of the other thread.
      
      Changed ha_myisammrg::lock_count() and ha_myisammrg::store_lock()
      to accept non-attached children. ha_myisammrg::lock_count() returns
      the number of MyISAM tables in the MERGE table so that the memory
      allocation done by get_lock_data() is done correctly, even if the
      children become attached before ha_myisammrg::store_lock() is
      called. ha_myisammrg::store_lock() will not return any lock if the
      children are not attached.
      
      This is however a change in the handler interface. lock_count()
      can now return a higher number than store_lock() stores locks.
      This is more safe than the reverse implementation would be.
      get_lock_data() in the SQL layer is adjusted accordingly. It sets
      MYSQL_LOCK::lock_count based on the number of locks returned by
      the handler::store_lock() calls, not based on the numbers returned
      by the handler::lock_count() calls. The latter are only used for
      allocation of memory now.
      
      No test case. The test suite cannot reliably run FLUSH between
      lock_count() and store_lock() of another thread. The bug report
      contains a program that can repeat the problem with some
      probability.
      
      
      include/myisammrg.h:
        Bug#30273 - merge tables: Can't lock file (errno: 155)
        Added mutex to struct st_myrg_info (MYRG_INFO).
      sql/handler.h:
        Bug#30273 - merge tables: Can't lock file (errno: 155)
        Extended comments for handler::lock_count() and
        handler::store_lock().
      sql/lock.cc:
        Bug#30273 - merge tables: Can't lock file (errno: 155)
        Changed get_lock_data() so that the final lock_count is taken
        from the number of locks returned from handler::store_lock()
        instead of from handler::lock_count().
      sql/sql_base.cc:
        Fixed a purecov comment. (unrelated to the rest of the changeset)
      storage/myisammrg/ha_myisammrg.cc:
        Bug#30273 - merge tables: Can't lock file (errno: 155)
        Changed ha_myisammrg::lock_count() and ha_myisammrg::store_lock()
        to accept non-attached children.
        Protected ha_myisammrg::store_lock() by MYRG_INFO::mutex.
      storage/myisammrg/myrg_close.c:
        Bug#30273 - merge tables: Can't lock file (errno: 155)
        Added MYRG_INFO::mutex destruction to myrg_parent_close().
      storage/myisammrg/myrg_open.c:
        Bug#30273 - merge tables: Can't lock file (errno: 155)
        Added MYRG_INFO::mutex initialization to myrg_parent_open().
        Protected myrg_attach_children() and myrg_detach_children()
        by MYRG_INFO::mutex.
        Fixed a purecov comment. (unrelated to the rest of the changeset)
      e223c320
  27. 11 Oct, 2007 1 commit
  28. 27 Aug, 2007 1 commit
    • unknown's avatar
      Bug#25164 create table `a` as select * from `A` hangs · 369a5f1c
      unknown authored
      The problem from a user's perspective: user creates table A, and then tries
      to CREATE TABLE a SELECT from A - and this causes a deadlock error, a hang,
      or fails with a debug assert, but only if the storage engine is InnoDB.
      
      The origin of the problem: InnoDB uses case-insensitive collation
      (system_charset_info) when looking up the internal table share, thus returning
      the same share for 'a' and 'A'.
      
      Cause of the user-visible behavior: since the same share is returned to SQL
      locking subsystem, it assumes that the same table is first locked (within the
      same session) for WRITE, and then for READ, and returns a deadlock error.
      However, the code is wrong in not properly cleaning up upon an error, leaving
      external locks in place, which leads to assertion failures and hangs.
      
      Fix that has been implemented: the SQL layer should properly propagate the
      deadlock error, cleaning up and freeing all resources.
      
      Further work towards a more complete solution: InnoDB should not use case
      insensitive collation for table share hash if table names on disk honor the case.
      
      
      mysql-test/r/innodb-deadlock.result:
        Bug#25164 test case result
      mysql-test/t/innodb-deadlock.test:
        Bug#25164 test case. The CREATE TABLE may fail depending on the character set
        of the system and filesystem, but it should never hang.
      sql/lock.cc:
        Unlock the storage engine "external" table level locks, if the MySQL thr_lock
        locking subsystem detects a deadlock error.
      369a5f1c