An error occurred fetching the project authors.
  1. 08 Feb, 2008 1 commit
    • unknown's avatar
      Fix for bug in log handler observed when running maria-big · d783c011
      unknown authored
      and then maria-connect under Windows 32-bit Visual Studio 2003.
      Build of all Maria unit tests under Windows.
      
      
      storage/maria/ma_loghandler.c:
        Assertion contributed by Sanja. Working around my_stat() bug on Windows
        (st_size is long i.e. 32-bit signed, the file was slightly bigger than
        2GB, so st_size was negative, TRANSLOG_PAGE_SIZE was int, so the
        comparison was done using long, so st_size was still negative in
        the comparison, which was incorrect). Two fixes: cast st_size to uint32
        (belt) and make TRANSLOG_PAGE_SIZE unsigned (suspenders).
        Of course 'long' is not enough for files >=4GB, but all log files of
        Maria are <4GB. The general problem of my_stat() has been passed on to
        our Windows team.
      storage/maria/ma_loghandler.h:
        use unsigned ints for constants which represent sizes of pages, files...
      storage/maria/unittest/CMakeLists.txt:
        a few unit tests were not built under Windows
      d783c011
  2. 20 Jan, 2008 1 commit
    • unknown's avatar
      - fix for segfault in rpl_trigger/rpl_found_rows with default engine=maria · 0825c485
      unknown authored
      (fix is keeping the real TRN through a disable_logging/reenable cycle)
      - fix for pagecache assertion failure in ps/type_ranges with default
      engine=maria (fix is in sql_insert.cc)
      - when reenabling logging we must either flush all dirty pages,
      or at least verify (in debug build) that there are none. For example
      a bulk insert with single UNDO_BULK_INSERT must flush them, no matter
      if it uses repair or not (bugfix)
      - UNDO_BULK_INSERT_WITH_REPAIR is also used with repair, changes name
      
      
      mysql-test/r/maria.result:
        tests for bugs fixed
      mysql-test/t/maria.test:
        tests for bugs fixed
      sql/sql_insert.cc:
        Bugfix: even if select_create::prepare() failed to create the 'table' object
        we still have to re-enable logging.
      storage/maria/ha_maria.cc:
        Bugfix: when a transactional table does a bulk insert without
        repair, it still sometimes skips logging of REDOs thus needs a full
        flush and sync at the end. Not if repair is done, as repair does
        it internally already (see end of maria_repair*()).
      storage/maria/ha_maria.h:
        variable now can have 3 states not 2
      storage/maria/ma_bitmap.c:
        name change
      storage/maria/ma_blockrec.c:
        name change
      storage/maria/ma_blockrec.h:
        name change
      storage/maria/ma_check.c:
        * When maria_repair() re-enables logging it does not need to ask for
        a flush&sync as it did it by itself already a few lines before.
        * the log record of bulk insert can be used even without repair
        * disable logging in maria_zerofill(): without that, it puts LSN pages
        in the cache, so when it flushes them it flushes the log; the change
        makes auto-ha_maria::zerofill-if-moved faster (no log flush).
      storage/maria/ma_key_recover.c:
        name change
      storage/maria/ma_loghandler.c:
        name change
      storage/maria/ma_loghandler.h:
        name change
      storage/maria/ma_pagecache.c:
        A function, to check in debug builds that no dirty pages exist for a file.
      storage/maria/ma_pagecache.h:
        new function (nothing in non-debug)
      storage/maria/ma_recovery.c:
        _ma_tmp_disable_logging() sets info->trn to dummy_transaction_object
        when needed now. The changes done here about info->trn are to allow
        a table to retain its original, real TRN through a disable/reenable
        cycle (see replication scenario in _ma_reenable_logging_for_table()).
        When we reenable, we offer the caller to flush and sync the table;
        if the caller doesn't accept our offer, we verify that it's ok
        (no REDOs => no dirty pages are allowed to exist).
      storage/maria/maria_chk.c:
        comment
      storage/maria/maria_def.h:
        new names
      mysql-test/suite/rpl/r/rpl_stm_maria.result:
        result (it used to crash)
      mysql-test/suite/rpl/t/rpl_stm_maria.test:
        Test of replication-specific Maria bug fixed
      0825c485
  3. 17 Jan, 2008 1 commit
    • unknown's avatar
      WL#3072 - Maria Recovery · 534417c2
      unknown authored
      Bulk insert: don't log REDO/UNDO for rows, log one UNDO which will
      truncate files; this is an optimization and a bugfix (table was left
      half-repaired by crash).
      Repair: mark table crashed-on-repair at start, bump skip_redo_lsn at start,
      this is easier for recovery (tells it to skip old REDOs or even UNDO
      phase) and user (tells it to repair) in case of crash, sync files
      in the end.
      Recovery skips missing or corrupted table and moves to next record
      (in REDO or UNDO phase) to be more robust; warns if happens in UNDO phase.
      Bugfix for UNDO_KEY_DELETE_WITH_ROOT (tested in ma_test_recovery)
      and maria_enable_indexes().
      Create missing bitmaps when needed (there can be more than one to create,
      in rare cases), log a record for this.
      
      
      include/myisamchk.h:
        new flag: bulk insert repair mustn't bump create_rename_lsn
      mysql-test/lib/mtr_report.pl:
        skip normal warning in maria-recovery.test
      mysql-test/r/maria-recovery.result:
        result: crash before bulk insert is committed, causes proper rollback,
        and crash right after OPTIMIZE replaces index file with new index file
        leads to table marked corrupted and recovery not failing.
      mysql-test/t/maria-recovery.test:
        - can't check the table or it would commit the transaction,
        but check is made after recovery.
        - test of crash before bulk-insert-with-repair is committed
        (to see if it is rolled back), and of crash after OPTIMIZE has replaced
        index file but not finished all operations (to see if recovery fails -
        it used to assert when trying to execute an old REDO on the new
        index).
      storage/maria/CMakeLists.txt:
        new file
      storage/maria/Makefile.am:
        new file
      storage/maria/ha_maria.cc:
        - If bulk insert on a transactional table using an index repair:
        table is initially empty, so don't log REDO/UNDO for data rows
        (optimization), just log an UNDO_BULK_INSERT_WITH_REPAIR
        which will, if executed, empty the data and index file. Re-enable
        logging in end_bulk_insert().
        - write log record for repair operation only after it's fully done,
        index sort including (maria_repair*() used to write the log record).
        - Adding back file->trn=NULL which was removed by mistake earlier.
      storage/maria/ha_maria.h:
        new member (see ha_maria.cc)
      storage/maria/ma_bitmap.c:
        Functions to create missing bitmaps:
        - one function which creates missing bitmaps in page cache, except
        the missing one with max offset which it does not put into page cache
        as it will be modified very soon.
        - one function which the one above calls, and creates bitmaps in page
        cache
        - one function to execute REDO_BITMAP_NEW_PAGE which uses the second
        one above.
      storage/maria/ma_blockrec.c:
        - when logging REDO_DELETE_ALL, not only 'records' and 'checksum'
        has to be reset under log's mutex.
        - execution of REDO_INSERT_ROW_BLOBS now checks the dirty pages' list
        - execution of UNDO_BULK_INSERT_WITH_REPAIR
      storage/maria/ma_blockrec.h:
        new functions
      storage/maria/ma_check.c:
        - table-flush-before-repair is moved to a separate function reused
        by maria_sort_index(); syncing is added
        - maria_repair() is allowed to re-enable logging only if it is the one
        which disabled it.
        - "_ma_flush_table_files_after_repair" was a bad name, it's not after
        repair now, and it should not sync as we do more changes to the files
        shortly after (sync is postponed to when writing the log record)
        - REDO_REPAIR record should be written only after all repair
        operations (in particular after sorting index in ha_mara::repair())
        - close to the end of repair by sort, flushing of pages must happen
        also in the non-quick case, to prepare for the sync at end.
        - in parallel repair, some page flushes are not needed as done
        by initialize_variables_for_repair().
      storage/maria/ma_create.c:
        Update skip_redo_lsn, create_rename_lsn optionally.
      storage/maria/ma_delete_all.c:
        Need to sync files at end of maria_delete_all_rows(), if transactional.
      storage/maria/ma_extra.c:
        During repair, we sometimes call _ma_flush_table_files() (via
        _ma_flush_table_files_before_swap()) while there is a WRITE_CACHE.
      storage/maria/ma_key_recover.c:
        - when we see CLR_END for UNDO_BULK_INSERT_WITH_REPAIR, re-enable
        indices.
        - fixing bug: _ma_apply_undo_key_delete() parsed UNDO_KEY_DELETE_WITH_ROOT
        wrongly, leading to recovery failure
      storage/maria/ma_key_recover.h:
        new prototype
      storage/maria/ma_locking.c:
        DBUG_VOID_RETURN missing
      storage/maria/ma_loghandler.c:
        UNDO for bulk insert with repair, and REDO for creating bitmaps.
        LOGREC_FIRST_FREE to not have to change the for() every time we
        add a new record type.
      storage/maria/ma_loghandler.h:
        new UNDO and REDO
      storage/maria/ma_open.c:
        Move share.kfile.file=kfile up a bit, so that _ma_update_state_lsns()
        can get its value, this fixes a bug where LSN_REPAIRED_BY_MARIA_CHK
        was not corrected on disk by maria_open().
        Store skip_redo_lsn in index' header.
        maria_enable_indexes() had a bug for BLOCK_RECORD, where an empty
        file has one page, not 0 bytes.
      storage/maria/ma_recovery.c:
        - Skip a corrupted, missing, or repaired-with-maria_chk, table in
        recovery: don't fail, just go to next REDO or UNDO; but if an UNDO
        is skipped in UNDO phase we issue warnings.
        - Skip REDO|UNDO in REDO phase if <skip_redo_lsn.
        - If UNDO phase fails, delete transactions to not make trnman
        assert.
        - Update skip_redo_lsn when playing REDO_CREATE_TABLE
        - Don't record UNDOs for old transactions which we don't know (long_trid==0)
        - Bugfix for UNDO_KEY_DELETE_WITH_ROOT (see ma_key_recover.c)
        - Execution of UNDO_BULK_INSERT_WITH_REPAIR
        - Don't try to find a page number in REDO_DELETE_ALL
        - Pieces moved to ma_recovery_util.c
      storage/maria/ma_rename.c:
        name change
      storage/maria/ma_static.c:
        I modified layout of the index' header (inserted skip_redo_lsn in its middle)
      storage/maria/ma_test2.c:
        allow breaking the test towards the end, tests execution of
        UNDO_KEY_DELETE_WITH_ROOT
      storage/maria/ma_test_recovery.expected:
        6 as testflag instead of 4
      storage/maria/ma_test_recovery:
        Increase the amount of rollback work to do when testing recovery
        with ma_test2; this reproduces the UNDO_KEY_DELETE_WITH_ROOT bug.
      storage/maria/maria_chk.c:
        skip_redo_lsn should be updated too, for consistency.
        Write a REDO_REPAIR after all operations (including sort-records)
        have been done.
        No reason to flush blocks after maria_chk_data_link() and
        maria_sort_records(), there is maria_close() in the end.
        write_log_record() is a function, to not clutter maria_chk().
      storage/maria/maria_def.h:
        New member skip_redo_lsn in the state, and comments
      storage/maria/maria_pack.c:
        skip_redo_lsn should be updated too, for consistency
      storage/maria/ma_recovery_util.c:
        _ma_redo_not_needed_for_page(), defined in ma_recovery.c, is needed
        by ma_blockrec.c; this causes link issues, resolved by putting
        _ma_redo_not_needed_for_page() into a new file (so that it is not
        in the same file as repair-related objects of ma_recovery.c).
      storage/maria/ma_recovery_util.h:
        new file
      534417c2
  4. 10 Jan, 2008 1 commit
    • unknown's avatar
      Fixed syncronization of buffer flush. · 9dd2d08c
      unknown authored
      Debug asserts/prints from Monty.
      No check of mutex operations (by Monty).
      
      
      storage/maria/ma_control_file.c:
        New external function to check mutex ownership.
      storage/maria/ma_loghandler.h:
        No check of mutex operations (by Monty).
        New external function to check mutex ownership.
      9dd2d08c
  5. 08 Jan, 2008 1 commit
    • unknown's avatar
      Use LOGREC_REDO_NEW_ROW_HEAD and LOGREC_REDO_NEW_ROW_TAIL when writing to a new page · e6bdb11f
      unknown authored
      This makes REDO safer as we then know when it's safe to ignore HA_ERR_WRONG_CRC from page cache
      Removed not used logged data for LOGREC_UNDO_ROW_DELETE and LOGREC_UNDO_ROW_UPDATE
      
      
      storage/maria/ma_blockrec.c:
        Use LOGREC_REDO_NEW_ROW_HEAD and LOGREC_REDO_NEW_ROW_TAIL when writing to a new page
        This makes REDO safer as we then know when it's safe to ignore HA_ERR_WRONG_CRC from page cache
      storage/maria/ma_blockrec.h:
        Fixed prototype
      storage/maria/ma_key_recover.c:
        Removed not used logged data for LOGREC_UNDO_ROW_DELETE and LOGREC_UNDO_ROW_UPDATE
      storage/maria/ma_loghandler.c:
        Use LOGREC_REDO_NEW_ROW_HEAD and LOGREC_REDO_NEW_ROW_TAIL when writing to a new page
      storage/maria/ma_loghandler.h:
        Added LOGREC_REDO_NEW_ROW_HEAD and LOGREC_REDO_NEW_ROW_TAIL
      storage/maria/ma_recovery.c:
        Added support for LOGREC_REDO_NEW_ROW_HEAD and LOGREC_REDO_NEW_ROW_TAIL
      e6bdb11f
  6. 21 Dec, 2007 1 commit
  7. 15 Dec, 2007 3 commits
    • unknown's avatar
      Pagecache callbacks support added. · 7b19ba34
      unknown authored
      Page CRC check based on pagecache support added.
      Loghandler pagecache callbacks support added.
      Loghandler filecache rewritten.
      Support of deletting all logs added.
      
      
      storage/maria/Makefile.am:
        New file with functions for CRC calculation.
      storage/maria/ma_bitmap.c:
        Page CRC support.
      storage/maria/ma_blockrec.c:
        Removed code replaced by pagecache callbacks.
      storage/maria/ma_check.c:
        Page CRC support.
      storage/maria/ma_create.c:
        Page CRC support.
      storage/maria/ma_loghandler.c:
        Pagecache callbacks support.
        New file cache support.
        Removing log files support.
      storage/maria/ma_loghandler.h:
        CRC_LENGTH replaced with CRC_SIZE
      storage/maria/ma_open.c:
        Page CRC support.
      storage/maria/ma_page.c:
        Page CRC support.
      storage/maria/ma_pagecache.c:
        Pagecache callbacks support.
      storage/maria/ma_pagecache.h:
        Pagecache callbacks support.
      storage/maria/ma_panic.c:
        Page CRC support.
      storage/maria/maria_chk.c:
        Page CRC support.
      storage/maria/maria_def.h:
        Page CRC support.
      storage/maria/unittest/Makefile.am:
        New test of removing logs.
      storage/maria/unittest/ma_maria_log_cleanup.c:
        Memory leack fixed.
      storage/maria/unittest/ma_pagecache_consist.c:
        Pagecache callbacks support.
      storage/maria/unittest/ma_pagecache_single.c:
        Pagecache callbacks support.
      storage/maria/unittest/ma_test_loghandler-t.c:
        Fixed the test error processing.
      storage/maria/unittest/ma_test_loghandler_first_lsn-t.c:
        Fixed the test error processing.
      storage/maria/unittest/ma_test_loghandler_max_lsn-t.c:
        Fixed the test error processing.
      storage/maria/unittest/ma_test_loghandler_multigroup-t.c:
        Fixed the test error processing.
      storage/maria/unittest/ma_test_loghandler_multithread-t.c:
        Fixed the test error processing.
      storage/maria/unittest/ma_test_loghandler_noflush-t.c:
        Fixed the test error processing.
      storage/maria/unittest/ma_test_loghandler_pagecache-t.c:
        Pagecache callbacks support.
      storage/maria/unittest/ma_test_loghandler_purge-t.c:
        Fixed the test error processing.
      storage/maria/unittest/test_file.c:
        Removed unneeded sync.
      7b19ba34
    • unknown's avatar
      Postmerge fix. · fabdf6df
      unknown authored
      storage/maria/ma_loghandler.c:
        Postmerge fix.
        New state of the log added to proicess correctly shutdown.
      storage/maria/ma_loghandler.h:
        New state of the log added to proicess correctly shutdown.
      fabdf6df
    • unknown's avatar
      Transaction log behaviour in case of write · d225521c
      unknown authored
        error fixed (switching to the read only mode).
      Added read only mode of transactions log handler.
      
      
      storage/maria/ha_maria.cc:
        Transaction log initialization parameters change.
      storage/maria/ma_check.c:
        New status variable of transactional log.
      storage/maria/ma_create.c:
        New status variable of transactional log.
      storage/maria/ma_loghandler.c:
        - New status variable added.
        - Checking the status variable in the loghandler
        interface functions added
        - All fails of loghandler functions revised.
        - UNRECOVERABLE_ERROR() removed.
        - Switching to read only mode added where it nead.
        - Checking of log state added before writes log content
          and changing status variables like sent_to_file, n_buffers_only, flushed.
        - Readonly loghandler initialization added.
        - Fixed problem with example table transactional log initialization.
      storage/maria/ma_loghandler.h:
        Readonly loghandler initialization added.
        Fixed problem with example table transactional log initialization.
        New status variable added.
      storage/maria/ma_open.c:
        New status variable of transactional log.
      storage/maria/ma_test1.c:
        Transaction log initialization parameters change.
      storage/maria/ma_test2.c:
        Transaction log initialization parameters change.
      storage/maria/maria_read_log.c:
        Transaction loghandler initialization in read only mode in
        case of only dysplay parameter.
      storage/maria/unittest/Makefile.am:
        Test of readonly mode added.
      storage/maria/unittest/ma_test_loghandler-t.c:
        Transaction log initialization parameters change.
        Fixed problem with example table transactional log initialization.
        Fixed incorrect fprintf call parameters.
      storage/maria/unittest/ma_test_loghandler_first_lsn-t.c:
        Transaction log initialization parameters change.
        Fixed problem with example table transactional log initialization.
      storage/maria/unittest/ma_test_loghandler_max_lsn-t.c:
        Transaction log initialization parameters change.
        Fixed problem with example table transactional log initialization.
      storage/maria/unittest/ma_test_loghandler_multigroup-t.c:
        Useing this test also as read only loghandler test.
        Transaction log initialization parameters change.
        Fixed problem with example table transactional log initialization.
      storage/maria/unittest/ma_test_loghandler_multithread-t.c:
        Transaction log initialization parameters change.
        Fixed problem with example table transactional log initialization.
      storage/maria/unittest/ma_test_loghandler_noflush-t.c:
        Transaction log initialization parameters change.
        Fixed problem with example table transactional log initialization.
      storage/maria/unittest/ma_test_loghandler_pagecache-t.c:
        Transaction log initialization parameters change.
        Fixed problem with example table transactional log initialization.
      storage/maria/unittest/ma_test_loghandler_purge-t.c:
        Transaction log initialization parameters change.
        Fixed problem with example table transactional log initialization.
      d225521c
  8. 13 Dec, 2007 1 commit
    • unknown's avatar
      Postreview changes. · ebafe3e4
      unknown authored
      mysql-test/r/maria-big.result:
        Change to make maria-purge more stable: increasing file size
          to let all records fit in one log.
      mysql-test/r/maria-connect.result:
        Change to make maria-purge more stable: increasing file size
          to let all records fit in one log.
      mysql-test/r/maria-purge.result:
        Change to make maria-purge more stable.
      mysql-test/r/maria.result:
        Change to make maria-purge more stable: increasing file size
          to let all records fit in one log.
      mysql-test/r/ps_maria.result:
        Change to make maria-purge more stable: increasing file size
          to let all records fit in one log.
      mysql-test/t/maria-big.test:
        Change to make maria-purge more stable: increasing file size
          to let all records fit in one log.
      mysql-test/t/maria-connect.test:
        Change to make maria-purge more stable: increasing file size
          to let all records fit in one log.
      mysql-test/t/maria-purge.test:
        Change to make maria-purge more stable.
      mysql-test/t/maria-recovery.test:
        Change to make maria-purge more stable: increasing file size
          to let all records fit in one log.
      mysql-test/t/maria.test:
        Change to make maria-purge more stable: increasing file size
          to let all records fit in one log.
      mysql-test/t/ps_maria.test:
        Change to make maria-purge more stable: increasing file size
          to let all records fit in one log.
      ebafe3e4
  9. 11 Dec, 2007 1 commit
    • unknown's avatar
      Post review fix (serg). · fed1d21b
      unknown authored
      Added more checks.
      
      
      storage/maria/ha_maria.cc:
        Let control size to my_get_opts().
      storage/maria/ma_loghandler.c:
        Let control size to my_get_opts().
        Removed unused function.
        Assert to check log file size.
        Additional compilation warnings.
      storage/maria/ma_loghandler.h:
        New log size bounds.
        Removed unused function.
      storage/maria/unittest/ma_test_loghandler-t.c:
        Fixed test to confirm new log size bounds.
      fed1d21b
  10. 10 Dec, 2007 2 commits
    • unknown's avatar
      WL#3072 Maria recovery: · 13f45b16
      unknown authored
      fix for bug: if a crash happened right after writing a REDO like this:
      REDO - UNDO - REDO*, then recovery would ignore the last REDO* (ok),
      rollback: REDO - UNDO - REDO* - REDO - CLR, and a next recovery would
      thus execute REDO* instead of skipping it again. Recovery now logs
      LOGREC_INCOMPLETE_GROUP when it meets REDO* for the first time,
      to draw a boundary and ensure it is always skipped. Tested by hand.
      Note: ma_test_all fails "maria_chk: error: Key 1 - Found too many records"
      not due to this patch (failed before).
      
      
      BitKeeper/triggers/post-commit:
        no truncation of the commit mail, or how to review patches?
      mysql-test/include/maria_verify_recovery.inc:
        let caller choose the statement used to crash (sometimes we
        want the crash to happen at special places)
      mysql-test/t/maria-recovery.test:
        user of maria_verify_recovery.inc now specifies statement which the
        script should use for crashing.
      storage/maria/ma_bitmap.c:
        it's easier to search for all places using functions from the bitmap
        module (like in ma_blockrec.c) if those exported functions all start
        with "_ma_bitmap": renaming some of them.
        Assertion that when we read a bitmap page, overwriting bitmap->map,
        we are not losing information (i.e. bitmap->changed is false).
      storage/maria/ma_blockrec.c:
        update to new names. Adding code (disabled, protected by a #ifdef)
        that I use to test certain crash scenarios (more to come).
      storage/maria/ma_blockrec.h:
        update to new names
      storage/maria/ma_checkpoint.c:
        update to new names
      storage/maria/ma_extra.c:
        update to new names
      storage/maria/ma_loghandler.c:
        new LOGREC_INCOMPLETE_GROUP
      storage/maria/ma_loghandler.h:
        new LOGREC_INCOMPLETE_GROUP
      storage/maria/ma_recovery.c:
        When at the end of the REDO phase we have identified some transactions
        with incomplete REDO groups (REDOs without an UNDO or CLR_END),
        for each of them we log LOGREC_INCOMPLETE_GROUP. This way, the
        upcoming UNDO phase can write more records for such transaction,
        a future recovery won't pair the incomplete group with the
        CLR_END (as there is LOGREC_INCOMPLETE_GROUP to draw a boundary).
      13f45b16
    • unknown's avatar
      Added MARIA_SHARE *share to a lot of places to make code simpler · 2f6f08ed
      unknown authored
      Changed info->s -> share to get more efficent code
      Updated arguments to page accessor functions to use MARIA_SHARE * instead of MARIA_HA *.
      Tested running tests in quick mode (no balance page on insert and only when critical on delete)
      Fixed bug in underflow handling in quick mode
      Fixed bug in log handler where it accessed not initialized variable
      Fixed bug in log handler where it didn't free mutex in unlikely error condition
      Removed double write of page in case of of some underflow conditions
      Added DBUG_PRINT in safemutex lock/unlock
      
      
      dbug/dbug.c:
        Compile without SAFE_MUTEX (to be able to use DBUG_PRINT in safe_mutex code)
        Use calls to get/set my_thread_var->dbug. (Make dbug independent of compile time options for mysys)
      include/my_pthread.h:
        Added prototypes for my_thread_var_get_dbug() & my_thread_var_set_dbug()
      mysql-test/lib/mtr_report.pl:
        Don't check warnings in log files if we are using --extern
      mysys/my_thr_init.c:
        Added my_thread_var_get_dbug() & my_thread_var_set_dbug()
      mysys/thr_mutex.c:
        Added DBUG printing of addresses to mutex for lock/unlock
      storage/maria/ma_blockrec.c:
        Fixed comment
      storage/maria/ma_check.c:
        Added MARIA_SHARE *share to a lot of places to make code simpler
        info->s -> share
        Updated arguments to page accessor functions
      storage/maria/ma_close.c:
        Indentation fixes
      storage/maria/ma_create.c:
        Calculate min_key_length correctly
      storage/maria/ma_dbug.c:
        Indentation fixes
      storage/maria/ma_delete.c:
        Added MARIA_SHARE *share to a lot of places to make code simpler
        info->s -> share
        Updated arguments to page accessor functions
        Removed some writing of key pages that underflow (will be written by caller)
        Fixed crashing bug in underflow handling when using quick mode
      storage/maria/ma_delete_all.c:
        Indentation fixes
      storage/maria/ma_dynrec.c:
        Indentation fixes
      storage/maria/ma_extra.c:
        Fixed indentation
        Removed old useless code
        Reset share->changed if we have written state
      storage/maria/ma_ft_update.c:
        Added MARIA_SHARE *share to a lot of places to make code simpler
        info->s -> share
        Updated arguments to page accessor functions
      storage/maria/ma_info.c:
        Indentation fixes
      storage/maria/ma_key_recover.c:
        Added MARIA_SHARE *share to a lot of places to make code simpler
        info->s -> share
        Updated arguments to page accessor functions
      storage/maria/ma_locking.c:
        Indentation fixes
      storage/maria/ma_loghandler.c:
        Removed wrapper functions translog_mutex_lock and translog_mutex_unlock (safemutex now does same kind of printing)
        Renamed LOGREC_REDO_INSERT_ROW_BLOB to LOGREC_REDO_INSERT_NOT_USED to mark it free
        Fixed some DBUG_PRINT to ensure that convert-dbug-for-diff works
        Fixed bug in translog_flush() that caused log to stop syncing to disk
        Added missing mutex_unlock in case of error
      storage/maria/ma_loghandler.h:
        Renamed LOGREC_REDO_INSERT_ROW_BLOB to LOGREC_REDO_INSERT_NOT_USED to mark it free
      storage/maria/ma_open.c:
        Indentation fixes
      storage/maria/ma_packrec.c:
        Indentation fixes
      storage/maria/ma_page.c:
        Added MARIA_SHARE *share to a lot of places to make code simpler
        info->s -> share
        Updated arguments to page accessor functions
        Added check that we never write a key page without content (except in recovery where a key page may temporary be without content)
      storage/maria/ma_preload.c:
        Updated arguments to page accessor functions
      storage/maria/ma_range.c:
        Updated arguments to page accessor functions
      storage/maria/ma_rkey.c:
        Indentation fixes
      storage/maria/ma_rprev.c:
        Indentation fixes
      storage/maria/ma_rt_index.c:
        Added MARIA_SHARE *share to a lot of places to make code simpler
        info->s -> share
        Updated arguments to page accessor functions
      storage/maria/ma_rt_index.h:
        Updated arguments to page accessor functions
      storage/maria/ma_rt_key.c:
        Added MARIA_SHARE *share to a lot of places to make code simpler
        info->s -> share
        Updated arguments to page accessor functions
      storage/maria/ma_rt_mbr.c:
        Added MARIA_SHARE *share to a lot of places to make code simpler
        info->s -> share
        Updated arguments to page accessor functions
      storage/maria/ma_rt_split.c:
        Added MARIA_SHARE *share to a lot of places to make code simpler
        info->s -> share
        Updated arguments to page accessor functions
      storage/maria/ma_search.c:
        Added MARIA_SHARE *share to a lot of places to make code simpler
        info->s -> share
        Updated arguments to page accessor functions
      storage/maria/ma_sort.c:
        Indentation fixes
      storage/maria/ma_statrec.c:
        Indentation fixes
      storage/maria/ma_test1.c:
        Added extra undo test
        Flush also keys in -u1, to ensure that the full log is flushed
      storage/maria/ma_test2.c:
        Added extra undo test
        Flush also keys in -u1, to ensure that the full log is flushed
      storage/maria/ma_test_recovery.expected:
        Updated results
      storage/maria/ma_test_recovery:
        Added extra undo test
      storage/maria/ma_update.c:
        Indentation fixes
      storage/maria/ma_write.c:
        Added MARIA_SHARE *share to a lot of places to make code simpler
        info->s -> share
        Updated arguments to page accessor functions
        Prepare for quick mode for insert (don't balance page)
      storage/maria/maria_chk.c:
        Added MARIA_SHARE *share to a lot of places to make code simpler
        info->s -> share
        Updated arguments to page accessor functions
      storage/maria/maria_def.h:
        Updated arguments to page accessor functions
      2f6f08ed
  11. 09 Dec, 2007 1 commit
    • unknown's avatar
      Manageable transactional log purge and file size · 771296eb
      unknown authored
      support added to maria.
      
      
      mysql-test/r/maria.result:
        New variables added.
      storage/maria/ha_maria.cc:
        Variable for transactional log purge method added.
        Variable for transactional log size added.
        SHOW for engine logs added.
        Log flush purge logs in case of "ondemand" type of
          log processing.
      storage/maria/ma_checkpoint.c:
        log purge call enabled.
      storage/maria/ma_loghandler.c:
        Support for different methods of log purge added.
        Functions for getting information about logs state added.
        Functions for getting/setting log size.
      storage/maria/ma_loghandler.h:
        Fixed defines.
        Functions for for transactional log mannegment added.
      storage/maria/ma_recovery.c:
        Dependence on TRANSLOG_FILE_SIZE removed.
      mysql-test/r/maria-purge.result:
        New BitKeeper file ``mysql-test/r/maria-purge.result''
      mysql-test/t/maria-purge.test:
        New BitKeeper file ``mysql-test/t/maria-purge.test''
      771296eb
  12. 06 Dec, 2007 1 commit
    • unknown's avatar
      Trivial postreview changes. · 21f63e41
      unknown authored
      storage/maria/ma_blockrec.c:
        Layout fix.
      storage/maria/ma_loghandler.c:
        Renamed struct variable 'class' to avoid problems with c++ compiler.
        Renamed translog_init_scanner to fit othere init functions naming pattern.
        translog_relative_LSN_encode can't return error for now.
        Other trivial postreview changes.
      storage/maria/ma_loghandler_lsn.h:
        trivial postreview changes.
      storage/maria/ma_recovery.c:
        Renamed translog_init_scanner to fit othere init functions naming pattern.
      storage/maria/unittest/ma_loghandler_examples.c:
        Renamed struct variable 'class' to avoid problems with c++ compiler.
      storage/maria/unittest/ma_test_loghandler-t.c:
        Renamed translog_init_scanner to fit othere init functions naming pattern.
      storage/maria/unittest/ma_test_loghandler_multigroup-t.c:
        Renamed translog_init_scanner to fit othere init functions naming pattern.
      storage/maria/unittest/ma_test_loghandler_multithread-t.c:
        Renamed translog_init_scanner to fit othere init functions naming pattern.
      21f63e41
  13. 04 Dec, 2007 1 commit
    • unknown's avatar
      Added error HA_ERR_FILE_TOO_SHORT to be used when files are shorter than... · ebf7ab7b
      unknown authored
      Added error HA_ERR_FILE_TOO_SHORT to be used when files are shorter than expected (by my_read/my_pread)
      Added debugger hook _my_dbug_put_break_here() that is called if we get a CRC that matches --debug-crc-break (my_crc_dbug_break)
      Fixed REDO_REPAIR to use all repair modes (repair, repair_by_sort, repair_paralell
      REDO_REPAIR now also logs used key map
      Fixed some bugs in REDO logging of key pages
      Better error messages from maria_read_log
      Added my_readwrite_flags to init_pagecache() to be able to get better error messages and simplify code.
      Don't allow pagecaches with less than 8 blocks (Causes strange crashes)
      Added EXTRA_DEBUG_KEY_CHANGES. When this is defined some REDO_INDEX entries contains page checksums (these are calculated and checked in DBUG mode, ignored otherwise)
      Fixed bug in ma_pagecache unit tests that caused program to sometimes fail
      Added some missing calls to MY_INIT() that caused some unit tests to fail
      Fixed that TRUNCATE works properly on temporary MyISAM files
      Updates some result files to new table checksums results (checksum when NULL fields are ignored)
      perl test-insert can be replayed with maria_read_log!
      
      
      sql/share/Makefile.am:
        Change mode to -rw-rw-r--
      BitKeeper/etc/ignore:
        added storage/maria/unittest/page_cache_test_file_1 storage/maria/unittest/pagecache_debug.log
      include/maria.h:
        Added maria_tmpdir
      include/my_base.h:
        Added error HA_ERR_FILE_TOO_SHORT
      include/my_sys.h:
        Added variable my_crc_dbug_check
        Added function my_dbug_put_break_here()
      include/myisamchk.h:
        Added org_key_map (Needed for writing REDO record for REPAIR)
      mysql-test/r/innodb.result:
        Updated to new checksum algorithm (NULL ignored)
      mysql-test/r/mix2_myisam.result:
        Updated to new checksum algorithm (NULL ignored)
      mysql-test/r/myisam.result:
        Updated to new checksum algorithm (NULL ignored)
      mysql-test/t/myisam.test:
        Added used table
      mysys/checksum.c:
        Added DBUG for checksum results
        Added debugger hook so that _my_dbug_put_break_here() is called if we get matching CRC
      mysys/lf_alloc-pin.c:
        Fixed compiler warning
      mysys/my_handler.c:
        Added new error message
      mysys/my_init.c:
        If my_progname is not given, use 'unknown' form my_progname_short
        Added debugger function my_debug_put_break_here()
      mysys/my_pread.c:
        In case of too short file when MY_NABP or MY_FNABP is specified, give error HA_ERR_FILE_TO_SHORT
      mysys/my_read.c:
        In case of too short file when MY_NABP or MY_FNABP is specified, give error HA_ERR_FILE_TO_SHORT
      sql/mysqld.cc:
        Added debug option --debug-crc-break
      sql/sql_parse.cc:
        Trivial optimization
      storage/maria/ha_maria.cc:
        Renamed variable to be more logical
        Ensure that param.testflag is correct when calling repair
        Added extra argument to init_pagecache
        Set default value for maria_tempdir
      storage/maria/ma_blockrec.c:
        Test for HA_ERR_FILE_TOO_SHORT instead for -1
      storage/maria/ma_cache.c:
        Test for HA_ERR_FILE_TOO_SHORT instead for -1
      storage/maria/ma_check.c:
        Set param->testflag to match how repair is run (needed for REDO logging)
        Simple optimization
        Moved flag if page is node from pagelength to keypage-flag byte
        Log used key map in REDO log.
      storage/maria/ma_delete.c:
        Remember previous UNDO entry when writing undo (for future CLR records)
        Moved flag if page is node from pagelength to keypage-flag byte
        Fixed some bugs in redo logging
        Added CRC for some translog REDO_INDEX entries
      storage/maria/ma_dynrec.c:
        Test for HA_ERR_FILE_TOO_SHORT instead for -1
      storage/maria/ma_ft_update.c:
        Fixed call to _ma_store_page_used()
      storage/maria/ma_key_recover.c:
        Added CRC for some translog REDO_INDEX entries
        Removed not needed pagecache_write() in _ma_apply_redo_index()
      storage/maria/ma_locking.c:
        Test for HA_ERR_FILE_TOO_SHORT instead for -1
      storage/maria/ma_loghandler.c:
        Added used key map to REDO_REPAIR_TABLE
      storage/maria/ma_loghandler.h:
        Added operation for checksum of key pages
      storage/maria/ma_open.c:
        Allocate storage for undo lsn pointers
      storage/maria/ma_pagecache.c:
        Remove not needed include file
        Change logging to use fd: for file descritors as other code
        Added my_readwrite_flags to init_pagecache() to be able to get better error messages for maria_chk/maria_read_log
        Don't allow pagecaches with less than 8 blocks
        Remove wrong DBUG_ASSERT()
      storage/maria/ma_pagecache.h:
        Added readwrite_flags
      storage/maria/ma_recovery.c:
        Better error messages for maria_read_log:
        - Added eprint() for printing error messages
        - Print extra \n before error message if we are printing %0 %10 ...
        
        Added used key_map to REDO_REPAIR log entry
        More DBUG
        Call same repair method that was used by mysqld
      storage/maria/ma_rt_index.c:
        Moved flag if page is node from pagelength to keypage-flag byte
      storage/maria/ma_rt_key.c:
        Fixed call to _ma_store_page_used()
      storage/maria/ma_rt_split.c:
        Moved flag if page is node from pagelength to keypage-flag byte
      storage/maria/ma_static.c:
        Added maria_tmpdir
      storage/maria/ma_test1.c:
        Updated call to init_pagecache()
      storage/maria/ma_test2.c:
        Updated call to init_pagecache()
      storage/maria/ma_test3.c:
        Updated call to init_pagecache()
      storage/maria/ma_write.c:
        Removed #ifdef NOT_YET
        Moved flag if page is node from pagelength to keypage-flag byte
        Fixed bug in  _ma_log_del_prefix()
      storage/maria/maria_chk.c:
        Fixed wrong min limit for page_buffer_size
        Updated call to init_pagecache()
      storage/maria/maria_def.h:
        Added EXTRA_DEBUG_KEY_CHANGES. When this is defined some REDO_INDEX entries contains page checksums
        Moved flag if page is node from pagelength to keypage-flag byte
      storage/maria/maria_ftdump.c:
        Updated call to init_pagecache()
      storage/maria/maria_pack.c:
        Updated call to init_pagecache()
        Reset share->state.create_rename_lsn & share->state.is_of_horizon
      storage/maria/maria_read_log.c:
        Better error messages
        Added --tmpdir option (needed to set temporary directory for REDO_REPAIR)
        Added --start-from-lsn
        Changed option for --display-only to 'd' (wanted to use -o for 'offset')
      storage/maria/unittest/lockman2-t.c:
        Added missing call to MY_INIT()
      storage/maria/unittest/ma_pagecache_consist.c:
        Updated call to init_pagecache()
      storage/maria/unittest/ma_pagecache_single.c:
        Fixed bug that caused program to sometimes fail
        Added some DBUG_ASSERTS()
        Changed some calls to malloc()/free() to my_malloc()/my_free()
        Create extra file to expose original hard-to-find bug
      storage/maria/unittest/ma_test_loghandler-t.c:
        Updated call to init_pagecache()
      storage/maria/unittest/ma_test_loghandler_first_lsn-t.c:
        Updated call to init_pagecache()
      storage/maria/unittest/ma_test_loghandler_max_lsn-t.c:
        Updated call to init_pagecache()
      storage/maria/unittest/ma_test_loghandler_multigroup-t.c:
        Updated call to init_pagecache()
      storage/maria/unittest/ma_test_loghandler_multithread-t.c:
        Updated call to init_pagecache()
      storage/maria/unittest/ma_test_loghandler_noflush-t.c:
        Updated call to init_pagecache()
      storage/maria/unittest/ma_test_loghandler_pagecache-t.c:
        Updated call to init_pagecache()
      storage/maria/unittest/ma_test_loghandler_purge-t.c:
        Updated call to init_pagecache()
      storage/maria/unittest/test_file.c:
        Changed malloc()/free() to my_malloc()/my_free()
        Fixed memory leak
        Changd logic a bit while trying to find bug in reset_file()
      storage/maria/unittest/trnman-t.c:
        Added missing call to MY_INIT()
      storage/myisam/mi_cache.c:
        Test for HA_ERR_FILE_TOO_SHORT instead for -1
      storage/myisam/mi_create.c:
        Removed O_EXCL to get TRUNCATE to work for temporary files
      storage/myisam/mi_dynrec.c:
        Test for HA_ERR_FILE_TOO_SHORT instead for -1
      storage/myisam/mi_locking.c:
        Test for HA_ERR_FILE_TOO_SHORT instead for -1
      mysql-test/r/old-mode.result:
        New BitKeeper file ``mysql-test/r/old-mode.result''
      mysql-test/t/old-mode-master.opt:
        New BitKeeper file ``mysql-test/t/old-mode-master.opt''
      mysql-test/t/old-mode.test:
        New BitKeeper file ``mysql-test/t/old-mode.test''
      ebf7ab7b
  14. 03 Dec, 2007 1 commit
    • unknown's avatar
      Added variable to control log directory syncs. · 3e90f0d8
      unknown authored
      Part of postreview fixes added.
      
      
      mysql-test/r/maria.result:
        Added variable to control log directory syncs.
      storage/maria/ha_maria.cc:
        Added variable to control log directory syncs.
      storage/maria/ma_loghandler.h:
        Added variable to control log directory syncs.
      storage/maria/ma_loghandler_lsn.h:
        postreview fix
      storage/maria/unittest/Makefile.am:
        New file with logrecord descriptors for tests.
      storage/maria/unittest/ma_test_loghandler-t.c:
        New file with logrecord descriptors for tests.
      storage/maria/unittest/ma_test_loghandler_first_lsn-t.c:
        New file with logrecord descriptors for tests.
      storage/maria/unittest/ma_test_loghandler_max_lsn-t.c:
        New file with logrecord descriptors for tests.
      storage/maria/unittest/ma_test_loghandler_multigroup-t.c:
        New file with logrecord descriptors for tests.
      storage/maria/unittest/ma_test_loghandler_multithread-t.c:
        New file with logrecord descriptors for tests.
      storage/maria/unittest/ma_test_loghandler_noflush-t.c:
        New file with logrecord descriptors for tests.
      storage/maria/unittest/ma_test_loghandler_pagecache-t.c:
        New file with logrecord descriptors for tests.
      storage/maria/unittest/ma_test_loghandler_purge-t.c:
        New file with logrecord descriptors for tests.
      storage/maria/unittest/ma_loghandler_examples.c:
        New BitKeeper file ``storage/maria/unittest/ma_loghandler_examples.c''
      3e90f0d8
  15. 28 Nov, 2007 1 commit
    • unknown's avatar
      Fixed repair_by_sort to work with BLOCK_RECORD · 4e0964cb
      unknown authored
      Fixed bugs in undo logging
      Fixed bug where head block was split before min_row_length (caused Maria to believe row was crashed on read)
      Reserved place for reference-transid on key pages (for packing of transids)
      ALTER TABLE and INSERT ... SELECT now uses fast creation of index
          
      Known bugs:
      ma_test_recovery fails because of a bug in redo handling when log is cut directly after a redo (Guilhem knows how to fix)
      ma_test_recovery.excepted is not totally correct, because of the above bug
      mysqld sometimes fails to restart; Fails with error "end_of_redo_phase: Assertion `long_trid != 0' failed"; Guilhem to investigate
      
      
      include/maria.h:
        Prototype changes
        Added current_filepos to st_maria_sort_info
      mysql-test/r/maria.result:
        Updated results that changes as alter table and insert ... select now uses fast creation of index
      mysys/mf_iocache.c:
        Reset variable to gurard against double invocation
      storage/maria/ma_bitmap.c:
        Added _ma_bitmap_reset_cache() (needed for repair)
      storage/maria/ma_blockrec.c:
        Simplify code
        More initial allocations
        Fixed bug where head block was split before min_row_length (caused Maria to believe row was crashed on read)
      storage/maria/ma_blockrec.h:
        Moved TRANSID_SIZE to maria_def.h
        Added prototype for new functions
      storage/maria/ma_check.c:
        Simplicy code
        Fixed repair_by_sort to work with BLOCK_RECORD
        - When using BLOCK_RECORD or UNPACK create new Maria handle
        - Use common initializer function
        - Align code with maria_repair()
        
        Made some changes to maria_repair_parallel() to use common initializer function
        Removed ASK_MONTY section by fixing noted problem
      storage/maria/ma_close.c:
        Moved check for readonly to _ma_state_info_write()
      storage/maria/ma_key_recover.c:
        Use different log entries if key root changes or not.
        This fixed some bugs when tree grows
      storage/maria/ma_key_recover.h:
        Added keynr to st_msg_to_write_hook_for_undo_key
      storage/maria/ma_loghandler.c:
        Added INIT_LOGREC_UNDO_KEY_INSERT_WITH_ROOT
      storage/maria/ma_loghandler.h:
        Added INIT_LOGREC_UNDO_KEY_INSERT_WITH_ROOT
      storage/maria/ma_open.c:
        Added TRANSID to all key pages (for future compressing of trans id's)
        For compressed records, alloc a bit bigger buffer to avoid valgrind warnings
        If table is opened readonly, don't update state
      storage/maria/ma_packrec.c:
        Allocate bigger array for bit unpacking to avoid valgrind errors
      storage/maria/ma_recovery.c:
        Added UNDO_KEY_INSERT_WITH_ROOT & UNDO_KEY_DELETE_WITH_ROOT
      storage/maria/ma_sort.c:
        More logging
      storage/maria/ma_test_all.sh:
        More tests
      storage/maria/ma_test_recovery.expected:
        Update results
        Note that this is not complete becasue of a bug in recovery
      storage/maria/ma_test_recovery:
        Removed recreation of index (not needed when we have redo for index pages)
      storage/maria/maria_chk.c:
        When using flag --read-only, don't update status for files
        When using --unpack, don't use REPAIR_BY_SORT if other repair option is given
        Enable repair_by_sort for BLOCK records
        Removed not needed newline at start of --describe
      storage/maria/maria_def.h:
        Support for TRANSID_SIZE to key pages
      storage/maria/maria_read_log.c:
        renamed --only-display to --display-only
      4e0964cb
  16. 14 Nov, 2007 2 commits
    • unknown's avatar
      First part of redo/undo for key pages · 21fd2a5a
      unknown authored
      Added key_nr to st_maria_keydef for faster keyinfo->keynr conversion
      For transactional tables, shift record number in keys up with 1 bit to have place to indicate if transid follows
      Checksum for MyISAM now ignores NULL and not used part of VARCHAR
      Renamed some variables that caused shadow compiler warnings
      Moved extra() call when waiting for tables to not be used to after tables are removed from cache.
      Fixed crashing bugs when using Maria TEMPORARY tables with TRUNCATE. Removed 'hack' code in sql directory to go around this bug.
      pagecache_unlock_by_ulink() now has extra argument to say if page was changed.
      Give error message if we fail to open control file
      Mark page cache variables as not flushable
      
      
      include/maria.h:
        Made min page cache larger (needed for pinning key page)
        Added key_nr to st_maria_keydef for faster keyinfo->keynr conversion
        Added write_comp_flag to move some runtime code to maria_open()
      include/my_base.h:
        Added new error message to be used when handler initialization failed
      include/my_global.h:
        Renamed dummy to swap_dummy to avoid conflicts with local 'dummy' variables
      include/my_handler.h:
        Added const to some parameters
      mysys/array.c:
        More DBUG
      mysys/my_error.c:
        Fixed indentation
      mysys/my_handler.c:
        Added const to some parameters
        Added missing error messages
      sql/field.h:
        Renamed variables to avoid variable shadowing
      sql/handler.h:
        Renamed parameter to avoid variable name conflict
      sql/item.h:
        Renamed variables to avoid variable shadowing
      sql/log_event_old.h:
        Renamed variables to avoid variable shadowing
      sql/set_var.h:
        Renamed variables to avoid variable shadowing
      sql/sql_delete.cc:
        Removed maria hack for temporary tables
        Fixed indentation
      sql/sql_table.cc:
        Moved extra() call when waiting for tables to not be used to after tables are removed from cache.
        This was needed to ensure we don't do a PREPARE_FOR_DROP or similar call while the table is still in use.
      sql/table.cc:
        Copy page_checksum from share
        Removed Maria hack
      storage/maria/Makefile.am:
        Added new files
      storage/maria/ha_maria.cc:
        Renamed records -> record_count and info -> create_info to avoid variable name conflicts
        Mark page cache variables as not flushable
      storage/maria/ma_blockrec.c:
        Moved _ma_unpin_all_pages() to ma_key_recover.c
        Moved init of info->pinned_pages to ma_open.c
        Moved _ma_finalize_row() to maria_key_recover.h
        Renamed some variables to avoid variable name conflicts
        Mark page_link.changed for blocks we change directly
        Simplify handling of undo link when writing LOGREC_UNDO_ROW_INSERT (old code crashed when having redo for index)
      storage/maria/ma_blockrec.h:
        Removed extra empty line
      storage/maria/ma_checkpoint.c:
        Remove not needed trnman.h
      storage/maria/ma_close.c:
        Free pinned pages (which are now always allocated)
      storage/maria/ma_control_file.c:
        Give error message if we fail to open control file
      storage/maria/ma_delete.c:
        Changes for redo logging (first part, logging of underflow not yet done)
        - Log undo-key-delete
        - Log delete of key
        - Updated arguments to _ma_fetch_keypage(), _ma_dispose(), _ma_write_keypage(), _ma_insert()
        - Added new arguments to some functions to be able to write redo information
        - Mark key pages as changed when we write with PAGECACHE_LOCK_LEFT_WRITELOCKED
        
        Remove one not needed _ma_write_keypage() in d_search() when upper level will do the write anyway
        Changed 2 bmove_upp() to bmove() as this made code easer to understand
        More function comments
        Indentation fixes
      storage/maria/ma_ft_update.c:
        New arguments to _ma_write_keypage()
      storage/maria/ma_loghandler.c:
        Fixed some DBUG_PRINT messages
        Simplify code
        Added new log entrys for key page redo
        Renamed some variables to avoid variable name shadowing
      storage/maria/ma_loghandler.h:
        Moved some defines here
        Added define for storing key number on key pages
        Added new translog record types
        Added enum for type of operations in LOGREC_REDO_INDEX
      storage/maria/ma_open.c:
        Always allocate info.pinned_pages (we need now also for normal key page usage)
        Update keyinfo->key_nr
        Added virtual functions to convert record position o number to be stored on key pages
        Update keyinfo->write_comp_flag to value of search flag to be used when writing key
      storage/maria/ma_page.c:
        Added redo for key pages
        - Extended _ma_fetch_keypage() with type of lock to put on page and address to used MARIA_PINNED_PAGE
        - _ma_fetch_keypage() now pin's pages if needed
        - Extended _ma_write_keypage() with type of locks to be used
        - ma_dispose() now locks info->s->state.key_del from other threads
        - ma_dispose() writes redo log record
        - ma_new() locks info->s->state.key_del from other threads if it was used
        - ma_new() now pins read page
        
        Other things:
        - Removed some not needed arguments from _ma_new() and _ma_dispose)
        - Added some new variables to simplify code
        - If EXTRA_DEBUG is used, do crc on full page to catch not unitialized bytes
      storage/maria/ma_pagecache.h:
        Applied patch from Sanja to add extra argument to pagecache_unlock_by_ulink() to mark if page was changed
        Added some defines for pagecache priority levels that one can use
      storage/maria/ma_range.c:
        Added new arguments for call to _ma_fetch_keypage()
      storage/maria/ma_recovery.c:
        - Added hooks for new translog types:
          REDO_INDEX, REDO_INDEX_NEW_PAGE, REDO_INDEX_FREE_PAGE, UNDO_KEY_INSERT, UNDO_KEY_DELETE and
          UNDO_KEY_DELETE_WITH_ROOT.
        - Moved variable declarations to start of function (portability fixes)
        - Removed some not needed initializations
        - Set only relevant state changes for each redo/undo entry
      storage/maria/lockman.c:
        Removed end space
      storage/maria/ma_check.c:
        Removed end space
      storage/maria/ma_create.c:
        Removed end space
      storage/maria/ma_locking.c:
        Removed end space
      storage/maria/ma_packrec.c:
        Removed end space
      storage/maria/ma_pagecache.c:
        Removed end space
      storage/maria/ma_panic.c:
        Removed end space
      storage/maria/ma_rt_index.c:
        Added new arguments for call to _ma_fetch_keypage(), _ma_write_keypage(), _ma_dispose() and _ma_new()
        Fixed indentation
      storage/maria/ma_rt_key.c:
        Added new arguments for call to _ma_fetch_keypage()
      storage/maria/ma_rt_split.c:
        Added new arguments for call to _ma_new()
        Use new keypage header
        Added new arguments for call to _ma_write_keypage()
      storage/maria/ma_search.c:
        Updated comments & indentation
        Added new arguments for call to _ma_fetch_keypage()
        Made some variables and arguments const
        Added virtual functions for converting row position to number to be stored in key
        use MARIA_RECORD_POS of record position instead of my_off_t
        Record in MARIA_KEY_PARAM how page was changed one key insert (needed for REDO)
      storage/maria/ma_sort.c:
        Removed end space
      storage/maria/ma_statrec.c:
        Updated arguments for call to _ma_rec_pos()
      storage/maria/ma_test1.c:
        Fixed too small buffer to init_pagecache()
        Fixed bug when using insert_count and test_flag
      storage/maria/ma_test2.c:
        Use more resonable pagecache size
        Remove not used code
        Reset blob_length to fix wrong output message
      storage/maria/ma_test_all.sh:
        Fixed wrong test
      storage/maria/ma_write.c:
        Lots of new code to handle REDO of key pages
        No logic changes because of REDO code, mostly adding new arguments and adding new code for logging 
        
        Added new arguments for calls to _ma_fetch_keypage(), _ma_write_keypage() and similar functions
        Move setting of comp_flag in ma_ck_wrte_btree() from runtime to maria_open()
        Zerofill new used pages for:
        - To remove possible sensitive data left in buffer
        - To get idenitical data on pages after running redo
        - Better compression of pages if archived
      storage/maria/maria_chk.c:
        Added information if table is crash safe
      storage/maria/maria_def.h:
        New virtual function to convert between record position on key and normal record position
        Aded mutex and extra variables to handle locking of share->state.key_del
        Moved some structure variables to get things more aligned
        Added extra arguments to MARIA_KEY_PARAM to be able to remember what was changed on key page on key insert
        Added argument to MARIA_PINNED_PAGE to indicate if page was changed
        Updated prototypes for functions
        Added some structures for signaling changes in REDO handling
      storage/maria/unittest/ma_pagecache_single.c:
        Updated arguments for changed function calls
      storage/myisam/mi_check.c:
        Made calc_check_checksum virtual
      storage/myisam/mi_checksum.c:
        Update checksums to ignore null columns
      storage/myisam/mi_create.c:
        Mark if table has null column (to know when we have to use mi_checksum())
      storage/myisam/mi_open.c:
        Added virtual function for calculating checksum to be able to easily ignore NULL fields
      storage/myisam/mi_test2.c:
        Fixed bug
      storage/myisam/myisamdef.h:
        Added virtual function for calculating checksum during check table
        Removed ha_key_cmp() as this is in handler.h
      storage/maria/ma_key_recover.c:
        New BitKeeper file ``storage/maria/ma_key_recover.c''
      storage/maria/ma_key_recover.h:
        New BitKeeper file ``storage/maria/ma_key_recover.h''
      storage/maria/ma_key_redo.c:
        New BitKeeper file ``storage/maria/ma_key_redo.c''
      21fd2a5a
    • unknown's avatar
      WL#3072 - Maria recovery · 1bc5e3b9
      unknown authored
      maria_read_log used to always print a warning message at startup
      to say it is unsafe if ALTER TABLE was used. Now it prints it only
      if the log does show the problem (=ALTER TABLE or CREATE SELECT, which
      both disable logging of REDO_INSERT*).
      For that, when ha_maria::external_lock() disables transactionality
      it writes a LOGREC_INCOMPLETE_LOG to the log, which "maria_read_log -a"
      picks up to write a warning.
      REPAIR TABLE also disables those REDO_INSERT* but as maria_read_log
      executes LOGREC_REDO_REPAIR no warning is needed.
      
      
      storage/maria/ha_maria.cc:
        as we now log a record when disabling transactionility, we need the
        TRN to be set up first
      storage/maria/ma_blockrec.c:
        comment
      storage/maria/ma_loghandler.c:
        new type of log record
      storage/maria/ma_loghandler.h:
        new type of log record
      storage/maria/ma_recovery.c:
        * maria_apply_log() now returns a count of warnings. What currently
        produces warnings is:
        - skipping applying UNDOs though there are some (=> inconsistent table)
        - replaying log (in maria_read_log) though the log contains some
        ALTER TABLE or CREATE SELECT (log misses REDO_INSERT* for those
        and is so incomplete).
        Count of warnings affects the final message of maria_read_log and
        recovery (though in recovery none of the two conditions above should
        happen).
        * maria_read_log used to always print a warning message at startup
        to say it is unsafe if ALTER TABLE was used. Now it prints it only
        if the log does show the problem, i.e. ALTER TABLE or CREATE SELECT
        was used (both disable logging of REDO_INSERT* as those records are
        not needed for recovery; those missing records in turn make
        recreation-from-scratch, via maria_read_log, impossible). For that,
        when ha_maria::external_lock() disables transactionality,
        _ma_tmp_disable_logging_for_table() writes a LOGREC_INCOMPLETE_LOG to
        the log, which maria_apply_log() picks up to write a warning.
      storage/maria/ma_recovery.h:
        maria_apply_log() returns a count of warnings
      storage/maria/maria_def.h:
        _ma_tmp_disable_logging_for_table() grows so becomes a function
      storage/maria/maria_read_log.c:
        maria_apply_log can now return a count of warnings, to temper the
        "SUCCESS" message printed in the end by maria_read_log.
        Advise users to make a backup first.
      1bc5e3b9
  17. 19 Oct, 2007 1 commit
    • unknown's avatar
      Merge some changes from sql directory in 5.1 tree · 13d53bf6
      unknown authored
      Changed format for REDO_INSERT_ROWS_BLOBS
      Fixed several bugs in handling of big blobs
      Added redo_free_head_or_tail() & redo_insert_row_blobs()
      Added uuid to control file
      maria_checks now verifies that not used part of bitmap is 0
      REDO_PURGE_BLOCKS -> REDO_FREE_BLOCKS
      Added REDO_FREE_HEAD_OR_TAIL
      Fixes problem when trying to read block outside of file during REDO
      
      
      include/my_global.h:
        STACK_DIRECTION is already set by configure
      mysql-test/r/maria.result:
        Updated results
      mysql-test/t/maria.test:
        Test shrinking of VARCHAR
      mysys/my_realloc.c:
        Fixed indentation
      mysys/safemalloc.c:
        Fixed indentation
      sql/filesort.cc:
        Removed some casts
      sql/mysqld.cc:
        Added missing setting of myisam_stats_method_str
      sql/uniques.cc:
        Removed some casts
      storage/maria/ma_bitmap.c:
        Added printing of bitmap (for debugging)
        Renamed _ma_print_bitmap() -> _ma_print_bitmap_changes()
        Added _ma_set_full_page_bits()
        Fixed bug in ma_bitmap_find_new_place() (affecting updates) when using big files
      storage/maria/ma_blockrec.c:
        Changed format for REDO_INSERT_ROWS_BLOBS
        Fixed several bugs in handling of big blobs
        Added code to fix some cases where redo when using blobs didn't produce idenital .MAD files as normal usage
        REDO_FREE_ROW_BLOCKS doesn't anymore change pages; We only mark things free in bitmap
        Remove TAIL and filler extents from REDO_FREE_BLOCKS log entry. (Fixed some asserts)
        REDO_PURGE_BLOCKS -> REDO_FREE_BLOCKS
        Delete tails in update. (Fixed bug when doing update that shrinks blob/varchar length)
        Fixed bug when doing insert in block outside of file size.
        Added redo_free_head_or_tail() & redo_insert_row_blobs()
        Added pagecache_unlock_by_link() when read fails.
        Much more comments, DBUG and ASSERT entries
      storage/maria/ma_blockrec.h:
        Prototypes of new functions
        Define of SUB_RANGE_SIZE & BLOCK_FILLER_SIZE
      storage/maria/ma_check.c:
        Verify that not used part of bitmap is 0
      storage/maria/ma_control_file.c:
        Added uuid to control file
      storage/maria/ma_loghandler.c:
        REDO_PURGE_BLOCKS -> REDO_FREE_BLOCKS
        Added REDO_FREE_HEAD_OR_TAIL
      storage/maria/ma_loghandler.h:
        REDO_PURGE_BLOCKS -> REDO_FREE_BLOCKS
        Added REDO_FREE_HEAD_OR_TAIL
      storage/maria/ma_pagecache.c:
        If we write full block, remove error flag for block.
        (Fixes problem when trying to read block outside of file)
      storage/maria/ma_recovery.c:
        REDO_PURGE_BLOCKS -> REDO_FREE_BLOCKS
        Added REDO_FREE_HEAD_OR_TAIL
      storage/maria/ma_test1.c:
        Allow option after 'b' to be compatible with ma_test2
        (This is just to simplify test scripts like ma_test_recovery)
      storage/maria/ma_test2.c:
        Default size of blob is now 1000 instead of 1
      storage/maria/ma_test_all.sh:
        Added test for bigger blobs
      storage/maria/ma_test_recovery.expected:
        Updated results
      storage/maria/ma_test_recovery:
        Added test for bigger blobs
      13d53bf6
  18. 09 Oct, 2007 1 commit
    • unknown's avatar
      Moved randomize and my_rnd under mysys · 496741d5
      unknown authored
      Added my_uuid
      Added pre-support for PAGE_CHECKSUM
      Added syntax for CREATE ... PAGE_CHECKSUM=# TABLE_CHECKSUM=#
      Reserved place for page checksums on index, bitmap and block pages
      Added index number to header of index pages
      Added linked list for free directory entries (speeds up inserts with BLOCK format)
      Calculate checksums in original column order (fixes bug with checksum on rows with BLOCK format)
      Cleaned up all index handling to use 'info->s->keypage_header' (variable size) as the header for index pages (before this was '2')
      Added 0xffffffff to end of index and block data bases and 0xfffffffe at end of bitmap pages when page checksums are not enabled
      Added _ma_get_page_used() and _ma_get_used_and_node() to simplify index page header handling
      rec_per_key_part is now in double precision
      Reserved place in index file for my_guid and nulls_per_key_part
      Give error HA_ERR_NEW_FILE if trying to open a Maria file with new, not yet supported extensions
      
      Lots of renames to increase readability:
      
      randomize() -> my_rnd_init()
      st_maria_info -> st_maria_handler
      st_maria_info -> MARIA_HA
      st_maria_isaminfo -> st_maria_info
      rand_struct -> my_rand_struct
      rec_per_key_rows -> records_at_analyze
      
      
      client/mysqladmin.cc:
        rand_struct -> my_rrnd_struct
      include/maria.h:
        st_maria_info -> MARIA_HA
        st_maria_isaminfo -> st_maria_info
        Changed analyze statistics to be of double precission
        Changed offset to field to be 32bits instead of 64 (safe as a record without blobs can't be that big)
      include/my_base.h:
        Added HA_OPTION_PAGE_CHECKSUM & HA_CREATE_PAGE_CHECKSUM
        Fixed comments
        Added HA_ERR_NEW_FILE
      include/my_sys.h:
        Added prototypes and structures for my_uuid() and my_rnd()
      include/myisamchk.h:
        Changed some buffers to size_t
        Added possibility to have key statistics with double precission
      include/mysql_com.h:
        Move rand functions to mysys
      libmysql/Makefile.shared:
        Added my_rnd
      mysql-test/r/maria.result:
        Updated results
      mysql-test/t/maria.test:
        More tests for checksum
      mysys/Makefile.am:
        Added my_rnd.c and my_uuid.c
      server-tools/instance-manager/listener.cc:
        Fixed include order (my_global.h should always be first)
      server-tools/instance-manager/mysql_connection.cc:
        Fixed include order (my_global.h should always be first)
        Use my_rnd_init()
      server-tools/instance-manager/mysql_connection.h:
        rand_struct -> my_rand_struct
      sql/handler.h:
        Added flag for page checksums
      sql/item_func.cc:
        Use new my_rnd() interface
      sql/item_func.h:
        Use new my_rnd() interface
      sql/item_strfunc.cc:
        Use new my_rnd() interface
      sql/lex.h:
        Added PAGE_CHECKSUM and TABLE_CHECKSUM
      sql/mysql_priv.h:
        Use new my_rnd() interface
      sql/mysqld.cc:
        Use new my_rnd() interface
      sql/password.c:
        Move my_rnd() to mysys
        Use new my_rnd() interface
      sql/sql_class.cc:
        Use new my_rnd() interface
      sql/sql_class.h:
        Use new my_rnd() interface
      sql/sql_crypt.cc:
        Use new my_rnd() interface
      sql/sql_crypt.h:
        Use new my_rnd() interface
      sql/sql_show.cc:
        Simpler handling of ha_choice_values
        Added PAGE_CHECKSUM
      sql/sql_table.cc:
        Enable correct checksum handling (for now) if not running in compatible mode
      sql/sql_yacc.yy:
        Added table option PAGE_CHECKSUM
        Added future compatible table option TABLE_CHECKSUM (alias for CHECKSUM)
        Added 'choice' target to simplify code
      sql/table.cc:
        Store flag for PAGE_CHECKSUM
      sql/table.h:
        Added support for PAGE_CHECKSUM
      storage/maria/ha_maria.cc:
        Remove protection for incompatbile frm and MAI
        (Slow, not needed test)
        Rec_per_key is now in double
        Remember row type for table
        Give warning if one Maria uses another row type than requested
        Removed some old ASK_MONTY entries (added comments instead)
        Added handling of PAGE_CHECKSUM flags
      storage/maria/ma_bitmap.c:
        Added page checksums to bitmap pages
        Added special bitmap marker for bitmap pages
        (Used to find bugs when running without page checksums)
      storage/maria/ma_blockrec.c:
        Added a free-link list over directory entries. This makes insert of small rows faster as we don't
        have to scan the whole directory to find a not used entry.
        Moved SANITY_CHECKS to maria_def.h
        Simplify code by introducing dir_entry_pos()
        Added support for PAGE_CHECKSUM
      storage/maria/ma_blockrec.h:
        Added DIR_FREE_SIZE (linked list of free directory entries)
        Added PAGE_CHECKSUM
        Added 'dir_entry_pos()'
      storage/maria/ma_check.c:
        Check that index pages has correct index number
        Calculate rec_per_key with double precission
        Simplify code by using '_ma_get_used_and_node()'
        Check free directory list
        Remove wrong end \n from messages
        maria_data_on_page() -> _ma_get_page_used()
        maria_putint() -> _ma_store_page_used()
        rec_per_key_rows -> records_at_analyze
      storage/maria/ma_checksum.c:
        Calculate checksum in original column order
      storage/maria/ma_create.c:
        Store original column order in index file
        Reserve place for nulls_per_key_part (future)
        Added support for PAGE_CHECKSUM
      storage/maria/ma_dbug.c:
        Fixed wrong debug output of key of type 'ulong'
      storage/maria/ma_delete.c:
        maria_data_on_page() -> _ma_get_used_and_node()
        maria_data_on_page() -> _ma_get_page_used()
        maria_putint() -> _ma_store_page_used()
        Added page header (index key number) to all index pages
        Reserved page for checksum on index pages
        Use keypage_header
      storage/maria/ma_ft_update.c:
        maria_putint() -> _ma_store_page_used()
        Store key number at start of page
      storage/maria/ma_loghandler.h:
        st_maria_info -> MARIA_HA
      storage/maria/ma_open.c:
        rec_per_key is now in double precission
        Added 'nulls_per_key_part'
        Added 'extra_options' (flags for future)
        Added support for PAGE_CHECKSUM
        Give error HA_ERR_NEW_FILE when using unsupported maria extensions
        Added comments
        Add maria_uuid to index file
        Added functions to store and read column_nr map.
        Changed some functions to return my_bool instead of uint
      storage/maria/ma_page.c:
        Added checks that pages has correct key nr
        Store 0xffffffff in checksum position if page checksums are not enabled
        Moved key-page-delete link to take into account keypage header
      storage/maria/ma_preload.c:
        Remove old MyISAM dependent code
        When scanning pages, only add pages to page cache for the requested index
      storage/maria/ma_range.c:
        maria_data_on_page() -> _ma_get_used_and_node()
        Use keypage_header
      storage/maria/ma_rt_index.c:
        Fixed indentation
      storage/maria/ma_rt_index.h:
        Added support for dynamic index page header
        Reserved place for PAGE_CHECKSUM
      storage/maria/ma_rt_key.c:
        Fixed indentation
        maria_data_on_page() -> _ma_get_page_used()
        maria_putint() -> maria_store_page_used()
      storage/maria/ma_rt_mbr.c:
        Fixed indentation
      storage/maria/ma_rt_split.c:
        Fixed indentation
        maria_data_on_page () -> _ma_get_page_used()
      storage/maria/ma_rt_test.c:
        Fixed indentation
      storage/maria/ma_search.c:
        Remove support of using -1 as 'last used index' to _ma_check_index()
        maria_data_on_page() -> _ma_get_page_used()
        maria_data_on_page() -> _ma_get_used_and_node()
        Use keypage_header
      storage/maria/ma_sort.c:
        Changed some buffers to size_t
        Changed rec_per_key_part to double
      storage/maria/ma_static.c:
        Removed NEAR
        Added maria_uuid
      storage/maria/ma_test2.c:
        Moevd testflag == 2 to correct place
        Remove test of reading with index number -1 (not supported anymore)
      storage/maria/ma_test_recovery.expected:
        Updated results
      storage/maria/ma_test_recovery:
        Changed tmp table names so that one can run maria_chk on them
      storage/maria/ma_write.c:
        Fixed indentation
        Use keypage_header
        Store index number on index pages
        maria_putint() -> _ma_store_page_used()
        maria_data_on_page() -> ma_get_used_and_node()
        maria_data_on_page() -> _ma_get_page_used()
        Added PAGE_CHECKSUM
        Added Maria handler to some functions
        Removed some not needed casts
      storage/maria/maria_chk.c:
        Added error handling for HA_ERR_NEW_FILE
        Added information about page checksums
        rec_per_key_part changed to double
        maria_data_on_page() -> _ma_get_page_used()
        Use keypage_header
      storage/maria/maria_def.h:
        Added IDENTICAL_PAGES_AFTER_RECOVERY and SANITY_CHECKS
        Changed rec_per_key_part to double
        Added nulls_per_key_part
        rec_per_key_rows -> records_at_analyze
        st_maria_info -> MARIA_HA
        Reserve place for new statistics variables, uuid, checksums per page etc.
        Removed NEAR tags
        Changed some prototypes to use my_bool and size_t
      storage/maria/maria_pack.c:
        st_maria_info -> MARIA_HA
        Fixed indentation
      storage/myisam/mi_dbug.c:
        Fix wrong debug output for ULONG
      mysys/my_rnd.c:
        New BitKeeper file ``mysys/my_rnd.c''
      mysys/my_uuid.c:
        New BitKeeper file ``mysys/my_uuid.c''
      496741d5
  19. 02 Oct, 2007 1 commit
    • unknown's avatar
      WL#3072 - Maria recovery. · d0b9387b
      unknown authored
      * Recovery of the table's live checksum (CREATE TABLE ... CHECKSUM=1)
      is achieved in this patch. The table's live checksum
      (info->s->state.state.checksum) is updated in inwrite_rec_hook's
      under the log mutex when writing UNDO_ROW_INSERT|UPDATE|DELETE
      and REDO_DELETE_ALL. The checksum variation caused by the operation
      is stored in these UNDOs, so that the REDO phase, when it sees such
      UNDOs, can update the live checksum if it is older (state.is_of_lsn is
      lower) than the record. It is also used, as a nice add-on with no
      cost, to do less row checksum computation during the UNDO phase
      (as we have it in the record already).
      Doing this work, it became pressing to move in-write hooks
      (write_hook_for_redo() et al) to ma_blockrec.c.
      The 'parts' argument of inwrite_rec_hook is unpredictable (it comes
      mangled at this stage, for example by LSN compression) so it is
      replaced by a 'void* hook_arg', which is used to pass down information,
      currently only to write_hook_for_clr_end() (previous undo_lsn and
      type of undone record).
      * If from ha_maria, we print to stderr how many seconds (with one
      fractional digit) the REDO phase took, same for UNDO phase and for
      final table close. Just to give an indication for debugging and maybe
      also for Support.
      
      
      storage/maria/ha_maria.cc:
        question for Monty
      storage/maria/ma_blockrec.c:
        * log in-write hooks (write_hook_for_redo() etc) move from
        ma_loghandler.c to here; this is natural: the hooks are coupled
        to their callers (functions in ma_blockrec.c).
        * translog_write_record() now has a new argument "hook_arg";
        using it to pass down to write_hook_for_clr_end() the transaction's
        previous_undo_lsn and the type of the being undone record, and also
        to pass down to all UNDOs the live checksum variation caused by the
        operation.
        * If table has live checksum, store in UNDO_ROW_INSERT|UPDATE|DELETE
        and in CLR_END the checksum variation ("delta") caused by the
        operation. For example if a DELETE caused the table's live checksum
        to change from 123 to 456, we store in the UNDO_ROW_DELETE, in 4 bytes,
        the value 333 (456-123).
        * Instead of hard-coded "1" as length of the place where we store
        the undone record's type in CLR_END, use a symbol CLR_TYPE_STORE_SIZE;
        use macros clr_type_store and clr_type_korr.
        * write_block_record() has a new parameter 'old_record_checksum'
        which is the pre-computed checksum of old_record; that value is used
        to update the table's live checksum when writing UNDO_ROW_UPDATE|CLR_END.
        * In allocate_write_block_record(), if we are executing UNDO_ROW_DELETE
        the row's checksum is already computed.
        * _ma_update_block_record2() now expect the new row's checksum into
        cur_row.checksum (was already true) and the old row's checksum into
        new_row.checksum (that's new). Its two callers, maria_update() and
        _ma_apply_undo_row_update(), honour this.
        * When executing an UNDO_ROW_INSERT|UPDATE|DELETE in UNDO phase, pick
        up the checksum delta from the log record. It is then used to update
        the table's live checksum when writing CLR_END, and saves us a
        computation of record.
      storage/maria/ma_blockrec.h:
        in-write hooks move from ma_loghandler.c
      storage/maria/ma_check.c:
        more straightforward size of buffer
      storage/maria/ma_checkpoint.c:
        <= is enough
      storage/maria/ma_commit.c:
        new prototype of translog_write_record()
      storage/maria/ma_create.c:
        new prototype of translog_write_record()
      storage/maria/ma_delete.c:
        The row's checksum must be computed before calling(*delete_record)(),
        not after, because it must be known inside _ma_delete_block_record()
        (to update the table's live checksum when writing UNDO_ROW_DELETE).
        If deleting from a transactional table, live checksum was already updated
        when writing UNDO_ROW_DELETE.
      storage/maria/ma_delete_all.c:
        @todo is now done (in ma_loghandler.c)
      storage/maria/ma_delete_table.c:
        new prototype of translog_write_record()
      storage/maria/ma_loghandler.c:
        * in-write hooks move to ma_blockrec.c.
        * translog_write_record() gets a new argument 'hook_arg' which is
        passed down to pre|inwrite_rec_hook. It is more useful that 'parts'
        for those hooks, because when those hooks are called, 'parts' has
        possibly been mangled (like with LSN compression) and is so
        unpredictable.
        * fix for compiler warning (unused buffer_start when compiling without
        debug support)
        * Because checksum delta is stored into UNDO_ROW_INSERT|UPDATE|DELETE
        and CLR_END, but only if the table has live checksum, these records
        are not PSEUDOFIXEDLENGTH anymore, they are now VARIABLE_LENGTH (their
        length is X if no live checksum and X+4 otherwise).
        * add an inwrite_rec_hook for UNDO_ROW_UPDATE, which updates the
        table's live checksum. Update it also in hooks of UNDO_ROW_INSERT|
        DELETE and REDO_DELETE_ALL and CLR_END.
        * Bugfix: when reading a record in translog_read_record(), it happened
        that "length" became negative, because the function assumed that
        the record extended beyond the page's end, whereas it may be shorter.
      storage/maria/ma_loghandler.h:
        * Instead of hard-coded "1" and "4", use symbols and macros
        to store/retrieve the type of record which the CLR_END corresponds
        to, and the checksum variation caused by the operation which logs the
        record
        * translog_write_record() gets a new argument 'hook_arg' which is
        passed down to pre|inwrite_rec_hook. It is more useful that 'parts'
        for those hooks, because when those hooks are called, 'parts' has
        possibly been mangled (like with LSN compression) and is so
        unpredictable.
      storage/maria/ma_open.c:
        fix for "empty body in if() statement" (when compiling without safemutex)
      storage/maria/ma_pagecache.c:
        <= is enough
      storage/maria/ma_recovery.c:
        * print the time that each recovery phase (REDO/UNDO/flush) took;
        this is enabled only when recovering from ha_maria. Is it printed
        n seconds with a fractional part of one digit (like 123.4 seconds).
        * In the REDO phase, update the table's live checksum by using
        the checksum delta stored in UNDO_ROW_INSERT|DELETE|UPDATE and CLR_END.
        Update it too when seeing REDO_DELETE_ALL.
        * In the UNDO phase, when executing UNDO_ROW_INSERT, if the table does
        not have live checksum then reading the record's header (as done by
        the master loop of run_undo_phase()) is enough; otherwise we
        do a translog_read_record() to have the checksum delta ready
        for _ma_apply_undo_row_insert().
        * When at the end of the REDO phase we notice that there is an unfinished
        group of REDOs, don't assert in debug binaries, as I verified that it
        can happen in real life (with kill -9)
        * removing ' in #error as it confuses gcc3
      storage/maria/ma_rename.c:
        new prototype of translog_write_record()
      storage/maria/ma_test_recovery.expected:
        Change in output of ma_test_recovery: now all live checksums of
        original tables equal those of tables recreated by the REDO phase
        and those of tables fixed by the UNDO phase. I.e. recovery of
        the live checksum looks like working (which was after all the only
        goal of this changeset).
        I checked by hand that it's not just all live checksums which are
        now 0 and that's why they match. They are the old values like
        3757530372. maria.test has hard-coded checksum values in its result
        file so checks this too.
      storage/maria/ma_update.c:
        * It's useless to put up HA_STATE_CHANGED in 'key_changed',
        as we put up HA_STATE_CHANGED in info->update anyway.
        * We need to compute the old and new rows' checksum before calling
        (*update_record)(), as checksum delta must be known when logging
        UNDO_ROW_UPDATE which is done by _ma_update_block_record(). Note that
        some functions change the 'newrec' record (at least _ma_check_unique()
        does) so we cannot move the checksum computation too early in the
        function.
      storage/maria/ma_write.c:
        If inserting into a transactional table, live's checksum was
        already updated when writing UNDO_ROW_INSERT. The multiplication
        is a trick to save an if().
      storage/maria/unittest/ma_test_loghandler-t.c:
        new prototype of translog_write_record()
      storage/maria/unittest/ma_test_loghandler_first_lsn-t.c:
        new prototype of translog_write_record()
      storage/maria/unittest/ma_test_loghandler_max_lsn-t.c:
        new prototype of translog_write_record()
      storage/maria/unittest/ma_test_loghandler_multigroup-t.c:
        new prototype of translog_write_record()
      storage/maria/unittest/ma_test_loghandler_multithread-t.c:
        new prototype of translog_write_record()
      storage/maria/unittest/ma_test_loghandler_noflush-t.c:
        new prototype of translog_write_record()
      storage/maria/unittest/ma_test_loghandler_pagecache-t.c:
        new prototype of translog_write_record()
      storage/maria/unittest/ma_test_loghandler_purge-t.c:
        new prototype of translog_write_record()
      storage/myisam/sort.c:
        fix for compiler warnings in pushbuild (write_merge_key* functions
        didn't have their declaration match MARIA_HA::write_key).
      d0b9387b
  20. 27 Sep, 2007 2 commits
    • unknown's avatar
      Faster translog_filename_by_fileno · 0de4c556
      unknown authored
      Fixed compiler warning
      PAGECACHE_PAGE_LINK -> PAGECACHE_BLOCK_LINK*
      
      
      storage/maria/ma_loghandler.h:
        PAGECACHE_PAGE_LINK -> PAGECACHE_BLOCK_LINK*
      storage/maria/ma_pagecache.c:
        PAGECACHE_PAGE_LINK -> PAGECACHE_BLOCK_LINK*
      storage/maria/ma_pagecache.h:
        PAGECACHE_PAGE_LINK -> PAGECACHE_BLOCK_LINK*
      storage/maria/maria_def.h:
        PAGECACHE_PAGE_LINK -> PAGECACHE_BLOCK_LINK*
      0de4c556
    • unknown's avatar
      Remove SAFE_MODE for opt_range as it disables UPDATE to use keys · c9c58163
      unknown authored
      REDO optimization (Bascily avoid moving blocks from/to pagecache)
      More command line arguments to maria_read_log
      Fixed recovery bug when recreating table
      
      
      sql/opt_range.cc:
        Remove SAFE_MODE for opt_range as it disables UPDATE to use keys
      storage/maria/ma_blockrec.c:
        REDO optimization
        Use new interface for pagecache_reads to avoid copying page buffers
      storage/maria/ma_loghandler.c:
        Patch from Sanja:
        - Added new parameter to translog_get_page to use direct links to pagecache
        - Changed scanner to be able to use direct links
        
        This avoids a lot of calls to bmove512() in page cache.
      storage/maria/ma_loghandler.h:
        Added direct link to pagecache objects
      storage/maria/ma_open.c:
        Added const to parameter
        Added missing braces
      storage/maria/ma_pagecache.c:
        From Sanja:
        - Added direct links to pagecache (from pagecache_read())
          Dirrect link means that on pagecache_read we get back a pointer to the pagecache buffer
        
        
        From Monty:
        - Fixed arguments to init_page_cache to handle big page caches
        - Fixed compiler warnings
        - Replaced PAGECACHE_PAGE_LINK with PAGECACHE_BLOCK_LINK * to catch errors
      storage/maria/ma_pagecache.h:
        Changed block numbers from int to long to be able to handle big page caches
        Changed some PAGECACHE_PAGE_LINK to PAGECACHE_BLOCK_LINK
      storage/maria/ma_recovery.c:
        Fixed recovery bug when recreating table (table was kept open)
        Moved some variables to function start (portability)
        Added space to some print messages
      storage/maria/maria_chk.c:
        key_buffer_size -> page_buffer_size
      storage/maria/maria_def.h:
        Changed default page_buffer_size to 10M
      storage/maria/maria_read_log.c:
        Added more startup options:
        --version
        --undo (apply undo)
        --page_cache_size (to run with big cache sizes)
        --silent (to not get any output from --apply)
      storage/maria/unittest/ma_control_file-t.c:
        Fixed compiler warning
      storage/maria/unittest/ma_test_loghandler-t.c:
        Added new argument to translog_init_scanner()
      storage/maria/unittest/ma_test_loghandler_multigroup-t.c:
        Added new argument to translog_init_scanner()
      storage/maria/unittest/ma_test_loghandler_multithread-t.c:
        Added new argument to translog_init_scanner()
      c9c58163
  21. 14 Sep, 2007 1 commit
  22. 12 Sep, 2007 1 commit
    • unknown's avatar
      WL#3071 Maria checkpoint · cec8ac3e
      unknown authored
      Finally this is the real checkpoint code.
      It however exhibits unstabilities when a checkpoint runs concurrently
      with data-modifying clients (table corruption, transaction log's
      assertions) so for now a checkpoint is taken only at startup after
      recovery and at shutdown, i.e. not in concurrent situations. Later
      we will let it run periodically, as well as flush dirty pages
      periodically (almost all needed code is there already, only pagecache
      code is written but not committed).
      WL#3072 Maria recovery
      * replacing UNDO_ROW_PURGE with CLR_END; testing of those CLR_END via
      ma_test2 which has INSERTs failing with duplicate keys.
      * replaying of REDO_RENAME_TABLE
      Now, off to test Recovery in ha_maria :)
      
      
      BitKeeper/deleted/.del-ma_least_recently_dirtied.c:
        Delete: storage/maria/ma_least_recently_dirtied.c
      BitKeeper/deleted/.del-ma_least_recently_dirtied.h:
        Delete: storage/maria/ma_least_recently_dirtied.h
      storage/maria/Makefile.am:
        compile Checkpoint module
      storage/maria/ha_maria.cc:
        When ha_maria starts, do a recovery from last checkpoint.
        Take a checkpoint when that recovery has ended and when ha_maria
        shuts down cleanly.
      storage/maria/ma_blockrec.c:
        * even if my_sync() fails we have to my_close() (otherwise we leak
        a descriptor)
        * UNDO_ROW_PURGE is replaced by a simple CLR_END for UNDO_ROW_INSERT,
        as promised in the old comment; it gives us skipping during the
        UNDO phase.
      storage/maria/ma_check.c:
        All REDOs before create_rename_lsn are ignored by Recovery. So
        create_rename_lsn must be set only after all data/index has been
        flushed and forced to disk. We thus move write_log_record_for_repair()
        to after _ma_flush_tables_files_after_repair().
      storage/maria/ma_checkpoint.c:
        Checkpoint module.
      storage/maria/ma_checkpoint.h:
        optional argument if caller wants a thread to periodically take
        checkpoints and flush dirty pages.
      storage/maria/ma_create.c:
        * no need to init some vars as the initial bzero(share) takes care of this.
        * update to new function's name
        * even if we fail in my_sync() we have to my_close()
      storage/maria/ma_extra.c:
        Checkpoint reads share->last_version under intern_lock, so we make
        maria_extra() update it under intern_lock. THR_LOCK_maria still needed
        because of _ma_test_if_reopen().
      storage/maria/ma_init.c:
        destroy checkpoint module when Maria shuts down.
      storage/maria/ma_loghandler.c:
        * UNDO_ROW_PURGE gone (see ma_blockrec.c)
        * we need to remember the LSN of the LOGREC_FILE_ID for a share,
        because this LSN is needed into the checkpoint record (Recovery wants
        to know the validity domain of an id->name mapping)
        * translog_get_horizon_no_lock() needed for Checkpoint
        * comment about failing assertion (Sanja knows)
        * translog_init_reader_data() thought that translog_read_record_header_scan()
        returns 0 in case of error, but 0 just means "0-length header".
        * translog_assign_id_to_share() now needs the MARIA_HA because
        LOGREC_FILE_ID uses a log-write hook.
        * Verify that (de)assignment of share->id happens only under intern_lock,
        as Checkpoint reads this id with intern_lock.
        * translog_purge() can accept TRANSLOG_ADDRESS, not necessarily
        a real LSN.
      storage/maria/ma_loghandler.h:
        prototype updates
      storage/maria/ma_open.c:
        no need to initialize "res"
      storage/maria/ma_pagecache.c:
        When taking a checkpoint, we don't need to know the maximum rec_lsn
        of dirty pages; this LSN was intended to be used in the two-checkpoint
        rule, but last_checkpoint_lsn is as good.
        4 bytes for stored_list_size is enough as PAGECACHE::blocks (number
        of blocks which the pagecache can contain) is int.
      storage/maria/ma_pagecache.h:
        new prototype
      storage/maria/ma_recovery.c:
        * added replaying of REDO_RENAME_TABLE
        * UNDO_ROW_PURGE gone (see ma_blockrec.c), replaced by CLR_END
        * Recovery from the last checkpoint record now possible
        * In new_table() we skip the table if the id->name mapping is older than
        create_rename_lsn (mapping dates from lsn_of_file_id).
        * in get_MARIA_HA_from_REDO_record() we skip the record
        if the id->name mapping is newer than the record (can happen if processing
        a record which is before the checkpoint record).
        * parse_checkpoint_record() has to return a LSN, that's what caller expects
      storage/maria/ma_rename.c:
        new function's name; log end zeroes of tables' names (ease recovery)
      storage/maria/ma_test2.c:
        * equivalent of ma_test1's --test-undo added (named -u here).
        * -t=1 now stops right after creating the table, so that
        we can test undoing of INSERTs with duplicate keys (which tests the
        CLR_END logged by _ma_write_abort_block_record()).
      storage/maria/ma_test_recovery.expected:
        Result of testing undoing of INSERTs with duplicate keys; there are
        some differences in maria_chk -dvv but they are normal (removing
        records does not shrink data/index file, does not put back the
        "analyzed, optimized keys"(etc) index state.
      storage/maria/ma_test_recovery:
        Test undoing of INSERTs with duplicate keys, using ma_test2;
        when such INSERT happens, it logs REDO_INSERT, UNDO_INSERT, REDO_DELETE,
        CLR_END; we abort after that, and test that CLR_END causes recovery
        to jump over UNDO_INSERT.
      storage/maria/ma_write.c:
        comment
      storage/maria/maria_chk.c:
        comment
      storage/maria/maria_def.h:
        * a new bit in MARIA_SHARE::in_checkpoint, used to build a list
        of unique shares during Checkpoint.
        * MARIA_SHARE::lsn_of_file_id added: the LSN of the last LOGREC_FILE_ID
        for this share; needed to know to which LSN domain the mappings
        found in the Checkpoint record apply (new mappings should not apply
        to old REDOs).
      storage/maria/trnman.c:
        * small changes to how trnman_collect_transactions() fills its buffer;
        it also uses a non-dummy lsn_read_non_atomic() found in ma_checkpoint.h
      cec8ac3e
  23. 04 Sep, 2007 2 commits
    • unknown's avatar
      Check of transaction log descriptor table consistance added. · 73b073ff
      unknown authored
      Small fixes made.
      
      
      storage/maria/ma_loghandler.c:
        Check of transaction log descriptor table consistance added.\
        Incorrect record description fixed.
        Compiler warning fixed.
      storage/maria/ma_loghandler.h:
        fixed ident.
      storage/maria/unittest/ma_test_loghandler-t.c:
        Suppressing of automatic record writing
      storage/maria/unittest/ma_test_loghandler_first_lsn-t.c:
        Suppressing of automatic record writing
      storage/maria/unittest/ma_test_loghandler_max_lsn-t.c:
        Suppressing of automatic record writing
      storage/maria/unittest/ma_test_loghandler_multigroup-t.c:
        Suppressing of automatic record writing
      storage/maria/unittest/ma_test_loghandler_multithread-t.c:
        Suppressing of automatic record writing
      storage/maria/unittest/ma_test_loghandler_noflush-t.c:
        Suppressing of automatic record writing
      storage/maria/unittest/ma_test_loghandler_pagecache-t.c:
        Suppressing of automatic record writing
      storage/maria/unittest/ma_test_loghandler_purge-t.c:
        Suppressing of automatic record writing
      73b073ff
    • unknown's avatar
      Added UNDO handling of insert during recovery · 03437ea0
      unknown authored
      storage/maria/ma_blockrec.c:
        Added UNDO handling of insert during recovery
        To do this, I also had to add write locking of tail pages during undo phase (As we need to access the same page twice if extents are split over two pages)
        Another way to handle the undo of insert would be to store the extent information as part of the UNDO_INSERT block.
      storage/maria/ma_blockrec.h:
        Added new prototype
      storage/maria/ma_loghandler.c:
        Changed type of CLR_END (to avoid crash in log handler)
        Removed not used variable
      storage/maria/ma_loghandler.h:
        Added TRN argument to record_execute_in_undo_phase()
      storage/maria/ma_pagecache.c:
        Hack for undo phase of recovery.  During REDO we work with PLAIN pages, but UNDO works with LSN pages, which caused an abort when trying to access a cached page.
      storage/maria/ma_recovery.c:
        Added execution of UNDO_ROW_INSERT
      storage/maria/ma_test1.c:
        Added option --test-undo for testing recovery with undo
      storage/maria/maria_read_log.c:
        Added processing of undos
      03437ea0
  24. 29 Aug, 2007 1 commit
    • unknown's avatar
      WL#3072 Maria recovery · e27890ca
      unknown authored
      * create page cache before initializing engine and not after, because
      Maria's recovery needs a page cache
      * make the creation of a bitmap page more crash-resistent
      * bugfix (see ma_blockrec.c)
      * back to old way: create an 8k bitmap page when creating table
      * preparations for the UNDO phase: recreate TRNs
      * preparations for Checkpoint: list of dirty pages, testing
      of rec_lsn to know if page should be skipped during Recovery
      (unused in this patch as no Checkpoint module pushed yet)
      * maria_chk tags repaired table with a special LSN
      * reworking all around in ma_recovery.c (less duplication)
      
      
      mysys/my_realloc.c:
        noted an issue in my_realloc()
      sql/mysqld.cc:
        page cache needs to be created before engines are initialized,
        because Maria's initialization may do a recovery which needs
        the page cache.
      storage/maria/ha_maria.cc:
        update to new prototype
      storage/maria/ma_bitmap.c:
        when creating the first bitmap page we used chsize to 8192 bytes then 
        pwrite (overwrite) the last 2 bytes (8191-8192). If crash between
        the two operations, this leaves a bitmap page full without its end
        marker. A later recovery may try to read this page and find it
        exists and misses a marker and conclude it's corrupted and fail.
        Changing the chsize to only 8190 bytes: recovery will then find
        the page is too short and recreate it entirely.
      storage/maria/ma_blockrec.c:
        Fix for a bug: when executing a REDO, if the data page is created,
        data_file_length was increased before _ma_bitmap_set():
        _ma_bitmap_set() called _ma_read_bitmap_page() which, due to the
        increased data_file_length, expected to find a bitmap page on disk
        with a correct end marker; if the bitmap page didn't exist already
        in fact, this failed. Fixed by increasing data_file_length only after
        _ma_read_bitmap_page() has created the new bitmap page correctly.
        This bug could happen every time a REDO is about creating a new
        bitmap page.
      storage/maria/ma_check.c:
        empty data file has a bitmap page
      storage/maria/ma_control_file.c:
        useless parameter to ma_control_file_create_or_open(), just
        test if this is recovery.
      storage/maria/ma_control_file.h:
        new prototype
      storage/maria/ma_create.c:
        Back to how it was before: maria_create() creates an 8k bitmap page.
        Thus (bugfix) data_file_length needs to reflect this instead of being 0.
      storage/maria/ma_loghandler.c:
        as ma_test1 and ma_test2 now use real transactions and not
        dummy_transaction_object, REDO for INSERT/UPDATE/DELETE are always
        about real transactions, can assert this.
        A function for Recovery to assign a short id to a table.
      storage/maria/ma_loghandler.h:
        new function
      storage/maria/ma_loghandler_lsn.h:
        maria_chk tags repaired tables with this LSN
      storage/maria/ma_open.c:
        * enforce that DMLs on transactional tables use real transactions
        and not dummy_transaction_object.
        * test if table was repaired with maria_chk (which has to been
        seen as an import of an external table into the server), test
        validity of create_rename_lsn (header corruption detection)
        * comments.
      storage/maria/ma_recovery.c:
        * preparations for the UNDO phase: recreate TRNs
        * preparations for Checkpoint: list of dirty pages, testing
        of rec_lsn to know if page should be skipped during Recovery
        (unused in this patch as no Checkpoint module pushed yet)
        * reworking all around (less duplication)
      storage/maria/ma_recovery.h:
        a parameter to say if the UNDO phase should be skipped
      storage/maria/maria_chk.c:
        tag repaired tables with a special LSN
      storage/maria/maria_read_log.c:
        * update to new prototype
        * no UNDO phase in maria_read_log for now
      storage/maria/trnman.c:
        * a function for Recovery to create a transaction (TRN), needed
        in the UNDO phase
        * a function for Recovery to grab an existing transaction, needed
        in the UNDO phase (rollback all existing transactions)
      storage/maria/trnman_public.h:
        new functions
      e27890ca
  25. 27 Aug, 2007 1 commit
    • unknown's avatar
      Very simple log files purger added. · 7c273b82
      unknown authored
      File number types fixed.
      
      
      storage/maria/unittest/Makefile.am:
        Test of purger added.
      storage/maria/unittest/ma_test_loghandler_purge-t.c:
        New BitKeeper file ``storage/maria/unittest/ma_test_loghandler_purge-t.c''
      7c273b82
  26. 19 Aug, 2007 1 commit
    • unknown's avatar
      Storing/getting maximum LSN of the record which parts written · d0398f72
      unknown authored
      in the file to the file header.
      
      
      storage/maria/ma_loghandler.h:
        Getting maximum LSN of the record which parts written
        in the file to the file header.
      storage/maria/unittest/Makefile.am:
        Test suite for getting max LSN added.
      storage/maria/unittest/ma_test_loghandler_first_lsn-t.c:
        Spelling fixed.
        Cleanup fixed.
      storage/maria/unittest/ma_test_loghandler_noflush-t.c:
        Cleanup fixed.
      storage/maria/unittest/ma_test_loghandler_max_lsn-t.c:
        New BitKeeper file ``storage/maria/unittest/ma_test_loghandler_max_lsn-t.c''
      d0398f72
  27. 13 Aug, 2007 1 commit
    • unknown's avatar
      First LSN calls added for transaction log. · 4cf6756e
      unknown authored
      storage/maria/ma_checkpoint.c:
        Definitions of LSN should be collected in
        the one file (ma_loghandler_lsn.h)
      storage/maria/ma_loghandler.c:
        New calls to get first theoretical and first real LSN.
      storage/maria/ma_loghandler.h:
        New calls to get first theoretical and first real LSN.
      storage/maria/ma_loghandler_lsn.h:
        Defined yet another impossible LSN to indicate error.
      storage/maria/ma_recovery.c:
        The first LSN call changed.
      storage/maria/maria_read_log.c:
        The first LSN call changed.
      storage/maria/unittest/Makefile.am:
        New unittest added.
      storage/maria/unittest/ma_test_loghandler_first_lsn-t.c:
        New BitKeeper file ``storage/maria/unittest/ma_test_loghandler_first_lsn-t.c''
      4cf6756e
  28. 01 Aug, 2007 1 commit
    • unknown's avatar
      WL#3072 Maria recovery: · 372b4ed4
      unknown authored
      preparation to enable recoverability of the table's state, more exactly
      info->state->checksum. This will require that info->state->checksum
      be updated in a inwrite_hook when writing an UNDO record, thus
      info->cur_row.checksum needs to be accessible to inwrite_hook, so we
      make translog_write_record() accept a MARIA_HA* (info) instead of
      MARIA_SHARE* (info->s); with this, we will be able to access
      info->cur_row.checksum. Old code which needed the MARIA_SHARE
      can derive it from MARIA_HA. Fix for typos and compiler warnings.
      
      
      storage/maria/ma_blockrec.c:
        fix for new loghandler API. Removing strange lines (how could gcc
        accept that?)
      storage/maria/ma_check.c:
        fix for new loghandler API
      storage/maria/ma_delete_all.c:
        fix for new loghandler API
      storage/maria/ma_loghandler.c:
        functions now take a MARIA_HA in argument, this is more powerful
        than a MARIA_SHARE (MARIA_SHARE can be derived from MARIA_HA, not
        the other way around). MARIA_HA will be needed to allow recoverability
        of the table's state.
        Fixing wrong DBUG_PRINT ('i' is not the id).
        When writing the LOGREC_FILE_ID, we don't have a MARIA_HA around,
        so we cannot ask translog_write_record() to store the id for us;
        we thus store the file's id by ourselves. Alternative would have been
        to pass MARIA_HA to translog_assign_id_to_share() but I didn't like it.
      storage/maria/ma_loghandler.h:
        new loghandler API
      storage/maria/tablockman.c:
        fix for compiler warning (intptr is int on my machine)
      372b4ed4
  29. 30 Jul, 2007 1 commit
    • unknown's avatar
      Applying Sanja's patch which makes the log handler not issue · 2cccfcd8
      unknown authored
      errors when reading a log record which has a 0-length header
      (like LOGREC_REDO_DROP_TABLE).
      
      
      storage/maria/ma_loghandler.c:
        Functions reading record's header now don't use 0 to indicate error,
        as some valid records have a 0-length header (like REDO_DROP_TABLE).
        Instead, negative values are used for EOF and error.
      storage/maria/ma_loghandler.h:
        functions to read record's header now return an int
        (either the length of this header (>=0) or some negative values
        for EOF or error).
      storage/maria/ma_recovery.c:
        update to the new log handler's behaviour. Note the @todo.
      storage/maria/maria_read_log.c:
        inform when program failed
      storage/maria/unittest/ma_test_loghandler-t.c:
        update to new log handler's API
      storage/maria/unittest/ma_test_loghandler_multigroup-t.c:
        update to new log handler's API
      storage/maria/unittest/ma_test_loghandler_multithread-t.c:
        update to new log handler's API
      2cccfcd8
  30. 26 Jul, 2007 1 commit
    • unknown's avatar
      WL#3072 - Maria recovery · 6bbca54d
      unknown authored
      Unit test for recovery: runs ma_test1 and ma_test2 (both only with
      INSERTs and DELETEs; UPDATEs disabled as not handled by recovery)
      then moves the tables elswhere; recreates tables from the log, and
      compares and fails if there is a difference. Passes now.
      Most of maria_read_log.c moved to ma_recovery.c, as it will be re-used
      for recovery-from-ha_maria.
      Bugfixes of applying of REDO_INSERT, REDO_PURGE_ROW.
      Applying of REDO_PURGE_BLOCKS, REDO_DELETE_ALL, REDO_DROP_TABLE,
      UNDO_ROW_INSERT (in REDO phase only, i.e. just doing records++),
      UNDO_ROW_DELETE, UNDO_ROW_PURGE.
      Code cleanups.
      Monty: please look for "QQ". Sanja: please look for "Sanja".
      Future tasks: recovery of the bitmap (easy), recovery of the state
      (make it idempotent), more REDOs (Monty to work on
      REDO_UPDATE?), UNDO phase...
      Pushing this cset as it looks safe, contains test and bugfixes which
      will help Monty implement applying of REDO_UPDATE.
      
      
      sql/handler.cc:
        typo
      storage/maria/Makefile.am:
        Adding ma_test_recovery (which ma_test_all invokes, and which can
        also be run alone). Most of maria_read_log.c moved to ma_recovery.c
      storage/maria/ha_maria.cc:
        comments
      storage/maria/ma_bitmap.c:
        fixing comments. 2 -> sizeof(maria_bitmap_marker).
        Bitmap-related part of _ma_initialize_datafile() moves in bitmap module.
        Now putting the "bm" signature when creating the first bitmap page
        (it used to happen only at next open, but that
        caused an annoying difference when testing Recovery if the original
        run didn't open the table, and it looks more
        logical like this: it goes to disk only with its signature correct);
        see the "QQ" comment towards the _ma_initialize_data_file() call
        in ma_create.c for more).
        When reading a bitmap page, verify its signature (happens when normally
        using the table or when CHECKing it; not when REPAIRing it).
      storage/maria/ma_blockrec.c:
        * no need to sync the data file if table is not transactional
        * Comments, code cleanup (log-related data moved to log-related code
        block, int5store->page_store).
        * Store the table's short id into LOGREC_UNDO_ROW_PURGE, like we
        do for other records (though this record will soon be replaced
        with a CLR).
        * If "page" is 1 it means the page which extends from byte
        page*block_size+1 to (page+1)*block_size (byte number 1 being
        the first byte of the file). The last byte of the file is
        data_file_length (same convention).
        A new page needs to be created if the last byte of the page is
        beyond the last byte of the file, i.e.
         (page+1)*block_size+1 > data_file_length, so we correct the test
        (bug found when testing log applying for ma_test1 -M -T --skip-update).
        * update the page's LSN when removing a row from it during
        execution of a REDO_PURGE_ROW record (bug found when testing log
        applying for ma_test1 -M -T --skip-update).
        * applying of REDO_PURGE_BLOCKs (limited to a one-page range for now).
      storage/maria/ma_blockrec.h:
        new functions. maria_bitmap_marker does not need to be exported.
      storage/maria/ma_close.c:
        we can always flush the table's state when closing the last instance
        of the table. And it is needed for maria_read_log (as it does
        not use maria_lock_database()).
      storage/maria/ma_control_file.c:
        when in Recovery, some assertions should not be used.
      storage/maria/ma_control_file.h:
        double-inclusion safe
      storage/maria/ma_create.c:
        during recovery, don't log records. Comments.
        Moving the creation of the first bitmap page to ma_bitmap.c
      storage/maria/ma_delete_table.c:
        during recovery, don't log records. Log the end-zero of the dropped
        table's name, so that recovery can use the string in place without
        extending it to fit an end zero.
      storage/maria/ma_loghandler.c:
        * inwrite_rec_hook also needs access to the MARIA_SHARE, like
        prewrite_rec_hook. This will be needed to update
        share->records_diff (in the upcoming patch "recovery of the state").
        * LOG_DESC::record_ends_group changed to an enum.
        * LOG_DESC for LOGREC_REDO_PURGE_BLOCKS and LOGREC_UNDO_ROW_PURGE
        corrected
        * Sanja please see the @todo LOG BUG
        * avoiding DBUG_RETURN(func()) as it gives confusing debug traces.
      storage/maria/ma_loghandler.h:
        - log write hooks called while the log's lock is held (inwrite_rec_hook)
        now need the MARIA_SHARE, like prewrite_rec_hook already had
        - instead of a bool saying if this record's type ends groups or not,
        we refine: it may not end a group, it may end a group, or it may
        be a group in itself. Imagine that we had a physical write failure
        to a table before we log the UNDO, we still end up in
        external_lock(F_UNLCK) and then we log a COMMIT: we don't want
        to consider this COMMIT as ending the group of REDOs (don't want
        to execute those REDOs during Recovery), that's why we say "COMMIT
        is a group in itself, it aborts any previous group". This also
        gives one more sanity check in maria_read_log.
      storage/maria/ma_recovery.c:
        New Recovery code, replacing the old pseudocode.
        Most of maria_read_log moved here.
        Call-able from ha_maria, but not enabled yet.
        Compared to the previous version of maria_read_log, some bugs have
        been fixed, debugging output can go to stdout or a disk file (for now
        it's useful for me, later it can be changed), execution of
        REDO_DROP_TABLE, REDO_DELETE_ALL, REDO_PURGE_BLOCKS has been added. Duplicate code
        has been factored into functions. We abort an unfinished group
        of records if we see a record which is a group in itself (like COMMIT).
        No need for maria_panic() after a bug (which caused tables to not
        be closed) was fixed; if there is yet another bug I prefer to see it.
        When opening a table for Recovery, set data_file_length
        and key_file_length to their real physical value (these are the
        easiest state members to restore :). Warn us if the last page
        was truncated (but Recovery handles it).
        MARIA_SHARE::state::state::records is now partly recovered (not
        idempotent, but works if recreating tables from scracth).
        When applying a REDO to a page, stamp it with the UNDO's LSN
        (current_group_end_lsn), not with the REDO's LSN; it makes
        the table more identical to the original table (easier to compare
        the two tables in the end).
        Big thing missing: some types of REDOs are not handled,
        and the UNDO phase does not exist (missing functions to execute UNDOs
        to actually rollback). So for now tests are only inserting/deleting
        a few 100 rows, closing the table and seeing if the log is applied ok;
        it works. UPDATE not handled.
      storage/maria/ma_recovery.h:
        new functions: ma_recover() for recovery from inside ha_maria;
        _ma_apply_log() for maria_read_log (ma_recover() calls _ma_apply_log()).
        Btw, we need to not use the word "recover" for REPAIR/maria_chk anymore.
      storage/maria/ma_rename.c:
        don't write log records during recovery
      storage/maria/ma_test2.c:
        - fail if maria_info() or other subtests find some wrong information
        - new option -g to skip updates.
        - init the translog before creating the table, so that log applying
        can work.
        - in "#if 0" you'll see some fixed bugs (will be removed).
      storage/maria/ma_test_all.sh:
        cleanup files. Test log applying.
      storage/maria/maria_read_log.c:
        most of the logic moves to ma_recovery.c to be shared between
        maria_read_log and recovery-from-inside-mysqld.
        See ma_recovery.c for additional changes made to the moved code.
      storage/maria/ma_test_recovery:
        unit test for Recovery. Tests insert and delete,
        REDO_UPDATE not yet coded.
        Script is called from ma_test_all. Can run standalone.
      6bbca54d
  31. 04 Jul, 2007 1 commit
  32. 03 Jul, 2007 1 commit
    • unknown's avatar
      Implement applying of REDO entries for · 85da5133
      unknown authored
      - LOGREC_REDO_INSERT_ROW_HEAD
      - LOGREC_REDO_INSERT_ROW_TAIL
      - LOGREC_REDO_PURGE_ROW_HEAD
      - LOGREC_REDO_PURGE_ROW_TAIL
      
      
      sql/sql_yacc.yy:
        Fixed typo in previous push
      storage/maria/ma_bitmap.c:
        Ensure we flush the new bitmap on close
      storage/maria/ma_blockrec.c:
        Implement applying of REDO entries for
        - LOGREC_REDO_INSERT_ROW_HEAD
        - LOGREC_REDO_INSERT_ROW_TAIL
        - LOGREC_REDO_PURGE_ROW_HEAD
        - LOGREC_REDO_PURGE_ROW_TAIL
        Split some functions into subfunctions to be able to reuse code
      storage/maria/ma_blockrec.h:
        Added prototypes for REDO applying functions
      storage/maria/ma_loghandler.h:
        Safety fix
      storage/maria/ma_loghandler_lsn.h:
        Avoid compiler warnings
      storage/maria/maria_read_log.c:
        Added hocks for:
        - REDO_INSERT_ROW_HEAD
        - REDO_INSERT_ROW_TAIL
        - REDO_PURGE_ROW_HEAD
        - REDO_PURGE_ROW_TAIL
        
        Added dummy hooks for:
        - UNDO_ROW_INSERT
        - UNDO_ROW_DELETE
        
        Changed to use maria_pagecache instead of own pagecache (fixed problem with unitialized share->pagecache)
        Use maria_panic() at end to ensure that all files are closed properly.
        Fixed option handling for --debug
      85da5133
  33. 02 Jul, 2007 1 commit
    • unknown's avatar
      Merged with mysql-5.1 main tree. · 631ecaab
      unknown authored
      BUILD/compile-pentium-debug-max:
        Added definition after macro was removed from main tree. This will
        be fixed back in main tree later.
      631ecaab
  34. 26 Jun, 2007 1 commit
    • unknown's avatar
      WL#3072 - Maria recovery · 1e73169a
      unknown authored
      fixes for build failures; copyrights; small bugfixes and comments
      
      
      mysys/Makefile.am:
        missing .h breaks building from tarball
      storage/maria/ma_loghandler.c:
        applying Serg's bugfix of trnman_new_trid() to translog_assign_id_to_share()
      storage/maria/ma_loghandler.h:
        copyright
      storage/maria/ma_loghandler_lsn.h:
        copyright
      storage/maria/maria_read_log.c:
        fix for compiler warnings. Comments.
        Close tables when program ends.
      1e73169a