An error occurred fetching the project authors.
  1. 26 Jul, 2007 1 commit
    • unknown's avatar
      WL#3072 - Maria recovery · fc08f82b
      unknown authored
      Unit test for recovery: runs ma_test1 and ma_test2 (both only with
      INSERTs and DELETEs; UPDATEs disabled as not handled by recovery)
      then moves the tables elswhere; recreates tables from the log, and
      compares and fails if there is a difference. Passes now.
      Most of maria_read_log.c moved to ma_recovery.c, as it will be re-used
      for recovery-from-ha_maria.
      Bugfixes of applying of REDO_INSERT, REDO_PURGE_ROW.
      Applying of REDO_PURGE_BLOCKS, REDO_DELETE_ALL, REDO_DROP_TABLE,
      UNDO_ROW_INSERT (in REDO phase only, i.e. just doing records++),
      UNDO_ROW_DELETE, UNDO_ROW_PURGE.
      Code cleanups.
      Monty: please look for "QQ". Sanja: please look for "Sanja".
      Future tasks: recovery of the bitmap (easy), recovery of the state
      (make it idempotent), more REDOs (Monty to work on
      REDO_UPDATE?), UNDO phase...
      Pushing this cset as it looks safe, contains test and bugfixes which
      will help Monty implement applying of REDO_UPDATE.
      
      
      sql/handler.cc:
        typo
      storage/maria/Makefile.am:
        Adding ma_test_recovery (which ma_test_all invokes, and which can
        also be run alone). Most of maria_read_log.c moved to ma_recovery.c
      storage/maria/ha_maria.cc:
        comments
      storage/maria/ma_bitmap.c:
        fixing comments. 2 -> sizeof(maria_bitmap_marker).
        Bitmap-related part of _ma_initialize_datafile() moves in bitmap module.
        Now putting the "bm" signature when creating the first bitmap page
        (it used to happen only at next open, but that
        caused an annoying difference when testing Recovery if the original
        run didn't open the table, and it looks more
        logical like this: it goes to disk only with its signature correct);
        see the "QQ" comment towards the _ma_initialize_data_file() call
        in ma_create.c for more).
        When reading a bitmap page, verify its signature (happens when normally
        using the table or when CHECKing it; not when REPAIRing it).
      storage/maria/ma_blockrec.c:
        * no need to sync the data file if table is not transactional
        * Comments, code cleanup (log-related data moved to log-related code
        block, int5store->page_store).
        * Store the table's short id into LOGREC_UNDO_ROW_PURGE, like we
        do for other records (though this record will soon be replaced
        with a CLR).
        * If "page" is 1 it means the page which extends from byte
        page*block_size+1 to (page+1)*block_size (byte number 1 being
        the first byte of the file). The last byte of the file is
        data_file_length (same convention).
        A new page needs to be created if the last byte of the page is
        beyond the last byte of the file, i.e.
         (page+1)*block_size+1 > data_file_length, so we correct the test
        (bug found when testing log applying for ma_test1 -M -T --skip-update).
        * update the page's LSN when removing a row from it during
        execution of a REDO_PURGE_ROW record (bug found when testing log
        applying for ma_test1 -M -T --skip-update).
        * applying of REDO_PURGE_BLOCKs (limited to a one-page range for now).
      storage/maria/ma_blockrec.h:
        new functions. maria_bitmap_marker does not need to be exported.
      storage/maria/ma_close.c:
        we can always flush the table's state when closing the last instance
        of the table. And it is needed for maria_read_log (as it does
        not use maria_lock_database()).
      storage/maria/ma_control_file.c:
        when in Recovery, some assertions should not be used.
      storage/maria/ma_control_file.h:
        double-inclusion safe
      storage/maria/ma_create.c:
        during recovery, don't log records. Comments.
        Moving the creation of the first bitmap page to ma_bitmap.c
      storage/maria/ma_delete_table.c:
        during recovery, don't log records. Log the end-zero of the dropped
        table's name, so that recovery can use the string in place without
        extending it to fit an end zero.
      storage/maria/ma_loghandler.c:
        * inwrite_rec_hook also needs access to the MARIA_SHARE, like
        prewrite_rec_hook. This will be needed to update
        share->records_diff (in the upcoming patch "recovery of the state").
        * LOG_DESC::record_ends_group changed to an enum.
        * LOG_DESC for LOGREC_REDO_PURGE_BLOCKS and LOGREC_UNDO_ROW_PURGE
        corrected
        * Sanja please see the @todo LOG BUG
        * avoiding DBUG_RETURN(func()) as it gives confusing debug traces.
      storage/maria/ma_loghandler.h:
        - log write hooks called while the log's lock is held (inwrite_rec_hook)
        now need the MARIA_SHARE, like prewrite_rec_hook already had
        - instead of a bool saying if this record's type ends groups or not,
        we refine: it may not end a group, it may end a group, or it may
        be a group in itself. Imagine that we had a physical write failure
        to a table before we log the UNDO, we still end up in
        external_lock(F_UNLCK) and then we log a COMMIT: we don't want
        to consider this COMMIT as ending the group of REDOs (don't want
        to execute those REDOs during Recovery), that's why we say "COMMIT
        is a group in itself, it aborts any previous group". This also
        gives one more sanity check in maria_read_log.
      storage/maria/ma_recovery.c:
        New Recovery code, replacing the old pseudocode.
        Most of maria_read_log moved here.
        Call-able from ha_maria, but not enabled yet.
        Compared to the previous version of maria_read_log, some bugs have
        been fixed, debugging output can go to stdout or a disk file (for now
        it's useful for me, later it can be changed), execution of
        REDO_DROP_TABLE, REDO_DELETE_ALL, REDO_PURGE_BLOCKS has been added. Duplicate code
        has been factored into functions. We abort an unfinished group
        of records if we see a record which is a group in itself (like COMMIT).
        No need for maria_panic() after a bug (which caused tables to not
        be closed) was fixed; if there is yet another bug I prefer to see it.
        When opening a table for Recovery, set data_file_length
        and key_file_length to their real physical value (these are the
        easiest state members to restore :). Warn us if the last page
        was truncated (but Recovery handles it).
        MARIA_SHARE::state::state::records is now partly recovered (not
        idempotent, but works if recreating tables from scracth).
        When applying a REDO to a page, stamp it with the UNDO's LSN
        (current_group_end_lsn), not with the REDO's LSN; it makes
        the table more identical to the original table (easier to compare
        the two tables in the end).
        Big thing missing: some types of REDOs are not handled,
        and the UNDO phase does not exist (missing functions to execute UNDOs
        to actually rollback). So for now tests are only inserting/deleting
        a few 100 rows, closing the table and seeing if the log is applied ok;
        it works. UPDATE not handled.
      storage/maria/ma_recovery.h:
        new functions: ma_recover() for recovery from inside ha_maria;
        _ma_apply_log() for maria_read_log (ma_recover() calls _ma_apply_log()).
        Btw, we need to not use the word "recover" for REPAIR/maria_chk anymore.
      storage/maria/ma_rename.c:
        don't write log records during recovery
      storage/maria/ma_test2.c:
        - fail if maria_info() or other subtests find some wrong information
        - new option -g to skip updates.
        - init the translog before creating the table, so that log applying
        can work.
        - in "#if 0" you'll see some fixed bugs (will be removed).
      storage/maria/ma_test_all.sh:
        cleanup files. Test log applying.
      storage/maria/maria_read_log.c:
        most of the logic moves to ma_recovery.c to be shared between
        maria_read_log and recovery-from-inside-mysqld.
        See ma_recovery.c for additional changes made to the moved code.
      storage/maria/ma_test_recovery:
        unit test for Recovery. Tests insert and delete,
        REDO_UPDATE not yet coded.
        Script is called from ma_test_all. Can run standalone.
      fc08f82b
  2. 02 Mar, 2007 1 commit
    • unknown's avatar
      GPL license update (same change as was done for all files in 5.1). · ee984206
      unknown authored
      storage/maria/Makefile.am:
        GPL license update
      storage/maria/ft_maria.c:
        GPL license update
      storage/maria/ha_maria.cc:
        GPL license update
      storage/maria/ha_maria.h:
        GPL license update
      storage/maria/lockman.c:
        GPL license update
      storage/maria/lockman.h:
        GPL license update
      storage/maria/ma_bitmap.c:
        GPL license update
      storage/maria/ma_blockrec.c:
        GPL license update
      storage/maria/ma_blockrec.h:
        GPL license update
      storage/maria/ma_cache.c:
        GPL license update
      storage/maria/ma_changed.c:
        GPL license update
      storage/maria/ma_check.c:
        GPL license update
      storage/maria/ma_checkpoint.c:
        GPL license update
      storage/maria/ma_checkpoint.h:
        GPL license update
      storage/maria/ma_checksum.c:
        GPL license update
      storage/maria/ma_close.c:
        GPL license update
      storage/maria/ma_control_file.c:
        GPL license update
      storage/maria/ma_control_file.h:
        GPL license update
      storage/maria/ma_create.c:
        GPL license update
      storage/maria/ma_dbug.c:
        GPL license update
      storage/maria/ma_delete.c:
        GPL license update
      storage/maria/ma_delete_all.c:
        GPL license update
      storage/maria/ma_delete_table.c:
        GPL license update
      storage/maria/ma_dynrec.c:
        GPL license update
      storage/maria/ma_extra.c:
        GPL license update
      storage/maria/ma_ft_boolean_search.c:
        GPL license update
      storage/maria/ma_ft_eval.c:
        GPL license update
      storage/maria/ma_ft_eval.h:
        GPL license update
      storage/maria/ma_ft_nlq_search.c:
        GPL license update
      storage/maria/ma_ft_parser.c:
        GPL license update
      storage/maria/ma_ft_stem.c:
        GPL license update
      storage/maria/ma_ft_test1.c:
        GPL license update
      storage/maria/ma_ft_test1.h:
        GPL license update
      storage/maria/ma_ft_update.c:
        GPL license update
      storage/maria/ma_ftdefs.h:
        GPL license update
      storage/maria/ma_fulltext.h:
        GPL license update
      storage/maria/ma_info.c:
        GPL license update
      storage/maria/ma_init.c:
        GPL license update
      storage/maria/ma_key.c:
        GPL license update
      storage/maria/ma_keycache.c:
        GPL license update
      storage/maria/ma_least_recently_dirtied.c:
        GPL license update
      storage/maria/ma_least_recently_dirtied.h:
        GPL license update
      storage/maria/ma_locking.c:
        GPL license update
      storage/maria/ma_open.c:
        GPL license update
      storage/maria/ma_packrec.c:
        GPL license update
      storage/maria/ma_page.c:
        GPL license update
      storage/maria/ma_panic.c:
        GPL license update
      storage/maria/ma_preload.c:
        GPL license update
      storage/maria/ma_range.c:
        GPL license update
      storage/maria/ma_recovery.c:
        GPL license update
      storage/maria/ma_recovery.h:
        GPL license update
      storage/maria/ma_rename.c:
        GPL license update
      storage/maria/ma_rfirst.c:
        GPL license update
      storage/maria/ma_rkey.c:
        GPL license update
      storage/maria/ma_rlast.c:
        GPL license update
      storage/maria/ma_rnext.c:
        GPL license update
      storage/maria/ma_rnext_same.c:
        GPL license update
      storage/maria/ma_rprev.c:
        GPL license update
      storage/maria/ma_rrnd.c:
        GPL license update
      storage/maria/ma_rsame.c:
        GPL license update
      storage/maria/ma_rsamepos.c:
        GPL license update
      storage/maria/ma_rt_index.c:
        GPL license update
      storage/maria/ma_rt_index.h:
        GPL license update
      storage/maria/ma_rt_key.c:
        GPL license update
      storage/maria/ma_rt_key.h:
        GPL license update
      storage/maria/ma_rt_mbr.c:
        GPL license update
      storage/maria/ma_rt_mbr.h:
        GPL license update
      storage/maria/ma_rt_split.c:
        GPL license update
      storage/maria/ma_rt_test.c:
        GPL license update
      storage/maria/ma_scan.c:
        GPL license update
      storage/maria/ma_search.c:
        GPL license update
      storage/maria/ma_sort.c:
        GPL license update
      storage/maria/ma_sp_defs.h:
        GPL license update
      storage/maria/ma_sp_key.c:
        GPL license update
      storage/maria/ma_sp_test.c:
        GPL license update
      storage/maria/ma_static.c:
        GPL license update
      storage/maria/ma_statrec.c:
        GPL license update
      storage/maria/ma_test1.c:
        GPL license update
      storage/maria/ma_test2.c:
        GPL license update
      storage/maria/ma_test3.c:
        GPL license update
      storage/maria/ma_unique.c:
        GPL license update
      storage/maria/ma_update.c:
        GPL license update
      storage/maria/ma_write.c:
        GPL license update
      storage/maria/maria_chk.c:
        GPL license update
      storage/maria/maria_def.h:
        GPL license update
      storage/maria/maria_ftdump.c:
        GPL license update
      storage/maria/maria_pack.c:
        GPL license update
      storage/maria/tablockman.c:
        GPL license update
      storage/maria/tablockman.h:
        GPL license update
      storage/maria/trnman.c:
        GPL license update
      storage/maria/trnman.h:
        GPL license update
      ee984206
  3. 14 Sep, 2006 1 commit
    • unknown's avatar
      WL#3071 Maria checkpoint: · 382b81d3
      unknown authored
          changing pseudocode to use the structures of the Maria pagecache
          ("pagecache->changed_blocks" etc) and other Maria structures
          inherited from MyISAM (THR_LOCK_maria etc).
      
      
      mysys/mf_pagecache.c:
        comment
      storage/maria/ma_checkpoint.c:
        changing pseudocode to use the structures of the Maria pagecache
        ("pagecache->changed_blocks" etc) and other Maria structures
        inherited from MyISAM (THR_LOCK_maria etc).
      storage/maria/ma_checkpoint.h:
        copyright
      storage/maria/ma_control_file.c:
        copyright
      storage/maria/ma_control_file.h:
        copyright
      storage/maria/ma_least_recently_dirtied.c:
        copyright
      storage/maria/ma_least_recently_dirtied.h:
        copyright
      storage/maria/ma_recovery.c:
        copyright
      storage/maria/ma_recovery.h:
        copyright
      storage/maria/unittest/Makefile.am:
        copyright
      382b81d3
  4. 01 Sep, 2006 1 commit
    • unknown's avatar
      WL#3234 "Maria - control file manager" · 6590935a
      unknown authored
      - fixes to the control file module
      - unit test for it
      - renames of all Maria files I created to start with ma_
      
      
      storage/maria/ma_checkpoint.c:
        Rename: storage/maria/checkpoint.c -> storage/maria/ma_checkpoint.c
      storage/maria/ma_checkpoint.h:
        Rename: storage/maria/checkpoint.h -> storage/maria/ma_checkpoint.h
      storage/maria/ma_least_recently_dirtied.c:
        Rename: storage/maria/least_recently_dirtied.c -> storage/maria/ma_least_recently_dirtied.c
      storage/maria/ma_least_recently_dirtied.h:
        Rename: storage/maria/least_recently_dirtied.h -> storage/maria/ma_least_recently_dirtied.h
      storage/maria/ma_recovery.c:
        Rename: storage/maria/recovery.c -> storage/maria/ma_recovery.c
      storage/maria/ma_recovery.h:
        Rename: storage/maria/recovery.h -> storage/maria/ma_recovery.h
      storage/maria/Makefile.am:
        control file module and its unit test program
      storage/maria/ma_control_file.c:
        DBUG_ tags. Fix for gcc warnings.
        log_no -> logno (I felt "_no" sounded like a standalone "No" word).
        ma_ prefix for some functions.
        last_checkpoint_lsn_at_startup -> last_checkpoint_lsn (no need
        to make special vars for the values at startup). Same for last_logno.
        ma_control_file_write_and_force() now updates last_checkpoint_lsn
        and last_logno, the idea being that they belong to the module,
        others should not update them.
        And thus when the module shuts down, it zeroes those vars.
      storage/maria/ma_control_file.h:
        importing structs from Sanja to get the control file module to compile;
        we'll remove that when Sanja pushes the log handler.
        CONTROL_FILE_IMPOSSIBLE_LOGNO is 0, not FFFFFFFF.
      storage/maria/ma_control_file_test.c:
        Unit test program for the Maria control file module.
        Modelled after other ma_test* files in this directory (so, does
        not follow the unit test framework recently introduced with libtap;
        TODO as a task on all ma_test* programs).
        We test that writing to the control file works, and re-reading from it
        too, we check (by reading the file by ourselves) that its content
        on disk is correct, and check that a corrupted control file is detected.
      6590935a
  5. 29 Aug, 2006 1 commit
    • unknown's avatar
      just comment changes and line formatting changes. · 2a15d49a
      unknown authored
      storage/maria/checkpoint.c:
        comments
      storage/maria/least_recently_dirtied.c:
        comments
      storage/maria/ma_check.c:
        line formatting changes neglected from my last merge
      storage/maria/ma_ft_parser.c:
        it reduces the diff of MyISAM vs Maria :)
      storage/maria/recovery.c:
        comments
      2a15d49a
  6. 26 Jul, 2006 1 commit
  7. 04 May, 2006 1 commit
  8. 27 Apr, 2006 1 commit
    • unknown's avatar
      Maria: first version of checkpoint (WL#3071), least-recently-dirtied page... · 81cb63ec
      unknown authored
      Maria: first version of checkpoint (WL#3071), least-recently-dirtied page flushing (WL#3261), recovery (WL#3072),
      control file (WL#3234), to serve as a detailed LLD. It looks like C code, but does not compile (no point in making it compile,
      as other modules on which I depend are not yet fully speficied or written); some pieces are not coded and just marked in comments.
      Files' organization (names, directories of C files) does not matter at this point.
      I don't think I had to commit so early, but it feels good to publish something, gives me the impression of moving forward :)
      
      
      storage/maria/checkpoint.c:
        WL#3071 Maria checkpoint, implementation
      storage/maria/checkpoint.h:
        WL#3071 Maria checkpoint, interface
      storage/maria/control_file.c:
        WL#3234 Maria control file, implementation
      storage/maria/control_file.h:
        WL#3234 Maria control file, interface
      storage/maria/least_recently_dirtied.c:
        WL#3261 Maria background flushing of least-recently-dirtied pages, implementation
      storage/maria/least_recently_dirtied.h:
        WL#3261 Maria background flushing of least-recently-dirtied pages, interface
      storage/maria/recovery.c:
        WL#3072 Maria recovery, implementation
      storage/maria/recovery.h:
        WL#3072 Maria recovery, interface
      81cb63ec