1. 14 Nov, 2007 1 commit
    • unknown's avatar
      First part of redo/undo for key pages · 21fd2a5a
      unknown authored
      Added key_nr to st_maria_keydef for faster keyinfo->keynr conversion
      For transactional tables, shift record number in keys up with 1 bit to have place to indicate if transid follows
      Checksum for MyISAM now ignores NULL and not used part of VARCHAR
      Renamed some variables that caused shadow compiler warnings
      Moved extra() call when waiting for tables to not be used to after tables are removed from cache.
      Fixed crashing bugs when using Maria TEMPORARY tables with TRUNCATE. Removed 'hack' code in sql directory to go around this bug.
      pagecache_unlock_by_ulink() now has extra argument to say if page was changed.
      Give error message if we fail to open control file
      Mark page cache variables as not flushable
      
      
      include/maria.h:
        Made min page cache larger (needed for pinning key page)
        Added key_nr to st_maria_keydef for faster keyinfo->keynr conversion
        Added write_comp_flag to move some runtime code to maria_open()
      include/my_base.h:
        Added new error message to be used when handler initialization failed
      include/my_global.h:
        Renamed dummy to swap_dummy to avoid conflicts with local 'dummy' variables
      include/my_handler.h:
        Added const to some parameters
      mysys/array.c:
        More DBUG
      mysys/my_error.c:
        Fixed indentation
      mysys/my_handler.c:
        Added const to some parameters
        Added missing error messages
      sql/field.h:
        Renamed variables to avoid variable shadowing
      sql/handler.h:
        Renamed parameter to avoid variable name conflict
      sql/item.h:
        Renamed variables to avoid variable shadowing
      sql/log_event_old.h:
        Renamed variables to avoid variable shadowing
      sql/set_var.h:
        Renamed variables to avoid variable shadowing
      sql/sql_delete.cc:
        Removed maria hack for temporary tables
        Fixed indentation
      sql/sql_table.cc:
        Moved extra() call when waiting for tables to not be used to after tables are removed from cache.
        This was needed to ensure we don't do a PREPARE_FOR_DROP or similar call while the table is still in use.
      sql/table.cc:
        Copy page_checksum from share
        Removed Maria hack
      storage/maria/Makefile.am:
        Added new files
      storage/maria/ha_maria.cc:
        Renamed records -> record_count and info -> create_info to avoid variable name conflicts
        Mark page cache variables as not flushable
      storage/maria/ma_blockrec.c:
        Moved _ma_unpin_all_pages() to ma_key_recover.c
        Moved init of info->pinned_pages to ma_open.c
        Moved _ma_finalize_row() to maria_key_recover.h
        Renamed some variables to avoid variable name conflicts
        Mark page_link.changed for blocks we change directly
        Simplify handling of undo link when writing LOGREC_UNDO_ROW_INSERT (old code crashed when having redo for index)
      storage/maria/ma_blockrec.h:
        Removed extra empty line
      storage/maria/ma_checkpoint.c:
        Remove not needed trnman.h
      storage/maria/ma_close.c:
        Free pinned pages (which are now always allocated)
      storage/maria/ma_control_file.c:
        Give error message if we fail to open control file
      storage/maria/ma_delete.c:
        Changes for redo logging (first part, logging of underflow not yet done)
        - Log undo-key-delete
        - Log delete of key
        - Updated arguments to _ma_fetch_keypage(), _ma_dispose(), _ma_write_keypage(), _ma_insert()
        - Added new arguments to some functions to be able to write redo information
        - Mark key pages as changed when we write with PAGECACHE_LOCK_LEFT_WRITELOCKED
        
        Remove one not needed _ma_write_keypage() in d_search() when upper level will do the write anyway
        Changed 2 bmove_upp() to bmove() as this made code easer to understand
        More function comments
        Indentation fixes
      storage/maria/ma_ft_update.c:
        New arguments to _ma_write_keypage()
      storage/maria/ma_loghandler.c:
        Fixed some DBUG_PRINT messages
        Simplify code
        Added new log entrys for key page redo
        Renamed some variables to avoid variable name shadowing
      storage/maria/ma_loghandler.h:
        Moved some defines here
        Added define for storing key number on key pages
        Added new translog record types
        Added enum for type of operations in LOGREC_REDO_INDEX
      storage/maria/ma_open.c:
        Always allocate info.pinned_pages (we need now also for normal key page usage)
        Update keyinfo->key_nr
        Added virtual functions to convert record position o number to be stored on key pages
        Update keyinfo->write_comp_flag to value of search flag to be used when writing key
      storage/maria/ma_page.c:
        Added redo for key pages
        - Extended _ma_fetch_keypage() with type of lock to put on page and address to used MARIA_PINNED_PAGE
        - _ma_fetch_keypage() now pin's pages if needed
        - Extended _ma_write_keypage() with type of locks to be used
        - ma_dispose() now locks info->s->state.key_del from other threads
        - ma_dispose() writes redo log record
        - ma_new() locks info->s->state.key_del from other threads if it was used
        - ma_new() now pins read page
        
        Other things:
        - Removed some not needed arguments from _ma_new() and _ma_dispose)
        - Added some new variables to simplify code
        - If EXTRA_DEBUG is used, do crc on full page to catch not unitialized bytes
      storage/maria/ma_pagecache.h:
        Applied patch from Sanja to add extra argument to pagecache_unlock_by_ulink() to mark if page was changed
        Added some defines for pagecache priority levels that one can use
      storage/maria/ma_range.c:
        Added new arguments for call to _ma_fetch_keypage()
      storage/maria/ma_recovery.c:
        - Added hooks for new translog types:
          REDO_INDEX, REDO_INDEX_NEW_PAGE, REDO_INDEX_FREE_PAGE, UNDO_KEY_INSERT, UNDO_KEY_DELETE and
          UNDO_KEY_DELETE_WITH_ROOT.
        - Moved variable declarations to start of function (portability fixes)
        - Removed some not needed initializations
        - Set only relevant state changes for each redo/undo entry
      storage/maria/lockman.c:
        Removed end space
      storage/maria/ma_check.c:
        Removed end space
      storage/maria/ma_create.c:
        Removed end space
      storage/maria/ma_locking.c:
        Removed end space
      storage/maria/ma_packrec.c:
        Removed end space
      storage/maria/ma_pagecache.c:
        Removed end space
      storage/maria/ma_panic.c:
        Removed end space
      storage/maria/ma_rt_index.c:
        Added new arguments for call to _ma_fetch_keypage(), _ma_write_keypage(), _ma_dispose() and _ma_new()
        Fixed indentation
      storage/maria/ma_rt_key.c:
        Added new arguments for call to _ma_fetch_keypage()
      storage/maria/ma_rt_split.c:
        Added new arguments for call to _ma_new()
        Use new keypage header
        Added new arguments for call to _ma_write_keypage()
      storage/maria/ma_search.c:
        Updated comments & indentation
        Added new arguments for call to _ma_fetch_keypage()
        Made some variables and arguments const
        Added virtual functions for converting row position to number to be stored in key
        use MARIA_RECORD_POS of record position instead of my_off_t
        Record in MARIA_KEY_PARAM how page was changed one key insert (needed for REDO)
      storage/maria/ma_sort.c:
        Removed end space
      storage/maria/ma_statrec.c:
        Updated arguments for call to _ma_rec_pos()
      storage/maria/ma_test1.c:
        Fixed too small buffer to init_pagecache()
        Fixed bug when using insert_count and test_flag
      storage/maria/ma_test2.c:
        Use more resonable pagecache size
        Remove not used code
        Reset blob_length to fix wrong output message
      storage/maria/ma_test_all.sh:
        Fixed wrong test
      storage/maria/ma_write.c:
        Lots of new code to handle REDO of key pages
        No logic changes because of REDO code, mostly adding new arguments and adding new code for logging 
        
        Added new arguments for calls to _ma_fetch_keypage(), _ma_write_keypage() and similar functions
        Move setting of comp_flag in ma_ck_wrte_btree() from runtime to maria_open()
        Zerofill new used pages for:
        - To remove possible sensitive data left in buffer
        - To get idenitical data on pages after running redo
        - Better compression of pages if archived
      storage/maria/maria_chk.c:
        Added information if table is crash safe
      storage/maria/maria_def.h:
        New virtual function to convert between record position on key and normal record position
        Aded mutex and extra variables to handle locking of share->state.key_del
        Moved some structure variables to get things more aligned
        Added extra arguments to MARIA_KEY_PARAM to be able to remember what was changed on key page on key insert
        Added argument to MARIA_PINNED_PAGE to indicate if page was changed
        Updated prototypes for functions
        Added some structures for signaling changes in REDO handling
      storage/maria/unittest/ma_pagecache_single.c:
        Updated arguments for changed function calls
      storage/myisam/mi_check.c:
        Made calc_check_checksum virtual
      storage/myisam/mi_checksum.c:
        Update checksums to ignore null columns
      storage/myisam/mi_create.c:
        Mark if table has null column (to know when we have to use mi_checksum())
      storage/myisam/mi_open.c:
        Added virtual function for calculating checksum to be able to easily ignore NULL fields
      storage/myisam/mi_test2.c:
        Fixed bug
      storage/myisam/myisamdef.h:
        Added virtual function for calculating checksum during check table
        Removed ha_key_cmp() as this is in handler.h
      storage/maria/ma_key_recover.c:
        New BitKeeper file ``storage/maria/ma_key_recover.c''
      storage/maria/ma_key_recover.h:
        New BitKeeper file ``storage/maria/ma_key_recover.h''
      storage/maria/ma_key_redo.c:
        New BitKeeper file ``storage/maria/ma_key_redo.c''
      21fd2a5a
  2. 19 Oct, 2007 5 commits
    • unknown's avatar
      Merge bk-internal.mysql.com:/home/bk/mysql-maria · 301ee8d9
      unknown authored
      into  mysql.com:/home/my/mysql-maria
      
      
      include/my_sys.h:
        Auto merged
      mysql-test/r/maria.result:
        Auto merged
      mysql-test/t/maria.test:
        Auto merged
      sql/handler.h:
        Auto merged
      sql/mysqld.cc:
        Auto merged
      storage/maria/ha_maria.cc:
        Auto merged
      storage/maria/ma_bitmap.c:
        Auto merged
      storage/maria/ma_blockrec.c:
        Auto merged
      storage/maria/ma_loghandler.c:
        Auto merged
      storage/maria/ma_pagecache.c:
        Auto merged
      storage/maria/ma_test1.c:
        Auto merged
      storage/maria/ma_test_recovery.expected:
        Auto merged
      storage/maria/ma_test_recovery:
        Auto merged
      sql/mysql_priv.h:
        manual merge
      storage/maria/ma_recovery.c:
        manual merge
      storage/maria/ma_test2.c:
        manual merge
      301ee8d9
    • unknown's avatar
      Merge some changes from sql directory in 5.1 tree · 13d53bf6
      unknown authored
      Changed format for REDO_INSERT_ROWS_BLOBS
      Fixed several bugs in handling of big blobs
      Added redo_free_head_or_tail() & redo_insert_row_blobs()
      Added uuid to control file
      maria_checks now verifies that not used part of bitmap is 0
      REDO_PURGE_BLOCKS -> REDO_FREE_BLOCKS
      Added REDO_FREE_HEAD_OR_TAIL
      Fixes problem when trying to read block outside of file during REDO
      
      
      include/my_global.h:
        STACK_DIRECTION is already set by configure
      mysql-test/r/maria.result:
        Updated results
      mysql-test/t/maria.test:
        Test shrinking of VARCHAR
      mysys/my_realloc.c:
        Fixed indentation
      mysys/safemalloc.c:
        Fixed indentation
      sql/filesort.cc:
        Removed some casts
      sql/mysqld.cc:
        Added missing setting of myisam_stats_method_str
      sql/uniques.cc:
        Removed some casts
      storage/maria/ma_bitmap.c:
        Added printing of bitmap (for debugging)
        Renamed _ma_print_bitmap() -> _ma_print_bitmap_changes()
        Added _ma_set_full_page_bits()
        Fixed bug in ma_bitmap_find_new_place() (affecting updates) when using big files
      storage/maria/ma_blockrec.c:
        Changed format for REDO_INSERT_ROWS_BLOBS
        Fixed several bugs in handling of big blobs
        Added code to fix some cases where redo when using blobs didn't produce idenital .MAD files as normal usage
        REDO_FREE_ROW_BLOCKS doesn't anymore change pages; We only mark things free in bitmap
        Remove TAIL and filler extents from REDO_FREE_BLOCKS log entry. (Fixed some asserts)
        REDO_PURGE_BLOCKS -> REDO_FREE_BLOCKS
        Delete tails in update. (Fixed bug when doing update that shrinks blob/varchar length)
        Fixed bug when doing insert in block outside of file size.
        Added redo_free_head_or_tail() & redo_insert_row_blobs()
        Added pagecache_unlock_by_link() when read fails.
        Much more comments, DBUG and ASSERT entries
      storage/maria/ma_blockrec.h:
        Prototypes of new functions
        Define of SUB_RANGE_SIZE & BLOCK_FILLER_SIZE
      storage/maria/ma_check.c:
        Verify that not used part of bitmap is 0
      storage/maria/ma_control_file.c:
        Added uuid to control file
      storage/maria/ma_loghandler.c:
        REDO_PURGE_BLOCKS -> REDO_FREE_BLOCKS
        Added REDO_FREE_HEAD_OR_TAIL
      storage/maria/ma_loghandler.h:
        REDO_PURGE_BLOCKS -> REDO_FREE_BLOCKS
        Added REDO_FREE_HEAD_OR_TAIL
      storage/maria/ma_pagecache.c:
        If we write full block, remove error flag for block.
        (Fixes problem when trying to read block outside of file)
      storage/maria/ma_recovery.c:
        REDO_PURGE_BLOCKS -> REDO_FREE_BLOCKS
        Added REDO_FREE_HEAD_OR_TAIL
      storage/maria/ma_test1.c:
        Allow option after 'b' to be compatible with ma_test2
        (This is just to simplify test scripts like ma_test_recovery)
      storage/maria/ma_test2.c:
        Default size of blob is now 1000 instead of 1
      storage/maria/ma_test_all.sh:
        Added test for bigger blobs
      storage/maria/ma_test_recovery.expected:
        Updated results
      storage/maria/ma_test_recovery:
        Added test for bigger blobs
      13d53bf6
    • unknown's avatar
      WL#3071 - Maria checkpoint · 1cede432
      unknown authored
      don't take a checkpoint at end of REDO phase because blocks' rec_lsn
      are not set. Fixes some assertion failures during tests of killing
      the UNDO phase (recovery then started from the end-of-REDO-phase
      checkpoint, wrongly ignoring REDOs before that record).
      
      
      storage/maria/ma_blockrec.c:
        note for a bug (mail sent to colleagues, put on todo)
      storage/maria/ma_recovery.c:
        this checkpoint at end of REDO phase is incorrect, disabling it.
        The one at end of UNDO phase remains.
      1cede432
    • unknown's avatar
      Merge gbichot@bk-internal.mysql.com:/home/bk/mysql-maria · 155365f0
      unknown authored
      into  gbichot4.local:/home/mysql_src/mysql-maria-for-undo-phase
      
      
      155365f0
    • unknown's avatar
      WL#3071 - Maria checkpoint · 77017191
      unknown authored
      - serializing calls to flush_pagecache_blocks_int() on the same file
      to avoid known concurrency bugs
      - having that, we can now enable the background thread, as the
      flushes it does are now supposedly safe in concurrent situations.
      - new type of flush FLUSH_KEEP_LAZY: when the background checkpoint
      thread is flushing a packet of dirty pages between two checkpoints,
      it uses this flush type, indeed if a file is already being flushed
      by another thread it's smarter to move on to the next file than wait.
      - maria_checkpoint_frequency renamed to maria_checkpoint_interval.
      
      
      include/my_sys.h:
        new type of flushing for the page cache: FLUSH_KEEP_LAZY
      mysql-test/r/maria.result:
        result update
      mysys/mf_keycache.c:
        indentation. No FLUSH_KEEP_LAZY support in key cache.
      storage/maria/ha_maria.cc:
        maria_checkpoint_frequency was somehow a hidden part of the
        Checkpoint API and that was not good. Now we have checkpoint_interval,
        local to ha_maria.cc, which serves as container for the user-visible
        maria_checkpoint_interval global variable; setting it calls
        update_checkpoint_interval which passes the new value to
        ma_checkpoint_init(). There is no hiding anymore.
        By default, enable background thread which does checkpoints
        every 30 seconds, and dirty page flush in between. That thread takes
        a checkpoint when it ends, so no need for maria_hton_panic to take one.
        The | is | and not ||, because maria_panic() must always be called.
        frequency->interval.
      storage/maria/ma_checkpoint.c:
        Use FLUSH_KEEP_LAZY for background thread when it flushes packets of
        dirty pages between two checkpoints: it is smarter to move on to
        the next file than wait for it to have been completely flushed, which
        may take long.
        Comments about flush concurrency bugs moved from ma_pagecache.c.
        Removing out-of-date comment.
        frequency->interval.
        create_background_thread -> (interval>0).
        In ma_checkpoint_background(), some variables need to be preserved
        between iterations.
      storage/maria/ma_checkpoint.h:
        new prototype
      storage/maria/ma_pagecache.c:
        - concurrent calls of flush_pagecache_blocks_int() on the same file
        cause bugs (see @note in that function); we fix them by serializing
        in this situation. For that we use a global hash of (file, wqueue).
        When flush_pagecache_blocks_int() starts it looks into the hash,
        using the file as key. If not found, it inserts (file,wqueue) into the
        hash, flushes the file, and finally removes itself from the hash and
        wakes up any waiter in the queue. If found, it adds itself to the
        wqueue and waits.
        - As a by-product, we can remove changed_blocks_is_incomplete
        and replace it by scanning the hash, replace the sleep() by a queue wait.
        - new type of flush FLUSH_KEEP_LAZY: when flushing a file, if it's
        already being flushed by another thread (even partially), return
        immediately.
      storage/maria/ma_pagecache.h:
        In pagecache, a hash of files currently being flushed (i.e. there
        is a call to flush_pagecache_blocks_int() for them).
      storage/maria/ma_recovery.c:
        new prototype
      storage/maria/ma_test1.c:
        new prototype
      storage/maria/ma_test2.c:
        new prototype
      77017191
  3. 18 Oct, 2007 2 commits
    • unknown's avatar
      mysys/my_getopt.c · fd4ca26d
      unknown authored
          always process uint/ulong using ulonglong (unsigned) code
          dbug printout for adjusted option values
      strings/llstr.c
          ullstr() - the unsigned brother of llstr()
      
      
      include/m_string.h:
        ullstr() - the unsigned brother of llstr()
      mysql-test/t/variables.test:
        test adjusted for 32bit
      mysys/my_getopt.c:
        always process uint/ulong using ulonglong (unsigned) code
        dbug printout for adjusted option values
      strings/llstr.c:
        ullstr() - the unsigned brother of llstr()
      fd4ca26d
    • unknown's avatar
      Merge gbichot@bk-internal.mysql.com:/home/bk/mysql-maria · 9bcbf851
      unknown authored
      into  gbichot4.local:/home/mysql_src/mysql-maria-tmp
      
      
      9bcbf851
  4. 17 Oct, 2007 5 commits
    • unknown's avatar
      Bugfix from Sanja (when truncating log at restart) · 9d6b8f1e
      unknown authored
      
      storage/maria/ma_loghandler.c:
        applying patch for bugfix, by Sanja.
      9d6b8f1e
    • unknown's avatar
      Merge gbichot@bk-internal.mysql.com:/home/bk/mysql-maria · 59b0303e
      unknown authored
      into  gbichot4.local:/home/mysql_src/mysql-maria-for-undo-phase
      
      
      storage/maria/ma_recovery.c:
        Auto merged
      59b0303e
    • unknown's avatar
      WL#3071 Maria checkpoint · 0f1feefa
      unknown authored
      Ability for flush_pagecache_blocks() to flush only certain pages of
      a file, as instructed by an option "filter" pointer-to-function argument;
      Checkpoint and background dirty page flushing use that to flush only
      pages which have been dirty for long enough and bitmap pages.
      Fix for a bug in flush_cached_blocks() (no idea if it could produce
      a bug in real life, but theoretically it is).
      Testing checkpoint in ma_test_recovery via ma_test1 and ma_test2.
      Background checkpoint & dirty pages flush thread is still disabled
      by default in ha_maria.
      
      
      mysql-test/r/maria.result:
        result update
      storage/maria/ha_maria.cc:
        blank after function comment
      storage/maria/ma_checkpoint.c:
        Using an enum instead of 0/1/2 (applying Sanja's review comments).
        The comment about "this is an horizon" can be removed as Sanja
        created translog_next_LSN() which parse_checkpoint_record() uses.
        Variables in ma_checkpoint_background() cannot be declared in the
        for() as their value must not be reset at each iteration!
      storage/maria/ma_pagecache.c:
        adding to flush_pagecache_blocks() optional arguments 'filter'
        (pointer to function) and 'filter_arg'; if filter!=NULL this function
        will be called for each block of the file and will reply if this
        block and following ones should be flushed or not (3 possible
        replies).
        Fixing a bug when flush_cached_blocks() skips a pinned page: it has
        to unset PCBLOCK_IN_FLUSH set by flush_pagecache_blocks_int().
      storage/maria/ma_pagecache.h:
        flush_pagecache_blocks() is changed to take "filter" and "filter_arg"
        arguments. "filter", if it is not NULL, may return one value
        among enum pagecache_flush_filter_result.
      storage/maria/ma_recovery.c:
        open_count=0 when closing tables at the end of recovery.
      storage/maria/ma_test1.c:
        Optional checkpoints (-H#) at various stages (stages similar
        to --testflag), for testing of checkpoints.
      storage/maria/ma_test2.c:
        Optional checkpoints (-H#) at various stages (stages similar
        to -t), for testing of checkpoints.
      storage/maria/ma_test_recovery.expected:
        Result update: the results of the additional test run with -H#
        (checkpoints) are added here. They are exactly identical to without
        checkpoints except that the index's Root (printed by maria_chk)
        is more correct when using checkpoints. This is because checkpoint
        flushed the state, so it happens to be correct, while no-checkpoint
        does not flush the state, and recovery does not recover indexes
        so Root is never fixed. When we recover indices, this will go away.
      storage/maria/ma_test_recovery:
        We duplicate the loop of tests to add an additional run with
        checkpoints at various stages, to see if maria_read_log
        uses them fine.
      0f1feefa
    • unknown's avatar
      Merge desktop.sanja.is.com.ua:/home/bell/mysql/bk/mysql-maria · 5c6922e3
      unknown authored
      into  desktop.sanja.is.com.ua:/home/bell/mysql/bk/work-maria-bug
      
      
      storage/maria/ma_recovery.c:
        Auto merged
      5c6922e3
    • unknown's avatar
      Checking that the very last record is fully written · de3178fd
      unknown authored
        on the loghandler start.
      Variable definition moved because it is C programm.
      
      
      storage/maria/ma_loghandler.c:
        Checking that the very last record is fully written
          on the loghandler start.
      storage/maria/ma_recovery.c:
        Variable definition moved because it is C programm.
      de3178fd
  5. 16 Oct, 2007 3 commits
    • unknown's avatar
      WL#3072 Maria recovery, cleanups, removing some out-of-date @todos · f2a675b3
      unknown authored
      
      storage/maria/ma_delete_all.c:
        comment (@todo gone, nothing we can do)
      storage/maria/ma_recovery.c:
        - fail if LOGREC_CREATE_TABLE contains some symlink info (that
        will be handled in a future version)
        - don't do buffer overrun when parsing a REDO record
        - out-of-date @todo
      f2a675b3
    • unknown's avatar
      Merge gbichot@bk-internal.mysql.com:/home/bk/mysql-maria · 694c722b
      unknown authored
      into  gbichot4.local:/home/mysql_src/mysql-maria-tmp
      
      
      694c722b
    • unknown's avatar
      WL#3072 Maria recovery · 08b71528
      unknown authored
      - fixes (in recovery, checkpoint, log handler) of bugs found
      during testing.
      - new option --check for maria_read_log: with --only-display (which only
      reads the header) it reads the full record, for debugging.
      
      
      storage/maria/ma_loghandler.c:
        importing patch from Sanja for bug of translog_next_LSN() found
        during recovery
      storage/maria/ma_loghandler_lsn.h:
        better types (0L is 4 bytes on some platforms, it causes problems
        when used into lsn_store(): right shift >= width of type.
      storage/maria/ma_pagecache.c:
        work around infamous "PAGECACHE_PLAIN_PAGE used for transactional
        tables in specialm case"; REDO phase disables logging and this causes
        pages to be PAGECACHE_PLAIN_PAGE, thus ignored wrongly by the
        checkpoint taken at the end of the REDO phase.
      storage/maria/ma_recovery.c:
        - a #ifdef which broke maria_read_log in non-debug builds (no output!)
        - support for maria_read_log --check
        - detect record corruption before opening the table
        - updating is_of_horizon requires writing the state
        - fix for wrong parsing of checkpoint record by recovery
      storage/maria/ma_recovery.h:
        support for maria_read_log --check
      storage/maria/maria_read_log.c:
        Option --check: --only-display only looks at the header;
        adding --check tries a translog_read_record() to see if record can
        be fully read (this is to find bugs).
      08b71528
  6. 15 Oct, 2007 1 commit
  7. 13 Oct, 2007 2 commits
  8. 12 Oct, 2007 6 commits
    • unknown's avatar
      different fix for strict-aliasing problem · 85a920dd
      unknown authored
      85a920dd
    • unknown's avatar
      bugs/warnings fixed in lf_alloc-pin.c: · c780abfe
      unknown authored
      1. available_stack_size() was getting the direction wrong,
         so alloca() was never used
      2. (char*) casts added to kill "break strict-aliasing rules" warnings
      3. s/node/node=0/ to kill "pointer casted to integer" warning
      4. added volatiles as appropriate to prevent gcc from moving
         assignment out of the loop
      
      
      mysys/lf_alloc-pin.c:
        bugs/warnings fixed:
        1. available_stack_size() was getting the direction wrong,
           so alloca() was never used
        2. (char*) casts added to kill "break strict-aliasing rules" warnings
        3. s/node/node=0/ to kill "pointer casted to integer" warning
        4. added volatiles as appropriate to prevent gcc from moving
           assignment out of the loop
      c780abfe
    • unknown's avatar
      Merge bk-internal.mysql.com:/home/bk/mysql-maria · 58cd4a52
      unknown authored
      into  janus.mylan:/usr/home/serg/Abk/mysql-maria
      
      
      58cd4a52
    • unknown's avatar
      make maria more pluggable · 47d48415
      unknown authored
      
      mysql-test/r/maria.result:
        test for maria variables
      mysql-test/t/maria.test:
        test for maria variables
      BitKeeper/etc/ignore:
        Added sql/link_sources ylwrap libmysql_r/link_sources to the ignore list
      sql/sql_delete.cc:
        fix incorrect check
      storage/maria/ha_maria.cc:
        maria status and pagecache variables
      47d48415
    • unknown's avatar
      int->my_bool transformation fixed (thanks Guilhem) · 928d7c23
      unknown authored
      Mutex owner checks added.
      
      
      928d7c23
    • unknown's avatar
      translog_next_LSN() changed to support pointer to the page end · 5eb78c51
      unknown authored
        which can happen when you save address of horizon
      
      
      5eb78c51
  9. 11 Oct, 2007 3 commits
    • unknown's avatar
      Merge desktop.sanja.is.com.ua:/home/bell/mysql/bk/mysql-maria · 212c7dfe
      unknown authored
      into  desktop.sanja.is.com.ua:/home/bell/mysql/bk/work-maria-filler
      
      
      212c7dfe
    • unknown's avatar
      Compiler warnings removed. · b47628f1
      unknown authored
      Added ability top change empty space filler of the loghandler.
      Fixed end of log reaction.
      Fixed memory corruprion bug caused by reading non-filled hage header.
      Added debug output.
      
      
      storage/maria/ma_bitmap.c:
        Compiler warnings removed.
      storage/maria/ma_blockrec.c:
        Compiler warnings removed.
      storage/maria/ma_loghandler.c:
        Added ability top change empty space filler of the loghandler.
        Fixed end of log reaction.
        Fixed memory corruprion bug caused by reading non-filled hage header.
        Added debug output.
      storage/maria/ma_loghandler_lsn.h:
        Compiler warnings removed.
      b47628f1
    • unknown's avatar
      WL#3072 Maria recovery. Bugfix (segfault of maria_read_log) · ea388ba2
      unknown authored
      
      storage/maria/ma_recovery.c:
        bugfix ("count" is meaningul only if printing messages)
      ea388ba2
  10. 10 Oct, 2007 2 commits
    • unknown's avatar
      WL#3072 Maria recovery. Improving the progress message (small fixes · 9a57cc3f
      unknown authored
      plus printing a downcount of tables to flush).
      
      
      storage/maria/ma_recovery.c:
        improving the progress message of recovery; it used to assume that
        if there is a non-zero UNDO phase then there was a non-REDO phase,
        and if there are tables to flush there was an UNDO phase,
        which is not always true. We now print a downcount of tables to flush
        (4 3 2 1 0).
      9a57cc3f
    • unknown's avatar
      Merge bk-internal.mysql.com:/home/bk/mysql-maria · df30832d
      unknown authored
      into  mysql.com:/home/my/mysql-maria
      
      
      client/mysqladmin.cc:
        Auto merged
      include/maria.h:
        Auto merged
      include/my_sys.h:
        Auto merged
      include/mysql_com.h:
        Auto merged
      mysql-test/r/maria.result:
        Auto merged
      server-tools/instance-manager/listener.cc:
        Auto merged
      sql/handler.h:
        Auto merged
      sql/item_func.cc:
        Auto merged
      sql/item_func.h:
        Auto merged
      sql/item_strfunc.cc:
        Auto merged
      sql/mysql_priv.h:
        Auto merged
      sql/mysqld.cc:
        Auto merged
      sql/sql_class.cc:
        Auto merged
      sql/sql_class.h:
        Auto merged
      sql/sql_show.cc:
        Auto merged
      sql/sql_table.cc:
        Auto merged
      sql/table.cc:
        Auto merged
      sql/table.h:
        Auto merged
      storage/maria/ma_bitmap.c:
        Auto merged
      storage/maria/ma_blockrec.c:
        Auto merged
      storage/maria/ma_blockrec.h:
        Auto merged
      storage/maria/ma_check.c:
        Auto merged
      storage/maria/ma_create.c:
        Auto merged
      storage/maria/ma_delete.c:
        Auto merged
      storage/maria/ma_loghandler.h:
        Auto merged
      storage/maria/ma_open.c:
        Auto merged
      storage/maria/ma_search.c:
        Auto merged
      storage/maria/ma_sort.c:
        Auto merged
      storage/maria/ma_test2.c:
        Auto merged
      storage/maria/ma_test_recovery.expected:
        Auto merged
      storage/maria/ma_write.c:
        Auto merged
      storage/maria/maria_chk.c:
        Auto merged
      storage/maria/maria_pack.c:
        Auto merged
      include/my_base.h:
        Trivial manual merge
      libmysql/Makefile.shared:
        Trivial manual merge
      sql/sql_yacc.yy:
        Manual merge
      storage/maria/ha_maria.cc:
        Trivial manual merge
      storage/maria/ma_page.c:
        Trivial manual merge
      storage/maria/maria_def.h:
        Trivial manual merge
      df30832d
  11. 09 Oct, 2007 3 commits
    • unknown's avatar
      Moved randomize and my_rnd under mysys · 496741d5
      unknown authored
      Added my_uuid
      Added pre-support for PAGE_CHECKSUM
      Added syntax for CREATE ... PAGE_CHECKSUM=# TABLE_CHECKSUM=#
      Reserved place for page checksums on index, bitmap and block pages
      Added index number to header of index pages
      Added linked list for free directory entries (speeds up inserts with BLOCK format)
      Calculate checksums in original column order (fixes bug with checksum on rows with BLOCK format)
      Cleaned up all index handling to use 'info->s->keypage_header' (variable size) as the header for index pages (before this was '2')
      Added 0xffffffff to end of index and block data bases and 0xfffffffe at end of bitmap pages when page checksums are not enabled
      Added _ma_get_page_used() and _ma_get_used_and_node() to simplify index page header handling
      rec_per_key_part is now in double precision
      Reserved place in index file for my_guid and nulls_per_key_part
      Give error HA_ERR_NEW_FILE if trying to open a Maria file with new, not yet supported extensions
      
      Lots of renames to increase readability:
      
      randomize() -> my_rnd_init()
      st_maria_info -> st_maria_handler
      st_maria_info -> MARIA_HA
      st_maria_isaminfo -> st_maria_info
      rand_struct -> my_rand_struct
      rec_per_key_rows -> records_at_analyze
      
      
      client/mysqladmin.cc:
        rand_struct -> my_rrnd_struct
      include/maria.h:
        st_maria_info -> MARIA_HA
        st_maria_isaminfo -> st_maria_info
        Changed analyze statistics to be of double precission
        Changed offset to field to be 32bits instead of 64 (safe as a record without blobs can't be that big)
      include/my_base.h:
        Added HA_OPTION_PAGE_CHECKSUM & HA_CREATE_PAGE_CHECKSUM
        Fixed comments
        Added HA_ERR_NEW_FILE
      include/my_sys.h:
        Added prototypes and structures for my_uuid() and my_rnd()
      include/myisamchk.h:
        Changed some buffers to size_t
        Added possibility to have key statistics with double precission
      include/mysql_com.h:
        Move rand functions to mysys
      libmysql/Makefile.shared:
        Added my_rnd
      mysql-test/r/maria.result:
        Updated results
      mysql-test/t/maria.test:
        More tests for checksum
      mysys/Makefile.am:
        Added my_rnd.c and my_uuid.c
      server-tools/instance-manager/listener.cc:
        Fixed include order (my_global.h should always be first)
      server-tools/instance-manager/mysql_connection.cc:
        Fixed include order (my_global.h should always be first)
        Use my_rnd_init()
      server-tools/instance-manager/mysql_connection.h:
        rand_struct -> my_rand_struct
      sql/handler.h:
        Added flag for page checksums
      sql/item_func.cc:
        Use new my_rnd() interface
      sql/item_func.h:
        Use new my_rnd() interface
      sql/item_strfunc.cc:
        Use new my_rnd() interface
      sql/lex.h:
        Added PAGE_CHECKSUM and TABLE_CHECKSUM
      sql/mysql_priv.h:
        Use new my_rnd() interface
      sql/mysqld.cc:
        Use new my_rnd() interface
      sql/password.c:
        Move my_rnd() to mysys
        Use new my_rnd() interface
      sql/sql_class.cc:
        Use new my_rnd() interface
      sql/sql_class.h:
        Use new my_rnd() interface
      sql/sql_crypt.cc:
        Use new my_rnd() interface
      sql/sql_crypt.h:
        Use new my_rnd() interface
      sql/sql_show.cc:
        Simpler handling of ha_choice_values
        Added PAGE_CHECKSUM
      sql/sql_table.cc:
        Enable correct checksum handling (for now) if not running in compatible mode
      sql/sql_yacc.yy:
        Added table option PAGE_CHECKSUM
        Added future compatible table option TABLE_CHECKSUM (alias for CHECKSUM)
        Added 'choice' target to simplify code
      sql/table.cc:
        Store flag for PAGE_CHECKSUM
      sql/table.h:
        Added support for PAGE_CHECKSUM
      storage/maria/ha_maria.cc:
        Remove protection for incompatbile frm and MAI
        (Slow, not needed test)
        Rec_per_key is now in double
        Remember row type for table
        Give warning if one Maria uses another row type than requested
        Removed some old ASK_MONTY entries (added comments instead)
        Added handling of PAGE_CHECKSUM flags
      storage/maria/ma_bitmap.c:
        Added page checksums to bitmap pages
        Added special bitmap marker for bitmap pages
        (Used to find bugs when running without page checksums)
      storage/maria/ma_blockrec.c:
        Added a free-link list over directory entries. This makes insert of small rows faster as we don't
        have to scan the whole directory to find a not used entry.
        Moved SANITY_CHECKS to maria_def.h
        Simplify code by introducing dir_entry_pos()
        Added support for PAGE_CHECKSUM
      storage/maria/ma_blockrec.h:
        Added DIR_FREE_SIZE (linked list of free directory entries)
        Added PAGE_CHECKSUM
        Added 'dir_entry_pos()'
      storage/maria/ma_check.c:
        Check that index pages has correct index number
        Calculate rec_per_key with double precission
        Simplify code by using '_ma_get_used_and_node()'
        Check free directory list
        Remove wrong end \n from messages
        maria_data_on_page() -> _ma_get_page_used()
        maria_putint() -> _ma_store_page_used()
        rec_per_key_rows -> records_at_analyze
      storage/maria/ma_checksum.c:
        Calculate checksum in original column order
      storage/maria/ma_create.c:
        Store original column order in index file
        Reserve place for nulls_per_key_part (future)
        Added support for PAGE_CHECKSUM
      storage/maria/ma_dbug.c:
        Fixed wrong debug output of key of type 'ulong'
      storage/maria/ma_delete.c:
        maria_data_on_page() -> _ma_get_used_and_node()
        maria_data_on_page() -> _ma_get_page_used()
        maria_putint() -> _ma_store_page_used()
        Added page header (index key number) to all index pages
        Reserved page for checksum on index pages
        Use keypage_header
      storage/maria/ma_ft_update.c:
        maria_putint() -> _ma_store_page_used()
        Store key number at start of page
      storage/maria/ma_loghandler.h:
        st_maria_info -> MARIA_HA
      storage/maria/ma_open.c:
        rec_per_key is now in double precission
        Added 'nulls_per_key_part'
        Added 'extra_options' (flags for future)
        Added support for PAGE_CHECKSUM
        Give error HA_ERR_NEW_FILE when using unsupported maria extensions
        Added comments
        Add maria_uuid to index file
        Added functions to store and read column_nr map.
        Changed some functions to return my_bool instead of uint
      storage/maria/ma_page.c:
        Added checks that pages has correct key nr
        Store 0xffffffff in checksum position if page checksums are not enabled
        Moved key-page-delete link to take into account keypage header
      storage/maria/ma_preload.c:
        Remove old MyISAM dependent code
        When scanning pages, only add pages to page cache for the requested index
      storage/maria/ma_range.c:
        maria_data_on_page() -> _ma_get_used_and_node()
        Use keypage_header
      storage/maria/ma_rt_index.c:
        Fixed indentation
      storage/maria/ma_rt_index.h:
        Added support for dynamic index page header
        Reserved place for PAGE_CHECKSUM
      storage/maria/ma_rt_key.c:
        Fixed indentation
        maria_data_on_page() -> _ma_get_page_used()
        maria_putint() -> maria_store_page_used()
      storage/maria/ma_rt_mbr.c:
        Fixed indentation
      storage/maria/ma_rt_split.c:
        Fixed indentation
        maria_data_on_page () -> _ma_get_page_used()
      storage/maria/ma_rt_test.c:
        Fixed indentation
      storage/maria/ma_search.c:
        Remove support of using -1 as 'last used index' to _ma_check_index()
        maria_data_on_page() -> _ma_get_page_used()
        maria_data_on_page() -> _ma_get_used_and_node()
        Use keypage_header
      storage/maria/ma_sort.c:
        Changed some buffers to size_t
        Changed rec_per_key_part to double
      storage/maria/ma_static.c:
        Removed NEAR
        Added maria_uuid
      storage/maria/ma_test2.c:
        Moevd testflag == 2 to correct place
        Remove test of reading with index number -1 (not supported anymore)
      storage/maria/ma_test_recovery.expected:
        Updated results
      storage/maria/ma_test_recovery:
        Changed tmp table names so that one can run maria_chk on them
      storage/maria/ma_write.c:
        Fixed indentation
        Use keypage_header
        Store index number on index pages
        maria_putint() -> _ma_store_page_used()
        maria_data_on_page() -> ma_get_used_and_node()
        maria_data_on_page() -> _ma_get_page_used()
        Added PAGE_CHECKSUM
        Added Maria handler to some functions
        Removed some not needed casts
      storage/maria/maria_chk.c:
        Added error handling for HA_ERR_NEW_FILE
        Added information about page checksums
        rec_per_key_part changed to double
        maria_data_on_page() -> _ma_get_page_used()
        Use keypage_header
      storage/maria/maria_def.h:
        Added IDENTICAL_PAGES_AFTER_RECOVERY and SANITY_CHECKS
        Changed rec_per_key_part to double
        Added nulls_per_key_part
        rec_per_key_rows -> records_at_analyze
        st_maria_info -> MARIA_HA
        Reserve place for new statistics variables, uuid, checksums per page etc.
        Removed NEAR tags
        Changed some prototypes to use my_bool and size_t
      storage/maria/maria_pack.c:
        st_maria_info -> MARIA_HA
        Fixed indentation
      storage/myisam/mi_dbug.c:
        Fix wrong debug output for ULONG
      mysys/my_rnd.c:
        New BitKeeper file ``mysys/my_rnd.c''
      mysys/my_uuid.c:
        New BitKeeper file ``mysys/my_uuid.c''
      496741d5
    • unknown's avatar
      WL#3071 - Maria checkpoint · 791b0aa0
      unknown authored
      * Preparation for having a background checkpoint thread:
      frequency of checkpoint taken by that thread is now configurable
      by the user: global variable maria_checkpoint_frequency, in seconds,
      default 30 (checkpoint every 30th second); 0 means no checkpoints
      (and thus no background thread, thus no background flushing, that
      will probably only be used for testing).
      * Don't take checkpoints in Recovery if it didn't do anything
      significant; thus no checkpoint after a clean shutdown/restart. The
      only checkpoint which is never skipped is the one at shutdown.
      * fix for a test failure (after-merge fix)
      
      
      include/maria.h:
        new variable
      mysql-test/suite/rpl/r/rpl_row_flsh_tbls.result:
        result update
      mysql-test/suite/rpl/t/rpl_row_flsh_tbls.test:
        position update (=after merge fix, as this position was already changed
        into 5.1 and not merged here, causing test to fail)
      storage/maria/ha_maria.cc:
        Checkpoint's frequency is now configurable by the user:
        global variable maria_checkpoint_frequency. Changing it on the fly
        requires us to shutdown/restart the background checkpoint thread,
        as the loop done in that thread assumes a constant checkpoint
        interval. Default value is 30: a checkpoint every 30 seconds (yes, I
        know, physicists will remind that it should be named "period" then).
        ha_maria now asks for a background checkpoint thread when it starts,
        but this is still overruled (disabled) in ma_checkpoint_init().
      storage/maria/ma_checkpoint.c:
        Checkpoint's frequency is now configurable by the user: background thread
        takes a checkpoint every maria_checkpoint_interval-th second.
        If that variable is 0, no checkpoints are taken.
        Note, I will enable the background thread only in a later changeset.
      storage/maria/ma_recovery.c:
        Don't take checkpoints at the end of the REDO phase and at the end of
        Recovery if Recovery didn't make anything significant (didn't open
        any tables, didn't rollback any transactions).
        With this, after a clean shutdown, Recovery shouldn't take any
        checkpoint, which makes starting faster (we save a few fsync()s of
        the log and control file).
      791b0aa0
    • unknown's avatar
      Merge gbichot@bk-internal.mysql.com:/home/bk/mysql-maria · 568e32c8
      unknown authored
      into  gbichot4.local:/home/mysql_src/mysql-maria-tmp
      
      
      storage/maria/ma_loghandler.c:
        Auto merged
      568e32c8
  12. 08 Oct, 2007 7 commits
    • unknown's avatar
      Merge desktop.sanja.is.com.ua:/home/bell/mysql/bk/mysql-maria · 494409bd
      unknown authored
      into  desktop.sanja.is.com.ua:/home/bell/mysql/bk/work-maria-bug
      
      
      494409bd
    • unknown's avatar
      Merge bk-internal.mysql.com:/home/bk/mysql-maria · 5cd55487
      unknown authored
      into  janus.mylan:/usr/home/serg/Abk/mysql-maria1
      
      
      5cd55487
    • unknown's avatar
      Merge janus.mylan:/usr/home/serg/Abk/mysql-5.1 · ceadd596
      unknown authored
      into  janus.mylan:/usr/home/serg/Abk/mysql-maria1
      
      
      Makefile.am:
        Auto merged
      client/mysqldump.c:
        Auto merged
      configure.in:
        Auto merged
      libmysqld/Makefile.am:
        Auto merged
      mysql-test/mysql-test-run.pl:
        Auto merged
      mysql-test/r/mysqldump.result:
        Auto merged
      mysql-test/t/disabled.def:
        Auto merged
      mysql-test/t/mysqldump.test:
        Auto merged
      mysys/mf_tempfile.c:
        Auto merged
      sql/Makefile.am:
        Auto merged
      sql/ha_partition.cc:
        Auto merged
      sql/ha_partition.h:
        Auto merged
      sql/handler.cc:
        Auto merged
      sql/lock.cc:
        Auto merged
      sql/sql_parse.cc:
        Auto merged
      sql/sql_plugin.cc:
        Auto merged
      sql/sql_select.cc:
        Auto merged
      sql/sql_table.cc:
        Auto merged
      sql/table.cc:
        Auto merged
      sql/table.h:
        Auto merged
      BUILD/compile-dist:
        SCCS merged
      ceadd596
    • unknown's avatar
      sql_plugin.cc: · 7bb3501f
      unknown authored
        fixed uninit memory access in SET pluginvar=DEFAULT
      innodb_mysql.test, innodb_mysql.result:
        test case for SET pluginvar=DEFAULT
      
      
      mysql-test/r/innodb_mysql.result:
        test case for SET pluginvar=DEFAULT
      mysql-test/t/innodb_mysql.test:
        test case for SET pluginvar=DEFAULT
      sql/sql_plugin.cc:
        fixed uninit memory access in SET pluginvar=DEFAULT
      7bb3501f
    • unknown's avatar
      rpl_mixed_dml.inc: · 85e96bf8
      unknown authored
        use mysqltest builtins instead of --exec. Fixes a failing test too :)
      
      
      mysql-test/suite/rpl/include/rpl_mixed_dml.inc:
        use mysqltest builtins instead of --exec. Fixes a failing test too :)
      85e96bf8
    • unknown's avatar
      Makefile.am: · 71bc644e
      unknown authored
        do link_sources only once
      
      
      sql/Makefile.am:
        do link_sources only once
      libmysql_r/Makefile.am:
        do link_sources only once
      71bc644e
    • unknown's avatar
      Comments added. · 5a0e10e2
      unknown authored
      moved debugging defines.
      Fixed buffer flushing (we copied last page
      content before it was actually written, now
      we abvance pointer in new buffer and unlock
      it while waiting for filling old buffer)
      
      
      5a0e10e2