Adding rec_lsn to Maria's page cache. Misc fixes to Checkpoint.


mysys/mf_pagecache.c:
  adding rec_lsn, the LSN when a page first became dirty.
  It is set when unlocking a page (TODO: should also be set when
  the unlocking is an implicit part of pagecache_write()).
  It is reset in link_to_file_list() and free_block()
  (one of which is used every time we flush a block).
  It is a ulonglong and not LSN, because its destination is comparisons
  for which ulonglong is better than a struct.
storage/maria/ma_checkpoint.c:
  misc fixes to Checkpoint (updates now that the transaction manager
  and the page cache are more known)
storage/maria/ma_close.c:
  an important note for the future.
storage/maria/ma_least_recently_dirtied.c:
  comment

mysys/lf_hash.c:
  fix for "statement with no effect" warning
storage/maria/lockman.c:
  fix for "statement with no effect" warning

And reducing the time taken by my_atomic-t.


include/lf.h:
  fix to be able to add a ";" at the end of the macro's invokation
  (removing it here, removes a warning from "gcc -ansi" about a
  standalone ";").
mysys/lf_hash.c:
  ";" is ok now after LF_REQUIRE_PINS
mysys/mf_pagecache.c:
  comment fix
unittest/mysys/my_atomic-t.c:
  decreasing number of iterations to make test take less time.

* merging changes done to the key cache since May 2006 into Maria
* enabling two small enough page cache's unit tests by default
* fix to have non-buffered output in unit tests (to not have a false
timeout killing in pushbuild) (patch given by Serg)
* removing some warnings of gcc -ansi


include/lf.h:
  getting rid of "warning: ISO C does not allow extra `;' outside of a function"
  (gcc -ansi)
mysys/lf_hash.c:
  getting rid of "warning: ISO C does not allow extra `;' outside of a function"
  (gcc -ansi)
mysys/mf_pagecache.c:
  Cosmetic changes to minimize the diff with the key cache.
  #define PAGECACHE_DEBUG_LOG is not needed (just define PAGECACHE_DEBUG
  if you want) (this change removes "warning: 'pagecache_debug_print'
  declared `static' but never defined").
  Importing changes made to mf_keycache.c since May 2006, into the page cache.
  Disabling online resizing in the page cache.
  Fix for "warning: ISO C90 forbids mixed declarations and code".
unittest/mysys/Makefile.am:
  Of the page cache's unit tests, two are small enough to run on pushbuild,
  renaming them to a -t suffix.
unittest/mytap/tap.c:
  pushbuild kills a test after seeing no output from it for 10 minutes;
  so we set the mytap framework to not buffer output (patch given by
  Serg) so that output is seen more frequently and not "all at the end
  of the test".

storage/maria/unittest/lockman-t.c:
  no //
storage/maria/unittest/lockman1-t.c:
  no //
storage/maria/unittest/lockman2-t.c:
  no //

storage/maria/lockman.c:
  no //
storage/maria/trnman.c:
  no //

don't run tests depending on the generic lock manager which will be
removed; don't run page cache unit tests by default (too intensive).


storage/maria/unittest/Makefile.am:
  don't run tests depending on the generic lock manager (it will 
  be removed in the end), one of them crashes.
unittest/mysys/Makefile.am:
  page cache tests put a too high load, causes problems on shared machines;
  so we still build them but give them a suffix so that they are
  not run by the default "test-unit" Makefile target.

unittest/mysys/Makefile.am:
  add test_file.h in the tarball

storage/maria/lockman.c:
  fix for -ansi (// changed to #warning)
storage/maria/trnman.c:
  fix for -ansi (// changed to #warning)

include/Makefile.am:
  put my_bit.h in the tarball (need it for mf_keycache.c)
mysys/lf_alloc-pin.c:
  // in .c file
mysys/lf_hash.c:
  // in .c file

Sync table files after CREATE (of non-temp table), DROP, RENAME,
TRUNCATE, sync directories and symlinks (for the 3 first commands).
Comments for future log records.
In ma_rename(), if rename of index works and then rename of data fails,
try to undo the rename of the index to leave a consistent state.


mysys/my_symlink.c:
  sync directory after creation of a symbolic link in it, if asked
mysys/my_sync.c:
  comment. Fix for when the file's name has no directory in it.
storage/maria/ma_create.c:
  sync files and links and dirs when creating a non-temporary table.
  Optimizations of the above to reduce syncs in the common cases:
  * if index file and data file have the exact same paths (regular
  and link), sync the directories (of regular and link) only once
  after creating the last file (the data file).
  * don't sync the data file if we didn't write to it (always true
  in our builds).
storage/maria/ma_delete_all.c:
  sync files after truncating a table
storage/maria/ma_delete_table.c:
  sync files and symbolic links and dirs after dropping a table
storage/maria/ma_extra.c:
  a function which wraps the sync of the index file and the sync of the
  data file.
storage/maria/ma_locking.c:
  using a wrapper function
storage/maria/ma_rename.c:
  sync files and symbolic links and dirs after renaming a table.
  If rename of index works and then rename of data fails, try to undo
  the rename of the index to leave a consistent state. That is just a
  try, it may fail...
storage/maria/ma_test3.c:
  warning to not pay attention to this test.
storage/maria/maria_def.h:
  declaration for the function added to ma_extra.c

make it return an error (except if certain errno), test this error in
callers. Do a single my_sync_dir() in my_rename() if possible.


include/my_global.h:
  better have a symbol name talking about the feature, use it in the
  code of the feature, and define the symbol once depending on the
  platform, rather than have the platform "tested" in the code
  of the feature several times.
include/my_sys.h:
  my_sync_dir() now can return error
mysys/my_create.c:
  my_sync_dir() can now return an error
mysys/my_delete.c:
  my_sync_dir() can now return an error
mysys/my_rename.c:
  my_sync_dir() can now return an error.
  Do a single sync if "from" and "to" are the same directory.
  #ifdef here to not even compile dirname_part() if useless.
mysys/my_sync.c:
  more comments.
  A compilation error if no way to make my_sync() work (I guess
  we don't want to ship a binary which cannot do any sync at all;
  users of strange OSes compile from source and can remove
  the #error).
  my_sync_dir() now returns an error (except for certain errno values
  considered ok; EIO "input/output error" is not ok).
sql/unireg.cc:
  my_sync_dir() now returns an error which must be tested

into  gbichot3.local:/home/mysql_src/mysql-maria

unittest/mysys/my_atomic-t.c:
  try both signed and unsigned arguments for atomic ops

into  gbichot3.local:/home/mysql_src/mysql-maria

1) on Mac OS X >=10.3, fcntl() is recommended over fsync (from the
man page: "[With fsync()] the disk drive may also re-order the data
so that later writes may be present while earlier writes are not.
Applications such as databases that require a strict ordering of writes
should use F_FULLFSYNC to ensure their data is written in the order
they expect"). I have seen two other pieces of software changing from
fsync to F_FULLFSYNC on Mac OS X.
2) to make a file creation/deletion/renaming durable on Linux (at least
ext2 as I have tested) (see "man fsync"), a fsync() on the directory
is needed: new functions to do that, and a flag MY_SYNC_DIR to do
it in my_create/my_delete/my_rename.
3) now using this directory syncing when creating he frm if
opt_sync_frm, and for Maria's control file when it is created.


include/my_sys.h:
  new flag to my_create/my_delete/my_rename, which asks to sync the
  directory after the operation is done (currently does nothing except
  on Linux)
libmysql/CMakeLists.txt:
  my_create() now depends on my_sync() so my_sync is needed for libmysql
libmysql/Makefile.shared:
  my_create() now depends on my_sync() so my_sync is needed for libmysql
mysys/my_create.c:
  my_create() can now sync the directory if asked for
mysys/my_delete.c:
  my_delete() can now sync the directory if asked for
mysys/my_open.c:
  it was a bug that my_close() is done on fd but a positive fd would
  still be returned, by my_register_filename().
mysys/my_rename.c:
  my_rename() can now sync the two directories (the one of "from" and
  the one of "to") if asked for.
mysys/my_sync.c:
  On recent Mac OS X, fcntl(F_FULLFSYNC) is recommended over fsync()
  (see "man fsync" on Mac OS X 10.3).
  my_sync_dir(): to sync a directory after a file creation/deletion/
  renaming; can be called directly or via MY_SYNC_DIR in my_create/
  my_delete/my_rename(). No-op except on Linux (see "man fsync" on Linux).
  my_sync_dir_from_file(): same as above, just more practical when the
  caller has a file name but no directory name ready.
  Should the #warning even be a #error? I mean do we want to release
  binaries which don't guarantee any durability?
sql/log.cc:
  a TODO for the future.
sql/unireg.cc:
  If we sync the frm it makes sense to also sync its creation in the
  directory.
storage/maria/ma_control_file.c:
  control file is vital, try to make it to disk

into  desktop.sanja.is.com.ua:/home/bell/mysql/bk/work-maria-pagecache

mysys/mf_pagecache.c:
  Unneed volitile definitions removed.
  Comments added.
  info_check_lock() code simplified.
  pagecache_delete_page() fixed.

into  janus.mylan:/usr/home/serg/Abk/mysql-maria

    #warning to warn (!) about current issues in the pagecache hindering
    checkpoint (we will have to fix them!).


mysys/mf_pagecache.c:
  fix for compiler warning (signed-ness).
  #warning to warn (!) about current issues in the pagecache hindering
  checkpoint (we will have to fix them!).

Moving the test_pagecache_ tests from mysys to unittest/mysys.
Means fixing includes to work with the new directory, some Makefile.am
editing, replacing memset() with bfill().
test_page_cache_*.c renamed to mf_pagecache_*-t.c (-t is the standard
suffix for tests in the mytap protocol).
Also added plan() and exit_status() calls to tests.
Sanja, I put some TODOs for you at the start of mf_pagecache_*.c


unittest/mysys/test_file.h:
  Rename: mysys/test_file.h -> unittest/mysys/test_file.h
mysys/Makefile.am:
  pagecache test files move to top/unittest/mysys
mysys/mf_pagecache.c:
  my_bit.h needed to compile.
unittest/mysys/Makefile.am:
  INCLUDES is a better place for includes than AM_CPPFLAGS (the latter
  get overriden by prog_CPPFLAGS, which is not desirable here).
  Adding pagecache's test programs (moved from mysys);
  test_pagecache_* has been renamed to mf_pagecache*-t
  (-t is the required suffix for test executables in the mytap framework).
unittest/mysys/mf_pagecache_consist.c:
  fixing includes to work with the new directory.
  The test must return an exit code informing if any part failed.
  TODOs for Sanja.
unittest/mysys/mf_pagecache_single.c:
  fixing includes to work with new directory.
  adding a plan() to account for the number of tests.
  Adding exit_status() to tell how many tests failed.
  memset() was giving a compilation warning (implicit declaration etc),
  properly due to me removing stdio.h etc, so I replaced it with bfill().
  TODOs for Sanja.
unittest/mysys/test_file.c:
  moved from mysys (piece of the page cache tests) and includes fixed.

tablockman: fixed a bug in finding a blocker lock


mysys/my_getsystime.c:
  this is no longer true
storage/maria/lockman.h:
  post-review fixes
storage/maria/tablockman.h:
  post-review fixes
storage/maria/unittest/lockman-t.c:
  post-review fixes
storage/maria/unittest/lockman1-t.c:
  post-review fixes
storage/maria/unittest/lockman2-t.c:
  post-review fixes
include/my_atomic.h:
  moved intptr definition to my_global.h
storage/maria/tablockman.c:
  post-review fixes
BUILD/SETUP.sh:
  add -DMY_LF_EXTRA_DEBUG to debug builds
include/atomic/nolock.h:
  suppress warning
include/my_global.h:
  suppress warning
mysys/lf_alloc-pin.c:
  post-review fixes
mysys/lf_dynarray.c:
  post-review fixes
mysys/lf_hash.c:
  post-review fixes
storage/maria/trnman.c:
  suppress warning
include/lf.h:
  post-review fix

storage/maria/CMakeLists.txt:
  Rename: storage/maria/_MakeLists.txt -> storage/maria/CMakeLists.txt

into  desktop.sanja.is.com.ua:/home/bell/mysql/bk/work-maria-pagecache

BitKeeper/deleted/.del-cmakelists_.txt:
  Delete: storage/maria/cmakelists_.txt

fixed bug in the pagecache which lead to assertion in multithread test


include/pagecache.h:
  post review fixes:
   - comments fixed
   - types fixed
   - keyword 'extern' added to all interface functions
mysys/mf_pagecache.c:
  postreview fixes:
   - comments fixed and added
   - types fixed
   - typo fixed
  Added write locking flag set when we took a block from LRU and going to free it (to prevent locking/using it for write)
mysys/test_pagecache_consist.c:
  pagecache size reduced (to be able reproduce problems found by Guilhem)
  typo in the comment fixed

into  janus.mylan:/usr/home/serg/Abk/mysql-maria

storage/maria/lockman.c:
  restore removed lines
storage/maria/unittest/lockman2-t.c:
  fix the plan

This makes an expected warning message about the index file's size, go away,
as intended.


storage/maria/ma_test_all.sh:
  importing change made to MyISAM's mi_test_all, into Maria's ma_test_all
  This makes an expected warning message about the index file's size, go away,
  as intended.

into  janus.mylan:/usr/home/serg/Abk/mysql-maria

storage/maria/tablockman.c:
  comments. bugfix - a special case in release_locks
storage/maria/unittest/lockman1-t.c:
  updated
storage/maria/unittest/lockman2-t.c:
  new tests

storage/maria/unittest/lockman1-t.c:
  New BitKeeper file ``storage/maria/unittest/lockman1-t.c''
storage/maria/tablockman.c:
  New BitKeeper file ``storage/maria/tablockman.c''
storage/maria/tablockman.h:
  New BitKeeper file ``storage/maria/tablockman.h''
storage/maria/unittest/lockman2-t.c:
  New BitKeeper file ``storage/maria/unittest/lockman2-t.c''

the unit test of the control file module.


storage/maria/trnman.c:
  fix for a bug (i has to be in [1..SHORT_TRID_MAX])
storage/maria/unittest/ma_control_file-t.c:
  corrupted checksum is in buffer[0], not in buffer[1]

"Changing MI_KEY_BLOCK_LENGTH makes a wrong myisamchk"
in the Maria tree as it is really needed to get "ma_test_all" to pass
(this bug showed up in Maria first, not in MyISAM).
Now ma_test_all does not have corruption messages about test2 anymore,
and shows the same output as mi_test_all except that
ma_test_all has this at the start:
lt-maria_chk: MARIA file test1
lt-maria_chk: warning: Size of indexfile is: 8192          Should be: 16384
MARIA-table 'test1' is usable but should be fixed
This was already true before importing the bugfix.
Wonder if normal.
NOTE: this bugfix is currently in 5.1-engines, in a few days
will be in the main 5.1, then we'll merge 5.1
into Maria: this will merge the bugfix into storage/myisam, but there
will be no need to apply it to storage/maria again. I just couldn't
wait a few days for the 5.1-engines->5.1 merge to be allowed.


mysql-test/r/maria.result:
  result update
mysql-test/t/maria.test:
  test for BUG#22119
storage/maria/ma_check.c:
  fix for BUG#22119

into  janus.mylan:/usr/home/serg/Abk/mysql-maria


storage/maria/trnman.c:
  Auto merged

---
comments


mysys/lf_alloc-pin.c:
  comments
mysys/lf_dynarray.c:
  comments
mysys/lf_hash.c:
  comments, charset-aware comparison
storage/maria/trnman.c:
  comments
storage/maria/unittest/lockman-t.c:
  test case for a bug
unittest/mysys/my_atomic-t.c:
  removed mistakenly copied line

include/my_atomic.h:
  comment