Commit dc16dc04 authored by Julien Muchembled's avatar Julien Muchembled

mariadb: build RocksDB without fallocate

https://github.com/facebook/rocksdb/commit/7291a3f813e563efbd6870465b1063a115480373
is not enough.

Anway, given how SST files are written, fallocate
is unlikely to be useful, even less on SSD.

RocksDB seems to have options to disable at runtime (see env/io_posix.cc)
but I see nothing for that in MariaDB configuration.

Steps to reproduce:

With a Btrfs partition (no specific mkfs option and for example 2GB size)
and following MariaDB configuration:

  [mysqld]
  datadir = /tmp/datadir
  disable-log-bin
  plugin-load = ha_rocksdb
  rocksdb_default_cf_options = compression=kNoCompression;bottommost_compression=kZSTD

execute the following queries with `mysql -uroot`:

> USE test;
> CREATE TABLE t(x MEDIUMBLOB) ENGINE=RocksDB;
> INSERT INTO t VALUES(repeat('.',16777215));
> INSERT INTO t SELECT x FROM t UNION ALL SELECT x FROM t;
> INSERT INTO t SELECT x FROM t UNION ALL SELECT x FROM t;
> INSERT INTO t SELECT x FROM t UNION ALL SELECT x FROM t;

Disk usage can be checked with:

> SHOW ENGINE RocksDB STATUS \G

Level    Files   Size     ...
------------------------- ...
  L0      1/0   144.00 MB ...
  L2      1/0   288.00 MB ...
 Sum      2/0   432.00 MB ...

and this can be compacted as follows:

> SET GLOBAL rocksdb_compact_cf='default';

> SHOW ENGINE RocksDB STATUS \G

Level    Files   Size     ...
------------------------- ...
  L0      0/0    0.00 KB  ...
  L1      0/0    0.00 KB  ...
  L2      1/0   16.28 KB  ...
 Sum      1/0   16.28 KB  ...

Now using `compsize` (on Debian, it's provided by btrfs-compsize package):

# with -DWITH_FALLOCATE=NO

$ compsize /tmp/datadir/#rocksdb/000029.sst
Type       Perc     Disk Usage   Uncompressed Referenced
TOTAL      100%       20K          20K          20K
none       100%       20K          20K          20K

# without -DWITH_FALLOCATE=NO

$ compsize /tmp/datadir/#rocksdb/000029.sst
Type       Perc     Disk Usage   Uncompressed Referenced
TOTAL      100%       35M          35M          20K
none       100%       35M          35M          20K
parent 8deaaf9f
...@@ -67,6 +67,10 @@ configure-options = ...@@ -67,6 +67,10 @@ configure-options =
-DCMAKE_INSTALL_RPATH=${:CMAKE_LIBRARY_PATH} -DCMAKE_INSTALL_RPATH=${:CMAKE_LIBRARY_PATH}
-DCMAKE_INCLUDE_PATH=${unixodbc:location}/include -DCMAKE_INCLUDE_PATH=${unixodbc:location}/include
-DCMAKE_LIBRARY_PATH=${unixodbc:location}/lib -DCMAKE_LIBRARY_PATH=${unixodbc:location}/lib
# for RocksDB - see
# https://lore.kernel.org/linux-btrfs/ed3642c2-682e-08a1-f18d-2d63409b7631@nexedi.com/T/
-DWITH_FALLOCATE=NO
##
CMAKE_CFLAGS = -I${bzip2:location}/include -I${jemalloc:location}/include -I${libaio:location}/include -I${libxml2:location}/include -I${ncurses:location}/include -I${openssl:location}/include -I${pcre:location}/include -I${readline5:location}/include -I${xz-utils:location}/include -I${zlib:location}/include -I${unixodbc:location}/include -I${lz4:location}/include -I${snappy:location}/include -I${zstd:location}/include CMAKE_CFLAGS = -I${bzip2:location}/include -I${jemalloc:location}/include -I${libaio:location}/include -I${libxml2:location}/include -I${ncurses:location}/include -I${openssl:location}/include -I${pcre:location}/include -I${readline5:location}/include -I${xz-utils:location}/include -I${zlib:location}/include -I${unixodbc:location}/include -I${lz4:location}/include -I${snappy:location}/include -I${zstd:location}/include
CMAKE_LIBRARY_PATH = ${bzip2:location}/lib:${jemalloc:location}/lib:${libaio:location}/lib:${libxml2:location}/lib:${ncurses:location}/lib:${openssl:location}/lib:${pcre:location}/lib:${readline5:location}/lib:${xz-utils:location}/lib:${zlib:location}/lib:${unixodbc:location}/lib:${lz4:location}/lib:${snappy:location}/lib:${zstd:location}/lib CMAKE_LIBRARY_PATH = ${bzip2:location}/lib:${jemalloc:location}/lib:${libaio:location}/lib:${libxml2:location}/lib:${ncurses:location}/lib:${openssl:location}/lib:${pcre:location}/lib:${readline5:location}/lib:${xz-utils:location}/lib:${zlib:location}/lib:${unixodbc:location}/lib:${lz4:location}/lib:${snappy:location}/lib:${zstd:location}/lib
environment = environment =
......
Markdown is supported
0%
or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment