1. 30 Mar, 2018 9 commits
  2. 29 Mar, 2018 31 commits
    • Varun Gupta's avatar
      MDEV-14592: Custom Aggregates Usage Status Variable · cbc45d29
      Varun Gupta authored
      Introduced new status variable for custom aggregate functions.
      cbc45d29
    • Marko Mäkelä's avatar
      Fix a WITH_WSREP=OFF warning · 87ee8563
      Marko Mäkelä authored
      87ee8563
    • Marko Mäkelä's avatar
      MDEV-12266: Change dict_table_t::space to fil_space_t* · 4cad4239
      Marko Mäkelä authored
      InnoDB always keeps all tablespaces in the fil_system cache.
      The fil_system.LRU is only for closing file handles; the
      fil_space_t and fil_node_t for all data files will remain
      in main memory. Between startup to shutdown, they can only be
      created and removed by DDL statements. Therefore, we can
      let dict_table_t::space point directly to the fil_space_t.
      
      dict_table_t::space_id: A numeric tablespace ID for the corner cases
      where we do not have a tablespace. The most prominent examples are
      ALTER TABLE...DISCARD TABLESPACE or a missing or corrupted file.
      
      There are a few functional differences; most notably:
      (1) DROP TABLE will delete matching .ibd and .cfg files,
      even if they were not attached to the data dictionary.
      (2) Some error messages will report file names instead of numeric IDs.
      
      There still are many functions that use numeric tablespace IDs instead
      of fil_space_t*, and many functions could be converted to fil_space_t
      member functions. Also, Tablespace and Datafile should be merged with
      fil_space_t and fil_node_t. page_id_t and buf_page_get_gen() could use
      fil_space_t& instead of a numeric ID, and after moving to a single
      buffer pool (MDEV-15058), buf_pool_t::page_hash could be moved to
      fil_space_t::page_hash.
      
      FilSpace: Remove. Only few calls to fil_space_acquire() will remain,
      and gradually they should be removed.
      
      mtr_t::set_named_space_id(ulint): Renamed from set_named_space(),
      to prevent accidental calls to this slower function. Very few
      callers remain.
      
      fseg_create(), fsp_reserve_free_extents(): Take fil_space_t*
      as a parameter instead of a space_id.
      
      fil_space_t::rename(): Wrapper for fil_rename_tablespace_check(),
      fil_name_write_rename(), fil_rename_tablespace(). Mariabackup
      passes the parameter log=false; InnoDB passes log=true.
      
      dict_mem_table_create(): Take fil_space_t* instead of space_id
      as parameter.
      
      dict_process_sys_tables_rec_and_mtr_commit(): Replace the parameter
      'status' with 'bool cached'.
      
      dict_get_and_save_data_dir_path(): Avoid copying the fil_node_t::name.
      
      fil_ibd_open(): Return the tablespace.
      
      fil_space_t::set_imported(): Replaces fil_space_set_imported().
      
      truncate_t: Change many member function parameters to fil_space_t*,
      and remove page_size parameters.
      
      row_truncate_prepare(): Merge to its only caller.
      
      row_drop_table_from_cache(): Assert that the table is persistent.
      
      dict_create_sys_indexes_tuple(): Write SYS_INDEXES.SPACE=FIL_NULL
      if the tablespace has been discarded.
      
      row_import_update_discarded_flag(): Remove a constant parameter.
      4cad4239
    • Marko Mäkelä's avatar
      55f4e480
    • Marko Mäkelä's avatar
      MDEV-12266: Skip a lookup when checking free space · c02c329a
      Marko Mäkelä authored
      fsp_get_available_space_in_free_extents(): Declare the function static
      in the only caller, and take const fil_space_t& as a parameter.
      c02c329a
    • Marko Mäkelä's avatar
      330ecb90
    • Marko Mäkelä's avatar
      MDEV-12266: Remove fil_system_t::named_spaces · 05863142
      Marko Mäkelä authored
      fil_space_get_by_name(): Remove.
      (Implement differently in mariabackup.)
      
      fil_ibd_open(): Check if the tablespace by the same ID already
      exists. If it is the same name, return success, else failure.
      05863142
    • Marko Mäkelä's avatar
      MDEV-12266: Make trx_rseg_t::space a pointer · 9043bec9
      Marko Mäkelä authored
      trx_rsegf_get(), trx_undo_get_first_rec(): Change the parameter to
      fil_space_t* so that fewer callers need to be adjusted.
      
      trx_undo_free_page(): Remove the redundant parameter 'space'.
      9043bec9
    • Marko Mäkelä's avatar
      MDEV-12266: Remove trx_undo_t::space · 39ed0743
      Marko Mäkelä authored
      39ed0743
    • Marko Mäkelä's avatar
      MDEV-12266: Clean up btr_search_drop_page_hash_when_freed() · 5dd5b6ca
      Marko Mäkelä authored
      Remove the parameter page_size, and pass a dummy page size
      to buf_page_get_gen() along with BUF_PEEK_IF_IN_POOL.
      5dd5b6ca
    • Marko Mäkelä's avatar
      MDEV-12266: Cleanup DISCARD TABLESPACE · e2bf76cb
      Marko Mäkelä authored
      fil_discard_tablespace(): Merge to row_discard_tablespace()
      which was the only caller.
      e2bf76cb
    • Marko Mäkelä's avatar
      MDEV-12266: Cleanup TRUNCATE · f8d1bd01
      Marko Mäkelä authored
      fil_recreate_table(), fil_recreate_tablespace(),
      fil_reinit_space_header_for_table(): Move the functions
      to the only calling module and declare static.
      
      fil_recreate_table(): Remove the constant parameter space_id=0,flags.
      f8d1bd01
    • Marko Mäkelä's avatar
      MDEV-12266: Clean up recv_recover_page() · f2a581e6
      Marko Mäkelä authored
      f2a581e6
    • Marko Mäkelä's avatar
      332e805e
    • Marko Mäkelä's avatar
      MDEV-12266: Remove dict_index_t::table_name · 428e0289
      Marko Mäkelä authored
      Replace index->table_name with index->table->name.
      428e0289
    • Marko Mäkelä's avatar
      MDEV-12266: Remove dict_index_t::space · 604fea1a
      Marko Mäkelä authored
      We can rely on the dict_table_t::space. All indexes of a table object
      are always in the same tablespace. (For fulltext indexes, the data is
      located in auxiliary tables, and these will continue to have their own
      table objects, separate from the main table.)
      604fea1a
    • Marko Mäkelä's avatar
      MDEV-12266: Add dict_index_t::set_modified(mtr) · e7980f9c
      Marko Mäkelä authored
      This is a non-functional change (pure refactoring).
      e7980f9c
    • Marko Mäkelä's avatar
    • Marko Mäkelä's avatar
      MDEV-12266: fsp_flags_try_adjust(): Remove a lookup · c577192d
      Marko Mäkelä authored
      fsp_header_init(): Take fil_space_t* as a parameter.
      c577192d
    • Marko Mäkelä's avatar
      MDEV-12266: Add fil_system.sys_space, temp_space · 2ac8b1a9
      Marko Mäkelä authored
      Add fil_system_t::sys_space, fil_system_t::temp_space.
      These will replace lookups for TRX_SYS_SPACE or SRV_TMP_SPACE_ID.
      
      mtr_t::m_undo_space, mtr_t::m_sys_space: Remove.
      
      mtr_t::set_sys_modified(): Remove.
      
      fil_space_get_type(), fil_space_get_n_reserved_extents(): Remove.
      
      fsp_header_get_tablespace_size(), fsp_header_inc_size():
      Merge to the only caller, innobase_start_or_create_for_mysql().
      2ac8b1a9
    • Marko Mäkelä's avatar
      Allocate the singleton fil_system statically · 600c85e8
      Marko Mäkelä authored
      fil_system_t::create(): Replaces fil_init(), fsp_init().
      
      fil_system_t::close(): Replaces fil_close().
      
      fil_system_t::max_n_open: Remove. Use srv_max_n_open_files directly.
      600c85e8
    • Daniel Black's avatar
      MDEV-13785: move defination HAVE_LARGE_PAGES -> HAVE_LINUX_LARGE_PAGES · 402c7584
      Daniel Black authored
      HAVE_LARGE_PAGES was always Linux but now there is
      HAVE_SOLARIS_LARGE_PAGES in the code base. Innodb was using HAVE_LINUX_LARGE_PAGES
      so keep this consistent everywhere.
      
      Test plan:
      
      $ grep Hugepagesize: /proc/meminfo
      Hugepagesize:       2048 kB
      
      $ sudo sysctl vm.nr_hugepages=1024
      vm.nr_hugepages = 1024
      $ sudo sysctl  kernel.shmmax=$(( 2 * 1024 *1024 * 1024 ))
      kernel.shmmax = 2147483648
      
      No errors in ouput:
      $ sql/mysqld --skip-networking --datadir=/tmp/datadir --log-bin=/tmp/datadir/mysqlbin --socket /tmp/s.sock --lc-messages-dir=${PWD}/sql/share --verbose --large-pages=1
      2018-03-23 12:51:18 139697428129984 [Note] sql/mysqld (mysqld 10.2.14-MariaDB-log) starting as process 25406 ...
      2018-03-23 12:51:18 139697428129984 [Note] InnoDB: Mutexes and rw_locks use GCC atomic builtins
      2018-03-23 12:51:18 139697428129984 [Note] InnoDB: Uses event mutexes
      2018-03-23 12:51:18 139697428129984 [Note] InnoDB: Compressed tables use zlib 1.2.11
      2018-03-23 12:51:18 139697428129984 [Note] InnoDB: Using Linux native AIO
      2018-03-23 12:51:18 139697428129984 [Note] InnoDB: Number of pools: 1
      2018-03-23 12:51:18 139697428129984 [Note] InnoDB: Using SSE2 crc32 instructions
      2018-03-23 12:51:18 139697428129984 [Note] InnoDB: Initializing buffer pool, total size = 128M, instances = 1, chunk size = 128M
      2018-03-23 12:51:18 139697428129984 [Note] InnoDB: Completed initialization of buffer pool
      2018-03-23 12:51:18 139696883590912 [Note] InnoDB: If the mysqld execution user is authorized, page cleaner thread priority can be changed. See the man page of setpriority().
      2018-03-23 12:51:18 139697428129984 [Note] InnoDB: Highest supported file format is Barracuda.
      2018-03-23 12:51:18 139697428129984 [Note] InnoDB: 128 out of 128 rollback segments are active.
      2018-03-23 12:51:18 139697428129984 [Note] InnoDB: Removed temporary tablespace data file: "ibtmp1"
      2018-03-23 12:51:18 139697428129984 [Note] InnoDB: Creating shared tablespace for temporary tables
      2018-03-23 12:51:18 139697428129984 [Note] InnoDB: Setting file './ibtmp1' size to 12 MB. Physically writing the file full; Please wait ...
      2018-03-23 12:51:18 139697428129984 [Note] InnoDB: File './ibtmp1' size is now 12 MB.
      2018-03-23 12:51:18 139697428129984 [Note] InnoDB: 5.7.21 started; log sequence number 1620099
      2018-03-23 12:51:18 139696713733888 [Note] InnoDB: Loading buffer pool(s) from /tmp/datadir/ib_buffer_pool
      2018-03-23 12:51:18 139696713733888 [Note] InnoDB: Buffer pool(s) load completed at 180323 12:51:18
      2018-03-23 12:51:18 139697428129984 [Note] Plugin 'FEEDBACK' is disabled.
      2018-03-23 12:51:18 139697428129984 [Note] Reading of all Master_info entries succeded
      2018-03-23 12:51:18 139697428129984 [Note] Added new Master_info '' to hash table
      2018-03-23 12:51:18 139697428129984 [Note] sql/mysqld: ready for connections.
      Version: '10.2.14-MariaDB-log'  socket: '/tmp/s.sock'  port: 0  Source distribution
      
      $ grep -i huge /proc/25406/smaps | grep -v ' 0 kB'
      Private_Hugetlb:    8192 kB
      Private_Hugetlb:    2048 kB
      $ grep huge /proc/25406/numa_maps
      7f0d74400000 default file=/SYSV00000000\040(deleted) huge
      7f0dbd200000 default file=/SYSV00000000\040(deleted) huge dirty=4 N0=4 kernelpagesize_kB=2048
      7f0dc5600000 default file=/SYSV00000000\040(deleted) huge
      7f0dd1200000 default file=/SYSV00000000\040(deleted) huge dirty=1 N0=1 kernelpagesize_kB=2048
      
      $ grep Huge  /proc/meminfo
      AnonHugePages:         0 kB
      ShmemHugePages:        0 kB
      HugePages_Total:     940
      HugePages_Free:      935
      HugePages_Rsvd:      177
      HugePages_Surp:        0
      Hugepagesize:       2048 kB
      
      Ran again with --memlock
      (note needs ulimit -l > size)
      
      $ grep Huge  /proc/meminfo
      AnonHugePages:         0 kB
      ShmemHugePages:        0 kB
      HugePages_Total:     940
      HugePages_Free:      758
      HugePages_Rsvd:        0
      HugePages_Surp:        0
      Hugepagesize:       2048 kB
      
      $ grep huge /proc/26020/numa_maps
      7fe870400000 default file=/SYSV00000000\040(deleted) huge dirty=62 N0=62 kernelpagesize_kB=2048
      7fe8b3a00000 default file=/SYSV00000000\040(deleted) huge dirty=66 N0=66 kernelpagesize_kB=2048
      7fe8bd600000 default file=/SYSV00000000\040(deleted) huge dirty=53 N0=53 kernelpagesize_kB=2048
      7fe8c8400000 default file=/SYSV00000000\040(deleted) huge dirty=1 N0=1 kernelpagesize_kB=2048
      
      $ grep -i huge /proc/26020/smaps | grep -v ' 0 kB'
      Private_Hugetlb:  126976 kB
      Private_Hugetlb:  135168 kB
      Private_Hugetlb:  108544 kB
      Private_Hugetlb:    2048 kB
      402c7584
    • Marko Mäkelä's avatar
      Remove dict_build_tablespace_for_table() · 1f0f7d59
      Marko Mäkelä authored
      create_table_info_t::create_table_def(): Assign the innodb_temporary
      tablespace directly.
      
      dict_build_tablespace_for_table(): Merge to the only remaining caller,
      dict_build_table_def_step().
      1f0f7d59
    • Marko Mäkelä's avatar
      Remove an unnecessary global declaration · 60ef478c
      Marko Mäkelä authored
      60ef478c
    • Marko Mäkelä's avatar
      d177a0ed
    • Marko Mäkelä's avatar
      Merge 10.2 into 10.3 · 980dd09b
      Marko Mäkelä authored
      980dd09b
    • Marko Mäkelä's avatar
      MDEV-15719: Adjust a test case · 014dfe47
      Marko Mäkelä authored
      014dfe47
    • Marko Mäkelä's avatar
      3eb73bf6
    • Marko Mäkelä's avatar
      row_drop_table_for_mysql(): Use a constant string · 622d21e2
      Marko Mäkelä authored
      It does not hurt to delete non-existing records from SYS_TABLESPACES
      and SYS_DATAFILES. Because MariaDB does not support CREATE TABLESPACE,
      only the system tablespace (space_id=0) can contain multiple tables.
      But, there are no entries for the system tablespace in these tables
      (which actually are stored inside the system tablespace).
      622d21e2
    • Sergei Petrunia's avatar
      MDEV-15472: Assertion `!is_set() || (m_status == DA_OK_BULK && is_bulk_op())' failure · b9227410
      Sergei Petrunia authored
      MariaDB differs from the upstream for "DDL-like" command. For these,
      it sets binlog_format=STATEMENT for the duration of the statement.
      This doesn't play well with MyRocks, which tries to prevent DML
      commands with binlog_format!=ROW.
      
      Also, if Locked_tables_list::reopen_tables() returned an error, then
      close_cached_tables should propagate the error condition and not silently
      consume it (it's difficult to have test coverage for this because this
      error condition is rare)
      b9227410
    • Monty's avatar
      MDEV-15149 Assert upon concurrent creating / querying sequences · 4277c173
      Monty authored
      MDEV-15117 Server crashes in in open_and_process_table or ASAN
      heap-use-after-free in is_temporary_table upon creating/flushing
      sequences.
      
      Problem was that sequence_insert closed and reopened the like table
      without proper locking.  Fixed by ensuring that the like table is
      not reopened in sequence_insert
      4277c173