1. 28 Apr, 2018 6 commits
  2. 27 Apr, 2018 1 commit
    • Alexander Barkov's avatar
      MDEV-16020 SP variables inside GROUP BY..WITH ROLLUP break replication · 96a301bb
      Alexander Barkov authored
      The code passing positions in the query to constructors of
      Rewritable_query_parameter descendants (e.g. Item_splocal)
      was not reliable. It used various Lex_input_stream methods:
      - get_tok_start()
      - get_tok_start_prev()
      - get_tok_end()
      - get_ptr()
      to find positions of the recently scanned tokens.
      
      The challenge was mostly to choose between get_tok_start()
      and get_tok_start_prev(), taking into account to the current
      grammar (depending if lookahead takes place before
      or after we read the positions in every particular rule).
      
      But this approach did not work at all in combination
      with token contractions, when MYSQLlex() translates
      two tokens into one token ID, for example:
         WITH ROLLUP -> WITH_ROLLUP_SYM
      
      As a result, the tokenizer is already one more token ahead.
      So in query fragment:
      
        "GROUP BY d, spvar WITH ROLLUP"
      
      get_tok_start() points to "ROLLUP".
      get_tok_start_prev() points to "WITH".
      
      As a result, it was "WITH" who was erroneously replaced
      to NAME_CONST() instead of "spvar".
      
      This patch modifies the code to do it a different way.
      
      Changes:
      
      1. For keywords and identifiers, the tokenizer now
      returns LEX_CTRING pointing directly to the query
      fragment. So query positions are now just available using:
      - $1.str           - for the beginning of a token
      - $1.str+$1.length - for the end of a token
      
      2. Identifiers are not allocated on the THD memory root
      in the tokenizer any more. Allocation is now done
      on later stages, in methods like LEX::create_item_ident().
      
      3. Two LEX_CSTRING based structures were added:
      - Lex_ident_cli_st - used to store the "client side"
        identifier representation, pointing to the
        query fragment. Note, these identifiers
        are encoded in @@character_set_client
        and can have broken byte sequences.
      
      - Lex_ident_sys_st - used to store the "server side"
        identifier representation, pointing to the
        THD allocated memory. This representation
        guarantees that the identifier was checked
        for being well-formed, and is encoded in utf8.
      
      4. To distinguish between two identifier types
         in the grammar, two Bison types were added:
         <ident_cli> and <ident_sys>
      
      5. All non-reserved keywords were marked as
         being of the type <ident_cli>.
         All reserved keywords are still of the type NONE.
      
      6. All curly brackets in rules collecting
         non-reserved keywords into non-terminal
         symbols were removed, e.g.:
      
         Was:
      
             keyword_sp_data_type:
               BIT_SYM           {}
             | BOOLEAN_SYM       {}
      
         Now:
      
             keyword_sp_data_type:
               BIT_SYM
             | BOOLEAN_SYM
      
        This is important NOT to have brackets here!!!!
        This is needed to make sure that the underlying
        Lex_ident_cli_ststructure correctly passes up to
        the calling rule.
      
      6. The code to scan identifiers and keywords
        was moved from lex_one_token() into new
        Lex_input_stream methods:
      
         scan_ident_sysvar()
         scan_ident_start()
         scan_ident_middle()
         scan_ident_delimited()
      
        This was done to:
        - get rid of enormous amount of references to &yylval->lex_str
        - and remove a lot of references like lip->xxx
      
      7. The allocating functionality which puts identifiers on the
         THD memory root now resides in methods of Lex_ident_sys_st,
         and in THD::to_ident_sys_alloc().
         get_quoted_token() was removed.
      
      8. Cleanup: check_simple_select() was moved as a method to LEX.
      
      9. Cleanup: Some more functionality was moved from *.yy
         to new methods were added to LEX:
           make_item_colon_ident_ident()
           make_item_func_call_generic()
           create_item_qualified_asterisk()
      96a301bb
  3. 26 Apr, 2018 24 commits
    • Igor Babaev's avatar
      MDEV-16038 Assertion `map->n_bits > 0' failed (my_bitmap.c:386: bitmap_is_clear_all) · 6c5e60f1
      Igor Babaev authored
      Rows with no elements are not allowed in a table value constructor
      unless it is used in an INSERT statement.
      6c5e60f1
    • Marko Mäkelä's avatar
      Merge 10.2 into 10.3 · 99fa7c6c
      Marko Mäkelä authored
      99fa7c6c
    • Marko Mäkelä's avatar
      MDEV-15914: Make mtr_buf_t a plain class · 2898c7ec
      Marko Mäkelä authored
      2898c7ec
    • Marko Mäkelä's avatar
      362151e8
    • Marko Mäkelä's avatar
      MDEV-15914: Use buf_block_t* for undo, not page_t* · 6f88bc45
      Marko Mäkelä authored
      trx_undof_page_add_undo_rec_log(): Write the undo page number
      directly from the buf_block_t descriptor, not by decoding the
      fields in the page frame.
      6f88bc45
    • Marko Mäkelä's avatar
      MDEV-15914: Restore MLOG_UNDO_INSERT · 76c62bc6
      Marko Mäkelä authored
      trx_undof_page_add_undo_rec_log(): Write the MLOG_UNDO_INSERT
      record instead of the equivalent MLOG_2BYTES and MLOG_WRITE_STRING.
      This essentially reverts commit 9ee8917d.
      
      In MariaDB 10.3, I attempted to simplify the crash recovery code
      by making use of lower-level redo log records. It turns out that
      we must keep the redo log parsing code in order to allow crash-upgrade
      from older MariaDB versions (MDEV-14848).
      
      Now, it further turns out that the InnoDB redo log record format is
      suboptimal for logging multiple changes to a single page. This simple
      change to the redo logging of undo log significantly affects the
      INSERT and UPDATE performance.
      
      Essentially, we wrote
      	(space_id,page_number,MLOG_2BYTES,2 bytes)
      	(space_id,page_number,MLOG_WRITE_STRING,N+4 bytes)
      instead of the previously written
      	(space_id,page_number,MLOG_UNDO_INSERT,N+2 bytes)
      
      The added redo log volume caused a single-threaded INSERT
      (without innodb_adaptive_hash_index) of
      1,000,000 rows to consume 11 seconds instead of 9 seconds,
      and a subsequent UPDATE of 30,000,000 rows to consume 64 seconds
      instead of 58 seconds. If we omitted all redo logging for the
      undo log, the INSERT would consume only 4 seconds.
      76c62bc6
    • Marko Mäkelä's avatar
      MDEV-15914: Remove trx_t::undo_mutex · 83bd4dd1
      Marko Mäkelä authored
      The trx_t::undo_mutex covered both some main-memory data structures
      (trx_undo_t) and access to undo pages. The trx_undo_t is only
      accessed by the thread that is associated with a running transaction.
      Likewise, each transaction has its private set of undo pages.
      The thread that is associated with an active transaction may
      lock multiple undo pages concurrently, but no other thread may
      lock multiple pages of a foreign transaction.
      
      Concurrent access to the undo logs of an active transaction is possible,
      but trx_undo_get_undo_rec_low() only locks one undo page at a time,
      without ever holding any undo_mutex.
      
      It seems that the trx_t::undo_mutex would have been necessary if
      multi-threaded execution or rollback of a single transaction
      had been implemented in InnoDB.
      83bd4dd1
    • Marko Mäkelä's avatar
      f7cac5e2
    • Marko Mäkelä's avatar
      MDEV-15914: Remove trx_undo_t::empty · ff0000cd
      Marko Mäkelä authored
      Use the value trx_undo_t::top_undo_no == IB_ID_MAX for indicating
      that an undo log is empty.
      ff0000cd
    • Marko Mäkelä's avatar
      c121574d
    • Marko Mäkelä's avatar
      MDEV-15914: Simplify row_search_mvcc() · 7b7e4d67
      Marko Mäkelä authored
      7b7e4d67
    • Marko Mäkelä's avatar
      MDEV-15914: Fast path for rw_trx_hash::find() · 30553aa3
      Marko Mäkelä authored
      rw_trx_hash_t::find(): Implement a fast path for looking up
      the current transaction. This helps transactions that visit
      rows modified by themselves.
      30553aa3
    • Marko Mäkelä's avatar
      Follow-up fix to MDEV-14705: Flush log at shutdown · b4c5e4a7
      Marko Mäkelä authored
      In the merge of commit e7f4e61f
      the call fil_flush_file_spaces(FIL_TYPE_LOG) is necessary.
      Tablespaces will be flushed as part of the redo log
      checkpoint, but the redo log will not necessarily
      be flushed, depending on innodb_flush_method.
      b4c5e4a7
    • Alexey Botchkov's avatar
      MDEV-15915 Add Feature_json status variable. · 6e42d19f
      Alexey Botchkov authored
      Related implementations.
      6e42d19f
    • Marko Mäkelä's avatar
      MDEV-16041 Do not write for null update (properly fix MySQL Bug#29157) · 5569b3eb
      Marko Mäkelä authored
      InnoDB takes a lot of time to perform null updates. The reason is that
      even though an empty update vector was created, InnoDB will go on to
      write undo log records and update the system columns
      DB_TRX_ID and DB_ROLL_PTR in the clustered index, and of course write
      redo log for all this.
      
      This could have been fixed properly in
      commit 54a492ec more than 10 years ago.
      5569b3eb
    • Monty's avatar
      Removed some old #if MYSQL_VERSION · cd48c1e2
      Monty authored
      - Removed never used warning that explicit_defaults_for_timestamp was
        not set
      cd48c1e2
    • Monty's avatar
      Fix a lot of compiler warnings found by -Wunused · 2ccd6716
      Monty authored
      2ccd6716
    • Oleksandr Byelkin's avatar
      MDEV-15732: Assertion `next_free_value % real_increment == offset &&... · 0bdc15d8
      Oleksandr Byelkin authored
      MDEV-15732: Assertion `next_free_value % real_increment == offset && next_free_value >= reserved_until' failed in sequence_definition::adjust_values upon SETVAL for sequence with INCREMENT 0
      
      there was a problem with "next_free_value >= reserved_until" condition:
      SEQUENCE::set_value handle next_free_value & reserved_until after adjust_values() call, so it is incorect to put assert on it in adjust_values()
      0bdc15d8
    • Daniel Black's avatar
      tests: func_math, correct crc32 for mariadb · 7f89d9c9
      Daniel Black authored
      Differences:
      
      MariaDB doesn't support a JSON type therefore the crc32 on those values
      are different.
      
      JSON extract syntax is different.
      
      loaddata_utf8 has 3 duplicate lines removed compared to MySQL version.
      7f89d9c9
    • Shivashankar Nagaraj's avatar
      BUG#26495791: CONTRIBUTION: EXPAND TEST SUITE TO INCLUDE CRC32 TESTS · 1a011e5b
      Shivashankar Nagaraj authored
      From mysql-server:
      09fdfad50764ff6809e7dd5300e9ce1ab727b62a
      e90ae1707e0ca46abc775d1680d1856c4be38b66
      
      described in http://github.com/mysql/mysql-server/pull/157
      
      Apart from external contribution I have added few more additional testcases
      for CRC32() function, which are given below.
      
      New Testcases added:
        ->Verify the crc value of various numeric and string data types(int,
            double, blob, text, json, enum, set)
        ->Verify the crc value when expressions having comparison_operators
            and logical_operators
        ->Verify the crc value for the expression having string_functions,
            arithmetic_functions, json_functions
        ->Verify the crc value for the expression having Geometry functions
            like POINT, LINESTRING, MULTILINESTRING, POLYGON, MULTIPOLYGON
        ->Verify the crc value generated from stored procedures, functions,
            triggers, prepare statement, views.
      
      Fix:
      Patch based on contribution by Daniel Black (Github user: grooverdan)
      
      Reviewed-by: Anitha Gopi anitha.gopi@oracle.com
      Reviewed-by: Srikanth B R srikanth.b.r@oracle.com
      
      RB: 17294
      1a011e5b
    • Marko Mäkelä's avatar
      innobase_init(): Remove some dead code · c5ea43fc
      Marko Mäkelä authored
      Remove the local variable srv_buf_pool_size_org, which was always 0.
      In MySQL 5.7, InnoDB was made a mandatory storage engine, which would
      force InnoDB to start up when executing
      	mysqld --verbose --help
      which is what mysql-test-run.pl is doing as a first step. With a
      large innodb_buffer_pool_size, this would take a long time.
      So, MySQL 5.7 includes a hack that starts up InnoDB with a smaller
      buffer pool when the option --verbose is present.
      c5ea43fc
    • Marko Mäkelä's avatar
      Remove dead code HAVE_LZO1X · 6e04af1b
      Marko Mäkelä authored
      MariaDB uses HAVE_LZO, not HAVE_LZO1X (which was never defined).
      Also, the variable srv_lzo_disabled was never defined or read
      (only declared and assigned to, in unreachable code).
      6e04af1b
    • Marko Mäkelä's avatar
      MDEV-15507 Assertion failed in dict_check_sys_tables on upgrade from 5.5 · c026e2c0
      Marko Mäkelä authored
      The InnoDB system table column SYS_TABLES.MIX_LEN was repurposed
      in InnoDB Plugin for MySQL 5.1, in
      commit 91111174 (MySQL 5.1.46).
      Until MySQL 5.6, it only contained a flag DICT_TF2_TEMPORARY.
      
      MySQL 5.6 introduced a number of flags that were transient
      in nature. One of these was introduced in 5.6.5, originally
      called DICT_TF2_USE_TABLESPACE and later renamed to
      DICT_TF2_USE_FILE_PER_TABLE. MySQL 5.7.6 introduced logic
      that insists that the flag be set for any table that does not
      reside in a shared tablespace, breaking upgrade from MySQL 5.5.
      
      MariaDB does not support shared tablespaces other than the
      InnoDB system tablespace. Also, some dependencies on
      SYS_TABLES.MIX_LEN were removed in an earlier fix:
      MDEV-13084 MariaDB 10.2 crashes on corrupted SYS_TABLES.MIX_LEN field
      (commit e813fe86).
      
      dict_check_sys_tables(): Remove a bogus debug assertion, and add a
      comment that explains how DICT_TF2_USE_FILE_PER_TABLE is used.
      
      dict_table_is_file_per_table(): Remove a bogus debug assertion.
      c026e2c0
    • Igor Babaev's avatar
      Fixed a compiler error · 5bba69f8
      Igor Babaev authored
      5bba69f8
  4. 25 Apr, 2018 5 commits
  5. 24 Apr, 2018 4 commits
    • Igor Babaev's avatar
      Changed the test case for MDEV-15571 · f033fbd9
      Igor Babaev authored
      It has been done to demonstrate that the fix of this bug is good for 10.3
      as well. The previous test case is not good for this purpose because
      10.2 and 10.3 use different rules for determining the types of recursive
      CTEs.
      f033fbd9
    • Marko Mäkelä's avatar
      Merge 10.2 into 10.3 · 7396dfcc
      Marko Mäkelä authored
      7396dfcc
    • Marko Mäkelä's avatar
      MDEV-15030 Add ASAN instrumentation to trx_t Pool · 7b5543b2
      Marko Mäkelä authored
      Pool::mem_free(): Poison the freed memory. Assert that it was
      fully initialized, because the reuse of trx_t objects will
      assume that the objects were previously initialized.
      
      Pool::~Pool(), Pool::get(): Unpoison the allocated memory,
      and mark it initialized.
      
      trx_free(): After invoking Pool::mem_free(), unpoison
      trx_t::mutex and trx_t::undo_mutex, because MutexMonitor
      will access these even for freed trx_t objects.
      7b5543b2
    • Marko Mäkelä's avatar
      8346acaf