Post-merge fix.

  Post-merge fix.

SP with local variables with non-ASCII names crashed the server.

The server replaces SP local variable names with NAME_CONST calls
when putting statements into the binary log. It used UTF8-encoded
item names as variable names for the replacement inside NAME_CONST
calls. However, statement string may be encoded by any
known character set by the SET NAMES statement.
The server used byte length of UTF8-encoded names to increment
the position in the query string that led to array index overrun.

Using view columns by their names during an execution of
a prepared SELECT statement or a SELECT statement inside
a SP caused a memory leak.

  Additional test case fix for bug #29338.

Optimization of queries with DETERMINISTIC functions in the
WHERE clause was not effective: sequential scan was always
used.
Now a SF with the DETERMINISTIC flags is treated as constant
when it's arguments are constants (or a SF doesn't has arguments).

fails if a database is not selected prior.

The problem manifested itself when a user tried to
create a routine that had non-fully-qualified identifiers in its bodies
and there was no current database selected.

This is a regression introduced by the fix for Bug 19022:

The patch for Bug 19022 changes the code to always produce a warning
if we can't resolve the current database in the parser. 
In this case this was not necessary, since even though the produced
parsed tree was incorrect, we never re-use sphead
that was obtained at first parsing of CREATE PROCEDURE.
The sphead that is anyhow used is always obtained through db_load_routine,
and there we change the current database to sphead->m_db before
calling yyparse.

The idea of the fix is to resolve the current database directly using 
lex->sphead->m_db member when parsing a stored routine body, when
such is present.

This patch removes the need to reset the current database
when loading a trigger or routine definition into SP cache.
The redundant code will be removed in 5.1.

'No database selected' is reported when calling stored procedures

Remove the offending warning introduced by the fix for Bug
25082
This minimal patch relies on the intrinsic knowledge of the fact that
mysql_change_db is never called with 'force_switch' set to TRUE
when such a warning may be needed:
 * every stored routine belongs to a database (unlike, e.g., a 
user defined function, which does not), so if we're activating the
database of a stored routine, it can never be NULL.
Therefore, this branch is never called for activation.
 * if we're restoring the 'old' current database after routine
execution is complete, we should not issue a warning, since it's OK to 
call a routine without having previously selected the current database.

TODO: 'force_switch' is an ambiguous flag, since we do not actually
have to 'force' the switch in case of stored routines at all.
When we activate the routine's database, we should perform
all the checks as in case of 'use db', and so we already do (in this
case 'force_switch' is unused).
When we load a routine into cache, we should not use mysql_change_db
at all, since there it's enough to call thd->reset_db(). We
do it this way for triggers, but code for routines is different (wrongly). 

TODO: bugs are lurking in replication, since it bypasses mysql_change_db
and calls thd->[re_]set_db to set the current database.
The latter does not change thd->db_charset, thd->sctx->db_access
and thd->variables.collation_database (and this may have nasty side
effects).

These todo items are to be addressed in a separate patch, if at all.

  - BUG#11986: Stored routines and triggers can fail if the code
    has a non-ascii symbol
  - BUG#16291: mysqldump corrupts string-constants with non-ascii-chars
  - BUG#19443: INFORMATION_SCHEMA does not support charsets properly
  - BUG#21249: Character set of SP-var can be ignored
  - BUG#25212: Character set of string constant is ignored (stored routines)
  - BUG#25221: Character set of string constant is ignored (triggers)

There were a few general problems that caused these bugs:
1. Character set information of the original (definition) query for views,
   triggers, stored routines and events was lost.
2. mysqldump output query in client character set, which can be
   inappropriate to encode definition-query.
3. INFORMATION_SCHEMA used strings with mixed encodings to display object
   definition;

1. No query-definition-character set.

In order to compile query into execution code, some extra data (such as
environment variables or the database character set) is used. The problem
here was that this context was not preserved. So, on the next load it can
differ from the original one, thus the result will be different.

The context contains the following data:
  - client character set;
  - connection collation (character set and collation);
  - collation of the owner database;

The fix is to store this context and use it each time we parse (compile)
and execute the object (stored routine, trigger, ...).

2. Wrong mysqldump-output.

The original query can contain several encodings (by means of character set
introducers). The problem here was that we tried to convert original query
to the mysqldump-client character set.

Moreover, we stored queries in different character sets for different
objects (views, for one, used UTF8, triggers used original character set).

The solution is
  - to store definition queries in the original character set;
  - to change SHOW CREATE statement to output definition query in the
    binary character set (i.e. without any conversion);
  - introduce SHOW CREATE TRIGGER statement;
  - to dump special statements to switch the context to the original one
    before dumping and restore it afterwards.

Note, in order to preserve the database collation at the creation time,
additional ALTER DATABASE might be used (to temporary switch the database
collation back to the original value). In this case, ALTER DATABASE
privilege will be required. This is a backward-incompatible change.

3. INFORMATION_SCHEMA showed non-UTF8 strings

The fix is to generate UTF8-query during the parsing, store it in the object
and show it in the INFORMATION_SCHEMA.

Basically, the idea is to create a copy of the original query convert it to
UTF8. Character set introducers are removed and all text literals are
converted to UTF8.

This UTF8 query is intended to provide user-readable output. It must not be
used to recreate the object.  Specialized SHOW CREATE statements should be
used for this.

The reason for this limitation is the following: the original query can
contain symbols from several character sets (by means of character set
introducers).

Example:

  - original query:
    CREATE VIEW v1 AS SELECT _cp1251 'Hello' AS c1;

  - UTF8 query (for INFORMATION_SCHEMA):
    CREATE VIEW v1 AS SELECT 'Hello' AS c1;

Fix some error messages so that all error codes are equivalent in 5.0 and 5.1

Bug 28127 (Some valid identifiers names are not parsed correctly)
Bug 26302 (MySQL server cuts off trailing "*/" from comments in SP/func)

This patch is the second part of a major cleanup, required to fix
Bug 25411 (trigger code truncated).

The root cause of the issue stems from the function skip_rear_comments,
which was a work around to remove "extra" "*/" characters from the query
text, when parsing a query and reusing the text fragments to represent a
view, trigger, function or stored procedure.
The reason for this work around is that "special comments",
like /*!50002 XXX */, were not parsed properly, so that a query like:
  AAA /*!50002 BBB */ CCC
would be seen by the parser as "AAA BBB */ CCC" when the current version
is greater or equal to 5.0.2

The root cause of this stems from how special comments are parsed.
Special comments are really out-of-bound text that appear inside a query,
that affects how the parser behave.
In nature, /*!50002 XXX */ in MySQL is similar to the C concept
of preprocessing :
  #if VERSION >= 50002
  XXX
  #endif

Depending on the current VERSION of the server, either the special comment
should be expanded or it should be ignored, but in all cases the "text" of
the query should be re-written to strip the "/*!50002" and "*/" markers,
which does not belong to the SQL language itself.

Prior to this fix, these markers would leak into :
- the storage format for VIEW,
- the storage format for FUNCTION,
- the storage format for FUNCTION parameters, in mysql.proc (param_list),
- the storage format for PROCEDURE,
- the storage format for PROCEDURE parameters, in mysql.proc (param_list),
- the storage format for TRIGGER,
- the binary log used for replication.

In all cases, not only this cause format corruption, but also provide a vector
for dormant security issues, by allowing to tunnel code that will be activated
after an upgrade.

The proper solution is to deal with special comments strictly during parsing,
when accepting a query from the outside world.
Once a query is parsed and an object is created with a persistant
representation, this object should not arbitrarily mutate after an upgrade.
In short, special comments are a useful but limited feature for MYSQLdump,
when used at an *interface* level to facilitate import/export,
but bloating the server *internal* storage format is *not* the proper way
to deal with configuration management of the user logic.

With this fix:
- the Lex_input_stream class now acts as a comment pre-processor,
and either expands or ignore special comments on the fly.
- MYSQLlex and sql_yacc.yy have been cleaned up to strictly use the
public interface of Lex_input_stream. In particular, how the input stream
accepts or rejects a character is private to Lex_input_stream, and the
internal buffer pointers of that class are strictly private, and should not
be tempered with during parsing.

This caused many changes mostly in sql_lex.cc.

During the code cleanup in case MY_LEX_NUMBER_IDENT,
Bug 28127 (Some valid identifiers names are not parsed correctly)
was found and fixed.

By parsing special comments properly, and removing the function
'skip_rear_comments' [sic],
Bug 26302 (MySQL server cuts off trailing "*/" from comments in SP/func)
has been fixed as well.

            SHOW CREATE TABLE fails

After merge fixes.

  The patch for WL 1563 added a new duplicate key error message so that the
  key name could be provided instead of the key number. But the error code
  for the new message was used even though that did not need to change.

  This could cause unnecessary problems for applications that used the old
  ER_DUP_ENTRY error code to detect duplicate key errors.

            SHOW CREATE TABLE fails

Underlying table names, that merge engine fails to open were not
reported.

With this fix CHECK TABLE issued against merge table reports all
underlying table names that it fails to open. Other statements
are unaffected, that is underlying table names are not included
into error message.

This fix doesn't solve SHOW CREATE TABLE issue.

  Post merge fix.

Integer values with 10 digits may or may not fit into an int column 
(e.g. 2147483647 vs 6147483647).
Thus when creating a temp table column for such an int we must
use bigint instead.
Fixed to use bigint.
Also subsituted a "magic number" with a named constant.

 longer showing SP names.
SHOW CREATE VIEW uses Item::print() methods to reconstruct the 
statement text from the parse tree.
The print() method for stored procedure calls needs allocate 
space to print the function's quoted name.
It was incorrectly calculating the length of the buffer needed 
(was too short).
Fixed to reflect the actual space needed.

Bug#4968 ""Stored procedure crash if cursor opened on altered table"
Bug#6895 "Prepared Statements: ALTER TABLE DROP COLUMN does nothing"
Bug#19182 "CREATE TABLE bar (m INT) SELECT n FROM foo; doesn't work from 
stored procedure."
Bug#19733 "Repeated alter, or repeated create/drop, fails"
Bug#22060 "ALTER TABLE x AUTO_INCREMENT=y in SP crashes server"
Bug#24879 "Prepared Statements: CREATE TABLE (UTF8 KEY) produces a 
growing key length" (this bug is not fixed in 5.0)

Re-execution of CREATE DATABASE, CREATE TABLE and ALTER TABLE 
statements in stored routines or as prepared statements caused
incorrect results (and crashes in versions prior to 5.0.25).

In 5.1 the problem occured only for CREATE DATABASE, CREATE TABLE
SELECT and CREATE TABLE with INDEX/DATA DIRECTOY options).
  
The problem of bugs 4968, 19733, 19282 and 6895 was that functions
mysql_prepare_table, mysql_create_table and mysql_alter_table are not
re-execution friendly: during their operation they modify contents
of LEX (members create_info, alter_info, key_list, create_list),
thus making the LEX unusable for the next execution.
In particular, these functions removed processed columns and keys from
create_list, key_list and drop_list. Search the code in sql_table.cc 
for drop_it.remove() and similar patterns to find evidence.
  
The fix is to supply to these functions a usable copy of each of the
above structures at every re-execution of an SQL statement. 
  
To simplify memory management, LEX::key_list and LEX::create_list
were added to LEX::alter_info, a fresh copy of which is created for
every execution.
  
The problem of crashing bug 22060 stemmed from the fact that the above 
metnioned functions were not only modifying HA_CREATE_INFO structure 
in LEX, but also were changing it to point to areas in volatile memory
of the execution memory root.
   
The patch solves this problem by creating and using an on-stack
copy of HA_CREATE_INFO in mysql_execute_command.

Additionally, this patch splits the part of mysql_alter_table
that analizes and rewrites information from the parser into
a separate function - mysql_prepare_alter_table, in analogy with
mysql_prepare_table, which is renamed to mysql_prepare_create_table.

Before this fix, the parser would sometime change where a token starts by
altering Lex_input_string::tok_start, which later confused the code in
sql_yacc.yy that needs to capture the source code of a SQL statement,
like to represent the body of a stored procedure.

This line of code in sql_lex.cc :

case MY_LEX_USER_VARIABLE_DELIMITER:
  lip->tok_start= lip->ptr; // Skip first `

would <skip the first back quote> ... and cause the bug reported.

In general, the responsibility of sql_lex.cc is to *find* where token are
in the SQL text, but is *not* to make up fake or incomplete tokens.
With a quoted label like `my_label`, the token starts on the first quote.
Extracting the token value should not change that (it did).

With this fix, the lexical analysis has been cleaned up to not change
lip->tok_start (in the case found for this bug).

The functions get_token() and get_quoted_token() now have an extra
parameters, used when some characters from the beginning of the token need
to be skipped when extracting a token value, like when extracting 'AB' from
'0xAB', for example, for a HEX_NUM token.

This exposed a bad assumption in Item_hex_string and Item_bin_string,
which has been fixed:

The assumption was that the string given, 'AB', was in fact preceded in
memory by '0x', which might be false (it can be preceded by "x'" and
followed by "'" -- or not be preceded by valid memory at all)

If a name is needed for Item_hex_string or Item_bin_string, the name is
taken from the original and true source code ('0xAB'), and assigned in
the select_item rule, instead of relying on assumptions related to how
memory is used.

of CHAR variable too great.

execution breaks replication.

When a stored routine is executed, we switch current
database to the database, in which the routine
has been created. When the stored routine finishes,
we switch back to the original database.

The problem was that if the original database does not
exist (anymore) after routine execution, we raised an error.

The fix is to report a warning, and switch to the NULL database.

- 1.84e+15 converted to unsigned bigint should be
  18400000000000000000 < 18446744073709551615.
- The test will still fail on windows, and is extracted
  into a new bug report.

Possible problems: function call could be eliminated from where class and only
be evaluated once; function can be evaluated during table and item setup phase which could
cause side effects not to be registered in binlog.

Fixed with introducing func_item_sp::used_tables() returning the correct table_map constant.

- Stored procedures returning unsinged values returns signed values if
  text protocol is used. The reason is that the stored proceedure item
  Item_func_sp wasn't initializing the member variables properly based
  on the information contained in the associated result field.
- The patch is to upon field-item association, ::fix_fields, initialize
  the member variables in appropriate order.
- Field type of an Item_func_sp was hard coded to MYSQL_TYPE_VARCHAR.
  This is changed to return the type of the actual result field.
- Member function name sp_result_field was refactored to the more 
  appropriate init_result_field.
- Member function name find_and_check_access was refactored to 
  sp_check_access.

Before this fix, the parser would accept illegal code in SQL exceptions
handlers, that later causes the runtime to crash when executing the code,
due to memory violations in the exception handler stack.

The root cause of the problem is instructions within an exception handler
that jumps to code located outside of the handler. This is illegal according
to the SQL 2003 standard, since labels located outside the handler are not
supposed to be visible (they are "out of scope"), so any instruction that
jumps to these labels, like ITERATE or LEAVE, should not parse.

The section of the standard that is relevant for this is :
  SQL:2003 SQL/PSM (ISO/IEC 9075-4:2003)
  section 13.1 <compound statement>,
  syntax rule 4
<quote>
  The scope of the <beginning label> is CS excluding every <SQL schema
  statement> contained in CS and excluding every
  <local handler declaration list> contained in CS. <beginning label> shall
  not be equivalent to any other <beginning label>s within that scope.
</quote>

With this fix, the C++ class sp_pcontext, which represent the "parsing
context" tree (a.k.a symbol table) of a stored procedure, has been changed
as follows:
- constructors have been cleaned up, so that only building a root node for
the tree is public; building nodes inside a tree is not public.
- a new member, m_label_scope, indicates if a given syntactic context
belongs to a DECLARE HANDLER block,
- label resolution, in the method find_label(), has been changed to
implement the restriction of scope regarding labels used in a compound
statement.

The actions in the parser, when parsing the body of a SQL exception handler,
have been changed as follows:
- the implementation of an exception handler (DECLARE HANDLER) now creates
explicitly a new sp_pcontext, to isolate the code inside the handler from
the containing compound statement context.
- registering exception handlers as a result occurs in the parent context,
see the rule sp_hcond_element
- the code in sp_hcond_list has been cleaned up, to avoid code duplication

In addition, the flags IN_SIMPLE_CASE and IN_HANDLER, declared in sp_head.h
have been removed, since they are unused and broken by design (as seen with
Bug 19194 (Right recursion in parser for CASE causes excessive stack usage,
limitation), representing a stack in a single flag is not possible.

Tests in sp-error have been added to show that illegal constructs are now
rejected.

Tests in sp have been added for code coverage, to show that ITERATE or LEAVE
statements are legal when jumping to a label in scope, inside the body of
an exception handler.

result.

For built-in functions like sqrt() function names are hard-coded and can be
compared by pointer. But this isn't the case for a used-defined stored
functions - names there are dynamical and should be compared as strings.

Now the Item_func::eq() function employs my_strcasecmp() function to compare
used-defined stored functions names.