Backport of fix for Bug#13581962

mysql-test/r/cast.result:
  Added test result for Bug#13581962,Bug#14096619
mysql-test/r/ctype_utf8mb4.result:
  Added test result for Bug#13581962,Bug#14096619
mysql-test/t/cast.test:
  Added test case for Bug#13581962,Bug#14096619
mysql-test/t/ctype_utf8mb4.test:
  Added test case for Bug#13581962,Bug#14096619
sql/item_func.h:
  limit max length by MY_INT64_NUM_DECIMAL_DIGITS