InnoDB: Performance optimizations based on OProfile analysis

innobase/btr/btr0btr.c:
  Eliminate some buf_frame_align() calls.
  Make use of the page_rec_is_infimum(), page_rec_is_supremum()
  and page_rec_is_user_rec() functions.
  Replace some index->table->comp with page_is_comp().
  Eliminate some variables to reduce register spilling on x86.
  Note that page_is_comp() may return nonzero instead of TRUE.
  Note that rec_offs_comp() may return nonzero instead of TRUE.
innobase/btr/btr0cur.c:
  Eliminate some buf_frame_align() calls.
  Replace some index->table->comp with
  page_is_comp() or rec_offs_comp().
  Eliminate some variables to reduce register spilling on x86.
  Note that page_is_comp() may return nonzero instead of TRUE.
  Note that rec_offs_comp() may return nonzero instead of TRUE.
  Remove an extra mem_heap_create() call from btr_cur_update_in_place().
  Add "page" parameter to lock_rec_store_on_page_infimum().
  Add some UNIV_LIKELY() and UNIV_UNLIKELY() hints.
  
  btr_estimate_number_of_different_key_vals(): Rename the
  offsets_* variables to be more descriptive and eliminate one
  rec_get_offsets() and one page_rec_get_next() call in the loop.
innobase/btr/btr0pcur.c:
  Eliminate some buf_frame_align() calls.
  Make use of the page_rec_is_infimum(), page_rec_is_supremum()
  and page_rec_is_user_rec() functions.
  Replace some index->table->comp with page_is_comp().
  Eliminate some variables to reduce register spilling on x86.
  Note that page_is_comp() may return nonzero instead of TRUE.
  Make some ut_a() assertions ut_ad() ones to improve performance.
  Add some UNIV_LIKELY() and UNIV_UNLIKELY() hints.
innobase/btr/btr0sea.c:
  Make use of page_rec_is_infimum() and page_rec_is_supremum().
  Eliminate some buf_frame_align() calls.
  Add some UNIV_UNLIKELY and UNIV_LIKELY hints.
  Turn some assertions into debug assertions.
innobase/dict/dict0crea.c:
  Replace FALSE/TRUE ibool comp with zero/nonzero ulint comp
innobase/ibuf/ibuf0ibuf.c:
  Make use of page_rec_is_infimum() and page_rec_is_supremum().
  Add some UNIV_UNLIKELY and UNIV_LIKELY hints.
  ibuf_get_merge_page_nos(): Rename parameter "first_rec" to "rec"
  and eliminate local variable "rec".
innobase/include/btr0btr.h:
  Replace FALSE/TRUE ibool comp with zero/nonzero ulint comp
innobase/include/buf0buf.h:
  Rename buf_frame_get_modify_clock()
  to buf_block_get_modify_clock().
innobase/include/buf0buf.ic:
  Rename buf_frame_get_modify_clock()
  to buf_block_get_modify_clock() and
  remove the buf_block_align() call.
innobase/include/lock0lock.h:
  lock_rec_store_on_page_infimum(): Add parameter "page"
innobase/include/mach0data.h:
  Add mach_encode_2() and mach_decode_2().
innobase/include/mach0data.ic:
  Add mach_encode_2() and mach_decode_2().
innobase/include/page0cur.h:
  Add const qualifier to page_cur_is_before_first()
  and page_cur_is_after_last().
innobase/include/page0cur.ic:
  Make use of page_rec_is_infimum() and page_rec_is_supremum().
innobase/include/page0page.h:
  Remove page_rec_is_first_user_rec() and page_rec_is_last_user_rec().
  Add page_rec_is_infimum() and page_rec_is_supremum().
  Replace FALSE/TRUE ibool comp with zero/nonzero ulint comp.
innobase/include/page0page.ic:
  Remove page_rec_is_first_user_rec() and page_rec_is_last_user_rec().
  Add page_rec_is_infimum() and page_rec_is_supremum().
  Replace FALSE/TRUE ibool comp with zero/nonzero ulint comp.
  Add UNIV_UNLIKELY, UNIV_LIKELY and UNIV_EXPECT hints.
  Reduce the number of buf_frame_align() calls.
innobase/include/rem0rec.ic:
  rec_offs_comp(): Return zero or nonzero instead of FALSE or TRUE.
innobase/include/row0mysql.h:
  Replace FALSE/TRUE ibool comp with zero/nonzero ulint comp.
innobase/lock/lock0lock.c:
  Replace FALSE/TRUE ibool comp with zero/nonzero ulint comp.
  Remove parameter "comp" from lock_rec_get_next(),
  lock_rec_has_expl() and lock_rec_other_has_expl_req().
  Add parameter "page" to lock_rec_store_on_page_infimum().
  Add UNIV_UNLIKELY hints.
  Reduce the number of buf_frame_align() calls.
  Make use of page_rec_is_infimum(), page_rec_is_supremum() and
  page_rec_is_user_rec().
  Move the "comp" flag outside some loops.
innobase/mtr/mtr0log.c:
  Replace index->table->comp with page_rec_is_comp().
innobase/page/page0cur.c:
  Replace index->table->comp with page_is_comp() or page_rec_is_comp().
  Eliminate some buf_frame_align() calls.
  Add some debug assertions.
innobase/page/page0page.c:
  Optimize page_dir_find_owner_slot(). Compare the record offset
  16 bits at a time, because that seems to be the only way to avoid
  register spilling on x86.
  Replace FALSE/TRUE ibool comp with zero/nonzero ulint comp.
  Remove parameter "page" of page_delete_rec_list_write_log().
  Make use of page_rec_is_infimum().
innobase/rem/rem0cmp.c:
  Replace FALSE/TRUE ibool comp with zero/nonzero ulint comp.
innobase/row/row0ins.c:
  Make use of page_rec_is_infimum() and page_rec_is_supremum().
  Reduce the amount of buf_frame_align() calls.
  row_ins_index_entry_low(): Disable assertion about column count
  unless #ifdef UNIV_DEBUG.
innobase/row/row0mysql.c:
  Replace FALSE/TRUE ibool comp with zero/nonzero ulint comp.
innobase/row/row0row.c:
  Eliminate some buf_frame_align() calls.
  Make use of page_rec_is_infimum().
innobase/row/row0sel.c:
  Make use of page_rec_is_supremum() and page_rec_is_infimum().
  Turn some assertions into debug assertions.
  Add UNIV_LIKELY and UNIV_UNLIKELY hints.
  
  row_search_for_mysql(): Eliminate local variables "moved",
  "cons_read_requires_clust_rec", "was_lock_wait", "shortcut",
  "success" and "comp". Replace some of them with goto's.
  Disable variable "cnt" unless #ifdef UNIV_SEARCH_DEBUG.
innobase/row/row0vers.c:
  Replace FALSE/TRUE ibool comp with zero/nonzero ulint comp.
  Replace index->table->comp with page_rec_is_comp().
  Eliminate some buf_frame_align() calls.

innobase/btr/btr0btr.c

@@ -138,13 +138,13 @@ btr_root_get(
 	ulint	space;
 	ulint	root_page_no;
 	page_t*	root;
-	ibool	comp = UT_LIST_GET_FIRST(tree->tree_indexes)->table->comp;
 	
 	space = dict_tree_get_space(tree);
 	root_page_no = dict_tree_get_page(tree);
 
 	root = btr_page_get(space, root_page_no, RW_X_LATCH, mtr);
-	ut_a(page_is_comp(root) == comp);
+	ut_a(!!page_is_comp(root) ==
+			UT_LIST_GET_FIRST(tree->tree_indexes)->table->comp);
 	
 	return(root);
 }
@@ -164,21 +164,19 @@ btr_get_prev_user_rec(
 	page_t*	page;
 	page_t*	prev_page;
 	ulint	prev_page_no;
-	rec_t*	prev_rec;
 	ulint	space;
 
-	page = buf_frame_align(rec);
-	
-	if (page_get_infimum_rec(page) != rec) {
+	if (!page_rec_is_infimum(rec)) {
 
-		prev_rec = page_rec_get_prev(rec);
+		rec_t*	prev_rec = page_rec_get_prev(rec);
 
-		if (page_get_infimum_rec(page) != prev_rec) {
+		if (!page_rec_is_infimum(prev_rec)) {
 
 			return(prev_rec);
 		}
 	}
 	
+	page = buf_frame_align(rec);
 	prev_page_no = btr_page_get_prev(page, mtr);
 	space = buf_frame_get_space_id(page);
 	
@@ -193,9 +191,7 @@ btr_get_prev_user_rec(
 		      				MTR_MEMO_PAGE_X_FIX)));
 		ut_a(page_is_comp(prev_page) == page_is_comp(page));
 
-		prev_rec = page_rec_get_prev(page_get_supremum_rec(prev_page));
-
-		return(prev_rec);
+		return(page_rec_get_prev(page_get_supremum_rec(prev_page)));
 	}
 
 	return(NULL);
@@ -216,21 +212,19 @@ btr_get_next_user_rec(
 	page_t*	page;
 	page_t*	next_page;
 	ulint	next_page_no;
-	rec_t*	next_rec;
 	ulint	space;
 
-	page = buf_frame_align(rec);
-	
-	if (page_get_supremum_rec(page) != rec) {
+	if (!page_rec_is_supremum(rec)) {
 
-		next_rec = page_rec_get_next(rec);
+		rec_t*	next_rec = page_rec_get_next(rec);
 
-		if (page_get_supremum_rec(page) != next_rec) {
+		if (!page_rec_is_supremum(next_rec)) {
 
 			return(next_rec);
 		}
 	}
 	
+	page = buf_frame_align(rec);
 	next_page_no = btr_page_get_next(page, mtr);
 	space = buf_frame_get_space_id(page);
 	
@@ -245,9 +239,7 @@ btr_get_next_user_rec(
 		      				MTR_MEMO_PAGE_X_FIX)));
 
 		ut_a(page_is_comp(next_page) == page_is_comp(page));
-		next_rec = page_rec_get_next(page_get_infimum_rec(next_page));
-
-		return(next_rec);
+		return(page_rec_get_next(page_get_infimum_rec(next_page)));
 	}
 
 	return(NULL);
@@ -574,8 +566,7 @@ btr_page_get_father_for_rec(
 
 	ut_ad(mtr_memo_contains(mtr, dict_tree_get_lock(tree),
 							MTR_MEMO_X_LOCK));
-	ut_a(user_rec != page_get_supremum_rec(page));
-	ut_a(user_rec != page_get_infimum_rec(page));
+	ut_a(page_rec_is_user_rec(user_rec));
 	
 	ut_ad(dict_tree_get_page(tree) != buf_frame_get_page_no(page));
 
@@ -599,6 +590,7 @@ btr_page_get_father_for_rec(
 
 	if (btr_node_ptr_get_child_page_no(node_ptr, offsets) !=
                                                 buf_frame_get_page_no(page)) {
+		rec_t*	print_rec;
 		fputs("InnoDB: Dump of the child page:\n", stderr);
 		buf_page_print(buf_frame_align(page));
 		fputs("InnoDB: Dump of the parent page:\n", stderr);
@@ -613,11 +605,10 @@ btr_page_get_father_for_rec(
 			(ulong)
 			btr_node_ptr_get_child_page_no(node_ptr, offsets),
 			(ulong) buf_frame_get_page_no(page));
-		offsets = rec_get_offsets(page_rec_get_next(
-				page_get_infimum_rec(page)), index,
+		print_rec = page_rec_get_next(page_get_infimum_rec(page));
+		offsets = rec_get_offsets(print_rec, index,
 				offsets, ULINT_UNDEFINED, &heap);
-		page_rec_print(page_rec_get_next(page_get_infimum_rec(page)),
-					offsets);
+		page_rec_print(print_rec, offsets);
 		offsets = rec_get_offsets(node_ptr, index, offsets,
 					ULINT_UNDEFINED, &heap);
 		page_rec_print(node_ptr, offsets);
@@ -664,7 +655,7 @@ btr_create(
 	ulint	type,	/* in: type of the index */
 	ulint	space,	/* in: space where created */
 	dulint	index_id,/* in: index id */
-	ibool	comp,	/* in: TRUE=compact page format */
+	ulint	comp,	/* in: nonzero=compact page format */
 	mtr_t*	mtr)	/* in: mini-transaction handle */
 {
 	ulint		page_no;
@@ -855,11 +846,12 @@ btr_page_reorganize_low(
 
 	ut_ad(mtr_memo_contains(mtr, buf_block_align(page),
 			      				MTR_MEMO_PAGE_X_FIX));
+	ut_ad(!!page_is_comp(page) == index->table->comp);
 	data_size1 = page_get_data_size(page);
 	max_ins_size1 = page_get_max_insert_size_after_reorganize(page, 1);
 
 	/* Write the log record */
-	mlog_open_and_write_index(mtr, page, index, index->table->comp
+	mlog_open_and_write_index(mtr, page, index, page_is_comp(page)
 			? MLOG_COMP_PAGE_REORGANIZE
 			: MLOG_PAGE_REORGANIZE, 0);
 
@@ -878,7 +870,7 @@ btr_page_reorganize_low(
 	/* Recreate the page: note that global data on page (possible
 	segment headers, next page-field, etc.) is preserved intact */
 
-	page_create(page, mtr, index->table->comp);
+	page_create(page, mtr, page_is_comp(page));
 	buf_block_align(page)->check_index_page_at_flush = TRUE;
 	
 	/* Copy the records from the temporary space to the recreated page;
@@ -1071,7 +1063,7 @@ btr_root_raise_and_insert(
 	as there is no lower alphabetical limit to records in the leftmost
 	node of a level: */
 
-	btr_set_min_rec_mark(node_ptr_rec, cursor->index->table->comp, mtr);
+	btr_set_min_rec_mark(node_ptr_rec, page_is_comp(root), mtr);
 		
 	/* Free the memory heap */
 	mem_heap_free(heap);
@@ -1152,7 +1144,6 @@ btr_page_get_split_rec_to_right(
 {
 	page_t*	page;
 	rec_t*	insert_point;
-	rec_t*	supremum;
 
 	page = btr_cur_get_page(cursor);
 	insert_point = btr_cur_get_rec(cursor);
@@ -1161,13 +1152,22 @@ btr_page_get_split_rec_to_right(
 	the previous insert on the same page, we assume that there is a
 	pattern of sequential inserts here. */
 
-	if (page_header_get_ptr(page, PAGE_LAST_INSERT) == insert_point) {
+	if (UNIV_LIKELY(page_header_get_ptr(page, PAGE_LAST_INSERT)
+				== insert_point)) {
 
-	     	supremum = page_get_supremum_rec(page);
-	     	 			
-		if (page_rec_get_next(insert_point) != supremum
-		    && page_rec_get_next(page_rec_get_next(insert_point))
-			!= supremum) {
+		rec_t*	next_rec;
+
+		next_rec = page_rec_get_next(insert_point);
+
+		if (page_rec_is_supremum(next_rec)) {
+		split_at_new:
+			/* Split at the new record to insert */
+	     		*split_rec = NULL;
+		} else {
+			rec_t*	next_next_rec = page_rec_get_next(next_rec);
+			if (page_rec_is_supremum(next_next_rec)) {
+				goto split_at_new;
+			}
 
 			/* If there are >= 2 user records up from the insert
 			point, split all but 1 off. We want to keep one because
@@ -1176,12 +1176,8 @@ btr_page_get_split_rec_to_right(
 			search position just by looking at the records on this
 			page. */
 		
-			*split_rec = page_rec_get_next(
-					page_rec_get_next(insert_point));
-		} else {
-			/* Else split at the new record to insert */
-	     		*split_rec = NULL;
-	     	}
+			*split_rec = next_next_rec;
+		}
 
 		return(TRUE);
 	}
@@ -1221,7 +1217,7 @@ btr_page_get_sure_split_rec(
 	page = btr_cur_get_page(cursor);
 	
 	insert_size = rec_get_converted_size(cursor->index, tuple);
-	free_space  = page_get_free_space_of_empty(cursor->index->table->comp);
+	free_space  = page_get_free_space_of_empty(page_is_comp(page));
 
 	/* free_space is now the free space of a created new page */
 
@@ -1283,11 +1279,8 @@ btr_page_get_sure_split_rec(
 				} else {
 					next_rec = page_rec_get_next(rec);
 				}
-				if (next_rec != page_get_supremum_rec(page)) {
-					if (UNIV_LIKELY_NULL(heap)) {
-						mem_heap_free(heap);
-					}
-					return(next_rec);
+				if (!page_rec_is_supremum(next_rec))) {
+					rec = next_rec;
 				}
                     	}
 
@@ -1330,13 +1323,12 @@ btr_page_insert_fits(
 
 	ut_ad(!split_rec == !offsets);
 	ut_ad(!offsets
-		|| cursor->index->table->comp == rec_offs_comp(offsets));
+		|| page_is_comp(page) == !!rec_offs_comp(offsets));
 	ut_ad(!offsets
 		|| rec_offs_validate(split_rec, cursor->index, offsets));
-	ut_ad(page_is_comp(page) == cursor->index->table->comp);
 
 	insert_size = rec_get_converted_size(cursor->index, tuple);
-	free_space  = page_get_free_space_of_empty(cursor->index->table->comp);
+	free_space  = page_get_free_space_of_empty(page_is_comp(page));
 
 	/* free_space is now the free space of a created new page */
 
@@ -1833,14 +1825,15 @@ void
 btr_set_min_rec_mark_log(
 /*=====================*/
 	rec_t*	rec,	/* in: record */
-	ibool	comp,	/* TRUE=compact record format */
+	ulint	comp,	/* nonzero=compact record format */
 	mtr_t*	mtr)	/* in: mtr */
 {
 	mlog_write_initial_log_record(rec,
 		comp ? MLOG_COMP_REC_MIN_MARK : MLOG_REC_MIN_MARK, mtr);
 
 	/* Write rec offset as a 2-byte ulint */
-	mlog_catenate_ulint(mtr, rec - buf_frame_align(rec), MLOG_2BYTES);
+	mlog_catenate_ulint(mtr, ut_align_offset(rec, UNIV_PAGE_SIZE),
+								MLOG_2BYTES);
 }
 
 /********************************************************************
@@ -1853,7 +1846,7 @@ btr_parse_set_min_rec_mark(
 			/* out: end of log record or NULL */
 	byte*	ptr,	/* in: buffer */
 	byte*	end_ptr,/* in: buffer end */
-	ibool	comp,	/* in: TRUE=compact page format */
+	ulint	comp,	/* in: nonzero=compact page format */
 	page_t*	page,	/* in: page or NULL */
 	mtr_t*	mtr)	/* in: mtr or NULL */
 {
@@ -1865,6 +1858,8 @@ btr_parse_set_min_rec_mark(
 	}
 
 	if (page) {
+		ut_a(!page_is_comp(page) == !comp);
+
 		rec = page + mach_read_from_2(ptr);
 
 		btr_set_min_rec_mark(rec, comp, mtr);
@@ -1880,7 +1875,7 @@ void
 btr_set_min_rec_mark(
 /*=================*/
 	rec_t*	rec,	/* in: record */
-	ibool	comp,	/* in: TRUE=compact page format */
+	ulint	comp,	/* in: nonzero=compact page format */
 	mtr_t*	mtr)	/* in: mtr */
 {
 	ulint	info_bits;
@@ -2009,11 +2004,12 @@ btr_compress(
 	ulint		max_ins_size;
 	ulint		max_ins_size_reorg;
 	ulint		level;
-	ibool		comp	= cursor->index->table->comp;
+	ulint		comp;
 
 	page = btr_cur_get_page(cursor);
 	tree = btr_cur_get_tree(cursor);
-	ut_a(comp == page_is_comp(page));
+	comp = page_is_comp(page);
+	ut_a(!!comp == cursor->index->table->comp);
 
 	ut_ad(mtr_memo_contains(mtr, dict_tree_get_lock(tree),
 							MTR_MEMO_X_LOCK));
@@ -2056,7 +2052,7 @@ btr_compress(
 	
 	n_recs = page_get_n_recs(page);
 	data_size = page_get_data_size(page);
-	ut_a(page_is_comp(merge_page) == page_is_comp(page));
+	ut_a(page_is_comp(merge_page) == comp);
 
 	max_ins_size_reorg = page_get_max_insert_size_after_reorganize(
 							merge_page, n_recs);
@@ -2251,10 +2247,9 @@ btr_discard_page(
 
 		node_ptr = page_rec_get_next(page_get_infimum_rec(merge_page));
 
-		ut_ad(node_ptr != page_get_supremum_rec(merge_page));
+		ut_ad(page_rec_is_user_rec(node_ptr));
 
-		btr_set_min_rec_mark(node_ptr,
-					cursor->index->table->comp, mtr);
+		btr_set_min_rec_mark(node_ptr, page_is_comp(merge_page), mtr);
 	}	
 	
 	btr_node_ptr_delete(tree, page, mtr);
@@ -2499,8 +2494,8 @@ btr_index_rec_validate(
 	*offsets_ = (sizeof offsets_) / sizeof *offsets_;
 
 	page = buf_frame_align(rec);
-	
-	if (index->type & DICT_UNIVERSAL) {
+
+	if (UNIV_UNLIKELY(index->type & DICT_UNIVERSAL)) {
 	        /* The insert buffer index tree can contain records from any
 	        other index: we cannot check the number of fields or
 	        their length */
@@ -2508,9 +2503,18 @@ btr_index_rec_validate(
 	        return(TRUE);
 	}
 
+	if (UNIV_UNLIKELY(!!page_is_comp(page) != index->table->comp)) {
+		btr_index_rec_validate_report(page, rec, index);
+		fprintf(stderr, "InnoDB: compact flag=%lu, should be %lu\n",
+			(ulong) !!page_is_comp(page),
+			(ulong) index->table->comp);
+		return(FALSE);
+	}
+
 	n = dict_index_get_n_fields(index);
 
-	if (!index->table->comp && rec_get_n_fields_old(rec) != n) {
+	if (!page_is_comp(page)
+			&& UNIV_UNLIKELY(rec_get_n_fields_old(rec) != n)) {
 		btr_index_rec_validate_report(page, rec, index);
 		fprintf(stderr, "InnoDB: has %lu fields, should have %lu\n",
 			(ulong) rec_get_n_fields_old(rec), (ulong) n);
@@ -2774,7 +2778,7 @@ loop:
 	if (level > 0 && left_page_no == FIL_NULL) {
 		ut_a(REC_INFO_MIN_REC_FLAG & rec_get_info_bits(
 			page_rec_get_next(page_get_infimum_rec(page)),
-				index->table->comp));
+				page_is_comp(page)));
 	}
 
 	if (buf_frame_get_page_no(page) != dict_tree_get_page(tree)) {
@@ -2930,7 +2934,7 @@ node_ptr_fails:
 	mtr_commit(&mtr);
 
 	if (right_page_no != FIL_NULL) {
-		ibool	comp = page_is_comp(page);
+		ulint	comp = page_is_comp(page);
 		mtr_start(&mtr);
 	
 		page = btr_page_get(space, right_page_no, RW_X_LATCH, &mtr);

innobase/btr/btr0cur.c

@@ -505,8 +505,9 @@ retry_page_get:
 
 			if (level > 0) {
 				/* x-latch the page */
-				ut_a(page_is_comp(btr_page_get(space,
-						page_no, RW_X_LATCH, mtr))
+				page = btr_page_get(space,
+						page_no, RW_X_LATCH, mtr);
+				ut_a(!!page_is_comp(page)
 						== index->table->comp);
 			}
 
@@ -961,7 +962,7 @@ calculate_sizes_again:
 	rec_size = rec_get_converted_size(index, entry);
 
 	if (rec_size >=
-		ut_min(page_get_free_space_of_empty(index->table->comp) / 2,
+		ut_min(page_get_free_space_of_empty(page_is_comp(page)) / 2,
 		REC_MAX_DATA_SIZE)) {
 
 		/* The record is so big that we have to store some fields
@@ -1027,7 +1028,7 @@ calculate_sizes_again:
 
 	*rec = page_cur_insert_rec_low(page_cursor, entry, index,
 							NULL, NULL, mtr);
-	if (!(*rec)) {
+	if (UNIV_UNLIKELY(!(*rec))) {
 		/* If the record did not fit, reorganize */
 		btr_page_reorganize(page, index, mtr);
 
@@ -1039,7 +1040,7 @@ calculate_sizes_again:
 
 		*rec = page_cur_tuple_insert(page_cursor, entry, index, mtr);
 
-		if (!*rec) {
+		if (UNIV_UNLIKELY(!*rec)) {
 			fputs("InnoDB: Error: cannot insert tuple ", stderr);
 			dtuple_print(stderr, entry);
 			fputs(" into ", stderr);
@@ -1166,7 +1167,7 @@ btr_cur_pessimistic_insert(
 	}
 
 	if (rec_get_converted_size(index, entry) >=
-		ut_min(page_get_free_space_of_empty(index->table->comp) / 2,
+		ut_min(page_get_free_space_of_empty(page_is_comp(page)) / 2,
 		REC_MAX_DATA_SIZE)) {
 
 		/* The record is so big that we have to store some fields
@@ -1293,9 +1294,11 @@ btr_cur_update_in_place_log(
 	mtr_t*		mtr)		/* in: mtr */
 {
 	byte*	log_ptr;
+	page_t*	page	= ut_align_down(rec, UNIV_PAGE_SIZE);
 	ut_ad(flags < 256);
+	ut_ad(!!page_is_comp(page) == index->table->comp);
 
-	log_ptr = mlog_open_and_write_index(mtr, rec, index, index->table->comp
+	log_ptr = mlog_open_and_write_index(mtr, rec, index, page_is_comp(page)
 			? MLOG_COMP_REC_UPDATE_IN_PLACE
 			: MLOG_REC_UPDATE_IN_PLACE,
 			1 + DATA_ROLL_PTR_LEN + 14 + 2 + MLOG_BUF_MARGIN);
@@ -1317,7 +1320,7 @@ btr_cur_update_in_place_log(
 
 	log_ptr = row_upd_write_sys_vals_to_log(index, trx, roll_ptr, log_ptr,
 									mtr);
-	mach_write_to_2(log_ptr, rec - buf_frame_align(rec));
+	mach_write_to_2(log_ptr, ut_align_offset(rec, UNIV_PAGE_SIZE));
 	log_ptr += 2;
 
 	row_upd_index_write_log(update, log_ptr, mtr);
@@ -1374,18 +1377,11 @@ btr_cur_parse_update_in_place(
 	
 	ptr = row_upd_index_parse(ptr, end_ptr, heap, &update);
 
-	if (ptr == NULL) {
-		mem_heap_free(heap);
-		
-		return(NULL);
+	if (!ptr || !page) {
+		goto func_exit;
 	}
 
-	if (!page) {
-		mem_heap_free(heap);
-
-		return(ptr);
-	}
-	
+	ut_a(!!page_is_comp(page) == index->table->comp);
 	rec = page + rec_offset;
 	
 	/* We do not need to reserve btr_search_latch, as the page is only
@@ -1400,6 +1396,7 @@ btr_cur_parse_update_in_place(
 
 	row_upd_rec_in_place(rec, offsets, update);
 
+func_exit:
 	mem_heap_free(heap);
 
 	return(ptr);
@@ -1438,7 +1435,6 @@ btr_cur_update_in_place(
 	rec = btr_cur_get_rec(cursor);
 	index = cursor->index;
 	trx = thr_get_trx(thr);
-	heap = mem_heap_create(100);
 	offsets = rec_get_offsets(rec, index, offsets, ULINT_UNDEFINED, &heap);
 
 	if (btr_cur_print_record_ops && thr) {
@@ -1449,7 +1445,7 @@ btr_cur_update_in_place(
 	/* Do lock checking and undo logging */
 	err = btr_cur_upd_lock_and_undo(flags, cursor, update, cmpl_info,
 							thr, &roll_ptr);
-	if (err != DB_SUCCESS) {
+	if (UNIV_UNLIKELY(err != DB_SUCCESS)) {
 
 		if (UNIV_LIKELY_NULL(heap)) {
 			mem_heap_free(heap);
@@ -1458,6 +1454,8 @@ btr_cur_update_in_place(
 	}
 
 	block = buf_block_align(rec);
+	ut_ad(!!page_is_comp(buf_block_get_frame(block))
+				== index->table->comp);
 
 	if (block->is_hashed) {
 		/* The function row_upd_changes_ord_field_binary works only
@@ -1481,7 +1479,8 @@ btr_cur_update_in_place(
 	/* FIXME: in a mixed tree, all records may not have enough ordering
 	fields for btr search: */
 
-	was_delete_marked = rec_get_deleted_flag(rec, index->table->comp);
+	was_delete_marked = rec_get_deleted_flag(rec,
+				page_is_comp(buf_block_get_frame(block)));
 
 	row_upd_rec_in_place(rec, offsets, update);
 
@@ -1491,7 +1490,8 @@ btr_cur_update_in_place(
 
 	btr_cur_update_in_place_log(flags, rec, index, update, trx, roll_ptr,
 									mtr);
-	if (was_delete_marked && !rec_get_deleted_flag(rec, index->table->comp)) {
+	if (was_delete_marked && !rec_get_deleted_flag(rec,
+				page_is_comp(buf_block_get_frame(block)))) {
 		/* The new updated record owns its possible externally
 		stored fields */
 
@@ -1597,7 +1597,7 @@ btr_cur_optimistic_update(
 	new_rec_size = rec_get_converted_size(index, new_entry);
 	
 	if (new_rec_size >=
-			page_get_free_space_of_empty(index->table->comp) / 2) {
+			page_get_free_space_of_empty(page_is_comp(page)) / 2) {
 
 		mem_heap_free(heap);		
 
@@ -1644,7 +1644,7 @@ btr_cur_optimistic_update(
 	explicit locks on rec, before deleting rec (see the comment in
 	.._pessimistic_update). */
 
-	lock_rec_store_on_page_infimum(rec);
+	lock_rec_store_on_page_infimum(page, rec);
 
 	btr_search_update_hash_on_delete(cursor);
 
@@ -1665,7 +1665,7 @@ btr_cur_optimistic_update(
 
 	ut_a(rec); /* <- We calculated above the insert would fit */
 
-	if (!rec_get_deleted_flag(rec, index->table->comp)) {
+	if (!rec_get_deleted_flag(rec, page_is_comp(page))) {
 		/* The new inserted record owns its possible externally
 		stored fields */
 
@@ -1814,7 +1814,7 @@ btr_cur_pessimistic_update(
 		}
 		
 		success = fsp_reserve_free_extents(&n_reserved,
-						cursor->index->space,
+						index->space,
 						n_extents, reserve_flag, mtr);
 		if (!success) {
 			err = DB_OUT_OF_FILE_SPACE;
@@ -1858,14 +1858,14 @@ btr_cur_pessimistic_update(
 
 	ext_vect = mem_heap_alloc(heap, sizeof(ulint)
 					* dict_index_get_n_fields(index));
-	ut_ad(!cursor->index->table->comp || !rec_get_node_ptr_flag(rec));
+	ut_ad(!page_is_comp(page) || !rec_get_node_ptr_flag(rec));
 	offsets = rec_get_offsets(rec, index, offsets,
 					ULINT_UNDEFINED, &heap);
 	n_ext_vect = btr_push_update_extern_fields(ext_vect, offsets, update);
 
-	if (rec_get_converted_size(index, new_entry) >=
-		ut_min(page_get_free_space_of_empty(index->table->comp) / 2,
-		REC_MAX_DATA_SIZE)) {
+	if (UNIV_UNLIKELY(rec_get_converted_size(index, new_entry) >=
+		ut_min(page_get_free_space_of_empty(page_is_comp(page)) / 2,
+		REC_MAX_DATA_SIZE))) {
 
                 big_rec_vec = dtuple_convert_big_rec(index, new_entry,
                 					ext_vect, n_ext_vect);
@@ -1887,7 +1887,7 @@ btr_cur_pessimistic_update(
 	delete the lock structs set on the root page even if the root
 	page carries just node pointers. */
 
-	lock_rec_store_on_page_infimum(rec);
+	lock_rec_store_on_page_infimum(buf_frame_align(rec), rec);
 
 	btr_search_update_hash_on_delete(cursor);
 
@@ -1965,8 +1965,7 @@ return_after_reservations:
 	mem_heap_free(heap);
 
 	if (n_extents > 0) {
-		fil_space_release_free_extents(cursor->index->space,
-							n_reserved);
+		fil_space_release_free_extents(index->space, n_reserved);
 	}
 
 	*big_rec = big_rec_vec;
@@ -1995,7 +1994,10 @@ btr_cur_del_mark_set_clust_rec_log(
 	ut_ad(flags < 256);
 	ut_ad(val <= 1);
 
-	log_ptr = mlog_open_and_write_index(mtr, rec, index, index->table->comp
+	ut_ad(!!page_rec_is_comp(rec) == index->table->comp);
+
+	log_ptr = mlog_open_and_write_index(mtr, rec, index,
+			page_rec_is_comp(rec)
 			? MLOG_COMP_REC_CLUST_DELETE_MARK
 			: MLOG_REC_CLUST_DELETE_MARK,
 			1 + 1 + DATA_ROLL_PTR_LEN + 14 + 2);
@@ -2012,7 +2014,7 @@ btr_cur_del_mark_set_clust_rec_log(
 
 	log_ptr = row_upd_write_sys_vals_to_log(index, trx, roll_ptr, log_ptr,
 									mtr);
-	mach_write_to_2(log_ptr, rec - buf_frame_align(rec));
+	mach_write_to_2(log_ptr, ut_align_offset(rec, UNIV_PAGE_SIZE));
 	log_ptr += 2;
 
 	mlog_close(mtr, log_ptr);
@@ -2039,6 +2041,8 @@ btr_cur_parse_del_mark_set_clust_rec(
 	ulint	offset;
 	rec_t*	rec;
 
+	ut_ad(!!page_is_comp(page) == index->table->comp);
+
 	if (end_ptr < ptr + 2) {
 
 		return(NULL);
@@ -2087,7 +2091,7 @@ btr_cur_parse_del_mark_set_clust_rec(
 		is only being recovered, and there cannot be a hash index to
 		it. */
 
-		rec_set_deleted_flag(rec, index->table->comp, val);
+		rec_set_deleted_flag(rec, page_is_comp(page), val);
 	}
 	
 	return(ptr);
@@ -2161,7 +2165,7 @@ btr_cur_del_mark_set_clust_rec(
 		rw_lock_x_lock(&btr_search_latch);
 	}
 
-	rec_set_deleted_flag(rec, index->table->comp, val);
+	rec_set_deleted_flag(rec, rec_offs_comp(offsets), val);
 
 	trx = thr_get_trx(thr);
 	
@@ -2486,6 +2490,7 @@ btr_cur_pessimistic_delete(
 	ulint		n_reserved;
 	ibool		success;
 	ibool		ret		= FALSE;
+	ulint		level;
 	mem_heap_t*	heap;
 	ulint*		offsets;
 	
@@ -2522,15 +2527,15 @@ btr_cur_pessimistic_delete(
 	/* Free externally stored fields if the record is neither
 	a node pointer nor in two-byte format.
 	This avoids an unnecessary loop. */
-	if (cursor->index->table->comp
+	if (page_is_comp(page)
 			? !rec_get_node_ptr_flag(rec)
 			: !rec_get_1byte_offs_flag(rec)) {
 		btr_rec_free_externally_stored_fields(cursor->index,
 					rec, offsets, in_rollback, mtr);
 	}
 
-	if ((page_get_n_recs(page) < 2)
-	    && (dict_tree_get_page(btr_cur_get_tree(cursor))
+	if (UNIV_UNLIKELY(page_get_n_recs(page) < 2)
+	    && UNIV_UNLIKELY(dict_tree_get_page(btr_cur_get_tree(cursor))
 					!= buf_frame_get_page_no(page))) {
 
 		/* If there is only one record, drop the whole page in
@@ -2545,9 +2550,13 @@ btr_cur_pessimistic_delete(
 	}
 
 	lock_update_delete(rec);
+	level = btr_page_get_level(page, mtr);
 
-	if ((btr_page_get_level(page, mtr) > 0)
-	    && (page_rec_get_next(page_get_infimum_rec(page)) == rec)) {
+	if (level > 0
+	    && UNIV_UNLIKELY(rec == page_rec_get_next(
+				page_get_infimum_rec(page)))) {
+
+		rec_t*	next_rec = page_rec_get_next(rec);
 
 		if (btr_page_get_prev(page, mtr) == FIL_NULL) {
 
@@ -2555,8 +2564,8 @@ btr_cur_pessimistic_delete(
 			non-leaf level, we must mark the new leftmost node
 			pointer as the predefined minimum record */
 
-			btr_set_min_rec_mark(page_rec_get_next(rec),
-					cursor->index->table->comp, mtr);
+			btr_set_min_rec_mark(next_rec, page_is_comp(page),
+						mtr);
 		} else {
 			/* Otherwise, if we delete the leftmost node pointer
 			on a page, we have to change the father node pointer
@@ -2566,13 +2575,12 @@ btr_cur_pessimistic_delete(
 			btr_node_ptr_delete(tree, page, mtr);
 
 			node_ptr = dict_tree_build_node_ptr(
-					tree, page_rec_get_next(rec),
+					tree, next_rec,
 					buf_frame_get_page_no(page),
-       					heap, btr_page_get_level(page, mtr));
+						heap, level);
 
 			btr_insert_on_non_leaf_level(tree,
-					btr_page_get_level(page, mtr) + 1,
-					node_ptr, mtr);
+					level + 1, node_ptr, mtr);
 		}
 	} 
 
@@ -2812,12 +2820,13 @@ btr_estimate_number_of_different_key_vals(
 	ulint		add_on;
 	mtr_t		mtr;
 	mem_heap_t*	heap		= NULL;
-	ulint		offsets1_[REC_OFFS_NORMAL_SIZE];
-	ulint		offsets2_[REC_OFFS_NORMAL_SIZE];
-	ulint*		offsets1	= offsets1_;
-	ulint*		offsets2	= offsets2_;
-	*offsets1_ = (sizeof offsets1_) / sizeof *offsets1_;
-	*offsets2_ = (sizeof offsets2_) / sizeof *offsets2_;
+	ulint		offsets_rec_[REC_OFFS_NORMAL_SIZE];
+	ulint		offsets_next_rec_[REC_OFFS_NORMAL_SIZE];
+	ulint*		offsets_rec	= offsets_rec_;
+	ulint*		offsets_next_rec= offsets_next_rec_;
+	*offsets_rec_ = (sizeof offsets_rec_) / sizeof *offsets_rec_;
+	*offsets_next_rec_ =
+			(sizeof offsets_next_rec_) / sizeof *offsets_next_rec_;
 
 	n_cols = dict_index_get_n_unique(index);
 
@@ -2830,6 +2839,7 @@ btr_estimate_number_of_different_key_vals(
 	/* We sample some pages in the index to get an estimate */
 	
 	for (i = 0; i < BTR_KEY_VAL_ESTIMATE_N_PAGES; i++) {
+		rec_t*	supremum;
 		mtr_start(&mtr);
 
 		btr_cur_open_at_rnd_pos(index, BTR_SEARCH_LEAF, &cursor, &mtr);
@@ -2842,26 +2852,29 @@ btr_estimate_number_of_different_key_vals(
 
 		page = btr_cur_get_page(&cursor);
 
-		rec = page_get_infimum_rec(page);
-		rec = page_rec_get_next(rec);
+		supremum = page_get_supremum_rec(page);
+		rec = page_rec_get_next(page_get_infimum_rec(page));
 
-		if (rec != page_get_supremum_rec(page)) {
+		if (rec != supremum) {
 			not_empty_flag = 1;
+			offsets_rec = rec_get_offsets(rec, index, offsets_rec,
+						ULINT_UNDEFINED, &heap);
 		}
-		
-		while (rec != page_get_supremum_rec(page)
-		       && page_rec_get_next(rec)
-					!= page_get_supremum_rec(page)) {
+
+		while (rec != supremum) {
 			rec_t*	next_rec = page_rec_get_next(rec);
+			if (next_rec == supremum) {
+				break;
+			}
+
 			matched_fields = 0;
 			matched_bytes = 0;
-			offsets1 = rec_get_offsets(rec, index, offsets1,
-						ULINT_UNDEFINED, &heap);
-			offsets2 = rec_get_offsets(next_rec, index, offsets2,
+			offsets_next_rec = rec_get_offsets(next_rec, index,
+						offsets_next_rec,
 						n_cols, &heap);
 
 			cmp_rec_rec_with_match(rec, next_rec,
-						offsets1, offsets2,
+						offsets_rec, offsets_next_rec,
 						index, &matched_fields,
 						&matched_bytes);
 
@@ -2874,9 +2887,17 @@ btr_estimate_number_of_different_key_vals(
 
 			total_external_size +=
 				btr_rec_get_externally_stored_len(
-								rec, offsets1);
+								rec, offsets_rec);
 			
-			rec = page_rec_get_next(rec);
+			rec = next_rec;
+			/* Initialize offsets_rec for the next round
+			and assign the old offsets_rec buffer to
+			offsets_next_rec. */
+			{
+				ulint*	offsets_tmp = offsets_rec;
+				offsets_rec = offsets_next_rec;
+				offsets_next_rec = offsets_tmp;
+			}
 		}
 		
 
@@ -2898,11 +2919,11 @@ btr_estimate_number_of_different_key_vals(
 			}
 		}
 
-		offsets1 = rec_get_offsets(rec, index, offsets1,
+		offsets_rec = rec_get_offsets(rec, index, offsets_rec,
 						ULINT_UNDEFINED, &heap);
 		total_external_size +=
 				btr_rec_get_externally_stored_len(rec,
-								offsets1);
+								offsets_rec);
 		mtr_commit(&mtr);
 	}
 
@@ -3598,7 +3619,7 @@ btr_rec_free_externally_stored_fields(
 							MTR_MEMO_PAGE_X_FIX));
 	/* Free possible externally stored fields in the record */
 
-	ut_ad(index->table->comp == rec_offs_comp(offsets));
+	ut_ad(index->table->comp == !!rec_offs_comp(offsets));
 	n_fields = rec_offs_n_fields(offsets);
 
 	for (i = 0; i < n_fields; i++) {

innobase/btr/btr0pcur.c

@@ -78,6 +78,7 @@ btr_pcur_store_position(
 	rec_t*		rec;
 	dict_tree_t*	tree;
 	page_t*		page;
+	ulint		offs;
 	
 	ut_a(cursor->pos_state == BTR_PCUR_IS_POSITIONED);
 	ut_ad(cursor->latch_mode != BTR_NO_LATCHES);
@@ -87,7 +88,8 @@ btr_pcur_store_position(
 	page_cursor = btr_pcur_get_page_cur(cursor);
 
 	rec = page_cur_get_rec(page_cursor);
-	page = buf_frame_align(rec);
+	page = ut_align_down(rec, UNIV_PAGE_SIZE);
+	offs = ut_align_offset(rec, UNIV_PAGE_SIZE);
 
 	ut_ad(mtr_memo_contains(mtr, buf_block_align(page),
 							MTR_MEMO_PAGE_S_FIX)
@@ -95,35 +97,33 @@ btr_pcur_store_position(
 							MTR_MEMO_PAGE_X_FIX));
 	ut_a(cursor->latch_mode != BTR_NO_LATCHES);
 
-	if (page_get_n_recs(page) == 0) {
+	if (UNIV_UNLIKELY(page_get_n_recs(page) == 0)) {
 		/* It must be an empty index tree; NOTE that in this case
 		we do not store the modify_clock, but always do a search
 		if we restore the cursor position */
 
-		ut_a(btr_page_get_next(page, mtr) == FIL_NULL
-		     && btr_page_get_prev(page, mtr) == FIL_NULL);
+		ut_a(btr_page_get_next(page, mtr) == FIL_NULL);
+		ut_a(btr_page_get_prev(page, mtr) == FIL_NULL);
 
-		if (rec == page_get_supremum_rec(page)) {
+		cursor->old_stored = BTR_PCUR_OLD_STORED;
 
-			cursor->rel_pos = BTR_PCUR_AFTER_LAST_IN_TREE;
-			cursor->old_stored = BTR_PCUR_OLD_STORED;
+		if (page_rec_is_supremum_low(offs)) {
 
-			return;
+			cursor->rel_pos = BTR_PCUR_AFTER_LAST_IN_TREE;
+		} else {
+			cursor->rel_pos = BTR_PCUR_BEFORE_FIRST_IN_TREE;
 		}
 
-		cursor->rel_pos = BTR_PCUR_BEFORE_FIRST_IN_TREE;
-		cursor->old_stored = BTR_PCUR_OLD_STORED;
-
 		return;
 	} 
 
-	if (rec == page_get_supremum_rec(page)) {
+	if (page_rec_is_supremum_low(offs)) {
 
 		rec = page_rec_get_prev(rec);
 
 		cursor->rel_pos = BTR_PCUR_AFTER;
 
-	} else if (rec == page_get_infimum_rec(page)) {
+	} else if (page_rec_is_infimum_low(offs)) {
 
 		rec = page_rec_get_next(rec);
 
@@ -139,7 +139,8 @@ btr_pcur_store_position(
 						&cursor->buf_size);
 
 	cursor->block_when_stored = buf_block_align(page);	
-	cursor->modify_clock = buf_frame_get_modify_clock(page);
+	cursor->modify_clock = buf_block_get_modify_clock(
+				cursor->block_when_stored);
 }
 
 /******************************************************************
@@ -202,12 +203,11 @@ btr_pcur_restore_position(
 	dtuple_t*	tuple;
 	ulint		mode;
 	ulint		old_mode;
-	ibool		from_left;
 	mem_heap_t*	heap;
 
-	ut_a(cursor->pos_state == BTR_PCUR_WAS_POSITIONED
+	ut_ad(cursor->pos_state == BTR_PCUR_WAS_POSITIONED
 			|| cursor->pos_state == BTR_PCUR_IS_POSITIONED);
-	if (cursor->old_stored != BTR_PCUR_OLD_STORED) {
+	if (UNIV_UNLIKELY(cursor->old_stored != BTR_PCUR_OLD_STORED)) {
 		ut_print_buf(stderr, (const byte*)cursor, sizeof(btr_pcur_t));
 		if (cursor->trx_if_known) {
 			trx_print(stderr, cursor->trx_if_known);
@@ -216,19 +216,14 @@ btr_pcur_restore_position(
 		ut_a(0);
 	}
 
-	if (cursor->rel_pos == BTR_PCUR_AFTER_LAST_IN_TREE
-	    || cursor->rel_pos == BTR_PCUR_BEFORE_FIRST_IN_TREE) {
+	if (UNIV_UNLIKELY(cursor->rel_pos == BTR_PCUR_AFTER_LAST_IN_TREE
+			|| cursor->rel_pos == BTR_PCUR_BEFORE_FIRST_IN_TREE)) {
 
 	    	/* In these cases we do not try an optimistic restoration,
 	    	but always do a search */
 
-	    	if (cursor->rel_pos == BTR_PCUR_BEFORE_FIRST_IN_TREE) {
-	    		from_left = TRUE;
-	    	} else {
-	    		from_left = FALSE;
-	    	}
-
-		btr_cur_open_at_index_side(from_left,
+		btr_cur_open_at_index_side(
+			cursor->rel_pos == BTR_PCUR_BEFORE_FIRST_IN_TREE,
 			btr_pcur_get_btr_cur(cursor)->index, latch_mode,
 					btr_pcur_get_btr_cur(cursor), mtr);
 
@@ -238,17 +233,18 @@ btr_pcur_restore_position(
 		return(FALSE);
 	}
 	
-	ut_a(cursor->old_rec);
-	ut_a(cursor->old_n_fields);
+	ut_ad(cursor->old_rec);
+	ut_ad(cursor->old_n_fields);
 
 	page = btr_cur_get_page(btr_pcur_get_btr_cur(cursor));
 
-	if (latch_mode == BTR_SEARCH_LEAF || latch_mode == BTR_MODIFY_LEAF) {
+	if (UNIV_LIKELY(latch_mode == BTR_SEARCH_LEAF)
+			|| UNIV_LIKELY(latch_mode == BTR_MODIFY_LEAF)) {
 		/* Try optimistic restoration */
 	    
-		if (buf_page_optimistic_get(latch_mode,
+		if (UNIV_LIKELY(buf_page_optimistic_get(latch_mode,
 					    cursor->block_when_stored, page,
-					    cursor->modify_clock, mtr)) {
+					    cursor->modify_clock, mtr))) {
 			cursor->pos_state = BTR_PCUR_IS_POSITIONED;
 #ifdef UNIV_SYNC_DEBUG
 			buf_page_dbg_add_level(page, SYNC_TREE_NODE);
@@ -297,7 +293,7 @@ btr_pcur_restore_position(
 	/* Save the old search mode of the cursor */
 	old_mode = cursor->search_mode;
 	
-	if (cursor->rel_pos == BTR_PCUR_ON) {
+	if (UNIV_LIKELY(cursor->rel_pos == BTR_PCUR_ON)) {
 		mode = PAGE_CUR_LE;
 	} else if (cursor->rel_pos == BTR_PCUR_AFTER) {
 		mode = PAGE_CUR_G;
@@ -323,12 +319,10 @@ btr_pcur_restore_position(
 		the cursor can now be on a different page! But we can retain
 		the value of old_rec */
 
-		cursor->modify_clock =
-			buf_frame_get_modify_clock(btr_pcur_get_page(cursor));
-
 		cursor->block_when_stored =
 			buf_block_align(btr_pcur_get_page(cursor));
-
+		cursor->modify_clock =
+			buf_block_get_modify_clock(cursor->block_when_stored);
 		cursor->old_stored = BTR_PCUR_OLD_STORED;
 
 		mem_heap_free(heap);

innobase/btr/btr0sea.c

@@ -544,7 +544,6 @@ btr_search_check_guess(
 				or PAGE_CUR_GE */
 	mtr_t*		mtr)	/* in: mtr */
 {
-	page_t*		page;
 	rec_t*		rec;
 	rec_t*		prev_rec;
 	rec_t*		next_rec;
@@ -561,7 +560,6 @@ btr_search_check_guess(
 	n_unique = dict_index_get_n_unique_in_tree(cursor->index);
 	
 	rec = btr_cur_get_rec(cursor);
-	page = buf_frame_align(rec);
 
 	ut_ad(page_rec_is_user_rec(rec));
 
@@ -612,12 +610,13 @@ btr_search_check_guess(
 
 	if ((mode == PAGE_CUR_G) || (mode == PAGE_CUR_GE)) {
 
-		ut_ad(rec != page_get_infimum_rec(page));
+		ut_ad(!page_rec_is_infimum(rec));
 		
 		prev_rec = page_rec_get_prev(rec);
 
-		if (prev_rec == page_get_infimum_rec(page)) {
-			success = btr_page_get_prev(page, mtr) == FIL_NULL;
+		if (page_rec_is_infimum(prev_rec)) {
+			success = btr_page_get_prev(
+				buf_frame_align(prev_rec), mtr) == FIL_NULL;
 			goto exit_func;
 		}
 
@@ -634,12 +633,13 @@ btr_search_check_guess(
 		goto exit_func;
 	}
 		
-	ut_ad(rec != page_get_supremum_rec(page));
+	ut_ad(!page_rec_is_supremum(rec));
 	
 	next_rec = page_rec_get_next(rec);
 
-	if (next_rec == page_get_supremum_rec(page)) {
-    		if (btr_page_get_next(page, mtr) == FIL_NULL) {
+	if (page_rec_is_supremum(next_rec)) {
+		if (btr_page_get_next(buf_frame_align(next_rec), mtr)
+							== FIL_NULL) {
 
 			cursor->up_match = 0;
 			success = TRUE;
@@ -694,7 +694,6 @@ btr_search_guess_on_hash(
 	buf_block_t*	block;
 	rec_t*		rec;
 	page_t*		page;
-	ibool		success;
 	ulint		fold;
 	ulint		tuple_n_fields;
 	dulint		tree_id;
@@ -710,7 +709,7 @@ btr_search_guess_on_hash(
 	/* Note that, for efficiency, the struct info may not be protected by
 	any latch here! */
 
-	if (info->n_hash_potential == 0) {
+	if (UNIV_UNLIKELY(info->n_hash_potential == 0)) {
 
 		return(FALSE);
 	}
@@ -720,12 +719,13 @@ btr_search_guess_on_hash(
 
 	tuple_n_fields = dtuple_get_n_fields(tuple);
 
-	if (tuple_n_fields < cursor->n_fields) {
+	if (UNIV_UNLIKELY(tuple_n_fields < cursor->n_fields)) {
 
 		return(FALSE);
 	}
 
-	if ((cursor->n_bytes > 0) && (tuple_n_fields <= cursor->n_fields)) {
+	if (UNIV_UNLIKELY(tuple_n_fields == cursor->n_fields)
+			&& (cursor->n_bytes > 0)) {
 
 	    	return(FALSE);
 	}
@@ -740,39 +740,31 @@ btr_search_guess_on_hash(
 	cursor->fold = fold;
 	cursor->flag = BTR_CUR_HASH;
 	
-	if (!has_search_latch) {
+	if (UNIV_LIKELY(!has_search_latch)) {
 		rw_lock_s_lock(&btr_search_latch);
 	}
 
-	ut_a(btr_search_latch.writer != RW_LOCK_EX);
-	ut_a(btr_search_latch.reader_count > 0);
+	ut_ad(btr_search_latch.writer != RW_LOCK_EX);
+	ut_ad(btr_search_latch.reader_count > 0);
 
 	rec = ha_search_and_get_data(btr_search_sys->hash_index, fold);
 
-	if (!rec) {
-		if (!has_search_latch) {
-			rw_lock_s_unlock(&btr_search_latch);
-		}
-		
-		goto failure;
+	if (UNIV_UNLIKELY(!rec)) {
+		goto failure_unlock;
 	}
 
 	page = buf_frame_align(rec);
 
-	if (!has_search_latch) {
+	if (UNIV_LIKELY(!has_search_latch)) {
 
-		success = buf_page_get_known_nowait(latch_mode, page,
+		if (UNIV_UNLIKELY(!buf_page_get_known_nowait(latch_mode, page,
 						BUF_MAKE_YOUNG,
 						__FILE__, __LINE__,
-						mtr);
-
-		rw_lock_s_unlock(&btr_search_latch);
-
-		if (!success) {
-
-			goto failure;
+						mtr))) {
+			goto failure_unlock;
 		}
 
+		rw_lock_s_unlock(&btr_search_latch);
 		can_only_compare_to_cursor_rec = FALSE;
 
 #ifdef UNIV_SYNC_DEBUG
@@ -782,8 +774,8 @@ btr_search_guess_on_hash(
 
 	block = buf_block_align(page);
 
-	if (block->state == BUF_BLOCK_REMOVE_HASH) {
-		if (!has_search_latch) {
+	if (UNIV_UNLIKELY(block->state == BUF_BLOCK_REMOVE_HASH)) {
+		if (UNIV_LIKELY(!has_search_latch)) {
 	
 			btr_leaf_page_release(page, latch_mode, mtr);
 		}
@@ -791,51 +783,33 @@ btr_search_guess_on_hash(
 		goto failure;
 	}
 
-	ut_a(block->state == BUF_BLOCK_FILE_PAGE);
-	ut_a(page_rec_is_user_rec(rec));	
+	ut_ad(block->state == BUF_BLOCK_FILE_PAGE);
+	ut_ad(page_rec_is_user_rec(rec));
 
 	btr_cur_position(index, rec, cursor);
 
 	/* Check the validity of the guess within the page */
 
-	if (0 != ut_dulint_cmp(tree_id, btr_page_get_index_id(page))) {
-
-		success = FALSE;
-/*
-		fprintf(stderr, "Tree id %lu, page index id %lu fold %lu\n",
-				ut_dulint_get_low(tree_id),
-				ut_dulint_get_low(btr_page_get_index_id(page)),
-				fold);
-*/				
-	} else {
-	        /* If we only have the latch on btr_search_latch, not on the
-		page, it only protects the columns of the record the cursor
-		is positioned on. We cannot look at the next of the previous
-		record to determine if our guess for the cursor position is
-		right. */
-
-		success = btr_search_check_guess(cursor,
-				               can_only_compare_to_cursor_rec,
-					       tuple, mode, mtr);
-	}
-	
-	if (!success) {
-		if (!has_search_latch) {
+	/* If we only have the latch on btr_search_latch, not on the
+	page, it only protects the columns of the record the cursor
+	is positioned on. We cannot look at the next of the previous
+	record to determine if our guess for the cursor position is
+	right. */
+	if (UNIV_EXPECT(ut_dulint_cmp(tree_id, btr_page_get_index_id(page)), 0)
+	    || !btr_search_check_guess(cursor, can_only_compare_to_cursor_rec,
+					       tuple, mode, mtr)) {
+		if (UNIV_LIKELY(!has_search_latch)) {
 		          btr_leaf_page_release(page, latch_mode, mtr);
 		}
 
 		goto failure;
 	}
 
-	if (info->n_hash_potential < BTR_SEARCH_BUILD_LIMIT + 5) {
+	if (UNIV_LIKELY(info->n_hash_potential < BTR_SEARCH_BUILD_LIMIT + 5)) {
 	
 		info->n_hash_potential++;
 	}
 
-	if (info->last_hash_succ != TRUE) {
-		info->last_hash_succ = TRUE;
-	}
-	
 #ifdef notdefined
 	/* These lines of code can be used in a debug version to check
 	the correctness of the searched cursor position: */
@@ -843,15 +817,14 @@ btr_search_guess_on_hash(
 	info->last_hash_succ = FALSE;
 
 	/* Currently, does not work if the following fails: */
-	ut_a(!has_search_latch);
+	ut_ad(!has_search_latch);
 	
 	btr_leaf_page_release(page, latch_mode, mtr);
 
 	btr_cur_search_to_nth_level(index, 0, tuple, mode, latch_mode,
 							&cursor2, 0, mtr);
 	if (mode == PAGE_CUR_GE
-		&& btr_cur_get_rec(&cursor2) == page_get_supremum_rec(
-			buf_frame_align(btr_cur_get_rec(&cursor2)))) {
+		&& page_rec_is_supremum(btr_cur_get_rec(&cursor2))) {
 
 		/* If mode is PAGE_CUR_GE, then the binary search
 		in the index tree may actually take us to the supremum
@@ -861,22 +834,22 @@ btr_search_guess_on_hash(
 
 		btr_pcur_open_on_user_rec(index, tuple, mode, latch_mode,
 				&pcur, mtr);
-		ut_a(btr_pcur_get_rec(&pcur) == btr_cur_get_rec(cursor));
+		ut_ad(btr_pcur_get_rec(&pcur) == btr_cur_get_rec(cursor));
 	} else {
-		ut_a(btr_cur_get_rec(&cursor2) == btr_cur_get_rec(cursor));
+		ut_ad(btr_cur_get_rec(&cursor2) == btr_cur_get_rec(cursor));
 	}
 
 	/* NOTE that it is theoretically possible that the above assertions
 	fail if the page of the cursor gets removed from the buffer pool
 	meanwhile! Thus it might not be a bug. */
-
-	info->last_hash_succ = TRUE;
 #endif
+	info->last_hash_succ = TRUE;
 
 #ifdef UNIV_SEARCH_PERF_STAT
 	btr_search_n_succ++;
 #endif
-	if (!has_search_latch && buf_block_peek_if_too_old(block)) {
+	if (UNIV_LIKELY(!has_search_latch)
+			&& buf_block_peek_if_too_old(block)) {
 
 		buf_page_make_young(page);
 	}	
@@ -889,6 +862,10 @@ btr_search_guess_on_hash(
 	return(TRUE);	
 
 	/*-------------------------------------------*/
+failure_unlock:
+	if (UNIV_LIKELY(!has_search_latch)) {
+		rw_lock_s_unlock(&btr_search_latch);
+	}
 failure:
 	info->n_hash_fail++;
 
@@ -917,7 +894,6 @@ btr_search_drop_page_hash_index(
 	ulint		n_fields;
 	ulint		n_bytes;
 	rec_t*		rec;
-	rec_t*		sup;
 	ulint		fold;
 	ulint		prev_fold;
 	dulint		tree_id;
@@ -968,12 +944,10 @@ btr_search_drop_page_hash_index(
 
 	n_cached = 0;
 
-	sup = page_get_supremum_rec(page);
-
 	rec = page_get_infimum_rec(page);
 	rec = page_rec_get_next(rec);
 
-	if (rec != sup) {
+	if (!page_rec_is_supremum(rec)) {
 		ut_a(n_fields <= rec_get_n_fields(rec, block->index));
 
 		if (n_bytes > 0) {
@@ -988,7 +962,7 @@ btr_search_drop_page_hash_index(
 	heap = NULL;
 	offsets = NULL;
 
-	while (rec != sup) {
+	while (!page_rec_is_supremum(rec)) {
 		/* FIXME: in a mixed tree, not all records may have enough
 		ordering fields: */
 		offsets = rec_get_offsets(rec, block->index,
@@ -1090,7 +1064,6 @@ btr_search_build_page_hash_index(
 	buf_block_t*	block;
 	rec_t*		rec;
 	rec_t*		next_rec;
-	rec_t*		sup;
 	ulint		fold;
 	ulint		next_fold;
 	dulint		tree_id;
@@ -1158,15 +1131,13 @@ btr_search_build_page_hash_index(
 
 	tree_id = btr_page_get_index_id(page);
 
-	sup = page_get_supremum_rec(page);
-
 	rec = page_get_infimum_rec(page);
 	rec = page_rec_get_next(rec);
 
 	offsets = rec_get_offsets(rec, index, offsets,
 					n_fields + (n_bytes > 0), &heap);
 
-	if (rec != sup) {
+	if (!page_rec_is_supremum(rec)) {
 		ut_a(n_fields <= rec_offs_n_fields(offsets));
 
 		if (n_bytes > 0) {
@@ -1188,7 +1159,7 @@ btr_search_build_page_hash_index(
 	for (;;) {
 		next_rec = page_rec_get_next(rec);
 
-		if (next_rec == sup) {
+		if (page_rec_is_supremum(next_rec)) {
 
 			if (side == BTR_SEARCH_RIGHT_SIDE) {
 	
@@ -1443,7 +1414,6 @@ btr_search_update_hash_on_insert(
 {
 	hash_table_t*	table; 
 	buf_block_t*	block;
-	page_t*		page;
 	rec_t*		rec;
 	rec_t*		ins_rec;
 	rec_t*		next_rec;
@@ -1488,19 +1458,18 @@ btr_search_update_hash_on_insert(
 	ins_rec = page_rec_get_next(rec);
 	next_rec = page_rec_get_next(ins_rec);
 
-	page = buf_frame_align(rec);
 	offsets = rec_get_offsets(ins_rec, cursor->index, offsets,
 					ULINT_UNDEFINED, &heap);
 	ins_fold = rec_fold(ins_rec, offsets, n_fields, n_bytes, tree_id);
 
-	if (next_rec != page_get_supremum_rec(page)) {
+	if (!page_rec_is_supremum(next_rec)) {
 		offsets = rec_get_offsets(next_rec, cursor->index, offsets,
 					n_fields + (n_bytes > 0), &heap);
 		next_fold = rec_fold(next_rec, offsets, n_fields,
 							n_bytes, tree_id);
 	}
 
-	if (rec != page_get_infimum_rec(page)) {
+	if (!page_rec_is_infimum(rec)) {
 		offsets = rec_get_offsets(rec, cursor->index, offsets,
 					n_fields + (n_bytes > 0), &heap);
 		fold = rec_fold(rec, offsets, n_fields, n_bytes, tree_id);
@@ -1534,7 +1503,7 @@ btr_search_update_hash_on_insert(
 	}
 
 check_next_rec:
-	if (next_rec == page_get_supremum_rec(page)) {
+	if (page_rec_is_supremum(next_rec)) {
 
 		if (side == BTR_SEARCH_RIGHT_SIDE) {
 

innobase/dict/dict0crea.c

@@ -736,7 +736,7 @@ dict_truncate_index_tree(
 	dulint		index_id;
 	byte*		ptr;
 	ulint		len;
-	ibool		comp;
+	ulint		comp;
 	dict_index_t*	index;
 
 #ifdef UNIV_SYNC_DEBUG

innobase/ibuf/ibuf0ibuf.c

@@ -1889,7 +1889,7 @@ ibuf_get_merge_page_nos(
 				contract the tree, FALSE if this is called
 				when a single page becomes full and we look
 				if it pays to read also nearby pages */
-	rec_t*		first_rec,/* in: record from which we read up and down
+	rec_t*		rec,	/* in: record from which we read up and down
 				in the chain of records */
 	ulint*		space_ids,/* in/out: space id's of the pages */
 	ib_longlong*	space_versions,/* in/out: tablespace version
@@ -1907,47 +1907,42 @@ ibuf_get_merge_page_nos(
 	ulint	first_space_id;
 	ulint	rec_page_no;
 	ulint	rec_space_id;
-	rec_t*	rec;
 	ulint	sum_volumes;
 	ulint	volume_for_page;
 	ulint	rec_volume;
 	ulint	limit;
-	page_t*	page;
 	ulint	n_pages;
 
 	*n_stored = 0;
 
 	limit = ut_min(IBUF_MAX_N_PAGES_MERGED, buf_pool->curr_size / 4);
 
-	page = buf_frame_align(first_rec);
-	
-	if (first_rec == page_get_supremum_rec(page)) {
+	if (page_rec_is_supremum(rec)) {
 
-		first_rec = page_rec_get_prev(first_rec);
+		rec = page_rec_get_prev(rec);
 	}
 
-	if (first_rec == page_get_infimum_rec(page)) {
+	if (page_rec_is_infimum(rec)) {
 
-		first_rec = page_rec_get_next(first_rec);
+		rec = page_rec_get_next(rec);
 	}
 
-	if (first_rec == page_get_supremum_rec(page)) {
+	if (page_rec_is_supremum(rec)) {
 
 		return(0);
 	}
 
-	rec = first_rec;
-	first_page_no = ibuf_rec_get_page_no(first_rec);
-	first_space_id = ibuf_rec_get_space(first_rec);
+	first_page_no = ibuf_rec_get_page_no(rec);
+	first_space_id = ibuf_rec_get_space(rec);
 	n_pages = 0;
 	prev_page_no = 0;
 	prev_space_id = 0;
 	
-	/* Go backwards from the first_rec until we reach the border of the
+	/* Go backwards from the first rec until we reach the border of the
 	'merge area', or the page start or the limit of storeable pages is
 	reached */
 
-	while ((rec != page_get_infimum_rec(page)) && (n_pages < limit)) {
+	while (!page_rec_is_infimum(rec) && UNIV_LIKELY(n_pages < limit)) {
 
 		rec_page_no = ibuf_rec_get_page_no(rec);
 		rec_space_id = ibuf_rec_get_space(rec);
@@ -1982,7 +1977,7 @@ ibuf_get_merge_page_nos(
 	volume_for_page = 0;
 	
 	while (*n_stored < limit) {
-		if (rec == page_get_supremum_rec(page)) {
+		if (page_rec_is_supremum(rec)) {
 			/* When no more records available, mark this with
 			another 'impossible' pair of space id, page no */
 			rec_page_no = 1;
@@ -2311,12 +2306,12 @@ ibuf_get_volume_buffered(
 
 	page = buf_frame_align(rec);
 
-	if (rec == page_get_supremum_rec(page)) {
+	if (page_rec_is_supremum(rec)) {
 		rec = page_rec_get_prev(rec);
 	}
 
 	for (;;) {
-		if (rec == page_get_infimum_rec(page)) {
+		if (page_rec_is_infimum(rec)) {
 
 			break;
 		}
@@ -2351,7 +2346,7 @@ ibuf_get_volume_buffered(
 	rec = page_rec_get_prev(rec);
 	
 	for (;;) {
-		if (rec == page_get_infimum_rec(prev_page)) {
+		if (page_rec_is_infimum(rec)) {
 
 			/* We cannot go to yet a previous page, because we
 			do not have the x-latch on it, and cannot acquire one
@@ -2374,12 +2369,12 @@ ibuf_get_volume_buffered(
 count_later:
 	rec = btr_pcur_get_rec(pcur);
 
-	if (rec != page_get_supremum_rec(page)) {
+	if (!page_rec_is_supremum(rec)) {
 		rec = page_rec_get_next(rec);
 	}
 
 	for (;;) {
-		if (rec == page_get_supremum_rec(page)) {
+		if (page_rec_is_supremum(rec)) {
 
 			break;
 		}
@@ -2414,7 +2409,7 @@ count_later:
 	rec = page_rec_get_next(rec);
 
 	for (;;) {
-		if (rec == page_get_supremum_rec(next_page)) {
+		if (page_rec_is_supremum(rec)) {
 
 			/* We give up */
 		
@@ -2815,7 +2810,7 @@ ibuf_insert_to_index_page(
 	ut_ad(ibuf_inside());
 	ut_ad(dtuple_check_typed(entry));
 
-	if (index->table->comp != page_is_comp(page)) {
+	if (UNIV_UNLIKELY(index->table->comp != !!page_is_comp(page))) {
 		fputs(
 "InnoDB: Trying to insert a record from the insert buffer to an index page\n"
 "InnoDB: but the 'compact' flag does not match!\n", stderr);
@@ -2824,7 +2819,8 @@ ibuf_insert_to_index_page(
 
 	rec = page_rec_get_next(page_get_infimum_rec(page));
 
-	if (rec_get_n_fields(rec, index) != dtuple_get_n_fields(entry)) {
+	if (UNIV_UNLIKELY(rec_get_n_fields(rec, index)
+			!= dtuple_get_n_fields(entry))) {
 		fputs(
 "InnoDB: Trying to insert a record from the insert buffer to an index page\n"
 "InnoDB: but the number of fields does not match!\n", stderr);
@@ -2861,8 +2857,8 @@ ibuf_insert_to_index_page(
 						PAGE_CUR_LE, &page_cur);
 
 			/* This time the record must fit */
-			if (!page_cur_tuple_insert(&page_cur, entry,
-								index, mtr)) {
+			if (UNIV_UNLIKELY(!page_cur_tuple_insert(
+					&page_cur, entry, index, mtr))) {
 
 				ut_print_timestamp(stderr);
 

innobase/include/btr0btr.h

@@ -168,7 +168,7 @@ btr_create(
 	ulint	type,	/* in: type of the index */
 	ulint	space,	/* in: space where created */
 	dulint	index_id,/* in: index id */
-	ibool	comp,	/* in: TRUE=compact page format */
+	ulint	comp,	/* in: nonzero=compact page format */
 	mtr_t*	mtr);	/* in: mini-transaction handle */
 /****************************************************************
 Frees a B-tree except the root page, which MUST be freed after this
@@ -276,7 +276,7 @@ void
 btr_set_min_rec_mark(
 /*=================*/
 	rec_t*	rec,	/* in: record */
-	ibool	comp,	/* in: TRUE=compact page format */
+	ulint	comp,	/* in: nonzero=compact page format */
 	mtr_t*	mtr);	/* in: mtr */
 /*****************************************************************
 Deletes on the upper level the node pointer to a page. */
@@ -336,7 +336,7 @@ btr_parse_set_min_rec_mark(
 			/* out: end of log record or NULL */
 	byte*	ptr,	/* in: buffer */
 	byte*	end_ptr,/* in: buffer end */
-	ibool	comp,	/* in: TRUE=compact page format */
+	ulint	comp,	/* in: nonzero=compact page format */
 	page_t*	page,	/* in: page or NULL */
 	mtr_t*	mtr);	/* in: mtr or NULL */
 /***************************************************************

innobase/include/buf0buf.h

@@ -382,10 +382,10 @@ Returns the value of the modify clock. The caller must have an s-lock
 or x-lock on the block. */
 UNIV_INLINE
 dulint
-buf_frame_get_modify_clock(
+buf_block_get_modify_clock(
 /*=======================*/
 				/* out: value */
-	buf_frame_t*	frame);	/* in: pointer to a frame */
+	buf_block_t*	block);	/* in: block */
 /************************************************************************
 Calculates a page checksum which is stored to the page when it is written
 to a file. Note that we must be careful to calculate the same value

innobase/include/buf0buf.ic

@@ -481,17 +481,11 @@ Returns the value of the modify clock. The caller must have an s-lock
 or x-lock on the block. */
 UNIV_INLINE
 dulint
-buf_frame_get_modify_clock(
+buf_block_get_modify_clock(
 /*=======================*/
 				/* out: value */
-	buf_frame_t*	frame)	/* in: pointer to a frame */
+	buf_block_t*	block)	/* in: block */
 {
-	buf_block_t*	block;
-
-	ut_ad(frame);
-
-	block = buf_block_align(frame);
-
 #ifdef UNIV_SYNC_DEBUG
 	ut_ad(rw_lock_own(&(block->lock), RW_LOCK_SHARED)
 	      || rw_lock_own(&(block->lock), RW_LOCK_EXCLUSIVE));

innobase/include/lock0lock.h

@@ -216,6 +216,7 @@ actual record is being moved. */
 void
 lock_rec_store_on_page_infimum(
 /*===========================*/
+	page_t*	page,	/* in: page containing the record */
 	rec_t*	rec);	/* in: record whose lock state is stored
 			on the infimum record of the same page; lock
 			bits are reset on the record */

innobase/include/mach0data.h

@@ -52,6 +52,27 @@ mach_read_from_2(
 /*=============*/
 			/* out: ulint integer, >= 0, < 64k */
 	byte*   b);      /* in: pointer to two bytes */
+
+/************************************************************
+The following function is used to convert a 16-bit data item
+to the canonical format, for fast bytewise equality test
+against memory. */
+UNIV_INLINE
+uint16
+mach_encode_2(
+/*==========*/
+			/* out: 16-bit integer in canonical format */
+	ulint	n);	/* in: integer in machine-dependent format */
+/************************************************************
+The following function is used to convert a 16-bit data item
+from the canonical format, for fast bytewise equality test
+against memory. */
+UNIV_INLINE
+ulint
+mach_decode_2(
+/*==========*/
+			/* out: integer in machine-dependent format */
+	uint16	n);	/* in: 16-bit integer in canonical format */
 /***********************************************************
 The following function is used to store data in 3 consecutive
 bytes. We store the most significant byte to the lowest address. */

innobase/include/mach0data.ic

@@ -68,6 +68,37 @@ mach_read_from_2(
 	      );
 }
 
+/************************************************************
+The following function is used to convert a 16-bit data item
+to the canonical format, for fast bytewise equality test
+against memory. */
+UNIV_INLINE
+uint16
+mach_encode_2(
+/*==========*/
+			/* out: 16-bit integer in canonical format */
+	ulint	n)	/* in: integer in machine-dependent format */
+{
+	uint16	ret;
+	ut_ad(2 == sizeof ret);
+	mach_write_to_2((byte*) &ret, n);
+	return(ret);
+}
+/************************************************************
+The following function is used to convert a 16-bit data item
+from the canonical format, for fast bytewise equality test
+against memory. */
+UNIV_INLINE
+ulint
+mach_decode_2(
+/*==========*/
+			/* out: integer in machine-dependent format */
+	uint16	n)	/* in: 16-bit integer in canonical format */
+{
+	ut_ad(2 == sizeof n);
+	return(mach_read_from_2((byte*) &n));
+}
+
 /***********************************************************
 The following function is used to store data in 3 consecutive
 bytes. We store the most significant byte to the lowest address. */

innobase/include/page0cur.h

@@ -78,16 +78,16 @@ UNIV_INLINE
 ibool
 page_cur_is_before_first(
 /*=====================*/
-				/* out: TRUE if at start */
-	page_cur_t*	cur);	/* in: cursor */
+					/* out: TRUE if at start */
+	const page_cur_t*	cur);	/* in: cursor */
 /*************************************************************
 Returns TRUE if the cursor is after last user record. */
 UNIV_INLINE
 ibool
 page_cur_is_after_last(
 /*===================*/
-				/* out: TRUE if at end */
-	page_cur_t*	cur);	/* in: cursor */
+					/* out: TRUE if at end */
+	const page_cur_t*	cur);	/* in: cursor */
 /**************************************************************
 Positions the cursor on the given record. */
 UNIV_INLINE

innobase/include/page0cur.ic

@@ -69,15 +69,10 @@ UNIV_INLINE
 ibool
 page_cur_is_before_first(
 /*=====================*/
-				/* out: TRUE if at start */
-	page_cur_t*	cur)	/* in: cursor */
+					/* out: TRUE if at start */
+	const page_cur_t*	cur)	/* in: cursor */
 {
-	if (page_get_infimum_rec(page_cur_get_page(cur)) == cur->rec) {
-
-		return(TRUE);
-	}
-
-	return(FALSE);
+	return(page_rec_is_infimum(cur->rec));
 }
 
 /*************************************************************
@@ -86,15 +81,10 @@ UNIV_INLINE
 ibool
 page_cur_is_after_last(
 /*===================*/
-				/* out: TRUE if at end */
-	page_cur_t*	cur)	/* in: cursor */
+					/* out: TRUE if at end */
+	const page_cur_t*	cur)	/* in: cursor */
 {
-	if (page_get_supremum_rec(page_cur_get_page(cur)) == cur->rec) {
-
-		return(TRUE);
-	}
-
-	return(FALSE);
+	return(page_rec_is_supremum(cur->rec));
 }
 
 /**************************************************************

innobase/include/page0page.h

@@ -373,13 +373,21 @@ page_dir_find_owner_slot(
 /****************************************************************
 Determine whether the page is in new-style compact format. */
 UNIV_INLINE
-ibool
+ulint
 page_is_comp(
 /*=========*/
-			/* out: TRUE if the page is in compact format
-			FALSE if it is in old-style format */
+			/* out: nonzero if the page is in compact
+			format, zero if it is in old-style format */
 	page_t*	page);	/* in: index page */
 /****************************************************************
+TRUE if the record is on a page in compact format. */
+UNIV_INLINE
+ulint
+page_rec_is_comp(
+/*=============*/
+				/* out: nonzero if in compact format */
+	const rec_t*	rec);	/* in: record */
+/****************************************************************
 Gets the pointer to the next record on the page. */
 UNIV_INLINE
 rec_t*
@@ -407,47 +415,55 @@ page_rec_get_prev(
 				/* out: pointer to previous record */
 	rec_t*		rec);	/* in: pointer to record,
 				must not be page infimum */
-
 /****************************************************************
 TRUE if the record is a user record on the page. */
 UNIV_INLINE
 ibool
-page_rec_is_user_rec(
-/*=================*/
+page_rec_is_user_rec_low(
+/*=====================*/
 			/* out: TRUE if a user record */
-	rec_t*	rec);	/* in: record */
+	ulint	offset);/* in: record offset on page */
 /****************************************************************
 TRUE if the record is the supremum record on a page. */
 UNIV_INLINE
 ibool
-page_rec_is_supremum(
-/*=================*/
+page_rec_is_supremum_low(
+/*=====================*/
 			/* out: TRUE if the supremum record */
-	rec_t*	rec);	/* in: record */
+	ulint	offset);/* in: record offset on page */
 /****************************************************************
 TRUE if the record is the infimum record on a page. */
 UNIV_INLINE
 ibool
-page_rec_is_infimum(
-/*================*/
+page_rec_is_infimum_low(
+/*=====================*/
 			/* out: TRUE if the infimum record */
-	rec_t*	rec);	/* in: record */
+	ulint	offset);/* in: record offset on page */
+
 /****************************************************************
-TRUE if the record is the first user record on the page. */
+TRUE if the record is a user record on the page. */
 UNIV_INLINE
 ibool
-page_rec_is_first_user_rec(
-/*=======================*/
-			/* out: TRUE if first user record */
-	rec_t*	rec);	/* in: record */
+page_rec_is_user_rec(
+/*=================*/
+				/* out: TRUE if a user record */
+	const rec_t*	rec);	/* in: record */
 /****************************************************************
-TRUE if the record is the last user record on the page. */
+TRUE if the record is the supremum record on a page. */
 UNIV_INLINE
 ibool
-page_rec_is_last_user_rec(
-/*======================*/
-			/* out: TRUE if last user record */
-	rec_t*	rec);	/* in: record */
+page_rec_is_supremum(
+/*=================*/
+				/* out: TRUE if the supremum record */
+	const rec_t*	rec);	/* in: record */
+/****************************************************************
+TRUE if the record is the infimum record on a page. */
+UNIV_INLINE
+ibool
+page_rec_is_infimum(
+/*================*/
+				/* out: TRUE if the infimum record */
+	const rec_t*	rec);	/* in: record */
 /*******************************************************************
 Looks for the record which owns the given record. */
 UNIV_INLINE
@@ -495,7 +511,7 @@ ulint
 page_get_free_space_of_empty(
 /*=========================*/
 			/* out: free space */
-	ibool	comp)	/* in: TRUE=compact page format */
+	ulint	comp)	/* in: nonzero=compact page format */
 		__attribute__((const));
 /****************************************************************
 Returns the sum of the sizes of the records in the record list
@@ -539,7 +555,7 @@ page_create(
 	buf_frame_t*	frame,		/* in: a buffer frame where the page is
 					created */
 	mtr_t*		mtr,		/* in: mini-transaction handle */
-	ibool		comp);		/* in: TRUE=compact page format */
+	ulint		comp);		/* in: nonzero=compact page format */
 /*****************************************************************
 Differs from page_copy_rec_list_end, because this function does not
 touch the lock table and max trx id on page. */
@@ -673,7 +689,7 @@ page_parse_create(
 			/* out: end of log record or NULL */
 	byte*	ptr,	/* in: buffer */
 	byte*	end_ptr,/* in: buffer end */
-	ibool	comp,	/* in: TRUE=compact page format */
+	ulint	comp,	/* in: nonzero=compact page format */
 	page_t*	page,	/* in: page or NULL */
 	mtr_t*	mtr);	/* in: mtr or NULL */
 /****************************************************************

innobase/include/page0page.ic

@@ -155,14 +155,27 @@ page_header_reset_last_insert(
 /****************************************************************
 Determine whether the page is in new-style compact format. */
 UNIV_INLINE
-ibool
+ulint
 page_is_comp(
 /*=========*/
-			/* out: TRUE if the page is in compact format
-			FALSE if it is in old-style format */
-	page_t* page)	/* in: index page */
+			/* out: nonzero if the page is in compact
+			format, zero if it is in old-style format */
+	page_t*	page)	/* in: index page */
+{
+	return(UNIV_EXPECT(page_header_get_field(page, PAGE_N_HEAP) & 0x8000,
+		0x8000));
+}
+
+/****************************************************************
+TRUE if the record is on a page in compact format. */
+UNIV_INLINE
+ulint
+page_rec_is_comp(
+/*=============*/
+				/* out: nonzero if in compact format */
+	const rec_t*	rec)	/* in: record */
 {
-	return(!!(page_header_get_field(page, PAGE_N_HEAP) & 0x8000));
+	return(page_is_comp(ut_align_down((rec_t*) rec, UNIV_PAGE_SIZE)));
 }
 
 /****************************************************************
@@ -205,112 +218,107 @@ page_get_supremum_rec(
 TRUE if the record is a user record on the page. */
 UNIV_INLINE
 ibool
-page_rec_is_user_rec(
-/*=================*/
+page_rec_is_user_rec_low(
+/*=====================*/
 			/* out: TRUE if a user record */
-	rec_t*	rec)	/* in: record */
+	ulint	offset)	/* in: record offset on page */
 {
-	ut_ad(rec);
-
-	if (rec == page_get_supremum_rec(buf_frame_align(rec))) {
-
-		return(FALSE);
-	}
-
-	if (rec == page_get_infimum_rec(buf_frame_align(rec))) {
-
-	     	return(FALSE);
-	}
+	ut_ad(offset >= PAGE_NEW_INFIMUM);
+#if PAGE_OLD_INFIMUM < PAGE_NEW_INFIMUM
+# error "PAGE_OLD_INFIMUM < PAGE_NEW_INFIMUM"
+#endif
+#if PAGE_OLD_SUPREMUM < PAGE_NEW_SUPREMUM
+# error "PAGE_OLD_SUPREMUM < PAGE_NEW_SUPREMUM"
+#endif
+#if PAGE_NEW_INFIMUM > PAGE_OLD_SUPREMUM
+# error "PAGE_NEW_INFIMUM > PAGE_OLD_SUPREMUM"
+#endif
+#if PAGE_OLD_INFIMUM > PAGE_NEW_SUPREMUM
+# error "PAGE_OLD_INFIMUM > PAGE_NEW_SUPREMUM"
+#endif
+#if PAGE_NEW_SUPREMUM > PAGE_OLD_SUPREMUM_END
+# error "PAGE_NEW_SUPREMUM > PAGE_OLD_SUPREMUM_END"
+#endif
+#if PAGE_OLD_SUPREMUM > PAGE_NEW_SUPREMUM_END
+# error "PAGE_OLD_SUPREMUM > PAGE_NEW_SUPREMUM_END"
+#endif
+	ut_ad(offset <= UNIV_PAGE_SIZE - PAGE_EMPTY_DIR_START);
 
-	return(TRUE);
+	return(UNIV_LIKELY(offset != PAGE_NEW_SUPREMUM)
+		&& UNIV_LIKELY(offset != PAGE_NEW_INFIMUM)
+		&& UNIV_LIKELY(offset != PAGE_OLD_INFIMUM)
+		&& UNIV_LIKELY(offset != PAGE_OLD_SUPREMUM));
 }
 
 /****************************************************************
 TRUE if the record is the supremum record on a page. */
 UNIV_INLINE
 ibool
-page_rec_is_supremum(
-/*=================*/
+page_rec_is_supremum_low(
+/*=====================*/
 			/* out: TRUE if the supremum record */
-	rec_t*	rec)	/* in: record */
+	ulint	offset)	/* in: record offset on page */
 {
-	ut_ad(rec);
+	ut_ad(offset >= PAGE_NEW_INFIMUM);
+	ut_ad(offset <= UNIV_PAGE_SIZE - PAGE_EMPTY_DIR_START);
 
-	if (rec == page_get_supremum_rec(buf_frame_align(rec))) {
-
-		return(TRUE);
-	}
-
-	return(FALSE);
+	return(UNIV_UNLIKELY(offset == PAGE_NEW_SUPREMUM)
+		|| UNIV_UNLIKELY(offset == PAGE_OLD_SUPREMUM));
 }
 
 /****************************************************************
 TRUE if the record is the infimum record on a page. */
 UNIV_INLINE
 ibool
-page_rec_is_infimum(
-/*================*/
+page_rec_is_infimum_low(
+/*=====================*/
 			/* out: TRUE if the infimum record */
-	rec_t*	rec)	/* in: record */
+	ulint	offset)	/* in: record offset on page */
 {
-	ut_ad(rec);
-
-	if (rec == page_get_infimum_rec(buf_frame_align(rec))) {
-
-		return(TRUE);
-	}
+	ut_ad(offset >= PAGE_NEW_INFIMUM);
+	ut_ad(offset <= UNIV_PAGE_SIZE - PAGE_EMPTY_DIR_START);
 
-	return(FALSE);
+	return(UNIV_UNLIKELY(offset == PAGE_NEW_INFIMUM)
+		|| UNIV_UNLIKELY(offset == PAGE_OLD_INFIMUM));
 }
 
 /****************************************************************
-TRUE if the record is the first user record on the page. */
+TRUE if the record is a user record on the page. */
 UNIV_INLINE
 ibool
-page_rec_is_first_user_rec(
-/*=======================*/
-			/* out: TRUE if first user record */
-	rec_t*	rec)	/* in: record */
+page_rec_is_user_rec(
+/*=================*/
+				/* out: TRUE if a user record */
+	const rec_t*	rec)	/* in: record */
 {
-	ut_ad(rec);
-
-	if (rec == page_get_supremum_rec(buf_frame_align(rec))) {
-
-		return(FALSE);
-	}
-
-	if (rec == page_rec_get_next(
-	     		page_get_infimum_rec(buf_frame_align(rec)))) {
-
-	     	return(TRUE);
-	}
-
-	return(FALSE);
+	return(page_rec_is_user_rec_low(
+			ut_align_offset(rec, UNIV_PAGE_SIZE)));
 }
 
 /****************************************************************
-TRUE if the record is the last user record on the page. */
+TRUE if the record is the supremum record on a page. */
 UNIV_INLINE
 ibool
-page_rec_is_last_user_rec(
-/*======================*/
-			/* out: TRUE if last user record */
-	rec_t*	rec)	/* in: record */
+page_rec_is_supremum(
+/*=================*/
+				/* out: TRUE if the supremum record */
+	const rec_t*	rec)	/* in: record */
 {
-	ut_ad(rec);
-
-	if (rec == page_get_supremum_rec(buf_frame_align(rec))) {
-
-		return(FALSE);
-	}
-
-	if (page_rec_get_next(rec)
-			== page_get_supremum_rec(buf_frame_align(rec))) {
-
-	     	return(TRUE);
-	}
+	return(page_rec_is_supremum_low(
+			ut_align_offset(rec, UNIV_PAGE_SIZE)));
+}
 
-	return(FALSE);
+/****************************************************************
+TRUE if the record is the infimum record on a page. */
+UNIV_INLINE
+ibool
+page_rec_is_infimum(
+/*================*/
+				/* out: TRUE if the infimum record */
+	const rec_t*	rec)	/* in: record */
+{
+	return(page_rec_is_infimum_low(
+			ut_align_offset(rec, UNIV_PAGE_SIZE)));
 }
 
 /*****************************************************************
@@ -340,22 +348,26 @@ page_cmp_dtuple_rec_with_match(
 				matched; when function returns contains the
 				value for current comparison */
 {
-	page_t*	page;
+	ulint	rec_offset;
 
 	ut_ad(dtuple_check_typed(dtuple));
 	ut_ad(rec_offs_validate(rec, NULL, offsets));
+	ut_ad(!rec_offs_comp(offsets) == !page_rec_is_comp(rec));
 
-	page = buf_frame_align(rec);	
+	rec_offset = ut_align_offset(rec, UNIV_PAGE_SIZE);
 
-	if (rec == page_get_infimum_rec(page)) {
+	if (UNIV_UNLIKELY(rec_offset == PAGE_NEW_INFIMUM)
+			|| UNIV_UNLIKELY(rec_offset == PAGE_OLD_INFIMUM)) {
 		return(1);
-	} else if (rec == page_get_supremum_rec(page)) {
+	}
+	if (UNIV_UNLIKELY(rec_offset == PAGE_NEW_SUPREMUM)
+			|| UNIV_UNLIKELY(rec_offset == PAGE_OLD_SUPREMUM)) {
 		return(-1);
-	} else {
-		return(cmp_dtuple_rec_with_match(dtuple, rec, offsets,
+	}
+
+	return(cmp_dtuple_rec_with_match(dtuple, rec, offsets,
 						matched_fields,
 						matched_bytes));
-	}
 }
 
 /*****************************************************************
@@ -482,7 +494,7 @@ page_dir_slot_set_rec(
 {
 	ut_ad(page_rec_check(rec));
 
-	mach_write_to_2(slot, rec - buf_frame_align(rec));
+	mach_write_to_2(slot, ut_align_offset(rec, UNIV_PAGE_SIZE));
 }
 
 /*******************************************************************
@@ -494,8 +506,8 @@ page_dir_slot_get_n_owned(
 					/* out: number of records */
 	page_dir_slot_t* 	slot)	/* in: page directory slot */
 {
-	return(rec_get_n_owned(page_dir_slot_get_rec(slot),
-		page_is_comp(buf_frame_align(slot))));
+	rec_t*	rec	= page_dir_slot_get_rec(slot);
+	return(rec_get_n_owned(rec, page_rec_is_comp(rec)));
 }
 
 /*******************************************************************
@@ -508,8 +520,8 @@ page_dir_slot_set_n_owned(
 	ulint			n)	/* in: number of records owned 
 					by the slot */
 {
-	rec_set_n_owned(page_dir_slot_get_rec(slot),
-		page_is_comp(buf_frame_align(slot)), n);
+	rec_t*	rec	= page_dir_slot_get_rec(slot);
+	rec_set_n_owned(rec, page_rec_is_comp(rec), n);
 }
 
 /****************************************************************
@@ -540,26 +552,25 @@ page_rec_get_next(
 
 	ut_ad(page_rec_check(rec));	
 
-	page = buf_frame_align(rec);
+	page = ut_align_down(rec, UNIV_PAGE_SIZE);
 
 	offs = rec_get_next_offs(rec, page_is_comp(page));
 
-	if (offs >= UNIV_PAGE_SIZE) {
+	if (UNIV_UNLIKELY(offs >= UNIV_PAGE_SIZE)) {
 		fprintf(stderr,
-"InnoDB: Next record offset is nonsensical %lu in record at offset %lu\n",
-		(ulong)offs, (ulong)(rec - page));
-		fprintf(stderr,
-"\nInnoDB: rec address %p, first buffer frame %p\n"
+"InnoDB: Next record offset is nonsensical %lu in record at offset %lu\n"
+"InnoDB: rec address %p, first buffer frame %p\n"
 "InnoDB: buffer pool high end %p, buf fix count %lu\n",
+				(ulong)offs, (ulong)(rec - page),
 				rec, buf_pool->frame_zero,
 				buf_pool->high_end,
 				(ulong)buf_block_align(rec)->buf_fix_count);
 		buf_page_print(page);
 
-		ut_a(0);
+		ut_error;
 	}
 
-	if (offs == 0) {
+	if (UNIV_UNLIKELY(offs == 0)) {
 		
 		return(NULL);
 	}
@@ -581,15 +592,12 @@ page_rec_set_next(
 	ulint	offs;
 
 	ut_ad(page_rec_check(rec));	
-	ut_a((next == NULL)
-	      || (buf_frame_align(rec) == buf_frame_align(next)));
-
-	page = buf_frame_align(rec);
-
-	ut_ad(rec != page_get_supremum_rec(page));
-	ut_ad(next != page_get_infimum_rec(page));
+	ut_ad(!page_rec_is_supremum(rec));
+	ut_ad(!page_rec_is_infimum(next));
+	page = ut_align_down(rec, UNIV_PAGE_SIZE);
 
 	if (next) {
+		ut_a(page == ut_align_down(next, UNIV_PAGE_SIZE));
 		offs = (ulint) (next - page);
 	} else {
 		offs = 0;
@@ -613,13 +621,12 @@ page_rec_get_prev(
 	rec_t*			rec2;
 	rec_t*			prev_rec = NULL;
 	page_t*			page;
-	ibool			comp;
 
 	ut_ad(page_rec_check(rec));	
 
 	page = buf_frame_align(rec);
 
-	ut_ad(rec != page_get_infimum_rec(page));
+	ut_ad(!page_rec_is_infimum(rec));
 
 	slot_no = page_dir_find_owner_slot(rec);
 
@@ -628,7 +635,6 @@ page_rec_get_prev(
 	slot = page_dir_get_nth_slot(page, slot_no - 1);
 	
 	rec2 = page_dir_slot_get_rec(slot);
-	comp = page_is_comp(page);
 	
 	while (rec != rec2) {
 		prev_rec = rec2;
@@ -649,13 +655,16 @@ page_rec_find_owner_rec(
 			/* out: the owner record */
 	rec_t*	rec)	/* in: the physical record */
 {
-	ibool	comp;
-
 	ut_ad(page_rec_check(rec));
-	comp = page_is_comp(buf_frame_align(rec));
 
-	while (rec_get_n_owned(rec, comp) == 0) {
-		rec = page_rec_get_next(rec);
+	if (page_rec_is_comp(rec)) {
+		while (rec_get_n_owned(rec, TRUE) == 0) {
+			rec = page_rec_get_next(rec);
+		}
+	} else {
+		while (rec_get_n_owned(rec, FALSE) == 0) {
+			rec = page_rec_get_next(rec);
+		}
 	}
 
 	return(rec);
@@ -691,10 +700,17 @@ ulint
 page_get_free_space_of_empty(
 /*=========================*/
 				/* out: free space */
-	ibool comp)		/* in: TRUE=compact page layout */
+	ulint	comp)		/* in: nonzero=compact page layout */
 {
+	if (UNIV_LIKELY(comp)) {
+		return((ulint)(UNIV_PAGE_SIZE
+		- PAGE_NEW_SUPREMUM_END
+		- PAGE_DIR
+		- 2 * PAGE_DIR_SLOT_SIZE));
+	}
+
 	return((ulint)(UNIV_PAGE_SIZE
-		- (comp ? PAGE_NEW_SUPREMUM_END : PAGE_OLD_SUPREMUM_END)
+		- PAGE_OLD_SUPREMUM_END
 		- PAGE_DIR
 		- 2 * PAGE_DIR_SLOT_SIZE));
 }
@@ -716,17 +732,21 @@ page_get_max_insert_size(
 {
 	ulint	occupied;
 	ulint	free_space;
-	ibool	comp;
 
-	comp = page_is_comp(page);
+	if (page_is_comp(page)) {
+		occupied = page_header_get_field(page, PAGE_HEAP_TOP)
+		- PAGE_NEW_SUPREMUM_END + page_dir_calc_reserved_space(
+			n_recs + page_dir_get_n_heap(page) - 2);
 
-	occupied = page_header_get_field(page, PAGE_HEAP_TOP)
-		- (comp ? PAGE_NEW_SUPREMUM_END : PAGE_OLD_SUPREMUM_END)
-		+ page_dir_calc_reserved_space(
+		free_space = page_get_free_space_of_empty(TRUE);
+	} else {
+		occupied = page_header_get_field(page, PAGE_HEAP_TOP)
+		- PAGE_OLD_SUPREMUM_END + page_dir_calc_reserved_space(
 			n_recs + page_dir_get_n_heap(page) - 2);
 
-	free_space = page_get_free_space_of_empty(comp);
-		     
+		free_space = page_get_free_space_of_empty(FALSE);
+	}
+
 	/* Above the 'n_recs +' part reserves directory space for the new
 	inserted records; the '- 2' excludes page infimum and supremum
 	records */
@@ -752,14 +772,11 @@ page_get_max_insert_size_after_reorganize(
 {
 	ulint	occupied;
 	ulint	free_space;
-	ibool	comp;
 
-	comp = page_is_comp(page);
-	
 	occupied = page_get_data_size(page)
 		+ page_dir_calc_reserved_space(n_recs + page_get_n_recs(page));
 
-	free_space = page_get_free_space_of_empty(comp);
+	free_space = page_get_free_space_of_empty(page_is_comp(page));
 
 	if (occupied > free_space) {
 
@@ -783,6 +800,7 @@ page_mem_free(
 	ulint		garbage;
 
 	ut_ad(rec_offs_validate(rec, NULL, offsets));
+	ut_ad(!rec_offs_comp(offsets) == !page_rec_is_comp(rec));
 	free = page_header_get_ptr(page, PAGE_FREE);
 
 	page_rec_set_next(rec, free);

innobase/include/rem0rec.ic

@@ -929,14 +929,14 @@ rec_get_nth_field(
 Determine if the offsets are for a record in the new
 compact format. */
 UNIV_INLINE
-ibool
+ulint
 rec_offs_comp(
 /*==========*/
-				/* out: TRUE if compact format */
+				/* out: nonzero if compact format */
 	const ulint*	offsets)/* in: array returned by rec_get_offsets() */
 {
 	ut_ad(rec_offs_validate(NULL, NULL, offsets));
-	return((*rec_offs_base(offsets) & REC_OFFS_COMPACT) != 0);
+	return(*rec_offs_base(offsets) & REC_OFFS_COMPACT);
 }
 
 /**********************************************************

innobase/include/row0mysql.h

@@ -110,7 +110,7 @@ row_mysql_store_col_in_innobase_format(
 					necessarily the length of the actual
 					payload data; if the column is a true
 					VARCHAR then this is irrelevant */
-	ibool		comp);		/* in: TRUE = compact format */
+	ulint		comp);		/* in: nonzero=compact format */
 /********************************************************************
 Handles user errors and lock waits detected by the database engine. */
 

innobase/lock/lock0lock.c

@@ -501,12 +501,7 @@ lock_clust_rec_cons_read_sees(
 
 	trx_id = row_get_rec_trx_id(rec, index, offsets);
 	
-	if (read_view_sees_trx_id(view, trx_id)) {
-
-		return(TRUE);
-	}
-
-	return(FALSE);
+	return(read_view_sees_trx_id(view, trx_id));
 }
 
 /*************************************************************************
@@ -1270,7 +1265,6 @@ lock_rec_get_next(
 /*==============*/
 			/* out: next lock, NULL if none exists */
 	rec_t*	rec,	/* in: record on a page */
-	ibool	comp,	/* in: TRUE=compact page format */
 	lock_t*	lock)	/* in: lock */
 {
 #ifdef UNIV_SYNC_DEBUG
@@ -1278,19 +1272,19 @@ lock_rec_get_next(
 #endif /* UNIV_SYNC_DEBUG */
 	ut_ad(lock_get_type(lock) == LOCK_REC);
 
-	for (;;) {
-		lock = lock_rec_get_next_on_page(lock);
-
-		if (lock == NULL) {
-
-			return(NULL);
-		}
-
-		if (lock_rec_get_nth_bit(lock, rec_get_heap_no(rec, comp))) {
-
-			return(lock);
-		}
+	if (page_rec_is_comp(rec)) {
+		do {
+			lock = lock_rec_get_next_on_page(lock);
+		} while (lock && !lock_rec_get_nth_bit(lock,
+					rec_get_heap_no(rec, TRUE)));
+	} else {
+		do {
+			lock = lock_rec_get_next_on_page(lock);
+		} while (lock && !lock_rec_get_nth_bit(lock,
+					rec_get_heap_no(rec, FALSE)));
 	}
+
+	return(lock);
 }
 
 /*************************************************************************
@@ -1303,22 +1297,24 @@ lock_rec_get_first(
 	rec_t*	rec)	/* in: record on a page */
 {
 	lock_t*	lock;
-	ibool	comp;
 
 #ifdef UNIV_SYNC_DEBUG
 	ut_ad(mutex_own(&kernel_mutex));
 #endif /* UNIV_SYNC_DEBUG */
 
 	lock = lock_rec_get_first_on_page(rec);
-	comp = page_is_comp(buf_frame_align(rec));
-
-	while (lock) {
-		if (lock_rec_get_nth_bit(lock, rec_get_heap_no(rec, comp))) {
-
-			break;
+	if (UNIV_LIKELY_NULL(lock)) {
+		if (page_rec_is_comp(rec)) {
+			while (lock && !lock_rec_get_nth_bit(lock,
+					rec_get_heap_no(rec, TRUE))) {
+				lock = lock_rec_get_next_on_page(lock);
+			}
+		} else {
+			while (lock && !lock_rec_get_nth_bit(lock,
+					rec_get_heap_no(rec, FALSE))) {
+				lock = lock_rec_get_next_on_page(lock);
+			}
 		}
-
-		lock = lock_rec_get_next_on_page(lock);
 	}
 
 	return(lock);
@@ -1480,7 +1476,6 @@ lock_rec_has_expl(
 			for a supremum record we regard this always a gap
 			type request */
 	rec_t*	rec,	/* in: record */
-	ibool	comp,	/* in: TRUE=compact page format */
 	trx_t*	trx)	/* in: transaction */
 {
 	lock_t*	lock;
@@ -1510,7 +1505,7 @@ lock_rec_has_expl(
 		    	return(lock);
 		}
 
-		lock = lock_rec_get_next(rec, comp, lock);
+		lock = lock_rec_get_next(rec, lock);
 	}
 
 	return(NULL);
@@ -1529,7 +1524,6 @@ lock_rec_other_has_expl_req(
 	ulint	wait,	/* in: LOCK_WAIT if also waiting locks are
 			taken into account, or 0 if not */
 	rec_t*	rec,	/* in: record to look at */	
-	ibool	comp,	/* in: TRUE=compact record format */
 	trx_t*	trx)	/* in: transaction, or NULL if requests by all
 			transactions are taken into account */
 {
@@ -1554,7 +1548,7 @@ lock_rec_other_has_expl_req(
 		    	return(lock);
 		}
 
-		lock = lock_rec_get_next(rec, comp, lock);
+		lock = lock_rec_get_next(rec, lock);
 	}
 
 	return(NULL);
@@ -1575,13 +1569,11 @@ lock_rec_other_has_conflicting(
 	trx_t*	trx)	/* in: our transaction */
 {
 	lock_t*	lock;
-	ibool	comp;
 #ifdef UNIV_SYNC_DEBUG
 	ut_ad(mutex_own(&kernel_mutex));
 #endif /* UNIV_SYNC_DEBUG */
 
 	lock = lock_rec_get_first(rec);
-	comp = page_is_comp(buf_frame_align(rec));
 
 	while (lock) {
 		if (lock_rec_has_to_wait(trx, mode, lock,
@@ -1590,7 +1582,7 @@ lock_rec_other_has_conflicting(
 			return(lock);
 		}
 		
-		lock = lock_rec_get_next(rec, comp, lock);
+		lock = lock_rec_get_next(rec, lock);
 	}
 
 	return(NULL);
@@ -1616,7 +1608,7 @@ lock_rec_find_similar_on_page(
 	ut_ad(mutex_own(&kernel_mutex));
 #endif /* UNIV_SYNC_DEBUG */
 
-	heap_no = rec_get_heap_no(rec, page_is_comp(buf_frame_align(rec)));
+	heap_no = rec_get_heap_no(rec, page_rec_is_comp(rec));
 	lock = lock_rec_get_first_on_page(rec);
 
 	while (lock != NULL) {
@@ -1718,6 +1710,8 @@ lock_rec_create(
 	page_no	= buf_frame_get_page_no(page);
 	heap_no = rec_get_heap_no(rec, page_is_comp(page));
 
+	ut_ad(!!page_is_comp(page) == index->table->comp);
+
 	/* If rec is the supremum record, then we reset the gap and
 	LOCK_REC_NOT_GAP bits, as all locks on the supremum are
 	automatically of the gap type */
@@ -1734,7 +1728,7 @@ lock_rec_create(
 
 	lock = mem_heap_alloc(trx->lock_heap, sizeof(lock_t) + n_bytes);
 	
-	if (lock == NULL) {
+	if (UNIV_UNLIKELY(lock == NULL)) {
 
 		return(NULL);
 	}
@@ -1835,7 +1829,7 @@ lock_rec_enqueue_waiting(
 
 		lock_reset_lock_and_trx_wait(lock);
 		lock_rec_reset_nth_bit(lock, rec_get_heap_no(rec,
-					page_is_comp(buf_frame_align(rec))));
+					page_rec_is_comp(rec)));
 
 		return(DB_DEADLOCK);
 	}
@@ -1885,7 +1879,6 @@ lock_rec_add_to_queue(
 	lock_t*	lock;
 	lock_t*	similar_lock	= NULL;
 	ulint	heap_no;
-	page_t*	page		= buf_frame_align(rec);
 	ibool	somebody_waits	= FALSE;
 	
 #ifdef UNIV_SYNC_DEBUG
@@ -1894,11 +1887,11 @@ lock_rec_add_to_queue(
 	ut_ad((type_mode & (LOCK_WAIT | LOCK_GAP))
 	      || ((type_mode & LOCK_MODE_MASK) != LOCK_S)
 	      || !lock_rec_other_has_expl_req(LOCK_X, 0, LOCK_WAIT,
-					rec, page_is_comp(page), trx));
+					rec, trx));
 	ut_ad((type_mode & (LOCK_WAIT | LOCK_GAP))
 	      || ((type_mode & LOCK_MODE_MASK) != LOCK_X)
 	      || !lock_rec_other_has_expl_req(LOCK_S, 0, LOCK_WAIT,
-					rec, page_is_comp(page), trx));
+					rec, trx));
 
 	type_mode = type_mode | LOCK_REC;
 
@@ -1907,7 +1900,7 @@ lock_rec_add_to_queue(
 	try to avoid unnecessary memory consumption of a new record lock
 	struct for a gap type lock */
 
-	if (rec == page_get_supremum_rec(page)) {
+	if (page_rec_is_supremum(rec)) {
 		ut_ad(!(type_mode & LOCK_REC_NOT_GAP));
 
 		/* There should never be LOCK_REC_NOT_GAP on a supremum
@@ -1918,7 +1911,7 @@ lock_rec_add_to_queue(
 
 	/* Look for a waiting lock request on the same record or on a gap */
 
-	heap_no = rec_get_heap_no(rec, page_is_comp(page));
+	heap_no = rec_get_heap_no(rec, page_rec_is_comp(rec));
 	lock = lock_rec_get_first_on_page(rec);
 
 	while (lock != NULL) {
@@ -1993,7 +1986,7 @@ lock_rec_lock_fast(
 			|| mode - (LOCK_MODE_MASK & mode) == 0
 			|| mode - (LOCK_MODE_MASK & mode) == LOCK_REC_NOT_GAP);
 			
-	heap_no = rec_get_heap_no(rec, page_is_comp(buf_frame_align(rec)));
+	heap_no = rec_get_heap_no(rec, page_rec_is_comp(rec));
 	
 	lock = lock_rec_get_first_on_page(rec);
 
@@ -2074,8 +2067,7 @@ lock_rec_lock_slow(
 			
 	trx = thr_get_trx(thr);
 		
-	if (lock_rec_has_expl(mode, rec,
-				page_is_comp(buf_frame_align(rec)), trx)) {
+	if (lock_rec_has_expl(mode, rec, trx)) {
 		/* The trx already has a strong enough lock on rec: do
 		nothing */
 
@@ -2392,14 +2384,12 @@ lock_rec_reset_and_release_wait(
 {
 	lock_t*	lock;
 	ulint	heap_no;
-	ibool	comp;
 
 #ifdef UNIV_SYNC_DEBUG
 	ut_ad(mutex_own(&kernel_mutex));
 #endif /* UNIV_SYNC_DEBUG */
 
-	comp = page_is_comp(buf_frame_align(rec));
-	heap_no = rec_get_heap_no(rec, comp);
+	heap_no = rec_get_heap_no(rec, page_rec_is_comp(rec));
 	
 	lock = lock_rec_get_first(rec);
 
@@ -2410,7 +2400,7 @@ lock_rec_reset_and_release_wait(
 			lock_rec_reset_nth_bit(lock, heap_no);
 		}
 
-		lock = lock_rec_get_next(rec, comp, lock);
+		lock = lock_rec_get_next(rec, lock);
 	}
 }	
 
@@ -2428,13 +2418,11 @@ lock_rec_inherit_to_gap(
 			the locks on this record */
 {
 	lock_t*	lock;
-	ibool	comp;
 #ifdef UNIV_SYNC_DEBUG
 	ut_ad(mutex_own(&kernel_mutex));
 #endif /* UNIV_SYNC_DEBUG */
 	
 	lock = lock_rec_get_first(rec);
-	comp = page_is_comp(buf_frame_align(rec));
 
 	while (lock != NULL) {
 		if (!lock_rec_get_insert_intention(lock)) {
@@ -2444,7 +2432,7 @@ lock_rec_inherit_to_gap(
 	 			     		heir, lock->index, lock->trx);
 	 	}
 	 	
-		lock = lock_rec_get_next(rec, comp, lock);
+		lock = lock_rec_get_next(rec, lock);
 	}
 }	
 
@@ -2461,13 +2449,11 @@ lock_rec_inherit_to_gap_if_gap_lock(
 			the locks on this record */
 {
 	lock_t*	lock;
-	ibool	comp;
 #ifdef UNIV_SYNC_DEBUG
 	ut_ad(mutex_own(&kernel_mutex));
 #endif /* UNIV_SYNC_DEBUG */
 	
 	lock = lock_rec_get_first(rec);
-	comp = page_is_comp(buf_frame_align(rec));
 
 	while (lock != NULL) {
 		if (!lock_rec_get_insert_intention(lock)
@@ -2479,7 +2465,7 @@ lock_rec_inherit_to_gap_if_gap_lock(
 	 			     		heir, lock->index, lock->trx);
 	 	}
 
-		lock = lock_rec_get_next(rec, comp, lock);
+		lock = lock_rec_get_next(rec, lock);
 	}
 }	
 
@@ -2493,7 +2479,7 @@ lock_rec_move(
 	rec_t*	receiver,	/* in: record which gets locks; this record
 				must have no lock requests on it! */
 	rec_t*	donator,	/* in: record which gives locks */
-	ibool	comp)		/* in: TRUE=compact page format */
+	ulint	comp)		/* in: nonzero=compact page format */
 {
 	lock_t*	lock;
 	ulint	heap_no;
@@ -2523,7 +2509,7 @@ lock_rec_move(
 
 		lock_rec_add_to_queue(type_mode, receiver, lock->index,
 								lock->trx);
-		lock = lock_rec_get_next(donator, comp, lock);
+		lock = lock_rec_get_next(donator, lock);
 	}
 
 	ut_ad(lock_rec_get_first(donator) == NULL);
@@ -2549,7 +2535,7 @@ lock_move_reorganize_page(
 	UT_LIST_BASE_NODE_T(lock_t)	old_locks;
 	mem_heap_t*	heap		= NULL;
 	rec_t*		sup;
-	ibool		comp;
+	ulint		comp;
 
 	lock_mutex_enter_kernel();
 
@@ -2668,8 +2654,9 @@ lock_move_rec_list_end(
 	ulint		heap_no;
 	rec_t*		sup;
 	ulint		type_mode;
-	ibool		comp;
-	
+	ulint		comp;
+	ut_ad(page == buf_frame_align(rec));
+
 	lock_mutex_enter_kernel();
 
 	/* Note: when we move locks from record to record, waiting locks
@@ -2754,7 +2741,7 @@ lock_move_rec_list_start(
 	page_cur_t	cur2;
 	ulint		heap_no;
 	ulint		type_mode;
-	ibool		comp;
+	ulint		comp;
 
 	ut_a(new_page);
 
@@ -2763,6 +2750,7 @@ lock_move_rec_list_start(
 	lock = lock_rec_get_first_on_page(page);
 	comp = page_is_comp(page);
 	ut_ad(comp == page_is_comp(new_page));
+	ut_ad(page == buf_frame_align(rec));
 
 	while (lock != NULL) {
 		
@@ -2821,7 +2809,7 @@ lock_update_split_right(
 	page_t*	right_page,	/* in: right page */
 	page_t*	left_page)	/* in: left page */
 {
-	ibool	comp;
+	ulint	comp;
 	lock_mutex_enter_kernel();
 	comp = page_is_comp(left_page);
 	ut_ad(comp == page_is_comp(right_page));
@@ -2884,7 +2872,7 @@ lock_update_root_raise(
 	page_t*	new_page,	/* in: index page to which copied */
 	page_t*	root)		/* in: root page */
 {
-	ibool	comp;
+	ulint	comp;
 	lock_mutex_enter_kernel();
 	comp = page_is_comp(root);
 	ut_ad(comp == page_is_comp(new_page));
@@ -2907,7 +2895,7 @@ lock_update_copy_and_discard(
 	page_t*	new_page,	/* in: index page to which copied */
 	page_t*	page)		/* in: index page; NOT the root! */
 {
-	ibool	comp;
+	ulint	comp;
 	lock_mutex_enter_kernel();
 	comp = page_is_comp(page);
 	ut_ad(comp == page_is_comp(new_page));
@@ -2954,31 +2942,34 @@ lock_update_merge_left(
 	page_t*	right_page)	/* in: merged index page which will be
 				discarded */
 {
-	ibool	comp;
+	rec_t*	left_next_rec;
+	rec_t*	left_supremum;
+	ulint	comp;
 	lock_mutex_enter_kernel();
 	comp = page_is_comp(left_page);
 	ut_ad(comp == page_is_comp(right_page));
+	ut_ad(left_page == buf_frame_align(orig_pred));
+
+	left_next_rec = page_rec_get_next(orig_pred);
+	left_supremum = page_get_supremum_rec(left_page);
 
-	if (page_rec_get_next(orig_pred) != page_get_supremum_rec(left_page)) {
+	if (UNIV_LIKELY(left_next_rec != left_supremum)) {
 
 		/* Inherit the locks on the supremum of the left page to the
 		first record which was moved from the right page */
 
-		lock_rec_inherit_to_gap(page_rec_get_next(orig_pred),
-					page_get_supremum_rec(left_page));
+		lock_rec_inherit_to_gap(left_next_rec, left_supremum);
 
 		/* Reset the locks on the supremum of the left page,
 		releasing waiting transactions */
 
-		lock_rec_reset_and_release_wait(page_get_supremum_rec(
-								left_page));
+		lock_rec_reset_and_release_wait(left_supremum);
 	}
 
 	/* Move the locks from the supremum of right page to the supremum
 	of the left page */
 	
-	lock_rec_move(page_get_supremum_rec(left_page),
-				page_get_supremum_rec(right_page), comp);
+	lock_rec_move(left_supremum, page_get_supremum_rec(right_page), comp);
 
 	lock_rec_free_all_from_discard_page(right_page);
 
@@ -3037,7 +3028,7 @@ lock_update_discard(
 
 		lock_rec_reset_and_release_wait(rec);
 
-		if (rec == page_get_supremum_rec(page)) {
+		if (page_rec_is_supremum(rec)) {
 
 			break;
 		}
@@ -3100,19 +3091,16 @@ actual record is being moved. */
 void
 lock_rec_store_on_page_infimum(
 /*===========================*/
+	page_t*	page,	/* in: page containing the record */
 	rec_t*	rec)	/* in: record whose lock state is stored
 			on the infimum record of the same page; lock
 			bits are reset on the record */
 {
-	page_t*	page;
-	ibool	comp;
-
-	page = buf_frame_align(rec);
-	comp = page_is_comp(page);
+	ut_ad(page == buf_frame_align(rec));
 
 	lock_mutex_enter_kernel();
 	
-	lock_rec_move(page_get_infimum_rec(page), rec, comp);
+	lock_rec_move(page_get_infimum_rec(page), rec, page_is_comp(page));
 
 	lock_mutex_exit_kernel();	
 }
@@ -3129,10 +3117,10 @@ lock_rec_restore_from_page_infimum(
 			whose infimum stored the lock state; lock bits are
 			reset on the infimum */ 
 {
-	ibool	comp;
+	ulint	comp;
 	lock_mutex_enter_kernel();
 	comp = page_is_comp(page);
-	ut_ad(comp == page_is_comp(buf_frame_align(rec)));
+	ut_ad(!comp == !page_rec_is_comp(rec));
 
 	lock_rec_move(rec, page_get_infimum_rec(page), comp);
 	
@@ -4483,15 +4471,14 @@ lock_rec_queue_validate(
 {
 	trx_t*	impl_trx;	
 	lock_t*	lock;
-	ibool	comp;
 
 	ut_a(rec);
 	ut_ad(rec_offs_validate(rec, index, offsets));
-	comp = page_is_comp(buf_frame_align(rec));
+	ut_ad(!page_rec_is_comp(rec) == !rec_offs_comp(offsets));
 
 	lock_mutex_enter_kernel();
 
-	if (page_rec_is_supremum(rec) || page_rec_is_infimum(rec)) {
+	if (!page_rec_is_user_rec(rec)) {
 
 		lock = lock_rec_get_first(rec);
 
@@ -4511,7 +4498,7 @@ lock_rec_queue_validate(
 				ut_a(lock->index == index);
 			}
 
-			lock = lock_rec_get_next(rec, comp, lock);
+			lock = lock_rec_get_next(rec, lock);
 		}
 
 		lock_mutex_exit_kernel();
@@ -4524,10 +4511,10 @@ lock_rec_queue_validate(
 		impl_trx = lock_clust_rec_some_has_impl(rec, index, offsets);
 
 		if (impl_trx && lock_rec_other_has_expl_req(LOCK_S, 0,
-				LOCK_WAIT, rec, comp, impl_trx)) {
+				LOCK_WAIT, rec, impl_trx)) {
 
 			ut_a(lock_rec_has_expl(LOCK_X | LOCK_REC_NOT_GAP, rec,
-							comp, impl_trx));
+							impl_trx));
 		}
 	}
 
@@ -4541,10 +4528,10 @@ lock_rec_queue_validate(
 				rec, index, offsets);
 
 		if (impl_trx && lock_rec_other_has_expl_req(LOCK_S, 0,
-				LOCK_WAIT, rec, comp, impl_trx)) {
+				LOCK_WAIT, rec, impl_trx)) {
 
 			ut_a(lock_rec_has_expl(LOCK_X | LOCK_REC_NOT_GAP,
-						rec, comp, impl_trx));
+						rec, impl_trx));
 		}
 	}
 
@@ -4561,21 +4548,23 @@ lock_rec_queue_validate(
 		}
 
 		if (!lock_rec_get_gap(lock) && !lock_get_wait(lock)) {
+
+			ulint	mode;
 		
 			if (lock_get_mode(lock) == LOCK_S) {
-				ut_a(!lock_rec_other_has_expl_req(LOCK_X,
-						0, 0, rec, comp, lock->trx));
+				mode = LOCK_X;
 			} else {
-				ut_a(!lock_rec_other_has_expl_req(LOCK_S,
-						0, 0, rec, comp, lock->trx));
+				mode = LOCK_S;
 			}
+			ut_a(!lock_rec_other_has_expl_req(mode,
+						0, 0, rec, lock->trx));
 
 		} else if (lock_get_wait(lock) && !lock_rec_get_gap(lock)) {
 
 			ut_a(lock_rec_has_to_wait_in_queue(lock));
 		}
 
-		lock = lock_rec_get_next(rec, comp, lock);
+		lock = lock_rec_get_next(rec, lock);
 	}
 
 	lock_mutex_exit_kernel();
@@ -4887,7 +4876,7 @@ lock_rec_convert_impl_to_expl(
 #endif /* UNIV_SYNC_DEBUG */
 	ut_ad(page_rec_is_user_rec(rec));
 	ut_ad(rec_offs_validate(rec, index, offsets));
-	ut_ad(page_is_comp(buf_frame_align(rec)) == index->table->comp);
+	ut_ad(!page_rec_is_comp(rec) == !rec_offs_comp(offsets));
 
 	if (index->type & DICT_CLUSTERED) {
 		impl_trx = lock_clust_rec_some_has_impl(rec, index, offsets);
@@ -4901,7 +4890,7 @@ lock_rec_convert_impl_to_expl(
 		record, set one for it */
 
 		if (!lock_rec_has_expl(LOCK_X | LOCK_REC_NOT_GAP, rec,
-					index->table->comp, impl_trx)) {
+					impl_trx)) {
 
 			lock_rec_add_to_queue(LOCK_REC | LOCK_X
 					      | LOCK_REC_NOT_GAP, rec, index,

innobase/mtr/mtr0log.c

@@ -15,6 +15,7 @@ Created 12/7/1995 Heikki Tuuri
 #include "buf0buf.h"
 #include "dict0boot.h"
 #include "log0recv.h"
+#include "page0page.h"
 
 /************************************************************
 Catenates n bytes to the mtr log. */
@@ -405,7 +406,9 @@ mlog_open_and_write_index(
 	const byte*	log_start;
 	const byte*	log_end;
 
-	if (!index->table->comp) {
+	ut_ad(!!page_rec_is_comp(rec) == index->table->comp);
+
+	if (!page_rec_is_comp(rec)) {
 		log_start = log_ptr = mlog_open(mtr, 11 + size);
 		if (!log_ptr) {
 			return(NULL); /* logging is disabled */
@@ -498,6 +501,8 @@ mlog_parse_index(
 	dict_table_t*	table;
 	dict_index_t*	ind;
 
+	ut_ad(comp == FALSE || comp == TRUE);
+
 	if (comp) {
 		if (end_ptr < ptr + 4) {
 			return(NULL);

innobase/page/page0cur.c

@@ -515,8 +515,12 @@ page_cur_insert_rec_write_log(
 	byte*	log_ptr;
 	byte*	log_end;
 	ulint	i;
+	ulint	comp;
 
 	ut_a(rec_size < UNIV_PAGE_SIZE);
+	ut_ad(buf_frame_align(insert_rec) == buf_frame_align(cursor_rec));
+	ut_ad(!page_rec_is_comp(insert_rec) == !index->table->comp);
+	comp = page_rec_is_comp(insert_rec);
 
 	{
 		mem_heap_t*	heap		= NULL;
@@ -565,7 +569,7 @@ page_cur_insert_rec_write_log(
 				ins_ptr++;
 				cur_ptr++;
 			} else if ((i < extra_size)
-				   && (i >= extra_size - (index->table->comp
+				   && (i >= extra_size - (comp
 					? REC_N_NEW_EXTRA_BYTES
 					: REC_N_OLD_EXTRA_BYTES))) {
 				i = extra_size;
@@ -580,7 +584,7 @@ page_cur_insert_rec_write_log(
 	if (mtr_get_log_mode(mtr) != MTR_LOG_SHORT_INSERTS) {
 
 		log_ptr = mlog_open_and_write_index(mtr, insert_rec, index,
-				index->table->comp
+				comp
 				? MLOG_COMP_REC_INSERT : MLOG_REC_INSERT,
 				2 + 5 + 1 + 5 + 5 + MLOG_BUF_MARGIN);
 
@@ -605,8 +609,8 @@ page_cur_insert_rec_write_log(
 		log_end = &log_ptr[5 + 1 + 5 + 5 + MLOG_BUF_MARGIN];
 	}
 
-	if ((rec_get_info_and_status_bits(insert_rec, index->table->comp) !=
-	     rec_get_info_and_status_bits(cursor_rec, index->table->comp))
+	if ((rec_get_info_and_status_bits(insert_rec, comp) !=
+	     rec_get_info_and_status_bits(cursor_rec, comp))
 	    || (extra_size != cur_extra_size)
 	    || (rec_size != cur_rec_size)) {
 
@@ -622,8 +626,7 @@ page_cur_insert_rec_write_log(
 	if (extra_info_yes) {
 		/* Write the info bits */
 		mach_write_to_1(log_ptr,
-			rec_get_info_and_status_bits(insert_rec,
-							index->table->comp));
+			rec_get_info_and_status_bits(insert_rec, comp));
 		log_ptr++;
 
 		/* Write the record origin offset */
@@ -757,6 +760,8 @@ page_cur_parse_insert_rec(
 		return(ptr + end_seg_len);
 	}
 
+	ut_ad(!!page_is_comp(page) == index->table->comp);
+
 	/* Read from the log the inserted index record end segment which
 	differs from the cursor record */
 
@@ -771,7 +776,7 @@ page_cur_parse_insert_rec(
 
 	if (extra_info_yes == 0) {
 		info_and_status_bits = rec_get_info_and_status_bits(
-					cursor_rec, index->table->comp);
+					cursor_rec, page_is_comp(page));
 		origin_offset = rec_offs_extra_size(offsets);
 		mismatch_index = rec_offs_size(offsets) - end_seg_len;
 	}
@@ -807,7 +812,7 @@ page_cur_parse_insert_rec(
 	ut_memcpy(buf, rec_get_start(cursor_rec, offsets), mismatch_index);
 	ut_memcpy(buf + mismatch_index, ptr, end_seg_len);
 
-	rec_set_info_and_status_bits(buf + origin_offset, index->table->comp,
+	rec_set_info_and_status_bits(buf + origin_offset, page_is_comp(page),
 							info_and_status_bits);
 
 	page_cur_position(cursor_rec, &cursor);
@@ -861,7 +866,7 @@ page_cur_insert_rec_low(
 	rec_t*		owner_rec;
 	ulint		n_owned;
 	mem_heap_t*	heap		= NULL;
-	ibool		comp		= index->table->comp;
+	ulint		comp;
 
 	ut_ad(cursor && mtr);
 	ut_ad(tuple || rec);
@@ -869,8 +874,8 @@ page_cur_insert_rec_low(
 	ut_ad(rec || dtuple_check_typed(tuple));
 
 	page = page_cur_get_page(cursor);
-
-	ut_ad(page_is_comp(page) == comp);
+	comp = page_is_comp(page);
+	ut_ad(index->table->comp == !!comp);
 
 	ut_ad(cursor->rec != page_get_supremum_rec(page));	
 
@@ -1000,8 +1005,10 @@ page_copy_rec_list_to_created_page_write_log(
 {
 	byte*	log_ptr;
 
+	ut_ad(!!page_is_comp(page) == index->table->comp);
+
 	log_ptr = mlog_open_and_write_index(mtr, page, index,
-			index->table->comp
+			page_is_comp(page)
 			? MLOG_COMP_LIST_END_COPY_CREATED
 			: MLOG_LIST_END_COPY_CREATED, 4);
 	ut_a(log_ptr);
@@ -1084,7 +1091,7 @@ page_copy_rec_list_end_to_created_page(
 	ulint	log_mode;
 	byte*	log_ptr;
 	ulint	log_data_len;
-	ibool		comp		= page_is_comp(page);
+	ulint		comp		= page_is_comp(page);
 	mem_heap_t*	heap		= NULL;
 	ulint		offsets_[REC_OFFS_NORMAL_SIZE];
 	ulint*		offsets		= offsets_;
@@ -1230,8 +1237,10 @@ page_cur_delete_rec_write_log(
 {
 	byte*	log_ptr;
 
+	ut_ad(!!page_rec_is_comp(rec) == index->table->comp);
+
 	log_ptr = mlog_open_and_write_index(mtr, rec, index,
-			index->table->comp
+			page_rec_is_comp(rec)
 			? MLOG_COMP_REC_DELETE
 			: MLOG_REC_DELETE, 2);
 
@@ -1242,7 +1251,7 @@ page_cur_delete_rec_write_log(
 	}
 
 	/* Write the cursor rec offset as a 2-byte ulint */
-	mach_write_to_2(log_ptr, rec - buf_frame_align(rec));
+	mach_write_to_2(log_ptr, ut_align_offset(rec, UNIV_PAGE_SIZE));
 
 	mlog_close(mtr, log_ptr + 2);
 }	
@@ -1320,6 +1329,7 @@ page_cur_delete_rec(
 	page = page_cur_get_page(cursor);
 	current_rec = cursor->rec;
 	ut_ad(rec_offs_validate(current_rec, index, offsets));
+	ut_ad(!!page_is_comp(page) == index->table->comp);
 
 	/* The record must not be the supremum or infimum record. */
 	ut_ad(current_rec != page_get_supremum_rec(page));	

innobase/page/page0page.c

@@ -72,65 +72,70 @@ page_dir_find_owner_slot(
 			/* out: the directory slot number */
 	rec_t*	rec)	/* in: the physical record */
 {
-	ulint			i;
-	ulint			steps		= 0;
-	page_t*			page;	
-	page_dir_slot_t*	slot;
-	rec_t*			original_rec	= rec;
-	ibool			comp;
-	
+	page_t*				page;
+	register uint16			rec_offs_bytes;
+	register page_dir_slot_t*	slot;
+	register const page_dir_slot_t*	first_slot;
+	register rec_t*			r = rec;
+
 	ut_ad(page_rec_check(rec));
 
 	page = buf_frame_align(rec);
-	comp = page_is_comp(page);
-
-	while (rec_get_n_owned(rec, comp) == 0) {
-		steps++;
-		rec = page_rec_get_next(rec);
+	first_slot = page_dir_get_nth_slot(page, 0);
+	slot = page_dir_get_nth_slot(page, page_dir_get_n_slots(page) - 1);
+
+	if (page_is_comp(page)) {
+		while (rec_get_n_owned(r, TRUE) == 0) {
+			r = page + rec_get_next_offs(r, TRUE);
+			ut_ad(r >= page + PAGE_NEW_SUPREMUM);
+			ut_ad(r < page + (UNIV_PAGE_SIZE - PAGE_DIR));
+		}
+	} else {
+		while (rec_get_n_owned(r, FALSE) == 0) {
+			r = page + rec_get_next_offs(r, FALSE);
+			ut_ad(r >= page + PAGE_OLD_SUPREMUM);
+			ut_ad(r < page + (UNIV_PAGE_SIZE - PAGE_DIR));
+		}
 	}
-	
-	page = buf_frame_align(rec);
 
-	i = page_dir_get_n_slots(page) - 1;
-	slot = page_dir_get_nth_slot(page, i); 
+	rec_offs_bytes = mach_encode_2(r - page);
 
-	while (page_dir_slot_get_rec(slot) != rec) {
+	while (UNIV_LIKELY(*(uint16*) slot != rec_offs_bytes)) {
 
- 		if (i == 0) {
+		if (UNIV_UNLIKELY(slot == first_slot)) {
 			fprintf(stderr,
 			"InnoDB: Probable data corruption on page %lu\n"
 			"InnoDB: Original record ",
 			(ulong) buf_frame_get_page_no(page));
 
-			if (comp) {
+			if (page_is_comp(page)) {
 				fputs("(compact record)", stderr);
 			} else {
-				rec_print_old(stderr, original_rec);
+				rec_print_old(stderr, rec);
 			}
 
-			fprintf(stderr, "\n"
-			"InnoDB: on that page. Steps %lu.\n", (ulong) steps);
-			fputs(
+			fputs("\n"
+			"InnoDB: on that page.\n"
 			"InnoDB: Cannot find the dir slot for record ",
 				stderr);
-			if (comp) {
+			if (page_is_comp(page)) {
 				fputs("(compact record)", stderr);
 			} else {
-				rec_print_old(stderr, rec);
+				rec_print_old(stderr, page
+					+ mach_decode_2(rec_offs_bytes));
 			}
 			fputs("\n"
 			"InnoDB: on that page!\n", stderr);
 
 			buf_page_print(page);
 
-	  		ut_error;
-	  	}
+			ut_error;
+		}
 
-		i--;
-		slot = page_dir_get_nth_slot(page, i); 
+		slot += PAGE_DIR_SLOT_SIZE;
 	}
 
-	return(i);
+	return(((ulint) (first_slot - slot)) / PAGE_DIR_SLOT_SIZE);
 }
 
 /******************************************************************
@@ -290,7 +295,7 @@ page_create_write_log(
 	buf_frame_t*	frame,	/* in: a buffer frame where the page is
 				created */
 	mtr_t*		mtr,	/* in: mini-transaction handle */
-	ibool		comp)	/* in: TRUE=compact page format */
+	ulint		comp)	/* in: nonzero=compact page format */
 {
 	mlog_write_initial_log_record(frame,
 			comp ? MLOG_COMP_PAGE_CREATE : MLOG_PAGE_CREATE, mtr);
@@ -305,7 +310,7 @@ page_parse_create(
 			/* out: end of log record or NULL */
 	byte*	ptr,	/* in: buffer */
 	byte*	end_ptr __attribute__((unused)), /* in: buffer end */
-	ibool	comp,	/* in: TRUE=compact page format */
+	ulint	comp,	/* in: nonzero=compact page format */
 	page_t*	page,	/* in: page or NULL */
 	mtr_t*	mtr)	/* in: mtr or NULL */
 {
@@ -330,7 +335,7 @@ page_create(
 	buf_frame_t*	frame,	/* in: a buffer frame where the page is
 				created */
 	mtr_t*		mtr,	/* in: mini-transaction handle */
-	ibool		comp)	/* in: TRUE=compact page format */
+	ulint		comp)	/* in: nonzero=compact page format */
 {
 	page_dir_slot_t* slot;
 	mem_heap_t*	heap;
@@ -396,9 +401,9 @@ page_create(
 	dtuple_set_info_bits(tuple, REC_STATUS_SUPREMUM);
 	field = dtuple_get_nth_field(tuple, 0);
 
-	dfield_set_data(field, "supremum", 9 - comp);
+	dfield_set_data(field, "supremum", comp ? 8 : 9);
 	dtype_set(dfield_get_type(field),
-		DATA_VARCHAR, DATA_ENGLISH | DATA_NOT_NULL, 9 - comp, 0);
+		DATA_VARCHAR, DATA_ENGLISH | DATA_NOT_NULL, comp ? 8 : 9, 0);
 
 	supremum_rec = rec_convert_dtuple_to_rec(heap_top, index, tuple);
 
@@ -478,10 +483,11 @@ page_copy_rec_list_end_no_locks(
 		page_cur_move_to_next(&cur1);
 	}
 
-	ut_a(index->table->comp == page_is_comp(page));
-	ut_a(index->table->comp == page_is_comp(new_page));
+	ut_a(!!page_is_comp(new_page) == index->table->comp);
+	ut_a(page_is_comp(new_page) == page_is_comp(page));
 	ut_a(mach_read_from_2(new_page + UNIV_PAGE_SIZE - 10) == (ulint)
-		(index->table->comp ? PAGE_NEW_INFIMUM : PAGE_OLD_INFIMUM));
+		(page_is_comp(new_page)
+		? PAGE_NEW_INFIMUM : PAGE_OLD_INFIMUM));
 
 	page_cur_set_before_first(new_page, &cur2);
 	
@@ -489,12 +495,15 @@ page_copy_rec_list_end_no_locks(
 
 	sup = page_get_supremum_rec(page);
 	
-	while (sup != page_cur_get_rec(&cur1)) {
+	for (;;) {
 		rec_t*	cur1_rec = page_cur_get_rec(&cur1);
+		if (cur1_rec == sup) {
+			break;
+		}
 		offsets = rec_get_offsets(cur1_rec, index, offsets,
 					ULINT_UNDEFINED, &heap);
-		if (!page_cur_rec_insert(&cur2, cur1_rec, index,
-							offsets, mtr)) {
+		if (UNIV_UNLIKELY(!page_cur_rec_insert(&cur2, cur1_rec, index,
+							offsets, mtr))) {
 			/* Track an assertion failure reported on the mailing
 			list on June 18th, 2003 */
 
@@ -619,7 +628,6 @@ UNIV_INLINE
 void
 page_delete_rec_list_write_log(
 /*===========================*/
-	page_t*		page,	/* in: index page */
 	rec_t*		rec,	/* in: record on page */
 	dict_index_t*	index,	/* in: record descriptor */
 	byte		type,	/* in: operation type:
@@ -632,10 +640,10 @@ page_delete_rec_list_write_log(
 		|| type == MLOG_COMP_LIST_END_DELETE
 		|| type == MLOG_COMP_LIST_START_DELETE);
 
-	log_ptr = mlog_open_and_write_index(mtr, page, index, type, 2);
+	log_ptr = mlog_open_and_write_index(mtr, rec, index, type, 2);
 	if (log_ptr) {
 		/* Write the parameter as a 2-byte ulint */
-		mach_write_to_2(log_ptr, rec - page);
+		mach_write_to_2(log_ptr, ut_align_offset(rec, UNIV_PAGE_SIZE));
 		mlog_close(mtr, log_ptr + 2);
 	}
 }
@@ -679,6 +687,8 @@ page_parse_delete_rec_list(
 		return(ptr);
 	}
 
+	ut_ad(!!page_is_comp(page) == index->table->comp);
+
 	if (type == MLOG_LIST_END_DELETE
 			|| type == MLOG_COMP_LIST_END_DELETE) {
 		page_delete_rec_list_end(page, page + offset, index,
@@ -716,7 +726,7 @@ page_delete_rec_list_end(
 	ulint	count;
 	ulint	n_owned;
 	rec_t*	sup;
-	ibool	comp;
+	ulint	comp;
 
 	/* Reset the last insert info in the page header and increment
 	the modify clock for the frame */
@@ -731,12 +741,12 @@ page_delete_rec_list_end(
 	
 	sup = page_get_supremum_rec(page);
 	
-	if (rec == page_get_infimum_rec(page)) {
+	comp = page_is_comp(page);
+	if (page_rec_is_infimum_low(rec - page)) {
 		rec = page_rec_get_next(rec);
 	}
 
-	comp = page_is_comp(page);
-	page_delete_rec_list_write_log(page, rec, index,
+	page_delete_rec_list_write_log(rec, index,
 		comp ? MLOG_COMP_LIST_END_DELETE : MLOG_LIST_END_DELETE, mtr);
 
 	if (rec == sup) {
@@ -841,13 +851,15 @@ page_delete_rec_list_start(
 	byte		type;
 	*offsets_ = (sizeof offsets_) / sizeof *offsets_;
 
-	if (index->table->comp) {
+	ut_ad(!!page_is_comp(page) == index->table->comp);
+
+	if (page_is_comp(page)) {
 		type = MLOG_COMP_LIST_START_DELETE;
 	} else {
 		type = MLOG_LIST_START_DELETE;
 	}
 
-	page_delete_rec_list_write_log(page, rec, index, type, mtr);
+	page_delete_rec_list_write_log(rec, index, type, mtr);
 
 	page_cur_set_before_first(page, &cur1);
 
@@ -1221,7 +1233,7 @@ page_rec_get_n_recs_before(
 	rec_t*			slot_rec;
 	page_t*			page;
 	ulint			i;
-	ibool			comp;
+	ulint			comp;
 	lint			n	= 0;
 
 	ut_ad(page_rec_check(rec));
@@ -1264,9 +1276,9 @@ page_rec_print(
 	rec_t*		rec,	/* in: physical record */
 	const ulint*	offsets)/* in: record descriptor */
 {
-	ibool	comp	= page_is_comp(buf_frame_align(rec));
+	ulint	comp	= page_is_comp(buf_frame_align(rec));
 
-	ut_a(comp == rec_offs_comp(offsets));
+	ut_a(!comp == !rec_offs_comp(offsets));
 	rec_print_new(stderr, rec, offsets);
 	fprintf(stderr,
      		"            n_owned: %lu; heap_no: %lu; next rec: %lu\n",
@@ -1335,7 +1347,7 @@ page_print_list(
 	ulint*		offsets		= offsets_;
 	*offsets_ = (sizeof offsets_) / sizeof *offsets_;
 
-	ut_a(page_is_comp(page) == index->table->comp);
+	ut_a(!!page_is_comp(page) == index->table->comp);
 
 	fprintf(stderr,
 		"--------------------------------\n"
@@ -1447,11 +1459,11 @@ page_rec_validate(
 	ulint	n_owned;
 	ulint	heap_no;
 	page_t*	page;
-	ibool	comp;
+	ulint	comp;
 
 	page = buf_frame_align(rec);
 	comp = page_is_comp(page);
-	ut_a(comp == rec_offs_comp(offsets));
+	ut_a(!comp == !rec_offs_comp(offsets));
 
 	page_rec_check(rec);
 	rec_validate(rec, offsets);
@@ -1528,7 +1540,7 @@ page_simple_validate(
 	ulint		count;
 	ulint		own_count;
 	ibool		ret	= FALSE;
-	ibool		comp	= page_is_comp(page);
+	ulint		comp	= page_is_comp(page);
 
 	/* Check first that the record heap and the directory do not
 	overlap. */
@@ -1725,11 +1737,11 @@ page_validate(
 	ulint		n_slots;
 	ibool		ret		= FALSE;
 	ulint		i;
-	ibool		comp		= page_is_comp(page);
+	ulint		comp		= page_is_comp(page);
 	ulint*		offsets		= NULL;
 	ulint*		old_offsets	= NULL;
 
-	if (comp != index->table->comp) {
+	if (!!comp != index->table->comp) {
 		fputs("InnoDB: 'compact format' flag mismatch\n", stderr);
 		goto func_exit2;
 	}
@@ -1810,8 +1822,7 @@ page_validate(
 			}
 		}
 
-		if ((rec != page_get_supremum_rec(page))
-		    && (rec != page_get_infimum_rec(page))) {
+		if (page_rec_is_user_rec(rec)) {
 
 			data_size += rec_offs_size(offsets);
 		}

innobase/rem/rem0cmp.c

@@ -727,7 +727,7 @@ cmp_rec_rec_with_match(
 	ulint	cur_bytes; 	/* number of already matched bytes in current
 				field */
 	int	ret = 3333;	/* return value */
-	ibool	comp;
+	ulint	comp;
 
 	ut_ad(rec1 && rec2 && index);
 	ut_ad(rec_offs_validate(rec1, index, offsets1));

innobase/row/row0ins.c

@@ -1255,9 +1255,11 @@ run_again:
 	/* Scan index records and check if there is a matching record */
 
 	for (;;) {
+		page_t*	page;
 		rec = btr_pcur_get_rec(&pcur);
+		page = buf_frame_align(rec);
 
-		if (rec == page_get_infimum_rec(buf_frame_align(rec))) {
+		if (rec == page_get_infimum_rec(page)) {
 
 			goto next_rec;
 		}
@@ -1265,7 +1267,7 @@ run_again:
 		offsets = rec_get_offsets(rec, check_index,
 					offsets, ULINT_UNDEFINED, &heap);
 
-		if (rec == page_get_supremum_rec(buf_frame_align(rec))) {
+		if (rec == page_get_supremum_rec(page)) {
 
 			err = row_ins_set_shared_rec_lock(LOCK_ORDINARY, rec,
 						check_index, offsets, thr);
@@ -1529,12 +1531,7 @@ row_ins_dupl_error_with_rec(
 	        }
 	}
 
-	if (!rec_get_deleted_flag(rec, index->table->comp)) {
-
-	        return(TRUE);
-	}
-
-	return(FALSE);
+	return(!rec_get_deleted_flag(rec, rec_offs_comp(offsets)));
 }	
 
 /*******************************************************************
@@ -1629,7 +1626,7 @@ row_ins_scan_sec_index_for_duplicate(
 			break;
 		}
 
-		if (rec == page_get_supremum_rec(buf_frame_align(rec))) {
+		if (page_rec_is_supremum(rec)) {
 		
 			goto next_rec;
 		}
@@ -1697,7 +1694,6 @@ row_ins_duplicate_error_in_clust(
 #ifndef UNIV_HOTBACKUP
 	ulint	err;
 	rec_t*	rec;
-	page_t*	page;
 	ulint	n_unique;
 	trx_t*	trx		= thr_get_trx(thr);
 	mem_heap_t*heap		= NULL;
@@ -1728,9 +1724,8 @@ row_ins_duplicate_error_in_clust(
 	if (cursor->low_match >= n_unique) {
 		
 		rec = btr_cur_get_rec(cursor);
-		page = buf_frame_align(rec);
 
-		if (rec != page_get_infimum_rec(page)) {
+		if (!page_rec_is_infimum(rec)) {
 			offsets = rec_get_offsets(rec, cursor->index, offsets,
 						ULINT_UNDEFINED, &heap);
 
@@ -1772,9 +1767,8 @@ row_ins_duplicate_error_in_clust(
 	if (cursor->up_match >= n_unique) {
 
 		rec = page_rec_get_next(btr_cur_get_rec(cursor));
-		page = buf_frame_align(rec);
 
-		if (rec != page_get_supremum_rec(page)) {
+		if (!page_rec_is_supremum(rec)) {
 			offsets = rec_get_offsets(rec, cursor->index, offsets,
 						ULINT_UNDEFINED, &heap);
 
@@ -1842,7 +1836,6 @@ row_ins_must_modify(
 {
 	ulint	enough_match;
 	rec_t*	rec;
-	page_t*	page;
 	
 	/* NOTE: (compare to the note in row_ins_duplicate_error) Because node
 	pointers on upper levels of the B-tree may match more to entry than
@@ -1856,9 +1849,8 @@ row_ins_must_modify(
 	if (cursor->low_match >= enough_match) {
 
 		rec = btr_cur_get_rec(cursor);
-		page = buf_frame_align(rec);
 
-		if (rec != page_get_infimum_rec(page)) {
+		if (!page_rec_is_infimum(rec)) {
 
 			return(ROW_INS_PREV);
 		}
@@ -1897,7 +1889,6 @@ row_ins_index_entry_low(
 	ulint		modify = 0; /* remove warning */
 	rec_t*		insert_rec;
 	rec_t*		rec;
-	rec_t*		first_rec;
 	ulint		err;
 	ulint		n_unique;
 	big_rec_t*	big_rec			= NULL;
@@ -1932,15 +1923,20 @@ row_ins_index_entry_low(
 		err = DB_SUCCESS;
 
 		goto function_exit;
-	}	
-					
-	first_rec = page_rec_get_next(page_get_infimum_rec(
-			buf_frame_align(btr_cur_get_rec(&cursor))));
+	}
 
-	if (!page_rec_is_supremum(first_rec)) {
-		ut_a(rec_get_n_fields(first_rec, index)
+#ifdef UNIV_DEBUG
+	{
+		page_t*	page = btr_cur_get_page(&cursor);
+		rec_t*	first_rec = page_rec_get_next(
+				page_get_infimum_rec(page));
+
+		if (UNIV_LIKELY(first_rec != page_get_supremum_rec(page))) {
+			ut_a(rec_get_n_fields(first_rec, index)
 			== dtuple_get_n_fields(entry));
+		}
 	}
+#endif
 
 	n_unique = dict_index_get_n_unique(index);
 

innobase/row/row0mysql.c

@@ -265,7 +265,7 @@ row_mysql_store_col_in_innobase_format(
 					necessarily the length of the actual
 					payload data; if the column is a true
 					VARCHAR then this is irrelevant */
-	ibool		comp)		/* in: TRUE = compact format */
+	ulint		comp)		/* in: nonzero=compact format */
 {
 	byte*		ptr 	= mysql_data;
 	dtype_t*	dtype;

innobase/row/row0row.c

@@ -616,7 +616,6 @@ row_search_on_row_ref(
 	ulint		low_match;	
 	rec_t*		rec;
 	dict_index_t*	index;
-	page_t*		page;	
 
 	ut_ad(dtuple_check_typed(ref));
 
@@ -629,9 +628,8 @@ row_search_on_row_ref(
 	low_match = btr_pcur_get_low_match(pcur);
 
 	rec = btr_pcur_get_rec(pcur);
-	page = buf_frame_align(rec);
 
-	if (rec == page_get_infimum_rec(page)) {
+	if (page_rec_is_infimum(rec)) {
 
 		return(FALSE);
 	}
@@ -702,7 +700,6 @@ row_search_index_entry(
 {
 	ulint	n_fields;
 	ulint	low_match;
-	page_t*	page;
 	rec_t*	rec;
 
 	ut_ad(dtuple_check_typed(entry));
@@ -711,11 +708,10 @@ row_search_index_entry(
 	low_match = btr_pcur_get_low_match(pcur);
 
 	rec = btr_pcur_get_rec(pcur);
-	page = buf_frame_align(rec);
 
 	n_fields = dtuple_get_n_fields(entry);
 
-	if (rec == page_get_infimum_rec(page)) {
+	if (page_rec_is_infimum(rec)) {
 
 		return(FALSE);
 	}

innobase/row/row0sel.c

@@ -1261,7 +1261,7 @@ rec_loop:
 	/* PHASE 1: Set a lock if specified */
 
 	if (!node->asc && cursor_just_opened
-		&& (rec != page_get_supremum_rec(buf_frame_align(rec)))) {
+		&& !page_rec_is_supremum(rec)) {
 
 		/* When we open a cursor for a descending search, we must set
 		a next-key lock on the successor record: otherwise it would
@@ -1299,7 +1299,7 @@ rec_loop:
 		}
 	}
 
-	if (rec == page_get_infimum_rec(buf_frame_align(rec))) {
+	if (page_rec_is_infimum(rec)) {
 
 		/* The infimum record on a page cannot be in the result set,
 		and neither can a record lock be placed on it: we skip such
@@ -1337,7 +1337,7 @@ rec_loop:
 		}
 	}
 
-	if (rec == page_get_supremum_rec(buf_frame_align(rec))) {
+	if (page_rec_is_supremum(rec)) {
 
 		/* A page supremum record cannot be in the result set: skip
 		it now when we have placed a possible lock on it */
@@ -2416,14 +2416,12 @@ row_sel_store_mysql_rec(
 	mem_heap_t*		extern_field_heap	= NULL;
 	byte*			data;
 	ulint			len;
-	byte*			blob_buf;
-	int			pad_char;
 	ulint			i;
 	
 	ut_ad(prebuilt->mysql_template);
 	ut_ad(rec_offs_validate(rec, NULL, offsets));
 
-	if (prebuilt->blob_heap != NULL) {
+	if (UNIV_LIKELY_NULL(prebuilt->blob_heap)) {
 		mem_heap_free(prebuilt->blob_heap);
 		prebuilt->blob_heap = NULL;
 	}
@@ -2435,7 +2433,8 @@ row_sel_store_mysql_rec(
 		data = rec_get_nth_field(rec, offsets,
 					templ->rec_field_no, &len);
 
-		if (rec_offs_nth_extern(offsets, templ->rec_field_no)) {
+		if (UNIV_UNLIKELY(rec_offs_nth_extern(offsets,
+						templ->rec_field_no))) {
 
 			/* Copy an externally stored field to the temporary
 			heap */
@@ -2456,7 +2455,7 @@ row_sel_store_mysql_rec(
 		}
 
 		if (len != UNIV_SQL_NULL) {
-			if (templ->type == DATA_BLOB) {
+			if (UNIV_UNLIKELY(templ->type == DATA_BLOB)) {
 
 				ut_a(prebuilt->templ_contains_blob);
 
@@ -2465,8 +2464,9 @@ row_sel_store_mysql_rec(
 				of 1000000 bytes. Since the test takes some
 				CPU time, we do not use it for small BLOBs. */
 
-				if (len > 2000000
-				    && !ut_test_malloc(len + 1000000)) {
+				if (UNIV_UNLIKELY(len > 2000000)
+				    && UNIV_UNLIKELY(!ut_test_malloc(
+							len + 1000000))) {
 
 					ut_print_timestamp(stderr);
 					fprintf(stderr,
@@ -2492,11 +2492,9 @@ row_sel_store_mysql_rec(
 						mem_heap_create(len);
 				}
 
-				blob_buf = mem_heap_alloc(prebuilt->blob_heap,
-									len);
-				ut_memcpy(blob_buf, data, len);
-
-				data = blob_buf;
+				data = memcpy(mem_heap_alloc(
+						prebuilt->blob_heap, len),
+						data, len);
 			}
 		
 			row_sel_field_store_in_mysql_format(
@@ -2521,41 +2519,45 @@ row_sel_store_mysql_rec(
 			account caused seg faults with NULL BLOB fields, and
 		        bug number 154 in the MySQL bug database: GROUP BY
 		        and DISTINCT could treat NULL values inequal. */
+			int	pad_char;
 
 			mysql_rec[templ->mysql_null_byte_offset] |=
 					(byte) (templ->mysql_null_bit_mask);
-			if (templ->type == DATA_VARCHAR
-			    || templ->type == DATA_CHAR
-			    || templ->type == DATA_BINARY
-			    || templ->type == DATA_FIXBINARY
-			    || templ->type == DATA_MYSQL
-			    || templ->type == DATA_VARMYSQL) {
+			switch (templ->type) {
+			case DATA_VARCHAR:
+			case DATA_CHAR:
+			case DATA_BINARY:
+			case DATA_FIXBINARY:
+			case DATA_MYSQL:
+			case DATA_VARMYSQL:
 			        /* MySQL pads all non-BLOB and non-TEXT
 				string types with space ' ' */
-			    
-				pad_char = ' ';
-			} else {
-				pad_char = '\0';
+				if (UNIV_UNLIKELY(templ->mbminlen == 2)) {
+					/* Treat UCS2 as a special case. */
+					data = mysql_rec
+						+ templ->mysql_col_offset;
+					len = templ->mysql_col_len;
+					/* There are two UCS2 bytes per char,
+					so the length has to be even. */
+					ut_a(!(len & 1));
+					/* Pad with 0x0020. */
+					while (len) {
+						*data++ = 0x00;
+						*data++ = 0x20;
+						len -= 2;
+					}
+					continue;
+				}
+				pad_char = 0x20;
+				break;
+			default:
+				pad_char = 0x00;
+				break;
 			}
 
-			/* Handle UCS2 strings differently. */
-			if (pad_char != '\0' && templ->mbminlen == 2) {
-				/* There are two bytes per char, so the length
-				has to be an even number. */
-				ut_a(!(templ->mysql_col_len & 1));
-				data = mysql_rec + templ->mysql_col_offset;
-				len = templ->mysql_col_len;
-				/* Pad with 0x0020. */
-				while (len >= 2) {
-					*data++ = 0x00;
-					*data++ = 0x20;
-					len -= 2;
-				}
-			} else {
-				ut_ad(!pad_char || templ->mbminlen == 1);
-				memset(mysql_rec + templ->mysql_col_offset,
+			ut_ad(!pad_char || templ->mbminlen == 1);
+			memset(mysql_rec + templ->mysql_col_offset,
 					pad_char, templ->mysql_col_len);
-			}
 		}
 	} 
 
@@ -2926,9 +2928,9 @@ row_sel_push_cache_row_for_mysql(
 
 	ut_ad(prebuilt->fetch_cache_first == 0);
 
-	if (!row_sel_store_mysql_rec(
+	if (UNIV_UNLIKELY(!row_sel_store_mysql_rec(
 			prebuilt->fetch_cache[prebuilt->n_fetch_cached],
-			prebuilt, rec, offsets)) {
+			prebuilt, rec, offsets))) {
 		ut_error;
 	}
 
@@ -3049,11 +3051,7 @@ row_search_for_mysql(
 	rec_t*		index_rec;
 	rec_t*		clust_rec;
 	rec_t*		old_vers;
-	ulint		err             = DB_SUCCESS;
-	ibool		moved;
-	ibool		cons_read_requires_clust_rec;
-	ibool		was_lock_wait;
-	ulint		shortcut;
+	ulint		err				= DB_SUCCESS;
 	ibool		unique_search			= FALSE;
 	ibool		unique_search_from_clust_index	= FALSE;
 	ibool		mtr_has_extra_clust_latch 	= FALSE;
@@ -3063,9 +3061,9 @@ row_search_for_mysql(
 					locking SELECT, and the isolation
 					level is <= TRX_ISO_READ_COMMITTED,
 					then this is set to FALSE */
-	ibool		success;
-	ibool		comp;
+#ifdef UNIV_SEARCH_DEBUG
 	ulint		cnt				= 0;
+#endif /* UNIV_SEARCH_DEBUG */
 	ulint		next_offs;
 	mtr_t		mtr;
 	mem_heap_t*	heap				= NULL;
@@ -3076,7 +3074,7 @@ row_search_for_mysql(
 	ut_ad(index && pcur && search_tuple);
 	ut_ad(trx->mysql_thread_id == os_thread_get_curr_id());
 
-	if (prebuilt->table->ibd_file_missing) {
+	if (UNIV_UNLIKELY(prebuilt->table->ibd_file_missing)) {
 	        ut_print_timestamp(stderr);
 	        fprintf(stderr, "  InnoDB: Error:\n"
 "InnoDB: MySQL is trying to use a table handle but the .ibd file for\n"
@@ -3090,7 +3088,7 @@ row_search_for_mysql(
 		return(DB_ERROR);
 	}
 
-	if (prebuilt->magic_n != ROW_PREBUILT_ALLOCATED) {
+	if (UNIV_UNLIKELY(prebuilt->magic_n != ROW_PREBUILT_ALLOCATED)) {
 		fprintf(stderr,
 		"InnoDB: Error: trying to free a corrupt\n"
 		"InnoDB: table handle. Magic n %lu, table name ",
@@ -3104,7 +3102,7 @@ row_search_for_mysql(
 	}
 
 	if (trx->n_mysql_tables_in_use == 0
-            && prebuilt->select_lock_type == LOCK_NONE) {
+	    && UNIV_UNLIKELY(prebuilt->select_lock_type == LOCK_NONE)) {
 		/* Note that if MySQL uses an InnoDB temp table that it
 		created inside LOCK TABLES, then n_mysql_tables_in_use can
 		be zero; in that case select_lock_type is set to LOCK_X in
@@ -3127,8 +3125,8 @@ row_search_for_mysql(
 	/* PHASE 0: Release a possible s-latch we are holding on the
 	adaptive hash index latch if there is someone waiting behind */
 
-	if (trx->has_search_latch
-	    && btr_search_latch.writer != RW_LOCK_NOT_LOCKED) {
+	if (UNIV_UNLIKELY(btr_search_latch.writer != RW_LOCK_NOT_LOCKED)
+	    && trx->has_search_latch) {
 
 		/* There is an x-latch request on the adaptive hash index:
 		release the s-latch to reduce starvation and wait for
@@ -3144,7 +3142,7 @@ row_search_for_mysql(
 	/*-------------------------------------------------------------*/
 	/* PHASE 1: Try to pop the row from the prefetch cache */
 
-	if (direction == 0) {
+	if (UNIV_UNLIKELY(direction == 0)) {
 		trx->op_info = "starting index read";
 	
 		prebuilt->n_rows_fetched = 0;
@@ -3162,8 +3160,8 @@ row_search_for_mysql(
 			prebuilt->fetch_direction = direction;
 		}
 
-		if (direction != prebuilt->fetch_direction) {
-			if (prebuilt->n_fetch_cached > 0) {
+		if (UNIV_UNLIKELY(direction != prebuilt->fetch_direction)) {
+			if (UNIV_UNLIKELY(prebuilt->n_fetch_cached > 0)) {
 				ut_error;
 				/* TODO: scrollable cursor: restore cursor to
 				the place of the latest returned row,
@@ -3175,7 +3173,7 @@ row_search_for_mysql(
 			prebuilt->n_fetch_cached = 0;
 			prebuilt->fetch_cache_first = 0;
 
-		} else if (prebuilt->n_fetch_cached > 0) {
+		} else if (UNIV_LIKELY(prebuilt->n_fetch_cached > 0)) {
 			row_sel_pop_cached_row_for_mysql(buf, prebuilt);
 
 			prebuilt->n_rows_fetched++;
@@ -3235,7 +3233,8 @@ row_search_for_mysql(
 		1 column. Return immediately if this is not a HANDLER
 		command. */
 
-		if (direction != 0 && !prebuilt->used_in_HANDLER) {
+		if (UNIV_UNLIKELY(direction != 0 &&
+				!prebuilt->used_in_HANDLER)) {
         
 			err = DB_RECORD_NOT_FOUND;
 			goto func_exit;
@@ -3253,9 +3252,9 @@ row_search_for_mysql(
 	cannot use the adaptive hash index in a search in the case the row
 	may be long and there may be externally stored fields */
 
-	if (unique_search
+	if (UNIV_UNLIKELY(direction == 0)
+	    && unique_search
 	    && index->type & DICT_CLUSTERED
-	    && direction == 0
 	    && !prebuilt->templ_contains_blob
 	    && !prebuilt->used_in_HANDLER
 	    && (prebuilt->mysql_row_len < UNIV_PAGE_SIZE / 8)) {
@@ -3287,9 +3286,9 @@ row_search_for_mysql(
 				trx->has_search_latch = TRUE;
 			}
 #endif
-			shortcut = row_sel_try_search_shortcut_for_mysql(&rec,
-					prebuilt, &offsets, &heap, &mtr);
-			if (shortcut == SEL_FOUND) {
+			switch (row_sel_try_search_shortcut_for_mysql(&rec,
+					prebuilt, &offsets, &heap, &mtr)) {
+			case SEL_FOUND:
 #ifdef UNIV_SEARCH_DEBUG
 				ut_a(0 == cmp_dtuple_rec(search_tuple,
 							rec, offsets));
@@ -3323,9 +3322,8 @@ row_search_for_mysql(
 				position */
 				err = DB_SUCCESS;
 				goto func_exit;
-			
-			} else if (shortcut == SEL_EXHAUSTED) {
 
+			case SEL_EXHAUSTED:
  				mtr_commit(&mtr);
 
 				/* ut_print_name(stderr, index->name);
@@ -3368,6 +3366,7 @@ shortcut_fails_too_big_rec:
 
 		/* Scan the MySQL query string; check if SELECT is the first
 	        word there */
+		ibool	success;
 
 		dict_accept(*trx->mysql_query_str, "SELECT", &success);
 
@@ -3383,7 +3382,7 @@ shortcut_fails_too_big_rec:
 	naturally moves upward (in fetch next) in alphabetical order,
 	otherwise downward */
 	
-	if (direction == 0) {
+	if (UNIV_UNLIKELY(direction == 0)) {
 		if (mode == PAGE_CUR_GE || mode == PAGE_CUR_G) {
 			moves_up = TRUE;
 		}
@@ -3397,10 +3396,9 @@ shortcut_fails_too_big_rec:
 
 	clust_index = dict_table_get_first_index(index->table);
 
-	if (direction != 0) {		
-		moved = sel_restore_position_for_mysql(BTR_SEARCH_LEAF, pcur,
-							moves_up, &mtr);
-		if (!moved) {
+	if (UNIV_LIKELY(direction != 0)) {
+		if (!sel_restore_position_for_mysql(BTR_SEARCH_LEAF, pcur,
+							moves_up, &mtr)) {
 			goto next_rec;
 		}
 
@@ -3441,11 +3439,13 @@ shortcut_fails_too_big_rec:
 		trx_assign_read_view(trx);
 		prebuilt->sql_stat_start = FALSE;
 	} else {
+		ulint	lock_mode;
 		if (prebuilt->select_lock_type == LOCK_S) {		
-			err = lock_table(0, index->table, LOCK_IS, thr);
+			lock_mode = LOCK_IS;
 		} else {
-			err = lock_table(0, index->table, LOCK_IX, thr);
+			lock_mode = LOCK_IX;
 		}
+		err = lock_table(0, index->table, lock_mode, thr);
 
 		if (err != DB_SUCCESS) {
 
@@ -3459,8 +3459,8 @@ rec_loop:
 	/* PHASE 4: Look for matching records in a loop */
 	
 	rec = btr_pcur_get_rec(pcur);
-	comp = index->table->comp;
-	ut_ad(comp == page_is_comp(buf_frame_align(rec)));
+	ut_ad(!page_rec_is_comp(rec) == !index->table->comp);
+#ifdef UNIV_SEARCH_DEBUG
 /*
 	fputs("Using ", stderr);
 	dict_index_name_print(stderr, index);
@@ -3468,7 +3468,9 @@ rec_loop:
 			buf_frame_get_page_no(buf_frame_align(rec)));
 	rec_print(rec);
 */
-	if (rec == page_get_infimum_rec(buf_frame_align(rec))) {
+#endif /* UNIV_SEARCH_DEBUG */
+
+	if (page_rec_is_infimum(rec)) {
 
 		/* The infimum record on a page cannot be in the result set,
 		and neither can a record lock be placed on it: we skip such
@@ -3477,10 +3479,11 @@ rec_loop:
 		goto next_rec;
 	}
 
-	if (rec == page_get_supremum_rec(buf_frame_align(rec))) {
+	if (page_rec_is_supremum(rec)) {
 
-		if (prebuilt->select_lock_type != LOCK_NONE
-		    && set_also_gap_locks) {
+		if (set_also_gap_locks
+		    && !srv_locks_unsafe_for_binlog
+		    && prebuilt->select_lock_type != LOCK_NONE) {
 
 			/* Try to place a lock on the index record */
 
@@ -3488,18 +3491,16 @@ rec_loop:
 			we do not lock gaps. Supremum record is really
 			a gap and therefore we do not set locks there. */
 			
-			if (!srv_locks_unsafe_for_binlog) {
-				offsets = rec_get_offsets(rec, index, offsets,
-						ULINT_UNDEFINED, &heap);
-				err = sel_set_rec_lock(rec, index, offsets,
-						prebuilt->select_lock_type,
-						LOCK_ORDINARY, thr);
-				if (err != DB_SUCCESS) {
+			offsets = rec_get_offsets(rec, index, offsets,
+					ULINT_UNDEFINED, &heap);
+			err = sel_set_rec_lock(rec, index, offsets,
+					prebuilt->select_lock_type,
+					LOCK_ORDINARY, thr);
 
-					goto lock_wait_or_error;
-				}
-			}
+			if (err != DB_SUCCESS) {
 
+				goto lock_wait_or_error;
+			}
 		}
 		/* A page supremum record cannot be in the result set: skip
 		it now that we have placed a possible lock on it */
@@ -3511,12 +3512,19 @@ rec_loop:
 	/* Do sanity checks in case our cursor has bumped into page
 	corruption */
 	
-	next_offs = rec_get_next_offs(rec, comp);
-
-	if (next_offs >= UNIV_PAGE_SIZE
-		|| next_offs <
-		(ulint) (comp ? PAGE_NEW_SUPREMUM : PAGE_OLD_SUPREMUM)) {
-
+	if (page_rec_is_comp(rec)) {
+		next_offs = rec_get_next_offs(rec, TRUE);
+		if (UNIV_UNLIKELY(next_offs < PAGE_NEW_SUPREMUM)) {
+			goto wrong_offs;
+		}
+	} else {
+		next_offs = rec_get_next_offs(rec, FALSE);
+		if (UNIV_UNLIKELY(next_offs < PAGE_OLD_SUPREMUM)) {
+			goto wrong_offs;
+		}
+	}
+	if (UNIV_UNLIKELY(next_offs >= UNIV_PAGE_SIZE - PAGE_DIR)) {
+	wrong_offs:
 		if (srv_force_recovery == 0 || moves_up == FALSE) {
 			ut_print_timestamp(stderr);
 			buf_page_print(buf_frame_align(rec));
@@ -3529,7 +3537,7 @@ rec_loop:
 			fprintf(stderr,
 "InnoDB: Index corruption: rec offs %lu next offs %lu, page no %lu,\n"
 "InnoDB: ",
-				(ulong) (rec - buf_frame_align(rec)),
+				(ulong) ut_align_offset(rec, UNIV_PAGE_SIZE),
 				(ulong) next_offs,
 				(ulong) buf_frame_get_page_no(rec));
 			dict_index_name_print(stderr, trx, index);
@@ -3547,7 +3555,7 @@ rec_loop:
 			fprintf(stderr,
 "InnoDB: Index corruption: rec offs %lu next offs %lu, page no %lu,\n"
 "InnoDB: ",
-			   (ulong) (rec - buf_frame_align(rec)),
+			   (ulong) ut_align_offset(rec, UNIV_PAGE_SIZE),
 			   (ulong) next_offs,
 			   (ulong) buf_frame_get_page_no(rec));
 			dict_index_name_print(stderr, trx, index);
@@ -3562,13 +3570,13 @@ rec_loop:
 
 	offsets = rec_get_offsets(rec, index, offsets, ULINT_UNDEFINED, &heap);
 
-	if (srv_force_recovery > 0) {
+	if (UNIV_UNLIKELY(srv_force_recovery > 0)) {
 		if (!rec_validate(rec, offsets)
 		|| !btr_index_rec_validate(rec, index, FALSE)) {
 			fprintf(stderr,
 "InnoDB: Index corruption: rec offs %lu next offs %lu, page no %lu,\n"
 "InnoDB: ",
-			   (ulong) (rec - buf_frame_align(rec)),
+			   (ulong) ut_align_offset(rec, UNIV_PAGE_SIZE),
 			   (ulong) next_offs,
 			   (ulong) buf_frame_get_page_no(rec));
 			dict_index_name_print(stderr, trx, index);
@@ -3594,25 +3602,22 @@ rec_loop:
 		
 		if (0 != cmp_dtuple_rec(search_tuple, rec, offsets)) {
 
-			if (prebuilt->select_lock_type != LOCK_NONE
-		    	    && set_also_gap_locks) {
+			if (set_also_gap_locks
+			    && !srv_locks_unsafe_for_binlog
+			    && prebuilt->select_lock_type != LOCK_NONE) {
 
 				/* Try to place a gap lock on the index 
 				record only if innodb_locks_unsafe_for_binlog
 				option is not set */
 
-				if (srv_locks_unsafe_for_binlog == FALSE) { 
-
-					err = sel_set_rec_lock(rec, index,
-						offsets,
+				err = sel_set_rec_lock(rec, index, offsets,
 						prebuilt->select_lock_type,
 						LOCK_GAP, thr);
-					if (err != DB_SUCCESS) {
 
-						goto lock_wait_or_error;
-					}
-				}
+				if (err != DB_SUCCESS) {
 
+					goto lock_wait_or_error;
+				}
 			}
 
 			btr_pcur_store_position(pcur, &mtr);
@@ -3628,25 +3633,22 @@ rec_loop:
 
 		if (!cmp_dtuple_is_prefix_of_rec(search_tuple, rec, offsets)) {
 			
-			if (prebuilt->select_lock_type != LOCK_NONE
-			    && set_also_gap_locks) {
+			if (set_also_gap_locks
+			    && !srv_locks_unsafe_for_binlog
+			    && prebuilt->select_lock_type != LOCK_NONE) {
 
 				/* Try to place a gap lock on the index 
 				record only if innodb_locks_unsafe_for_binlog
 				option is not set */
 
-				if (srv_locks_unsafe_for_binlog == FALSE) {
-
-					err = sel_set_rec_lock(rec, index,
-						offsets,
+				err = sel_set_rec_lock(rec, index, offsets,
 						prebuilt->select_lock_type,
 						LOCK_GAP, thr);
-					if (err != DB_SUCCESS) {
 
-						goto lock_wait_or_error;
-					}
-				}
+				if (err != DB_SUCCESS) {
 
+					goto lock_wait_or_error;
+				}
 			}
 
 			btr_pcur_store_position(pcur, &mtr);
@@ -3662,29 +3664,25 @@ rec_loop:
 	/* We are ready to look at a possible new index entry in the result
 	set: the cursor is now placed on a user record */
 
-	cons_read_requires_clust_rec = FALSE;
-
 	if (prebuilt->select_lock_type != LOCK_NONE) {
 		/* Try to place a lock on the index record; note that delete
 		marked records are a special case in a unique search. If there
 		is a non-delete marked record, then it is enough to lock its
 		existence with LOCK_REC_NOT_GAP. */
 
+		/* If innodb_locks_unsafe_for_binlog option is used,
+		we lock only the record, i.e., next-key locking is
+		not used. */
+
 		ulint	lock_type;
 
 		if (!set_also_gap_locks
-		    || (unique_search && !rec_get_deleted_flag(rec, comp))) {
-			lock_type = LOCK_REC_NOT_GAP;
+		    || srv_locks_unsafe_for_binlog
+		    || (unique_search && !UNIV_UNLIKELY(rec_get_deleted_flag(
+					rec, page_rec_is_comp(rec))))) {
+			goto no_gap_lock;
 		} else {
-			/* If innodb_locks_unsafe_for_binlog option is used, 
-			we lock only the record, i.e., next-key locking is
-			not used. */
-
-	                if (srv_locks_unsafe_for_binlog) {
-				lock_type = LOCK_REC_NOT_GAP;
-			} else {
-				lock_type = LOCK_ORDINARY;
- 			}
+			lock_type = LOCK_ORDINARY;
 		}
 
 		/* If we are doing a 'greater or equal than a primary key
@@ -3704,7 +3702,7 @@ rec_loop:
 		    && dtuple_get_n_fields_cmp(search_tuple)
 		       == dict_index_get_n_unique(index)
 		    && 0 == cmp_dtuple_rec(search_tuple, rec, offsets)) {
-
+		no_gap_lock:
 			lock_type = LOCK_REC_NOT_GAP;
 		}
 
@@ -3732,7 +3730,7 @@ rec_loop:
 			high force recovery level set, we try to avoid crashes
 			by skipping this lookup */
 
-			if (srv_force_recovery < 5
+			if (UNIV_LIKELY(srv_force_recovery < 5)
                             && !lock_clust_rec_cons_read_sees(rec, index,
 						offsets, trx->read_view)) {
 
@@ -3763,13 +3761,15 @@ rec_loop:
 			have to look also into the clustered index: this
 			is necessary, because we can only get the undo
 			information via the clustered index record. */
-			
-			cons_read_requires_clust_rec = TRUE;
+
+			/* Get the clustered index record if needed */
+			index_rec = rec;
+			ut_ad(index != clust_index);
+			goto requires_clust_rec;
 		}
 	}
 
-	if (rec_get_deleted_flag(rec, comp)
-			&& !cons_read_requires_clust_rec) {
+	if (UNIV_UNLIKELY(rec_get_deleted_flag(rec, page_rec_is_comp(rec)))) {
 
 		/* The record is delete-marked: we can skip it if this is
 		not a consistent read which might see an earlier version
@@ -3783,14 +3783,14 @@ rec_loop:
 
 	index_rec = rec;
 
-	/* Before and after the following "if" block, "offsets" will be
-	related to "rec", which may be in "index", a secondary index or
-	the clustered index ("clust_index").  However, after this "if" block,
-	"rec" may be pointing to "clust_rec" of "clust_index". */
-	ut_ad(rec_offs_validate(rec, index, offsets));
-
-	if (index != clust_index && (cons_read_requires_clust_rec
-				|| prebuilt->need_to_access_clustered)) {
+	if (index != clust_index && prebuilt->need_to_access_clustered) {
+	requires_clust_rec:
+		/* Before and after this "if" block, "offsets" will be
+		related to "rec", which may be in a secondary index "index" or
+		the clustered index ("clust_index").  However, after this
+		"if" block, "rec" may be pointing to
+		"clust_rec" of "clust_index". */
+		ut_ad(rec_offs_validate(rec, index, offsets));
 
 		/* It was a non-clustered index and we must fetch also the
 		clustered index record */
@@ -3812,7 +3812,8 @@ rec_loop:
 			goto next_rec;
 		}
 
-		if (rec_get_deleted_flag(clust_rec, comp)) {
+		if (UNIV_UNLIKELY(rec_get_deleted_flag(clust_rec,
+					page_rec_is_comp(clust_rec)))) {
 
 			/* The record is delete marked: we can skip it */
 
@@ -3833,7 +3834,8 @@ rec_loop:
 				rec == clust_rec ? clust_index : index,
 				offsets));
 
-	if (prebuilt->n_rows_fetched >= MYSQL_FETCH_CACHE_THRESHOLD
+	if ((match_mode == ROW_SEL_EXACT
+		|| prebuilt->n_rows_fetched >= MYSQL_FETCH_CACHE_THRESHOLD)
 			&& prebuilt->select_lock_type == LOCK_NONE
 			&& !prebuilt->templ_contains_blob
 			&& !prebuilt->clust_index_was_generated
@@ -3908,7 +3910,7 @@ next_rec:
 	/*-------------------------------------------------------------*/
 	/* PHASE 5: Move the cursor to the next index record */
 	
-	if (mtr_has_extra_clust_latch) {
+	if (UNIV_UNLIKELY(mtr_has_extra_clust_latch)) {
 		/* We must commit mtr if we are moving to the next
 		non-clustered index record, because we could break the
 		latching order if we would access a different clustered
@@ -3920,34 +3922,38 @@ next_rec:
 		mtr_has_extra_clust_latch = FALSE;
 	
 		mtr_start(&mtr);
-		moved = sel_restore_position_for_mysql(BTR_SEARCH_LEAF, pcur,
-							moves_up, &mtr);
-		if (moved) {
+		if (sel_restore_position_for_mysql(BTR_SEARCH_LEAF, pcur,
+							moves_up, &mtr)) {
+#ifdef UNIV_SEARCH_DEBUG
 			cnt++;
+#endif /* UNIV_SEARCH_DEBUG */
 
 			goto rec_loop;
 		}
 	}
 
 	if (moves_up) {		
-		moved = btr_pcur_move_to_next(pcur, &mtr);
-	} else {
-		moved = btr_pcur_move_to_prev(pcur, &mtr);
-	}
+		if (UNIV_UNLIKELY(!btr_pcur_move_to_next(pcur, &mtr))) {
+		not_moved:
+			btr_pcur_store_position(pcur, &mtr);
 
-	if (!moved) {
-		btr_pcur_store_position(pcur, &mtr);
+			if (match_mode != 0) {
+				err = DB_RECORD_NOT_FOUND;
+			} else {
+				err = DB_END_OF_INDEX;
+			}
 
-		if (match_mode != 0) {
-			err = DB_RECORD_NOT_FOUND;
-		} else {
-			err = DB_END_OF_INDEX;
+			goto normal_return;
+		}
+	} else {
+		if (UNIV_UNLIKELY(!btr_pcur_move_to_prev(pcur, &mtr))) {
+			goto not_moved;
 		}
-
-		goto normal_return;
 	}
 
+#ifdef UNIV_SEARCH_DEBUG
 	cnt++;
+#endif /* UNIV_SEARCH_DEBUG */
 
 	goto rec_loop;
 
@@ -3965,11 +3971,10 @@ lock_wait_or_error:
 
 	que_thr_stop_for_mysql(thr);
 
-  thr->lock_state= QUE_THR_LOCK_ROW;
-	was_lock_wait = row_mysql_handle_errors(&err, trx, thr, NULL);
-  thr->lock_state= QUE_THR_LOCK_NOLOCK;
+	thr->lock_state = QUE_THR_LOCK_ROW;
 
-	if (was_lock_wait) {
+	if (row_mysql_handle_errors(&err, trx, thr, NULL)) {
+		thr->lock_state = QUE_THR_LOCK_NOLOCK;
 		mtr_start(&mtr);
 
 		sel_restore_position_for_mysql(BTR_SEARCH_LEAF, pcur,
@@ -3979,9 +3984,13 @@ lock_wait_or_error:
 		goto rec_loop;
 	}
 
+	thr->lock_state = QUE_THR_LOCK_NOLOCK;
+
+#ifdef UNIV_SEARCH_DEBUG
 /*	fputs("Using ", stderr);
 	dict_index_name_print(stderr, index);
 	fprintf(stderr, " cnt %lu ret value %lu err\n", cnt, err); */
+#endif /* UNIV_SEARCH_DEBUG */
 	goto func_exit;
 
 normal_return:
@@ -3996,9 +4005,11 @@ normal_return:
 		err = DB_SUCCESS;
 	}
 
+#ifdef UNIV_SEARCH_DEBUG
 /*	fputs("Using ", stderr);
 	dict_index_name_print(stderr, index);
 	fprintf(stderr, " cnt %lu ret value %lu err\n", cnt, err); */
+#endif /* UNIV_SEARCH_DEBUG */
 	if (err == DB_SUCCESS) {
 		srv_n_rows_read++;
 	}

innobase/row/row0vers.c

@@ -61,7 +61,7 @@ row_vers_impl_x_locked_off_kernel(
 	ibool		rec_del;
 	ulint		err;
 	mtr_t		mtr;
-	ibool		comp;
+	ulint		comp;
 	
 #ifdef UNIV_SYNC_DEBUG
 	ut_ad(mutex_own(&kernel_mutex));
@@ -121,10 +121,10 @@ row_vers_impl_x_locked_off_kernel(
 		goto exit_func;
 	}
 
-	comp = index->table->comp;
+	comp = page_rec_is_comp(rec);
 	ut_ad(index->table == clust_index->table);
-	ut_ad(comp == page_is_comp(buf_frame_align(rec)));
-	ut_ad(comp == page_is_comp(buf_frame_align(clust_rec)));
+	ut_ad(!!comp == index->table->comp);
+	ut_ad(!comp == !page_rec_is_comp(clust_rec));
 
 	/* We look up if some earlier version, which was modified by the trx_id
 	transaction, of the clustered index record would require rec to be in
@@ -310,7 +310,7 @@ row_vers_old_has_index_entry(
 	dtuple_t*	row;
 	dtuple_t*	entry;
 	ulint		err;
-	ibool		comp;
+	ulint		comp;
 
 	ut_ad(mtr_memo_contains(mtr, buf_block_align(rec), MTR_MEMO_PAGE_X_FIX)
 	   	|| mtr_memo_contains(mtr, buf_block_align(rec),
@@ -322,8 +322,8 @@ row_vers_old_has_index_entry(
 
 	clust_index = dict_table_get_first_index(index->table);
 
-	comp = index->table->comp;
-	ut_ad(comp == page_is_comp(buf_frame_align(rec)));
+	comp = page_rec_is_comp(rec);
+	ut_ad(!index->table->comp == !comp);
 	heap = mem_heap_create(1024);
 	clust_offsets = rec_get_offsets(rec, clust_index, NULL,
 					ULINT_UNDEFINED, &heap);