fix the EDOM errors by makeing the key compare function implement a total order on the key space (including key prefixes), and using DB_SET_RANGE in place of DB_SET for cursor operations.  closes #576 closes #587



git-svn-id: file:///svn/mysql/tokudb-engine/src@3211 c7de825b-a66e-492c-adef-691d508d4ae1