Commit bbdc2821 authored by Olaf Kirch's avatar Olaf Kirch Committed by Roland Dreier

mlx4_core: Avoid recycling old FMR R_Keys too soon

When a FMR is unmapped, mlx4 resets the map count to 0, and clears the
upper part of the R_Key which is used as the sequence counter.

This poses a problem for RDS, which uses ib_fmr_unmap as a fence
operation.  RDS assumes that after issuing an unmap, the old R_Keys
will be invalid for a "reasonable" period of time. For instance,
Oracle processes uses shared memory buffers allocated from a pool of
buffers.  When a process dies, we want to reclaim these buffers -- but
we must make sure there are no pending RDMA operations to/from those
buffers.  The only way to achieve that is by using unmap and sync the
TPT.

However, when the sequence count is reset on unmap, there is a high
likelihood that a new mapping will be given the same R_Key that was
issued a few milliseconds ago.

To prevent this, don't reset the sequence count when unmapping a FMR.
Signed-off-by: default avatarOlaf Kirch <olaf.kirch@oracle.com>
Signed-off-by: default avatarRoland Dreier <rolandd@cisco.com>
parent d227fa72
...@@ -607,15 +607,9 @@ EXPORT_SYMBOL_GPL(mlx4_fmr_enable); ...@@ -607,15 +607,9 @@ EXPORT_SYMBOL_GPL(mlx4_fmr_enable);
void mlx4_fmr_unmap(struct mlx4_dev *dev, struct mlx4_fmr *fmr, void mlx4_fmr_unmap(struct mlx4_dev *dev, struct mlx4_fmr *fmr,
u32 *lkey, u32 *rkey) u32 *lkey, u32 *rkey)
{ {
u32 key;
if (!fmr->maps) if (!fmr->maps)
return; return;
key = key_to_hw_index(fmr->mr.key);
key &= dev->caps.num_mpts - 1;
*lkey = *rkey = fmr->mr.key = hw_index_to_key(key);
fmr->maps = 0; fmr->maps = 0;
*(u8 *) fmr->mpt = MLX4_MPT_STATUS_SW; *(u8 *) fmr->mpt = MLX4_MPT_STATUS_SW;
......
Markdown is supported
0%
or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment