Commit 8ac75b96 authored by Ian Munsie's avatar Ian Munsie Committed by Michael Ellerman

cxl: Use call_rcu to reduce latency when releasing the afu fd

The afu fd release path was identified as a significant bottleneck in
the overall performance of cxl. While an optimal AFU design would
minimise the need to close & reopen the AFU fd, it is not always
practical to avoid.

The bottleneck seems to be down to the call to synchronize_rcu(), which
will block until every other thread is guaranteed to be out of an RCU
critical section. Replace it with call_rcu() to free the context
structures later so we can return to the application sooner.

This reduces the time spent in the fd release path from 13356 usec to
13.3 usec - about a 100x speed up.
Reported-by: default avatarFei K Chen <uchen@cn.ibm.com>
Signed-off-by: default avatarIan Munsie <imunsie@au1.ibm.com>
Signed-off-by: default avatarMichael Ellerman <mpe@ellerman.id.au>
parent e36f6fe1
...@@ -232,12 +232,9 @@ void cxl_context_detach_all(struct cxl_afu *afu) ...@@ -232,12 +232,9 @@ void cxl_context_detach_all(struct cxl_afu *afu)
mutex_unlock(&afu->contexts_lock); mutex_unlock(&afu->contexts_lock);
} }
void cxl_context_free(struct cxl_context *ctx) static void reclaim_ctx(struct rcu_head *rcu)
{ {
mutex_lock(&ctx->afu->contexts_lock); struct cxl_context *ctx = container_of(rcu, struct cxl_context, rcu);
idr_remove(&ctx->afu->contexts_idr, ctx->pe);
mutex_unlock(&ctx->afu->contexts_lock);
synchronize_rcu();
free_page((u64)ctx->sstp); free_page((u64)ctx->sstp);
ctx->sstp = NULL; ctx->sstp = NULL;
...@@ -245,3 +242,11 @@ void cxl_context_free(struct cxl_context *ctx) ...@@ -245,3 +242,11 @@ void cxl_context_free(struct cxl_context *ctx)
put_pid(ctx->pid); put_pid(ctx->pid);
kfree(ctx); kfree(ctx);
} }
void cxl_context_free(struct cxl_context *ctx)
{
mutex_lock(&ctx->afu->contexts_lock);
idr_remove(&ctx->afu->contexts_idr, ctx->pe);
mutex_unlock(&ctx->afu->contexts_lock);
call_rcu(&ctx->rcu, reclaim_ctx);
}
...@@ -457,6 +457,8 @@ struct cxl_context { ...@@ -457,6 +457,8 @@ struct cxl_context {
bool pending_irq; bool pending_irq;
bool pending_fault; bool pending_fault;
bool pending_afu_err; bool pending_afu_err;
struct rcu_head rcu;
}; };
struct cxl { struct cxl {
......
Markdown is supported
0%
or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment