Commit 81a70637 authored by Andrew Morton's avatar Andrew Morton Committed by Linus Torvalds

[PATCH] Speed up direct-io hugetlbpage handling

This patch short-circuits all the direct-io page dirtying logic for
higher-order pages.  Without this, we pointlessly bounce BIOs up to keventd
all the time.
parent 69d85819
...@@ -532,6 +532,12 @@ void bio_unmap_user(struct bio *bio, int write_to_vm) ...@@ -532,6 +532,12 @@ void bio_unmap_user(struct bio *bio, int write_to_vm)
* check that the pages are still dirty. If so, fine. If not, redirty them * check that the pages are still dirty. If so, fine. If not, redirty them
* in process context. * in process context.
* *
* We special-case compound pages here: normally this means reads into hugetlb
* pages. The logic in here doesn't really work right for compound pages
* because the VM does not uniformly chase down the head page in all cases.
* But dirtiness of compound pages is pretty meaningless anyway: the VM doesn't
* handle them at all. So we skip compound pages here at an early stage.
*
* Note that this code is very hard to test under normal circumstances because * Note that this code is very hard to test under normal circumstances because
* direct-io pins the pages with get_user_pages(). This makes * direct-io pins the pages with get_user_pages(). This makes
* is_page_cache_freeable return false, and the VM will not clean the pages. * is_page_cache_freeable return false, and the VM will not clean the pages.
...@@ -553,8 +559,8 @@ void bio_set_pages_dirty(struct bio *bio) ...@@ -553,8 +559,8 @@ void bio_set_pages_dirty(struct bio *bio)
for (i = 0; i < bio->bi_vcnt; i++) { for (i = 0; i < bio->bi_vcnt; i++) {
struct page *page = bvec[i].bv_page; struct page *page = bvec[i].bv_page;
if (page) if (page && !PageCompound(page))
set_page_dirty_lock(bvec[i].bv_page); set_page_dirty_lock(page);
} }
} }
...@@ -620,7 +626,7 @@ void bio_check_pages_dirty(struct bio *bio) ...@@ -620,7 +626,7 @@ void bio_check_pages_dirty(struct bio *bio)
for (i = 0; i < bio->bi_vcnt; i++) { for (i = 0; i < bio->bi_vcnt; i++) {
struct page *page = bvec[i].bv_page; struct page *page = bvec[i].bv_page;
if (PageDirty(page)) { if (PageDirty(page) || PageCompound(page)) {
page_cache_release(page); page_cache_release(page);
bvec[i].bv_page = NULL; bvec[i].bv_page = NULL;
} else { } else {
......
Markdown is supported
0%
or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment