Commit fc5f9d5f authored by Baoquan He's avatar Baoquan He Committed by Ingo Molnar

x86/mm: Fix boot crash caused by incorrect loop count calculation in sync_global_pgds()

Jeff Moyer reported that on his system with two memory regions 0~64G and
1T~1T+192G, and kernel option "memmap=192G!1024G" added, enabling KASLR
will make the system hang intermittently during boot. While adding 'nokaslr'
won't.

The back trace is:

 Oops: 0000 [#1] SMP

 RIP: memcpy_erms()
 [ .... ]
 Call Trace:
  pmem_rw_page()
  bdev_read_page()
  do_mpage_readpage()
  mpage_readpages()
  blkdev_readpages()
  __do_page_cache_readahead()
  force_page_cache_readahead()
  page_cache_sync_readahead()
  generic_file_read_iter()
  blkdev_read_iter()
  __vfs_read()
  vfs_read()
  SyS_read()
  entry_SYSCALL_64_fastpath()

This crash happens because the for loop count calculation in sync_global_pgds()
is not correct. When a mapping area crosses PGD entries, we should
calculate the starting address of region which next PGD covers and assign
it to next for loop count, but not add PGDIR_SIZE directly. The old
code works right only if the mapping area is an exact multiple of PGDIR_SIZE,
otherwize the end region could be skipped so that it can't be synchronized
to all other processes from kernel PGD init_mm.pgd.

In Jeff's system, emulated pmem area [1024G, 1216G) is smaller than
PGDIR_SIZE. While 'nokaslr' works because PAGE_OFFSET is 1T aligned, it
makes this area be mapped inside one PGD entry. With KASLR enabled,
this area could cross two PGD entries, then the next PGD entry won't
be synced to all other processes. That is why we saw empty PGD.

Fix it.
Reported-by: default avatarJeff Moyer <jmoyer@redhat.com>
Signed-off-by: default avatarBaoquan He <bhe@redhat.com>
Cc: Andrew Morton <akpm@linux-foundation.org>
Cc: Andy Lutomirski <luto@kernel.org>
Cc: Borislav Petkov <bp@alien8.de>
Cc: Brian Gerst <brgerst@gmail.com>
Cc: Dan Williams <dan.j.williams@intel.com>
Cc: Dave Hansen <dave.hansen@linux.intel.com>
Cc: Dave Young <dyoung@redhat.com>
Cc: Denys Vlasenko <dvlasenk@redhat.com>
Cc: H. Peter Anvin <hpa@zytor.com>
Cc: Jinbum Park <jinb.park7@gmail.com>
Cc: Josh Poimboeuf <jpoimboe@redhat.com>
Cc: Kees Cook <keescook@chromium.org>
Cc: Kirill A. Shutemov <kirill.shutemov@linux.intel.com>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Thomas Garnier <thgarnie@google.com>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: Yasuaki Ishimatsu <yasu.isimatu@gmail.com>
Cc: Yinghai Lu <yinghai@kernel.org>
Link: http://lkml.kernel.org/r/1493864747-8506-1-git-send-email-bhe@redhat.comSigned-off-by: default avatarIngo Molnar <mingo@kernel.org>
parent 415812f2
...@@ -94,10 +94,10 @@ __setup("noexec32=", nonx32_setup); ...@@ -94,10 +94,10 @@ __setup("noexec32=", nonx32_setup);
*/ */
void sync_global_pgds(unsigned long start, unsigned long end) void sync_global_pgds(unsigned long start, unsigned long end)
{ {
unsigned long address; unsigned long addr;
for (address = start; address <= end; address += PGDIR_SIZE) { for (addr = start; addr <= end; addr = ALIGN(addr + 1, PGDIR_SIZE)) {
pgd_t *pgd_ref = pgd_offset_k(address); pgd_t *pgd_ref = pgd_offset_k(addr);
const p4d_t *p4d_ref; const p4d_t *p4d_ref;
struct page *page; struct page *page;
...@@ -106,7 +106,7 @@ void sync_global_pgds(unsigned long start, unsigned long end) ...@@ -106,7 +106,7 @@ void sync_global_pgds(unsigned long start, unsigned long end)
* handle synchonization on p4d level. * handle synchonization on p4d level.
*/ */
BUILD_BUG_ON(pgd_none(*pgd_ref)); BUILD_BUG_ON(pgd_none(*pgd_ref));
p4d_ref = p4d_offset(pgd_ref, address); p4d_ref = p4d_offset(pgd_ref, addr);
if (p4d_none(*p4d_ref)) if (p4d_none(*p4d_ref))
continue; continue;
...@@ -117,8 +117,8 @@ void sync_global_pgds(unsigned long start, unsigned long end) ...@@ -117,8 +117,8 @@ void sync_global_pgds(unsigned long start, unsigned long end)
p4d_t *p4d; p4d_t *p4d;
spinlock_t *pgt_lock; spinlock_t *pgt_lock;
pgd = (pgd_t *)page_address(page) + pgd_index(address); pgd = (pgd_t *)page_address(page) + pgd_index(addr);
p4d = p4d_offset(pgd, address); p4d = p4d_offset(pgd, addr);
/* the pgt_lock only for Xen */ /* the pgt_lock only for Xen */
pgt_lock = &pgd_page_get_mm(page)->page_table_lock; pgt_lock = &pgd_page_get_mm(page)->page_table_lock;
spin_lock(pgt_lock); spin_lock(pgt_lock);
......
Markdown is supported
0%
or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment