• Marc Zyngier's avatar
    arm64: KVM: Fix stage-2 PGD allocation to have per-page refcounting · a987370f
    Marc Zyngier authored
    We're using __get_free_pages with to allocate the guest's stage-2
    PGD. The standard behaviour of this function is to return a set of
    pages where only the head page has a valid refcount.
    
    This behaviour gets us into trouble when we're trying to increment
    the refcount on a non-head page:
    
    page:ffff7c00cfb693c0 count:0 mapcount:0 mapping:          (null) index:0x0
    flags: 0x4000000000000000()
    page dumped because: VM_BUG_ON_PAGE((*({ __attribute__((unused)) typeof((&page->_count)->counter) __var = ( typeof((&page->_count)->counter)) 0; (volatile typeof((&page->_count)->counter) *)&((&page->_count)->counter); })) <= 0)
    BUG: failure at include/linux/mm.h:548/get_page()!
    Kernel panic - not syncing: BUG!
    CPU: 1 PID: 1695 Comm: kvm-vcpu-0 Not tainted 4.0.0-rc1+ #3825
    Hardware name: APM X-Gene Mustang board (DT)
    Call trace:
    [<ffff80000008a09c>] dump_backtrace+0x0/0x13c
    [<ffff80000008a1e8>] show_stack+0x10/0x1c
    [<ffff800000691da8>] dump_stack+0x74/0x94
    [<ffff800000690d78>] panic+0x100/0x240
    [<ffff8000000a0bc4>] stage2_get_pmd+0x17c/0x2bc
    [<ffff8000000a1dc4>] kvm_handle_guest_abort+0x4b4/0x6b0
    [<ffff8000000a420c>] handle_exit+0x58/0x180
    [<ffff80000009e7a4>] kvm_arch_vcpu_ioctl_run+0x114/0x45c
    [<ffff800000099df4>] kvm_vcpu_ioctl+0x2e0/0x754
    [<ffff8000001c0a18>] do_vfs_ioctl+0x424/0x5c8
    [<ffff8000001c0bfc>] SyS_ioctl+0x40/0x78
    CPU0: stopping
    
    A possible approach for this is to split the compound page using
    split_page() at allocation time, and change the teardown path to
    free one page at a time.  It turns out that alloc_pages_exact() and
    free_pages_exact() does exactly that.
    
    While we're at it, the PGD allocation code is reworked to reduce
    duplication.
    
    This has been tested on an X-Gene platform with a 4kB/48bit-VA host
    kernel, and kvmtool hacked to place memory in the second page of
    the hardware PGD (PUD for the host kernel). Also regression-tested
    on a Cubietruck (Cortex-A7).
    
     [ Reworked to use alloc_pages_exact() and free_pages_exact() and to
       return pointers directly instead of by reference as arguments
        - Christoffer ]
    Reported-by: default avatarMark Rutland <mark.rutland@arm.com>
    Signed-off-by: default avatarMarc Zyngier <marc.zyngier@arm.com>
    Signed-off-by: default avatarChristoffer Dall <christoffer.dall@linaro.org>
    a987370f
mmu.c 48.1 KB