• Chao Yu's avatar
    f2fs: quota: fix potential deadlock · 9de71ede
    Chao Yu authored
    xfstest generic/587 reports a deadlock issue as below:
    
    ======================================================
    WARNING: possible circular locking dependency detected
    5.14.0-rc1 #69 Not tainted
    ------------------------------------------------------
    repquota/8606 is trying to acquire lock:
    ffff888022ac9320 (&sb->s_type->i_mutex_key#18){+.+.}-{3:3}, at: f2fs_quota_sync+0x207/0x300 [f2fs]
    
    but task is already holding lock:
    ffff8880084bcde8 (&sbi->quota_sem){.+.+}-{3:3}, at: f2fs_quota_sync+0x59/0x300 [f2fs]
    
    which lock already depends on the new lock.
    
    the existing dependency chain (in reverse order) is:
    
    -> #2 (&sbi->quota_sem){.+.+}-{3:3}:
           __lock_acquire+0x648/0x10b0
           lock_acquire+0x128/0x470
           down_read+0x3b/0x2a0
           f2fs_quota_sync+0x59/0x300 [f2fs]
           f2fs_quota_on+0x48/0x100 [f2fs]
           do_quotactl+0x5e3/0xb30
           __x64_sys_quotactl+0x23a/0x4e0
           do_syscall_64+0x3b/0x90
           entry_SYSCALL_64_after_hwframe+0x44/0xae
    
    -> #1 (&sbi->cp_rwsem){++++}-{3:3}:
           __lock_acquire+0x648/0x10b0
           lock_acquire+0x128/0x470
           down_read+0x3b/0x2a0
           f2fs_unlink+0x353/0x670 [f2fs]
           vfs_unlink+0x1c7/0x380
           do_unlinkat+0x413/0x4b0
           __x64_sys_unlinkat+0x50/0xb0
           do_syscall_64+0x3b/0x90
           entry_SYSCALL_64_after_hwframe+0x44/0xae
    
    -> #0 (&sb->s_type->i_mutex_key#18){+.+.}-{3:3}:
           check_prev_add+0xdc/0xb30
           validate_chain+0xa67/0xb20
           __lock_acquire+0x648/0x10b0
           lock_acquire+0x128/0x470
           down_write+0x39/0xc0
           f2fs_quota_sync+0x207/0x300 [f2fs]
           do_quotactl+0xaff/0xb30
           __x64_sys_quotactl+0x23a/0x4e0
           do_syscall_64+0x3b/0x90
           entry_SYSCALL_64_after_hwframe+0x44/0xae
    
    other info that might help us debug this:
    
    Chain exists of:
      &sb->s_type->i_mutex_key#18 --> &sbi->cp_rwsem --> &sbi->quota_sem
    
     Possible unsafe locking scenario:
    
           CPU0                    CPU1
           ----                    ----
      lock(&sbi->quota_sem);
                                   lock(&sbi->cp_rwsem);
                                   lock(&sbi->quota_sem);
      lock(&sb->s_type->i_mutex_key#18);
    
     *** DEADLOCK ***
    
    3 locks held by repquota/8606:
     #0: ffff88801efac0e0 (&type->s_umount_key#53){++++}-{3:3}, at: user_get_super+0xd9/0x190
     #1: ffff8880084bc380 (&sbi->cp_rwsem){++++}-{3:3}, at: f2fs_quota_sync+0x3e/0x300 [f2fs]
     #2: ffff8880084bcde8 (&sbi->quota_sem){.+.+}-{3:3}, at: f2fs_quota_sync+0x59/0x300 [f2fs]
    
    stack backtrace:
    CPU: 6 PID: 8606 Comm: repquota Not tainted 5.14.0-rc1 #69
    Hardware name: innotek GmbH VirtualBox/VirtualBox, BIOS VirtualBox 12/01/2006
    Call Trace:
     dump_stack_lvl+0xce/0x134
     dump_stack+0x17/0x20
     print_circular_bug.isra.0.cold+0x239/0x253
     check_noncircular+0x1be/0x1f0
     check_prev_add+0xdc/0xb30
     validate_chain+0xa67/0xb20
     __lock_acquire+0x648/0x10b0
     lock_acquire+0x128/0x470
     down_write+0x39/0xc0
     f2fs_quota_sync+0x207/0x300 [f2fs]
     do_quotactl+0xaff/0xb30
     __x64_sys_quotactl+0x23a/0x4e0
     do_syscall_64+0x3b/0x90
     entry_SYSCALL_64_after_hwframe+0x44/0xae
    RIP: 0033:0x7f883b0b4efe
    
    The root cause is ABBA deadlock of inode lock and cp_rwsem,
    reorder locks in f2fs_quota_sync() as below to fix this issue:
    - lock inode
    - lock cp_rwsem
    - lock quota_sem
    
    Fixes: db6ec53b ("f2fs: add a rw_sem to cover quota flag changes")
    Signed-off-by: default avatarChao Yu <chao@kernel.org>
    Signed-off-by: default avatarJaegeuk Kim <jaegeuk@kernel.org>
    9de71ede
super.c 116 KB