1. 16 Nov, 2021 3 commits
    • Filipe Manana's avatar
      btrfs: silence lockdep when reading chunk tree during mount · 4d9380e0
      Filipe Manana authored
      Often some test cases like btrfs/161 trigger lockdep splats that complain
      about possible unsafe lock scenario due to the fact that during mount,
      when reading the chunk tree we end up calling blkdev_get_by_path() while
      holding a read lock on a leaf of the chunk tree. That produces a lockdep
      splat like the following:
      
      [ 3653.683975] ======================================================
      [ 3653.685148] WARNING: possible circular locking dependency detected
      [ 3653.686301] 5.15.0-rc7-btrfs-next-103 #1 Not tainted
      [ 3653.687239] ------------------------------------------------------
      [ 3653.688400] mount/447465 is trying to acquire lock:
      [ 3653.689320] ffff8c6b0c76e528 (&disk->open_mutex){+.+.}-{3:3}, at: blkdev_get_by_dev.part.0+0xe7/0x320
      [ 3653.691054]
                     but task is already holding lock:
      [ 3653.692155] ffff8c6b0a9f39e0 (btrfs-chunk-00){++++}-{3:3}, at: __btrfs_tree_read_lock+0x24/0x110 [btrfs]
      [ 3653.693978]
                     which lock already depends on the new lock.
      
      [ 3653.695510]
                     the existing dependency chain (in reverse order) is:
      [ 3653.696915]
                     -> #3 (btrfs-chunk-00){++++}-{3:3}:
      [ 3653.698053]        down_read_nested+0x4b/0x140
      [ 3653.698893]        __btrfs_tree_read_lock+0x24/0x110 [btrfs]
      [ 3653.699988]        btrfs_read_lock_root_node+0x31/0x40 [btrfs]
      [ 3653.701205]        btrfs_search_slot+0x537/0xc00 [btrfs]
      [ 3653.702234]        btrfs_insert_empty_items+0x32/0x70 [btrfs]
      [ 3653.703332]        btrfs_init_new_device+0x563/0x15b0 [btrfs]
      [ 3653.704439]        btrfs_ioctl+0x2110/0x3530 [btrfs]
      [ 3653.705405]        __x64_sys_ioctl+0x83/0xb0
      [ 3653.706215]        do_syscall_64+0x3b/0xc0
      [ 3653.706990]        entry_SYSCALL_64_after_hwframe+0x44/0xae
      [ 3653.708040]
                     -> #2 (sb_internal#2){.+.+}-{0:0}:
      [ 3653.708994]        lock_release+0x13d/0x4a0
      [ 3653.709533]        up_write+0x18/0x160
      [ 3653.710017]        btrfs_sync_file+0x3f3/0x5b0 [btrfs]
      [ 3653.710699]        __loop_update_dio+0xbd/0x170 [loop]
      [ 3653.711360]        lo_ioctl+0x3b1/0x8a0 [loop]
      [ 3653.711929]        block_ioctl+0x48/0x50
      [ 3653.712442]        __x64_sys_ioctl+0x83/0xb0
      [ 3653.712991]        do_syscall_64+0x3b/0xc0
      [ 3653.713519]        entry_SYSCALL_64_after_hwframe+0x44/0xae
      [ 3653.714233]
                     -> #1 (&lo->lo_mutex){+.+.}-{3:3}:
      [ 3653.715026]        __mutex_lock+0x92/0x900
      [ 3653.715648]        lo_open+0x28/0x60 [loop]
      [ 3653.716275]        blkdev_get_whole+0x28/0x90
      [ 3653.716867]        blkdev_get_by_dev.part.0+0x142/0x320
      [ 3653.717537]        blkdev_open+0x5e/0xa0
      [ 3653.718043]        do_dentry_open+0x163/0x390
      [ 3653.718604]        path_openat+0x3f0/0xa80
      [ 3653.719128]        do_filp_open+0xa9/0x150
      [ 3653.719652]        do_sys_openat2+0x97/0x160
      [ 3653.720197]        __x64_sys_openat+0x54/0x90
      [ 3653.720766]        do_syscall_64+0x3b/0xc0
      [ 3653.721285]        entry_SYSCALL_64_after_hwframe+0x44/0xae
      [ 3653.721986]
                     -> #0 (&disk->open_mutex){+.+.}-{3:3}:
      [ 3653.722775]        __lock_acquire+0x130e/0x2210
      [ 3653.723348]        lock_acquire+0xd7/0x310
      [ 3653.723867]        __mutex_lock+0x92/0x900
      [ 3653.724394]        blkdev_get_by_dev.part.0+0xe7/0x320
      [ 3653.725041]        blkdev_get_by_path+0xb8/0xd0
      [ 3653.725614]        btrfs_get_bdev_and_sb+0x1b/0xb0 [btrfs]
      [ 3653.726332]        open_fs_devices+0xd7/0x2c0 [btrfs]
      [ 3653.726999]        btrfs_read_chunk_tree+0x3ad/0x870 [btrfs]
      [ 3653.727739]        open_ctree+0xb8e/0x17bf [btrfs]
      [ 3653.728384]        btrfs_mount_root.cold+0x12/0xde [btrfs]
      [ 3653.729130]        legacy_get_tree+0x30/0x50
      [ 3653.729676]        vfs_get_tree+0x28/0xc0
      [ 3653.730192]        vfs_kern_mount.part.0+0x71/0xb0
      [ 3653.730800]        btrfs_mount+0x11d/0x3a0 [btrfs]
      [ 3653.731427]        legacy_get_tree+0x30/0x50
      [ 3653.731970]        vfs_get_tree+0x28/0xc0
      [ 3653.732486]        path_mount+0x2d4/0xbe0
      [ 3653.732997]        __x64_sys_mount+0x103/0x140
      [ 3653.733560]        do_syscall_64+0x3b/0xc0
      [ 3653.734080]        entry_SYSCALL_64_after_hwframe+0x44/0xae
      [ 3653.734782]
                     other info that might help us debug this:
      
      [ 3653.735784] Chain exists of:
                       &disk->open_mutex --> sb_internal#2 --> btrfs-chunk-00
      
      [ 3653.737123]  Possible unsafe locking scenario:
      
      [ 3653.737865]        CPU0                    CPU1
      [ 3653.738435]        ----                    ----
      [ 3653.739007]   lock(btrfs-chunk-00);
      [ 3653.739449]                                lock(sb_internal#2);
      [ 3653.740193]                                lock(btrfs-chunk-00);
      [ 3653.740955]   lock(&disk->open_mutex);
      [ 3653.741431]
                      *** DEADLOCK ***
      
      [ 3653.742176] 3 locks held by mount/447465:
      [ 3653.742739]  #0: ffff8c6acf85c0e8 (&type->s_umount_key#44/1){+.+.}-{3:3}, at: alloc_super+0xd5/0x3b0
      [ 3653.744114]  #1: ffffffffc0b28f70 (uuid_mutex){+.+.}-{3:3}, at: btrfs_read_chunk_tree+0x59/0x870 [btrfs]
      [ 3653.745563]  #2: ffff8c6b0a9f39e0 (btrfs-chunk-00){++++}-{3:3}, at: __btrfs_tree_read_lock+0x24/0x110 [btrfs]
      [ 3653.747066]
                     stack backtrace:
      [ 3653.747723] CPU: 4 PID: 447465 Comm: mount Not tainted 5.15.0-rc7-btrfs-next-103 #1
      [ 3653.748873] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS rel-1.14.0-0-g155821a1990b-prebuilt.qemu.org 04/01/2014
      [ 3653.750592] Call Trace:
      [ 3653.750967]  dump_stack_lvl+0x57/0x72
      [ 3653.751526]  check_noncircular+0xf3/0x110
      [ 3653.752136]  ? stack_trace_save+0x4b/0x70
      [ 3653.752748]  __lock_acquire+0x130e/0x2210
      [ 3653.753356]  lock_acquire+0xd7/0x310
      [ 3653.753898]  ? blkdev_get_by_dev.part.0+0xe7/0x320
      [ 3653.754596]  ? lock_is_held_type+0xe8/0x140
      [ 3653.755125]  ? blkdev_get_by_dev.part.0+0xe7/0x320
      [ 3653.755729]  ? blkdev_get_by_dev.part.0+0xe7/0x320
      [ 3653.756338]  __mutex_lock+0x92/0x900
      [ 3653.756794]  ? blkdev_get_by_dev.part.0+0xe7/0x320
      [ 3653.757400]  ? do_raw_spin_unlock+0x4b/0xa0
      [ 3653.757930]  ? _raw_spin_unlock+0x29/0x40
      [ 3653.758437]  ? bd_prepare_to_claim+0x129/0x150
      [ 3653.758999]  ? trace_module_get+0x2b/0xd0
      [ 3653.759508]  ? try_module_get.part.0+0x50/0x80
      [ 3653.760072]  blkdev_get_by_dev.part.0+0xe7/0x320
      [ 3653.760661]  ? devcgroup_check_permission+0xc1/0x1f0
      [ 3653.761288]  blkdev_get_by_path+0xb8/0xd0
      [ 3653.761797]  btrfs_get_bdev_and_sb+0x1b/0xb0 [btrfs]
      [ 3653.762454]  open_fs_devices+0xd7/0x2c0 [btrfs]
      [ 3653.763055]  ? clone_fs_devices+0x8f/0x170 [btrfs]
      [ 3653.763689]  btrfs_read_chunk_tree+0x3ad/0x870 [btrfs]
      [ 3653.764370]  ? kvm_sched_clock_read+0x14/0x40
      [ 3653.764922]  open_ctree+0xb8e/0x17bf [btrfs]
      [ 3653.765493]  ? super_setup_bdi_name+0x79/0xd0
      [ 3653.766043]  btrfs_mount_root.cold+0x12/0xde [btrfs]
      [ 3653.766780]  ? rcu_read_lock_sched_held+0x3f/0x80
      [ 3653.767488]  ? kfree+0x1f2/0x3c0
      [ 3653.767979]  legacy_get_tree+0x30/0x50
      [ 3653.768548]  vfs_get_tree+0x28/0xc0
      [ 3653.769076]  vfs_kern_mount.part.0+0x71/0xb0
      [ 3653.769718]  btrfs_mount+0x11d/0x3a0 [btrfs]
      [ 3653.770381]  ? rcu_read_lock_sched_held+0x3f/0x80
      [ 3653.771086]  ? kfree+0x1f2/0x3c0
      [ 3653.771574]  legacy_get_tree+0x30/0x50
      [ 3653.772136]  vfs_get_tree+0x28/0xc0
      [ 3653.772673]  path_mount+0x2d4/0xbe0
      [ 3653.773201]  __x64_sys_mount+0x103/0x140
      [ 3653.773793]  do_syscall_64+0x3b/0xc0
      [ 3653.774333]  entry_SYSCALL_64_after_hwframe+0x44/0xae
      [ 3653.775094] RIP: 0033:0x7f648bc45aaa
      
      This happens because through btrfs_read_chunk_tree(), which is called only
      during mount, ends up acquiring the mutex open_mutex of a block device
      while holding a read lock on a leaf of the chunk tree while other paths
      need to acquire other locks before locking extent buffers of the chunk
      tree.
      
      Since at mount time when we call btrfs_read_chunk_tree() we know that
      we don't have other tasks running in parallel and modifying the chunk
      tree, we can simply skip locking of chunk tree extent buffers. So do
      that and move the assertion that checks the fs is not yet mounted to the
      top block of btrfs_read_chunk_tree(), with a comment before doing it.
      Signed-off-by: default avatarFilipe Manana <fdmanana@suse.com>
      Signed-off-by: default avatarDavid Sterba <dsterba@suse.com>
      4d9380e0
    • Nikolay Borisov's avatar
      btrfs: fix memory ordering between normal and ordered work functions · 45da9c17
      Nikolay Borisov authored
      Ordered work functions aren't guaranteed to be handled by the same thread
      which executed the normal work functions. The only way execution between
      normal/ordered functions is synchronized is via the WORK_DONE_BIT,
      unfortunately the used bitops don't guarantee any ordering whatsoever.
      
      This manifested as seemingly inexplicable crashes on ARM64, where
      async_chunk::inode is seen as non-null in async_cow_submit which causes
      submit_compressed_extents to be called and crash occurs because
      async_chunk::inode suddenly became NULL. The call trace was similar to:
      
          pc : submit_compressed_extents+0x38/0x3d0
          lr : async_cow_submit+0x50/0xd0
          sp : ffff800015d4bc20
      
          <registers omitted for brevity>
      
          Call trace:
           submit_compressed_extents+0x38/0x3d0
           async_cow_submit+0x50/0xd0
           run_ordered_work+0xc8/0x280
           btrfs_work_helper+0x98/0x250
           process_one_work+0x1f0/0x4ac
           worker_thread+0x188/0x504
           kthread+0x110/0x114
           ret_from_fork+0x10/0x18
      
      Fix this by adding respective barrier calls which ensure that all
      accesses preceding setting of WORK_DONE_BIT are strictly ordered before
      setting the flag. At the same time add a read barrier after reading of
      WORK_DONE_BIT in run_ordered_work which ensures all subsequent loads
      would be strictly ordered after reading the bit. This in turn ensures
      are all accesses before WORK_DONE_BIT are going to be strictly ordered
      before any access that can occur in ordered_func.
      Reported-by: default avatarChris Murphy <lists@colorremedies.com>
      Fixes: 08a9ff32 ("btrfs: Added btrfs_workqueue_struct implemented ordered execution based on kernel workqueue")
      CC: stable@vger.kernel.org # 4.4+
      Link: https://bugzilla.redhat.com/show_bug.cgi?id=2011928Reviewed-by: default avatarJosef Bacik <josef@toxicpanda.com>
      Tested-by: default avatarChris Murphy <chris@colorremedies.com>
      Signed-off-by: default avatarNikolay Borisov <nborisov@suse.com>
      Signed-off-by: default avatarDavid Sterba <dsterba@suse.com>
      45da9c17
    • Qu Wenruo's avatar
      btrfs: fix a out-of-bound access in copy_compressed_data_to_page() · 6f019c0e
      Qu Wenruo authored
      [BUG]
      The following script can cause btrfs to crash:
      
        $ mount -o compress-force=lzo $DEV /mnt
        $ dd if=/dev/urandom of=/mnt/foo bs=4k count=1
        $ sync
      
      The call trace looks like this:
      
        general protection fault, probably for non-canonical address 0xe04b37fccce3b000: 0000 [#1] PREEMPT SMP NOPTI
        CPU: 5 PID: 164 Comm: kworker/u20:3 Not tainted 5.15.0-rc7-custom+ #4
        Workqueue: btrfs-delalloc btrfs_work_helper [btrfs]
        RIP: 0010:__memcpy+0x12/0x20
        Call Trace:
         lzo_compress_pages+0x236/0x540 [btrfs]
         btrfs_compress_pages+0xaa/0xf0 [btrfs]
         compress_file_range+0x431/0x8e0 [btrfs]
         async_cow_start+0x12/0x30 [btrfs]
         btrfs_work_helper+0xf6/0x3e0 [btrfs]
         process_one_work+0x294/0x5d0
         worker_thread+0x55/0x3c0
         kthread+0x140/0x170
         ret_from_fork+0x22/0x30
        ---[ end trace 63c3c0f131e61982 ]---
      
      [CAUSE]
      In lzo_compress_pages(), parameter @out_pages is not only an output
      parameter (for the number of compressed pages), but also an input
      parameter, as the upper limit of compressed pages we can utilize.
      
      In commit d4088803 ("btrfs: subpage: make lzo_compress_pages()
      compatible"), the refactoring doesn't take @out_pages as an input, thus
      completely ignoring the limit.
      
      And for compress-force case, we could hit incompressible data that
      compressed size would go beyond the page limit, and cause the above
      crash.
      
      [FIX]
      Save @out_pages as @max_nr_page, and pass it to lzo_compress_pages(),
      and check if we're beyond the limit before accessing the pages.
      
      Note: this also fixes crash on 32bit architectures that was suspected to
      be caused by merge of btrfs patches to 5.16-rc1. Reported in
      https://lore.kernel.org/all/20211104115001.GU20319@twin.jikos.cz/ .
      Reported-by: default avatarOmar Sandoval <osandov@fb.com>
      Fixes: d4088803 ("btrfs: subpage: make lzo_compress_pages() compatible")
      Reviewed-by: default avatarOmar Sandoval <osandov@fb.com>
      Reviewed-by: default avatarJosef Bacik <josef@toxicpanda.com>
      Signed-off-by: default avatarQu Wenruo <wqu@suse.com>
      Reviewed-by: default avatarDavid Sterba <dsterba@suse.com>
      [ add note ]
      Signed-off-by: default avatarDavid Sterba <dsterba@suse.com>
      6f019c0e
  2. 29 Oct, 2021 7 commits
    • Filipe Manana's avatar
      btrfs: remove root argument from check_item_in_log() · d1ed82f3
      Filipe Manana authored
      The root argument passed to check_item_in_log() always matches the root
      of the given directory, so it can be eliminated.
      Signed-off-by: default avatarFilipe Manana <fdmanana@suse.com>
      Reviewed-by: default avatarDavid Sterba <dsterba@suse.com>
      Signed-off-by: default avatarDavid Sterba <dsterba@suse.com>
      d1ed82f3
    • Filipe Manana's avatar
      btrfs: remove root argument from add_link() · 6d9cc072
      Filipe Manana authored
      The root argument for tree-log.c:add_link() always matches the root of the
      given directory and the given inode, so it can eliminated.
      Signed-off-by: default avatarFilipe Manana <fdmanana@suse.com>
      Reviewed-by: default avatarDavid Sterba <dsterba@suse.com>
      Signed-off-by: default avatarDavid Sterba <dsterba@suse.com>
      6d9cc072
    • Filipe Manana's avatar
      btrfs: remove root argument from btrfs_unlink_inode() · 4467af88
      Filipe Manana authored
      The root argument passed to btrfs_unlink_inode() and its callee,
      __btrfs_unlink_inode(), always matches the root of the given directory and
      the given inode. So remove the argument and make __btrfs_unlink_inode()
      use the root of the directory.
      Signed-off-by: default avatarFilipe Manana <fdmanana@suse.com>
      Reviewed-by: default avatarDavid Sterba <dsterba@suse.com>
      Signed-off-by: default avatarDavid Sterba <dsterba@suse.com>
      4467af88
    • Filipe Manana's avatar
      btrfs: remove root argument from drop_one_dir_item() · 9798ba24
      Filipe Manana authored
      The root argument for drop_one_dir_item() always matches the root of the
      given directory inode, since each log tree is associated to one and only
      one subvolume/root, so remove the argument.
      Signed-off-by: default avatarFilipe Manana <fdmanana@suse.com>
      Reviewed-by: default avatarDavid Sterba <dsterba@suse.com>
      Signed-off-by: default avatarDavid Sterba <dsterba@suse.com>
      9798ba24
    • Li Zhang's avatar
      btrfs: clear MISSING device status bit in btrfs_close_one_device · 5d03dbeb
      Li Zhang authored
      Reported bug: https://github.com/kdave/btrfs-progs/issues/389
      
      There's a problem with scrub reporting aborted status but returning
      error code 0, on a filesystem with missing and readded device.
      
      Roughly these steps:
      
      - mkfs -d raid1 dev1 dev2
      - fill with data
      - unmount
      - make dev1 disappear
      - mount -o degraded
      - copy more data
      - make dev1 appear again
      
      Running scrub afterwards reports that the command was aborted, but the
      system log message says the exit code was 0.
      
      It seems that the cause of the error is decrementing
      fs_devices->missing_devices but not clearing device->dev_state.  Every
      time we umount filesystem, it would call close_ctree, And it would
      eventually involve btrfs_close_one_device to close the device, but it
      only decrements fs_devices->missing_devices but does not clear the
      device BTRFS_DEV_STATE_MISSING bit. Worse, this bug will cause Integer
      Overflow, because every time umount, fs_devices->missing_devices will
      decrease. If fs_devices->missing_devices value hit 0, it would overflow.
      
      With added debugging:
      
         loop1: detected capacity change from 0 to 20971520
         BTRFS: device fsid 56ad51f1-5523-463b-8547-c19486c51ebb devid 1 transid 21 /dev/loop1 scanned by systemd-udevd (2311)
         loop2: detected capacity change from 0 to 20971520
         BTRFS: device fsid 56ad51f1-5523-463b-8547-c19486c51ebb devid 2 transid 17 /dev/loop2 scanned by systemd-udevd (2313)
         BTRFS info (device loop1): flagging fs with big metadata feature
         BTRFS info (device loop1): allowing degraded mounts
         BTRFS info (device loop1): using free space tree
         BTRFS info (device loop1): has skinny extents
         BTRFS info (device loop1):  before clear_missing.00000000f706684d /dev/loop1 0
         BTRFS warning (device loop1): devid 2 uuid 6635ac31-56dd-4852-873b-c60f5e2d53d2 is missing
         BTRFS info (device loop1):  before clear_missing.0000000000000000 /dev/loop2 1
         BTRFS info (device loop1): flagging fs with big metadata feature
         BTRFS info (device loop1): allowing degraded mounts
         BTRFS info (device loop1): using free space tree
         BTRFS info (device loop1): has skinny extents
         BTRFS info (device loop1):  before clear_missing.00000000f706684d /dev/loop1 0
         BTRFS warning (device loop1): devid 2 uuid 6635ac31-56dd-4852-873b-c60f5e2d53d2 is missing
         BTRFS info (device loop1):  before clear_missing.0000000000000000 /dev/loop2 0
         BTRFS info (device loop1): flagging fs with big metadata feature
         BTRFS info (device loop1): allowing degraded mounts
         BTRFS info (device loop1): using free space tree
         BTRFS info (device loop1): has skinny extents
         BTRFS info (device loop1):  before clear_missing.00000000f706684d /dev/loop1 18446744073709551615
         BTRFS warning (device loop1): devid 2 uuid 6635ac31-56dd-4852-873b-c60f5e2d53d2 is missing
         BTRFS info (device loop1):  before clear_missing.0000000000000000 /dev/loop2 18446744073709551615
      
      If fs_devices->missing_devices is 0, next time it would be 18446744073709551615
      
      After apply this patch, the fs_devices->missing_devices seems to be
      right:
      
        $ truncate -s 10g test1
        $ truncate -s 10g test2
        $ losetup /dev/loop1 test1
        $ losetup /dev/loop2 test2
        $ mkfs.btrfs -draid1 -mraid1 /dev/loop1 /dev/loop2 -f
        $ losetup -d /dev/loop2
        $ mount -o degraded /dev/loop1 /mnt/1
        $ umount /mnt/1
        $ mount -o degraded /dev/loop1 /mnt/1
        $ umount /mnt/1
        $ mount -o degraded /dev/loop1 /mnt/1
        $ umount /mnt/1
        $ dmesg
      
         loop1: detected capacity change from 0 to 20971520
         loop2: detected capacity change from 0 to 20971520
         BTRFS: device fsid 15aa1203-98d3-4a66-bcae-ca82f629c2cd devid 1 transid 5 /dev/loop1 scanned by mkfs.btrfs (1863)
         BTRFS: device fsid 15aa1203-98d3-4a66-bcae-ca82f629c2cd devid 2 transid 5 /dev/loop2 scanned by mkfs.btrfs (1863)
         BTRFS info (device loop1): flagging fs with big metadata feature
         BTRFS info (device loop1): allowing degraded mounts
         BTRFS info (device loop1): disk space caching is enabled
         BTRFS info (device loop1): has skinny extents
         BTRFS info (device loop1):  before clear_missing.00000000975bd577 /dev/loop1 0
         BTRFS warning (device loop1): devid 2 uuid 8b333791-0b3f-4f57-b449-1c1ab6b51f38 is missing
         BTRFS info (device loop1):  before clear_missing.0000000000000000 /dev/loop2 1
         BTRFS info (device loop1): checking UUID tree
         BTRFS info (device loop1): flagging fs with big metadata feature
         BTRFS info (device loop1): allowing degraded mounts
         BTRFS info (device loop1): disk space caching is enabled
         BTRFS info (device loop1): has skinny extents
         BTRFS info (device loop1):  before clear_missing.00000000975bd577 /dev/loop1 0
         BTRFS warning (device loop1): devid 2 uuid 8b333791-0b3f-4f57-b449-1c1ab6b51f38 is missing
         BTRFS info (device loop1):  before clear_missing.0000000000000000 /dev/loop2 1
         BTRFS info (device loop1): flagging fs with big metadata feature
         BTRFS info (device loop1): allowing degraded mounts
         BTRFS info (device loop1): disk space caching is enabled
         BTRFS info (device loop1): has skinny extents
         BTRFS info (device loop1):  before clear_missing.00000000975bd577 /dev/loop1 0
         BTRFS warning (device loop1): devid 2 uuid 8b333791-0b3f-4f57-b449-1c1ab6b51f38 is missing
         BTRFS info (device loop1):  before clear_missing.0000000000000000 /dev/loop2 1
      
      CC: stable@vger.kernel.org # 4.19+
      Signed-off-by: default avatarLi Zhang <zhanglikernel@gmail.com>
      Reviewed-by: default avatarDavid Sterba <dsterba@suse.com>
      Signed-off-by: default avatarDavid Sterba <dsterba@suse.com>
      5d03dbeb
    • Anand Jain's avatar
      btrfs: call btrfs_check_rw_degradable only if there is a missing device · 5c78a5e7
      Anand Jain authored
      In open_ctree() in btrfs_check_rw_degradable() [1], we check each block
      group individually if at least the minimum number of devices is available
      for that profile. If all the devices are available, then we don't have to
      check degradable.
      
      [1]
      open_ctree()
      ::
      3559 if (!sb_rdonly(sb) && !btrfs_check_rw_degradable(fs_info, NULL)) {
      
      Also before calling btrfs_check_rw_degradable() in open_ctee() at the
      line number shown below [2] we call btrfs_read_chunk_tree() and down to
      add_missing_dev() to record number of missing devices.
      
      [2]
      open_ctree()
      ::
      3454         ret = btrfs_read_chunk_tree(fs_info);
      
      btrfs_read_chunk_tree()
        read_one_chunk() / read_one_dev()
          add_missing_dev()
      
      So, check if there is any missing device before btrfs_check_rw_degradable()
      in open_ctree().
      
      Also, with this the mount command could save ~16ms.[3] in the most
      common case, that is no device is missing.
      
      [3]
       1) * 16934.96 us | btrfs_check_rw_degradable [btrfs]();
      
      CC: stable@vger.kernel.org # 4.19+
      Reviewed-by: default avatarJosef Bacik <josef@toxicpanda.com>
      Signed-off-by: default avatarAnand Jain <anand.jain@oracle.com>
      Reviewed-by: default avatarDavid Sterba <dsterba@suse.com>
      Signed-off-by: default avatarDavid Sterba <dsterba@suse.com>
      5c78a5e7
    • David Sterba's avatar
      btrfs: send: prepare for v2 protocol · e77fbf99
      David Sterba authored
      This is preparatory work for send protocol update to version 2 and
      higher.
      
      We have many pending protocol update requests but still don't have the
      basic protocol rev in place, the first thing that must happen is to do
      the actual versioning support.
      
      The protocol version is u32 and is a new member in the send ioctl
      struct. Validity of the version field is backed by a new flag bit. Old
      kernels would fail when a higher version is requested. Version protocol
      0 will pick the highest supported version, BTRFS_SEND_STREAM_VERSION,
        that's also exported in sysfs.
      
      The version is still unchanged and will be increased once we have new
      incompatible commands or stream updates.
      Reviewed-by: default avatarNikolay Borisov <nborisov@suse.com>
      Signed-off-by: default avatarDavid Sterba <dsterba@suse.com>
      e77fbf99
  3. 26 Oct, 2021 30 commits
    • Anand Jain's avatar
      btrfs: fix comment about sector sizes supported in 64K systems · 50780d9b
      Anand Jain authored
      Commit 95ea0486 ("btrfs: allow read-write for 4K sectorsize on 64K
      page size systems") added write support for 4K sectorsize on a 64K
      systems. Fix the now stale comments.
      Reviewed-by: default avatarJosef Bacik <josef@toxicpanda.com>
      Signed-off-by: default avatarAnand Jain <anand.jain@oracle.com>
      Reviewed-by: default avatarDavid Sterba <dsterba@suse.com>
      Signed-off-by: default avatarDavid Sterba <dsterba@suse.com>
      50780d9b
    • Josef Bacik's avatar
      btrfs: update device path inode time instead of bd_inode · 54fde91f
      Josef Bacik authored
      Christoph pointed out that I'm updating bdev->bd_inode for the device
      time when we remove block devices from a btrfs file system, however this
      isn't actually exposed to anything.  The inode we want to update is the
      one that's associated with the path to the device, usually on devtmpfs,
      so that blkid notices the difference.
      
      We still don't want to do the blkdev_open, so use kern_path() to get the
      path to the given device and do the update time on that inode.
      
      Fixes: 8f96a5bf ("btrfs: update the bdev time directly when closing")
      Reported-by: default avatarChristoph Hellwig <hch@lst.de>
      Signed-off-by: default avatarJosef Bacik <josef@toxicpanda.com>
      Signed-off-by: default avatarDavid Sterba <dsterba@suse.com>
      54fde91f
    • Josef Bacik's avatar
      fs: export an inode_update_time helper · e60feb44
      Josef Bacik authored
      If you already have an inode and need to update the time on the inode
      there is no way to do this properly.  Export this helper to allow file
      systems to update time on the inode so the appropriate handler is
      called, either ->update_time or generic_update_time.
      Signed-off-by: default avatarJosef Bacik <josef@toxicpanda.com>
      Reviewed-by: default avatarDavid Sterba <dsterba@suse.com>
      Signed-off-by: default avatarDavid Sterba <dsterba@suse.com>
      e60feb44
    • Omar Sandoval's avatar
      btrfs: fix deadlock when defragging transparent huge pages · 24bcb454
      Omar Sandoval authored
      Attempting to defragment a Btrfs file containing a transparent huge page
      immediately deadlocks with the following stack trace:
      
        #0  context_switch (kernel/sched/core.c:4940:2)
        #1  __schedule (kernel/sched/core.c:6287:8)
        #2  schedule (kernel/sched/core.c:6366:3)
        #3  io_schedule (kernel/sched/core.c:8389:2)
        #4  wait_on_page_bit_common (mm/filemap.c:1356:4)
        #5  __lock_page (mm/filemap.c:1648:2)
        #6  lock_page (./include/linux/pagemap.h:625:3)
        #7  pagecache_get_page (mm/filemap.c:1910:4)
        #8  find_or_create_page (./include/linux/pagemap.h:420:9)
        #9  defrag_prepare_one_page (fs/btrfs/ioctl.c:1068:9)
        #10 defrag_one_range (fs/btrfs/ioctl.c:1326:14)
        #11 defrag_one_cluster (fs/btrfs/ioctl.c:1421:9)
        #12 btrfs_defrag_file (fs/btrfs/ioctl.c:1523:9)
        #13 btrfs_ioctl_defrag (fs/btrfs/ioctl.c:3117:9)
        #14 btrfs_ioctl (fs/btrfs/ioctl.c:4872:10)
        #15 vfs_ioctl (fs/ioctl.c:51:10)
        #16 __do_sys_ioctl (fs/ioctl.c:874:11)
        #17 __se_sys_ioctl (fs/ioctl.c:860:1)
        #18 __x64_sys_ioctl (fs/ioctl.c:860:1)
        #19 do_syscall_x64 (arch/x86/entry/common.c:50:14)
        #20 do_syscall_64 (arch/x86/entry/common.c:80:7)
        #21 entry_SYSCALL_64+0x7c/0x15b (arch/x86/entry/entry_64.S:113)
      
      A huge page is represented by a compound page, which consists of a
      struct page for each PAGE_SIZE page within the huge page. The first
      struct page is the "head page", and the remaining are "tail pages".
      
      Defragmentation attempts to lock each page in the range. However,
      lock_page() on a tail page actually locks the corresponding head page.
      So, if defragmentation tries to lock more than one struct page in a
      compound page, it tries to lock the same head page twice and deadlocks
      with itself.
      
      Ideally, we should be able to defragment transparent huge pages.
      However, THP for filesystems is currently read-only, so a lot of code is
      not ready to use huge pages for I/O. For now, let's just return
      ETXTBUSY.
      
      This can be reproduced with the following on a kernel with
      CONFIG_READ_ONLY_THP_FOR_FS=y:
      
        $ cat create_thp_file.c
        #include <fcntl.h>
        #include <stdbool.h>
        #include <stdio.h>
        #include <stdint.h>
        #include <stdlib.h>
        #include <unistd.h>
        #include <sys/mman.h>
      
        static const char zeroes[1024 * 1024];
        static const size_t FILE_SIZE = 2 * 1024 * 1024;
      
        int main(int argc, char **argv)
        {
                if (argc != 2) {
                        fprintf(stderr, "usage: %s PATH\n", argv[0]);
                        return EXIT_FAILURE;
                }
                int fd = creat(argv[1], 0777);
                if (fd == -1) {
                        perror("creat");
                        return EXIT_FAILURE;
                }
                size_t written = 0;
                while (written < FILE_SIZE) {
                        ssize_t ret = write(fd, zeroes,
                                            sizeof(zeroes) < FILE_SIZE - written ?
                                            sizeof(zeroes) : FILE_SIZE - written);
                        if (ret < 0) {
                                perror("write");
                                return EXIT_FAILURE;
                        }
                        written += ret;
                }
                close(fd);
                fd = open(argv[1], O_RDONLY);
                if (fd == -1) {
                        perror("open");
                        return EXIT_FAILURE;
                }
      
                /*
                 * Reserve some address space so that we can align the file mapping to
                 * the huge page size.
                 */
                void *placeholder_map = mmap(NULL, FILE_SIZE * 2, PROT_NONE,
                                             MAP_PRIVATE | MAP_ANONYMOUS, -1, 0);
                if (placeholder_map == MAP_FAILED) {
                        perror("mmap (placeholder)");
                        return EXIT_FAILURE;
                }
      
                void *aligned_address =
                        (void *)(((uintptr_t)placeholder_map + FILE_SIZE - 1) & ~(FILE_SIZE - 1));
      
                void *map = mmap(aligned_address, FILE_SIZE, PROT_READ | PROT_EXEC,
                                 MAP_SHARED | MAP_FIXED, fd, 0);
                if (map == MAP_FAILED) {
                        perror("mmap");
                        return EXIT_FAILURE;
                }
                if (madvise(map, FILE_SIZE, MADV_HUGEPAGE) < 0) {
                        perror("madvise");
                        return EXIT_FAILURE;
                }
      
                char *line = NULL;
                size_t line_capacity = 0;
                FILE *smaps_file = fopen("/proc/self/smaps", "r");
                if (!smaps_file) {
                        perror("fopen");
                        return EXIT_FAILURE;
                }
                for (;;) {
                        for (size_t off = 0; off < FILE_SIZE; off += 4096)
                                ((volatile char *)map)[off];
      
                        ssize_t ret;
                        bool this_mapping = false;
                        while ((ret = getline(&line, &line_capacity, smaps_file)) > 0) {
                                unsigned long start, end, huge;
                                if (sscanf(line, "%lx-%lx", &start, &end) == 2) {
                                        this_mapping = (start <= (uintptr_t)map &&
                                                        (uintptr_t)map < end);
                                } else if (this_mapping &&
                                           sscanf(line, "FilePmdMapped: %ld", &huge) == 1 &&
                                           huge > 0) {
                                        return EXIT_SUCCESS;
                                }
                        }
      
                        sleep(6);
                        rewind(smaps_file);
                        fflush(smaps_file);
                }
        }
        $ ./create_thp_file huge
        $ btrfs fi defrag -czstd ./huge
      Reviewed-by: default avatarJosef Bacik <josef@toxicpanda.com>
      Signed-off-by: default avatarOmar Sandoval <osandov@fb.com>
      Reviewed-by: default avatarDavid Sterba <dsterba@suse.com>
      Signed-off-by: default avatarDavid Sterba <dsterba@suse.com>
      24bcb454
    • Anand Jain's avatar
      btrfs: sysfs: convert scnprintf and snprintf to sysfs_emit · 020e5277
      Anand Jain authored
      Commit 2efc459d ("sysfs: Add sysfs_emit and sysfs_emit_at to format
      sysfs out") merged in 5.10 introduced two new functions sysfs_emit() and
      sysfs_emit_at() which are aware of the PAGE_SIZE limit of the output
      buffer.
      
      Use the above two new functions instead of scnprintf() and snprintf()
      in various sysfs show().
      Reviewed-by: default avatarJosef Bacik <josef@toxicpanda.com>
      Signed-off-by: default avatarAnand Jain <anand.jain@oracle.com>
      Reviewed-by: default avatarDavid Sterba <dsterba@suse.com>
      Signed-off-by: default avatarDavid Sterba <dsterba@suse.com>
      020e5277
    • Qu Wenruo's avatar
      btrfs: make btrfs_super_block size match BTRFS_SUPER_INFO_SIZE · 38732474
      Qu Wenruo authored
      It's a common practice to avoid use sizeof(struct btrfs_super_block)
      (3531), but to use BTRFS_SUPER_INFO_SIZE (4096).
      
      The problem is that, sizeof(struct btrfs_super_block) doesn't match
      BTRFS_SUPER_INFO_SIZE from the very beginning.
      
      Furthermore, for all call sites except selftests, we always allocate
      BTRFS_SUPER_INFO_SIZE space for super block, there isn't any real reason
      to use the smaller value, and it doesn't really save any space.
      
      So let's get rid of such confusing behavior, and unify those two values.
      
      This modification also adds a new static_assert() to verify the size,
      and moves the BTRFS_SUPER_INFO_* macros to the definition of
      btrfs_super_block for the static_assert().
      Reviewed-by: default avatarJosef Bacik <josef@toxicpanda.com>
      Signed-off-by: default avatarQu Wenruo <wqu@suse.com>
      Reviewed-by: default avatarDavid Sterba <dsterba@suse.com>
      Signed-off-by: default avatarDavid Sterba <dsterba@suse.com>
      38732474
    • Filipe Manana's avatar
      btrfs: update comments for chunk allocation -ENOSPC cases · ecd84d54
      Filipe Manana authored
      Update the comments at btrfs_chunk_alloc() and do_chunk_alloc() that
      describe which cases can lead to a failure to allocate metadata and system
      space despite having previously reserved space. This adds one more reason
      that I previously forgot to mention.
      Reviewed-by: default avatarJosef Bacik <josef@toxicpanda.com>
      Signed-off-by: default avatarFilipe Manana <fdmanana@suse.com>
      Signed-off-by: default avatarDavid Sterba <dsterba@suse.com>
      ecd84d54
    • Filipe Manana's avatar
      btrfs: fix deadlock between chunk allocation and chunk btree modifications · 2bb2e00e
      Filipe Manana authored
      When a task is doing some modification to the chunk btree and it is not in
      the context of a chunk allocation or a chunk removal, it can deadlock with
      another task that is currently allocating a new data or metadata chunk.
      
      These contexts are the following:
      
      * When relocating a system chunk, when we need to COW the extent buffers
        that belong to the chunk btree;
      
      * When adding a new device (ioctl), where we need to add a new device item
        to the chunk btree;
      
      * When removing a device (ioctl), where we need to remove a device item
        from the chunk btree;
      
      * When resizing a device (ioctl), where we need to update a device item in
        the chunk btree and may need to relocate a system chunk that lies beyond
        the new device size when shrinking a device.
      
      The problem happens due to a sequence of steps like the following:
      
      1) Task A starts a data or metadata chunk allocation and it locks the
         chunk mutex;
      
      2) Task B is relocating a system chunk, and when it needs to COW an extent
         buffer of the chunk btree, it has locked both that extent buffer as
         well as its parent extent buffer;
      
      3) Since there is not enough available system space, either because none
         of the existing system block groups have enough free space or because
         the only one with enough free space is in RO mode due to the relocation,
         task B triggers a new system chunk allocation. It blocks when trying to
         acquire the chunk mutex, currently held by task A;
      
      4) Task A enters btrfs_chunk_alloc_add_chunk_item(), in order to insert
         the new chunk item into the chunk btree and update the existing device
         items there. But in order to do that, it has to lock the extent buffer
         that task B locked at step 2, or its parent extent buffer, but task B
         is waiting on the chunk mutex, which is currently locked by task A,
         therefore resulting in a deadlock.
      
      One example report when the deadlock happens with system chunk relocation:
      
        INFO: task kworker/u9:5:546 blocked for more than 143 seconds.
              Not tainted 5.15.0-rc3+ #1
        "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
        task:kworker/u9:5    state:D stack:25936 pid:  546 ppid:     2 flags:0x00004000
        Workqueue: events_unbound btrfs_async_reclaim_metadata_space
        Call Trace:
         context_switch kernel/sched/core.c:4940 [inline]
         __schedule+0xcd9/0x2530 kernel/sched/core.c:6287
         schedule+0xd3/0x270 kernel/sched/core.c:6366
         rwsem_down_read_slowpath+0x4ee/0x9d0 kernel/locking/rwsem.c:993
         __down_read_common kernel/locking/rwsem.c:1214 [inline]
         __down_read kernel/locking/rwsem.c:1223 [inline]
         down_read_nested+0xe6/0x440 kernel/locking/rwsem.c:1590
         __btrfs_tree_read_lock+0x31/0x350 fs/btrfs/locking.c:47
         btrfs_tree_read_lock fs/btrfs/locking.c:54 [inline]
         btrfs_read_lock_root_node+0x8a/0x320 fs/btrfs/locking.c:191
         btrfs_search_slot_get_root fs/btrfs/ctree.c:1623 [inline]
         btrfs_search_slot+0x13b4/0x2140 fs/btrfs/ctree.c:1728
         btrfs_update_device+0x11f/0x500 fs/btrfs/volumes.c:2794
         btrfs_chunk_alloc_add_chunk_item+0x34d/0xea0 fs/btrfs/volumes.c:5504
         do_chunk_alloc fs/btrfs/block-group.c:3408 [inline]
         btrfs_chunk_alloc+0x84d/0xf50 fs/btrfs/block-group.c:3653
         flush_space+0x54e/0xd80 fs/btrfs/space-info.c:670
         btrfs_async_reclaim_metadata_space+0x396/0xa90 fs/btrfs/space-info.c:953
         process_one_work+0x9df/0x16d0 kernel/workqueue.c:2297
         worker_thread+0x90/0xed0 kernel/workqueue.c:2444
         kthread+0x3e5/0x4d0 kernel/kthread.c:319
         ret_from_fork+0x1f/0x30 arch/x86/entry/entry_64.S:295
        INFO: task syz-executor:9107 blocked for more than 143 seconds.
              Not tainted 5.15.0-rc3+ #1
        "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
        task:syz-executor    state:D stack:23200 pid: 9107 ppid:  7792 flags:0x00004004
        Call Trace:
         context_switch kernel/sched/core.c:4940 [inline]
         __schedule+0xcd9/0x2530 kernel/sched/core.c:6287
         schedule+0xd3/0x270 kernel/sched/core.c:6366
         schedule_preempt_disabled+0xf/0x20 kernel/sched/core.c:6425
         __mutex_lock_common kernel/locking/mutex.c:669 [inline]
         __mutex_lock+0xc96/0x1680 kernel/locking/mutex.c:729
         btrfs_chunk_alloc+0x31a/0xf50 fs/btrfs/block-group.c:3631
         find_free_extent_update_loop fs/btrfs/extent-tree.c:3986 [inline]
         find_free_extent+0x25cb/0x3a30 fs/btrfs/extent-tree.c:4335
         btrfs_reserve_extent+0x1f1/0x500 fs/btrfs/extent-tree.c:4415
         btrfs_alloc_tree_block+0x203/0x1120 fs/btrfs/extent-tree.c:4813
         __btrfs_cow_block+0x412/0x1620 fs/btrfs/ctree.c:415
         btrfs_cow_block+0x2f6/0x8c0 fs/btrfs/ctree.c:570
         btrfs_search_slot+0x1094/0x2140 fs/btrfs/ctree.c:1768
         relocate_tree_block fs/btrfs/relocation.c:2694 [inline]
         relocate_tree_blocks+0xf73/0x1770 fs/btrfs/relocation.c:2757
         relocate_block_group+0x47e/0xc70 fs/btrfs/relocation.c:3673
         btrfs_relocate_block_group+0x48a/0xc60 fs/btrfs/relocation.c:4070
         btrfs_relocate_chunk+0x96/0x280 fs/btrfs/volumes.c:3181
         __btrfs_balance fs/btrfs/volumes.c:3911 [inline]
         btrfs_balance+0x1f03/0x3cd0 fs/btrfs/volumes.c:4301
         btrfs_ioctl_balance+0x61e/0x800 fs/btrfs/ioctl.c:4137
         btrfs_ioctl+0x39ea/0x7b70 fs/btrfs/ioctl.c:4949
         vfs_ioctl fs/ioctl.c:51 [inline]
         __do_sys_ioctl fs/ioctl.c:874 [inline]
         __se_sys_ioctl fs/ioctl.c:860 [inline]
         __x64_sys_ioctl+0x193/0x200 fs/ioctl.c:860
         do_syscall_x64 arch/x86/entry/common.c:50 [inline]
         do_syscall_64+0x35/0xb0 arch/x86/entry/common.c:80
         entry_SYSCALL_64_after_hwframe+0x44/0xae
      
      So fix this by making sure that whenever we try to modify the chunk btree
      and we are neither in a chunk allocation context nor in a chunk remove
      context, we reserve system space before modifying the chunk btree.
      Reported-by: default avatarHao Sun <sunhao.th@gmail.com>
      Link: https://lore.kernel.org/linux-btrfs/CACkBjsax51i4mu6C0C3vJqQN3NR_iVuucoeG3U1HXjrgzn5FFQ@mail.gmail.com/
      Fixes: 79bd3712 ("btrfs: rework chunk allocation to avoid exhaustion of the system chunk array")
      CC: stable@vger.kernel.org # 5.14+
      Reviewed-by: default avatarJosef Bacik <josef@toxicpanda.com>
      Signed-off-by: default avatarFilipe Manana <fdmanana@suse.com>
      Signed-off-by: default avatarDavid Sterba <dsterba@suse.com>
      2bb2e00e
    • Johannes Thumshirn's avatar
      btrfs: zoned: use greedy gc for auto reclaim · 2ca0ec77
      Johannes Thumshirn authored
      Currently auto reclaim of unusable zones reclaims the block-groups in
      the order they have been added to the reclaim list.
      
      Change this to a greedy algorithm by sorting the list so we have the
      block-groups with the least amount of valid bytes reclaimed first.
      
      Note: we can't splice the block groups from reclaim_bgs to let the sort
      happen outside of the lock. The block groups can be still in use by
      other parts eg. via bg_list and we must hold unused_bgs_lock while
      processing them.
      Signed-off-by: default avatarJohannes Thumshirn <johannes.thumshirn@wdc.com>
      Reviewed-by: default avatarDavid Sterba <dsterba@suse.com>
      [ write note and comment why we can't splice the list ]
      Signed-off-by: default avatarDavid Sterba <dsterba@suse.com>
      2ca0ec77
    • Christoph Hellwig's avatar
      btrfs: check-integrity: stop storing the block device name in btrfsic_dev_state · 813ebc16
      Christoph Hellwig authored
      Just use the %pg format specifier in all the debug printks previously
      using it.  Note that both bdevname and the %pg specifier never print
      a pathname, so the kbasename call wasn't needed to start with.
      Reviewed-by: default avatarNikolay Borisov <nborisov@suse.com>
      Signed-off-by: default avatarChristoph Hellwig <hch@lst.de>
      Reviewed-by: default avatarDavid Sterba <dsterba@suse.com>
      [ adjust messages and indentation ]
      Signed-off-by: default avatarDavid Sterba <dsterba@suse.com>
      813ebc16
    • Josef Bacik's avatar
      btrfs: use btrfs_get_dev_args_from_path in dev removal ioctls · 1a15eb72
      Josef Bacik authored
      For device removal and replace we call btrfs_find_device_by_devspec,
      which if we give it a device path and nothing else will call
      btrfs_get_dev_args_from_path, which opens the block device and reads the
      super block and then looks up our device based on that.
      
      However at this point we're holding the sb write "lock", so reading the
      block device pulls in the dependency of ->open_mutex, which produces the
      following lockdep splat
      
      ======================================================
      WARNING: possible circular locking dependency detected
      5.14.0-rc2+ #405 Not tainted
      ------------------------------------------------------
      losetup/11576 is trying to acquire lock:
      ffff9bbe8cded938 ((wq_completion)loop0){+.+.}-{0:0}, at: flush_workqueue+0x67/0x5e0
      
      but task is already holding lock:
      ffff9bbe88e4fc68 (&lo->lo_mutex){+.+.}-{3:3}, at: __loop_clr_fd+0x41/0x660 [loop]
      
      which lock already depends on the new lock.
      
      the existing dependency chain (in reverse order) is:
      
      -> #4 (&lo->lo_mutex){+.+.}-{3:3}:
             __mutex_lock+0x7d/0x750
             lo_open+0x28/0x60 [loop]
             blkdev_get_whole+0x25/0xf0
             blkdev_get_by_dev.part.0+0x168/0x3c0
             blkdev_open+0xd2/0xe0
             do_dentry_open+0x161/0x390
             path_openat+0x3cc/0xa20
             do_filp_open+0x96/0x120
             do_sys_openat2+0x7b/0x130
             __x64_sys_openat+0x46/0x70
             do_syscall_64+0x38/0x90
             entry_SYSCALL_64_after_hwframe+0x44/0xae
      
      -> #3 (&disk->open_mutex){+.+.}-{3:3}:
             __mutex_lock+0x7d/0x750
             blkdev_get_by_dev.part.0+0x56/0x3c0
             blkdev_get_by_path+0x98/0xa0
             btrfs_get_bdev_and_sb+0x1b/0xb0
             btrfs_find_device_by_devspec+0x12b/0x1c0
             btrfs_rm_device+0x127/0x610
             btrfs_ioctl+0x2a31/0x2e70
             __x64_sys_ioctl+0x80/0xb0
             do_syscall_64+0x38/0x90
             entry_SYSCALL_64_after_hwframe+0x44/0xae
      
      -> #2 (sb_writers#12){.+.+}-{0:0}:
             lo_write_bvec+0xc2/0x240 [loop]
             loop_process_work+0x238/0xd00 [loop]
             process_one_work+0x26b/0x560
             worker_thread+0x55/0x3c0
             kthread+0x140/0x160
             ret_from_fork+0x1f/0x30
      
      -> #1 ((work_completion)(&lo->rootcg_work)){+.+.}-{0:0}:
             process_one_work+0x245/0x560
             worker_thread+0x55/0x3c0
             kthread+0x140/0x160
             ret_from_fork+0x1f/0x30
      
      -> #0 ((wq_completion)loop0){+.+.}-{0:0}:
             __lock_acquire+0x10ea/0x1d90
             lock_acquire+0xb5/0x2b0
             flush_workqueue+0x91/0x5e0
             drain_workqueue+0xa0/0x110
             destroy_workqueue+0x36/0x250
             __loop_clr_fd+0x9a/0x660 [loop]
             block_ioctl+0x3f/0x50
             __x64_sys_ioctl+0x80/0xb0
             do_syscall_64+0x38/0x90
             entry_SYSCALL_64_after_hwframe+0x44/0xae
      
      other info that might help us debug this:
      
      Chain exists of:
        (wq_completion)loop0 --> &disk->open_mutex --> &lo->lo_mutex
      
       Possible unsafe locking scenario:
      
             CPU0                    CPU1
             ----                    ----
        lock(&lo->lo_mutex);
                                     lock(&disk->open_mutex);
                                     lock(&lo->lo_mutex);
        lock((wq_completion)loop0);
      
       *** DEADLOCK ***
      
      1 lock held by losetup/11576:
       #0: ffff9bbe88e4fc68 (&lo->lo_mutex){+.+.}-{3:3}, at: __loop_clr_fd+0x41/0x660 [loop]
      
      stack backtrace:
      CPU: 0 PID: 11576 Comm: losetup Not tainted 5.14.0-rc2+ #405
      Hardware name: QEMU Standard PC (Q35 + ICH9, 2009), BIOS 1.13.0-2.fc32 04/01/2014
      Call Trace:
       dump_stack_lvl+0x57/0x72
       check_noncircular+0xcf/0xf0
       ? stack_trace_save+0x3b/0x50
       __lock_acquire+0x10ea/0x1d90
       lock_acquire+0xb5/0x2b0
       ? flush_workqueue+0x67/0x5e0
       ? lockdep_init_map_type+0x47/0x220
       flush_workqueue+0x91/0x5e0
       ? flush_workqueue+0x67/0x5e0
       ? verify_cpu+0xf0/0x100
       drain_workqueue+0xa0/0x110
       destroy_workqueue+0x36/0x250
       __loop_clr_fd+0x9a/0x660 [loop]
       ? blkdev_ioctl+0x8d/0x2a0
       block_ioctl+0x3f/0x50
       __x64_sys_ioctl+0x80/0xb0
       do_syscall_64+0x38/0x90
       entry_SYSCALL_64_after_hwframe+0x44/0xae
      RIP: 0033:0x7f31b02404cb
      
      Instead what we want to do is populate our device lookup args before we
      grab any locks, and then pass these args into btrfs_rm_device().  From
      there we can find the device and do the appropriate removal.
      Suggested-by: default avatarAnand Jain <anand.jain@oracle.com>
      Reviewed-by: default avatarAnand Jain <anand.jain@oracle.com>
      Signed-off-by: default avatarJosef Bacik <josef@toxicpanda.com>
      Signed-off-by: default avatarDavid Sterba <dsterba@suse.com>
      1a15eb72
    • Josef Bacik's avatar
      btrfs: add a btrfs_get_dev_args_from_path helper · faa775c4
      Josef Bacik authored
      We are going to want to populate our device lookup args outside of any
      locks and then do the actual device lookup later, so add a helper to do
      this work and make btrfs_find_device_by_devspec() use this helper for
      now.
      Reviewed-by: default avatarNikolay Borisov <nborisov@suse.com>
      Reviewed-by: default avatarAnand Jain <anand.jain@oracle.com>
      Signed-off-by: default avatarJosef Bacik <josef@toxicpanda.com>
      Signed-off-by: default avatarDavid Sterba <dsterba@suse.com>
      faa775c4
    • Josef Bacik's avatar
      btrfs: handle device lookup with btrfs_dev_lookup_args · 562d7b15
      Josef Bacik authored
      We have a lot of device lookup functions that all do something slightly
      different.  Clean this up by adding a struct to hold the different
      lookup criteria, and then pass this around to btrfs_find_device() so it
      can do the proper matching based on the lookup criteria.
      Reviewed-by: default avatarAnand Jain <anand.jain@oracle.com>
      Signed-off-by: default avatarJosef Bacik <josef@toxicpanda.com>
      Reviewed-by: default avatarDavid Sterba <dsterba@suse.com>
      Signed-off-by: default avatarDavid Sterba <dsterba@suse.com>
      562d7b15
    • Josef Bacik's avatar
      btrfs: do not call close_fs_devices in btrfs_rm_device · 8b41393f
      Josef Bacik authored
      There's a subtle case where if we're removing the seed device from a
      file system we need to free its private copy of the fs_devices.  However
      we do not need to call close_fs_devices(), because at this point there
      are no devices left to close as we've closed the last one.  The only
      thing that close_fs_devices() does is decrement ->opened, which should
      be 1.  We want to avoid calling close_fs_devices() here because it has a
      lockdep_assert_held(&uuid_mutex), and we are going to stop holding the
      uuid_mutex in this path.
      
      So simply decrement the  ->opened counter like we should, and then clean
      up like normal.  Also add a comment explaining what we're doing here as
      I initially removed this code erroneously.
      Reviewed-by: default avatarAnand Jain <anand.jain@oracle.com>
      Signed-off-by: default avatarJosef Bacik <josef@toxicpanda.com>
      Signed-off-by: default avatarDavid Sterba <dsterba@suse.com>
      8b41393f
    • Anand Jain's avatar
      btrfs: add comments for device counts in struct btrfs_fs_devices · add9745a
      Anand Jain authored
      A bug was was checking a wrong device count before we delete the struct
      btrfs_fs_devices in btrfs_rm_device(). To avoid future confusion and
      easy reference add a comment about the various device counts that we have
      in the struct btrfs_fs_devices.
      Signed-off-by: default avatarAnand Jain <anand.jain@oracle.com>
      Signed-off-by: default avatarDavid Sterba <dsterba@suse.com>
      add9745a
    • Anand Jain's avatar
      btrfs: use num_device to check for the last surviving seed device · 8e906945
      Anand Jain authored
      For both sprout and seed fsids,
       btrfs_fs_devices::num_devices provides device count including missing
       btrfs_fs_devices::open_devices provides device count excluding missing
      
      We create a dummy struct btrfs_device for the missing device, so
      num_devices != open_devices when there is a missing device.
      
      In btrfs_rm_devices() we wrongly check for %cur_devices->open_devices
      before freeing the seed fs_devices. Instead we should check for
      %cur_devices->num_devices.
      Signed-off-by: default avatarAnand Jain <anand.jain@oracle.com>
      Signed-off-by: default avatarDavid Sterba <dsterba@suse.com>
      8e906945
    • Filipe Manana's avatar
      btrfs: fix lost error handling when replaying directory deletes · 10adb115
      Filipe Manana authored
      At replay_dir_deletes(), if find_dir_range() returns an error we break out
      of the main while loop and then assign a value of 0 (success) to the 'ret'
      variable, resulting in completely ignoring that an error happened. Fix
      that by jumping to the 'out' label when find_dir_range() returns an error
      (negative value).
      
      CC: stable@vger.kernel.org # 4.4+
      Reviewed-by: default avatarJosef Bacik <josef@toxicpanda.com>
      Signed-off-by: default avatarFilipe Manana <fdmanana@suse.com>
      Signed-off-by: default avatarDavid Sterba <dsterba@suse.com>
      10adb115
    • Qu Wenruo's avatar
      btrfs: remove btrfs_bio::logical member · f4f39fc5
      Qu Wenruo authored
      The member btrfs_bio::logical is only initialized by two call sites:
      
      - btrfs_repair_one_sector()
        No corresponding site to utilize it.
      
      - btrfs_submit_direct()
        The corresponding site to utilize it is btrfs_check_read_dio_bio().
      
      However for btrfs_check_read_dio_bio(), we can grab the file_offset from
      btrfs_dio_private::file_offset directly.
      
      Thus it turns out we don't really need that btrfs_bio::logical member at
      all.
      
      For btrfs_bio, the logical bytenr can be fetched from its
      bio->bi_iter.bi_sector directly.
      
      So let's just remove the member to save 8 bytes for structure btrfs_bio.
      Signed-off-by: default avatarQu Wenruo <wqu@suse.com>
      Signed-off-by: default avatarDavid Sterba <dsterba@suse.com>
      f4f39fc5
    • Qu Wenruo's avatar
      btrfs: rename btrfs_dio_private::logical_offset to file_offset · 47926ab5
      Qu Wenruo authored
      The naming of "logical_offset" can be confused with logical bytenr of
      the dio range.
      
      In fact it's file offset, and the naming "file_offset" is already widely
      used in all other sites.
      
      Just do the rename to avoid confusion.
      Signed-off-by: default avatarQu Wenruo <wqu@suse.com>
      Reviewed-by: default avatarDavid Sterba <dsterba@suse.com>
      Signed-off-by: default avatarDavid Sterba <dsterba@suse.com>
      47926ab5
    • Christoph Hellwig's avatar
      btrfs: use bvec_kmap_local in btrfs_csum_one_bio · 3dcfbcce
      Christoph Hellwig authored
      Using local kmaps slightly reduces the chances to stray writes, and
      the bvec interface cleans up the code a little bit.
      Reviewed-by: default avatarNikolay Borisov <nborisov@suse.com>
      Signed-off-by: default avatarChristoph Hellwig <hch@lst.de>
      Reviewed-by: default avatarDavid Sterba <dsterba@suse.com>
      Signed-off-by: default avatarDavid Sterba <dsterba@suse.com>
      3dcfbcce
    • Anand Jain's avatar
      btrfs: reduce btrfs_update_block_group alloc argument to bool · 11b66fa6
      Anand Jain authored
      btrfs_update_block_group() accounts for the number of bytes allocated or
      freed. Argument @alloc specifies whether the call is for alloc or free.
      Convert the argument @alloc type from int to bool.
      Reviewed-by: default avatarSu Yue <l@damenly.su>
      Signed-off-by: default avatarAnand Jain <anand.jain@oracle.com>
      Reviewed-by: default avatarDavid Sterba <dsterba@suse.com>
      Signed-off-by: default avatarDavid Sterba <dsterba@suse.com>
      11b66fa6
    • Nikolay Borisov's avatar
      btrfs: make btrfs_ref::real_root optional · eed2037f
      Nikolay Borisov authored
      Now that real_root is only used in ref-verify core gate it behind
      CONFIG_BTRFS_FS_REF_VERIFY ifdef. This shrinks the size of pending
      delayed refs by 8 bytes per ref, of which we can have many at any one
      time depending on intensity of the workload. Also change the comment
      about the member as it no longer deals with qgroups.
      Signed-off-by: default avatarNikolay Borisov <nborisov@suse.com>
      Reviewed-by: default avatarDavid Sterba <dsterba@suse.com>
      Signed-off-by: default avatarDavid Sterba <dsterba@suse.com>
      eed2037f
    • Nikolay Borisov's avatar
      btrfs: pull up qgroup checks from delayed-ref core to init time · 681145d4
      Nikolay Borisov authored
      Instead of checking whether qgroup processing for a dealyed ref has to
      happen in the core of delayed ref, simply pull the check at init time of
      respective delayed ref structures. This eliminates the final use of
      real_root in delayed-ref core paving the way to making this member
      optional.
      Signed-off-by: default avatarNikolay Borisov <nborisov@suse.com>
      Reviewed-by: default avatarDavid Sterba <dsterba@suse.com>
      Signed-off-by: default avatarDavid Sterba <dsterba@suse.com>
      681145d4
    • Nikolay Borisov's avatar
      btrfs: add additional parameters to btrfs_init_tree_ref/btrfs_init_data_ref · f42c5da6
      Nikolay Borisov authored
      In order to make 'real_root' used only in ref-verify it's required to
      have the necessary context to perform the same checks that this member
      is used for. So add 'mod_root' which will contain the root on behalf of
      which a delayed ref was created and a 'skip_group' parameter which
      will contain callsite-specific override of skip_qgroup.
      Signed-off-by: default avatarNikolay Borisov <nborisov@suse.com>
      Reviewed-by: default avatarDavid Sterba <dsterba@suse.com>
      Signed-off-by: default avatarDavid Sterba <dsterba@suse.com>
      f42c5da6
    • Nikolay Borisov's avatar
      btrfs: rely on owning_root field in btrfs_add_delayed_tree_ref to detect CHUNK_ROOT · d55b9e68
      Nikolay Borisov authored
      The real_root field is going to be used only by ref-verify tool so limit
      its use outside of it. Blocks belonging to the chunk root will always
      have it as an owner so the check is equivalent.
      Signed-off-by: default avatarNikolay Borisov <nborisov@suse.com>
      Reviewed-by: default avatarDavid Sterba <dsterba@suse.com>
      Signed-off-by: default avatarDavid Sterba <dsterba@suse.com>
      d55b9e68
    • Nikolay Borisov's avatar
      btrfs: rename root fields in delayed refs structs · 113479d5
      Nikolay Borisov authored
      Both data and metadata delayed ref structures have fields named
      root/ref_root respectively. Those are somewhat cryptic and don't really
      convey the real meaning. In fact those roots are really the original
      owners of the respective block (i.e in case of a snapshot a data delayed
      ref will contain the original root that owns the given block). Rename
      those fields accordingly and adjust comments.
      Signed-off-by: default avatarNikolay Borisov <nborisov@suse.com>
      Reviewed-by: default avatarDavid Sterba <dsterba@suse.com>
      Signed-off-by: default avatarDavid Sterba <dsterba@suse.com>
      113479d5
    • Josef Bacik's avatar
      btrfs: do not infinite loop in data reclaim if we aborted · 0e24f6d8
      Josef Bacik authored
      Error injection stressing uncovered a busy loop in our data reclaim
      loop.  There are two cases here, one where we loop creating block groups
      until space_info->full is set, or in the main loop we will skip erroring
      out any tickets if space_info->full == 0.  Unfortunately if we aborted
      the transaction then we will never allocate chunks or reclaim any space
      and thus never get ->full, and you'll see stack traces like this:
      
        watchdog: BUG: soft lockup - CPU#0 stuck for 26s! [kworker/u4:4:139]
        CPU: 0 PID: 139 Comm: kworker/u4:4 Tainted: G        W         5.13.0-rc1+ #328
        Hardware name: QEMU Standard PC (Q35 + ICH9, 2009), BIOS 1.13.0-2.fc32 04/01/2014
        Workqueue: events_unbound btrfs_async_reclaim_data_space
        RIP: 0010:btrfs_join_transaction+0x12/0x20
        RSP: 0018:ffffb2b780b77de0 EFLAGS: 00000246
        RAX: ffffb2b781863d58 RBX: 0000000000000000 RCX: 0000000000000000
        RDX: 0000000000000801 RSI: ffff987952b57400 RDI: ffff987940aa3000
        RBP: ffff987954d55000 R08: 0000000000000001 R09: ffff98795539e8f0
        R10: 000000000000000f R11: 000000000000000f R12: ffffffffffffffff
        R13: ffff987952b574c8 R14: ffff987952b57400 R15: 0000000000000008
        FS:  0000000000000000(0000) GS:ffff9879bbc00000(0000) knlGS:0000000000000000
        CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
        CR2: 00007f0703da4000 CR3: 0000000113398004 CR4: 0000000000370ef0
        Call Trace:
         flush_space+0x4a8/0x660
         btrfs_async_reclaim_data_space+0x55/0x130
         process_one_work+0x1e9/0x380
         worker_thread+0x53/0x3e0
         ? process_one_work+0x380/0x380
         kthread+0x118/0x140
         ? __kthread_bind_mask+0x60/0x60
         ret_from_fork+0x1f/0x30
      
      Fix this by checking to see if we have a btrfs fs error in either of the
      reclaim loops, and if so fail the tickets and bail.  In addition to
      this, fix maybe_fail_all_tickets() to not try to grant tickets if we've
      aborted, simply fail everything.
      Reviewed-by: default avatarNikolay Borisov <nborisov@suse.com>
      Reviewed-by: default avatarFilipe Manana <fdmanana@suse.com>
      Signed-off-by: default avatarJosef Bacik <josef@toxicpanda.com>
      Signed-off-by: default avatarDavid Sterba <dsterba@suse.com>
      0e24f6d8
    • Josef Bacik's avatar
      btrfs: add a BTRFS_FS_ERROR helper · 84961539
      Josef Bacik authored
      We have a few flags that are inconsistently used to describe the fs in
      different states of failure.  As of 5963ffca ("btrfs: always abort
      the transaction if we abort a trans handle") we will always set
      BTRFS_FS_STATE_ERROR if we abort, so we don't have to check both ABORTED
      and ERROR to see if things have gone wrong.  Add a helper to check
      BTRFS_FS_STATE_ERROR and then convert all checkers of FS_STATE_ERROR to
      use the helper.
      
      The TRANS_ABORTED bit check was added in af722733 ("Btrfs: clean up
      resources during umount after trans is aborted") but is not actually
      specific.
      Reviewed-by: default avatarAnand Jain <anand.jain@oracle.com>
      Reviewed-by: default avatarNikolay Borisov <nborisov@suse.com>
      Signed-off-by: default avatarJosef Bacik <josef@toxicpanda.com>
      Reviewed-by: default avatarDavid Sterba <dsterba@suse.com>
      Signed-off-by: default avatarDavid Sterba <dsterba@suse.com>
      84961539
    • Josef Bacik's avatar
      btrfs: change error handling for btrfs_delete_*_in_log · 9a35fc95
      Josef Bacik authored
      Currently we will abort the transaction if we get a random error (like
      -EIO) while trying to remove the directory entries from the root log
      during rename.
      
      However since these are simply log tree related errors, we can mark the
      trans as needing a full commit.  Then if the error was truly
      catastrophic we'll hit it during the normal commit and abort as
      appropriate.
      Reviewed-by: default avatarNikolay Borisov <nborisov@suse.com>
      Reviewed-by: default avatarFilipe Manana <fdmanana@suse.com>
      Signed-off-by: default avatarJosef Bacik <josef@toxicpanda.com>
      Signed-off-by: default avatarDavid Sterba <dsterba@suse.com>
      9a35fc95
    • Josef Bacik's avatar
      btrfs: change handle_fs_error in recover_log_trees to aborts · ba51e2a1
      Josef Bacik authored
      During inspection of the return path for replay I noticed that we don't
      actually abort the transaction if we get a failure during replay.  This
      isn't a problem necessarily, as we properly return the error and will
      fail to mount.  However we still leave this dangling transaction that
      could conceivably be committed without thinking there was an error.
      
      We were using btrfs_handle_fs_error() here, but that pre-dates the
      transaction abort code.  Simply replace the btrfs_handle_fs_error()
      calls with transaction aborts, so we still know where exactly things
      went wrong, and add a few in some other un-handled error cases.
      Reviewed-by: default avatarNikolay Borisov <nborisov@suse.com>
      Reviewed-by: default avatarJohannes Thumshirn <johannes.thumshirn@wdc.com>
      Signed-off-by: default avatarJosef Bacik <josef@toxicpanda.com>
      Reviewed-by: default avatarDavid Sterba <dsterba@suse.com>
      Signed-off-by: default avatarDavid Sterba <dsterba@suse.com>
      ba51e2a1