1. 08 Feb, 2021 40 commits
    • Qu Wenruo's avatar
      btrfs: introduce the skeleton of btrfs_subpage structure · cac06d84
      Qu Wenruo authored
      For sectorsize < page size support, we need a structure to record extra
      status info for each sector of a page.
      
      Introduce the skeleton structure, all subpage related code would go to
      subpage.[ch].
      Reviewed-by: default avatarJosef Bacik <josef@toxicpanda.com>
      Signed-off-by: default avatarQu Wenruo <wqu@suse.com>
      Reviewed-by: default avatarDavid Sterba <dsterba@suse.com>
      Signed-off-by: default avatarDavid Sterba <dsterba@suse.com>
      cac06d84
    • Qu Wenruo's avatar
      btrfs: set UNMAPPED bit early in btrfs_clone_extent_buffer() for subpage support · 62c053fb
      Qu Wenruo authored
      For the incoming subpage support, UNMAPPED extent buffer will have
      different behavior in btrfs_release_extent_buffer().
      
      This means we need to set UNMAPPED bit early before calling
      btrfs_release_extent_buffer().
      
      Currently there is only one caller which relies on
      btrfs_release_extent_buffer() in its error path while set UNMAPPED bit
      late:
      - btrfs_clone_extent_buffer()
      
      Make it subpage compatible by setting the UNMAPPED bit early, since
      we're here, also move the UPTODATE bit early.
      
      There is another caller, __alloc_dummy_extent_buffer(), setting
      UNMAPPED bit late, but that function clean up the allocated page
      manually, thus no need for any modification.
      Reviewed-by: default avatarJosef Bacik <josef@toxicpanda.com>
      Signed-off-by: default avatarQu Wenruo <wqu@suse.com>
      Reviewed-by: default avatarDavid Sterba <dsterba@suse.com>
      Signed-off-by: default avatarDavid Sterba <dsterba@suse.com>
      62c053fb
    • Qu Wenruo's avatar
      btrfs: merge PAGE_CLEAR_DIRTY and PAGE_SET_WRITEBACK to PAGE_START_WRITEBACK · 6869b0a8
      Qu Wenruo authored
      PAGE_CLEAR_DIRTY and PAGE_SET_WRITEBACK are two defines used in
      __process_pages_contig(), to let the function know to clear page dirty
      bit and then set page writeback.
      
      However page writeback and dirty bits are conflicting (at least for
      sector size == PAGE_SIZE case), this means these two have to be always
      updated together.
      
      This means we can merge PAGE_CLEAR_DIRTY and PAGE_SET_WRITEBACK to
      PAGE_START_WRITEBACK.
      Reviewed-by: default avatarJosef Bacik <josef@toxicpanda.com>
      Signed-off-by: default avatarQu Wenruo <wqu@suse.com>
      Reviewed-by: default avatarDavid Sterba <dsterba@suse.com>
      Signed-off-by: default avatarDavid Sterba <dsterba@suse.com>
      6869b0a8
    • Filipe Manana's avatar
      btrfs: make concurrent fsyncs wait less when waiting for a transaction commit · d0c2f4fa
      Filipe Manana authored
      Often an fsync needs to fallback to a transaction commit for several
      reasons (to ensure consistency after a power failure, a new block group
      was allocated or a temporary error such as ENOMEM or ENOSPC happened).
      
      In that case the log is marked as needing a full commit and any concurrent
      tasks attempting to log inodes or commit the log will also fallback to the
      transaction commit. When this happens they all wait for the task that first
      started the transaction commit to finish the transaction commit - however
      they wait until the full transaction commit happens, which is not needed,
      as they only need to wait for the superblocks to be persisted and not for
      unpinning all the extents pinned during the transaction's lifetime, which
      even for short lived transactions can be a few thousand and take some
      significant amount of time to complete - for dbench workloads I have
      observed up to 4~5 milliseconds of time spent unpinning extents in the
      worst cases, and the number of pinned extents was between 2 to 3 thousand.
      
      So allow fsync tasks to skip waiting for the unpinning of extents when
      they call btrfs_commit_transaction() and they were not the task that
      started the transaction commit (that one has to do it, the alternative
      would be to offload the transaction commit to another task so that it
      could avoid waiting for the extent unpinning or offload the extent
      unpinning to another task).
      
      This patch is part of a patchset comprised of the following patches:
      
        btrfs: remove unnecessary directory inode item update when deleting dir entry
        btrfs: stop setting nbytes when filling inode item for logging
        btrfs: avoid logging new ancestor inodes when logging new inode
        btrfs: skip logging directories already logged when logging all parents
        btrfs: skip logging inodes already logged when logging new entries
        btrfs: remove unnecessary check_parent_dirs_for_sync()
        btrfs: make concurrent fsyncs wait less when waiting for a transaction commit
      
      After applying the entire patchset, dbench shows improvements in respect
      to throughput and latency. The script used to measure it is the following:
      
        $ cat dbench-test.sh
        #!/bin/bash
      
        DEV=/dev/sdk
        MNT=/mnt/sdk
        MOUNT_OPTIONS="-o ssd"
        MKFS_OPTIONS="-m single -d single"
      
        echo "performance" | tee /sys/devices/system/cpu/cpu*/cpufreq/scaling_governor
      
        umount $DEV &> /dev/null
        mkfs.btrfs -f $MKFS_OPTIONS $DEV
        mount $MOUNT_OPTIONS $DEV $MNT
      
        dbench -D $MNT -t 300 64
      
        umount $MNT
      
      The test was run on a physical machine with 12 cores (Intel corei7), 64G
      of ram, using a NVMe device and a non-debug kernel configuration (Debian's
      default configuration).
      
      Before applying patchset, 32 clients:
      
       Operation      Count    AvgLat    MaxLat
       ----------------------------------------
       NTCreateX    9627107     0.153    61.938
       Close        7072076     0.001     3.175
       Rename        407633     1.222    44.439
       Unlink       1943895     0.658    44.440
       Deltree          256    17.339   110.891
       Mkdir            128     0.003     0.009
       Qpathinfo    8725406     0.064    17.850
       Qfileinfo    1529516     0.001     2.188
       Qfsinfo      1599884     0.002     1.457
       Sfileinfo     784200     0.005     3.562
       Find         3373513     0.411    30.312
       WriteX       4802132     0.053    29.054
       ReadX       15089959     0.002     5.801
       LockX          31344     0.002     0.425
       UnlockX        31344     0.001     0.173
       Flush         674724     5.952   341.830
      
      Throughput 1008.02 MB/sec  32 clients  32 procs  max_latency=341.833 ms
      
      After applying patchset, 32 clients:
      
      After patchset, with 32 clients:
      
       Operation      Count    AvgLat    MaxLat
       ----------------------------------------
       NTCreateX    9931568     0.111    25.597
       Close        7295730     0.001     2.171
       Rename        420549     0.982    49.714
       Unlink       2005366     0.497    39.015
       Deltree          256    11.149    89.242
       Mkdir            128     0.002     0.014
       Qpathinfo    9001863     0.049    20.761
       Qfileinfo    1577730     0.001     2.546
       Qfsinfo      1650508     0.002     3.531
       Sfileinfo     809031     0.005     5.846
       Find         3480259     0.309    23.977
       WriteX       4952505     0.043    41.283
       ReadX       15568127     0.002     5.476
       LockX          32338     0.002     0.978
       UnlockX        32338     0.001     2.032
       Flush         696017     7.485   228.835
      
      Throughput 1049.91 MB/sec  32 clients  32 procs  max_latency=228.847 ms
      
       --> +4.1% throughput, -39.6% max latency
      
      Before applying patchset, 64 clients:
      
       Operation      Count    AvgLat    MaxLat
       ----------------------------------------
       NTCreateX    8956748     0.342   108.312
       Close        6579660     0.001     3.823
       Rename        379209     2.396    81.897
       Unlink       1808625     1.108   131.148
       Deltree          256    25.632   172.176
       Mkdir            128     0.003     0.018
       Qpathinfo    8117615     0.131    55.916
       Qfileinfo    1423495     0.001     2.635
       Qfsinfo      1488496     0.002     5.412
       Sfileinfo     729472     0.007     8.643
       Find         3138598     0.855    78.321
       WriteX       4470783     0.102    79.442
       ReadX       14038139     0.002     7.578
       LockX          29158     0.002     0.844
       UnlockX        29158     0.001     0.567
       Flush         627746    14.168   506.151
      
      Throughput 924.738 MB/sec  64 clients  64 procs  max_latency=506.154 ms
      
      After applying patchset, 64 clients:
      
       Operation      Count    AvgLat    MaxLat
       ----------------------------------------
       NTCreateX    9069003     0.303    43.193
       Close        6662328     0.001     3.888
       Rename        383976     2.194    46.418
       Unlink       1831080     1.022    43.873
       Deltree          256    24.037   155.763
       Mkdir            128     0.002     0.005
       Qpathinfo    8219173     0.137    30.233
       Qfileinfo    1441203     0.001     3.204
       Qfsinfo      1507092     0.002     4.055
       Sfileinfo     738775     0.006     5.431
       Find         3177874     0.936    38.170
       WriteX       4526152     0.084    39.518
       ReadX       14213562     0.002    24.760
       LockX          29522     0.002     1.221
       UnlockX        29522     0.001     0.694
       Flush         635652    14.358   422.039
      
      Throughput 990.13 MB/sec  64 clients  64 procs  max_latency=422.043 ms
      
       --> +6.8% throughput, -18.1% max latency
      Reviewed-by: default avatarJosef Bacik <josef@toxicpanda.com>
      Signed-off-by: default avatarFilipe Manana <fdmanana@suse.com>
      Signed-off-by: default avatarDavid Sterba <dsterba@suse.com>
      d0c2f4fa
    • Filipe Manana's avatar
      btrfs: remove unnecessary check_parent_dirs_for_sync() · 64d6b281
      Filipe Manana authored
      Whenever we fsync an inode, if it is a directory, a regular file that was
      created in the current transaction or has last_unlink_trans set to the
      generation of the current transaction, we check if any of its ancestor
      inodes (and the inode itself if it is a directory) can not be logged and
      need a fallback to a full transaction commit - if so, we return with a
      value of 1 in order to fallback to a transaction commit.
      
      However we often do not need to fallback to a transaction commit because:
      
      1) The ancestor inode is not an immediate parent, and therefore there is
         not an explicit request to log it and it is not needed neither to
         guarantee the consistency of the inode originally asked to be logged
         (fsynced) nor its immediate parent;
      
      2) The ancestor inode was already logged before, in which case any link,
         unlink or rename operation updates the log as needed.
      
      So for these two cases we can avoid an unnecessary transaction commit.
      Therefore remove check_parent_dirs_for_sync() and add a check at the top
      of btrfs_log_inode() to make us fallback immediately to a transaction
      commit when we are logging a directory inode that can not be logged and
      needs a full transaction commit. All we need to protect is the case where
      after renaming a file someone fsyncs only the old directory, which would
      result is losing the renamed file after a log replay.
      
      This patch is part of a patchset comprised of the following patches:
      
        btrfs: remove unnecessary directory inode item update when deleting dir entry
        btrfs: stop setting nbytes when filling inode item for logging
        btrfs: avoid logging new ancestor inodes when logging new inode
        btrfs: skip logging directories already logged when logging all parents
        btrfs: skip logging inodes already logged when logging new entries
        btrfs: remove unnecessary check_parent_dirs_for_sync()
        btrfs: make concurrent fsyncs wait less when waiting for a transaction commit
      
      Performance results, after applying all patches, are mentioned in the
      change log of the last patch.
      Reviewed-by: default avatarJosef Bacik <josef@toxicpanda.com>
      Signed-off-by: default avatarFilipe Manana <fdmanana@suse.com>
      Signed-off-by: default avatarDavid Sterba <dsterba@suse.com>
      64d6b281
    • Filipe Manana's avatar
      btrfs: skip logging inodes already logged when logging new entries · 0e44cb3f
      Filipe Manana authored
      When logging new directory entries of a directory, we log the inodes of
      new dentries and the inodes of dentries pointing to directories that
      may have been created in past transactions. For the case of directories
      we log in full mode, which can be particularly expensive for large
      directories.
      
      We do use btrfs_inode_in_log() to skip already logged inodes, however for
      that helper to return true, it requires that the log transaction used to
      log the inode to be already committed. This means that when we have more
      than one task using the same log transaction we can end up logging an
      inode multiple times, which is a waste of time and not necessary since
      the log will be committed by one of the tasks and the others will wait for
      the log transaction to be committed before returning to user space.
      
      So simply replace the use of btrfs_inode_in_log() with the new helper
      function need_log_inode(), introduced in a previous commit.
      
      This patch is part of a patchset comprised of the following patches:
      
        btrfs: remove unnecessary directory inode item update when deleting dir entry
        btrfs: stop setting nbytes when filling inode item for logging
        btrfs: avoid logging new ancestor inodes when logging new inode
        btrfs: skip logging directories already logged when logging all parents
        btrfs: skip logging inodes already logged when logging new entries
        btrfs: remove unnecessary check_parent_dirs_for_sync()
        btrfs: make concurrent fsyncs wait less when waiting for a transaction commit
      
      Performance results, after applying all patches, are mentioned in the
      change log of the last patch.
      Reviewed-by: default avatarJosef Bacik <josef@toxicpanda.com>
      Signed-off-by: default avatarFilipe Manana <fdmanana@suse.com>
      Signed-off-by: default avatarDavid Sterba <dsterba@suse.com>
      0e44cb3f
    • Filipe Manana's avatar
      btrfs: skip logging directories already logged when logging all parents · 3e6a86a1
      Filipe Manana authored
      Some times when we fsync an inode we need to do a full log of all its
      ancestors (due to unlink, link or rename operations), which can be an
      expensive operation, specially if the directories are large.
      
      However if we find an ancestor directory inode that is already logged in
      the current transaction, and has no inserted/updated/deleted xattrs since
      it was last logged, we can skip logging the directory again. We are safe
      to skip that since we know that for logged directories, any link, unlink
      or rename operations that implicate the directory will update the log as
      necessary.
      
      So use the helper need_log_dir(), introduced in a previous commit, to
      detect already logged directories that can be skipped.
      
      This patch is part of a patchset comprised of the following patches:
      
        btrfs: remove unnecessary directory inode item update when deleting dir entry
        btrfs: stop setting nbytes when filling inode item for logging
        btrfs: avoid logging new ancestor inodes when logging new inode
        btrfs: skip logging directories already logged when logging all parents
        btrfs: skip logging inodes already logged when logging new entries
        btrfs: remove unnecessary check_parent_dirs_for_sync()
        btrfs: make concurrent fsyncs wait less when waiting for a transaction commit
      
      Performance results, after applying all patches, are mentioned in the
      change log of the last patch.
      Reviewed-by: default avatarJosef Bacik <josef@toxicpanda.com>
      Signed-off-by: default avatarFilipe Manana <fdmanana@suse.com>
      Signed-off-by: default avatarDavid Sterba <dsterba@suse.com>
      3e6a86a1
    • Filipe Manana's avatar
      btrfs: avoid logging new ancestor inodes when logging new inode · ab12313a
      Filipe Manana authored
      When we fsync a new file, created in the current transaction, we check
      all its ancestor inodes and always log them if they were created in the
      current transaction - even if we have already logged them before, which
      is a waste of time.
      
      So avoid logging new ancestor inodes if they were already logged before
      and have no xattrs added/updated/removed since they were last logged.
      
      This patch is part of a patchset comprised of the following patches:
      
        btrfs: remove unnecessary directory inode item update when deleting dir entry
        btrfs: stop setting nbytes when filling inode item for logging
        btrfs: avoid logging new ancestor inodes when logging new inode
        btrfs: skip logging directories already logged when logging all parents
        btrfs: skip logging inodes already logged when logging new entries
        btrfs: remove unnecessary check_parent_dirs_for_sync()
        btrfs: make concurrent fsyncs wait less when waiting for a transaction commit
      
      Performance results, after applying all patches, are mentioned in the
      change log of the last patch.
      Reviewed-by: default avatarJosef Bacik <josef@toxicpanda.com>
      Signed-off-by: default avatarFilipe Manana <fdmanana@suse.com>
      Signed-off-by: default avatarDavid Sterba <dsterba@suse.com>
      ab12313a
    • Filipe Manana's avatar
      btrfs: stop setting nbytes when filling inode item for logging · e593e54e
      Filipe Manana authored
      When we fill an inode item for logging we are setting its nbytes field
      with the value returned by inode_get_bytes() (a VFS API), however we do
      not need it because it is not used during log replay. In fact, for fast
      fsyncs, when we call inode_get_bytes() we may even get an outdated value
      for nbytes because the nbytes field of the inode is only updated when
      ordered extents complete, and a fast fsync only waits for writeback to
      complete, it does not wait for ordered extent completion.
      
      So just remove the setup of nbytes and add an explicit comment mentioning
      why we do not set it. This also avoids adding contention on the inode's
      i_lock (VFS) with concurrent stat() calls, since that spinlock is used by
      inode_get_bytes() which is also called by our stat callback
      (btrfs_getattr()).
      
      This patch is part of a patchset comprised of the following patches:
      
        btrfs: remove unnecessary directory inode item update when deleting dir entry
        btrfs: stop setting nbytes when filling inode item for logging
        btrfs: avoid logging new ancestor inodes when logging new inode
        btrfs: skip logging directories already logged when logging all parents
        btrfs: skip logging inodes already logged when logging new entries
        btrfs: remove unnecessary check_parent_dirs_for_sync()
        btrfs: make concurrent fsyncs wait less when waiting for a transaction commit
      
      Performance results, after applying all patches, are mentioned in the
      change log of the last patch.
      Reviewed-by: default avatarJosef Bacik <josef@toxicpanda.com>
      Signed-off-by: default avatarFilipe Manana <fdmanana@suse.com>
      Signed-off-by: default avatarDavid Sterba <dsterba@suse.com>
      e593e54e
    • Filipe Manana's avatar
      btrfs: remove unnecessary directory inode item update when deleting dir entry · ddffcf6f
      Filipe Manana authored
      When we remove a directory entry, as part of an unlink operation, if the
      directory was logged before we must remove the directory index items from
      the log. We are also updating the inode item of the directory to update
      its i_size, but that is not necessary because during log replay we do not
      need it and we correctly adjust the i_size in the inode item of the
      subvolume as we process directory index items and replay deletes.
      
      This is not needed since commit d555438b ("Btrfs: drop dir i_size
      when adding new names on replay"), where we explicitly ignore the i_size
      of directory inode items on log replay. Before that we used it but it
      was buggy as mentioned in that commit's change log (i_size got a larger
      value then it should have).
      
      So stop updating the i_size of the directory inode item in the log, as
      that is a waste of time, adds more log contention to the log tree and
      often results in COWing more extent buffers for the log tree.
      
      This code path is triggered often during dbench workloads for example.
      This patch is part of a patchset comprised of the following patches:
      
        btrfs: remove unnecessary directory inode item update when deleting dir entry
        btrfs: stop setting nbytes when filling inode item for logging
        btrfs: avoid logging new ancestor inodes when logging new inode
        btrfs: skip logging directories already logged when logging all parents
        btrfs: skip logging inodes already logged when logging new entries
        btrfs: remove unnecessary check_parent_dirs_for_sync()
        btrfs: make concurrent fsyncs wait less when waiting for a transaction commit
      
      Performance results, after applying all patches, are mentioned in the
      change log of the last patch.
      Reviewed-by: default avatarJosef Bacik <josef@toxicpanda.com>
      Signed-off-by: default avatarFilipe Manana <fdmanana@suse.com>
      Signed-off-by: default avatarDavid Sterba <dsterba@suse.com>
      ddffcf6f
    • Michal Rostecki's avatar
      btrfs: let callers of btrfs_get_io_geometry pass the em · 42034313
      Michal Rostecki authored
      Before this change, the btrfs_get_io_geometry() function was calling
      btrfs_get_chunk_map() to get the extent mapping, necessary for
      calculating the I/O geometry. It was using that extent mapping only
      internally and freeing the pointer after its execution.
      
      That resulted in calling btrfs_get_chunk_map() de facto twice by the
      __btrfs_map_block() function. It was calling btrfs_get_io_geometry()
      first and then calling btrfs_get_chunk_map() directly to get the extent
      mapping, used by the rest of the function.
      
      Change that to passing the extent mapping to the btrfs_get_io_geometry()
      function as an argument.
      
      This could improve performance in some cases.  For very large
      filesystems, i.e. several thousands of allocated chunks, not only this
      avoids searching two times the rbtree, saving time, it may also help
      reducing contention on the lock that protects the tree - thinking of
      writeback starting for multiple inodes, other tasks allocating or
      removing chunks, and anything else that requires access to the rbtree.
      Reviewed-by: default avatarFilipe Manana <fdmanana@suse.com>
      Signed-off-by: default avatarMichal Rostecki <mrostecki@suse.com>
      Reviewed-by: default avatarDavid Sterba <dsterba@suse.com>
      [ add Filipe's analysis ]
      Signed-off-by: default avatarDavid Sterba <dsterba@suse.com>
      42034313
    • Qu Wenruo's avatar
      btrfs: fix double accounting of ordered extent for subpage case in btrfs_invalidapge · 951c80f8
      Qu Wenruo authored
      Commit dbfdb6d1 ("Btrfs: Search for all ordered extents that could
      span across a page") make btrfs_invalidapage() to search all ordered
      extents.
      
      The offending code looks like this:
      
        again:
      	  start = page_start;
      	  ordered = btrfs_lookup_ordered_range(inode, start, page_end - start + 1);
      	  if (ordred) {
      		  end = min(page_end,
      			    ordered->file_offset + ordered->num_bytes - 1);
      
      		  /* Do the cleanup */
      
      		  start = end + 1;
      		  if (start < page_end)
      			  goto again;
      	  }
      
      The behavior is indeed necessary for the incoming subpage support, but
      when it iterates through all the ordered extents, it also resets the
      search range @start.
      
      This means, for the following cases, we can double account the ordered
      extents, causing its bytes_left underflow:
      
      	Page offset
      	0		16K		32K
      	|<--- OE 1  --->|<--- OE 2 ---->|
      
      As the first iteration will find ordered extent (OE) 1, which doesn't
      cover the full page, thus after cleanup code, we need to retry again.
      But again label will reset start to page_start, and we got OE 1 again,
      which causes double accounting on OE 1, and cause OE 1's byte_left to
      underflow.
      
      This problem can only happen for subpage case, as for regular sectorsize
      == PAGE_SIZE case, we will always find a OE ends at or after page end,
      thus no way to trigger the problem.
      
      Move the again label after start = page_start.  There will be more
      comprehensive rework to convert the open coded loop to a proper while
      loop for subpage support.
      
      Fixes: dbfdb6d1 ("Btrfs: Search for all ordered extents that could span across a page")
      Reviewed-by: default avatarFilipe Manana <fdmanana@suse.com>
      Signed-off-by: default avatarQu Wenruo <wqu@suse.com>
      Signed-off-by: default avatarDavid Sterba <dsterba@suse.com>
      951c80f8
    • Abaci Team's avatar
      btrfs: simplify condition in __btrfs_run_delayed_items · a4559e6f
      Abaci Team authored
      Fix the following coccicheck warnings:
      
      ./fs/btrfs/delayed-inode.c:1157:39-41: WARNING !A || A && B is
      equivalent to !A || B.
      Reported-by: default avatarAbaci Robot <abaci@linux.alibaba.com>
      Suggested-by: default avatarJiapeng Zhong <oswb@linux.alibaba.com>
      Reviewed-by: default avatarJosef Bacik <josef@toxicpanda.com>
      Signed-off-by: default avatarAbaci Team <abaci-bugfix@linux.alibaba.com>
      Reviewed-by: default avatarDavid Sterba <dsterba@suse.com>
      Signed-off-by: default avatarDavid Sterba <dsterba@suse.com>
      a4559e6f
    • Filipe Manana's avatar
      btrfs: remove wrong comment for can_nocow_extent() · 2965194b
      Filipe Manana authored
      The comment for can_nocow_extent() says that the function will flush
      ordered extents, however that never happens and was never true before the
      comment was added in commit e4ecaf90 ("btrfs: add comments for
      btrfs_check_can_nocow() and can_nocow_extent()"). This is true only for
      the function btrfs_check_can_nocow(), which after that commit was renamed
      to check_can_nocow(). So just remove that part of the comment.
      Reviewed-by: default avatarJosef Bacik <josef@toxicpanda.com>
      Signed-off-by: default avatarFilipe Manana <fdmanana@suse.com>
      Reviewed-by: default avatarDavid Sterba <dsterba@suse.com>
      Signed-off-by: default avatarDavid Sterba <dsterba@suse.com>
      2965194b
    • Josef Bacik's avatar
      btrfs: add a trace class for dumping the current ENOSPC state · e5ad49e2
      Josef Bacik authored
      Often when I'm debugging ENOSPC related issues I have to resort to
      printing the entire ENOSPC state with trace_printk() in different spots.
      This gets pretty annoying, so add a trace state that does this for us.
      Then add a trace point at the end of preemptive flushing so you can see
      the state of the space_info when we decide to exit preemptive flushing.
      This helped me figure out we weren't kicking in the preemptive flushing
      soon enough.
      Reviewed-by: default avatarNikolay Borisov <nborisov@suse.com>
      Signed-off-by: default avatarJosef Bacik <josef@toxicpanda.com>
      Reviewed-by: default avatarDavid Sterba <dsterba@suse.com>
      Signed-off-by: default avatarDavid Sterba <dsterba@suse.com>
      e5ad49e2
    • Josef Bacik's avatar
      btrfs: adjust the flush trace point to include the source · 4b02b00f
      Josef Bacik authored
      Since we have normal ticketed flushing and preemptive flushing, adjust
      the tracepoint so that we know the source of the flushing action to make
      it easier to debug problems.
      Reviewed-by: default avatarNikolay Borisov <nborisov@suse.com>
      Signed-off-by: default avatarJosef Bacik <josef@toxicpanda.com>
      Reviewed-by: default avatarDavid Sterba <dsterba@suse.com>
      Signed-off-by: default avatarDavid Sterba <dsterba@suse.com>
      4b02b00f
    • Josef Bacik's avatar
      btrfs: implement space clamping for preemptive flushing · 88a777a6
      Josef Bacik authored
      Starting preemptive flushing at 50% of available free space is a good
      start, but some workloads are particularly abusive and can quickly
      overwhelm the preemptive flushing code and drive us into using tickets.
      
      Handle this by clamping down on our threshold for starting and
      continuing to run preemptive flushing.  This is particularly important
      for our overcommit case, as we can really drive the file system into
      overages and then it's more difficult to pull it back as we start to
      actually fill up the file system.
      
      The clamping is essentially 2^CLAMP, but we start at 1 so whatever we
      calculate for overcommit is the baseline.
      Reviewed-by: default avatarNikolay Borisov <nborisov@suse.com>
      Signed-off-by: default avatarJosef Bacik <josef@toxicpanda.com>
      Signed-off-by: default avatarDavid Sterba <dsterba@suse.com>
      88a777a6
    • Josef Bacik's avatar
      btrfs: simplify the logic in need_preemptive_flushing · 2e294c60
      Josef Bacik authored
      A lot of this was added all in one go with no explanation, and is a bit
      unwieldy and confusing.  Simplify the logic to start preemptive flushing
      if we've reserved more than half of our available free space.
      Reviewed-by: default avatarNikolay Borisov <nborisov@suse.com>
      Signed-off-by: default avatarJosef Bacik <josef@toxicpanda.com>
      Signed-off-by: default avatarDavid Sterba <dsterba@suse.com>
      2e294c60
    • Josef Bacik's avatar
      btrfs: rework btrfs_calc_reclaim_metadata_size · 9f42d377
      Josef Bacik authored
      Currently btrfs_calc_reclaim_metadata_size does two things, it returns
      the space currently required for flushing by the tickets, and if there
      are no tickets it calculates a value for the preemptive flushing.
      
      However for the normal ticketed flushing we really only care about the
      space required for tickets.  We will accidentally come in and flush one
      time, but as soon as we see there are no tickets we bail out of our
      flushing.
      
      Fix this by making btrfs_calc_reclaim_metadata_size really only tell us
      what is required for flushing if we have people waiting on space.  Then
      move the preemptive flushing logic into need_preemptive_reclaim().  We
      ignore btrfs_calc_reclaim_metadata_size() in need_preemptive_reclaim()
      because if we are in this path then we made our reservation and there
      are not pending tickets currently, so we do not need to check it, simply
      do the fuzzy logic to check if we're getting low on space.
      Reviewed-by: default avatarNikolay Borisov <nborisov@suse.com>
      Signed-off-by: default avatarJosef Bacik <josef@toxicpanda.com>
      Signed-off-by: default avatarDavid Sterba <dsterba@suse.com>
      9f42d377
    • Josef Bacik's avatar
      btrfs: check reclaim_size in need_preemptive_reclaim · f205edf7
      Josef Bacik authored
      If we're flushing space for tickets then we have
      space_info->reclaim_size set and we do not need to do background
      reclaim.
      Reviewed-by: default avatarNikolay Borisov <nborisov@suse.com>
      Signed-off-by: default avatarJosef Bacik <josef@toxicpanda.com>
      Signed-off-by: default avatarDavid Sterba <dsterba@suse.com>
      f205edf7
    • Josef Bacik's avatar
      btrfs: rename need_do_async_reclaim · ae7913ba
      Josef Bacik authored
      All of our normal flushing is asynchronous reclaim, so this helper is
      poorly named.  This is more checking if we need to preemptively flush
      space, so rename it to need_preemptive_reclaim.
      
      Also switch it to bool and make it plain static as followup patches will
      move more code here.
      Reviewed-by: default avatarNikolay Borisov <nborisov@suse.com>
      Signed-off-by: default avatarJosef Bacik <josef@toxicpanda.com>
      Reviewed-by: default avatarDavid Sterba <dsterba@suse.com>
      Signed-off-by: default avatarDavid Sterba <dsterba@suse.com>
      ae7913ba
    • Josef Bacik's avatar
      btrfs: improve preemptive background space flushing · 576fa348
      Josef Bacik authored
      Currently if we ever have to flush space because we do not have enough
      we allocate a ticket and attach it to the space_info, and then
      systematically flush things in the filesystem that hold space
      reservations until our space is reclaimed.
      
      However this has a latency cost, we must go to sleep and wait for the
      flushing to make progress before we are woken up and allowed to continue
      doing our work.
      
      In order to address that we used to kick off the async worker to flush
      space preemptively, so that we could be reclaiming space hopefully
      before any tasks needed to stop and wait for space to reclaim.
      
      When I introduced the ticketed ENOSPC stuff this broke slightly in the
      fact that we were using tickets to indicate if we were done flushing.
      No tickets, no more flushing.  However this meant that we essentially
      never preemptively flushed.  This caused a write performance regression
      that Nikolay noticed in an unrelated patch that removed the committing
      of the transaction during btrfs_end_transaction.
      
      The behavior that happened pre that patch was btrfs_end_transaction()
      would see that we were low on space, and it would commit the
      transaction.  This was bad because in this particular case you could end
      up with thousands and thousands of transactions being committed during
      the 5 minute reproducer.  With the patch to remove this behavior we got
      much more sane transaction commits, but we ended up slower because we
      would write for a while, flush, write for a while, flush again.
      
      To address this we need to reinstate a preemptive flushing mechanism.
      However it is distinctly different from our ticketing flushing in that
      it doesn't have tickets to base it's decisions on.  Instead of bolting
      this logic into our existing flushing work, add another worker to handle
      this preemptive flushing.  Here we will attempt to be slightly
      intelligent about the things that we flushing, attempting to balance
      between whichever pool is taking up the most space.
      Reviewed-by: default avatarNikolay Borisov <nborisov@suse.com>
      Signed-off-by: default avatarJosef Bacik <josef@toxicpanda.com>
      Signed-off-by: default avatarDavid Sterba <dsterba@suse.com>
      576fa348
    • Josef Bacik's avatar
      btrfs: introduce a FORCE_COMMIT_TRANS flush operation · f00c42dd
      Josef Bacik authored
      Solely for preemptive flushing, we want to be able to force the
      transaction commit without any of the ambiguity of
      may_commit_transaction().  This is because may_commit_transaction()
      checks tickets and such, and in preemptive flushing we already know
      it'll be helpful, so use this to keep the code nice and clean and
      straightforward.
      Reviewed-by: default avatarNikolay Borisov <nborisov@suse.com>
      Signed-off-by: default avatarJosef Bacik <josef@toxicpanda.com>
      [ add comment ]
      Signed-off-by: default avatarDavid Sterba <dsterba@suse.com>
      f00c42dd
    • Josef Bacik's avatar
      btrfs: track ordered bytes instead of just dio ordered bytes · 5deb17e1
      Josef Bacik authored
      We track dio_bytes because the shrink delalloc code needs to know if we
      have more DIO in flight than we have normal buffered IO.  The reason for
      this is because we can't "flush" DIO, we have to just wait on the
      ordered extents to finish.
      
      However this is true of all ordered extents.  If we have more ordered
      space outstanding than dirty pages we should be waiting on ordered
      extents.  We already are ok on this front technically, because we always
      do a FLUSH_DELALLOC_WAIT loop, but I want to use the ordered counter in
      the preemptive flushing code as well, so change this to count all
      ordered bytes instead of just DIO ordered bytes.
      Reviewed-by: default avatarNikolay Borisov <nborisov@suse.com>
      Signed-off-by: default avatarJosef Bacik <josef@toxicpanda.com>
      Signed-off-by: default avatarDavid Sterba <dsterba@suse.com>
      5deb17e1
    • Josef Bacik's avatar
      btrfs: add a trace point for reserve tickets · ac1ea10e
      Josef Bacik authored
      While debugging a ENOSPC related performance problem I needed to see the
      time difference between start and end of a reserve ticket, so add a
      trace point to report when we handle a reserve ticket.
      
      I opted to spit out start_ns itself without calculating the difference
      because there could be a gap between enabling the tracepoint and setting
      start_ns.  Doing it this way allows us to filter on 0 start_ns so we
      don't get bogus entries, and we can easily calculate the time difference
      with bpftrace or something else.
      Reviewed-by: default avatarNikolay Borisov <nborisov@suse.com>
      Signed-off-by: default avatarJosef Bacik <josef@toxicpanda.com>
      Reviewed-by: default avatarDavid Sterba <dsterba@suse.com>
      Signed-off-by: default avatarDavid Sterba <dsterba@suse.com>
      ac1ea10e
    • Josef Bacik's avatar
      btrfs: make flush_space take a enum btrfs_flush_state instead of int · 91e79a83
      Josef Bacik authored
      I got a automated message from somebody who runs clang against our
      kernels and it's because I used the wrong enum type for what I passed
      into flush_space, caught by -Wenum-conversion.  Change the argument to
      be explicitly the enum we're expecting to make everything consistent.
      Maybe eventually gcc will catch errors like this.
      Reviewed-by: default avatarNikolay Borisov <nborisov@suse.com>
      Signed-off-by: default avatarJosef Bacik <josef@toxicpanda.com>
      Reviewed-by: default avatarDavid Sterba <dsterba@suse.com>
      Signed-off-by: default avatarDavid Sterba <dsterba@suse.com>
      91e79a83
    • Roman Anasal's avatar
      btrfs: send: use struct send_ctx *sctx for btrfs_compare_trees and changed_cb · 88980383
      Roman Anasal authored
      btrfs_compare_trees and changed_cb use a void *ctx parameter instead of
      struct send_ctx *sctx but when used in changed_cb it is immediately
      cast to `struct send_ctx *sctx = ctx;`.
      
      changed_cb is only ever called from btrfs_compare_trees and full_send_tree:
      - full_send_tree already passes a struct send_ctx *sctx
      - btrfs_compare_trees is only called by send_subvol with a struct send_ctx *sctx
      - void *ctx in btrfs_compare_trees is only used to be passed to changed_cb
      
      So casting to/from void *ctx seems unnecessary and directly using
      struct send_ctx *sctx instead provides better type-safety.
      
      The original reason for using void *ctx in the first place seems to have
      been dropped with 1b51d6fc ("btrfs: send: remove indirect callback
      parameter for changed_cb").
      Signed-off-by: default avatarRoman Anasal <roman.anasal@bdsu.de>
      Reviewed-by: default avatarDavid Sterba <dsterba@suse.com>
      Signed-off-by: default avatarDavid Sterba <dsterba@suse.com>
      88980383
    • Josef Bacik's avatar
      btrfs: run delayed refs less often in commit_cowonly_roots · 488bc2a2
      Josef Bacik authored
      We love running delayed refs in commit_cowonly_roots, but it is a bit
      excessive.  I was seeing cases of running 3 or 4 refs a few times in a
      row during this time.  Instead simply:
      
      - update all of the roots first
      - then run delayed refs
      - then handle the empty block groups case
      - and then if we have any more dirty roots do the whole thing again
      
      This allows us to be much more efficient with our delayed ref running,
      as we can batch a few more operations at once.
      Reviewed-by: default avatarNikolay Borisov <nborisov@suse.com>
      Signed-off-by: default avatarJosef Bacik <josef@toxicpanda.com>
      Signed-off-by: default avatarDavid Sterba <dsterba@suse.com>
      488bc2a2
    • Josef Bacik's avatar
      btrfs: stop running all delayed refs during snapshot · dac348e9
      Josef Bacik authored
      This was added in commit 361048f5 ("Btrfs: fix full backref problem
      when inserting shared block reference") to address a problem where we
      hit the following BUG_ON() in alloc_reserved_tree_block
      
              if (node->type == BTRFS_SHARED_BLOCK_REF_KEY) {
                      BUG_ON(!(flags & BTRFS_BLOCK_FLAG_FULL_BACKREF));
      
      However this BUG_ON() is bogus, and was removed by previous commit:
      
        btrfs: remove bogus BUG_ON in alloc_reserved_tree_block
      
      We no longer need to run delayed refs because of this, and can remove
      this flushing here.
      Signed-off-by: default avatarJosef Bacik <josef@toxicpanda.com>
      Signed-off-by: default avatarDavid Sterba <dsterba@suse.com>
      dac348e9
    • Josef Bacik's avatar
      btrfs: remove bogus BUG_ON in alloc_reserved_tree_block · b7774425
      Josef Bacik authored
      The fix 361048f5 ("Btrfs: fix full backref problem when inserting
      shared block reference") added a delayed ref flushing at subvolume
      creation time in order to avoid hitting this particular BUG_ON().
      
      Before this fix, we were tripping the BUG_ON() by
      
      1. Modify snapshot A, which creates blocks with a normal reference for
         snapshot A, as A is the owner of these blocks.  We now have delayed
         refs for these blocks.
      2. Create a snapshot of A named B, which pushes references for the
         children blocks of the root node for the new root B, thus creating
         more delayed refs for newly allocated blocks.
      3. A is modified, and because the metadata blocks can now be shared, it
         must push FULL_BACKREF references to the children of any block that A
         COWs down it's path to its target key.
      4. Delayed refs are run.  Because these are newly allocated blocks, we
         have ->must_insert_reserved reserved set on the delayed ref head, we
         call into alloc_reserved_tree_block() to add the extent item, and
         then add our ref.  At the time of this fix, we were ordering
         FULL_BACKREF delayed ref operations first, so we'd go to add this
         reference and then BUG_ON() because we didn't have the FULL_BACKREF
         flag set.
      
      The patch fixed this problem by making sure we ran the delayed refs
      before we had the chance to modify A.  This meant that any *new* blocks
      would have had their extent items created _before_ we would ever
      actually COW down and generate FULL_BACKREF entries.  Thus the problem
      went away.
      
      However this BUG_ON() is actually completely bogus.  The existence of a
      full backref doesn't necessarily mean that FULL_BACKREF must be set on
      that block, it must only be set on the actual parent itself.  Consider
      the example provided above.  If we COW down one path from A, any nodes
      are going to have a FULL_BACKREF ref pushed down to _all_ of their
      children, but not all of the children are going to have FULL_BACKREF
      set.  It is completely valid to have an extent item with normal and full
      backrefs without FULL_BACKREF actually set on the block itself.
      
      As a final note, I have been testing with the patch (applied after this
      one)
      
        btrfs: stop running all delayed refs during snapshot
      
      which removed this flushing.  My test was a torture test which did a lot
      of operations while snapshotting and deleting snapshots as well as
      relocation, and I never tripped this BUG_ON().  This is actually because
      at the time of 361048f5, we ordered SHARED keys _before_ normal
      references, and thus they would get run first.  However currently they
      are ordered _after_ normal references, so we'd do the initial creation
      without having a shared reference, and thus not hit this BUG_ON(), which
      explains why I didn't start hitting this problem during my testing with
      my other patch applied.
      Signed-off-by: default avatarJosef Bacik <josef@toxicpanda.com>
      Signed-off-by: default avatarDavid Sterba <dsterba@suse.com>
      b7774425
    • Josef Bacik's avatar
      btrfs: move delayed ref flushing for qgroup into qgroup helper · 2a4d84c1
      Josef Bacik authored
      The commit d6726335 ("btrfs: qgroup: Make snapshot accounting work
      with new extent-oriented qgroup.") added a flush of the delayed refs
      during snapshot creation in order to get the qgroup accounting properly.
      However this code has changed and been moved to it's own helper that is
      skipped if qgroups are turned off.  Move the flushing to the helper, as
      we do not need it when qgroups are turned off.
      
      Also add a comment explaining why it exists, and why it doesn't actually
      save us.  This will be helpful later when we try to fix qgroup
      accounting properly.
      Signed-off-by: default avatarJosef Bacik <josef@toxicpanda.com>
      Reviewed-by: default avatarDavid Sterba <dsterba@suse.com>
      Signed-off-by: default avatarDavid Sterba <dsterba@suse.com>
      2a4d84c1
    • Josef Bacik's avatar
      btrfs: only run delayed refs once before committing · ad368f33
      Josef Bacik authored
      We try to pre-flush the delayed refs when committing, because we want to
      do as little work as possible in the critical section of the transaction
      commit.
      
      However doing this twice can lead to very long transaction commit delays
      as other threads are allowed to continue to generate more delayed refs,
      which potentially delays the commit by multiple minutes in very extreme
      cases.
      
      So simply stick to one pre-flush, and then continue the rest of the
      transaction commit.
      Reviewed-by: default avatarNikolay Borisov <nborisov@suse.com>
      Signed-off-by: default avatarJosef Bacik <josef@toxicpanda.com>
      Signed-off-by: default avatarDavid Sterba <dsterba@suse.com>
      ad368f33
    • Josef Bacik's avatar
      btrfs: delayed refs pre-flushing should only run the heads we have · 61a56a99
      Josef Bacik authored
      Previously our delayed ref running used the total number of items as the
      items to run.  However we changed that to number of heads to run with
      the delayed_refs_rsv, as generally we want to run all of the operations
      for one bytenr.
      
      But with btrfs_run_delayed_refs(trans, 0) we set our count to 2x the
      number of items that we have.  This is generally fine, but if we have
      some operation generation loads of delayed refs while we're doing this
      pre-flushing in the transaction commit, we'll just spin forever doing
      delayed refs.
      
      Fix this to simply pick the number of delayed refs we currently have,
      that way we do not end up doing a lot of extra work that's being
      generated in other threads.
      Reviewed-by: default avatarNikolay Borisov <nborisov@suse.com>
      Signed-off-by: default avatarJosef Bacik <josef@toxicpanda.com>
      Signed-off-by: default avatarDavid Sterba <dsterba@suse.com>
      61a56a99
    • Josef Bacik's avatar
      btrfs: only let one thread pre-flush delayed refs in commit · e19eb11f
      Josef Bacik authored
      I've been running a stress test that runs 20 workers in their own
      subvolume, which are running an fsstress instance with 4 threads per
      worker, which is 80 total fsstress threads.  In addition to this I'm
      running balance in the background as well as creating and deleting
      snapshots.  This test takes around 12 hours to run normally, going
      slower and slower as the test goes on.
      
      The reason for this is because fsstress is running fsync sometimes, and
      because we're messing with block groups we often fall through to
      btrfs_commit_transaction, so will often have 20-30 threads all calling
      btrfs_commit_transaction at the same time.
      
      These all get stuck contending on the extent tree while they try to run
      delayed refs during the initial part of the commit.
      
      This is suboptimal, really because the extent tree is a single point of
      failure we only want one thread acting on that tree at once to reduce
      lock contention.
      
      Fix this by making the flushing mechanism a bit operation, to make it
      easy to use test_and_set_bit() in order to make sure only one task does
      this initial flush.
      
      Once we're into the transaction commit we only have one thread doing
      delayed ref running, it's just this initial pre-flush that is
      problematic.  With this patch my stress test takes around 90 minutes to
      run, instead of 12 hours.
      
      The memory barrier is not necessary for the flushing bit as it's
      ordered, unlike plain int. The transaction state accessed in
      btrfs_should_end_transaction could be affected by that too as it's not
      always used under transaction lock. Upon Nikolay's analysis in [1]
      it's not necessary:
      
        In should_end_transaction it's read without holding any locks. (U)
      
        It's modified in btrfs_cleanup_transaction without holding the
        fs_info->trans_lock (U), but the STATE_ERROR flag is going to be set.
      
        set in cleanup_transaction under fs_info->trans_lock (L)
        set in btrfs_commit_trans to COMMIT_START under fs_info->trans_lock.(L)
        set in btrfs_commit_trans to COMMIT_DOING under fs_info->trans_lock.(L)
        set in btrfs_commit_trans to COMMIT_UNBLOCK under
        fs_info->trans_lock.(L)
      
        set in btrfs_commit_trans to COMMIT_COMPLETED without locks but at this
        point the transaction is finished and fs_info->running_trans is NULL (U
        but irrelevant).
      
        So by the looks of it we can have a concurrent READ race with a WRITE,
        due to reads not taking a lock. In this case what we want to ensure is
        we either see new or old state. I consulted with Will Deacon and he said
        that in such a case we'd want to annotate the accesses to ->state with
        (READ|WRITE)_ONCE so as to avoid a theoretical tear, in this case I
        don't think this could happen but I imagine at some point KCSAN would
        flag such an access as racy (which it is).
      
      [1] https://lore.kernel.org/linux-btrfs/e1fd5cc1-0f28-f670-69f4-e9958b4964e6@suse.comReviewed-by: default avatarNikolay Borisov <nborisov@suse.com>
      Signed-off-by: default avatarJosef Bacik <josef@toxicpanda.com>
      [ add comments regarding memory barrier ]
      Signed-off-by: default avatarDavid Sterba <dsterba@suse.com>
      e19eb11f
    • Josef Bacik's avatar
      btrfs: do not block on deleted bgs mutex in the cleaner · ddfd08cb
      Josef Bacik authored
      While running some stress tests I started getting hung task messages.
      This is because the delete unused block groups code has to take the
      delete_unused_bgs_mutex to do it's work, which is taken by balance to
      make sure we don't delete block groups while we're balancing.
      
      The problem is that balance can take a while, and so we were getting
      hung task warnings.  We don't need to block and run these things, and
      the cleaner is needed to do other work, so trylock on this mutex and
      just bail if we can't acquire it right away.
      Reviewed-by: default avatarNikolay Borisov <nborisov@suse.com>
      Signed-off-by: default avatarJosef Bacik <josef@toxicpanda.com>
      Reviewed-by: default avatarDavid Sterba <dsterba@suse.com>
      Signed-off-by: default avatarDavid Sterba <dsterba@suse.com>
      ddfd08cb
    • Josef Bacik's avatar
      btrfs: abort the transaction if we fail to inc ref in btrfs_copy_root · 867ed321
      Josef Bacik authored
      While testing my error handling patches, I added a error injection site
      at btrfs_inc_extent_ref, to validate the error handling I added was
      doing the correct thing.  However I hit a pretty ugly corruption while
      doing this check, with the following error injection stack trace:
      
      btrfs_inc_extent_ref
        btrfs_copy_root
          create_reloc_root
            btrfs_init_reloc_root
      	btrfs_record_root_in_trans
      	  btrfs_start_transaction
      	    btrfs_update_inode
      	      btrfs_update_time
      		touch_atime
      		  file_accessed
      		    btrfs_file_mmap
      
      This is because we do not catch the error from btrfs_inc_extent_ref,
      which in practice would be ENOMEM, which means we lose the extent
      references for a root that has already been allocated and inserted,
      which is the problem.  Fix this by aborting the transaction if we fail
      to do the reference modification.
      
      CC: stable@vger.kernel.org # 4.4+
      Signed-off-by: default avatarJosef Bacik <josef@toxicpanda.com>
      Reviewed-by: default avatarDavid Sterba <dsterba@suse.com>
      Signed-off-by: default avatarDavid Sterba <dsterba@suse.com>
      867ed321
    • Josef Bacik's avatar
      btrfs: add asserts for deleting backref cache nodes · eddda68d
      Josef Bacik authored
      A weird KASAN problem that Zygo reported could have been easily caught
      if we checked for basic things in our backref freeing code.  We have two
      methods of freeing a backref node
      
      - btrfs_backref_free_node: this just is kfree() essentially.
      - btrfs_backref_drop_node: this actually unlinks the node and cleans up
        everything and then calls btrfs_backref_free_node().
      
      We should mostly be using btrfs_backref_drop_node(), to make sure the
      node is properly unlinked from the backref cache, and only use
      btrfs_backref_free_node() when we know the node isn't actually linked to
      the backref cache.  We made a mistake here and thus got the KASAN splat.
      
      Make this style of issue easier to find by adding some ASSERT()'s to
      btrfs_backref_free_node() and adjusting our deletion stuff to properly
      init the list so we can rely on list_empty() checks working properly.
      
        BUG: KASAN: use-after-free in btrfs_backref_cleanup_node+0x18a/0x420
        Read of size 8 at addr ffff888112402950 by task btrfs/28836
      
        CPU: 0 PID: 28836 Comm: btrfs Tainted: G        W         5.10.0-e35f27394290-for-next+ #23
        Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.12.0-1 04/01/2014
        Call Trace:
         dump_stack+0xbc/0xf9
         ? btrfs_backref_cleanup_node+0x18a/0x420
         print_address_description.constprop.8+0x21/0x210
         ? record_print_text.cold.34+0x11/0x11
         ? btrfs_backref_cleanup_node+0x18a/0x420
         ? btrfs_backref_cleanup_node+0x18a/0x420
         kasan_report.cold.10+0x20/0x37
         ? btrfs_backref_cleanup_node+0x18a/0x420
         __asan_load8+0x69/0x90
         btrfs_backref_cleanup_node+0x18a/0x420
         btrfs_backref_release_cache+0x83/0x1b0
         relocate_block_group+0x394/0x780
         ? merge_reloc_roots+0x4a0/0x4a0
         btrfs_relocate_block_group+0x26e/0x4c0
         btrfs_relocate_chunk+0x52/0x120
         btrfs_balance+0xe2e/0x1900
         ? check_flags.part.50+0x6c/0x1e0
         ? btrfs_relocate_chunk+0x120/0x120
         ? kmem_cache_alloc_trace+0xa06/0xcb0
         ? _copy_from_user+0x83/0xc0
         btrfs_ioctl_balance+0x3a7/0x460
         btrfs_ioctl+0x24c8/0x4360
         ? __kasan_check_read+0x11/0x20
         ? check_chain_key+0x1f4/0x2f0
         ? __asan_loadN+0xf/0x20
         ? btrfs_ioctl_get_supported_features+0x30/0x30
         ? kvm_sched_clock_read+0x18/0x30
         ? check_chain_key+0x1f4/0x2f0
         ? lock_downgrade+0x3f0/0x3f0
         ? handle_mm_fault+0xad6/0x2150
         ? do_vfs_ioctl+0xfc/0x9d0
         ? ioctl_file_clone+0xe0/0xe0
         ? check_flags.part.50+0x6c/0x1e0
         ? check_flags.part.50+0x6c/0x1e0
         ? check_flags+0x26/0x30
         ? lock_is_held_type+0xc3/0xf0
         ? syscall_enter_from_user_mode+0x1b/0x60
         ? do_syscall_64+0x13/0x80
         ? rcu_read_lock_sched_held+0xa1/0xd0
         ? __kasan_check_read+0x11/0x20
         ? __fget_light+0xae/0x110
         __x64_sys_ioctl+0xc3/0x100
         do_syscall_64+0x37/0x80
         entry_SYSCALL_64_after_hwframe+0x44/0xa9
        RIP: 0033:0x7f4c4bdfe427
        RSP: 002b:00007fff33ee6df8 EFLAGS: 00000202 ORIG_RAX: 0000000000000010
        RAX: ffffffffffffffda RBX: 00007fff33ee6e98 RCX: 00007f4c4bdfe427
        RDX: 00007fff33ee6e98 RSI: 00000000c4009420 RDI: 0000000000000003
        RBP: 0000000000000003 R08: 0000000000000003 R09: 0000000000000078
        R10: fffffffffffff59d R11: 0000000000000202 R12: 0000000000000001
        R13: 0000000000000000 R14: 00007fff33ee8a34 R15: 0000000000000001
      
        Allocated by task 28836:
         kasan_save_stack+0x21/0x50
         __kasan_kmalloc.constprop.18+0xbe/0xd0
         kasan_kmalloc+0x9/0x10
         kmem_cache_alloc_trace+0x410/0xcb0
         btrfs_backref_alloc_node+0x46/0xf0
         btrfs_backref_add_tree_node+0x60d/0x11d0
         build_backref_tree+0xc5/0x700
         relocate_tree_blocks+0x2be/0xb90
         relocate_block_group+0x2eb/0x780
         btrfs_relocate_block_group+0x26e/0x4c0
         btrfs_relocate_chunk+0x52/0x120
         btrfs_balance+0xe2e/0x1900
         btrfs_ioctl_balance+0x3a7/0x460
         btrfs_ioctl+0x24c8/0x4360
         __x64_sys_ioctl+0xc3/0x100
         do_syscall_64+0x37/0x80
         entry_SYSCALL_64_after_hwframe+0x44/0xa9
      
        Freed by task 28836:
         kasan_save_stack+0x21/0x50
         kasan_set_track+0x20/0x30
         kasan_set_free_info+0x1f/0x30
         __kasan_slab_free+0xf3/0x140
         kasan_slab_free+0xe/0x10
         kfree+0xde/0x200
         btrfs_backref_error_cleanup+0x452/0x530
         build_backref_tree+0x1a5/0x700
         relocate_tree_blocks+0x2be/0xb90
         relocate_block_group+0x2eb/0x780
         btrfs_relocate_block_group+0x26e/0x4c0
         btrfs_relocate_chunk+0x52/0x120
         btrfs_balance+0xe2e/0x1900
         btrfs_ioctl_balance+0x3a7/0x460
         btrfs_ioctl+0x24c8/0x4360
         __x64_sys_ioctl+0xc3/0x100
         do_syscall_64+0x37/0x80
         entry_SYSCALL_64_after_hwframe+0x44/0xa9
      
        The buggy address belongs to the object at ffff888112402900
         which belongs to the cache kmalloc-128 of size 128
        The buggy address is located 80 bytes inside of
         128-byte region [ffff888112402900, ffff888112402980)
        The buggy address belongs to the page:
        page:0000000028b1cd08 refcount:1 mapcount:0 mapping:0000000000000000 index:0xffff888131c810c0 pfn:0x112402
        flags: 0x17ffe0000000200(slab)
        raw: 017ffe0000000200 ffffea000424f308 ffffea0007d572c8 ffff888100040440
        raw: ffff888131c810c0 ffff888112402000 0000000100000009 0000000000000000
        page dumped because: kasan: bad access detected
      
        Memory state around the buggy address:
         ffff888112402800: fa fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb
         ffff888112402880: fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc
        >ffff888112402900: fa fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb
                                                         ^
         ffff888112402980: fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc
         ffff888112402a00: fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb
      
      Link: https://lore.kernel.org/linux-btrfs/20201208194607.GI31381@hungrycats.org/
      CC: stable@vger.kernel.org # 5.10+
      Signed-off-by: default avatarJosef Bacik <josef@toxicpanda.com>
      Reviewed-by: default avatarDavid Sterba <dsterba@suse.com>
      Signed-off-by: default avatarDavid Sterba <dsterba@suse.com>
      eddda68d
    • Josef Bacik's avatar
      btrfs: do not warn if we can't find the reloc root when looking up backref · f78743fb
      Josef Bacik authored
      The backref code is looking for a reloc_root that corresponds to the
      given fs root.  However any number of things could have gone wrong while
      initializing that reloc_root, like ENOMEM while trying to allocate the
      root itself, or EIO while trying to write the root item.  This would
      result in no corresponding reloc_root being in the reloc root cache, and
      thus would return NULL when we do the find_reloc_root() call.
      
      Because of this we do not want to WARN_ON().  This presumably was meant
      to catch developer errors, cases where we messed up adding the reloc
      root.  However we can easily hit this case with error injection, and
      thus should not do a WARN_ON().
      
      CC: stable@vger.kernel.org # 5.10+
      Reported-by: default avatarZygo Blaxell <ce3g8jdj@umail.furryterror.org>
      Signed-off-by: default avatarJosef Bacik <josef@toxicpanda.com>
      Reviewed-by: default avatarDavid Sterba <dsterba@suse.com>
      Signed-off-by: default avatarDavid Sterba <dsterba@suse.com>
      f78743fb
    • Josef Bacik's avatar
      btrfs: splice remaining dirty_bg's onto the transaction dirty bg list · 938fcbfb
      Josef Bacik authored
      While doing error injection testing with my relocation patches I hit the
      following assert:
      
        assertion failed: list_empty(&block_group->dirty_list), in fs/btrfs/block-group.c:3356
        ------------[ cut here ]------------
        kernel BUG at fs/btrfs/ctree.h:3357!
        invalid opcode: 0000 [#1] SMP NOPTI
        CPU: 0 PID: 24351 Comm: umount Tainted: G        W         5.10.0-rc3+ #193
        Hardware name: QEMU Standard PC (Q35 + ICH9, 2009), BIOS 1.13.0-2.fc32 04/01/2014
        RIP: 0010:assertfail.constprop.0+0x18/0x1a
        RSP: 0018:ffffa09b019c7e00 EFLAGS: 00010282
        RAX: 0000000000000056 RBX: ffff8f6492c18000 RCX: 0000000000000000
        RDX: ffff8f64fbc27c60 RSI: ffff8f64fbc19050 RDI: ffff8f64fbc19050
        RBP: ffff8f6483bbdc00 R08: 0000000000000000 R09: 0000000000000000
        R10: ffffa09b019c7c38 R11: ffffffff85d70928 R12: ffff8f6492c18100
        R13: ffff8f6492c18148 R14: ffff8f6483bbdd70 R15: dead000000000100
        FS:  00007fbfda4cdc40(0000) GS:ffff8f64fbc00000(0000) knlGS:0000000000000000
        CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
        CR2: 00007fbfda666fd0 CR3: 000000013cf66002 CR4: 0000000000370ef0
        Call Trace:
         btrfs_free_block_groups.cold+0x55/0x55
         close_ctree+0x2c5/0x306
         ? fsnotify_destroy_marks+0x14/0x100
         generic_shutdown_super+0x6c/0x100
         kill_anon_super+0x14/0x30
         btrfs_kill_super+0x12/0x20
         deactivate_locked_super+0x36/0xa0
         cleanup_mnt+0x12d/0x190
         task_work_run+0x5c/0xa0
         exit_to_user_mode_prepare+0x1b1/0x1d0
         syscall_exit_to_user_mode+0x54/0x280
         entry_SYSCALL_64_after_hwframe+0x44/0xa9
      
      This happened because I injected an error in btrfs_cow_block() while
      running the dirty block groups.  When we run the dirty block groups, we
      splice the list onto a local list to process.  However if an error
      occurs, we only cleanup the transactions dirty block group list, not any
      pending block groups we have on our locally spliced list.
      
      In fact if we fail to allocate a path in this function we'll also fail
      to clean up the splice list.
      
      Fix this by splicing the list back onto the transaction dirty block
      group list so that the block groups are cleaned up.  Then add a 'out'
      label and have the error conditions jump to out so that the errors are
      handled properly.  This also has the side-effect of fixing a problem
      where we would clear 'ret' on error because we unconditionally ran
      btrfs_run_delayed_refs().
      
      CC: stable@vger.kernel.org # 4.4+
      Signed-off-by: default avatarJosef Bacik <josef@toxicpanda.com>
      Reviewed-by: default avatarDavid Sterba <dsterba@suse.com>
      Signed-off-by: default avatarDavid Sterba <dsterba@suse.com>
      938fcbfb
    • Josef Bacik's avatar
      btrfs: fix reloc root leak with 0 ref reloc roots on recovery · c78a10ae
      Josef Bacik authored
      When recovering a relocation, if we run into a reloc root that has 0
      refs we simply add it to the reloc_control->reloc_roots list, and then
      clean it up later.  The problem with this is __del_reloc_root() doesn't
      do anything if the root isn't in the radix tree, which in this case it
      won't be because we never call __add_reloc_root() on the reloc_root.
      
      This exit condition simply isn't correct really.  During normal
      operation we can remove ourselves from the rb tree and then we're meant
      to clean up later at merge_reloc_roots() time, and this happens
      correctly.  During recovery we're depending on free_reloc_roots() to
      drop our references, but we're short-circuiting.
      
      Fix this by continuing to check if we're on the list and dropping
      ourselves from the reloc_control root list and dropping our reference
      appropriately.  Change the corresponding BUG_ON() to an ASSERT() that
      does the correct thing if we aren't in the rb tree.
      
      CC: stable@vger.kernel.org # 4.4+
      Signed-off-by: default avatarJosef Bacik <josef@toxicpanda.com>
      Reviewed-by: default avatarDavid Sterba <dsterba@suse.com>
      Signed-off-by: default avatarDavid Sterba <dsterba@suse.com>
      c78a10ae