• Filipe Manana's avatar
    btrfs: send: skip resolution of our own backref when finding clone source · adf02418
    Filipe Manana authored
    When doing backref walking to determine a source range to clone from, it
    is worthless to collect and resolve our own data backref, as we can't
    obviously use it as a clone source and it represents the range we want to
    clone into. Collecting the backref implies doing the extra work to resolve
    it, doing the search for a file extent item in a subvolume tree, etc.
    Skipping the data backref is valid as long as we only have the send root
    as the single clone root, otherwise the leaf with the file extent item may
    be accessible from another clone root due to shared subtrees created by
    snapshots, and therefore we have to collect the backref and resolve it.
    
    So add a callback to the backref walking code to guide it to skip data
    backrefs.
    
    This change is part of a patchset comprised of the following patches:
    
      01/17 btrfs: fix inode list leak during backref walking at resolve_indirect_refs()
      02/17 btrfs: fix inode list leak during backref walking at find_parent_nodes()
      03/17 btrfs: fix ulist leaks in error paths of qgroup self tests
      04/17 btrfs: remove pointless and double ulist frees in error paths of qgroup tests
      05/17 btrfs: send: avoid unnecessary path allocations when finding extent clone
      06/17 btrfs: send: update comment at find_extent_clone()
      07/17 btrfs: send: drop unnecessary backref context field initializations
      08/17 btrfs: send: avoid unnecessary backref lookups when finding clone source
      09/17 btrfs: send: optimize clone detection to increase extent sharing
      10/17 btrfs: use a single argument for extent offset in backref walking functions
      11/17 btrfs: use a structure to pass arguments to backref walking functions
      12/17 btrfs: reuse roots ulist on each leaf iteration for iterate_extent_inodes()
      13/17 btrfs: constify ulist parameter of ulist_next()
      14/17 btrfs: send: cache leaf to roots mapping during backref walking
      15/17 btrfs: send: skip unnecessary backref iterations
      16/17 btrfs: send: avoid double extent tree search when finding clone source
      17/17 btrfs: send: skip resolution of our own backref when finding clone source
    
    The following test was run on non-debug kernel (Debian's default kernel
    config) before and after applying the patchset:
    
       $ cat test-send-many-shared-extents.sh
       #!/bin/bash
    
       DEV=/dev/sdh
       MNT=/mnt/sdh
    
       umount $DEV &> /dev/null
       mkfs.btrfs -f $DEV
       mount $DEV $MNT
    
       num_files=50000
       num_clones_per_file=50
    
       for ((i = 1; i <= $num_files; i++)); do
           xfs_io -f -c "pwrite 0 64K" $MNT/file_$i > /dev/null
           echo -ne "\r$i files created..."
       done
       echo
    
       btrfs subvolume snapshot -r $MNT $MNT/snap1
    
       cloned=0
       for ((i = 1; i <= $num_clones_per_file; i++)); do
           for ((j = 1; j <= $num_files; j++)); do
               cp --reflink=always $MNT/file_$j $MNT/file_${j}_clone_${i}
               cloned=$((cloned + 1))
               echo -ne "\r$cloned / $((num_files * num_clones_per_file)) clone operations"
           done
       done
       echo
    
       btrfs subvolume snapshot -r $MNT $MNT/snap2
    
       # Unmount and mount again to clear all cached metadata (and data).
       umount $DEV
       mount $DEV $MNT
    
       start=$(date +%s%N)
       btrfs send $MNT/snap2 > /dev/null
       end=$(date +%s%N)
    
       dur=$(( (end - start) / 1000000000 ))
       echo -e "\nFull send took $dur seconds"
    
       # Unmount and mount again to clear all cached metadata (and data).
       umount $DEV
       mount $DEV $MNT
    
       start=$(date +%s%N)
       btrfs send -p $MNT/snap1 $MNT/snap2 > /dev/null
       end=$(date +%s%N)
    
       dur=$(( (end - start) / 1000000000 ))
       echo -e "\nIncremental send took $dur seconds"
    
       umount $MNT
    
    Before applying the patchset:
    
       (...)
       Full send took 1108 seconds
       (...)
       Incremental send took 1135 seconds
    
    After applying the whole patchset:
    
       (...)
       Full send took 268 seconds            (-75.8%)
       (...)
       Incremental send took 316 seconds     (-72.2%)
    Signed-off-by: default avatarFilipe Manana <fdmanana@suse.com>
    Signed-off-by: default avatarDavid Sterba <dsterba@suse.com>
    adf02418
backref.h 17 KB