1. 21 Sep, 2013 16 commits
  2. 02 Sep, 2013 4 commits
  3. 01 Sep, 2013 20 commits
    • Filipe David Borba Manana's avatar
      Btrfs: optimize key searches in btrfs_search_slot · d7396f07
      Filipe David Borba Manana authored
      When the binary search returns 0 (exact match), the target key
      will necessarily be at slot 0 of all nodes below the current one,
      so in this case the binary search is not needed because it will
      always return 0, and we waste time doing it, holding node locks
      for longer than necessary, etc.
      
      Below follow histograms with the times spent on the current approach of
      doing a binary search when the previous binary search returned 0, and
      times for the new approach, which directly picks the first item/child
      node in the leaf/node.
      
      Current approach:
      
      Count: 6682
      Range: 35.000 - 8370.000; Mean: 85.837; Median: 75.000; Stddev: 106.429
      Percentiles:  90th: 124.000; 95th: 145.000; 99th: 206.000
        35.000 -   61.080:  1235 ################
        61.080 -  106.053:  4207 #####################################################
       106.053 -  183.606:  1122 ##############
       183.606 -  317.341:   111 #
       317.341 -  547.959:     6 |
       547.959 - 8370.000:     1 |
      
      Approach proposed by this patch:
      
      Count: 6682
      Range:  6.000 - 135.000; Mean: 16.690; Median: 16.000; Stddev:  7.160
      Percentiles:  90th: 23.000; 95th: 27.000; 99th: 40.000
         6.000 -    8.418:    58 #
         8.418 -   11.670:  1149 #########################
        11.670 -   16.046:  2418 #####################################################
        16.046 -   21.934:  2098 ##############################################
        21.934 -   29.854:   744 ################
        29.854 -   40.511:   154 ###
        40.511 -   54.848:    41 #
        54.848 -   74.136:     5 |
        74.136 -  100.087:     9 |
       100.087 -  135.000:     6 |
      
      These samples were captured during a run of the btrfs tests 001, 002 and
      004 in the xfstests, with a leaf/node size of 4Kb.
      Signed-off-by: default avatarFilipe David Borba Manana <fdmanana@gmail.com>
      Signed-off-by: default avatarJosef Bacik <jbacik@fusionio.com>
      Signed-off-by: default avatarChris Mason <chris.mason@fusionio.com>
      d7396f07
    • Josef Bacik's avatar
      Btrfs: don't use an async starter for most of our workers · 45d5fd14
      Josef Bacik authored
      We only need an async starter if we can't make a GFP_NOFS allocation in our
      current path.  This is the case for the endio stuff since it happens in IRQ
      context, but things like the caching thread workers and the delalloc flushers we
      can easily make this allocation and start threads right away.  Also change the
      worker count for the caching thread pool.  Traditionally we limited this to 2
      since we took read locks while caching, but nowadays we do this lockless so
      there's no reason to limit the number of caching threads.  Thanks,
      Signed-off-by: default avatarJosef Bacik <jbacik@fusionio.com>
      Signed-off-by: default avatarChris Mason <chris.mason@fusionio.com>
      45d5fd14
    • Josef Bacik's avatar
      Btrfs: only update disk_i_size as we remove extents · 7f4f6e0a
      Josef Bacik authored
      This fixes a problem where if we fail a truncate we will leave the i_size set
      where we wanted to truncate to instead of where we were able to truncate to.
      Fix this by making btrfs_truncate_inode_items do the disk_i_size update as it
      removes extents, that way it will always be consistent with where its extents
      are.  Then if the truncate fails at all we can update the in-ram i_size with
      what we have on disk and delete the orphan item.  Thanks,
      Signed-off-by: default avatarJosef Bacik <jbacik@fusionio.com>
      Signed-off-by: default avatarChris Mason <chris.mason@fusionio.com>
      7f4f6e0a
    • Filipe David Borba Manana's avatar
      Btrfs: fix deadlock in uuid scan kthread · f45388f3
      Filipe David Borba Manana authored
      If there's an ongoing transaction when the uuid scan kthread attempts
      to create one, the kthread will block, waiting for that transaction to
      finish while it's keeping locks on the tree root, and in turn the existing
      transaction is waiting for those locks to be free.
      
      The stack trace reported by the kernel follows.
      
      [36700.671601] INFO: task btrfs-uuid:15480 blocked for more than 120 seconds.
      [36700.671602] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
      [36700.671602] btrfs-uuid      D 0000000000000000     0 15480      2 0x00000000
      [36700.671604]  ffff880710bd5b88 0000000000000046 ffff8803d36ba850 0000000000030000
      [36700.671605]  ffff8806d76dc530 ffff880710bd5fd8 ffff880710bd5fd8 ffff880710bd5fd8
      [36700.671607]  ffff8808098ac530 ffff8806d76dc530 ffff880710bd5b98 ffff8805e4508e40
      [36700.671608] Call Trace:
      [36700.671610]  [<ffffffff816f36b9>] schedule+0x29/0x70
      [36700.671620]  [<ffffffffa05a3bdf>] wait_current_trans.isra.33+0xbf/0x120 [btrfs]
      [36700.671623]  [<ffffffff81066760>] ? add_wait_queue+0x60/0x60
      [36700.671629]  [<ffffffffa05a5b06>] start_transaction+0x3d6/0x530 [btrfs]
      [36700.671636]  [<ffffffffa05bb1f4>] ? btrfs_get_token_32+0x64/0xf0 [btrfs]
      [36700.671642]  [<ffffffffa05a5fbb>] btrfs_start_transaction+0x1b/0x20 [btrfs]
      [36700.671649]  [<ffffffffa05c8a81>] btrfs_uuid_scan_kthread+0x211/0x3d0 [btrfs]
      [36700.671655]  [<ffffffffa05c8870>] ? __btrfs_open_devices+0x2a0/0x2a0 [btrfs]
      [36700.671657]  [<ffffffff81065fa0>] kthread+0xc0/0xd0
      [36700.671659]  [<ffffffff81065ee0>] ? flush_kthread_worker+0xb0/0xb0
      [36700.671661]  [<ffffffff816fcd1c>] ret_from_fork+0x7c/0xb0
      [36700.671662]  [<ffffffff81065ee0>] ? flush_kthread_worker+0xb0/0xb0
      [36700.671663] INFO: task btrfs:15481 blocked for more than 120 seconds.
      [36700.671664] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
      [36700.671665] btrfs           D 0000000000000000     0 15481  15212 0x00000004
      [36700.671666]  ffff880248cbf4c8 0000000000000086 ffff8803d36ba700 ffff8801dbd5c280
      [36700.671668]  ffff880807815c40 ffff880248cbffd8 ffff880248cbffd8 ffff880248cbffd8
      [36700.671669]  ffff8805e86a0000 ffff880807815c40 ffff880248cbf4d8 ffff8801dbd5c280
      [36700.671670] Call Trace:
      [36700.671672]  [<ffffffff816f36b9>] schedule+0x29/0x70
      [36700.671679]  [<ffffffffa05d9b0d>] btrfs_tree_lock+0x6d/0x230 [btrfs]
      [36700.671680]  [<ffffffff81066760>] ? add_wait_queue+0x60/0x60
      [36700.671685]  [<ffffffffa0582829>] btrfs_search_slot+0x999/0xb00 [btrfs]
      [36700.671691]  [<ffffffffa05bd9de>] ? btrfs_lookup_first_ordered_extent+0x5e/0xb0 [btrfs]
      [36700.671698]  [<ffffffffa05e3e54>] __btrfs_write_out_cache+0x8c4/0xa80 [btrfs]
      [36700.671704]  [<ffffffffa05e4362>] btrfs_write_out_cache+0xb2/0xf0 [btrfs]
      [36700.671710]  [<ffffffffa05c4441>] ? free_extent_buffer+0x61/0xc0 [btrfs]
      [36700.671716]  [<ffffffffa0594c82>] btrfs_write_dirty_block_groups+0x562/0x650 [btrfs]
      [36700.671723]  [<ffffffffa0610092>] commit_cowonly_roots+0x171/0x24b [btrfs]
      [36700.671729]  [<ffffffffa05a4dde>] btrfs_commit_transaction+0x4fe/0xa10 [btrfs]
      [36700.671735]  [<ffffffffa0610af3>] create_subvol+0x5c0/0x636 [btrfs]
      [36700.671742]  [<ffffffffa05d49ff>] btrfs_mksubvol.isra.60+0x33f/0x3f0 [btrfs]
      [36700.671747]  [<ffffffffa05d4bf2>] btrfs_ioctl_snap_create_transid+0x142/0x190 [btrfs]
      [36700.671752]  [<ffffffffa05d4c6c>] ? btrfs_ioctl_snap_create+0x2c/0x80 [btrfs]
      [36700.671757]  [<ffffffffa05d4c9e>] btrfs_ioctl_snap_create+0x5e/0x80 [btrfs]
      [36700.671759]  [<ffffffff8113a764>] ? handle_pte_fault+0x84/0x920
      [36700.671764]  [<ffffffffa05d87eb>] btrfs_ioctl+0xf0b/0x1d00 [btrfs]
      [36700.671766]  [<ffffffff8113c120>] ? handle_mm_fault+0x210/0x310
      [36700.671768]  [<ffffffff816f83a4>] ? __do_page_fault+0x284/0x4e0
      [36700.671770]  [<ffffffff81180aa6>] do_vfs_ioctl+0x96/0x550
      [36700.671772]  [<ffffffff81170fe3>] ? __sb_end_write+0x33/0x70
      [36700.671774]  [<ffffffff81180ff1>] SyS_ioctl+0x91/0xb0
      [36700.671775]  [<ffffffff816fcdc2>] system_call_fastpath+0x16/0x1b
      Signed-off-by: default avatarFilipe David Borba Manana <fdmanana@gmail.com>
      Signed-off-by: default avatarJosef Bacik <jbacik@fusionio.com>
      Signed-off-by: default avatarChris Mason <chris.mason@fusionio.com>
      f45388f3
    • Ilya Dryomov's avatar
      Btrfs: stop refusing the relocation of chunk 0 · 795a3321
      Ilya Dryomov authored
      AFAICT chunk 0 is no longer special, and so it should be restriped just
      like every other chunk.  One reason for this change is us refusing the
      relocation can lead to filesystems that can only be mounted ro, and
      never rw -- see the bugzilla [1] for details.  The other reason is that
      device removal code is already doing this: it will happily relocate
      chunk 0 is part of shrinking the device.
      
      [1] https://bugzilla.kernel.org/show_bug.cgi?id=60594Reported-by: default avatarXavier Bassery <xavier@bartica.org>
      Signed-off-by: default avatarIlya Dryomov <idryomov@gmail.com>
      Signed-off-by: default avatarJosef Bacik <jbacik@fusionio.com>
      Signed-off-by: default avatarChris Mason <chris.mason@fusionio.com>
      795a3321
    • Filipe David Borba Manana's avatar
    • Andy Shevchenko's avatar
      btrfs: reuse kbasename helper · ed84885d
      Andy Shevchenko authored
      To get name of the file from a pathname let's use kbasename() helper. It allows
      to simplify code a bit.
      Signed-off-by: default avatarAndy Shevchenko <andriy.shevchenko@linux.intel.com>
      Signed-off-by: default avatarJosef Bacik <jbacik@fusionio.com>
      Signed-off-by: default avatarChris Mason <chris.mason@fusionio.com>
      ed84885d
    • Anand Jain's avatar
      btrfs: return btrfs error code for dev excl ops err · e57138b3
      Anand Jain authored
      now threads can return BTRFS_ERROR_DEV_EXCL_RUN_IN_PROGRESS
      as defined in btrfs.h for the dev excl operation error in
      the FS, which means with this kernel would stop logging
      (almost an user error) into the /var/log/messages
      
      v2: accepts Josef' comment
      Signed-off-by: default avatarAnand Jain <anand.jain@oracle.com>
      Signed-off-by: default avatarJosef Bacik <jbacik@fusionio.com>
      Signed-off-by: default avatarChris Mason <chris.mason@fusionio.com>
      e57138b3
    • Josef Bacik's avatar
      Btrfs: allow partial ordered extent completion · 77cef2ec
      Josef Bacik authored
      We currently have this problem where you can truncate pages that have not yet
      been written for an ordered extent.  We do this because the truncate will be
      coming behind to clean us up anyway so what's the harm right?  Well if truncate
      fails for whatever reason we leave an orphan item around for the file to be
      cleaned up later.  But if the user goes and truncates up the file and tries to
      read from the area that had been discarded previously they will get a csum error
      because we never actually wrote that data out.
      
      This patch fixes this by allowing us to either discard the ordered extent
      completely, by which I mean we just free up the space we had allocated and not
      add the file extent, or adjust the length of the file extent we write.  We do
      this by setting the length we truncated down to in the ordered extent, and then
      we set the file extent length and ram bytes to this length.  The total disk
      space stays unchanged since we may be compressed and we can't just chop off the
      disk space, but at least this way the file extent only points to the valid data.
      Then when the file extent is free'd the extent and csums will be freed normally.
      
      This patch is needed for the next series which will give us more graceful
      recovery of failed truncates.  Thanks,
      Signed-off-by: default avatarJosef Bacik <jbacik@fusionio.com>
      Signed-off-by: default avatarChris Mason <chris.mason@fusionio.com>
      77cef2ec
    • Josef Bacik's avatar
      Btrfs: convert all bug_ons in free-space-cache.c · b12d6869
      Josef Bacik authored
      All of these are logic checks to make sure we're not breaking anything, so
      convert them over to ASSERT().  Thanks,
      Signed-off-by: default avatarJosef Bacik <jbacik@fusionio.com>
      Signed-off-by: default avatarChris Mason <chris.mason@fusionio.com>
      b12d6869
    • Josef Bacik's avatar
      Btrfs: add support for asserts · 2e17c7c6
      Josef Bacik authored
      One of the complaints we get a lot is how many BUG_ON()'s we have.  So to help
      with this I'm introducing a kconfig option to enable/disable a new ASSERT()
      mechanism much like what XFS does.  This will allow us developers to still get
      our nice panics but allow users/distros to compile them out.  With this we can
      go through and convert any BUG_ON()'s that we have to catch actual programming
      mistakes to the new ASSERT() and then fix everybody else to return errors.  This
      will also allow developers to leave sanity checks in their new code to make sure
      we don't trip over problems while testing stuff and vetting new features.
      Thanks,
      Signed-off-by: default avatarJosef Bacik <jbacik@fusionio.com>
      Signed-off-by: default avatarChris Mason <chris.mason@fusionio.com>
      2e17c7c6
    • Josef Bacik's avatar
      Btrfs: adjust the fs_devices->missing count on unmount · 726551eb
      Josef Bacik authored
      I noticed that if I tried to mount a file system with -o degraded after having
      done it once already we would fail to mount.  This is because the
      fs_devices->missing count was getting bumped everytime we mounted, but not
      getting reset whenever we unmounted.  To fix this we just drop the missing count
      as we're closing devices to make sure this doesn't happen.  Thanks,
      Signed-off-by: default avatarJosef Bacik <jbacik@fusionio.com>
      Signed-off-by: default avatarChris Mason <chris.mason@fusionio.com>
      726551eb
    • Stefan Behrens's avatar
      Btrf: cleanup: don't check for root_refs == 0 twice · 23fa76b0
      Stefan Behrens authored
      btrfs_read_fs_root_no_name() already checks if btrfs_root_refs()
      is zero and returns ENOENT in this case. There is no need to do
      it again in three more places.
      Signed-off-by: default avatarStefan Behrens <sbehrens@giantdisaster.de>
      Signed-off-by: default avatarJosef Bacik <jbacik@fusionio.com>
      Signed-off-by: default avatarChris Mason <chris.mason@fusionio.com>
      23fa76b0
    • Stefan Behrens's avatar
      Btrfs: fix for patch "cleanup: don't check the same thing twice" · 48475471
      Stefan Behrens authored
      Mitch Harder noticed that the patch 3c64a1ab mentioned in the subject
      line was causing a kernel BUG() on snapshot deletion.
      
      The patch was wrong. It did not handle cached roots correctly. The
      check for root_refs == 0 was removed everywhere where
      btrfs_read_fs_root_no_name() had been used to retrieve the root,
      because this check was already dealt with in
      btrfs_read_fs_root_no_name(). But in the case when the root was
      found in the cache, there was no such check.
      
      This patch adds the missing check in the case where the root is
      found in the cache.
      Reported-by: default avatarMitch Harder <mitch.harder@sabayonlinux.org>
      Signed-off-by: default avatarStefan Behrens <sbehrens@giantdisaster.de>
      Reviewed-by: default avatarMiao Xie <miaox@cn.fujitsu.com>
      Signed-off-by: default avatarJosef Bacik <jbacik@fusionio.com>
      Signed-off-by: default avatarChris Mason <chris.mason@fusionio.com>
      48475471
    • Stefan Behrens's avatar
      Btrfs: get rid of one BUG() in write_all_supers() · 9d565ba4
      Stefan Behrens authored
      The second round uses btrfs_error() and return -EIO, the first round
      can handle write errors the same way.
      Signed-off-by: default avatarStefan Behrens <sbehrens@giantdisaster.de>
      Signed-off-by: default avatarJosef Bacik <jbacik@fusionio.com>
      Signed-off-by: default avatarChris Mason <chris.mason@fusionio.com>
      9d565ba4
    • Wang Shilong's avatar
      Btrfs: allocate prelim_ref with a slab allocater · b9e9a6cb
      Wang Shilong authored
      struct __prelim_ref is allocated and freed frequently when
      walking backref tree, using slab allocater can not only
      speed up allocating but also detect memory leaks.
      Signed-off-by: default avatarWang Shilong <wangsl.fnst@cn.fujitsu.com>
      Reviewed-by: default avatarMiao Xie <miaox@cn.fujitsu.com>
      Reviewed-by: default avatarJan Schmidt <list.btrfs@jan-o-sch.net>
      Signed-off-by: default avatarJosef Bacik <jbacik@fusionio.com>
      Signed-off-by: default avatarChris Mason <chris.mason@fusionio.com>
      b9e9a6cb
    • Wang Shilong's avatar
      Btrfs: pass gfp_t to __add_prelim_ref() to avoid always using GFP_ATOMIC · 742916b8
      Wang Shilong authored
      Currently, only add_delayed_refs have to allocate with GFP_ATOMIC,
      So just pass arg 'gfp_t' to decide which allocation mode.
      Signed-off-by: default avatarWang Shilong <wangsl.fnst@cn.fujitsu.com>
      Reviewed-by: default avatarMiao Xie <miaox@cn.fujitsu.com>
      Reviewed-by: default avatarJan Schmidt <list.btrfs@jan-o-sch.net>
      Signed-off-by: default avatarJosef Bacik <jbacik@fusionio.com>
      Signed-off-by: default avatarChris Mason <chris.mason@fusionio.com>
      742916b8
    • Filipe David Borba Manana's avatar
      Btrfs: fix race conditions in BTRFS_IOC_FS_INFO ioctl · f7171750
      Filipe David Borba Manana authored
      The handler for the ioctl BTRFS_IOC_FS_INFO was reading the
      number of devices before acquiring the device list mutex.
      
      This could lead to inconsistent results because the update of
      the device list and the number of devices counter (amongst other
      counters related to the device list) are updated in volumes.c
      while holding the device list mutex - except for 2 places, one
      was volumes.c:btrfs_prepare_sprout() and the other was
      volumes.c:device_list_add().
      
      For example, if we have 2 devices, with IDs 1 and 2 and then add
      a new device, with ID 3, and while adding the device is in progress
      an BTRFS_IOC_FS_INFO ioctl arrives, it could return a number of
      devices of 2 and a max dev id of 3. This would be incorrect.
      
      Also, this ioctl handler was reading the fsid while it can be
      updated concurrently. This can happen when while a new device is
      being added and the current filesystem is in seeding mode.
      Example:
      
      $ mkfs.btrfs -f /dev/sdb1
      $ mkfs.btrfs -f /dev/sdb2
      $ btrfstune -S 1 /dev/sdb1
      $ mount /dev/sdb1 /mnt/test
      $ btrfs device add /dev/sdb2 /mnt/test
      
      If during the last step a BTRFS_IOC_FS_INFO ioctl was requested, it
      could read an fsid that was never valid (some bits part of the old
      fsid and others part of the new fsid). Also, it could read a number
      of devices that doesn't match the number of devices in the list and
      the max device id, as explained before.
      Signed-off-by: default avatarFilipe David Borba Manana <fdmanana@gmail.com>
      Signed-off-by: default avatarJosef Bacik <jbacik@fusionio.com>
      Signed-off-by: default avatarChris Mason <chris.mason@fusionio.com>
      f7171750
    • Filipe David Borba Manana's avatar
      Btrfs: fix race between removing a dev and writing sbs · d7306801
      Filipe David Borba Manana authored
      This change fixes an issue when removing a device and writing
      all super blocks run simultaneously. Here's the steps necessary
      for the issue to happen:
      
      1) disk-io.c:write_all_supers() gets a number of N devices from the
         super_copy, so it will not panic if it fails to write super blocks
         for N - 1 devices;
      
      2) Then it tries to acquire the device_list_mutex, but blocks because
         volumes.c:btrfs_rm_device() got it first;
      
      3) btrfs_rm_device() removes the device from the list, then unlocks the
         mutex and after the unlock it updates the number of devices in
         super_copy to N - 1.
      
      4) write_all_supers() finally acquires the mutex, iterates over all the
         devices in the list and gets N - 1 errors, that is, it failed to write
         super blocks to all the devices;
      
      5) Because write_all_supers() thinks there are a total of N devices, it
         considers N - 1 errors to be ok, and therefore won't panic.
      
      So this change just makes sure that write_all_supers() reads the number
      of devices from super_copy after it acquires the device_list_mutex.
      Conversely, it changes btrfs_rm_device() to update the number of devices
      in super_copy before it releases the device list mutex.
      
      The code path to add a new device (volumes.c:btrfs_init_new_device),
      already has the right behaviour: it updates the number of devices in
      super_copy while holding the device_list_mutex.
      
      The only code path that doesn't lock the device list mutex
      before updating the number of devices in the super copy is
      disk-io.c:next_root_backup(), called by open_ctree() during
      mount time where concurrency issues can't happen.
      Signed-off-by: default avatarFilipe David Borba Manana <fdmanana@gmail.com>
      Signed-off-by: default avatarJosef Bacik <jbacik@fusionio.com>
      Signed-off-by: default avatarChris Mason <chris.mason@fusionio.com>
      d7306801
    • Josef Bacik's avatar
      Btrfs: remove ourselves from the cluster list under lock · b8d0c69b
      Josef Bacik authored
      A user was reporting weird warnings from btrfs_put_delayed_ref() and I noticed
      that we were doing this list_del_init() on our head ref outside of
      delayed_refs->lock.  This is a problem if we have people still on the list, we
      could end up modifying old pointers and such.  Fix this by removing us from the
      list before we do our run_delayed_ref on our head ref.  Thanks,
      Signed-off-by: default avatarJosef Bacik <jbacik@fusionio.com>
      Signed-off-by: default avatarChris Mason <chris.mason@fusionio.com>
      b8d0c69b