1. 10 Mar, 2016 11 commits
    • Mike Snitzer's avatar
    • Mike Snitzer's avatar
      dm: return error if bio_integrity_clone() fails in clone_bio() · c80914e8
      Mike Snitzer authored
      clone_bio() now checks if bio_integrity_clone() returned an error rather
      than just drop it on the floor.
      Signed-off-by: default avatarMike Snitzer <snitzer@redhat.com>
      c80914e8
    • Joe Thornber's avatar
      dm thin metadata: don't issue prefetches if a transaction abort has failed · 2eae9e44
      Joe Thornber authored
      If a transaction abort has failed then we can no longer use the metadata
      device.  Typically this happens if the superblock is unreadable.
      
      This fix addresses a crash seen during metadata device failure testing.
      
      Fixes: 8a01a6af ("dm thin: prefetch missing metadata pages")
      Cc: stable@vger.kernel.org # 3.19+
      Signed-off-by: default avatarJoe Thornber <ejt@redhat.com>
      Signed-off-by: default avatarMike Snitzer <snitzer@redhat.com>
      2eae9e44
    • DingXiang's avatar
      dm snapshot: disallow the COW and origin devices from being identical · 4df2bf46
      DingXiang authored
      Otherwise loading a "snapshot" table using the same device for the
      origin and COW devices, e.g.:
      
      echo "0 20971520 snapshot 253:3 253:3 P 8" | dmsetup create snap
      
      will trigger:
      
      BUG: unable to handle kernel NULL pointer dereference at 0000000000000098
      [ 1958.979934] IP: [<ffffffffa040efba>] dm_exception_store_set_chunk_size+0x7a/0x110 [dm_snapshot]
      [ 1958.989655] PGD 0
      [ 1958.991903] Oops: 0000 [#1] SMP
      ...
      [ 1959.059647] CPU: 9 PID: 3556 Comm: dmsetup Tainted: G          IO    4.5.0-rc5.snitm+ #150
      ...
      [ 1959.083517] task: ffff8800b9660c80 ti: ffff88032a954000 task.ti: ffff88032a954000
      [ 1959.091865] RIP: 0010:[<ffffffffa040efba>]  [<ffffffffa040efba>] dm_exception_store_set_chunk_size+0x7a/0x110 [dm_snapshot]
      [ 1959.104295] RSP: 0018:ffff88032a957b30  EFLAGS: 00010246
      [ 1959.110219] RAX: 0000000000000000 RBX: 0000000000000008 RCX: 0000000000000001
      [ 1959.118180] RDX: 0000000000000000 RSI: 0000000000000008 RDI: ffff880329334a00
      [ 1959.126141] RBP: ffff88032a957b50 R08: 0000000000000000 R09: 0000000000000001
      [ 1959.134102] R10: 000000000000000a R11: f000000000000000 R12: ffff880330884d80
      [ 1959.142061] R13: 0000000000000008 R14: ffffc90001c13088 R15: ffff880330884d80
      [ 1959.150021] FS:  00007f8926ba3840(0000) GS:ffff880333440000(0000) knlGS:0000000000000000
      [ 1959.159047] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
      [ 1959.165456] CR2: 0000000000000098 CR3: 000000032f48b000 CR4: 00000000000006e0
      [ 1959.173415] Stack:
      [ 1959.175656]  ffffc90001c13040 ffff880329334a00 ffff880330884ed0 ffff88032a957bdc
      [ 1959.183946]  ffff88032a957bb8 ffffffffa040f225 ffff880329334a30 ffff880300000000
      [ 1959.192233]  ffffffffa04133e0 ffff880329334b30 0000000830884d58 00000000569c58cf
      [ 1959.200521] Call Trace:
      [ 1959.203248]  [<ffffffffa040f225>] dm_exception_store_create+0x1d5/0x240 [dm_snapshot]
      [ 1959.211986]  [<ffffffffa040d310>] snapshot_ctr+0x140/0x630 [dm_snapshot]
      [ 1959.219469]  [<ffffffffa0005c44>] ? dm_split_args+0x64/0x150 [dm_mod]
      [ 1959.226656]  [<ffffffffa0005ea7>] dm_table_add_target+0x177/0x440 [dm_mod]
      [ 1959.234328]  [<ffffffffa0009203>] table_load+0x143/0x370 [dm_mod]
      [ 1959.241129]  [<ffffffffa00090c0>] ? retrieve_status+0x1b0/0x1b0 [dm_mod]
      [ 1959.248607]  [<ffffffffa0009e35>] ctl_ioctl+0x255/0x4d0 [dm_mod]
      [ 1959.255307]  [<ffffffff813304e2>] ? memzero_explicit+0x12/0x20
      [ 1959.261816]  [<ffffffffa000a0c3>] dm_ctl_ioctl+0x13/0x20 [dm_mod]
      [ 1959.268615]  [<ffffffff81215eb6>] do_vfs_ioctl+0xa6/0x5c0
      [ 1959.274637]  [<ffffffff81120d2f>] ? __audit_syscall_entry+0xaf/0x100
      [ 1959.281726]  [<ffffffff81003176>] ? do_audit_syscall_entry+0x66/0x70
      [ 1959.288814]  [<ffffffff81216449>] SyS_ioctl+0x79/0x90
      [ 1959.294450]  [<ffffffff8167e4ae>] entry_SYSCALL_64_fastpath+0x12/0x71
      ...
      [ 1959.323277] RIP  [<ffffffffa040efba>] dm_exception_store_set_chunk_size+0x7a/0x110 [dm_snapshot]
      [ 1959.333090]  RSP <ffff88032a957b30>
      [ 1959.336978] CR2: 0000000000000098
      [ 1959.344121] ---[ end trace b049991ccad1169e ]---
      
      Fixes: https://bugzilla.redhat.com/show_bug.cgi?id=1195899
      Cc: stable@vger.kernel.org
      Signed-off-by: default avatarDing Xiang <dingxiang@huawei.com>
      Signed-off-by: default avatarMike Snitzer <snitzer@redhat.com>
      4df2bf46
    • Joe Thornber's avatar
      dm cache: make the 'mq' policy an alias for 'smq' · 9ed84698
      Joe Thornber authored
      smq seems to be performing better than the old mq policy in all
      situations, as well as using a quarter of the memory.
      
      Make 'mq' an alias for 'smq' when choosing a cache policy.  The tunables
      that were present for the old mq are faked, and have no effect.  mq
      should be considered deprecated now.
      Signed-off-by: default avatarJoe Thornber <ejt@redhat.com>
      Signed-off-by: default avatarMike Snitzer <snitzer@redhat.com>
      9ed84698
    • Bob Liu's avatar
      dm: drop unnecessary assignment of md->queue · e233d800
      Bob Liu authored
      md->queue and q are the same thing in dm_old_init_request_queue() and
      dm_mq_init_request_queue().
      
      Also drop the temporary 'struct request_queue *q' in
      dm_old_init_request_queue().
      Signed-off-by: default avatarBob Liu <bob.liu@oracle.com>
      Signed-off-by: default avatarMike Snitzer <snitzer@redhat.com>
      e233d800
    • Mike Snitzer's avatar
      dm: reorder 'struct mapped_device' members to fix alignment and holes · 032482fd
      Mike Snitzer authored
      Saves 16 bytes by eliminating 4 4byte holes but more importantly:
      numerous members that crossed cachelines were fixed.
      Signed-off-by: default avatarMike Snitzer <snitzer@redhat.com>
      032482fd
    • Mike Snitzer's avatar
      dm: remove dummy definition of 'struct dm_table' · 1d3aa6f6
      Mike Snitzer authored
      Change the map pointer in 'struct mapped_device' from 'struct dm_table
      __rcu *' to 'void __rcu *' to avoid the need for the dummy definition.
      Signed-off-by: default avatarMike Snitzer <snitzer@redhat.com>
      1d3aa6f6
    • Mike Snitzer's avatar
      dm: add 'dm_numa_node' module parameter · 115485e8
      Mike Snitzer authored
      Allows user to control which NUMA node the memory for DM device
      structures (e.g. mapped_device, request_queue, gendisk, blk_mq_tag_set)
      is allocated from.
      
      Defaults to NUMA_NO_NODE (-1).  Allowable range is from -1 until the
      last online NUMA node id.
      Signed-off-by: default avatarMike Snitzer <snitzer@redhat.com>
      115485e8
    • Mike Snitzer's avatar
    • Mike Snitzer's avatar
      dm mpath: cleanup reinstate_path() et al based on code review · ec31f3f7
      Mike Snitzer authored
      fail_path() will print a "Failing path ..." message but reinstate_path()
      doesn't print a "Reinstating path ...".  Add that message to
      reinstate_path() to add symmetry and aid system debugging.
      
      Remove reinstate_path()'s check for the path_selector providing
      .reinstate_path hook.  All path selectors provide this and any future
      ones must too.
      
      activate_path() calls pg_init_done() with SCSI_DH_DEV_OFFLINED but
      pg_init_done() doesn't expicitly handle it in its swicth statement.  Add
      SCSI_DH_DEV_OFFLINED to the default case.
      Signed-off-by: default avatarMike Snitzer <snitzer@redhat.com>
      ec31f3f7
  2. 23 Feb, 2016 15 commits
  3. 22 Feb, 2016 11 commits
    • Mike Snitzer's avatar
      dm: allocate blk_mq_tag_set rather than embed in mapped_device · 1c357a1e
      Mike Snitzer authored
      The blk_mq_tag_set is only needed for dm-mq support.  There is point
      wasting space in 'struct mapped_device' for non-dm-mq devices.
      Signed-off-by: default avatarMike Snitzer <snitzer@redhat.com>
      Signed-off-by: Dan Carpenter <dan.carpenter@oracle.com> # check kzalloc return
      1c357a1e
    • Mike Snitzer's avatar
      dm: add 'dm_mq_nr_hw_queues' and 'dm_mq_queue_depth' module params · faad87df
      Mike Snitzer authored
      Allow user to change these values via module params or sysfs.
      
      'dm_mq_nr_hw_queues' defaults to 1 (max 32).
      
      'dm_mq_queue_depth' defaults to 2048 (up from 64, which proved far too
      small under moderate sized workloads -- the dm-multipath device would
      continuously block waiting for tags (requests) to become available).
      The maximum is BLK_MQ_MAX_DEPTH (currently 10240).
      
      Keep in mind the total number of pre-allocated requests per
      request-based dm-mq device is 'dm_mq_nr_hw_queues' * 'dm_mq_queue_depth'
      (currently 2048).
      Signed-off-by: default avatarMike Snitzer <snitzer@redhat.com>
      faad87df
    • Mike Snitzer's avatar
      dm: optimize dm_request_fn() · c91852ff
      Mike Snitzer authored
      DM multipath is the only request-based DM target -- which only supports
      tables with a single target that is immutable.  Leverage this fact in
      dm_request_fn().
      Signed-off-by: default avatarMike Snitzer <snitzer@redhat.com>
      c91852ff
    • Mike Snitzer's avatar
      dm: optimize dm_mq_queue_rq() · 16f12266
      Mike Snitzer authored
      DM multipath is the only dm-mq target.  But that aside, request-based DM
      only supports tables with a single target that is immutable.  Leverage
      this fact in dm_mq_queue_rq() by using the 'immutable_target' stored in
      the mapped_device when the table was made active.  This saves the need
      to even take the read-side of the SRCU via dm_{get,put}_live_table.
      
      If the active DM table does not have an immutable target (e.g. "error"
      target was swapped in) then fallback to the slow-path where the target
      is looked up from the live table.
      Signed-off-by: default avatarMike Snitzer <snitzer@redhat.com>
      16f12266
    • Mike Snitzer's avatar
      dm: set DM_TARGET_WILDCARD feature on "error" target · f083b09b
      Mike Snitzer authored
      The DM_TARGET_WILDCARD feature indicates that the "error" target may
      replace any target; even immutable targets.  This feature will be useful
      to preserve the ability to replace the "multipath" target even once it
      is formally converted over to having the DM_TARGET_IMMUTABLE feature.
      
      Also, implicit in the DM_TARGET_WILDCARD feature flag being set is that
      .map, .map_rq, .clone_and_map_rq and .release_clone_rq are all defined
      in the target_type.
      Signed-off-by: default avatarMike Snitzer <snitzer@redhat.com>
      f083b09b
    • Mike Snitzer's avatar
      dm: cleanup dm_any_congested() · e522c039
      Mike Snitzer authored
      The request-based DM support for checking queue congestion doesn't
      require access to the live DM table.
      Signed-off-by: default avatarMike Snitzer <snitzer@redhat.com>
      e522c039
    • Mike Snitzer's avatar
      dm: remove unused dm_get_rq_mapinfo() · ae6ad75e
      Mike Snitzer authored
      Signed-off-by: default avatarMike Snitzer <snitzer@redhat.com>
      ae6ad75e
    • Mike Snitzer's avatar
      dm: fix excessive dm-mq context switching · 6acfe68b
      Mike Snitzer authored
      Request-based DM's blk-mq support (dm-mq) was reported to be 50% slower
      than if an underlying null_blk device were used directly.  One of the
      reasons for this drop in performance is that blk_insert_clone_request()
      was calling blk_mq_insert_request() with @async=true.  This forced the
      use of kblockd_schedule_delayed_work_on() to run the blk-mq hw queues
      which ushered in ping-ponging between process context (fio in this case)
      and kblockd's kworker to submit the cloned request.  The ftrace
      function_graph tracer showed:
      
        kworker-2013  =>   fio-12190
        fio-12190    =>  kworker-2013
        ...
        kworker-2013  =>   fio-12190
        fio-12190    =>  kworker-2013
        ...
      
      Fixing blk_insert_clone_request()'s blk_mq_insert_request() call to
      _not_ use kblockd to submit the cloned requests isn't enough to
      eliminate the observed context switches.
      
      In addition to this dm-mq specific blk-core fix, there are 2 DM core
      fixes to dm-mq that (when paired with the blk-core fix) completely
      eliminate the observed context switching:
      
      1)  don't blk_mq_run_hw_queues in blk-mq request completion
      
          Motivated by desire to reduce overhead of dm-mq, punting to kblockd
          just increases context switches.
      
          In my testing against a really fast null_blk device there was no benefit
          to running blk_mq_run_hw_queues() on completion (and no other blk-mq
          driver does this).  So hopefully this change doesn't induce the need for
          yet another revert like commit 621739b0 !
      
      2)  use blk_mq_complete_request() in dm_complete_request()
      
          blk_complete_request() doesn't offer the traditional q->mq_ops vs
          .request_fn branching pattern that other historic block interfaces
          do (e.g. blk_get_request).  Using blk_mq_complete_request() for
          blk-mq requests is important for performance.  It should be noted
          that, like blk_complete_request(), blk_mq_complete_request() doesn't
          natively handle partial completions -- but the request-based
          DM-multipath target does provide the required partial completion
          support by dm.c:end_clone_bio() triggering requeueing of the request
          via dm-mpath.c:multipath_end_io()'s return of DM_ENDIO_REQUEUE.
      
      dm-mq fix #2 is _much_ more important than #1 for eliminating the
      context switches.
      Before: cpu          : usr=15.10%, sys=59.39%, ctx=7905181, majf=0, minf=475
      After:  cpu          : usr=20.60%, sys=79.35%, ctx=2008, majf=0, minf=472
      
      With these changes multithreaded async read IOPs improved from ~950K
      to ~1350K for this dm-mq stacked on null_blk test-case.  The raw read
      IOPs of the underlying null_blk device for the same workload is ~1950K.
      
      Fixes: 7fb4898e ("block: add blk-mq support to blk_insert_cloned_request()")
      Fixes: bfebd1cd ("dm: add full blk-mq support to request-based DM")
      Cc: stable@vger.kernel.org # 4.1+
      Reported-by: default avatarSagi Grimberg <sagig@dev.mellanox.co.il>
      Signed-off-by: default avatarMike Snitzer <snitzer@redhat.com>
      Acked-by: default avatarJens Axboe <axboe@kernel.dk>
      6acfe68b
    • Mike Snitzer's avatar
      dm: fix sparse "unexpected unlock" warnings in ioctl code · 956a4025
      Mike Snitzer authored
      Rename dm_get_live_table_for_ioctl to dm_grab_bdev_for_ioctl and have it
      do the dm_{get,put}_live_table() rather than split those operations.
      
      The dm_grab_bdev_for_ioctl() callers only care about the block_device
      associated with a singleton DM device so there isn't any need to retain
      a reference to the live DM table.  It is sufficient to:
      1) dm_get_live_table()
      2) bdgrab() the bdev associated with the singleton table's target
      3) dm_put_live_table()
      4) bdput() the bdev
      Signed-off-by: default avatarMike Snitzer <snitzer@redhat.com>
      956a4025
    • Mike Snitzer's avatar
      dm: do not return target from dm_get_live_table_for_ioctl() · 66482026
      Mike Snitzer authored
      None of the callers actually used the returned target.
      Also, just reuse bdev pointer passed to dm_blk_ioctl().
      Signed-off-by: default avatarMike Snitzer <snitzer@redhat.com>
      66482026
    • Mike Snitzer's avatar
      dm: fix dm_rq_target_io leak on faults with .request_fn DM w/ blk-mq paths · 4328daa2
      Mike Snitzer authored
      Using request-based DM mpath configured with the following stacking
      (.request_fn DM mpath ontop of scsi-mq paths):
      
      echo Y > /sys/module/scsi_mod/parameters/use_blk_mq
      echo N > /sys/module/dm_mod/parameters/use_blk_mq
      
      'struct dm_rq_target_io' would leak if a request is requeued before a
      blk-mq clone is allocated (or fails to allocate).  free_rq_tio()
      wasn't being called.
      
      kmemleak reported:
      
      unreferenced object 0xffff8800b90b98c0 (size 112):
        comm "kworker/7:1H", pid 5692, jiffies 4295056109 (age 78.589s)
        hex dump (first 32 bytes):
          00 d0 5c 2c 03 88 ff ff 40 00 bf 01 00 c9 ff ff  ..\,....@.......
          e0 d9 b1 34 00 88 ff ff 00 00 00 00 00 00 00 00  ...4............
        backtrace:
          [<ffffffff81672b6e>] kmemleak_alloc+0x4e/0xb0
          [<ffffffff811dbb63>] kmem_cache_alloc+0xc3/0x1e0
          [<ffffffff8117eae5>] mempool_alloc_slab+0x15/0x20
          [<ffffffff8117ec1e>] mempool_alloc+0x6e/0x170
          [<ffffffffa00029ac>] dm_old_prep_fn+0x3c/0x180 [dm_mod]
          [<ffffffff812fbd78>] blk_peek_request+0x168/0x290
          [<ffffffffa0003e62>] dm_request_fn+0xb2/0x1b0 [dm_mod]
          [<ffffffff812f66e3>] __blk_run_queue+0x33/0x40
          [<ffffffff812f9585>] blk_delay_work+0x25/0x40
          [<ffffffff81096fff>] process_one_work+0x14f/0x3d0
          [<ffffffff81097715>] worker_thread+0x125/0x4b0
          [<ffffffff8109ce88>] kthread+0xd8/0xf0
          [<ffffffff8167cb8f>] ret_from_fork+0x3f/0x70
          [<ffffffffffffffff>] 0xffffffffffffffff
      
      crash> struct -o dm_rq_target_io
      struct dm_rq_target_io {
          ...
      }
      SIZE: 112
      
      Fixes: e5863d9a ("dm: allocate requests in target when stacking on blk-mq devices")
      Cc: stable@vger.kernel.org # 4.0+
      Signed-off-by: default avatarMike Snitzer <snitzer@redhat.com>
      4328daa2
  4. 20 Feb, 2016 3 commits
    • Linus Torvalds's avatar
      Linux 4.5-rc5 · 81f70ba2
      Linus Torvalds authored
      81f70ba2
    • Linus Torvalds's avatar
      Merge branch 'x86-urgent-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip · 0389075e
      Linus Torvalds authored
      Pull x86 fixes from Ingo Molnar:
       "This is unusually large, partly due to the EFI fixes that prevent
        accidental deletion of EFI variables through efivarfs that may brick
        machines.  These fixes are somewhat involved to maintain compatibility
        with existing install methods and other usage modes, while trying to
        turn off the 'rm -rf' bricking vector.
      
        Other fixes are for large page ioremap()s and for non-temporal
        user-memcpy()s"
      
      * 'x86-urgent-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip:
        x86/mm: Fix vmalloc_fault() to handle large pages properly
        hpet: Drop stale URLs
        x86/uaccess/64: Handle the caching of 4-byte nocache copies properly in __copy_user_nocache()
        x86/uaccess/64: Make the __copy_user_nocache() assembly code more readable
        lib/ucs2_string: Correct ucs2 -> utf8 conversion
        efi: Add pstore variables to the deletion whitelist
        efi: Make efivarfs entries immutable by default
        efi: Make our variable validation list include the guid
        efi: Do variable name validation tests in utf8
        efi: Use ucs2_as_utf8 in efivarfs instead of open coding a bad version
        lib/ucs2_string: Add ucs2 -> utf8 helper functions
      0389075e
    • Linus Torvalds's avatar
      Merge branch 'perf-urgent-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip · 06b74c65
      Linus Torvalds authored
      Pull perf fixes from Ingo Molnar:
       "A handful of CPU hotplug related fixes"
      
      * 'perf-urgent-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip:
        perf/core: Plug potential memory leak in CPU_UP_PREPARE
        perf/core: Remove the bogus and dangerous CPU_DOWN_FAILED hotplug state
        perf/core: Remove bogus UP_CANCELED hotplug state
        perf/x86/amd/uncore: Plug reference leak
      06b74c65