1. 03 May, 2018 6 commits
    • Coly Li's avatar
      bcache: use pr_info() to inform duplicated CACHE_SET_IO_DISABLE set · 09a44ca2
      Coly Li authored
      It is possible that multiple I/O requests hits on failed cache device or
      backing device, therefore it is quite common that CACHE_SET_IO_DISABLE is
      set already when a task tries to set the bit from bch_cache_set_error().
      Currently the message "CACHE_SET_IO_DISABLE already set" is printed by
      pr_warn(), which might mislead users to think a serious fault happens in
      source code.
      
      This patch uses pr_info() to print the information in such situation,
      avoid extra worries. This information is helpful to understand bcache
      behavior in cache device failures, so I still keep them in source code.
      
      Fixes: 771f393e ("bcache: add CACHE_SET_IO_DISABLE to struct cache_set flags")
      Signed-off-by: default avatarColy Li <colyli@suse.de>
      Reviewed-by: default avatarHannes Reinecke <hare@suse.com>
      Signed-off-by: default avatarJens Axboe <axboe@kernel.dk>
      09a44ca2
    • Coly Li's avatar
      bcache: set dc->io_disable to true in conditional_stop_bcache_device() · 4fd8e138
      Coly Li authored
      Commit 7e027ca4 ("bcache: add stop_when_cache_set_failed option to
      backing device") adds stop_when_cache_set_failed option and stops bcache
      device if stop_when_cache_set_failed is auto and there is dirty data on
      broken cache device. There might exists a small time gap that the cache
      set is released and set to NULL but bcache device is not released yet
      (because they are released in parallel). During this time gap, dc->c is
      NULL so CACHE_SET_IO_DISABLE won't be checked, and dc->io_disable is still
      false, so new coming I/O requests will be accepted and directly go into
      backing device as no cache set attached to. If there is dirty data on
      cache device, this behavior may introduce potential inconsistent data.
      
      This patch sets dc->io_disable to true before calling bcache_device_stop()
      to make sure the backing device will reject new coming I/O request as
      well, so even in the small time gap no I/O will directly go into backing
      device to corrupt data consistency.
      
      Fixes: 7e027ca4 ("bcache: add stop_when_cache_set_failed option to backing device")
      Signed-off-by: default avatarColy Li <colyli@suse.de>
      Reviewed-by: default avatarHannes Reinecke <hare@suse.com>
      Signed-off-by: default avatarJens Axboe <axboe@kernel.dk>
      4fd8e138
    • Coly Li's avatar
      bcache: add wait_for_kthread_stop() in bch_allocator_thread() · ecb2ba8c
      Coly Li authored
      When CACHE_SET_IO_DISABLE is set on cache set flags, bcache allocator
      thread routine bch_allocator_thread() may stop the while-loops and
      exit. Then it is possible to observe the following kernel oops message,
      
      [  631.068366] bcache: bch_btree_insert() error -5
      [  631.069115] bcache: cached_dev_detach_finish() Caching disabled for sdf
      [  631.070220] BUG: unable to handle kernel NULL pointer dereference at 0000000000000000
      [  631.070250] PGD 0 P4D 0
      [  631.070261] Oops: 0002 [#1] SMP PTI
      [snipped]
      [  631.070578] Workqueue: events cache_set_flush [bcache]
      [  631.070597] RIP: 0010:exit_creds+0x1b/0x50
      [  631.070610] RSP: 0018:ffffc9000705fe08 EFLAGS: 00010246
      [  631.070626] RAX: 0000000000000001 RBX: ffff880a622ad300 RCX: 000000000000000b
      [  631.070645] RDX: 0000000000000601 RSI: 000000000000000c RDI: 0000000000000000
      [  631.070663] RBP: ffff880a622ad300 R08: ffffea00190c66e0 R09: 0000000000000200
      [  631.070682] R10: ffff880a48123000 R11: ffff880000000000 R12: 0000000000000000
      [  631.070700] R13: ffff880a4b160e40 R14: ffff880a4b160000 R15: 0ffff880667e2530
      [  631.070719] FS:  0000000000000000(0000) GS:ffff880667e00000(0000) knlGS:0000000000000000
      [  631.070740] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
      [  631.070755] CR2: 0000000000000000 CR3: 000000000200a001 CR4: 00000000003606e0
      [  631.070774] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
      [  631.070793] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
      [  631.070811] Call Trace:
      [  631.070828]  __put_task_struct+0x55/0x160
      [  631.070845]  kthread_stop+0xee/0x100
      [  631.070863]  cache_set_flush+0x11d/0x1a0 [bcache]
      [  631.070879]  process_one_work+0x146/0x340
      [  631.070892]  worker_thread+0x47/0x3e0
      [  631.070906]  kthread+0xf5/0x130
      [  631.070917]  ? max_active_store+0x60/0x60
      [  631.070930]  ? kthread_bind+0x10/0x10
      [  631.070945]  ret_from_fork+0x35/0x40
      [snipped]
      [  631.071017] RIP: exit_creds+0x1b/0x50 RSP: ffffc9000705fe08
      [  631.071033] CR2: 0000000000000000
      [  631.071045] ---[ end trace 011c63a24b22c927 ]---
      [  631.071085] bcache: bcache_device_free() bcache0 stopped
      
      The reason is when cache_set_flush() tries to call kthread_stop() to stop
      allocator thread, but it exits already due to cache device I/O errors.
      
      This patch adds wait_for_kthread_stop() at tail of bch_allocator_thread(),
      to prevent the thread routine exiting directly. Then the allocator thread
      can be blocked at wait_for_kthread_stop() and wait for cache_set_flush()
      to stop it by calling kthread_stop().
      
      changelog:
      v3: add Reviewed-by from Hannnes.
      v2: not directly return from allocator_wait(), move 'return 0' to tail of
          bch_allocator_thread().
      v1: initial version.
      
      Fixes: 771f393e ("bcache: add CACHE_SET_IO_DISABLE to struct cache_set flags")
      Signed-off-by: default avatarColy Li <colyli@suse.de>
      Reviewed-by: default avatarHannes Reinecke <hare@suse.com>
      Signed-off-by: default avatarJens Axboe <axboe@kernel.dk>
      ecb2ba8c
    • Coly Li's avatar
      bcache: count backing device I/O error for writeback I/O · bf78980f
      Coly Li authored
      Commit c7b7bd07 ("bcache: add io_disable to struct cached_dev")
      counts backing device I/O requets and set dc->io_disable to true if error
      counters exceeds dc->io_error_limit. But it only counts I/O errors for
      regular I/O request, neglects errors of write back I/Os when backing device
      is offline.
      
      This patch counts the errors of writeback I/Os, in dirty_endio() if
      bio->bi_status is  not 0, it means error happens when writing dirty keys
      to backing device, then bch_count_backing_io_errors() is called.
      
      By this fix, even there is no reqular I/O request coming, if writeback I/O
      errors exceed dc->io_error_limit, the bcache device may still be stopped
      for the broken backing device.
      
      Fixes: c7b7bd07 ("bcache: add io_disable to struct cached_dev")
      Signed-off-by: default avatarColy Li <colyli@suse.de>
      Reviewed-by: default avatarHannes Reinecke <hare@suse.com>
      Signed-off-by: default avatarJens Axboe <axboe@kernel.dk>
      bf78980f
    • Coly Li's avatar
      bcache: set CACHE_SET_IO_DISABLE in bch_cached_dev_error() · 6147305c
      Coly Li authored
      Commit c7b7bd07 ("bcache: add io_disable to struct cached_dev") tries
      to stop bcache device by calling bcache_device_stop() when too many I/O
      errors happened on backing device. But if there is internal I/O happening
      on cache device (writeback scan, garbage collection, etc), a regular I/O
      request triggers the internal I/Os may still holds a refcount of dc->count,
      and the refcount may only be dropped after the internal I/O stopped.
      
      By this patch, bch_cached_dev_error() will check if the backing device is
      attached to a cache set, if yes that CACHE_SET_IO_DISABLE will be set to
      flags of this cache set. Then internal I/Os on cache device will be
      rejected and stopped immediately, and the bcache device can be stopped.
      
      For people who are not familiar with the interesting refcount dependance,
      let me explain a bit more how the fix works. Example the writeback thread
      will scan cache device for dirty data writeback purpose. Before it stopps,
      it holds a refcount of dc->count. When CACHE_SET_IO_DISABLE bit is set,
      the internal I/O will stopped and the while-loop in bch_writeback_thread()
      quits and calls cached_dev_put() to drop dc->count. If this is the last
      refcount to drop, then cached_dev_detach_finish() will be called. In this
      call back function, in turn closure_put(dc->disk.cl) is called to drop a
      refcount of closure dc->disk.cl. If this is the last refcount of this
      closure to drop, then cached_dev_flush() will be called. Then the cached
      device is freed. So if CACHE_SET_IO_DISABLE is not set, the bache device
      can not be stopped until all inernal cache device I/O stopped. For large
      size cache device, and writeback thread competes locks with gc thread,
      there might be a quite long time to wait.
      
      Fixes: c7b7bd07 ("bcache: add io_disable to struct cached_dev")
      Signed-off-by: default avatarColy Li <colyli@suse.de>
      Reviewed-by: default avatarHannes Reinecke <hare@suse.com>
      Signed-off-by: default avatarJens Axboe <axboe@kernel.dk>
      6147305c
    • Coly Li's avatar
      bcache: store disk name in struct cache and struct cached_dev · 6e916a7e
      Coly Li authored
      Current code uses bdevname() or bio_devname() to reference gendisk
      disk name when bcache needs to display the disk names in kernel message.
      It was safe before bcache device failure handling patch set merged in,
      because when devices are failed, there was deadlock to prevent bcache
      printing error messages with gendisk disk name. But after the failure
      handling patch set merged, the deadlock is fixed, so it is possible
      that the gendisk structure bdev->hd_disk is released when bdevname() is
      called to reference bdev->bd_disk->disk_name[]. This is why I receive
      bug report of NULL pointers deference panic.
      
      This patch stores gendisk disk name in a buffer inside struct cache and
      struct cached_dev, then print out the offline device name won't reference
      bdev->hd_disk anymore. And this patch also avoids extra function calls
      of bdevname() and bio_devnmae().
      
      Changelog:
      v3, add Reviewed-by from Hannes.
      v2, call bdevname() earlier in register_bdev()
      v1, first version with segguestion from Junhui Tang.
      
      Fixes: c7b7bd07 ("bcache: add io_disable to struct cached_dev")
      Fixes: 5138ac67 ("bcache: fix misleading error message in bch_count_io_errors()")
      Signed-off-by: default avatarColy Li <colyli@suse.de>
      Reviewed-by: default avatarHannes Reinecke <hare@suse.com>
      Signed-off-by: default avatarJens Axboe <axboe@kernel.dk>
      6e916a7e
  2. 26 Apr, 2018 6 commits
    • Omar Sandoval's avatar
      blk-mq: fix sysfs inflight counter · bf0ddaba
      Omar Sandoval authored
      When the blk-mq inflight implementation was added, /proc/diskstats was
      converted to use it, but /sys/block/$dev/inflight was not. Fix it by
      adding another helper to count in-flight requests by data direction.
      
      Fixes: f299b7c7 ("blk-mq: provide internal in-flight variant")
      Signed-off-by: default avatarOmar Sandoval <osandov@fb.com>
      Signed-off-by: default avatarJens Axboe <axboe@kernel.dk>
      bf0ddaba
    • Omar Sandoval's avatar
      blk-mq: count allocated but not started requests in iostats inflight · 6131837b
      Omar Sandoval authored
      In the legacy block case, we increment the counter right after we
      allocate the request, not when the driver handles it. In both the legacy
      and blk-mq cases, part_inc_in_flight() is called from
      blk_account_io_start() right after we've allocated the request. blk-mq
      only considers requests started requests as inflight, but this is
      inconsistent with the legacy definition and the intention in the code.
      This removes the started condition and instead counts all allocated
      requests.
      
      Fixes: f299b7c7 ("blk-mq: provide internal in-flight variant")
      Signed-off-by: default avatarOmar Sandoval <osandov@fb.com>
      Signed-off-by: default avatarJens Axboe <axboe@kernel.dk>
      6131837b
    • Linus Torvalds's avatar
      Merge tag 'for_v4.17-rc3' of git://git.kernel.org/pub/scm/linux/kernel/git/jack/linux-fs · 69bfd470
      Linus Torvalds authored
      Pull fsnotify fix from Jan Kara:
       "A fix of a fsnotify race causing panics / softlockups"
      
      * tag 'for_v4.17-rc3' of git://git.kernel.org/pub/scm/linux/kernel/git/jack/linux-fs:
        fsnotify: Fix fsnotify_mark_connector race
      69bfd470
    • Linus Torvalds's avatar
      Merge tag 'scsi-fixes' of git://git.kernel.org/pub/scm/linux/kernel/git/jejb/scsi · 3442097b
      Linus Torvalds authored
      Pull SCSI fixes from James Bottomley:
       "Eight bug fixes, one spelling update and one tracepoint addition.
      
        The most serious is probably the mptsas write same fix because it
        means anyone using these controllers sees errors when modern
        filesystems try to issue discards"
      
      * tag 'scsi-fixes' of git://git.kernel.org/pub/scm/linux/kernel/git/jejb/scsi:
        scsi: target: fix crash with iscsi target and dvd
        scsi: sd_zbc: Avoid that resetting a zone fails sporadically
        scsi: sd: Defer spinning up drive while SANITIZE is in progress
        scsi: megaraid_sas: Do not log an error if FW successfully initializes.
        scsi: ufs: add trace event for ufs upiu
        scsi: core: remove reference to scsi_show_extd_sense()
        scsi: mptsas: Disable WRITE SAME
        scsi: fnic: fix spelling mistake in fnic stats "Abord" -> "Abort"
        scsi: scsi_debug: IMMED related delay adjustments
        scsi: iscsi: respond to netlink with unicast when appropriate
      3442097b
    • Linus Torvalds's avatar
      Merge tag 'for-linus-20180425' of git://git.kernel.dk/linux-block · 8fba70b0
      Linus Torvalds authored
      Pull block updates from Jens Axboe:
       "I ended up sitting on this about a week longer than I wanted to, since
        we were hashing out details with a timeout change. I've now killed
        that patch, so we can flush the existing queue in due time.
      
        This contains:
      
         - Fix for an old regression, where entering the queue can be
           disturbed by a signal to the process. This can cause spurious EIO.
           Fix from Alan Jenkins.
      
         - cdrom information leak fix from Dan.
      
         - Trivial helper for testing queue FUA from Dave Chinner, part of his
           O_DIRECT FUA series.
      
         - Series of swim fixes from Finn that actually makes it work again.
      
         - Loop O_DIRECT corruption fix, which caused data corruption in
           production for us. From me.
      
         - BFQ crash fix from me.
      
         - bcache maintainer update. Michael no longer has the time to do it,
           Coly has stepped up to serve as the new maintainer.
      
         - blkcg locking fixes from Jiang Biao.
      
         - Revert of a change from this merge window from Ming, that causes an
           issue on some hardware.
      
         - Minor clarification doc addition from Linus Walleij"
      
      * tag 'for-linus-20180425' of git://git.kernel.dk/linux-block: (22 commits)
        Revert "blk-mq: remove code for dealing with remapping queue"
        block: mq: Add some minor doc for core structs
        bcache: mark Coly Li as bcache maintainer
        MAINTAINERS: Remove me as maintainer of bcache
        blkcg: init root blkcg_gq under lock
        blkcg: small fix on comment in blkcg_init_queue
        blkcg: don't hold blkcg lock when deactivating policy
        block: add blk_queue_fua() helper function
        cdrom: information leak in cdrom_ioctl_media_changed()
        bfq-iosched: ensure to clear bic/bfqq pointers when preparing request
        blk-mq: start request gstate with gen 1
        block/swim: Select appropriate drive on device open
        block/swim: Fix IO error at end of medium
        block/swim: Check drive type
        block/swim: Rename macros to avoid inconsistent inverted logic
        block/swim: Don't log an error message for an invalid ioctl
        block/swim: Remove extra put_disk() call from error path
        block/swim: Fix array bounds check
        m68k/mac: Don't remap SWIM MMIO region
        loop: handle short DIO reads
        ...
      8fba70b0
    • Linus Torvalds's avatar
      Merge tag 'riscv-for-linus-4.17-rc3' of... · c6dc3e71
      Linus Torvalds authored
      Merge tag 'riscv-for-linus-4.17-rc3' of git://git.kernel.org/pub/scm/linux/kernel/git/palmer/riscv-linux
      
      Pull RISC-V fixes from Palmer Dabbelt:
       "This contains three small fixes related to the RISC-V port that I'd
        like to target for 4.17-rc3:
      
         - a Kconfig cleanup to select DMA_DIRECT_OPS instead of redefining it
           in arch/riscv
      
         - the removal of asm/handle_irq.h, which doesn't exist, from our arch
           header list
      
         - the addition of "-no-pie" the link rules for our VDSO-related
           files, which fixes the build on systems where PIE is enabled by
           default"
      
      * tag 'riscv-for-linus-4.17-rc3' of git://git.kernel.org/pub/scm/linux/kernel/git/palmer/riscv-linux:
        RISC-V: build vdso-dummy.o with -no-pie
        riscv: there is no <asm/handle_irq.h>
        riscv: select DMA_DIRECT_OPS instead of redefining it
      c6dc3e71
  3. 25 Apr, 2018 6 commits
  4. 24 Apr, 2018 22 commits