1. 23 Oct, 2018 10 commits
    • Chao Yu's avatar
      f2fs: guarantee journalled quota data by checkpoint · af033b2a
      Chao Yu authored
      For journalled quota mode, let checkpoint to flush dquot dirty data
      and quota file data to guarntee persistence of all quota sysfile in
      last checkpoint, by this way, we can avoid corrupting quota sysfile
      when encountering SPO.
      
      The implementation is as below:
      
      1. add a global state SBI_QUOTA_NEED_FLUSH to indicate that there is
      cached dquot metadata changes in quota subsystem, and later checkpoint
      should:
       a) flush dquot metadata into quota file.
       b) flush quota file to storage to keep file usage be consistent.
      
      2. add a global state SBI_QUOTA_NEED_REPAIR to indicate that quota
      operation failed due to -EIO or -ENOSPC, so later,
       a) checkpoint will skip syncing dquot metadata.
       b) CP_QUOTA_NEED_FSCK_FLAG will be set in last cp pack to give a
          hint for fsck repairing.
      
      3. add a global state SBI_QUOTA_SKIP_FLUSH, in checkpoint, if quota
      data updating is very heavy, it may cause hungtask in block_operation().
      To avoid this, if our retry time exceed threshold, let's just skip
      flushing and retry in next checkpoint().
      Signed-off-by: default avatarWeichao Guo <guoweichao@huawei.com>
      Signed-off-by: default avatarChao Yu <yuchao0@huawei.com>
      [Jaegeuk Kim: avoid warnings and set fsck flag]
      Signed-off-by: default avatarJaegeuk Kim <jaegeuk@kernel.org>
      af033b2a
    • Sheng Yong's avatar
      f2fs: cleanup dirty pages if recover failed · 26b5a079
      Sheng Yong authored
      During recover, we will try to create new dentries for inodes with
      dentry_mark. But if the parent is missing (e.g. killed by fsck),
      recover will break. But those recovered dirty pages are not cleanup.
      This will hit f2fs_bug_on:
      
      [   53.519566] F2FS-fs (loop0): Found nat_bits in checkpoint
      [   53.539354] F2FS-fs (loop0): recover_inode: ino = 5, name = file, inline = 3
      [   53.539402] F2FS-fs (loop0): recover_dentry: ino = 5, name = file, dir = 0, err = -2
      [   53.545760] F2FS-fs (loop0): Cannot recover all fsync data errno=-2
      [   53.546105] F2FS-fs (loop0): access invalid blkaddr:4294967295
      [   53.546171] WARNING: CPU: 1 PID: 1798 at fs/f2fs/checkpoint.c:163 f2fs_is_valid_blkaddr+0x26c/0x320
      [   53.546174] Modules linked in:
      [   53.546183] CPU: 1 PID: 1798 Comm: mount Not tainted 4.19.0-rc2+ #1
      [   53.546186] Hardware name: innotek GmbH VirtualBox/VirtualBox, BIOS VirtualBox 12/01/2006
      [   53.546191] RIP: 0010:f2fs_is_valid_blkaddr+0x26c/0x320
      [   53.546195] Code: 85 bb 00 00 00 48 89 df 88 44 24 07 e8 ad a8 db ff 48 8b 3b 44 89 e1 48 c7 c2 40 03 72 a9 48 c7 c6 e0 01 72 a9 e8 84 3c ff ff <0f> 0b 0f b6 44 24 07 e9 8a 00 00 00 48 8d bf 38 01 00 00 e8 7c a8
      [   53.546201] RSP: 0018:ffff88006c067768 EFLAGS: 00010282
      [   53.546208] RAX: 0000000000000000 RBX: ffff880068844200 RCX: ffffffffa83e1a33
      [   53.546211] RDX: 0000000000000000 RSI: 0000000000000008 RDI: ffff88006d51e590
      [   53.546215] RBP: 0000000000000005 R08: ffffed000daa3cb3 R09: ffffed000daa3cb3
      [   53.546218] R10: 0000000000000001 R11: ffffed000daa3cb2 R12: 00000000ffffffff
      [   53.546221] R13: ffff88006a1f8000 R14: 0000000000000200 R15: 0000000000000009
      [   53.546226] FS:  00007fb2f3646840(0000) GS:ffff88006d500000(0000) knlGS:0000000000000000
      [   53.546229] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
      [   53.546234] CR2: 00007f0fd77f0008 CR3: 00000000687e6002 CR4: 00000000000206e0
      [   53.546237] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
      [   53.546240] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
      [   53.546242] Call Trace:
      [   53.546248]  f2fs_submit_page_bio+0x95/0x740
      [   53.546253]  read_node_page+0x161/0x1e0
      [   53.546271]  ? truncate_node+0x650/0x650
      [   53.546283]  ? add_to_page_cache_lru+0x12c/0x170
      [   53.546288]  ? pagecache_get_page+0x262/0x2d0
      [   53.546292]  __get_node_page+0x200/0x660
      [   53.546302]  f2fs_update_inode_page+0x4a/0x160
      [   53.546306]  f2fs_write_inode+0x86/0xb0
      [   53.546317]  __writeback_single_inode+0x49c/0x620
      [   53.546322]  writeback_single_inode+0xe4/0x1e0
      [   53.546326]  sync_inode_metadata+0x93/0xd0
      [   53.546330]  ? sync_inode+0x10/0x10
      [   53.546342]  ? do_raw_spin_unlock+0xed/0x100
      [   53.546347]  f2fs_sync_inode_meta+0xe0/0x130
      [   53.546351]  f2fs_fill_super+0x287d/0x2d10
      [   53.546367]  ? vsnprintf+0x742/0x7a0
      [   53.546372]  ? f2fs_commit_super+0x180/0x180
      [   53.546379]  ? up_write+0x20/0x40
      [   53.546385]  ? set_blocksize+0x5f/0x140
      [   53.546391]  ? f2fs_commit_super+0x180/0x180
      [   53.546402]  mount_bdev+0x181/0x200
      [   53.546406]  mount_fs+0x94/0x180
      [   53.546411]  vfs_kern_mount+0x6c/0x1e0
      [   53.546415]  do_mount+0xe5e/0x1510
      [   53.546420]  ? fs_reclaim_release+0x9/0x30
      [   53.546424]  ? copy_mount_string+0x20/0x20
      [   53.546428]  ? fs_reclaim_acquire+0xd/0x30
      [   53.546435]  ? __might_sleep+0x2c/0xc0
      [   53.546440]  ? ___might_sleep+0x53/0x170
      [   53.546453]  ? __might_fault+0x4c/0x60
      [   53.546468]  ? _copy_from_user+0x95/0xa0
      [   53.546474]  ? memdup_user+0x39/0x60
      [   53.546478]  ksys_mount+0x88/0xb0
      [   53.546482]  __x64_sys_mount+0x5d/0x70
      [   53.546495]  do_syscall_64+0x65/0x130
      [   53.546503]  entry_SYSCALL_64_after_hwframe+0x44/0xa9
      [   53.547639] ---[ end trace b804d1ea2fec893e ]---
      
      So if recover fails, we need to drop all recovered data.
      Signed-off-by: default avatarSheng Yong <shengyong1@huawei.com>
      Reviewed-by: default avatarChao Yu <yuchao0@huawei.com>
      Signed-off-by: default avatarJaegeuk Kim <jaegeuk@kernel.org>
      26b5a079
    • Sahitya Tummala's avatar
      f2fs: fix data corruption issue with hardware encryption · 1e78e8bd
      Sahitya Tummala authored
      Direct IO can be used in case of hardware encryption. The following
      scenario results into data corruption issue in this path -
      
      Thread A -                          Thread B-
      -> write file#1 in direct IO
                                          -> GC gets kicked in
                                          -> GC submitted bio on meta mapping
      				       for file#1, but pending completion
      -> write file#1 again with new data
         in direct IO
                                          -> GC bio gets completed now
                                          -> GC writes old data to the new
                                             location and thus file#1 is
      				       corrupted.
      
      Fix this by submitting and waiting for pending io on meta mapping
      for direct IO case in f2fs_map_blocks().
      Signed-off-by: default avatarSahitya Tummala <stummala@codeaurora.org>
      Reviewed-by: default avatarChao Yu <yuchao0@huawei.com>
      Signed-off-by: default avatarJaegeuk Kim <jaegeuk@kernel.org>
      1e78e8bd
    • Chao Yu's avatar
      f2fs: fix to recover inode->i_flags of inode block during POR · 0c093b59
      Chao Yu authored
      Testcase to reproduce this bug:
      1. mkfs.f2fs /dev/sdd
      2. mount -t f2fs /dev/sdd /mnt/f2fs
      3. touch /mnt/f2fs/file
      4. sync
      5. chattr +a /mnt/f2fs/file
      6. xfs_io -a /mnt/f2fs/file -c "fsync"
      7. godown /mnt/f2fs
      8. umount /mnt/f2fs
      9. mount -t f2fs /dev/sdd /mnt/f2fs
      10. xfs_io /mnt/f2fs/file
      
      There is no error when opening this file w/o O_APPEND, but actually,
      we expect the correct result should be:
      
      /mnt/f2fs/file: Operation not permitted
      
      The root cause is, in recover_inode(), we recover inode->i_flags more
      than F2FS_I(inode)->i_flags, so fix it.
      Signed-off-by: default avatarChao Yu <yuchao0@huawei.com>
      Signed-off-by: default avatarJaegeuk Kim <jaegeuk@kernel.org>
      0c093b59
    • Chao Yu's avatar
      f2fs: spread f2fs_set_inode_flags() · 9149a5eb
      Chao Yu authored
      This patch changes codes as below:
      - use f2fs_set_inode_flags() to update i_flags atomically to avoid
      potential race.
      - synchronize F2FS_I(inode)->i_flags to inode->i_flags in
      f2fs_new_inode().
      - use f2fs_set_inode_flags() to simply codes in f2fs_quota_{on,off}.
      Signed-off-by: default avatarChao Yu <yuchao0@huawei.com>
      Signed-off-by: default avatarJaegeuk Kim <jaegeuk@kernel.org>
      9149a5eb
    • Chao Yu's avatar
      f2fs: fix to spread clear_cold_data() · 2baf0781
      Chao Yu authored
      We need to drop PG_checked flag on page as well when we clear PG_uptodate
      flag, in order to avoid treating the page as GCing one later.
      Signed-off-by: default avatarWeichao Guo <guoweichao@huawei.com>
      Signed-off-by: default avatarChao Yu <yuchao0@huawei.com>
      Signed-off-by: default avatarJaegeuk Kim <jaegeuk@kernel.org>
      2baf0781
    • Jaegeuk Kim's avatar
      Revert "f2fs: fix to clear PG_checked flag in set_page_dirty()" · 164a63fa
      Jaegeuk Kim authored
      This reverts commit 66110abc.
      
      If we clear the cold data flag out of the writeback flow, we can miscount
      -1 by end_io, which incurs a deadlock caused by all I/Os being blocked during
      heavy GC.
      
      Balancing F2FS Async:
       - IO (CP:    1, Data:   -1, Flush: (   0    0    1), Discard: (   ...
      
      GC thread:                              IRQ
      - move_data_page()
       - set_page_dirty()
        - clear_cold_data()
                                              - f2fs_write_end_io()
                                               - type = WB_DATA_TYPE(page);
                                                 here, we get wrong type
                                               - dec_page_count(sbi, type);
       - f2fs_wait_on_page_writeback()
      
      Cc: <stable@vger.kernel.org>
      Reported-and-Tested-by: default avatarPark Ju Hyung <qkrwngud825@gmail.com>
      Reviewed-by: default avatarChao Yu <yuchao0@huawei.com>
      Signed-off-by: default avatarJaegeuk Kim <jaegeuk@kernel.org>
      164a63fa
    • Jaegeuk Kim's avatar
      f2fs: account read IOs and use IO counts for is_idle · 5f9abab4
      Jaegeuk Kim authored
      This patch adds issued read IO counts which is under block layer.
      
      Chao modified a bit, since:
      
      Below race can cause reversed reference on F2FS_RD_DATA, there is
      the same issue in f2fs_submit_page_bio(), fix them by relocate
      __submit_bio() and inc_page_count.
      
      Thread A			Thread B
      - f2fs_write_begin
       - f2fs_submit_page_read
       - __submit_bio
      				- f2fs_read_end_io
      				 - __read_end_io
      				 - dec_page_count(, F2FS_RD_DATA)
       - inc_page_count(, F2FS_RD_DATA)
      Signed-off-by: default avatarChao Yu <yuchao0@huawei.com>
      Signed-off-by: default avatarJaegeuk Kim <jaegeuk@kernel.org>
      5f9abab4
    • Chao Yu's avatar
      f2fs: fix to account IO correctly for cgroup writeback · 78efac53
      Chao Yu authored
      Now, we have supported cgroup writeback, it depends on correctly IO
      account of specified filesystem.
      
      But in commit d1b3e72d ("f2fs: submit bio of in-place-update pages"),
      we split write paths from f2fs_submit_page_mbio() to two:
      - f2fs_submit_page_bio() for IPU path
      - f2fs_submit_page_bio() for OPU path
      
      But still we account write IO only in f2fs_submit_page_mbio(), result in
      incorrect IO account, fix it by adding missing IO account in IPU path.
      
      Fixes: d1b3e72d ("f2fs: submit bio of in-place-update pages")
      Signed-off-by: default avatarChao Yu <yuchao0@huawei.com>
      Signed-off-by: default avatarJaegeuk Kim <jaegeuk@kernel.org>
      78efac53
    • Chao Yu's avatar
      f2fs: fix to account IO correctly · 4c58ed07
      Chao Yu authored
      Below race can cause reversed reference on dirty count, fix it by
      relocating __submit_bio() and inc_page_count().
      
      Thread A				Thread B
      - f2fs_inplace_write_data
       - f2fs_submit_page_bio
        - __submit_bio
      					- f2fs_write_end_io
      					 - dec_page_count
        - inc_page_count
      
      Cc: <stable@vger.kernel.org>
      Fixes: d1b3e72d ("f2fs: submit bio of in-place-update pages")
      Signed-off-by: default avatarChao Yu <yuchao0@huawei.com>
      Signed-off-by: default avatarJaegeuk Kim <jaegeuk@kernel.org>
      4c58ed07
  2. 17 Oct, 2018 1 commit
  3. 16 Oct, 2018 10 commits
  4. 03 Oct, 2018 1 commit
    • Jaegeuk Kim's avatar
      f2fs: clear PageError on the read path · fb7d70db
      Jaegeuk Kim authored
      When running fault injection test, I hit somewhat wrong behavior in f2fs_gc ->
      gc_data_segment():
      
      0. fault injection generated some PageError'ed pages
      
      1. gc_data_segment
       -> f2fs_get_read_data_page(REQ_RAHEAD)
      
      2. move_data_page
       -> f2fs_get_lock_data_page()
        -> f2f_get_read_data_page()
         -> f2fs_submit_page_read()
          -> submit_bio(READ)
        -> return EIO due to PageError
        -> fail to move data
      Signed-off-by: default avatarJaegeuk Kim <jaegeuk@kernel.org>
      fb7d70db
  5. 01 Oct, 2018 6 commits
  6. 30 Sep, 2018 1 commit
  7. 29 Sep, 2018 1 commit
  8. 28 Sep, 2018 2 commits
  9. 26 Sep, 2018 8 commits