- 29 Apr, 2019 40 commits
-
-
Nikolay Borisov authored
Currently unallocated chunks are always trimmed. For example 2 consecutive trims on large storage would trim freespace twice irrespective of whether the space was actually allocated or not between those trims. Optimise this behavior by exploiting the newly introduced alloc_state tree of btrfs_device. A new CHUNK_TRIMMED bit is used to mark those unallocated chunks which have been trimmed and have not been allocated afterwards. On chunk allocation the respective underlying devices' physical space will have its CHUNK_TRIMMED flag cleared. This avoids submitting discards for space which hasn't been changed since the last time discard was issued. This applies to the single mount period of the filesystem as the information is not stored permanently. Signed-off-by:
Nikolay Borisov <nborisov@suse.com> Signed-off-by:
David Sterba <dsterba@suse.com>
-
Nikolay Borisov authored
This is used in more than one places so let's factor it out in ctree.h. No functional changes. Signed-off-by:
-
Nikolay Borisov authored
Now that these functions no longer require a handle to transaction to inspect pending/pinned chunks the argument can be removed. At the same time also remove any surrounding code which acquired the handle. Signed-off-by:
-
Jeff Mahoney authored
The pending chunks list contains chunks that are allocated in the current transaction but haven't been created yet. The pinned chunks list contains chunks that are being released in the current transaction. Both describe chunks that are not reflected on disk as in use but are unavailable just the same. The pending chunks list is anchored by the transaction handle, which means that we need to hold a reference to a transaction when working with the list. The way we use them is by iterating over both lists to perform comparisons on the stripes they describe for each device. This is backwards and requires that we keep a transaction handle open while we're trimming. This patchset adds an extent_io_tree to btrfs_device that maintains the allocation state of the device. Extents are set dirty when chunks are first allocated -- when the extent maps are added to the mapping tree. They're cleared when last removed -- when the extent maps are removed from the mapping tree. This matches the lifespan of the pending and pinned chunks list and allows us to do trims on unallocated space safely without pinning the transaction for what may be a lengthy operation. We can also use this io tree to mark which chunks have already been trimmed so we don't repeat the operation. Signed-off-by:
Jeff Mahoney <jeffm@suse.com> Signed-off-by:
-
Nikolay Borisov authored
Following the introduction of the alloc_state tree, some of the callees of btrfs_mapping_tree_free will have to interact with the btrfs_device of the constituent devices. Enable this by moving the code responsible for freeing devices after the last user (btrfs_mapping_tree_free). Otherwise the kernel could crash due to use-after-free. Signed-off-by:
-
Nikolay Borisov authored
btrfs_device structs are freed from RCU context since device iteration is protected by RCU. Currently this is achieved by using call_rcu since no blocking functions are called within btrfs_free_device. Future refactoring of pending/pinned chunks will require calling sleeping functions. This patch is in preparation for these changes by simply switching from RCU callbacks to explicit calls of synchronize_rcu and calling btrfs_free_device directly. This is functionally equivalent, making sure that there are no readers at that time. Signed-off-by:
-
Nikolay Borisov authored
It will be used in a future patch that will require modifying an extent_io_tree struct under a spinlock. Signed-off-by:
-
Nikolay Borisov authored
Rather than hijacking the existing defines let's just define new bits, with more descriptive names. Instead of using yet more (currently at 18) bits for the new flags, use the fact those flags will be specific to the device allocation tree so define them using existing EXTENT_* flags. Signed-off-by:
-
Nikolay Borisov authored
Chunks read from disk currently don't get their ->orig_block_len member set, in contrast when a new chunk is allocated, the respective extent_map's ->orig_block_len is assigned the size of the stripe of this chunk. Let's apply the same strategy for chunks which are read from disk, not only does this codify the invariant that ->orig_block_len always contains the size of the stripe for a chunk (when the em belongs to the mapping tree). But it's also a preparatory patch for further work around tracking chunk allocation in an extent tree rather than pinned/pending lists. Signed-off-by:
-
Nikolay Borisov authored
This function is going to be used to clear out the device extent allocation information. Give it a more generic name and export it. This is in preparation to replacing the pending/pinned chunk lists with an extent tree. No functional changes. Signed-off-by:
-
Nikolay Borisov authored
During device shrink pinned/pending chunks (i.e. those which have been deleted/created respectively, in the current transaction and haven't touched disk) need to be accounted when doing device shrink. Presently this happens after the main relocation loop in btrfs_shrink_device, which could lead to making another go in the body of the function. Since there is no hard requirement to perform pinned/pending chunks handling after the relocation loop, move the code before it. This leads to simplifying the code flow around - i.e. no need to use 'goto again'. A notable side effect of this change is that modification of the device's size requires a transaction to be started and committed before the relocation loop starts. This is necessary to ensure that relocation process sees the shrunk device size. Signed-off-by:
-
Nikolay Borisov authored
We currently overload the pending_chunks list to handle updating btrfs_device->commit_bytes used. We don't actually care about the extent mapping or even the device mapping for the chunk - we just need the device, and we can end up processing it multiple times. The fs_devices->resized_list does more or less the same thing, but with the disk size. They are called consecutively during commit and have more or less the same purpose. We can combine the two lists into a single list that attaches to the transaction and contains a list of devices that need updating. Since we always add the device to a list when we change bytes_used or disk_total_size, there's no harm in copying both values at once. Signed-off-by:
-
Nikolay Borisov authored
Up until now trimming the freespace was done irrespective of what the arguments of the FITRIM ioctl were. For example fstrim's -o/-l arguments will be entirely ignored. Fix it by correctly handling those paramter. This requires breaking if the found freespace extent is after the end of the passed range as well as completing trim after trimming fstrim_range::len bytes. Fixes: 499f377f ("btrfs: iterate over unused chunk space in FITRIM") CC: stable@vger.kernel.org # 4.4+ Signed-off-by:
-
Robbie Ko authored
Improve clone_range in two scenarios. 1. Remove the limit of inode size when find clone inodes We can do partial clone, so there is no need to limit the size of the candidate inode. When clone a range, we clone the legal range only by bytenr, offset, len, inode size. 2. In the scenarios of rewrite or clone_range, data_offset rarely matches exactly, so the chance of a clone is missed. e.g. 1. Write a 1M file dd if=/dev/zero of=1M bs=1M count=1 2. Clone 1M file cp --reflink 1M clone 3. Rewrite 4k on the clone file dd if=/dev/zero of=clone bs=4k count=1 conv=notrunc The disk layout is as follows: item 16 key (257 EXTENT_DATA 0) itemoff 15353 itemsize 53 extent data disk byte 1103101952 nr 1048576 extent data offset 0 nr 1048576 ram 1048576 extent compression(none) ... item 22 key (258 EXTENT_DATA 0) itemoff 14959 itemsize 53 extent data disk byte 1104150528 nr 4096 extent data offset 0 nr 4096 ram 4096 extent compression(none) item 23 key (258 EXTENT_DATA 4096) itemoff 14906 itemsize 53 extent data disk byte 1103101952 nr 1048576 extent data offset 4096 nr 1044480 ram 1048576 extent compression(none) When send, inode 258 file offset 4096~1048576 (item 23) has a chance to clone_range, but because data_offset does not match inode 257 (item 16), it causes missed clone and can only transfer actual data. Improve the problem by judging whether the current data_offset has overlap with the file extent item, and if so, adjusting offset and extent_len so that we can clone correctly. Reviewed-by:Filipe Manana <fdmanana@suse.com> Signed-off-by:
Robbie Ko <robbieko@synology.com> Signed-off-by:
-
Anand Jain authored
When an inode inherits property from its parent, we call btrfs_set_prop(). btrfs_set_prop() does an elaborate checks, which is not required in the context of inheriting a property. Instead just open-code only the required items from btrfs_set_prop() and then call btrfs_setxattr() directly. So now the only user of btrfs_set_prop() is gone, (except for the wraper function btrfs_set_prop_trans()). Reviewed-by:
Anand Jain <anand.jain@oracle.com> Reviewed-by:
-
Anand Jain authored
@device_name in mount_subvol() is not used, drop it. Also see: 5bedc48a ("btrfs: drop unused parameters from mount_subvol"). Signed-off-by:
-
David Sterba authored
We can read fs_info from extent buffer and can drop it from the parameters. Signed-off-by: -
David Sterba authored
We can read fs_info from extent buffer and can drop it from the parameters. Signed-off-by: -
David Sterba authored
We can read fs_info from extent buffer and can drop it from the parameters. Signed-off-by: -
David Sterba authored
We can read fs_info from extent buffer and can drop it from the parameters. Signed-off-by: -
David Sterba authored
We can read fs_info from extent buffer and can drop it from the parameters. Signed-off-by: -
David Sterba authored
We can read fs_info from extent buffer and can drop it from the parameters. Signed-off-by: -
David Sterba authored
We can read fs_info from extent buffer and can drop it from the parameters. Signed-off-by: -
David Sterba authored
We can read fs_info from extent buffer and can drop it from the parameters. Signed-off-by: -
David Sterba authored
We can read fs_info from extent buffer and can drop it from the parameters. Signed-off-by: -
David Sterba authored
We can read fs_info from extent buffer and can drop it from the parameters. Signed-off-by: -
David Sterba authored
We can read fs_info from extent buffer and can drop it from the parameters. Signed-off-by: -
David Sterba authored
We can read fs_info from extent buffer and can drop it from the parameters. Signed-off-by: -
David Sterba authored
We can read fs_info from extent buffer and can drop it from the parameters. Signed-off-by: -
David Sterba authored
We can read fs_info from extent buffer and can drop it from the parameters. Signed-off-by: -
Qu Wenruo authored
[BUG] When accessing a file on a crafted image, btrfs can crash in block layer: BUG: unable to handle kernel NULL pointer dereference at 0000000000000008 PGD 136501067 P4D 136501067 PUD 124519067 PMD 0 CPU: 3 PID: 0 Comm: swapper/3 Not tainted 5.0.0-rc8-default #252 RIP: 0010:end_bio_extent_readpage+0x144/0x700 Call Trace: <IRQ> blk_update_request+0x8f/0x350 blk_mq_end_request+0x1a/0x120 blk_done_softirq+0x99/0xc0 __do_softirq+0xc7/0x467 irq_exit+0xd1/0xe0 call_function_single_interrupt+0xf/0x20 </IRQ> RIP: 0010:default_idle+0x1e/0x170 [CAUSE] The crafted image has a tricky corruption, the INODE_ITEM has a different type against its parent dir: item 20 key (268 INODE_ITEM 0) itemoff 2808 itemsize 160 generation 13 transid 13 size 1048576 nbytes 1048576 block group 0 mode 121644 links 1 uid 0 gid 0 rdev 0 sequence 9 flags 0x0(none) This mode number 0120000 means it's a symlink. But the dir item think it's still a regular file: item 8 key (264 DIR_INDEX 5) itemoff 3707 itemsize 32 location key (268 INODE_ITEM 0) type FILE transid 13 data_len 0 name_len 2 name: f4 item 40 key (264 DIR_ITEM 51821248) itemoff 1573 itemsize 32 location key (268 INODE_ITEM 0) type FILE transid 13 data_len 0 name_len 2 name: f4 For symlink, we don't set BTRFS_I(inode)->io_tree.ops and leave it empty, as symlink is only designed to have inlined extent, all handled by tree block read. Thus no need to trigger btrfs_submit_bio_hook() for inline file extent. However end_bio_extent_readpage() expects tree->ops populated, as it's reading regular data extent. This causes NULL pointer dereference. [FIX] This patch fixes the problem in two ways: - Verify inode mode against its dir item when looking up inode So in btrfs_lookup_dentry() if we find inode mode mismatch with dir item, we error out so that corrupted inode will not be accessed. - Verify inode mode when getting extent mapping Only regular file should have regular or preallocated extent. If we found regular/preallocated file extent for symlink or the rest, we error out before submitting the read bio. With this fix that crafted image can be rejected gracefully: BTRFS critical (device loop0): inode mode mismatch with dir: inode mode=0121644 btrfs type=7 dir type=1 Reported-by:Yoon Jungyeon <jungyeon@gatech.edu> Link: https://bugzilla.kernel.org/show_bug.cgi?id=202763Reviewed-by:
Qu Wenruo <wqu@suse.com> Signed-off-by:
-
Qu Wenruo authored
There is a report in kernel bugzilla about mismatch file type in dir item and inode item. This inspires us to check inode mode in inode item. This patch will check the following members: - inode key objectid Should be ROOT_DIR_DIR or [256, (u64)-256] or FREE_INO. - inode key offset Should be 0 - inode item generation - inode item transid No newer than sb generation + 1. The +1 is for log tree. - inode item mode No unknown bits. No invalid S_IF* bit. NOTE: S_IFMT check is not enough, need to check every know type. - inode item nlink Dir should have no more link than 1. - inode item flags Reviewed-by:
Johannes Thumshirn <jthumshirn@suse.de> Signed-off-by:
-
Qu Wenruo authored
Btrfs-progs already have a comprehensive type checker, to ensure there is only 0 (SINGLE profile) or 1 (DUP/RAID0/1/5/6/10) bit set for chunk profile bits. Do the same work for kernel. Reported-by:
-
Qu Wenruo authored
[BUG] For fuzzed image whose DEV_ITEM has invalid total_bytes as 0, then kernel will just panic: BUG: unable to handle kernel NULL pointer dereference at 0000000000000098 #PF error: [normal kernel read fault] PGD 800000022b2bd067 P4D 800000022b2bd067 PUD 22b2bc067 PMD 0 Oops: 0000 [#1] SMP PTI CPU: 0 PID: 1106 Comm: mount Not tainted 5.0.0-rc8+ #9 RIP: 0010:btrfs_verify_dev_extents+0x2a5/0x5a0 Call Trace: open_ctree+0x160d/0x2149 btrfs_mount_root+0x5b2/0x680 [CAUSE] If device extent verification finds a deivce with 0 total_bytes, then it assumes it's a seed dummy, then search for seed devices. But in this case, there is no seed device at all, causing NULL pointer. [FIX] Since this is caused by fuzzed image, let's go the tree-check way, just add a new verification for device item. Reported-by:
-
Qu Wenruo authored
Since we have btrfs_check_chunk_valid() in tree-checker, let's do chunk item verification in tree-checker too. Since the tree-checker is run at endio time, if one chunk leaf fails chunk verification, we can still retry the other copy, making btrfs more robust to fuzzed image as we may still get a good chunk item. Also since we have done chunk verification in tree block read time, skip the btrfs_check_chunk_valid() call in read_one_chunk() if we're reading chunk items from leaf. Reviewed-by:
-
Qu Wenruo authored
To follow the standard behavior of tree-checker. Reviewed-by:
-
Qu Wenruo authored
Old error message would be something like: BTRFS error (device dm-3): invalid chunk num_stipres: 0 New error message would be: Btrfs critical (device dm-3): corrupt superblock syschunk array: chunk_start=2097152, invalid chunk num_stripes: 0 Or Btrfs critical (device dm-3): corrupt leaf: root=3 block=8388608 slot=3 chunk_start=2097152, invalid chunk num_stripes: 0 And for certain error message, also output expected value. The error message levels are changed from error to critical. Reviewed-by:
-
Qu Wenruo authored
By function, chunk item verification is more suitable to be done inside tree-checker. So move btrfs_check_chunk_valid() to tree-checker.c and export it. And since it's now moved to tree-checker, also add a better comment for what this function is doing. Reviewed-by:
-
David Sterba authored
The commit fcebe456 ("Btrfs: rework qgroup accounting") reworked qgroups and added some new structures. Another rework of qgroup mechanics e69bcee3 ("btrfs: qgroup: Cleanup the old ref_node-oriented mechanism.") stopped using them and left uncleaned. Reviewed-by:
-
David Sterba authored
We can read fs_info from extent buffer and can drop it from the parameters. Signed-off-by:
-
