- 30 Oct, 2017 40 commits
-
-
Liu Bo authored
By setting compression for a defrag task, the task will start IO at the end of defrag. After the combo of filemap_flush(), we've already made sure that dirty pages have made progress via async compress thread because the second filemap_flush() will wait for page lock, which won't be unlocked until those pages have been marked as writeback and ordered extents have been queued. And this is for per-inode defrag, it's not helpful to wait on a global %async_delalloc_pages and %nr_async_submits from fs_info. Although waiting on %nr_async_submits means that all bios are submitted down to per-device schedule IO lists, it doesn't wait for their completions, thus users still need to do fsync/sync to make sure the data is on disk. While with this change, it makes sure that pages are marked with writeback bits and will be submitted asynchronously shortly, therefore, the behavior of defrag option '-c' remains unchanged. Signed-off-by:
Liu Bo <bo.li.liu@oracle.com> Signed-off-by:
David Sterba <dsterba@suse.com>
-
Liu Bo authored
This was intended to congest higher layers to not send bios, but as 1) the congested bit has been taken by writeback Async bios come from buffered writes and DIO writes. For DIO writes, we want to submit them ASAP, while for buffered writes, writeback uses balance_dirty_pages() to throttle how much dirty pages we can have. 2) and no one is waiting for %nr_async_bios down to zero, Historically, it was introduced along with changes which let checksumming workload spread accross different cpus. And at that time, pdflush was used instead of per-bdi flushing, perhaps pdflush did not have the necessary information for writeback to do throttling. We can safely remove them now. Signed-off-by:
-
Qu Wenruo authored
Output the invalid member name and its bad value, along with its expected value range or alignment. Signed-off-by:
Qu Wenruo <quwenruo.btrfs@gmx.com> Signed-off-by:
-
Qu Wenruo authored
Output the bad value and expected good value (or its alignment). Signed-off-by:
-
Qu Wenruo authored
Enhance the output to print: 1) the eason 2) the ad value, if reason is not sufficient 3) good value (range) Signed-off-by:
-
Qu Wenruo authored
Use inline function to replace macro since we don't need stringification. (Macro still exists until all callers get updated) And add more info about the error, and replace EIO with EUCLEAN. For nr_items error, report if it's too large or too small, and output the valid value range. For node block pointer, added a new alignment checker. For key order, also output the next key to make the problem more obvious. Signed-off-by:
-
Qu Wenruo authored
It's no doubt the comprehensive tree block checker will become larger, so moving them into their own files is quite reasonable. Signed-off-by:
-
Timofey Titovets authored
Remove dead assigment of num_bytes. Also as num_bytes only used in the will_compress block as copy of total_in just replace that with total_in and drop num_bytes entirely. Signed-off-by:
Timofey Titovets <nefelim4ag@gmail.com> Reviewed-by:
Nikolay Borisov <nborisov@suse.com> Reviewed-by:
-
Rakesh Pandit authored
Commit a53f4f8e ("btrfs: Don't call btrfs_start_transaction() on frozen fs to avoid deadlock.") started using internal calls and we replace them with more suitable ones. Signed-off-by:
Rakesh Pandit <rakesh@tuxera.com> Reviewed-by:
-
Hans van Kranenburg authored
Currently struct names for sysfs are generated only based on the attribute names. This means that attribute names cannot be reused in multiple places throughout the complete btrfs sysfs hierarchy. E.g. allocation/data/total_bytes and allocation/data/single/total_bytes result in the same struct name btrfs_attr_total_bytes. A workaround for this case was made in the past by ad hoc creating an extra macro wrapper, BTRFS_RAID_ATTR, that inserts some extra text in the struct name. Instead of polluting sysfs.h with such kind of extra macro definitions, and only doing so when there are collisions, use a prefix which gets inserted in the struct name, so we keep everything nicely grouped together by default. Current collections of attributes are: * (the toplevel, empty prefix) * allocation * space_info * raid * features Signed-off-by:
Hans van Kranenburg <hans.van.kranenburg@mendix.com> Reviewed-by:
-
Thomas Meyer authored
Bool initializations should use true and false. Bool tests don't need comparisons. Signed-off-by:
Thomas Meyer <thomas@m3y3r.de> Reviewed-by:
-
Nikolay Borisov authored
If btrfs_transaction_commit fails it will proceed to call cleanup_transaction, which in turn already does btrfs_abort_transaction. So let's remove the unnecessary code duplication. Also let's be explicit about handling failure of btrfs_uuid_tree_add by calling btrfs_end_transaction. Signed-off-by:
-
Nikolay Borisov authored
btrfs_udpate_root can fail and it aborts the transaction, the correct way to handle an aborted transaction is to explicitly end with btrfs_end_transaction. Even now the code is correct since btrfs_commit_transaction would handle an aborted transaction but this is more of an implementation detail. So let's be explicit in handling failure in btrfs_update_root. Furthermore btrfs_commit_transaction can also fail and by ignoring it's return value we could have left the in-memory copy of the root item in an inconsistent state. So capture the error value which allows us to correctly revert the RO/RW flags in case of commit failure. Signed-off-by:
-
Anand Jain authored
btrfs_init_new_device() calls btrfs_attach_transaction() to commit sys chunks, and it should error out if it fails. Signed-off-by:
Anand Jain <anand.jain@oracle.com> Reviewed-by:
-
Anand Jain authored
Instead of BUG_ON return error to the caller. And handle the fail condition by calling the abort transaction and going through the error path. Signed-off-by:
-
Anand Jain authored
When new device is being added to seed FS, seed FS is marked writable, but when we fail to bring in the new device, we missed to undo the writable part. This patch fixes it. Signed-off-by:
-
Qu Wenruo authored
EXTENT_CSUM checker is a relatively easy one, only needs to check: 1) Objectid Fixed to BTRFS_EXTENT_CSUM_OBJECTID 2) Key offset alignment Must be aligned to sectorsize 3) Item size alignedment Must be aligned to csum size Signed-off-by:
-
Qu Wenruo authored
Add extra checks for item with EXTENT_DATA type. This checks the following thing: 0) Key offset All key offsets must be aligned to sectorsize. Inline extent must have 0 for key offset. 1) Item size Uncompressed inline file extent size must match item size. (Compressed inline file extent has no information about its on-disk size.) Regular/preallocated file extent size must be a fixed value. 2) Every member of regular file extent item Including alignment for bytenr and offset, possible value for compression/encryption/type. 3) Type/compression/encode must be one of the valid values. This should be the most comprehensive and strict check in the context of btrfs_item for EXTENT_DATA. Signed-off-by:
-
Qu Wenruo authored
Function check_leaf() checks if any item pointer points outside of the leaf, but it doesn't check if the pointer overlaps with the item itself. Normally only the last item may be the victim, but adding such check is never a bad idea anyway. Signed-off-by:
-
Qu Wenruo authored
Current check_leaf() function does a good job checking key order and item offset/size. However it only checks from slot 0 to the last but one slot, this is good but makes later expansion hard. So this refactoring iterates from slot 0 to the last slot. For key comparison, it uses a key with all 0 as initial key, so all valid keys should be larger than that. And for item size/offset checks, it compares current item end with previous item offset. For slot 0, use leaf end as a special case. This makes later item/key offset checks and item size checks easier to be implemented. Also, makes check_leaf() to return -EUCLEAN other than -EIO to indicate error. Signed-off-by:
-
Timofey Titovets authored
Was added in: c8b97818 "Btrfs: Add zlib compression support" Survive to near time (from 08.10.2008). Because 'start' checked for zero before branch, so it's safe to remove that subtraction. Signed-off-by:
Satoru Takeuchi <satoru.takeuchi@gmail.com> Reviewed-by:
-
Josef Bacik authored
Nikolay reported that generic/273 was failing currently with ENOSPC. Turns out this is because we get to the point where the outstanding reservations are greater than the pinned space on the fs. This is a mistake, previously we used the current reservation amount in may_commit_transaction, not the entire outstanding reservation amount. Fix this to find the minimum byte size needed to make progress in flushing, and pass that into may_commit_transaction. From there we can make a smarter decision on whether to commit the transaction or not. This fixes the failure in generic/273. From Nikolai, IOW: when we go to the final stage of deciding whether to do trans commit, instead of passing all the reservations from all tickets we just pass the reservation for the current ticket. Otherwise, in case all reservations exceed pinned space, then we don't commit transaction and fail prematurely. Before we passed num_bytes from flush_space, where num_bytes was the sum of all pending reserverations, but now all we do is take the first ticket and commit the trans if we can satisfy that. Fixes: 957780eb ("Btrfs: introduce ticketed enospc infrastructure") Cc: stable@vger.kernel.org # 4.8 Reported-by:
Josef Bacik <jbacik@fb.com> Reviewed-by:
-
Kuanling Huang authored
By analyzing the perf on btrfs send, we found it take large amount of cpu time on page_cache_sync_readahead. This effort can be reduced after switching to asynchronous one. Overall performance gain on HDD and SSD were 9 and 15 percent if simply send a large file. Signed-off-by:
Kuanling Huang <peterh@synology.com> Reviewed-by:
-
Liu Bo authored
The local bio_list may have pending bios when doing cleanup, it can end up with memory leak if they don't get freed. Signed-off-by:
-
Colin Ian King authored
Don't populate the read-only array types on the stack, instead make it static const. Makes the object code smaller by nearly 60 bytes: Before: text data bss dec hex filename 90536 6552 64 97152 17b80 fs/btrfs/ioctl.o After: text data bss dec hex filename 90414 6616 64 97094 17b46 fs/btrfs/ioctl.o Signed-off-by:
Colin Ian King <colin.king@canonical.com> Reviewed-by:
-
Allen Pais authored
Forward the correct return value -ENOMEM from btrfsic_dev_state_alloc() too. Signed-off-by:
Allen Pais <allen.lkml@gmail.com> Reviewed-by:
-
Liu Bo authored
We've seen the following backtrace stack in ftrace or dmesg log, kworker/u16:10-4244 [000] 241942.480955: function: btrfs_put_ordered_extent kworker/u16:10-4244 [000] 241942.480956: kernel_stack: <stack trace> => finish_ordered_fn (ffffffffa0384475) => btrfs_scrubparity_helper (ffffffffa03ca577) <-----"incorrect" => btrfs_freespace_write_helper (ffffffffa03ca98e) <-----"correct" => process_one_work (ffffffff81117b2f) => worker_thread (ffffffff81118c2a) => kthread (ffffffff81121de0) => ret_from_fork (ffffffff81d7087a) btrfs_freespace_write_helper is actually calling normal_worker_helper instead of btrfs_scrubparity_helper, so somehow kernel has parsed the incorrect function address while unwinding the stack, btrfs_scrubparity_helper really shouldn't be shown up. It's caused by compiler doing inline for our helper function, adding a noinline tag can fix that. Signed-off-by:
-
Liu Bo authored
Since both committing transaction and writing log-tree are doing plugging on metadata IO, we can unify to use %sync_writers to benefit both cases, instead of checking bio_flags while writing meta blocks of log-tree. We can remove this bio_flags because in order to write dirty blocks, log tree also uses btrfs_write_marked_extents(), inside which we have enabled %sync_writers, therefore, every write goes in a synchronous way, so does checksuming. Please also note that, bio_flags is applied per-context while %sync_writers is applied per-inode, so this might incur some overhead, ie. 1) while log tree is flushing its dirty blocks via btrfs_write_marked_extents(), in which %sync_writers is increased by one. 2) in the meantime, some writeback operations may happen upon btrfs's metadata inode, so these writes go synchronously, too. However, AFAICS, the overhead is not a big one while the win is that we unify the two places that needs synchronous way and remove a special hack/flag. This removes the bio_flags related stuff for writing log-tree. Signed-off-by:
-
Liu Bo authored
We have started plug in btrfs_write_and_wait_marked_extents() but the generated IOs actually go to device's schedule IO list where the work is doing in another task, thus the started plug doesn't make any sense. And since we wait for IOs immediately after writing meta blocks, it's the same case as writing log tree, doing sync submit can merge more IOs. Signed-off-by:
-
Satoru Takeuchi authored
Signed-off-by:
-
Colin Ian King authored
There are checks on fs_info in __btrfs_panic to avoid dereferencing a null fs_info, however, there is a call to btrfs_crit that may also dereference a null fs_info. Fix this by adding a check to see if fs_info is null and only print the s_id if fs_info is non-null. Detected by CoverityScan CID#401973 ("Dereference after null check") Fixes: efe120a0 ("Btrfs: convert printk to btrfs_ and fix BTRFS prefix") Signed-off-by: -
Christos Gkekas authored
The value of variable 'can_recover' is never used after being set, thus it should be removed, as it was never used since the first commit 68a7342c ("Btrfs: cleanup orphaned root orphan item"). Signed-off-by:
Christos Gkekas <chris.gekas@gmail.com> Reviewed-by:
-
Christophe JAILLET authored
If 'btrfs_alloc_path()' fails, we must free the resources already allocated, as done in the other error handling paths in this function. Signed-off-by:
Christophe JAILLET <christophe.jaillet@wanadoo.fr> Reviewed-by:
-
Nikolay Borisov authored
__link_block_group is called from only 2 places and at each call site the space_info being passed is the same as the space info assigned to the passed cache struct. Let's remove the redundant argument and make the function reference the space_info from the passed block_group_cache. No functional changes Signed-off-by:
-
Nikolay Borisov authored
Currently the code executes add_extent_mapping and if it is successful it links the new mapping, it then proceeds to unlock the extent mapping tree and check for failure and handle them. Instead, rework the code to only perform a single check if add_extent_mapping has failed and handle it, otherwise the code continues in a linear fashion. No functional changes Signed-off-by:
-
Nikolay Borisov authored
Introduced by 5a5f79b5 ("Btrfs: allow unaligned DIO") and never used. The buffered fallback from unaligned DIO works as expected. Signed-off-by:
-
Nikolay Borisov authored
btrfs_changed_cb_t represents the signature of the callback being passed to btrfs_compare_trees. Currently there is only one such callback, namely changed_cb in send.c. This function doesn't really uses the first 2 parameters, i.e. the roots. Since there are not other functions implementing the btrfs_changed_cb_t let's remove the unused parameters from the prototype and implementation. Signed-off-by:
-
Nikolay Borisov authored
iterate_dir_item:found_key - introduced in 31db9f7c ("Btrfs: introduce BTRFS_IOC_SEND for btrfs send/receive"), yet never used. record_ref:num - ditto This is a first pass with the low-hanging fruit. There are still quite a few unsued parameters in some function which have to abide by a callback interface. Signed-off-by:
-
Nikolay Borisov authored
Src was initially part of 31ff1cd2 ("Btrfs: Copy into the log tree in big batches"), however 16e7549f ("Btrfs: incompatible format change to remove hole extents") changed parameters passed to copy_items which made the src variable redundant. Signed-off-by:
-
Liu Bo authored
While we submit direct writes, if the inode is flagged with nodatasum, there's no benefit to submit asynchronously, because a) we don't have to calculate checksum across processors, b) and direct IO has started a plug, but async submit makes us queue IO on each device's scheduled IO list instead of DIO's plug list, so that IOs get much less merges in general. Lets use sync submit for nodatasum inodes. Signed-off-by:
-
