Commit 79952bdc authored by Linus Torvalds's avatar Linus Torvalds

Merge tag 'f2fs-for-6.12-rc1' of git://git.kernel.org/pub/scm/linux/kernel/git/jaegeuk/f2fs

Pull f2fs updates from Jaegeuk Kim:
 "The main changes include converting major IO paths to use folio, and
  adding various knobs to control GC more flexibly for Zoned devices.

  In addition, there are several patches to address corner cases of
  atomic file operations and better support for file pinning on zoned
  device.

  Enhancement:
   - add knobs to tune foreground/background GCs for Zoned devices
   - convert IO paths to use folio
   - reduce expensive checkpoint trigger frequency
   - allow F2FS_IPU_NOCACHE for pinned file
   - forcibly migrate to secure space for zoned device file pinning
   - get rid of buffer_head use
   - add write priority option based on zone UFS
   - get rid of online repair on corrupted directory

  Bug fixes:
   - fix to don't panic system for no free segment fault injection
   - fix to don't set SB_RDONLY in f2fs_handle_critical_error()
   - avoid unused block when dio write in LFS mode
   - compress: don't redirty sparse cluster during {,de}compress
   - check discard support for conventional zones
   - atomic: prevent atomic file from being dirtied before commit
   - atomic: fix to check atomic_file in f2fs ioctl interfaces
   - atomic: fix to forbid dio in atomic_file
   - atomic: fix to truncate pagecache before on-disk metadata truncation
   - atomic: create COW inode from parent dentry
   - atomic: fix to avoid racing w/ GC
   - atomic: require FMODE_WRITE for atomic write ioctls
   - fix to wait page writeback before setting gcing flag
   - fix to avoid racing in between read and OPU dio write, dio completion
   - fix several potential integer overflows in file offsets and dir_block_index
   - fix to avoid use-after-free in f2fs_stop_gc_thread()

  As usual, there are several code clean-ups and refactorings"

* tag 'f2fs-for-6.12-rc1' of git://git.kernel.org/pub/scm/linux/kernel/git/jaegeuk/f2fs: (60 commits)
  f2fs: allow F2FS_IPU_NOCACHE for pinned file
  f2fs: forcibly migrate to secure space for zoned device file pinning
  f2fs: remove unused parameters
  f2fs: fix to don't panic system for no free segment fault injection
  f2fs: fix to don't set SB_RDONLY in f2fs_handle_critical_error()
  f2fs: add valid block ratio not to do excessive GC for one time GC
  f2fs: create gc_no_zoned_gc_percent and gc_boost_zoned_gc_percent
  f2fs: do FG_GC when GC boosting is required for zoned devices
  f2fs: increase BG GC migration window granularity when boosted for zoned devices
  f2fs: add reserved_segments sysfs node
  f2fs: introduce migration_window_granularity
  f2fs: make BG GC more aggressive for zoned devices
  f2fs: avoid unused block when dio write in LFS mode
  f2fs: fix to check atomic_file in f2fs ioctl interfaces
  f2fs: get rid of online repaire on corrupted directory
  f2fs: prevent atomic file from being dirtied before commit
  f2fs: get rid of page->index
  f2fs: convert read_node_page() to use folio
  f2fs: convert __write_node_page() to use folio
  f2fs: convert f2fs_write_data_page() to use folio
  ...
parents fa8380a0 ae87b9c2
...@@ -579,6 +579,12 @@ Description: When ATGC is on, it controls age threshold to bypass GCing young ...@@ -579,6 +579,12 @@ Description: When ATGC is on, it controls age threshold to bypass GCing young
candidates whose age is not beyond the threshold, by default it was candidates whose age is not beyond the threshold, by default it was
initialized as 604800 seconds (equals to 7 days). initialized as 604800 seconds (equals to 7 days).
What: /sys/fs/f2fs/<disk>/atgc_enabled
Date: Feb 2024
Contact: "Jinbao Liu" <liujinbao1@xiaomi.com>
Description: It represents whether ATGC is on or off. The value is 1 which
indicates that ATGC is on, and 0 indicates that it is off.
What: /sys/fs/f2fs/<disk>/gc_reclaimed_segments What: /sys/fs/f2fs/<disk>/gc_reclaimed_segments
Date: July 2021 Date: July 2021
Contact: "Daeho Jeong" <daehojeong@google.com> Contact: "Daeho Jeong" <daehojeong@google.com>
...@@ -763,3 +769,53 @@ Date: November 2023 ...@@ -763,3 +769,53 @@ Date: November 2023
Contact: "Chao Yu" <chao@kernel.org> Contact: "Chao Yu" <chao@kernel.org>
Description: It controls to enable/disable IO aware feature for background discard. Description: It controls to enable/disable IO aware feature for background discard.
By default, the value is 1 which indicates IO aware is on. By default, the value is 1 which indicates IO aware is on.
What: /sys/fs/f2fs/<disk>/blkzone_alloc_policy
Date: July 2024
Contact: "Yuanhong Liao" <liaoyuanhong@vivo.com>
Description: The zone UFS we are currently using consists of two parts:
conventional zones and sequential zones. It can be used to control which part
to prioritize for writes, with a default value of 0.
======================== =========================================
value description
blkzone_alloc_policy = 0 Prioritize writing to sequential zones
blkzone_alloc_policy = 1 Only allow writing to sequential zones
blkzone_alloc_policy = 2 Prioritize writing to conventional zones
======================== =========================================
What: /sys/fs/f2fs/<disk>/migration_window_granularity
Date: September 2024
Contact: "Daeho Jeong" <daehojeong@google.com>
Description: Controls migration window granularity of garbage collection on large
section. it can control the scanning window granularity for GC migration
in a unit of segment, while migration_granularity controls the number
of segments which can be migrated at the same turn.
What: /sys/fs/f2fs/<disk>/reserved_segments
Date: September 2024
Contact: "Daeho Jeong" <daehojeong@google.com>
Description: In order to fine tune GC behavior, we can control the number of
reserved segments.
What: /sys/fs/f2fs/<disk>/gc_no_zoned_gc_percent
Date: September 2024
Contact: "Daeho Jeong" <daehojeong@google.com>
Description: If the percentage of free sections over total sections is above this
number, F2FS do not garbage collection for zoned devices through the
background GC thread. the default number is "60".
What: /sys/fs/f2fs/<disk>/gc_boost_zoned_gc_percent
Date: September 2024
Contact: "Daeho Jeong" <daehojeong@google.com>
Description: If the percentage of free sections over total sections is under this
number, F2FS boosts garbage collection for zoned devices through the
background GC thread. the default number is "25".
What: /sys/fs/f2fs/<disk>/gc_valid_thresh_ratio
Date: September 2024
Contact: "Daeho Jeong" <daehojeong@google.com>
Description: It controls the valid block ratio threshold not to trigger excessive GC
for zoned deivces. The initial value of it is 95(%). F2FS will stop the
background GC thread from intiating GC for sections having valid blocks
exceeding the ratio.
...@@ -99,7 +99,7 @@ static struct page *__get_meta_page(struct f2fs_sb_info *sbi, pgoff_t index, ...@@ -99,7 +99,7 @@ static struct page *__get_meta_page(struct f2fs_sb_info *sbi, pgoff_t index,
} }
if (unlikely(!PageUptodate(page))) { if (unlikely(!PageUptodate(page))) {
f2fs_handle_page_eio(sbi, page->index, META); f2fs_handle_page_eio(sbi, page_folio(page), META);
f2fs_put_page(page, 1); f2fs_put_page(page, 1);
return ERR_PTR(-EIO); return ERR_PTR(-EIO);
} }
...@@ -345,30 +345,31 @@ static int __f2fs_write_meta_page(struct page *page, ...@@ -345,30 +345,31 @@ static int __f2fs_write_meta_page(struct page *page,
enum iostat_type io_type) enum iostat_type io_type)
{ {
struct f2fs_sb_info *sbi = F2FS_P_SB(page); struct f2fs_sb_info *sbi = F2FS_P_SB(page);
struct folio *folio = page_folio(page);
trace_f2fs_writepage(page_folio(page), META); trace_f2fs_writepage(folio, META);
if (unlikely(f2fs_cp_error(sbi))) { if (unlikely(f2fs_cp_error(sbi))) {
if (is_sbi_flag_set(sbi, SBI_IS_CLOSE)) { if (is_sbi_flag_set(sbi, SBI_IS_CLOSE)) {
ClearPageUptodate(page); folio_clear_uptodate(folio);
dec_page_count(sbi, F2FS_DIRTY_META); dec_page_count(sbi, F2FS_DIRTY_META);
unlock_page(page); folio_unlock(folio);
return 0; return 0;
} }
goto redirty_out; goto redirty_out;
} }
if (unlikely(is_sbi_flag_set(sbi, SBI_POR_DOING))) if (unlikely(is_sbi_flag_set(sbi, SBI_POR_DOING)))
goto redirty_out; goto redirty_out;
if (wbc->for_reclaim && page->index < GET_SUM_BLOCK(sbi, 0)) if (wbc->for_reclaim && folio->index < GET_SUM_BLOCK(sbi, 0))
goto redirty_out; goto redirty_out;
f2fs_do_write_meta_page(sbi, page, io_type); f2fs_do_write_meta_page(sbi, folio, io_type);
dec_page_count(sbi, F2FS_DIRTY_META); dec_page_count(sbi, F2FS_DIRTY_META);
if (wbc->for_reclaim) if (wbc->for_reclaim)
f2fs_submit_merged_write_cond(sbi, NULL, page, 0, META); f2fs_submit_merged_write_cond(sbi, NULL, page, 0, META);
unlock_page(page); folio_unlock(folio);
if (unlikely(f2fs_cp_error(sbi))) if (unlikely(f2fs_cp_error(sbi)))
f2fs_submit_merged_write(sbi, META); f2fs_submit_merged_write(sbi, META);
...@@ -1551,7 +1552,7 @@ static int do_checkpoint(struct f2fs_sb_info *sbi, struct cp_control *cpc) ...@@ -1551,7 +1552,7 @@ static int do_checkpoint(struct f2fs_sb_info *sbi, struct cp_control *cpc)
blk = start_blk + BLKS_PER_SEG(sbi) - nm_i->nat_bits_blocks; blk = start_blk + BLKS_PER_SEG(sbi) - nm_i->nat_bits_blocks;
for (i = 0; i < nm_i->nat_bits_blocks; i++) for (i = 0; i < nm_i->nat_bits_blocks; i++)
f2fs_update_meta_page(sbi, nm_i->nat_bits + f2fs_update_meta_page(sbi, nm_i->nat_bits +
(i << F2FS_BLKSIZE_BITS), blk + i); F2FS_BLK_TO_BYTES(i), blk + i);
} }
/* write out checkpoint buffer at block 0 */ /* write out checkpoint buffer at block 0 */
......
...@@ -90,11 +90,13 @@ bool f2fs_is_compressed_page(struct page *page) ...@@ -90,11 +90,13 @@ bool f2fs_is_compressed_page(struct page *page)
static void f2fs_set_compressed_page(struct page *page, static void f2fs_set_compressed_page(struct page *page,
struct inode *inode, pgoff_t index, void *data) struct inode *inode, pgoff_t index, void *data)
{ {
attach_page_private(page, (void *)data); struct folio *folio = page_folio(page);
folio_attach_private(folio, (void *)data);
/* i_crypto_info and iv index */ /* i_crypto_info and iv index */
page->index = index; folio->index = index;
page->mapping = inode->i_mapping; folio->mapping = inode->i_mapping;
} }
static void f2fs_drop_rpages(struct compress_ctx *cc, int len, bool unlock) static void f2fs_drop_rpages(struct compress_ctx *cc, int len, bool unlock)
...@@ -160,17 +162,17 @@ void f2fs_destroy_compress_ctx(struct compress_ctx *cc, bool reuse) ...@@ -160,17 +162,17 @@ void f2fs_destroy_compress_ctx(struct compress_ctx *cc, bool reuse)
cc->cluster_idx = NULL_CLUSTER; cc->cluster_idx = NULL_CLUSTER;
} }
void f2fs_compress_ctx_add_page(struct compress_ctx *cc, struct page *page) void f2fs_compress_ctx_add_page(struct compress_ctx *cc, struct folio *folio)
{ {
unsigned int cluster_ofs; unsigned int cluster_ofs;
if (!f2fs_cluster_can_merge_page(cc, page->index)) if (!f2fs_cluster_can_merge_page(cc, folio->index))
f2fs_bug_on(F2FS_I_SB(cc->inode), 1); f2fs_bug_on(F2FS_I_SB(cc->inode), 1);
cluster_ofs = offset_in_cluster(cc, page->index); cluster_ofs = offset_in_cluster(cc, folio->index);
cc->rpages[cluster_ofs] = page; cc->rpages[cluster_ofs] = folio_page(folio, 0);
cc->nr_rpages++; cc->nr_rpages++;
cc->cluster_idx = cluster_idx(cc, page->index); cc->cluster_idx = cluster_idx(cc, folio->index);
} }
#ifdef CONFIG_F2FS_FS_LZO #ifdef CONFIG_F2FS_FS_LZO
...@@ -879,7 +881,7 @@ static bool cluster_has_invalid_data(struct compress_ctx *cc) ...@@ -879,7 +881,7 @@ static bool cluster_has_invalid_data(struct compress_ctx *cc)
f2fs_bug_on(F2FS_I_SB(cc->inode), !page); f2fs_bug_on(F2FS_I_SB(cc->inode), !page);
/* beyond EOF */ /* beyond EOF */
if (page->index >= nr_pages) if (page_folio(page)->index >= nr_pages)
return true; return true;
} }
return false; return false;
...@@ -945,7 +947,7 @@ static int __f2fs_get_cluster_blocks(struct inode *inode, ...@@ -945,7 +947,7 @@ static int __f2fs_get_cluster_blocks(struct inode *inode,
unsigned int cluster_size = F2FS_I(inode)->i_cluster_size; unsigned int cluster_size = F2FS_I(inode)->i_cluster_size;
int count, i; int count, i;
for (i = 1, count = 1; i < cluster_size; i++) { for (i = 0, count = 0; i < cluster_size; i++) {
block_t blkaddr = data_blkaddr(dn->inode, dn->node_page, block_t blkaddr = data_blkaddr(dn->inode, dn->node_page,
dn->ofs_in_node + i); dn->ofs_in_node + i);
...@@ -956,8 +958,8 @@ static int __f2fs_get_cluster_blocks(struct inode *inode, ...@@ -956,8 +958,8 @@ static int __f2fs_get_cluster_blocks(struct inode *inode,
return count; return count;
} }
static int __f2fs_cluster_blocks(struct inode *inode, static int __f2fs_cluster_blocks(struct inode *inode, unsigned int cluster_idx,
unsigned int cluster_idx, bool compr_blks) enum cluster_check_type type)
{ {
struct dnode_of_data dn; struct dnode_of_data dn;
unsigned int start_idx = cluster_idx << unsigned int start_idx = cluster_idx <<
...@@ -978,10 +980,12 @@ static int __f2fs_cluster_blocks(struct inode *inode, ...@@ -978,10 +980,12 @@ static int __f2fs_cluster_blocks(struct inode *inode,
} }
if (dn.data_blkaddr == COMPRESS_ADDR) { if (dn.data_blkaddr == COMPRESS_ADDR) {
if (compr_blks) if (type == CLUSTER_COMPR_BLKS)
ret = __f2fs_get_cluster_blocks(inode, &dn); ret = 1 + __f2fs_get_cluster_blocks(inode, &dn);
else else if (type == CLUSTER_IS_COMPR)
ret = 1; ret = 1;
} else if (type == CLUSTER_RAW_BLKS) {
ret = __f2fs_get_cluster_blocks(inode, &dn);
} }
fail: fail:
f2fs_put_dnode(&dn); f2fs_put_dnode(&dn);
...@@ -991,7 +995,16 @@ static int __f2fs_cluster_blocks(struct inode *inode, ...@@ -991,7 +995,16 @@ static int __f2fs_cluster_blocks(struct inode *inode,
/* return # of compressed blocks in compressed cluster */ /* return # of compressed blocks in compressed cluster */
static int f2fs_compressed_blocks(struct compress_ctx *cc) static int f2fs_compressed_blocks(struct compress_ctx *cc)
{ {
return __f2fs_cluster_blocks(cc->inode, cc->cluster_idx, true); return __f2fs_cluster_blocks(cc->inode, cc->cluster_idx,
CLUSTER_COMPR_BLKS);
}
/* return # of raw blocks in non-compressed cluster */
static int f2fs_decompressed_blocks(struct inode *inode,
unsigned int cluster_idx)
{
return __f2fs_cluster_blocks(inode, cluster_idx,
CLUSTER_RAW_BLKS);
} }
/* return whether cluster is compressed one or not */ /* return whether cluster is compressed one or not */
...@@ -999,7 +1012,16 @@ int f2fs_is_compressed_cluster(struct inode *inode, pgoff_t index) ...@@ -999,7 +1012,16 @@ int f2fs_is_compressed_cluster(struct inode *inode, pgoff_t index)
{ {
return __f2fs_cluster_blocks(inode, return __f2fs_cluster_blocks(inode,
index >> F2FS_I(inode)->i_log_cluster_size, index >> F2FS_I(inode)->i_log_cluster_size,
false); CLUSTER_IS_COMPR);
}
/* return whether cluster contains non raw blocks or not */
bool f2fs_is_sparse_cluster(struct inode *inode, pgoff_t index)
{
unsigned int cluster_idx = index >> F2FS_I(inode)->i_log_cluster_size;
return f2fs_decompressed_blocks(inode, cluster_idx) !=
F2FS_I(inode)->i_cluster_size;
} }
static bool cluster_may_compress(struct compress_ctx *cc) static bool cluster_may_compress(struct compress_ctx *cc)
...@@ -1093,7 +1115,7 @@ static int prepare_compress_overwrite(struct compress_ctx *cc, ...@@ -1093,7 +1115,7 @@ static int prepare_compress_overwrite(struct compress_ctx *cc,
if (PageUptodate(page)) if (PageUptodate(page))
f2fs_put_page(page, 1); f2fs_put_page(page, 1);
else else
f2fs_compress_ctx_add_page(cc, page); f2fs_compress_ctx_add_page(cc, page_folio(page));
} }
if (!f2fs_cluster_is_empty(cc)) { if (!f2fs_cluster_is_empty(cc)) {
...@@ -1123,7 +1145,7 @@ static int prepare_compress_overwrite(struct compress_ctx *cc, ...@@ -1123,7 +1145,7 @@ static int prepare_compress_overwrite(struct compress_ctx *cc,
} }
f2fs_wait_on_page_writeback(page, DATA, true, true); f2fs_wait_on_page_writeback(page, DATA, true, true);
f2fs_compress_ctx_add_page(cc, page); f2fs_compress_ctx_add_page(cc, page_folio(page));
if (!PageUptodate(page)) { if (!PageUptodate(page)) {
release_and_retry: release_and_retry:
...@@ -1523,7 +1545,8 @@ static int f2fs_write_raw_pages(struct compress_ctx *cc, ...@@ -1523,7 +1545,8 @@ static int f2fs_write_raw_pages(struct compress_ctx *cc,
if (!clear_page_dirty_for_io(cc->rpages[i])) if (!clear_page_dirty_for_io(cc->rpages[i]))
goto continue_unlock; goto continue_unlock;
ret = f2fs_write_single_data_page(cc->rpages[i], &submitted, ret = f2fs_write_single_data_page(page_folio(cc->rpages[i]),
&submitted,
NULL, NULL, wbc, io_type, NULL, NULL, wbc, io_type,
compr_blocks, false); compr_blocks, false);
if (ret) { if (ret) {
......
This diff is collapsed.
...@@ -275,7 +275,7 @@ static void update_mem_info(struct f2fs_sb_info *sbi) ...@@ -275,7 +275,7 @@ static void update_mem_info(struct f2fs_sb_info *sbi)
/* build nm */ /* build nm */
si->base_mem += sizeof(struct f2fs_nm_info); si->base_mem += sizeof(struct f2fs_nm_info);
si->base_mem += __bitmap_size(sbi, NAT_BITMAP); si->base_mem += __bitmap_size(sbi, NAT_BITMAP);
si->base_mem += (NM_I(sbi)->nat_bits_blocks << F2FS_BLKSIZE_BITS); si->base_mem += F2FS_BLK_TO_BYTES(NM_I(sbi)->nat_bits_blocks);
si->base_mem += NM_I(sbi)->nat_blocks * si->base_mem += NM_I(sbi)->nat_blocks *
f2fs_bitmap_size(NAT_ENTRY_PER_BLOCK); f2fs_bitmap_size(NAT_ENTRY_PER_BLOCK);
si->base_mem += NM_I(sbi)->nat_blocks / 8; si->base_mem += NM_I(sbi)->nat_blocks / 8;
......
...@@ -166,7 +166,8 @@ static unsigned long dir_block_index(unsigned int level, ...@@ -166,7 +166,8 @@ static unsigned long dir_block_index(unsigned int level,
unsigned long bidx = 0; unsigned long bidx = 0;
for (i = 0; i < level; i++) for (i = 0; i < level; i++)
bidx += dir_buckets(i, dir_level) * bucket_blocks(i); bidx += mul_u32_u32(dir_buckets(i, dir_level),
bucket_blocks(i));
bidx += idx * bucket_blocks(level); bidx += idx * bucket_blocks(level);
return bidx; return bidx;
} }
...@@ -841,6 +842,7 @@ void f2fs_delete_entry(struct f2fs_dir_entry *dentry, struct page *page, ...@@ -841,6 +842,7 @@ void f2fs_delete_entry(struct f2fs_dir_entry *dentry, struct page *page,
struct f2fs_dentry_block *dentry_blk; struct f2fs_dentry_block *dentry_blk;
unsigned int bit_pos; unsigned int bit_pos;
int slots = GET_DENTRY_SLOTS(le16_to_cpu(dentry->name_len)); int slots = GET_DENTRY_SLOTS(le16_to_cpu(dentry->name_len));
pgoff_t index = page_folio(page)->index;
int i; int i;
f2fs_update_time(F2FS_I_SB(dir), REQ_TIME); f2fs_update_time(F2FS_I_SB(dir), REQ_TIME);
...@@ -866,8 +868,8 @@ void f2fs_delete_entry(struct f2fs_dir_entry *dentry, struct page *page, ...@@ -866,8 +868,8 @@ void f2fs_delete_entry(struct f2fs_dir_entry *dentry, struct page *page,
set_page_dirty(page); set_page_dirty(page);
if (bit_pos == NR_DENTRY_IN_BLOCK && if (bit_pos == NR_DENTRY_IN_BLOCK &&
!f2fs_truncate_hole(dir, page->index, page->index + 1)) { !f2fs_truncate_hole(dir, index, index + 1)) {
f2fs_clear_page_cache_dirty_tag(page); f2fs_clear_page_cache_dirty_tag(page_folio(page));
clear_page_dirty_for_io(page); clear_page_dirty_for_io(page);
ClearPageUptodate(page); ClearPageUptodate(page);
clear_page_private_all(page); clear_page_private_all(page);
......
...@@ -366,7 +366,7 @@ static unsigned int __free_extent_tree(struct f2fs_sb_info *sbi, ...@@ -366,7 +366,7 @@ static unsigned int __free_extent_tree(struct f2fs_sb_info *sbi,
static void __drop_largest_extent(struct extent_tree *et, static void __drop_largest_extent(struct extent_tree *et,
pgoff_t fofs, unsigned int len) pgoff_t fofs, unsigned int len)
{ {
if (fofs < et->largest.fofs + et->largest.len && if (fofs < (pgoff_t)et->largest.fofs + et->largest.len &&
fofs + len > et->largest.fofs) { fofs + len > et->largest.fofs) {
et->largest.len = 0; et->largest.len = 0;
et->largest_updated = true; et->largest_updated = true;
...@@ -456,7 +456,7 @@ static bool __lookup_extent_tree(struct inode *inode, pgoff_t pgofs, ...@@ -456,7 +456,7 @@ static bool __lookup_extent_tree(struct inode *inode, pgoff_t pgofs,
if (type == EX_READ && if (type == EX_READ &&
et->largest.fofs <= pgofs && et->largest.fofs <= pgofs &&
et->largest.fofs + et->largest.len > pgofs) { (pgoff_t)et->largest.fofs + et->largest.len > pgofs) {
*ei = et->largest; *ei = et->largest;
ret = true; ret = true;
stat_inc_largest_node_hit(sbi); stat_inc_largest_node_hit(sbi);
......
This diff is collapsed.
This diff is collapsed.
...@@ -81,6 +81,8 @@ static int gc_thread_func(void *data) ...@@ -81,6 +81,8 @@ static int gc_thread_func(void *data)
continue; continue;
} }
gc_control.one_time = false;
/* /*
* [GC triggering condition] * [GC triggering condition]
* 0. GC is not conducted currently. * 0. GC is not conducted currently.
...@@ -116,15 +118,30 @@ static int gc_thread_func(void *data) ...@@ -116,15 +118,30 @@ static int gc_thread_func(void *data)
goto next; goto next;
} }
if (has_enough_invalid_blocks(sbi)) if (f2fs_sb_has_blkzoned(sbi)) {
if (has_enough_free_blocks(sbi,
gc_th->no_zoned_gc_percent)) {
wait_ms = gc_th->no_gc_sleep_time;
f2fs_up_write(&sbi->gc_lock);
goto next;
}
if (wait_ms == gc_th->no_gc_sleep_time)
wait_ms = gc_th->max_sleep_time;
}
if (need_to_boost_gc(sbi)) {
decrease_sleep_time(gc_th, &wait_ms); decrease_sleep_time(gc_th, &wait_ms);
else if (f2fs_sb_has_blkzoned(sbi))
gc_control.one_time = true;
} else {
increase_sleep_time(gc_th, &wait_ms); increase_sleep_time(gc_th, &wait_ms);
}
do_gc: do_gc:
stat_inc_gc_call_count(sbi, foreground ? stat_inc_gc_call_count(sbi, foreground ?
FOREGROUND : BACKGROUND); FOREGROUND : BACKGROUND);
sync_mode = F2FS_OPTION(sbi).bggc_mode == BGGC_MODE_SYNC; sync_mode = (F2FS_OPTION(sbi).bggc_mode == BGGC_MODE_SYNC) ||
gc_control.one_time;
/* foreground GC was been triggered via f2fs_balance_fs() */ /* foreground GC was been triggered via f2fs_balance_fs() */
if (foreground) if (foreground)
...@@ -179,9 +196,21 @@ int f2fs_start_gc_thread(struct f2fs_sb_info *sbi) ...@@ -179,9 +196,21 @@ int f2fs_start_gc_thread(struct f2fs_sb_info *sbi)
return -ENOMEM; return -ENOMEM;
gc_th->urgent_sleep_time = DEF_GC_THREAD_URGENT_SLEEP_TIME; gc_th->urgent_sleep_time = DEF_GC_THREAD_URGENT_SLEEP_TIME;
gc_th->valid_thresh_ratio = DEF_GC_THREAD_VALID_THRESH_RATIO;
if (f2fs_sb_has_blkzoned(sbi)) {
gc_th->min_sleep_time = DEF_GC_THREAD_MIN_SLEEP_TIME_ZONED;
gc_th->max_sleep_time = DEF_GC_THREAD_MAX_SLEEP_TIME_ZONED;
gc_th->no_gc_sleep_time = DEF_GC_THREAD_NOGC_SLEEP_TIME_ZONED;
gc_th->no_zoned_gc_percent = LIMIT_NO_ZONED_GC;
gc_th->boost_zoned_gc_percent = LIMIT_BOOST_ZONED_GC;
} else {
gc_th->min_sleep_time = DEF_GC_THREAD_MIN_SLEEP_TIME; gc_th->min_sleep_time = DEF_GC_THREAD_MIN_SLEEP_TIME;
gc_th->max_sleep_time = DEF_GC_THREAD_MAX_SLEEP_TIME; gc_th->max_sleep_time = DEF_GC_THREAD_MAX_SLEEP_TIME;
gc_th->no_gc_sleep_time = DEF_GC_THREAD_NOGC_SLEEP_TIME; gc_th->no_gc_sleep_time = DEF_GC_THREAD_NOGC_SLEEP_TIME;
gc_th->no_zoned_gc_percent = 0;
gc_th->boost_zoned_gc_percent = 0;
}
gc_th->gc_wake = false; gc_th->gc_wake = false;
...@@ -339,7 +368,7 @@ static unsigned int get_cb_cost(struct f2fs_sb_info *sbi, unsigned int segno) ...@@ -339,7 +368,7 @@ static unsigned int get_cb_cost(struct f2fs_sb_info *sbi, unsigned int segno)
unsigned char age = 0; unsigned char age = 0;
unsigned char u; unsigned char u;
unsigned int i; unsigned int i;
unsigned int usable_segs_per_sec = f2fs_usable_segs_in_sec(sbi, segno); unsigned int usable_segs_per_sec = f2fs_usable_segs_in_sec(sbi);
for (i = 0; i < usable_segs_per_sec; i++) for (i = 0; i < usable_segs_per_sec; i++)
mtime += get_seg_entry(sbi, start + i)->mtime; mtime += get_seg_entry(sbi, start + i)->mtime;
...@@ -368,6 +397,11 @@ static inline unsigned int get_gc_cost(struct f2fs_sb_info *sbi, ...@@ -368,6 +397,11 @@ static inline unsigned int get_gc_cost(struct f2fs_sb_info *sbi,
if (p->alloc_mode == SSR) if (p->alloc_mode == SSR)
return get_seg_entry(sbi, segno)->ckpt_valid_blocks; return get_seg_entry(sbi, segno)->ckpt_valid_blocks;
if (p->one_time_gc && (get_valid_blocks(sbi, segno, true) >=
CAP_BLKS_PER_SEC(sbi) * sbi->gc_thread->valid_thresh_ratio /
100))
return UINT_MAX;
/* alloc_mode == LFS */ /* alloc_mode == LFS */
if (p->gc_mode == GC_GREEDY) if (p->gc_mode == GC_GREEDY)
return get_valid_blocks(sbi, segno, true); return get_valid_blocks(sbi, segno, true);
...@@ -742,7 +776,7 @@ static int f2fs_gc_pinned_control(struct inode *inode, int gc_type, ...@@ -742,7 +776,7 @@ static int f2fs_gc_pinned_control(struct inode *inode, int gc_type,
*/ */
int f2fs_get_victim(struct f2fs_sb_info *sbi, unsigned int *result, int f2fs_get_victim(struct f2fs_sb_info *sbi, unsigned int *result,
int gc_type, int type, char alloc_mode, int gc_type, int type, char alloc_mode,
unsigned long long age) unsigned long long age, bool one_time)
{ {
struct dirty_seglist_info *dirty_i = DIRTY_I(sbi); struct dirty_seglist_info *dirty_i = DIRTY_I(sbi);
struct sit_info *sm = SIT_I(sbi); struct sit_info *sm = SIT_I(sbi);
...@@ -759,6 +793,7 @@ int f2fs_get_victim(struct f2fs_sb_info *sbi, unsigned int *result, ...@@ -759,6 +793,7 @@ int f2fs_get_victim(struct f2fs_sb_info *sbi, unsigned int *result,
p.alloc_mode = alloc_mode; p.alloc_mode = alloc_mode;
p.age = age; p.age = age;
p.age_threshold = sbi->am.age_threshold; p.age_threshold = sbi->am.age_threshold;
p.one_time_gc = one_time;
retry: retry:
select_policy(sbi, gc_type, type, &p); select_policy(sbi, gc_type, type, &p);
...@@ -1670,13 +1705,14 @@ static int gc_data_segment(struct f2fs_sb_info *sbi, struct f2fs_summary *sum, ...@@ -1670,13 +1705,14 @@ static int gc_data_segment(struct f2fs_sb_info *sbi, struct f2fs_summary *sum,
} }
static int __get_victim(struct f2fs_sb_info *sbi, unsigned int *victim, static int __get_victim(struct f2fs_sb_info *sbi, unsigned int *victim,
int gc_type) int gc_type, bool one_time)
{ {
struct sit_info *sit_i = SIT_I(sbi); struct sit_info *sit_i = SIT_I(sbi);
int ret; int ret;
down_write(&sit_i->sentry_lock); down_write(&sit_i->sentry_lock);
ret = f2fs_get_victim(sbi, victim, gc_type, NO_CHECK_TYPE, LFS, 0); ret = f2fs_get_victim(sbi, victim, gc_type, NO_CHECK_TYPE,
LFS, 0, one_time);
up_write(&sit_i->sentry_lock); up_write(&sit_i->sentry_lock);
return ret; return ret;
} }
...@@ -1684,30 +1720,49 @@ static int __get_victim(struct f2fs_sb_info *sbi, unsigned int *victim, ...@@ -1684,30 +1720,49 @@ static int __get_victim(struct f2fs_sb_info *sbi, unsigned int *victim,
static int do_garbage_collect(struct f2fs_sb_info *sbi, static int do_garbage_collect(struct f2fs_sb_info *sbi,
unsigned int start_segno, unsigned int start_segno,
struct gc_inode_list *gc_list, int gc_type, struct gc_inode_list *gc_list, int gc_type,
bool force_migrate) bool force_migrate, bool one_time)
{ {
struct page *sum_page; struct page *sum_page;
struct f2fs_summary_block *sum; struct f2fs_summary_block *sum;
struct blk_plug plug; struct blk_plug plug;
unsigned int segno = start_segno; unsigned int segno = start_segno;
unsigned int end_segno = start_segno + SEGS_PER_SEC(sbi); unsigned int end_segno = start_segno + SEGS_PER_SEC(sbi);
unsigned int sec_end_segno;
int seg_freed = 0, migrated = 0; int seg_freed = 0, migrated = 0;
unsigned char type = IS_DATASEG(get_seg_entry(sbi, segno)->type) ? unsigned char type = IS_DATASEG(get_seg_entry(sbi, segno)->type) ?
SUM_TYPE_DATA : SUM_TYPE_NODE; SUM_TYPE_DATA : SUM_TYPE_NODE;
unsigned char data_type = (type == SUM_TYPE_DATA) ? DATA : NODE; unsigned char data_type = (type == SUM_TYPE_DATA) ? DATA : NODE;
int submitted = 0; int submitted = 0;
if (__is_large_section(sbi)) if (__is_large_section(sbi)) {
end_segno = rounddown(end_segno, SEGS_PER_SEC(sbi)); sec_end_segno = rounddown(end_segno, SEGS_PER_SEC(sbi));
/* /*
* zone-capacity can be less than zone-size in zoned devices, * zone-capacity can be less than zone-size in zoned devices,
* resulting in less than expected usable segments in the zone, * resulting in less than expected usable segments in the zone,
* calculate the end segno in the zone which can be garbage collected * calculate the end segno in the zone which can be garbage
* collected
*/ */
if (f2fs_sb_has_blkzoned(sbi)) if (f2fs_sb_has_blkzoned(sbi))
end_segno -= SEGS_PER_SEC(sbi) - sec_end_segno -= SEGS_PER_SEC(sbi) -
f2fs_usable_segs_in_sec(sbi, segno); f2fs_usable_segs_in_sec(sbi);
if (gc_type == BG_GC || one_time) {
unsigned int window_granularity =
sbi->migration_window_granularity;
if (f2fs_sb_has_blkzoned(sbi) &&
!has_enough_free_blocks(sbi,
sbi->gc_thread->boost_zoned_gc_percent))
window_granularity *=
BOOST_GC_MULTIPLE;
end_segno = start_segno + window_granularity;
}
if (end_segno > sec_end_segno)
end_segno = sec_end_segno;
}
sanity_check_seg_type(sbi, get_seg_entry(sbi, segno)->type); sanity_check_seg_type(sbi, get_seg_entry(sbi, segno)->type);
...@@ -1786,7 +1841,8 @@ static int do_garbage_collect(struct f2fs_sb_info *sbi, ...@@ -1786,7 +1841,8 @@ static int do_garbage_collect(struct f2fs_sb_info *sbi,
if (__is_large_section(sbi)) if (__is_large_section(sbi))
sbi->next_victim_seg[gc_type] = sbi->next_victim_seg[gc_type] =
(segno + 1 < end_segno) ? segno + 1 : NULL_SEGNO; (segno + 1 < sec_end_segno) ?
segno + 1 : NULL_SEGNO;
skip: skip:
f2fs_put_page(sum_page, 0); f2fs_put_page(sum_page, 0);
} }
...@@ -1863,7 +1919,7 @@ int f2fs_gc(struct f2fs_sb_info *sbi, struct f2fs_gc_control *gc_control) ...@@ -1863,7 +1919,7 @@ int f2fs_gc(struct f2fs_sb_info *sbi, struct f2fs_gc_control *gc_control)
goto stop; goto stop;
} }
retry: retry:
ret = __get_victim(sbi, &segno, gc_type); ret = __get_victim(sbi, &segno, gc_type, gc_control->one_time);
if (ret) { if (ret) {
/* allow to search victim from sections has pinned data */ /* allow to search victim from sections has pinned data */
if (ret == -ENODATA && gc_type == FG_GC && if (ret == -ENODATA && gc_type == FG_GC &&
...@@ -1875,17 +1931,21 @@ int f2fs_gc(struct f2fs_sb_info *sbi, struct f2fs_gc_control *gc_control) ...@@ -1875,17 +1931,21 @@ int f2fs_gc(struct f2fs_sb_info *sbi, struct f2fs_gc_control *gc_control)
} }
seg_freed = do_garbage_collect(sbi, segno, &gc_list, gc_type, seg_freed = do_garbage_collect(sbi, segno, &gc_list, gc_type,
gc_control->should_migrate_blocks); gc_control->should_migrate_blocks,
gc_control->one_time);
if (seg_freed < 0) if (seg_freed < 0)
goto stop; goto stop;
total_freed += seg_freed; total_freed += seg_freed;
if (seg_freed == f2fs_usable_segs_in_sec(sbi, segno)) { if (seg_freed == f2fs_usable_segs_in_sec(sbi)) {
sec_freed++; sec_freed++;
total_sec_freed++; total_sec_freed++;
} }
if (gc_control->one_time)
goto stop;
if (gc_type == FG_GC) { if (gc_type == FG_GC) {
sbi->cur_victim_sec = NULL_SEGNO; sbi->cur_victim_sec = NULL_SEGNO;
...@@ -2010,8 +2070,7 @@ int f2fs_gc_range(struct f2fs_sb_info *sbi, ...@@ -2010,8 +2070,7 @@ int f2fs_gc_range(struct f2fs_sb_info *sbi,
.iroot = RADIX_TREE_INIT(gc_list.iroot, GFP_NOFS), .iroot = RADIX_TREE_INIT(gc_list.iroot, GFP_NOFS),
}; };
do_garbage_collect(sbi, segno, &gc_list, FG_GC, do_garbage_collect(sbi, segno, &gc_list, FG_GC, true, false);
dry_run_sections == 0);
put_gc_inode(&gc_list); put_gc_inode(&gc_list);
if (!dry_run && get_valid_blocks(sbi, segno, true)) if (!dry_run && get_valid_blocks(sbi, segno, true))
......
...@@ -15,16 +15,27 @@ ...@@ -15,16 +15,27 @@
#define DEF_GC_THREAD_MAX_SLEEP_TIME 60000 #define DEF_GC_THREAD_MAX_SLEEP_TIME 60000
#define DEF_GC_THREAD_NOGC_SLEEP_TIME 300000 /* wait 5 min */ #define DEF_GC_THREAD_NOGC_SLEEP_TIME 300000 /* wait 5 min */
/* GC sleep parameters for zoned deivces */
#define DEF_GC_THREAD_MIN_SLEEP_TIME_ZONED 10
#define DEF_GC_THREAD_MAX_SLEEP_TIME_ZONED 20
#define DEF_GC_THREAD_NOGC_SLEEP_TIME_ZONED 60000
/* choose candidates from sections which has age of more than 7 days */ /* choose candidates from sections which has age of more than 7 days */
#define DEF_GC_THREAD_AGE_THRESHOLD (60 * 60 * 24 * 7) #define DEF_GC_THREAD_AGE_THRESHOLD (60 * 60 * 24 * 7)
#define DEF_GC_THREAD_CANDIDATE_RATIO 20 /* select 20% oldest sections as candidates */ #define DEF_GC_THREAD_CANDIDATE_RATIO 20 /* select 20% oldest sections as candidates */
#define DEF_GC_THREAD_MAX_CANDIDATE_COUNT 10 /* select at most 10 sections as candidates */ #define DEF_GC_THREAD_MAX_CANDIDATE_COUNT 10 /* select at most 10 sections as candidates */
#define DEF_GC_THREAD_AGE_WEIGHT 60 /* age weight */ #define DEF_GC_THREAD_AGE_WEIGHT 60 /* age weight */
#define DEF_GC_THREAD_VALID_THRESH_RATIO 95 /* do not GC over 95% valid block ratio for one time GC */
#define DEFAULT_ACCURACY_CLASS 10000 /* accuracy class */ #define DEFAULT_ACCURACY_CLASS 10000 /* accuracy class */
#define LIMIT_INVALID_BLOCK 40 /* percentage over total user space */ #define LIMIT_INVALID_BLOCK 40 /* percentage over total user space */
#define LIMIT_FREE_BLOCK 40 /* percentage over invalid + free space */ #define LIMIT_FREE_BLOCK 40 /* percentage over invalid + free space */
#define LIMIT_NO_ZONED_GC 60 /* percentage over total user space of no gc for zoned devices */
#define LIMIT_BOOST_ZONED_GC 25 /* percentage over total user space of boosted gc for zoned devices */
#define DEF_MIGRATION_WINDOW_GRANULARITY_ZONED 3
#define BOOST_GC_MULTIPLE 5
#define DEF_GC_FAILED_PINNED_FILES 2048 #define DEF_GC_FAILED_PINNED_FILES 2048
#define MAX_GC_FAILED_PINNED_FILES USHRT_MAX #define MAX_GC_FAILED_PINNED_FILES USHRT_MAX
...@@ -51,6 +62,11 @@ struct f2fs_gc_kthread { ...@@ -51,6 +62,11 @@ struct f2fs_gc_kthread {
* caller of f2fs_balance_fs() * caller of f2fs_balance_fs()
* will wait on this wait queue. * will wait on this wait queue.
*/ */
/* for gc control for zoned devices */
unsigned int no_zoned_gc_percent;
unsigned int boost_zoned_gc_percent;
unsigned int valid_thresh_ratio;
}; };
struct gc_inode_list { struct gc_inode_list {
...@@ -152,6 +168,12 @@ static inline void decrease_sleep_time(struct f2fs_gc_kthread *gc_th, ...@@ -152,6 +168,12 @@ static inline void decrease_sleep_time(struct f2fs_gc_kthread *gc_th,
*wait -= min_time; *wait -= min_time;
} }
static inline bool has_enough_free_blocks(struct f2fs_sb_info *sbi,
unsigned int limit_perc)
{
return free_sections(sbi) > ((sbi->total_sections * limit_perc) / 100);
}
static inline bool has_enough_invalid_blocks(struct f2fs_sb_info *sbi) static inline bool has_enough_invalid_blocks(struct f2fs_sb_info *sbi)
{ {
block_t user_block_count = sbi->user_block_count; block_t user_block_count = sbi->user_block_count;
...@@ -167,3 +189,10 @@ static inline bool has_enough_invalid_blocks(struct f2fs_sb_info *sbi) ...@@ -167,3 +189,10 @@ static inline bool has_enough_invalid_blocks(struct f2fs_sb_info *sbi)
free_user_blocks(sbi) < free_user_blocks(sbi) <
limit_free_user_blocks(invalid_user_blocks)); limit_free_user_blocks(invalid_user_blocks));
} }
static inline bool need_to_boost_gc(struct f2fs_sb_info *sbi)
{
if (f2fs_sb_has_blkzoned(sbi))
return !has_enough_free_blocks(sbi, LIMIT_BOOST_ZONED_GC);
return has_enough_invalid_blocks(sbi);
}
...@@ -260,35 +260,34 @@ int f2fs_convert_inline_inode(struct inode *inode) ...@@ -260,35 +260,34 @@ int f2fs_convert_inline_inode(struct inode *inode)
return err; return err;
} }
int f2fs_write_inline_data(struct inode *inode, struct page *page) int f2fs_write_inline_data(struct inode *inode, struct folio *folio)
{ {
struct dnode_of_data dn; struct f2fs_sb_info *sbi = F2FS_I_SB(inode);
int err; struct page *ipage;
set_new_dnode(&dn, inode, NULL, NULL, 0); ipage = f2fs_get_node_page(sbi, inode->i_ino);
err = f2fs_get_dnode_of_data(&dn, 0, LOOKUP_NODE); if (IS_ERR(ipage))
if (err) return PTR_ERR(ipage);
return err;
if (!f2fs_has_inline_data(inode)) { if (!f2fs_has_inline_data(inode)) {
f2fs_put_dnode(&dn); f2fs_put_page(ipage, 1);
return -EAGAIN; return -EAGAIN;
} }
f2fs_bug_on(F2FS_I_SB(inode), page->index); f2fs_bug_on(F2FS_I_SB(inode), folio->index);
f2fs_wait_on_page_writeback(dn.inode_page, NODE, true, true); f2fs_wait_on_page_writeback(ipage, NODE, true, true);
memcpy_from_page(inline_data_addr(inode, dn.inode_page), memcpy_from_folio(inline_data_addr(inode, ipage),
page, 0, MAX_INLINE_DATA(inode)); folio, 0, MAX_INLINE_DATA(inode));
set_page_dirty(dn.inode_page); set_page_dirty(ipage);
f2fs_clear_page_cache_dirty_tag(page); f2fs_clear_page_cache_dirty_tag(folio);
set_inode_flag(inode, FI_APPEND_WRITE); set_inode_flag(inode, FI_APPEND_WRITE);
set_inode_flag(inode, FI_DATA_EXIST); set_inode_flag(inode, FI_DATA_EXIST);
clear_page_private_inline(dn.inode_page); clear_page_private_inline(ipage);
f2fs_put_dnode(&dn); f2fs_put_page(ipage, 1);
return 0; return 0;
} }
......
...@@ -7,7 +7,6 @@ ...@@ -7,7 +7,6 @@
*/ */
#include <linux/fs.h> #include <linux/fs.h>
#include <linux/f2fs_fs.h> #include <linux/f2fs_fs.h>
#include <linux/buffer_head.h>
#include <linux/writeback.h> #include <linux/writeback.h>
#include <linux/sched/mm.h> #include <linux/sched/mm.h>
#include <linux/lz4.h> #include <linux/lz4.h>
...@@ -35,6 +34,11 @@ void f2fs_mark_inode_dirty_sync(struct inode *inode, bool sync) ...@@ -35,6 +34,11 @@ void f2fs_mark_inode_dirty_sync(struct inode *inode, bool sync)
if (f2fs_inode_dirtied(inode, sync)) if (f2fs_inode_dirtied(inode, sync))
return; return;
if (f2fs_is_atomic_file(inode)) {
set_inode_flag(inode, FI_ATOMIC_DIRTIED);
return;
}
mark_inode_dirty_sync(inode); mark_inode_dirty_sync(inode);
} }
...@@ -175,7 +179,8 @@ bool f2fs_inode_chksum_verify(struct f2fs_sb_info *sbi, struct page *page) ...@@ -175,7 +179,8 @@ bool f2fs_inode_chksum_verify(struct f2fs_sb_info *sbi, struct page *page)
if (provided != calculated) if (provided != calculated)
f2fs_warn(sbi, "checksum invalid, nid = %lu, ino_of_node = %x, %x vs. %x", f2fs_warn(sbi, "checksum invalid, nid = %lu, ino_of_node = %x, %x vs. %x",
page->index, ino_of_node(page), provided, calculated); page_folio(page)->index, ino_of_node(page),
provided, calculated);
return provided == calculated; return provided == calculated;
} }
......
...@@ -457,62 +457,6 @@ struct dentry *f2fs_get_parent(struct dentry *child) ...@@ -457,62 +457,6 @@ struct dentry *f2fs_get_parent(struct dentry *child)
return d_obtain_alias(f2fs_iget(child->d_sb, ino)); return d_obtain_alias(f2fs_iget(child->d_sb, ino));
} }
static int __recover_dot_dentries(struct inode *dir, nid_t pino)
{
struct f2fs_sb_info *sbi = F2FS_I_SB(dir);
struct qstr dot = QSTR_INIT(".", 1);
struct f2fs_dir_entry *de;
struct page *page;
int err = 0;
if (f2fs_readonly(sbi->sb)) {
f2fs_info(sbi, "skip recovering inline_dots inode (ino:%lu, pino:%u) in readonly mountpoint",
dir->i_ino, pino);
return 0;
}
if (!S_ISDIR(dir->i_mode)) {
f2fs_err(sbi, "inconsistent inode status, skip recovering inline_dots inode (ino:%lu, i_mode:%u, pino:%u)",
dir->i_ino, dir->i_mode, pino);
set_sbi_flag(sbi, SBI_NEED_FSCK);
return -ENOTDIR;
}
err = f2fs_dquot_initialize(dir);
if (err)
return err;
f2fs_balance_fs(sbi, true);
f2fs_lock_op(sbi);
de = f2fs_find_entry(dir, &dot, &page);
if (de) {
f2fs_put_page(page, 0);
} else if (IS_ERR(page)) {
err = PTR_ERR(page);
goto out;
} else {
err = f2fs_do_add_link(dir, &dot, NULL, dir->i_ino, S_IFDIR);
if (err)
goto out;
}
de = f2fs_find_entry(dir, &dotdot_name, &page);
if (de)
f2fs_put_page(page, 0);
else if (IS_ERR(page))
err = PTR_ERR(page);
else
err = f2fs_do_add_link(dir, &dotdot_name, NULL, pino, S_IFDIR);
out:
if (!err)
clear_inode_flag(dir, FI_INLINE_DOTS);
f2fs_unlock_op(sbi);
return err;
}
static struct dentry *f2fs_lookup(struct inode *dir, struct dentry *dentry, static struct dentry *f2fs_lookup(struct inode *dir, struct dentry *dentry,
unsigned int flags) unsigned int flags)
{ {
...@@ -522,7 +466,6 @@ static struct dentry *f2fs_lookup(struct inode *dir, struct dentry *dentry, ...@@ -522,7 +466,6 @@ static struct dentry *f2fs_lookup(struct inode *dir, struct dentry *dentry,
struct dentry *new; struct dentry *new;
nid_t ino = -1; nid_t ino = -1;
int err = 0; int err = 0;
unsigned int root_ino = F2FS_ROOT_INO(F2FS_I_SB(dir));
struct f2fs_filename fname; struct f2fs_filename fname;
trace_f2fs_lookup_start(dir, dentry, flags); trace_f2fs_lookup_start(dir, dentry, flags);
...@@ -558,17 +501,6 @@ static struct dentry *f2fs_lookup(struct inode *dir, struct dentry *dentry, ...@@ -558,17 +501,6 @@ static struct dentry *f2fs_lookup(struct inode *dir, struct dentry *dentry,
goto out; goto out;
} }
if ((dir->i_ino == root_ino) && f2fs_has_inline_dots(dir)) {
err = __recover_dot_dentries(dir, root_ino);
if (err)
goto out_iput;
}
if (f2fs_has_inline_dots(inode)) {
err = __recover_dot_dentries(inode, dir->i_ino);
if (err)
goto out_iput;
}
if (IS_ENCRYPTED(dir) && if (IS_ENCRYPTED(dir) &&
(S_ISDIR(inode->i_mode) || S_ISLNK(inode->i_mode)) && (S_ISDIR(inode->i_mode) || S_ISLNK(inode->i_mode)) &&
!fscrypt_has_permitted_context(dir, inode)) { !fscrypt_has_permitted_context(dir, inode)) {
......
...@@ -20,7 +20,7 @@ ...@@ -20,7 +20,7 @@
#include "iostat.h" #include "iostat.h"
#include <trace/events/f2fs.h> #include <trace/events/f2fs.h>
#define on_f2fs_build_free_nids(nmi) mutex_is_locked(&(nm_i)->build_lock) #define on_f2fs_build_free_nids(nm_i) mutex_is_locked(&(nm_i)->build_lock)
static struct kmem_cache *nat_entry_slab; static struct kmem_cache *nat_entry_slab;
static struct kmem_cache *free_nid_slab; static struct kmem_cache *free_nid_slab;
...@@ -123,7 +123,7 @@ bool f2fs_available_free_memory(struct f2fs_sb_info *sbi, int type) ...@@ -123,7 +123,7 @@ bool f2fs_available_free_memory(struct f2fs_sb_info *sbi, int type)
static void clear_node_page_dirty(struct page *page) static void clear_node_page_dirty(struct page *page)
{ {
if (PageDirty(page)) { if (PageDirty(page)) {
f2fs_clear_page_cache_dirty_tag(page); f2fs_clear_page_cache_dirty_tag(page_folio(page));
clear_page_dirty_for_io(page); clear_page_dirty_for_io(page);
dec_page_count(F2FS_P_SB(page), F2FS_DIRTY_NODES); dec_page_count(F2FS_P_SB(page), F2FS_DIRTY_NODES);
} }
...@@ -919,7 +919,7 @@ static int truncate_node(struct dnode_of_data *dn) ...@@ -919,7 +919,7 @@ static int truncate_node(struct dnode_of_data *dn)
clear_node_page_dirty(dn->node_page); clear_node_page_dirty(dn->node_page);
set_sbi_flag(sbi, SBI_IS_DIRTY); set_sbi_flag(sbi, SBI_IS_DIRTY);
index = dn->node_page->index; index = page_folio(dn->node_page)->index;
f2fs_put_page(dn->node_page, 1); f2fs_put_page(dn->node_page, 1);
invalidate_mapping_pages(NODE_MAPPING(sbi), invalidate_mapping_pages(NODE_MAPPING(sbi),
...@@ -1369,6 +1369,7 @@ struct page *f2fs_new_node_page(struct dnode_of_data *dn, unsigned int ofs) ...@@ -1369,6 +1369,7 @@ struct page *f2fs_new_node_page(struct dnode_of_data *dn, unsigned int ofs)
*/ */
static int read_node_page(struct page *page, blk_opf_t op_flags) static int read_node_page(struct page *page, blk_opf_t op_flags)
{ {
struct folio *folio = page_folio(page);
struct f2fs_sb_info *sbi = F2FS_P_SB(page); struct f2fs_sb_info *sbi = F2FS_P_SB(page);
struct node_info ni; struct node_info ni;
struct f2fs_io_info fio = { struct f2fs_io_info fio = {
...@@ -1381,21 +1382,21 @@ static int read_node_page(struct page *page, blk_opf_t op_flags) ...@@ -1381,21 +1382,21 @@ static int read_node_page(struct page *page, blk_opf_t op_flags)
}; };
int err; int err;
if (PageUptodate(page)) { if (folio_test_uptodate(folio)) {
if (!f2fs_inode_chksum_verify(sbi, page)) { if (!f2fs_inode_chksum_verify(sbi, page)) {
ClearPageUptodate(page); folio_clear_uptodate(folio);
return -EFSBADCRC; return -EFSBADCRC;
} }
return LOCKED_PAGE; return LOCKED_PAGE;
} }
err = f2fs_get_node_info(sbi, page->index, &ni, false); err = f2fs_get_node_info(sbi, folio->index, &ni, false);
if (err) if (err)
return err; return err;
/* NEW_ADDR can be seen, after cp_error drops some dirty node pages */ /* NEW_ADDR can be seen, after cp_error drops some dirty node pages */
if (unlikely(ni.blk_addr == NULL_ADDR || ni.blk_addr == NEW_ADDR)) { if (unlikely(ni.blk_addr == NULL_ADDR || ni.blk_addr == NEW_ADDR)) {
ClearPageUptodate(page); folio_clear_uptodate(folio);
return -ENOENT; return -ENOENT;
} }
...@@ -1492,7 +1493,7 @@ static struct page *__get_node_page(struct f2fs_sb_info *sbi, pgoff_t nid, ...@@ -1492,7 +1493,7 @@ static struct page *__get_node_page(struct f2fs_sb_info *sbi, pgoff_t nid,
out_put_err: out_put_err:
/* ENOENT comes from read_node_page which is not an error. */ /* ENOENT comes from read_node_page which is not an error. */
if (err != -ENOENT) if (err != -ENOENT)
f2fs_handle_page_eio(sbi, page->index, NODE); f2fs_handle_page_eio(sbi, page_folio(page), NODE);
f2fs_put_page(page, 1); f2fs_put_page(page, 1);
return ERR_PTR(err); return ERR_PTR(err);
} }
...@@ -1535,7 +1536,7 @@ static void flush_inline_data(struct f2fs_sb_info *sbi, nid_t ino) ...@@ -1535,7 +1536,7 @@ static void flush_inline_data(struct f2fs_sb_info *sbi, nid_t ino)
if (!clear_page_dirty_for_io(page)) if (!clear_page_dirty_for_io(page))
goto page_out; goto page_out;
ret = f2fs_write_inline_data(inode, page); ret = f2fs_write_inline_data(inode, page_folio(page));
inode_dec_dirty_pages(inode); inode_dec_dirty_pages(inode);
f2fs_remove_dirty_inode(inode); f2fs_remove_dirty_inode(inode);
if (ret) if (ret)
...@@ -1608,6 +1609,7 @@ static int __write_node_page(struct page *page, bool atomic, bool *submitted, ...@@ -1608,6 +1609,7 @@ static int __write_node_page(struct page *page, bool atomic, bool *submitted,
enum iostat_type io_type, unsigned int *seq_id) enum iostat_type io_type, unsigned int *seq_id)
{ {
struct f2fs_sb_info *sbi = F2FS_P_SB(page); struct f2fs_sb_info *sbi = F2FS_P_SB(page);
struct folio *folio = page_folio(page);
nid_t nid; nid_t nid;
struct node_info ni; struct node_info ni;
struct f2fs_io_info fio = { struct f2fs_io_info fio = {
...@@ -1624,15 +1626,15 @@ static int __write_node_page(struct page *page, bool atomic, bool *submitted, ...@@ -1624,15 +1626,15 @@ static int __write_node_page(struct page *page, bool atomic, bool *submitted,
}; };
unsigned int seq; unsigned int seq;
trace_f2fs_writepage(page_folio(page), NODE); trace_f2fs_writepage(folio, NODE);
if (unlikely(f2fs_cp_error(sbi))) { if (unlikely(f2fs_cp_error(sbi))) {
/* keep node pages in remount-ro mode */ /* keep node pages in remount-ro mode */
if (F2FS_OPTION(sbi).errors == MOUNT_ERRORS_READONLY) if (F2FS_OPTION(sbi).errors == MOUNT_ERRORS_READONLY)
goto redirty_out; goto redirty_out;
ClearPageUptodate(page); folio_clear_uptodate(folio);
dec_page_count(sbi, F2FS_DIRTY_NODES); dec_page_count(sbi, F2FS_DIRTY_NODES);
unlock_page(page); folio_unlock(folio);
return 0; return 0;
} }
...@@ -1646,7 +1648,7 @@ static int __write_node_page(struct page *page, bool atomic, bool *submitted, ...@@ -1646,7 +1648,7 @@ static int __write_node_page(struct page *page, bool atomic, bool *submitted,
/* get old block addr of this node page */ /* get old block addr of this node page */
nid = nid_of_node(page); nid = nid_of_node(page);
f2fs_bug_on(sbi, page->index != nid); f2fs_bug_on(sbi, folio->index != nid);
if (f2fs_get_node_info(sbi, nid, &ni, !do_balance)) if (f2fs_get_node_info(sbi, nid, &ni, !do_balance))
goto redirty_out; goto redirty_out;
...@@ -1660,10 +1662,10 @@ static int __write_node_page(struct page *page, bool atomic, bool *submitted, ...@@ -1660,10 +1662,10 @@ static int __write_node_page(struct page *page, bool atomic, bool *submitted,
/* This page is already truncated */ /* This page is already truncated */
if (unlikely(ni.blk_addr == NULL_ADDR)) { if (unlikely(ni.blk_addr == NULL_ADDR)) {
ClearPageUptodate(page); folio_clear_uptodate(folio);
dec_page_count(sbi, F2FS_DIRTY_NODES); dec_page_count(sbi, F2FS_DIRTY_NODES);
f2fs_up_read(&sbi->node_write); f2fs_up_read(&sbi->node_write);
unlock_page(page); folio_unlock(folio);
return 0; return 0;
} }
...@@ -1674,7 +1676,7 @@ static int __write_node_page(struct page *page, bool atomic, bool *submitted, ...@@ -1674,7 +1676,7 @@ static int __write_node_page(struct page *page, bool atomic, bool *submitted,
goto redirty_out; goto redirty_out;
} }
if (atomic && !test_opt(sbi, NOBARRIER) && !f2fs_sb_has_blkzoned(sbi)) if (atomic && !test_opt(sbi, NOBARRIER))
fio.op_flags |= REQ_PREFLUSH | REQ_FUA; fio.op_flags |= REQ_PREFLUSH | REQ_FUA;
/* should add to global list before clearing PAGECACHE status */ /* should add to global list before clearing PAGECACHE status */
...@@ -1684,7 +1686,7 @@ static int __write_node_page(struct page *page, bool atomic, bool *submitted, ...@@ -1684,7 +1686,7 @@ static int __write_node_page(struct page *page, bool atomic, bool *submitted,
*seq_id = seq; *seq_id = seq;
} }
set_page_writeback(page); folio_start_writeback(folio);
fio.old_blkaddr = ni.blk_addr; fio.old_blkaddr = ni.blk_addr;
f2fs_do_write_node_page(nid, &fio); f2fs_do_write_node_page(nid, &fio);
...@@ -1697,7 +1699,7 @@ static int __write_node_page(struct page *page, bool atomic, bool *submitted, ...@@ -1697,7 +1699,7 @@ static int __write_node_page(struct page *page, bool atomic, bool *submitted,
submitted = NULL; submitted = NULL;
} }
unlock_page(page); folio_unlock(folio);
if (unlikely(f2fs_cp_error(sbi))) { if (unlikely(f2fs_cp_error(sbi))) {
f2fs_submit_merged_write(sbi, NODE); f2fs_submit_merged_write(sbi, NODE);
...@@ -1711,7 +1713,7 @@ static int __write_node_page(struct page *page, bool atomic, bool *submitted, ...@@ -1711,7 +1713,7 @@ static int __write_node_page(struct page *page, bool atomic, bool *submitted,
return 0; return 0;
redirty_out: redirty_out:
redirty_page_for_writepage(wbc, page); folio_redirty_for_writepage(wbc, folio);
return AOP_WRITEPAGE_ACTIVATE; return AOP_WRITEPAGE_ACTIVATE;
} }
...@@ -1867,7 +1869,7 @@ int f2fs_fsync_node_pages(struct f2fs_sb_info *sbi, struct inode *inode, ...@@ -1867,7 +1869,7 @@ int f2fs_fsync_node_pages(struct f2fs_sb_info *sbi, struct inode *inode,
} }
if (!ret && atomic && !marked) { if (!ret && atomic && !marked) {
f2fs_debug(sbi, "Retry to write fsync mark: ino=%u, idx=%lx", f2fs_debug(sbi, "Retry to write fsync mark: ino=%u, idx=%lx",
ino, last_page->index); ino, page_folio(last_page)->index);
lock_page(last_page); lock_page(last_page);
f2fs_wait_on_page_writeback(last_page, NODE, true, true); f2fs_wait_on_page_writeback(last_page, NODE, true, true);
set_page_dirty(last_page); set_page_dirty(last_page);
...@@ -3166,7 +3168,7 @@ static int __get_nat_bitmaps(struct f2fs_sb_info *sbi) ...@@ -3166,7 +3168,7 @@ static int __get_nat_bitmaps(struct f2fs_sb_info *sbi)
nm_i->nat_bits_blocks = F2FS_BLK_ALIGN((nat_bits_bytes << 1) + 8); nm_i->nat_bits_blocks = F2FS_BLK_ALIGN((nat_bits_bytes << 1) + 8);
nm_i->nat_bits = f2fs_kvzalloc(sbi, nm_i->nat_bits = f2fs_kvzalloc(sbi,
nm_i->nat_bits_blocks << F2FS_BLKSIZE_BITS, GFP_KERNEL); F2FS_BLK_TO_BYTES(nm_i->nat_bits_blocks), GFP_KERNEL);
if (!nm_i->nat_bits) if (!nm_i->nat_bits)
return -ENOMEM; return -ENOMEM;
...@@ -3185,7 +3187,7 @@ static int __get_nat_bitmaps(struct f2fs_sb_info *sbi) ...@@ -3185,7 +3187,7 @@ static int __get_nat_bitmaps(struct f2fs_sb_info *sbi)
if (IS_ERR(page)) if (IS_ERR(page))
return PTR_ERR(page); return PTR_ERR(page);
memcpy(nm_i->nat_bits + (i << F2FS_BLKSIZE_BITS), memcpy(nm_i->nat_bits + F2FS_BLK_TO_BYTES(i),
page_address(page), F2FS_BLKSIZE); page_address(page), F2FS_BLKSIZE);
f2fs_put_page(page, 1); f2fs_put_page(page, 1);
} }
......
...@@ -199,6 +199,10 @@ void f2fs_abort_atomic_write(struct inode *inode, bool clean) ...@@ -199,6 +199,10 @@ void f2fs_abort_atomic_write(struct inode *inode, bool clean)
clear_inode_flag(inode, FI_ATOMIC_COMMITTED); clear_inode_flag(inode, FI_ATOMIC_COMMITTED);
clear_inode_flag(inode, FI_ATOMIC_REPLACE); clear_inode_flag(inode, FI_ATOMIC_REPLACE);
clear_inode_flag(inode, FI_ATOMIC_FILE); clear_inode_flag(inode, FI_ATOMIC_FILE);
if (is_inode_flag_set(inode, FI_ATOMIC_DIRTIED)) {
clear_inode_flag(inode, FI_ATOMIC_DIRTIED);
f2fs_mark_inode_dirty_sync(inode, true);
}
stat_dec_atomic_inode(inode); stat_dec_atomic_inode(inode);
F2FS_I(inode)->atomic_write_task = NULL; F2FS_I(inode)->atomic_write_task = NULL;
...@@ -366,6 +370,10 @@ static int __f2fs_commit_atomic_write(struct inode *inode) ...@@ -366,6 +370,10 @@ static int __f2fs_commit_atomic_write(struct inode *inode)
} else { } else {
sbi->committed_atomic_block += fi->atomic_write_cnt; sbi->committed_atomic_block += fi->atomic_write_cnt;
set_inode_flag(inode, FI_ATOMIC_COMMITTED); set_inode_flag(inode, FI_ATOMIC_COMMITTED);
if (is_inode_flag_set(inode, FI_ATOMIC_DIRTIED)) {
clear_inode_flag(inode, FI_ATOMIC_DIRTIED);
f2fs_mark_inode_dirty_sync(inode, true);
}
} }
__complete_revoke_list(inode, &revoke_list, ret ? true : false); __complete_revoke_list(inode, &revoke_list, ret ? true : false);
...@@ -1282,6 +1290,13 @@ static int __submit_discard_cmd(struct f2fs_sb_info *sbi, ...@@ -1282,6 +1290,13 @@ static int __submit_discard_cmd(struct f2fs_sb_info *sbi,
wait_list, issued); wait_list, issued);
return 0; return 0;
} }
/*
* Issue discard for conventional zones only if the device
* supports discard.
*/
if (!bdev_max_discard_sectors(bdev))
return -EOPNOTSUPP;
} }
#endif #endif
...@@ -2686,22 +2701,47 @@ static int get_new_segment(struct f2fs_sb_info *sbi, ...@@ -2686,22 +2701,47 @@ static int get_new_segment(struct f2fs_sb_info *sbi,
goto got_it; goto got_it;
} }
#ifdef CONFIG_BLK_DEV_ZONED
/* /*
* If we format f2fs on zoned storage, let's try to get pinned sections * If we format f2fs on zoned storage, let's try to get pinned sections
* from beginning of the storage, which should be a conventional one. * from beginning of the storage, which should be a conventional one.
*/ */
if (f2fs_sb_has_blkzoned(sbi)) { if (f2fs_sb_has_blkzoned(sbi)) {
segno = pinning ? 0 : max(first_zoned_segno(sbi), *newseg); /* Prioritize writing to conventional zones */
if (sbi->blkzone_alloc_policy == BLKZONE_ALLOC_PRIOR_CONV || pinning)
segno = 0;
else
segno = max(first_zoned_segno(sbi), *newseg);
hint = GET_SEC_FROM_SEG(sbi, segno); hint = GET_SEC_FROM_SEG(sbi, segno);
} }
#endif
find_other_zone: find_other_zone:
secno = find_next_zero_bit(free_i->free_secmap, MAIN_SECS(sbi), hint); secno = find_next_zero_bit(free_i->free_secmap, MAIN_SECS(sbi), hint);
#ifdef CONFIG_BLK_DEV_ZONED
if (secno >= MAIN_SECS(sbi) && f2fs_sb_has_blkzoned(sbi)) {
/* Write only to sequential zones */
if (sbi->blkzone_alloc_policy == BLKZONE_ALLOC_ONLY_SEQ) {
hint = GET_SEC_FROM_SEG(sbi, first_zoned_segno(sbi));
secno = find_next_zero_bit(free_i->free_secmap, MAIN_SECS(sbi), hint);
} else
secno = find_first_zero_bit(free_i->free_secmap,
MAIN_SECS(sbi));
if (secno >= MAIN_SECS(sbi)) {
ret = -ENOSPC;
f2fs_bug_on(sbi, 1);
goto out_unlock;
}
}
#endif
if (secno >= MAIN_SECS(sbi)) { if (secno >= MAIN_SECS(sbi)) {
secno = find_first_zero_bit(free_i->free_secmap, secno = find_first_zero_bit(free_i->free_secmap,
MAIN_SECS(sbi)); MAIN_SECS(sbi));
if (secno >= MAIN_SECS(sbi)) { if (secno >= MAIN_SECS(sbi)) {
ret = -ENOSPC; ret = -ENOSPC;
f2fs_bug_on(sbi, 1);
goto out_unlock; goto out_unlock;
} }
} }
...@@ -2743,10 +2783,8 @@ static int get_new_segment(struct f2fs_sb_info *sbi, ...@@ -2743,10 +2783,8 @@ static int get_new_segment(struct f2fs_sb_info *sbi,
out_unlock: out_unlock:
spin_unlock(&free_i->segmap_lock); spin_unlock(&free_i->segmap_lock);
if (ret == -ENOSPC) { if (ret == -ENOSPC)
f2fs_stop_checkpoint(sbi, false, STOP_CP_REASON_NO_SEGMENT); f2fs_stop_checkpoint(sbi, false, STOP_CP_REASON_NO_SEGMENT);
f2fs_bug_on(sbi, 1);
}
return ret; return ret;
} }
...@@ -3052,7 +3090,8 @@ static int get_ssr_segment(struct f2fs_sb_info *sbi, int type, ...@@ -3052,7 +3090,8 @@ static int get_ssr_segment(struct f2fs_sb_info *sbi, int type,
sanity_check_seg_type(sbi, seg_type); sanity_check_seg_type(sbi, seg_type);
/* f2fs_need_SSR() already forces to do this */ /* f2fs_need_SSR() already forces to do this */
if (!f2fs_get_victim(sbi, &segno, BG_GC, seg_type, alloc_mode, age)) { if (!f2fs_get_victim(sbi, &segno, BG_GC, seg_type,
alloc_mode, age, false)) {
curseg->next_segno = segno; curseg->next_segno = segno;
return 1; return 1;
} }
...@@ -3079,7 +3118,8 @@ static int get_ssr_segment(struct f2fs_sb_info *sbi, int type, ...@@ -3079,7 +3118,8 @@ static int get_ssr_segment(struct f2fs_sb_info *sbi, int type,
for (; cnt-- > 0; reversed ? i-- : i++) { for (; cnt-- > 0; reversed ? i-- : i++) {
if (i == seg_type) if (i == seg_type)
continue; continue;
if (!f2fs_get_victim(sbi, &segno, BG_GC, i, alloc_mode, age)) { if (!f2fs_get_victim(sbi, &segno, BG_GC, i,
alloc_mode, age, false)) {
curseg->next_segno = segno; curseg->next_segno = segno;
return 1; return 1;
} }
...@@ -3522,7 +3562,8 @@ static int __get_segment_type_6(struct f2fs_io_info *fio) ...@@ -3522,7 +3562,8 @@ static int __get_segment_type_6(struct f2fs_io_info *fio)
if (file_is_cold(inode) || f2fs_need_compress_data(inode)) if (file_is_cold(inode) || f2fs_need_compress_data(inode))
return CURSEG_COLD_DATA; return CURSEG_COLD_DATA;
type = __get_age_segment_type(inode, fio->page->index); type = __get_age_segment_type(inode,
page_folio(fio->page)->index);
if (type != NO_CHECK_TYPE) if (type != NO_CHECK_TYPE)
return type; return type;
...@@ -3781,7 +3822,7 @@ static void do_write_page(struct f2fs_summary *sum, struct f2fs_io_info *fio) ...@@ -3781,7 +3822,7 @@ static void do_write_page(struct f2fs_summary *sum, struct f2fs_io_info *fio)
f2fs_up_read(&fio->sbi->io_order_lock); f2fs_up_read(&fio->sbi->io_order_lock);
} }
void f2fs_do_write_meta_page(struct f2fs_sb_info *sbi, struct page *page, void f2fs_do_write_meta_page(struct f2fs_sb_info *sbi, struct folio *folio,
enum iostat_type io_type) enum iostat_type io_type)
{ {
struct f2fs_io_info fio = { struct f2fs_io_info fio = {
...@@ -3790,20 +3831,20 @@ void f2fs_do_write_meta_page(struct f2fs_sb_info *sbi, struct page *page, ...@@ -3790,20 +3831,20 @@ void f2fs_do_write_meta_page(struct f2fs_sb_info *sbi, struct page *page,
.temp = HOT, .temp = HOT,
.op = REQ_OP_WRITE, .op = REQ_OP_WRITE,
.op_flags = REQ_SYNC | REQ_META | REQ_PRIO, .op_flags = REQ_SYNC | REQ_META | REQ_PRIO,
.old_blkaddr = page->index, .old_blkaddr = folio->index,
.new_blkaddr = page->index, .new_blkaddr = folio->index,
.page = page, .page = folio_page(folio, 0),
.encrypted_page = NULL, .encrypted_page = NULL,
.in_list = 0, .in_list = 0,
}; };
if (unlikely(page->index >= MAIN_BLKADDR(sbi))) if (unlikely(folio->index >= MAIN_BLKADDR(sbi)))
fio.op_flags &= ~REQ_META; fio.op_flags &= ~REQ_META;
set_page_writeback(page); folio_start_writeback(folio);
f2fs_submit_page_write(&fio); f2fs_submit_page_write(&fio);
stat_inc_meta_count(sbi, page->index); stat_inc_meta_count(sbi, folio->index);
f2fs_update_iostat(sbi, NULL, io_type, F2FS_BLKSIZE); f2fs_update_iostat(sbi, NULL, io_type, F2FS_BLKSIZE);
} }
...@@ -5381,8 +5422,7 @@ unsigned int f2fs_usable_blks_in_seg(struct f2fs_sb_info *sbi, ...@@ -5381,8 +5422,7 @@ unsigned int f2fs_usable_blks_in_seg(struct f2fs_sb_info *sbi,
return BLKS_PER_SEG(sbi); return BLKS_PER_SEG(sbi);
} }
unsigned int f2fs_usable_segs_in_sec(struct f2fs_sb_info *sbi, unsigned int f2fs_usable_segs_in_sec(struct f2fs_sb_info *sbi)
unsigned int segno)
{ {
if (f2fs_sb_has_blkzoned(sbi)) if (f2fs_sb_has_blkzoned(sbi))
return CAP_SEGS_PER_SEC(sbi); return CAP_SEGS_PER_SEC(sbi);
......
...@@ -188,6 +188,7 @@ struct victim_sel_policy { ...@@ -188,6 +188,7 @@ struct victim_sel_policy {
unsigned int min_segno; /* segment # having min. cost */ unsigned int min_segno; /* segment # having min. cost */
unsigned long long age; /* mtime of GCed section*/ unsigned long long age; /* mtime of GCed section*/
unsigned long long age_threshold;/* age threshold */ unsigned long long age_threshold;/* age threshold */
bool one_time_gc; /* one time GC */
}; };
struct seg_entry { struct seg_entry {
...@@ -430,7 +431,7 @@ static inline void __set_free(struct f2fs_sb_info *sbi, unsigned int segno) ...@@ -430,7 +431,7 @@ static inline void __set_free(struct f2fs_sb_info *sbi, unsigned int segno)
unsigned int secno = GET_SEC_FROM_SEG(sbi, segno); unsigned int secno = GET_SEC_FROM_SEG(sbi, segno);
unsigned int start_segno = GET_SEG_FROM_SEC(sbi, secno); unsigned int start_segno = GET_SEG_FROM_SEC(sbi, secno);
unsigned int next; unsigned int next;
unsigned int usable_segs = f2fs_usable_segs_in_sec(sbi, segno); unsigned int usable_segs = f2fs_usable_segs_in_sec(sbi);
spin_lock(&free_i->segmap_lock); spin_lock(&free_i->segmap_lock);
clear_bit(segno, free_i->free_segmap); clear_bit(segno, free_i->free_segmap);
...@@ -464,7 +465,7 @@ static inline void __set_test_and_free(struct f2fs_sb_info *sbi, ...@@ -464,7 +465,7 @@ static inline void __set_test_and_free(struct f2fs_sb_info *sbi,
unsigned int secno = GET_SEC_FROM_SEG(sbi, segno); unsigned int secno = GET_SEC_FROM_SEG(sbi, segno);
unsigned int start_segno = GET_SEG_FROM_SEC(sbi, secno); unsigned int start_segno = GET_SEG_FROM_SEC(sbi, secno);
unsigned int next; unsigned int next;
unsigned int usable_segs = f2fs_usable_segs_in_sec(sbi, segno); unsigned int usable_segs = f2fs_usable_segs_in_sec(sbi);
spin_lock(&free_i->segmap_lock); spin_lock(&free_i->segmap_lock);
if (test_and_clear_bit(segno, free_i->free_segmap)) { if (test_and_clear_bit(segno, free_i->free_segmap)) {
......
...@@ -11,7 +11,6 @@ ...@@ -11,7 +11,6 @@
#include <linux/fs_context.h> #include <linux/fs_context.h>
#include <linux/sched/mm.h> #include <linux/sched/mm.h>
#include <linux/statfs.h> #include <linux/statfs.h>
#include <linux/buffer_head.h>
#include <linux/kthread.h> #include <linux/kthread.h>
#include <linux/parser.h> #include <linux/parser.h>
#include <linux/mount.h> #include <linux/mount.h>
...@@ -707,6 +706,11 @@ static int parse_options(struct super_block *sb, char *options, bool is_remount) ...@@ -707,6 +706,11 @@ static int parse_options(struct super_block *sb, char *options, bool is_remount)
if (!strcmp(name, "on")) { if (!strcmp(name, "on")) {
F2FS_OPTION(sbi).bggc_mode = BGGC_MODE_ON; F2FS_OPTION(sbi).bggc_mode = BGGC_MODE_ON;
} else if (!strcmp(name, "off")) { } else if (!strcmp(name, "off")) {
if (f2fs_sb_has_blkzoned(sbi)) {
f2fs_warn(sbi, "zoned devices need bggc");
kfree(name);
return -EINVAL;
}
F2FS_OPTION(sbi).bggc_mode = BGGC_MODE_OFF; F2FS_OPTION(sbi).bggc_mode = BGGC_MODE_OFF;
} else if (!strcmp(name, "sync")) { } else if (!strcmp(name, "sync")) {
F2FS_OPTION(sbi).bggc_mode = BGGC_MODE_SYNC; F2FS_OPTION(sbi).bggc_mode = BGGC_MODE_SYNC;
...@@ -2561,7 +2565,7 @@ static int f2fs_remount(struct super_block *sb, int *flags, char *data) ...@@ -2561,7 +2565,7 @@ static int f2fs_remount(struct super_block *sb, int *flags, char *data)
static void f2fs_shutdown(struct super_block *sb) static void f2fs_shutdown(struct super_block *sb)
{ {
f2fs_do_shutdown(F2FS_SB(sb), F2FS_GOING_DOWN_NOSYNC, false); f2fs_do_shutdown(F2FS_SB(sb), F2FS_GOING_DOWN_NOSYNC, false, false);
} }
#ifdef CONFIG_QUOTA #ifdef CONFIG_QUOTA
...@@ -3318,29 +3322,47 @@ loff_t max_file_blocks(struct inode *inode) ...@@ -3318,29 +3322,47 @@ loff_t max_file_blocks(struct inode *inode)
* fit within U32_MAX + 1 data units. * fit within U32_MAX + 1 data units.
*/ */
result = min(result, (((loff_t)U32_MAX + 1) * 4096) >> F2FS_BLKSIZE_BITS); result = min(result, F2FS_BYTES_TO_BLK(((loff_t)U32_MAX + 1) * 4096));
return result; return result;
} }
static int __f2fs_commit_super(struct buffer_head *bh, static int __f2fs_commit_super(struct f2fs_sb_info *sbi, struct folio *folio,
struct f2fs_super_block *super) pgoff_t index, bool update)
{ {
lock_buffer(bh); struct bio *bio;
if (super)
memcpy(bh->b_data + F2FS_SUPER_OFFSET, super, sizeof(*super));
set_buffer_dirty(bh);
unlock_buffer(bh);
/* it's rare case, we can do fua all the time */ /* it's rare case, we can do fua all the time */
return __sync_dirty_buffer(bh, REQ_SYNC | REQ_PREFLUSH | REQ_FUA); blk_opf_t opf = REQ_OP_WRITE | REQ_SYNC | REQ_PREFLUSH | REQ_FUA;
int ret;
folio_lock(folio);
folio_wait_writeback(folio);
if (update)
memcpy(F2FS_SUPER_BLOCK(folio, index), F2FS_RAW_SUPER(sbi),
sizeof(struct f2fs_super_block));
folio_mark_dirty(folio);
folio_clear_dirty_for_io(folio);
folio_start_writeback(folio);
folio_unlock(folio);
bio = bio_alloc(sbi->sb->s_bdev, 1, opf, GFP_NOFS);
/* it doesn't need to set crypto context for superblock update */
bio->bi_iter.bi_sector = SECTOR_FROM_BLOCK(folio_index(folio));
if (!bio_add_folio(bio, folio, folio_size(folio), 0))
f2fs_bug_on(sbi, 1);
ret = submit_bio_wait(bio);
folio_end_writeback(folio);
return ret;
} }
static inline bool sanity_check_area_boundary(struct f2fs_sb_info *sbi, static inline bool sanity_check_area_boundary(struct f2fs_sb_info *sbi,
struct buffer_head *bh) struct folio *folio, pgoff_t index)
{ {
struct f2fs_super_block *raw_super = (struct f2fs_super_block *) struct f2fs_super_block *raw_super = F2FS_SUPER_BLOCK(folio, index);
(bh->b_data + F2FS_SUPER_OFFSET);
struct super_block *sb = sbi->sb; struct super_block *sb = sbi->sb;
u32 segment0_blkaddr = le32_to_cpu(raw_super->segment0_blkaddr); u32 segment0_blkaddr = le32_to_cpu(raw_super->segment0_blkaddr);
u32 cp_blkaddr = le32_to_cpu(raw_super->cp_blkaddr); u32 cp_blkaddr = le32_to_cpu(raw_super->cp_blkaddr);
...@@ -3356,9 +3378,9 @@ static inline bool sanity_check_area_boundary(struct f2fs_sb_info *sbi, ...@@ -3356,9 +3378,9 @@ static inline bool sanity_check_area_boundary(struct f2fs_sb_info *sbi,
u32 segment_count = le32_to_cpu(raw_super->segment_count); u32 segment_count = le32_to_cpu(raw_super->segment_count);
u32 log_blocks_per_seg = le32_to_cpu(raw_super->log_blocks_per_seg); u32 log_blocks_per_seg = le32_to_cpu(raw_super->log_blocks_per_seg);
u64 main_end_blkaddr = main_blkaddr + u64 main_end_blkaddr = main_blkaddr +
(segment_count_main << log_blocks_per_seg); ((u64)segment_count_main << log_blocks_per_seg);
u64 seg_end_blkaddr = segment0_blkaddr + u64 seg_end_blkaddr = segment0_blkaddr +
(segment_count << log_blocks_per_seg); ((u64)segment_count << log_blocks_per_seg);
if (segment0_blkaddr != cp_blkaddr) { if (segment0_blkaddr != cp_blkaddr) {
f2fs_info(sbi, "Mismatch start address, segment0(%u) cp_blkaddr(%u)", f2fs_info(sbi, "Mismatch start address, segment0(%u) cp_blkaddr(%u)",
...@@ -3415,7 +3437,7 @@ static inline bool sanity_check_area_boundary(struct f2fs_sb_info *sbi, ...@@ -3415,7 +3437,7 @@ static inline bool sanity_check_area_boundary(struct f2fs_sb_info *sbi,
set_sbi_flag(sbi, SBI_NEED_SB_WRITE); set_sbi_flag(sbi, SBI_NEED_SB_WRITE);
res = "internally"; res = "internally";
} else { } else {
err = __f2fs_commit_super(bh, NULL); err = __f2fs_commit_super(sbi, folio, index, false);
res = err ? "failed" : "done"; res = err ? "failed" : "done";
} }
f2fs_info(sbi, "Fix alignment : %s, start(%u) end(%llu) block(%u)", f2fs_info(sbi, "Fix alignment : %s, start(%u) end(%llu) block(%u)",
...@@ -3428,12 +3450,11 @@ static inline bool sanity_check_area_boundary(struct f2fs_sb_info *sbi, ...@@ -3428,12 +3450,11 @@ static inline bool sanity_check_area_boundary(struct f2fs_sb_info *sbi,
} }
static int sanity_check_raw_super(struct f2fs_sb_info *sbi, static int sanity_check_raw_super(struct f2fs_sb_info *sbi,
struct buffer_head *bh) struct folio *folio, pgoff_t index)
{ {
block_t segment_count, segs_per_sec, secs_per_zone, segment_count_main; block_t segment_count, segs_per_sec, secs_per_zone, segment_count_main;
block_t total_sections, blocks_per_seg; block_t total_sections, blocks_per_seg;
struct f2fs_super_block *raw_super = (struct f2fs_super_block *) struct f2fs_super_block *raw_super = F2FS_SUPER_BLOCK(folio, index);
(bh->b_data + F2FS_SUPER_OFFSET);
size_t crc_offset = 0; size_t crc_offset = 0;
__u32 crc = 0; __u32 crc = 0;
...@@ -3591,7 +3612,7 @@ static int sanity_check_raw_super(struct f2fs_sb_info *sbi, ...@@ -3591,7 +3612,7 @@ static int sanity_check_raw_super(struct f2fs_sb_info *sbi,
} }
/* check CP/SIT/NAT/SSA/MAIN_AREA area boundary */ /* check CP/SIT/NAT/SSA/MAIN_AREA area boundary */
if (sanity_check_area_boundary(sbi, bh)) if (sanity_check_area_boundary(sbi, folio, index))
return -EFSCORRUPTED; return -EFSCORRUPTED;
return 0; return 0;
...@@ -3786,6 +3807,8 @@ static void init_sb_info(struct f2fs_sb_info *sbi) ...@@ -3786,6 +3807,8 @@ static void init_sb_info(struct f2fs_sb_info *sbi)
sbi->next_victim_seg[FG_GC] = NULL_SEGNO; sbi->next_victim_seg[FG_GC] = NULL_SEGNO;
sbi->max_victim_search = DEF_MAX_VICTIM_SEARCH; sbi->max_victim_search = DEF_MAX_VICTIM_SEARCH;
sbi->migration_granularity = SEGS_PER_SEC(sbi); sbi->migration_granularity = SEGS_PER_SEC(sbi);
sbi->migration_window_granularity = f2fs_sb_has_blkzoned(sbi) ?
DEF_MIGRATION_WINDOW_GRANULARITY_ZONED : SEGS_PER_SEC(sbi);
sbi->seq_file_ra_mul = MIN_RA_MUL; sbi->seq_file_ra_mul = MIN_RA_MUL;
sbi->max_fragment_chunk = DEF_FRAGMENT_SIZE; sbi->max_fragment_chunk = DEF_FRAGMENT_SIZE;
sbi->max_fragment_hole = DEF_FRAGMENT_SIZE; sbi->max_fragment_hole = DEF_FRAGMENT_SIZE;
...@@ -3938,7 +3961,7 @@ static int read_raw_super_block(struct f2fs_sb_info *sbi, ...@@ -3938,7 +3961,7 @@ static int read_raw_super_block(struct f2fs_sb_info *sbi,
{ {
struct super_block *sb = sbi->sb; struct super_block *sb = sbi->sb;
int block; int block;
struct buffer_head *bh; struct folio *folio;
struct f2fs_super_block *super; struct f2fs_super_block *super;
int err = 0; int err = 0;
...@@ -3947,32 +3970,32 @@ static int read_raw_super_block(struct f2fs_sb_info *sbi, ...@@ -3947,32 +3970,32 @@ static int read_raw_super_block(struct f2fs_sb_info *sbi,
return -ENOMEM; return -ENOMEM;
for (block = 0; block < 2; block++) { for (block = 0; block < 2; block++) {
bh = sb_bread(sb, block); folio = read_mapping_folio(sb->s_bdev->bd_mapping, block, NULL);
if (!bh) { if (IS_ERR(folio)) {
f2fs_err(sbi, "Unable to read %dth superblock", f2fs_err(sbi, "Unable to read %dth superblock",
block + 1); block + 1);
err = -EIO; err = PTR_ERR(folio);
*recovery = 1; *recovery = 1;
continue; continue;
} }
/* sanity checking of raw super */ /* sanity checking of raw super */
err = sanity_check_raw_super(sbi, bh); err = sanity_check_raw_super(sbi, folio, block);
if (err) { if (err) {
f2fs_err(sbi, "Can't find valid F2FS filesystem in %dth superblock", f2fs_err(sbi, "Can't find valid F2FS filesystem in %dth superblock",
block + 1); block + 1);
brelse(bh); folio_put(folio);
*recovery = 1; *recovery = 1;
continue; continue;
} }
if (!*raw_super) { if (!*raw_super) {
memcpy(super, bh->b_data + F2FS_SUPER_OFFSET, memcpy(super, F2FS_SUPER_BLOCK(folio, block),
sizeof(*super)); sizeof(*super));
*valid_super_block = block; *valid_super_block = block;
*raw_super = super; *raw_super = super;
} }
brelse(bh); folio_put(folio);
} }
/* No valid superblock */ /* No valid superblock */
...@@ -3986,7 +4009,8 @@ static int read_raw_super_block(struct f2fs_sb_info *sbi, ...@@ -3986,7 +4009,8 @@ static int read_raw_super_block(struct f2fs_sb_info *sbi,
int f2fs_commit_super(struct f2fs_sb_info *sbi, bool recover) int f2fs_commit_super(struct f2fs_sb_info *sbi, bool recover)
{ {
struct buffer_head *bh; struct folio *folio;
pgoff_t index;
__u32 crc = 0; __u32 crc = 0;
int err; int err;
...@@ -4004,22 +4028,24 @@ int f2fs_commit_super(struct f2fs_sb_info *sbi, bool recover) ...@@ -4004,22 +4028,24 @@ int f2fs_commit_super(struct f2fs_sb_info *sbi, bool recover)
} }
/* write back-up superblock first */ /* write back-up superblock first */
bh = sb_bread(sbi->sb, sbi->valid_super_block ? 0 : 1); index = sbi->valid_super_block ? 0 : 1;
if (!bh) folio = read_mapping_folio(sbi->sb->s_bdev->bd_mapping, index, NULL);
return -EIO; if (IS_ERR(folio))
err = __f2fs_commit_super(bh, F2FS_RAW_SUPER(sbi)); return PTR_ERR(folio);
brelse(bh); err = __f2fs_commit_super(sbi, folio, index, true);
folio_put(folio);
/* if we are in recovery path, skip writing valid superblock */ /* if we are in recovery path, skip writing valid superblock */
if (recover || err) if (recover || err)
return err; return err;
/* write current valid superblock */ /* write current valid superblock */
bh = sb_bread(sbi->sb, sbi->valid_super_block); index = sbi->valid_super_block;
if (!bh) folio = read_mapping_folio(sbi->sb->s_bdev->bd_mapping, index, NULL);
return -EIO; if (IS_ERR(folio))
err = __f2fs_commit_super(bh, F2FS_RAW_SUPER(sbi)); return PTR_ERR(folio);
brelse(bh); err = __f2fs_commit_super(sbi, folio, index, true);
folio_put(folio);
return err; return err;
} }
...@@ -4173,12 +4199,14 @@ void f2fs_handle_critical_error(struct f2fs_sb_info *sbi, unsigned char reason, ...@@ -4173,12 +4199,14 @@ void f2fs_handle_critical_error(struct f2fs_sb_info *sbi, unsigned char reason,
} }
f2fs_warn(sbi, "Remounting filesystem read-only"); f2fs_warn(sbi, "Remounting filesystem read-only");
/* /*
* Make sure updated value of ->s_mount_flags will be visible before * We have already set CP_ERROR_FLAG flag to stop all updates
* ->s_flags update * to filesystem, so it doesn't need to set SB_RDONLY flag here
* because the flag should be set covered w/ sb->s_umount semaphore
* via remount procedure, otherwise, it will confuse code like
* freeze_super() which will lead to deadlocks and other problems.
*/ */
smp_wmb();
sb->s_flags |= SB_RDONLY;
} }
static void f2fs_record_error_work(struct work_struct *work) static void f2fs_record_error_work(struct work_struct *work)
...@@ -4219,6 +4247,7 @@ static int f2fs_scan_devices(struct f2fs_sb_info *sbi) ...@@ -4219,6 +4247,7 @@ static int f2fs_scan_devices(struct f2fs_sb_info *sbi)
sbi->aligned_blksize = true; sbi->aligned_blksize = true;
#ifdef CONFIG_BLK_DEV_ZONED #ifdef CONFIG_BLK_DEV_ZONED
sbi->max_open_zones = UINT_MAX; sbi->max_open_zones = UINT_MAX;
sbi->blkzone_alloc_policy = BLKZONE_ALLOC_PRIOR_SEQ;
#endif #endif
for (i = 0; i < max_devices; i++) { for (i = 0; i < max_devices; i++) {
......
...@@ -170,6 +170,12 @@ static ssize_t undiscard_blks_show(struct f2fs_attr *a, ...@@ -170,6 +170,12 @@ static ssize_t undiscard_blks_show(struct f2fs_attr *a,
SM_I(sbi)->dcc_info->undiscard_blks); SM_I(sbi)->dcc_info->undiscard_blks);
} }
static ssize_t atgc_enabled_show(struct f2fs_attr *a,
struct f2fs_sb_info *sbi, char *buf)
{
return sysfs_emit(buf, "%d\n", sbi->am.atgc_enabled ? 1 : 0);
}
static ssize_t gc_mode_show(struct f2fs_attr *a, static ssize_t gc_mode_show(struct f2fs_attr *a,
struct f2fs_sb_info *sbi, char *buf) struct f2fs_sb_info *sbi, char *buf)
{ {
...@@ -182,50 +188,50 @@ static ssize_t features_show(struct f2fs_attr *a, ...@@ -182,50 +188,50 @@ static ssize_t features_show(struct f2fs_attr *a,
int len = 0; int len = 0;
if (f2fs_sb_has_encrypt(sbi)) if (f2fs_sb_has_encrypt(sbi))
len += scnprintf(buf, PAGE_SIZE - len, "%s", len += sysfs_emit_at(buf, len, "%s",
"encryption"); "encryption");
if (f2fs_sb_has_blkzoned(sbi)) if (f2fs_sb_has_blkzoned(sbi))
len += scnprintf(buf + len, PAGE_SIZE - len, "%s%s", len += sysfs_emit_at(buf, len, "%s%s",
len ? ", " : "", "blkzoned"); len ? ", " : "", "blkzoned");
if (f2fs_sb_has_extra_attr(sbi)) if (f2fs_sb_has_extra_attr(sbi))
len += scnprintf(buf + len, PAGE_SIZE - len, "%s%s", len += sysfs_emit_at(buf, len, "%s%s",
len ? ", " : "", "extra_attr"); len ? ", " : "", "extra_attr");
if (f2fs_sb_has_project_quota(sbi)) if (f2fs_sb_has_project_quota(sbi))
len += scnprintf(buf + len, PAGE_SIZE - len, "%s%s", len += sysfs_emit_at(buf, len, "%s%s",
len ? ", " : "", "projquota"); len ? ", " : "", "projquota");
if (f2fs_sb_has_inode_chksum(sbi)) if (f2fs_sb_has_inode_chksum(sbi))
len += scnprintf(buf + len, PAGE_SIZE - len, "%s%s", len += sysfs_emit_at(buf, len, "%s%s",
len ? ", " : "", "inode_checksum"); len ? ", " : "", "inode_checksum");
if (f2fs_sb_has_flexible_inline_xattr(sbi)) if (f2fs_sb_has_flexible_inline_xattr(sbi))
len += scnprintf(buf + len, PAGE_SIZE - len, "%s%s", len += sysfs_emit_at(buf, len, "%s%s",
len ? ", " : "", "flexible_inline_xattr"); len ? ", " : "", "flexible_inline_xattr");
if (f2fs_sb_has_quota_ino(sbi)) if (f2fs_sb_has_quota_ino(sbi))
len += scnprintf(buf + len, PAGE_SIZE - len, "%s%s", len += sysfs_emit_at(buf, len, "%s%s",
len ? ", " : "", "quota_ino"); len ? ", " : "", "quota_ino");
if (f2fs_sb_has_inode_crtime(sbi)) if (f2fs_sb_has_inode_crtime(sbi))
len += scnprintf(buf + len, PAGE_SIZE - len, "%s%s", len += sysfs_emit_at(buf, len, "%s%s",
len ? ", " : "", "inode_crtime"); len ? ", " : "", "inode_crtime");
if (f2fs_sb_has_lost_found(sbi)) if (f2fs_sb_has_lost_found(sbi))
len += scnprintf(buf + len, PAGE_SIZE - len, "%s%s", len += sysfs_emit_at(buf, len, "%s%s",
len ? ", " : "", "lost_found"); len ? ", " : "", "lost_found");
if (f2fs_sb_has_verity(sbi)) if (f2fs_sb_has_verity(sbi))
len += scnprintf(buf + len, PAGE_SIZE - len, "%s%s", len += sysfs_emit_at(buf, len, "%s%s",
len ? ", " : "", "verity"); len ? ", " : "", "verity");
if (f2fs_sb_has_sb_chksum(sbi)) if (f2fs_sb_has_sb_chksum(sbi))
len += scnprintf(buf + len, PAGE_SIZE - len, "%s%s", len += sysfs_emit_at(buf, len, "%s%s",
len ? ", " : "", "sb_checksum"); len ? ", " : "", "sb_checksum");
if (f2fs_sb_has_casefold(sbi)) if (f2fs_sb_has_casefold(sbi))
len += scnprintf(buf + len, PAGE_SIZE - len, "%s%s", len += sysfs_emit_at(buf, len, "%s%s",
len ? ", " : "", "casefold"); len ? ", " : "", "casefold");
if (f2fs_sb_has_readonly(sbi)) if (f2fs_sb_has_readonly(sbi))
len += scnprintf(buf + len, PAGE_SIZE - len, "%s%s", len += sysfs_emit_at(buf, len, "%s%s",
len ? ", " : "", "readonly"); len ? ", " : "", "readonly");
if (f2fs_sb_has_compression(sbi)) if (f2fs_sb_has_compression(sbi))
len += scnprintf(buf + len, PAGE_SIZE - len, "%s%s", len += sysfs_emit_at(buf, len, "%s%s",
len ? ", " : "", "compression"); len ? ", " : "", "compression");
len += scnprintf(buf + len, PAGE_SIZE - len, "%s%s", len += sysfs_emit_at(buf, len, "%s%s",
len ? ", " : "", "pin_file"); len ? ", " : "", "pin_file");
len += scnprintf(buf + len, PAGE_SIZE - len, "\n"); len += sysfs_emit_at(buf, len, "\n");
return len; return len;
} }
...@@ -323,17 +329,14 @@ static ssize_t f2fs_sbi_show(struct f2fs_attr *a, ...@@ -323,17 +329,14 @@ static ssize_t f2fs_sbi_show(struct f2fs_attr *a,
int hot_count = sbi->raw_super->hot_ext_count; int hot_count = sbi->raw_super->hot_ext_count;
int len = 0, i; int len = 0, i;
len += scnprintf(buf + len, PAGE_SIZE - len, len += sysfs_emit_at(buf, len, "cold file extension:\n");
"cold file extension:\n");
for (i = 0; i < cold_count; i++) for (i = 0; i < cold_count; i++)
len += scnprintf(buf + len, PAGE_SIZE - len, "%s\n", len += sysfs_emit_at(buf, len, "%s\n", extlist[i]);
extlist[i]);
len += scnprintf(buf + len, PAGE_SIZE - len, len += sysfs_emit_at(buf, len, "hot file extension:\n");
"hot file extension:\n");
for (i = cold_count; i < cold_count + hot_count; i++) for (i = cold_count; i < cold_count + hot_count; i++)
len += scnprintf(buf + len, PAGE_SIZE - len, "%s\n", len += sysfs_emit_at(buf, len, "%s\n", extlist[i]);
extlist[i]);
return len; return len;
} }
...@@ -561,6 +564,11 @@ static ssize_t __sbi_store(struct f2fs_attr *a, ...@@ -561,6 +564,11 @@ static ssize_t __sbi_store(struct f2fs_attr *a,
return -EINVAL; return -EINVAL;
} }
if (!strcmp(a->attr.name, "migration_window_granularity")) {
if (t == 0 || t > SEGS_PER_SEC(sbi))
return -EINVAL;
}
if (!strcmp(a->attr.name, "gc_urgent")) { if (!strcmp(a->attr.name, "gc_urgent")) {
if (t == 0) { if (t == 0) {
sbi->gc_mode = GC_NORMAL; sbi->gc_mode = GC_NORMAL;
...@@ -627,6 +635,15 @@ static ssize_t __sbi_store(struct f2fs_attr *a, ...@@ -627,6 +635,15 @@ static ssize_t __sbi_store(struct f2fs_attr *a,
} }
#endif #endif
#ifdef CONFIG_BLK_DEV_ZONED
if (!strcmp(a->attr.name, "blkzone_alloc_policy")) {
if (t < BLKZONE_ALLOC_PRIOR_SEQ || t > BLKZONE_ALLOC_PRIOR_CONV)
return -EINVAL;
sbi->blkzone_alloc_policy = t;
return count;
}
#endif
#ifdef CONFIG_F2FS_FS_COMPRESSION #ifdef CONFIG_F2FS_FS_COMPRESSION
if (!strcmp(a->attr.name, "compr_written_block") || if (!strcmp(a->attr.name, "compr_written_block") ||
!strcmp(a->attr.name, "compr_saved_block")) { !strcmp(a->attr.name, "compr_saved_block")) {
...@@ -775,7 +792,8 @@ static ssize_t __sbi_store(struct f2fs_attr *a, ...@@ -775,7 +792,8 @@ static ssize_t __sbi_store(struct f2fs_attr *a,
if (!strcmp(a->attr.name, "ipu_policy")) { if (!strcmp(a->attr.name, "ipu_policy")) {
if (t >= BIT(F2FS_IPU_MAX)) if (t >= BIT(F2FS_IPU_MAX))
return -EINVAL; return -EINVAL;
if (t && f2fs_lfs_mode(sbi)) /* allow F2FS_IPU_NOCACHE only for IPU in the pinned file */
if (f2fs_lfs_mode(sbi) && (t & ~BIT(F2FS_IPU_NOCACHE)))
return -EINVAL; return -EINVAL;
SM_I(sbi)->ipu_policy = (unsigned int)t; SM_I(sbi)->ipu_policy = (unsigned int)t;
return count; return count;
...@@ -960,6 +978,9 @@ GC_THREAD_RW_ATTR(gc_urgent_sleep_time, urgent_sleep_time); ...@@ -960,6 +978,9 @@ GC_THREAD_RW_ATTR(gc_urgent_sleep_time, urgent_sleep_time);
GC_THREAD_RW_ATTR(gc_min_sleep_time, min_sleep_time); GC_THREAD_RW_ATTR(gc_min_sleep_time, min_sleep_time);
GC_THREAD_RW_ATTR(gc_max_sleep_time, max_sleep_time); GC_THREAD_RW_ATTR(gc_max_sleep_time, max_sleep_time);
GC_THREAD_RW_ATTR(gc_no_gc_sleep_time, no_gc_sleep_time); GC_THREAD_RW_ATTR(gc_no_gc_sleep_time, no_gc_sleep_time);
GC_THREAD_RW_ATTR(gc_no_zoned_gc_percent, no_zoned_gc_percent);
GC_THREAD_RW_ATTR(gc_boost_zoned_gc_percent, boost_zoned_gc_percent);
GC_THREAD_RW_ATTR(gc_valid_thresh_ratio, valid_thresh_ratio);
/* SM_INFO ATTR */ /* SM_INFO ATTR */
SM_INFO_RW_ATTR(reclaim_segments, rec_prefree_segments); SM_INFO_RW_ATTR(reclaim_segments, rec_prefree_segments);
...@@ -969,6 +990,7 @@ SM_INFO_GENERAL_RW_ATTR(min_fsync_blocks); ...@@ -969,6 +990,7 @@ SM_INFO_GENERAL_RW_ATTR(min_fsync_blocks);
SM_INFO_GENERAL_RW_ATTR(min_seq_blocks); SM_INFO_GENERAL_RW_ATTR(min_seq_blocks);
SM_INFO_GENERAL_RW_ATTR(min_hot_blocks); SM_INFO_GENERAL_RW_ATTR(min_hot_blocks);
SM_INFO_GENERAL_RW_ATTR(min_ssr_sections); SM_INFO_GENERAL_RW_ATTR(min_ssr_sections);
SM_INFO_GENERAL_RW_ATTR(reserved_segments);
/* DCC_INFO ATTR */ /* DCC_INFO ATTR */
DCC_INFO_RW_ATTR(max_small_discards, max_discards); DCC_INFO_RW_ATTR(max_small_discards, max_discards);
...@@ -1001,6 +1023,7 @@ F2FS_SBI_RW_ATTR(gc_pin_file_thresh, gc_pin_file_threshold); ...@@ -1001,6 +1023,7 @@ F2FS_SBI_RW_ATTR(gc_pin_file_thresh, gc_pin_file_threshold);
F2FS_SBI_RW_ATTR(gc_reclaimed_segments, gc_reclaimed_segs); F2FS_SBI_RW_ATTR(gc_reclaimed_segments, gc_reclaimed_segs);
F2FS_SBI_GENERAL_RW_ATTR(max_victim_search); F2FS_SBI_GENERAL_RW_ATTR(max_victim_search);
F2FS_SBI_GENERAL_RW_ATTR(migration_granularity); F2FS_SBI_GENERAL_RW_ATTR(migration_granularity);
F2FS_SBI_GENERAL_RW_ATTR(migration_window_granularity);
F2FS_SBI_GENERAL_RW_ATTR(dir_level); F2FS_SBI_GENERAL_RW_ATTR(dir_level);
#ifdef CONFIG_F2FS_IOSTAT #ifdef CONFIG_F2FS_IOSTAT
F2FS_SBI_GENERAL_RW_ATTR(iostat_enable); F2FS_SBI_GENERAL_RW_ATTR(iostat_enable);
...@@ -1033,6 +1056,7 @@ F2FS_SBI_GENERAL_RW_ATTR(warm_data_age_threshold); ...@@ -1033,6 +1056,7 @@ F2FS_SBI_GENERAL_RW_ATTR(warm_data_age_threshold);
F2FS_SBI_GENERAL_RW_ATTR(last_age_weight); F2FS_SBI_GENERAL_RW_ATTR(last_age_weight);
#ifdef CONFIG_BLK_DEV_ZONED #ifdef CONFIG_BLK_DEV_ZONED
F2FS_SBI_GENERAL_RO_ATTR(unusable_blocks_per_sec); F2FS_SBI_GENERAL_RO_ATTR(unusable_blocks_per_sec);
F2FS_SBI_GENERAL_RW_ATTR(blkzone_alloc_policy);
#endif #endif
/* STAT_INFO ATTR */ /* STAT_INFO ATTR */
...@@ -1072,6 +1096,7 @@ F2FS_GENERAL_RO_ATTR(encoding); ...@@ -1072,6 +1096,7 @@ F2FS_GENERAL_RO_ATTR(encoding);
F2FS_GENERAL_RO_ATTR(mounted_time_sec); F2FS_GENERAL_RO_ATTR(mounted_time_sec);
F2FS_GENERAL_RO_ATTR(main_blkaddr); F2FS_GENERAL_RO_ATTR(main_blkaddr);
F2FS_GENERAL_RO_ATTR(pending_discard); F2FS_GENERAL_RO_ATTR(pending_discard);
F2FS_GENERAL_RO_ATTR(atgc_enabled);
F2FS_GENERAL_RO_ATTR(gc_mode); F2FS_GENERAL_RO_ATTR(gc_mode);
#ifdef CONFIG_F2FS_STAT_FS #ifdef CONFIG_F2FS_STAT_FS
F2FS_GENERAL_RO_ATTR(moved_blocks_background); F2FS_GENERAL_RO_ATTR(moved_blocks_background);
...@@ -1116,6 +1141,9 @@ static struct attribute *f2fs_attrs[] = { ...@@ -1116,6 +1141,9 @@ static struct attribute *f2fs_attrs[] = {
ATTR_LIST(gc_min_sleep_time), ATTR_LIST(gc_min_sleep_time),
ATTR_LIST(gc_max_sleep_time), ATTR_LIST(gc_max_sleep_time),
ATTR_LIST(gc_no_gc_sleep_time), ATTR_LIST(gc_no_gc_sleep_time),
ATTR_LIST(gc_no_zoned_gc_percent),
ATTR_LIST(gc_boost_zoned_gc_percent),
ATTR_LIST(gc_valid_thresh_ratio),
ATTR_LIST(gc_idle), ATTR_LIST(gc_idle),
ATTR_LIST(gc_urgent), ATTR_LIST(gc_urgent),
ATTR_LIST(reclaim_segments), ATTR_LIST(reclaim_segments),
...@@ -1138,8 +1166,10 @@ static struct attribute *f2fs_attrs[] = { ...@@ -1138,8 +1166,10 @@ static struct attribute *f2fs_attrs[] = {
ATTR_LIST(min_seq_blocks), ATTR_LIST(min_seq_blocks),
ATTR_LIST(min_hot_blocks), ATTR_LIST(min_hot_blocks),
ATTR_LIST(min_ssr_sections), ATTR_LIST(min_ssr_sections),
ATTR_LIST(reserved_segments),
ATTR_LIST(max_victim_search), ATTR_LIST(max_victim_search),
ATTR_LIST(migration_granularity), ATTR_LIST(migration_granularity),
ATTR_LIST(migration_window_granularity),
ATTR_LIST(dir_level), ATTR_LIST(dir_level),
ATTR_LIST(ram_thresh), ATTR_LIST(ram_thresh),
ATTR_LIST(ra_nid_pages), ATTR_LIST(ra_nid_pages),
...@@ -1187,6 +1217,7 @@ static struct attribute *f2fs_attrs[] = { ...@@ -1187,6 +1217,7 @@ static struct attribute *f2fs_attrs[] = {
#endif #endif
#ifdef CONFIG_BLK_DEV_ZONED #ifdef CONFIG_BLK_DEV_ZONED
ATTR_LIST(unusable_blocks_per_sec), ATTR_LIST(unusable_blocks_per_sec),
ATTR_LIST(blkzone_alloc_policy),
#endif #endif
#ifdef CONFIG_F2FS_FS_COMPRESSION #ifdef CONFIG_F2FS_FS_COMPRESSION
ATTR_LIST(compr_written_block), ATTR_LIST(compr_written_block),
...@@ -1200,6 +1231,7 @@ static struct attribute *f2fs_attrs[] = { ...@@ -1200,6 +1231,7 @@ static struct attribute *f2fs_attrs[] = {
ATTR_LIST(atgc_candidate_count), ATTR_LIST(atgc_candidate_count),
ATTR_LIST(atgc_age_weight), ATTR_LIST(atgc_age_weight),
ATTR_LIST(atgc_age_threshold), ATTR_LIST(atgc_age_threshold),
ATTR_LIST(atgc_enabled),
ATTR_LIST(seq_file_ra_mul), ATTR_LIST(seq_file_ra_mul),
ATTR_LIST(gc_segment_mode), ATTR_LIST(gc_segment_mode),
ATTR_LIST(gc_reclaimed_segments), ATTR_LIST(gc_reclaimed_segments),
......
...@@ -74,7 +74,7 @@ static int pagecache_write(struct inode *inode, const void *buf, size_t count, ...@@ -74,7 +74,7 @@ static int pagecache_write(struct inode *inode, const void *buf, size_t count,
struct address_space *mapping = inode->i_mapping; struct address_space *mapping = inode->i_mapping;
const struct address_space_operations *aops = mapping->a_ops; const struct address_space_operations *aops = mapping->a_ops;
if (pos + count > inode->i_sb->s_maxbytes) if (pos + count > F2FS_BLK_TO_BYTES(max_file_blocks(inode)))
return -EFBIG; return -EFBIG;
while (count) { while (count) {
...@@ -237,7 +237,8 @@ static int f2fs_get_verity_descriptor(struct inode *inode, void *buf, ...@@ -237,7 +237,8 @@ static int f2fs_get_verity_descriptor(struct inode *inode, void *buf,
pos = le64_to_cpu(dloc.pos); pos = le64_to_cpu(dloc.pos);
/* Get the descriptor */ /* Get the descriptor */
if (pos + size < pos || pos + size > inode->i_sb->s_maxbytes || if (pos + size < pos ||
pos + size > F2FS_BLK_TO_BYTES(max_file_blocks(inode)) ||
pos < f2fs_verity_metadata_pos(inode) || size > INT_MAX) { pos < f2fs_verity_metadata_pos(inode) || size > INT_MAX) {
f2fs_warn(F2FS_I_SB(inode), "invalid verity xattr"); f2fs_warn(F2FS_I_SB(inode), "invalid verity xattr");
f2fs_handle_error(F2FS_I_SB(inode), f2fs_handle_error(F2FS_I_SB(inode),
......
...@@ -629,6 +629,7 @@ static int __f2fs_setxattr(struct inode *inode, int index, ...@@ -629,6 +629,7 @@ static int __f2fs_setxattr(struct inode *inode, int index,
const char *name, const void *value, size_t size, const char *name, const void *value, size_t size,
struct page *ipage, int flags) struct page *ipage, int flags)
{ {
struct f2fs_sb_info *sbi = F2FS_I_SB(inode);
struct f2fs_xattr_entry *here, *last; struct f2fs_xattr_entry *here, *last;
void *base_addr, *last_base_addr; void *base_addr, *last_base_addr;
int found, newsize; int found, newsize;
...@@ -772,9 +773,18 @@ static int __f2fs_setxattr(struct inode *inode, int index, ...@@ -772,9 +773,18 @@ static int __f2fs_setxattr(struct inode *inode, int index,
if (index == F2FS_XATTR_INDEX_ENCRYPTION && if (index == F2FS_XATTR_INDEX_ENCRYPTION &&
!strcmp(name, F2FS_XATTR_NAME_ENCRYPTION_CONTEXT)) !strcmp(name, F2FS_XATTR_NAME_ENCRYPTION_CONTEXT))
f2fs_set_encrypted_inode(inode); f2fs_set_encrypted_inode(inode);
if (S_ISDIR(inode->i_mode))
set_sbi_flag(F2FS_I_SB(inode), SBI_NEED_CP);
if (!S_ISDIR(inode->i_mode))
goto same;
/*
* In restrict mode, fsync() always try to trigger checkpoint for all
* metadata consistency, in other mode, it triggers checkpoint when
* parent's xattr metadata was updated.
*/
if (F2FS_OPTION(sbi).fsync_mode == FSYNC_MODE_STRICT)
set_sbi_flag(sbi, SBI_NEED_CP);
else
f2fs_add_ino_entry(sbi, inode->i_ino, XATTR_DIR_INO);
same: same:
if (is_inode_flag_set(inode, FI_ACL_MODE)) { if (is_inode_flag_set(inode, FI_ACL_MODE)) {
inode->i_mode = F2FS_I(inode)->i_acl_mode; inode->i_mode = F2FS_I(inode)->i_acl_mode;
......
...@@ -19,7 +19,6 @@ ...@@ -19,7 +19,6 @@
#define F2FS_BLKSIZE_BITS PAGE_SHIFT /* bits for F2FS_BLKSIZE */ #define F2FS_BLKSIZE_BITS PAGE_SHIFT /* bits for F2FS_BLKSIZE */
#define F2FS_MAX_EXTENSION 64 /* # of extension entries */ #define F2FS_MAX_EXTENSION 64 /* # of extension entries */
#define F2FS_EXTENSION_LEN 8 /* max size of extension */ #define F2FS_EXTENSION_LEN 8 /* max size of extension */
#define F2FS_BLK_ALIGN(x) (((x) + F2FS_BLKSIZE - 1) >> F2FS_BLKSIZE_BITS)
#define NULL_ADDR ((block_t)0) /* used as block_t addresses */ #define NULL_ADDR ((block_t)0) /* used as block_t addresses */
#define NEW_ADDR ((block_t)-1) /* used as block_t addresses */ #define NEW_ADDR ((block_t)-1) /* used as block_t addresses */
...@@ -28,6 +27,7 @@ ...@@ -28,6 +27,7 @@
#define F2FS_BYTES_TO_BLK(bytes) ((bytes) >> F2FS_BLKSIZE_BITS) #define F2FS_BYTES_TO_BLK(bytes) ((bytes) >> F2FS_BLKSIZE_BITS)
#define F2FS_BLK_TO_BYTES(blk) ((blk) << F2FS_BLKSIZE_BITS) #define F2FS_BLK_TO_BYTES(blk) ((blk) << F2FS_BLKSIZE_BITS)
#define F2FS_BLK_END_BYTES(blk) (F2FS_BLK_TO_BYTES(blk + 1) - 1) #define F2FS_BLK_END_BYTES(blk) (F2FS_BLK_TO_BYTES(blk + 1) - 1)
#define F2FS_BLK_ALIGN(x) (F2FS_BYTES_TO_BLK((x) + F2FS_BLKSIZE - 1))
/* 0, 1(node nid), 2(meta nid) are reserved node id */ /* 0, 1(node nid), 2(meta nid) are reserved node id */
#define F2FS_RESERVED_NODE_NUM 3 #define F2FS_RESERVED_NODE_NUM 3
...@@ -278,7 +278,7 @@ struct node_footer { ...@@ -278,7 +278,7 @@ struct node_footer {
#define F2FS_INLINE_DATA 0x02 /* file inline data flag */ #define F2FS_INLINE_DATA 0x02 /* file inline data flag */
#define F2FS_INLINE_DENTRY 0x04 /* file inline dentry flag */ #define F2FS_INLINE_DENTRY 0x04 /* file inline dentry flag */
#define F2FS_DATA_EXIST 0x08 /* file inline data exist flag */ #define F2FS_DATA_EXIST 0x08 /* file inline data exist flag */
#define F2FS_INLINE_DOTS 0x10 /* file having implicit dot dentries */ #define F2FS_INLINE_DOTS 0x10 /* file having implicit dot dentries (obsolete) */
#define F2FS_EXTRA_ATTR 0x20 /* file having extra attribute */ #define F2FS_EXTRA_ATTR 0x20 /* file having extra attribute */
#define F2FS_PIN_FILE 0x40 /* file should not be gced */ #define F2FS_PIN_FILE 0x40 /* file should not be gced */
#define F2FS_COMPRESS_RELEASED 0x80 /* file released compressed blocks */ #define F2FS_COMPRESS_RELEASED 0x80 /* file released compressed blocks */
......
...@@ -139,7 +139,8 @@ TRACE_DEFINE_ENUM(EX_BLOCK_AGE); ...@@ -139,7 +139,8 @@ TRACE_DEFINE_ENUM(EX_BLOCK_AGE);
{ CP_NODE_NEED_CP, "node needs cp" }, \ { CP_NODE_NEED_CP, "node needs cp" }, \
{ CP_FASTBOOT_MODE, "fastboot mode" }, \ { CP_FASTBOOT_MODE, "fastboot mode" }, \
{ CP_SPEC_LOG_NUM, "log type is 2" }, \ { CP_SPEC_LOG_NUM, "log type is 2" }, \
{ CP_RECOVER_DIR, "dir needs recovery" }) { CP_RECOVER_DIR, "dir needs recovery" }, \
{ CP_XATTR_DIR, "dir's xattr updated" })
#define show_shutdown_mode(type) \ #define show_shutdown_mode(type) \
__print_symbolic(type, \ __print_symbolic(type, \
......
Markdown is supported
0%
or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment