The 'space_fixup' flag can be set in the superblock of a new filesystem by
mkfs.ubifs to indicate that any eraseblocks with free space remaining should be
fixed-up the first time it's mounted (after which the flag is un-set). This
means that the UBIFS image has been flashed by a "dumb" flasher and the free
space has been actually programmed (writing all 0xFFs), so this free space
cannot be used. UBIFS fixes the free space up by re-writing the contents of all
LEBs with free space using the atomic LEB change UBI operation.

Artem: improved commit message, add some more commentaries to the code.
Signed-off-by: Matthew L. Creech <mlcreech@gmail.com>
Signed-off-by: Artem Bityutskiy <Artem.Bityutskiy@nokia.com>

We'll need to use the 'next_log_lnum()' helper function from log.c in the fixup
code, so let's move it to misc.h. IOW, this is a preparation to the following
free space fixup changes.
Signed-off-by: Artem Bityutskiy <Artem.Bityutskiy@nokia.com>

This patch improves UBIFS recovery and teaches it to expect corruption only
in the last buds. Indeed, currently we just recover all buds, which is
incorrect because only the last buds can have corruptions in case of a power
cut. So it is inconsistent with the rest of the recovery strategy which tries
hard to distinguish between corruptions cause by power cuts and other types of
corruptions.

This patch also adds one quirk - a bit older UBIFS was could have corruption in
the next to last bud because of the way it switched buds: when bud A is full,
it first searched for the next bud B, the wrote a reference node to the log
about B, and then synchronized the write-buffer of A. So we could end up with
buds A and B, where B is the last, but A had corruption. The UBIFS behavior
was fixed, though, so currently it always first synchronizes A's write-buffer
and only after this adds B to the log. However, to be make sure that we handle
unclean (after a power cut) UBIFS images belonging to older UBIFS - we need to
add a quirk and keep it for some time: we need to check for the situation
described above.

Thankfully, it is easy to check for that situation. When UBIFS adds B to the
log, it always first unmaps B, then maps it, and then syncs A's write-buffer.
Thus, in that situation we can check that B is empty, in which case it is OK to
have corruption in A. To check that B is empty it is enough to just read the
first few bytes of the bud and compare them with 0xFFs. This quirk may be
removed in a couple of years.
Signed-off-by: Artem Bityutskiy <Artem.Bityutskiy@nokia.com>

Currently when UBIFS fills up the current bud (which is the last in the journal
head) and switches to the next bud, it first writes the log reference node for
the next bud and only after this synchronizes the write-buffer of the previous
bud. This is not a big deal, but an unclean power cut may lead to a situation
when we have corruption in a next-to-last bud, although it is much more logical
that we have to have corruption only in the last bud.

This patch also removes write-buffer synchronization from
'ubifs_wbuf_seek_nolock()' because this is not needed anymore (we synchronize
the write-buffer explicitly everywhere now) and also because this is just
prone to various errors.
Signed-off-by: Artem Bityutskiy <Artem.Bityutskiy@nokia.com>

Remove a 'BUG()' statement when we are unable to find a bud and add a
similar 'ubifs_assert()' statement instead.
Signed-off-by: Artem Bityutskiy <Artem.Bityutskiy@nokia.com>

This is a minor preparation patch which changes 'replay_bud()' interface -
instead of passing bud lnum, offs, jhead, etc directly, pass a pointer to the
bud entry which contains all the information. The bud entry will be also needed
in one of the following patches.
Signed-off-by: Artem Bityutskiy <Artem.Bityutskiy@nokia.com>

This patch simplifies replay even further - it removes the replay tree and
adds the replay list instead. Indeed, we just do not need to use a tree here -
all we need to do is to add all nodes to the list and then sort it. Using
RB-tree is an overkill - more code and slower. And since we replay buds in
order, we expect the nodes to follow in _mostly_ sorted order, so the merge
sort becomes much cheaper in average than an RB-tree.
Signed-off-by: Artem Bityutskiy <Artem.Bityutskiy@nokia.com>

This patch simplifies the replay code and makes it smaller. First of all, we
can notice that we do not really need to create bud replay entries and insert
them to the replay tree, because the only reason we do this is to set buds
lprops correctly at the end. Instead, we can just walk the list of buds at the
very end and set lprops for each bud. This allows us to get rid of whole
'insert_ref_node()' function, the 'REPLAY_REF' flag, and several fields in
'struct replay_entry'. Then we can also notice that we do not need the 'flags'
'struct replay_entry' field, because there is only one flag -
'REPLAY_DELETION'. Instead, we can just add a 'deletion' bit fields. As a
result, this patch deletes much more lines that in adds.
Signed-off-by: Artem Bityutskiy <Artem.Bityutskiy@nokia.com>

This is just a small preparation patch which adds 'free' and 'drity' fields to
'struct bud_entry'. They will be used to set bud lprops.
Signed-off-by: Artem Bityutskiy <Artem.Bityutskiy@nokia.com>

This is patch removes an unnecessary 'offs' variable from 'ubifs_wbuf_write_nolock()'
- we can just keep 'wbuf->offs' up-to-date instead. This patch is very minor
the only motivation for it was that it is cleaner to keep wbuf->offs up-to-date
by the time we call 'ubifs_leb_write()'.
Signed-off-by: Artem Bityutskiy <Artem.Bityutskiy@nokia.com>

Commit 52c6e6f9 provides misleading infomation
in the commit messages - buds are replied in order. And the real reason why
that fix helped is probably because it made sure we seek head even in read-only
mode (so deferred recovery will have seeked heads).

This patch adds an assertion which will fire if we reply buds out of order.
Signed-off-by: Artem Bityutskiy <Artem.Bityutskiy@nokia.com>

This is a minor change which makes 2 functions static because they
are not used outside the gc.c file: 'data_nodes_cmp()' and
'nondata_nodes_cmp()'.
Signed-off-by: Artem Bityutskiy <Artem.Bityutskiy@nokia.com>

This is a tiny clean-up patch which improves replay commentaries.
Signed-off-by: Artem Bityutskiy <Artem.Bityutskiy@nokia.com>

Print a bit more information is some recovery and replay paths.
Signed-off-by: Artem Bityutskiy <Artem.Bityutskiy@nokia.com>

Signed-off-by: Artem Bityutskiy <Artem.Bityutskiy@nokia.com>

Now we return all errors from 'scan_check_cb()' directly, so we do not need
'struct scan_check_data' any more, and this patch removes it.
Signed-off-by: Artem Bityutskiy <Artem.Bityutskiy@nokia.com>

Simplify error path in 'scan_check_cb()' and stop using the special 'data->err'
field, but instead return the error code directly.
Signed-off-by: Artem Bityutskiy <Artem.Bityutskiy@nokia.com>

When doing the lprops extra check ('dbg_check_lprops()') we scan whole media.
We even scan empty and freeable LEBs which may contain garbage, which we handle
after scanning. This patch teach the lprops checking function
('scan_check_cb()') to avoid scanning for free and freeable LEBs and save time.
Signed-off-by: Artem Bityutskiy <Artem.Bityutskiy@nokia.com>

When re-mounting from R/O mode to R/W mode and the LEB count in the superblock
is not up-to date, because for the underlying UBI volume became larger, we
re-write the superblock. We allocate RAM for these purposes, but never free it.
So this is a memory leak, although very rare one.
Signed-off-by: Artem Bityutskiy <Artem.Bityutskiy@nokia.com>
Cc: stable@kernel.org

This patch fixes a problem with the following symptoms:

UBIFS: deferred recovery completed
UBIFS error (pid 15676): dbg_check_synced_i_size: ui_size is 11481088, synced_i_size is 11459081, but inode is clean
UBIFS error (pid 15676): dbg_check_synced_i_size: i_ino 128, i_mode 0x81a4, i_size 11481088

It happens when additional debugging checks are enabled and we are recovering
from a power cut. When we fixup corrupted inode size during recovery, we change
them in-place and we change ui_size as well, but not synced_i_size, which
causes this failure. This patch makes sure we change both fields and fixes the
issue.
Signed-off-by: Artem Bityutskiy <Artem.Bityutskiy@nokia.com>

When the debugging self-checks are enabled, we go trough whole file-system
after mount and check/validate every single node referred to by the index.
This is implemented by the 'dbg_check_filesystem()' function. However, this
function fails if we mount "unclean" file-system, i.e., if we mount the
file-system after a power cut. It fails with the following symptoms:

UBIFS DBG (pid 8171): ubifs_recover_size: ino 937 size 3309925 -> 3317760
UBIFS: recovery deferred
UBIFS error (pid 8171): check_leaf: data node at LEB 1000:0 is not within inode size 3309925

The reason of failure is that recovery fixed up the inode size in memory, but
not on the flash so far. So the value on the flash is incorrect so far,
and would be corrected when we re-mount R/W. But 'check_leaf()' ignores
this fact and tries to validate the size of the on-flash inode, which is
incorrect, so it fails.

This patch teaches the checking code to look at the VFS inode cache first,
and if there is the inode in question, use that inode instead of the inode
on the flash media. This fixes the issue.
Signed-off-by: Artem Bityutskiy <Artem.Bityutskiy@nokia.com>

In 'ubifs_recover_size()' we have an "if (!e->inode && c->ro_mount)" statement.
But if 'c->ro_mount' is true, then '!e->inode' must always be true as well. So
we can remove the unnecessary '!e->inode' test and put an
'ubifs_assert(!e->inode)' instead.

This patch also removes an extra trailing white-space in a debugging print,
as well as adds few empty lines to 'ubifs_recover_size()' to make it a bit more
readable.
Signed-off-by: Artem Bityutskiy <Artem.Bityutskiy@nokia.com>

When recovering the inode size, one of the debugging messages was printed
incorrecly, this patches fixes it.
Signed-off-by: Artem Bityutskiy <Artem.Bityutskiy@nokia.com>

This commits refactors and cleans up 'ubifs_rcvry_gc_commit()' which was quite
untidy, also removes the commentary which was not 100% correct.
Signed-off-by: Artem Bityutskiy <Artem.Bityutskiy@nokia.com>

Split the 'ubifs_rcvry_gc_commit()' function and introduce a 'grab_empty_leb()'
heler. This cleans 'ubifs_rcvry_gc_commit()' a little and makes it a bit less
of spagetti.

Also, add a commentary which explains why it is crucial to first search for an
empty LEB and then run commit.
Signed-off-by: Artem Bityutskiy <Artem.Bityutskiy@nokia.com>

When UBIFS is in the failure mode (used for power cut emulation testing) we for
some reasons do not dump the stack in many places, e.g., in assertions.
Probably at early days we had too many of them and disabled this to make the
development easier, but then never enabled. Nowadays I sometimes observe
assertion failures during power cut testing, but the useful stackdump is not
printed, which is bad. This patch makes UBIFS always print the stackdump when
debugging is enabled.
Signed-off-by: Artem Bityutskiy <Artem.Bityutskiy@nokia.com>

If we fail to recover the gc_lnum we just return an error and it then
it is difficult to figure out why this happened. This patch adds useful
debugging information which should make it easier to debug the failure.
Signed-off-by: Artem Bityutskiy <Artem.Bityutskiy@nokia.com>

This patch removes a piece of code in 'ubifs_rcvry_gc_commit()' which is never
executed. We call 'ubifs_find_dirty_leb()' function with min_space =
wbuf->offs, so if it returns us an LEB, it is guaranteed to have at lease
'wbuf->offs' bytes of free+dirty space. So we can remove the subsequent code
which deals with "returned LEB has less than 'wbuf->offs' bytes of free+dirty
space". This simplifies 'ubifs_rcvry_gc_commit()' a little.
Signed-off-by: Artem Bityutskiy <Artem.Bityutskiy@nokia.com>

We have duplicated code in 'ubifs_garbage_collect()' and
'ubifs_rcvry_gc_commit()', which is about handling the special case of free
LEB. In both cases we just want to garbage-collect the LEB using
'ubifs_garbage_collect_leb()'.

This patch teaches 'ubifs_garbage_collect_leb()' to handle free LEB's so that
the caller does not have to do this and the duplicated code is removed.
Signed-off-by: Artem Bityutskiy <Artem.Bityutskiy@nokia.com>

Remove the following commentary from 'ubifs_file_mmap()':

/* 'generic_file_mmap()' takes care of NOMMU case */

I do not understand what it means, and I could not find anything relater to
NOMMU in 'generic_file_mmap()'.
Signed-off-by: Artem Bityutskiy <Artem.Bityutskiy@nokia.com>

This patch is a tiny improvement which removes few bytes of code.
UBIFS debugfs files are non-seekable and the file position is ignored,
so do not increase it in the write handler.
Signed-off-by: Artem Bityutskiy <Artem.Bityutskiy@nokia.com>

The 'dbg_dump_lprop()' is trying to detect journal head LEBs when printing,
so it looks at the write-buffers. However, if we are in R/O mode, we
de-allocate the write-buffers, so 'dbg_dump_lprop()' oopses. This patch fixes
the issue.

Note, this patch is not critical, it is only about the debugging code path, and
it is unlikely that anyone but UBIFS developers would ever hit this issue.
Signed-off-by: Artem Bityutskiy <Artem.Bityutskiy@nokia.com>

We have our own flags indicating R/O mode, and c->ro_mode is equivalent
to MS_RDONLY. Let's be consistent and use UBIFS flags everywhere.
This patch is just a minor cleanup.

Additionally, add a comment that we are surprised with VFS behavior -
as a reminder to look at this some day.
Signed-off-by: Artem Bityutskiy <Artem.Bityutskiy@nokia.com>

When the debugging failure emulation is enabled and UBIFS decides to
emulate an I/O error, it uses EIO error code. In which case UBIFS
switches into R/O mode later on. The for the user-space is that when
a failure is emulated, the file-system sometimes returns EIO and
sometimes EROFS. This makes it more difficult to implement user-space
tests for the failure mode. Let's be consistent and return EROFS in
all the cases.

This patch is an improvement for the debugging code and does not affect
the functionality at all.
Signed-off-by: Artem Bityutskiy <Artem.Bityutskiy@nokia.com>

This is just a tiny clean-up patch. The variable name for empty address
space operations is "empty_aops". Let's use consistent names for empty
inode and file operations: "empty_iops" and "empty_fops", instead of
inconsistent "none_inode_operations" and "none_file_operations".

Artem: re-write the commit message.
Signed-off-by: Sedat Dilek <sedat.dilek@gmail.com>
Signed-off-by: Artem Bityutskiy <Artem.Bityutskiy@nokia.com>

Try to improve UBIFS testing coverage by randomly picking LEBs to
store in lsave, rather than picking them optimally. Create a debugging
version of 'populate_lsave()' for these purposes and enable it when
general debugging self-checks are enabled.
Signed-off-by: Artem Bityutskiy <Artem.Bityutskiy@nokia.com>

UBIFS can force itself to use the 'in-the-gaps' commit method - the last resort
method which is normally invoced very very rarely. Currently this "force
int-the-gaps" debugging feature is a separate test mode. But it is a bit saner
to make it to be the "general" self-test check instead.

This patch is just a clean-up which should make the debugging code look a bit
nicer and easier to use - we have way too many debugging options.
Signed-off-by: Artem Bityutskiy <Artem.Bityutskiy@nokia.com>

This patch improves the 'dbg_check_space_info()' function which checks
whether the amount of space before re-mounting and after re-mounting
is the same (remounting from R/O to R/W modes and vice-versa).

The problem is that 'dbg_check_space_info()' does not save the budgeting
information before re-mounting, so when an error is reported, we do not
know why the amount of free space changed.

This patches makes the following changes:

1. Teaches 'dbg_dump_budg()' function to accept a 'struct ubifs_budg_info'
   argument and print out the this argument. This way we may ask it to
   print any saved budgeting info, no only the current one.
2. Accordingly changes all the callers of 'dbg_dump_budg()' to comply with
   the changed interface.
3. Introduce a 'saved_bi' (saved budgeting info) field to
   'struct ubifs_debug_info' and save the budgeting info before re-mounting
   there.
4. Change 'dbg_check_space_info()' and make it print both old and new
   budgeting information.
5. Additionally, save 'c->igx_gc_cnt' and print it if and error happens. This
   value contributes to the amount of free space, so we have to print it.
Signed-off-by: Artem Bityutskiy <Artem.Bityutskiy@nokia.com>

Re-arrange the budget dump and make sure we first dump all
the 'struct ubifs_budg_info' fields, and then the other information.
Additionally, print the 'uncommitted_idx' variable.

This change is required for to the following dumping function
enhancement where it will be possible to dump saved
'struct ubifs_budg_info' objects, not only the current one.
Signed-off-by: Artem Bityutskiy <Artem.Bityutskiy@nokia.com>

The current 'dbg_dump_budg()' calling convention is that the
'c->space_lock' spinlock is held. However, none of the callers
actually use it from contects which have 'c->space_lock' locked,
so all callers have to explicitely lock and unlock the spinlock.
This is not very sensible convention. This patch changes it and
makes 'dbg_dump_budg()' lock the spinlock instead of imposing this
to the callers. This simplifies the code a little.
Signed-off-by: Artem Bityutskiy <Artem.Bityutskiy@nokia.com>