This patch adds support for resizing file systems with the meta_bg and
64bit features.

[ Added a fix by tytso to fix a divide by zero when resizing a
  filesystem from 14 TB to 18TB.  Also fixed overhead accounting for
  meta_bg file systems.]
Signed-off-by: Yongqiang Yang <xiaoqiangnk@gmail.com>
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>

Previously we allocated the s_group_info array with enough space for
any future possible growth of the file system via online resize.  This
is unfortunate because it wastes memory, and it doesn't work for the
meta_bg scheme, since there is no limit based on the number of
reserved gdt blocks.  So add the code to grow the s_group_info array
as needed.
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>

Previously, we allocated the s_flex_groups array to the maximum size
that the file system could be resized.  There was two problems with
this approach.  First, it wasted memory in the common case where the
file system was not resized.  Secondly, once we start allowing online
resizing using the meta_bg scheme, there is no maximum size that the
file system can be resized.  So instead, we need to grow the
s_flex_groups at inline resize time.
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>

The resize code was needlessly writing the backup block group
descriptor blocks multiple times (once per block group) during an
online resize.
Signed-off-by: Yongqiang Yang <xiaoqiangnk@gmail.com>
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Cc: stable@vger.kernel.org

The resize code was copying blocks at the beginning of each block
group in order to copy the superblock and block group descriptor table
(gdt) blocks.  This was, unfortunately, being done even for block
groups that did not have super blocks or gdt blocks.  This is a
complete waste of perfectly good I/O bandwidth, to skip writing those
blocks for sparse bg's.
Signed-off-by: Yongqiang Yang <xiaoqiangnk@gmail.com>
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Cc: stable@vger.kernel.org

Avoid changing o_blocks_count, since it is used later when reporting
old blocks count in debug mode.
Signed-off-by: Yongqiang Yang <xiaoqiangnk@gmail.com>
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>

If the last group does not have enough space for group tables, ignore
it instead of calling BUG_ON().
Reported-by: Daniel Drake <dsd@laptop.org>
Signed-off-by: Yongqiang Yang <xiaoqiangnk@gmail.com>
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Cc: stable@vger.kernel.org

In patch cb20d518, ext4_set_bh_endio
and ext4_end_io_buffer_write are declared at the beginning of inode.c,
and again later on in the middle of the file.  Remove the second set
of duplicated function declarations.
Signed-off-by: Zheng Liu <wenqing.lz@taobao.com>
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>

Signed-off-by: Wang Sheng-Hui <shhuiw@gmail.com>
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>

While performing punch hole for an inode, i_disksize is not changed.
So, there is no need to add the inode to orphan list.
Signed-off-by: Ashish Sangwan <ashish.sangwan2@gmail.com>
Signed-off-by: Namjae Jeon <linkinjeon@gmail.com>
Acked-by: Zheng Liu <wenqing.lz@taobao.com>
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>

This sequence:

# truncate --size=1g fsfile
# mkfs.ext4 -F fsfile
# mount -o loop,ro fsfile /mnt
# umount /mnt
# dmesg | tail

results in an IO error when unmounting the RO filesystem:

[  318.020828] Buffer I/O error on device loop1, logical block 196608
[  318.027024] lost page write due to I/O error on loop1
[  318.032088] JBD2: Error -5 detected when updating journal superblock for loop1-8.

This was a regression introduced by commit 24bcc89c: "jbd2: split
updating of journal superblock and marking journal empty".
Signed-off-by: Eric Sandeen <sandeen@redhat.com>
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Cc: stable@vger.kernel.org

Fixes the following sparse warning:
fs/ext4/super.c:1672:45: warning: Using plain integer as NULL pointer
Signed-off-by: Sachin Kamat <sachin.kamat@linaro.org>
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>

We don't need lock_super()/unlock_super() any more, since the places
where it is used, we are protected by the s_umount r/w semaphore.
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Cc: Marco Stornelli <marco.stornelli@gmail.com>

In the very unlikely case that kset_create_and_add() fails when the
ext4.ko module is being loaded (or during kernel startup) set err so
that it's clear that the module load failed.

https://bugzilla.kernel.org/show_bug.cgi?id=27912Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>

All the routines call mb_find_extent are setting argument 'order' to 0
just like:

	mb_find_extent(e4b, 0, ex.fe_start, ex.fe_len, &ex);

therefore the useless argument should be removed.
Signed-off-by: Robin Dong <sanbai@taobao.com>
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>

Signed-off-by: Robin Dong <sanbai@taobao.com>
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>

    
blkdev_issue_flush() can fail; make sure the error gets properly
propagated.
    
This is a port of the equivalent ext3 patch from commit 44f4f729.
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>

blkdev_issue_flush() can fail; make sure the error gets properly
propagated.

This is a port of the equivalent jbd patch from commit 349ecd6a.
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>

Currently in ext4 the length of zero-out chunk is set to 7 file system
blocks.  But if an inode has uninitailized extents from using
fallocate to preallocate space, and the workload issues many random
writes, this can cause a fragmented extent tree that will
unnecessarily grow the extent tree.

So create a new sysfs tunable, extent_max_zeroout_kb, which controls
the maximum size where blocks will be zeroed out instead of creating a
new uninitialized extent.  The default of this has been sent to 32kb.

CC: Zach Brown <zab@zabbo.net>
CC: Andreas Dilger <adilger@dilger.ca>
Signed-off-by: Zheng Liu <wenqing.lz@taobao.com>
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>

Signed-off-by: Anatol Pomozov <anatol.pomozov@gmail.com>
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>

Most hardware architectures require that data (including struct fields)
have to be aligned in memory. To make it happen compiler inserts padding
between struct fields if they are not aligned correctly.

Reorder fields to remove paddings and make structures denser. Making data
smaller saves some memory that is very important for trace events.
Tracing buffer has limited size and making objects smaller we can put more
of them without overflowing the tracing buffer.

To find data struct holes I used 'pahole -H 1 -E -I vmlinux.o' from
'dwarves' package.
Signed-off-by: Anatol Pomozov <anatol.pomozov@gmail.com>
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>

Very large directories can cause significant performance problems, or
perhaps even invoke the OOM killer, if the process is running in a
highly constrained memory environment (whether it is VM's with a small
amount of memory or in a small memory cgroup).

So it is useful, in cloud server/data center environments, to be able
to set a filesystem-wide cap on the maximum size of a directory, to
ensure that directories never get larger than a sane size.  We do this
via a new mount option, max_dir_size_kb.  If there is an attempt to
grow the directory larger than max_dir_size_kb, the system call will
return ENOSPC instead.

Google-Bug-Id: 6863013
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>

Add a short circuit check to ext4_mb_group_group() so that we don't
bother to load the block bitmap for a block group which does not have
any space available.  (Or which does not have enough space until we
are in desperation mode, i.e., when cr == 3.)

Resolves-bug: https://bugzilla.kernel.org/show_bug.cgi?id=45741
Reported-by: mirek@me.com
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>

If an inode has more than 4 extents, but then later some of the
extents are merged together, we can optimize the file system by moving
the extents up into the inode, and discarding the extent tree block.
This is important, because if there are a large number of inodes with
an external extent tree blocks where the contents could fit in the
inode, this can significantly increase the fsck time of the file
system.

Google-Bug-Id: 6801242
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>

Commit 968dee77: "ext4: fix hole punch failure when depth is greater
than 0" introduced a regression in v3.5.1/v3.6-rc1 which caused kernel
crashes when users ran run "rm -rf" on large directory hierarchy on
ext4 filesystems on RAID devices:

    BUG: unable to handle kernel NULL pointer dereference at 0000000000000028

    Process rm (pid: 18229, threadinfo ffff8801276bc000, task ffff880123631710)
    Call Trace:
     [<ffffffff81236483>] ? __ext4_handle_dirty_metadata+0x83/0x110
     [<ffffffff812353d3>] ext4_ext_truncate+0x193/0x1d0
     [<ffffffff8120a8cf>] ? ext4_mark_inode_dirty+0x7f/0x1f0
     [<ffffffff81207e05>] ext4_truncate+0xf5/0x100
     [<ffffffff8120cd51>] ext4_evict_inode+0x461/0x490
     [<ffffffff811a1312>] evict+0xa2/0x1a0
     [<ffffffff811a1513>] iput+0x103/0x1f0
     [<ffffffff81196d84>] do_unlinkat+0x154/0x1c0
     [<ffffffff8118cc3a>] ? sys_newfstatat+0x2a/0x40
     [<ffffffff81197b0b>] sys_unlinkat+0x1b/0x50
     [<ffffffff816135e9>] system_call_fastpath+0x16/0x1b
    Code: 8b 4d 20 0f b7 41 02 48 8d 04 40 48 8d 04 81 49 89 45 18 0f b7 49 02 48 83 c1 01 49 89 4d 00 e9 ae f8 ff ff 0f 1f 00 49 8b 45 28 <48> 8b 40 28 49 89 45 20 e9 85 f8 ff ff 0f 1f 80 00 00 00

    RIP  [<ffffffff81233164>] ext4_ext_remove_space+0xa34/0xdf0

This could be reproduced as follows:

The problem in commit 968dee77 was that caused the variable 'i' to
be left uninitialized if the truncate required more space than was
available in the journal.  This resulted in the function
ext4_ext_truncate_extend_restart() returning -EAGAIN, which caused
ext4_ext_remove_space() to restart the truncate operation after
starting a new jbd2 handle.
Reported-by: Maciej Żenczykowski <maze@google.com>
Reported-by: Marti Raudsepp <marti@juffo.org>
Tested-by: Fengguang Wu <fengguang.wu@intel.com>
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Cc: stable@vger.kernel.org

Commit 8aeb00ff: "ext4: fix overhead calculation used by
ext4_statfs()" introduced a O(n**2) calculation which makes very large
file systems take forever to mount.  Fix this with an optimization for
non-bigalloc file systems.  (For bigalloc file systems the overhead
needs to be set in the the superblock.)
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Cc: stable@vger.kernel.org

While in ext4_validate_block_bitmap(), if an block allocation bitmap
is found to be invalid, we call ext4_error() while the block group is
still locked.  This causes ext4_commit_super() to call a function
which might sleep while in an atomic context.

There's no need to keep the block group locked at this point, so hoist
the ext4_error() call up to ext4_validate_block_bitmap() and release
the block group spinlock before calling ext4_error().

The reported stack trace can be found at:

	http://article.gmane.org/gmane.comp.file-systems.ext4/33731Reported-by: Dave Jones <davej@redhat.com>
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Cc: stable@vger.kernel.org

Commit 03179fe9 introduced a kmemcheck complaint in
ext4_da_get_block_prep() because we save and restore
ei->i_da_metadata_calc_last_lblock even though it is left
uninitialized in the case where i_da_metadata_calc_len is zero.

This doesn't hurt anything, but silencing the kmemcheck complaint
makes it easier for people to find real bugs.

Addresses https://bugzilla.kernel.org/show_bug.cgi?id=45631
(which is marked as a regression).
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Cc: stable@vger.kernel.org

After we transfer set the EXT4_ERROR_FS bit in the file system
superblock, it's not enough to call jbd2_journal_clear_err() to clear
the error indication from journal superblock --- we need to call
jbd2_journal_update_sb_errno() as well.  Otherwise, when the root file
system is mounted read-only, the journal is replayed, and the error
indicator is transferred to the superblock --- but the s_errno field
in the jbd2 superblock is left set (since although we cleared it in
memory, we never flushed it out to disk).

This can end up confusing e2fsck.  We should make e2fsck more robust
in this case, but the kernel shouldn't be leaving things in this
confused state, either.
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Cc: stable@kernel.org

Pull OLPC platform updates from Andres Salomon:
 "These move the OLPC Embedded Controller driver out of
  arch/x86/platform and into drivers/platform/olpc.

  OLPC machines are now ARM-based (which means lots of x86 and ARM
  changes), but are typically pretty self-contained..  so it makes more
  sense to go through a separate OLPC tree after getting the appropriate
  review/ACKs."

* 'for-linus-3.6' of git://dev.laptop.org/users/dilinger/linux-olpc:
  x86: OLPC: move s/r-related EC cmds to EC driver
  Platform: OLPC: move global variables into priv struct
  Platform: OLPC: move debugfs support from x86 EC driver
  x86: OLPC: switch over to using new EC driver on x86
  Platform: OLPC: add a suspended flag to the EC driver
  Platform: OLPC: turn EC driver into a platform_driver
  Platform: OLPC: allow EC cmd to be overridden, and create a workqueue to call it
  drivers: OLPC: update various drivers to include olpc-ec.h
  Platform: OLPC: add a stub to drivers/platform/ for the OLPC EC driver

Pull arm-soc Marvell Orion device-tree updates from Olof Johansson:
 "This contains a set of device-tree conversions for Marvell Orion
  platforms that were staged early but took a few tries to get the
  branch into a format where it was suitable for us to pick up.

  Given that most people working on these platforms are hobbyists with
  limited time, we were a bit more flexible with merging it even though
  it came in late."

* tag 'dt2' of git://git.kernel.org/pub/scm/linux/kernel/git/arm/arm-soc: (21 commits)
  ARM: Kirkwood: Replace mrvl with marvell
  ARM: Kirkwood: Describe GoFlex Net LEDs and SATA in DT.
  ARM: Kirkwood: Describe Dreamplug LEDs in DT.
  ARM: Kirkwood: Describe iConnects LEDs in DT.
  ARM: Kirkwood: Describe iConnects temperature sensor in DT.
  ARM: Kirkwood: Describe IB62x0 LEDs in DT.
  ARM: Kirkwood: Describe IB62x0 gpio-keys in DT.
  ARM: Kirkwood: Describe DNS32? gpio-keys in DT.
  ARM: Kirkwood: Move common portions into a kirkwood-dnskw.dtsi
  ARM: Kirkwood: Replace DNS-320/DNS-325 leds with dt bindings
  ARM: Kirkwood: Describe DNS325 temperature sensor in DT.
  ARM: Kirkwood: Use DT to configure SATA device.
  ARM: kirkwood: use devicetree for SPI on dreamplug
  ARM: kirkwood: Add LS-XHL and LS-CHLv2 support
  ARM: Kirkwood: Initial DTS support for Kirkwood GoFlex Net
  ARM: Kirkwood: Add basic device tree support for QNAP TS219.
  ATA: sata_mv: Add device tree support
  ARM: Orion: DTify the watchdog timer.
  ARM: Orion: Add arch support needed for I2C via DT.
  ARM: kirkwood: use devicetree for orion-spi
  ...

Conflicts:
	drivers/watchdog/orion_wdt.c

Pull arm-soc cpuidle enablement for OMAP from Olof Johansson:
 "Coupled cpuidle was meant to merge for 3.5 through Len Brown's tree,
  but didn't go in because the pull request ended up rejected.  So it
  just got merged, and we got this staged branch that enables the
  coupled cpuidle code on OMAP.

  With a stable git workflow from the other maintainer we could have
  staged this earlier, but that wasn't the case so we have had to merge
  it late.

  The alternative is to hold it off until 3.7 but given that the code is
  well-isolated to OMAP and they are eager to see it go in, I didn't
  push back hard in that direction."

* tag 'pm2' of git://git.kernel.org/pub/scm/linux/kernel/git/arm/arm-soc:
  ARM: OMAP4: CPUidle: Open broadcast clock-event device.
  ARM: OMAP4: CPUidle: add synchronization for coupled idle states
  ARM: OMAP4: CPUidle: Use coupled cpuidle states to implement SMP cpuidle.
  ARM: OMAP: timer: allow gp timer clock-event to be used on both cpus

Pull ARM SoC fixes from Olof Johansson:
 "A few fixes for merge window fallout, and a bugfix for timer resume on
  PRIMA2."

* tag 'fixes-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/arm/arm-soc:
  ARM: mmp: add missing irqs.h
  arm: mvebu: fix typo in .dtsi comment for Armada XP SoCs
  ARM: PRIMA2: delete redundant codes to restore LATCHED when timer resumes
  ARM: mxc: Include missing irqs.h header

Pull SuperH fixes from Paul Mundt.

* tag 'sh-for-linus' of git://github.com/pmundt/linux-sh: (24 commits)
  sh: explicitly include sh_dma.h in setup-sh7722.c
  sh: ecovec: care CN5 VBUS if USB host mode
  sh: sh7724: fixup renesas_usbhs clock settings
  sh: intc: initial irqdomain support.
  sh: pfc: Fix up init ordering mess.
  serial: sh-sci: fix compilation breakage, when DMA is enabled
  dmaengine: shdma: restore partial transfer calculation
  sh: modify the sh_dmae_slave_config for RSPI in setup-sh7757
  sh: Fix up recursive fault in oops with unset TTB.
  sh: pfc: Build fix for pinctrl_remove_gpio_range() changes.
  sh: select the fixed regulator driver on several boards
  sh: ecovec: switch MMC power control to regulators
  sh: add fixed voltage regulators to se7724
  sh: add fixed voltage regulators to sdk7786
  sh: add fixed voltage regulators to rsk
  sh: add fixed voltage regulators to migor
  sh: add fixed voltage regulators to kfr2r09
  sh: add fixed voltage regulators to ap325rxa
  sh: add fixed voltage regulators to sh7757lcr
  sh: add fixed voltage regulators to sh2007
  ...

Pull additional md update from NeilBrown:
 "This contains a few patches that depend on plugging changes in the
  block layer so needed to wait for those.

  It also contains a Kconfig fix for the new RAID10 support in dm-raid."

* tag 'md-3.6' of git://neil.brown.name/md:
  md/dm-raid: DM_RAID should select MD_RAID10
  md/raid1: submit IO from originating thread instead of md thread.
  raid5: raid5d handle stripe in batch way
  raid5: make_request use batch stripe release

Pull two ceph fixes from Sage Weil:
 "The first patch fixes up the old crufty open intent code to use the
  atomic_open stuff properly, and the second fixes a possible null deref
  and memory leak with the crypto keys."

* 'for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/sage/ceph-client:
  libceph: fix crypto key null deref, memory leak
  ceph: simplify+fix atomic_open

Merge tag 'ecryptfs-3.6-rc1-fixes' of git://git.kernel.org/pub/scm/linux/kernel/git/tyhicks/ecryptfs

Pull ecryptfs fixes from Tyler Hicks:
 - Fixes a bug when the lower filesystem mount options include 'acl',
   but the eCryptfs mount options do not
 - Cleanups in the messaging code
 - Better handling of empty files in the lower filesystem to improve
   usability.  Failed file creations are now cleaned up and empty lower
   files are converted into eCryptfs during open().
 - The write-through cache changes are being reverted due to bugs that
   are not easy to fix.  Stability outweighs the performance
   enhancements here.
 - Improvement to the mount code to catch unsupported ciphers specified
   in the mount options

* tag 'ecryptfs-3.6-rc1-fixes' of git://git.kernel.org/pub/scm/linux/kernel/git/tyhicks/ecryptfs:
  eCryptfs: check for eCryptfs cipher support at mount
  eCryptfs: Revert to a writethrough cache model
  eCryptfs: Initialize empty lower files when opening them
  eCryptfs: Unlink lower inode when ecryptfs_create() fails
  eCryptfs: Make all miscdev functions use daemon ptr in file private_data
  eCryptfs: Remove unused messaging declarations and function
  eCryptfs: Copy up POSIX ACL and read-only flags from lower mount

Pull CIFS update from Steve French:
 "Adds SMB2 rmdir/mkdir capability to the SMB2/SMB2.1 support in cifs.

  I am holding up a few more days on merging the remainder of the
  SMB2/SMB2.1 enablement although it is nearing review completion, in
  order to address some review comments from Jeff Layton on a few of the
  subsequent SMB2 patches, and also to debug an unrelated cifs problem
  that Pavel discovered."

* 'for-next' of git://git.samba.org/sfrench/cifs-2.6:
  CIFS: Add SMB2 support for rmdir
  CIFS: Move rmdir code to ops struct
  CIFS: Add SMB2 support for mkdir operation
  CIFS: Separate protocol specific part from mkdir
  CIFS: Simplify cifs_mkdir call

Borislav Petkov reports that the new warning added in commit
88fdf75d ("mm: warn if pg_data_t isn't initialized with zero")
triggers for him, and it is the node_start_pfn field that has already
been initialized once.

The call trace looks like this:

  x86_64_start_kernel ->
    x86_64_start_reservations ->
    start_kernel ->
    setup_arch ->
    paging_init ->
    zone_sizes_init ->
    free_area_init_nodes ->
    free_area_init_node

and (with the warning replaced by debug output), Borislav sees

  On node 0 totalpages: 4193848
    DMA zone: 64 pages used for memmap
    DMA zone: 6 pages reserved
    DMA zone: 3890 pages, LIFO batch:0
    DMA32 zone: 16320 pages used for memmap
    DMA32 zone: 798464 pages, LIFO batch:31
    Normal zone: 52736 pages used for memmap
    Normal zone: 3322368 pages, LIFO batch:31
  free_area_init_node: pgdat->node_start_pfn: 4423680      <----
  On node 1 totalpages: 4194304
    Normal zone: 65536 pages used for memmap
    Normal zone: 4128768 pages, LIFO batch:31
  free_area_init_node: pgdat->node_start_pfn: 8617984      <----
  On node 2 totalpages: 4194304
    Normal zone: 65536 pages used for memmap
    Normal zone: 4128768 pages, LIFO batch:31
  free_area_init_node: pgdat->node_start_pfn: 12812288     <----
  On node 3 totalpages: 4194304
    Normal zone: 65536 pages used for memmap
    Normal zone: 4128768 pages, LIFO batch:31

so remove the bogus warning for now to avoid annoying people.  Minchan
Kim is looking at it.
Reported-by: Borislav Petkov <bp@amd64.org>
Cc: Minchan Kim <minchan@kernel.org>
Cc: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>