- 27 Jul, 2018 40 commits
-
-
Gao Xiang authored
This patch adds an optional choice which can be enabled by users in order to cache both incomplete ends of compressed clusters as a complement to the in-place decompression in order to boost random read, but it costs more memory than the in-place decompression only. Signed-off-by:
Gao Xiang <gaoxiang25@huawei.com> Signed-off-by:
Greg Kroah-Hartman <gregkh@linuxfoundation.org>
-
Gao Xiang authored
This patch introduces the basic in-place VLE decompression implementation for the erofs file system. Compared with fixed-sized input compression, it implements what we call 'the variable-length extent compression' which specifies the same output size for each compression block to make the full use of IO bandwidth (which means almost all data from block device can be directly used for decomp- ression), improve the real (rather than just via data caching, which costs more memory) random read and keep the relatively lower compression ratios (it saves more storage space than fixed-sized input compression which is also configured with the same input block size), as illustrated below: |--- variable-length extent ---|------ VLE ------|--- VLE ---| /> clusterofs /> clusterofs /> clusterofs /> clusterofs ++---|-------++-----------++---------|-++-----------++-|---------++-| ...|| | || || | || || | || | ... original data ++---|-------++-----------++---------|-++-----------++-|---------++-| ++->cluster<-++->cluster<-++->cluster<-++->cluster<-++->cluster<-++ size size size size size \ / / / \ / / / \ / / / ++-----------++-----------++-----------++ ... || || || || ... compressed clusters ++-----------++-----------++-----------++ ++->cluster<-++->cluster<-++->cluster<-++ size size size The main point of 'in-place' refers to the decompression mode: Instead of allocating independent compressed pages and data structures, it reuses the allocated file cache pages at most to store its compressed data and the corresponding pagevec in a time-sharing approach by default, which will be useful for low memory scenario. In the end, unlike the other filesystems with (de)compression support using a relatively large compression block size, which reads and decompresses >= 128KB at once, and gains a more good-looking random read (In fact it collects small random reads into large sequential reads and caches all decompressed data in memory, but it is unacceptable especially for embedded devices with limited memory, and it is not the real random read), we select a universal small-sized 4KB compressed cluster, which is the smallest page size for most architectures, and all compressed clusters can be read and decompressed independently, which ensures random read number for all use cases. Signed-off-by: -
Gao Xiang authored
This patch introduces another concept used by the unzip subsystem called 'workstation'. It can be seen as a sparse array that stores pointers pointed to data structures related to the corresponding physical blocks. All lookup cases are protected by RCU read lock. Besides, reference count and spin_lock are also introduced to manage its lifetime and serialize all update operations. 'workstation' is currently implemented on the in-kernel radix tree approach for backward compatibility. With the evolution of linux kernel, it could be migrated into XArray implementation in the future. Signed-off-by:
-
Gao Xiang authored
This patch adds a dedicated shrinker targeting to free unneeded memory consumed by a number of erofs in-memory data structures. Like F2FS and UBIFS, it also adds: - sbi->umount_mutex to avoid races on shrinker and put_super - sbi->shrinker_run_no to not revisit recently scaned objects Signed-off-by:
-
Gao Xiang authored
In order to introducing shrinker solution for erofs, let's manage all mounted erofs instances at first. Signed-off-by:
-
Gao Xiang authored
Currently, this patch only simply implements LZ4 decompressor due to its development priority. In the future, erofs will support more compression algorithm and format other than LZ4, thus a generic decompressor interface will be needed. Signed-off-by:
-
Gao Xiang authored
We have to reduce the memory cost as much as possible, so we don't want to decompress more data beyond the output buffer size, however "LZ4_decompress_safe_partial" doesn't guarantee to stop at the arbitary end position, but stop just after its current LZ4 "sequence" is completed. Link: https://groups.google.com/forum/#!topic/lz4c/_3kkz5N6n00 Therefore, I hacked the LZ4 decompression logic by hand, probably NOT the fastest approach, and hope for better implementation. Signed-off-by:
Miao Xie <miaoxie@huawei.com> Signed-off-by:
Chao Yu <yuchao0@huawei.com> Signed-off-by:
-
Gao Xiang authored
The unzip subsystem also uses these functions, let's export them to internal.h. Signed-off-by:
-
Gao Xiang authored
This patch introduces an temporary _on-stack_ page pool to reuse the freed page directly as much as it can for better performance and release all pages at a time, it also slightly reduces the possibility of the potential memory allocation failure. Signed-off-by:
-
Gao Xiang authored
This patch introduces an iterable L2P mapping operation 'erofs_map_blocks_iter'. Compared with 'erofs_map_blocks', it avoids a number of redundant 'release and regrab' processes if they request the same meta page. Signed-off-by:
-
Gao Xiang authored
For each compressed cluster, there is a straight-forward way of allocating a fixed or variable-sized (for VLE) array to record the corresponding file pages for its decompression if we decide to decompress these pages asynchronously (eg. read-ahead case), however it could take much extra on-heap memory compared with traditional uncompressed filesystems. This patch introduces a pagevec solution to reuse some allocated file page in the time-sharing approach storing parts of the array itself in order to minimize the extra memory overhead, thus only a constant and small-sized array used for booting the whole array itself up will be needed. Signed-off-by:
-
Gao Xiang authored
Currently kernel has scattered tagged pointer usages hacked by hand in plain code, without a unique and portable functionset to highlight the tagged pointer itself and wrap these hacked code in order to clean up all over meaningless magic masks. Therefore, this patch introduces simple generic methods to fold tags into a pointer integer. It currently supports the last n bits of the pointer for tags, which can be selected by users. In addition, it will also be used for the upcoming EROFS filesystem, which heavily uses tagged pointer approach for high performance and reducing extra memory allocation. Link: https://en.wikipedia.org/wiki/Tagged_pointerSigned-off-by:
-
Chao Yu authored
Add basic tracepoints for ->readpage{,s}, ->lookup, ->destroy_inode, fill_inode and map_blocks. Reviewed-by: -
Chao Yu authored
This patch introduces error injection infrastructure, with it, we can inject error in any kernel exported common functions which erofs used, so that it can force erofs running into error paths, it turns out that tests can cover real rare paths more easily to find bugs. Reviewed-by:
-
Chao Yu authored
This patch adds to support special inode, such as block dev, char, socket, pipe inode. Reviewed-by:
-
Gao Xiang authored
This implements xattr and acl functionalities. Inline and shared xattrs are introduced for flexibility. Specifically, if the same xattr occurs for many times in a large number of inodes or the value of a xattr is so large that it isn't suitable to be inlined, a shared xattr kept in the xattr meta will be used instead. Signed-off-by:
-
Gao Xiang authored
This commit adds Makefile and Kconfig for erofs, and updates Makefile and Kconfig files in the fs directory. Signed-off-by:
-
Gao Xiang authored
This commit adds functions that transfer names to inodes. Signed-off-by:
-
Gao Xiang authored
This adds functions for directory, mainly readdir. Signed-off-by:
-
Gao Xiang authored
This adds core functions to get, read an inode. Signed-off-by:
-
Gao Xiang authored
This commit adds functions for meta and raw data, and also provides address_space_operations for raw data access. Signed-off-by:
-
Gao Xiang authored
This commit adds erofs super block operations, including (u)mount, remount_fs, show_options, statfs, in addition to some private icache management functions. Signed-off-by:
-
Gao Xiang authored
- erofs_sb_info: contains erofs-specific in-memory information. - erofs_vnode: contains vfs_inode and other fs-specific information. same as super block, the only one in-memory definition exists. - erofs_map_blocks plays a role in the file L2P mapping Signed-off-by:
-
Gao Xiang authored
This commit adds the on-disk layout header file of erofs. Note that the on-disk layout is still WIP, and some fields are reserved for the future use by design. Any comments are welcome. Thanks-to: Li Guifu <liguifu2@huawei.com> Thanks-to: Sun Qiuyang <sunqiuyang@huawei.com> Signed-off-by:
-
Ioana Ciornei authored
Document nested structs per kernel-doc requirements by moving all comments before the actual struct. Signed-off-by:
Ioana Ciornei <ioana.ciornei@nxp.com> Signed-off-by:
-
Ioana Ciornei authored
Document nested structs per kernel-doc requirements by moving all comments before the actual struct. Signed-off-by:
-
Ioana Radulescu authored
We never really used the driver version, so no point in keeping it around. Signed-off-by:
Ioana Radulescu <ruxandra.radulescu@nxp.com> Signed-off-by:
-
Ioana Radulescu authored
In our documentation, we claim to use a 5-tuple key for Rx hash distribution of flows. The code however configures a key composed of all supported header fields. Update the Rx hash key to contain only the documented fields: {IP src, IP dst, IP nextproto, L4 src, L4 dst}, which was the original intention and makes most sense as a default. Signed-off-by: -
Ioana Radulescu authored
Error message was referencing wrong function, fix it. Signed-off-by:
-
Georgios Tsotsos authored
Adding appropriate SPDX-License-Identifier (GPL-2) that were missing from code, header and make files. Signed-off-by:
Georgios Tsotsos <tsotsos@gmail.com> Signed-off-by:
-
Todd Poynor authored
Print device addresses as unsigned long, not as kernel pointers. Signed-off-by:
Todd Poynor <toddpoynor@google.com> Signed-off-by:
-
Todd Poynor authored
Gasket/apex drivers now use standard logging, remove TODO entry for this. Signed-off-by:
-
Todd Poynor authored
Gasket logging functions no longer used. Signed-off-by:
-
Todd Poynor authored
Drop gasket logging calls in favor of standard logging. Signed-off-by:
-
Todd Poynor authored
Drop gasket logging calls in favor of standard logging. Signed-off-by:
-
Todd Poynor authored
Replace gasket logging calls with standard logging calls. Signed-off-by:
-
Todd Poynor authored
Replace gasket logging calls with standard logging calls. Signed-off-by:
-
Todd Poynor authored
Convert gasket logging calls to standard functions. Signed-off-by:
-
Todd Poynor authored
Use standard logging functions, drop use of gasket log functions. Signed-off-by:
-
Todd Poynor authored
Save the struct device pointer to a gasket device in gasket's metadata, to facilitate use of standard logging calls and in anticipation of non-PCI gasket devices in the future. Signed-off-by:
-
