- 08 Oct, 2021 15 commits
-
-
Quentin Monnet authored
With "make install", bpftool installs its binary and its bash completion file. Usually, this is what we want. But a few components in the kernel repository (namely, BPF iterators and selftests) also install bpftool locally before using it. In such a case, bash completion is not necessary and is just a useless build artifact. Let's add an "install-bin" target to bpftool, to offer a way to install the binary only. Signed-off-by:
Quentin Monnet <quentin@isovalent.com> Signed-off-by:
Andrii Nakryiko <andrii@kernel.org> Link: https://lore.kernel.org/bpf/20211007194438.34443-13-quentin@isovalent.com
-
Quentin Monnet authored
The script test_bpftool_build.sh attempts to build bpftool in the various supported ways, to make sure nothing breaks. One of those ways is to run "make tools/bpf" from the root of the kernel repository. This command builds bpftool, along with the other tools under tools/bpf, and runqslower in particular. After running the command and upon a successful bpftool build, the script attempts to cleanup the generated objects. However, after building with this target and in the case of runqslower, the files are not cleaned up as expected. This is because the "tools/bpf" target sets $(OUTPUT) to .../tools/bpf/runqslower/ when building the tool, causing the object files to be placed directly under the runqslower directory. But when running "cd tools/bpf; make clean", the value for $(OUTPUT) is set to ".output" (relative to the runqslower directory) by runqslower's Makefile, and this is where the Makefile looks for files to clean up. We cannot easily fix in the root Makefile (where "tools/bpf" is defined) or in tools/scripts/Makefile.include (setting $(OUTPUT)), where changing the way the output variables are passed would likely have consequences elsewhere. We could change runqslower's Makefile to build in the repository instead of in a dedicated ".output/", but doing so just to accommodate a test script doesn't sound great. Instead, let's just make sure that we clean up runqslower properly by adding the correct command to the script. This will attempt to clean runqslower twice: the first try with command "cd tools/bpf; make clean" will search for tools/bpf/runqslower/.output and fail to clean it (but will still clean the other tools, in particular bpftool), the second one (added in this commit) sets the $(OUTPUT) variable like for building with the "tool/bpf" target and should succeed. Signed-off-by:
-
Quentin Monnet authored
In samples/bpf/Makefile, libbpf has a FORCE dependency that force it to be rebuilt. I read this as a way to keep the library up-to-date, given that we do not have, in samples/bpf, a list of the source files for libbpf itself. However, a better approach would be to use the "$(wildcard ...)" function from make, and to have libbpf depend on all the .c and .h files in its directory. This is what samples/bpf/Makefile does for bpftool, and also what the BPF selftests' Makefile does for libbpf. Let's update the Makefile to avoid rebuilding libbpf all the time (and bpftool on top of it). Signed-off-by:
-
Quentin Monnet authored
API headers from libbpf should not be accessed directly from the source directory. Instead, they should be exported with "make install_headers". Make sure that samples/bpf/Makefile installs the headers properly when building. The object compiled from and exported by libbpf are now placed into a subdirectory of sample/bpf/ instead of remaining in tools/lib/bpf/. We attempt to remove this directory on "make clean". However, the "clean" target re-enters the samples/bpf/ directory from the root of the repository ("$(MAKE) -C ../../ M=$(CURDIR) clean"), in such a way that $(srctree) and $(src) are not defined, making it impossible to use $(LIBBPF_OUTPUT) and $(LIBBPF_DESTDIR) in the recipe. So we only attempt to clean $(CURDIR)/libbpf, which is the default value. Add a dependency on libbpf's headers for the $(TRACE_HELPERS). We also change the output directory for bpftool, to place the generated objects under samples/bpf/bpftool/ instead of building in bpftool's directory directly. Doing so, we make sure bpftool reuses the libbpf library previously compiled and installed. Signed-off-by: -
Quentin Monnet authored
Update samples/bpf/.gitignore to ignore files generated when building the samples. Add: - vmlinux.h - the generated skeleton files (*.skel.h) - the samples/bpf/libbpf/ and .../bpftool/ directories, in preparation of a future commit which introduces a local output directory for building libbpf and bpftool. Signed-off-by: -
Quentin Monnet authored
API headers from libbpf should not be accessed directly from the library's source directory. Instead, they should be exported with "make install_headers". Let's make sure that bpf/preload/iterators/Makefile installs the headers properly when building. Signed-off-by:
-
Quentin Monnet authored
API headers from libbpf should not be accessed directly from the library's source directory. Instead, they should be exported with "make install_headers". Let's make sure that bpf/preload/Makefile installs the headers properly when building. Note that we declare an additional dependency for iterators/iterators.o: having $(LIBBPF_A) as a dependency to "$(obj)/bpf_preload_umd" is not sufficient, as it makes it required only at the linking step. But we need libbpf to be compiled, and in particular its headers to be exported, before we attempt to compile iterators.o. The issue would not occur before this commit, because libbpf's headers were not exported and were always available under tools/lib/bpf. Signed-off-by:
-
Quentin Monnet authored
API headers from libbpf should not be accessed directly from the library's source directory. Instead, they should be exported with "make install_headers". Let's make sure that runqslower installs the headers properly when building. We use a libbpf_hdrs target to mark the logical dependency on libbpf's headers export for a number of object files, even though the headers should have been exported at this time (since bpftool needs them, and is required to generate the skeleton or the vmlinux.h). When descending from a parent Makefile, the specific output directories for building the library and exporting the headers are configurable with BPFOBJ_OUTPUT and BPF_DESTDIR, respectively. This is in addition to OUTPUT, on top of which those variables are constructed by default. Also adjust the Makefile for the BPF selftests. We pass a number of variables to the "make" invocation, because we want to point runqslower to the (target) libbpf shared with other tools, instead of building its own version. In addition, runqslower relies on (target) bpftool, and we also want to pass the proper variables to its Makefile so that bpftool itself reuses the same libbpf. Signed-off-by:
-
Quentin Monnet authored
API headers from libbpf should not be accessed directly from the library's source directory. Instead, they should be exported with "make install_headers". Let's make sure that resolve_btfids installs the headers properly when building. When descending from a parent Makefile, the specific output directories for building the library and exporting the headers are configurable with LIBBPF_OUT and LIBBPF_DESTDIR, respectively. This is in addition to OUTPUT, on top of which those variables are constructed by default. Also adjust the Makefile for the BPF selftests in order to point to the (target) libbpf shared with other tools, instead of building a version specific to resolve_btfids. Remove libbpf's order-only dependencies on the include directories (they are created by libbpf and don't need to exist beforehand). Signed-off-by:
-
Quentin Monnet authored
Bpftool relies on libbpf, therefore it relies on a number of headers from the library and must be linked against the library. The Makefile for bpftool exposes these objects by adding tools/lib as an include directory ("-I$(srctree)/tools/lib"). This is a working solution, but this is not the cleanest one. The risk is to involuntarily include objects that are not intended to be exposed by the libbpf. The headers needed to compile bpftool should in fact be "installed" from libbpf, with its "install_headers" Makefile target. In addition, there is one header which is internal to the library and not supposed to be used by external applications, but that bpftool uses anyway. Adjust the Makefile in order to install the header files properly before compiling bpftool. Also copy the additional internal header file (nlattr.h), but call it out explicitly. Build (and install headers) in a subdirectory under bpftool/ instead of tools/lib/bpf/. When descending from a parent Makefile, this is configurable by setting the OUTPUT, LIBBPF_OUTPUT and LIBBPF_DESTDIR variables. Also adjust the Makefile for BPF selftests, so as to reuse the (host) libbpf compiled earlier and to avoid compiling a separate version of the library just for bpftool. Signed-off-by: -
Quentin Monnet authored
It seems that the header file was never necessary to compile bpftool, and it is not part of the headers exported from libbpf. Let's remove the includes from prog.c and gen.c. Fixes: d510296d ("bpftool: Use syscall/loader program in "prog load" and "gen skeleton" command.") Signed-off-by:
-
Quentin Monnet authored
The "install_headers" target in libbpf's Makefile would unconditionally export all API headers to the target directory. When those headers are installed to compile another application, this means that make always finds newer dependencies for the source files relying on those headers, and deduces that the targets should be rebuilt. Avoid that by making "install_headers" depend on the source header files, and (re-)install them only when necessary. Signed-off-by:
-
Yucong Sun authored
New clang version changed ([0]) type name in dwarf from "long int" to "long", this is causing btf_dump tests to fail. [0] https://github.com/llvm/llvm-project/commit/f6a561c4d6754b13165a49990e8365d819f64c86Signed-off-by:
Yucong Sun <sunyucong@gmail.com> Signed-off-by:
-
Dave Marchevsky authored
Since commit 6c4fc209 ("bpf: remove useless version check for prog load") these "version" sections, which result in bpf_attr.kern_version being set, have been unnecessary. Remove them so that it's obvious to folks using selftests as a guide that "modern" BPF progs don't need this section. Signed-off-by:
Dave Marchevsky <davemarchevsky@fb.com> Signed-off-by:
-
Song Liu authored
VMs running on upstream 5.12+ kernel support LBR. However, bpf_get_branch_snapshot couldn't stop the LBR before too many entries are flushed. Skip the hit/waste test for VMs before we find a proper fix for LBR in VM. Fixes: 025bd7c7 ("selftests/bpf: Add test for bpf_get_branch_snapshot") Signed-off-by:
Song Liu <songliubraving@fb.com> Signed-off-by:
-
- 07 Oct, 2021 4 commits
-
-
Johan Almbladh authored
This patch adds new tests for the two-instruction LD_IMM64. The new tests verify the operation with immediate values of different byte patterns. Mainly intended to cover JITs that want to be clever when loading 64-bit constants. Signed-off-by:
Johan Almbladh <johan.almbladh@anyfinetworks.com> Signed-off-by:
Daniel Borkmann <daniel@iogearbox.net> Link: https://lore.kernel.org/bpf/20211007143006.634308-1-johan.almbladh@anyfinetworks.com
-
Johan Almbladh authored
This patch shaves off a few instructions when loading sparse 64-bit constants to register. The change is covered by additional tests in lib/test_bpf.c. Signed-off-by:
-
Johan Almbladh authored
This patch removes a stale Makefile reference to the cBPF JIT that was removed. Fixes: ebcbacfa ("mips, bpf: Remove old BPF JIT implementations") Signed-off-by:
-
Jie Meng authored
Introduce a single reg version of maybe_emit_mod() and factor out common code in more cases. Signed-off-by:
Jie Meng <jmeng@fb.com> Signed-off-by:
-
- 06 Oct, 2021 21 commits
-
-
Andrii Nakryiko authored
Hengqi Chen says: ==================== bpf_{map,program}__{prev,next} don't follow the libbpf API naming convention. Deprecate them and replace them with a new set of APIs named bpf_object__{prev,next}_{program,map}. v1->v2: [0] * Addressed Andrii's comments [0]: https://patchwork.kernel.org/project/netdevbpf/patch/20210906165456.325999-1-hengqi.chen@gmail.com/ ==================== Signed-off-by: -
Hengqi Chen authored
BPF objects are not reloadable after unload. Users are expected to use bpf_object__close() to unload and free up resources in one operation. No need to expose bpf_object__unload() as a public API, deprecate it ([0]). Add bpf_object__unload() as an alias to internal bpf_object_unload() and replace all bpf_object__unload() uses to avoid compilation errors. [0] Closes: https://github.com/libbpf/libbpf/issues/290Signed-off-by:
Hengqi Chen <hengqi.chen@gmail.com> Signed-off-by:
-
Hengqi Chen authored
Replace deprecated bpf_{map,program}__next APIs with newly added bpf_object__next_{map,program} APIs, so that no compilation warnings emit. Signed-off-by: -
Grant Seltzer authored
This adds a section to the documentation for libbpf naming convention which describes how to document API features in libbpf, specifically the format of which API doc comments need to conform to. Signed-off-by:
Grant Seltzer <grantseltzer@gmail.com> Signed-off-by:
-
Hengqi Chen authored
Deprecate bpf_{map,program}__{prev,next} APIs. Replace them with a new set of APIs named bpf_object__{prev,next}_{program,map} which follow the libbpf API naming convention ([0]). No functionality changes. [0] Closes: https://github.com/libbpf/libbpf/issues/296Signed-off-by: -
Jiri Olsa authored
Currently the recursion test is hooking __htab_map_lookup_elem function, which is invoked both from bpf_prog and bpf syscall. But in our kernel build, the __htab_map_lookup_elem gets inlined within the htab_map_lookup_elem, so it's not trigered and the test fails. Fixing this by using htab_map_delete_elem, which is not inlined for bpf_prog calls (like htab_map_lookup_elem is) and is used directly as pointer for map_delete_elem, so it won't disappear by inlining. Signed-off-by:
Jiri Olsa <jolsa@kernel.org> Signed-off-by:
-
Quentin Monnet authored
Recent-ish versions of make do no longer consider number signs ("#") as comment symbols when they are inserted inside of a macro reference or in a function invocation. In such cases, the symbols should not be escaped. There are a few occurrences of "\#" in libbpf's and samples' Makefiles. In the former, the backslash is harmless, because grep associates no particular meaning to the escaped symbol and reads it as a regular "#". In samples' Makefile, recent versions of make will pass the backslash down to the compiler, making the probe fail all the time and resulting in the display of a warning about "make headers_install" being required, even after headers have been installed. A similar issue has been addressed at some other locations by commit 9564a8cf ("Kbuild: fix # escaping in .cmd files for future Make"). Let's address it for libbpf's and samples' Makefiles in the same fashion, by using a "$(pound)" variable (pulled from tools/scripts/Makefile.include for libbpf, or re-defined for the samples). Reference for the change in make: https://git.savannah.gnu.org/cgit/make.git/commit/?id=c6966b323811c37acedff05b57 Fixes: 2f383041 ("libbpf: Make libbpf_version.h non-auto-generated") Fixes: 07c3bbdb ("samples: bpf: print a warning about headers_install") Signed-off-by: -
Daniel Borkmann authored
The BPF core defines a __weak bpf_jit_compile() dummy function already which should only be overridden by JITs if they actually implement a legacy cBPF JIT. Given arm implements an eBPF JIT, this stub is not needed. Now that MIPS cBPF JIT is finally gone, the only JIT left that is still implementing bpf_jit_compile() is the sparc32 one. Signed-off-by:
-
Daniel Borkmann authored
Johan Almbladh says: ==================== This is an implementation of an eBPF JIT for MIPS I-V and MIPS32/64 r1-r6. The new JIT is written from scratch, but uses the same overall structure as other eBPF JITs. Before, the MIPS JIT situation looked like this. - 32-bit: MIPS32, cBPF-only, tests fail - 64-bit: MIPS64r2-r6, eBPF, tests fail, incomplete eBPF ISA support The new JIT implementation raises the bar to the following level. - 32/64-bit: all MIPS ISA, eBPF, all tests pass, full eBPF ISA support Overview -------- The implementation supports all 32-bit and 64-bit eBPF instructions defined as of this writing, including the recently-added atomics. It is intended to provide good performance for native word size operations, while also being complete so the JIT never has to fall back to the interpreter. The new JIT replaces the current cBPF and eBPF JITs for MIPS. The implementation is divided into separate files as follows. The source files contains comments describing internal mechanisms and details on things like eBPF-to-CPU register mappings, so I won't repeat that here. - jit_comp.[ch] code shared between 32-bit and 64-bit JITs - jit_comp32.c 32-bit JIT implementation - jit_comp64.c 64-bit JIT implementation Both the 32-bit and 64-bit versions map all eBPF registers to native MIPS CPU registers. There are also enough unmapped CPU registers available to allow all eBPF operations implemented natively by the JIT to use only CPU registers without having to resort to stack scratch space. Some operations are deemed too complex to implement natively in the JIT. Those are instead implemented as a function call to a helper that performs the operation. This is done in the following cases. - 64-bit div and mod on a 32-bit CPU - 64-bit atomics on a 32-bit CPU - 32-bit atomics on a 32-bit CPU that lacks ll/sc instructions CPU errata workarounds ---------------------- The JIT implements workarounds for R10000, Loongson-2F and Loongson-3 CPU errata. For the Loongson workarounds, I have used the public information available on the matter. Link: https://sourceware.org/legacy-ml/binutils/2009-11/msg00387.html Testing ------- During the development of the JIT, I have added a number of new test cases to the test_bpf.ko test suite to be able to verify correctness of JIT implementations in a more systematic way. The new additions increase the test suite roughly three-fold, with many of the new tests being very extensive and even exhaustive when feasible. Link: https://lore.kernel.org/bpf/20211001130348.3670534-1-johan.almbladh@anyfinetworks.com/ Link: https://lore.kernel.org/bpf/20210914091842.4186267-1-johan.almbladh@anyfinetworks.com/ Link: https://lore.kernel.org/bpf/20210809091829.810076-1-johan.almbladh@anyfinetworks.com/ The JIT has been tested by running the test_bpf.ko test suite in QEMU with the following MIPS ISAs, in both big and little endian mode, with and without JIT hardening enabled. MIPS32r2, MIPS32r6, MIPS64r2, MIPS64r6 For the QEMU r2 targets, the correctness of pre-r2 code emitted has been tested by manually overriding each of the following macros with 0. cpu_has_llsc, cpu_has_mips_2, cpu_has_mips_r1, cpu_has_mips_r2 Similarly, CPU errata workaround code has been tested by enabling the each of the following configurations for the MIPS64r2 targets. CONFIG_WAR_R10000 CONFIG_CPU_LOONGSON3_WORKAROUNDS CONFIG_CPU_NOP_WORKAROUNDS CONFIG_CPU_JUMP_WORKAROUNDS The JIT passes all tests in all configurations. Below is the summary for MIPS32r2 in little endian mode. test_bpf: Summary: 1006 PASSED, 0 FAILED, [994/994 JIT'ed] test_bpf: test_tail_calls: Summary: 8 PASSED, 0 FAILED, [8/8 JIT'ed] test_bpf: test_skb_segment: Summary: 2 PASSED, 0 FAILED According to MIPS ISA reference documentation, the result of a 32-bit ALU arithmetic operation on a 64-bit CPU is unpredictable if an operand register value is not properly sign-extended to 64 bits. To verify the code emitted by the JIT, the code generation engine in QEMU was modifed to flip all low 32 bits if the above condition was not met. With this trip-wire installed, the kernel booted properly in qemu-system-mips64el and all test_bpf.ko tests passed. Remaining features ------------------ While the JIT is complete is terms of eBPF ISA support, this series does not include support for BPF-to-BPF calls and BPF trampolines. Those features are planned to be added in another patch series. The BPF_ST | BPF_NOSPEC instruction currently emits nothing. This is consistent with the behavior if the MIPS interpreter and the existing eBPF JIT. Why not build on the existing eBPF JIT? --------------------------------------- The existing eBPF JIT was originally written for MIPS64. An effort was made to add MIPS32 support to it in commit 716850ab ("MIPS: eBPF: Initial eBPF support for MIPS32 architecture."). That turned out to contain a number of flaws, so eBPF support for MIPS32 was disabled in commit 36366e36 ("MIPS: BPF: Restore MIPS32 cBPF JIT"). Link: https://lore.kernel.org/bpf/5deaa994.1c69fb81.97561.647e@mx.google.com/ The current eBPF JIT for MIPS64 lacks a lot of functionality regarding ALU32, JMP32 and atomic operations. It also lacks 32-bit CPU support on a fundamental level, for example 32-bit CPU register mappings and o32 ABI calling conventions. For optimization purposes, it tracks register usage through the program control flow in order to do zero-extension and sign- extension only when necessary, a static analysis of sorts. In my opinion, having this kind of complexity in JITs, and for which there is not adequate test coverage, is a problem. Such analysis should be done by the verifier, if needed at all. Finally, when I run the BPF test suite test_bpf.ko on the current JIT, there are errors and warnings. I believe that an eBPF JIT should strive to be correct, complete and optimized, and in that order. The JIT runs after the verifer has audited the program and given its approval. If the JIT then emits code that does something else, it will undermine the eBPF security model. A simple implementation is easier to get correct than a complex one. Furthermore, the real performance hit is not an extra CPU instruction here and there, but when the JIT bails on an unimplemented eBPF instruction and cause the whole program to fall back to the interpreter. My reasoning here boils down to the following. * The JIT should not contain a static analyzer that tracks branches. * It is acceptable to emit possibly superfluous sign-/zero-extensions for ALU32 and JMP32 operations on a 64-bit MIPS to guarantee correctness. * The JIT should handle all eBPF instructions on all MIPS CPUs. I conclude that the current eBPF MIPS JIT is complex, incomplete and incorrect. For the reasons stated above, I decided to not use the existing JIT implementation. ==================== Signed-off-by:
-
Johan Almbladh authored
This patch removes the old 32-bit cBPF and 64-bit eBPF JIT implementations. They are replaced by a new eBPF implementation that supports both 32-bit and 64-bit MIPS CPUs. Signed-off-by:
-
Johan Almbladh authored
This patch enables the new eBPF JITs for 32-bit and 64-bit MIPS. It also disables the old cBPF JIT to so cBPF programs are converted to use the new JIT. Workarounds for R4000 CPU errata are not implemented by the JIT, so the JIT is disabled if any of those workarounds are configured. Signed-off-by:
-
Johan Almbladh authored
This patch adds workarounds for the following CPU errata to the MIPS eBPF JIT, if enabled in the kernel configuration. - R10000 ll/sc weak ordering - Loongson-3 ll/sc weak ordering - Loongson-2F jump hang The Loongson-2F nop errata is implemented in uasm, which the JIT uses, so no additional mitigations are needed for that. Signed-off-by:
Jiaxun Yang <jiaxun.yang@flygoat.com> Link: https://lore.kernel.org/bpf/20211005165408.2305108-6-johan.almbladh@anyfinetworks.com
-
Johan Almbladh authored
This is an implementation on of an eBPF JIT for 64-bit MIPS III-V and MIPS64r1-r6. It uses the same framework introduced by the 32-bit JIT. Signed-off-by:
-
Johan Almbladh authored
This is an implementation of an eBPF JIT for 32-bit MIPS I-V and MIPS32. The implementation supports all 32-bit and 64-bit ALU and JMP operations, including the recently-added atomics. 64-bit div/mod and 64-bit atomics are implemented using function calls to math64 and atomic64 functions, respectively. All 32-bit operations are implemented natively by the JIT, except if the CPU lacks ll/sc instructions. Register mapping ================ All 64-bit eBPF registers are mapped to native 32-bit MIPS register pairs, and does not use any stack scratch space for register swapping. This means that all eBPF register data is kept in CPU registers all the time, and this simplifies the register management a lot. It also reduces the JIT's pressure on temporary registers since we do not have to move data around. Native register pairs are ordered according to CPU endiannes, following the O32 calling convention for passing 64-bit arguments and return values. The eBPF return value, arguments and callee-saved registers are mapped to their native MIPS equivalents. Since the 32 highest bits in the eBPF FP (frame pointer) register are always zero, only one general-purpose register is actually needed for the mapping. The MIPS fp register is used for this purpose. The high bits are mapped to MIPS register r0. This saves us one CPU register, which is much needed for temporaries, while still allowing us to treat the R10 (FP) register just like any other eBPF register in the JIT. The MIPS gp (global pointer) and at (assembler temporary) registers are used as internal temporary registers for constant blinding. CPU registers t6-t9 are used internally by the JIT when constructing more complex 64-bit operations. This is precisely what is needed - two registers to store an operand value, and two more as scratch registers when performing the operation. The register mapping is shown below. R0 - $v1, $v0 return value R1 - $a1, $a0 argument 1, passed in registers R2 - $a3, $a2 argument 2, passed in registers R3 - $t1, $t0 argument 3, passed on stack R4 - $t3, $t2 argument 4, passed on stack R5 - $t4, $t3 argument 5, passed on stack R6 - $s1, $s0 callee-saved R7 - $s3, $s2 callee-saved R8 - $s5, $s4 callee-saved R9 - $s7, $s6 callee-saved FP - $r0, $fp 32-bit frame pointer AX - $gp, $at constant-blinding $t6 - $t9 unallocated, JIT temporaries Jump offsets ============ The JIT tries to map all conditional JMP operations to MIPS conditional PC-relative branches. The MIPS branch offset field is 18 bits, in bytes, which is equivalent to the eBPF 16-bit instruction offset. However, since the JIT may emit more than one CPU instruction per eBPF instruction, the field width may overflow. If that happens, the JIT converts the long conditional jump to a short PC-relative branch with the condition inverted, jumping over a long unconditional absolute jmp (j). This conversion will change the instruction offset mapping used for jumps, and may in turn result in more branch offset overflows. The JIT therefore dry-runs the translation until no more branches are converted and the offsets do not change anymore. There is an upper bound on this of course, and if the JIT hits that limit, the last two iterations are run with all branches being converted. Tail call count =============== The current tail call count is stored in the 16-byte area of the caller's stack frame that is reserved for the callee in the o32 ABI. The value is initialized in the prologue, and propagated to the tail-callee by skipping the initialization instructions when emitting the tail call. Signed-off-by: -
Johan Almbladh authored
This patch implements a workaround for the Loongson-2F nop in generated, code, if the existing option CONFIG_CPU_NOP_WORKAROUND is set. Before, the binutils option -mfix-loongson2f-nop was enabled, but no workaround was done when emitting MIPS code. Now, the nop pseudo instruction is emitted as "or ax,ax,zero" instead of the default "sll zero,zero,0". This is consistent with the workaround implemented by binutils. Signed-off-by:
-
Tony Ambardar authored
Enable the 'muhu' instruction, complementing the existing 'mulu', needed to implement a MIPS32 BPF JIT. Also fix a typo in the existing definition of 'dmulu'. Signed-off-by:
Tony Ambardar <Tony.Ambardar@gmail.com> Signed-off-by:
-
Andrii Nakryiko authored
Add a test that validates that btf__add_btf() API is correctly copying all the types from the source BTF into destination BTF object and adjusts type IDs and string offsets properly. Signed-off-by:
-
Andrii Nakryiko authored
Next patch will need to reuse BTF generation logic, which tests every supported BTF kind, for testing btf__add_btf() APIs. So restructure existing selftests and make it as a single subtest that uses bulk VALIDATE_RAW_BTF() macro for raw BTF dump checking. Signed-off-by:
-
Andrii Nakryiko authored
Add a bulk copying api, btf__add_btf(), that speeds up and simplifies appending entire contents of one BTF object to another one, taking care of copying BTF type data, adjusting resulting BTF type IDs according to their new locations in the destination BTF object, as well as copying and deduplicating all the referenced strings and updating all the string offsets in new BTF types as appropriate. This API is intended to be used from tools that are generating and otherwise manipulating BTFs generically, such as pahole. In pahole's case, this API is useful for speeding up parallelized BTF encoding, as it allows pahole to offload all the intricacies of BTF type copying to libbpf and handle the parallelization aspects of the process. Signed-off-by:
-
Jie Meng authored
Instead of unconditionally performing push/pop on %rax/%rdx in case of division/modulo, we can save a few bytes in case of destination register being either BPF r0 (%rax) or r3 (%rdx) since the result is written in there anyway. Also, we do not need to copy the source to %r11 unless the source is either %rax, %rdx or an immediate. For example, before the patch: 22: push %rax 23: push %rdx 24: mov %rsi,%r11 27: xor %edx,%edx 29: div %r11 2c: mov %rax,%r11 2f: pop %rdx 30: pop %rax 31: mov %r11,%rax After: 22: push %rdx 23: xor %edx,%edx 25: div %rsi 28: pop %rdx Signed-off-by:
Alexei Starovoitov <ast@kernel.org> Signed-off-by:
-
Andrey Ignatov authored
Similarly to 09772d92 ("bpf: avoid retpoline for lookup/update/delete calls on maps") and 84430d42 ("bpf, verifier: avoid retpoline for map push/pop/peek operation") avoid indirect call while calling bpf_for_each_map_elem. Before (a program fragment): ; if (rules_map) { 142: (15) if r4 == 0x0 goto pc+8 143: (bf) r3 = r10 ; bpf_for_each_map_elem(rules_map, process_each_rule, &ctx, 0); 144: (07) r3 += -24 145: (bf) r1 = r4 146: (18) r2 = subprog[+5] 148: (b7) r4 = 0 149: (85) call bpf_for_each_map_elem#143680 <-- indirect call via helper After (same program fragment): ; if (rules_map) { 142: (15) if r4 == 0x0 goto pc+8 143: (bf) r3 = r10 ; bpf_for_each_map_elem(rules_map, process_each_rule, &ctx, 0); 144: (07) r3 += -24 145: (bf) r1 = r4 146: (18) r2 = subprog[+5] 148: (b7) r4 = 0 149: (85) call bpf_for_each_array_elem#170336 <-- direct call On a benchmark that calls bpf_for_each_map_elem() once and does many other things (mostly checking fields in skb) with CONFIG_RETPOLINE=y it makes program faster. Before: ============================================================================ Benchmark.cpp time/iter iters/s ============================================================================ IngressMatchByRemoteEndpoint 80.78ns 12.38M IngressMatchByRemoteIP 80.66ns 12.40M IngressMatchByRemotePort 80.87ns 12.37M After: ============================================================================ Benchmark.cpp time/iter iters/s ============================================================================ IngressMatchByRemoteEndpoint 73.49ns 13.61M IngressMatchByRemoteIP 71.48ns 13.99M IngressMatchByRemotePort 70.39ns 14.21M Signed-off-by:
Andrey Ignatov <rdna@fb.com> Signed-off-by:
-
