• Andrii Nakryiko's avatar
    bpf: add special internal-only MOV instruction to resolve per-CPU addrs · 7bdbf744
    Andrii Nakryiko authored
    Add a new BPF instruction for resolving absolute addresses of per-CPU
    data from their per-CPU offsets. This instruction is internal-only and
    users are not allowed to use them directly. They will only be used for
    internal inlining optimizations for now between BPF verifier and BPF JITs.
    
    We use a special BPF_MOV | BPF_ALU64 | BPF_X form with insn->off field
    set to BPF_ADDR_PERCPU = -1. I used negative offset value to distinguish
    them from positive ones used by user-exposed instructions.
    
    Such instruction performs a resolution of a per-CPU offset stored in
    a register to a valid kernel address which can be dereferenced. It is
    useful in any use case where absolute address of a per-CPU data has to
    be resolved (e.g., in inlining bpf_map_lookup_elem()).
    
    BPF disassembler is also taught to recognize them to support dumping
    final BPF assembly code (non-JIT'ed version).
    
    Add arch-specific way for BPF JITs to mark support for this instructions.
    
    This patch also adds support for these instructions in x86-64 BPF JIT.
    Signed-off-by: default avatarAndrii Nakryiko <andrii@kernel.org>
    Acked-by: default avatarJohn Fastabend <john.fastabend@gmail.com>
    Link: https://lore.kernel.org/r/20240402021307.1012571-2-andrii@kernel.orgSigned-off-by: default avatarAlexei Starovoitov <ast@kernel.org>
    7bdbf744
core.c 77.4 KB