• Daniel Borkmann's avatar
    bpf: Constant map key tracking for prog array pokes · d2e4c1e6
    Daniel Borkmann authored
    Add tracking of constant keys into tail call maps. The signature of
    bpf_tail_call_proto is that arg1 is ctx, arg2 map pointer and arg3
    is a index key. The direct call approach for tail calls can be enabled
    if the verifier asserted that for all branches leading to the tail call
    helper invocation, the map pointer and index key were both constant
    and the same.
    
    Tracking of map pointers we already do from prior work via c93552c4
    ("bpf: properly enforce index mask to prevent out-of-bounds speculation")
    and 09772d92 ("bpf: avoid retpoline for lookup/update/ delete calls
    on maps").
    
    Given the tail call map index key is not on stack but directly in the
    register, we can add similar tracking approach and later in fixup_bpf_calls()
    add a poke descriptor to the progs poke_tab with the relevant information
    for the JITing phase.
    
    We internally reuse insn->imm for the rewritten BPF_JMP | BPF_TAIL_CALL
    instruction in order to point into the prog's poke_tab, and keep insn->imm
    as 0 as indicator that current indirect tail call emission must be used.
    Note that publishing to the tracker must happen at the end of fixup_bpf_calls()
    since adding elements to the poke_tab reallocates its memory, so we need
    to wait until its in final state.
    
    Future work can generalize and add similar approach to optimize plain
    array map lookups. Difference there is that we need to look into the key
    value that sits on stack. For clarity in bpf_insn_aux_data, map_state
    has been renamed into map_ptr_state, so we get map_{ptr,key}_state as
    trackers.
    Signed-off-by: default avatarDaniel Borkmann <daniel@iogearbox.net>
    Signed-off-by: default avatarAlexei Starovoitov <ast@kernel.org>
    Acked-by: default avatarAndrii Nakryiko <andriin@fb.com>
    Link: https://lore.kernel.org/bpf/e8db37f6b2ae60402fa40216c96738ee9b316c32.1574452833.git.daniel@iogearbox.net
    d2e4c1e6
verifier.c 280 KB