1. 19 Nov, 2021 1 commit
    • John Fastabend's avatar
      bpf, sockmap: Attach map progs to psock early for feature probes · 38207a5e
      John Fastabend authored
      When a TCP socket is added to a sock map we look at the programs attached
      to the map to determine what proto op hooks need to be changed. Before
      the patch in the 'fixes' tag there were only two categories -- the empty
      set of programs or a TX policy. In any case the base set handled the
      receive case.
      
      After the fix we have an optimized program for receive that closes a small,
      but possible, race on receive. This program is loaded only when the map the
      psock is being added to includes a RX policy. Otherwise, the race is not
      possible so we don't need to handle the race condition.
      
      In order for the call to sk_psock_init() to correctly evaluate the above
      conditions all progs need to be set in the psock before the call. However,
      in the current code this is not the case. We end up evaluating the
      requirements on the old prog state. If your psock is attached to multiple
      maps -- for example a tx map and rx map -- then the second update would pull
      in the correct maps. But, the other pattern with a single rx enabled map
      the correct receive hooks are not used. The result is the race fixed by the
      patch in the fixes tag below may still be seen in this case.
      
      To fix we simply set all psock->progs before doing the call into
      sock_map_init(). With this the init() call gets the full list of programs
      and chooses the correct proto ops on the first iteration instead of
      requiring the second update to pull them in. This fixes the race case when
      only a single map is used.
      
      Fixes: c5d2177a ("bpf, sockmap: Fix race in ingress receive verdict with redirect to self")
      Signed-off-by: default avatarJohn Fastabend <john.fastabend@gmail.com>
      Signed-off-by: default avatarDaniel Borkmann <daniel@iogearbox.net>
      Link: https://lore.kernel.org/bpf/20211119181418.353932-2-john.fastabend@gmail.com
      38207a5e
  2. 17 Nov, 2021 11 commits
  3. 16 Nov, 2021 7 commits
    • David S. Miller's avatar
      Merge branch '40GbE' of git://git.kernel.org/pub/scm/linux/kernel/git/tnguy/net-queue · 848e5d66
      David S. Miller authored
      Tony Nguyen says:
      
      ====================
      Intel Wired LAN Driver Updates 2021-11-15
      
      This series contains updates to iavf driver only.
      
      Mateusz adds a wait for reset completion when changing queue count which
      could otherwise cause issues with VF reset.
      
      Nick adds a null check for vf_res in iavf_fix_features(), corrects
      ordering of function calls to resolve dependency issues, and prevents
      possible freeing of a lock which isn't being held.
      
      Piotr fixes logic that did not allow setting all multicast mode without
      promiscuous mode.
      
      Jake prevents possible accidental freeing of filter structure.
      
      Mitch adds null checks for key and indir parameters in iavf_get_rxfh().
      
      Surabhi adds an additional check that would, previously, cause the driver
      to print a false error due to values obtained while the VF is in reset.
      
      Grzegorz prevents a queue request of 0 which would cause queue count to
      reset to default values.
      
      Akeem restores VLAN filters when bringing the interface back up.
      ====================
      Signed-off-by: default avatarDavid S. Miller <davem@davemloft.net>
      848e5d66
    • Cong Wang's avatar
      udp: Validate checksum in udp_read_sock() · 099f896f
      Cong Wang authored
      It turns out the skb's in sock receive queue could have bad checksums, as
      both ->poll() and ->recvmsg() validate checksums. We have to do the same
      for ->read_sock() path too before they are redirected in sockmap.
      
      Fixes: d7f57118 ("udp: Implement ->read_sock() for sockmap")
      Reported-by: default avatarDaniel Borkmann <daniel@iogearbox.net>
      Signed-off-by: default avatarCong Wang <cong.wang@bytedance.com>
      Signed-off-by: default avatarDaniel Borkmann <daniel@iogearbox.net>
      Acked-by: default avatarJohn Fastabend <john.fastabend@gmail.com>
      Link: https://lore.kernel.org/bpf/20211115044006.26068-1-xiyou.wangcong@gmail.com
      099f896f
    • Daniel Borkmann's avatar
      bpf: Fix toctou on read-only map's constant scalar tracking · 353050be
      Daniel Borkmann authored
      Commit a23740ec ("bpf: Track contents of read-only maps as scalars") is
      checking whether maps are read-only both from BPF program side and user space
      side, and then, given their content is constant, reading out their data via
      map->ops->map_direct_value_addr() which is then subsequently used as known
      scalar value for the register, that is, it is marked as __mark_reg_known()
      with the read value at verification time. Before a23740ec, the register
      content was marked as an unknown scalar so the verifier could not make any
      assumptions about the map content.
      
      The current implementation however is prone to a TOCTOU race, meaning, the
      value read as known scalar for the register is not guaranteed to be exactly
      the same at a later point when the program is executed, and as such, the
      prior made assumptions of the verifier with regards to the program will be
      invalid which can cause issues such as OOB access, etc.
      
      While the BPF_F_RDONLY_PROG map flag is always fixed and required to be
      specified at map creation time, the map->frozen property is initially set to
      false for the map given the map value needs to be populated, e.g. for global
      data sections. Once complete, the loader "freezes" the map from user space
      such that no subsequent updates/deletes are possible anymore. For the rest
      of the lifetime of the map, this freeze one-time trigger cannot be undone
      anymore after a successful BPF_MAP_FREEZE cmd return. Meaning, any new BPF_*
      cmd calls which would update/delete map entries will be rejected with -EPERM
      since map_get_sys_perms() removes the FMODE_CAN_WRITE permission. This also
      means that pending update/delete map entries must still complete before this
      guarantee is given. This corner case is not an issue for loaders since they
      create and prepare such program private map in successive steps.
      
      However, a malicious user is able to trigger this TOCTOU race in two different
      ways: i) via userfaultfd, and ii) via batched updates. For i) userfaultfd is
      used to expand the competition interval, so that map_update_elem() can modify
      the contents of the map after map_freeze() and bpf_prog_load() were executed.
      This works, because userfaultfd halts the parallel thread which triggered a
      map_update_elem() at the time where we copy key/value from the user buffer and
      this already passed the FMODE_CAN_WRITE capability test given at that time the
      map was not "frozen". Then, the main thread performs the map_freeze() and
      bpf_prog_load(), and once that had completed successfully, the other thread
      is woken up to complete the pending map_update_elem() which then changes the
      map content. For ii) the idea of the batched update is similar, meaning, when
      there are a large number of updates to be processed, it can increase the
      competition interval between the two. It is therefore possible in practice to
      modify the contents of the map after executing map_freeze() and bpf_prog_load().
      
      One way to fix both i) and ii) at the same time is to expand the use of the
      map's map->writecnt. The latter was introduced in fc970227 ("bpf: Add mmap()
      support for BPF_MAP_TYPE_ARRAY") and further refined in 1f6cb19b ("bpf:
      Prevent re-mmap()'ing BPF map as writable for initially r/o mapping") with
      the rationale to make a writable mmap()'ing of a map mutually exclusive with
      read-only freezing. The counter indicates writable mmap() mappings and then
      prevents/fails the freeze operation. Its semantics can be expanded beyond
      just mmap() by generally indicating ongoing write phases. This would essentially
      span any parallel regular and batched flavor of update/delete operation and
      then also have map_freeze() fail with -EBUSY. For the check_mem_access() in
      the verifier we expand upon the bpf_map_is_rdonly() check ensuring that all
      last pending writes have completed via bpf_map_write_active() test. Once the
      map->frozen is set and bpf_map_write_active() indicates a map->writecnt of 0
      only then we are really guaranteed to use the map's data as known constants.
      For map->frozen being set and pending writes in process of still being completed
      we fall back to marking that register as unknown scalar so we don't end up
      making assumptions about it. With this, both TOCTOU reproducers from i) and
      ii) are fixed.
      
      Note that the map->writecnt has been converted into a atomic64 in the fix in
      order to avoid a double freeze_mutex mutex_{un,}lock() pair when updating
      map->writecnt in the various map update/delete BPF_* cmd flavors. Spanning
      the freeze_mutex over entire map update/delete operations in syscall side
      would not be possible due to then causing everything to be serialized.
      Similarly, something like synchronize_rcu() after setting map->frozen to wait
      for update/deletes to complete is not possible either since it would also
      have to span the user copy which can sleep. On the libbpf side, this won't
      break d66562fb ("libbpf: Add BPF object skeleton support") as the
      anonymous mmap()-ed "map initialization image" is remapped as a BPF map-backed
      mmap()-ed memory where for .rodata it's non-writable.
      
      Fixes: a23740ec ("bpf: Track contents of read-only maps as scalars")
      Reported-by: w1tcher.bupt@gmail.com
      Signed-off-by: default avatarDaniel Borkmann <daniel@iogearbox.net>
      Acked-by: default avatarAndrii Nakryiko <andrii@kernel.org>
      Signed-off-by: default avatarAlexei Starovoitov <ast@kernel.org>
      353050be
    • Alexander Lobakin's avatar
      samples/bpf: Fix build error due to -isystem removal · 6060a6cb
      Alexander Lobakin authored
      Since recent Kbuild updates we no longer include files from compiler
      directories. However, samples/bpf/hbm_kern.h hasn't been tuned for
      this (LLVM 13):
      
        CLANG-bpf  samples/bpf/hbm_out_kern.o
      In file included from samples/bpf/hbm_out_kern.c:55:
      samples/bpf/hbm_kern.h:12:10: fatal error: 'stddef.h' file not found
               ^~~~~~~~~~
      1 error generated.
        CLANG-bpf  samples/bpf/hbm_edt_kern.o
      In file included from samples/bpf/hbm_edt_kern.c:53:
      samples/bpf/hbm_kern.h:12:10: fatal error: 'stddef.h' file not found
               ^~~~~~~~~~
      1 error generated.
      
      It is enough to just drop both stdbool.h and stddef.h from includes
      to fix those.
      
      Fixes: 04e85bbf ("isystem: delete global -isystem compile option")
      Signed-off-by: default avatarAlexander Lobakin <alexandr.lobakin@intel.com>
      Signed-off-by: default avatarDaniel Borkmann <daniel@iogearbox.net>
      Reviewed-by: default avatarMichal Swiatkowski <michal.swiatkowski@linux.intel.com>
      Link: https://lore.kernel.org/bpf/20211115130741.3584-1-alexandr.lobakin@intel.comSigned-off-by: default avatarAlexei Starovoitov <ast@kernel.org>
      6060a6cb
    • Alexei Starovoitov's avatar
      Merge branch 'Forbid bpf_ktime_get_coarse_ns and bpf_timer_* in tracing progs' · 9e4dc892
      Alexei Starovoitov authored
      Dmitrii Banshchikov says:
      
      ====================
      
      Various locking issues are possible with bpf_ktime_get_coarse_ns() and
      bpf_timer_* set of helpers.
      
      syzbot found a locking issue with bpf_ktime_get_coarse_ns() helper executed in
      BPF_PROG_TYPE_PERF_EVENT prog type - [1]. The issue is possible because the
      helper uses non fast version of time accessor that isn't safe for any context.
      The helper was added because it provided performance benefits in comparison to
      bpf_ktime_get_ns() helper.
      
      A similar locking issue is possible with bpf_timer_* set of helpers when used
      in tracing progs.
      
      The solution is to restrict use of the helpers in tracing progs.
      
      In the [1] discussion it was stated that bpf_spin_lock related helpers shall
      also be excluded for tracing progs. The verifier has a compatibility check
      between a map and a program. If a tracing program tries to use a map which
      value has struct bpf_spin_lock the verifier fails that is why bpf_spin_lock is
      already restricted.
      
      Patch 1 restricts helpers
      Patch 2 adds tests
      
      v1 -> v2:
       * Limit the helpers via func proto getters instead of allowed callback
       * Add note about helpers' restrictions to linux/bpf.h
       * Add Fixes tag
       * Remove extra \0 from btf_str_sec
       * Beside asm tests add prog tests
       * Trim CC
      
      1. https://lore.kernel.org/all/00000000000013aebd05cff8e064@google.com/
      ====================
      Signed-off-by: default avatarAlexei Starovoitov <ast@kernel.org>
      9e4dc892
    • Dmitrii Banshchikov's avatar
      selftests/bpf: Add tests for restricted helpers · e60e6962
      Dmitrii Banshchikov authored
      This patch adds tests that bpf_ktime_get_coarse_ns(), bpf_timer_* and
      bpf_spin_lock()/bpf_spin_unlock() helpers are forbidden in tracing progs
      as their use there may result in various locking issues.
      Signed-off-by: default avatarDmitrii Banshchikov <me@ubique.spb.ru>
      Signed-off-by: default avatarAlexei Starovoitov <ast@kernel.org>
      Link: https://lore.kernel.org/bpf/20211113142227.566439-3-me@ubique.spb.ru
      e60e6962
    • Dmitrii Banshchikov's avatar
      bpf: Forbid bpf_ktime_get_coarse_ns and bpf_timer_* in tracing progs · 5e0bc308
      Dmitrii Banshchikov authored
      Use of bpf_ktime_get_coarse_ns() and bpf_timer_* helpers in tracing
      progs may result in locking issues.
      
      bpf_ktime_get_coarse_ns() uses ktime_get_coarse_ns() time accessor that
      isn't safe for any context:
      ======================================================
      WARNING: possible circular locking dependency detected
      5.15.0-syzkaller #0 Not tainted
      ------------------------------------------------------
      syz-executor.4/14877 is trying to acquire lock:
      ffffffff8cb30008 (tk_core.seq.seqcount){----}-{0:0}, at: ktime_get_coarse_ts64+0x25/0x110 kernel/time/timekeeping.c:2255
      
      but task is already holding lock:
      ffffffff90dbf200 (&obj_hash[i].lock){-.-.}-{2:2}, at: debug_object_deactivate+0x61/0x400 lib/debugobjects.c:735
      
      which lock already depends on the new lock.
      
      the existing dependency chain (in reverse order) is:
      
      -> #1 (&obj_hash[i].lock){-.-.}-{2:2}:
             lock_acquire+0x19f/0x4d0 kernel/locking/lockdep.c:5625
             __raw_spin_lock_irqsave include/linux/spinlock_api_smp.h:110 [inline]
             _raw_spin_lock_irqsave+0xd1/0x120 kernel/locking/spinlock.c:162
             __debug_object_init+0xd9/0x1860 lib/debugobjects.c:569
             debug_hrtimer_init kernel/time/hrtimer.c:414 [inline]
             debug_init kernel/time/hrtimer.c:468 [inline]
             hrtimer_init+0x20/0x40 kernel/time/hrtimer.c:1592
             ntp_init_cmos_sync kernel/time/ntp.c:676 [inline]
             ntp_init+0xa1/0xad kernel/time/ntp.c:1095
             timekeeping_init+0x512/0x6bf kernel/time/timekeeping.c:1639
             start_kernel+0x267/0x56e init/main.c:1030
             secondary_startup_64_no_verify+0xb1/0xbb
      
      -> #0 (tk_core.seq.seqcount){----}-{0:0}:
             check_prev_add kernel/locking/lockdep.c:3051 [inline]
             check_prevs_add kernel/locking/lockdep.c:3174 [inline]
             validate_chain+0x1dfb/0x8240 kernel/locking/lockdep.c:3789
             __lock_acquire+0x1382/0x2b00 kernel/locking/lockdep.c:5015
             lock_acquire+0x19f/0x4d0 kernel/locking/lockdep.c:5625
             seqcount_lockdep_reader_access+0xfe/0x230 include/linux/seqlock.h:103
             ktime_get_coarse_ts64+0x25/0x110 kernel/time/timekeeping.c:2255
             ktime_get_coarse include/linux/timekeeping.h:120 [inline]
             ktime_get_coarse_ns include/linux/timekeeping.h:126 [inline]
             ____bpf_ktime_get_coarse_ns kernel/bpf/helpers.c:173 [inline]
             bpf_ktime_get_coarse_ns+0x7e/0x130 kernel/bpf/helpers.c:171
             bpf_prog_a99735ebafdda2f1+0x10/0xb50
             bpf_dispatcher_nop_func include/linux/bpf.h:721 [inline]
             __bpf_prog_run include/linux/filter.h:626 [inline]
             bpf_prog_run include/linux/filter.h:633 [inline]
             BPF_PROG_RUN_ARRAY include/linux/bpf.h:1294 [inline]
             trace_call_bpf+0x2cf/0x5d0 kernel/trace/bpf_trace.c:127
             perf_trace_run_bpf_submit+0x7b/0x1d0 kernel/events/core.c:9708
             perf_trace_lock+0x37c/0x440 include/trace/events/lock.h:39
             trace_lock_release+0x128/0x150 include/trace/events/lock.h:58
             lock_release+0x82/0x810 kernel/locking/lockdep.c:5636
             __raw_spin_unlock_irqrestore include/linux/spinlock_api_smp.h:149 [inline]
             _raw_spin_unlock_irqrestore+0x75/0x130 kernel/locking/spinlock.c:194
             debug_hrtimer_deactivate kernel/time/hrtimer.c:425 [inline]
             debug_deactivate kernel/time/hrtimer.c:481 [inline]
             __run_hrtimer kernel/time/hrtimer.c:1653 [inline]
             __hrtimer_run_queues+0x2f9/0xa60 kernel/time/hrtimer.c:1749
             hrtimer_interrupt+0x3b3/0x1040 kernel/time/hrtimer.c:1811
             local_apic_timer_interrupt arch/x86/kernel/apic/apic.c:1086 [inline]
             __sysvec_apic_timer_interrupt+0xf9/0x270 arch/x86/kernel/apic/apic.c:1103
             sysvec_apic_timer_interrupt+0x8c/0xb0 arch/x86/kernel/apic/apic.c:1097
             asm_sysvec_apic_timer_interrupt+0x12/0x20
             __raw_spin_unlock_irqrestore include/linux/spinlock_api_smp.h:152 [inline]
             _raw_spin_unlock_irqrestore+0xd4/0x130 kernel/locking/spinlock.c:194
             try_to_wake_up+0x702/0xd20 kernel/sched/core.c:4118
             wake_up_process kernel/sched/core.c:4200 [inline]
             wake_up_q+0x9a/0xf0 kernel/sched/core.c:953
             futex_wake+0x50f/0x5b0 kernel/futex/waitwake.c:184
             do_futex+0x367/0x560 kernel/futex/syscalls.c:127
             __do_sys_futex kernel/futex/syscalls.c:199 [inline]
             __se_sys_futex+0x401/0x4b0 kernel/futex/syscalls.c:180
             do_syscall_x64 arch/x86/entry/common.c:50 [inline]
             do_syscall_64+0x44/0xd0 arch/x86/entry/common.c:80
             entry_SYSCALL_64_after_hwframe+0x44/0xae
      
      There is a possible deadlock with bpf_timer_* set of helpers:
      hrtimer_start()
        lock_base();
        trace_hrtimer...()
          perf_event()
            bpf_run()
              bpf_timer_start()
                hrtimer_start()
                  lock_base()         <- DEADLOCK
      
      Forbid use of bpf_ktime_get_coarse_ns() and bpf_timer_* helpers in
      BPF_PROG_TYPE_KPROBE, BPF_PROG_TYPE_TRACEPOINT, BPF_PROG_TYPE_PERF_EVENT
      and BPF_PROG_TYPE_RAW_TRACEPOINT prog types.
      
      Fixes: d0551261 ("bpf: Add bpf_ktime_get_coarse_ns helper")
      Fixes: b00628b1 ("bpf: Introduce bpf timers.")
      Reported-by: syzbot+43fd005b5a1b4d10781e@syzkaller.appspotmail.com
      Signed-off-by: default avatarDmitrii Banshchikov <me@ubique.spb.ru>
      Signed-off-by: default avatarAlexei Starovoitov <ast@kernel.org>
      Link: https://lore.kernel.org/bpf/20211113142227.566439-2-me@ubique.spb.ru
      5e0bc308
  4. 15 Nov, 2021 21 commits