1. 14 Feb, 2023 1 commit
    • Daniel Bristot de Oliveira's avatar
      rtla: Add hwnoise tool · 1f428356
      Daniel Bristot de Oliveira authored
      The hwnoise tool is a special mode for the osnoise top tool.
      
      hwnoise dispatches the osnoise tracer and displays a summary of the noise.
      The difference is that it runs the tracer with the OSNOISE_IRQ_DISABLE
      option set, thus only allowing only hardware-related noise, resulting in
      a simplified output. hwnoise has the same features of osnoise.
      
      An example of the tool's output:
      
       # rtla hwnoise -c 1-11 -T 1 -d 10m -q
                                                 Hardware-related Noise
       duration:   0 00:10:00 | time is in us
       CPU Period       Runtime        Noise  % CPU Aval   Max Noise   Max Single          HW          NMI
         1 #599       599000000          138    99.99997           3            3           4           74
         2 #599       599000000           85    99.99998           3            3           4           75
         3 #599       599000000           86    99.99998           4            3           6           75
         4 #599       599000000           81    99.99998           4            4           2           75
         5 #599       599000000           85    99.99998           2            2           2           75
      
      Link: https://lkml.kernel.org/r/2d6f49a6f3a4f8b51b2c806458b1cff71ad4d014.1675805361.git.bristot@kernel.orgSigned-off-by: default avatarDaniel Bristot de Oliveira <bristot@kernel.org>
      Cc: Daniel Bristot de Oliveira <bristot@kernel.org>
      Cc: Juri Lelli <juri.lelli@redhat.com>
      Cc: Clark Williams <williams@redhat.com>
      Cc: Bagas Sanjaya <bagasdotme@gmail.com>
      Cc: Jonathan Corbet <corbet@lwn.net>
      Signed-off-by: default avatarSteven Rostedt (Google) <rostedt@goodmis.org>
      1f428356
  2. 02 Feb, 2023 3 commits
    • Daniel Bristot de Oliveira's avatar
      Documentation/rtla: Add timerlat-top auto-analysis options · ce6cc6f7
      Daniel Bristot de Oliveira authored
      Add the new options to the man page, as well as updating the
      example to include the new output.
      
      Link: https://lkml.kernel.org/r/a3f5fb49432a55c3323b18725fc6e702f16ccc79.1675179318.git.bristot@kernel.org
      
      Cc: Daniel Bristot de Oliveira <bristot@kernel.org>
      Cc: Jonathan Corbet <corbet@lwn.net>
      Signed-off-by: default avatarDaniel Bristot de Oliveira <bristot@kernel.org>
      Signed-off-by: default avatarSteven Rostedt (Google) <rostedt@goodmis.org>
      ce6cc6f7
    • Daniel Bristot de Oliveira's avatar
      rtla/timerlat: Add auto-analysis support to timerlat top · 5def33df
      Daniel Bristot de Oliveira authored
      Currently, timerlat top displays the timerlat tracer latency results, saving
      the intuitive timerlat trace for the developer to analyze.
      
      This patch goes a step forward in the automaton of the scheduling latency
      analysis by providing a summary of the root cause of a latency higher than
      the passed "stop tracing" parameter if the trace stops.
      
      The output is intuitive enough for non-expert users to have a general idea
      of the root cause by looking at each factor's contribution percentage while
      keeping the technical detail in the output for more expert users to start
      an in dept debug or to correlate a root cause with an existing one.
      
      The terminology is in line with recent industry and academic publications
      to facilitate the understanding of both audiences.
      
      Here is one example of tool output:
       ----------------------------------------- %< -----------------------------------------------------
        # taskset -c 0 timerlat -a 40 -c 1-23 -q
                                           Timer Latency
          0 00:00:12   |          IRQ Timer Latency (us)        |         Thread Timer Latency (us)
        CPU COUNT      |      cur       min       avg       max |      cur       min       avg       max
          1 #12322     |        0         0         1        15 |       10         3         9        31
          2 #12322     |        3         0         1        12 |       10         3         9        23
          3 #12322     |        1         0         1        21 |        8         2         8        34
          4 #12322     |        1         0         1        17 |       10         2        11        33
          5 #12322     |        0         0         1        12 |        8         3         8        25
          6 #12322     |        1         0         1        14 |       16         3        11        35
          7 #12322     |        0         0         1        14 |        9         2         8        29
          8 #12322     |        1         0         1        22 |        9         3         9        34
          9 #12322     |        0         0         1        14 |        8         2         8        24
         10 #12322     |        1         0         0        12 |        9         3         8        24
         11 #12322     |        0         0         0        15 |        6         2         7        29
         12 #12321     |        1         0         0        13 |        5         3         8        23
         13 #12319     |        0         0         1        14 |        9         3         9        26
         14 #12321     |        1         0         0        13 |        6         2         8        24
         15 #12321     |        1         0         1        15 |       12         3        11        27
         16 #12318     |        0         0         1        13 |        7         3        10        24
         17 #12319     |        0         0         1        13 |       11         3         9        25
         18 #12318     |        0         0         0        12 |        8         2         8        20
         19 #12319     |        0         0         1        18 |       10         2         9        28
         20 #12317     |        0         0         0        20 |        9         3         8        34
         21 #12318     |        0         0         0        13 |        8         3         8        28
         22 #12319     |        0         0         1        11 |        8         3        10        22
         23 #12320     |       28         0         1        28 |       41         3        11        41
        rtla timerlat hit stop tracing
        ## CPU 23 hit stop tracing, analyzing it ##
        IRQ handler delay:				      	    27.49 us (65.52 %)
        IRQ latency:						    28.13 us
        Timerlat IRQ duration:				     9.59 us (22.85 %)
        Blocking thread:					     3.79 us (9.03 %)
      			objtool:49256    		     3.79 us
          Blocking thread stacktrace
      		-> timerlat_irq
      		-> __hrtimer_run_queues
      		-> hrtimer_interrupt
      		-> __sysvec_apic_timer_interrupt
      		-> sysvec_apic_timer_interrupt
      		-> asm_sysvec_apic_timer_interrupt
      		-> _raw_spin_unlock_irqrestore
      		-> cgroup_rstat_flush_locked
      		-> cgroup_rstat_flush_irqsafe
      		-> mem_cgroup_flush_stats
      		-> mem_cgroup_wb_stats
      		-> balance_dirty_pages
      		-> balance_dirty_pages_ratelimited_flags
      		-> btrfs_buffered_write
      		-> btrfs_do_write_iter
      		-> vfs_write
      		-> __x64_sys_pwrite64
      		-> do_syscall_64
      		-> entry_SYSCALL_64_after_hwframe
        ------------------------------------------------------------------------
          Thread latency:					    41.96 us (100%)
      
        The system has exit from idle latency!
          Max timerlat IRQ latency from idle: 17.48 us in cpu 4
        Saving trace to timerlat_trace.txt
       ----------------------------------------- >% -----------------------------------------------------
      
      In this case, the major factor was the delay suffered by the IRQ handler
      that handles timerlat wakeup: 65.52 %. This can be caused by the
      current thread masking interrupts, which can be seen in the blocking
      thread stacktrace: the current thread (objtool:49256) disabled interrupts
      via raw spin lock operations inside mem cgroup, while doing write
      syscall in a btrfs file system.
      
      A simple search for the function name on Google shows that this is
      a legit case for disabling the interrupts:
      
        cgroup: Use irqsave in cgroup_rstat_flush_locked()
        lore.kernel.org/linux-mm/20220301122143.1521823-2-bigeasy@linutronix.de/
      
      The output also prints other reasons for the latency root cause, such as:
      
        - an IRQ that happened before the IRQ handler that caused delays
        - The interference from NMI, IRQ, Softirq, and Threads
      
      The details about how these factors affect the scheduling latency
      can be found here:
      
         https://bristot.me/demystifying-the-real-time-linux-latency/
      
      Link: https://lkml.kernel.org/r/3d45f40e630317f51ac6d678e2d96d310e495729.1675179318.git.bristot@kernel.org
      
      Cc: Daniel Bristot de Oliveira <bristot@kernel.org>
      Cc: Jonathan Corbet <corbet@lwn.net>
      Signed-off-by: default avatarDaniel Bristot de Oliveira <bristot@kernel.org>
      Signed-off-by: default avatarSteven Rostedt (Google) <rostedt@goodmis.org>
      5def33df
    • Daniel Bristot de Oliveira's avatar
      rtla/timerlat: Add auto-analysis core · 27e348b2
      Daniel Bristot de Oliveira authored
      Currently, timerlat displays a summary of the timerlat tracer results
      saving the trace if the system hits a stop condition.
      
      While this represented a huge step forward, the root cause was not
      that is accessible to non-expert users.
      
      The auto-analysis fulfill this gap by parsing the trace timerlat runs,
      printing an intuitive auto-analysis.
      
      Link: https://lkml.kernel.org/r/1ee073822f6a2cbb33da0c817331d0d4045e837f.1675179318.git.bristot@kernel.org
      
      Cc: Daniel Bristot de Oliveira <bristot@kernel.org>
      Cc: Jonathan Corbet <corbet@lwn.net>
      Signed-off-by: default avatarDaniel Bristot de Oliveira <bristot@kernel.org>
      Signed-off-by: default avatarSteven Rostedt (Google) <rostedt@goodmis.org>
      27e348b2
  3. 01 Feb, 2023 3 commits
  4. 29 Jan, 2023 6 commits
  5. 28 Jan, 2023 7 commits
    • Linus Torvalds's avatar
      Fix up more non-executable files marked executable · c9661827
      Linus Torvalds authored
      Joe found another DT file that shouldn't be executable, and that
      frustrated me enough that I went hunting with this script:
      
          git ls-files -s |
              grep '^100755' |
              cut -f2 |
              xargs grep -L '^#!'
      
      and that found another file that shouldn't have been marked executable
      either, despite being in the scripts directory.
      
      Maybe these two are the last ones at least for now.  But I'm sure we'll
      be back in a few years, fixing things up again.
      
      Fixes: 8c6789f4 ("ASoC: dt-bindings: Add Everest ES8326 audio CODEC")
      Fixes: 4d8e5cd2 ("locking/atomics: Fix scripts/atomic/ script permissions")
      Reported-by: default avatarJoe Perches <joe@perches.com>
      Signed-off-by: default avatarLinus Torvalds <torvalds@linux-foundation.org>
      c9661827
    • Linus Torvalds's avatar
      Merge tag '6.2-rc5-ksmbd-server-fixes' of git://git.samba.org/ksmbd · 2543fdbd
      Linus Torvalds authored
      Pull ksmbd server fixes from Steve French:
       "Four smb3 server fixes, all also for stable:
      
         - fix for signing bug
      
         - fix to more strictly check packet length
      
         - add a max connections parm to limit simultaneous connections
      
         - fix error message flood that can occur with newer Samba xattr
           format"
      
      * tag '6.2-rc5-ksmbd-server-fixes' of git://git.samba.org/ksmbd:
        ksmbd: downgrade ndr version error message to debug
        ksmbd: limit pdu length size according to connection status
        ksmbd: do not sign response to session request for guest login
        ksmbd: add max connections parameter
      2543fdbd
    • Linus Torvalds's avatar
      Merge tag '6.2-rc5-smb3-client-fixes' of git://git.samba.org/sfrench/cifs-2.6 · 5af6ce70
      Linus Torvalds authored
      Pull cifs fix from Steve French:
       "Fix for reconnect oops in smbdirect (RDMA), also is marked for stable"
      
      * tag '6.2-rc5-smb3-client-fixes' of git://git.samba.org/sfrench/cifs-2.6:
        cifs: Fix oops due to uncleared server->smbd_conn in reconnect
      5af6ce70
    • Linus Torvalds's avatar
      Merge tag 'block-6.2-2023-01-27' of git://git.kernel.dk/linux · 90aaef4e
      Linus Torvalds authored
      Pull block fixes from Jens Axboe:
       "Minor tweaks for this release:
      
         - NVMe pull request via Christoph:
              - Flush initial scan_work for async probe (Keith Busch)
              - Fix passthrough csi check (Keith Busch)
              - Fix nvme-fc initialization order (Ross Lagerwall)
      
         - Fix for tearing down non-started device in ublk (Ming)"
      
      * tag 'block-6.2-2023-01-27' of git://git.kernel.dk/linux:
        block: ublk: move ublk_chr_class destroying after devices are removed
        nvme: fix passthrough csi check
        nvme-pci: flush initial scan_work for async probe
        nvme-fc: fix initialization order
      90aaef4e
    • Linus Torvalds's avatar
      Merge tag 'io_uring-6.2-2023-01-27' of git://git.kernel.dk/linux · f851453b
      Linus Torvalds authored
      Pull io_uring fixes from Jens Axboe:
       "Two small fixes for this release:
      
         - Sanitize how async prep is done for drain requests, so we ensure
           that it always gets done (Dylan)
      
         - A ring provided buffer recycling fix for multishot receive (me)"
      
      * tag 'io_uring-6.2-2023-01-27' of git://git.kernel.dk/linux:
        io_uring: always prep_async for drain requests
        io_uring/net: cache provided buffer group value for multishot receives
      f851453b
    • Linus Torvalds's avatar
      Merge tag 'hardening-v6.2-rc6' of git://git.kernel.org/pub/scm/linux/kernel/git/kees/linux · 28cca23d
      Linus Torvalds authored
      Pull hardening fixes from Kees Cook:
      
       - Split slow memcpy tests into MEMCPY_SLOW_KUNIT_TEST
      
       - Reorganize gcc-plugin includes for GCC 13
      
       - Silence bcache memcpy run-time false positive warnings
      
      * tag 'hardening-v6.2-rc6' of git://git.kernel.org/pub/scm/linux/kernel/git/kees/linux:
        bcache: Silence memcpy() run-time false positive warnings
        gcc-plugins: Reorganize gimple includes for GCC 13
        kunit: memcpy: Split slow memcpy tests into MEMCPY_SLOW_KUNIT_TEST
      28cca23d
    • Linus Torvalds's avatar
      Merge tag 'trace-v6.2-rc5' of git://git.kernel.org/pub/scm/linux/kernel/git/trace/linux-trace · d786f0fe
      Linus Torvalds authored
      Pull tracing fixes from Steven Rostedt:
      
       - Fix filter memory leak by calling ftrace_free_filter()
      
       - Initialize trace_printk() earlier so that ftrace_dump_on_oops shows
         data on early crashes.
      
       - Update the outdated instructions in scripts/tracing/ftrace-bisect.sh
      
       - Add lockdep_is_held() to fix lockdep warning
      
       - Add allocation failure check in create_hist_field()
      
       - Don't initialize pointer that gets set right away in enabled_monitors_write()
      
       - Update MAINTAINER entries
      
       - Fix help messages in Kconfigs
      
       - Fix kernel-doc header for update_preds()
      
      * tag 'trace-v6.2-rc5' of git://git.kernel.org/pub/scm/linux/kernel/git/trace/linux-trace:
        bootconfig: Update MAINTAINERS file to add tree and mailing list
        rv: remove redundant initialization of pointer ptr
        ftrace: Maintain samples/ftrace
        tracing/filter: fix kernel-doc warnings
        lib: Kconfig: fix spellos
        trace_events_hist: add check for return value of 'create_hist_field'
        tracing/osnoise: Use built-in RCU list checking
        tracing: Kconfig: Fix spelling/grammar/punctuation
        ftrace/scripts: Update the instructions for ftrace-bisect.sh
        tracing: Make sure trace_printk() can output as soon as it can be used
        ftrace: Export ftrace_free_filter() to modules
      d786f0fe
  6. 27 Jan, 2023 20 commits