- 27 Dec, 2022 28 commits
-
-
Nicholas Piggin authored
The archs that use cputime_to_nsecs() internally provide their own definition and don't need the fallback. cputime_to_usecs() unused except in this fallback, and is not defined anywhere. This removes the final remnant of the cputime_t code from the kernel. Signed-off-by:
Nicholas Piggin <npiggin@gmail.com> Signed-off-by:
Peter Zijlstra (Intel) <peterz@infradead.org> Acked-by:
Alexander Gordeev <agordeev@linux.ibm.com> Link: https://lore.kernel.org/r/20221220070705.2958959-1-npiggin@gmail.com
-
Hao Jia authored
When select_idle_capacity() starts scanning for an idle CPU, it starts with target CPU that has already been checked in select_idle_sibling(). So we start checking from the next CPU and try the target CPU at the end. Similarly for task_numa_assign(), we have just checked numa_migrate_on of dst_cpu, so start from the next CPU. This also works for steal_cookie_task(), the first scan must fail and start directly from the next one. Signed-off-by:
Hao Jia <jiahao.os@bytedance.com> Signed-off-by:
Vincent Guittot <vincent.guittot@linaro.org> Acked-by:
Mel Gorman <mgorman@techsingularity.net> Link: https://lore.kernel.org/r/20221216062406.7812-3-jiahao.os@bytedance.com
-
Hao Jia authored
In update_numa_stats() we try to find an idle cpu on the NUMA node, preferably an idle core. we can stop looking for the next idle core or idle cpu after finding an idle core. But we can't stop the whole loop of scanning the CPU, because we need to calculate approximate NUMA stats at a point in time. For example, the src and dst nr_running is needed by task_numa_find_cpu(). Signed-off-by:
-
Matthew Wilcox (Oracle) authored
With a modified container_of() that preserves constness, the compiler finds some pointers which should have been marked as const. task_of() also needs to become const-preserving for the !FAIR_GROUP_SCHED case so that cfs_rq_of() can take a const argument. No change to generated code. Signed-off-by:
Matthew Wilcox (Oracle) <willy@infradead.org> Signed-off-by:
-
Mathieu Desnoyers authored
Add mm_numa_cid tests to the run_param_test.sh test script. Signed-off-by:
Mathieu Desnoyers <mathieu.desnoyers@efficios.com> Signed-off-by:
-
Mathieu Desnoyers authored
Add the mm_cid field to the rseq_update event, allowing tracers to follow which mm_cid is observed by user-space, and whether negative mm_cid values are visible in case of internal scheduler implementation issues. Signed-off-by:
-
Mathieu Desnoyers authored
Report and abort when a negative concurrency ID value is observed by the spinlock test. Signed-off-by:
-
Mathieu Desnoyers authored
Adapt to the rseq.h API changes introduced by commits "selftests/rseq: <arch>: Template memory ordering and percpu access mode". Build a new param_test_mm_cid, param_test_mm_cid_benchmark, and param_test_mm_cid_compare_twice executables to test the new "mm_cid" rseq field. Signed-off-by:
-
Mathieu Desnoyers authored
Adapt to the rseq.h API changes introduced by commits "selftests/rseq: <arch>: Template memory ordering and percpu access mode". Build a new basic_percpu_ops_mm_cid_test to test the new "mm_cid" rseq field. Signed-off-by:
-
Mathieu Desnoyers authored
Introduce a rseq-riscv-bits.h template header which is internally included to generate the static inline functions covering: - relaxed and release memory ordering, - per-cpu-id and per-mm-cid per-cpu data access. Signed-off-by:
-
Mathieu Desnoyers authored
Introduce a rseq-s390-bits.h template header which is internally included to generate the static inline functions covering: - relaxed and release memory ordering, - per-cpu-id and per-mm-cid per-cpu data access. Signed-off-by:
-
Mathieu Desnoyers authored
Introduce a rseq-ppc-bits.h template header which is internally included to generate the static inline functions covering: - relaxed and release memory ordering, - per-cpu-id and per-mm-cid per-cpu data access. Signed-off-by:
-
Mathieu Desnoyers authored
Introduce a rseq-mips-bits.h template header which is internally included to generate the static inline functions covering: - relaxed and release memory ordering, - per-cpu-id and per-mm-cid per-cpu data access. Signed-off-by:
-
Mathieu Desnoyers authored
Introduce a rseq-arm64-bits.h template header which is internally included to generate the static inline functions covering: - relaxed and release memory ordering, - per-cpu-id and per-mm-cid per-cpu data access. Signed-off-by:
-
Mathieu Desnoyers authored
Introduce a rseq-arm-bits.h template header which is internally included to generate the static inline functions covering: - relaxed and release memory ordering, - per-cpu-id and per-mm-cid per-cpu data access. Signed-off-by:
-
Mathieu Desnoyers authored
Introduce a rseq-x86-bits.h template header which is internally included to generate the static inline functions covering: - relaxed and release memory ordering, - per-cpu-id and per-mm-cid per-cpu data access. This introduces changes to the rseq.h selftests API which require to update the rseq selftest programs. Similar API/templating changes need to be done for other architectures. Signed-off-by:
-
Mathieu Desnoyers authored
Add support for the mm_cid field (per-memory-map concurrency ID) of struct rseq to rseq selftests. Signed-off-by:
-
Mathieu Desnoyers authored
This code is not currently build by the test Makefile, adds complexity, and is not overall useful considering that the abort handling loops to retry the fast-path. Signed-off-by:
-
Mathieu Desnoyers authored
If a memory map has fewer threads than there are cores on the system, or is limited to run on few cores concurrently through sched affinity or cgroup cpusets, the concurrency IDs will be values close to 0, thus allowing efficient use of user-space memory for per-cpu data structures. Signed-off-by:
-
Mathieu Desnoyers authored
This feature allows the scheduler to expose a per-memory map concurrency ID to user-space. This concurrency ID is within the possible cpus range, and is temporarily (and uniquely) assigned while threads are actively running within a memory map. If a memory map has fewer threads than cores, or is limited to run on few cores concurrently through sched affinity or cgroup cpusets, the concurrency IDs will be values close to 0, thus allowing efficient use of user-space memory for per-cpu data structures. This feature is meant to be exposed by a new rseq thread area field. The primary purpose of this feature is to do the heavy-lifting needed by memory allocators to allow them to use per-cpu data structures efficiently in the following situations: - Single-threaded applications, - Multi-threaded applications on large systems (many cores) with limited cpu affinity mask, - Multi-threaded applications on large systems (many cores) with restricted cgroup cpuset per container. One of the key concern from scheduler maintainers is the overhead associated with additional spin locks or atomic operations in the scheduler fast-path. This is why the following optimization is implemented. On context switch between threads belonging to the same memory map, transfer the mm_cid from prev to next without any atomic ops. This takes care of use-cases involving frequent context switch between threads belonging to the same memory map. Additional optimizations can be done if the spin locks added when context switching between threads belonging to different memory maps end up being a performance bottleneck. Those are left out of this patch though. A performance impact would have to be clearly demonstrated to justify the added complexity. The credit goes to Paul Turner (Google) for the original virtual cpu id idea. This feature is implemented based on the discussions with Paul Turner and Peter Oskolkov (Google), but I took the liberty to implement scheduler fast-path optimizations and my own NUMA-awareness scheme. The rumor has it that Google have been running a rseq vcpu_id extension internally in production for a year. The tcmalloc source code indeed has comments hinting at a vcpu_id prototype extension to the rseq system call [1]. The following benchmarks do not show any significant overhead added to the scheduler context switch by this feature: * perf bench sched messaging (process) Baseline: 86.5±0.3 ms With mm_cid: 86.7±2.6 ms * perf bench sched messaging (threaded) Baseline: 84.3±3.0 ms With mm_cid: 84.7±2.6 ms * hackbench (process) Baseline: 82.9±2.7 ms With mm_cid: 82.9±2.9 ms * hackbench (threaded) Baseline: 85.2±2.6 ms With mm_cid: 84.4±2.9 ms [1] https://github.com/google/tcmalloc/blob/master/tcmalloc/internal/linux_syscall_support.h#L26Signed-off-by:
-
Mathieu Desnoyers authored
Test the NUMA node id extension rseq field. Compare it against the value returned by the getcpu(2) system call while pinned on a specific core. Signed-off-by:
-
Mathieu Desnoyers authored
Use the ELF auxiliary vector AT_RSEQ_FEATURE_SIZE to detect the RSEQ features supported by the kernel. Signed-off-by:
-
Mathieu Desnoyers authored
Adding the NUMA node id to struct rseq is a straightforward thing to do, and a good way to figure out if anything in the user-space ecosystem prevents extending struct rseq. This NUMA node id field allows memory allocators such as tcmalloc to take advantage of fast access to the current NUMA node id to perform NUMA-aware memory allocation. It can also be useful for implementing fast-paths for NUMA-aware user-space mutexes. It also allows implementing getcpu(2) purely in user-space. Signed-off-by:
-
Mathieu Desnoyers authored
Introduce the extensible rseq ABI, where the feature size supported by the kernel and the required alignment are communicated to user-space through ELF auxiliary vectors. This allows user-space to call rseq registration with a rseq_len of either 32 bytes for the original struct rseq size (which includes padding), or larger. If rseq_len is larger than 32 bytes, then it must be large enough to contain the feature size communicated to user-space through ELF auxiliary vectors. Signed-off-by:
-
Mathieu Desnoyers authored
Export the rseq feature size supported by the kernel as well as the required allocation alignment for the rseq per-thread area to user-space through ELF auxiliary vector entries. This is part of the extensible rseq ABI. Signed-off-by:
-
Mathieu Desnoyers authored
When linking the selftests against a libc which does not handle rseq registration (before 2.35), rseq thread registration silently succeed even with CONFIG_RSEQ=n because it erroneously thinks that libc is handling rseq registration. This is caused by setting the rseq ownership flag only after the rseq_available() check. It should rather be set before the rseq_available() check. Set the rseq_size to 0 (error value) immediately after the rseq_available() check fails rather than in the thread registration functions. Signed-off-by:
-
Josh Don authored
CFS bandwidth currently distributes new runtime and unthrottles cfs_rq's inline in an hrtimer callback. Runtime distribution is a per-cpu operation, and unthrottling is a per-cgroup operation, since a tg walk is required. On machines with a large number of cpus and large cgroup hierarchies, this cpus*cgroups work can be too much to do in a single hrtimer callback: since IRQ are disabled, hard lockups may easily occur. Specifically, we've found this scalability issue on configurations with 256 cpus, O(1000) cgroups in the hierarchy being throttled, and high memory bandwidth usage. To fix this, we can instead unthrottle cfs_rq's asynchronously via a CSD. Each cpu is responsible for unthrottling itself, thus sharding the total work more fairly across the system, and avoiding hard lockups. Signed-off-by:
Josh Don <joshdon@google.com> Signed-off-by:
-
Bing Huang authored
init_defrootdomain is only used in initialization Signed-off-by:
Bing Huang <huangbing@kylinos.cn> Signed-off-by:
Steven Rostedt (Google) <rostedt@goodmis.org> Reviewed-by:
Randy Dunlap <rdunlap@infradead.org> Reviewed-by:
Valentin Schneider <vschneid@redhat.com> Link: https://lkml.kernel.org/r/20221118034208.267330-1-huangbing775@126.com
-
- 25 Dec, 2022 2 commits
-
-
Linus Torvalds authored
-
Steven Rostedt (Google) authored
Due to several bugs caused by timers being re-armed after they are shutdown and just before they are freed, a new state of timers was added called "shutdown". After a timer is set to this state, then it can no longer be re-armed. The following script was run to find all the trivial locations where del_timer() or del_timer_sync() is called in the same function that the object holding the timer is freed. It also ignores any locations where the timer->function is modified between the del_timer*() and the free(), as that is not considered a "trivial" case. This was created by using a coccinelle script and the following commands: $ cat timer.cocci @@ expression ptr, slab; identifier timer, rfield; @@ ( - del_timer(&ptr->timer); + timer_shutdown(&ptr->timer); | - del_timer_sync(&ptr->timer); + timer_shutdown_sync(&ptr->timer); ) ... when strict when != ptr->timer ( kfree_rcu(ptr, rfield); | kmem_cache_free(slab, ptr); | kfree(ptr); ) $ spatch timer.cocci . > /tmp/t.patch $ patch -p1 < /tmp/t.patch Link: https://lore.kernel.org/lkml/20221123201306.823305113@linutronix.de/Signed-off-by:Linus Torvalds <torvalds@linux-foundation.org>
-
- 23 Dec, 2022 10 commits
-
-
git://git.kernel.org/pub/scm/linux/kernel/git/broonie/spiLinus Torvalds authored
Pull spi fix from Mark Brown: "One driver specific change here which handles the case where a SPI device for some reason tries to change the bus speed during a message on fsl_spi hardware, this should be very unusual" * tag 'spi-fix-v6.2-rc1' of git://git.kernel.org/pub/scm/linux/kernel/git/broonie/spi: spi: fsl_spi: Don't change speed while chipselect is active
-
Linus Torvalds authored
Merge tag 'regulator-fix-v6.2-rc1' of git://git.kernel.org/pub/scm/linux/kernel/git/broonie/regulator Pull regulator fixes from Mark Brown: "Two core fixes here, one for a long standing race which some Qualcomm systems have started triggering with their UFS driver and another fixing a problem with supply lookup introduced by the fixes for devm related use after free issues that were introduced in this merge window" * tag 'regulator-fix-v6.2-rc1' of git://git.kernel.org/pub/scm/linux/kernel/git/broonie/regulator: regulator: core: fix deadlock on regulator enable regulator: core: Fix resolve supply lookup issue
-
git://git.kernel.org/pub/scm/linux/kernel/git/jlawall/linuxLinus Torvalds authored
Pull coccicheck update from Julia Lawall: "Modernize use of grep in coccicheck: Use 'grep -E' instead of 'egrep'" * tag 'coccinelle-6.2' of git://git.kernel.org/pub/scm/linux/kernel/git/jlawall/linux: scripts: coccicheck: use "grep -E" instead of "egrep"
-
git://git.kernel.org/pub/scm/linux/kernel/git/kees/linuxLinus Torvalds authored
Pull kernel hardening fixes from Kees Cook: - Fix CFI failure with KASAN (Sami Tolvanen) - Fix LKDTM + CFI under GCC 7 and 8 (Kristina Martsenko) - Limit CONFIG_ZERO_CALL_USED_REGS to Clang > 15.0.6 (Nathan Chancellor) - Ignore "contents" argument in LoadPin's LSM hook handling - Fix paste-o in /sys/kernel/warn_count API docs - Use READ_ONCE() consistently for oops/warn limit reading * tag 'hardening-v6.2-rc1-fixes' of git://git.kernel.org/pub/scm/linux/kernel/git/kees/linux: cfi: Fix CFI failure with KASAN exit: Use READ_ONCE() for all oops/warn limit reads security: Restrict CONFIG_ZERO_CALL_USED_REGS to gcc or clang > 15.0.6 lkdtm: cfi: Make PAC test work with GCC 7 and 8 docs: Fix path paste-o for /sys/kernel/warn_count LoadPin: Ignore the "contents" argument of the LSM hooks
-
git://git.kernel.org/pub/scm/linux/kernel/git/kees/linuxLinus Torvalds authored
Pull pstore fixes from Kees Cook: - Switch pmsg_lock to an rt_mutex to avoid priority inversion (John Stultz) - Correctly assign mem_type property (Luca Stefani) * tag 'pstore-v6.2-rc1-fixes' of git://git.kernel.org/pub/scm/linux/kernel/git/kees/linux: pstore: Properly assign mem_type property pstore: Make sure CONFIG_PSTORE_PMSG selects CONFIG_RT_MUTEXES pstore: Switch pmsg_lock to an rt_mutex to avoid priority inversion
-
git://git.infradead.org/users/hch/dma-mappingLinus Torvalds authored
Pull dma-mapping fixes from Christoph Hellwig: "Fix up the sound code to not pass __GFP_COMP to the non-coherent DMA allocator, as it copes with that just as badly as the coherent allocator, and then add a check to make sure no one passes the flag ever again" * tag 'dma-mapping-2022-12-23' of git://git.infradead.org/users/hch/dma-mapping: dma-mapping: reject GFP_COMP for noncoherent allocations ALSA: memalloc: don't use GFP_COMP for non-coherent dma allocations
-
https://github.com/martinetd/linuxLinus Torvalds authored
Pull 9p updates from Dominique Martinet: - improve p9_check_errors to check buffer size instead of msize when possible (e.g. not zero-copy) - some more syzbot and KCSAN fixes - minor headers include cleanup * tag '9p-for-6.2-rc1' of https://github.com/martinetd/linux: 9p/client: fix data race on req->status net/9p: fix response size check in p9_check_errors() net/9p: distinguish zero-copy requests 9p/xen: do not memcpy header into req->rc 9p: set req refcount to zero to avoid uninitialized usage 9p/net: Remove unneeded idr.h #include 9p/fs: Remove unneeded idr.h #include
-
git://git.kernel.org/pub/scm/linux/kernel/git/tiwai/soundLinus Torvalds authored
Pull more sound updates from Takashi Iwai: "A few more updates for 6.2: most of changes are about ASoC device-specific fixes. - Lots of ASoC Intel AVS extensions and refactoring - Quirks for ASoC Intel SOF as well as regression fixes - ASoC Mediatek and Rockchip fixes - Intel HD-audio HDMI workarounds - Usual HD- and USB-audio device-specific quirks" * tag 'sound-6.2-rc1-2' of git://git.kernel.org/pub/scm/linux/kernel/git/tiwai/sound: (54 commits) ALSA: usb-audio: Add new quirk FIXED_RATE for JBL Quantum810 Wireless ALSA: azt3328: Remove the unused function snd_azf3328_codec_outl() ASoC: lochnagar: Fix unused lochnagar_of_match warning ASoC: Intel: Add HP Stream 8 to bytcr_rt5640.c ASoC: SOF: mediatek: initialize panic_info to zero ASoC: rt5670: Remove unbalanced pm_runtime_put() ASoC: Intel: bytcr_rt5640: Add quirk for the Advantech MICA-071 tablet ASoC: Intel: soc-acpi: update codec addr on 0C11/0C4F product ASoC: rockchip: spdif: Add missing clk_disable_unprepare() in rk_spdif_runtime_resume() ASoC: wm8994: Fix potential deadlock ASoC: mediatek: mt8195: add sof be ops to check audio active ASoC: SOF: Revert: "core: unregister clients and machine drivers in .shutdown" ASoC: SOF: Intel: pci-tgl: unblock S5 entry if DMA stop has failed" ALSA: hda/hdmi: fix stream-id config keep-alive for rt suspend ALSA: hda/hdmi: set default audio parameters for KAE silent-stream ALSA: hda/hdmi: fix i915 silent stream programming flow ALSA: hda: Error out if invalid stream is being setup ASoC: dt-bindings: fsl-sai: Reinstate i.MX93 SAI compatible string ASoC: soc-pcm.c: Clear DAIs parameters after stream_active is updated ASoC: codecs: wcd-clsh: Remove the unused function ...
-
git://anongit.freedesktop.org/drm/drmLinus Torvalds authored
Pull drm fixes from Dave Airlie: "Holiday fixes! Two batches from amd, and one group of i915 changes. amdgpu: - Spelling fix - BO pin fix - Properly handle polaris 10/11 overlap asics - GMC9 fix - SR-IOV suspend fix - DCN 3.1.4 fix - KFD userptr locking fix - SMU13.x fixes - GDS/GWS/OA handling fix - Reserved VMID handling fixes - FRU EEPROM fix - BO validation fixes - Avoid large variable on the stack - S0ix fixes - SMU 13.x fixes - VCN fix - Add missing fence reference amdkfd: - Fix init vm error handling - Fix double release of compute pasid i915 - Documentation fixes - OA-perf related fix - VLV/CHV HDMI/DP audio fix - Display DDI/Transcoder fix - Migrate fixes" * tag 'drm-next-2022-12-23' of git://anongit.freedesktop.org/drm/drm: (39 commits) drm/amdgpu: grab extra fence reference for drm_sched_job_add_dependency drm/amdgpu: enable VCN DPG for GC IP v11.0.4 drm/amdgpu: skip mes self test after s0i3 resume for MES IP v11.0 drm/amd/pm: correct the fan speed retrieving in PWM for some SMU13 asics drm/amd/pm: bump SMU13.0.0 driver_if header to version 0x34 drm/amdgpu: skip MES for S0ix as well since it's part of GFX drm/amd/pm: avoid large variable on kernel stack drm/amdkfd: Fix double release compute pasid drm/amdkfd: Fix kfd_process_device_init_vm error handling drm/amd/pm: update SMU13.0.0 reported maximum shader clock drm/amd/pm: correct SMU13.0.0 pstate profiling clock settings drm/amd/pm: enable GPO dynamic control support for SMU13.0.7 drm/amd/pm: enable GPO dynamic control support for SMU13.0.0 drm/amdgpu: revert "generally allow over-commit during BO allocation" drm/amdgpu: Remove unnecessary domain argument drm/amdgpu: Fix size validation for non-exclusive domains (v4) drm/amdgpu: Check if fru_addr is not NULL (v2) drm/i915/ttm: consider CCS for backup objects drm/i915/migrate: fix corner case in CCS aux copying drm/amdgpu: rework reserved VMID handling ...
-
git://git.kernel.org/pub/scm/linux/kernel/git/mips/linuxLinus Torvalds authored
Pull MIPS fixes from Thomas Bogendoerfer: "Fixes due to DT changes" * tag 'mips_6.2_1' of git://git.kernel.org/pub/scm/linux/kernel/git/mips/linux: MIPS: dts: bcm63268: Add missing properties to the TWD node MIPS: ralink: mt7621: avoid to init common ralink reset controller
-
