Skip to content

L
linux
Commit 927ba67a authored by Linus Torvalds's avatar Linus Torvalds
Browse files
Options

Merge branch 'timers-core-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip 

Pull timer updates from Thomas Gleixner:
 "The timer and timekeeping departement delivers:

  Core:

   - The consolidation of the VDSO code into a generic library including
     the conversion of x86 and ARM64. Conversion of ARM and MIPS are en
     route through the relevant maintainer trees and should end up in
     5.4.

     This gets rid of the unnecessary different copies of the same code
     and brings all architectures on the same level of VDSO
     functionality.

   - Make the NTP user space interface more robust by restricting the
     TAI offset to prevent undefined behaviour. Includes a selftest.

   - Validate user input in the compat settimeofday() syscall to catch
     invalid values which would be turned into valid values by a
     multiplication overflow

   - Consolidate the time accessors

   - Small fixes, improvements and cleanups all over the place

  Drivers:

   - Support for the NXP system counter, TI davinci timer

   - Move the Microsoft HyperV clocksource/events code into the
     drivers/clocksource directory so it can be shared between x86 and
     ARM64.

   - Overhaul of the Tegra driver

   - Delay timer support for IXP4xx

   - Small fixes, improvements and cleanups as usual"

* 'timers-core-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip: (71 commits)
  time: Validate user input in compat_settimeofday()
  timer: Document TIMER_PINNED
  clocksource/drivers: Continue making Hyper-V clocksource ISA agnostic
  clocksource/drivers: Make Hyper-V clocksource ISA agnostic
  MAINTAINERS: Fix Andy's surname and the directory entries of VDSO
  hrtimer: Use a bullet for the returns bullet list
  arm64: vdso: Fix compilation with clang older than 8
  arm64: compat: Fix __arch_get_hw_counter() implementation
  arm64: Fix __arch_get_hw_counter() implementation
  lib/vdso: Make delta calculation work correctly
  MAINTAINERS: Add entry for the generic VDSO library
  arm64: compat: No need for pre-ARMv7 barriers on an ARMv8 system
  arm64: vdso: Remove unnecessary asm-offsets.c definitions
  vdso: Remove superfluous #ifdef __KERNEL__ in vdso/datapage.h
  clocksource/drivers/davinci: Add support for clocksource
  clocksource/drivers/davinci: Add support for clockevents
  clocksource/drivers/tegra: Set up maximum-ticks limit properly
  clocksource/drivers/tegra: Cycles can't be 0
  clocksource/drivers/tegra: Restore base address before cleanup
  clocksource/drivers/tegra: Add verbose definition for 1MHz constant
  ...
parents 2a1ccd31 9176ab1b
Expand all Hide whitespace changes
Inline Side-by-side
Showing with 3929 additions and 1359 deletions
Documentation/core-api/timekeeping.rst
View file @ 927ba67a
......@@ -65,7 +65,7 @@ different format depending on what is required by the user:
.. c:function:: u64 ktime_get_ns( void )
u64 ktime_get_boottime_ns( void )
u64 ktime_get_real_ns( void )
u64 ktime_get_tai_ns( void )
u64 ktime_get_clocktai_ns( void )
u64 ktime_get_raw_ns( void )
Same as the plain ktime_get functions, but returning a u64 number
......@@ -99,16 +99,20 @@ Coarse and fast_ns access
Some additional variants exist for more specialized cases:
.. c:function:: ktime_t ktime_get_coarse_boottime( void )
.. c:function:: ktime_t ktime_get_coarse( void )
ktime_t ktime_get_coarse_boottime( void )
ktime_t ktime_get_coarse_real( void )
ktime_t ktime_get_coarse_clocktai( void )
ktime_t ktime_get_coarse_raw( void )
.. c:function:: u64 ktime_get_coarse_ns( void )
u64 ktime_get_coarse_boottime_ns( void )
u64 ktime_get_coarse_real_ns( void )
u64 ktime_get_coarse_clocktai_ns( void )
.. c:function:: void ktime_get_coarse_ts64( struct timespec64 * )
void ktime_get_coarse_boottime_ts64( struct timespec64 * )
void ktime_get_coarse_real_ts64( struct timespec64 * )
void ktime_get_coarse_clocktai_ts64( struct timespec64 * )
void ktime_get_coarse_raw_ts64( struct timespec64 * )
These are quicker than the non-coarse versions, but less accurate,
corresponding to CLOCK_MONONOTNIC_COARSE and CLOCK_REALTIME_COARSE
......
Documentation/devicetree/bindings/timer/nxp,sysctr-timer.txt 0 → 100644
View file @ 927ba67a
NXP System Counter Module(sys_ctr)
The system counter(sys_ctr) is a programmable system counter which provides
a shared time base to Cortex A15, A7, A53, A73, etc. it is intended for use in
applications where the counter is always powered and support multiple,
unrelated clocks. The compare frame inside can be used for timer purpose.
Required properties:
- compatible : should be "nxp,sysctr-timer"
- reg : Specifies the base physical address and size of the comapre
frame and the counter control, read & compare.
- interrupts : should be the first compare frames' interrupt
- clocks : Specifies the counter clock.
- clock-names: Specifies the clock's name of this module
Example:
system_counter: timer@306a0000 {
compatible = "nxp,sysctr-timer";
reg = <0x306a0000 0x20000>;/* system-counter-rd & compare */
clocks = <&clk_8m>;
clock-names = "per";
interrupts = <GIC_SPI 47 IRQ_TYPE_LEVEL_HIGH>;
};
MAINTAINERS
View file @ 927ba67a
......@@ -6687,6 +6687,18 @@ L: kvm@vger.kernel.org
S: Supported
F: drivers/uio/uio_pci_generic.c
GENERIC VDSO LIBRARY:
M: Andy Lutomirski <luto@kernel.org>
M: Thomas Gleixner <tglx@linutronix.de>
M: Vincenzo Frascino <vincenzo.frascino@arm.com>
L: linux-kernel@vger.kernel.org
T: git git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip.git timers/vdso
S: Maintained
F: lib/vdso/
F: kernel/time/vsyscall.c
F: include/vdso/
F: include/asm-generic/vdso/vsyscall.h
GENWQE (IBM Generic Workqueue Card)
M: Frank Haverkamp <haver@linux.ibm.com>
S: Supported
......@@ -7324,6 +7336,7 @@ F: arch/x86/include/asm/trace/hyperv.h
F: arch/x86/include/asm/hyperv-tlfs.h
F: arch/x86/kernel/cpu/mshyperv.c
F: arch/x86/hyperv
F: drivers/clocksource/hyperv_timer.c
F: drivers/hid/hid-hyperv.c
F: drivers/hv/
F: drivers/input/serio/hyperv-keyboard.c
......@@ -7334,6 +7347,7 @@ F: drivers/uio/uio_hv_generic.c
F: drivers/video/fbdev/hyperv_fb.c
F: drivers/iommu/hyperv_iommu.c
F: net/vmw_vsock/hyperv_transport.c
F: include/clocksource/hyperv_timer.h
F: include/linux/hyperv.h
F: include/uapi/linux/hyperv.h
F: tools/hv/
......
arch/arm/include/asm/arch_timer.h
View file @ 927ba67a
......@@ -4,6 +4,7 @@
#include <asm/barrier.h>
#include <asm/errno.h>
#include <asm/hwcap.h>
#include <linux/clocksource.h>
#include <linux/init.h>
#include <linux/types.h>
......@@ -124,6 +125,15 @@ static inline void arch_timer_set_cntkctl(u32 cntkctl)
isb();
}
static inline void arch_timer_set_evtstrm_feature(void)
{
elf_hwcap |= HWCAP_EVTSTRM;
}
static inline bool arch_timer_have_evtstrm_feature(void)
{
return elf_hwcap & HWCAP_EVTSTRM;
}
#endif
#endif
arch/arm64/Kconfig
View file @ 927ba67a
......@@ -108,6 +108,8 @@ config ARM64
select GENERIC_STRNCPY_FROM_USER
select GENERIC_STRNLEN_USER
select GENERIC_TIME_VSYSCALL
select GENERIC_GETTIMEOFDAY
select GENERIC_COMPAT_VDSO if (!CPU_BIG_ENDIAN && COMPAT)
select HANDLE_DOMAIN_IRQ
select HARDIRQS_SW_RESEND
select HAVE_PCI
......@@ -161,6 +163,7 @@ config ARM64
select HAVE_SYSCALL_TRACEPOINTS
select HAVE_KPROBES
select HAVE_KRETPROBES
select HAVE_GENERIC_VDSO
select IOMMU_DMA if IOMMU_SUPPORT
select IRQ_DOMAIN
select IRQ_FORCED_THREADING
......
arch/arm64/Makefile
View file @ 927ba67a
......@@ -49,10 +49,26 @@ $(warning Detected assembler with broken .inst; disassembly will be unreliable)
endif
endif
KBUILD_CFLAGS += -mgeneral-regs-only $(lseinstr) $(brokengasinst)
ifeq ($(CONFIG_GENERIC_COMPAT_VDSO), y)
CROSS_COMPILE_COMPAT ?= $(CONFIG_CROSS_COMPILE_COMPAT_VDSO:"%"=%)
ifeq ($(CONFIG_CC_IS_CLANG), y)
$(warning CROSS_COMPILE_COMPAT is clang, the compat vDSO will not be built)
else ifeq ($(CROSS_COMPILE_COMPAT),)
$(warning CROSS_COMPILE_COMPAT not defined or empty, the compat vDSO will not be built)
else ifeq ($(shell which $(CROSS_COMPILE_COMPAT)gcc 2> /dev/null),)
$(error $(CROSS_COMPILE_COMPAT)gcc not found, check CROSS_COMPILE_COMPAT)
else
export CROSS_COMPILE_COMPAT
export CONFIG_COMPAT_VDSO := y
compat_vdso := -DCONFIG_COMPAT_VDSO=1
endif
endif
KBUILD_CFLAGS += -mgeneral-regs-only $(lseinstr) $(brokengasinst) $(compat_vdso)
KBUILD_CFLAGS += -fno-asynchronous-unwind-tables
KBUILD_CFLAGS += $(call cc-disable-warning, psabi)
KBUILD_AFLAGS += $(lseinstr) $(brokengasinst)
KBUILD_AFLAGS += $(lseinstr) $(brokengasinst) $(compat_vdso)
KBUILD_CFLAGS += $(call cc-option,-mabi=lp64)
KBUILD_AFLAGS += $(call cc-option,-mabi=lp64)
......@@ -164,6 +180,9 @@ ifeq ($(KBUILD_EXTMOD),)
prepare: vdso_prepare
vdso_prepare: prepare0
$(Q)$(MAKE) $(build)=arch/arm64/kernel/vdso include/generated/vdso-offsets.h
$(if $(CONFIG_COMPAT_VDSO),$(Q)$(MAKE) \
$(build)=arch/arm64/kernel/vdso32 \
include/generated/vdso32-offsets.h)
endif
define archhelp
......
arch/arm64/include/asm/arch_timer.h
View file @ 927ba67a
......@@ -9,6 +9,7 @@
#define __ASM_ARCH_TIMER_H
#include <asm/barrier.h>
#include <asm/hwcap.h>
#include <asm/sysreg.h>
#include <linux/bug.h>
......@@ -229,4 +230,16 @@ static inline int arch_timer_arch_init(void)
return 0;
}
static inline void arch_timer_set_evtstrm_feature(void)
{
cpu_set_named_feature(EVTSTRM);
#ifdef CONFIG_COMPAT
compat_elf_hwcap |= COMPAT_HWCAP_EVTSTRM;
#endif
}
static inline bool arch_timer_have_evtstrm_feature(void)
{
return cpu_have_named_feature(EVTSTRM);
}
#endif
arch/arm64/include/asm/elf.h
View file @ 927ba67a
......@@ -202,7 +202,21 @@ typedef compat_elf_greg_t compat_elf_gregset_t[COMPAT_ELF_NGREG];
({ \
set_thread_flag(TIF_32BIT); \
})
#ifdef CONFIG_GENERIC_COMPAT_VDSO
#define COMPAT_ARCH_DLINFO \
do { \
/* \
* Note that we use Elf64_Off instead of elf_addr_t because \
* elf_addr_t in compat is defined as Elf32_Addr and casting \
* current->mm->context.vdso to it triggers a cast warning of \
* cast from pointer to integer of different size. \
*/ \
NEW_AUX_ENT(AT_SYSINFO_EHDR, \
(Elf64_Off)current->mm->context.vdso); \
} while (0)
#else
#define COMPAT_ARCH_DLINFO
#endif
extern int aarch32_setup_additional_pages(struct linux_binprm *bprm,
int uses_interp);
#define compat_arch_setup_additional_pages \
......
arch/arm64/include/asm/signal32.h
View file @ 927ba67a
......@@ -9,6 +9,52 @@
#ifdef CONFIG_COMPAT
#include <linux/compat.h>
struct compat_sigcontext {
/* We always set these two fields to 0 */
compat_ulong_t trap_no;
compat_ulong_t error_code;
compat_ulong_t oldmask;
compat_ulong_t arm_r0;
compat_ulong_t arm_r1;
compat_ulong_t arm_r2;
compat_ulong_t arm_r3;
compat_ulong_t arm_r4;
compat_ulong_t arm_r5;
compat_ulong_t arm_r6;
compat_ulong_t arm_r7;
compat_ulong_t arm_r8;
compat_ulong_t arm_r9;
compat_ulong_t arm_r10;
compat_ulong_t arm_fp;
compat_ulong_t arm_ip;
compat_ulong_t arm_sp;
compat_ulong_t arm_lr;
compat_ulong_t arm_pc;
compat_ulong_t arm_cpsr;
compat_ulong_t fault_address;
};
struct compat_ucontext {
compat_ulong_t uc_flags;
compat_uptr_t uc_link;
compat_stack_t uc_stack;
struct compat_sigcontext uc_mcontext;
compat_sigset_t uc_sigmask;
int __unused[32 - (sizeof(compat_sigset_t) / sizeof(int))];
compat_ulong_t uc_regspace[128] __attribute__((__aligned__(8)));
};
struct compat_sigframe {
struct compat_ucontext uc;
compat_ulong_t retcode[2];
};
struct compat_rt_sigframe {
struct compat_siginfo info;
struct compat_sigframe sig;
};
int compat_setup_frame(int usig, struct ksignal *ksig, sigset_t *set,
struct pt_regs *regs);
int compat_setup_rt_frame(int usig, struct ksignal *ksig, sigset_t *set,
......
arch/arm64/include/asm/unistd.h
View file @ 927ba67a
......@@ -22,8 +22,13 @@
#define __NR_compat_exit 1
#define __NR_compat_read 3
#define __NR_compat_write 4
#define __NR_compat_gettimeofday 78
#define __NR_compat_sigreturn 119
#define __NR_compat_rt_sigreturn 173
#define __NR_compat_clock_getres 247
#define __NR_compat_clock_gettime 263
#define __NR_compat_clock_gettime64 403
#define __NR_compat_clock_getres_time64 406
/*
* The following SVCs are ARM private.
......
arch/arm64/include/asm/vdso.h
View file @ 927ba67a
......@@ -17,6 +17,9 @@
#ifndef __ASSEMBLY__
#include <generated/vdso-offsets.h>
#ifdef CONFIG_COMPAT_VDSO
#include <generated/vdso32-offsets.h>
#endif
#define VDSO_SYMBOL(base, name) \
({ \
......
arch/arm64/include/asm/vdso/compat_barrier.h 0 → 100644
View file @ 927ba67a
/* SPDX-License-Identifier: GPL-2.0 */
/*
* Copyright (C) 2018 ARM Limited
*/
#ifndef __COMPAT_BARRIER_H
#define __COMPAT_BARRIER_H
#ifndef __ASSEMBLY__
/*
* Warning: This code is meant to be used with
* ENABLE_COMPAT_VDSO only.
*/
#ifndef ENABLE_COMPAT_VDSO
#error This header is meant to be used with ENABLE_COMPAT_VDSO only
#endif
#ifdef dmb
#undef dmb
#endif
#define dmb(option) __asm__ __volatile__ ("dmb " #option : : : "memory")
#if __LINUX_ARM_ARCH__ >= 8
#define aarch32_smp_mb() dmb(ish)
#define aarch32_smp_rmb() dmb(ishld)
#define aarch32_smp_wmb() dmb(ishst)
#else
#define aarch32_smp_mb() dmb(ish)
#define aarch32_smp_rmb() aarch32_smp_mb()
#define aarch32_smp_wmb() dmb(ishst)
#endif
#undef smp_mb
#undef smp_rmb
#undef smp_wmb
#define smp_mb() aarch32_smp_mb()
#define smp_rmb() aarch32_smp_rmb()
#define smp_wmb() aarch32_smp_wmb()
#endif /* !__ASSEMBLY__ */
#endif /* __COMPAT_BARRIER_H */
arch/arm64/include/asm/vdso/compat_gettimeofday.h 0 → 100644
View file @ 927ba67a
/* SPDX-License-Identifier: GPL-2.0 */
/*
* Copyright (C) 2018 ARM Limited
*/
#ifndef __ASM_VDSO_GETTIMEOFDAY_H
#define __ASM_VDSO_GETTIMEOFDAY_H
#ifndef __ASSEMBLY__
#include <asm/unistd.h>
#include <uapi/linux/time.h>
#include <asm/vdso/compat_barrier.h>
#define __VDSO_USE_SYSCALL ULLONG_MAX
#define VDSO_HAS_CLOCK_GETRES 1
static __always_inline
int gettimeofday_fallback(struct __kernel_old_timeval *_tv,
struct timezone *_tz)
{
register struct timezone *tz asm("r1") = _tz;
register struct __kernel_old_timeval *tv asm("r0") = _tv;
register long ret asm ("r0");
register long nr asm("r7") = __NR_compat_gettimeofday;
asm volatile(
" swi #0\n"
: "=r" (ret)
: "r" (tv), "r" (tz), "r" (nr)
: "memory");
return ret;
}
static __always_inline
long clock_gettime_fallback(clockid_t _clkid, struct __kernel_timespec *_ts)
{
register struct __kernel_timespec *ts asm("r1") = _ts;
register clockid_t clkid asm("r0") = _clkid;
register long ret asm ("r0");
register long nr asm("r7") = __NR_compat_clock_gettime64;
asm volatile(
" swi #0\n"
: "=r" (ret)
: "r" (clkid), "r" (ts), "r" (nr)
: "memory");
return ret;
}
static __always_inline
int clock_getres_fallback(clockid_t _clkid, struct __kernel_timespec *_ts)
{
register struct __kernel_timespec *ts asm("r1") = _ts;
register clockid_t clkid asm("r0") = _clkid;
register long ret asm ("r0");
register long nr asm("r7") = __NR_compat_clock_getres_time64;
/* The checks below are required for ABI consistency with arm */
if ((_clkid >= MAX_CLOCKS) && (_ts == NULL))
return -EINVAL;
asm volatile(
" swi #0\n"
: "=r" (ret)
: "r" (clkid), "r" (ts), "r" (nr)
: "memory");
return ret;
}
static __always_inline u64 __arch_get_hw_counter(s32 clock_mode)
{
u64 res;
/*
* clock_mode == 0 implies that vDSO are enabled otherwise
* fallback on syscall.
*/
if (clock_mode)
return __VDSO_USE_SYSCALL;
/*
* This isb() is required to prevent that the counter value
* is speculated.
*/
isb();
asm volatile("mrrc p15, 1, %Q0, %R0, c14" : "=r" (res));
/*
* This isb() is required to prevent that the seq lock is
* speculated.
*/
isb();
return res;
}
static __always_inline const struct vdso_data *__arch_get_vdso_data(void)
{
const struct vdso_data *ret;
/*
* This simply puts &_vdso_data into ret. The reason why we don't use
* `ret = _vdso_data` is that the compiler tends to optimise this in a
* very suboptimal way: instead of keeping &_vdso_data in a register,
* it goes through a relocation almost every time _vdso_data must be
* accessed (even in subfunctions). This is both time and space
* consuming: each relocation uses a word in the code section, and it
* has to be loaded at runtime.
*
* This trick hides the assignment from the compiler. Since it cannot
* track where the pointer comes from, it will only use one relocation
* where __arch_get_vdso_data() is called, and then keep the result in
* a register.
*/
asm volatile("mov %0, %1" : "=r"(ret) : "r"(_vdso_data));
return ret;
}
#endif /* !__ASSEMBLY__ */
#endif /* __ASM_VDSO_GETTIMEOFDAY_H */
arch/arm64/include/asm/vdso/gettimeofday.h 0 → 100644
View file @ 927ba67a
/* SPDX-License-Identifier: GPL-2.0 */
/*
* Copyright (C) 2018 ARM Limited
*/
#ifndef __ASM_VDSO_GETTIMEOFDAY_H
#define __ASM_VDSO_GETTIMEOFDAY_H
#ifndef __ASSEMBLY__
#include <asm/unistd.h>
#include <uapi/linux/time.h>
#define __VDSO_USE_SYSCALL ULLONG_MAX
#define VDSO_HAS_CLOCK_GETRES 1
static __always_inline
int gettimeofday_fallback(struct __kernel_old_timeval *_tv,
struct timezone *_tz)
{
register struct timezone *tz asm("x1") = _tz;
register struct __kernel_old_timeval *tv asm("x0") = _tv;
register long ret asm ("x0");
register long nr asm("x8") = __NR_gettimeofday;
asm volatile(
" svc #0\n"
: "=r" (ret)
: "r" (tv), "r" (tz), "r" (nr)
: "memory");
return ret;
}
static __always_inline
long clock_gettime_fallback(clockid_t _clkid, struct __kernel_timespec *_ts)
{
register struct __kernel_timespec *ts asm("x1") = _ts;
register clockid_t clkid asm("x0") = _clkid;
register long ret asm ("x0");
register long nr asm("x8") = __NR_clock_gettime;
asm volatile(
" svc #0\n"
: "=r" (ret)
: "r" (clkid), "r" (ts), "r" (nr)
: "memory");
return ret;
}
static __always_inline
int clock_getres_fallback(clockid_t _clkid, struct __kernel_timespec *_ts)
{
register struct __kernel_timespec *ts asm("x1") = _ts;
register clockid_t clkid asm("x0") = _clkid;
register long ret asm ("x0");
register long nr asm("x8") = __NR_clock_getres;
asm volatile(
" svc #0\n"
: "=r" (ret)
: "r" (clkid), "r" (ts), "r" (nr)
: "memory");
return ret;
}
static __always_inline u64 __arch_get_hw_counter(s32 clock_mode)
{
u64 res;
/*
* clock_mode == 0 implies that vDSO are enabled otherwise
* fallback on syscall.
*/
if (clock_mode)
return __VDSO_USE_SYSCALL;
/*
* This isb() is required to prevent that the counter value
* is speculated.
*/
isb();
asm volatile("mrs %0, cntvct_el0" : "=r" (res) :: "memory");
/*
* This isb() is required to prevent that the seq lock is
* speculated.#
*/
isb();
return res;
}
static __always_inline
const struct vdso_data *__arch_get_vdso_data(void)
{
return _vdso_data;
}
#endif /* !__ASSEMBLY__ */
#endif /* __ASM_VDSO_GETTIMEOFDAY_H */
arch/arm64/include/asm/vdso/vsyscall.h 0 → 100644
View file @ 927ba67a
/* SPDX-License-Identifier: GPL-2.0 */
#ifndef __ASM_VDSO_VSYSCALL_H
#define __ASM_VDSO_VSYSCALL_H
#ifndef __ASSEMBLY__
#include <linux/timekeeper_internal.h>
#include <vdso/datapage.h>
#define VDSO_PRECISION_MASK ~(0xFF00ULL<<48)
extern struct vdso_data *vdso_data;
/*
* Update the vDSO data page to keep in sync with kernel timekeeping.
*/
static __always_inline
struct vdso_data *__arm64_get_k_vdso_data(void)
{
return vdso_data;
}
#define __arch_get_k_vdso_data __arm64_get_k_vdso_data
static __always_inline
int __arm64_get_clock_mode(struct timekeeper *tk)
{
u32 use_syscall = !tk->tkr_mono.clock->archdata.vdso_direct;
return use_syscall;
}
#define __arch_get_clock_mode __arm64_get_clock_mode
static __always_inline
int __arm64_use_vsyscall(struct vdso_data *vdata)
{
return !vdata[CS_HRES_COARSE].clock_mode;
}
#define __arch_use_vsyscall __arm64_use_vsyscall
static __always_inline
void __arm64_update_vsyscall(struct vdso_data *vdata, struct timekeeper *tk)
{
vdata[CS_HRES_COARSE].mask = VDSO_PRECISION_MASK;
vdata[CS_RAW].mask = VDSO_PRECISION_MASK;
}
#define __arch_update_vsyscall __arm64_update_vsyscall
/* The asm-generic header needs to be included after the definitions above */
#include <asm-generic/vdso/vsyscall.h>
#endif /* !__ASSEMBLY__ */
#endif /* __ASM_VDSO_VSYSCALL_H */
arch/arm64/kernel/Makefile
View file @ 927ba67a
......@@ -28,7 +28,10 @@ $(obj)/%.stub.o: $(obj)/%.o FORCE
$(call if_changed,objcopy)
obj-$(CONFIG_COMPAT) += sys32.o signal32.o \
sigreturn32.o sys_compat.o
sys_compat.o
ifneq ($(CONFIG_COMPAT_VDSO), y)
obj-$(CONFIG_COMPAT) += sigreturn32.o
endif
obj-$(CONFIG_KUSER_HELPERS) += kuser32.o
obj-$(CONFIG_FUNCTION_TRACER) += ftrace.o entry-ftrace.o
obj-$(CONFIG_MODULES) += module.o
......@@ -62,6 +65,7 @@ obj-$(CONFIG_ARM64_SSBD) += ssbd.o
obj-$(CONFIG_ARM64_PTR_AUTH) += pointer_auth.o
obj-y += vdso/ probes/
obj-$(CONFIG_COMPAT_VDSO) += vdso32/
head-y := head.o
extra-y += $(head-y) vmlinux.lds
......
arch/arm64/kernel/asm-offsets.c
View file @ 927ba67a
......@@ -18,9 +18,9 @@
#include <asm/fixmap.h>
#include <asm/thread_info.h>
#include <asm/memory.h>
#include <asm/signal32.h>
#include <asm/smp_plat.h>
#include <asm/suspend.h>
#include <asm/vdso_datapage.h>
#include <linux/kbuild.h>
#include <linux/arm-smccc.h>
......@@ -66,6 +66,11 @@ int main(void)
DEFINE(S_STACKFRAME, offsetof(struct pt_regs, stackframe));
DEFINE(S_FRAME_SIZE, sizeof(struct pt_regs));
BLANK();
#ifdef CONFIG_COMPAT
DEFINE(COMPAT_SIGFRAME_REGS_OFFSET, offsetof(struct compat_sigframe, uc.uc_mcontext.arm_r0));
DEFINE(COMPAT_RT_SIGFRAME_REGS_OFFSET, offsetof(struct compat_rt_sigframe, sig.uc.uc_mcontext.arm_r0));
BLANK();
#endif
DEFINE(MM_CONTEXT_ID, offsetof(struct mm_struct, context.id.counter));
BLANK();
DEFINE(VMA_VM_MM, offsetof(struct vm_area_struct, vm_mm));
......@@ -80,33 +85,6 @@ int main(void)
BLANK();
DEFINE(PREEMPT_DISABLE_OFFSET, PREEMPT_DISABLE_OFFSET);
BLANK();
DEFINE(CLOCK_REALTIME, CLOCK_REALTIME);
DEFINE(CLOCK_MONOTONIC, CLOCK_MONOTONIC);
DEFINE(CLOCK_MONOTONIC_RAW, CLOCK_MONOTONIC_RAW);
DEFINE(CLOCK_REALTIME_RES, offsetof(struct vdso_data, hrtimer_res));
DEFINE(CLOCK_REALTIME_COARSE, CLOCK_REALTIME_COARSE);
DEFINE(CLOCK_MONOTONIC_COARSE,CLOCK_MONOTONIC_COARSE);
DEFINE(CLOCK_COARSE_RES, LOW_RES_NSEC);
DEFINE(NSEC_PER_SEC, NSEC_PER_SEC);
BLANK();
DEFINE(VDSO_CS_CYCLE_LAST, offsetof(struct vdso_data, cs_cycle_last));
DEFINE(VDSO_RAW_TIME_SEC, offsetof(struct vdso_data, raw_time_sec));
DEFINE(VDSO_XTIME_CLK_SEC, offsetof(struct vdso_data, xtime_clock_sec));
DEFINE(VDSO_XTIME_CRS_SEC, offsetof(struct vdso_data, xtime_coarse_sec));
DEFINE(VDSO_XTIME_CRS_NSEC, offsetof(struct vdso_data, xtime_coarse_nsec));
DEFINE(VDSO_WTM_CLK_SEC, offsetof(struct vdso_data, wtm_clock_sec));
DEFINE(VDSO_TB_SEQ_COUNT, offsetof(struct vdso_data, tb_seq_count));
DEFINE(VDSO_CS_MONO_MULT, offsetof(struct vdso_data, cs_mono_mult));
DEFINE(VDSO_CS_SHIFT, offsetof(struct vdso_data, cs_shift));
DEFINE(VDSO_TZ_MINWEST, offsetof(struct vdso_data, tz_minuteswest));
DEFINE(VDSO_USE_SYSCALL, offsetof(struct vdso_data, use_syscall));
BLANK();
DEFINE(TVAL_TV_SEC, offsetof(struct timeval, tv_sec));
DEFINE(TSPEC_TV_SEC, offsetof(struct timespec, tv_sec));
BLANK();
DEFINE(TZ_MINWEST, offsetof(struct timezone, tz_minuteswest));
DEFINE(TZ_DSTTIME, offsetof(struct timezone, tz_dsttime));
BLANK();
DEFINE(CPU_BOOT_STACK, offsetof(struct secondary_data, stack));
DEFINE(CPU_BOOT_TASK, offsetof(struct secondary_data, task));
BLANK();
......
arch/arm64/kernel/signal32.c
View file @ 927ba67a
......@@ -18,42 +18,7 @@
#include <asm/traps.h>
#include <linux/uaccess.h>
#include <asm/unistd.h>
struct compat_sigcontext {
/* We always set these two fields to 0 */
compat_ulong_t trap_no;
compat_ulong_t error_code;
compat_ulong_t oldmask;
compat_ulong_t arm_r0;
compat_ulong_t arm_r1;
compat_ulong_t arm_r2;
compat_ulong_t arm_r3;
compat_ulong_t arm_r4;
compat_ulong_t arm_r5;
compat_ulong_t arm_r6;
compat_ulong_t arm_r7;
compat_ulong_t arm_r8;
compat_ulong_t arm_r9;
compat_ulong_t arm_r10;
compat_ulong_t arm_fp;
compat_ulong_t arm_ip;
compat_ulong_t arm_sp;
compat_ulong_t arm_lr;
compat_ulong_t arm_pc;
compat_ulong_t arm_cpsr;
compat_ulong_t fault_address;
};
struct compat_ucontext {
compat_ulong_t uc_flags;
compat_uptr_t uc_link;
compat_stack_t uc_stack;
struct compat_sigcontext uc_mcontext;
compat_sigset_t uc_sigmask;
int __unused[32 - (sizeof (compat_sigset_t) / sizeof (int))];
compat_ulong_t uc_regspace[128] __attribute__((__aligned__(8)));
};
#include <asm/vdso.h>
struct compat_vfp_sigframe {
compat_ulong_t magic;
......@@ -81,16 +46,6 @@ struct compat_aux_sigframe {
unsigned long end_magic;
} __attribute__((__aligned__(8)));
struct compat_sigframe {
struct compat_ucontext uc;
compat_ulong_t retcode[2];
};
struct compat_rt_sigframe {
struct compat_siginfo info;
struct compat_sigframe sig;
};
#define _BLOCKABLE (~(sigmask(SIGKILL) | sigmask(SIGSTOP)))
static inline int put_sigset_t(compat_sigset_t __user *uset, sigset_t *set)
......@@ -387,6 +342,30 @@ static void compat_setup_return(struct pt_regs *regs, struct k_sigaction *ka,
retcode = ptr_to_compat(ka->sa.sa_restorer);
} else {
/* Set up sigreturn pointer */
#ifdef CONFIG_COMPAT_VDSO
void *vdso_base = current->mm->context.vdso;
void *vdso_trampoline;
if (ka->sa.sa_flags & SA_SIGINFO) {
if (thumb) {
vdso_trampoline = VDSO_SYMBOL(vdso_base,
compat_rt_sigreturn_thumb);
} else {
vdso_trampoline = VDSO_SYMBOL(vdso_base,
compat_rt_sigreturn_arm);
}
} else {
if (thumb) {
vdso_trampoline = VDSO_SYMBOL(vdso_base,
compat_sigreturn_thumb);
} else {
vdso_trampoline = VDSO_SYMBOL(vdso_base,
compat_sigreturn_arm);
}
}
retcode = ptr_to_compat(vdso_trampoline) + thumb;
#else
unsigned int idx = thumb << 1;
if (ka->sa.sa_flags & SA_SIGINFO)
......@@ -394,6 +373,7 @@ static void compat_setup_return(struct pt_regs *regs, struct k_sigaction *ka,
retcode = (unsigned long)current->mm->context.vdso +
(idx << 2) + thumb;
#endif
}
regs->regs[0] = usig;
......
arch/arm64/kernel/vdso.c
View file @ 927ba67a
This diff is collapsed.
arch/arm64/kernel/vdso/Makefile
View file @ 927ba67a
......@@ -6,7 +6,12 @@
# Heavily based on the vDSO Makefiles for other archs.
#
obj-vdso := gettimeofday.o note.o sigreturn.o
# Absolute relocation type $(ARCH_REL_TYPE_ABS) needs to be defined before
# the inclusion of generic Makefile.
ARCH_REL_TYPE_ABS := R_AARCH64_JUMP_SLOT|R_AARCH64_GLOB_DAT|R_AARCH64_ABS64
include $(srctree)/lib/vdso/Makefile
obj-vdso := vgettimeofday.o note.o sigreturn.o
# Build rules
targets := $(obj-vdso) vdso.so vdso.so.dbg
......@@ -15,6 +20,31 @@ obj-vdso := $(addprefix $(obj)/, $(obj-vdso))
ldflags-y := -shared -nostdlib -soname=linux-vdso.so.1 --hash-style=sysv \
--build-id -n -T
ccflags-y := -fno-common -fno-builtin -fno-stack-protector -ffixed-x18
ccflags-y += -DDISABLE_BRANCH_PROFILING
VDSO_LDFLAGS := -Bsymbolic
CFLAGS_REMOVE_vgettimeofday.o = $(CC_FLAGS_FTRACE) -Os
KBUILD_CFLAGS += $(DISABLE_LTO)
KASAN_SANITIZE := n
UBSAN_SANITIZE := n
OBJECT_FILES_NON_STANDARD := y
KCOV_INSTRUMENT := n
ifeq ($(c-gettimeofday-y),)
CFLAGS_vgettimeofday.o = -O2 -mcmodel=tiny
else
CFLAGS_vgettimeofday.o = -O2 -mcmodel=tiny -include $(c-gettimeofday-y)
endif
# Clang versions less than 8 do not support -mcmodel=tiny
ifeq ($(CONFIG_CC_IS_CLANG), y)
ifeq ($(shell test $(CONFIG_CLANG_VERSION) -lt 80000; echo $$?),0)
CFLAGS_REMOVE_vgettimeofday.o += -mcmodel=tiny
endif
endif
# Disable gcov profiling for VDSO code
GCOV_PROFILE := n
......@@ -28,6 +58,7 @@ $(obj)/vdso.o : $(obj)/vdso.so
# Link rule for the .so file, .lds has to be first
$(obj)/vdso.so.dbg: $(obj)/vdso.lds $(obj-vdso) FORCE
$(call if_changed,ld)
$(call if_changed,vdso_check)
# Strip rule for the .so file
$(obj)/%.so: OBJCOPYFLAGS := -S
......@@ -42,13 +73,9 @@ quiet_cmd_vdsosym = VDSOSYM $@
include/generated/vdso-offsets.h: $(obj)/vdso.so.dbg FORCE
$(call if_changed,vdsosym)
# Assembly rules for the .S files
$(obj-vdso): %.o: %.S FORCE
$(call if_changed_dep,vdsoas)
# Actual build commands
quiet_cmd_vdsoas = VDSOA $@
cmd_vdsoas = $(CC) $(a_flags) -c -o $@ $<
quiet_cmd_vdsocc = VDSOCC $@
cmd_vdsocc = $(CC) $(a_flags) $(c_flags) -c -o $@ $<
# Install commands for the unstripped file
quiet_cmd_vdso_install = INSTALL $@
......
arch/arm64/kernel/vdso/gettimeofday.S
View file @ 927ba67a
/* SPDX-License-Identifier: GPL-2.0-only */
/*
* Userspace implementations of gettimeofday() and friends.
*
* Copyright (C) 2012 ARM Limited
*
* Author: Will Deacon <will.deacon@arm.com>
*/
#include <linux/linkage.h>
#include <asm/asm-offsets.h>
#include <asm/unistd.h>
#define NSEC_PER_SEC_LO16 0xca00
#define NSEC_PER_SEC_HI16 0x3b9a
vdso_data .req x6
seqcnt .req w7
w_tmp .req w8
x_tmp .req x8
/*
* Conventions for macro arguments:
* - An argument is write-only if its name starts with "res".
* - All other arguments are read-only, unless otherwise specified.
*/
.macro seqcnt_acquire
9999: ldr seqcnt, [vdso_data, #VDSO_TB_SEQ_COUNT]
tbnz seqcnt, #0, 9999b
dmb ishld
.endm
.macro seqcnt_check fail
dmb ishld
ldr w_tmp, [vdso_data, #VDSO_TB_SEQ_COUNT]
cmp w_tmp, seqcnt
b.ne \fail
.endm
.macro syscall_check fail
ldr w_tmp, [vdso_data, #VDSO_USE_SYSCALL]
cbnz w_tmp, \fail
.endm
.macro get_nsec_per_sec res
mov \res, #NSEC_PER_SEC_LO16
movk \res, #NSEC_PER_SEC_HI16, lsl #16
.endm
/*
* Returns the clock delta, in nanoseconds left-shifted by the clock
* shift.
*/
.macro get_clock_shifted_nsec res, cycle_last, mult
/* Read the virtual counter. */
isb
mrs x_tmp, cntvct_el0
/* Calculate cycle delta and convert to ns. */
sub \res, x_tmp, \cycle_last
/* We can only guarantee 56 bits of precision. */
movn x_tmp, #0xff00, lsl #48
and \res, x_tmp, \res
mul \res, \res, \mult
/*
* Fake address dependency from the value computed from the counter
* register to subsequent data page accesses so that the sequence
* locking also orders the read of the counter.
*/
and x_tmp, \res, xzr
add vdso_data, vdso_data, x_tmp
.endm
/*
* Returns in res_{sec,nsec} the REALTIME timespec, based on the
* "wall time" (xtime) and the clock_mono delta.
*/
.macro get_ts_realtime res_sec, res_nsec, \
clock_nsec, xtime_sec, xtime_nsec, nsec_to_sec
add \res_nsec, \clock_nsec, \xtime_nsec
udiv x_tmp, \res_nsec, \nsec_to_sec
add \res_sec, \xtime_sec, x_tmp
msub \res_nsec, x_tmp, \nsec_to_sec, \res_nsec
.endm
/*
* Returns in res_{sec,nsec} the timespec based on the clock_raw delta,
* used for CLOCK_MONOTONIC_RAW.
*/
.macro get_ts_clock_raw res_sec, res_nsec, clock_nsec, nsec_to_sec
udiv \res_sec, \clock_nsec, \nsec_to_sec
msub \res_nsec, \res_sec, \nsec_to_sec, \clock_nsec
.endm
/* sec and nsec are modified in place. */
.macro add_ts sec, nsec, ts_sec, ts_nsec, nsec_to_sec
/* Add timespec. */
add \sec, \sec, \ts_sec
add \nsec, \nsec, \ts_nsec
/* Normalise the new timespec. */
cmp \nsec, \nsec_to_sec
b.lt 9999f
sub \nsec, \nsec, \nsec_to_sec
add \sec, \sec, #1
9999:
cmp \nsec, #0
b.ge 9998f
add \nsec, \nsec, \nsec_to_sec
sub \sec, \sec, #1
9998:
.endm
.macro clock_gettime_return, shift=0
.if \shift == 1
lsr x11, x11, x12
.endif
stp x10, x11, [x1, #TSPEC_TV_SEC]
mov x0, xzr
ret
.endm
.macro jump_slot jumptable, index, label
.if (. - \jumptable) != 4 * (\index)
.error "Jump slot index mismatch"
.endif
b \label
.endm
.text
/* int __kernel_gettimeofday(struct timeval *tv, struct timezone *tz); */
ENTRY(__kernel_gettimeofday)
.cfi_startproc
adr vdso_data, _vdso_data
/* If tv is NULL, skip to the timezone code. */
cbz x0, 2f
/* Compute the time of day. */
1: seqcnt_acquire
syscall_check fail=4f
ldr x10, [vdso_data, #VDSO_CS_CYCLE_LAST]
/* w11 = cs_mono_mult, w12 = cs_shift */
ldp w11, w12, [vdso_data, #VDSO_CS_MONO_MULT]
ldp x13, x14, [vdso_data, #VDSO_XTIME_CLK_SEC]
get_nsec_per_sec res=x9
lsl x9, x9, x12
get_clock_shifted_nsec res=x15, cycle_last=x10, mult=x11
seqcnt_check fail=1b
get_ts_realtime res_sec=x10, res_nsec=x11, \
clock_nsec=x15, xtime_sec=x13, xtime_nsec=x14, nsec_to_sec=x9
/* Convert ns to us. */
mov x13, #1000
lsl x13, x13, x12
udiv x11, x11, x13
stp x10, x11, [x0, #TVAL_TV_SEC]
2:
/* If tz is NULL, return 0. */
cbz x1, 3f
ldp w4, w5, [vdso_data, #VDSO_TZ_MINWEST]
stp w4, w5, [x1, #TZ_MINWEST]
3:
mov x0, xzr
ret
4:
/* Syscall fallback. */
mov x8, #__NR_gettimeofday
svc #0
ret
.cfi_endproc
ENDPROC(__kernel_gettimeofday)
#define JUMPSLOT_MAX CLOCK_MONOTONIC_COARSE
/* int __kernel_clock_gettime(clockid_t clock_id, struct timespec *tp); */
ENTRY(__kernel_clock_gettime)
.cfi_startproc
cmp w0, #JUMPSLOT_MAX
b.hi syscall
adr vdso_data, _vdso_data
adr x_tmp, jumptable
add x_tmp, x_tmp, w0, uxtw #2
br x_tmp
ALIGN
jumptable:
jump_slot jumptable, CLOCK_REALTIME, realtime
jump_slot jumptable, CLOCK_MONOTONIC, monotonic
b syscall
b syscall
jump_slot jumptable, CLOCK_MONOTONIC_RAW, monotonic_raw
jump_slot jumptable, CLOCK_REALTIME_COARSE, realtime_coarse
jump_slot jumptable, CLOCK_MONOTONIC_COARSE, monotonic_coarse
.if (. - jumptable) != 4 * (JUMPSLOT_MAX + 1)
.error "Wrong jumptable size"
.endif
ALIGN
realtime:
seqcnt_acquire
syscall_check fail=syscall
ldr x10, [vdso_data, #VDSO_CS_CYCLE_LAST]
/* w11 = cs_mono_mult, w12 = cs_shift */
ldp w11, w12, [vdso_data, #VDSO_CS_MONO_MULT]
ldp x13, x14, [vdso_data, #VDSO_XTIME_CLK_SEC]
/* All computations are done with left-shifted nsecs. */
get_nsec_per_sec res=x9
lsl x9, x9, x12
get_clock_shifted_nsec res=x15, cycle_last=x10, mult=x11
seqcnt_check fail=realtime
get_ts_realtime res_sec=x10, res_nsec=x11, \
clock_nsec=x15, xtime_sec=x13, xtime_nsec=x14, nsec_to_sec=x9
clock_gettime_return, shift=1
ALIGN
monotonic:
seqcnt_acquire
syscall_check fail=syscall
ldr x10, [vdso_data, #VDSO_CS_CYCLE_LAST]
/* w11 = cs_mono_mult, w12 = cs_shift */
ldp w11, w12, [vdso_data, #VDSO_CS_MONO_MULT]
ldp x13, x14, [vdso_data, #VDSO_XTIME_CLK_SEC]
ldp x3, x4, [vdso_data, #VDSO_WTM_CLK_SEC]
/* All computations are done with left-shifted nsecs. */
lsl x4, x4, x12
get_nsec_per_sec res=x9
lsl x9, x9, x12
get_clock_shifted_nsec res=x15, cycle_last=x10, mult=x11
seqcnt_check fail=monotonic
get_ts_realtime res_sec=x10, res_nsec=x11, \
clock_nsec=x15, xtime_sec=x13, xtime_nsec=x14, nsec_to_sec=x9
add_ts sec=x10, nsec=x11, ts_sec=x3, ts_nsec=x4, nsec_to_sec=x9
clock_gettime_return, shift=1
ALIGN
monotonic_raw:
seqcnt_acquire
syscall_check fail=syscall
ldr x10, [vdso_data, #VDSO_CS_CYCLE_LAST]
/* w11 = cs_raw_mult, w12 = cs_shift */
ldp w12, w11, [vdso_data, #VDSO_CS_SHIFT]
ldp x13, x14, [vdso_data, #VDSO_RAW_TIME_SEC]
/* All computations are done with left-shifted nsecs. */
get_nsec_per_sec res=x9
lsl x9, x9, x12
get_clock_shifted_nsec res=x15, cycle_last=x10, mult=x11
seqcnt_check fail=monotonic_raw
get_ts_clock_raw res_sec=x10, res_nsec=x11, \
clock_nsec=x15, nsec_to_sec=x9
add_ts sec=x10, nsec=x11, ts_sec=x13, ts_nsec=x14, nsec_to_sec=x9
clock_gettime_return, shift=1
ALIGN
realtime_coarse:
seqcnt_acquire
ldp x10, x11, [vdso_data, #VDSO_XTIME_CRS_SEC]
seqcnt_check fail=realtime_coarse
clock_gettime_return
ALIGN
monotonic_coarse:
seqcnt_acquire
ldp x10, x11, [vdso_data, #VDSO_XTIME_CRS_SEC]
ldp x13, x14, [vdso_data, #VDSO_WTM_CLK_SEC]
seqcnt_check fail=monotonic_coarse
/* Computations are done in (non-shifted) nsecs. */
get_nsec_per_sec res=x9
add_ts sec=x10, nsec=x11, ts_sec=x13, ts_nsec=x14, nsec_to_sec=x9
clock_gettime_return
ALIGN
syscall: /* Syscall fallback. */
mov x8, #__NR_clock_gettime
svc #0
ret
.cfi_endproc
ENDPROC(__kernel_clock_gettime)
/* int __kernel_clock_getres(clockid_t clock_id, struct timespec *res); */
ENTRY(__kernel_clock_getres)
.cfi_startproc
cmp w0, #CLOCK_REALTIME
ccmp w0, #CLOCK_MONOTONIC, #0x4, ne
ccmp w0, #CLOCK_MONOTONIC_RAW, #0x4, ne
b.ne 1f
adr vdso_data, _vdso_data
ldr w2, [vdso_data, #CLOCK_REALTIME_RES]
b 2f
1:
cmp w0, #CLOCK_REALTIME_COARSE
ccmp w0, #CLOCK_MONOTONIC_COARSE, #0x4, ne
b.ne 4f
ldr x2, 5f
2:
cbz x1, 3f
stp xzr, x2, [x1]
3: /* res == NULL. */
mov w0, wzr
ret
4: /* Syscall fallback. */
mov x8, #__NR_clock_getres
svc #0
ret
5:
.quad CLOCK_COARSE_RES
.cfi_endproc
ENDPROC(__kernel_clock_getres)
arch/arm64/kernel/vdso/vgettimeofday.c 0 → 100644
View file @ 927ba67a
// SPDX-License-Identifier: GPL-2.0
/*
* ARM64 userspace implementations of gettimeofday() and similar.
*
* Copyright (C) 2018 ARM Limited
*
*/
#include <linux/time.h>
#include <linux/types.h>
int __kernel_clock_gettime(clockid_t clock,
struct __kernel_timespec *ts)
{
return __cvdso_clock_gettime(clock, ts);
}
int __kernel_gettimeofday(struct __kernel_old_timeval *tv,
struct timezone *tz)
{
return __cvdso_gettimeofday(tv, tz);
}
int __kernel_clock_getres(clockid_t clock_id,
struct __kernel_timespec *res)
{
return __cvdso_clock_getres(clock_id, res);
}
arch/arm64/kernel/vdso32/Makefile 0 → 100644
View file @ 927ba67a
# SPDX-License-Identifier: GPL-2.0
#
# Makefile for vdso32
#
# Absolute relocation type $(ARCH_REL_TYPE_ABS) needs to be defined before
# the inclusion of generic Makefile.
ARCH_REL_TYPE_ABS := R_ARM_JUMP_SLOT|R_ARM_GLOB_DAT|R_ARM_ABS32
include $(srctree)/lib/vdso/Makefile
COMPATCC := $(CROSS_COMPILE_COMPAT)gcc
# Same as cc-*option, but using COMPATCC instead of CC
cc32-option = $(call try-run,\
$(COMPATCC) $(1) -c -x c /dev/null -o "$$TMP",$(1),$(2))
cc32-disable-warning = $(call try-run,\
$(COMPATCC) -W$(strip $(1)) -c -x c /dev/null -o "$$TMP",-Wno-$(strip $(1)))
cc32-ldoption = $(call try-run,\
$(COMPATCC) $(1) -nostdlib -x c /dev/null -o "$$TMP",$(1),$(2))
# We cannot use the global flags to compile the vDSO files, the main reason
# being that the 32-bit compiler may be older than the main (64-bit) compiler
# and therefore may not understand flags set using $(cc-option ...). Besides,
# arch-specific options should be taken from the arm Makefile instead of the
# arm64 one.
# As a result we set our own flags here.
# From top-level Makefile
# NOSTDINC_FLAGS
VDSO_CPPFLAGS := -nostdinc -isystem $(shell $(COMPATCC) -print-file-name=include)
VDSO_CPPFLAGS += $(LINUXINCLUDE)
VDSO_CPPFLAGS += $(KBUILD_CPPFLAGS)
# Common C and assembly flags
# From top-level Makefile
VDSO_CAFLAGS := $(VDSO_CPPFLAGS)
VDSO_CAFLAGS += $(call cc32-option,-fno-PIE)
ifdef CONFIG_DEBUG_INFO
VDSO_CAFLAGS += -g
endif
ifeq ($(shell $(CONFIG_SHELL) $(srctree)/scripts/gcc-goto.sh $(COMPATCC)), y)
VDSO_CAFLAGS += -DCC_HAVE_ASM_GOTO
endif
# From arm Makefile
VDSO_CAFLAGS += $(call cc32-option,-fno-dwarf2-cfi-asm)
VDSO_CAFLAGS += -mabi=aapcs-linux -mfloat-abi=soft
ifeq ($(CONFIG_CPU_BIG_ENDIAN), y)
VDSO_CAFLAGS += -mbig-endian
else
VDSO_CAFLAGS += -mlittle-endian
endif
# From arm vDSO Makefile
VDSO_CAFLAGS += -fPIC -fno-builtin -fno-stack-protector
VDSO_CAFLAGS += -DDISABLE_BRANCH_PROFILING
# Try to compile for ARMv8. If the compiler is too old and doesn't support it,
# fall back to v7. There is no easy way to check for what architecture the code
# is being compiled, so define a macro specifying that (see arch/arm/Makefile).
VDSO_CAFLAGS += $(call cc32-option,-march=armv8-a -D__LINUX_ARM_ARCH__=8,\
-march=armv7-a -D__LINUX_ARM_ARCH__=7)
VDSO_CFLAGS := $(VDSO_CAFLAGS)
VDSO_CFLAGS += -DENABLE_COMPAT_VDSO=1
# KBUILD_CFLAGS from top-level Makefile
VDSO_CFLAGS += -Wall -Wundef -Wstrict-prototypes -Wno-trigraphs \
-fno-strict-aliasing -fno-common \
-Werror-implicit-function-declaration \
-Wno-format-security \
-std=gnu89
VDSO_CFLAGS += -O2
# Some useful compiler-dependent flags from top-level Makefile
VDSO_CFLAGS += $(call cc32-option,-Wdeclaration-after-statement,)
VDSO_CFLAGS += $(call cc32-option,-Wno-pointer-sign)
VDSO_CFLAGS += $(call cc32-option,-fno-strict-overflow)
VDSO_CFLAGS += $(call cc32-option,-Werror=strict-prototypes)
VDSO_CFLAGS += $(call cc32-option,-Werror=date-time)
VDSO_CFLAGS += $(call cc32-option,-Werror=incompatible-pointer-types)
# The 32-bit compiler does not provide 128-bit integers, which are used in
# some headers that are indirectly included from the vDSO code.
# This hack makes the compiler happy and should trigger a warning/error if
# variables of such type are referenced.
VDSO_CFLAGS += -D__uint128_t='void*'
# Silence some warnings coming from headers that operate on long's
# (on GCC 4.8 or older, there is unfortunately no way to silence this warning)
VDSO_CFLAGS += $(call cc32-disable-warning,shift-count-overflow)
VDSO_CFLAGS += -Wno-int-to-pointer-cast
VDSO_AFLAGS := $(VDSO_CAFLAGS)
VDSO_AFLAGS += -D__ASSEMBLY__
VDSO_LDFLAGS := $(VDSO_CPPFLAGS)
# From arm vDSO Makefile
VDSO_LDFLAGS += -Wl,-Bsymbolic -Wl,--no-undefined -Wl,-soname=linux-vdso.so.1
VDSO_LDFLAGS += -Wl,-z,max-page-size=4096 -Wl,-z,common-page-size=4096
VDSO_LDFLAGS += -nostdlib -shared -mfloat-abi=soft
VDSO_LDFLAGS += $(call cc32-ldoption,-Wl$(comma)--hash-style=sysv)
VDSO_LDFLAGS += $(call cc32-ldoption,-Wl$(comma)--build-id)
VDSO_LDFLAGS += $(call cc32-ldoption,-fuse-ld=bfd)
# Borrow vdsomunge.c from the arm vDSO
# We have to use a relative path because scripts/Makefile.host prefixes
# $(hostprogs-y) with $(obj)
munge := ../../../arm/vdso/vdsomunge
hostprogs-y := $(munge)
c-obj-vdso := note.o
c-obj-vdso-gettimeofday := vgettimeofday.o
asm-obj-vdso := sigreturn.o
ifneq ($(c-gettimeofday-y),)
VDSO_CFLAGS_gettimeofday_o += -include $(c-gettimeofday-y)
endif
VDSO_CFLAGS_REMOVE_vgettimeofday.o = $(CC_FLAGS_FTRACE) -Os
# Build rules
targets := $(c-obj-vdso) $(c-obj-vdso-gettimeofday) $(asm-obj-vdso) vdso.so vdso.so.dbg vdso.so.raw
c-obj-vdso := $(addprefix $(obj)/, $(c-obj-vdso))
c-obj-vdso-gettimeofday := $(addprefix $(obj)/, $(c-obj-vdso-gettimeofday))
asm-obj-vdso := $(addprefix $(obj)/, $(asm-obj-vdso))
obj-vdso := $(c-obj-vdso) $(c-obj-vdso-gettimeofday) $(asm-obj-vdso)
obj-y += vdso.o
extra-y += vdso.lds
CPPFLAGS_vdso.lds += -P -C -U$(ARCH)
# Force dependency (vdso.s includes vdso.so through incbin)
$(obj)/vdso.o: $(obj)/vdso.so
include/generated/vdso32-offsets.h: $(obj)/vdso.so.dbg FORCE
$(call if_changed,vdsosym)
# Strip rule for vdso.so
$(obj)/vdso.so: OBJCOPYFLAGS := -S
$(obj)/vdso.so: $(obj)/vdso.so.dbg FORCE
$(call if_changed,objcopy)
$(obj)/vdso.so.dbg: $(obj)/vdso.so.raw $(obj)/$(munge) FORCE
$(call if_changed,vdsomunge)
# Link rule for the .so file, .lds has to be first
$(obj)/vdso.so.raw: $(src)/vdso.lds $(obj-vdso) FORCE
$(call if_changed,vdsold)
$(call if_changed,vdso_check)
# Compilation rules for the vDSO sources
$(c-obj-vdso): %.o: %.c FORCE
$(call if_changed_dep,vdsocc)
$(c-obj-vdso-gettimeofday): %.o: %.c FORCE
$(call if_changed_dep,vdsocc_gettimeofday)
$(asm-obj-vdso): %.o: %.S FORCE
$(call if_changed_dep,vdsoas)
# Actual build commands
quiet_cmd_vdsold = VDSOL $@
cmd_vdsold = $(COMPATCC) -Wp,-MD,$(depfile) $(VDSO_LDFLAGS) \
-Wl,-T $(filter %.lds,$^) $(filter %.o,$^) -o $@
quiet_cmd_vdsocc = VDSOC $@
cmd_vdsocc = $(COMPATCC) -Wp,-MD,$(depfile) $(VDSO_CFLAGS) -c -o $@ $<
quiet_cmd_vdsocc_gettimeofday = VDSOC_GTD $@
cmd_vdsocc_gettimeofday = $(COMPATCC) -Wp,-MD,$(depfile) $(VDSO_CFLAGS) $(VDSO_CFLAGS_gettimeofday_o) -c -o $@ $<
quiet_cmd_vdsoas = VDSOA $@
cmd_vdsoas = $(COMPATCC) -Wp,-MD,$(depfile) $(VDSO_AFLAGS) -c -o $@ $<
quiet_cmd_vdsomunge = MUNGE $@
cmd_vdsomunge = $(obj)/$(munge) $< $@
# Generate vDSO offsets using helper script (borrowed from the 64-bit vDSO)
gen-vdsosym := $(srctree)/$(src)/../vdso/gen_vdso_offsets.sh
quiet_cmd_vdsosym = VDSOSYM $@
# The AArch64 nm should be able to read an AArch32 binary
cmd_vdsosym = $(NM) $< | $(gen-vdsosym) | LC_ALL=C sort > $@
# Install commands for the unstripped file
quiet_cmd_vdso_install = INSTALL $@
cmd_vdso_install = cp $(obj)/$@.dbg $(MODLIB)/vdso/vdso32.so
vdso.so: $(obj)/vdso.so.dbg
@mkdir -p $(MODLIB)/vdso
$(call cmd,vdso_install)
vdso_install: vdso.so
arch/arm64/kernel/vdso32/note.c 0 → 100644
View file @ 927ba67a
// SPDX-License-Identifier: GPL-2.0
/*
* Copyright (C) 2012-2018 ARM Limited
*
* This supplies .note.* sections to go into the PT_NOTE inside the vDSO text.
* Here we can supply some information useful to userland.
*/
#include <linux/uts.h>
#include <linux/version.h>
#include <linux/elfnote.h>
#include <linux/build-salt.h>
ELFNOTE32("Linux", 0, LINUX_VERSION_CODE);
BUILD_SALT;
arch/arm64/kernel/vdso32/sigreturn.S 0 → 100644
View file @ 927ba67a
/* SPDX-License-Identifier: GPL-2.0 */
/*
* This file provides both A32 and T32 versions, in accordance with the
* arm sigreturn code.
*
* Copyright (C) 2018 ARM Limited
*/
#include <linux/linkage.h>
#include <asm/asm-offsets.h>
#include <asm/unistd.h>
#define ARM_ENTRY(name) \
ENTRY(name)
#define ARM_ENDPROC(name) \
.type name, %function; \
END(name)
.text
.arm
.fnstart
.save {r0-r15}
.pad #COMPAT_SIGFRAME_REGS_OFFSET
nop
ARM_ENTRY(__kernel_sigreturn_arm)
mov r7, #__NR_compat_sigreturn
svc #0
.fnend
ARM_ENDPROC(__kernel_sigreturn_arm)
.fnstart
.save {r0-r15}
.pad #COMPAT_RT_SIGFRAME_REGS_OFFSET
nop
ARM_ENTRY(__kernel_rt_sigreturn_arm)
mov r7, #__NR_compat_rt_sigreturn
svc #0
.fnend
ARM_ENDPROC(__kernel_rt_sigreturn_arm)
.thumb
.fnstart
.save {r0-r15}
.pad #COMPAT_SIGFRAME_REGS_OFFSET
nop
ARM_ENTRY(__kernel_sigreturn_thumb)
mov r7, #__NR_compat_sigreturn
svc #0
.fnend
ARM_ENDPROC(__kernel_sigreturn_thumb)
.fnstart
.save {r0-r15}
.pad #COMPAT_RT_SIGFRAME_REGS_OFFSET
nop
ARM_ENTRY(__kernel_rt_sigreturn_thumb)
mov r7, #__NR_compat_rt_sigreturn
svc #0
.fnend
ARM_ENDPROC(__kernel_rt_sigreturn_thumb)
arch/arm64/kernel/vdso32/vdso.S 0 → 100644
View file @ 927ba67a
/* SPDX-License-Identifier: GPL-2.0 */
/*
* Copyright (C) 2012 ARM Limited
*/
#include <linux/init.h>
#include <linux/linkage.h>
#include <linux/const.h>
#include <asm/page.h>
.globl vdso32_start, vdso32_end
.section .rodata
.balign PAGE_SIZE
vdso32_start:
.incbin "arch/arm64/kernel/vdso32/vdso.so"
.balign PAGE_SIZE
vdso32_end:
.previous
arch/arm64/kernel/vdso32/vdso.lds.S 0 → 100644
View file @ 927ba67a
/* SPDX-License-Identifier: GPL-2.0 */
/*
* Adapted from arm64 version.
*
* GNU linker script for the VDSO library.
* Heavily based on the vDSO linker scripts for other archs.
*
* Copyright (C) 2012-2018 ARM Limited
*/
#include <linux/const.h>
#include <asm/page.h>
#include <asm/vdso.h>
OUTPUT_FORMAT("elf32-littlearm", "elf32-bigarm", "elf32-littlearm")
OUTPUT_ARCH(arm)
SECTIONS
{
PROVIDE_HIDDEN(_vdso_data = . - PAGE_SIZE);
. = VDSO_LBASE + SIZEOF_HEADERS;
.hash : { *(.hash) } :text
.gnu.hash : { *(.gnu.hash) }
.dynsym : { *(.dynsym) }
.dynstr : { *(.dynstr) }
.gnu.version : { *(.gnu.version) }
.gnu.version_d : { *(.gnu.version_d) }
.gnu.version_r : { *(.gnu.version_r) }
.note : { *(.note.*) } :text :note
.dynamic : { *(.dynamic) } :text :dynamic
.rodata : { *(.rodata*) } :text
.text : { *(.text*) } :text =0xe7f001f2
.got : { *(.got) }
.rel.plt : { *(.rel.plt) }
/DISCARD/ : {
*(.note.GNU-stack)
*(.data .data.* .gnu.linkonce.d.* .sdata*)
*(.bss .sbss .dynbss .dynsbss)
}
}
/*
* We must supply the ELF program headers explicitly to get just one
* PT_LOAD segment, and set the flags explicitly to make segments read-only.
*/
PHDRS
{
text PT_LOAD FLAGS(5) FILEHDR PHDRS; /* PF_R|PF_X */
dynamic PT_DYNAMIC FLAGS(4); /* PF_R */
note PT_NOTE FLAGS(4); /* PF_R */
}
VERSION
{
LINUX_2.6 {
global:
__vdso_clock_gettime;
__vdso_gettimeofday;
__vdso_clock_getres;
__kernel_sigreturn_arm;
__kernel_sigreturn_thumb;
__kernel_rt_sigreturn_arm;
__kernel_rt_sigreturn_thumb;
__vdso_clock_gettime64;
local: *;
};
}
/*
* Make the sigreturn code visible to the kernel.
*/
VDSO_compat_sigreturn_arm = __kernel_sigreturn_arm;
VDSO_compat_sigreturn_thumb = __kernel_sigreturn_thumb;
VDSO_compat_rt_sigreturn_arm = __kernel_rt_sigreturn_arm;
VDSO_compat_rt_sigreturn_thumb = __kernel_rt_sigreturn_thumb;
arch/arm64/kernel/vdso32/vgettimeofday.c 0 → 100644
View file @ 927ba67a
// SPDX-License-Identifier: GPL-2.0
/*
* ARM64 compat userspace implementations of gettimeofday() and similar.
*
* Copyright (C) 2018 ARM Limited
*
*/
#include <linux/time.h>
#include <linux/types.h>
int __vdso_clock_gettime(clockid_t clock,
struct old_timespec32 *ts)
{
/* The checks below are required for ABI consistency with arm */
if ((u32)ts >= TASK_SIZE_32)
return -EFAULT;
return __cvdso_clock_gettime32(clock, ts);
}
int __vdso_clock_gettime64(clockid_t clock,
struct __kernel_timespec *ts)
{
/* The checks below are required for ABI consistency with arm */
if ((u32)ts >= TASK_SIZE_32)
return -EFAULT;
return __cvdso_clock_gettime(clock, ts);
}
int __vdso_gettimeofday(struct __kernel_old_timeval *tv,
struct timezone *tz)
{
return __cvdso_gettimeofday(tv, tz);
}
int __vdso_clock_getres(clockid_t clock_id,
struct old_timespec32 *res)
{
/* The checks below are required for ABI consistency with arm */
if ((u32)res >= TASK_SIZE_32)
return -EFAULT;
return __cvdso_clock_getres_time32(clock_id, res);
}
/* Avoid unresolved references emitted by GCC */
void __aeabi_unwind_cpp_pr0(void)
{
}
void __aeabi_unwind_cpp_pr1(void)
{
}
void __aeabi_unwind_cpp_pr2(void)
{
}
arch/x86/Kconfig
View file @ 927ba67a
......@@ -17,6 +17,7 @@ config X86_32
select HAVE_DEBUG_STACKOVERFLOW
select MODULES_USE_ELF_REL
select OLD_SIGACTION
select GENERIC_VDSO_32
config X86_64
def_bool y
......@@ -121,6 +122,7 @@ config X86
select GENERIC_STRNCPY_FROM_USER
select GENERIC_STRNLEN_USER
select GENERIC_TIME_VSYSCALL
select GENERIC_GETTIMEOFDAY
select HARDLOCKUP_CHECK_TIMESTAMP if X86_64
select HAVE_ACPI_APEI if ACPI
select HAVE_ACPI_APEI_NMI if ACPI
......@@ -202,6 +204,7 @@ config X86
select HAVE_SYSCALL_TRACEPOINTS
select HAVE_UNSTABLE_SCHED_CLOCK
select HAVE_USER_RETURN_NOTIFIER
select HAVE_GENERIC_VDSO
select HOTPLUG_SMT if SMP
select IRQ_FORCED_THREADING
select NEED_SG_DMA_LENGTH
......
arch/x86/entry/vdso/Makefile
View file @ 927ba67a
......@@ -3,6 +3,12 @@
# Building vDSO images for x86.
#
# Absolute relocation type $(ARCH_REL_TYPE_ABS) needs to be defined before
# the inclusion of generic Makefile.
ARCH_REL_TYPE_ABS := R_X86_64_JUMP_SLOT|R_X86_64_GLOB_DAT|R_X86_64_RELATIVE|
ARCH_REL_TYPE_ABS += R_386_GLOB_DAT|R_386_JMP_SLOT|R_386_RELATIVE
include $(srctree)/lib/vdso/Makefile
KBUILD_CFLAGS += $(DISABLE_LTO)
KASAN_SANITIZE := n
UBSAN_SANITIZE := n
......@@ -51,6 +57,7 @@ VDSO_LDFLAGS_vdso.lds = -m elf_x86_64 -soname linux-vdso.so.1 --no-undefined \
$(obj)/vdso64.so.dbg: $(obj)/vdso.lds $(vobjs) FORCE
$(call if_changed,vdso)
$(call if_changed,vdso_check)
HOST_EXTRACFLAGS += -I$(srctree)/tools/include -I$(srctree)/include/uapi -I$(srctree)/arch/$(SUBARCH)/include/uapi
hostprogs-y += vdso2c
......@@ -121,6 +128,7 @@ $(obj)/%.so: $(obj)/%.so.dbg FORCE
$(obj)/vdsox32.so.dbg: $(obj)/vdsox32.lds $(vobjx32s) FORCE
$(call if_changed,vdso)
$(call if_changed,vdso_check)
CPPFLAGS_vdso32.lds = $(CPPFLAGS_vdso.lds)
VDSO_LDFLAGS_vdso32.lds = -m elf_i386 -soname linux-gate.so.1
......@@ -160,6 +168,7 @@ $(obj)/vdso32.so.dbg: FORCE \
$(obj)/vdso32/system_call.o \
$(obj)/vdso32/sigreturn.o
$(call if_changed,vdso)
$(call if_changed,vdso_check)
#
# The DSO images are built using a special linker script.
......
arch/x86/entry/vdso/vclock_gettime.c
View file @ 927ba67a
// SPDX-License-Identifier: GPL-2.0-only
/*
* Copyright 2006 Andi Kleen, SUSE Labs.
*
* Fast user context implementation of clock_gettime, gettimeofday, and time.
*
* Copyright 2006 Andi Kleen, SUSE Labs.
* Copyright 2019 ARM Limited
*
* 32 Bit compat layer by Stefani Seibold <stefani@seibold.net>
* sponsored by Rohde & Schwarz GmbH & Co. KG Munich/Germany
*
* The code should have no internal unresolved relocations.
* Check with readelf after changing.
*/
#include <uapi/linux/time.h>
#include <asm/vgtod.h>
#include <asm/vvar.h>
#include <asm/unistd.h>
#include <asm/msr.h>
#include <asm/pvclock.h>
#include <asm/mshyperv.h>
#include <linux/math64.h>
#include <linux/time.h>
#include <linux/kernel.h>
#include <linux/types.h>
#define gtod (&VVAR(vsyscall_gtod_data))
#include "../../../../lib/vdso/gettimeofday.c"
extern int __vdso_clock_gettime(clockid_t clock, struct timespec *ts);
extern int __vdso_gettimeofday(struct timeval *tv, struct timezone *tz);
extern int __vdso_gettimeofday(struct __kernel_old_timeval *tv, struct timezone *tz);
extern time_t __vdso_time(time_t *t);
#ifdef CONFIG_PARAVIRT_CLOCK
extern u8 pvclock_page[PAGE_SIZE]
__attribute__((visibility("hidden")));
#endif
#ifdef CONFIG_HYPERV_TSCPAGE
extern u8 hvclock_page[PAGE_SIZE]
__attribute__((visibility("hidden")));
#endif
#ifndef BUILD_VDSO32
notrace static long vdso_fallback_gettime(long clock, struct timespec *ts)
int __vdso_gettimeofday(struct __kernel_old_timeval *tv, struct timezone *tz)
{
long ret;
asm ("syscall" : "=a" (ret), "=m" (*ts) :
"0" (__NR_clock_gettime), "D" (clock), "S" (ts) :
"rcx", "r11");
return ret;
return __cvdso_gettimeofday(tv, tz);
}
#else
int gettimeofday(struct __kernel_old_timeval *, struct timezone *)
__attribute__((weak, alias("__vdso_gettimeofday")));
notrace static long vdso_fallback_gettime(long clock, struct timespec *ts)
time_t __vdso_time(time_t *t)
{
long ret;
asm (
"mov %%ebx, %%edx \n"
"mov %[clock], %%ebx \n"
"call __kernel_vsyscall \n"
"mov %%edx, %%ebx \n"
: "=a" (ret), "=m" (*ts)
: "0" (__NR_clock_gettime), [clock] "g" (clock), "c" (ts)
: "edx");
return ret;
return __cvdso_time(t);
}
#endif
time_t time(time_t *t) __attribute__((weak, alias("__vdso_time")));
#ifdef CONFIG_PARAVIRT_CLOCK
static notrace const struct pvclock_vsyscall_time_info *get_pvti0(void)
{
return (const struct pvclock_vsyscall_time_info *)&pvclock_page;
}
static notrace u64 vread_pvclock(void)
{
const struct pvclock_vcpu_time_info *pvti = &get_pvti0()->pvti;
u32 version;
u64 ret;
/*
* Note: The kernel and hypervisor must guarantee that cpu ID
* number maps 1:1 to per-CPU pvclock time info.
*
* Because the hypervisor is entirely unaware of guest userspace
* preemption, it cannot guarantee that per-CPU pvclock time
* info is updated if the underlying CPU changes or that that
* version is increased whenever underlying CPU changes.
*
* On KVM, we are guaranteed that pvti updates for any vCPU are
* atomic as seen by *all* vCPUs. This is an even stronger
* guarantee than we get with a normal seqlock.
*
* On Xen, we don't appear to have that guarantee, but Xen still
* supplies a valid seqlock using the version field.
*
* We only do pvclock vdso timing at all if
* PVCLOCK_TSC_STABLE_BIT is set, and we interpret that bit to
* mean that all vCPUs have matching pvti and that the TSC is
* synced, so we can just look at vCPU 0's pvti.
*/
do {
version = pvclock_read_begin(pvti);
if (unlikely(!(pvti->flags & PVCLOCK_TSC_STABLE_BIT)))
return U64_MAX;
ret = __pvclock_read_cycles(pvti, rdtsc_ordered());
} while (pvclock_read_retry(pvti, version));
return ret;
}
#endif
#ifdef CONFIG_HYPERV_TSCPAGE
static notrace u64 vread_hvclock(void)
{
const struct ms_hyperv_tsc_page *tsc_pg =
(const struct ms_hyperv_tsc_page *)&hvclock_page;
#if defined(CONFIG_X86_64) && !defined(BUILD_VDSO32_64)
/* both 64-bit and x32 use these */
extern int __vdso_clock_gettime(clockid_t clock, struct __kernel_timespec *ts);
extern int __vdso_clock_getres(clockid_t clock, struct __kernel_timespec *res);
return hv_read_tsc_page(tsc_pg);
}
#endif
notrace static inline u64 vgetcyc(int mode)
int __vdso_clock_gettime(clockid_t clock, struct __kernel_timespec *ts)
{
if (mode == VCLOCK_TSC)
return (u64)rdtsc_ordered();
/*
* For any memory-mapped vclock type, we need to make sure that gcc
* doesn't cleverly hoist a load before the mode check. Otherwise we
* might end up touching the memory-mapped page even if the vclock in
* question isn't enabled, which will segfault. Hence the barriers.
*/
#ifdef CONFIG_PARAVIRT_CLOCK
if (mode == VCLOCK_PVCLOCK) {
barrier();
return vread_pvclock();
}
#endif
#ifdef CONFIG_HYPERV_TSCPAGE
if (mode == VCLOCK_HVCLOCK) {
barrier();
return vread_hvclock();
}
#endif
return U64_MAX;
return __cvdso_clock_gettime(clock, ts);
}
notrace static int do_hres(clockid_t clk, struct timespec *ts)
{
struct vgtod_ts *base = &gtod->basetime[clk];
u64 cycles, last, sec, ns;
unsigned int seq;
do {
seq = gtod_read_begin(gtod);
cycles = vgetcyc(gtod->vclock_mode);
ns = base->nsec;
last = gtod->cycle_last;
if (unlikely((s64)cycles < 0))
return vdso_fallback_gettime(clk, ts);
if (cycles > last)
ns += (cycles - last) * gtod->mult;
ns >>= gtod->shift;
sec = base->sec;
} while (unlikely(gtod_read_retry(gtod, seq)));
/*
* Do this outside the loop: a race inside the loop could result
* in __iter_div_u64_rem() being extremely slow.
*/
ts->tv_sec = sec + __iter_div_u64_rem(ns, NSEC_PER_SEC, &ns);
ts->tv_nsec = ns;
return 0;
}
int clock_gettime(clockid_t, struct __kernel_timespec *)
__attribute__((weak, alias("__vdso_clock_gettime")));
notrace static void do_coarse(clockid_t clk, struct timespec *ts)
int __vdso_clock_getres(clockid_t clock,
struct __kernel_timespec *res)
{
struct vgtod_ts *base = &gtod->basetime[clk];
unsigned int seq;
do {
seq = gtod_read_begin(gtod);
ts->tv_sec = base->sec;
ts->tv_nsec = base->nsec;
} while (unlikely(gtod_read_retry(gtod, seq)));
return __cvdso_clock_getres(clock, res);
}
int clock_getres(clockid_t, struct __kernel_timespec *)
__attribute__((weak, alias("__vdso_clock_getres")));
notrace int __vdso_clock_gettime(clockid_t clock, struct timespec *ts)
#else
/* i386 only */
extern int __vdso_clock_gettime(clockid_t clock, struct old_timespec32 *ts);
extern int __vdso_clock_getres(clockid_t clock, struct old_timespec32 *res);
int __vdso_clock_gettime(clockid_t clock, struct old_timespec32 *ts)
{
unsigned int msk;
/* Sort out negative (CPU/FD) and invalid clocks */
if (unlikely((unsigned int) clock >= MAX_CLOCKS))
return vdso_fallback_gettime(clock, ts);
/*
* Convert the clockid to a bitmask and use it to check which
* clocks are handled in the VDSO directly.
*/
msk = 1U << clock;
if (likely(msk & VGTOD_HRES)) {
return do_hres(clock, ts);
} else if (msk & VGTOD_COARSE) {
do_coarse(clock, ts);
return 0;
}
return vdso_fallback_gettime(clock, ts);
return __cvdso_clock_gettime32(clock, ts);
}
int clock_gettime(clockid_t, struct timespec *)
int clock_gettime(clockid_t, struct old_timespec32 *)
__attribute__((weak, alias("__vdso_clock_gettime")));
notrace int __vdso_gettimeofday(struct timeval *tv, struct timezone *tz)
int __vdso_clock_gettime64(clockid_t clock, struct __kernel_timespec *ts)
{
if (likely(tv != NULL)) {
struct timespec *ts = (struct timespec *) tv;
do_hres(CLOCK_REALTIME, ts);
tv->tv_usec /= 1000;
}
if (unlikely(tz != NULL)) {
tz->tz_minuteswest = gtod->tz_minuteswest;
tz->tz_dsttime = gtod->tz_dsttime;
}
return 0;
return __cvdso_clock_gettime(clock, ts);
}
int gettimeofday(struct timeval *, struct timezone *)
__attribute__((weak, alias("__vdso_gettimeofday")));
/*
* This will break when the xtime seconds get inaccurate, but that is
* unlikely
*/
notrace time_t __vdso_time(time_t *t)
{
/* This is atomic on x86 so we don't need any locks. */
time_t result = READ_ONCE(gtod->basetime[CLOCK_REALTIME].sec);
int clock_gettime64(clockid_t, struct __kernel_timespec *)
__attribute__((weak, alias("__vdso_clock_gettime64")));
if (t)
*t = result;
return result;
int __vdso_clock_getres(clockid_t clock, struct old_timespec32 *res)
{
return __cvdso_clock_getres_time32(clock, res);
}
time_t time(time_t *t)
__attribute__((weak, alias("__vdso_time")));
int clock_getres(clockid_t, struct old_timespec32 *)
__attribute__((weak, alias("__vdso_clock_getres")));
#endif
arch/x86/entry/vdso/vdso.lds.S
View file @ 927ba67a
......@@ -25,6 +25,8 @@ VERSION {
__vdso_getcpu;
time;
__vdso_time;
clock_getres;
__vdso_clock_getres;
local: *;
};
}
arch/x86/entry/vdso/vdso32/vdso32.lds.S
View file @ 927ba67a
......@@ -26,6 +26,8 @@ VERSION
__vdso_clock_gettime;
__vdso_gettimeofday;
__vdso_time;
__vdso_clock_getres;
__vdso_clock_gettime64;
};
LINUX_2.5 {
......
arch/x86/entry/vdso/vdsox32.lds.S
View file @ 927ba67a
......@@ -21,6 +21,7 @@ VERSION {
__vdso_gettimeofday;
__vdso_getcpu;
__vdso_time;
__vdso_clock_getres;
local: *;
};
}
arch/x86/entry/vdso/vma.c
View file @ 927ba67a
......@@ -22,7 +22,7 @@
#include <asm/page.h>
#include <asm/desc.h>
#include <asm/cpufeature.h>
#include <asm/mshyperv.h>
#include <clocksource/hyperv_timer.h>
#if defined(CONFIG_X86_64)
unsigned int __read_mostly vdso64_enabled = 1;
......
arch/x86/entry/vsyscall/Makefile
View file @ 927ba67a
......@@ -2,7 +2,5 @@
#
# Makefile for the x86 low level vsyscall code
#
obj-y := vsyscall_gtod.o
obj-$(CONFIG_X86_VSYSCALL_EMULATION) += vsyscall_64.o vsyscall_emu_64.o
arch/x86/entry/vsyscall/vsyscall_gtod.c deleted 100644 → 0
View file @ 2a1ccd31
// SPDX-License-Identifier: GPL-2.0
/*
* Copyright (C) 2001 Andrea Arcangeli <andrea@suse.de> SuSE
* Copyright 2003 Andi Kleen, SuSE Labs.
*
* Modified for x86 32 bit architecture by
* Stefani Seibold <stefani@seibold.net>
* sponsored by Rohde & Schwarz GmbH & Co. KG Munich/Germany
*
* Thanks to hpa@transmeta.com for some useful hint.
* Special thanks to Ingo Molnar for his early experience with
* a different vsyscall implementation for Linux/IA32 and for the name.
*
*/
#include <linux/timekeeper_internal.h>
#include <asm/vgtod.h>
#include <asm/vvar.h>
int vclocks_used __read_mostly;
DEFINE_VVAR(struct vsyscall_gtod_data, vsyscall_gtod_data);
void update_vsyscall_tz(void)
{
vsyscall_gtod_data.tz_minuteswest = sys_tz.tz_minuteswest;
vsyscall_gtod_data.tz_dsttime = sys_tz.tz_dsttime;
}
void update_vsyscall(struct timekeeper *tk)
{
int vclock_mode = tk->tkr_mono.clock->archdata.vclock_mode;
struct vsyscall_gtod_data *vdata = &vsyscall_gtod_data;
struct vgtod_ts *base;
u64 nsec;
/* Mark the new vclock used. */
BUILD_BUG_ON(VCLOCK_MAX >= 32);
WRITE_ONCE(vclocks_used, READ_ONCE(vclocks_used) | (1 << vclock_mode));
gtod_write_begin(vdata);
/* copy vsyscall data */
vdata->vclock_mode = vclock_mode;
vdata->cycle_last = tk->tkr_mono.cycle_last;
vdata->mask = tk->tkr_mono.mask;
vdata->mult = tk->tkr_mono.mult;
vdata->shift = tk->tkr_mono.shift;
base = &vdata->basetime[CLOCK_REALTIME];
base->sec = tk->xtime_sec;
base->nsec = tk->tkr_mono.xtime_nsec;
base = &vdata->basetime[CLOCK_TAI];
base->sec = tk->xtime_sec + (s64)tk->tai_offset;
base->nsec = tk->tkr_mono.xtime_nsec;
base = &vdata->basetime[CLOCK_MONOTONIC];
base->sec = tk->xtime_sec + tk->wall_to_monotonic.tv_sec;
nsec = tk->tkr_mono.xtime_nsec;
nsec += ((u64)tk->wall_to_monotonic.tv_nsec << tk->tkr_mono.shift);
while (nsec >= (((u64)NSEC_PER_SEC) << tk->tkr_mono.shift)) {
nsec -= ((u64)NSEC_PER_SEC) << tk->tkr_mono.shift;
base->sec++;
}
base->nsec = nsec;
base = &vdata->basetime[CLOCK_REALTIME_COARSE];
base->sec = tk->xtime_sec;
base->nsec = tk->tkr_mono.xtime_nsec >> tk->tkr_mono.shift;
base = &vdata->basetime[CLOCK_MONOTONIC_COARSE];
base->sec = tk->xtime_sec + tk->wall_to_monotonic.tv_sec;
nsec = tk->tkr_mono.xtime_nsec >> tk->tkr_mono.shift;
nsec += tk->wall_to_monotonic.tv_nsec;
while (nsec >= NSEC_PER_SEC) {
nsec -= NSEC_PER_SEC;
base->sec++;
}
base->nsec = nsec;
gtod_write_end(vdata);
}
arch/x86/hyperv/hv_init.c
View file @ 927ba67a
......@@ -17,64 +17,13 @@
#include <linux/version.h>
#include <linux/vmalloc.h>
#include <linux/mm.h>
#include <linux/clockchips.h>
#include <linux/hyperv.h>
#include <linux/slab.h>
#include <linux/cpuhotplug.h>
#ifdef CONFIG_HYPERV_TSCPAGE
static struct ms_hyperv_tsc_page *tsc_pg;
struct ms_hyperv_tsc_page *hv_get_tsc_page(void)
{
return tsc_pg;
}
EXPORT_SYMBOL_GPL(hv_get_tsc_page);
static u64 read_hv_clock_tsc(struct clocksource *arg)
{
u64 current_tick = hv_read_tsc_page(tsc_pg);
if (current_tick == U64_MAX)
rdmsrl(HV_X64_MSR_TIME_REF_COUNT, current_tick);
return current_tick;
}
static struct clocksource hyperv_cs_tsc = {
.name = "hyperv_clocksource_tsc_page",
.rating = 400,
.read = read_hv_clock_tsc,
.mask = CLOCKSOURCE_MASK(64),
.flags = CLOCK_SOURCE_IS_CONTINUOUS,
};
#endif
static u64 read_hv_clock_msr(struct clocksource *arg)
{
u64 current_tick;
/*
* Read the partition counter to get the current tick count. This count
* is set to 0 when the partition is created and is incremented in
* 100 nanosecond units.
*/
rdmsrl(HV_X64_MSR_TIME_REF_COUNT, current_tick);
return current_tick;
}
static struct clocksource hyperv_cs_msr = {
.name = "hyperv_clocksource_msr",
.rating = 400,
.read = read_hv_clock_msr,
.mask = CLOCKSOURCE_MASK(64),
.flags = CLOCK_SOURCE_IS_CONTINUOUS,
};
#include <clocksource/hyperv_timer.h>
void *hv_hypercall_pg;
EXPORT_SYMBOL_GPL(hv_hypercall_pg);
struct clocksource *hyperv_cs;
EXPORT_SYMBOL_GPL(hyperv_cs);
u32 *hv_vp_index;
EXPORT_SYMBOL_GPL(hv_vp_index);
......@@ -343,42 +292,8 @@ void __init hyperv_init(void)
x86_init.pci.arch_init = hv_pci_init;
/*
* Register Hyper-V specific clocksource.
*/
#ifdef CONFIG_HYPERV_TSCPAGE
if (ms_hyperv.features & HV_MSR_REFERENCE_TSC_AVAILABLE) {
union hv_x64_msr_hypercall_contents tsc_msr;
tsc_pg = __vmalloc(PAGE_SIZE, GFP_KERNEL, PAGE_KERNEL);
if (!tsc_pg)
goto register_msr_cs;
hyperv_cs = &hyperv_cs_tsc;
rdmsrl(HV_X64_MSR_REFERENCE_TSC, tsc_msr.as_uint64);
tsc_msr.enable = 1;
tsc_msr.guest_physical_address = vmalloc_to_pfn(tsc_pg);
wrmsrl(HV_X64_MSR_REFERENCE_TSC, tsc_msr.as_uint64);
hyperv_cs_tsc.archdata.vclock_mode = VCLOCK_HVCLOCK;
clocksource_register_hz(&hyperv_cs_tsc, NSEC_PER_SEC/100);
return;
}
register_msr_cs:
#endif
/*
* For 32 bit guests just use the MSR based mechanism for reading
* the partition counter.
*/
hyperv_cs = &hyperv_cs_msr;
if (ms_hyperv.features & HV_MSR_TIME_REF_COUNT_AVAILABLE)
clocksource_register_hz(&hyperv_cs_msr, NSEC_PER_SEC/100);
/* Register Hyper-V specific clocksource */
hv_init_clocksource();
return;
remove_cpuhp_state:
......
arch/x86/include/asm/hyperv-tlfs.h
View file @ 927ba67a
......@@ -401,6 +401,12 @@ enum HV_GENERIC_SET_FORMAT {
#define HV_STATUS_INVALID_CONNECTION_ID 18
#define HV_STATUS_INSUFFICIENT_BUFFERS 19
/*
* The Hyper-V TimeRefCount register and the TSC
* page provide a guest VM clock with 100ns tick rate
*/
#define HV_CLOCK_HZ (NSEC_PER_SEC/100)
typedef struct _HV_REFERENCE_TSC_PAGE {
__u32 tsc_sequence;
__u32 res1;
......
arch/x86/include/asm/mshyperv.h
View file @ 927ba67a
......@@ -105,6 +105,17 @@ static inline void vmbus_signal_eom(struct hv_message *msg, u32 old_msg_type)
#define hv_get_crash_ctl(val) \
rdmsrl(HV_X64_MSR_CRASH_CTL, val)
#define hv_get_time_ref_count(val) \
rdmsrl(HV_X64_MSR_TIME_REF_COUNT, val)
#define hv_get_reference_tsc(val) \
rdmsrl(HV_X64_MSR_REFERENCE_TSC, val)
#define hv_set_reference_tsc(val) \
wrmsrl(HV_X64_MSR_REFERENCE_TSC, val)
#define hv_set_clocksource_vdso(val) \
((val).archdata.vclock_mode = VCLOCK_HVCLOCK)
#define hv_get_raw_timer() rdtsc_ordered()
void hyperv_callback_vector(void);
void hyperv_reenlightenment_vector(void);
#ifdef CONFIG_TRACING
......@@ -133,7 +144,6 @@ static inline void hv_disable_stimer0_percpu_irq(int irq) {}
#if IS_ENABLED(CONFIG_HYPERV)
extern struct clocksource *hyperv_cs;
extern void *hv_hypercall_pg;
extern void __percpu **hyperv_pcpu_input_arg;
......@@ -387,73 +397,4 @@ static inline int hyperv_flush_guest_mapping_range(u64 as,
}
#endif /* CONFIG_HYPERV */
#ifdef CONFIG_HYPERV_TSCPAGE
struct ms_hyperv_tsc_page *hv_get_tsc_page(void);
static inline u64 hv_read_tsc_page_tsc(const struct ms_hyperv_tsc_page *tsc_pg,
u64 *cur_tsc)
{
u64 scale, offset;
u32 sequence;
/*
* The protocol for reading Hyper-V TSC page is specified in Hypervisor
* Top-Level Functional Specification ver. 3.0 and above. To get the
* reference time we must do the following:
* - READ ReferenceTscSequence
* A special '0' value indicates the time source is unreliable and we
* need to use something else. The currently published specification
* versions (up to 4.0b) contain a mistake and wrongly claim '-1'
* instead of '0' as the special value, see commit c35b82ef0294.
* - ReferenceTime =
* ((RDTSC() * ReferenceTscScale) >> 64) + ReferenceTscOffset
* - READ ReferenceTscSequence again. In case its value has changed
* since our first reading we need to discard ReferenceTime and repeat
* the whole sequence as the hypervisor was updating the page in
* between.
*/
do {
sequence = READ_ONCE(tsc_pg->tsc_sequence);
if (!sequence)
return U64_MAX;
/*
* Make sure we read sequence before we read other values from
* TSC page.
*/
smp_rmb();
scale = READ_ONCE(tsc_pg->tsc_scale);
offset = READ_ONCE(tsc_pg->tsc_offset);
*cur_tsc = rdtsc_ordered();
/*
* Make sure we read sequence after we read all other values
* from TSC page.
*/
smp_rmb();
} while (READ_ONCE(tsc_pg->tsc_sequence) != sequence);
return mul_u64_u64_shr(*cur_tsc, scale, 64) + offset;
}
static inline u64 hv_read_tsc_page(const struct ms_hyperv_tsc_page *tsc_pg)
{
u64 cur_tsc;
return hv_read_tsc_page_tsc(tsc_pg, &cur_tsc);
}
#else
static inline struct ms_hyperv_tsc_page *hv_get_tsc_page(void)
{
return NULL;
}
static inline u64 hv_read_tsc_page_tsc(const struct ms_hyperv_tsc_page *tsc_pg,
u64 *cur_tsc)
{
BUG();
return U64_MAX;
}
#endif
#endif
arch/x86/include/asm/pvclock.h
View file @ 927ba67a
......@@ -2,7 +2,7 @@
#ifndef _ASM_X86_PVCLOCK_H
#define _ASM_X86_PVCLOCK_H
#include <linux/clocksource.h>
#include <asm/clocksource.h>
#include <asm/pvclock-abi.h>
/* some helper functions for xen and kvm pv clock sources */
......
arch/x86/include/asm/vdso/gettimeofday.h 0 → 100644
View file @ 927ba67a
/* SPDX-License-Identifier: GPL-2.0 */
/*
* Fast user context implementation of clock_gettime, gettimeofday, and time.
*
* Copyright (C) 2019 ARM Limited.
* Copyright 2006 Andi Kleen, SUSE Labs.
* 32 Bit compat layer by Stefani Seibold <stefani@seibold.net>
* sponsored by Rohde & Schwarz GmbH & Co. KG Munich/Germany
*/
#ifndef __ASM_VDSO_GETTIMEOFDAY_H
#define __ASM_VDSO_GETTIMEOFDAY_H
#ifndef __ASSEMBLY__
#include <uapi/linux/time.h>
#include <asm/vgtod.h>
#include <asm/vvar.h>
#include <asm/unistd.h>
#include <asm/msr.h>
#include <asm/pvclock.h>
#include <clocksource/hyperv_timer.h>
#define __vdso_data (VVAR(_vdso_data))
#define VDSO_HAS_TIME 1
#define VDSO_HAS_CLOCK_GETRES 1
/*
* Declare the memory-mapped vclock data pages. These come from hypervisors.
* If we ever reintroduce something like direct access to an MMIO clock like
* the HPET again, it will go here as well.
*
* A load from any of these pages will segfault if the clock in question is
* disabled, so appropriate compiler barriers and checks need to be used
* to prevent stray loads.
*
* These declarations MUST NOT be const. The compiler will assume that
* an extern const variable has genuinely constant contents, and the
* resulting code won't work, since the whole point is that these pages
* change over time, possibly while we're accessing them.
*/
#ifdef CONFIG_PARAVIRT_CLOCK
/*
* This is the vCPU 0 pvclock page. We only use pvclock from the vDSO
* if the hypervisor tells us that all vCPUs can get valid data from the
* vCPU 0 page.
*/
extern struct pvclock_vsyscall_time_info pvclock_page
__attribute__((visibility("hidden")));
#endif
#ifdef CONFIG_HYPERV_TSCPAGE
extern struct ms_hyperv_tsc_page hvclock_page
__attribute__((visibility("hidden")));
#endif
#ifndef BUILD_VDSO32
static __always_inline
long clock_gettime_fallback(clockid_t _clkid, struct __kernel_timespec *_ts)
{
long ret;
asm ("syscall" : "=a" (ret), "=m" (*_ts) :
"0" (__NR_clock_gettime), "D" (_clkid), "S" (_ts) :
"rcx", "r11");
return ret;
}
static __always_inline
long gettimeofday_fallback(struct __kernel_old_timeval *_tv,
struct timezone *_tz)
{
long ret;
asm("syscall" : "=a" (ret) :
"0" (__NR_gettimeofday), "D" (_tv), "S" (_tz) : "memory");
return ret;
}
static __always_inline
long clock_getres_fallback(clockid_t _clkid, struct __kernel_timespec *_ts)
{
long ret;
asm ("syscall" : "=a" (ret), "=m" (*_ts) :
"0" (__NR_clock_getres), "D" (_clkid), "S" (_ts) :
"rcx", "r11");
return ret;
}
#else
static __always_inline
long clock_gettime_fallback(clockid_t _clkid, struct __kernel_timespec *_ts)
{
long ret;
asm (
"mov %%ebx, %%edx \n"
"mov %[clock], %%ebx \n"
"call __kernel_vsyscall \n"
"mov %%edx, %%ebx \n"
: "=a" (ret), "=m" (*_ts)
: "0" (__NR_clock_gettime64), [clock] "g" (_clkid), "c" (_ts)
: "edx");
return ret;
}
static __always_inline
long gettimeofday_fallback(struct __kernel_old_timeval *_tv,
struct timezone *_tz)
{
long ret;
asm(
"mov %%ebx, %%edx \n"
"mov %2, %%ebx \n"
"call __kernel_vsyscall \n"
"mov %%edx, %%ebx \n"
: "=a" (ret)
: "0" (__NR_gettimeofday), "g" (_tv), "c" (_tz)
: "memory", "edx");
return ret;
}
static __always_inline long
clock_getres_fallback(clockid_t _clkid, struct __kernel_timespec *_ts)
{
long ret;
asm (
"mov %%ebx, %%edx \n"
"mov %[clock], %%ebx \n"
"call __kernel_vsyscall \n"
"mov %%edx, %%ebx \n"
: "=a" (ret), "=m" (*_ts)
: "0" (__NR_clock_getres_time64), [clock] "g" (_clkid), "c" (_ts)
: "edx");
return ret;
}
#endif
#ifdef CONFIG_PARAVIRT_CLOCK
static u64 vread_pvclock(void)
{
const struct pvclock_vcpu_time_info *pvti = &pvclock_page.pvti;
u32 version;
u64 ret;
/*
* Note: The kernel and hypervisor must guarantee that cpu ID
* number maps 1:1 to per-CPU pvclock time info.
*
* Because the hypervisor is entirely unaware of guest userspace
* preemption, it cannot guarantee that per-CPU pvclock time
* info is updated if the underlying CPU changes or that that
* version is increased whenever underlying CPU changes.
*
* On KVM, we are guaranteed that pvti updates for any vCPU are
* atomic as seen by *all* vCPUs. This is an even stronger
* guarantee than we get with a normal seqlock.
*
* On Xen, we don't appear to have that guarantee, but Xen still
* supplies a valid seqlock using the version field.
*
* We only do pvclock vdso timing at all if
* PVCLOCK_TSC_STABLE_BIT is set, and we interpret that bit to
* mean that all vCPUs have matching pvti and that the TSC is
* synced, so we can just look at vCPU 0's pvti.
*/
do {
version = pvclock_read_begin(pvti);
if (unlikely(!(pvti->flags & PVCLOCK_TSC_STABLE_BIT)))
return U64_MAX;
ret = __pvclock_read_cycles(pvti, rdtsc_ordered());
} while (pvclock_read_retry(pvti, version));
return ret;
}
#endif
#ifdef CONFIG_HYPERV_TSCPAGE
static u64 vread_hvclock(void)
{
return hv_read_tsc_page(&hvclock_page);
}
#endif
static inline u64 __arch_get_hw_counter(s32 clock_mode)
{
if (clock_mode == VCLOCK_TSC)
return (u64)rdtsc_ordered();
/*
* For any memory-mapped vclock type, we need to make sure that gcc
* doesn't cleverly hoist a load before the mode check. Otherwise we
* might end up touching the memory-mapped page even if the vclock in
* question isn't enabled, which will segfault. Hence the barriers.
*/
#ifdef CONFIG_PARAVIRT_CLOCK
if (clock_mode == VCLOCK_PVCLOCK) {
barrier();
return vread_pvclock();
}
#endif
#ifdef CONFIG_HYPERV_TSCPAGE
if (clock_mode == VCLOCK_HVCLOCK) {
barrier();
return vread_hvclock();
}
#endif
return U64_MAX;
}
static __always_inline const struct vdso_data *__arch_get_vdso_data(void)
{
return __vdso_data;
}
/*
* x86 specific delta calculation.
*
* The regular implementation assumes that clocksource reads are globally
* monotonic. The TSC can be slightly off across sockets which can cause
* the regular delta calculation (@cycles - @last) to return a huge time
* jump.
*
* Therefore it needs to be verified that @cycles are greater than
* @last. If not then use @last, which is the base time of the current
* conversion period.
*
* This variant also removes the masking of the subtraction because the
* clocksource mask of all VDSO capable clocksources on x86 is U64_MAX
* which would result in a pointless operation. The compiler cannot
* optimize it away as the mask comes from the vdso data and is not compile
* time constant.
*/
static __always_inline
u64 vdso_calc_delta(u64 cycles, u64 last, u64 mask, u32 mult)
{
if (cycles > last)
return (cycles - last) * mult;
return 0;
}
#define vdso_calc_delta vdso_calc_delta
#endif /* !__ASSEMBLY__ */
#endif /* __ASM_VDSO_GETTIMEOFDAY_H */
arch/x86/include/asm/vdso/vsyscall.h 0 → 100644
View file @ 927ba67a
/* SPDX-License-Identifier: GPL-2.0 */
#ifndef __ASM_VDSO_VSYSCALL_H
#define __ASM_VDSO_VSYSCALL_H
#ifndef __ASSEMBLY__
#include <linux/hrtimer.h>
#include <linux/timekeeper_internal.h>
#include <vdso/datapage.h>
#include <asm/vgtod.h>
#include <asm/vvar.h>
int vclocks_used __read_mostly;
DEFINE_VVAR(struct vdso_data, _vdso_data);
/*
* Update the vDSO data page to keep in sync with kernel timekeeping.
*/
static __always_inline
struct vdso_data *__x86_get_k_vdso_data(void)
{
return _vdso_data;
}
#define __arch_get_k_vdso_data __x86_get_k_vdso_data
static __always_inline
int __x86_get_clock_mode(struct timekeeper *tk)
{
int vclock_mode = tk->tkr_mono.clock->archdata.vclock_mode;
/* Mark the new vclock used. */
BUILD_BUG_ON(VCLOCK_MAX >= 32);
WRITE_ONCE(vclocks_used, READ_ONCE(vclocks_used) | (1 << vclock_mode));
return vclock_mode;
}
#define __arch_get_clock_mode __x86_get_clock_mode
/* The asm-generic header needs to be included after the definitions above */
#include <asm-generic/vdso/vsyscall.h>
#endif /* !__ASSEMBLY__ */
#endif /* __ASM_VDSO_VSYSCALL_H */
arch/x86/include/asm/vgtod.h
View file @ 927ba67a
......@@ -3,7 +3,9 @@
#define _ASM_X86_VGTOD_H
#include <linux/compiler.h>
#include <linux/clocksource.h>
#include <asm/clocksource.h>
#include <vdso/datapage.h>
#include <vdso/helpers.h>
#include <uapi/linux/time.h>
......@@ -13,81 +15,10 @@ typedef u64 gtod_long_t;
typedef unsigned long gtod_long_t;
#endif
/*
* There is one of these objects in the vvar page for each
* vDSO-accelerated clockid. For high-resolution clocks, this encodes
* the time corresponding to vsyscall_gtod_data.cycle_last. For coarse
* clocks, this encodes the actual time.
*
* To confuse the reader, for high-resolution clocks, nsec is left-shifted
* by vsyscall_gtod_data.shift.
*/
struct vgtod_ts {
u64 sec;
u64 nsec;
};
#define VGTOD_BASES (CLOCK_TAI + 1)
#define VGTOD_HRES (BIT(CLOCK_REALTIME) | BIT(CLOCK_MONOTONIC) | BIT(CLOCK_TAI))
#define VGTOD_COARSE (BIT(CLOCK_REALTIME_COARSE) | BIT(CLOCK_MONOTONIC_COARSE))
/*
* vsyscall_gtod_data will be accessed by 32 and 64 bit code at the same time
* so be carefull by modifying this structure.
*/
struct vsyscall_gtod_data {
unsigned int seq;
int vclock_mode;
u64 cycle_last;
u64 mask;
u32 mult;
u32 shift;
struct vgtod_ts basetime[VGTOD_BASES];
int tz_minuteswest;
int tz_dsttime;
};
extern struct vsyscall_gtod_data vsyscall_gtod_data;
extern int vclocks_used;
static inline bool vclock_was_used(int vclock)
{
return READ_ONCE(vclocks_used) & (1 << vclock);
}
static inline unsigned int gtod_read_begin(const struct vsyscall_gtod_data *s)
{
unsigned int ret;
repeat:
ret = READ_ONCE(s->seq);
if (unlikely(ret & 1)) {
cpu_relax();
goto repeat;
}
smp_rmb();
return ret;
}
static inline int gtod_read_retry(const struct vsyscall_gtod_data *s,
unsigned int start)
{
smp_rmb();
return unlikely(s->seq != start);
}
static inline void gtod_write_begin(struct vsyscall_gtod_data *s)
{
++s->seq;
smp_wmb();
}
static inline void gtod_write_end(struct vsyscall_gtod_data *s)
{
smp_wmb();
++s->seq;
}
#endif /* _ASM_X86_VGTOD_H */
arch/x86/include/asm/vvar.h
View file @ 927ba67a
......@@ -32,19 +32,20 @@
extern char __vvar_page;
#define DECLARE_VVAR(offset, type, name) \
extern type vvar_ ## name __attribute__((visibility("hidden")));
extern type vvar_ ## name[CS_BASES] \
__attribute__((visibility("hidden")));
#define VVAR(name) (vvar_ ## name)
#define DEFINE_VVAR(type, name) \
type name \
type name[CS_BASES] \
__attribute__((section(".vvar_" #name), aligned(16))) __visible
#endif
/* DECLARE_VVAR(offset, type, name) */
DECLARE_VVAR(128, struct vsyscall_gtod_data, vsyscall_gtod_data)
DECLARE_VVAR(128, struct vdso_data, _vdso_data)
#undef DECLARE_VVAR
......
arch/x86/kernel/cpu/mshyperv.c
View file @ 927ba67a
......@@ -17,6 +17,7 @@
#include <linux/irq.h>
#include <linux/kexec.h>
#include <linux/i8253.h>
#include <linux/random.h>
#include <asm/processor.h>
#include <asm/hypervisor.h>
#include <asm/hyperv-tlfs.h>
......@@ -80,6 +81,7 @@ __visible void __irq_entry hv_stimer0_vector_handler(struct pt_regs *regs)
inc_irq_stat(hyperv_stimer0_count);
if (hv_stimer0_handler)
hv_stimer0_handler();
add_interrupt_randomness(HYPERV_STIMER0_VECTOR, 0);
ack_APIC_irq();
exiting_irq();
......@@ -89,7 +91,7 @@ __visible void __irq_entry hv_stimer0_vector_handler(struct pt_regs *regs)
int hv_setup_stimer0_irq(int *irq, int *vector, void (*handler)(void))
{
*vector = HYPERV_STIMER0_VECTOR;
*irq = 0; /* Unused on x86/x64 */
*irq = -1; /* Unused on x86/x64 */
hv_stimer0_handler = handler;
return 0;
}
......
arch/x86/kernel/pvclock.c
View file @ 927ba67a
......@@ -3,6 +3,7 @@
*/
#include <linux/clocksource.h>
#include <linux/kernel.h>
#include <linux/percpu.h>
#include <linux/notifier.h>
......
arch/x86/kvm/pmu.c
View file @ 927ba67a
......@@ -261,10 +261,10 @@ static int kvm_pmu_rdpmc_vmware(struct kvm_vcpu *vcpu, unsigned idx, u64 *data)
ctr_val = rdtsc();
break;
case VMWARE_BACKDOOR_PMC_REAL_TIME:
ctr_val = ktime_get_boot_ns();
ctr_val = ktime_get_boottime_ns();
break;
case VMWARE_BACKDOOR_PMC_APPARENT_TIME:
ctr_val = ktime_get_boot_ns() +
ctr_val = ktime_get_boottime_ns() +
vcpu->kvm->arch.kvmclock_offset;
break;
default:
......
arch/x86/kvm/x86.c
View file @ 927ba67a
......@@ -67,6 +67,7 @@
#include <asm/mshyperv.h>
#include <asm/hypervisor.h>
#include <asm/intel_pt.h>
#include <clocksource/hyperv_timer.h>
#define CREATE_TRACE_POINTS
#include "trace.h"
......@@ -1728,7 +1729,7 @@ void kvm_write_tsc(struct kvm_vcpu *vcpu, struct msr_data *msr)
raw_spin_lock_irqsave(&kvm->arch.tsc_write_lock, flags);
offset = kvm_compute_tsc_offset(vcpu, data);
ns = ktime_get_boot_ns();
ns = ktime_get_boottime_ns();
elapsed = ns - kvm->arch.last_tsc_nsec;
if (vcpu->arch.virtual_tsc_khz) {
......@@ -2070,7 +2071,7 @@ u64 get_kvmclock_ns(struct kvm *kvm)
spin_lock(&ka->pvclock_gtod_sync_lock);
if (!ka->use_master_clock) {
spin_unlock(&ka->pvclock_gtod_sync_lock);
return ktime_get_boot_ns() + ka->kvmclock_offset;
return ktime_get_boottime_ns() + ka->kvmclock_offset;
}
hv_clock.tsc_timestamp = ka->master_cycle_now;
......@@ -2086,7 +2087,7 @@ u64 get_kvmclock_ns(struct kvm *kvm)
&hv_clock.tsc_to_system_mul);
ret = __pvclock_read_cycles(&hv_clock, rdtsc());
} else
ret = ktime_get_boot_ns() + ka->kvmclock_offset;
ret = ktime_get_boottime_ns() + ka->kvmclock_offset;
put_cpu();
......@@ -2185,7 +2186,7 @@ static int kvm_guest_time_update(struct kvm_vcpu *v)
}
if (!use_master_clock) {
host_tsc = rdtsc();
kernel_ns = ktime_get_boot_ns();
kernel_ns = ktime_get_boottime_ns();
}
tsc_timestamp = kvm_read_l1_tsc(v, host_tsc);
......@@ -9015,7 +9016,7 @@ int kvm_arch_hardware_enable(void)
* before any KVM threads can be running. Unfortunately, we can't
* bring the TSCs fully up to date with real time, as we aren't yet far
* enough into CPU bringup that we know how much real time has actually
* elapsed; our helper function, ktime_get_boot_ns() will be using boot
* elapsed; our helper function, ktime_get_boottime_ns() will be using boot
* variables that haven't been updated yet.
*
* So we simply find the maximum observed TSC above, then record the
......@@ -9243,7 +9244,7 @@ int kvm_arch_init_vm(struct kvm *kvm, unsigned long type)
mutex_init(&kvm->arch.apic_map_lock);
spin_lock_init(&kvm->arch.pvclock_gtod_sync_lock);
kvm->arch.kvmclock_offset = -ktime_get_boot_ns();
kvm->arch.kvmclock_offset = -ktime_get_boottime_ns();
pvclock_update_vm_gtod_copy(kvm);
kvm->arch.guest_can_read_msr_platform_info = true;
......
drivers/clocksource/Kconfig
View file @ 927ba67a
......@@ -43,6 +43,11 @@ config BCM_KONA_TIMER
help
Enables the support for the BCM Kona mobile timer driver.
config DAVINCI_TIMER
bool "Texas Instruments DaVinci timer driver" if COMPILE_TEST
help
Enables the support for the TI DaVinci timer driver.
config DIGICOLOR_TIMER
bool "Digicolor timer driver" if COMPILE_TEST
select CLKSRC_MMIO
......@@ -140,7 +145,7 @@ config TEGRA_TIMER
bool "Tegra timer driver" if COMPILE_TEST
select CLKSRC_MMIO
select TIMER_OF
depends on ARM || ARM64
depends on ARCH_TEGRA || COMPILE_TEST
help
Enables support for the Tegra driver.
......@@ -617,6 +622,13 @@ config CLKSRC_IMX_TPM
Enable this option to use IMX Timer/PWM Module (TPM) timer as
clocksource.
config TIMER_IMX_SYS_CTR
bool "i.MX system counter timer" if COMPILE_TEST
select TIMER_OF
help
Enable this option to use i.MX system counter timer as a
clockevent.
config CLKSRC_ST_LPC
bool "Low power clocksource found in the LPC" if COMPILE_TEST
select TIMER_OF if OF
......
drivers/clocksource/Makefile
View file @ 927ba67a
......@@ -15,6 +15,7 @@ obj-$(CONFIG_SH_TIMER_TMU) += sh_tmu.o
obj-$(CONFIG_EM_TIMER_STI) += em_sti.o
obj-$(CONFIG_CLKBLD_I8253) += i8253.o
obj-$(CONFIG_CLKSRC_MMIO) += mmio.o
obj-$(CONFIG_DAVINCI_TIMER) += timer-davinci.o
obj-$(CONFIG_DIGICOLOR_TIMER) += timer-digicolor.o
obj-$(CONFIG_OMAP_DM_TIMER) += timer-ti-dm.o
obj-$(CONFIG_DW_APB_TIMER) += dw_apb_timer.o
......@@ -36,7 +37,7 @@ obj-$(CONFIG_U300_TIMER) += timer-u300.o
obj-$(CONFIG_SUN4I_TIMER) += timer-sun4i.o
obj-$(CONFIG_SUN5I_HSTIMER) += timer-sun5i.o
obj-$(CONFIG_MESON6_TIMER) += timer-meson6.o
obj-$(CONFIG_TEGRA_TIMER) += timer-tegra20.o
obj-$(CONFIG_TEGRA_TIMER) += timer-tegra.o
obj-$(CONFIG_VT8500_TIMER) += timer-vt8500.o
obj-$(CONFIG_NSPIRE_TIMER) += timer-zevio.o
obj-$(CONFIG_BCM_KONA_TIMER) += bcm_kona_timer.o
......@@ -74,6 +75,7 @@ obj-$(CONFIG_CLKSRC_MIPS_GIC) += mips-gic-timer.o
obj-$(CONFIG_CLKSRC_TANGO_XTAL) += timer-tango-xtal.o
obj-$(CONFIG_CLKSRC_IMX_GPT) += timer-imx-gpt.o
obj-$(CONFIG_CLKSRC_IMX_TPM) += timer-imx-tpm.o
obj-$(CONFIG_TIMER_IMX_SYS_CTR) += timer-imx-sysctr.o
obj-$(CONFIG_ASM9260_TIMER) += asm9260_timer.o
obj-$(CONFIG_H8300_TMR8) += h8300_timer8.o
obj-$(CONFIG_H8300_TMR16) += h8300_timer16.o
......@@ -84,3 +86,4 @@ obj-$(CONFIG_ATCPIT100_TIMER) += timer-atcpit100.o
obj-$(CONFIG_RISCV_TIMER) += timer-riscv.o
obj-$(CONFIG_CSKY_MP_TIMER) += timer-mp-csky.o
obj-$(CONFIG_GX6605S_TIMER) += timer-gx6605s.o
obj-$(CONFIG_HYPERV_TIMER) += hyperv_timer.o
drivers/clocksource/arc_timer.c
View file @ 927ba67a
......@@ -13,6 +13,7 @@
*/
#include <linux/interrupt.h>
#include <linux/bits.h>
#include <linux/clk.h>
#include <linux/clk-provider.h>
#include <linux/clocksource.h>
......@@ -139,7 +140,7 @@ static u64 arc_read_rtc(struct clocksource *cs)
l = read_aux_reg(AUX_RTC_LOW);
h = read_aux_reg(AUX_RTC_HIGH);
status = read_aux_reg(AUX_RTC_CTRL);
} while (!(status & _BITUL(31)));
} while (!(status & BIT(31)));
return (((u64)h) << 32) | l;
}
......
drivers/clocksource/arm_arch_timer.c
View file @ 927ba67a
......@@ -801,14 +801,7 @@ static void arch_timer_evtstrm_enable(int divider)
cntkctl |= (divider << ARCH_TIMER_EVT_TRIGGER_SHIFT)
| ARCH_TIMER_VIRT_EVT_EN;
arch_timer_set_cntkctl(cntkctl);
#ifdef CONFIG_ARM64
cpu_set_named_feature(EVTSTRM);
#else
elf_hwcap |= HWCAP_EVTSTRM;
#endif
#ifdef CONFIG_COMPAT
compat_elf_hwcap |= COMPAT_HWCAP_EVTSTRM;
#endif
arch_timer_set_evtstrm_feature();
cpumask_set_cpu(smp_processor_id(), &evtstrm_available);
}
......@@ -1037,11 +1030,7 @@ static int arch_timer_cpu_pm_notify(struct notifier_block *self,
} else if (action == CPU_PM_ENTER_FAILED || action == CPU_PM_EXIT) {
arch_timer_set_cntkctl(__this_cpu_read(saved_cntkctl));
#ifdef CONFIG_ARM64
if (cpu_have_named_feature(EVTSTRM))
#else
if (elf_hwcap & HWCAP_EVTSTRM)
#endif
if (arch_timer_have_evtstrm_feature())
cpumask_set_cpu(smp_processor_id(), &evtstrm_available);
}
return NOTIFY_OK;
......
drivers/clocksource/exynos_mct.c
View file @ 927ba67a
......@@ -206,7 +206,7 @@ static void exynos4_frc_resume(struct clocksource *cs)
static struct clocksource mct_frc = {
.name = "mct-frc",
.rating = 400,
.rating = 450, /* use value higher than ARM arch timer */
.read = exynos4_frc_read,
.mask = CLOCKSOURCE_MASK(32),
.flags = CLOCK_SOURCE_IS_CONTINUOUS,
......@@ -461,7 +461,7 @@ static int exynos4_mct_starting_cpu(unsigned int cpu)
evt->set_state_oneshot_stopped = set_state_shutdown;
evt->tick_resume = set_state_shutdown;
evt->features = CLOCK_EVT_FEAT_PERIODIC | CLOCK_EVT_FEAT_ONESHOT;
evt->rating = 450;
evt->rating = 500; /* use value higher than ARM arch timer */
exynos4_mct_write(TICK_BASE_CNT, mevt->base + MCT_L_TCNTB_OFFSET);
......
drivers/clocksource/hyperv_timer.c 0 → 100644
View file @ 927ba67a
// SPDX-License-Identifier: GPL-2.0
/*
* Clocksource driver for the synthetic counter and timers
* provided by the Hyper-V hypervisor to guest VMs, as described
* in the Hyper-V Top Level Functional Spec (TLFS). This driver
* is instruction set architecture independent.
*
* Copyright (C) 2019, Microsoft, Inc.
*
* Author: Michael Kelley <mikelley@microsoft.com>
*/
#include <linux/percpu.h>
#include <linux/cpumask.h>
#include <linux/clockchips.h>
#include <linux/clocksource.h>
#include <linux/sched_clock.h>
#include <linux/mm.h>
#include <clocksource/hyperv_timer.h>
#include <asm/hyperv-tlfs.h>
#include <asm/mshyperv.h>
static struct clock_event_device __percpu *hv_clock_event;
/*
* If false, we're using the old mechanism for stimer0 interrupts
* where it sends a VMbus message when it expires. The old
* mechanism is used when running on older versions of Hyper-V
* that don't support Direct Mode. While Hyper-V provides
* four stimer's per CPU, Linux uses only stimer0.
*/
static bool direct_mode_enabled;
static int stimer0_irq;
static int stimer0_vector;
static int stimer0_message_sint;
/*
* ISR for when stimer0 is operating in Direct Mode. Direct Mode
* does not use VMbus or any VMbus messages, so process here and not
* in the VMbus driver code.
*/
void hv_stimer0_isr(void)
{
struct clock_event_device *ce;
ce = this_cpu_ptr(hv_clock_event);
ce->event_handler(ce);
}
EXPORT_SYMBOL_GPL(hv_stimer0_isr);
static int hv_ce_set_next_event(unsigned long delta,
struct clock_event_device *evt)
{
u64 current_tick;
current_tick = hyperv_cs->read(NULL);
current_tick += delta;
hv_init_timer(0, current_tick);
return 0;
}
static int hv_ce_shutdown(struct clock_event_device *evt)
{
hv_init_timer(0, 0);
hv_init_timer_config(0, 0);
if (direct_mode_enabled)
hv_disable_stimer0_percpu_irq(stimer0_irq);
return 0;
}
static int hv_ce_set_oneshot(struct clock_event_device *evt)
{
union hv_stimer_config timer_cfg;
timer_cfg.as_uint64 = 0;
timer_cfg.enable = 1;
timer_cfg.auto_enable = 1;
if (direct_mode_enabled) {
/*
* When it expires, the timer will directly interrupt
* on the specified hardware vector/IRQ.
*/
timer_cfg.direct_mode = 1;
timer_cfg.apic_vector = stimer0_vector;
hv_enable_stimer0_percpu_irq(stimer0_irq);
} else {
/*
* When it expires, the timer will generate a VMbus message,
* to be handled by the normal VMbus interrupt handler.
*/
timer_cfg.direct_mode = 0;
timer_cfg.sintx = stimer0_message_sint;
}
hv_init_timer_config(0, timer_cfg.as_uint64);
return 0;
}
/*
* hv_stimer_init - Per-cpu initialization of the clockevent
*/
void hv_stimer_init(unsigned int cpu)
{
struct clock_event_device *ce;
/*
* Synthetic timers are always available except on old versions of
* Hyper-V on x86. In that case, just return as Linux will use a
* clocksource based on emulated PIT or LAPIC timer hardware.
*/
if (!(ms_hyperv.features & HV_MSR_SYNTIMER_AVAILABLE))
return;
ce = per_cpu_ptr(hv_clock_event, cpu);
ce->name = "Hyper-V clockevent";
ce->features = CLOCK_EVT_FEAT_ONESHOT;
ce->cpumask = cpumask_of(cpu);
ce->rating = 1000;
ce->set_state_shutdown = hv_ce_shutdown;
ce->set_state_oneshot = hv_ce_set_oneshot;
ce->set_next_event = hv_ce_set_next_event;
clockevents_config_and_register(ce,
HV_CLOCK_HZ,
HV_MIN_DELTA_TICKS,
HV_MAX_MAX_DELTA_TICKS);
}
EXPORT_SYMBOL_GPL(hv_stimer_init);
/*
* hv_stimer_cleanup - Per-cpu cleanup of the clockevent
*/
void hv_stimer_cleanup(unsigned int cpu)
{
struct clock_event_device *ce;
/* Turn off clockevent device */
if (ms_hyperv.features & HV_MSR_SYNTIMER_AVAILABLE) {
ce = per_cpu_ptr(hv_clock_event, cpu);
hv_ce_shutdown(ce);
}
}
EXPORT_SYMBOL_GPL(hv_stimer_cleanup);
/* hv_stimer_alloc - Global initialization of the clockevent and stimer0 */
int hv_stimer_alloc(int sint)
{
int ret;
hv_clock_event = alloc_percpu(struct clock_event_device);
if (!hv_clock_event)
return -ENOMEM;
direct_mode_enabled = ms_hyperv.misc_features &
HV_STIMER_DIRECT_MODE_AVAILABLE;
if (direct_mode_enabled) {
ret = hv_setup_stimer0_irq(&stimer0_irq, &stimer0_vector,
hv_stimer0_isr);
if (ret) {
free_percpu(hv_clock_event);
hv_clock_event = NULL;
return ret;
}
}
stimer0_message_sint = sint;
return 0;
}
EXPORT_SYMBOL_GPL(hv_stimer_alloc);
/* hv_stimer_free - Free global resources allocated by hv_stimer_alloc() */
void hv_stimer_free(void)
{
if (direct_mode_enabled && (stimer0_irq != 0)) {
hv_remove_stimer0_irq(stimer0_irq);
stimer0_irq = 0;
}
free_percpu(hv_clock_event);
hv_clock_event = NULL;
}
EXPORT_SYMBOL_GPL(hv_stimer_free);
/*
* Do a global cleanup of clockevents for the cases of kexec and
* vmbus exit
*/
void hv_stimer_global_cleanup(void)
{
int cpu;
struct clock_event_device *ce;
if (ms_hyperv.features & HV_MSR_SYNTIMER_AVAILABLE) {
for_each_present_cpu(cpu) {
ce = per_cpu_ptr(hv_clock_event, cpu);
clockevents_unbind_device(ce, cpu);
}
}
hv_stimer_free();
}
EXPORT_SYMBOL_GPL(hv_stimer_global_cleanup);
/*
* Code and definitions for the Hyper-V clocksources. Two
* clocksources are defined: one that reads the Hyper-V defined MSR, and
* the other that uses the TSC reference page feature as defined in the
* TLFS. The MSR version is for compatibility with old versions of
* Hyper-V and 32-bit x86. The TSC reference page version is preferred.
*/
struct clocksource *hyperv_cs;
EXPORT_SYMBOL_GPL(hyperv_cs);
#ifdef CONFIG_HYPERV_TSCPAGE
static struct ms_hyperv_tsc_page *tsc_pg;
struct ms_hyperv_tsc_page *hv_get_tsc_page(void)
{
return tsc_pg;
}
EXPORT_SYMBOL_GPL(hv_get_tsc_page);
static u64 notrace read_hv_sched_clock_tsc(void)
{
u64 current_tick = hv_read_tsc_page(tsc_pg);
if (current_tick == U64_MAX)
hv_get_time_ref_count(current_tick);
return current_tick;
}
static u64 read_hv_clock_tsc(struct clocksource *arg)
{
return read_hv_sched_clock_tsc();
}
static struct clocksource hyperv_cs_tsc = {
.name = "hyperv_clocksource_tsc_page",
.rating = 400,
.read = read_hv_clock_tsc,
.mask = CLOCKSOURCE_MASK(64),
.flags = CLOCK_SOURCE_IS_CONTINUOUS,
};
#endif
static u64 notrace read_hv_sched_clock_msr(void)
{
u64 current_tick;
/*
* Read the partition counter to get the current tick count. This count
* is set to 0 when the partition is created and is incremented in
* 100 nanosecond units.
*/
hv_get_time_ref_count(current_tick);
return current_tick;
}
static u64 read_hv_clock_msr(struct clocksource *arg)
{
return read_hv_sched_clock_msr();
}
static struct clocksource hyperv_cs_msr = {
.name = "hyperv_clocksource_msr",
.rating = 400,
.read = read_hv_clock_msr,
.mask = CLOCKSOURCE_MASK(64),
.flags = CLOCK_SOURCE_IS_CONTINUOUS,
};
#ifdef CONFIG_HYPERV_TSCPAGE
static bool __init hv_init_tsc_clocksource(void)
{
u64 tsc_msr;
phys_addr_t phys_addr;
if (!(ms_hyperv.features & HV_MSR_REFERENCE_TSC_AVAILABLE))
return false;
tsc_pg = vmalloc(PAGE_SIZE);
if (!tsc_pg)
return false;
hyperv_cs = &hyperv_cs_tsc;
phys_addr = page_to_phys(vmalloc_to_page(tsc_pg));
/*
* The Hyper-V TLFS specifies to preserve the value of reserved
* bits in registers. So read the existing value, preserve the
* low order 12 bits, and add in the guest physical address
* (which already has at least the low 12 bits set to zero since
* it is page aligned). Also set the "enable" bit, which is bit 0.
*/
hv_get_reference_tsc(tsc_msr);
tsc_msr &= GENMASK_ULL(11, 0);
tsc_msr = tsc_msr | 0x1 | (u64)phys_addr;
hv_set_reference_tsc(tsc_msr);
hv_set_clocksource_vdso(hyperv_cs_tsc);
clocksource_register_hz(&hyperv_cs_tsc, NSEC_PER_SEC/100);
/* sched_clock_register is needed on ARM64 but is a no-op on x86 */
sched_clock_register(read_hv_sched_clock_tsc, 64, HV_CLOCK_HZ);
return true;
}
#else
static bool __init hv_init_tsc_clocksource(void)
{
return false;
}
#endif
void __init hv_init_clocksource(void)
{
/*
* Try to set up the TSC page clocksource. If it succeeds, we're
* done. Otherwise, set up the MSR clocksoruce. At least one of
* these will always be available except on very old versions of
* Hyper-V on x86. In that case we won't have a Hyper-V
* clocksource, but Linux will still run with a clocksource based
* on the emulated PIT or LAPIC timer.
*/
if (hv_init_tsc_clocksource())
return;
if (!(ms_hyperv.features & HV_MSR_TIME_REF_COUNT_AVAILABLE))
return;
hyperv_cs = &hyperv_cs_msr;
clocksource_register_hz(&hyperv_cs_msr, NSEC_PER_SEC/100);
/* sched_clock_register is needed on ARM64 but is a no-op on x86 */
sched_clock_register(read_hv_sched_clock_msr, 64, HV_CLOCK_HZ);
}
EXPORT_SYMBOL_GPL(hv_init_clocksource);
drivers/clocksource/timer-davinci.c 0 → 100644
View file @ 927ba67a
This diff is collapsed.
drivers/clocksource/timer-imx-sysctr.c 0 → 100644
View file @ 927ba67a
// SPDX-License-Identifier: GPL-2.0+
//
// Copyright 2017-2019 NXP
#include <linux/interrupt.h>
#include <linux/clockchips.h>
#include <linux/of_address.h>
#include <linux/of_irq.h>
#include "timer-of.h"
#define CMP_OFFSET 0x10000
#define CNTCV_LO 0x8
#define CNTCV_HI 0xc
#define CMPCV_LO (CMP_OFFSET + 0x20)
#define CMPCV_HI (CMP_OFFSET + 0x24)
#define CMPCR (CMP_OFFSET + 0x2c)
#define SYS_CTR_EN 0x1
#define SYS_CTR_IRQ_MASK 0x2
static void __iomem *sys_ctr_base;
static u32 cmpcr;
static void sysctr_timer_enable(bool enable)
{
writel(enable ? cmpcr | SYS_CTR_EN : cmpcr, sys_ctr_base + CMPCR);
}
static void sysctr_irq_acknowledge(void)
{
/*
* clear the enable bit(EN =0) will clear
* the status bit(ISTAT = 0), then the interrupt
* signal will be negated(acknowledged).
*/
sysctr_timer_enable(false);
}
static inline u64 sysctr_read_counter(void)
{
u32 cnt_hi, tmp_hi, cnt_lo;
do {
cnt_hi = readl_relaxed(sys_ctr_base + CNTCV_HI);
cnt_lo = readl_relaxed(sys_ctr_base + CNTCV_LO);
tmp_hi = readl_relaxed(sys_ctr_base + CNTCV_HI);
} while (tmp_hi != cnt_hi);
return ((u64) cnt_hi << 32) | cnt_lo;
}
static int sysctr_set_next_event(unsigned long delta,
struct clock_event_device *evt)
{
u32 cmp_hi, cmp_lo;
u64 next;
sysctr_timer_enable(false);
next = sysctr_read_counter();
next += delta;
cmp_hi = (next >> 32) & 0x00fffff;
cmp_lo = next & 0xffffffff;
writel_relaxed(cmp_hi, sys_ctr_base + CMPCV_HI);
writel_relaxed(cmp_lo, sys_ctr_base + CMPCV_LO);
sysctr_timer_enable(true);
return 0;
}
static int sysctr_set_state_oneshot(struct clock_event_device *evt)
{
return 0;
}
static int sysctr_set_state_shutdown(struct clock_event_device *evt)
{
sysctr_timer_enable(false);
return 0;
}
static irqreturn_t sysctr_timer_interrupt(int irq, void *dev_id)
{
struct clock_event_device *evt = dev_id;
sysctr_irq_acknowledge();
evt->event_handler(evt);
return IRQ_HANDLED;
}
static struct timer_of to_sysctr = {
.flags = TIMER_OF_IRQ | TIMER_OF_CLOCK | TIMER_OF_BASE,
.clkevt = {
.name = "i.MX system counter timer",
.features = CLOCK_EVT_FEAT_ONESHOT |
CLOCK_EVT_FEAT_DYNIRQ,
.set_state_oneshot = sysctr_set_state_oneshot,
.set_next_event = sysctr_set_next_event,
.set_state_shutdown = sysctr_set_state_shutdown,
.rating = 200,
},
.of_irq = {
.handler = sysctr_timer_interrupt,
.flags = IRQF_TIMER | IRQF_IRQPOLL,
},
.of_clk = {
.name = "per",
},
};
static void __init sysctr_clockevent_init(void)
{
to_sysctr.clkevt.cpumask = cpumask_of(0);
clockevents_config_and_register(&to_sysctr.clkevt,
timer_of_rate(&to_sysctr),
0xff, 0x7fffffff);
}
static int __init sysctr_timer_init(struct device_node *np)
{
int ret = 0;
ret = timer_of_init(np, &to_sysctr);
if (ret)
return ret;
sys_ctr_base = timer_of_base(&to_sysctr);
cmpcr = readl(sys_ctr_base + CMPCR);
cmpcr &= ~SYS_CTR_EN;
sysctr_clockevent_init();
return 0;
}
TIMER_OF_DECLARE(sysctr_timer, "nxp,sysctr-timer", sysctr_timer_init);
drivers/clocksource/timer-ixp4xx.c
View file @ 927ba67a
......@@ -75,14 +75,19 @@ to_ixp4xx_timer(struct clock_event_device *evt)
return container_of(evt, struct ixp4xx_timer, clkevt);
}
static u64 notrace ixp4xx_read_sched_clock(void)
static unsigned long ixp4xx_read_timer(void)
{
return __raw_readl(local_ixp4xx_timer->base + IXP4XX_OSTS_OFFSET);
}
static u64 notrace ixp4xx_read_sched_clock(void)
{
return ixp4xx_read_timer();
}
static u64 ixp4xx_clocksource_read(struct clocksource *c)
{
return __raw_readl(local_ixp4xx_timer->base + IXP4XX_OSTS_OFFSET);
return ixp4xx_read_timer();
}
static irqreturn_t ixp4xx_timer_interrupt(int irq, void *dev_id)
......@@ -224,6 +229,13 @@ static __init int ixp4xx_timer_register(void __iomem *base,
sched_clock_register(ixp4xx_read_sched_clock, 32, timer_freq);
#ifdef CONFIG_ARM
/* Also use this timer for delays */
tmr->delay_timer.read_current_timer = ixp4xx_read_timer;
tmr->delay_timer.freq = timer_freq;
register_current_timer_delay(&tmr->delay_timer);
#endif
return 0;
}
......
drivers/clocksource/timer-meson6.c
View file @ 927ba67a
// SPDX-License-Identifier: GPL-2.0
/*
* Amlogic Meson6 SoCs timer handling.
*
* Copyright (C) 2014 Carlo Caione <carlo@caione.org>
*
* Based on code from Amlogic, Inc
*
* This file is licensed under the terms of the GNU General Public
* License version 2. This program is licensed "as is" without any
* warranty of any kind, whether express or implied.
*/
#include <linux/bitfield.h>
......
drivers/gpu/drm/amd/amdkfd/kfd_chardev.c
View file @ 927ba67a
......@@ -837,7 +837,7 @@ static int kfd_ioctl_get_clock_counters(struct file *filep,
/* No access to rdtsc. Using raw monotonic time */
args->cpu_clock_counter = ktime_get_raw_ns();
args->system_clock_counter = ktime_get_boot_ns();
args->system_clock_counter = ktime_get_boottime_ns();
/* Since the counter is in nano-seconds we use 1GHz frequency */
args->system_clock_freq = 1000000000;
......
drivers/hv/Kconfig
View file @ 927ba67a
......@@ -10,6 +10,9 @@ config HYPERV
Select this option to run Linux as a Hyper-V client operating
system.
config HYPERV_TIMER
def_bool HYPERV
config HYPERV_TSCPAGE
def_bool HYPERV && X86_64
......
drivers/hv/hv.c
View file @ 927ba67a
This diff is collapsed.
drivers/hv/hv_util.c
View file @ 927ba67a
......@@ -17,6 +17,7 @@
#include <linux/hyperv.h>
#include <linux/clockchips.h>
#include <linux/ptp_clock_kernel.h>
#include <clocksource/hyperv_timer.h>
#include <asm/mshyperv.h>
#include "hyperv_vmbus.h"
......
drivers/hv/hyperv_vmbus.h
View file @ 927ba67a
......@@ -138,7 +138,6 @@ struct hv_per_cpu_context {
* per-cpu list of the channels based on their CPU affinity.
*/
struct list_head chan_list;
struct clock_event_device *clk_evt;
};
struct hv_context {
......@@ -176,8 +175,6 @@ extern int hv_synic_init(unsigned int cpu);
extern int hv_synic_cleanup(unsigned int cpu);
extern void hv_synic_clockevents_cleanup(void);
/* Interface */
void hv_ringbuffer_pre_init(struct vmbus_channel *channel);
......
drivers/hv/vmbus_drv.c
View file @ 927ba67a
......@@ -30,6 +30,7 @@
#include <linux/kdebug.h>
#include <linux/efi.h>
#include <linux/random.h>
#include <clocksource/hyperv_timer.h>
#include "hyperv_vmbus.h"
struct vmbus_dynid {
......@@ -955,17 +956,6 @@ static void vmbus_onmessage_work(struct work_struct *work)
kfree(ctx);
}
static void hv_process_timer_expiration(struct hv_message *msg,
struct hv_per_cpu_context *hv_cpu)
{
struct clock_event_device *dev = hv_cpu->clk_evt;
if (dev->event_handler)
dev->event_handler(dev);
vmbus_signal_eom(msg, HVMSG_TIMER_EXPIRED);
}
void vmbus_on_msg_dpc(unsigned long data)
{
struct hv_per_cpu_context *hv_cpu = (void *)data;
......@@ -1159,9 +1149,10 @@ static void vmbus_isr(void)
/* Check if there are actual msgs to be processed */
if (msg->header.message_type != HVMSG_NONE) {
if (msg->header.message_type == HVMSG_TIMER_EXPIRED)
hv_process_timer_expiration(msg, hv_cpu);
else
if (msg->header.message_type == HVMSG_TIMER_EXPIRED) {
hv_stimer0_isr();
vmbus_signal_eom(msg, HVMSG_TIMER_EXPIRED);
} else
tasklet_schedule(&hv_cpu->msg_dpc);
}
......@@ -1263,14 +1254,19 @@ static int vmbus_bus_init(void)
ret = hv_synic_alloc();
if (ret)
goto err_alloc;
ret = hv_stimer_alloc(VMBUS_MESSAGE_SINT);
if (ret < 0)
goto err_alloc;
/*
* Initialize the per-cpu interrupt state and
* connect to the host.
* Initialize the per-cpu interrupt state and stimer state.
* Then connect to the host.
*/
ret = cpuhp_setup_state(CPUHP_AP_ONLINE_DYN, "hyperv/vmbus:online",
hv_synic_init, hv_synic_cleanup);
if (ret < 0)
goto err_alloc;
goto err_cpuhp;
hyperv_cpuhp_online = ret;
ret = vmbus_connect();
......@@ -1318,6 +1314,8 @@ static int vmbus_bus_init(void)
err_connect:
cpuhp_remove_state(hyperv_cpuhp_online);
err_cpuhp:
hv_stimer_free();
err_alloc:
hv_synic_free();
hv_remove_vmbus_irq();
......@@ -2064,7 +2062,7 @@ static struct acpi_driver vmbus_acpi_driver = {
static void hv_kexec_handler(void)
{
hv_synic_clockevents_cleanup();
hv_stimer_global_cleanup();
vmbus_initiate_unload(false);
vmbus_connection.conn_state = DISCONNECTED;
/* Make sure conn_state is set as hv_synic_cleanup checks for it */
......@@ -2075,6 +2073,8 @@ static void hv_kexec_handler(void)
static void hv_crash_handler(struct pt_regs *regs)
{
int cpu;
vmbus_initiate_unload(true);
/*
* In crash handler we can't schedule synic cleanup for all CPUs,
......@@ -2082,7 +2082,9 @@ static void hv_crash_handler(struct pt_regs *regs)
* for kdump.
*/
vmbus_connection.conn_state = DISCONNECTED;
hv_synic_cleanup(smp_processor_id());
cpu = smp_processor_id();
hv_stimer_cleanup(cpu);
hv_synic_cleanup(cpu);
hyperv_cleanup();
};
......@@ -2131,7 +2133,7 @@ static void __exit vmbus_exit(void)
hv_remove_kexec_handler();
hv_remove_crash_handler();
vmbus_connection.conn_state = DISCONNECTED;
hv_synic_clockevents_cleanup();
hv_stimer_global_cleanup();
vmbus_disconnect();
hv_remove_vmbus_irq();
for_each_online_cpu(cpu) {
......
drivers/iio/humidity/dht11.c
View file @ 927ba67a
......@@ -149,7 +149,7 @@ static int dht11_decode(struct dht11 *dht11, int offset)
return -EIO;
}
dht11->timestamp = ktime_get_boot_ns();
dht11->timestamp = ktime_get_boottime_ns();
if (hum_int < 4) { /* DHT22: 100000 = (3*256+232)*100 */
dht11->temperature = (((temp_int & 0x7f) << 8) + temp_dec) *
((temp_int & 0x80) ? -100 : 100);
......@@ -177,7 +177,7 @@ static irqreturn_t dht11_handle_irq(int irq, void *data)
/* TODO: Consider making the handler safe for IRQ sharing */
if (dht11->num_edges < DHT11_EDGES_PER_READ && dht11->num_edges >= 0) {
dht11->edges[dht11->num_edges].ts = ktime_get_boot_ns();
dht11->edges[dht11->num_edges].ts = ktime_get_boottime_ns();
dht11->edges[dht11->num_edges++].value =
gpio_get_value(dht11->gpio);
......@@ -196,7 +196,7 @@ static int dht11_read_raw(struct iio_dev *iio_dev,
int ret, timeres, offset;
mutex_lock(&dht11->lock);
if (dht11->timestamp + DHT11_DATA_VALID_TIME < ktime_get_boot_ns()) {
if (dht11->timestamp + DHT11_DATA_VALID_TIME < ktime_get_boottime_ns()) {
timeres = ktime_get_resolution_ns();
dev_dbg(dht11->dev, "current timeresolution: %dns\n", timeres);
if (timeres > DHT11_MIN_TIMERES) {
......@@ -322,7 +322,7 @@ static int dht11_probe(struct platform_device *pdev)
return -EINVAL;
}
dht11->timestamp = ktime_get_boot_ns() - DHT11_DATA_VALID_TIME - 1;
dht11->timestamp = ktime_get_boottime_ns() - DHT11_DATA_VALID_TIME - 1;
dht11->num_edges = -1;
platform_set_drvdata(pdev, iio);
......
drivers/iio/industrialio-core.c
View file @ 927ba67a
......@@ -228,9 +228,9 @@ s64 iio_get_time_ns(const struct iio_dev *indio_dev)
ktime_get_coarse_ts64(&tp);
return timespec64_to_ns(&tp);
case CLOCK_BOOTTIME:
return ktime_get_boot_ns();
return ktime_get_boottime_ns();
case CLOCK_TAI:
return ktime_get_tai_ns();
return ktime_get_clocktai_ns();
default:
BUG();
}
......
drivers/infiniband/hw/mlx4/alias_GUID.c
View file @ 927ba67a
......@@ -310,7 +310,7 @@ static void aliasguid_query_handler(int status,
if (status) {
pr_debug("(port: %d) failed: status = %d\n",
cb_ctx->port, status);
rec->time_to_run = ktime_get_boot_ns() + 1 * NSEC_PER_SEC;
rec->time_to_run = ktime_get_boottime_ns() + 1 * NSEC_PER_SEC;
goto out;
}
......@@ -416,7 +416,7 @@ static void aliasguid_query_handler(int status,
be64_to_cpu((__force __be64)rec->guid_indexes),
be64_to_cpu((__force __be64)applied_guid_indexes),
be64_to_cpu((__force __be64)declined_guid_indexes));
rec->time_to_run = ktime_get_boot_ns() +
rec->time_to_run = ktime_get_boottime_ns() +
resched_delay_sec * NSEC_PER_SEC;
} else {
rec->status = MLX4_GUID_INFO_STATUS_SET;
......@@ -709,7 +709,7 @@ static int get_low_record_time_index(struct mlx4_ib_dev *dev, u8 port,
}
}
if (resched_delay_sec) {
u64 curr_time = ktime_get_boot_ns();
u64 curr_time = ktime_get_boottime_ns();
*resched_delay_sec = (low_record_time < curr_time) ? 0 :
div_u64((low_record_time - curr_time), NSEC_PER_SEC);
......
drivers/leds/trigger/ledtrig-activity.c
View file @ 927ba67a
......@@ -70,7 +70,7 @@ static void led_activity_function(struct timer_list *t)
* down to 16us, ensuring we won't overflow 32-bit computations below
* even up to 3k CPUs, while keeping divides cheap on smaller systems.
*/
curr_boot = ktime_get_boot_ns() * cpus;
curr_boot = ktime_get_boottime_ns() * cpus;
diff_boot = (curr_boot - activity_data->last_boot) >> 16;
diff_used = (curr_used - activity_data->last_used) >> 16;
activity_data->last_boot = curr_boot;
......
drivers/net/wireless/intel/iwlwifi/mvm/ftm-initiator.c
View file @ 927ba67a
......@@ -93,7 +93,7 @@ void iwl_mvm_ftm_restart(struct iwl_mvm *mvm)
struct cfg80211_pmsr_result result = {
.status = NL80211_PMSR_STATUS_FAILURE,
.final = 1,
.host_time = ktime_get_boot_ns(),
.host_time = ktime_get_boottime_ns(),
.type = NL80211_PMSR_TYPE_FTM,
};
int i;
......
drivers/net/wireless/intel/iwlwifi/mvm/rx.c
View file @ 927ba67a
This diff is collapsed.
drivers/net/wireless/mac80211_hwsim.c
View file @ 927ba67a
This diff is collapsed.
drivers/net/wireless/ti/wlcore/main.c
View file @ 927ba67a
This diff is collapsed.
drivers/net/wireless/ti/wlcore/rx.c
View file @ 927ba67a
This diff is collapsed.
drivers/net/wireless/ti/wlcore/tx.c
View file @ 927ba67a
This diff is collapsed.
drivers/net/wireless/virt_wifi.c
View file @ 927ba67a
This diff is collapsed.
include/asm-generic/vdso/vsyscall.h 0 → 100644
View file @ 927ba67a
This diff is collapsed.
include/clocksource/hyperv_timer.h 0 → 100644
View file @ 927ba67a
This diff is collapsed.
include/clocksource/timer-davinci.h 0 → 100644
View file @ 927ba67a
This diff is collapsed.
include/linux/cpuhotplug.h
View file @ 927ba67a
This diff is collapsed.
include/linux/hrtimer.h
View file @ 927ba67a
This diff is collapsed.
include/linux/hrtimer_defs.h 0 → 100644
View file @ 927ba67a
This diff is collapsed.
include/linux/timekeeping.h
View file @ 927ba67a
This diff is collapsed.
include/linux/timer.h
View file @ 927ba67a
This diff is collapsed.
include/net/cfg80211.h
View file @ 927ba67a
This diff is collapsed.
include/vdso/datapage.h 0 → 100644
View file @ 927ba67a
This diff is collapsed.
include/vdso/helpers.h 0 → 100644
View file @ 927ba67a
This diff is collapsed.
include/vdso/vsyscall.h 0 → 100644
View file @ 927ba67a
This diff is collapsed.
kernel/bpf/syscall.c
View file @ 927ba67a
This diff is collapsed.
kernel/events/core.c
View file @ 927ba67a
This diff is collapsed.
kernel/fork.c
View file @ 927ba67a
This diff is collapsed.
kernel/time/Makefile
View file @ 927ba67a
This diff is collapsed.
kernel/time/alarmtimer.c
View file @ 927ba67a
This diff is collapsed.
kernel/time/clocksource.c
View file @ 927ba67a
This diff is collapsed.
kernel/time/hrtimer.c
View file @ 927ba67a
This diff is collapsed.
kernel/time/ntp.c
View file @ 927ba67a
This diff is collapsed.
kernel/time/posix-timers.c
View file @ 927ba67a
This diff is collapsed.
kernel/time/time.c
View file @ 927ba67a
This diff is collapsed.
kernel/time/timekeeping.c
View file @ 927ba67a
This diff is collapsed.
kernel/time/timer_list.c
View file @ 927ba67a
This diff is collapsed.
kernel/time/vsyscall.c 0 → 100644
View file @ 927ba67a
This diff is collapsed.
lib/Kconfig
View file @ 927ba67a
This diff is collapsed.
lib/vdso/Kconfig 0 → 100644
View file @ 927ba67a
This diff is collapsed.
lib/vdso/Makefile 0 → 100644
View file @ 927ba67a
This diff is collapsed.
lib/vdso/gettimeofday.c 0 → 100644
View file @ 927ba67a
This diff is collapsed.
tools/testing/selftests/timers/freq-step.c
View file @ 927ba67a
This diff is collapsed.
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment
GitLab Nexedi Edition | About GitLab | About Nexedi | 沪ICP备2021021310号-2 | 沪ICP备2021021310号-7