• Thomas Gleixner's avatar
    lib/vdso: Make delta calculation work correctly · 9d90b93b
    Thomas Gleixner authored
    The x86 vdso implementation on which the generic vdso library is based on
    has subtle (unfortunately undocumented) twists:
    
     1) The code assumes that the clocksource mask is U64_MAX which means that
        no bits are masked. Which is true for any valid x86 VDSO clocksource.
        Stupidly it still did the mask operation for no reason and at the wrong
        place right after reading the clocksource.
    
     2) It contains a sanity check to catch the case where slightly
        unsynchronized TSC values can be observed which would cause the delta
        calculation to make a huge jump. It therefore checks whether the
        current TSC value is larger than the value on which the current
        conversion is based on. If it's not larger the base value is used to
        prevent time jumps.
    
    #1 Is not only stupid for the X86 case because it does the masking for no
    reason it is also completely wrong for clocksources with a smaller mask
    which can legitimately wrap around during a conversion period. The core
    timekeeping code does it correct by applying the mask after the delta
    calculation:
    
    	(now - base) & mask
    
    #2 is equally broken for clocksources which have smaller masks and can wrap
    around during a conversion period because there the now > base check is
    just wrong and causes stale time stamps and time going backwards issues.
    
    Unbreak it by:
    
      1) Removing the mask operation from the clocksource read which makes the
         fallback detection work for all clocksources
    
      2) Replacing the conditional delta calculation with a overrideable inline
         function.
    
    #2 could reuse clocksource_delta() from the timekeeping code but that
    results in a significant performance hit for the x86 VSDO. The timekeeping
    core code must have the non optimized version as it has to operate
    correctly with clocksources which have smaller masks as well to handle the
    case where TSC is discarded as timekeeper clocksource and replaced by HPET
    or pmtimer. For the VDSO there is no replacement clocksource. If TSC is
    unusable the syscall is enforced which does the right thing.
    
    To accommodate to the needs of various architectures provide an
    override-able inline function which defaults to the regular delta
    calculation with masking:
    
    	(now - base) & mask
    
    Override it for x86 with the non-masking and checking version.
    
    This unbreaks the ARM64 syscall fallback operation, allows to use
    clocksources with arbitrary width and preserves the performance
    optimization for x86.
    Signed-off-by: default avatarThomas Gleixner <tglx@linutronix.de>
    Reviewed-by: default avatarVincenzo Frascino <vincenzo.frascino@arm.com>
    Cc: linux-arch@vger.kernel.org
    Cc: LAK <linux-arm-kernel@lists.infradead.org>
    Cc: linux-mips@vger.kernel.org
    Cc: linux-kselftest@vger.kernel.org
    Cc: catalin.marinas@arm.com
    Cc: Will Deacon <will.deacon@arm.com>
    Cc: Arnd Bergmann <arnd@arndb.de>
    Cc: linux@armlinux.org.uk
    Cc: Ralf Baechle <ralf@linux-mips.org>
    Cc: paul.burton@mips.com
    Cc: Daniel Lezcano <daniel.lezcano@linaro.org>
    Cc: salyzyn@android.com
    Cc: pcc@google.com
    Cc: shuah@kernel.org
    Cc: 0x7f454c46@gmail.com
    Cc: linux@rasmusvillemoes.dk
    Cc: huw@codeweavers.com
    Cc: sthotton@marvell.com
    Cc: andre.przywara@arm.com
    Cc: Andy Lutomirski <luto@kernel.org>
    Link: https://lkml.kernel.org/r/alpine.DEB.2.21.1906261159230.32342@nanos.tec.linutronix.de
    9d90b93b
gettimeofday.c 5.45 KB