• Dave Chinner's avatar
    xfs: don't take a spinlock unconditionally in the DIO fastpath · 977ec4dd
    Dave Chinner authored
    Because this happens at high thread counts on high IOPS devices
    doing mixed read/write AIO-DIO to a single file at about a million
    iops:
    
       64.09%     0.21%  [kernel]            [k] io_submit_one
       - 63.87% io_submit_one
          - 44.33% aio_write
             - 42.70% xfs_file_write_iter
                - 41.32% xfs_file_dio_write_aligned
                   - 25.51% xfs_file_write_checks
                      - 21.60% _raw_spin_lock
                         - 21.59% do_raw_spin_lock
                            - 19.70% __pv_queued_spin_lock_slowpath
    
    This also happens of the IO completion IO path:
    
       22.89%     0.69%  [kernel]            [k] xfs_dio_write_end_io
       - 22.49% xfs_dio_write_end_io
          - 21.79% _raw_spin_lock
             - 20.97% do_raw_spin_lock
                - 20.10% __pv_queued_spin_lock_slowpath
    
    IOWs, fio is burning ~14 whole CPUs on this spin lock.
    
    So, do an unlocked check against inode size first, then if we are
    at/beyond EOF, take the spinlock and recheck. This makes the
    spinlock disappear from the overwrite fastpath.
    
    I'd like to report that fixing this makes things go faster. It
    doesn't - it just exposes the the XFS_ILOCK as the next severe
    contention point doing extent mapping lookups, and that now burns
    all the 14 CPUs this spinlock was burning.
    Signed-off-by: default avatarDave Chinner <dchinner@redhat.com>
    Reviewed-by: default avatarCarlos Maiolino <cmaiolino@redhat.com>
    Reviewed-by: default avatarDarrick J. Wong <djwong@kernel.org>
    Signed-off-by: default avatarDarrick J. Wong <djwong@kernel.org>
    977ec4dd
xfs_file.c 37.3 KB