• Asias He's avatar
    virtio-blk: Disable callback in virtblk_done() · bb811108
    Asias He authored
    This reduces unnecessary interrupts that host could send to guest while
    guest is in the progress of irq handling.
    
    If one vcpu is handling the irq, while another interrupt comes, in
    handle_edge_irq(), the guest will mask the interrupt via mask_msi_irq()
    which is a very heavy operation that goes all the way down to host.
    
     Here are some performance numbers on qemu:
    
     Before:
     -------------------------------------
       seq-read  : io=0 B, bw=269730KB/s, iops=67432 , runt= 62200msec
       seq-write : io=0 B, bw=339716KB/s, iops=84929 , runt= 49386msec
       rand-read : io=0 B, bw=270435KB/s, iops=67608 , runt= 62038msec
       rand-write: io=0 B, bw=354436KB/s, iops=88608 , runt= 47335msec
         clat (usec): min=101 , max=138052 , avg=14822.09, stdev=11771.01
         clat (usec): min=96 , max=81543 , avg=11798.94, stdev=7735.60
         clat (usec): min=128 , max=140043 , avg=14835.85, stdev=11765.33
         clat (usec): min=109 , max=147207 , avg=11337.09, stdev=5990.35
       cpu          : usr=15.93%, sys=60.37%, ctx=7764972, majf=0, minf=54
       cpu          : usr=32.73%, sys=120.49%, ctx=7372945, majf=0, minf=1
       cpu          : usr=18.84%, sys=58.18%, ctx=7775420, majf=0, minf=1
       cpu          : usr=24.20%, sys=59.85%, ctx=8307886, majf=0, minf=0
       vdb: ios=8389107/8368136, merge=0/0, ticks=19457874/14616506,
     in_queue=34206098, util=99.68%
      43: interrupt in total: 887320
     fio --exec_prerun="echo 3 > /proc/sys/vm/drop_caches" --group_reporting
     --ioscheduler=noop --thread --bs=4k --size=512MB --direct=1 --numjobs=16
     --ioengine=libaio --iodepth=64 --loops=3 --ramp_time=0
     --filename=/dev/vdb --name=seq-read --stonewall --rw=read
     --name=seq-write --stonewall --rw=write --name=rnd-read --stonewall
     --rw=randread --name=rnd-write --stonewall --rw=randwrite
    
     After:
     -------------------------------------
       seq-read  : io=0 B, bw=309503KB/s, iops=77375 , runt= 54207msec
       seq-write : io=0 B, bw=448205KB/s, iops=112051 , runt= 37432msec
       rand-read : io=0 B, bw=311254KB/s, iops=77813 , runt= 53902msec
       rand-write: io=0 B, bw=377152KB/s, iops=94287 , runt= 44484msec
         clat (usec): min=81 , max=90588 , avg=12946.06, stdev=9085.94
         clat (usec): min=57 , max=72264 , avg=8967.97, stdev=5951.04
         clat (usec): min=29 , max=101046 , avg=12889.95, stdev=9067.91
         clat (usec): min=52 , max=106152 , avg=10660.56, stdev=4778.19
       cpu          : usr=15.05%, sys=57.92%, ctx=77109411, majf=0, minf=54
       cpu          : usr=26.78%, sys=101.40%, ctx=7387891, majf=0, minf=2
       cpu          : usr=19.03%, sys=58.17%, ctx=7681976, majf=0, minf=8
       cpu          : usr=24.65%, sys=58.34%, ctx=8442632, majf=0, minf=4
       vdb: ios=8389086/8361888, merge=0/0, ticks=17243780/12742010,
     in_queue=30078377, util=99.59%
      43: interrupt in total: 1259639
     fio --exec_prerun="echo 3 > /proc/sys/vm/drop_caches" --group_reporting
     --ioscheduler=noop --thread --bs=4k --size=512MB --direct=1 --numjobs=16
     --ioengine=libaio --iodepth=64 --loops=3 --ramp_time=0
     --filename=/dev/vdb --name=seq-read --stonewall --rw=read
     --name=seq-write --stonewall --rw=write --name=rnd-read --stonewall
     --rw=randread --name=rnd-write --stonewall --rw=randwrite
    Signed-off-by: default avatarAsias He <asias@redhat.com>
    Signed-off-by: default avatarRusty Russell <rusty@rustcorp.com.au>
    bb811108
virtio_blk.c 24.6 KB