Export eventfd_wake_count so that some modules can use
the eventfd_signal_count() to check whether the
eventfd_signal() call should be deferred to a safe context.

NB(mst): this patch is not needed in Linus tree since there
eventfd_signal_count() has been superseded by an already exported
eventfd_signal_allowed().
Signed-off-by: Xie Yongji <xieyongji@bytedance.com>
Link: https://lore.kernel.org/r/20210831103634.33-3-xieyongji@bytedance.comAcked-by: Jason Wang <jasowang@redhat.com>
Signed-off-by: Michael S. Tsirkin <mst@redhat.com>

Export alloc_iova_fast() and free_iova_fast() so that
some modules can make use of the per-CPU cache to get
rid of rbtree spinlock in alloc_iova() and free_iova()
during IOVA allocation.
Signed-off-by: Xie Yongji <xieyongji@bytedance.com>
Acked-by: Jason Wang <jasowang@redhat.com>
Acked-by: Will Deacon <will@kernel.org>
Link: https://lore.kernel.org/r/20210831103634.33-2-xieyongji@bytedance.comSigned-off-by: Michael S. Tsirkin <mst@redhat.com>

Usually we use "likely/unlikely" to optimize the fast path. Remove
redundant "likely/unlikely" statements in the control path to simplify
the code and make it easier to read.
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Signed-off-by: Max Gurtovoy <mgurtovoy@nvidia.com>
Link: https://lore.kernel.org/r/20210905085717.7427-1-mgurtovoy@nvidia.comSigned-off-by: Michael S. Tsirkin <mst@redhat.com>
Reviewed-by: Chaitanya Kulkarni <ckulkarnilinux@gmail.com>
Reviewed-by: Christoph Hellwig <hch@lst.de>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>

Use the helper virtio_find_vqs().
Signed-off-by: Xianting Tian <xianting.tian@linux.alibaba.com>
Reviewed-by: David Hildenbrand <david@redhat.com>
Acked-by: Jason Wang <jasowang@redhat.com>
Link: https://lore.kernel.org/r/20210723054259.2779-1-xianting.tian@linux.alibaba.comSigned-off-by: Michael S. Tsirkin <mst@redhat.com>

it's a nice refactor to make use of
PFN_PHYS/PFN_UP/PFN_DOWN helper macro
Signed-off-by: Cai Huoqing <caihuoqing@baidu.com>
Link: https://lore.kernel.org/r/20210802013717.851-1-caihuoqing@baidu.comAcked-by: Jason Wang <jasowang@redhat.com>
Signed-off-by: Michael S. Tsirkin <mst@redhat.com>

Set 'MSG_EOR' in one of message sent, check that 'MSG_EOR'
is visible in corresponding message at receiver.
Signed-off-by: Arseny Krasnov <arseny.krasnov@kaspersky.com>
Reviewed-by: Stefano Garzarella <sgarzare@redhat.com>
Link: https://lore.kernel.org/r/20210903123321.3273866-1-arseny.krasnov@kaspersky.comSigned-off-by: Michael S. Tsirkin <mst@redhat.com>

Record is supported via MSG_EOR flag, while current logic operates
with message, so rename variables from 'record' to 'message'.
Signed-off-by: Arseny Krasnov <arseny.krasnov@kaspersky.com>
Reviewed-by: Stefano Garzarella <sgarzare@redhat.com>
Link: https://lore.kernel.org/r/20210903123306.3273757-1-arseny.krasnov@kaspersky.comSigned-off-by: Michael S. Tsirkin <mst@redhat.com>

If packet has 'EOR' bit - set MSG_EOR in 'recvmsg()' flags.
Signed-off-by: Arseny Krasnov <arseny.krasnov@kaspersky.com>
Reviewed-by: Stefano Garzarella <sgarzare@redhat.com>
Link: https://lore.kernel.org/r/20210903123251.3273639-1-arseny.krasnov@kaspersky.comSigned-off-by: Michael S. Tsirkin <mst@redhat.com>

'MSG_EOR' handling has similar logic as 'MSG_EOM' - if bit present
in packet's header, reset it to 0. Then restore it back if packet
processing wasn't completed. Instead of bool variable for each
flag, bit mask variable was added: it has logical OR of 'MSG_EOR'
and 'MSG_EOM' if needed, to restore flags, this variable is ORed
with flags field of packet.
Signed-off-by: Arseny Krasnov <arseny.krasnov@kaspersky.com>
Link: https://lore.kernel.org/r/20210903123238.3273526-1-arseny.krasnov@kaspersky.comSigned-off-by: Michael S. Tsirkin <mst@redhat.com>
Reviewed-by: Stefano Garzarella <sgarzare@redhat.com>

This bit is used to handle POSIX MSG_EOR flag passed from
userspace in 'send*()' system calls. It marks end of each
record and is visible to receiver using 'recvmsg()' system
call.
Signed-off-by: Arseny Krasnov <arseny.krasnov@kaspersky.com>
Reviewed-by: Stefano Garzarella <sgarzare@redhat.com>
Link: https://lore.kernel.org/r/20210903123225.3273425-1-arseny.krasnov@kaspersky.comSigned-off-by: Michael S. Tsirkin <mst@redhat.com>

This current implemented bit is used to mark end of messages
('EOM' - end of message), not records('EOR' - end of record).
Also rename 'record' to 'message' in implementation as it is
different things.
Signed-off-by: Arseny Krasnov <arseny.krasnov@kaspersky.com>
Reviewed-by: Stefano Garzarella <sgarzare@redhat.com>
Link: https://lore.kernel.org/r/20210903123109.3273053-1-arseny.krasnov@kaspersky.comSigned-off-by: Michael S. Tsirkin <mst@redhat.com>

Bind the virtio devices with their of_node. This will help users of the
virtio devices to mention their dependencies on the device in the DT
itself. Like GPIO pin users can use the phandle of the device node, or
the node may contain more subnodes to add i2c or spi eeproms and other
users.
Reviewed-by: Arnd Bergmann <arnd@arndb.de>
Signed-off-by: Viresh Kumar <viresh.kumar@linaro.org>
Link: https://lore.kernel.org/r/94c12705602929968477aaf27e02439eb7a7f253.1627362340.git.viresh.kumar@linaro.orgSigned-off-by: Michael S. Tsirkin <mst@redhat.com>

This synchronizes the virtio ids with the latest list from virtio
specification.
Signed-off-by: Viresh Kumar <viresh.kumar@linaro.org>
Link: https://lore.kernel.org/r/61b27e3bc61fb0c9f067001e95cfafc5d37d414a.1627362340.git.viresh.kumar@linaro.orgSigned-off-by: Michael S. Tsirkin <mst@redhat.com>

This patch adds binding for virtio GPIO controller, it is based on
virtio-device bindings.
Reviewed-by: Arnd Bergmann <arnd@arndb.de>
Signed-off-by: Viresh Kumar <viresh.kumar@linaro.org>
Link: https://lore.kernel.org/r/acf7402ef4aabc0ad6295c32846f2bef1cd9b56a.1627362340.git.viresh.kumar@linaro.orgSigned-off-by: Michael S. Tsirkin <mst@redhat.com>
Reviewed-by: Rob Herring <robh@kernel.org>

This patch adds binding for virtio I2C device, it is based on
virtio-device bindings.
Acked-by: Wolfram Sang <wsa@kernel.org>
Reviewed-by: Arnd Bergmann <arnd@arndb.de>
Signed-off-by: Viresh Kumar <viresh.kumar@linaro.org>
Link: https://lore.kernel.org/r/33c317b95097ce491845c697db1e8285e3ec1d41.1627362340.git.viresh.kumar@linaro.orgSigned-off-by: Michael S. Tsirkin <mst@redhat.com>
Reviewed-by: Rob Herring <robh@kernel.org>

Allow virtio device sub-nodes to be added to the virtio mmio or pci
nodes. The compatible property for virtio device must be of the format
"virtio,device<ID>", where ID is virtio device ID in hexadecimal format.
Signed-off-by: Viresh Kumar <viresh.kumar@linaro.org>
Link: https://lore.kernel.org/r/d8319fd18df7086b12cdcc23193c313893aa071a.1627362340.git.viresh.kumar@linaro.orgSigned-off-by: Michael S. Tsirkin <mst@redhat.com>
Reviewed-by: Arnd Bergmann <arnd@arndb.de>
Reviewed-by: Rob Herring <robh@kernel.org>

The size passed to alloc_iova() should be the size of page frames.
Now we use byte granularity for the iova domain, so it's safe to
pass the size in bytes to alloc_iova(). But it would be better to use
iova_shift() for the size to avoid future bugs if we change granularity.
Signed-off-by: Xie Yongji <xieyongji@bytedance.com>
Reviewed-by: Stefano Garzarella <sgarzare@redhat.com>
Link: https://lore.kernel.org/r/20210809100923.38-1-xieyongji@bytedance.comSigned-off-by: Michael S. Tsirkin <mst@redhat.com>
Acked-by: Jason Wang <jasowang@redhat.com>

use SPDX-License-Identifier instead of a verbose license text
Signed-off-by: Cai Huoqing <caihuoqing@baidu.com>
Link: https://lore.kernel.org/r/20210821123320.734-1-caihuoqing@baidu.comSigned-off-by: Michael S. Tsirkin <mst@redhat.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>

Multiqueue support requires additional virtio_net_q objects to be added
or removed per the configured number of queue pairs. In addition the RQ
tables needs to be modified to match the number of configured receive
queues so the packets are dispatched to the right virtqueue according to
the hash result.

Note: qemu v6.0.0 is broken when the device requests more than two data
queues; no net device will be created for the vdpa device. To avoid
this, one should specify mq=off to qemu. In this case it will end up
with a single queue.
Signed-off-by: Eli Cohen <elic@nvidia.com>
Link: https://lore.kernel.org/r/20210823052123.14909-7-elic@nvidia.comSigned-off-by: Michael S. Tsirkin <mst@redhat.com>

Add support to handle control virtqueue configurations per virtio
specification. The control virtqueue is implemented in software and no
hardware offloading is involved.

Control VQ configuration need task context, therefore all configurations
are handled in a workqueue created for the purpose.

Modifications are made to the memory registration code to allow for
saving a copy of itolb to be used by the control VQ to access the vring.

The max number of data virtqueus supported by the driver has been
updated to 2 since multiqueue is not supported at this stage and we need
to ensure consistency of VQ indices mapping to either data or control
VQ.
Signed-off-by: Eli Cohen <elic@nvidia.com>
Link: https://lore.kernel.org/r/20210823052123.14909-6-elic@nvidia.comSigned-off-by: Michael S. Tsirkin <mst@redhat.com>

Following patches add control virtuqeue and multiqueue support. We want
to verify that the index value to callbacks referencing a virtqueue is
valid.

The logic defining valid indices is as follows:
CVQ clear: 0 and 1.
CVQ set, MQ clear: 0, 1 and 2
CVQ set, MQ set: 0..nvq where nvq is whatever provided to
_vdpa_register_device()
Signed-off-by: Eli Cohen <elic@nvidia.com>
Link: https://lore.kernel.org/r/20210823052123.14909-5-elic@nvidia.comSigned-off-by: Michael S. Tsirkin <mst@redhat.com>

Instead, define an array of struct vdpa_callback on struct mlx5_vdpa_net
and use it to store callbacks for any virtqueue provided. This is
required due to the fact that callback configurations arrive before feature
negotiation. With control VQ and multiqueue introduced next we want to
save the information until after feature negotiation where we know the
CVQ index.
Signed-off-by: Eli Cohen <elic@nvidia.com>
Link: https://lore.kernel.org/r/20210823052123.14909-4-elic@nvidia.comSigned-off-by: Michael S. Tsirkin <mst@redhat.com>

Use struct mlx5_vdpa_dev as an argument to setup_driver() and a few
others in preparation to control virtqueue support in a subsequent
patch. The control virtqueue is part of struct mlx5_vdpa_dev so this is
required.
Signed-off-by: Eli Cohen <elic@nvidia.com>
Link: https://lore.kernel.org/r/20210823052123.14909-3-elic@nvidia.comSigned-off-by: Michael S. Tsirkin <mst@redhat.com>

linux/if_vlan.h is not required.
Remove it.
Signed-off-by: Eli Cohen <elic@nvidia.com>
Link: https://lore.kernel.org/r/20210823052123.14909-2-elic@nvidia.comSigned-off-by: Michael S. Tsirkin <mst@redhat.com>

This commit enbales multi-queue and control vq
features for ifcvf
Signed-off-by: Zhu Lingshan <lingshan.zhu@intel.com>
Link: https://lore.kernel.org/r/20210818095714.3220-3-lingshan.zhu@intel.comSigned-off-by: Michael S. Tsirkin <mst@redhat.com>
Acked-by: Jason Wang <jasowang@redhat.com>

To enable this multi-queue feature for ifcvf, this commit
intends to detect and use the onboard number of queues
directly than IFCVF_MAX_QUEUE_PAIRS = 1 (removed)
Signed-off-by: Zhu Lingshan <lingshan.zhu@intel.com>
Link: https://lore.kernel.org/r/20210818095714.3220-2-lingshan.zhu@intel.comSigned-off-by: Michael S. Tsirkin <mst@redhat.com>
Acked-by: Jason Wang <jasowang@redhat.com>

This commit implements the management netlink framework for ifcvf,
including register and add / remove a device

It works with iproute2:
[root@localhost lszhu]# vdpa mgmtdev show -jp
{
    "mgmtdev": {
        "pci/0000:01:00.5": {
            "supported_classes": [ "net" ]
        },
        "pci/0000:01:00.6": {
            "supported_classes": [ "net" ]
        }
    }
}

[root@localhost lszhu]# vdpa dev add mgmtdev pci/0000:01:00.5 name vdpa0
[root@localhost lszhu]# vdpa dev add mgmtdev pci/0000:01:00.6 name vdpa1
Signed-off-by: Zhu Lingshan <lingshan.zhu@intel.com>
Acked-by: Jason Wang <jasowang@redhat.com>
Link: https://lore.kernel.org/r/20210812032454.24486-3-lingshan.zhu@intel.comSigned-off-by: Michael S. Tsirkin <mst@redhat.com>

This commit introduces a new function get_dev_type() which returns
the virtio device id of a device, to avoid duplicated code.
Signed-off-by: Zhu Lingshan <lingshan.zhu@intel.com>
Acked-by: Jason Wang <jasowang@redhat.com>
Link: https://lore.kernel.org/r/20210812032454.24486-2-lingshan.zhu@intel.comSigned-off-by: Michael S. Tsirkin <mst@redhat.com>

Pull clk fix from Stephen Boyd:
 "One hotfix for a NULL pointer deref in the Renesas usb clk driver"

* tag 'clk-fixes-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/clk/linux:
  clk: renesas: rcar-usb2-clock-sel: Fix kernel NULL pointer dereference

Pull scheduler fixes from Borislav Petkov:

 - Have get_push_task() check whether current has migration disabled and
   thus avoid useless invocations of the migration thread

 - Rework initialization flow so that all rq->core's are initialized,
   even of CPUs which have not been onlined yet, so that iterating over
   them all works as expected

* tag 'sched_urgent_for_v5.14' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip:
  sched: Fix get_push_task() vs migrate_disable()
  sched: Fix Core-wide rq->lock for uninitialized CPUs

Pull irq fix from Borislav Petkov:

 - Have msix_mask_all() check a global control which says whether MSI-X
   masking should be done and thus make it usable on Xen-PV too

* tag 'irq_urgent_for_v5.14' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip:
  PCI/MSI: Skip masking MSI-X on Xen PV

Pull perf fixes from Borislav Petkov:

 - Prevent the amd/power module from being removed while in use

 - Mark AMD IBS as not supporting content exclusion

 - Add a workaround for AMD erratum #1197 where IBS registers might not
   be restored properly after exiting CC6 state

 - Fix a potential truncation of a 32-bit variable due to shifting

 - Read the correct bits describing the number of configurable address
   ranges on Intel PT

* tag 'perf_urgent_for_v5.14' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip:
  perf/x86/amd/power: Assign pmu.module
  perf/x86/amd/ibs: Extend PERF_PMU_CAP_NO_EXCLUDE to IBS Op
  perf/x86/amd/ibs: Work around erratum #1197
  perf/x86/intel/uncore: Fix integer overflow on 23 bit left shift of a u32
  perf/x86/intel/pt: Fix mask of num_address_ranges

Pull x86 fixes from Borislav Petkov:

 - Fix build error on RHEL where -Werror=maybe-uninitialized is set.

 - Restore the firmware's IDT when calling EFI boot services and before
   ExitBootServices() has been called. This fixes a boot failure on what
   appears to be a tablet with 32-bit UEFI running a 64-bit kernel.

* tag 'x86_urgent_for_v5.14' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip:
  x86/resctrl: Fix a maybe-uninitialized build warning treated as error
  x86/efi: Restore Firmware IDT before calling ExitBootServices()

This reverts commit 83af58f8.

It turns out that at least the assembly implementation for strncpy() was
buggy.  Revert the whole commit and return back to the default coding.
Signed-off-by: Helge Deller <deller@gmx.de>
Cc: <stable@vger.kernel.org> # v5.4+
Cc: Rasmus Villemoes <linux@rasmusvillemoes.dk>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>

The probe was manually passing NULL instead of dev to devm_clk_hw_register.
This caused a Unable to handle kernel NULL pointer dereference error.
Fix this by passing 'dev'.
Signed-off-by: Adam Ford <aford173@gmail.com>
Fixes: a20a40a8 ("clk: renesas: rcar-usb2-clock-sel: Fix error handling in .probe()")
Reviewed-by: Geert Uytterhoeven <geert+renesas@glider.be>
Signed-off-by: Stephen Boyd <sboyd@kernel.org>

Pull SCSI fix from James Bottomley:
 "A single fix for a race introduced by a fix that went into 5.14-rc5"

* tag 'scsi-fixes' of git://git.kernel.org/pub/scm/linux/kernel/git/jejb/scsi:
  scsi: core: Fix hang of freezing queue between blocking and running device

Pull USB fixes from Greg KH:
 "Here are a few tiny USB fixes for reported issues with some USB
  drivers.

  These fixes include:

   - gadget driver fixes for regressions

   - tcpm driver fix

   - dwc3 driver fixes

   - xhci renesas firmware loading fix, again.

   - usb serial option driver device id addition

   - usb serial ch341 revert for regression

  All all of these have been in linux-next with no reported problems"

* tag 'usb-5.14' of git://git.kernel.org/pub/scm/linux/kernel/git/gregkh/usb:
  usb: gadget: u_audio: fix race condition on endpoint stop
  usb: gadget: f_uac2: fixup feedback endpoint stop
  usb: typec: tcpm: Raise vdm_sm_running flag only when VDM SM is running
  usb: renesas-xhci: Prefer firmware loading on unknown ROM state
  usb: dwc3: gadget: Stop EP0 transfers during pullup disable
  usb: dwc3: gadget: Fix dwc3_calc_trbs_left()
  Revert "USB: serial: ch341: fix character loss at high transfer rates"
  USB: serial: option: add new VID/PID to support Fibocom FG150

Pull powerpc fixes from Michael Ellerman:

 - Fix scv implicit soft-mask table for relocated (eg. kdump) kernels

 - Re-enable ARCH_ENABLE_SPLIT_PMD_PTLOCK, which was disabled due to a
   typo

Thanks to Lukas Bulwahn, Nicholas Piggin, and Daniel Axtens.

* tag 'powerpc-5.14-7' of git://git.kernel.org/pub/scm/linux/kernel/git/powerpc/linux:
  powerpc/64s: Fix scv implicit soft-mask table for relocated kernels
  powerpc: Re-enable ARCH_ENABLE_SPLIT_PMD_PTLOCK

Pull block fixes from Jens Axboe:

 - Revert the mq-deadline priority handling, it's causing serious
   performance regressions. While experimental patches exists to fix
   this up, it's too late to do so now. Revert it and re-do it properly
   for 5.15 instead.

 - Fix a NULL vs IS_ERR() regression in this release (Dan)

 - Fix a mq-deadline accounting regression in this release (Bart)

 - Mark cryptoloop as deprecated. It's broken and dm-crypt fully
   supports it, and it's actively intefering with loop. Plan on removal
   for 5.16 (Christoph)

* tag 'block-5.14-2021-08-27' of git://git.kernel.dk/linux-block:
  cryptoloop: add a deprecation warning
  pd: fix a NULL vs IS_ERR() check
  Revert "block/mq-deadline: Prioritize high-priority requests"
  mq-deadline: Fix request accounting