Commit 84cc6674 authored by Linus Torvalds's avatar Linus Torvalds

Merge tag 'for_linus' of git://git.kernel.org/pub/scm/linux/kernel/git/mst/vhost

Pull virtio updates from Michael Tsirkin:

 - device feature provisioning in ifcvf, mlx5

 - new SolidNET driver

 - support for zoned block device in virtio blk

 - numa support in virtio pmem

 - VIRTIO_F_RING_RESET support in vhost-net

 - more debugfs entries in mlx5

 - resume support in vdpa

 - completion batching in virtio blk

 - cleanup of dma api use in vdpa

 - now simulating more features in vdpa-sim

 - documentation, features, fixes all over the place

* tag 'for_linus' of git://git.kernel.org/pub/scm/linux/kernel/git/mst/vhost: (64 commits)
  vdpa/mlx5: support device features provisioning
  vdpa/mlx5: make MTU/STATUS presence conditional on feature bits
  vdpa: validate device feature provisioning against supported class
  vdpa: validate provisioned device features against specified attribute
  vdpa: conditionally read STATUS in config space
  vdpa: fix improper error message when adding vdpa dev
  vdpa/mlx5: Initialize CVQ iotlb spinlock
  vdpa/mlx5: Don't clear mr struct on destroy MR
  vdpa/mlx5: Directly assign memory key
  tools/virtio: enable to build with retpoline
  vringh: fix a typo in comments for vringh_kiov
  vhost-vdpa: print warning when vhost_vdpa_alloc_domain fails
  scsi: virtio_scsi: fix handling of kmalloc failure
  vdpa: Fix a couple of spelling mistakes in some messages
  vhost-net: support VIRTIO_F_RING_RESET
  vhost-scsi: convert sysfs snprintf and sprintf to sysfs_emit
  vdpa: mlx5: support per virtqueue dma device
  vdpa: set dma mask for vDPA device
  virtio-vdpa: support per vq dma device
  vdpa: introduce get_vq_dma_device()
  ...
parents 49d57592 deeacf35
......@@ -108,6 +108,7 @@ available subsections can be seen below.
vfio-mediated-device
vfio
vfio-pci-device-specific-driver-acceptance
virtio/index
xilinx/index
xillybus
zorro
......
.. SPDX-License-Identifier: GPL-2.0
======
Virtio
======
.. toctree::
:maxdepth: 1
virtio
writing_virtio_drivers
.. SPDX-License-Identifier: GPL-2.0
.. _virtio:
===============
Virtio on Linux
===============
Introduction
============
Virtio is an open standard that defines a protocol for communication
between drivers and devices of different types, see Chapter 5 ("Device
Types") of the virtio spec (`[1]`_). Originally developed as a standard
for paravirtualized devices implemented by a hypervisor, it can be used
to interface any compliant device (real or emulated) with a driver.
For illustrative purposes, this document will focus on the common case
of a Linux kernel running in a virtual machine and using paravirtualized
devices provided by the hypervisor, which exposes them as virtio devices
via standard mechanisms such as PCI.
Device - Driver communication: virtqueues
=========================================
Although the virtio devices are really an abstraction layer in the
hypervisor, they're exposed to the guest as if they are physical devices
using a specific transport method -- PCI, MMIO or CCW -- that is
orthogonal to the device itself. The virtio spec defines these transport
methods in detail, including device discovery, capabilities and
interrupt handling.
The communication between the driver in the guest OS and the device in
the hypervisor is done through shared memory (that's what makes virtio
devices so efficient) using specialized data structures called
virtqueues, which are actually ring buffers [#f1]_ of buffer descriptors
similar to the ones used in a network device:
.. kernel-doc:: include/uapi/linux/virtio_ring.h
:identifiers: struct vring_desc
All the buffers the descriptors point to are allocated by the guest and
used by the host either for reading or for writing but not for both.
Refer to Chapter 2.5 ("Virtqueues") of the virtio spec (`[1]`_) for the
reference definitions of virtqueues and "Virtqueues and virtio ring: How
the data travels" blog post (`[2]`_) for an illustrated overview of how
the host device and the guest driver communicate.
The :c:type:`vring_virtqueue` struct models a virtqueue, including the
ring buffers and management data. Embedded in this struct is the
:c:type:`virtqueue` struct, which is the data structure that's
ultimately used by virtio drivers:
.. kernel-doc:: include/linux/virtio.h
:identifiers: struct virtqueue
The callback function pointed by this struct is triggered when the
device has consumed the buffers provided by the driver. More
specifically, the trigger will be an interrupt issued by the hypervisor
(see vring_interrupt()). Interrupt request handlers are registered for
a virtqueue during the virtqueue setup process (transport-specific).
.. kernel-doc:: drivers/virtio/virtio_ring.c
:identifiers: vring_interrupt
Device discovery and probing
============================
In the kernel, the virtio core contains the virtio bus driver and
transport-specific drivers like `virtio-pci` and `virtio-mmio`. Then
there are individual virtio drivers for specific device types that are
registered to the virtio bus driver.
How a virtio device is found and configured by the kernel depends on how
the hypervisor defines it. Taking the `QEMU virtio-console
<https://gitlab.com/qemu-project/qemu/-/blob/master/hw/char/virtio-console.c>`__
device as an example. When using PCI as a transport method, the device
will present itself on the PCI bus with vendor 0x1af4 (Red Hat, Inc.)
and device id 0x1003 (virtio console), as defined in the spec, so the
kernel will detect it as it would do with any other PCI device.
During the PCI enumeration process, if a device is found to match the
virtio-pci driver (according to the virtio-pci device table, any PCI
device with vendor id = 0x1af4)::
/* Qumranet donated their vendor ID for devices 0x1000 thru 0x10FF. */
static const struct pci_device_id virtio_pci_id_table[] = {
{ PCI_DEVICE(PCI_VENDOR_ID_REDHAT_QUMRANET, PCI_ANY_ID) },
{ 0 }
};
then the virtio-pci driver is probed and, if the probing goes well, the
device is registered to the virtio bus::
static int virtio_pci_probe(struct pci_dev *pci_dev,
const struct pci_device_id *id)
{
...
if (force_legacy) {
rc = virtio_pci_legacy_probe(vp_dev);
/* Also try modern mode if we can't map BAR0 (no IO space). */
if (rc == -ENODEV || rc == -ENOMEM)
rc = virtio_pci_modern_probe(vp_dev);
if (rc)
goto err_probe;
} else {
rc = virtio_pci_modern_probe(vp_dev);
if (rc == -ENODEV)
rc = virtio_pci_legacy_probe(vp_dev);
if (rc)
goto err_probe;
}
...
rc = register_virtio_device(&vp_dev->vdev);
When the device is registered to the virtio bus the kernel will look
for a driver in the bus that can handle the device and call that
driver's ``probe`` method.
At this point, the virtqueues will be allocated and configured by
calling the appropriate ``virtio_find`` helper function, such as
virtio_find_single_vq() or virtio_find_vqs(), which will end up calling
a transport-specific ``find_vqs`` method.
References
==========
_`[1]` Virtio Spec v1.2:
https://docs.oasis-open.org/virtio/virtio/v1.2/virtio-v1.2.html
.. Check for later versions of the spec as well.
_`[2]` Virtqueues and virtio ring: How the data travels
https://www.redhat.com/en/blog/virtqueues-and-virtio-ring-how-data-travels
.. rubric:: Footnotes
.. [#f1] that's why they may be also referred to as virtrings.
.. SPDX-License-Identifier: GPL-2.0
.. _writing_virtio_drivers:
======================
Writing Virtio Drivers
======================
Introduction
============
This document serves as a basic guideline for driver programmers that
need to hack a new virtio driver or understand the essentials of the
existing ones. See :ref:`Virtio on Linux <virtio>` for a general
overview of virtio.
Driver boilerplate
==================
As a bare minimum, a virtio driver needs to register in the virtio bus
and configure the virtqueues for the device according to its spec, the
configuration of the virtqueues in the driver side must match the
virtqueue definitions in the device. A basic driver skeleton could look
like this::
#include <linux/virtio.h>
#include <linux/virtio_ids.h>
#include <linux/virtio_config.h>
#include <linux/module.h>
/* device private data (one per device) */
struct virtio_dummy_dev {
struct virtqueue *vq;
};
static void virtio_dummy_recv_cb(struct virtqueue *vq)
{
struct virtio_dummy_dev *dev = vq->vdev->priv;
char *buf;
unsigned int len;
while ((buf = virtqueue_get_buf(dev->vq, &len)) != NULL) {
/* process the received data */
}
}
static int virtio_dummy_probe(struct virtio_device *vdev)
{
struct virtio_dummy_dev *dev = NULL;
/* initialize device data */
dev = kzalloc(sizeof(struct virtio_dummy_dev), GFP_KERNEL);
if (!dev)
return -ENOMEM;
/* the device has a single virtqueue */
dev->vq = virtio_find_single_vq(vdev, virtio_dummy_recv_cb, "input");
if (IS_ERR(dev->vq)) {
kfree(dev);
return PTR_ERR(dev->vq);
}
vdev->priv = dev;
/* from this point on, the device can notify and get callbacks */
virtio_device_ready(vdev);
return 0;
}
static void virtio_dummy_remove(struct virtio_device *vdev)
{
struct virtio_dummy_dev *dev = vdev->priv;
/*
* disable vq interrupts: equivalent to
* vdev->config->reset(vdev)
*/
virtio_reset_device(vdev);
/* detach unused buffers */
while ((buf = virtqueue_detach_unused_buf(dev->vq)) != NULL) {
kfree(buf);
}
/* remove virtqueues */
vdev->config->del_vqs(vdev);
kfree(dev);
}
static const struct virtio_device_id id_table[] = {
{ VIRTIO_ID_DUMMY, VIRTIO_DEV_ANY_ID },
{ 0 },
};
static struct virtio_driver virtio_dummy_driver = {
.driver.name = KBUILD_MODNAME,
.driver.owner = THIS_MODULE,
.id_table = id_table,
.probe = virtio_dummy_probe,
.remove = virtio_dummy_remove,
};
module_virtio_driver(virtio_dummy_driver);
MODULE_DEVICE_TABLE(virtio, id_table);
MODULE_DESCRIPTION("Dummy virtio driver");
MODULE_LICENSE("GPL");
The device id ``VIRTIO_ID_DUMMY`` here is a placeholder, virtio drivers
should be added only for devices that are defined in the spec, see
include/uapi/linux/virtio_ids.h. Device ids need to be at least reserved
in the virtio spec before being added to that file.
If your driver doesn't have to do anything special in its ``init`` and
``exit`` methods, you can use the module_virtio_driver() helper to
reduce the amount of boilerplate code.
The ``probe`` method does the minimum driver setup in this case
(memory allocation for the device data) and initializes the
virtqueue. virtio_device_ready() is used to enable the virtqueue and to
notify the device that the driver is ready to manage the device
("DRIVER_OK"). The virtqueues are anyway enabled automatically by the
core after ``probe`` returns.
.. kernel-doc:: include/linux/virtio_config.h
:identifiers: virtio_device_ready
In any case, the virtqueues need to be enabled before adding buffers to
them.
Sending and receiving data
==========================
The virtio_dummy_recv_cb() callback in the code above will be triggered
when the device notifies the driver after it finishes processing a
descriptor or descriptor chain, either for reading or writing. However,
that's only the second half of the virtio device-driver communication
process, as the communication is always started by the driver regardless
of the direction of the data transfer.
To configure a buffer transfer from the driver to the device, first you
have to add the buffers -- packed as `scatterlists` -- to the
appropriate virtqueue using any of the virtqueue_add_inbuf(),
virtqueue_add_outbuf() or virtqueue_add_sgs(), depending on whether you
need to add one input `scatterlist` (for the device to fill in), one
output `scatterlist` (for the device to consume) or multiple
`scatterlists`, respectively. Then, once the virtqueue is set up, a call
to virtqueue_kick() sends a notification that will be serviced by the
hypervisor that implements the device::
struct scatterlist sg[1];
sg_init_one(sg, buffer, BUFLEN);
virtqueue_add_inbuf(dev->vq, sg, 1, buffer, GFP_ATOMIC);
virtqueue_kick(dev->vq);
.. kernel-doc:: drivers/virtio/virtio_ring.c
:identifiers: virtqueue_add_inbuf
.. kernel-doc:: drivers/virtio/virtio_ring.c
:identifiers: virtqueue_add_outbuf
.. kernel-doc:: drivers/virtio/virtio_ring.c
:identifiers: virtqueue_add_sgs
Then, after the device has read or written the buffers prepared by the
driver and notifies it back, the driver can call virtqueue_get_buf() to
read the data produced by the device (if the virtqueue was set up with
input buffers) or simply to reclaim the buffers if they were already
consumed by the device:
.. kernel-doc:: drivers/virtio/virtio_ring.c
:identifiers: virtqueue_get_buf_ctx
The virtqueue callbacks can be disabled and re-enabled using the
virtqueue_disable_cb() and the family of virtqueue_enable_cb() functions
respectively. See drivers/virtio/virtio_ring.c for more details:
.. kernel-doc:: drivers/virtio/virtio_ring.c
:identifiers: virtqueue_disable_cb
.. kernel-doc:: drivers/virtio/virtio_ring.c
:identifiers: virtqueue_enable_cb
But note that some spurious callbacks can still be triggered under
certain scenarios. The way to disable callbacks reliably is to reset the
device or the virtqueue (virtio_reset_device()).
References
==========
_`[1]` Virtio Spec v1.2:
https://docs.oasis-open.org/virtio/virtio/v1.2/virtio-v1.2.html
Check for later versions of the spec as well.
......@@ -22057,6 +22057,7 @@ S: Maintained
F: Documentation/ABI/testing/sysfs-bus-vdpa
F: Documentation/ABI/testing/sysfs-class-vduse
F: Documentation/devicetree/bindings/virtio/
F: Documentation/driver-api/virtio/
F: drivers/block/virtio_blk.c
F: drivers/crypto/virtio/
F: drivers/net/virtio_net.c
......@@ -22077,6 +22078,10 @@ IFCVF VIRTIO DATA PATH ACCELERATOR
R: Zhu Lingshan <lingshan.zhu@intel.com>
F: drivers/vdpa/ifcvf/
SNET DPU VIRTIO DATA PATH ACCELERATOR
R: Alvaro Karsz <alvaro.karsz@solid-run.com>
F: drivers/vdpa/solidrun/
VIRTIO BALLOON
M: "Michael S. Tsirkin" <mst@redhat.com>
M: David Hildenbrand <david@redhat.com>
......
This diff is collapsed.
......@@ -32,7 +32,6 @@ static int init_vq(struct virtio_pmem *vpmem)
static int virtio_pmem_probe(struct virtio_device *vdev)
{
struct nd_region_desc ndr_desc = {};
int nid = dev_to_node(&vdev->dev);
struct nd_region *nd_region;
struct virtio_pmem *vpmem;
struct resource res;
......@@ -79,7 +78,15 @@ static int virtio_pmem_probe(struct virtio_device *vdev)
dev_set_drvdata(&vdev->dev, vpmem->nvdimm_bus);
ndr_desc.res = &res;
ndr_desc.numa_node = nid;
ndr_desc.numa_node = memory_add_physaddr_to_nid(res.start);
ndr_desc.target_node = phys_to_target_node(res.start);
if (ndr_desc.target_node == NUMA_NO_NODE) {
ndr_desc.target_node = ndr_desc.numa_node;
dev_dbg(&vdev->dev, "changing target node from %d to %d",
NUMA_NO_NODE, ndr_desc.target_node);
}
ndr_desc.flush = async_pmem_flush;
ndr_desc.provider_data = vdev;
set_bit(ND_REGION_PAGEMAP, &ndr_desc.flags);
......
......@@ -5366,6 +5366,14 @@ DECLARE_PCI_FIXUP_EARLY(PCI_VENDOR_ID_AMD, 0x7901, quirk_no_flr);
DECLARE_PCI_FIXUP_EARLY(PCI_VENDOR_ID_INTEL, 0x1502, quirk_no_flr);
DECLARE_PCI_FIXUP_EARLY(PCI_VENDOR_ID_INTEL, 0x1503, quirk_no_flr);
/* FLR may cause the SolidRun SNET DPU (rev 0x1) to hang */
static void quirk_no_flr_snet(struct pci_dev *dev)
{
if (dev->revision == 0x1)
quirk_no_flr(dev);
}
DECLARE_PCI_FIXUP_EARLY(PCI_VENDOR_ID_SOLIDRUN, 0x1000, quirk_no_flr_snet);
static void quirk_no_ext_tags(struct pci_dev *pdev)
{
struct pci_host_bridge *bridge = pci_find_host_bridge(pdev->bus);
......
......@@ -330,7 +330,7 @@ static void virtscsi_handle_param_change(struct virtio_scsi *vscsi,
scsi_device_put(sdev);
}
static void virtscsi_rescan_hotunplug(struct virtio_scsi *vscsi)
static int virtscsi_rescan_hotunplug(struct virtio_scsi *vscsi)
{
struct scsi_device *sdev;
struct Scsi_Host *shost = virtio_scsi_host(vscsi->vdev);
......@@ -338,6 +338,11 @@ static void virtscsi_rescan_hotunplug(struct virtio_scsi *vscsi)
int result, inquiry_len, inq_result_len = 256;
char *inq_result = kmalloc(inq_result_len, GFP_KERNEL);
if (!inq_result) {
kfree(inq_result);
return -ENOMEM;
}
shost_for_each_device(sdev, shost) {
inquiry_len = sdev->inquiry_len ? sdev->inquiry_len : 36;
......@@ -366,6 +371,7 @@ static void virtscsi_rescan_hotunplug(struct virtio_scsi *vscsi)
}
kfree(inq_result);
return 0;
}
static void virtscsi_handle_event(struct work_struct *work)
......@@ -377,9 +383,13 @@ static void virtscsi_handle_event(struct work_struct *work)
if (event->event &
cpu_to_virtio32(vscsi->vdev, VIRTIO_SCSI_T_EVENTS_MISSED)) {
int ret;
event->event &= ~cpu_to_virtio32(vscsi->vdev,
VIRTIO_SCSI_T_EVENTS_MISSED);
virtscsi_rescan_hotunplug(vscsi);
ret = virtscsi_rescan_hotunplug(vscsi);
if (ret)
return;
scsi_scan_host(virtio_scsi_host(vscsi->vdev));
}
......
......@@ -71,6 +71,18 @@ config MLX5_VDPA_NET
be executed by the hardware. It also supports a variety of stateless
offloads depending on the actual device used and firmware version.
config MLX5_VDPA_STEERING_DEBUG
bool "expose steering counters on debugfs"
select MLX5_VDPA
help
Expose RX steering counters in debugfs to aid in debugging. For each VLAN
or non VLAN interface, two hardware counters are added to the RX flow
table: one for unicast and one for multicast.
The counters counts the number of packets and bytes and exposes them in
debugfs. Once can read the counters using, e.g.:
cat /sys/kernel/debug/mlx5/mlx5_core.sf.1/vdpa-0/rx/untagged/ucast/packets
cat /sys/kernel/debug/mlx5/mlx5_core.sf.1/vdpa-0/rx/untagged/mcast/bytes
config VP_VDPA
tristate "Virtio PCI bridge vDPA driver"
select VIRTIO_PCI_LIB
......@@ -86,4 +98,22 @@ config ALIBABA_ENI_VDPA
VDPA driver for Alibaba ENI (Elastic Network Interface) which is built upon
virtio 0.9.5 specification.
config SNET_VDPA
tristate "SolidRun's vDPA driver for SolidNET"
depends on PCI_MSI && PCI_IOV && (HWMON || HWMON=n)
# This driver MAY create a HWMON device.
# Depending on (HWMON || HWMON=n) ensures that:
# If HWMON=n the driver can be compiled either as a module or built-in.
# If HWMON=y the driver can be compiled either as a module or built-in.
# If HWMON=m the driver is forced to be compiled as a module.
# By doing so, IS_ENABLED can be used instead of IS_REACHABLE
help
vDPA driver for SolidNET DPU.
With this driver, the VirtIO dataplane can be
offloaded to a SolidNET DPU.
This driver includes a HW monitor device that
reads health values from the DPU.
endif # VDPA
......@@ -6,3 +6,4 @@ obj-$(CONFIG_IFCVF) += ifcvf/
obj-$(CONFIG_MLX5_VDPA) += mlx5/
obj-$(CONFIG_VP_VDPA) += virtio_pci/
obj-$(CONFIG_ALIBABA_ENI_VDPA) += alibaba/
obj-$(CONFIG_SNET_VDPA) += solidrun/
......@@ -10,11 +10,6 @@
#include "ifcvf_base.h"
struct ifcvf_adapter *vf_to_adapter(struct ifcvf_hw *hw)
{
return container_of(hw, struct ifcvf_adapter, vf);
}
u16 ifcvf_set_vq_vector(struct ifcvf_hw *hw, u16 qid, int vector)
{
struct virtio_pci_common_cfg __iomem *cfg = hw->common_cfg;
......@@ -37,8 +32,6 @@ u16 ifcvf_set_config_vector(struct ifcvf_hw *hw, int vector)
static void __iomem *get_cap_addr(struct ifcvf_hw *hw,
struct virtio_pci_cap *cap)
{
struct ifcvf_adapter *ifcvf;
struct pci_dev *pdev;
u32 length, offset;
u8 bar;
......@@ -46,17 +39,14 @@ static void __iomem *get_cap_addr(struct ifcvf_hw *hw,
offset = le32_to_cpu(cap->offset);
bar = cap->bar;
ifcvf= vf_to_adapter(hw);
pdev = ifcvf->pdev;
if (bar >= IFCVF_PCI_MAX_RESOURCE) {
IFCVF_DBG(pdev,
IFCVF_DBG(hw->pdev,
"Invalid bar number %u to get capabilities\n", bar);
return NULL;
}
if (offset + length > pci_resource_len(pdev, bar)) {
IFCVF_DBG(pdev,
if (offset + length > pci_resource_len(hw->pdev, bar)) {
IFCVF_DBG(hw->pdev,
"offset(%u) + len(%u) overflows bar%u's capability\n",
offset, length, bar);
return NULL;
......@@ -92,6 +82,7 @@ int ifcvf_init_hw(struct ifcvf_hw *hw, struct pci_dev *pdev)
IFCVF_ERR(pdev, "Failed to read PCI capability list\n");
return -EIO;
}
hw->pdev = pdev;
while (pos) {
ret = ifcvf_read_config_range(pdev, (u32 *)&cap,
......@@ -215,15 +206,13 @@ u64 ifcvf_get_hw_features(struct ifcvf_hw *hw)
u64 ifcvf_get_features(struct ifcvf_hw *hw)
{
return hw->hw_features;
return hw->dev_features;
}
int ifcvf_verify_min_features(struct ifcvf_hw *hw, u64 features)
{
struct ifcvf_adapter *ifcvf = vf_to_adapter(hw);
if (!(features & BIT_ULL(VIRTIO_F_ACCESS_PLATFORM)) && features) {
IFCVF_ERR(ifcvf->pdev, "VIRTIO_F_ACCESS_PLATFORM is not negotiated\n");
IFCVF_ERR(hw->pdev, "VIRTIO_F_ACCESS_PLATFORM is not negotiated\n");
return -EINVAL;
}
......@@ -232,13 +221,11 @@ int ifcvf_verify_min_features(struct ifcvf_hw *hw, u64 features)
u32 ifcvf_get_config_size(struct ifcvf_hw *hw)
{
struct ifcvf_adapter *adapter;
u32 net_config_size = sizeof(struct virtio_net_config);
u32 blk_config_size = sizeof(struct virtio_blk_config);
u32 cap_size = hw->cap_dev_config_size;
u32 config_size;
adapter = vf_to_adapter(hw);
/* If the onboard device config space size is greater than
* the size of struct virtio_net/blk_config, only the spec
* implementing contents size is returned, this is very
......@@ -253,7 +240,7 @@ u32 ifcvf_get_config_size(struct ifcvf_hw *hw)
break;
default:
config_size = 0;
IFCVF_ERR(adapter->pdev, "VIRTIO ID %u not supported\n", hw->dev_type);
IFCVF_ERR(hw->pdev, "VIRTIO ID %u not supported\n", hw->dev_type);
}
return config_size;
......@@ -301,14 +288,11 @@ static void ifcvf_set_features(struct ifcvf_hw *hw, u64 features)
static int ifcvf_config_features(struct ifcvf_hw *hw)
{
struct ifcvf_adapter *ifcvf;
ifcvf = vf_to_adapter(hw);
ifcvf_set_features(hw, hw->req_features);
ifcvf_add_status(hw, VIRTIO_CONFIG_S_FEATURES_OK);
if (!(ifcvf_get_status(hw) & VIRTIO_CONFIG_S_FEATURES_OK)) {
IFCVF_ERR(ifcvf->pdev, "Failed to set FEATURES_OK status\n");
IFCVF_ERR(hw->pdev, "Failed to set FEATURES_OK status\n");
return -EIO;
}
......
......@@ -19,6 +19,7 @@
#include <uapi/linux/virtio_blk.h>
#include <uapi/linux/virtio_config.h>
#include <uapi/linux/virtio_pci.h>
#include <uapi/linux/vdpa.h>
#define N3000_DEVICE_ID 0x1041
#define N3000_SUBSYS_DEVICE_ID 0x001A
......@@ -38,9 +39,6 @@
#define IFCVF_DBG(pdev, fmt, ...) dev_dbg(&pdev->dev, fmt, ##__VA_ARGS__)
#define IFCVF_INFO(pdev, fmt, ...) dev_info(&pdev->dev, fmt, ##__VA_ARGS__)
#define ifcvf_private_to_vf(adapter) \
(&((struct ifcvf_adapter *)adapter)->vf)
/* all vqs and config interrupt has its own vector */
#define MSIX_VECTOR_PER_VQ_AND_CONFIG 1
/* all vqs share a vector, and config interrupt has a separate vector */
......@@ -78,6 +76,8 @@ struct ifcvf_hw {
u32 dev_type;
u64 req_features;
u64 hw_features;
/* provisioned device features */
u64 dev_features;
struct virtio_pci_common_cfg __iomem *common_cfg;
void __iomem *dev_cfg;
struct vring_info vring[IFCVF_MAX_QUEUES];
......@@ -89,12 +89,13 @@ struct ifcvf_hw {
u16 nr_vring;
/* VIRTIO_PCI_CAP_DEVICE_CFG size */
u32 cap_dev_config_size;
struct pci_dev *pdev;
};
struct ifcvf_adapter {
struct vdpa_device vdpa;
struct pci_dev *pdev;
struct ifcvf_hw vf;
struct ifcvf_hw *vf;
};
struct ifcvf_vring_lm_cfg {
......@@ -109,6 +110,7 @@ struct ifcvf_lm_cfg {
struct ifcvf_vdpa_mgmt_dev {
struct vdpa_mgmt_dev mdev;
struct ifcvf_hw vf;
struct ifcvf_adapter *adapter;
struct pci_dev *pdev;
};
......
This diff is collapsed.
subdir-ccflags-y += -I$(srctree)/drivers/vdpa/mlx5/core
obj-$(CONFIG_MLX5_VDPA_NET) += mlx5_vdpa.o
mlx5_vdpa-$(CONFIG_MLX5_VDPA_NET) += net/mlx5_vnet.o core/resources.o core/mr.o
mlx5_vdpa-$(CONFIG_MLX5_VDPA_NET) += net/mlx5_vnet.o core/resources.o core/mr.o net/debug.o
......@@ -503,7 +503,6 @@ void mlx5_vdpa_destroy_mr(struct mlx5_vdpa_dev *mvdev)
else
destroy_dma_mr(mvdev, mr);
memset(mr, 0, sizeof(*mr));
mr->initialized = false;
out:
mutex_unlock(&mr->mkey_mtx);
......
......@@ -213,7 +213,7 @@ int mlx5_vdpa_create_mkey(struct mlx5_vdpa_dev *mvdev, u32 *mkey, u32 *in,
return err;
mkey_index = MLX5_GET(create_mkey_out, lout, mkey_index);
*mkey |= mlx5_idx_to_mkey(mkey_index);
*mkey = mlx5_idx_to_mkey(mkey_index);
return 0;
}
......@@ -233,6 +233,7 @@ static int init_ctrl_vq(struct mlx5_vdpa_dev *mvdev)
if (!mvdev->cvq.iotlb)
return -ENOMEM;
spin_lock_init(&mvdev->cvq.iommu_lock);
vringh_set_iotlb(&mvdev->cvq.vring, mvdev->cvq.iotlb, &mvdev->cvq.iommu_lock);
return 0;
......
// SPDX-License-Identifier: GPL-2.0 OR Linux-OpenIB
/* Copyright (c) 2022, NVIDIA CORPORATION & AFFILIATES. All rights reserved. */
#include <linux/debugfs.h>
#include <linux/mlx5/fs.h>
#include "mlx5_vnet.h"
static int tirn_show(struct seq_file *file, void *priv)
{
struct mlx5_vdpa_net *ndev = file->private;
seq_printf(file, "0x%x\n", ndev->res.tirn);
return 0;
}
DEFINE_SHOW_ATTRIBUTE(tirn);
void mlx5_vdpa_remove_tirn(struct mlx5_vdpa_net *ndev)
{
if (ndev->debugfs)
debugfs_remove(ndev->res.tirn_dent);
}
void mlx5_vdpa_add_tirn(struct mlx5_vdpa_net *ndev)
{
ndev->res.tirn_dent = debugfs_create_file("tirn", 0444, ndev->rx_dent,
ndev, &tirn_fops);
}
static int rx_flow_table_show(struct seq_file *file, void *priv)
{
struct mlx5_vdpa_net *ndev = file->private;
seq_printf(file, "0x%x\n", mlx5_flow_table_id(ndev->rxft));
return 0;
}
DEFINE_SHOW_ATTRIBUTE(rx_flow_table);
void mlx5_vdpa_remove_rx_flow_table(struct mlx5_vdpa_net *ndev)
{
if (ndev->debugfs)
debugfs_remove(ndev->rx_table_dent);
}
void mlx5_vdpa_add_rx_flow_table(struct mlx5_vdpa_net *ndev)
{
ndev->rx_table_dent = debugfs_create_file("table_id", 0444, ndev->rx_dent,
ndev, &rx_flow_table_fops);
}
#if defined(CONFIG_MLX5_VDPA_STEERING_DEBUG)
static int packets_show(struct seq_file *file, void *priv)
{
struct mlx5_vdpa_counter *counter = file->private;
u64 packets;
u64 bytes;
int err;
err = mlx5_fc_query(counter->mdev, counter->counter, &packets, &bytes);
if (err)
return err;
seq_printf(file, "0x%llx\n", packets);
return 0;
}
static int bytes_show(struct seq_file *file, void *priv)
{
struct mlx5_vdpa_counter *counter = file->private;
u64 packets;
u64 bytes;
int err;
err = mlx5_fc_query(counter->mdev, counter->counter, &packets, &bytes);
if (err)
return err;
seq_printf(file, "0x%llx\n", bytes);
return 0;
}
DEFINE_SHOW_ATTRIBUTE(packets);
DEFINE_SHOW_ATTRIBUTE(bytes);
static void add_counter_node(struct mlx5_vdpa_counter *counter,
struct dentry *parent)
{
debugfs_create_file("packets", 0444, parent, counter,
&packets_fops);
debugfs_create_file("bytes", 0444, parent, counter,
&bytes_fops);
}
void mlx5_vdpa_add_rx_counters(struct mlx5_vdpa_net *ndev,
struct macvlan_node *node)
{
static const char *ut = "untagged";
char vidstr[9];
u16 vid;
node->ucast_counter.mdev = ndev->mvdev.mdev;
node->mcast_counter.mdev = ndev->mvdev.mdev;
if (node->tagged) {
vid = key2vid(node->macvlan);
snprintf(vidstr, sizeof(vidstr), "0x%x", vid);
} else {
strcpy(vidstr, ut);
}
node->dent = debugfs_create_dir(vidstr, ndev->rx_dent);
if (IS_ERR(node->dent)) {
node->dent = NULL;
return;
}
node->ucast_counter.dent = debugfs_create_dir("ucast", node->dent);
if (IS_ERR(node->ucast_counter.dent))
return;
add_counter_node(&node->ucast_counter, node->ucast_counter.dent);
node->mcast_counter.dent = debugfs_create_dir("mcast", node->dent);
if (IS_ERR(node->mcast_counter.dent))
return;
add_counter_node(&node->mcast_counter, node->mcast_counter.dent);
}
void mlx5_vdpa_remove_rx_counters(struct mlx5_vdpa_net *ndev,
struct macvlan_node *node)
{
if (node->dent && ndev->debugfs)
debugfs_remove_recursive(node->dent);
}
#endif
void mlx5_vdpa_add_debugfs(struct mlx5_vdpa_net *ndev)
{
struct mlx5_core_dev *mdev;
mdev = ndev->mvdev.mdev;
ndev->debugfs = debugfs_create_dir(dev_name(&ndev->mvdev.vdev.dev),
mlx5_debugfs_get_dev_root(mdev));
if (!IS_ERR(ndev->debugfs))
ndev->rx_dent = debugfs_create_dir("rx", ndev->debugfs);
}
void mlx5_vdpa_remove_debugfs(struct dentry *dbg)
{
debugfs_remove_recursive(dbg);
}
This diff is collapsed.
/* SPDX-License-Identifier: GPL-2.0 OR Linux-OpenIB */
/* Copyright (c) 2022, NVIDIA CORPORATION & AFFILIATES. All rights reserved. */
#ifndef __MLX5_VNET_H__
#define __MLX5_VNET_H__
#include "mlx5_vdpa.h"
#define to_mlx5_vdpa_ndev(__mvdev) \
container_of(__mvdev, struct mlx5_vdpa_net, mvdev)
#define to_mvdev(__vdev) container_of((__vdev), struct mlx5_vdpa_dev, vdev)
struct mlx5_vdpa_net_resources {
u32 tisn;
u32 tdn;
u32 tirn;
u32 rqtn;
bool valid;
struct dentry *tirn_dent;
};
#define MLX5V_MACVLAN_SIZE 256
static inline u16 key2vid(u64 key)
{
return (u16)(key >> 48) & 0xfff;
}
struct mlx5_vdpa_net {
struct mlx5_vdpa_dev mvdev;
struct mlx5_vdpa_net_resources res;
struct virtio_net_config config;
struct mlx5_vdpa_virtqueue *vqs;
struct vdpa_callback *event_cbs;
/* Serialize vq resources creation and destruction. This is required
* since memory map might change and we need to destroy and create
* resources while driver in operational.
*/
struct rw_semaphore reslock;
struct mlx5_flow_table *rxft;
struct dentry *rx_dent;
struct dentry *rx_table_dent;
bool setup;
u32 cur_num_vqs;
u32 rqt_size;
bool nb_registered;
struct notifier_block nb;
struct vdpa_callback config_cb;
struct mlx5_vdpa_wq_ent cvq_ent;
struct hlist_head macvlan_hash[MLX5V_MACVLAN_SIZE];
struct dentry *debugfs;
};
struct mlx5_vdpa_counter {
struct mlx5_fc *counter;
struct dentry *dent;
struct mlx5_core_dev *mdev;
};
struct macvlan_node {
struct hlist_node hlist;
struct mlx5_flow_handle *ucast_rule;
struct mlx5_flow_handle *mcast_rule;
u64 macvlan;
struct mlx5_vdpa_net *ndev;
bool tagged;
#if defined(CONFIG_MLX5_VDPA_STEERING_DEBUG)
struct dentry *dent;
struct mlx5_vdpa_counter ucast_counter;
struct mlx5_vdpa_counter mcast_counter;
#endif
};
void mlx5_vdpa_add_debugfs(struct mlx5_vdpa_net *ndev);
void mlx5_vdpa_remove_debugfs(struct dentry *dbg);
void mlx5_vdpa_add_rx_flow_table(struct mlx5_vdpa_net *ndev);
void mlx5_vdpa_remove_rx_flow_table(struct mlx5_vdpa_net *ndev);
void mlx5_vdpa_add_tirn(struct mlx5_vdpa_net *ndev);
void mlx5_vdpa_remove_tirn(struct mlx5_vdpa_net *ndev);
#if defined(CONFIG_MLX5_VDPA_STEERING_DEBUG)
void mlx5_vdpa_add_rx_counters(struct mlx5_vdpa_net *ndev,
struct macvlan_node *node);
void mlx5_vdpa_remove_rx_counters(struct mlx5_vdpa_net *ndev,
struct macvlan_node *node);
#else
static inline void mlx5_vdpa_add_rx_counters(struct mlx5_vdpa_net *ndev,
struct macvlan_node *node) {}
static inline void mlx5_vdpa_remove_rx_counters(struct mlx5_vdpa_net *ndev,
struct macvlan_node *node) {}
#endif
#endif /* __MLX5_VNET_H__ */
# SPDX-License-Identifier: GPL-2.0
obj-$(CONFIG_SNET_VDPA) += snet_vdpa.o
snet_vdpa-$(CONFIG_SNET_VDPA) += snet_main.o
ifdef CONFIG_HWMON
snet_vdpa-$(CONFIG_SNET_VDPA) += snet_hwmon.o
endif
// SPDX-License-Identifier: GPL-2.0-only
/*
* SolidRun DPU driver for control plane
*
* Copyright (C) 2022 SolidRun
*
* Author: Alvaro Karsz <alvaro.karsz@solid-run.com>
*
*/
#include <linux/hwmon.h>
#include "snet_vdpa.h"
/* Monitor offsets */
#define SNET_MON_TMP0_IN_OFF 0x00
#define SNET_MON_TMP0_MAX_OFF 0x08
#define SNET_MON_TMP0_CRIT_OFF 0x10
#define SNET_MON_TMP1_IN_OFF 0x18
#define SNET_MON_TMP1_CRIT_OFF 0x20
#define SNET_MON_CURR_IN_OFF 0x28
#define SNET_MON_CURR_MAX_OFF 0x30
#define SNET_MON_CURR_CRIT_OFF 0x38
#define SNET_MON_PWR_IN_OFF 0x40
#define SNET_MON_VOLT_IN_OFF 0x48
#define SNET_MON_VOLT_CRIT_OFF 0x50
#define SNET_MON_VOLT_LCRIT_OFF 0x58
static void snet_hwmon_read_reg(struct psnet *psnet, u32 reg, long *out)
{
*out = psnet_read64(psnet, psnet->cfg.hwmon_off + reg);
}
static umode_t snet_howmon_is_visible(const void *data,
enum hwmon_sensor_types type,
u32 attr, int channel)
{
return 0444;
}
static int snet_howmon_read(struct device *dev, enum hwmon_sensor_types type,
u32 attr, int channel, long *val)
{
struct psnet *psnet = dev_get_drvdata(dev);
int ret = 0;
switch (type) {
case hwmon_in:
switch (attr) {
case hwmon_in_lcrit:
snet_hwmon_read_reg(psnet, SNET_MON_VOLT_LCRIT_OFF, val);
break;
case hwmon_in_crit:
snet_hwmon_read_reg(psnet, SNET_MON_VOLT_CRIT_OFF, val);
break;
case hwmon_in_input:
snet_hwmon_read_reg(psnet, SNET_MON_VOLT_IN_OFF, val);
break;
default:
ret = -EOPNOTSUPP;
break;
}
break;
case hwmon_power:
switch (attr) {
case hwmon_power_input:
snet_hwmon_read_reg(psnet, SNET_MON_PWR_IN_OFF, val);
break;
default:
ret = -EOPNOTSUPP;
break;
}
break;
case hwmon_curr:
switch (attr) {
case hwmon_curr_input:
snet_hwmon_read_reg(psnet, SNET_MON_CURR_IN_OFF, val);
break;
case hwmon_curr_max:
snet_hwmon_read_reg(psnet, SNET_MON_CURR_MAX_OFF, val);
break;
case hwmon_curr_crit:
snet_hwmon_read_reg(psnet, SNET_MON_CURR_CRIT_OFF, val);
break;
default:
ret = -EOPNOTSUPP;
break;
}
break;
case hwmon_temp:
switch (attr) {
case hwmon_temp_input:
if (channel == 0)
snet_hwmon_read_reg(psnet, SNET_MON_TMP0_IN_OFF, val);
else
snet_hwmon_read_reg(psnet, SNET_MON_TMP1_IN_OFF, val);
break;
case hwmon_temp_max:
if (channel == 0)
snet_hwmon_read_reg(psnet, SNET_MON_TMP0_MAX_OFF, val);
else
ret = -EOPNOTSUPP;
break;
case hwmon_temp_crit:
if (channel == 0)
snet_hwmon_read_reg(psnet, SNET_MON_TMP0_CRIT_OFF, val);
else
snet_hwmon_read_reg(psnet, SNET_MON_TMP1_CRIT_OFF, val);
break;
default:
ret = -EOPNOTSUPP;
break;
}
break;
default:
ret = -EOPNOTSUPP;
break;
}
return ret;
}
static int snet_hwmon_read_string(struct device *dev,
enum hwmon_sensor_types type, u32 attr,
int channel, const char **str)
{
int ret = 0;
switch (type) {
case hwmon_in:
*str = "main_vin";
break;
case hwmon_power:
*str = "soc_pin";
break;
case hwmon_curr:
*str = "soc_iin";
break;
case hwmon_temp:
if (channel == 0)
*str = "power_stage_temp";
else
*str = "ic_junction_temp";
break;
default:
ret = -EOPNOTSUPP;
break;
}
return ret;
}
static const struct hwmon_ops snet_hwmon_ops = {
.is_visible = snet_howmon_is_visible,
.read = snet_howmon_read,
.read_string = snet_hwmon_read_string
};
static const struct hwmon_channel_info *snet_hwmon_info[] = {
HWMON_CHANNEL_INFO(temp, HWMON_T_INPUT | HWMON_T_MAX | HWMON_T_CRIT | HWMON_T_LABEL,
HWMON_T_INPUT | HWMON_T_CRIT | HWMON_T_LABEL),
HWMON_CHANNEL_INFO(power, HWMON_P_INPUT | HWMON_P_LABEL),
HWMON_CHANNEL_INFO(curr, HWMON_C_INPUT | HWMON_C_MAX | HWMON_C_CRIT | HWMON_C_LABEL),
HWMON_CHANNEL_INFO(in, HWMON_I_INPUT | HWMON_I_CRIT | HWMON_I_LCRIT | HWMON_I_LABEL),
NULL
};
static const struct hwmon_chip_info snet_hwmono_info = {
.ops = &snet_hwmon_ops,
.info = snet_hwmon_info,
};
/* Create an HW monitor device */
void psnet_create_hwmon(struct pci_dev *pdev)
{
struct device *hwmon;
struct psnet *psnet = pci_get_drvdata(pdev);
snprintf(psnet->hwmon_name, SNET_NAME_SIZE, "snet_%s", pci_name(pdev));
hwmon = devm_hwmon_device_register_with_info(&pdev->dev, psnet->hwmon_name, psnet,
&snet_hwmono_info, NULL);
/* The monitor is not mandatory, Just alert user in case of an error */
if (IS_ERR(hwmon))
SNET_WARN(pdev, "Failed to create SNET hwmon, error %ld\n", PTR_ERR(hwmon));
}
This diff is collapsed.
/* SPDX-License-Identifier: GPL-2.0-only */
/*
* SolidRun DPU driver for control plane
*
* Copyright (C) 2022 SolidRun
*
* Author: Alvaro Karsz <alvaro.karsz@solid-run.com>
*
*/
#ifndef _SNET_VDPA_H_
#define _SNET_VDPA_H_
#include <linux/vdpa.h>
#include <linux/pci.h>
#define SNET_NAME_SIZE 256
#define SNET_ERR(pdev, fmt, ...) dev_err(&(pdev)->dev, "%s"fmt, "snet_vdpa: ", ##__VA_ARGS__)
#define SNET_WARN(pdev, fmt, ...) dev_warn(&(pdev)->dev, "%s"fmt, "snet_vdpa: ", ##__VA_ARGS__)
#define SNET_INFO(pdev, fmt, ...) dev_info(&(pdev)->dev, "%s"fmt, "snet_vdpa: ", ##__VA_ARGS__)
#define SNET_DBG(pdev, fmt, ...) dev_dbg(&(pdev)->dev, "%s"fmt, "snet_vdpa: ", ##__VA_ARGS__)
#define SNET_HAS_FEATURE(s, f) ((s)->negotiated_features & BIT_ULL(f))
/* VQ struct */
struct snet_vq {
/* VQ callback */
struct vdpa_callback cb;
/* desc base address */
u64 desc_area;
/* device base address */
u64 device_area;
/* driver base address */
u64 driver_area;
/* Queue size */
u32 num;
/* Serial ID for VQ */
u32 sid;
/* is ready flag */
bool ready;
/* IRQ number */
u32 irq;
/* IRQ index, DPU uses this to parse data from MSI-X table */
u32 irq_idx;
/* IRQ name */
char irq_name[SNET_NAME_SIZE];
/* pointer to mapped PCI BAR register used by this VQ to kick */
void __iomem *kick_ptr;
};
struct snet {
/* vdpa device */
struct vdpa_device vdpa;
/* Config callback */
struct vdpa_callback cb;
/* array of virqueues */
struct snet_vq **vqs;
/* Used features */
u64 negotiated_features;
/* Device serial ID */
u32 sid;
/* device status */
u8 status;
/* boolean indicating if snet config was passed to the device */
bool dpu_ready;
/* IRQ number */
u32 cfg_irq;
/* IRQ index, DPU uses this to parse data from MSI-X table */
u32 cfg_irq_idx;
/* IRQ name */
char cfg_irq_name[SNET_NAME_SIZE];
/* BAR to access the VF */
void __iomem *bar;
/* PCI device */
struct pci_dev *pdev;
/* Pointer to snet pdev parent device */
struct psnet *psnet;
/* Pointer to snet config device */
struct snet_dev_cfg *cfg;
};
struct snet_dev_cfg {
/* Device ID following VirtIO spec. */
u32 virtio_id;
/* Number of VQs for this device */
u32 vq_num;
/* Size of every VQ */
u32 vq_size;
/* Virtual Function id */
u32 vfid;
/* Device features, following VirtIO spec */
u64 features;
/* Reserved for future usage */
u32 rsvd[6];
/* VirtIO device specific config size */
u32 cfg_size;
/* VirtIO device specific config address */
void __iomem *virtio_cfg;
} __packed;
struct snet_cfg {
/* Magic key */
u32 key;
/* Size of total config in bytes */
u32 cfg_size;
/* Config version */
u32 cfg_ver;
/* Number of Virtual Functions to create */
u32 vf_num;
/* BAR to use for the VFs */
u32 vf_bar;
/* Where should we write the SNET's config */
u32 host_cfg_off;
/* Max. allowed size for a SNET's config */
u32 max_size_host_cfg;
/* VirtIO config offset in BAR */
u32 virtio_cfg_off;
/* Offset in PCI BAR for VQ kicks */
u32 kick_off;
/* Offset in PCI BAR for HW monitoring */
u32 hwmon_off;
/* Offset in PCI BAR for SNET messages */
u32 msg_off;
/* Config general flags - enum snet_cfg_flags */
u32 flags;
/* Reserved for future usage */
u32 rsvd[6];
/* Number of snet devices */
u32 devices_num;
/* The actual devices */
struct snet_dev_cfg **devs;
} __packed;
/* SolidNET PCIe device, one device per PCIe physical function */
struct psnet {
/* PCI BARs */
void __iomem *bars[PCI_STD_NUM_BARS];
/* Negotiated config version */
u32 negotiated_cfg_ver;
/* Next IRQ index to use in case when the IRQs are allocated from this device */
u32 next_irq;
/* BAR number used to communicate with the device */
u8 barno;
/* spinlock to protect data that can be changed by SNET devices */
spinlock_t lock;
/* Pointer to the device's config read from BAR */
struct snet_cfg cfg;
/* Name of monitor device */
char hwmon_name[SNET_NAME_SIZE];
};
enum snet_cfg_flags {
/* Create a HWMON device */
SNET_CFG_FLAG_HWMON = BIT(0),
/* USE IRQs from the physical function */
SNET_CFG_FLAG_IRQ_PF = BIT(1),
};
#define PSNET_FLAG_ON(p, f) ((p)->cfg.flags & (f))
static inline u32 psnet_read32(struct psnet *psnet, u32 off)
{
return ioread32(psnet->bars[psnet->barno] + off);
}
static inline u32 snet_read32(struct snet *snet, u32 off)
{
return ioread32(snet->bar + off);
}
static inline void snet_write32(struct snet *snet, u32 off, u32 val)
{
iowrite32(val, snet->bar + off);
}
static inline u64 psnet_read64(struct psnet *psnet, u32 off)
{
u64 val;
/* 64bits are written in 2 halves, low part first */
val = (u64)psnet_read32(psnet, off);
val |= ((u64)psnet_read32(psnet, off + 4) << 32);
return val;
}
static inline void snet_write64(struct snet *snet, u32 off, u64 val)
{
/* The DPU expects a 64bit integer in 2 halves, the low part first */
snet_write32(snet, off, (u32)val);
snet_write32(snet, off + 4, (u32)(val >> 32));
}
#if IS_ENABLED(CONFIG_HWMON)
void psnet_create_hwmon(struct pci_dev *pdev);
#endif
#endif //_SNET_VDPA_H_
......@@ -39,6 +39,11 @@ static int vdpa_dev_probe(struct device *d)
u32 max_num, min_num = 1;
int ret = 0;
d->dma_mask = &d->coherent_dma_mask;
ret = dma_set_mask_and_coherent(d, DMA_BIT_MASK(64));
if (ret)
return ret;
max_num = ops->get_vq_num_max(vdev);
if (ops->get_vq_num_min)
min_num = ops->get_vq_num_min(vdev);
......@@ -460,12 +465,28 @@ static int vdpa_nl_mgmtdev_handle_fill(struct sk_buff *msg, const struct vdpa_mg
return 0;
}
static u64 vdpa_mgmtdev_get_classes(const struct vdpa_mgmt_dev *mdev,
unsigned int *nclasses)
{
u64 supported_classes = 0;
unsigned int n = 0;
for (int i = 0; mdev->id_table[i].device; i++) {
if (mdev->id_table[i].device > 63)
continue;
supported_classes |= BIT_ULL(mdev->id_table[i].device);
n++;
}
if (nclasses)
*nclasses = n;
return supported_classes;
}
static int vdpa_mgmtdev_fill(const struct vdpa_mgmt_dev *mdev, struct sk_buff *msg,
u32 portid, u32 seq, int flags)
{
u64 supported_classes = 0;
void *hdr;
int i = 0;
int err;
hdr = genlmsg_put(msg, portid, seq, &vdpa_nl_family, flags, VDPA_CMD_MGMTDEV_NEW);
......@@ -475,14 +496,9 @@ static int vdpa_mgmtdev_fill(const struct vdpa_mgmt_dev *mdev, struct sk_buff *m
if (err)
goto msg_err;
while (mdev->id_table[i].device) {
if (mdev->id_table[i].device <= 63)
supported_classes |= BIT_ULL(mdev->id_table[i].device);
i++;
}
if (nla_put_u64_64bit(msg, VDPA_ATTR_MGMTDEV_SUPPORTED_CLASSES,
supported_classes, VDPA_ATTR_UNSPEC)) {
vdpa_mgmtdev_get_classes(mdev, NULL),
VDPA_ATTR_UNSPEC)) {
err = -EMSGSIZE;
goto msg_err;
}
......@@ -566,13 +582,25 @@ vdpa_nl_cmd_mgmtdev_get_dumpit(struct sk_buff *msg, struct netlink_callback *cb)
BIT_ULL(VDPA_ATTR_DEV_NET_CFG_MTU) | \
BIT_ULL(VDPA_ATTR_DEV_NET_CFG_MAX_VQP))
/*
* Bitmask for all per-device features: feature bits VIRTIO_TRANSPORT_F_START
* through VIRTIO_TRANSPORT_F_END are unset, i.e. 0xfffffc000fffffff for
* all 64bit features. If the features are extended beyond 64 bits, or new
* "holes" are reserved for other type of features than per-device, this
* macro would have to be updated.
*/
#define VIRTIO_DEVICE_F_MASK (~0ULL << (VIRTIO_TRANSPORT_F_END + 1) | \
((1ULL << VIRTIO_TRANSPORT_F_START) - 1))
static int vdpa_nl_cmd_dev_add_set_doit(struct sk_buff *skb, struct genl_info *info)
{
struct vdpa_dev_set_config config = {};
struct nlattr **nl_attrs = info->attrs;
struct vdpa_mgmt_dev *mdev;
unsigned int ncls = 0;
const u8 *macaddr;
const char *name;
u64 classes;
int err = 0;
if (!info->attrs[VDPA_ATTR_DEV_NAME])
......@@ -601,8 +629,26 @@ static int vdpa_nl_cmd_dev_add_set_doit(struct sk_buff *skb, struct genl_info *i
config.mask |= BIT_ULL(VDPA_ATTR_DEV_NET_CFG_MAX_VQP);
}
if (nl_attrs[VDPA_ATTR_DEV_FEATURES]) {
u64 missing = 0x0ULL;
config.device_features =
nla_get_u64(nl_attrs[VDPA_ATTR_DEV_FEATURES]);
if (nl_attrs[VDPA_ATTR_DEV_NET_CFG_MACADDR] &&
!(config.device_features & BIT_ULL(VIRTIO_NET_F_MAC)))
missing |= BIT_ULL(VIRTIO_NET_F_MAC);
if (nl_attrs[VDPA_ATTR_DEV_NET_CFG_MTU] &&
!(config.device_features & BIT_ULL(VIRTIO_NET_F_MTU)))
missing |= BIT_ULL(VIRTIO_NET_F_MTU);
if (nl_attrs[VDPA_ATTR_DEV_NET_CFG_MAX_VQP] &&
config.net.max_vq_pairs > 1 &&
!(config.device_features & BIT_ULL(VIRTIO_NET_F_MQ)))
missing |= BIT_ULL(VIRTIO_NET_F_MQ);
if (missing) {
NL_SET_ERR_MSG_FMT_MOD(info->extack,
"Missing features 0x%llx for provided attributes",
missing);
return -EINVAL;
}
config.mask |= BIT_ULL(VDPA_ATTR_DEV_FEATURES);
}
......@@ -622,13 +668,33 @@ static int vdpa_nl_cmd_dev_add_set_doit(struct sk_buff *skb, struct genl_info *i
err = PTR_ERR(mdev);
goto err;
}
if ((config.mask & mdev->config_attr_mask) != config.mask) {
NL_SET_ERR_MSG_MOD(info->extack,
"All provided attributes are not supported");
NL_SET_ERR_MSG_FMT_MOD(info->extack,
"Some provided attributes are not supported: 0x%llx",
config.mask & ~mdev->config_attr_mask);
err = -EOPNOTSUPP;
goto err;
}
classes = vdpa_mgmtdev_get_classes(mdev, &ncls);
if (config.mask & VDPA_DEV_NET_ATTRS_MASK &&
!(classes & BIT_ULL(VIRTIO_ID_NET))) {
NL_SET_ERR_MSG_MOD(info->extack,
"Network class attributes provided on unsupported management device");
err = -EINVAL;
goto err;
}
if (!(config.mask & VDPA_DEV_NET_ATTRS_MASK) &&
config.mask & BIT_ULL(VDPA_ATTR_DEV_FEATURES) &&
classes & BIT_ULL(VIRTIO_ID_NET) && ncls > 1 &&
config.device_features & VIRTIO_DEVICE_F_MASK) {
NL_SET_ERR_MSG_MOD(info->extack,
"Management device supports multi-class while device features specified are ambiguous");
err = -EINVAL;
goto err;
}
err = mdev->ops->dev_add(mdev, name, &config);
err:
up_write(&vdpa_dev_lock);
......@@ -841,18 +907,25 @@ static int vdpa_dev_net_mac_config_fill(struct sk_buff *msg, u64 features,
sizeof(config->mac), config->mac);
}
static int vdpa_dev_net_status_config_fill(struct sk_buff *msg, u64 features,
const struct virtio_net_config *config)
{
u16 val_u16;
if ((features & BIT_ULL(VIRTIO_NET_F_STATUS)) == 0)
return 0;
val_u16 = __virtio16_to_cpu(true, config->status);
return nla_put_u16(msg, VDPA_ATTR_DEV_NET_STATUS, val_u16);
}
static int vdpa_dev_net_config_fill(struct vdpa_device *vdev, struct sk_buff *msg)
{
struct virtio_net_config config = {};
u64 features_device;
u16 val_u16;
vdev->config->get_config(vdev, 0, &config, sizeof(config));
val_u16 = __virtio16_to_cpu(true, config.status);
if (nla_put_u16(msg, VDPA_ATTR_DEV_NET_STATUS, val_u16))
return -EMSGSIZE;
features_device = vdev->config->get_device_features(vdev);
if (nla_put_u64_64bit(msg, VDPA_ATTR_DEV_FEATURES, features_device,
......@@ -865,6 +938,9 @@ static int vdpa_dev_net_config_fill(struct vdpa_device *vdev, struct sk_buff *ms
if (vdpa_dev_net_mac_config_fill(msg, features_device, &config))
return -EMSGSIZE;
if (vdpa_dev_net_status_config_fill(msg, features_device, &config))
return -EMSGSIZE;
return vdpa_dev_net_mq_config_fill(msg, features_device, &config);
}
......@@ -1011,7 +1087,7 @@ static int vdpa_dev_vendor_stats_fill(struct vdpa_device *vdev,
switch (device_id) {
case VIRTIO_ID_NET:
if (index > VIRTIO_NET_CTRL_MQ_VQ_PAIRS_MAX) {
NL_SET_ERR_MSG_MOD(info->extack, "queue index excceeds max value");
NL_SET_ERR_MSG_MOD(info->extack, "queue index exceeds max value");
err = -ERANGE;
break;
}
......
This diff is collapsed.
......@@ -37,6 +37,7 @@ struct vdpasim_dev_attr {
struct vdpa_mgmt_dev *mgmt_dev;
const char *name;
u64 supported_features;
size_t alloc_size;
size_t config_size;
size_t buffer_size;
int nvqs;
......@@ -47,6 +48,9 @@ struct vdpasim_dev_attr {
work_func_t work_fn;
void (*get_config)(struct vdpasim *vdpasim, void *config);
void (*set_config)(struct vdpasim *vdpasim, const void *config);
int (*get_stats)(struct vdpasim *vdpasim, u16 idx,
struct sk_buff *msg,
struct netlink_ext_ack *extack);
};
/* State of each vdpasim device */
......@@ -60,13 +64,14 @@ struct vdpasim {
/* virtio config according to device type */
void *config;
struct vhost_iotlb *iommu;
struct iova_domain iova;
bool *iommu_pt;
void *buffer;
u32 status;
u32 generation;
u64 features;
u32 groups;
bool running;
bool pending_kick;
/* spinlock to synchronize iommu table */
spinlock_t iommu_lock;
};
......
......@@ -378,6 +378,7 @@ static int vdpasim_blk_dev_add(struct vdpa_mgmt_dev *mdev, const char *name,
dev_attr.nvqs = VDPASIM_BLK_VQ_NUM;
dev_attr.ngroups = VDPASIM_BLK_GROUP_NUM;
dev_attr.nas = VDPASIM_BLK_AS_NUM;
dev_attr.alloc_size = sizeof(struct vdpasim);
dev_attr.config_size = sizeof(struct virtio_blk_config);
dev_attr.get_config = vdpasim_blk_get_config;
dev_attr.work_fn = vdpasim_blk_work;
......
......@@ -15,6 +15,7 @@
#include <linux/etherdevice.h>
#include <linux/vringh.h>
#include <linux/vdpa.h>
#include <net/netlink.h>
#include <uapi/linux/virtio_net.h>
#include <uapi/linux/vdpa.h>
......@@ -27,6 +28,7 @@
#define VDPASIM_NET_FEATURES (VDPASIM_FEATURES | \
(1ULL << VIRTIO_NET_F_MAC) | \
(1ULL << VIRTIO_NET_F_STATUS) | \
(1ULL << VIRTIO_NET_F_MTU) | \
(1ULL << VIRTIO_NET_F_CTRL_VQ) | \
(1ULL << VIRTIO_NET_F_CTRL_MAC_ADDR))
......@@ -36,6 +38,34 @@
#define VDPASIM_NET_AS_NUM 2
#define VDPASIM_NET_GROUP_NUM 2
struct vdpasim_dataq_stats {
struct u64_stats_sync syncp;
u64 pkts;
u64 bytes;
u64 drops;
u64 errors;
u64 overruns;
};
struct vdpasim_cq_stats {
struct u64_stats_sync syncp;
u64 requests;
u64 successes;
u64 errors;
};
struct vdpasim_net{
struct vdpasim vdpasim;
struct vdpasim_dataq_stats tx_stats;
struct vdpasim_dataq_stats rx_stats;
struct vdpasim_cq_stats cq_stats;
};
static struct vdpasim_net *sim_to_net(struct vdpasim *vdpasim)
{
return container_of(vdpasim, struct vdpasim_net, vdpasim);
}
static void vdpasim_net_complete(struct vdpasim_virtqueue *vq, size_t len)
{
/* Make sure data is wrote before advancing index */
......@@ -96,9 +126,11 @@ static virtio_net_ctrl_ack vdpasim_handle_ctrl_mac(struct vdpasim *vdpasim,
static void vdpasim_handle_cvq(struct vdpasim *vdpasim)
{
struct vdpasim_virtqueue *cvq = &vdpasim->vqs[2];
struct vdpasim_net *net = sim_to_net(vdpasim);
virtio_net_ctrl_ack status = VIRTIO_NET_ERR;
struct virtio_net_ctrl_hdr ctrl;
size_t read, write;
u64 requests = 0, errors = 0, successes = 0;
int err;
if (!(vdpasim->features & (1ULL << VIRTIO_NET_F_CTRL_VQ)))
......@@ -114,10 +146,13 @@ static void vdpasim_handle_cvq(struct vdpasim *vdpasim)
if (err <= 0)
break;
++requests;
read = vringh_iov_pull_iotlb(&cvq->vring, &cvq->in_iov, &ctrl,
sizeof(ctrl));
if (read != sizeof(ctrl))
if (read != sizeof(ctrl)) {
++errors;
break;
}
switch (ctrl.class) {
case VIRTIO_NET_CTRL_MAC:
......@@ -127,6 +162,11 @@ static void vdpasim_handle_cvq(struct vdpasim *vdpasim)
break;
}
if (status == VIRTIO_NET_OK)
++successes;
else
++errors;
/* Make sure data is wrote before advancing index */
smp_wmb();
......@@ -144,6 +184,12 @@ static void vdpasim_handle_cvq(struct vdpasim *vdpasim)
cvq->cb(cvq->private);
local_bh_enable();
}
u64_stats_update_begin(&net->cq_stats.syncp);
net->cq_stats.requests += requests;
net->cq_stats.errors += errors;
net->cq_stats.successes += successes;
u64_stats_update_end(&net->cq_stats.syncp);
}
static void vdpasim_net_work(struct work_struct *work)
......@@ -151,8 +197,10 @@ static void vdpasim_net_work(struct work_struct *work)
struct vdpasim *vdpasim = container_of(work, struct vdpasim, work);
struct vdpasim_virtqueue *txq = &vdpasim->vqs[1];
struct vdpasim_virtqueue *rxq = &vdpasim->vqs[0];
struct vdpasim_net *net = sim_to_net(vdpasim);
ssize_t read, write;
int pkts = 0;
u64 tx_pkts = 0, rx_pkts = 0, tx_bytes = 0, rx_bytes = 0;
u64 rx_drops = 0, rx_overruns = 0, rx_errors = 0, tx_errors = 0;
int err;
spin_lock(&vdpasim->lock);
......@@ -171,14 +219,21 @@ static void vdpasim_net_work(struct work_struct *work)
while (true) {
err = vringh_getdesc_iotlb(&txq->vring, &txq->out_iov, NULL,
&txq->head, GFP_ATOMIC);
if (err <= 0)
if (err <= 0) {
if (err)
++tx_errors;
break;
}
++tx_pkts;
read = vringh_iov_pull_iotlb(&txq->vring, &txq->out_iov,
vdpasim->buffer,
PAGE_SIZE);
tx_bytes += read;
if (!receive_filter(vdpasim, read)) {
++rx_drops;
vdpasim_net_complete(txq, 0);
continue;
}
......@@ -186,19 +241,25 @@ static void vdpasim_net_work(struct work_struct *work)
err = vringh_getdesc_iotlb(&rxq->vring, NULL, &rxq->in_iov,
&rxq->head, GFP_ATOMIC);
if (err <= 0) {
++rx_overruns;
vdpasim_net_complete(txq, 0);
break;
}
write = vringh_iov_push_iotlb(&rxq->vring, &rxq->in_iov,
vdpasim->buffer, read);
if (write <= 0)
if (write <= 0) {
++rx_errors;
break;
}
++rx_pkts;
rx_bytes += write;
vdpasim_net_complete(txq, 0);
vdpasim_net_complete(rxq, write);
if (++pkts > 4) {
if (tx_pkts > 4) {
schedule_work(&vdpasim->work);
goto out;
}
......@@ -206,6 +267,145 @@ static void vdpasim_net_work(struct work_struct *work)
out:
spin_unlock(&vdpasim->lock);
u64_stats_update_begin(&net->tx_stats.syncp);
net->tx_stats.pkts += tx_pkts;
net->tx_stats.bytes += tx_bytes;
net->tx_stats.errors += tx_errors;
u64_stats_update_end(&net->tx_stats.syncp);
u64_stats_update_begin(&net->rx_stats.syncp);
net->rx_stats.pkts += rx_pkts;
net->rx_stats.bytes += rx_bytes;
net->rx_stats.drops += rx_drops;
net->rx_stats.errors += rx_errors;
net->rx_stats.overruns += rx_overruns;
u64_stats_update_end(&net->rx_stats.syncp);
}
static int vdpasim_net_get_stats(struct vdpasim *vdpasim, u16 idx,
struct sk_buff *msg,
struct netlink_ext_ack *extack)
{
struct vdpasim_net *net = sim_to_net(vdpasim);
u64 rx_pkts, rx_bytes, rx_errors, rx_overruns, rx_drops;
u64 tx_pkts, tx_bytes, tx_errors, tx_drops;
u64 cq_requests, cq_successes, cq_errors;
unsigned int start;
int err = -EMSGSIZE;
switch(idx) {
case 0:
do {
start = u64_stats_fetch_begin(&net->rx_stats.syncp);
rx_pkts = net->rx_stats.pkts;
rx_bytes = net->rx_stats.bytes;
rx_errors = net->rx_stats.errors;
rx_overruns = net->rx_stats.overruns;
rx_drops = net->rx_stats.drops;
} while (u64_stats_fetch_retry(&net->rx_stats.syncp, start));
if (nla_put_string(msg, VDPA_ATTR_DEV_VENDOR_ATTR_NAME,
"rx packets"))
break;
if (nla_put_u64_64bit(msg, VDPA_ATTR_DEV_VENDOR_ATTR_VALUE,
rx_pkts, VDPA_ATTR_PAD))
break;
if (nla_put_string(msg, VDPA_ATTR_DEV_VENDOR_ATTR_NAME,
"rx bytes"))
break;
if (nla_put_u64_64bit(msg, VDPA_ATTR_DEV_VENDOR_ATTR_VALUE,
rx_bytes, VDPA_ATTR_PAD))
break;
if (nla_put_string(msg, VDPA_ATTR_DEV_VENDOR_ATTR_NAME,
"rx errors"))
break;
if (nla_put_u64_64bit(msg, VDPA_ATTR_DEV_VENDOR_ATTR_VALUE,
rx_errors, VDPA_ATTR_PAD))
break;
if (nla_put_string(msg, VDPA_ATTR_DEV_VENDOR_ATTR_NAME,
"rx overruns"))
break;
if (nla_put_u64_64bit(msg, VDPA_ATTR_DEV_VENDOR_ATTR_VALUE,
rx_overruns, VDPA_ATTR_PAD))
break;
if (nla_put_string(msg, VDPA_ATTR_DEV_VENDOR_ATTR_NAME,
"rx drops"))
break;
if (nla_put_u64_64bit(msg, VDPA_ATTR_DEV_VENDOR_ATTR_VALUE,
rx_drops, VDPA_ATTR_PAD))
break;
err = 0;
break;
case 1:
do {
start = u64_stats_fetch_begin(&net->tx_stats.syncp);
tx_pkts = net->tx_stats.pkts;
tx_bytes = net->tx_stats.bytes;
tx_errors = net->tx_stats.errors;
tx_drops = net->tx_stats.drops;
} while (u64_stats_fetch_retry(&net->tx_stats.syncp, start));
if (nla_put_string(msg, VDPA_ATTR_DEV_VENDOR_ATTR_NAME,
"tx packets"))
break;
if (nla_put_u64_64bit(msg, VDPA_ATTR_DEV_VENDOR_ATTR_VALUE,
tx_pkts, VDPA_ATTR_PAD))
break;
if (nla_put_string(msg, VDPA_ATTR_DEV_VENDOR_ATTR_NAME,
"tx bytes"))
break;
if (nla_put_u64_64bit(msg, VDPA_ATTR_DEV_VENDOR_ATTR_VALUE,
tx_bytes, VDPA_ATTR_PAD))
break;
if (nla_put_string(msg, VDPA_ATTR_DEV_VENDOR_ATTR_NAME,
"tx errors"))
break;
if (nla_put_u64_64bit(msg, VDPA_ATTR_DEV_VENDOR_ATTR_VALUE,
tx_errors, VDPA_ATTR_PAD))
break;
if (nla_put_string(msg, VDPA_ATTR_DEV_VENDOR_ATTR_NAME,
"tx drops"))
break;
if (nla_put_u64_64bit(msg, VDPA_ATTR_DEV_VENDOR_ATTR_VALUE,
tx_drops, VDPA_ATTR_PAD))
break;
err = 0;
break;
case 2:
do {
start = u64_stats_fetch_begin(&net->cq_stats.syncp);
cq_requests = net->cq_stats.requests;
cq_successes = net->cq_stats.successes;
cq_errors = net->cq_stats.errors;
} while (u64_stats_fetch_retry(&net->cq_stats.syncp, start));
if (nla_put_string(msg, VDPA_ATTR_DEV_VENDOR_ATTR_NAME,
"cvq requests"))
break;
if (nla_put_u64_64bit(msg, VDPA_ATTR_DEV_VENDOR_ATTR_VALUE,
cq_requests, VDPA_ATTR_PAD))
break;
if (nla_put_string(msg, VDPA_ATTR_DEV_VENDOR_ATTR_NAME,
"cvq successes"))
break;
if (nla_put_u64_64bit(msg, VDPA_ATTR_DEV_VENDOR_ATTR_VALUE,
cq_successes, VDPA_ATTR_PAD))
break;
if (nla_put_string(msg, VDPA_ATTR_DEV_VENDOR_ATTR_NAME,
"cvq errors"))
break;
if (nla_put_u64_64bit(msg, VDPA_ATTR_DEV_VENDOR_ATTR_VALUE,
cq_errors, VDPA_ATTR_PAD))
break;
err = 0;
break;
default:
err = -EINVAL;
break;
}
return err;
}
static void vdpasim_net_get_config(struct vdpasim *vdpasim, void *config)
......@@ -242,6 +442,7 @@ static int vdpasim_net_dev_add(struct vdpa_mgmt_dev *mdev, const char *name,
const struct vdpa_dev_set_config *config)
{
struct vdpasim_dev_attr dev_attr = {};
struct vdpasim_net *net;
struct vdpasim *simdev;
int ret;
......@@ -252,9 +453,11 @@ static int vdpasim_net_dev_add(struct vdpa_mgmt_dev *mdev, const char *name,
dev_attr.nvqs = VDPASIM_NET_VQ_NUM;
dev_attr.ngroups = VDPASIM_NET_GROUP_NUM;
dev_attr.nas = VDPASIM_NET_AS_NUM;
dev_attr.alloc_size = sizeof(struct vdpasim_net);
dev_attr.config_size = sizeof(struct virtio_net_config);
dev_attr.get_config = vdpasim_net_get_config;
dev_attr.work_fn = vdpasim_net_work;
dev_attr.get_stats = vdpasim_net_get_stats;
dev_attr.buffer_size = PAGE_SIZE;
simdev = vdpasim_create(&dev_attr, config);
......@@ -267,6 +470,12 @@ static int vdpasim_net_dev_add(struct vdpa_mgmt_dev *mdev, const char *name,
if (ret)
goto reg_err;
net = sim_to_net(simdev);
u64_stats_init(&net->tx_stats.syncp);
u64_stats_init(&net->rx_stats.syncp);
u64_stats_init(&net->cq_stats.syncp);
return 0;
reg_err:
......
......@@ -73,7 +73,8 @@ enum {
VHOST_NET_FEATURES = VHOST_FEATURES |
(1ULL << VHOST_NET_F_VIRTIO_NET_HDR) |
(1ULL << VIRTIO_NET_F_MRG_RXBUF) |
(1ULL << VIRTIO_F_ACCESS_PLATFORM)
(1ULL << VIRTIO_F_ACCESS_PLATFORM) |
(1ULL << VIRTIO_F_RING_RESET)
};
enum {
......@@ -1645,7 +1646,7 @@ static int vhost_net_set_features(struct vhost_net *n, u64 features)
goto out_unlock;
if ((features & (1ULL << VIRTIO_F_ACCESS_PLATFORM))) {
if (vhost_init_device_iotlb(&n->dev, true))
if (vhost_init_device_iotlb(&n->dev))
goto out_unlock;
}
......
......@@ -2105,7 +2105,7 @@ static ssize_t vhost_scsi_tpg_attrib_fabric_prot_type_show(
struct vhost_scsi_tpg *tpg = container_of(se_tpg,
struct vhost_scsi_tpg, se_tpg);
return sprintf(page, "%d\n", tpg->tv_fabric_prot_type);
return sysfs_emit(page, "%d\n", tpg->tv_fabric_prot_type);
}
CONFIGFS_ATTR(vhost_scsi_tpg_attrib_, fabric_prot_type);
......@@ -2215,7 +2215,7 @@ static ssize_t vhost_scsi_tpg_nexus_show(struct config_item *item, char *page)
mutex_unlock(&tpg->tv_tpg_mutex);
return -ENODEV;
}
ret = snprintf(page, PAGE_SIZE, "%s\n",
ret = sysfs_emit(page, "%s\n",
tv_nexus->tvn_se_sess->se_node_acl->initiatorname);
mutex_unlock(&tpg->tv_tpg_mutex);
......@@ -2440,7 +2440,7 @@ static void vhost_scsi_drop_tport(struct se_wwn *wwn)
static ssize_t
vhost_scsi_wwn_version_show(struct config_item *item, char *page)
{
return sprintf(page, "TCM_VHOST fabric module %s on %s/%s"
return sysfs_emit(page, "TCM_VHOST fabric module %s on %s/%s"
"on "UTS_RELEASE"\n", VHOST_SCSI_VERSION, utsname()->sysname,
utsname()->machine);
}
......
......@@ -333,13 +333,10 @@ static long vhost_test_ioctl(struct file *f, unsigned int ioctl,
return -EFAULT;
return 0;
case VHOST_SET_FEATURES:
printk(KERN_ERR "1\n");
if (copy_from_user(&features, featurep, sizeof features))
return -EFAULT;
printk(KERN_ERR "2\n");
if (features & ~VHOST_FEATURES)
return -EOPNOTSUPP;
printk(KERN_ERR "3\n");
return vhost_test_set_features(n, features);
case VHOST_RESET_OWNER:
return vhost_test_reset_owner(n);
......
......@@ -359,6 +359,14 @@ static bool vhost_vdpa_can_suspend(const struct vhost_vdpa *v)
return ops->suspend;
}
static bool vhost_vdpa_can_resume(const struct vhost_vdpa *v)
{
struct vdpa_device *vdpa = v->vdpa;
const struct vdpa_config_ops *ops = vdpa->config;
return ops->resume;
}
static long vhost_vdpa_get_features(struct vhost_vdpa *v, u64 __user *featurep)
{
struct vdpa_device *vdpa = v->vdpa;
......@@ -498,6 +506,21 @@ static long vhost_vdpa_suspend(struct vhost_vdpa *v)
return ops->suspend(vdpa);
}
/* After a successful return of this ioctl the device resumes processing
* virtqueue descriptors. The device becomes fully operational the same way it
* was before it was suspended.
*/
static long vhost_vdpa_resume(struct vhost_vdpa *v)
{
struct vdpa_device *vdpa = v->vdpa;
const struct vdpa_config_ops *ops = vdpa->config;
if (!ops->resume)
return -EOPNOTSUPP;
return ops->resume(vdpa);
}
static long vhost_vdpa_vring_ioctl(struct vhost_vdpa *v, unsigned int cmd,
void __user *argp)
{
......@@ -606,11 +629,15 @@ static long vhost_vdpa_unlocked_ioctl(struct file *filep,
if (copy_from_user(&features, featurep, sizeof(features)))
return -EFAULT;
if (features & ~(VHOST_VDPA_BACKEND_FEATURES |
BIT_ULL(VHOST_BACKEND_F_SUSPEND)))
BIT_ULL(VHOST_BACKEND_F_SUSPEND) |
BIT_ULL(VHOST_BACKEND_F_RESUME)))
return -EOPNOTSUPP;
if ((features & BIT_ULL(VHOST_BACKEND_F_SUSPEND)) &&
!vhost_vdpa_can_suspend(v))
return -EOPNOTSUPP;
if ((features & BIT_ULL(VHOST_BACKEND_F_RESUME)) &&
!vhost_vdpa_can_resume(v))
return -EOPNOTSUPP;
vhost_set_backend_features(&v->vdev, features);
return 0;
}
......@@ -662,6 +689,8 @@ static long vhost_vdpa_unlocked_ioctl(struct file *filep,
features = VHOST_VDPA_BACKEND_FEATURES;
if (vhost_vdpa_can_suspend(v))
features |= BIT_ULL(VHOST_BACKEND_F_SUSPEND);
if (vhost_vdpa_can_resume(v))
features |= BIT_ULL(VHOST_BACKEND_F_RESUME);
if (copy_to_user(featurep, &features, sizeof(features)))
r = -EFAULT;
break;
......@@ -677,6 +706,9 @@ static long vhost_vdpa_unlocked_ioctl(struct file *filep,
case VHOST_VDPA_SUSPEND:
r = vhost_vdpa_suspend(v);
break;
case VHOST_VDPA_RESUME:
r = vhost_vdpa_resume(v);
break;
default:
r = vhost_dev_ioctl(&v->vdev, cmd, argp);
if (r == -ENOIOCTLCMD)
......@@ -1119,8 +1151,11 @@ static int vhost_vdpa_alloc_domain(struct vhost_vdpa *v)
if (!bus)
return -EFAULT;
if (!device_iommu_capable(dma_dev, IOMMU_CAP_CACHE_COHERENCY))
if (!device_iommu_capable(dma_dev, IOMMU_CAP_CACHE_COHERENCY)) {
dev_warn_once(&v->dev,
"Failed to allocate domain, device is not IOMMU cache coherent capable\n");
return -ENOTSUPP;
}
v->domain = iommu_domain_alloc(bus);
if (!v->domain)
......
......@@ -1730,7 +1730,7 @@ long vhost_vring_ioctl(struct vhost_dev *d, unsigned int ioctl, void __user *arg
}
EXPORT_SYMBOL_GPL(vhost_vring_ioctl);
int vhost_init_device_iotlb(struct vhost_dev *d, bool enabled)
int vhost_init_device_iotlb(struct vhost_dev *d)
{
struct vhost_iotlb *niotlb, *oiotlb;
int i;
......
......@@ -222,7 +222,7 @@ ssize_t vhost_chr_read_iter(struct vhost_dev *dev, struct iov_iter *to,
int noblock);
ssize_t vhost_chr_write_iter(struct vhost_dev *dev,
struct iov_iter *from);
int vhost_init_device_iotlb(struct vhost_dev *d, bool enabled);
int vhost_init_device_iotlb(struct vhost_dev *d);
void vhost_iotlb_map_free(struct vhost_iotlb *iotlb,
struct vhost_iotlb_map *map);
......
......@@ -793,7 +793,7 @@ static int vhost_vsock_set_features(struct vhost_vsock *vsock, u64 features)
}
if ((features & (1ULL << VIRTIO_F_ACCESS_PLATFORM))) {
if (vhost_init_device_iotlb(&vsock->dev, true))
if (vhost_init_device_iotlb(&vsock->dev))
goto err;
}
......
This diff is collapsed.
......@@ -135,6 +135,7 @@ virtio_vdpa_setup_vq(struct virtio_device *vdev, unsigned int index,
{
struct virtio_vdpa_device *vd_dev = to_virtio_vdpa_device(vdev);
struct vdpa_device *vdpa = vd_get_vdpa(vdev);
struct device *dma_dev;
const struct vdpa_config_ops *ops = vdpa->config;
struct virtio_vdpa_vq_info *info;
struct vdpa_callback cb;
......@@ -175,9 +176,15 @@ virtio_vdpa_setup_vq(struct virtio_device *vdev, unsigned int index,
/* Create the vring */
align = ops->get_vq_align(vdpa);
vq = vring_create_virtqueue(index, max_num, align, vdev,
true, may_reduce_num, ctx,
virtio_vdpa_notify, callback, name);
if (ops->get_vq_dma_dev)
dma_dev = ops->get_vq_dma_dev(vdpa, index);
else
dma_dev = vdpa_get_dma_dev(vdpa);
vq = vring_create_virtqueue_dma(index, max_num, align, vdev,
true, may_reduce_num, ctx,
virtio_vdpa_notify, callback,
name, dma_dev);
if (!vq) {
err = -ENOMEM;
goto error_new_virtqueue;
......
......@@ -3094,6 +3094,8 @@
#define PCI_VENDOR_ID_3COM_2 0xa727
#define PCI_VENDOR_ID_SOLIDRUN 0xd063
#define PCI_VENDOR_ID_DIGIUM 0xd161
#define PCI_DEVICE_ID_DIGIUM_HFC4S 0xb410
......
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
Markdown is supported
0%
or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment