- 06 Dec, 2022 1 commit
-
-
Dave Jiang authored
Change names for granularity macros to clearly indicate which variable is encoded and which is the actual granularity. granularity == interleave granularity eig == encoded interleave granularity Reviewed-by:
Jonathan Cameron <Jonathan.Cameron@huawei.com> Reviewed-by:
Alison Schofield <alison.schofield@intel.com> Signed-off-by:
Dave Jiang <dave.jiang@intel.com> Link: https://lore.kernel.org/r/167027493237.3124429.8948852388671827664.stgit@djiang5-desk3.ch.intel.comSigned-off-by:
Dan Williams <dan.j.williams@intel.com>
-
- 05 Dec, 2022 12 commits
-
-
Robert Richter authored
After parsing for a CHBCR in cxl_get_chbcr() the case of (ctx.chbcr == CXL_RESOURCE_NONE) is a slighly different error reason than the !ctx.chbcr case. In the first case the CHBS was found but the CHBCR was invalid or something else failed to determine it, while in the latter case no CHBS entry exists at all. Update the warning message to reflect this. The log messages for both cases can be differentiated now and the reason for a failure can be determined better. Signed-off-by:
Robert Richter <rrichter@amd.com> Reviewed-by:
-
Dan Williams authored
The typical environment where cxl_test is run, QEMU, does not support cpu_cache_invalidate_memregion(). Add the 'test' bypass symbols to the configuration check. Reported-by:
-
Alison Schofield authored
The BIOS provided CXIMS (CXL XOR Interleave Math Structure) is required for calculating a targets position in an interleave list during region creation. The CXL driver expects to discover a CXIMS that matches the HBIG (Host Bridge Interleave Granularity) and stores the xormaps found in that CXIMS for retrieval during region creation. If there is no CXIMS for an HBIG, no maps are stored. That leads to a NULL pointer dereference at xormap retrieval during region creation. Add a check during ACPI probe for the case of no matching CXIMS. Emit an error message and fail to add the decoder. Fixes: f9db85bf ("cxl/acpi: Support CXL XOR Interleave Math (CXIMS)") Suggested-by:
-
Colin Ian King authored
There is a spelling mistake in a dev_warn message. Fix it. Signed-off-by:
Colin Ian King <colin.i.king@gmail.com> Link: https://lore.kernel.org/r/20221205091819.1943564-1-colin.i.king@gmail.comSigned-off-by:
-
Dan Williams authored
The 0day robot belatedly points out that @addr is not properly tagged as an iomap pointer: "drivers/cxl/core/regs.c:332:14: sparse: sparse: incorrect type in assignment (different address spaces) @@ expected void *addr @@ got void [noderef] __iomem * @@" Fixes: 1168271ca054 ("cxl/acpi: Extract component registers of restricted hosts from RCRB") Reported-by:kernel test robot <lkp@intel.com> Reviewed-by:
-
Dan Williams authored
Pick up support for "XOR" interleave math when parsing ACPI CFMWS window structures. Fix up conflicts with the RCH emulation already pending in cxl/next.
-
Dan Williams authored
Pick up CXL AER handling and correctable error extensions. Resolve conflicts with cxl_pmem_wq reworks and RCH support.
-
Dan Williams authored
Pick CXL PMEM security commands for v6.2. Resolve conflicts with the removal of the cxl_pmem_wq.
-
Terry Bowman authored
ACPI uses the CXL _OSC support method to communicate the available CXL functionality to FW. The CXL _OSC support method includes a field to indicate the OS is capable of RCD mode. FW can potentially change it's operation depending on the _OSC support method reported by the OS. The ACPI driver currently only sets the ACPI _OSC support method to indicate CXL VH mode. Change the capability reported to also include CXL RCD mode. [1] CXL3.0 Table 9-26 'Interpretation of CXL _OSC Support Field' Signed-off-by:
Terry Bowman <terry.bowman@amd.com> [rrichter@amd.com: Reworded patch description.] Signed-off-by:
-
Dan Williams authored
In an RCH topology a CXL host-bridge as Root Complex Integrated Endpoint the represents the memory expander. Unlike a VH topology there is no CXL/PCIE Root Port that host the endpoint. The CXL subsystem maps this as the CXL root object (ACPI0017 on ACPI based systems) targeting the host-bridge as a dport, per usual, but then that dport directly hosts the endpoint port. Mock up that configuration with a 4th host-bridge that has a 'cxl_rcd' device instance as its immediate child. Reviewed-by:
-
Dan Williams authored
Unlike a CXL memory expander in a VH topology that has at least one intervening 'struct cxl_port' instance between itself and the CXL root device, an RCD attaches one-level higher. For example: VH ┌──────────┐ │ ACPI0017 │ │ root0 │ └─────┬────┘ │ ┌─────┴────┐ │ dport0 │ ┌─────┤ ACPI0016 ├─────┐ │ │ port1 │ │ │ └────┬─────┘ │ │ │ │ ┌──┴───┐ ┌──┴───┐ ┌───┴──┐ │dport0│ │dport1│ │dport2│ │ RP0 │ │ RP1 │ │ RP2 │ └──────┘ └──┬───┘ └──────┘ │ ┌───┴─────┐ │endpoint0│ │ port2 │ └─────────┘ ...vs: RCH ┌──────────┐ │ ACPI0017 │ │ root0 │ └────┬─────┘ │ ┌───┴────┐ │ dport0 │ │ACPI0016│ └───┬────┘ │ ┌────┴─────┐ │endpoint0 │ │ port1 │ └──────────┘ So arrange for endpoint port in the RCH/RCD case to appear directly connected to the host-bridge in its singular role as a dport. Compare that to the VH case where the host-bridge serves a dual role as a 'cxl_dport' for the CXL root device *and* a 'cxl_port' upstream port for the Root Ports in the Root Complex that are modeled as 'cxl_dport' instances in the CXL topology. Another deviation from the VH case is that RCDs may need to look up their component registers from the Root Complex Register Block (RCRB). That platform firmware specified RCRB area is cached by the cxl_acpi driver and conveyed via the host-bridge dport to the cxl_mem driver to perform the cxl_rcrb_to_component() lookup for the endpoint port (See 9.11.8 CXL Devices Attached to an RCH for the lookup of the upstream port component registers). Tested-by: -
Dan Williams authored
tl;dr: Clean up an unnecessary export and enable cxl_test. An RCD (Restricted CXL Device), in contrast to a typical CXL device in a VH topology, obtains its component registers from the bottom half of the associated CXL host bridge RCRB (Root Complex Register Block). In turn this means that cxl_rcrb_to_component() needs to be called from devm_cxl_add_endpoint(). Presently devm_cxl_add_endpoint() is part of the CXL core, but the only user is the CXL mem module. Move it from cxl_core to cxl_mem to not only get rid of an unnecessary export, but to also enable its call out to cxl_rcrb_to_component(), in a subsequent patch, to be mocked by cxl_test. Recall that cxl_test can only mock exported symbols, and since cxl_rcrb_to_component() is itself inside the core, all callers must be outside of cxl_core to allow cxl_test to mock it. Reviewed-by:
-
- 04 Dec, 2022 2 commits
-
-
Alison Schofield authored
Expand the cxl_test topology to include CFMWS's that use XOR math for interleave arithmetic, as defined in the CXL Specification 3.0. With this expanded topology, cxl_test is useful for testing: x1,x2,x4 ways with XOR interleave arithmetic. Define the additional XOR CFMWS entries to appear only with the module parameter interleave_arithmetic=1. The cxl_test default continues to be modulo math. modprobe cxl_test interleave_arithmetic=1 Signed-off-by:
-
Alison Schofield authored
When the CFMWS is using XOR math, parse the corresponding CXIMS structure and store the xormaps in the root decoder structure. Use the xormaps in a new lookup, cxl_hb_xor(), to find a targets entry in the host bridge interleave target list. Defined in CXL Specfication 3.0 Section: 9.17.1 Signed-off-by:
-
- 03 Dec, 2022 24 commits
-
-
Dan Williams authored
Pick up: f350c68e ("ACPICA: Add CXL 3.0 structures (CXIMS & RDPAS) to the CEDT table") ...to build the new XOR interleave math support for the CXL Fixed Memory Window Structures.
-
Dave Jiang authored
Add AER error handler callback to read the RAS capability structure correctable error (CE) status register for the CXL device. Log the error as a trace event and clear the error. For CXL devices, the driver also needs to write back to the status register to clear the unmasked correctable errors. See CXL spec rev3.0 8.2.4.16 for RAS capability structure CE Status Register. Suggested-by:
-
Dave Jiang authored
Some new devices such as CXL devices may want to record additional error information on a corrected error. Add a callback to allow the PCI device driver to do additional logging such as providing additional stats for user space RAS monitoring. For CXL device, this is actually a need due to CXL needing to write to the CXL RAS capability structure correctable error status register in order to clear the unmasked correctable errors. See CXL spec rev3.0 8.2.4.16. Suggested-by:
Kuppuswamy Sathyanarayanan <sathyanarayanan.kuppuswamy@linux.intel.com> Reviewed-by:
Bjorn Helgaas <bhelgaas@google.com> Signed-off-by:
-
Dan Williams authored
Add nominal error handling that tears down CXL.mem in response to error notifications that imply a device reset. Given some CXL.mem may be operating as System RAM, there is a high likelihood that these error events are fatal. However, if the system survives the notification the expectation is that the driver behavior is equivalent to a hot-unplug and re-plug of an endpoint. Note that this does not change the mask values from the default. That awaits CXL _OSC support to determine whether platform firmware is in control of the mask registers. Reviewed-by:
-
Dave Jiang authored
Add tracepoint events for recording the CXL uncorrectable and correctable errors. For uncorrectable errors, there is additional data of 512B from the header log register (CXL spec rev3 8.2.4.16.7). The trace event will intake a dynamic array that will dump the entire Header Log data. If multiple errors are set in the status register, then the 'first error' field (CXL spec rev3 v8.2.4.16.6) is read from the Error Capabilities and Control Register in order to determine the error. This implementation does not include CXL IDE Error details. Cc: Steven Rostedt <rostedt@goodmis.org> Reviewed-by:
Steven Rostedt (Google) <rostedt@goodmis.org> Link: https://lore.kernel.org/r/166974413388.1608150.5875712482260436188.stgit@djiang5-desk3.ch.intel.comSigned-off-by:
-
Dan Williams authored
The RAS Capability Structure has some ancillary information that may be relevant with respect to AER events, link and protcol error status registers. Map the RAS Capability Registers in support of defining a 'struct pci_error_handlers' instance for the cxl_pci driver. Reviewed-by:
-
Dan Williams authored
The RAS Capabilitiy Structure is a CXL Component register capability block. Unlike the HDM Decoder Capability, it will be referenced by the cxl_pci driver in response to PCIe AER events. Due to this it is no longer the case that cxl_map_component_regs() can assume that it should map all component registers. Plumb a bitmask of capability ids to map through cxl_map_component_regs(). For symmetry cxl_probe_device_regs() is updated to populate @id in 'struct cxl_reg_map' even though cxl_map_device_regs() does not have a need to map a subset of the device registers per caller. Reviewed-by:
-
Dan Williams authored
Update the port driver to use cxl_map_component_registers() so that the component register block can be shared between the cxl_pci driver and the cxl_port driver. I.e. stop the port driver from reserving the entire component register block for itself via request_region() when it only needs the HDM Decoder Capability subset. Reviewed-by:
-
Dan Williams authored
There is no need to carry the barno and the block offset through the stack, just convert them to a resource base immediately. Reviewed-by:
-
Dan Williams authored
The component registers are currently unused by the cxl_pci driver. Only the physical address base of the component registers is conveyed to the cxl_mem driver. Just call cxl_map_device_registers() directly. Reviewed-by:
-
Dan Williams authored
Use a loop to reduce the duplicated code in cxl_map_device_regs(). This is in preparation for deleting cxl_map_regs(). Reviewed-by:
-
Dan Williams authored
Rather then duplicating the setting of valid, length, and offset for each type, just convey a pointer to the register map to common code. Yes, the change in cxl_probe_component_regs() does not save any lines of code, but it is preparation for adding another component register type to map (RAS Capability Structure). Reviewed-by:
-
Robert Richter authored
A downstream port must be connected to a component register block. For restricted hosts the base address is determined from the RCRB. The RCRB is provided by the host's CEDT CHBS entry. Rework CEDT parser to get the RCRB and add code to extract the component register block from it. RCRB's BAR[0..1] point to the component block containing CXL subsystem component registers. MEMBAR extraction follows the PCI base spec here, esp. 64 bit extraction and memory range alignment (6.0, 7.5.1.2.1). The RCRB base address is cached in the cxl_dport per-host bridge so that the upstream port component registers can be retrieved later by an RCD (RCIEP) associated with the host bridge. Note: Right now the component register block is used for HDM decoder capability only which is optional for RCDs. If unsupported by the RCD, the HDM init will fail. It is future work to bypass it in this case. Co-developed-by:
-
Dan Williams authored
A "DPA invalidation event" is any scenario where the contents of a DPA (Device Physical Address) is modified in a way that is incoherent with CPU caches, or if the HPA (Host Physical Address) to DPA association changes due to a remapping event. PMEM security events like Unlock and Passphrase Secure Erase already manage caches through LIBNVDIMM, so that leaves HPA to DPA remap events that need cache management by the CXL core. Those only happen when the boot time CXL configuration has changed. That event occurs when userspace attaches an endpoint decoder to a region configuration, and that region is subsequently activated. The implications of not invalidating caches between remap events is that reads from the region at different points in time may return different results due to stale cached data from the previous HPA to DPA mapping. Without a guarantee that the region contents after cxl_region_probe() are written before being read (a layering-violation assumption that cxl_region_probe() can not make) the CXL subsystem needs to ensure that reads that precede writes see consistent results. A CONFIG_CXL_REGION_INVALIDATION_TEST option is added to support debug and unit testing of the CXL implementation in QEMU or other environments where cpu_cache_has_invalidate_memregion() returns false. This may prove too restrictive for QEMU where the HDM decoders are emulated, but in that case the CXL subsystem needs some new mechanism / indication that the HDM decoder is emulated and not a passthrough of real hardware. Reviewed-by:
-
Dan Williams authored
Now that cpu_cache_invalidate_memregion() is generically available, use it to centralize CPU cache management in the nvdimm region driver. This trades off removing redundant per-dimm CPU cache flushing with an opportunistic flush on every region disable event to cover the case of sensitive dirty data in the cache being written back to media after a secure erase / overwrite event. Reviewed-by:
Davidlohr Bueso <dave@stgolabs.net> Reviewed-by:
-
Dan Williams authored
Preclude the possibility of user tooling sending device secrets in the clear into the kernel by marking the security commands as exclusive. This mandates the usage of the keyctl ABI for managing the device passphrase. Reviewed-by:
-
Dan Williams authored
cxl_region_probe() allows for regions not in the 'commit' state to be enabled. Fail probe when the region is not committed otherwise the kernel may indicate that an address range is active when none of the decoders are active. Fixes: 8d48817d ("cxl/region: Add region driver boiler plate") Cc: <stable@vger.kernel.org> Reviewed-by:
-
Dave Jiang authored
Set the cxlds->serial as the dimm_id to be fed to __nvdimm_create(). The security code uses that as the key description for the security key of the memory device. The nvdimm unlock code cannot find the respective key without the dimm_id. Reviewed-by:
-
Robert Richter authored
A port of a CXL host bridge links to the bridge's ACPI device (&adev->dev) with its corresponding uport/dport device (uport_dev and dport_dev respectively). The device is not a direct parent device in the PCI topology as pdev->dev.parent points to a PCI bridge's (struct pci_host_bridge) device. The following CXL memory device hierarchy would be valid for an endpoint once an RCD EP would be enabled (note this will be done in a later patch): VH mode: cxlmd->dev.parent->parent ^^^\^^^^^^\ ^^^^^^\ \ \ pci_dev (Type 1, Downstream Port) \ pci_dev (Type 0, PCI Express Endpoint) cxl mem device RCD mode: cxlmd->dev.parent->parent ^^^\^^^^^^\ ^^^^^^\ \ \ pci_host_bridge \ pci_dev (Type 0, RCiEP) cxl mem device In VH mode a downstream port is created by port enumeration and thus always exists. Now, in RCD mode the host bridge also already exists but it references to an ACPI device. A port lookup by the PCI device's parent device will fail as a direct link to the registered port is missing. The ACPI device of the bridge must be determined first. To prevent this, change port registration of a CXL host to use the bridge device instead. Do this also for the VH case as port topology will better reflect the PCI topology then. Signed-off-by: -
Dan Williams authored
Accept any cxl_test topology device as the first argument in cxl_chbs_context. This is in preparation for reworking the detection of the component registers across VH and RCH topologies. Move mock_acpi_table_parse_cedt() beneath the definition of is_mock_port() and use is_mock_port() instead of the explicit mock cxl_acpi device check. Acked-by:
-
Dan Williams authored
Now that the cxl_mem driver has a need to take the root device lock, the cxl_bus_rescan() needs to run outside of the root lock context. That need arises from RCH topologies and the locking that the cxl_mem driver does to attach a descendant to an upstream port. In the RCH case the lock needed is the CXL root device lock [1]. Link: http://lore.kernel.org/r/166993045621.1882361.1730100141527044744.stgit@dwillia2-xfh.jf.intel.com [1] Tested-by:
-
Dan Williams authored
Now that cxl_nvdimm and cxl_pmem_region objects are torn down sychronously with the removal of either the bridge, or an endpoint, the cxl_pmem_wq infrastructure can be jettisoned. Tested-by:
-
Dan Williams authored
The three objects 'struct cxl_nvdimm_bridge', 'struct cxl_nvdimm', and 'struct cxl_pmem_region' manage CXL persistent memory resources. The bridge represents base platform resources, the nvdimm represents one or more endpoints, and the region is a collection of nvdimms that contribute to an assembled address range. Their relationship is such that a region is torn down if any component endpoints are removed. All regions and endpoints are torn down if the foundational bridge device goes down. A workqueue was deployed to manage these interdependencies, but it is difficult to reason about, and fragile. A recent attempt to take the CXL root device lock in the cxl_mem driver was reported by lockdep as colliding with the flush_work() in the cxl_pmem flows. Instead of the workqueue, arrange for all pmem/nvdimm devices to be torn down immediately and hierarchically. A similar change is made to both the 'cxl_nvdimm' and 'cxl_pmem_region' objects. For bisect-ability both changes are made in the same patch which unfortunately makes the patch bigger than desired. Arrange for cxl_memdev and cxl_region to register a cxl_nvdimm and cxl_pmem_region as a devres release action of the bridge device. Additionally, include a devres release action of the cxl_memdev or cxl_region device that triggers the bridge's release action if an endpoint exits before the bridge. I.e. this allows either unplugging the bridge, or unplugging and endpoint to result in the same cleanup actions. To keep the patch smaller the cleanup of the now defunct workqueue infrastructure is saved for a follow-on patch. Tested-by:
-
Dan Williams authored
Now that a cxl_nvdimm object can only experience ->remove() via an unregistration event (because the cxl_nvdimm bind attributes are suppressed), additional cleanups are possible. It is already the case that the removal of a cxl_memdev object triggers ->remove() on any associated region. With that mechanism in place there is no need for the cxl_nvdimm removal to trigger the same. Just rely on cxl_region_detach() to tear down the whole cxl_pmem_region. Tested-by:
-
- 01 Dec, 2022 1 commit
-
-
Dan Williams authored
The 'struct cxl_nvdimm_bridge' object advertises platform CXL PMEM resources. It coordinates with libnvdimm to attach nvdimm devices and regions for each corresponding CXL object. That coordination is complicated, i.e. difficult to reason about, and it turns out redundant. It is already the case that the CXL core knows how to tear down a cxl_region when a cxl_memdev goes through ->remove(), so that pathway can be extended to directly cleanup cxl_nvdimm and cxl_pmem_region objects. Towards the goal of ripping out the cxl_nvdimm_bridge state machine, arrange for cxl_acpi to optionally pre-load the cxl_pmem driver so that the nvdimm bridge is active synchronously with devm_cxl_add_nvdimm_bridge(), and remove all the bind attributes for the cxl_nvdimm* objects since the cxl root device and cxl_memdev bind attributes are sufficient. Tested-by:
-
