- 04 Mar, 2021 40 commits
-
-
Mike Christie authored
This patch adds plug/unplug callouts for tcmu, so we can avoid the number of times we switch to userspace. Using this driver with tcm_loop is a common config, and dependng on the nr_hw_queues (nr_hw_queues=1 performs much better) and fio jobs (lower num jobs around 4) this patch can increase IOPS by only around 5-10% because we hit other issues like the big per tcmu device mutex. Link: https://lore.kernel.org/r/20210227170006.5077-24-michael.christie@oracle.comReviewed-by: Bodo Stroesser <bostroesser@gmail.com> Signed-off-by: Mike Christie <michael.christie@oracle.com> Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
-
Mike Christie authored
This patch adds plug/unplug callouts for iblock. For an initiator driver like iSCSI which wants to pass multiple cmds to its xmit thread instead of one cmd at a time, this increases IOPS by around 10% with vhost-scsi (combined with the last patches we can see a total 40-50% increase). For driver combos like tcm_loop and faster drivers like the iSER initiator, we can still see IOPS increase by 20-30% when tcm_loop's nr_hw_queues setting is also increased. Link: https://lore.kernel.org/r/20210227170006.5077-23-michael.christie@oracle.comSigned-off-by: Mike Christie <michael.christie@oracle.com> Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
-
Mike Christie authored
target_core_iblock is plugging and unplugging on every command and this is causing perf issues for drivers that prefer batched cmds. With recent patches we can now take multiple cmds from a fabric driver queue and then pass them down the backend drivers in a batch. This patch adds this support by adding 2 callouts to the backend for plugging and unplugging the device. Subsequent commits will add support for iblock and tcmu device plugging. Link: https://lore.kernel.org/r/20210227170006.5077-22-michael.christie@oracle.comReviewed-by: Bodo Stroesser <bostroesser@gmail.com> Signed-off-by: Mike Christie <michael.christie@oracle.com> Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
-
Mike Christie authored
We have a couple holes in the cmd flags definitions. This cleans up the definitions to fix that and make it easier to read. Link: https://lore.kernel.org/r/20210227170006.5077-21-michael.christie@oracle.comReviewed-by: Chaitanya Kulkarni <chaitanya.kulkarni@wdc.com> Reviewed-by: Christoph Hellwig <hch@lst.de> Reviewed-by: Himanshu Madhani <himanshu.madhani@oracle.com> Signed-off-by: Mike Christie <michael.christie@oracle.com> Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
-
Mike Christie authored
Convert loop to use the LIO wq cmd submission helper. Link: https://lore.kernel.org/r/20210227170006.5077-20-michael.christie@oracle.comTested-by: Laurence Oberman <loberman@redhat.com> Reviewed-by: Bodo Stroesser <bostroesser@gmail.com> Signed-off-by: Mike Christie <michael.christie@oracle.com> Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
-
Mike Christie authored
Make tcm_loop use the block layer cmd allocator for se_cmds instead of using the tcm_loop_cmd_cache. In the future when we can use the host tags for internal requests like TMFs we can completely kill the tcm_loop_cmd_cache. Link: https://lore.kernel.org/r/20210227170006.5077-19-michael.christie@oracle.comTested-by: Laurence Oberman <loberman@redhat.com> Reviewed-by: Christoph Hellwig <hch@lst.de> Signed-off-by: Mike Christie <michael.christie@oracle.com> Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
-
Mike Christie authored
Convert vhost-scsi to use the LIO wq cmd submission helper. Link: https://lore.kernel.org/r/20210227170006.5077-18-michael.christie@oracle.comSigned-off-by: Mike Christie <michael.christie@oracle.com> Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com> Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
-
Mike Christie authored
loop and vhost/scsi do their target cmd submission from driver workqueues. This allows them to avoid an issue where the backend may block waiting for resources like tags/requests, mem/locks, etc and that ends up blocking their entire submission path and for the case of vhost-scsi both the submission and completion path. This patch adds a helper drivers can use to submit from a LIO workqueue. This code will then be extended in the next patches to fix the plugging of backend devices. We are only converting vhost/loop initially, but the workqueue based submission will work for other drivers and have similar benefits where the main target loops will not end up blocking one some backend resource. Link: https://lore.kernel.org/r/20210227170006.5077-17-michael.christie@oracle.comTested-by: Laurence Oberman <loberman@redhat.com> Reviewed-by: Bodo Stroesser <bostroesser@gmail.com> Signed-off-by: Mike Christie <michael.christie@oracle.com> Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
-
Mike Christie authored
tcm_loop could be used like a normal block device, so we can't use GFP_KERNEL and should use GFP_NOIO. This adds a gfp_t arg to target_cmd_init_cdb() and converts the users. For every driver but loop GFP_KERNEL is kept. This will also be useful in subsequent patches where loop needs to do target_submit_prep() from interrupt context to get a ref to the se_device, and so it will need to use GFP_ATOMIC. Link: https://lore.kernel.org/r/20210227170006.5077-16-michael.christie@oracle.comTested-by: Laurence Oberman <loberman@redhat.com> Reviewed-by: Christoph Hellwig <hch@lst.de> Signed-off-by: Mike Christie <michael.christie@oracle.com> Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
-
Mike Christie authored
Convert target_submit_cmd() to do its own calls and then remove target_submit_cmd_map_sgls() since no one uses it. Link: https://lore.kernel.org/r/20210227170006.5077-15-michael.christie@oracle.comTested-by: Laurence Oberman <loberman@redhat.com> Reviewed-by: Bodo Stroesser <bostroesser@gmail.com> Signed-off-by: Mike Christie <michael.christie@oracle.com> Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
-
Mike Christie authored
target_submit_cmd() is now only for simple drivers that do their own sync during shutdown and do not use target_stop_session(). tcm_fc uses target_stop_session() to sync session shutdown with LIO core, so we use target_init_cmd(), target_submit_prep(), target_submit(), because target_init_cmd() will now detect the target_stop_session() call and return an error. Link: https://lore.kernel.org/r/20210227170006.5077-14-michael.christie@oracle.comReviewed-by: Christoph Hellwig <hch@lst.de> Signed-off-by: Mike Christie <michael.christie@oracle.com> Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
-
Mike Christie authored
target_submit_cmd_map_sgls() is being removed, so convert xen to the new submission API. This has it use target_init_cmd(), target_submit_prep(), or target_submit() because we need to have LIO core map sgls which is now done in target_submit_prep(). target_init_cmd() will never fail for xen because it does its own sync during session shutdown, so we can remove that code. Note: xen never calls target_stop_session() so target_submit_cmd_map_sgls() never failed (in the new API target_init_cmd() handles target_stop_session() being called when cmds are being submitted). If it were to have used target_stop_session() and got an error, we would have hit a refcount bug like xen and usb, because it does: if (rc < 0) { transport_send_check_condition_and_sense(se_cmd, TCM_LOGICAL_UNIT_COMMUNICATION_FAILURE, 0); transport_generic_free_cmd(se_cmd, 0); } transport_send_check_condition_and_sense() calls queue_status which calls scsiback_cmd_done->target_put_sess_cmd. We do an extra transport_generic_free_cmd() call above which would have dropped the refcount to -1 and the refcount code would spit out errors. Link: https://lore.kernel.org/r/20210227170006.5077-13-michael.christie@oracle.comReviewed-by: Christoph Hellwig <hch@lst.de> Signed-off-by: Mike Christie <michael.christie@oracle.com> Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
-
Mike Christie authored
target_submit_cmd_map_sgls() is being removed, so convert vhost-scsi to the new submission API. This has it use target_init_cmd(), target_submit_prep(), target_submit() because we need to have LIO core map sgls which is now done in target_submit_prep(), and in the next patches we will do the target_submit step from the LIO workqueue. Note: vhost-scsi never calls target_stop_session() so target_submit_cmd_map_sgls() never failed (in the new API target_init_cmd() handles target_stop_session() being called when cmds are being submitted). If it were to have used target_stop_session() and got an error, we would have hit a refcount bug like xen and usb, because it does: if (rc < 0) { transport_send_check_condition_and_sense(se_cmd, TCM_LOGICAL_UNIT_COMMUNICATION_FAILURE, 0); transport_generic_free_cmd(se_cmd, 0); } transport_send_check_condition_and_sense() calls queue_status which does transport_generic_free_cmd(), and then we do an extra transport_generic_free_cmd() call above which would have dropped the refcount to -1 and the refcount code would spit out errors. Link: https://lore.kernel.org/r/20210227170006.5077-12-michael.christie@oracle.com Cc: "Michael S. Tsirkin" <mst@redhat.com> Cc: Stefan Hajnoczi <stefanha@redhat.com> Reviewed-by: Christoph Hellwig <hch@lst.de> Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com> Signed-off-by: Mike Christie <michael.christie@oracle.com> Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
-
Mike Christie authored
target_submit_cmd() is now only for simple drivers that do their own sync during shutdown and do not use target_stop_session(). It will never return a failure, so we can remove that code from the driver. Note: Before these patches target_submit_cmd() would never return an error for usb since it does not use target_stop_session(). If it did then we would have hit a refcount error here: transport_send_check_condition_and_sense(se_cmd, TCM_UNSUPPORTED_SCSI_OPCODE, 1); transport_generic_free_cmd(&cmd->se_cmd, 0); transport_send_check_condition_and_sense() calls queue_status and the driver can sometimes do transport_generic_free_cmd() from there via uasp_status_data_cmpl(). In that case, the above transport_generic_free_cmd() would then hit a refcount error. So that other use of the above error path in the driver is also probably wrong, but someone with the hardware needs to fix that. Link: https://lore.kernel.org/r/20210227170006.5077-11-michael.christie@oracle.com Cc: Felipe Balbi <balbi@kernel.org> Reviewed-by: Christoph Hellwig <hch@lst.de> Signed-off-by: Mike Christie <michael.christie@oracle.com> Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
-
Mike Christie authored
target_submit_cmd() is now only for simple drivers that do their own sync during shutdown and do not use target_stop_session(). It will never return a failure, so we can remove that code from the driver. Link: https://lore.kernel.org/r/20210227170006.5077-10-michael.christie@oracle.com Cc: Chris Boot <bootc@bootc.net> Reviewed-by: Christoph Hellwig <hch@lst.de> Signed-off-by: Mike Christie <michael.christie@oracle.com> Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
-
Mike Christie authored
target_submit_cmd_map_sgls() is being removed, so convert loop to the new submission API. Even though loop does its own shutdown sync, this has loop use target_init_cmd()/target_submit_prep()/target_submit() since it needed to map sgls and in the next patches it will use the API to use LIO's workqueue. Link: https://lore.kernel.org/r/20210227170006.5077-9-michael.christie@oracle.comTested-by: Laurence Oberman <loberman@redhat.com> Reviewed-by: Christoph Hellwig <hch@lst.de> Signed-off-by: Mike Christie <michael.christie@oracle.com> Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
-
Mike Christie authored
target_submit_cmd() is now only for simple drivers that do their own sync during shutdown and do not use target_stop_session(). tcm_qla2xxx uses target_stop_session() to sync session shutdown with LIO core, so we use target_init_cmd()/target_submit_prep()/target_submit(), because target_init_cmd() will detect the target_stop_session() call and return an error. Link: https://lore.kernel.org/r/20210227170006.5077-8-michael.christie@oracle.com Cc: Nilesh Javali <njavali@marvell.com> Tested-by: Laurence Oberman <loberman@redhat.com> Reviewed-by: Christoph Hellwig <hch@lst.de> Reviewed-by: Himanshu Madhani <himanshu.madhani@oracle.com> Signed-off-by: Mike Christie <michael.christie@oracle.com> Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
-
Mike Christie authored
target_submit_cmd() is now only for simple drivers that do their own sync during shutdown and do not use target_stop_session(). It will never return a failure, so we can remove that code from the driver. Link: https://lore.kernel.org/r/20210227170006.5077-7-michael.christie@oracle.com Cc: Michael Cyr <mikecyr@linux.ibm.com> Reviewed-by: Christoph Hellwig <hch@lst.de> Signed-off-by: Mike Christie <michael.christie@oracle.com> Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
-
Mike Christie authored
target_submit_cmd_map_sgls() is being removed, so convert srpt to the new submission API. srpt uses target_stop_session() to sync session shutdown with LIO core, so we use target_init_cmd()/target_submit_prep()/target_submit(), because target_init_cmd() will detect the target_stop_session() call and return an error. Link: https://lore.kernel.org/r/20210227170006.5077-6-michael.christie@oracle.com Cc: Bart Van Assche <bvanassche@acm.org> Reviewed-by: Christoph Hellwig <hch@lst.de> Reviewed-by: Bart Van Assche <bvanassche@acm.org> Signed-off-by: Mike Christie <michael.christie@oracle.com> Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
-
Mike Christie authored
This breaks up target_submit_cmd_map_sgls() into 3 helpers: - target_init_cmd(): Do the basic general setup and get a refcount to the session to make sure the caller can execute the cmd. - target_submit_prep(): Do the mapping, cdb processing and get a ref to the LUN. - target_submit(): Pass the cmd to LIO core for execution. The above functions must be used by drivers that either: 1. Rely on LIO for session shutdown synchronization by calling target_stop_session(). 2. Need to map sgls. When the next patches are applied then simple drivers that do not need the extra functionality above can use target_submit_cmd() and not worry about failures being returned and how to handle them, since many drivers were getting this wrong and would have hit refcount bugs. Also, by breaking target_submit_cmd_map_sgls() up into these 3 helper functions, we can allow the later patches to do the init/prep from interrupt context and then do the submission from a workqueue. Link: https://lore.kernel.org/r/20210227170006.5077-5-michael.christie@oracle.com Cc: Bart Van Assche <bvanassche@acm.org> Cc: Juergen Gross <jgross@suse.com> Cc: Hannes Reinecke <hare@suse.de> Cc: Nilesh Javali <njavali@marvell.com> Cc: Michael Cyr <mikecyr@linux.ibm.com> Cc: Chris Boot <bootc@bootc.net> Cc: Felipe Balbi <balbi@kernel.org> Cc: "Michael S. Tsirkin" <mst@redhat.com> Cc: Stefan Hajnoczi <stefanha@redhat.com> Tested-by: Laurence Oberman <loberman@redhat.com> Reviewed-by: Christoph Hellwig <hch@lst.de> Reviewed-by: Himanshu Madhani <himanshu.madhani@oracle.com> Signed-off-by: Mike Christie <michael.christie@oracle.com> Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
-
Mike Christie authored
Rename transport_init_se_cmd() to __target_init_cmd() to reflect that it is more of an internal function that drivers should normally not use and because we are going to add a new init function in the next patches. Link: https://lore.kernel.org/r/20210227170006.5077-4-michael.christie@oracle.comReviewed-by: Christoph Hellwig <hch@lst.de> Reviewed-by: Himanshu Madhani <himanshu.madhani@oracle.com> Signed-off-by: Mike Christie <michael.christie@oracle.com> Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
-
Mike Christie authored
The kref_get_unless_zero() use in target_get_sess_cmd() was added in: commit 1b4c59b7 ("target: fix potential race window in target_sess_cmd_list_waiting()")' but it does not seem to do anything. The original patch might have thought we could have added the cmd to the sess_wait_list and then target_wait_for_sess_cmds could do a put before target_get_sess_cmd did its get. That wouldn't happen because we do the get first then grab the sess lock and put it on the list. It is also not needed now, because the sess_cmd_list does not exist anymore and we instead wait on the session cmd_count. The other problem with the commit is that several target_submit_cmd_map_sgls()/target_submit_cmd() callers do not handle the error case properly if it were to ever happen. These drivers think they have their normal refcount on the cmd and in many cases do a transport_generic_free_cmd() plus target_put_sess_cmd() so they would have fired off the refcount WARN/BUGs. This patch just changes the kref_get_unless_zero() to kref_get(). Link: https://lore.kernel.org/r/20210227170006.5077-3-michael.christie@oracle.comTested-by: Laurence Oberman <loberman@redhat.com> Reviewed-by: Christoph Hellwig <hch@lst.de> Reviewed-by: Himanshu Madhani <himanshu.madhani@oracle.com> Signed-off-by: Mike Christie <michael.christie@oracle.com> Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
-
Mike Christie authored
Prepare to split target_submit_cmd_map_sgls() so the initialization and submission part can be called at different times. If the init part fails we can reference the t_task_cdb early in some of the logging and tracing code. Move it to transport_init_se_cmd() so we don't hit NULL pointer crashes. Link: https://lore.kernel.org/r/20210227170006.5077-2-michael.christie@oracle.comTested-by: Laurence Oberman <loberman@redhat.com> Reviewed-by: Christoph Hellwig <hch@lst.de> Reviewed-by: Himanshu Madhani <himanshu.madhani@oracle.com> Signed-off-by: Mike Christie <michael.christie@oracle.com> Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
-
Ming Lei authored
SCSI currently uses an atomic variable to track queue depth for each attached device. The queue depth depends on many factors such as transport type and device implementation. In addition, the SCSI device queue depth is not a static entity but changes over time as a result of congestion management. While blk-mq currently tracks queue depth for each hctx, it can't easily be changed to accommodate the SCSI per-device requirement. The current approach of using an atomic variable doesn't scale well when there are lots of CPU cores and the disk is very fast. IOPS can be substantially impacted by the atomic in the hot path. Replace the atomic variable sdev->device_busy with an sbitmap for tracking the SCSI device queue depth. It has been observed that IOPS is improved ~30% by this patchset in the following test: 1) test machine(32 logical CPU cores) Thread(s) per core: 2 Core(s) per socket: 8 Socket(s): 2 NUMA node(s): 2 Model name: Intel(R) Xeon(R) Silver 4110 CPU @ 2.10GHz 2) setup scsi_debug: modprobe scsi_debug virtual_gb=128 max_luns=1 submit_queues=32 delay=0 max_queue=256 3) fio script: fio --rw=randread --size=128G --direct=1 --ioengine=libaio --iodepth=2048 \ --numjobs=32 --bs=4k --group_reporting=1 --group_reporting=1 --runtime=60 \ --loops=10000 --name=job1 --filename=/dev/sdN [mkp: fix device_busy reference in mpt3sas] Link: https://lore.kernel.org/r/20210122023317.687987-14-ming.lei@redhat.com Link: https://lore.kernel.org/linux-block/20200119071432.18558-6-ming.lei@redhat.com/ Cc: Omar Sandoval <osandov@fb.com> Cc: Kashyap Desai <kashyap.desai@broadcom.com> Cc: Sumanesh Samanta <sumanesh.samanta@broadcom.com> Cc: Ewan D. Milne <emilne@redhat.com> Cc: Hannes Reinecke <hare@suse.de> Tested-by: Sumanesh Samanta <sumanesh.samanta@broadcom.com> Reviewed-by: Hannes Reinecke <hare@suse.de> Signed-off-by: Ming Lei <ming.lei@redhat.com> Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
-
Ming Lei authored
Limit SCSI device's queue depth to max(host->can_queue, 1024) in scsi_change_queue_depth(). 1024 is big enough for saturating current fast SCSI LUN(SSD or RAID volume on multiple SSDs). Also single hardware queue depth is usually enough for saturating single LUN because per-core performance is often considered in storage design. This patch is needed for replacing sdev->device_busy with sbitmap which has to be pre-allocated with reasonable max depth. Link: https://lore.kernel.org/r/20210122023317.687987-13-ming.lei@redhat.com Cc: Omar Sandoval <osandov@fb.com> Cc: Kashyap Desai <kashyap.desai@broadcom.com> Cc: Sumanesh Samanta <sumanesh.samanta@broadcom.com> Cc: Ewan D. Milne <emilne@redhat.com> Tested-by: Sumanesh Samanta <sumanesh.samanta@broadcom.com> Reviewed-by: Hannes Reinecke <hare@suse.de> Signed-off-by: Ming Lei <ming.lei@redhat.com> Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
-
Ming Lei authored
Add scsi_device_busy() helper to prepare drivers for tracking device queue depth via sbitmap_queue. Link: https://lore.kernel.org/r/20210122023317.687987-12-ming.lei@redhat.com Cc: Omar Sandoval <osandov@fb.com> Cc: Kashyap Desai <kashyap.desai@broadcom.com> Cc: Sumanesh Samanta <sumanesh.samanta@broadcom.com> Cc: Ewan D. Milne <emilne@redhat.com> Tested-by: Sumanesh Samanta <sumanesh.samanta@broadcom.com> Reviewed-by: Hannes Reinecke <hare@suse.de> Signed-off-by: Ming Lei <ming.lei@redhat.com> Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
-
Kashyap Desai authored
Use local tracking of per-sdev outstanding command since sdev_busy in SCSI mid layer is improved for performance reason using sbitmap (earlier it was atomic variable). Link: https://lore.kernel.org/r/20210122023317.687987-11-ming.lei@redhat.com Cc: Omar Sandoval <osandov@fb.com> Cc: Kashyap Desai <kashyap.desai@broadcom.com> Cc: Sumanesh Samanta <sumanesh.samanta@broadcom.com> Cc: Ewan D. Milne <emilne@redhat.com> Reviewed-by: Hannes Reinecke <hare@suse.de> Signed-off-by: Kashyap Desai <kashyap.desai@broadcom.com> Signed-off-by: Ming Lei <ming.lei@redhat.com> Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
-
Ming Lei authored
The following three fields of scsi_host_template are referenced in the SCSI I/O submission hot path. Put them together in one cacheline: - cmd_size - queuecommand - commit_rqs Link: https://lore.kernel.org/r/20210122023317.687987-10-ming.lei@redhat.com Cc: Omar Sandoval <osandov@fb.com> Cc: Kashyap Desai <kashyap.desai@broadcom.com> Cc: Sumanesh Samanta <sumanesh.samanta@broadcom.com> Cc: Ewan D. Milne <emilne@redhat.com> Cc: Hannes Reinecke <hare@suse.de> Tested-by: Sumanesh Samanta <sumanesh.samanta@broadcom.com> Reviewed-by: Hannes Reinecke <hare@suse.de> Reviewed-by: Christoph Hellwig <hch@lst.de> Signed-off-by: Ming Lei <ming.lei@redhat.com> Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
-
Ming Lei authored
SCSI uses a global atomic variable to track queue depth for each LUN/request queue. This doesn't scale well when there are lots of CPU cores and the disk is very fast. It has been observed that IOPS is affected a lot by tracking queue depth via sdev->device_busy in the I/O path. Return budget token from .get_budget callback. The budget token can be passed to driver so that we can replace the atomic variable with sbitmap_queue and alleviate the scaling problems that way. Link: https://lore.kernel.org/r/20210122023317.687987-9-ming.lei@redhat.com Cc: Omar Sandoval <osandov@fb.com> Cc: Kashyap Desai <kashyap.desai@broadcom.com> Cc: Sumanesh Samanta <sumanesh.samanta@broadcom.com> Cc: Ewan D. Milne <emilne@redhat.com> Tested-by: Sumanesh Samanta <sumanesh.samanta@broadcom.com> Reviewed-by: Hannes Reinecke <hare@suse.de> Signed-off-by: Ming Lei <ming.lei@redhat.com> Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
-
Ming Lei authored
Since SCSI is the only driver which requires dispatch budget move the token from struct request to struct scsi_cmnd. Link: https://lore.kernel.org/r/20210122023317.687987-8-ming.lei@redhat.com Cc: Omar Sandoval <osandov@fb.com> Cc: Kashyap Desai <kashyap.desai@broadcom.com> Cc: Sumanesh Samanta <sumanesh.samanta@broadcom.com> Cc: Ewan D. Milne <emilne@redhat.com> Cc: Hannes Reinecke <hare@suse.de> Tested-by: Sumanesh Samanta <sumanesh.samanta@broadcom.com> Reviewed-by: Hannes Reinecke <hare@suse.de> Signed-off-by: Ming Lei <ming.lei@redhat.com> Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
-
Ming Lei authored
Move code for calculating default shift into a public helper which can be used by SCSI. Link: https://lore.kernel.org/r/20210122023317.687987-7-ming.lei@redhat.com Cc: Omar Sandoval <osandov@fb.com> Cc: Kashyap Desai <kashyap.desai@broadcom.com> Cc: Sumanesh Samanta <sumanesh.samanta@broadcom.com> Cc: Ewan D. Milne <emilne@redhat.com> Tested-by: Sumanesh Samanta <sumanesh.samanta@broadcom.com> Reviewed-by: Hannes Reinecke <hare@suse.de> Signed-off-by: Ming Lei <ming.lei@redhat.com> Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
-
Ming Lei authored
SCSI's .device_busy will be converted to sbitmap and sbitmap_weight is needed. Export the helper. The only existing user of sbitmap_weight() uses it to find out how many bits are set and not cleared. Align sbitmap_weight() meaning with this usage model. Link: https://lore.kernel.org/r/20210122023317.687987-6-ming.lei@redhat.com Cc: Omar Sandoval <osandov@fb.com> Cc: Kashyap Desai <kashyap.desai@broadcom.com> Cc: Sumanesh Samanta <sumanesh.samanta@broadcom.com> Cc: Ewan D. Milne <emilne@redhat.com> Tested-by: Sumanesh Samanta <sumanesh.samanta@broadcom.com> Reviewed-by: Hannes Reinecke <hare@suse.de> Signed-off-by: Ming Lei <ming.lei@redhat.com> Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
-
Ming Lei authored
Allocation hint should have belonged to sbitmap. Also, when sbitmap's depth is high and there is no need to use mulitple wakeup queues, user can benefit from percpu allocation hint too. Move allocation hint into sbitmap, then SCSI device queue can benefit from allocation hint when converting to plain sbitmap. Convert vhost/scsi.c to use sbitmap allocation with percpu alloc hint. This is more efficient than the previous approach. Link: https://lore.kernel.org/r/20210122023317.687987-5-ming.lei@redhat.com Cc: Omar Sandoval <osandov@fb.com> Cc: Kashyap Desai <kashyap.desai@broadcom.com> Cc: Sumanesh Samanta <sumanesh.samanta@broadcom.com> Cc: Ewan D. Milne <emilne@redhat.com> Cc: Mike Christie <michael.christie@oracle.com> Cc: virtualization@lists.linux-foundation.org Tested-by: Sumanesh Samanta <sumanesh.samanta@broadcom.com> Reviewed-by: Hannes Reinecke <hare@suse.de> Signed-off-by: Ming Lei <ming.lei@redhat.com> Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
-
Ming Lei authored
Add helpers for updating allocation hint so that we can avoid duplicate code. Prepare for moving allocation hint into sbitmap. Link: https://lore.kernel.org/r/20210122023317.687987-4-ming.lei@redhat.com Cc: Omar Sandoval <osandov@fb.com> Cc: Kashyap Desai <kashyap.desai@broadcom.com> Cc: Sumanesh Samanta <sumanesh.samanta@broadcom.com> Cc: Ewan D. Milne <emilne@redhat.com> Cc: Hannes Reinecke <hare@suse.de> Tested-by: Sumanesh Samanta <sumanesh.samanta@broadcom.com> Reviewed-by: Hannes Reinecke <hare@suse.de> Signed-off-by: Ming Lei <ming.lei@redhat.com> Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
-
Ming Lei authored
Currently the allocation round_robin info is maintained by sbitmap_queue. However, bit allocation really belongs to sbitmap. Move it there. Link: https://lore.kernel.org/r/20210122023317.687987-3-ming.lei@redhat.com Cc: Omar Sandoval <osandov@fb.com> Cc: Kashyap Desai <kashyap.desai@broadcom.com> Cc: Sumanesh Samanta <sumanesh.samanta@broadcom.com> Cc: Ewan D. Milne <emilne@redhat.com> Cc: Hannes Reinecke <hare@suse.de> Cc: virtualization@lists.linux-foundation.org Tested-by: Sumanesh Samanta <sumanesh.samanta@broadcom.com> Reviewed-by: Hannes Reinecke <hare@suse.de> Signed-off-by: Ming Lei <ming.lei@redhat.com> Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
-
Ming Lei authored
No one uses this helper any more, so kill it. Link: https://lore.kernel.org/r/20210122023317.687987-2-ming.lei@redhat.com Cc: Omar Sandoval <osandov@fb.com> Cc: Kashyap Desai <kashyap.desai@broadcom.com> Cc: Sumanesh Samanta <sumanesh.samanta@broadcom.com> Cc: Ewan D. Milne <emilne@redhat.com> Cc: Hannes Reinecke <hare@suse.de> Tested-by: Sumanesh Samanta <sumanesh.samanta@broadcom.com> Reviewed-by: Hannes Reinecke <hare@suse.de> Reviewed-by: Johannes Thumshirn <johannes.thumshirn@wdc.com> Signed-off-by: Ming Lei <ming.lei@redhat.com> Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
-
Adrian Hunter authored
An enabled user-specified exception event that does not clear quickly will repeatedly cause the handler to run. That could unduly disturb the driver behaviour being tested or debugged. To prevent that add debugfs file exception_event_rate_limit_ms. When a exception event happens, it is disabled, and then after a period of time (default 20ms) the exception event is enabled again. Note that if the driver also has that exception event enabled, it will not be disabled. Link: https://lore.kernel.org/r/20210209062437.6954-5-adrian.hunter@intel.comAcked-by: Bean Huo <beanhuo@micron.com> Signed-off-by: Adrian Hunter <adrian.hunter@intel.com> Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
-
Adrian Hunter authored
Allow users to enable specific exception events via debugfs. The bits enabled by the driver ee_drv_ctrl are separated from the bits enabled by the user ee_usr_ctrl. The control mask ee_mask_ctrl is the logical-or of those two. A mutex is needed to ensure that the masks match what was written to the device. Link: https://lore.kernel.org/r/20210209062437.6954-4-adrian.hunter@intel.comAcked-by: Bean Huo <beanhuo@micron.com> Signed-off-by: Adrian Hunter <adrian.hunter@intel.com> Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
-
Adrian Hunter authored
For readability and completeness, add exception event definitions. Link: https://lore.kernel.org/r/20210209062437.6954-3-adrian.hunter@intel.comReviewed-by: Bean Huo <beanhuo@micron.com> Signed-off-by: Adrian Hunter <adrian.hunter@intel.com> Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
-
Adrian Hunter authored
Currently, exception event status can be read from wExceptionEventStatus attribute (sysfs file attributes/exception_event_status under the UFS host controller device directory). Polling that attribute to track UFS exception events is impractical, so add a tracepoint to track exception events for testing and debugging purposes. Note, by the time the exception event status is read, the exception event may have cleared, so the value can be zero - see example below. Note also, only enabled exception events can be reported. A subsequent patch adds the ability for users to enable selected exception events via debugfs. Example with driver instrumented to enable all exception events: # echo 1 > /sys/kernel/debug/tracing/events/ufs/ufshcd_exception_event/enable ... do some I/O ... # cat /sys/kernel/debug/tracing/trace # tracer: nop # # entries-in-buffer/entries-written: 3/3 #P:5 # # _-----=> irqs-off # / _----=> need-resched # | / _---=> hardirq/softirq # || / _--=> preempt-depth # ||| / delay # TASK-PID CPU# |||| TIMESTAMP FUNCTION # | | | |||| | | kworker/2:2-173 [002] .... 731.486419: ufshcd_exception_event: 0000:00:12.5: status 0x0 kworker/2:2-173 [002] .... 732.608918: ufshcd_exception_event: 0000:00:12.5: status 0x4 kworker/2:2-173 [002] .... 732.609312: ufshcd_exception_event: 0000:00:12.5: status 0x4 Link: https://lore.kernel.org/r/20210209062437.6954-2-adrian.hunter@intel.comReviewed-by: Avri Altman <avri.altman@wdc.com> Reviewed-by: Bean Huo <beanhuo@micron.com> Signed-off-by: Adrian Hunter <adrian.hunter@intel.com> Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
-