Bart Van Assche [Mon, 9 Aug 2021 23:03:15 +0000 (16:03 -0700)]
scsi: NCR5380: Use sc_data_direction instead of rq_data_dir()
This patch prepares for the removal of the request pointer from struct
scsi_cmnd and does not change any functionality.
Link: https://lore.kernel.org/r/20210809230355.8186-13-bvanassche@acm.org Cc: Michael Schmitz <schmitzmic@gmail.com> Suggested-by: Finn Thain <fthain@linux-m68k.org> Acked-by: Finn Thain <fthain@linux-m68k.org> Reviewed-by: Hannes Reinecke <hare@suse.de> Signed-off-by: Bart Van Assche <bvanassche@acm.org> Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
Bart Van Assche [Mon, 9 Aug 2021 23:03:13 +0000 (16:03 -0700)]
scsi: zfcp: Use scsi_cmd_to_rq() instead of scsi_cmnd.request
Prepare for removal of the request pointer by using scsi_cmd_to_rq()
instead. This patch does not change any functionality.
Link: https://lore.kernel.org/r/20210809230355.8186-11-bvanassche@acm.org Acked-by: Benjamin Block <bblock@linux.ibm.com> Reviewed-by: Hannes Reinecke <hare@suse.de> Signed-off-by: Bart Van Assche <bvanassche@acm.org> Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
Bart Van Assche [Mon, 9 Aug 2021 23:03:11 +0000 (16:03 -0700)]
scsi: RDMA/iser: Use scsi_cmd_to_rq() instead of scsi_cmnd.request
Prepare for removal of the request pointer by using scsi_cmd_to_rq()
instead. This patch does not change any functionality.
Link: https://lore.kernel.org/r/20210809230355.8186-9-bvanassche@acm.org Reviewed-by: Max Gurtovoy <mgurtovoy@nvidia.com> Reviewed-by: Sagi Grimberg <sagi@grimberg.me> Reviewed-by: Hannes Reinecke <hare@suse.de> Signed-off-by: Bart Van Assche <bvanassche@acm.org> Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
Bart Van Assche [Mon, 9 Aug 2021 23:03:10 +0000 (16:03 -0700)]
scsi: ata: Use scsi_cmd_to_rq() instead of scsi_cmnd.request
Prepare for removal of the request pointer by using scsi_cmd_to_rq()
instead. This patch does not change any functionality.
Link: https://lore.kernel.org/r/20210809230355.8186-8-bvanassche@acm.org Cc: Jens Axboe <axboe@kernel.dk> Cc: Christoph Hellwig <hch@lst.de> Cc: Hannes Reinecke <hare@suse.de> Cc: Ming Lei <ming.lei@redhat.com> Reviewed-by: Hannes Reinecke <hare@suse.de> Signed-off-by: Bart Van Assche <bvanassche@acm.org> Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
Bart Van Assche [Mon, 9 Aug 2021 23:03:09 +0000 (16:03 -0700)]
scsi: scsi_transport_spi: Use scsi_cmd_to_rq() instead of scsi_cmnd.request
Prepare for removal of the request pointer by using scsi_cmd_to_rq()
instead. This patch does not change any functionality.
Link: https://lore.kernel.org/r/20210809230355.8186-7-bvanassche@acm.org Cc: Christoph Hellwig <hch@lst.de> Cc: Hannes Reinecke <hare@suse.de> Cc: Ming Lei <ming.lei@redhat.com> Reviewed-by: Hannes Reinecke <hare@suse.de> Signed-off-by: Bart Van Assche <bvanassche@acm.org> Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
Bart Van Assche [Mon, 9 Aug 2021 23:03:08 +0000 (16:03 -0700)]
scsi: scsi_transport_fc: Use scsi_cmd_to_rq() instead of scsi_cmnd.request
Prepare for removal of the request pointer by using scsi_cmd_to_rq()
instead. This patch does not change any functionality.
Link: https://lore.kernel.org/r/20210809230355.8186-6-bvanassche@acm.org Cc: Christoph Hellwig <hch@lst.de> Cc: Hannes Reinecke <hare@suse.de> Cc: Ming Lei <ming.lei@redhat.com> Reviewed-by: Hannes Reinecke <hare@suse.de> Signed-off-by: Bart Van Assche <bvanassche@acm.org> Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
Bart Van Assche [Mon, 9 Aug 2021 23:03:07 +0000 (16:03 -0700)]
scsi: sr: Use scsi_cmd_to_rq() instead of scsi_cmnd.request
Prepare for removal of the request pointer by using scsi_cmd_to_rq()
instead. This patch does not change any functionality.
Link: https://lore.kernel.org/r/20210809230355.8186-5-bvanassche@acm.org Cc: Christoph Hellwig <hch@lst.de> Cc: Hannes Reinecke <hare@suse.de> Cc: Ming Lei <ming.lei@redhat.com> Reviewed-by: Hannes Reinecke <hare@suse.de> Signed-off-by: Bart Van Assche <bvanassche@acm.org> Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
Bart Van Assche [Mon, 9 Aug 2021 23:03:06 +0000 (16:03 -0700)]
scsi: sd: Use scsi_cmd_to_rq() instead of scsi_cmnd.request
Prepare for removal of the request pointer by using scsi_cmd_to_rq()
instead. This patch does not change any functionality.
Link: https://lore.kernel.org/r/20210809230355.8186-4-bvanassche@acm.org Cc: Christoph Hellwig <hch@lst.de> Cc: Hannes Reinecke <hare@suse.de> Cc: Ming Lei <ming.lei@redhat.com> Reviewed-by: Hannes Reinecke <hare@suse.de> Signed-off-by: Bart Van Assche <bvanassche@acm.org> Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
Bart Van Assche [Mon, 9 Aug 2021 23:03:05 +0000 (16:03 -0700)]
scsi: core: Use scsi_cmd_to_rq() instead of scsi_cmnd.request
Prepare for removal of the request pointer by using scsi_cmd_to_rq()
instead. Cast away constness where necessary when passing a SCSI command
pointer to scsi_cmd_to_rq(). This patch does not change any functionality.
Link: https://lore.kernel.org/r/20210809230355.8186-3-bvanassche@acm.org Cc: Christoph Hellwig <hch@lst.de> Cc: Hannes Reinecke <hare@suse.de> Cc: Ming Lei <ming.lei@redhat.com> Reviewed-by: Hannes Reinecke <hare@suse.de> Signed-off-by: Bart Van Assche <bvanassche@acm.org> Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
Bart Van Assche [Mon, 9 Aug 2021 23:03:04 +0000 (16:03 -0700)]
scsi: core: Introduce the scsi_cmd_to_rq() function
The 'request' member of struct scsi_cmnd is superfluous. The struct request
and struct scsi_cmnd data structures are adjacent and hence the request
pointer can be derived easily from a scsi_cmnd pointer. Introduce a helper
function that performs that conversion in a type-safe way. This patch is
the first step towards removing the request member from struct
scsi_cmnd. Making that change has the following advantages:
- This is a performance optimization since adding an offset to a pointer
takes less time than dereferencing a pointer.
- struct scsi_cmnd becomes smaller.
Link: https://lore.kernel.org/r/20210809230355.8186-2-bvanassche@acm.org Cc: Christoph Hellwig <hch@lst.de> Cc: Hannes Reinecke <hare@suse.de> Cc: Ming Lei <ming.lei@redhat.com> Reviewed-by: Hannes Reinecke <hare@suse.de> Signed-off-by: Bart Van Assche <bvanassche@acm.org> Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
Avri Altman [Sun, 8 Aug 2021 09:00:24 +0000 (12:00 +0300)]
scsi: ufs: ufshpb: Do not report victim error in HCM
In host control mode, eviction is perceived as an extreme measure. There
are several conditions that both the entering and exiting regions should
meet, so that eviction will take place.
The common case however, is that those conditions are rarely met, so it is
normal that the act of eviction fails. Therefore, do not report an error
in host control mode if eviction fails.
Avri Altman [Sun, 8 Aug 2021 09:00:23 +0000 (12:00 +0300)]
scsi: ufs: ufshpb: Verify that 'num_inflight_map_req' is non-negative
'num_inflight_map_req' should not be negative. It is incremented and
decremented without any protection, allowing it theoretically to be
negative, should some weird unbalanced count occur.
Verify that the those calls are properly serialized.
Avri Altman [Sun, 8 Aug 2021 09:00:22 +0000 (12:00 +0300)]
scsi: ufs: ufshpb: Use a correct max multi chunk
In HPB2.0, if pre_req_min_tr_len < transfer_len < pre_req_max_tr_len, the
driver is expected to send a HPB-WRITE-BUFFER companion to HPB-READ.
The upper bound should fit into a single byte, regardless of bMAX_
DATA_SIZE_FOR_HPB_SINGLE_CMD which being an attribute (u32) can be
significantly larger.
To further illustrate the issue, consider the following scenario:
- SCSI_DEFAULT_MAX_SECTORS is 1024 limiting the I/O chunks to 512KB
- The OEM changes scsi_host_template .max_sectors to be 2048 which allows
for 1MB requests: transfer_len = 256
Avri Altman [Sun, 8 Aug 2021 09:00:21 +0000 (12:00 +0300)]
scsi: ufs: ufshpb: Rewind the read timeout on every read
The purpose of the "cold"-timer is not to hang-on to active regions with no
reads. Therefore the read timeout should be rewound on every read, and not
just when the region is activated.
Adrian Hunter [Fri, 6 Aug 2021 13:04:41 +0000 (16:04 +0300)]
scsi: ufshcd: Fix device links when BOOT WLUN fails to probe
Managed device links are deleted by device_del(). However it is possible to
add a device link to a consumer before device_add(), and then discovering
an error prevents the device from being used. In that case normally
references to the device would be dropped and the device would be deleted.
However the device link holds a reference to the device, so the device link
and device remain indefinitely (unless the supplier is deleted).
For UFSHCD, if a LUN fails to probe (e.g. absent BOOT WLUN), the device
will not have been registered but can still have a device link holding a
reference to the device. The unwanted device link will prevent runtime
suspend indefinitely.
Amend device link removal to accept removal of a link with an unregistered
consumer device (suggested by Rafael), and fix UFSHCD by explicitly
deleting the device link when SCSI destroys the SCSI device.
Link: https://lore.kernel.org/r/a1c9bac8-b560-b662-f0aa-58c7e000cbbd@intel.com Fixes: b294ff3e3449 ("scsi: ufs: core: Enable power management for wlun") Reviewed-by: Rafael J. Wysocki <rafael@kernel.org> Signed-off-by: Adrian Hunter <adrian.hunter@intel.com> Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
Colin Ian King [Fri, 6 Aug 2021 14:43:01 +0000 (15:43 +0100)]
scsi: ufs: Fix unsigned int compared with less than zero
Variable 'tag' is currently an unsigned int and is being compared to less
than zero, this check is always false. Fix this by making 'tag' an int.
Link: https://lore.kernel.org/r/20210806144301.19864-1-colin.king@canonical.com Fixes: 4728ab4a8e64 ("scsi: ufs: Remove ufshcd_valid_tag()") Reviewed-by: Bart Van Assche <bvanassche@acm.org> Signed-off-by: Colin Ian King <colin.king@canonical.com> Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
Addresses-Coverity: ("Macro compares unsigned to 0")
Similarly to AHCI, introduce the device sysfs attribute
sas_ncq_prio_supported to advertise if a SATA device supports the NCQ
priority feature. Without this new attribute, the user can only discover if
a SATA device supports NCQ priority by trying to enable the feature use
with the sas_ncq_prio_enable sysfs device attribute, which fails when the
device does not support high prioity commands.
scsi: mpt3sas: Use firmware recommended queue depth
Currently, the mpt3sas driver sets the default queue depth based on the
physical interface of the attached device:
- SAS : 254
- SATA: 32
- NVMe: 128
The IOC firmware provides a recommended queue depth for each device through
SAS IO Unit Page1 for SAS/SATA and PCIe IO Unit Page 1 for NVMe devices.
If the host sets the queue depth greater than the firmware recommended
value, then the IOC places the I/Os above the recommended queue depth in an
internal pending queue. This consumes outstanding host-credit/resources,
thereby leading to potential starvation of other devices.
To avoid this, use the device depth recommended by the IOC firmware.
Enable the driver to work in non-IRQ mode, i.e. there will not be any MSI-X
vectors associated with queues dedicated to polling. The IOC hardware is
single submission queue and multiple reply queue. However, using the shared
host tagset support it is possible to simulate multiple hardware queues.
When poll_queues are enabled through the module parameter, the driver will
allocate extra reply queues without an MSI-X association. All I/O
completion on these queues will be done through the iopoll interface.
For Micron UFS devices the L2P entry need to be byteswapped before sending
an HPB READ command to the UFS device. Add the quirk
UFS_DEVICE_QUIRK_SWAP_L2P_ENTRY_FOR_HPB_READ to address this.
Colin Ian King [Wed, 4 Aug 2021 13:13:44 +0000 (14:13 +0100)]
scsi: qla2xxx: Remove redundant initialization of variable num_cnt
The variable num_cnt is being initialized with a value that is never read,
it is being updated later on. The assignment is redundant and can be
removed.
Link: https://lore.kernel.org/r/20210804131344.112635-1-colin.king@canonical.com Reviewed-by: Himanshu Madhani <himanshu.madhani@oracle.com> Signed-off-by: Colin Ian King <colin.king@canonical.com> Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
Addresses-Coverity: ("Unused value")
David Disseldorp [Wed, 28 Jul 2021 11:53:53 +0000 (13:53 +0200)]
scsi: target: sbp: Drop incorrect ASC/ASCQ usage
The se_cmd scsi_asc and scsi_ascq members are only used for tracking ALUA
SCSI sense detail between target_core_alua and translate_sense_reason(), so
they're effectively always zero here.
Link: https://lore.kernel.org/r/20210728115353.2396-3-ddiss@suse.de Cc: Chris Boot <bootc@bootc.net> Reviewed-by: Mike Christie <michael.christie@oracle.com> Signed-off-by: David Disseldorp <ddiss@suse.de> Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
David Disseldorp [Wed, 28 Jul 2021 11:53:52 +0000 (13:53 +0200)]
scsi: target: core: Avoid using lun_tg_pt_gp after unlock
core_alua_state_lba_dependent() currently uses lun->lun_tg_pt_gp without
holding the lun_tg_pt_gp_lock. The lock is taken in the caller, so obtain
the needed tg_pt_gp_id there instead.
Link: https://lore.kernel.org/r/20210728115353.2396-2-ddiss@suse.de Cc: Hannes Reinecke <hare@suse.de> Reviewed-by: Mike Christie <michael.christie@oracle.com> Signed-off-by: David Disseldorp <ddiss@suse.de> Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
Bodo Stroesser [Tue, 13 Jul 2021 17:50:21 +0000 (19:50 +0200)]
scsi: target: tcmu: Add new feature KEEP_BUF
When running command pipelining for WRITE direction commands (e.g. tape
device write), userspace sends cmd completion to cmd ring before processing
write data. In that case userspace has to copy data before sending
completion, because cmd completion also implicitly releases the data buffer
in data area.
The new feature KEEP_BUF allows userspace to optionally keep the buffer
after completion by setting new bit TCMU_UFLAG_KEEP_BUF in
tcmu_cmd_entry_hdr->uflags. In that case buffer has to be released
explicitly by writing the cmd_id to new action item free_kept_buf.
All kept buffers are released during reset_ring and if userspace closes uio
device (tcmu_release).
scsi: ufs: Retry aborted SCSI commands instead of completing these successfully
Neither SAM nor the UFS standard require that the UFS controller fills in
the completion status of commands that have been aborted (LUN RESET aborts
pending commands). Hence do not rely on the completion status provided by
the UFS controller for aborted commands but instead ask the SCSI core to
retry SCSI commands that have been aborted.
Link: https://lore.kernel.org/r/20210722033439.26550-18-bvanassche@acm.org Cc: Adrian Hunter <adrian.hunter@intel.com> Cc: Stanley Chu <stanley.chu@mediatek.com> Cc: Can Guo <cang@codeaurora.org> Cc: Asutosh Das <asutoshd@codeaurora.org> Cc: Avri Altman <avri.altman@wdc.com> Reviewed-by: Bean Huo <beanhuo@micron.com> Signed-off-by: Bart Van Assche <bvanassche@acm.org> Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
scsi: ufs: Synchronize SCSI and UFS error handling
Use the SCSI error handler instead of a custom error handling strategy.
This change reduces the number of potential races in the UFS drivers since
the UFS error handler and the SCSI error handler no longer run
concurrently.
Link: https://lore.kernel.org/r/20210722033439.26550-17-bvanassche@acm.org Cc: Adrian Hunter <adrian.hunter@intel.com> Cc: Stanley Chu <stanley.chu@mediatek.com> Cc: Can Guo <cang@codeaurora.org> Cc: Asutosh Das <asutoshd@codeaurora.org> Cc: Avri Altman <avri.altman@wdc.com> Tested-by: Bean Huo <beanhuo@micron.com> Reviewed-by: Bean Huo <beanhuo@micron.com> Signed-off-by: Bart Van Assche <bvanassche@acm.org> Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
Clearing a unit attention synchronously from inside the UFS error handler
may trigger the following deadlock:
- ufshcd_err_handler() calls ufshcd_err_handling_unprepare() and the
latter function calls ufshcd_clear_ua_wluns().
- ufshcd_clear_ua_wluns() submits a REQUEST SENSE command and that command
activates the SCSI error handler.
- The SCSI error handler calls ufshcd_host_reset_and_restore().
- ufshcd_host_reset_and_restore() executes the following code:
ufshcd_schedule_eh_work(hba); flush_work(&hba->eh_work);
This sequence results in a deadlock (circular wait). Fix this by requesting
sense data asynchronously.
Link: https://lore.kernel.org/r/20210722033439.26550-16-bvanassche@acm.org Cc: Adrian Hunter <adrian.hunter@intel.com> Cc: Stanley Chu <stanley.chu@mediatek.com> Cc: Can Guo <cang@codeaurora.org> Cc: Asutosh Das <asutoshd@codeaurora.org> Cc: Avri Altman <avri.altman@wdc.com> Reviewed-by: Bean Huo <beanhuo@micron.com> Signed-off-by: Bart Van Assche <bvanassche@acm.org> Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
- Return FAILED instead of SUCCESS if the abort handler notices that a
SCSI command has already been completed. Returning SUCCESS in this case
triggers a use-after-free and may trigger a kernel crash.
- Fix the code for aborting SCSI commands submitted to a WLUN.
The current approach for aborting SCSI commands that have been submitted to
a WLUN and that timed out is as follows:
- Report to the SCSI core that the command has completed successfully.
Let the block layer free any data buffers associated with the command.
- Mark the command as outstanding in 'outstanding_reqs'.
- If the block layer tries to reuse the tag associated with the aborted
command, busy-wait until the tag is freed.
This approach can result in:
- Memory corruption if the controller accesses the data buffer after the
block layer has freed the associated data buffers.
- A race condition if ufshcd_queuecommand() or ufshcd_exec_dev_cmd()
checks the bit that corresponds to an aborted command in
'outstanding_reqs' after it has been cleared and before it is reset.
- High energy consumption if ufshcd_queuecommand() repeatedly returns
SCSI_MLQUEUE_HOST_BUSY.
Fix this by reporting to the SCSI error handler that aborting a SCSI
command failed if the SCSI command was submitted to a WLUN.
Link: https://lore.kernel.org/r/20210722033439.26550-15-bvanassche@acm.org Fixes: 7a7e66c65d41 ("scsi: ufs: Fix a race condition between ufshcd_abort() and eh_work()") Cc: Adrian Hunter <adrian.hunter@intel.com> Cc: Stanley Chu <stanley.chu@mediatek.com> Cc: Can Guo <cang@codeaurora.org> Cc: Asutosh Das <asutoshd@codeaurora.org> Cc: Avri Altman <avri.altman@wdc.com> Reviewed-by: Bean Huo <beanhuo@micron.com> Signed-off-by: Bart Van Assche <bvanassche@acm.org> Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
Use a spinlock to protect hba->outstanding_reqs instead of using atomic
operations to update this member variable.
This patch is a performance improvement because it reduces the number of
atomic operations in the hot path (test_and_clear_bit()) and because it
reduces the lock contention on the SCSI host lock. On my test setup this
patch improves IOPS by about 1%.
Link: https://lore.kernel.org/r/20210722033439.26550-14-bvanassche@acm.org Cc: Adrian Hunter <adrian.hunter@intel.com> Cc: Stanley Chu <stanley.chu@mediatek.com> Cc: Can Guo <cang@codeaurora.org> Cc: Asutosh Das <asutoshd@codeaurora.org> Cc: Avri Altman <avri.altman@wdc.com> Reviewed-by: Daejun Park <daejun7.park@samsung.com> Reviewed-by: Bean Huo <beanhuo@micron.com> Signed-off-by: Bart Van Assche <bvanassche@acm.org> Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
scsi: ufs: Optimize serialization of setup_xfer_req() calls
Reduce the number of times the host lock is taken in the hot path.
Additionally, inline ufshcd_vops_setup_xfer_req() because that function is
too short to keep it.
Link: https://lore.kernel.org/r/20210722033439.26550-13-bvanassche@acm.org Fixes: a45f937110fa ("scsi: ufs: Optimize host lock on transfer requests send/compl paths") Cc: Jaegeuk Kim <jaegeuk@kernel.org> Cc: Stanley Chu <stanley.chu@mediatek.com> Cc: Can Guo <cang@codeaurora.org> Cc: Bean Huo <beanhuo@micron.com> Cc: Asutosh Das <asutoshd@codeaurora.org> Reviewed-by: Daejun Park <daejun7.park@samsung.com> Reviewed-by: Bean Huo <beanhuo@micron.com> Signed-off-by: Bart Van Assche <bvanassche@acm.org> Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
scsi: ufs: Revert "Utilize Transfer Request List Completion Notification Register"
Using the UTRLCNR register involves two MMIO accesses in the hot path while
using the doorbell register only involves a single MMIO access. Since MMIO
accesses take time, do not use the UTRLCNR register. The spinlock
contention on the SCSI host lock that is reintroduced by this commit will
be addressed later.
Link: https://lore.kernel.org/r/20210722033439.26550-12-bvanassche@acm.org Cc: Adrian Hunter <adrian.hunter@intel.com> Cc: Stanley Chu <stanley.chu@mediatek.com> Cc: Can Guo <cang@codeaurora.org> Cc: Asutosh Das <asutoshd@codeaurora.org> Cc: Avri Altman <avri.altman@wdc.com> Tested-by: Bean Huo <beanhuo@micron.com> Signed-off-by: Bart Van Assche <bvanassche@acm.org> Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
Inline ufshcd_outstanding_req_clear() since it only has one caller and
since its body is only one line long.
Link: https://lore.kernel.org/r/20210722033439.26550-11-bvanassche@acm.org Cc: Adrian Hunter <adrian.hunter@intel.com> Cc: Stanley Chu <stanley.chu@mediatek.com> Cc: Can Guo <cang@codeaurora.org> Cc: Asutosh Das <asutoshd@codeaurora.org> Reviewed-by: Avri Altman <avri.altman@wdc.com> Reviewed-by: Bean Huo <beanhuo@micron.com> Signed-off-by: Bart Van Assche <bvanassche@acm.org> Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
From Documentation/memory-barriers.txt: "Note that, when using writel(), a
prior wmb() is not needed to guarantee that the cache coherent memory
writes have completed before writing to the MMIO region."
In other words, calling wmb() before writel() is not necessary. Hence
remove the wmb() calls that precede a writel() call. Remove the wmb() calls
that precede a ufshcd_send_command() call since the latter function uses
writel(). Remove the wmb() call from ufshcd_wait_for_dev_cmd() since the
following chain of events guarantees that the CPU will see up-to-date LRB
values:
- UFS controller writes to host memory.
- UFS controller posts completion interrupt after the memory writes from
the previous step are visible to the CPU.
- complete(hba->dev_cmd.complete) is called from the UFS interrupt handler.
- The wait_for_completion(hba->dev_cmd.complete) call in
ufshcd_wait_for_dev_cmd() returns.
Link: https://lore.kernel.org/r/20210722033439.26550-10-bvanassche@acm.org Cc: Adrian Hunter <adrian.hunter@intel.com> Cc: Stanley Chu <stanley.chu@mediatek.com> Cc: Can Guo <cang@codeaurora.org> Cc: Asutosh Das <asutoshd@codeaurora.org> Cc: Avri Altman <avri.altman@wdc.com> Tested-by: Avri altman <avri.altman@wdc.com> Reviewed-by: Avri Altman <avri.altman@wdc.com> Reviewed-by: Daejun Park <daejun7.park@samsung.com> Signed-off-by: Bart Van Assche <bvanassche@acm.org> Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
scsi: ufs: Improve static type checking for the host controller state
Assign a name to the enumeration type for UFS host controller states and
remove the default clause from switch statements on this enumeration type
to make the compiler warn about unhandled enumeration labels.
Link: https://lore.kernel.org/r/20210722033439.26550-9-bvanassche@acm.org Cc: Can Guo <cang@codeaurora.org> Reviewed-by: Avri Altman <avri.altman@wdc.com> Reviewed-by: Keoseong Park <keosung.park@samsung.com> Reviewed-by: Daejun Park <daejun7.park@samsung.com> Signed-off-by: Bart Van Assche <bvanassche@acm.org> Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
scsi: ufs: Verify UIC locking requirements at runtime
Instead of documenting the locking requirements of the UIC code as
comments, use lockdep_assert_held() such that lockdep verifies the lockdep
requirements at runtime if lockdep is enabled.
Link: https://lore.kernel.org/r/20210722033439.26550-8-bvanassche@acm.org Cc: Adrian Hunter <adrian.hunter@intel.com> Cc: Stanley Chu <stanley.chu@mediatek.com> Cc: Can Guo <cang@codeaurora.org> Cc: Asutosh Das <asutoshd@codeaurora.org> Reviewed-by: Avri Altman <avri.altman@wdc.com> Reviewed-by: Bean Huo <beanhuo@micron.com> Reviewed-by: Daejun Park <daejun7.park@samsung.com> Signed-off-by: Bart Van Assche <bvanassche@acm.org> Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
scsi_add_host() allocates shost->can_queue tags. ufshcd_init() sets
shost->can_queue to hba->nutrs. In other words, we know that tag values
will less than hba->nutrs. Hence remove the checks that verify that
blk_get_request() returns a tag less than hba->nutrs. This check was
introduced by commit 14497328b6a6 ("scsi: ufs: verify command tag
validity").
Keep the tag >= 0 check because it helps to detect use-after-free issues.
Link: https://lore.kernel.org/r/20210722033439.26550-7-bvanassche@acm.org CC: Avri Altman <avri.altman@wdc.com> Cc: Alim Akhtar <alim.akhtar@samsung.com> Reviewed-by: Bean Huo <beanhuo@micron.com> Reviewed-by: Daejun Park <daejun7.park@samsung.com> Signed-off-by: Bart Van Assche <bvanassche@acm.org> Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
scsi: ufs: Use DECLARE_COMPLETION_ONSTACK() where appropriate
From Documentation/scheduler/completion.rst: "When a completion is declared
as a local variable within a function, then the initialization should
always use DECLARE_COMPLETION_ONSTACK() explicitly, not just to make
lockdep happy, but also to make it clear that limited scope had been
considered and is intentional."
Link: https://lore.kernel.org/r/20210722033439.26550-6-bvanassche@acm.org Cc: Adrian Hunter <adrian.hunter@intel.com> Cc: Stanley Chu <stanley.chu@mediatek.com> Cc: Can Guo <cang@codeaurora.org> Cc: Asutosh Das <asutoshd@codeaurora.org> Cc: Avri Altman <avri.altman@wdc.com> Reviewed-by: Avri Altman <avri.altman@wdc.com> Reviewed-by: Bean Huo <beanhuo@micron.com> Reviewed-by: Daejun Park <daejun7.park@samsung.com> Signed-off-by: Bart Van Assche <bvanassche@acm.org> Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
scsi: ufs: Rename the second ufshcd_probe_hba() argument
Rename the second argument of ufshcd_probe_hba() such that the name of that
argument reflects its purpose instead of how the function is called. See
also commit 1b9e21412f72 ("scsi: ufs: Split ufshcd_probe_hba() based on its
called flow").
Link: https://lore.kernel.org/r/20210722033439.26550-5-bvanassche@acm.org Cc: Asutosh Das <asutoshd@codeaurora.org> Cc: Can Guo <cang@codeaurora.org> Reviewed-by: Avri Altman <avri.altman@wdc.com> Reviewed-by: Bean Huo <beanhuo@micron.com> Reviewed-by: Daejun Park <daejun7.park@samsung.com> Reviewed-by: Stanley Chu <stanley.chu@mediatek.com> Signed-off-by: Bart Van Assche <bvanassche@acm.org> Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
scsi: ufs: Only include power management code if necessary
This patch slightly reduces the UFS driver size if built with power
management support disabled.
Link: https://lore.kernel.org/r/20210722033439.26550-4-bvanassche@acm.org Cc: Adrian Hunter <adrian.hunter@intel.com> Cc: Stanley Chu <stanley.chu@mediatek.com> Cc: Can Guo <cang@codeaurora.org> Cc: Asutosh Das <asutoshd@codeaurora.org> Reviewed-by: Avri Altman <avri.altman@wdc.com> Reviewed-by: Bean Huo <beanhuo@micron.com> Reviewed-by: Daejun Park <daejun7.park@samsung.com> Reviewed-by: Stanley Chu <stanley.chu@mediatek.com> Signed-off-by: Bart Van Assche <bvanassche@acm.org> Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
scsi: ufs: Reduce power management code duplication
Move the dev_get_drvdata() calls into the ufshcd_{system,runtime}_*()
functions. Remove ufshcd_runtime_idle() since it is empty. This patch does
not change any functionality.
Link: https://lore.kernel.org/r/20210722033439.26550-3-bvanassche@acm.org Cc: Adrian Hunter <adrian.hunter@intel.com> Cc: Stanley Chu <stanley.chu@mediatek.com> Cc: Can Guo <cang@codeaurora.org> Cc: Asutosh Das <asutoshd@codeaurora.org> Reviewed-by: Avri Altman <avri.altman@wdc.com> Reviewed-by: Bean Huo <beanhuo@micron.com> Reviewed-by: Daejun Park <daejun7.park@samsung.com> Reviewed-by: Stanley Chu <stanley.chu@mediatek.com> Signed-off-by: Bart Van Assche <bvanassche@acm.org> Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
scsi: ufs: Fix memory corruption by ufshcd_read_desc_param()
If param_offset > buff_len then the memcpy() statement in
ufshcd_read_desc_param() corrupts memory since it copies 256 + buff_len -
param_offset bytes into a buffer with size buff_len. Since param_offset <
256 this results in writing past the bound of the output buffer.
Link: https://lore.kernel.org/r/20210722033439.26550-2-bvanassche@acm.org Fixes: cbe193f6f093 ("scsi: ufs: Fix potential NULL pointer access during memcpy") Reviewed-by: Avri Altman <avri.altman@wdc.com> Reviewed-by: Daejun Park <daejun7.park@samsung.com> Signed-off-by: Bart Van Assche <bvanassche@acm.org> Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
scsi: ufs: ufshpb: Limit the number of in-flight map requests
In host control mode the host is the originator of map requests. To not
flood the device with map requests, use a simple throttling mechanism that
limits the number of in-flight map requests.
In order not to hang on to "cold" regions, we inactivate a region that has
had no READ access for a predefined amount of time - READ_TO_MS. For that
purpose monitor the active regions list, polling it on every
POLLING_INTERVAL_MS. On timeout expiry add the region to the
"to-be-inactivated" list unless it is clean and did not exhaust its
READ_TO_EXPIRIES - another parameter.
In host control mode, reads are the major source of activation trials.
Keep track of those reads counters, for both active as well inactive
regions.
We reset the read counter upon write - we are only interested in "clean"
reads.
Keep those counters normalized, as we are using those reads as a
comparative score, to make various decisions. If during consecutive
normalizations an active region has exhaust its reads - inactivate it.
While at it, protect the {active,inactive}_count stats by adding them into
the applicable handler.
scsi: ufs: ufshpb: Transform set_dirty to iterate_rgn
Given a transfer length, set_dirty meticulously iterates over all the
entries, across subregions and regions if needed. Currently its only use is
to mark dirty blocks, but HCM may benefit from it as well to manage its
read counters.
scsi: ufs: ufshpb: Add host control mode support to rsp_upiu
In device control mode, the device may recommend the host to either
activate or inactivate a region, and the host should follow. Meaning those
are not actually recommendations, but more of instructions.
Conversely, in host control mode, the recommendation protocol is slightly
changed:
a) The device may only recommend the host to update a subregion of an
already-active region. And,
b) The device may *not* recommend to inactivate a region.
Furthermore, in host control mode, the host may choose not to follow any of
the device's recommendations. However, in case of a recommendation to
update an active and clean subregion, it is better to follow those
recommendation because otherwise the host has no other way to know that
some internal relocation took place.
Daejun Park [Mon, 12 Jul 2021 09:00:25 +0000 (18:00 +0900)]
scsi: ufs: ufshpb: Add HPB 2.0 support
Version 2.0 of HBP supports reads of varying sizes from 4KB to 1MB.
A read operation <= 32KB is supported as single HPB read. A read between
36KB and 1MB is supported by a combination of write buffer command and HPB
read command to deliver more PPN. The write buffer commands may not be
issued immediately due to busy tags. To use HPB read more aggressively, the
driver can requeue the write buffer command. The requeue threshold is
implemented as timeout and can be modified with requeue_timeout_ms entry in
sysfs.
[mkp: REQ_OP_DRV_* and blk_rq_is_passthrough()]
Link: https://lore.kernel.org/r/20210712090025epcms2p3b3d94f6f1b2cfa394e3d9ba130ca0fa7@epcms2p3 Tested-by: Can Guo <cang@codeaurora.org> Tested-by: Stanley Chu <stanley.chu@mediatek.com> Reviewed-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org> Reviewed-by: Can Guo <cang@codeaurora.org> Reviewed-by: Bean Huo <beanhuo@micron.com> Reviewed-by: Stanley Chu <stanley.chu@mediatek.com> Signed-off-by: Daejun Park <daejun7.park@samsung.com> Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
Daejun Park [Mon, 12 Jul 2021 08:59:36 +0000 (17:59 +0900)]
scsi: ufs: ufshpb: Prepare HPB read for cached sub-region
If the logical address of a read I/O belongs to an active sub-region, the
HPB driver modifies the read I/O command to an HPB read. The driver
modifies the UFS UPIU instead of modifying the existing SCSI command.
In HPB version 1.0, the maximum read I/O size that can be converted to HPB
read is 4KB.
The dirty map of the active sub-region prevents an incorrect HPB read that
has stale physical page number which is updated by previous write I/O.
[mkp: REQ_OP_DRV_* and blk_rq_is_passthrough()]
Link: https://lore.kernel.org/r/20210712085936epcms2p4b0ec5c8cecdeea6cc043d684363842b6@epcms2p4 Tested-by: Bean Huo <beanhuo@micron.com> Tested-by: Can Guo <cang@codeaurora.org> Tested-by: Stanley Chu <stanley.chu@mediatek.com> Reviewed-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org> Reviewed-by: Can Guo <cang@codeaurora.org> Reviewed-by: Bart Van Assche <bvanassche@acm.org> Reviewed-by: Bean Huo <beanhuo@micron.com> Reviewed-by: Stanley Chu <stanley.chu@mediatek.com> Acked-by: Avri Altman <Avri.Altman@wdc.com> Signed-off-by: Daejun Park <daejun7.park@samsung.com> Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
Daejun Park [Mon, 12 Jul 2021 08:58:59 +0000 (17:58 +0900)]
scsi: ufs: ufshpb: L2P map management for HPB read
Implement L2P map management in HPB.
The HPB divides logical addresses into several regions. A region consists
of several sub-regions. The sub-region is a basic unit where L2P mapping is
managed. The driver loads L2P mapping data of each sub-region. The loaded
sub-region is called active-state. The HPB driver unloads L2P mapping data
as region unit. The unloaded region is called inactive-state.
Sub-region/region candidates to be loaded and unloaded are delivered from
the UFS device. The UFS device delivers the recommended active sub-region
and inactivate region to the driver using sense data. The HPB module
performs L2P mapping management on the host through the delivered
information.
A pinned region is a preset region on the UFS device that is always
in activate-state.
The data structures for map data requests and L2P mappings use the mempool
API, minimizing allocation overhead while avoiding static allocation.
The mininum size of the memory pool used in the HPB is implemented
as a module parameter so that it can be configurable by the user.
To guarantee a minimum memory pool size of 4MB: ufshpb_host_map_kbytes=4096.
The map_work manages active/inactive via 2 "to-do" lists:
- hpb->lh_inact_rgn: regions to be inactivated
- hpb->lh_act_srgn: subregions to be activated
These lists are maintained on I/O completion.
[mkp: switch to REQ_OP_DRV_*]
Link: https://lore.kernel.org/r/20210712085859epcms2p36e420f19564f6cd0c4a45d54949619eb@epcms2p3 Tested-by: Bean Huo <beanhuo@micron.com> Tested-by: Can Guo <cang@codeaurora.org> Tested-by: Stanley Chu <stanley.chu@mediatek.com> Reviewed-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org> Reviewed-by: Bart Van Assche <bvanassche@acm.org> Reviewed-by: Can Guo <cang@codeaurora.org> Reviewed-by: Bean Huo <beanhuo@micron.com> Reviewed-by: Stanley Chu <stanley.chu@mediatek.com> Acked-by: Avri Altman <Avri.Altman@wdc.com> Signed-off-by: Daejun Park <daejun7.park@samsung.com> Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
Implement Host Performance Buffer (HPB) initialization and add function
calls to UFS core driver.
NAND flash-based storage devices, including UFS, have mechanisms to
translate logical addresses of I/O requests to the corresponding physical
addresses of the flash storage. In UFS, logical-to-physical-address (L2P)
map data, which is required to identify the physical address for the
requested I/Os, can only be partially stored in SRAM from NAND flash. Due
to this partial loading, accessing the flash address area, where the L2P
information for that address is not loaded in the SRAM, can result in
serious performance degradation.
The basic concept of HPB is to cache L2P mapping entries in host system
memory so that both physical block address (PBA) and logical block address
(LBA) can be delivered in HPB read command. The HPB read command allows to
read data faster than a regular read command in UFS since it provides the
physical address (HPB Entry) of the desired logical block in addition to
its logical address. The UFS device can access the physical block in NAND
directly without searching and uploading L2P mapping table. This improves
read performance because the NAND read operation for uploading L2P mapping
table is removed.
In HPB initialization, the host checks if the UFS device supports HPB
feature and retrieves related device capabilities. Then, HPB parameters are
configured in the device.
Total start-up time of popular applications was measured and the difference
observed between HPB being enabled and disabled. Popular applications are
12 game apps and 24 non-game apps. Each test cycle consists of running 36
applications in sequence. We repeated the cycle for observing performance
improvement by L2P mapping cache hit in HPB.
Link: https://lore.kernel.org/r/20210712085830epcms2p8c1288b7f7a81b044158a18232617b572@epcms2p8 Reported-by: kernel test robot <lkp@intel.com> Tested-by: Bean Huo <beanhuo@micron.com> Tested-by: Can Guo <cang@codeaurora.org> Tested-by: Stanley Chu <stanley.chu@mediatek.com> Reviewed-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org> Reviewed-by: Bart Van Assche <bvanassche@acm.org> Reviewed-by: Can Guo <cang@codeaurora.org> Reviewed-by: Bean Huo <beanhuo@micron.com> Reviewed-by: Stanley Chu <stanley.chu@mediatek.com> Acked-by: Avri Altman <Avri.Altman@wdc.com> Signed-off-by: Daejun Park <daejun7.park@samsung.com> Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
Dwaipayan Ray [Fri, 16 Jul 2021 11:28:52 +0000 (16:58 +0530)]
scsi: qla4xxx: Convert uses of __constant_cpu_to_<foo> to cpu_to_<foo>
The macros cpu_to_le16() and cpu_to_le32() have special cases for
constants. Their __constant_<foo> versions are not required.
On little endian systems, both cpu_to_le16() and __constant_cpu_to_le16()
expand to the same expression. Same is the case with cpu_to_le32().
On big endian systems, cpu_to_le16() expands to __swab16() which has a
__builtin_constant_p check. Similarly, cpu_to_le32() expands to __swab32().
Consequently these macros can be safely used with constants, and hence all
those uses are converted. This was discovered as a part of a checkpatch
evaluation, looking at all reports of WARNING:CONSTANT_CONVERSION error
type.
Colin Ian King [Fri, 30 Jul 2021 09:50:31 +0000 (10:50 +0100)]
scsi: BusLogic: Use %X for u32 sized integer rather than %lX
An earlier fix changed the print format specifier for adapter->bios_addr to
use %lX. However, the integer is a u32 so the fix was wrong. Fix this by
using the correct %X format specifier.
Link: https://lore.kernel.org/r/20210730095031.26981-1-colin.king@canonical.com Fixes: 43622697117c ("scsi: BusLogic: use %lX for unsigned long rather than %X") Acked-by: Khalid Aziz <khalid@gonehiking.org> Signed-off-by: Colin Ian King <colin.king@canonical.com> Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
Addresses-Coverity: ("Invalid type in argument")
Existing blogic_msg() invocations do not appear to overrun its internal
buffer of a fixed length of 100, which would cause stack corruption, but
it's easy to miss with possible further updates and a fix is cheap in
performance terms, so limit the output produced into the buffer by using
vscnprintf() rather than vsprintf().
Also diagnostic output such as with the BusLogic=TraceConfiguration
parameter is affected and becomes vertical and therefore hard to read.
This has now been corrected, e.g.:
Link: https://lore.kernel.org/r/alpine.DEB.2.21.2104201940430.44318@angie.orcam.me.uk Fixes: 4bcc595ccd80 ("printk: reinstate KERN_CONT for printing continuation lines") Cc: stable@vger.kernel.org # v4.9+ Acked-by: Khalid Aziz <khalid@gonehiking.org> Signed-off-by: Maciej W. Rozycki <macro@orcam.me.uk> Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
scsi: bsg-lib: Fix commands without data transfer in bsg_transport_sg_io_fn()
Set ret to 0 after the initial permission checks to avoid leaking -EPERM
for commands without data transfer.
Link: https://lore.kernel.org/r/20210731074027.1185545-3-hch@lst.de Fixes: 75ca56409e5b ("scsi: bsg: Move the whole request execution into the SCSI/transport handlers") Signed-off-by: Christoph Hellwig <hch@lst.de> Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
scsi: bsg: Fix commands without data transfer in scsi_bsg_sg_io_fn()
Set ret to 0 after the initial permission checks to avoid leaking -EPERM
for commands without data transfer.
Link: https://lore.kernel.org/r/20210731074027.1185545-2-hch@lst.de Fixes: 75ca56409e5b ("scsi: bsg: Move the whole request execution into the SCSI/transport handlers") Signed-off-by: Christoph Hellwig <hch@lst.de> Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
Allow UFS suspend/resume callbacks to run in parallel with other
suspend/resume callbacks. This can recoup dozens of milliseconds on the
resume path if UFS hardware needs to be powered back on.
Suspending and resuming asynchronously is safe to do so long as the driver
callbacks only depend on resources made available by either a) parent
devices or b) devices explicitly marked as suppliers with device_link_add.
Link: https://lore.kernel.org/r/20210728012743.1063928-1-paillon@google.com Cc: Bjorn Helgaas <bhelgaas@google.com> Cc: Jaegeuk Kim <jaegeuk@kernel.org> Cc: Bart Van Assche <bvanassche@acm.org> Cc: Adrian Hunter <adrian.hunter@intel.com> Cc: Stanley Chu <stanley.chu@mediatek.com> Cc: Can Guo <cang@codeaurora.org> Cc: Asutosh Das <asutoshd@codeaurora.org> Cc: Avri Altman <avri.altman@wdc.com> Cc: Martin K. Petersen <martin.petersen@oracle.com> Reviewed-by: Bart Van Assche <bvanassche@acm.org> Signed-off-by: Vincent Palomares <paillon@google.com> Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
James Smart [Fri, 30 Jul 2021 16:33:09 +0000 (09:33 -0700)]
scsi: lpfc: Fix possible ABBA deadlock in nvmet_xri_aborted()
The lpfc_sli4_nvmet_xri_aborted() routine takes out the abts_buf_list_lock
and traverses the buffer contexts to match the xri. Upon match, it then
takes the context lock before potentially removing the context from the
associated buffer list. This violates the lock hierarchy used elsewhere in
the driver of locking context, then the abts_buf_list_lock - thus a
possible deadlock.
Resolve by: after matching, release the abts_buf_list_lock, then take the
context lock, and if to be deleted from the list, retake the
abts_buf_list_lock, maintaining lock hierarchy. This matches same list lock
hierarchy as elsewhere in the driver
scsi: block: Remove the remaining SG_IO-related fields from struct request_queue
Move the sg_timeout and sg_reserved_size fields into the bsg_device and
scsi_device structures as they have nothing to do with generic block I/O.
Note that these values are now separate for bsg vs. SCSI device node
access, but that just matches how /dev/sg vs the other nodes has always
behaved.
Use the per-device cdev_device_interface to store the bsg data in the char
device inode, and thus remove the need to embedd the bsg_class_device
structure in the request_queue.
scsi: sr: cdrom: Move cdrom_read_cdda_bpc() into the sr driver
cdrom_read_cdda_bpc() relies on sending SCSI command to the low level
driver using a REQ_OP_SCSI_IN request. This isn't generic block layer
functionality, so move the actual low-level code into the sr driver and
call it through a new read_cdda_bpc method in the cdrom_device_ops
structure.
With this the CDROM code does not have to pull in scsi_normalize_sense()
and depend on CONFIG_SCSI_COMMON.
projects / thirdparty / kernel / linux.git / log
