We don't want to wait for conn_logout_comp from isert_comp_wq
context as this blocks further completions from being processed.
Instead we wait for it conditionally (if logout response was
actually posted) in wait_conn. This wait should normally happen
immediately as it occurs after we consumed all the completions
(including flush errors) and conn_logout_comp should have been
completed.
Signed-off-by: Sagi Grimberg <sagig@mellanox.com>
Signed-off-by: Nicholas Bellinger <nab@linux-iscsi.org>

Might result in a deadlock where completion context waits for
session commands release where the later might need a final
completion for it.
Signed-off-by: Sagi Grimberg <sagig@mellanox.com>
Signed-off-by: Nicholas Bellinger <nab@linux-iscsi.org>

In order to reduce the contention on CQ locking (present
in some LLDDs) we poll in batches of 16 work completion items.
Signed-off-by: Sagi Grimberg <sagig@mellanox.com>
Signed-off-by: Nicholas Bellinger <nab@linux-iscsi.org>

In case the CQ is packed with completions, we can't just
hog the CPU forever. Poll until a sufficient budget (currently
hard-coded to 64k completions) and if budget is exhausted, bailout
and give a chance to other threads.
Signed-off-by: Sagi Grimberg <sagig@mellanox.com>
Signed-off-by: Nicholas Bellinger <nab@linux-iscsi.org>

In order to know that we consumed all the connection completions
we maintain atomic post_send_buf_count for each IO post send. But
we can know that if we post a "beacon" (zero length RECV work request)
after we move the QP into error state and the target does not serve
any new IO. When we consume it, we know we finished all the connection
completion and we can go ahead and destroy stuff.

In error completion handler we now just need to check for ISERT_BEACON_WRID
to arrive and then wait for session commands to cleanup and complete
conn_wait_comp_err.

We reserve another CQ and QP entries to fit the zero length post recv.
Signed-off-by: Sagi Grimberg <sagig@mellanox.com>
Signed-off-by: Nicholas Bellinger <nab@linux-iscsi.org>

We are calling session reinstatement, wait_conn will start
connection termination.
Signed-off-by: Sagi Grimberg <sagig@mellanox.com>
Signed-off-by: Nicholas Bellinger <nab@linux-iscsi.org>

Using TX and RX CQs attached to the same vector might
create a throttling effect coming from the serial processing
of a work-queue. Use one CQ instead, it will do better in interrupt
processing and it provides a simpler code. Also, We get rid of
redundant isert_rx_wq.

Next we can remove the atomic post_send_buf_count from the IO path.
Signed-off-by: Sagi Grimberg <sagig@mellanox.com>
Signed-off-by: Nicholas Bellinger <nab@linux-iscsi.org>

A pre-step before going to a single CQ.
Also this makes the code a little more simple to
read.
Signed-off-by: Sagi Grimberg <sagig@mellanox.com>
Signed-off-by: Nicholas Bellinger <nab@linux-iscsi.org>

Nit, uintptr_t is designed for pointer casting, use it.
Signed-off-by: Sagi Grimberg <sagig@mellanox.com>
Signed-off-by: Nicholas Bellinger <nab@linux-iscsi.org>

As a pre-step to a single CQ, we unite the error completion
handlers to a single handler.

This patch does not change any functionality.
Signed-off-by: Sagi Grimberg <sagig@mellanox.com>
Signed-off-by: Nicholas Bellinger <nab@linux-iscsi.org>

It is disabled at the moment, we will get that back
in once the target is more stable.

This reverts commit 95b60f07
"Add support for completion interrupt coalescing"
Signed-off-by: Sagi Grimberg <sagig@mellanox.com>
Signed-off-by: Nicholas Bellinger <nab@linux-iscsi.org>

Currently we have no way to tell that the target stack is in shutdown
sequence. In case we have open connections, the initiator immediately
attempts to reconnect in a DDOS attack style, so we may end up
terminating the iser enabled network portal while it's np_accept_list
still have pending connections.

The workaround is simply release all the connections in the list.
A proper fix will be to start shutdown sequence by shutting the
network portal to avoid initiator immediate reconnect attempts.

But the temporary work around seems to work at this point, so I think
we can do this for now...
Reported-by: Slava Shwartsman <valyushash@gmail.com>
Signed-off-by: Sagi Grimberg <sagig@mellanox.com>
Signed-off-by: Nicholas Bellinger <nab@linux-iscsi.org>

iSER will report supported protection operations based on
the tpg attribute t10_pi settings and HCA PI offload capabilities.
If the HCA does not support PI offload or tpg attribute t10_pi is
not set, we fall to SW PI mode.

In order to do that, we move iscsit_get_sup_prot_ops after connection
tpg assignment.
Signed-off-by: Sagi Grimberg <sagig@mellanox.com>
Cc: <stable@vger.kernel.org> # v3.14+
Signed-off-by: Nicholas Bellinger <nab@linux-iscsi.org>

Fallback to software mode DIF if HCA does not support
PI (without crashing obviously). It is still possible to
run with backend protection and an unprotected frontend,
so looking at the command prot_op is not enough. Check
device PI capability on a per-IO basis (isert_prot_cmd
inline static) to determine if we need to handle protection
information.

Trace:
BUG: unable to handle kernel NULL pointer dereference at 0000000000000010
IP: [<ffffffffa037f8b1>] isert_reg_sig_mr+0x351/0x3b0 [ib_isert]
Call Trace:
 [<ffffffff812b003a>] ? swiotlb_map_sg_attrs+0x7a/0x130
 [<ffffffffa038184d>] isert_reg_rdma+0x2fd/0x370 [ib_isert]
 [<ffffffff8108f2ec>] ? idle_balance+0x6c/0x2c0
 [<ffffffffa0382b68>] isert_put_datain+0x68/0x210 [ib_isert]
 [<ffffffffa02acf5b>] lio_queue_data_in+0x2b/0x30 [iscsi_target_mod]
 [<ffffffffa02306eb>] target_complete_ok_work+0x21b/0x310 [target_core_mod]
 [<ffffffff8106ece2>] process_one_work+0x182/0x3b0
 [<ffffffff8106fda0>] worker_thread+0x120/0x3c0
 [<ffffffff8106fc80>] ? maybe_create_worker+0x190/0x190
 [<ffffffff8107594e>] kthread+0xce/0xf0
 [<ffffffff81075880>] ? kthread_freezable_should_stop+0x70/0x70
 [<ffffffff8159a22c>] ret_from_fork+0x7c/0xb0
 [<ffffffff81075880>] ? kthread_freezable_should_stop+0x70/0x70
Reported-by: Slava Shwartsman <valyushash@gmail.com>
Signed-off-by: Sagi Grimberg <sagig@mellanox.com>
Cc: <stable@vger.kernel.org> # v3.14+
Signed-off-by: Nicholas Bellinger <nab@linux-iscsi.org>

This patch converts to allocate PI contexts dynamically in order
avoid a potentially bogus np->tpg_np and associated NULL pointer
dereference in isert_connect_request() during iser-target endpoint
shutdown with multiple network portals.

Also, there is really no need to allocate these at connection
establishment since it is not guaranteed that all the IOs on
that connection will be to a PI formatted device.

We can do it in a lazy fashion so the initial burst will have a
transient slow down, but very fast all IOs will allocate a PI
context.

Squashed:

iser-target: Centralize PI context handling code
Signed-off-by: Sagi Grimberg <sagig@mellanox.com>
Cc: <stable@vger.kernel.org> # v3.14+
Signed-off-by: Nicholas Bellinger <nab@linux-iscsi.org>

In situations such as bond failover, The new session establishment
implicitly invokes the termination of the old connection.

So, we don't want to wait for the old connection wait_conn to completely
terminate before we accept the new connection and post a login response.

The solution is to deffer the comp_wait completion and the conn_put to
a work so wait_conn will effectively be non-blocking (flush errors are
assumed to come very fast).

We allocate isert_release_wq with WQ_UNBOUND and WQ_UNBOUND_MAX_ACTIVE
to spread the concurrency of release works.
Reported-by: Slava Shwartsman <valyushash@gmail.com>
Signed-off-by: Sagi Grimberg <sagig@mellanox.com>
Cc: <stable@vger.kernel.org> # v3.10+
Signed-off-by: Nicholas Bellinger <nab@linux-iscsi.org>

The np listener cm_id will also get ADDR_CHANGE event
upcall (in case it is bound to a specific IP). Handle
it correctly by creating a new cm_id and implicitly
destroy the old one.

Since this is the second event a listener np cm_id may
encounter, we move the np cm_id event handling to a
routine.

Squashed:

iser-target: Move cma_id setup to a function
Reported-by: Slava Shwartsman <valyushash@gmail.com>
Signed-off-by: Sagi Grimberg <sagig@mellanox.com>
Cc: <stable@vger.kernel.org> # v3.10+
Signed-off-by: Nicholas Bellinger <nab@linux-iscsi.org>

Take isert_conn pointer from cm_id->qp->qp_context. This
will allow us to know that the cm_id context is always
the network portal. This will make the cm_id event check
(connection or network portal) more reliable.

In order to avoid a NULL dereference in cma_id->qp->qp_context
we destroy the qp after we destroy the cm_id (and make the
dereference safe). session stablishment/teardown sequences
can happen in parallel, we should take into account that
connected_handler might race with connection teardown flow.

Also, protect isert_conn->conn_device->active_qps decrement
within the error patch during QP creation failure and the
normal teardown path in isert_connect_release().

Squashed:

iser-target: Decrement completion context active_qps in error flow
Signed-off-by: Sagi Grimberg <sagig@mellanox.com>
Cc: <stable@vger.kernel.org> # v3.10+
Signed-off-by: Nicholas Bellinger <nab@linux-iscsi.org>

There is no point in accepting a new CM request only
when we are completely done with the last iscsi login.
Instead we accept immediately, this will also cause the
CM connection to reach connected state and the initiator
is allowed to send the first login. We mark that we got
the initial login and let iscsi layer pick it up when it
gets there.

This reduces the parallel login sequence by a factor of
more then 4 (and more for multi-login) and also prevents
the initiator (who does all logins in parallel) from
giving up on login timeout expiration.

In order to support multiple login requests sequence (CHAP)
we call isert_rx_login_req from isert_rx_completion insead
of letting isert_get_login_rx call it.

Squashed:

iser-target: Use kref_get_unless_zero in connected_handler
iser-target: Acquire conn_mutex when changing connection state
iser-target: Reject connect request in failure path
Signed-off-by: Sagi Grimberg <sagig@mellanox.com>
Cc: <stable@vger.kernel.org> # v3.10+
Signed-off-by: Nicholas Bellinger <nab@linux-iscsi.org>

ISER_CONN_UP state is not sufficient to know if
we should wait for completion of flush errors and
disconnected_handler event.

Instead, split it to 2 states:
- ISER_CONN_UP: Got to CM connected phase, This state
indicates that we need to wait for a CM disconnect
event before going to teardown.

- ISER_CONN_FULL_FEATURE: Got to full feature phase
after we posted login response, This state indicates
that we posted recv buffers and we need to wait for
flush completions before going to teardown.

Also avoid deffering disconnected handler to a work,
and handle it within disconnected handler.
More work here is needed to handle DEVICE_REMOVAL event
correctly (cleanup all resources).

Squashed:

iser-target: Don't deffer disconnected handler to a work
Signed-off-by: Sagi Grimberg <sagig@mellanox.com>
Cc: <stable@vger.kernel.org> # v3.10+
Signed-off-by: Nicholas Bellinger <nab@linux-iscsi.org>

Since commit 0fc4ea70 ("Target/iser: Don't put isert_conn inside
disconnected handler") we put the conn kref in isert_wait_conn, so we
need .wait_conn to be invoked also in the error path.

Introduce call to isert_conn_terminate (called under lock)
which transitions the connection state to TERMINATING and calls
rdma_disconnect. If the state is already teminating, just bail
out back (temination started).

Also, make sure to destroy the connection when getting a connect
error event if didn't get to connected (state UP). Same for the
handling of REJECTED and UNREACHABLE cma events.

Squashed:

iscsi-target: Add call to wait_conn in establishment error flow
Reported-by: Slava Shwartsman <valyushash@gmail.com>
Signed-off-by: Sagi Grimberg <sagig@mellanox.com>
Cc: <stable@vger.kernel.org> # v3.10+
Signed-off-by: Nicholas Bellinger <nab@linux-iscsi.org>

Delete a local array that is only used to be initialized by memset.

A semantic patch that makes this change is as follows:
(http://coccinelle.lip6.fr/)

// <smpl>
@@
identifier x;
type T;
@@

{
... when any
-T x[...];
<+... when != x
- memset(x,...);
...+>
}
// </smpl>
Signed-off-by: Julia Lawall <Julia.Lawall@lip6.fr>
Signed-off-by: Nicholas Bellinger <nab@linux-iscsi.org>

In iscsi_target_init_module() unwind transport protocol registration in case
that iscsit_load_discovery_tpg() failed.
Signed-off-by: Lino Sanfilippo <LinoSanfilippo@gmx.de>
Reviewed-by: Sagi Grimberg <sagig@mellanox.com>
Signed-off-by: Nicholas Bellinger <nab@linux-iscsi.org>

tcm_loop has the I_T nexus associated with the HBA. This causes
commands to become misdirected if the HBA has more than one
target portal group; any command is then being sent to the
first target portal group instead of the correct one.

The nexus needs to be associated with the target portal group
instead.
Signed-off-by: Hannes Reinecke <hare@suse.de>
Cc: <stable@vger.kernel.org> # 3.0+
Signed-off-by: Nicholas Bellinger <nab@linux-iscsi.org>

Now that PSCSI is only exposing four hw_* read-only device attributes,
go ahead and drop the left-over -> legacy PHBA_PDEV checks in various
se_dev_set_* code, since it's now only used by virtual devices.
Signed-off-by: Nicholas Bellinger <nab@linux-iscsi.org>

Now that backend drivers are populating their own device attributes,
go ahead and remove left-over definitions + internal attribute list
of device attributes from target_core_configfs.c code

Also update TB_CIT_SETUP(dev_attrib,...) to signal ct_attr = NULL.
Signed-off-by: Nicholas Bellinger <nab@linux-iscsi.org>

This patch converts PSCSI to use an external set of device attributes,
and utilizes target_core_backend_configfs.h macros to generate a default
set of configfs extended-attr handlers.

It calls target_core_setup_sub_cits() to setup the initial config_item_type
based on existing target_core_configfs.c defaults, and using configfs_attribute
for hw_pi_prot_type, hw_block_size, hw_max_sectors, hw_queue_depth populates
pscsi_backend_dev_attrs[]

Only these four hw_* read-only device attributes are exports for PSCSI.
Signed-off-by: Nicholas Bellinger <nab@linux-iscsi.org>

This patch converts TCM-USER to use an external set of device attributes,
and utilizes target_core_backend_configfs.h macros to generate a default
set of configfs extended-attr handlers.

It calls target_core_setup_sub_cits() to setup the initial config_item_type
based on existing target_core_configfs.c defaults, and using configfs_attribute
generated by DEF_TB_DEFAULT_ATTRIBS(tcmu) populates tcmu_backend_dev_attrs[]

It introduces no function change for existing TCMU device attributes.
Signed-off-by: Nicholas Bellinger <nab@linux-iscsi.org>

This patch converts RAMDISK to use an external set of device attributes,
and utilizes target_core_backend_configfs.h macros to generate a default
set of configfs extended-attr handlers.

It calls target_core_setup_sub_cits() to setup the initial config_item_type
based on existing target_core_configfs.c defaults, and using configfs_attribute
generated by DEF_TB_DEFAULT_ATTRIBS(rc_mcp) populates rd_mcp_backend_dev_attrs[]

It introduces no functional change for existing RD_MCP device attributes.
Signed-off-by: Nicholas Bellinger <nab@linux-iscsi.org>

This patch converts FILEIO to use an external set of device attributes,
and utilizes target_core_backend_configfs.h macros to generate a default
set of configfs extended-attr handlers.

It calls target_core_setup_sub_cits() to setup the initial config_item_type
based on existing target_core_configfs.c defaults, and using configfs_attribute
generated by DEF_TB_DEFAULT_ATTRIBS(fileio) populates fileio_backend_dev_attrs[]

It introduces no functional change for existing FILEIO device attributes.
Signed-off-by: Nicholas Bellinger <nab@linux-iscsi.org>

This patch converts IBLOCK to use an external set of device attributes,
and utilizes target_core_backend_configfs.h macros to generate a default
set of configfs extended-attr handlers.

It calls target_core_setup_sub_cits() to setup the initial config_item_type
based on existing target_core_configfs.c defaults, and using configfs_attribute
generated by DEF_TB_DEFAULT_ATTRIBS(iblock) populates iblock_backend_dev_attrs[]

It introduces no functional change for existing IBLOCK device attributes.
Signed-off-by: Nicholas Bellinger <nab@linux-iscsi.org>

This helper macro adds the default set of 30 device attributes for
virtual devices from existing target_core_configfs.c code, and moves
the definitions into a single macro to create the structs necessary
for backend drivers.

It allows them to populate their own external struct configfs_attribute
for se_subsystem_api->tb_cits.tb_dev_attrib_cit.ct_attrs.
Signed-off-by: Nicholas Bellinger <nab@linux-iscsi.org>

Now that target_core_backend_configfs.h macros will be using these
se_dev_set attribute functions externally to allow backend drivers
to populate different attributes, go ahead and add EXPORT_SYMBOL()
for the existing default set of 30 device attributes.

Also update target_core_backend.h with proper function prototypes.
Signed-off-by: Nicholas Bellinger <nab@linux-iscsi.org>

This patch adds a number of configfs e-attr macros following
what existing target_core_configfs.c code does for internal
target_backend_dev_attrib setup, and similar to how target
fabric drivers allow for external config_item_type + cit->ct_attrs.
assignment.

This is useful for backend drivers like PSCSI who need to only
expose a small subset of device attributes, while still retaining
a default list of attributes for other backend drivers like
IBLOCK, FILEIO, RAMDISK, and TCMU.
Signed-off-by: Nicholas Bellinger <nab@linux-iscsi.org>

This patch adds support for dev_stat_cit as an external config_item_type
using TB_CIT_SETUP() helper macro, and sets only ct_group_ops following
existing code.

It updates target_core_make_subdev() + target_core_setup_dev_attrib_cit() +
struct target_backend_cits, and drops left-over target_core_dev_stat_cit
from target_core_configfs.c code and update comments.

This patch introduces no functional change from existing code.
Signed-off-by: Nicholas Bellinger <nab@linux-iscsi.org>

This patch adds support for dev_alua_tg_pt_gps_cit as an external config_item_type
using TB_CIT_SETUP() helper macro, and sets only ct_group_ops following
existing code.

It updates target_core_make_subdev() + target_core_setup_dev_attrib_cit() +
struct target_backend_cits, and drops left-over target_core_dev_alua_tg_pt_gps_cit
from target_core_configfs.c code and update comments.

This patch introduces no functional change from existing code.
Signed-off-by: Nicholas Bellinger <nab@linux-iscsi.org>

This patch adds support for dev_wwn_cit as an external config_item_type
using TB_CIT_SETUP() helper macro, and sets both ct_item_ops + ct_attr
following existing code.

It updates target_core_make_subdev() + target_core_setup_dev_attrib_cit() +
struct target_backend_cits, and drops left-over target_core_dev_wwn_cit
from target_core_configfs.c code and update comments.

This patch introduces no functional change from existing code.
Signed-off-by: Nicholas Bellinger <nab@linux-iscsi.org>

This patch adds support for dev_pr_cit as an external config_item_type
using TB_CIT_SETUP() helper macro, and sets both ct_item_ops + ct_attr
following existing code.

It updates target_core_make_subdev() + target_core_setup_dev_attrib_cit() +
struct target_backend_cits, and drops left-over target_core_dev_pr_cit
from target_core_configfs.c code and update comments.

This patch introduces no functional change from existing code.
Signed-off-by: Nicholas Bellinger <nab@linux-iscsi.org>

This patch adds support for dev_attrib_cit as an external config_item_type
using TB_CIT_SETUP() helper macro, and sets both ct_item_ops + ct_attr
following existing code.

It updates target_core_make_subdev() + target_core_setup_dev_attrib_cit() +
struct target_backend_cits, and drops left-over target_core_dev_attrib_cit
from target_core_configfs.c code and update comments.

This patch introduces no functional change from existing code.
Signed-off-by: Nicholas Bellinger <nab@linux-iscsi.org>

This patch adds initial support for dev_cit as external config_item_type.

This includes a new struct target_backend_cits to hold the external CITs
within struct se_subsystem_api, and target_core_setup_sub_cits() to be
used by backend drivers ahead of transport_subsystem_register().

It adds a TB_CIT_SETUP() helper following target_core_fabric_configfs.c
to perform the config_item_type assignments.

Also, drop left-over target_core_dev_cit from target_core_configfs.c code
and update comments.
Signed-off-by: Nicholas Bellinger <nab@linux-iscsi.org>