Return-Path: X-Sieve: cmu-sieve 2.0 Return-Path: Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id h0EFY5t02687 for ips-outgoing; Tue, 14 Jan 2003 10:34:05 -0500 (EST) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from d12lmsgate-2.de.ibm.com (d12lmsgate-2.de.ibm.com [194.196.100.235]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id h0EFY2W02676 for ; Tue, 14 Jan 2003 10:34:02 -0500 (EST) Received: from d12relay02.de.ibm.com (d12relay02.de.ibm.com [9.165.215.23]) by d12lmsgate-2.de.ibm.com (8.12.3/8.12.3) with ESMTP id h0EFXsqM011492; Tue, 14 Jan 2003 16:33:54 +0100 Received: from d10ml001.telaviv.ibm.com (d10ml001.telaviv.ibm.com [9.148.216.55]) by d12relay02.de.ibm.com (8.12.3/NCO/VER6.4) with ESMTP id h0EFXrmZ121226; Tue, 14 Jan 2003 16:33:53 +0100 In-Reply-To: <000501c2bbda$be5b8a10$6403a8c0@ivvt2dxrc11> To: Cc: Black_David@emc.com, dcuddihy@attotech.com, "Elliott, Robert \(Server Storage\)" , ips@ece.cmu.edu Subject: RE: iSCSI: Implicit Termination of Tasks MIME-Version: 1.0 X-Mailer: Lotus Notes Release 6.0 September 26, 2002 From: "Julian Satran" Message-ID: Date: Tue, 14 Jan 2003 17:33:50 +0200 X-MIMETrack: Serialize by Router on D10ML001/10/M/IBM(Release 5.0.9a |January 7, 2002) at 14/01/2003 17:33:53, Serialize complete at 14/01/2003 17:33:53 Content-Type: multipart/alternative; boundary="=_alternative 0054CCE4C2256CAE_=" Sender: owner-ips@ece.cmu.edu Precedence: bulk This is a multipart message in MIME format. --=_alternative 0054CCE4C2256CAE_= Content-Type: text/plain; charset="US-ASCII" Eddy, Setting the UA is a purely SCSI matter and so is the code used. There is no iSCSI state surviving and we assume that the isCSI initiator produces some termination code for the SCSI initiator. It is not clear to me than anything besides "ACA active" code is needed. If the target messes with command from other initiators (single queue-per-target) those should be ready to accept the consequences of a common queue. UA is used (IMHO) mostly in cases of reset - that kill commands in other queues even when the operation is queue-per-initiator. Julo "Eddy Quicksall" 14/01/03 16:39 Please respond to To Julian Satran/Haifa/IBM@IBMIL cc , , , "Elliott, Robert \(Server Storage\)" Subject RE: iSCSI: Implicit Termination of Tasks >>nothing specific has to survive for ACA For ACA, doesn't the next command get the UA then the additional commands get ACA Active until ACA is cleared (all assuming NACA, the ACA attribute and QErr are being used appropriately)? So, doesn't the UA have to survive? Eddy -----Original Message----- From: Julian Satran [mailto:Julian_Satran@il.ibm.com] Sent: Tuesday, January 14, 2003 3:50 AM To: Elliott, Robert (Server Storage) Cc: Black_David@emc.com; dcuddihy@attotech.com; Eddy Quicksall; ips@ece.cmu.edu Subject: RE: iSCSI: Implicit Termination of Tasks Rob, This is perfectly reasonable. The clearing effects will wipe-out anyhow whatever was specific to the code for session termination and nothing specific has to survive for ACA. Regards, Julo "Elliott, Robert (Server Storage)" 14/01/03 09:59 To "Eddy Quicksall" , , , Julian Satran/Haifa/IBM@IBMIL, cc Subject RE: iSCSI: Implicit Termination of Tasks A few comments on this thread: * Autosense protocols simply do not have CA. At the T10 CAP meeting this week, Ralph Weber is proposing removing all references to CA from SAM-3 (see 03-002r2 on http://www.t10.org). As soon as a command completes, it is assumed that the status and sense data will be returned together. If they can't be delivered, it doesn't matter. * the clearing effects table in FCP-2 was removed because it had several incorrect entries and was missing quite a few additional SCSI items that are also affected by logouts. T10 proposals 02-143r3 and 02-232r2 have the details and include large tables listing all the considered SCSI effects. These proposals have been incorporated into the latest SAM-3, SPC-3, SBC-2, and SSC-2 standards. * Most protocols don't detail how their hypothetical SCSI layer confirms the abort of tasks to the protocol layer by specifying a CHECK CONDITION, much less one with a specific sense key and additional sense code. This level of detail is internal only and seems irrelevent to the iSCSI wire protocol. Specifying a "unit attention" sense key could cause particular confusion, since unit attentions are supposed to be cleared once reported. On the iSCSI wire, the goal is to generate a unit attention with an additional sense code whose ASC field is 29h (e.g. I_T NEXUS LOSS, POWER ON OCCURRED, etc.) on the next command. To minimize changes to iSCSI at this point, I recommend changing the wording like this (changed text noted between _ characters) so only status is mentioned and it is only mentioned as an example: If the tasks terminated in the above cases a), b, c) and d)are SCSI tasks, they must be internally terminated _with CHECK CONDITION status with a sense key of unit attention and ASC/ASCQ values of 0x6E/0x00 (COMMAND TO LOGICAL UNIT FAILED)_. This status is only meaningful for appropriately handling the internal SCSI state and SCSI side effects with respect to ordering (e.g., queued commands and ACA) because this status is never communicated back as a terminating status to the initiator. To: If the tasks terminated in the above cases a), b, c) and d) are SCSI tasks, they must be internally terminated _as if with CHECK CONDITION status_. This status is only meaningful for appropriately handling the internal SCSI state and SCSI side effects with respect to ordering (e.g., queued commands and ACA) because this status is never communicated back as a terminating status to the initiator. --- Rob Elliott, HP Industry Standard Server Storage elliott@hp.com -----Original Message----- From: Eddy Quicksall [mailto:eddy_quicksall@ivivity.com] Sent: Thursday, January 09, 2003 2:39 PM To: dcuddihy@attotech.com; ips@ece.cmu.edu Subject: RE: iSCSI: Implicit Termination of Tasks The way I figured it would work is that the UA would get set and would get reported when the next command arrives. Consider this: connection 0 and 1 are all quite connection 0 has a command x on the wire connection 0 is lost the UA is set command y comes in connection 1 command y gets the UA or ACA Active That way the upper layer at the initiator can figure out how to recover. Is this not the idea? Eddy -----Original Message----- From: dcuddihy@attotech.com [mailto:dcuddihy@attotech.com] Sent: Thursday, January 09, 2003 3:05 PM To: ips@ece.cmu.edu Subject: RE: iSCSI: Implicit Termination of Tasks David, Regarding Autosense clearing the CA: The problem that I have with this is that the CA should get cleared when the sense data is reported, not when the sense data is generated. In this case, the autosense cannot be reported (the port is logged out), so the CA may not be cleared. It doesn't seem to me that generating a sense data response, even for an interface in which autosense is mandatory, implicitly clears a CA. By the way, there was a question of Request Sense commands enqueued when a termination occurs .. my understanding is that the sense data need not be saved for a subsequent request sense command if it has been reported via autosense, so the request sense returns NO_SENSE. Regarding FCP handling of port logout and ACA: For us, ACA is a non-issue-- we don't support it, but it should be cleared only for the initiator which generated the log event. (ACA support seems to be a point of contention among SCSI implementers... I know it's in all the specs, but I have yet to see it implemented.) Rev 7A of FCP-2 has a nice chart of how task managment is affected via the various log events. I'm not why it was removed in rev 8, but "FCP-2 Rev 7A (1/1/2001)" Table 4 might help. regards, david David J Cuddihy Principal Engineer ATTO Technology, Inc. http://www.attotech.com/fcbridge.html dcuddihy@attotech.com ------------------------------------------------------------------------ ---- --------------------- ------------------------------------------------------------------------ ---- --------------------- David, > Forgive me for being confused, but what is the reason for the Check > Condition being issued? Good question ... > Say a target implements a blocking CA, (TST 0 Qerr 0) which would disallow > other initiator's tasks from running until the check condition could > be reported, or another command from the CA'd initiator completes with > good status. An internal, non- reported check-condition would cause > all other initiators' tasks to be blocked until someone times out and > issues a lun reset to the device, or a command from the first > initiator completed (not likely, since that initiator logged out and all its tasks have been > terminated ). All this caused by bad behavior on behalf of one > initiator. That shouldn't happen. iSCSI REQUIRES use of Autosense, which will immediately clear the CA by generating a response to report the sense. That response gets bit-bucketed as undeliverable for the implicit termination scenarios in Section 6.5, but its generation clears the CA. Not the most obvious thing to do/explain, though ... > Implicit termination is handled in FCP without the check condition > being issued... all the tasks from the 'logged out' initiator are > terminated, reservations cleared, etc. and a UA is generated for that > initiator. Every other initiator goes along with no problem, including > those sharing the lun with the initiator who abruptly logged out. > Seems to work. That looks like a cleaner way to accomplish this and avoids some of the confusion over what ASC/ASCQ values to use. Does FCP also avoid ACA in this circumstance (assuming that NACA is set to use ACA), and if so, can you suggest some specific text for iSCSI? Thanks, --David ---------------------------------------------------- David L. Black, Senior Technologist EMC Corporation, 176 South St., Hopkinton, MA 01748 +1 (508) 293-7953 **NEW** FAX: +1 (508) 293-7786 black_david@emc.com Mobile: +1 (978) 394-7754 ---------------------------------------------------- --=_alternative 0054CCE4C2256CAE_= Content-Type: text/html; charset="US-ASCII"
Eddy,

Setting the UA is a purely SCSI matter and so is the code used. There is no iSCSI state surviving and we assume that the isCSI initiator
produces some termination code for the SCSI initiator. It is not clear to me than anything besides "ACA active" code is needed.
If the target messes with command from other initiators (single queue-per-target)  those should be ready to accept the consequences of a common queue.
UA is used (IMHO) mostly in cases of reset - that kill commands in other queues even when the operation is queue-per-initiator.

Julo


"Eddy Quicksall" <Eddy_Quicksall@iVivity.com>

14/01/03 16:39
Please respond to
<Eddy_Quicksall@iVivity.com>
To
Julian Satran/Haifa/IBM@IBMIL
cc
<Black_David@emc.com>, <dcuddihy@attotech.com>, <ips@ece.cmu.edu>, "Elliott, Robert \(Server Storage\)" <Elliott@hp.com>
Subject
RE: iSCSI: Implicit Termination of Tasks





>>nothing specific has to survive for ACA
 
For ACA, doesn't the next command get the UA then the additional commands get ACA Active until ACA is cleared (all assuming NACA, the ACA attribute and QErr are being used appropriately)?
 
So, doesn't the UA have to survive?
 
Eddy
-----Original Message-----
From: Julian Satran [mailto:Julian_Satran@il.ibm.com]
Sent: Tuesday, January 14, 2003 3:50 AM
To: Elliott, Robert (Server Storage)
Cc: Black_David@emc.com; dcuddihy@attotech.com; Eddy Quicksall; ips@ece.cmu.edu
Subject: RE: iSCSI: Implicit Termination of Tasks

Rob,

This is perfectly reasonable.
The clearing effects will wipe-out anyhow whatever was specific to the code for session termination
and nothing specific has to survive for ACA.

Regards,
Julo

"Elliott, Robert (Server Storage)" <Elliott@hp.com>

14/01/03 09:59


To
"Eddy Quicksall" <eddy_quicksall@ivivity.com>, <dcuddihy@attotech.com>, <ips@ece.cmu.edu>, Julian Satran/Haifa/IBM@IBMIL, <Black_David@emc.com>
cc
Subject
RE: iSCSI: Implicit Termination of Tasks







A few comments on this thread:

* Autosense protocols simply do not have CA.  At the T10 CAP meeting
this week, Ralph Weber is proposing removing all references to CA from
SAM-3 (see 03-002r2 on http://www.t10.org).  As soon as a command
completes, it is assumed that the status and sense data will be returned
together.  If they can't be delivered, it doesn't matter.

* the clearing effects table in FCP-2 was removed because it had several
incorrect entries and was missing quite a few additional SCSI items that
are also affected by logouts.  T10 proposals 02-143r3 and 02-232r2 have
the details and include large tables listing all the considered SCSI
effects.  These proposals have been incorporated into the latest SAM-3,
SPC-3, SBC-2, and SSC-2 standards.

* Most protocols don't detail how their hypothetical SCSI layer confirms
the abort of tasks to the protocol layer by specifying a CHECK
CONDITION, much less one with a specific sense key and additional sense
code.  This level of detail is internal only and seems irrelevent to the
iSCSI wire protocol. Specifying a "unit attention" sense key could cause
particular confusion, since unit attentions are supposed to be cleared
once reported.  

On the iSCSI wire, the goal is to generate a unit attention with an
additional sense code whose ASC field is 29h (e.g. I_T NEXUS LOSS, POWER
ON OCCURRED, etc.) on the next command.

To minimize changes to iSCSI at this point, I recommend changing
the wording like this (changed text noted between _ characters)
so only status is mentioned and it is only mentioned as an example:

If the tasks terminated in the above cases a), b, c) and d)are SCSI
tasks, they must be internally terminated _with CHECK CONDITION
status with a sense key of unit attention and ASC/ASCQ values of
0x6E/0x00 (COMMAND TO LOGICAL UNIT FAILED)_. This status is only
meaningful for appropriately handling the internal SCSI state and
SCSI side effects with respect to ordering (e.g., queued commands
and ACA) because this status is never communicated back as a
terminating status to the initiator.

To:
If the tasks terminated in the above cases a), b, c) and d) are SCSI
tasks, they must be internally terminated _as if with CHECK CONDITION
status_.  This status is only
meaningful for appropriately handling the internal SCSI state and
SCSI side effects with respect to ordering (e.g., queued commands
and ACA) because this status is never communicated back as a
terminating status to the initiator.

---
Rob Elliott, HP Industry Standard Server Storage
elliott@hp.com

-----Original Message-----
From: Eddy Quicksall [mailto:eddy_quicksall@ivivity.com]
Sent: Thursday, January 09, 2003 2:39 PM
To: dcuddihy@attotech.com; ips@ece.cmu.edu
Subject: RE: iSCSI: Implicit Termination of Tasks


The way I figured it would work is that the UA would get set and would
get reported when the next command arrives.

Consider this:

connection 0 and 1 are all quite
connection 0 has a command x on the wire
connection 0 is lost
the UA is set
command y comes in connection 1
command y gets the UA or ACA Active

That way the upper layer at the initiator can figure out how to recover.

Is this not the idea?

Eddy

-----Original Message-----
From: dcuddihy@attotech.com [mailto:dcuddihy@attotech.com]
Sent: Thursday, January 09, 2003 3:05 PM
To: ips@ece.cmu.edu
Subject: RE: iSCSI: Implicit Termination of Tasks


David,

Regarding Autosense clearing the CA:

The problem that I have with this is that the CA should get cleared when
the sense data is reported, not when the sense data is generated.  In
this case, the autosense cannot be reported (the port is logged out), so
the CA may not be cleared. It doesn't seem to me that generating a sense
data response, even for an interface in which autosense is mandatory,
implicitly clears a CA.

By the way, there was a question of Request Sense commands enqueued when
a termination occurs .. my understanding is that the sense data need not
be saved for a subsequent request sense command if it has been reported
via autosense, so the request sense returns NO_SENSE.

Regarding FCP handling of port logout and ACA:

For us, ACA is a non-issue-- we don't support it, but it should be
cleared only for the initiator which generated the log event.  (ACA
support seems to be a point of contention among SCSI implementers... I
know it's in all the specs, but I have yet to see it implemented.)

Rev 7A of FCP-2 has a nice chart of how task managment is affected via
the various log events.  I'm not why it was removed in rev 8, but "FCP-2
Rev 7A (1/1/2001)" Table 4 might help.

regards,

david




David J Cuddihy
Principal Engineer
ATTO Technology, Inc.
http://www.attotech.com/fcbridge.html
dcuddihy@attotech.com

------------------------------------------------------------------------
----
---------------------


------------------------------------------------------------------------
----
---------------------

David,

> Forgive me for being confused, but what is the reason for the Check
> Condition being issued?

Good question ...

> Say a target implements a blocking CA, (TST 0 Qerr 0) which would
disallow
> other initiator's tasks from running until the check condition could
> be reported, or another command from the CA'd initiator completes with

> good status.  An internal, non- reported check-condition would cause
> all other initiators' tasks to be blocked until  someone times out and

> issues a lun reset to the device, or a command from the first
> initiator completed (not likely, since that initiator logged out and
all its tasks have been
> terminated ).    All this caused by bad behavior on behalf of one
> initiator.

That shouldn't happen.  iSCSI REQUIRES use of Autosense, which will
immediately clear the CA by generating a response to report the sense.
That response gets bit-bucketed as undeliverable for the implicit
termination scenarios in Section 6.5, but its generation clears the CA.
Not the most obvious thing to do/explain, though ...

> Implicit termination is handled in FCP without the check condition
> being issued... all the tasks from the 'logged out' initiator are
> terminated, reservations cleared, etc.  and a UA is generated for that

> initiator. Every other initiator goes along with no problem, including

> those sharing the lun with the initiator who abruptly logged out.  
> Seems to work.

That looks like a cleaner way to accomplish this and avoids some of the
confusion over what ASC/ASCQ values to use.  Does FCP also avoid ACA in
this circumstance (assuming that NACA is set to use ACA), and if so, can
you suggest some specific text for iSCSI?

Thanks,
--David
----------------------------------------------------
David L. Black, Senior Technologist
EMC Corporation, 176 South St., Hopkinton, MA  01748
+1 (508) 293-7953 **NEW**     FAX: +1 (508) 293-7786
black_david@emc.com        Mobile: +1 (978) 394-7754
----------------------------------------------------
--=_alternative 0054CCE4C2256CAE_=--