Return-Path: X-Sieve: cmu-sieve 2.0 Return-Path: Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id h08GrAm17080 for ips-outgoing; Wed, 8 Jan 2003 11:53:10 -0500 (EST) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from mail.ivivity.com ([64.238.111.99]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id h08Gr4W17069; Wed, 8 Jan 2003 11:53:04 -0500 (EST) Received: by ATLOPS with Internet Mail Service (5.5.2653.19) id ; Wed, 8 Jan 2003 11:53:03 -0500 Message-ID: From: Eddy Quicksall To: Julian Satran , "David J. Brown" Cc: Black_David@emc.com, ips@ece.cmu.edu, owner-ips@ece.cmu.edu Subject: RE: iSCSI: Implicit Termination of Tasks Date: Wed, 8 Jan 2003 11:53:03 -0500 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2653.19) Content-Type: multipart/alternative; boundary="----_=_NextPart_001_01C2B736.688FB280" Sender: owner-ips@ece.cmu.edu Precedence: bulk This message is in MIME format. Since your mail reader does not understand this format, some or all of this message may not be legible. ------_=_NextPart_001_01C2B736.688FB280 Content-Type: text/plain; charset="iso-8859-1" The "with respect to ordering" should probably be removed as it may be misleading to some cases not yet thought of. Eddy -----Original Message----- From: Julian Satran [mailto:Julian_Satran@il.ibm.com] Sent: Wednesday, January 08, 2003 10:55 AM To: David J. Brown Cc: Black_David@emc.com; ips@ece.cmu.edu; owner-ips@ece.cmu.edu Subject: RE: iSCSI: Implicit Termination of Tasks oops - thanks - Julo "David J. Brown" 08/01/03 17:04 To Julian Satran/Haifa/IBM@IBMIL, Black_David@emc.com cc ips@ece.cmu.edu, owner-ips@ece.cmu.edu Subject RE: iSCSI: Implicit Termination of Tasks Minor nit: Please change "active tasks in three cases" to "active tasks in four cases". dj -----Original Message----- From: Julian Satran [mailto:Julian_Satran@il.ibm.com] Sent: Wednesday, January 08, 2003 1:59 AM To: Black_David@emc.com Cc: david.brown@trebia.com; ips@ece.cmu.edu; owner-ips@ece.cmu.edu Subject: RE: iSCSI: Implicit Termination of Tasks How about the following text: If the tasks terminated in the above cases a), b, c) and d)are SCSI tasks, they must be internally terminated with CHECK CONDITION status with a sense key of unit attention and ASC/ASCQ values of 0x6E/0x00 (COMMAND TO LOGICAL UNIT FAILED). This status is only meaningful for appropriately handling the internal SCSI state and SCSI side effects with respect to ordering (e.g., queued commands and ACA) because this status is never communicated back as a terminating status to the initiator. Please pay attention to the fact that the statement about the status not being returned is unchanged. 10.14.5 now reads also: Implicit termination of tasks A target implicitly terminates the active tasks in three cases due to iSCSI protocol: When a connection is implicitly or explicitly logged out with the reason code of "Close the connection" and there are active tasks allegiant to that connection. When a connection fails and eventually the connection state times out (state transition M1 in Section 7.2.2 State Transition Descriptions for Initiators and Targets) and there are active tasks allegiant to that connection. When a successful recovery Logout is performed while there are active tasks allegiant to that connection, and those tasks eventually time out after the Time2Wait and Time2Retain periods without allegiance reassignment. When a connection is implicitly or explicitly logged out with the reason code of "Close the session" and there are active tasks in that session. If the tasks terminated in any of the above cases are SCSI tasks, they must be internally terminated with CHECK CONDITION status with a sense key of unit attention and ASC/ASCQ values of 0x6E/0x00 (COMMAND TO LOGICAL UNIT FAILED). This status is only meaningful for appropriately handling the internal SCSI state and SCSI side effects with respect to ordering (e.g., queued commands and ACA) because this status is never communicated back as a terminating status to the initiator. Julo Black_David@emc.com Sent by: owner-ips@ece.cmu.edu 08/01/03 03:57 To david.brown@trebia.com, ips@ece.cmu.edu cc Subject RE: iSCSI: Implicit Termination of Tasks Folks, We need to drive this issue to closure. The gory details of the cross-initiator effects of task termination with CHECK CONDITION are summarized in Table 25 of Section 5.9.1.2 of SAM-2, and the next four subsections describe how this affects new tasks. As David Brown pointed out, the important behavior to note is that when TST is 0, any such termination affects *all* tasks regardless of initiator, and if it results in an ACA, that affects *all* existing and new tasks until the ACA cleared. FWIW, iSCSI's requirement that autosense MUST be used eliminates the possibility of a new task being created while CA is in effect as the CA is immediately cleared by autosense. Given this, especially its ACA aspects, and the fact that iSCSI needs to support ACA, the distinction currently made in Section 6.5 between connection closure and session closure seems wrong - I'm inclined to agree with David Brown and Eddy Quicksall that all four cases in Section 6.5 need to implicitly terminate the tasks with CHECK CONDITION for consistency, and I also think that David Brown's add text observing that there may be cross-initiator side effects depending on how CA and ACA are handled is reasonable, but I would limit it to about what I just wrote plus advice to consult SAM-2. Anyone want to disagree? Thanks, --David > I don't understand why the generation of CHECK CONDITIONs is immaterial when > a session is closed. When a check condition generates a CA or ACA, it can > affect tasks from other initiators, too. Look at the TST and QErr bits in > the Control mode page. > > If the TST bit, in the Control mode page, is zero and the QErr data field, > in the same mode page, contains 01b, then a check condition causes all tasks > from all initiators to be aborted. > > If TST is zero and QErr is not 01b, then all tasks from other initiators are > temporarily blocked by a CA or ACA. The situation in which an ACA was > generated and the session was closed is particularly problematic, because > the initiator that caused the ACA is gone. One of the other initiators > would need to reset the unit, or send a PERSISTENT RESERVE OUT, to clear the > ACA. > > It wouldn't have done any harm to make the last paragraph of 6.5 apply to > clause (d), namely session closure. It certainly would have been helped if > the paragraph referred to CA and ACA explicitly. Such a statement would > have made clear the underlying assumption, namely that a check condition > never affects other initiators, as long as the target > implements autosense. > > Bottom line: If I understand the SAM and SPC specs correctly, the current > draft of the iSCSI spec is incorrect. Clause (d) of 6.5 should cause the > tasks to be terminated with a check condition, just like clauses (a), (b), > and (c). The check condition is visible externally in two situations: > > 1. Closing a session will cause tasks from other sessions to be aborted, if > QErr contains 01b and TST contains 0. > 2. Closing a session will cause tasks from other sessions to be blocked, if > QErr contains 00b (or 011b) and TST contains 0 and any of the terminated > commands had the NACA bit set. > > dj > > > -----Original Message----- > From: Eddy Quicksall [mailto:Eddy_Quicksall@ivivity.com] > Sent: Tuesday, January 07, 2003 8:52 AM > To: 'Mallikarjun C.'; ips@ece.cmu.edu > Subject: RE: iSCSI: Implicit Termination of Tasks > > > If it never makes it back to the initiator, then it is an > implementation > issue. As long as the target maintains all ordering > requirements (iSCSI out > of order commands and SCSI queued commands), all should be > well and there > would be no requirement imposed in the spec to implement this. > > If would be better to have just given the paragraph below as an > implementation note. As long as we all understand what your > intent was, then > we should be OK (and you have now explained that). > > Thanks > > Eddy > > -----Original Message----- > From: Mallikarjun C. [mailto:cbm@rose.hp.com] > Sent: Monday, January 06, 2003 9:15 PM > To: Eddy_Quicksall@iVivity.com; ips@ece.cmu.edu > Subject: Re: iSCSI: Implicit Termination of Tasks > > > Eddy, > > iSCSI's design goal was to ensure command ordering > even in the face of errors - ultimately, we wanted the SCSI > execution ordering to be correct while running on iSCSI. > This should be able to be guaranteed by the iSCSI targets, > while initiators may/may not employ command ordering. > > For this to happen, iSCSI must both maintain its delivery > ordering, and require SCSI CHECK CONDITIONs to > be locally triggered on the target on iSCSI exception > conditions such as connection terminations. The CHECK > CONDITION would then cause the ACA/persistent-UA to > be invoked if so desired by the initiator. > > This is not "a method of implementing", it's required behavior > to meet iSCSI's architectural guarantee of command ordering. > -- > Mallikarjun > > Mallikarjun Chadalapaka > Networked Storage Architecture > Network Storage Solutions > Hewlett-Packard MS 5668 > Roseville CA 95747 > cbm@rose.hp.com > > > ----- Original Message ----- > From: "Eddy Quicksall" > To: "'Mallikarjun C.'" ; > Sent: Monday, January 06, 2003 12:46 PM > Subject: RE: iSCSI: Implicit Termination of Tasks > > > > Yes, I noticed my typo of "two" vs. "three" but too late. > > > > Below you say that the ordering is iSCSI ordering but section 6.5 is > saying > > SCSI ordering (queued commands is SCSI). So, that leaves me a little > > confused as to what this paragraph is trying to say, especially when > the > > "status is never communicated back ... to the initiator". > > > > I'm probably mis-understanding something. It appears as > though this is > just > > a method of implementing within the target. > > > > Eddy > > > > -----Original Message----- > > From: Mallikarjun C. [mailto:cbm@rose.hp.com] > > Sent: Monday, January 06, 2003 3:12 PM > > To: Eddy_Quicksall@ivivity.com; ips@ece.cmu.edu > > Subject: Re: iSCSI: Implicit Termination of Tasks > > > > > > Comments below. > > -- > > Mallikarjun > > > > Mallikarjun Chadalapaka > > Networked Storage Architecture > > Network Storage Solutions > > Hewlett-Packard MS 5668 > > Roseville CA 95747 > > cbm@rose.hp.com > > > > ----- Original Message ----- > > From: "Eddy Quicksall" > > To: > > Sent: Monday, January 06, 2003 5:16 AM > > Subject: iSCSI: Implicit Termination of Tasks > > > > > > > There are two sections titled Implicit Termination of > Tasks but they > > are > > > slightly different. Which is correct? > > > > Both are correct and consistent. > > > > > > > > Section 6.5 lists 4 items but section 10.14.5 only lists two. > > > > I'm seeing three in 10.14.5..... > > > > Even though 6.5 lists all the four cases, it makes it clear that the > > check condition is to be employed only for three cases - and > > only those three are listed by 10.14.5. Perhaps the text could have > > been a little bit more explicit about this distinction. > > > > > > > > If 6.5 is correct, why is item D not included in the unit > attention? > > SAM-3 > > > says: > > > > It's not so much a SAM issue. The issue we considered was how to > ensure > > iSCSI-standard ordered delivery of commands in the face of errors. > The > > ordered delivery of commands does not make sense for case (d) - that > of > > creating a new session - ordering is not guaranteed anyway across > > sessions. > > > > > ------_=_NextPart_001_01C2B736.688FB280 Content-Type: text/html; charset="iso-8859-1"

The "with respect to ordering" should probably be removed as it may be misleading to some cases not yet thought of.

Eddy

-----Original Message-----
From: Julian Satran [mailto:Julian_Satran@il.ibm.com]
Sent: Wednesday, January 08, 2003 10:55 AM
To: David J. Brown
Cc: Black_David@emc.com; ips@ece.cmu.edu; owner-ips@ece.cmu.edu
Subject: RE: iSCSI: Implicit Termination of Tasks

oops - thanks - Julo

"David J. Brown" <david.brown@trebia.com>
08/01/03 17:04

To
Julian Satran/Haifa/IBM@IBMIL, Black_David@emc.com

cc
ips@ece.cmu.edu, owner-ips@ece.cmu.edu

Subject
RE: iSCSI: Implicit Termination of Tasks

Minor nit: Please change "active tasks in three cases" to "active tasks in four cases".

dj
-----Original Message-----
From: Julian Satran [mailto:Julian_Satran@il.ibm.com]
Sent: Wednesday, January 08, 2003 1:59 AM
To: Black_David@emc.com
Cc: david.brown@trebia.com; ips@ece.cmu.edu; owner-ips@ece.cmu.edu
Subject: RE: iSCSI: Implicit Termination of Tasks

How about the following text:

If the tasks terminated in the above cases a), b, c) and d)are SCSI tasks, they must be internally terminated with CHECK CONDITION status with a sense key of unit attention and ASC/ASCQ values of 0x6E/0x00 (COMMAND TO LOGICAL UNIT FAILED). This status is only meaningful for appropriately handling the internal SCSI state and SCSI side effects with respect to ordering (e.g., queued commands and ACA) because this status is never communicated back as a terminating status to the initiator.

Please pay attention to the fact that the statement about the status not being returned is unchanged.

10.14.5 now reads also:

Implicit termination of tasks
A target implicitly terminates the active tasks in three cases due to iSCSI protocol:

When a connection is implicitly or explicitly logged out with the reason code of "Close the connection" and there are active tasks allegiant to that connection.

When a connection fails and eventually the connection state times out (state transition M1 in Section 7.2.2 State Transition Descriptions for Initiators and Targets) and there are active tasks allegiant to that connection.

When a successful recovery Logout is performed while there are active tasks allegiant to that connection, and those tasks eventually time out after the Time2Wait and Time2Retain periods without allegiance reassignment.

When a connection is implicitly or explicitly logged out with the reason code of "Close the session" and there are active tasks in that session.

If the tasks terminated in any of the above cases are SCSI tasks, they must be internally terminated with CHECK CONDITION status with a sense key of unit attention and ASC/ASCQ values of 0x6E/0x00 (COMMAND TO LOGICAL UNIT FAILED). This status is only meaningful for appropriately handling the internal SCSI state and SCSI side effects with respect to ordering (e.g., queued commands and ACA) because this status is never communicated back as a terminating status to the initiator.

Julo

Black_David@emc.com
Sent by: owner-ips@ece.cmu.edu
08/01/03 03:57

To
david.brown@trebia.com, ips@ece.cmu.edu

cc

Subject
RE: iSCSI: Implicit Termination of Tasks

Folks, We need to drive this issue to closure. The gory details of the cross-initiator effects of task termination with CHECK CONDITION are summarized in Table 25 of Section 5.9.1.2 of SAM-2, and the next four subsections describe how this affects new tasks. As David Brown pointed out, the important behavior to note is that when TST is 0, any such termination affects *all* tasks regardless of initiator, and if it results in an ACA, that affects *all* existing and new tasks until the ACA cleared. FWIW, iSCSI's requirement that autosense MUST be used eliminates the possibility of a new task being created while CA is in effect as the CA is immediately cleared by autosense. Given this, especially its ACA aspects, and the fact that iSCSI needs to support ACA, the distinction currently made in Section 6.5 between connection closure and session closure seems wrong - I'm inclined to agree with David Brown and Eddy Quicksall that all four cases in Section 6.5 need to implicitly terminate the tasks with CHECK CONDITION for consistency, and I also think that David Brown's add text observing that there may be cross-initiator side effects depending on how CA and ACA are handled is reasonable, but I would limit it to about what I just wrote plus advice to consult SAM-2. Anyone want to disagree? Thanks, --David > I don't understand why the generation of CHECK CONDITIONs is immaterial when > a session is closed. When a check condition generates a CA or ACA, it can > affect tasks from other initiators, too. Look at the TST and QErr bits in > the Control mode page. > > If the TST bit, in the Control mode page, is zero and the QErr data field, > in the same mode page, contains 01b, then a check condition causes all tasks > from all initiators to be aborted. > > If TST is zero and QErr is not 01b, then all tasks from other initiators are > temporarily blocked by a CA or ACA. The situation in which an ACA was > generated and the session was closed is particularly problematic, because > the initiator that caused the ACA is gone. One of the other initiators > would need to reset the unit, or send a PERSISTENT RESERVE OUT, to clear the > ACA. > > It wouldn't have done any harm to make the last paragraph of 6.5 apply to > clause (d), namely session closure. It certainly would have been helped if > the paragraph referred to CA and ACA explicitly. Such a statement would > have made clear the underlying assumption, namely that a check condition > never affects other initiators, as long as the target > implements autosense. > > Bottom line: If I understand the SAM and SPC specs correctly, the current > draft of the iSCSI spec is incorrect. Clause (d) of 6.5 should cause the > tasks to be terminated with a check condition, just like clauses (a), (b), > and (c). The check condition is visible externally in two situations: > > 1. Closing a session will cause tasks from other sessions to be aborted, if > QErr contains 01b and TST contains 0. > 2. Closing a session will cause tasks from other sessions to be blocked, if > QErr contains 00b (or 011b) and TST contains 0 and any of the terminated > commands had the NACA bit set. > > dj > > > -----Original Message----- > From: Eddy Quicksall [mailto:Eddy_Quicksall@ivivity.com] > Sent: Tuesday, January 07, 2003 8:52 AM > To: 'Mallikarjun C.'; ips@ece.cmu.edu > Subject: RE: iSCSI: Implicit Termination of Tasks > > > If it never makes it back to the initiator, then it is an > implementation > issue. As long as the target maintains all ordering > requirements (iSCSI out > of order commands and SCSI queued commands), all should be > well and there > would be no requirement imposed in the spec to implement this. > > If would be better to have just given the paragraph below as an > implementation note. As long as we all understand what your > intent was, then > we should be OK (and you have now explained that). > > Thanks > > Eddy > > -----Original Message----- > From: Mallikarjun C. [mailto:cbm@rose.hp.com] > Sent: Monday, January 06, 2003 9:15 PM > To: Eddy_Quicksall@iVivity.com; ips@ece.cmu.edu > Subject: Re: iSCSI: Implicit Termination of Tasks > > > Eddy, > > iSCSI's design goal was to ensure command ordering > even in the face of errors - ultimately, we wanted the SCSI > execution ordering to be correct while running on iSCSI. > This should be able to be guaranteed by the iSCSI targets, > while initiators may/may not employ command ordering. > > For this to happen, iSCSI must both maintain its delivery > ordering, and require SCSI CHECK CONDITIONs to > be locally triggered on the target on iSCSI exception > conditions such as connection terminations. The CHECK > CONDITION would then cause the ACA/persistent-UA to > be invoked if so desired by the initiator. > > This is not "a method of implementing", it's required behavior > to meet iSCSI's architectural guarantee of command ordering. > -- > Mallikarjun > > Mallikarjun Chadalapaka > Networked Storage Architecture > Network Storage Solutions > Hewlett-Packard MS 5668 > Roseville CA 95747 > cbm@rose.hp.com > > > ----- Original Message ----- > From: "Eddy Quicksall" <Eddy_Quicksall@iVivity.com> > To: "'Mallikarjun C.'" <cbm@rose.hp.com>; <ips@ece.cmu.edu> > Sent: Monday, January 06, 2003 12:46 PM > Subject: RE: iSCSI: Implicit Termination of Tasks > > > > Yes, I noticed my typo of "two" vs. "three" but too late. > > > > Below you say that the ordering is iSCSI ordering but section 6.5 is > saying > > SCSI ordering (queued commands is SCSI). So, that leaves me a little > > confused as to what this paragraph is trying to say, especially when > the > > "status is never communicated back ... to the initiator". > > > > I'm probably mis-understanding something. It appears as > though this is > just > > a method of implementing within the target. > > > > Eddy > > > > -----Original Message----- > > From: Mallikarjun C. [mailto:cbm@rose.hp.com] > > Sent: Monday, January 06, 2003 3:12 PM > > To: Eddy_Quicksall@ivivity.com; ips@ece.cmu.edu > > Subject: Re: iSCSI: Implicit Termination of Tasks > > > > > > Comments below. > > -- > > Mallikarjun > > > > Mallikarjun Chadalapaka > > Networked Storage Architecture > > Network Storage Solutions > > Hewlett-Packard MS 5668 > > Roseville CA 95747 > > cbm@rose.hp.com > > > > ----- Original Message ----- > > From: "Eddy Quicksall" <Eddy_Quicksall@ivivity.com> > > To: <ips@ece.cmu.edu> > > Sent: Monday, January 06, 2003 5:16 AM > > Subject: iSCSI: Implicit Termination of Tasks > > > > > > > There are two sections titled Implicit Termination of > Tasks but they > > are > > > slightly different. Which is correct? > > > > Both are correct and consistent. > > > > > > > > Section 6.5 lists 4 items but section 10.14.5 only lists two. > > > > I'm seeing three in 10.14.5..... > > > > Even though 6.5 lists all the four cases, it makes it clear that the > > check condition is to be employed only for three cases - and > > only those three are listed by 10.14.5. Perhaps the text could have > > been a little bit more explicit about this distinction. > > > > > > > > If 6.5 is correct, why is item D not included in the unit > attention? > > SAM-3 > > > says: > > > > It's not so much a SAM issue. The issue we considered was how to > ensure > > iSCSI-standard ordered delivery of commands in the face of errors. > The > > ordered delivery of commands does not make sense for case (d) - that > of > > creating a new session - ordering is not guaranteed anyway across > > sessions. > > > > >

------_=_NextPart_001_01C2B736.688FB280--