From ips-bounces@ietf.org Sun Dec 02 22:20:56 2007 Return-path: Received: from [127.0.0.1] (helo=stiedprmman1.va.neustar.com) by megatron.ietf.org with esmtp (Exim 4.43) id 1Iz1rP-0000FU-Gm; Sun, 02 Dec 2007 22:20:43 -0500 Received: from ips by megatron.ietf.org with local (Exim 4.43) id 1Iz1rO-0000FC-7B for ips-confirm+ok@megatron.ietf.org; Sun, 02 Dec 2007 22:20:42 -0500 Received: from [10.90.34.44] (helo=chiedprmail1.ietf.org) by megatron.ietf.org with esmtp (Exim 4.43) id 1Iz1rN-0000Ct-Rn for ips@ietf.org; Sun, 02 Dec 2007 22:20:41 -0500 Received: from ausc60pc101.us.dell.com ([143.166.85.206]) by chiedprmail1.ietf.org with esmtp (Exim 4.43) id 1Iz1rL-00017J-Fh for ips@ietf.org; Sun, 02 Dec 2007 22:20:39 -0500 DomainKey-Signature: s=smtpout; d=dell.com; c=nofws; q=dns; h=X-IronPort-AV:X-MimeOLE:Content-class:MIME-Version: Content-Type:Subject:Date:Message-ID:X-MS-Has-Attach: X-MS-TNEF-Correlator:Thread-Topic:Thread-Index:From:To:Cc: Return-Path:X-OriginalArrivalTime; b=Ij3UYQF8iAGTKyC4xy01fjMYTPPCYn3UFSccuHgathSu1rxW/S91hZYl 3L8KzBmge2VgRTFGZRnvFq1CS+3CpK54na4hqOqNRWh+49+ssEPvUCZJx FTj6JhZa8symopz; X-IronPort-AV: E=Sophos;i="4.23,241,1194242400"; d="scan'208,217";a="454365117" X-MimeOLE: Produced By Microsoft Exchange V6.5 Content-class: urn:content-classes:message MIME-Version: 1.0 Date: Sun, 2 Dec 2007 21:20:37 -0600 Message-ID: <5DDAB7BA7BDB58439DD0EED0B8E9A3AEFD9CFF@ausx3mpc102.aus.amer.dell.com> X-MS-Has-Attach: X-MS-TNEF-Correlator: Thread-Topic: Immediate Commands in iSCSI Thread-Index: Acg1FA80xnV1/ZyrQrWRgdXVcTU4Tg== From: To: X-OriginalArrivalTime: 03 Dec 2007 03:20:38.0123 (UTC) FILETIME=[79C3CBB0:01C8355B] X-Spam-Score: 0.0 (/) X-Scan-Signature: 36fb765c89ed47dab364ab702a78e8fd Cc: Nam_Nguyen@Dell.com Subject: [Ips] Immediate Commands in iSCSI X-BeenThere: ips@ietf.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: IP Storage List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Content-Type: multipart/mixed; boundary="===============2074212125==" Errors-To: ips-bounces@ietf.org This is a multi-part message in MIME format. --===============2074212125== Content-class: urn:content-classes:message Content-Type: multipart/alternative; boundary="----_=_NextPart_001_01C8355B.798A1ACC" This is a multi-part message in MIME format. ------_=_NextPart_001_01C8355B.798A1ACC Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable RFC 3720 states: The number of commands used for immediate delivery is not limited and their delivery for execution is not acknowledged through the numbering scheme. Immediate commands MAY be rejected by the iSCSI target layer due to a lack of resources. An iSCSI target MUST be able to handle at least one immediate task management command and one immediate non-task-management iSCSI command per connection at any time. =20 With storage systems that have very large number of logical units, we run into an issue when we use LU Reset or Abort Task for reset recovery. I couldn't find a way to tell an initiator how many immediate commands a target can accept. The issue with this is that if we have to issue multiple task management commands and it is more than what the target can accept, it will reject the task management request. Since we don't know why it was rejected, we will have to escalate the reset. This can be very disruptive with nodes that have very large number of logical units. There may be two ways to resolve this: =20 1) Define a new Text negotiation key for number of outstanding immediate commands supported. Unlike non-immediate (data) commands where we can send TASK SET FULL, there is no way for a target to report in the Task Management Function Response PDU that is unable to process the current request before it is out of resources. The default vaule for this new key can be 1, so if this key is not negotiated initiators can assume one per connection. 2) Define a new value for the Response field in the Task Management Function Response PDU for reporting that the target cannot process the request currently. =20 Any suggestions on how to resolve this? =20 Thanks, Jacob ------_=_NextPart_001_01C8355B.798A1ACC Content-Type: text/html; charset="us-ascii" Content-Transfer-Encoding: quoted-printable

RFC 3720 states:

The number of commands used for immediate = delivery is not limited and their delivery for execution is not acknowledged = through the numbering scheme. Immediate commands MAY be rejected by the iSCSI target = layer due to a lack of resources. An iSCSI target MUST be able to handle at = least one immediate task management command and one immediate non-task-management = iSCSI command per connection at any time.

 

With storage systems that have very large number of = logical units, we run into an issue when we use LU Reset or Abort Task for reset recovery. I couldn’t find a way to tell an initiator how many = immediate commands a target can accept. The issue with this is that if we have to = issue multiple task management commands and it is more than what the target = can accept, it will reject the task management request. Since we don’t = know why it was rejected, we will have to escalate the reset. This can be = very disruptive with nodes that have very large number of logical units. = There may be two ways to resolve this:

 

1)      Define a new Text negotiation key for number of outstanding immediate commands supported. Unlike non-immediate (data) = commands where we can send TASK SET FULL, there is no way for a target to report in the = Task Management Function Response PDU that is unable to process the current = request before it is out of resources.  The default vaule for this new key = can be 1, so if this key is not negotiated initiators can assume one per = connection.

2)      Define a new value for the Response field in the = Task Management Function Response PDU for reporting that the target cannot = process the request currently.

 

Any suggestions on how to resolve = this?

 

Thanks,

Jacob

------_=_NextPart_001_01C8355B.798A1ACC-- --===============2074212125== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline _______________________________________________ Ips mailing list Ips@ietf.org https://www1.ietf.org/mailman/listinfo/ips --===============2074212125==-- From ips-bounces@ietf.org Mon Dec 03 00:54:18 2007 Return-path: Received: from [127.0.0.1] (helo=stiedprmman1.va.neustar.com) by megatron.ietf.org with esmtp (Exim 4.43) id 1Iz4G1-00079C-13; Mon, 03 Dec 2007 00:54:17 -0500 Received: from ips by megatron.ietf.org with local (Exim 4.43) id 1Iz4Fz-00078q-Mh for ips-confirm+ok@megatron.ietf.org; Mon, 03 Dec 2007 00:54:15 -0500 Received: from [10.91.34.44] (helo=ietf-mx.ietf.org) by megatron.ietf.org with esmtp (Exim 4.43) id 1Iz4Fz-00078J-Bw for ips@ietf.org; Mon, 03 Dec 2007 00:54:15 -0500 Received: from smtpoutm.mac.com ([17.148.16.80]) by ietf-mx.ietf.org with esmtp (Exim 4.43) id 1Iz4Fy-0004oC-TD for ips@ietf.org; Mon, 03 Dec 2007 00:54:15 -0500 Received: from mac.com (asmtp005-s [10.150.69.68]) by smtpoutm.mac.com (Xserve/smtpout017/MantshX 4.0) with ESMTP id lB35sD2R002873; Sun, 2 Dec 2007 21:54:13 -0800 (PST) Received: from waterdeep.home.54thstreetzoo.com (wsip-72-214-2-130.sd.sd.cox.net [72.214.2.130]) (authenticated bits=0) by mac.com (Xserve/asmtp005/MantshX 4.0) with ESMTP id lB35sB9D019111 (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=NO); Sun, 2 Dec 2007 21:54:11 -0800 (PST) Message-Id: <3269CD91-0A2D-4B6C-B148-CDA8BDE07256@mac.com> From: William Stouder-Studenmund To: Jacob_Cherian@Dell.com In-Reply-To: <5DDAB7BA7BDB58439DD0EED0B8E9A3AEFD9CFF@ausx3mpc102.aus.amer.dell.com> Content-Type: text/plain; charset=WINDOWS-1252; format=flowed; delsp=yes Content-Transfer-Encoding: quoted-printable Mime-Version: 1.0 (Apple Message framework v915) Subject: Re: [Ips] Immediate Commands in iSCSI Date: Sun, 2 Dec 2007 21:54:11 -0800 References: <5DDAB7BA7BDB58439DD0EED0B8E9A3AEFD9CFF@ausx3mpc102.aus.amer.dell.com> X-Mailer: Apple Mail (2.915) X-Spam-Score: 0.0 (/) X-Scan-Signature: e8a67952aa972b528dd04570d58ad8fe Cc: IPS , Nam_Nguyen@Dell.com X-BeenThere: ips@ietf.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: IP Storage List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: ips-bounces@ietf.org On Dec 2, 2007, at 7:20 PM, Jacob_Cherian@Dell.com wrote: > RFC 3720 states: > The number of commands used for immediate delivery is not limited =20 > and their delivery for execution is not acknowledged through the =20 > numbering scheme. Immediate commands MAY be rejected by the iSCSI =20 > target layer due to a lack of resources. An iSCSI target MUST be =20 > able to handle at least one immediate task management command and =20 > one immediate non-task-management iSCSI command per connection at =20 > any time. > > With storage systems that have very large number of logical units, =20 > we run into an issue when we use LU Reset or Abort Task for reset =20 > recovery. I couldn=92t find a way to tell an initiator how many =20 > immediate commands a target can accept. The issue with this is that =20= > if we have to issue multiple task management commands and it is more =20= > than what the target can accept, it will reject the task management =20= > request. Since we don=92t know why it was rejected, we will have to =20= > escalate the reset. This can be very disruptive with nodes that have =20= > very large number of logical units. There may be two ways to resolve =20= > this: > > 1) Define a new Text negotiation key for number of outstanding =20= > immediate commands supported. Unlike non-immediate (data) commands =20 > where we can send TASK SET FULL, there is no way for a target to =20 > report in the Task Management Function Response PDU that is unable =20 > to process the current request before it is out of resources. The =20 > default vaule for this new key can be 1, so if this key is not =20 > negotiated initiators can assume one per connection. If we were to add this key, I think there should be no default. No =20 negotiation result should leave both parties with the current "try it =20= and see what happens" behavior. > 2) Define a new value for the Response field in the Task =20 > Management Function Response PDU for reporting that the target =20 > cannot process the request currently. > > Any suggestions on how to resolve this? Since we're talking about changing the protocol, why not just fix the =20= problem? The problem is a lack of clarity about why an operation =20 failed, so let's add an extra response code to indicate a transient =20 issue. Something to the effect of, "I'm rejecting/failing this, but I =20= think the reasons was transient and may not hold true if you try the =20 command again." I'm not sure if anything other than TMR needs this treatment. Take care, Bill= _______________________________________________ Ips mailing list Ips@ietf.org https://www1.ietf.org/mailman/listinfo/ips From ips-bounces@ietf.org Mon Dec 03 02:23:38 2007 Return-path: Received: from [127.0.0.1] (helo=stiedprmman1.va.neustar.com) by megatron.ietf.org with esmtp (Exim 4.43) id 1Iz5eR-00077D-2V; Mon, 03 Dec 2007 02:23:35 -0500 Received: from ips by megatron.ietf.org with local (Exim 4.43) id 1Iz5eP-00076t-PE for ips-confirm+ok@megatron.ietf.org; Mon, 03 Dec 2007 02:23:33 -0500 Received: from [10.91.34.44] (helo=ietf-mx.ietf.org) by megatron.ietf.org with esmtp (Exim 4.43) id 1Iz5eO-00076j-VH for ips@ietf.org; Mon, 03 Dec 2007 02:23:32 -0500 Received: from mtagate7.de.ibm.com ([195.212.29.156]) by ietf-mx.ietf.org with esmtp (Exim 4.43) id 1Iz5eK-00022j-BJ for ips@ietf.org; Mon, 03 Dec 2007 02:23:32 -0500 Received: from d12nrmr1607.megacenter.de.ibm.com (d12nrmr1607.megacenter.de.ibm.com [9.149.167.49]) by mtagate7.de.ibm.com (8.13.8/8.13.8) with ESMTP id lB37NQV1177752 for ; Mon, 3 Dec 2007 07:23:26 GMT Received: from d12av02.megacenter.de.ibm.com (d12av02.megacenter.de.ibm.com [9.149.165.228]) by d12nrmr1607.megacenter.de.ibm.com (8.13.8/8.13.8/NCO v8.7) with ESMTP id lB37NQj12916542 for ; Mon, 3 Dec 2007 08:23:26 +0100 Received: from d12av02.megacenter.de.ibm.com (loopback [127.0.0.1]) by d12av02.megacenter.de.ibm.com (8.12.11.20060308/8.13.3) with ESMTP id lB37NQrE006437 for ; Mon, 3 Dec 2007 08:23:26 +0100 Received: from d12mc102.megacenter.de.ibm.com (d12mc102.megacenter.de.ibm.com [9.149.167.114]) by d12av02.megacenter.de.ibm.com (8.12.11.20060308/8.12.11) with ESMTP id lB37NQ1Y006432; Mon, 3 Dec 2007 08:23:26 +0100 In-Reply-To: <5DDAB7BA7BDB58439DD0EED0B8E9A3AEFD9CFF@ausx3mpc102.aus.amer.dell.com> To: MIME-Version: 1.0 Subject: Re: [Ips] Immediate Commands in iSCSI X-KeepSent: 2A29EEC8:F7179BE8-C22573A6:00279830; type=4; name=$KeepSent X-Mailer: Lotus Notes Release 7.0.3 September 26, 2007 From: Julian Satran Message-ID: Date: Mon, 3 Dec 2007 09:23:24 +0200 X-MIMETrack: Serialize by Router on D12MC102/12/M/IBM(Release 8.0|August 02, 2007) at 03/12/2007 09:23:26, Serialize complete at 03/12/2007 09:23:26 X-Spam-Score: -4.0 (----) X-Scan-Signature: 72dbfff5c6b8ad2b1b727c13be042129 Cc: ips@ietf.org, Nam_Nguyen@Dell.com X-BeenThere: ips@ietf.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: IP Storage List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Content-Type: multipart/mixed; boundary="===============1738579005==" Errors-To: ips-bounces@ietf.org This is a multipart message in MIME format. --===============1738579005== Content-Type: multipart/related; boundary="=_related 0028976FC22573A6_=" This is a multipart message in MIME format. --=_related 0028976FC22573A6_= Content-Type: multipart/alternative; boundary="=_alternative 0028976FC22573A6_=" --=_alternative 0028976FC22573A6_= Content-Type: text/plain; charset="ISO-8859-1" Content-Transfer-Encoding: quoted-printable It looks to me that the Reject response with reason 0x06 should be enough = for the initiator to handle the situation you describe. If the TM are the cause then each of those has a response indicating=20 termination. After receiving a reject an initiator should reissue new TMs=20 only after getting back one or more responses. The initiator may even=20 accurately guess the number of commands the targets is capable of handling = per connection and per LU. The only case in which this handling may be=20 awkward is over a WAN but if you are operating over a WAN and TM=20 performance is your only worry you are in good shape :-) Julo [Ips] Immediate Commands in iSCSI Jacob=5FCherian=20 to: ips 12/03/07 05:23 AM Cc: Nam=5FNguyen RFC 3720 states: The number of commands used for immediate delivery is not limited and=20 their delivery for execution is not acknowledged through the numbering=20 scheme. Immediate commands MAY be rejected by the iSCSI target layer due=20 to a lack of resources. An iSCSI target MUST be able to handle at least=20 one immediate task management command and one immediate=20 non-task-management iSCSI command per connection at any time. =20 With storage systems that have very large number of logical units, we run=20 into an issue when we use LU Reset or Abort Task for reset recovery. I=20 couldn?t find a way to tell an initiator how many immediate commands a=20 target can accept. The issue with this is that if we have to issue=20 multiple task management commands and it is more than what the target can=20 accept, it will reject the task management request. Since we don?t know=20 why it was rejected, we will have to escalate the reset. This can be very=20 disruptive with nodes that have very large number of logical units. There=20 may be two ways to resolve this: =20 1) Define a new Text negotiation key for number of outstanding=20 immediate commands supported. Unlike non-immediate (data) commands where=20 we can send TASK SET FULL, there is no way for a target to report in the=20 Task Management Function Response PDU that is unable to process the=20 current request before it is out of resources. The default vaule for this = new key can be 1, so if this key is not negotiated initiators can assume=20 one per connection. 2) Define a new value for the Response field in the Task Management=20 Function Response PDU for reporting that the target cannot process the=20 request currently. =20 Any suggestions on how to resolve this? =20 Thanks, Jacob =5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F= =5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F Ips mailing list Ips@ietf.org https://www1.ietf.org/mailman/listinfo/ips --=_alternative 0028976FC22573A6_= Content-Type: text/html; charset="ISO-8859-1" Content-Transfer-Encoding: quoted-printable
It looks to  me that the Reject response with reason 0x06 should be enough for the initiator to handle the situation you describe.
If the TM are the cause then each of those has a response indicating termination. After receiving a reject an initiator should reissue new TMs only after getting back one or more respon= ses.  The initiator may even accurately guess the number of commands the targets is capable of handling per connection and per LU. The only case in which this handling may be awkward is over a WAN but if you are operating over a WAN and TM performance is your only worry you are in good shape :-)

Julo


[Ips] Immediate Comm= ands in iSCSI

Jacob=5FCherian <= /b> to: ips
12/03/07 05:23 AM

Cc: Nam=5FNguyen







RFC 3720 states:

The number of commands used for immediate delivery is not limited and their delivery for execution is not acknowledged through the numbering scheme. Immediate commands MAY be rejected by the iSCSI target layer due to a lack of resources. An iSCSI target MUST be able to handle at least one immediate task management command and one immediate non-task-m= anagement iSCSI command per connection at any time.

 

With storage systems that have very large number of logical units, we run into an issue when we use LU Reset or Abort Task for reset recovery. I couldn’t find a way to tell an initiator how ma= ny immediate commands a target can accept. The issue with this is that if we have to issue multiple task management commands and it is more than what the target can accept, it will reject the task management request. Since we don’t know why it was rejected, we will have to escalate the reset. This can be very disruptive with nodes that have very large number of logical units. There may be two ways to resolve this:

 

1)      Define a new Text negotiation key for number of outstanding immediate commands supported. Unlike non-immediate (data) commands where we can send TASK SET FULL, there is no way for a target to report in the Task Management Function Response PDU that is unable to process the current request before it is out of resou= rces.  The default vaule for this new key can be 1, so if this key is not negotiated initiators can assume one per connection.

2)      Define a new value for the Response field in the Task Management Function Response PDU for reporting that the target cannot process the request currently.

 

Any suggestions on how to resolve this?

 

Thanks,

Jacob
=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F= =5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F= =5F=5F=5F=5F=5F
Ips mailing list
Ips@ietf.org
https://www1.ietf.org/mailman/listinfo/ips

--=_alternative 0028976FC22573A6_=-- --=_related 0028976FC22573A6_= Content-Type: image/gif Content-ID: <_1_1EE038681EE0349400289552C22573A6> Content-Transfer-Encoding: base64 R0lGODlhZAABAOcAAMDAwAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAACH5BAEAAAAALAAAAABkAAEA QAgSAAEIHEiwoMGDCBMqXMiwYcKAADs= --=_related 0028976FC22573A6_=-- --===============1738579005== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline _______________________________________________ Ips mailing list Ips@ietf.org https://www1.ietf.org/mailman/listinfo/ips --===============1738579005==-- From ips-bounces@ietf.org Mon Dec 03 09:05:49 2007 Return-path: Received: from [127.0.0.1] (helo=stiedprmman1.va.neustar.com) by megatron.ietf.org with esmtp (Exim 4.43) id 1IzBvT-0003xn-Gi; Mon, 03 Dec 2007 09:05:35 -0500 Received: from ips by megatron.ietf.org with local (Exim 4.43) id 1IzBvS-0003xR-92 for ips-confirm+ok@megatron.ietf.org; Mon, 03 Dec 2007 09:05:34 -0500 Received: from [10.90.34.44] (helo=chiedprmail1.ietf.org) by megatron.ietf.org with esmtp (Exim 4.43) id 1IzBvR-0003x7-TY for ips@ietf.org; Mon, 03 Dec 2007 09:05:33 -0500 Received: from ausc60ps301.us.dell.com ([143.166.148.206]) by chiedprmail1.ietf.org with esmtp (Exim 4.43) id 1IzBvP-0006Rq-9l for ips@ietf.org; Mon, 03 Dec 2007 09:05:31 -0500 DomainKey-Signature: s=smtpout; d=dell.com; c=nofws; q=dns; h=X-IronPort-AV:X-MimeOLE:Content-class:MIME-Version: Content-Type:Subject:Date:Message-ID:In-Reply-To: X-MS-Has-Attach:X-MS-TNEF-Correlator:Thread-Topic: Thread-Index:References:From:To:Cc:Return-Path: X-OriginalArrivalTime; b=WWKYL3fnp926DcG/TJh2CjvG3iAkBxH6SemxpYoQwck1fPAtSJyUUqma LLXjslO3a6hAblsfWy0CT+o+sSHXoCLOk4gfZM/glgtmtLnV2JqIlSPi9 M15AYHNnrSGQHKk; X-IronPort-AV: E=Sophos;i="4.23,244,1194242400"; d="gif'147?scan'147,208,217,147";a="433185393" X-MimeOLE: Produced By Microsoft Exchange V6.5 Content-class: urn:content-classes:message MIME-Version: 1.0 Subject: RE: [Ips] Immediate Commands in iSCSI Date: Mon, 3 Dec 2007 08:05:28 -0600 Message-ID: <5DDAB7BA7BDB58439DD0EED0B8E9A3AEFD9D2E@ausx3mpc102.aus.amer.dell.com> In-Reply-To: X-MS-Has-Attach: yes X-MS-TNEF-Correlator: Thread-Topic: [Ips] Immediate Commands in iSCSI Thread-Index: Acg1fWkJDi06/gPCSV+IzBXYlbPg+QAN5FtA References: <5DDAB7BA7BDB58439DD0EED0B8E9A3AEFD9CFF@ausx3mpc102.aus.amer.dell.com> From: To: X-OriginalArrivalTime: 03 Dec 2007 14:05:30.0418 (UTC) FILETIME=[902B7920:01C835B5] X-Spam-Score: 0.0 (/) X-Scan-Signature: 90fbdaf3c139ce803b358d56775b59ed Cc: ips@ietf.org, Nam_Nguyen@Dell.com X-BeenThere: ips@ietf.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: IP Storage List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Content-Type: multipart/mixed; boundary="===============1824597339==" Errors-To: ips-bounces@ietf.org This is a multi-part message in MIME format. --===============1824597339== Content-class: urn:content-classes:message Content-Type: multipart/related; type="multipart/alternative"; boundary="----_=_NextPart_001_01C835B5.8FBAB70D" This is a multi-part message in MIME format. ------_=_NextPart_001_01C835B5.8FBAB70D Content-Type: multipart/alternative; boundary="----_=_NextPart_002_01C835B5.8FBAB70D" ------_=_NextPart_002_01C835B5.8FBAB70D Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable The goal of the new status or key is to help the initiator to know the correct number of immediate commands that the target supports. The issue with overloading a status that does clearly define the error condition is that it does help in achieving interoperability between the myriad of initiators and targets that are out there. I was looking to see if there was another way that I had not considered. =20 Jacob =20 =20 From: Julian Satran [mailto:Julian_Satran@il.ibm.com]=20 Sent: Monday, December 03, 2007 1:23 AM To: Cherian, Jacob Cc: ips@ietf.org; Nguyen, Nam Subject: Re: [Ips] Immediate Commands in iSCSI =20 It looks to me that the Reject response with reason 0x06 should be enough for the initiator to handle the situation you describe.=20 If the TM are the cause then each of those has a response indicating termination. After receiving a reject an initiator should reissue new TMs only after getting back one or more responses. The initiator may even accurately guess the number of commands the targets is capable of handling per connection and per LU. The only case in which this handling may be awkward is over a WAN but if you are operating over a WAN and TM performance is your only worry you are in good shape :-)=20 Julo=20 =20 [Ips] Immediate Commands in iSCSI =20 Jacob_Cherian=20 to:=20 ips=20 12/03/07 05:23 AM =20 Cc:=20 Nam_Nguyen =09 =20 ________________________________ RFC 3720 states:=20 The number of commands used for immediate delivery is not limited and their delivery for execution is not acknowledged through the numbering scheme. Immediate commands MAY be rejected by the iSCSI target layer due to a lack of resources. An iSCSI target MUST be able to handle at least one immediate task management command and one immediate non-task-management iSCSI command per connection at any time.=20 =20 With storage systems that have very large number of logical units, we run into an issue when we use LU Reset or Abort Task for reset recovery. I couldn't find a way to tell an initiator how many immediate commands a target can accept. The issue with this is that if we have to issue multiple task management commands and it is more than what the target can accept, it will reject the task management request. Since we don't know why it was rejected, we will have to escalate the reset. This can be very disruptive with nodes that have very large number of logical units. There may be two ways to resolve this:=20 =20 1) Define a new Text negotiation key for number of outstanding immediate commands supported. Unlike non-immediate (data) commands where we can send TASK SET FULL, there is no way for a target to report in the Task Management Function Response PDU that is unable to process the current request before it is out of resources. The default vaule for this new key can be 1, so if this key is not negotiated initiators can assume one per connection.=20 2) Define a new value for the Response field in the Task Management Function Response PDU for reporting that the target cannot process the request currently.=20 =20 Any suggestions on how to resolve this?=20 =20 Thanks,=20 Jacob=20 _______________________________________________ Ips mailing list Ips@ietf.org https://www1.ietf.org/mailman/listinfo/ips =20 ------_=_NextPart_002_01C835B5.8FBAB70D Content-Type: text/html; charset="us-ascii" Content-Transfer-Encoding: quoted-printable

The goal of the new status or key is to help the = initiator to know the correct number of immediate commands that the target supports. The = issue with overloading a status that does clearly define the error condition = is that it does help in achieving interoperability between the myriad of = initiators and targets that are out there. I was looking to see if there was another = way that I had not considered.

 

Jacob

 

 

From:= Julian = Satran [mailto:Julian_Satran@il.ibm.com]
Sent: Monday, December 03, 2007 1:23 AM
To: Cherian, Jacob
Cc: ips@ietf.org; Nguyen, Nam
Subject: Re: [Ips] Immediate Commands in = iSCSI

 


It = looks to  me that the Reject response with reason 0x06 should be enough for = the initiator to handle the situation you describe.
If the = TM are the cause then each of those has a response indicating termination. = After receiving a reject an initiator should reissue new TMs only after = getting back one or more responses.  The initiator may even accurately guess the = number of commands the targets is capable of handling per connection and per = LU. The only case in which this handling may be awkward is over a WAN but if you = are operating over a WAN and TM performance is your only worry you are in = good shape :-)

Julo =

[Ips] Immediate Commands in iSCSI

 

Jacob_Cherian

to:

ips

12/03/07 = 05:23 AM

 

Cc:

Nam_Nguyen

 





RFC 3720 states:

The number of commands used for = immediate delivery is not limited and their delivery for execution is not = acknowledged through the numbering scheme. Immediate commands MAY be rejected by the = iSCSI target layer due to a lack of resources. An iSCSI target MUST be able to handle = at least one immediate task management command and one immediate non-task-management iSCSI command per connection at any = time.

 

With storage systems that have very = large number of logical units, we run into an issue when we use LU Reset or Abort = Task for reset recovery. I couldn’t find a way to tell an initiator how = many immediate commands a target can accept. The issue with this is that if we have to = issue multiple task management commands and it is more than what the target = can accept, it will reject the task management request. Since we don’t = know why it was rejected, we will have to escalate the reset. This can be very = disruptive with nodes that have very large number of logical units. There may be = two ways to resolve this:

 

1)      = Define a new Text negotiation key for number of outstanding immediate commands supported. Unlike non-immediate (data) commands where we can send TASK = SET FULL, there is no way for a target to report in the Task Management = Function Response PDU that is unable to process the current request before it is = out of resources.  The default vaule for this new key can be 1, so if this = key is not negotiated initiators can assume one per connection. =

2)      = Define a new value for the Response field in the Task Management Function = Response PDU for reporting that the target cannot process the request = currently.

 

Any suggestions on how to resolve = this?

 

Thanks,

Jacob
______________________________________________= _
Ips mailing list
Ips@ietf.org
https://www1.ietf.org/mailman/listinfo/ips

------_=_NextPart_002_01C835B5.8FBAB70D-- ------_=_NextPart_001_01C835B5.8FBAB70D Content-Type: image/gif; name="image001.gif" Content-Transfer-Encoding: base64 Content-ID: Content-Description: image001.gif Content-Location: image001.gif R0lGODlhZAABAOcAAMDAwAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAACH5BAEAAAAALAAAAABkAAEA QAgSAAEIHEiwoMGDCBMqXMiwYcKAADs= ------_=_NextPart_001_01C835B5.8FBAB70D-- --===============1824597339== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline _______________________________________________ Ips mailing list Ips@ietf.org https://www1.ietf.org/mailman/listinfo/ips --===============1824597339==-- From ips-bounces@ietf.org Mon Dec 03 09:42:49 2007 Return-path: Received: from [127.0.0.1] (helo=stiedprmman1.va.neustar.com) by megatron.ietf.org with esmtp (Exim 4.43) id 1IzCVM-0004Q8-2R; Mon, 03 Dec 2007 09:42:40 -0500 Received: from ips by megatron.ietf.org with local (Exim 4.43) id 1IzCVK-0004LZ-0B for ips-confirm+ok@megatron.ietf.org; Mon, 03 Dec 2007 09:42:38 -0500 Received: from [10.91.34.44] (helo=ietf-mx.ietf.org) by megatron.ietf.org with esmtp (Exim 4.43) id 1IzCVJ-0004Jn-Kq for ips@ietf.org; Mon, 03 Dec 2007 09:42:37 -0500 Received: from mtagate1.de.ibm.com ([195.212.29.150]) by ietf-mx.ietf.org with esmtp (Exim 4.43) id 1IzCVD-0008LF-9W for ips@ietf.org; Mon, 03 Dec 2007 09:42:37 -0500 Received: from d12nrmr1607.megacenter.de.ibm.com (d12nrmr1607.megacenter.de.ibm.com [9.149.167.49]) by mtagate1.de.ibm.com (8.13.8/8.13.8) with ESMTP id lB3EgUdH145614 for ; Mon, 3 Dec 2007 14:42:30 GMT Received: from d12av02.megacenter.de.ibm.com (d12av02.megacenter.de.ibm.com [9.149.165.228]) by d12nrmr1607.megacenter.de.ibm.com (8.13.8/8.13.8/NCO v8.7) with ESMTP id lB3EgTpB3010772 for ; Mon, 3 Dec 2007 15:42:30 +0100 Received: from d12av02.megacenter.de.ibm.com (loopback [127.0.0.1]) by d12av02.megacenter.de.ibm.com (8.12.11.20060308/8.13.3) with ESMTP id lB3EgTeV004554 for ; Mon, 3 Dec 2007 15:42:29 +0100 Received: from d12mc102.megacenter.de.ibm.com (d12mc102.megacenter.de.ibm.com [9.149.167.114]) by d12av02.megacenter.de.ibm.com (8.12.11.20060308/8.12.11) with ESMTP id lB3EgTQE004551; Mon, 3 Dec 2007 15:42:29 +0100 In-Reply-To: <5DDAB7BA7BDB58439DD0EED0B8E9A3AEFD9D2E@ausx3mpc102.aus.amer.dell.com> To: MIME-Version: 1.0 Subject: RE: [Ips] Immediate Commands in iSCSI X-KeepSent: 93E9CD5B:0C677223-C22573A6:004FE0BF; type=4; name=$KeepSent X-Mailer: Lotus Notes Release 7.0.3 September 26, 2007 From: Julian Satran Message-ID: Date: Mon, 3 Dec 2007 16:42:29 +0200 X-MIMETrack: Serialize by Router on D12MC102/12/M/IBM(Release 8.0|August 02, 2007) at 03/12/2007 16:42:29, Serialize complete at 03/12/2007 16:42:29 X-Spam-Score: -4.0 (----) X-Scan-Signature: 958aa603499a3de6b2b87d68741ed60e Cc: ips@ietf.org, Nam_Nguyen@Dell.com X-BeenThere: ips@ietf.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: IP Storage List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Content-Type: multipart/mixed; boundary="===============0594521496==" Errors-To: ips-bounces@ietf.org This is a multipart message in MIME format. --===============0594521496== Content-Type: multipart/related; boundary="=_related 0050CA43C22573A6_=" This is a multipart message in MIME format. --=_related 0050CA43C22573A6_= Content-Type: multipart/alternative; boundary="=_alternative 0050CA43C22573A6_=" --=_alternative 0050CA43C22573A6_= Content-Type: text/plain; charset="ISO-8859-1" Content-Transfer-Encoding: quoted-printable Jacob, I do not want to appear offensive - but did you consider the "larger=20 picture"? Assuming that the negotiation key or status is there and=20 assuming that the target talks to 7 (seven) different initiators what=20 would it tell each of them (it has 7 different sessions)? Should he=20 reserve a quota for each of them - so that it can meet what it declared?=20 What should a target do with commands that exceed the number?=20 In iSCSI as a rule we avoided setting or negotiating parameters that we=20 don't need to set/negotiate and I think the number of accepted immediate=20 commands is one of them. Julo RE: [Ips] Immediate Commands in iSCSI Jacob=5FCherian=20 to: Julian Satran 12/03/07 04:06 PM Cc: ips, Nam=5FNguyen The goal of the new status or key is to help the initiator to know the=20 correct number of immediate commands that the target supports. The issue=20 with overloading a status that does clearly define the error condition is=20 that it does help in achieving interoperability between the myriad of=20 initiators and targets that are out there. I was looking to see if there=20 was another way that I had not considered. =20 Jacob =20 =20 From: Julian Satran [mailto:Julian=5FSatran@il.ibm.com]=20 Sent: Monday, December 03, 2007 1:23 AM To: Cherian, Jacob Cc: ips@ietf.org; Nguyen, Nam Subject: Re: [Ips] Immediate Commands in iSCSI =20 It looks to me that the Reject response with reason 0x06 should be enough = for the initiator to handle the situation you describe.=20 If the TM are the cause then each of those has a response indicating=20 termination. After receiving a reject an initiator should reissue new TMs=20 only after getting back one or more responses. The initiator may even=20 accurately guess the number of commands the targets is capable of handling = per connection and per LU. The only case in which this handling may be=20 awkward is over a WAN but if you are operating over a WAN and TM=20 performance is your only worry you are in good shape :-)=20 Julo=20 [Ips] Immediate Commands in iSCSI =20 Jacob=5FCherian=20 to:=20 ips=20 12/03/07 05:23 AM =20 Cc:=20 Nam=5FNguyen =20 RFC 3720 states:=20 The number of commands used for immediate delivery is not limited and=20 their delivery for execution is not acknowledged through the numbering=20 scheme. Immediate commands MAY be rejected by the iSCSI target layer due=20 to a lack of resources. An iSCSI target MUST be able to handle at least=20 one immediate task management command and one immediate=20 non-task-management iSCSI command per connection at any time.=20 =20 With storage systems that have very large number of logical units, we run=20 into an issue when we use LU Reset or Abort Task for reset recovery. I=20 couldn?t find a way to tell an initiator how many immediate commands a=20 target can accept. The issue with this is that if we have to issue=20 multiple task management commands and it is more than what the target can=20 accept, it will reject the task management request. Since we don?t know=20 why it was rejected, we will have to escalate the reset. This can be very=20 disruptive with nodes that have very large number of logical units. There=20 may be two ways to resolve this:=20 =20 1) Define a new Text negotiation key for number of outstanding=20 immediate commands supported. Unlike non-immediate (data) commands where=20 we can send TASK SET FULL, there is no way for a target to report in the=20 Task Management Function Response PDU that is unable to process the=20 current request before it is out of resources. The default vaule for this = new key can be 1, so if this key is not negotiated initiators can assume=20 one per connection.=20 2) Define a new value for the Response field in the Task Management=20 Function Response PDU for reporting that the target cannot process the=20 request currently.=20 =20 Any suggestions on how to resolve this?=20 =20 Thanks,=20 Jacob=20 =5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F= =5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F Ips mailing list Ips@ietf.org https://www1.ietf.org/mailman/listinfo/ips=20 --=_alternative 0050CA43C22573A6_= Content-Type: text/html; charset="ISO-8859-1" Content-Transfer-Encoding: quoted-printable
Jacob,

I do not want to appear offensive - but did you consider the "larger picture"? Assuming that the negotiation key or status is there and assuming that the target talks to 7 (seven) different initiators what would it tell each of them (it has 7 different sessions)? Should he reserve a quota for each of them - so that it can meet what it declared? What should a target do with commands that exceed the number?

In iSCSI as a rule we avoided setting or negotiating parameters that we don't need to set/negotiate and I think the number of accepted immediate commands is one of them.

Julo


RE: [Ips] Immediate = Commands in iSCSI

Jacob=5FCherian <= /b> to: Julian Satran
12/03/07 04:06 PM

Cc: ips, Nam=5FNguyen






The goal of the new = status or key is to help the initiator to know the correct number of immediate commands that the target supports. The issue with overloading a status that does clearly define the error condition is that it does help in achiev= ing interoperability between the myriad of initiators and targets that are out there. I was looking to see if there was another way that I had not considered.
 
Jacob
 
 
From: Julian Satran [mailto:Julia= n=5FSatran@il.ibm.com]
Sent: Monday, December 03, 2007 1:23 AM
To: Cherian, Jacob
Cc: ips@ietf.org; Nguyen, Nam
Subject: Re: [Ips] Immediate Commands in iSCSI
 

It looks to  me that the Reject response with reason 0x06 should be enough for the initiator to handle the situation you describe.
If the TM are the cause then each of those has a response indicating termin= ation. After receiving a reject an initiator should reissue new TMs only after getting back one or more responses.  The initiator may even accurately guess the number of commands the targets is capable of handling per connect= ion and per LU. The only case in which this handling may be awkward is over a WAN but if you are operating over a WAN and TM performance is your only worry you are in good shape :-)

Julo


[Ips] Immediate Commands = in iSCSI

 


Jacob=5FCh= erian to: ips
12/03/07 05:23 AM

 


Cc: Nam=5FNguyen



 







RFC 3720 states:

The number of commands used for immediate delivery is not limited and their delivery for execution is not acknowledged through the numbering scheme. Immediate commands MAY be rejected by the iSCSI target layer due to a lack of resources. An iSCSI target MUST be able to handle at least one immediate task management command and one immediate non-task-management iSCSI command per connection at any time.

 

With storage systems that have very large number of logical units, we run into an issue when we use LU Reset or Abort Task for reset recovery. I couldn’t find a way to tell an initiator how many immediate commands a target can accept. The issue with this is that if we have to issue multiple task management commands and it is more than what the target can accept, it will reject the task management request. Since we don’t know why it was rejected, we will have to escalate the reset. This can be very disrupti= ve with nodes that have very large number of logical units. There may be two ways to resolve this:

 

1)      Define a new Text negotiation key for number of outstanding immediate commands supported. Unlike non-immediate (data) comma= nds where we can send TASK SET FULL, there is no way for a target to report in the Task Management Function Response PDU that is unable to process the current request before it is out of resources.  The default vaule for this new key can be 1, so if this key is not negotiated initiators can assume one per connection.
2)      Define a new value for the Response field in the Task Management Function Response PDU for reporting that the target cannot process the request currently.
 

Any suggestions on how to resolve this?

 

Thanks,

Jacob
=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F= =5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F
Ips mailing list
Ips@ietf.org=
https://www1.ietf.org/mailman/lis= tinfo/ips
--=_alternative 0050CA43C22573A6_=-- --=_related 0050CA43C22573A6_= Content-Type: image/gif Content-ID: <_1_2027C71C2027C3480050C7F1C22573A6> Content-Transfer-Encoding: base64 R0lGODlhZAABAOcAAMDAwAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAACH5BAEAAAAALAAAAABkAAEA QAgSAAEIHEiwoMGDCBMqXMiwYcKAADs= --=_related 0050CA43C22573A6_= Content-Type: image/gif Content-ID: <_1_2028DD082028D9640050C7F1C22573A6> Content-Transfer-Encoding: base64 R0lGODlhZAABAOcAAMDAwAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAACH5BAEAAAAALAAAAABkAAEA QAgSAAEIHEiwoMGDCBMqXMiwYcKAADs= --=_related 0050CA43C22573A6_=-- --===============0594521496== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline _______________________________________________ Ips mailing list Ips@ietf.org https://www1.ietf.org/mailman/listinfo/ips --===============0594521496==-- From ips-bounces@ietf.org Mon Dec 03 15:15:32 2007 Return-path: Received: from [127.0.0.1] (helo=stiedprmman1.va.neustar.com) by megatron.ietf.org with esmtp (Exim 4.43) id 1IzHhP-0003Bs-Sq; Mon, 03 Dec 2007 15:15:27 -0500 Received: from ips by megatron.ietf.org with local (Exim 4.43) id 1IzHhN-00038Q-Tz for ips-confirm+ok@megatron.ietf.org; Mon, 03 Dec 2007 15:15:25 -0500 Received: from [10.90.34.44] (helo=chiedprmail1.ietf.org) by megatron.ietf.org with esmtp (Exim 4.43) id 1IzHhN-0002zD-Ae for ips@ietf.org; Mon, 03 Dec 2007 15:15:25 -0500 Received: from ausc60pc101.us.dell.com ([143.166.85.206]) by chiedprmail1.ietf.org with esmtp (Exim 4.43) id 1IzHhK-0004ZH-9W for ips@ietf.org; Mon, 03 Dec 2007 15:15:22 -0500 DomainKey-Signature: s=smtpout; d=dell.com; c=nofws; q=dns; h=X-IronPort-AV:X-MimeOLE:Content-class:MIME-Version: Content-Type:Subject:Date:Message-ID:In-Reply-To: X-MS-Has-Attach:X-MS-TNEF-Correlator:Thread-Topic: Thread-Index:References:From:To:Cc:Return-Path: X-OriginalArrivalTime; b=qF0zSoIzu0YNW8ezwq/qTy6Lzpt9k0dpbapfPkGb4LAMYDHG2xcTUqBR 6XEpI8PsLo+6R7pTTFAK392B+aHwf9DjcihB4fGZLRhDvwgyu7avG2lvi qFx5Ik/Ghtw6Pw6; X-IronPort-AV: E=Sophos;i="4.23,244,1194242400"; d="gif'147?scan'147,208,217,147";a="456232986" X-MimeOLE: Produced By Microsoft Exchange V6.5 Content-class: urn:content-classes:message MIME-Version: 1.0 Subject: RE: [Ips] Immediate Commands in iSCSI Date: Mon, 3 Dec 2007 14:15:19 -0600 Message-ID: <5DDAB7BA7BDB58439DD0EED0B8E9A3AEFD9F00@ausx3mpc102.aus.amer.dell.com> In-Reply-To: X-MS-Has-Attach: yes X-MS-TNEF-Correlator: Thread-Topic: [Ips] Immediate Commands in iSCSI Thread-Index: Acg1usC8FBkl9pixR+yYp4g1A9OrcAACBnIA References: <5DDAB7BA7BDB58439DD0EED0B8E9A3AEFD9D2E@ausx3mpc102.aus.amer.dell.com> From: To: X-OriginalArrivalTime: 03 Dec 2007 20:15:21.0393 (UTC) FILETIME=[3B058A10:01C835E9] X-Spam-Score: 0.0 (/) X-Scan-Signature: 7b1e34d2462e88c1c4311f8189331c59 Cc: ips@ietf.org, Nam_Nguyen@Dell.com X-BeenThere: ips@ietf.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: IP Storage List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Content-Type: multipart/mixed; boundary="===============1942487721==" Errors-To: ips-bounces@ietf.org This is a multi-part message in MIME format. --===============1942487721== Content-class: urn:content-classes:message Content-Type: multipart/related; type="multipart/alternative"; boundary="----_=_NextPart_001_01C835E9.3A9E9DB5" This is a multi-part message in MIME format. ------_=_NextPart_001_01C835E9.3A9E9DB5 Content-Type: multipart/alternative; boundary="----_=_NextPart_002_01C835E9.3A9E9DB5" ------_=_NextPart_002_01C835E9.3A9E9DB5 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable Julian, No offense taken. The scenario that you brought up was considered since most configurations with iSCSI are with multiple hosts. However, the allocation of immediate commands to connection/sessions/hosts depends on the actual implementation. RFC 3720 already requires support for at least one per connection. Even we cannot agree on a key, having a more granular status for a rejected TMF that says the target is currently busy processing a TMF and cannot process another at this time (but will be able to sometime in the future), helps initiators to wait and retry, instead of assuming a failure and escalating the reset.=20 =20 Jacob =20 From: Julian Satran [mailto:Julian_Satran@il.ibm.com]=20 Sent: Monday, December 03, 2007 8:42 AM To: Cherian, Jacob Cc: ips@ietf.org; Nguyen, Nam Subject: RE: [Ips] Immediate Commands in iSCSI =20 Jacob,=20 I do not want to appear offensive - but did you consider the "larger picture"? Assuming that the negotiation key or status is there and assuming that the target talks to 7 (seven) different initiators what would it tell each of them (it has 7 different sessions)? Should he reserve a quota for each of them - so that it can meet what it declared? What should a target do with commands that exceed the number?=20 In iSCSI as a rule we avoided setting or negotiating parameters that we don't need to set/negotiate and I think the number of accepted immediate commands is one of them.=20 Julo=20 =20 RE: [Ips] Immediate Commands in iSCSI =20 Jacob_Cherian=20 to:=20 Julian Satran=20 12/03/07 04:06 PM =20 Cc:=20 ips, Nam_Nguyen =09 =20 ________________________________ The goal of the new status or key is to help the initiator to know the correct number of immediate commands that the target supports. The issue with overloading a status that does clearly define the error condition is that it does help in achieving interoperability between the myriad of initiators and targets that are out there. I was looking to see if there was another way that I had not considered.=20 =20 Jacob=20 =20 =20 From: Julian Satran [mailto:Julian_Satran@il.ibm.com]=20 Sent: Monday, December 03, 2007 1:23 AM To: Cherian, Jacob Cc: ips@ietf.org; Nguyen, Nam Subject: Re: [Ips] Immediate Commands in iSCSI=20 =20 It looks to me that the Reject response with reason 0x06 should be enough for the initiator to handle the situation you describe.=20 If the TM are the cause then each of those has a response indicating termination. After receiving a reject an initiator should reissue new TMs only after getting back one or more responses. The initiator may even accurately guess the number of commands the targets is capable of handling per connection and per LU. The only case in which this handling may be awkward is over a WAN but if you are operating over a WAN and TM performance is your only worry you are in good shape :-)=20 Julo=20 =20 [Ips] Immediate Commands in iSCSI =20 =20 Jacob_Cherian=20 to:=20 ips=20 12/03/07 05:23 AM =20 =20 Cc:=20 Nam_Nguyen =09 =20 =20 ________________________________ RFC 3720 states:=20 The number of commands used for immediate delivery is not limited and their delivery for execution is not acknowledged through the numbering scheme. Immediate commands MAY be rejected by the iSCSI target layer due to a lack of resources. An iSCSI target MUST be able to handle at least one immediate task management command and one immediate non-task-management iSCSI command per connection at any time.=20 =20 With storage systems that have very large number of logical units, we run into an issue when we use LU Reset or Abort Task for reset recovery. I couldn't find a way to tell an initiator how many immediate commands a target can accept. The issue with this is that if we have to issue multiple task management commands and it is more than what the target can accept, it will reject the task management request. Since we don't know why it was rejected, we will have to escalate the reset. This can be very disruptive with nodes that have very large number of logical units. There may be two ways to resolve this:=20 =20 1) Define a new Text negotiation key for number of outstanding immediate commands supported. Unlike non-immediate (data) commands where we can send TASK SET FULL, there is no way for a target to report in the Task Management Function Response PDU that is unable to process the current request before it is out of resources. The default vaule for this new key can be 1, so if this key is not negotiated initiators can assume one per connection.=20 2) Define a new value for the Response field in the Task Management Function Response PDU for reporting that the target cannot process the request currently.=20 =20 Any suggestions on how to resolve this?=20 =20 Thanks,=20 Jacob=20 _______________________________________________ Ips mailing list Ips@ietf.org https://www1.ietf.org/mailman/listinfo/ips =20 ------_=_NextPart_002_01C835E9.3A9E9DB5 Content-Type: text/html; charset="us-ascii" Content-Transfer-Encoding: quoted-printable

Julian,

    No offense taken. The scenario that = you brought up was considered since most configurations with iSCSI are with multiple hosts. However, the allocation of immediate commands to = connection/sessions/hosts depends on the actual implementation. RFC 3720 already requires support = for at least one per connection. Even we cannot agree on a key, having a more = granular status for a rejected TMF that says the target is currently busy = processing a TMF and cannot process another at this time (but will be able to = sometime in the future), helps initiators to wait and retry, instead of assuming a = failure and escalating the reset.

 

 Jacob

 

From:= Julian = Satran [mailto:Julian_Satran@il.ibm.com]
Sent: Monday, December 03, 2007 8:42 AM
To: Cherian, Jacob
Cc: ips@ietf.org; Nguyen, Nam
Subject: RE: [Ips] Immediate Commands in = iSCSI

 


Jacob,=

I do = not want to appear offensive - but did you consider the "larger = picture"? Assuming that the negotiation key or status is there and assuming that = the target talks to 7 (seven) different initiators what would it tell each = of them (it has 7 different sessions)? Should he reserve a quota for each of = them - so that it can meet what it declared? What should a target do with commands = that exceed the number?

In = iSCSI as a rule we avoided setting or negotiating parameters that we don't need to set/negotiate and I think the number of accepted immediate commands is = one of them.

Julo =

RE: [Ips] Immediate Commands in iSCSI

 

Jacob_Cherian

to:

Julian Satran

12/03/07 = 04:06 PM

 

Cc:

ips, Nam_Nguyen

 




The goal of the new status or key is to help the initiator to know the = correct number of immediate commands that the target supports. The issue with overloading a status that does clearly define the error condition is = that it does help in achieving interoperability between the myriad of initiators = and targets that are out there. I was looking to see if there was another = way that I had not considered.
 
Jacob
 
 
From:= Julian = Satran [mailto:Julian_Satran@il.ibm.com]
Sent: Monday, December 03, 2007 1:23 AM
To: Cherian, Jacob
Cc: ips@ietf.org; Nguyen, Nam
Subject: Re: [Ips] Immediate Commands in iSCSI
 

It looks to  me that the Reject response with reason 0x06 should be = enough for the initiator to handle the situation you describe.
If the TM are the cause then each of those has a response indicating termination. After receiving a reject an initiator should reissue new = TMs only after getting back one or more responses.  The initiator may even accurately guess the number of commands the targets is capable of = handling per connection and per LU. The only case in which this handling may be = awkward is over a WAN but if you are operating over a WAN and TM performance is = your only worry you are in good shape :-)

Julo

 

[Ips] Immediate Commands in iSCSI


 

 

Jacob_Cherian

to:

ips

12/03/07 = 05:23 AM


 

 

Cc:

Nam_Nguyen


 

 






RFC 3720 states:

The number of commands used for immediate delivery is not limited and = their delivery for execution is not acknowledged through the numbering scheme. Immediate commands MAY be rejected by the iSCSI target layer due to a = lack of resources. An iSCSI target MUST be able to handle at least one immediate = task management command and one immediate non-task-management iSCSI command = per connection at any time.

 

With storage systems that have very large number of logical units, we = run into an issue when we use LU Reset or Abort Task for reset recovery. I couldn’t find a way to tell an initiator how many immediate = commands a target can accept. The issue with this is that if we have to issue = multiple task management commands and it is more than what the target can accept, = it will reject the task management request. Since we don’t know why = it was rejected, we will have to escalate the reset. This can be very = disruptive with nodes that have very large number of logical units. There may be two = ways to resolve this:

 

1)      Define a new Text negotiation key for number of outstanding immediate commands supported. Unlike non-immediate (data) = commands where we can send TASK SET FULL, there is no way for a target to report = in the Task Management Function Response PDU that is unable to process the = current request before it is out of resources.  The default vaule for this = new key can be 1, so if this key is not negotiated initiators can assume one per connection.

2)      Define a new value for the Response field in the = Task Management Function Response PDU for reporting that the target cannot = process the request currently.

 

Any suggestions on how to resolve this?

 

Thanks,

Jacob
_______________________________________________
Ips mailing list
Ips@ietf.org
https://www1.ietf.org/mailman/listinfo/ips

------_=_NextPart_002_01C835E9.3A9E9DB5-- ------_=_NextPart_001_01C835E9.3A9E9DB5 Content-Type: image/gif; name="image001.gif" Content-Transfer-Encoding: base64 Content-ID: Content-Description: image001.gif Content-Location: image001.gif R0lGODlhZAABAOcAAMDAwAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAACH5BAEAAAAALAAAAABkAAEA QAgSAAEIHEiwoMGDCBMqXMiwYcKAADs= ------_=_NextPart_001_01C835E9.3A9E9DB5-- --===============1942487721== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline _______________________________________________ Ips mailing list Ips@ietf.org https://www1.ietf.org/mailman/listinfo/ips --===============1942487721==-- From ips-bounces@ietf.org Tue Dec 04 08:47:45 2007 Return-path: Received: from [127.0.0.1] (helo=stiedprmman1.va.neustar.com) by megatron.ietf.org with esmtp (Exim 4.43) id 1IzY7i-0007Ms-QN; Tue, 04 Dec 2007 08:47:42 -0500 Received: from ips by megatron.ietf.org with local (Exim 4.43) id 1IzRIX-0002FN-S5 for ips-confirm+ok@megatron.ietf.org; Tue, 04 Dec 2007 01:30:25 -0500 Received: from ips by megatron.ietf.org with local (Exim 4.43) id 1IzRIX-0002F8-GW for ips@ietf.org; Tue, 04 Dec 2007 01:30:25 -0500 Received: from [10.90.34.44] (helo=chiedprmail1.ietf.org) by megatron.ietf.org with esmtp (Exim 4.43) id 1IzRHn-0007ih-31 for ips@ietf.org; Tue, 04 Dec 2007 01:29:39 -0500 Received: from smtp-outbound-1.vmware.com ([65.113.40.141]) by chiedprmail1.ietf.org with esmtp (Exim 4.43) id 1IzRHk-00034L-NL for ips@ietf.org; Tue, 04 Dec 2007 01:29:36 -0500 Received: from mailhost2.vmware.com (mailhost2.vmware.com [10.16.64.160]) by smtp-outbound-1.vmware.com (Postfix) with ESMTP id 11F3161A3 for ; Mon, 3 Dec 2007 22:29:36 -0800 (PST) Received: from PA-EXCH03.vmware.com (pa-exch-node03.vmware.com [10.16.64.142]) by mailhost2.vmware.com (Postfix) with ESMTP id 14DD33C694 for ; Mon, 3 Dec 2007 22:29:36 -0800 (PST) X-MimeOLE: Produced By Microsoft Exchange V6.5 Content-class: urn:content-classes:message MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable Date: Mon, 3 Dec 2007 22:29:31 -0800 Message-ID: <882A4B7A51024043B9E5EA13382DF21102FD4F73@PA-EXCH03.vmware.com> X-MS-Has-Attach: X-MS-TNEF-Correlator: Thread-Topic: Pending R2T's at the initiator and initiator issuing "abort task" Thread-Index: Acg2PwdN6097jtEiShaiAssPQMIgog== From: "Prasanna Aithal" To: X-Spam-Score: 0.0 (/) X-Scan-Signature: d6b246023072368de71562c0ab503126 X-TMDA-Confirmed: Tue, 04 Dec 2007 01:30:25 -0500 X-Mailman-Approved-At: Tue, 04 Dec 2007 08:47:40 -0500 Subject: [Ips] Pending R2T's at the initiator and initiator issuing "abort task" X-BeenThere: ips@ietf.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: IP Storage List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: ips-bounces@ietf.org RFC 3720, Section 10.5.1 says: =20 "For ABORT TASK SET and CLEAR TASK SET, the issuing initiator MUST continue to respond to all valid target transfer tags (received via R2T, Text Response, NOP-In, or SCSI Data-In PDUs) related to the affected task set, even after issuing the task management request. The issuing initiator SHOULD however terminate (i.e., by setting the F-bit to 1) these response sequences as quickly as possible. The target on its part MUST wait for responses on all affected target transfer tags before acting on either of these two task management requests. In case all or part of the response sequence is not received (due to digest errors) for a valid TTT, the target MAY treat it as a case of within-command error recovery class (see Section 6.1.4.1 Recovery Within-command) if it is supporting ErrorRecoveryLevel >=3D 1, or alternatively may drop the connection to complete the requested task set function." What is expected of the initiators in case of "abort task"?. Does the responding to "all valid target tags" apply to "abort task" also?. _______________________________________________ Ips mailing list Ips@ietf.org https://www1.ietf.org/mailman/listinfo/ips From ips-bounces@ietf.org Tue Dec 04 10:05:53 2007 Return-path: Received: from [127.0.0.1] (helo=stiedprmman1.va.neustar.com) by megatron.ietf.org with esmtp (Exim 4.43) id 1IzZLM-0003s8-8L; Tue, 04 Dec 2007 10:05:52 -0500 Received: from ips by megatron.ietf.org with local (Exim 4.43) id 1IzZLK-0003ry-ON for ips-confirm+ok@megatron.ietf.org; Tue, 04 Dec 2007 10:05:50 -0500 Received: from [10.91.34.44] (helo=ietf-mx.ietf.org) by megatron.ietf.org with esmtp (Exim 4.43) id 1IzZLK-0003rn-Em for ips@ietf.org; Tue, 04 Dec 2007 10:05:50 -0500 Received: from mexforward.lss.emc.com ([128.222.32.20]) by ietf-mx.ietf.org with esmtp (Exim 4.43) id 1IzZLK-00057x-1F for ips@ietf.org; Tue, 04 Dec 2007 10:05:50 -0500 Received: from hop04-l1d11-si01.isus.emc.com (HOP04-L1D11-SI01.isus.emc.com [10.254.111.54]) by mexforward.lss.emc.com (Switch-3.2.5/Switch-3.1.7) with ESMTP id lB4F5lwI006332 for ; Tue, 4 Dec 2007 10:05:49 -0500 (EST) Received: from mailhub.lss.emc.com (nirah.lss.emc.com [10.254.144.13]) by hop04-l1d11-si01.isus.emc.com (Tablus Interceptor) for ; Tue, 4 Dec 2007 10:05:47 -0500 Received: from corpussmtp2.corp.emc.com (corpussmtp2.corp.emc.com [128.221.14.146]) by mailhub.lss.emc.com (Switch-3.2.5/Switch-3.1.7) with ESMTP id lB4F4kCt022136 for ; Tue, 4 Dec 2007 10:05:45 -0500 (EST) From: Black_David@emc.com Received: from CORPUSMX20A.corp.emc.com ([128.221.62.13]) by corpussmtp2.corp.emc.com with Microsoft SMTPSVC(6.0.3790.1830); Tue, 4 Dec 2007 10:04:58 -0500 X-MimeOLE: Produced By Microsoft Exchange V6.5 Content-class: urn:content-classes:message MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable Subject: RE: [Ips] Pending R2T's at the initiator and initiator issuing "aborttask" Date: Tue, 4 Dec 2007 10:04:56 -0500 Message-ID: In-Reply-To: <882A4B7A51024043B9E5EA13382DF21102FD4F73@PA-EXCH03.vmware.com> X-MS-Has-Attach: X-MS-TNEF-Correlator: Thread-Topic: [Ips] Pending R2T's at the initiator and initiator issuing "aborttask" thread-index: Acg2PwdN6097jtEiShaiAssPQMIgogAQh35Q References: <882A4B7A51024043B9E5EA13382DF21102FD4F73@PA-EXCH03.vmware.com> To: , X-OriginalArrivalTime: 04 Dec 2007 15:04:58.0641 (UTC) FILETIME=[0968D810:01C83687] X-PMX-Version: 4.7.1.128075, Antispam-Engine: 2.5.1.298604, Antispam-Data: 2007.8.30.51425 X-PerlMx-Spam: Gauge=, SPAM=0%, Reason='EMC_BODY_1+ -3, EMC_FROM_0+ -3, NO_REAL_NAME 0, __C230066_P5 0, __CT 0, __CTE 0, __CT_TEXT_PLAIN 0, __HAS_MSGID 0, __IMS_MSGID 0, __MIME_TEXT_ONLY 0, __MIME_VERSION 0, __SANE_MSGID 0' X-Tablus-Inspected: yes X-Tablus-Classifications: public X-Tablus-Action: allow X-Spam-Score: -4.0 (----) X-Scan-Signature: c1c65599517f9ac32519d043c37c5336 Cc: X-BeenThere: ips@ietf.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: IP Storage List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: ips-bounces@ietf.org Prasanna, > RFC 3720, Section 10.5.1 says: > =20 > "For ABORT TASK SET and CLEAR TASK SET, the issuing initiator MUST > continue to respond to all valid target transfer tags (received via > R2T, Text Response, NOP-In, or SCSI Data-In PDUs) related to the > affected task set, even after issuing the task management request. > The issuing initiator SHOULD however terminate (i.e., by setting the > F-bit to 1) these response sequences as quickly as possible. The > target on its part MUST wait for responses on all affected target > transfer tags before acting on either of these two task management > requests. In case all or part of the response sequence is not > received (due to digest errors) for a valid TTT, the target MAY treat > it as a case of within-command error recovery class (see Section > 6.1.4.1 Recovery Within-command) if it is supporting > ErrorRecoveryLevel >=3D 1, or alternatively may drop the connection to > complete the requested task set function." Please be aware that the above text was modified by Section 4 of RFC 5048, although those modifications do not affect what is being asked about here: =20 > What is expected of the initiators in case of "abort task"?. Does the > responding to "all valid target tags" apply to "abort task" also?. No, ABORT TASK can only be used by an initiator to abort one of its own commands - it is required to be issued on the same TCP connection as the command to be aborted: If the connection is still active (it is not undergoing an implicit or explicit logout), ABORT TASK MUST be issued on the same connection to which the task to be aborted is allegiant at the time the Task Management Request is issued. This avoids any need to complete outstanding transfers because the ABORT TASK request is synchronized with those transfers (which also have to be on the same connection). Thanks, --David ---------------------------------------------------- David L. Black, Distinguished Engineer EMC Corporation, 176 South St., Hopkinton, MA 01748 +1 (508) 293-7953 FAX: +1 (508) 293-7786 black_david@emc.com Mobile: +1 (978) 394-7754 ---------------------------------------------------- _______________________________________________ Ips mailing list Ips@ietf.org https://www1.ietf.org/mailman/listinfo/ips From ips-bounces@ietf.org Tue Dec 04 20:08:05 2007 Return-path: Received: from [127.0.0.1] (helo=stiedprmman1.va.neustar.com) by megatron.ietf.org with esmtp (Exim 4.43) id 1Izik9-0006Bl-2N; Tue, 04 Dec 2007 20:08:05 -0500 Received: from ips by megatron.ietf.org with local (Exim 4.43) id 1Izik7-00068G-NH for ips-confirm+ok@megatron.ietf.org; Tue, 04 Dec 2007 20:08:03 -0500 Received: from [10.91.34.44] (helo=ietf-mx.ietf.org) by megatron.ietf.org with esmtp (Exim 4.43) id 1Izik7-00067T-Cn for ips@ietf.org; Tue, 04 Dec 2007 20:08:03 -0500 Received: from blu139-omc3-s5.blu139.hotmail.com ([65.55.175.205]) by ietf-mx.ietf.org with esmtp (Exim 4.43) id 1Izik4-0004k3-Ov for ips@ietf.org; Tue, 04 Dec 2007 20:08:03 -0500 Received: from BLU117-W3 ([65.55.162.189]) by blu139-omc3-s5.blu139.hotmail.com with Microsoft SMTPSVC(6.0.3790.3959); Tue, 4 Dec 2007 17:08:00 -0800 Message-ID: X-Originating-IP: [61.14.27.42] From: Ken Sandars To: , Subject: RE: [Ips] Immediate Commands in iSCSI Date: Wed, 5 Dec 2007 12:08:00 +1100 Importance: Normal In-Reply-To: <5DDAB7BA7BDB58439DD0EED0B8E9A3AEFD9F00@ausx3mpc102.aus.amer.dell.com> References: <5DDAB7BA7BDB58439DD0EED0B8E9A3AEFD9D2E@ausx3mpc102.aus.amer.dell.com> <5DDAB7BA7BDB58439DD0EED0B8E9A3AEFD9F00@ausx3mpc102.aus.amer.dell.com> MIME-Version: 1.0 X-OriginalArrivalTime: 05 Dec 2007 01:08:00.0478 (UTC) FILETIME=[477707E0:01C836DB] X-Spam-Score: 2.2 (++) X-Scan-Signature: eda4528ec6326d3504ad3a985d220c76 Cc: ips@ietf.org, nam_nguyen@dell.com X-BeenThere: ips@ietf.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: IP Storage List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Content-Type: multipart/mixed; boundary="===============0175238084==" Errors-To: ips-bounces@ietf.org --===============0175238084== Content-Type: multipart/alternative; boundary="_44fbb6ae-6d5b-4897-b310-34a25e73e823_" --_44fbb6ae-6d5b-4897-b310-34a25e73e823_ Content-Type: text/plain; charset="Windows-1252" Content-Transfer-Encoding: quoted-printable =0A= =0A= =0A= Hi Jacob, A Reject PDU which indicates its reason as "Immediate Command Reject - too = many immediate commands" seems pretty granular to me. :-) That's what the r= eject code 0x06 means (as Julo had recommended). The initiator may resubmit= the PDU whenever it deems appropriate. Does that make sense? IMHO, a target implementation constrains the number of non-TMF commands and= might throttle repeated TMF commands when they are in progress (and the cl= earing effects are taking a long time to resolve). Even with a large number= of LU's it would be unusual to constrain the number of concurrent LU Reset= operations. Have you got a trace showing this behaviour? HTH, Ken Subject: RE: [Ips] Immediate Commands in iSCSI Date: Mon, 3 Dec 2007 14:15:19 -0600 From: Jacob_Cherian@Dell.com To: Julian_Satran@il.ibm.com CC: ips@ietf.org; Nam_Nguyen@Dell.com =0A= =0A= =0A= =0A= =0A= =0A= =0A= =0A= =0A= =0A= =0A= =0A= =0A= =0A= =0A= =0A= =0A= =0A= Julian,=0A= =0A= No offense taken. The scenario that you=0A= brought up was considered since most configurations with iSCSI are with=0A= multiple hosts. However, the allocation of immediate commands to connection= /sessions/hosts=0A= depends on the actual implementation. RFC 3720 already requires support for= at=0A= least one per connection. Even we cannot agree on a key, having a more gran= ular=0A= status for a rejected TMF that says the target is currently busy processing= a=0A= TMF and cannot process another at this time (but will be able to sometime i= n=0A= the future), helps initiators to wait and retry, instead of assuming a fail= ure=0A= and escalating the reset. =0A= =0A= =0A= =0A= Jacob=0A= =0A= =0A= =0A= =0A= =0A= From: Julian Satran=0A= [mailto:Julian_Satran@il.ibm.com]=20 =0A= Sent: Monday, December 03, 2007 8:42 AM =0A= To: Cherian, Jacob =0A= Cc: ips@ietf.org; Nguyen, Nam =0A= Subject: RE: [Ips] Immediate Commands in iSCSI=0A= =0A= =0A= =0A= =0A= =0A= =0A= Jacob,=20 =0A= =0A= I do not want=0A= to appear offensive - but did you consider the "larger picture"?=0A= Assuming that the negotiation key or status is there and assuming that the= =0A= target talks to 7 (seven) different initiators what would it tell each of t= hem=0A= (it has 7 different sessions)? Should he reserve a quota for each of them -= so=0A= that it can meet what it declared? What should a target do with commands th= at=0A= exceed the number?=20 =0A= =0A= In iSCSI as a rule=0A= we avoided setting or negotiating parameters that we don't need to=0A= set/negotiate and I think the number of accepted immediate commands is one = of=0A= them.=20 =0A= =0A= Julo=20 =0A= =0A= =0A= =0A= =0A= =0A= =0A= =0A= =0A= =0A= =0A= =0A= =0A= RE:=0A= [Ips] Immediate Commands in iSCSI=0A= =0A= =0A= =0A= =0A= =0A= =0A= =0A= Jacob_Cherian =0A= =0A= =0A= to: =0A= =0A= =0A= Julian=0A= Satran =0A= =0A= =0A= 12/03/07 04:06 PM=0A= =0A= =0A= =0A= =0A= =0A= =0A= =0A= =0A= =0A= =0A= Cc: =0A= =0A= =0A= ips,=0A= Nam_Nguyen=0A= =0A= =0A= =0A= =0A= =0A= =0A= =0A= =0A= =0A= =0A= =0A= =0A= =0A= =0A= =0A= =0A= =0A= =0A= =0A= =0A= =0A= =0A= The=0A= goal of the new status or key is to help the initiator to know the correct= =0A= number of immediate commands that the target supports. The issue with=0A= overloading a status that does clearly define the error condition is that i= t=0A= does help in achieving interoperability between the myriad of initiators an= d=0A= targets that are out there. I was looking to see if there was another way t= hat=0A= I had not considered.=20 =0A= =0A= =0A= Jacob=0A= =0A= =0A= =0A= =0A= =0A= From: Julian Satran=0A= [mailto:Julian_Satran@il.ibm.com]=20 =0A= Sent: Monday, December 03, 2007 1:23 AM =0A= To: Cherian, Jacob =0A= Cc: ips@ietf.org; Nguyen, Nam =0A= Subject: Re: [Ips] Immediate Commands in iSCSI=20 =0A= =20 =0A= =0A= It looks to me that the Reject response with reason 0x06 should be enough= =0A= for the initiator to handle the situation you describe.=20 =0A= If the TM are the cause then each of those has a response indicating=0A= termination. After receiving a reject an initiator should reissue new TMs o= nly=0A= after getting back one or more responses. The initiator may even=0A= accurately guess the number of commands the targets is capable of handling = per=0A= connection and per LU. The only case in which this handling may be awkward = is=0A= over a WAN but if you are operating over a WAN and TM performance is your o= nly=0A= worry you are in good shape :-)=20 =0A= =0A= Julo =0A= =0A= =0A= =0A= =0A= =0A= =0A= =0A= =0A= =0A= =0A= =0A= [Ips]=0A= Immediate Commands in iSCSI=0A= =0A= =0A= =0A= =20 =0A= =0A= =20 =0A= =0A= =0A= =0A= Jacob_Cherian =0A= =0A= =0A= to: =0A= =0A= =0A= ips=0A= =0A= =0A= =0A= 12/03/07 05:23 AM=0A= =0A= =0A= =0A= =20 =0A= =20 =0A= =20 =0A= =0A= =0A= =0A= =0A= =0A= =0A= Cc: =0A= =0A= =0A= Nam_Nguyen=0A= =0A= =0A= =0A= =0A= =0A= =0A= =0A= =0A= =0A= =0A= =0A= =0A= =20 =0A= =0A= =0A= =0A= =0A= =0A= =0A= =0A= =0A= =0A= =0A= =0A= =0A= =0A= =0A= RFC 3720 states:=20 =0A= =0A= The number of commands used for immediate delivery is not limited and their= =0A= delivery for execution is not acknowledged through the numbering scheme.=0A= Immediate commands MAY be rejected by the iSCSI target layer due to a lack = of=0A= resources. An iSCSI target MUST be able to handle at least one immediate ta= sk=0A= management command and one immediate non-task-management iSCSI command per= =0A= connection at any time.=20 =0A= =0A= =20 =0A= =0A= With storage systems that have very large number of logical units, we run i= nto=0A= an issue when we use LU Reset or Abort Task for reset recovery. I=0A= couldn=92t find a way to tell an initiator how many immediate commands a=0A= target can accept. The issue with this is that if we have to issue multiple= =0A= task management commands and it is more than what the target can accept, it= =0A= will reject the task management request. Since we don=92t know why it was= =0A= rejected, we will have to escalate the reset. This can be very disruptive w= ith=0A= nodes that have very large number of logical units. There may be two ways t= o=0A= resolve this:=20 =0A= =0A= =20 =0A= =0A= 1) Define a new Text negotiation key for number of=0A= outstanding immediate commands supported. Unlike non-immediate (data) comma= nds=0A= where we can send TASK SET FULL, there is no way for a target to report in = the=0A= Task Management Function Response PDU that is unable to process the current= =0A= request before it is out of resources. The default vaule for this new key= =0A= can be 1, so if this key is not negotiated initiators can assume one per=0A= connection.=20 =0A= =0A= 2) Define a new value for the Response field in the Task=0A= Management Function Response PDU for reporting that the target cannot proce= ss=0A= the request currently.=20 =0A= =0A= =20 =0A= =0A= Any suggestions on how to resolve this?=20 =0A= =0A= =20 =0A= =0A= Thanks,=20 =0A= =0A= Jacob=20 =0A= _______________________________________________ =0A= Ips mailing list =0A= Ips@ietf.org =0A= https://www1.ietf.org/mailman/listinfo/ips=0A= =0A= =0A= =0A= =0A= =0A= =0A= =0A= _________________________________________________________________ It's simple! Sell your car for just $30 at CarPoint.com.au http://a.ninemsn.com.au/b.aspx?URL=3Dhttp%3A%2F%2Fsecure%2Dau%2Eimrworldwid= e%2Ecom%2Fcgi%2Dbin%2Fa%2Fci%5F450304%2Fet%5F2%2Fcg%5F801459%2Fpi%5F1004813= %2Fai%5F859641&_t=3D762955845&_r=3Dtig_OCT07&_m=3DEXT= --_44fbb6ae-6d5b-4897-b310-34a25e73e823_ Content-Type: text/html; charset="Windows-1252" Content-Transfer-Encoding: quoted-printable
=0A= =0A=
=0A=
Hi Jacob,

A Reject PDU which indica= tes its reason as "Immediate Command Reject - too many immediate commands" = seems pretty granular to me. :-) That's what the reject code 0x06 means (as= Julo had recommended). The initiator may resubmit the PDU whenever it deem= s appropriate. Does that make sense?

IMHO, a target implementation c= onstrains the number of non-TMF commands and might throttle repeated TMF co= mmands when they are in progress (and the clearing effects are taking a lon= g time to resolve). Even with a large number of LU's it would be unusual to= constrain the number of concurrent LU Reset operations. Have you got a tra= ce showing this behaviour?

HTH,
Ken

Subject: RE: [Ips] Immediate Commands in iSCSI
Da= te: Mon, 3 Dec 2007 14:15:19 -0600
From: Jacob_Cherian@Dell.com
To: J= ulian_Satran@il.ibm.com
CC: ips@ietf.org; Nam_Nguyen@Dell.com

=0A= =0A= =0A= = =0A= =0A= =0A= =0A= =0A= =0A= =0A= =0A= =0A= =0A= =0A= =0A= =0A=
=0A= =0A=

Julian,

=0A= =0A=

    No offense= taken. The scenario that you=0A= brought up was considered since most configurations with iSCSI are with=0A= multiple hosts. However, the allocation of immediate commands to connection= /sessions/hosts=0A= depends on the actual implementation. RFC 3720 already requires support for= at=0A= least one per connection. Even we cannot agree on a key, having a more gran= ular=0A= status for a rejected TMF that says the target is currently busy processing= a=0A= TMF and cannot process another at this time (but will be able to sometime i= n=0A= the future), helps initiators to wait and retry, instead of assuming a fail= ure=0A= and escalating the reset.

=0A= =0A=

 

=0A= =0A=

 Jacob

=0A= =0A=

 

=0A= =0A=
=0A= =0A=

From: Julian Satran=0A= [mailto:Julian_Satran@il.ibm.com]
=0A= Sent: Monday, December 03, 2007 8:42 AM
=0A= To: Cherian, Jacob
=0A= Cc: ips@ietf.org; Nguyen, Nam
=0A= Subject: RE: [Ips] Immediate Commands in iSCSI

=0A= =0A=
=0A= =0A=

 

=0A= =0A=


=0A= Jacob,<= /span>
=0A=
=0A= I do no= t want=0A= to appear offensive - but did you consider the "larger picture"?=0A= Assuming that the negotiation key or status is there and assuming that the= =0A= target talks to 7 (seven) different initiators what would it tell each of t= hem=0A= (it has 7 different sessions)? Should he reserve a quota for each of them -= so=0A= that it can meet what it declared? What should a target do with commands th= at=0A= exceed the number?
=0A=
=0A= In iSCS= I as a rule=0A= we avoided setting or negotiating parameters that we don't need to=0A= set/negotiate and I think the number of accepted immediate commands is one = of=0A= them.
=0A=
=0A= Julo
=0A=
=0A=

=0A= =0A= =0A= =0A= =0A= =0A= =0A=
=0A=

=0A= 		=0A= =0A= =0A= =0A= =0A=
=0A=

RE:=0A= [Ips] Immediate Commands in iSCSI

=0A=
=0A=

 

=0A= =0A= =0A= =0A= =0A= =0A= =0A= =0A=
=0A=

Jacob_Cherian =

=0A= 		=0A=

to:

=0A= 		=0A=

Julian=0A= Satran

=0A= 		=0A=

= 12/03/= 07 04:06 PM

=0A=
=0A=

 

=0A= =0A= =0A= =0A= =0A= =0A=
=0A= =0A= =0A= =0A= =0A= =0A=
=0A=

Cc:

=0A= 		=0A=

ips,=0A= Nam_Nguyen

=0A=
=0A=
=0A=
=0A= =0A=

 

=0A= =0A=
= =0A= =0A=
=0A= =0A=
=0A= =0A=


=0A=
=0A=
=0A= The=0A= goal of the new status or key is to help the initiator to know the correct= =0A= number of immediate commands that the target supports. The issue with=0A= overloading a status that does clearly define the error condition is that i= t=0A= does help in achieving interoperability between the myriad of initiators an= d=0A= targets that are out there. I was looking to see if there was another way t= hat=0A= I had not considered.
=0A=  =0A=
=0A= Jacob=0A=
=0A=  =0A=
=0A=  =0A=
=0A= Fro= m: Julian Satran=0A= [mailto:Julian_Satran@il.ibm.com]
=0A= Sent: Monday, December 03, 2007 1:23 AM
=0A= To: Cherian, Jacob
=0A= Cc: ips@ietf.org; Nguyen, Nam
=0A= Subject: Re: [Ips] Immediate Commands in iSCSI
=0A=  
=0A=
=0A= It looks to  me that the Reject response with reason 0x06 should be en= ough=0A= for the initiator to handle the situation you describe.
=0A= If the TM are the cause then each of those has a response indicating=0A= termination. After receiving a reject an initiator should reissue new TMs o= nly=0A= after getting back one or more responses.  The initiator may even=0A= accurately guess the number of commands the targets is capable of handling = per=0A= connection and per LU. The only case in which this handling may be awkward = is=0A= over a WAN but if you are operating over a WAN and TM performance is your o= nly=0A= worry you are in good shape :-)
=0A=
=0A= Julo

=0A= =0A= =0A= =0A= =0A= =0A= =0A=
=0A=

=0A= 		=0A=

 

=0A= =0A= =0A= =0A= =0A=
=0A=

[Ips]=0A= Immediate Commands in iSCSI

=0A=
=0A=


=0A=  

=0A=  
=0A= =0A= =0A= =0A= =0A= =0A= =0A= =0A=
=0A=

Jacob_Cherian =

=0A= 		=0A=

to:

=0A= 		=0A=

ips=0A=

=0A= 		=0A=

= 12/03/= 07 05:23 AM

=0A=
=0A=
=0A=  
=0A=  
=0A= =0A= =0A= =0A= =0A= =0A=
=0A= =0A= =0A= =0A= =0A= =0A=
=0A=

Cc:

=0A= 		=0A=

Nam_Nguyen

=0A=
=0A=
=0A=
=0A= =0A=
=0A=  
=0A= =0A=

&n= bsp;

=0A= =0A=
= =0A= =0A=
=0A= =0A=
=0A= =0A=


=0A=
=0A=
=0A=
=0A=
=0A= RFC 3720 states:
=0A=
=0A= The number of commands used for immediate delivery is not limited and their= =0A= delivery for execution is not acknowledged through the numbering scheme.=0A= Immediate commands MAY be rejected by the iSCSI target layer due to a lack = of=0A= resources. An iSCSI target MUST be able to handle at least one immediate ta= sk=0A= management command and one immediate non-task-management iSCSI command per= =0A= connection at any time.
=0A=
=0A=  
=0A=
=0A= With storage systems that have very large number of logical units, we run i= nto=0A= an issue when we use LU Reset or Abort Task for reset recovery. I=0A= couldn=92t find a way to tell an initiator how many immediate commands a=0A= target can accept. The issue with this is that if we have to issue multiple= =0A= task management commands and it is more than what the target can accept, it= =0A= will reject the task management request. Since we don=92t know why it was= =0A= rejected, we will have to escalate the reset. This can be very disruptive w= ith=0A= nodes that have very large number of logical units. There may be two ways t= o=0A= resolve this:
=0A=
=0A=  
=0A=
=0A= 1)      Define a new Text negotiation key for number of=0A= outstanding immediate commands supported. Unlike non-immediate (data) comma= nds=0A= where we can send TASK SET FULL, there is no way for a target to report in = the=0A= Task Management Function Response PDU that is unable to process the current= =0A= request before it is out of resources.  The default vaule for this new= key=0A= can be 1, so if this key is not negotiated initiators can assume one per=0A= connection.
=0A=
=0A= 2)      Define a new value for the Response field in the Tas= k=0A= Management Function Response PDU for reporting that the target cannot proce= ss=0A= the request currently.
=0A=
=0A=  
=0A=
=0A= Any suggestions on how to resolve this?
=0A=
=0A=  
=0A=
=0A= Thanks,
=0A=
=0A= Jacob <= br>=0A= _______________________________________________
=0A= Ips mailing list
=0A= Ips@ietf.org
=0A= https= ://www1.ietf.org/mailman/listinfo/ips=0A=

=0A= =0A=
=0A= =0A= =0A= =0A= =0A=

Sell your car for just $30 at CarPoint.com.au. It's simple! = --_44fbb6ae-6d5b-4897-b310-34a25e73e823_-- --===============0175238084== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline _______________________________________________ Ips mailing list Ips@ietf.org https://www1.ietf.org/mailman/listinfo/ips --===============0175238084==-- From ips-bounces@ietf.org Thu Dec 06 04:10:54 2007 Return-path: Received: from [127.0.0.1] (helo=stiedprmman1.va.neustar.com) by megatron.ietf.org with esmtp (Exim 4.43) id 1J0Ckt-0006vb-Iv; Thu, 06 Dec 2007 04:10:51 -0500 Received: from ips by megatron.ietf.org with local (Exim 4.43) id 1J0Ckr-0006vT-0p for ips-confirm+ok@megatron.ietf.org; Thu, 06 Dec 2007 04:10:49 -0500 Received: from [10.91.34.44] (helo=ietf-mx.ietf.org) by megatron.ietf.org with esmtp (Exim 4.43) id 1J0Ckp-0006vI-Vq for ips@ietf.org; Thu, 06 Dec 2007 04:10:47 -0500 Received: from rv-out-0910.google.com ([209.85.198.190]) by ietf-mx.ietf.org with esmtp (Exim 4.43) id 1J0Ckj-0003W6-HW for ips@ietf.org; Thu, 06 Dec 2007 04:10:47 -0500 Received: by rv-out-0910.google.com with SMTP id l15so154412rvb for ; Thu, 06 Dec 2007 01:10:41 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:message-id:date:from:to:subject:cc:in-reply-to:mime-version:content-type:references; bh=tcri5mfyA9CJxEfdZ4Qe84lV1VrqtICAFZr7kGrfvYQ=; b=mKBonzBPBtdZLmhFws1ow/SPZEeNG0NVzD8mc/ltwFfWBf1nv8DEnU6MWvdYfzrSVgSSZeY0oK/ogHvoo8MejVUvTsCMAxUGibaymcRhoWGxzbfrRJjzMzH1p70D/tK5jLWZl9nWBtvAW8g1j+oFUeGJ0tcDgUkOBB9jQOP/zGU= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=message-id:date:from:to:subject:cc:in-reply-to:mime-version:content-type:references; b=Rwir01cEG5d7eW4+AkNXXUcfxBMAqP78VcT6oLjjHo6SNy0y26lqYwBzWLhv/33BtMijv2/8lYY+0RUWct4Jp97tUBLkgID766FdLVcoLrE5wtsnhzqwe8iya7j2hBhIZEUhvDzUXNG/vsMv+RRkU6ZlRspIreHv4n/0z4rcGL8= Received: by 10.141.153.16 with SMTP id f16mr1809548rvo.1196932240917; Thu, 06 Dec 2007 01:10:40 -0800 (PST) Received: by 10.140.127.16 with HTTP; Thu, 6 Dec 2007 01:10:40 -0800 (PST) Message-ID: Date: Thu, 6 Dec 2007 14:40:40 +0530 From: "Jijo Chacko" To: "Ken Sandars" Subject: Re: [Ips] Immediate Commands in iSCSI In-Reply-To: MIME-Version: 1.0 References: <5DDAB7BA7BDB58439DD0EED0B8E9A3AEFD9D2E@ausx3mpc102.aus.amer.dell.com> <5DDAB7BA7BDB58439DD0EED0B8E9A3AEFD9F00@ausx3mpc102.aus.amer.dell.com> X-Spam-Score: 0.0 (/) X-Scan-Signature: dae47ebd0d959deee2d6f67621ddb2e3 Cc: ips@ietf.org, jacob_cherian@dell.com, nam_nguyen@dell.com, julian_satran@il.ibm.com X-BeenThere: ips@ietf.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: IP Storage List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Content-Type: multipart/mixed; boundary="===============1617781316==" Errors-To: ips-bounces@ietf.org --===============1617781316== Content-Type: multipart/alternative; boundary="----=_Part_4462_11068876.1196932240895" ------=_Part_4462_11068876.1196932240895 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Content-Disposition: inline A related question - from the above discussions, i assume that there is no, application-level, end-to-end flow control from the initiator-to-target, as it relates to queued commands. As TCP flow control layers already allowed the command-bytes in to the host, we have the memory buffers at the TCP layer, and later commands are just ignored, with ICR return code 0x06. And initiator re-submits, target re-rejects, and this repeats this is not good for internet, as it relates to congestion. When tagged queueing/no of outstanding commands support added to SCSI, iSCSI was not in anybody's mind(IMHO).. please dis-agree with me here. And strongly feel that, we need to have some way of throttling the initiator to prevent from swamping the target, or target implementation code should have the logic to re-submit the un-honored commands from the TCP buffers(or local buffers), before returning an error code to the initiator. or there can be an iSCSI, pending-coomands cache at the target stack, to solve this problem. In essence, IMHO, we should do some fixes for target iSCSI implementation, to provide some staging buffers for already tcp-received commands. We should not be returning wrong error code to the initiator, if we already received the command at the TCP layer, and the target has the ability to apply it. We should be returning the error code only if target failed to apply it. IN the SCSI world, as the number of initiators are typically just 1, and the data is not travelling across WAN, target can return "Too many commands pending" errors.. as it has only local effect, having few meters. Please correct me, if i am wrong... thanks, Jijo Chacko Bangalore. On Dec 5, 2007 6:38 AM, Ken Sandars wrote: > Hi Jacob, > > A Reject PDU which indicates its reason as "Immediate Command Reject - too > many immediate commands" seems pretty granular to me. :-) That's what the > reject code 0x06 means (as Julo had recommended). The initiator may resubmit > the PDU whenever it deems appropriate. Does that make sense? > > IMHO, a target implementation constrains the number of non-TMF commands > and might throttle repeated TMF commands when they are in progress (and the > clearing effects are taking a long time to resolve). Even with a large > number of LU's it would be unusual to constrain the number of concurrent LU > Reset operations. Have you got a trace showing this behaviour? > > HTH, > Ken > > ------------------------------ > Subject: RE: [Ips] Immediate Commands in iSCSI > Date: Mon, 3 Dec 2007 14:15:19 -0600 > From: Jacob_Cherian@Dell.com > To: Julian_Satran@il.ibm.com > CC: ips@ietf.org; Nam_Nguyen@Dell.com > > Julian, > > No offense taken. The scenario that you brought up was considered > since most configurations with iSCSI are with multiple hosts. However, the > allocation of immediate commands to connection/sessions/hosts depends on the > actual implementation. RFC 3720 already requires support for at least one > per connection. Even we cannot agree on a key, having a more granular status > for a rejected TMF that says the target is currently busy processing a TMF > and cannot process another at this time (but will be able to sometime in the > future), helps initiators to wait and retry, instead of assuming a failure > and escalating the reset. > > > > Jacob** > > > > *From:* Julian Satran [mailto:Julian_Satran@il.ibm.com] > *Sent:* Monday, December 03, 2007 8:42 AM > *To:* Cherian, Jacob > *Cc:* ips@ietf.org; Nguyen, Nam > *Subject:* RE: [Ips] Immediate Commands in iSCSI > > > > > Jacob, > > I do not want to appear offensive - but did you consider the "larger > picture"? Assuming that the negotiation key or status is there and assuming > that the target talks to 7 (seven) different initiators what would it tell > each of them (it has 7 different sessions)? Should he reserve a quota for > each of them - so that it can meet what it declared? What should a target do > with commands that exceed the number? > > In iSCSI as a rule we avoided setting or negotiating parameters that we > don't need to set/negotiate and I think the number of accepted immediate > commands is one of them. > > Julo > > *RE: [Ips] Immediate Commands in iSCSI* > > > > *Jacob_Cherian * > > to: > > Julian Satran > > 12/03/07 04:06 PM > > > > Cc: > > ips, Nam_Nguyen > > > ------------------------------ > > > > > The goal of the new status or key is to help the initiator to know the > correct number of immediate commands that the target supports. The issue > with overloading a status that does clearly define the error condition is > that it does help in achieving interoperability between the myriad of > initiators and targets that are out there. I was looking to see if there was > another way that I had not considered. > > Jacob > > > *From:* Julian Satran [mailto:Julian_Satran@il.ibm.com] * > Sent:* Monday, December 03, 2007 1:23 AM* > To:* Cherian, Jacob* > Cc:* ips@ietf.org; Nguyen, Nam* > Subject:* Re: [Ips] Immediate Commands in iSCSI > > > It looks to me that the Reject response with reason 0x06 should be enough > for the initiator to handle the situation you describe. > If the TM are the cause then each of those has a response indicating > termination. After receiving a reject an initiator should reissue new TMs > only after getting back one or more responses. The initiator may even > accurately guess the number of commands the targets is capable of handling > per connection and per LU. The only case in which this handling may be > awkward is over a WAN but if you are operating over a WAN and TM performance > is your only worry you are in good shape :-) > > Julo > > > > *[Ips] Immediate Commands in iSCSI* > > > > > > *Jacob_Cherian * > > to: > > ips > > 12/03/07 05:23 AM > > > > > Cc: > > Nam_Nguyen > > > > > ------------------------------ > > > > > > > RFC 3720 states: > > The number of commands used for immediate delivery is not limited and > their delivery for execution is not acknowledged through the numbering > scheme. Immediate commands MAY be rejected by the iSCSI target layer due to > a lack of resources. An iSCSI target MUST be able to handle at least one > immediate task management command and one immediate non-task-management > iSCSI command per connection at any time. > > > > With storage systems that have very large number of logical units, we run > into an issue when we use LU Reset or Abort Task for reset recovery. I > couldn't find a way to tell an initiator how many immediate commands a > target can accept. The issue with this is that if we have to issue multiple > task management commands and it is more than what the target can accept, it > will reject the task management request. Since we don't know why it was > rejected, we will have to escalate the reset. This can be very disruptive > with nodes that have very large number of logical units. There may be two > ways to resolve this: > > > > 1) Define a new Text negotiation key for number of outstanding > immediate commands supported. Unlike non-immediate (data) commands where we > can send TASK SET FULL, there is no way for a target to report in the Task > Management Function Response PDU that is unable to process the current > request before it is out of resources. The default vaule for this new key > can be 1, so if this key is not negotiated initiators can assume one per > connection. > > 2) Define a new value for the Response field in the Task Management > Function Response PDU for reporting that the target cannot process the > request currently. > > > > Any suggestions on how to resolve this? > > > > Thanks, > > Jacob > _______________________________________________ > Ips mailing list > Ips@ietf.org* > *https://www1.ietf.org/mailman/listinfo/ips > > > ------------------------------ > Sell your car for just $30 at CarPoint.com.au. It's simple! > > _______________________________________________ > Ips mailing list > Ips@ietf.org > https://www1.ietf.org/mailman/listinfo/ips > > ------=_Part_4462_11068876.1196932240895 Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Content-Disposition: inline
A related question - from the above discussions, i assume that there is no, application-level, end-to-end flow control from the initiator-to-target, as it relates to queued commands. As TCP flow control layers already allowed the command-bytes in to the host,  we have the memory buffers at the TCP layer, and later commands are just ignored, with ICR return code 0x06. And initiator re-submits, target re-rejects,  and this repeats this is not good for internet, as it relates to congestion. When tagged queueing/no of outstanding commands support added to SCSI, iSCSI was not in anybody's mind(IMHO).. please dis-agree with me here.

And strongly feel that, we need to have some way of throttling the initiator to prevent from swamping the target, or  target implementation code should have the logic to re-submit the un-honored commands from the TCP buffers(or local buffers), before returning an error code to the initiator. or there can be an iSCSI, pending-coomands cache at the target stack, to solve this problem.

In essence, IMHO, we should do some fixes for target iSCSI implementation, to provide some staging buffers for already tcp-received commands. We should not be returning wrong error code to the initiator, if we already received the command at the TCP layer, and the target has the ability to apply it. We should be returning the error code only if target failed to apply it.

IN the SCSI world, as the number of initiators are typically just 1, and the data is not travelling across WAN, target can  return "Too many commands pending" errors.. as it has only local effect, having few meters.

Please correct me, if i am wrong...

thanks,
Jijo Chacko
Bangalore.

On Dec 5, 2007 6:38 AM, Ken Sandars <kensandars@hotmail.com > wrote:
Hi Jacob,

A Reject PDU which indicates its reason as "Immediate Command Reject - too many immediate commands" seems pretty granular to me. :-) That's what the reject code 0x06 means (as Julo had recommended). The initiator may resubmit the PDU whenever it deems appropriate. Does that make sense?

IMHO, a target implementation constrains the number of non-TMF commands and might throttle repeated TMF commands when they are in progress (and the clearing effects are taking a long time to resolve). Even with a large number of LU's it would be unusual to constrain the number of concurrent LU Reset operations. Have you got a trace showing this behaviour?

HTH,
Ken

Subject: RE: [Ips] Immediate Commands in iSCSI
Date: Mon, 3 Dec 2007 14:15:19 -0600
From: Jacob_Cherian@Dell.com
To: Julian_Satran@il.ibm.com
CC: ips@ietf.org; Nam_Nguyen@Dell.com

Julian,

    No offense taken. The scenario that you brought up was considered since most configurations with iSCSI are with multiple hosts. However, the allocation of immediate commands to connection/sessions/hosts depends on the actual implementation. RFC 3720 already requires support for at least one per connection. Even we cannot agree on a key, having a more granular status for a rejected TMF that says the target is currently busy processing a TMF and cannot process another at this time (but will be able to sometime in the future), helps initiators to wait and retry, instead of assuming a failure and escalating the reset.

 

 Jacob

 

From: Julian Satran [mailto:Julian_Satran@il.ibm.com]
Sent: Monday, December 03, 2007 8:42 AM
To: Cherian, Jacob
Cc: ips@ietf.org; Nguyen, Nam
Subject: RE: [Ips] Immediate Commands in iSCSI

 


Jacob,

I do not want to appear offensive - but did you consider the "larger picture"? Assuming that the negotiation key or status is there and assuming that the target talks to 7 (seven) different initiators what would it tell each of them (it has 7 different sessions)? Should he reserve a quota for each of them - so that it can meet what it declared? What should a target do with commands that exceed the number?

In iSCSI as a rule we avoided setting or negotiating parameters that we don't need to set/negotiate and I think the number of accepted immediate commands is one of them.

Julo

RE: [Ips] Immediate Commands in iSCSI

 

Jacob_Cherian

to:

Julian Satran

12/03/07 04:06 PM

 

Cc:

ips, Nam_Nguyen

 




The goal of the new status or key is to help the initiator to know the correct number of immediate commands that the target supports. The issue with overloading a status that does clearly define the error condition is that it does help in achieving interoperability between the myriad of initiators and targets that are out there. I was looking to see if there was another way that I had not considered.
 
Jacob
 
 
From: Julian Satran [mailto:Julian_Satran@il.ibm.com]
Sent: Monday, December 03, 2007 1:23 AM
To: Cherian, Jacob
Cc: ips@ietf.org; Nguyen, Nam
Subject: Re: [Ips] Immediate Commands in iSCSI
 

It looks to  me that the Reject response with reason 0x06 should be enough for the initiator to handle the situation you describe.
If the TM are the cause then each of those has a response indicating termination. After receiving a reject an initiator should reissue new TMs only after getting back one or more responses.  The initiator may even accurately guess the number of commands the targets is capable of handling per connection and per LU. The only case in which this handling may be awkward is over a WAN but if you are operating over a WAN and TM performance is your only worry you are in good shape :-)

Julo

 

[Ips] Immediate Commands in iSCSI


 

 

Jacob_Cherian

to:

ips

12/03/07 05:23 AM

 
 

Cc:

Nam_Nguyen

 

 






RFC 3720 states:

The number of commands used for immediate delivery is not limited and their delivery for execution is not acknowledged through the numbering scheme. Immediate commands MAY be rejected by the iSCSI target layer due to a lack of resources. An iSCSI target MUST be able to handle at least one immediate task management command and one immediate non-task-management iSCSI command per connection at any time.

 

With storage systems that have very large number of logical units, we run into an issue when we use LU Reset or Abort Task for reset recovery. I couldn't find a way to tell an initiator how many immediate commands a target can accept. The issue with this is that if we have to issue multiple task management commands and it is more than what the target can accept, it will reject the task management request. Since we don't know why it was rejected, we will have to escalate the reset. This can be very disruptive with nodes that have very large number of logical units. There may be two ways to resolve this:

 

1)      Define a new Text negotiation key for number of outstanding immediate commands supported. Unlike non-immediate (data) commands where we can send TASK SET FULL, there is no way for a target to report in the Task Management Function Response PDU that is unable to process the current request before it is out of resources.  The default vaule for this new key can be 1, so if this key is not negotiated initiators can assume one per connection.

2)      Define a new value for the Response field in the Task Management Function Response PDU for reporting that the target cannot process the request currently.

 

Any suggestions on how to resolve this?

 

Thanks,

Jacob
_______________________________________________
Ips mailing list
Ips@ietf.org
https://www1.ietf.org/mailman/listinfo/ips


Sell your car for just $30 at CarPoint.com.au. It's simple!

_______________________________________________
Ips mailing list
Ips@ietf.org
https://www1.ietf.org/mailman/listinfo/ips


------=_Part_4462_11068876.1196932240895-- --===============1617781316== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline _______________________________________________ Ips mailing list Ips@ietf.org https://www1.ietf.org/mailman/listinfo/ips --===============1617781316==-- From ips-bounces@ietf.org Thu Dec 06 09:46:49 2007 Return-path: Received: from [127.0.0.1] (helo=stiedprmman1.va.neustar.com) by megatron.ietf.org with esmtp (Exim 4.43) id 1J0Hzy-0006ht-0p; Thu, 06 Dec 2007 09:46:46 -0500 Received: from ips by megatron.ietf.org with local (Exim 4.43) id 1J0Hzv-0006em-WE for ips-confirm+ok@megatron.ietf.org; Thu, 06 Dec 2007 09:46:44 -0500 Received: from [10.91.34.44] (helo=ietf-mx.ietf.org) by megatron.ietf.org with esmtp (Exim 4.43) id 1J0Hzv-0006d2-Kh for ips@ietf.org; Thu, 06 Dec 2007 09:46:43 -0500 Received: from mexforward.lss.emc.com ([128.222.32.20]) by ietf-mx.ietf.org with esmtp (Exim 4.43) id 1J0Hzq-0003UF-Ur for ips@ietf.org; Thu, 06 Dec 2007 09:46:42 -0500 Received: from hop04-l1d11-si02.isus.emc.com (HOP04-L1D11-SI02.isus.emc.com [10.254.111.55]) by mexforward.lss.emc.com (Switch-3.2.5/Switch-3.1.7) with ESMTP id lB6EkZo6000416; Thu, 6 Dec 2007 09:46:35 -0500 (EST) Received: from mailhub.lss.emc.com (nirah.lss.emc.com [10.254.144.13]) by hop04-l1d11-si02.isus.emc.com (Tablus Interceptor); Thu, 6 Dec 2007 09:46:32 -0500 Received: from corpussmtp3.corp.emc.com (corpussmtp3.corp.emc.com [10.254.64.53]) by mailhub.lss.emc.com (Switch-3.2.5/Switch-3.1.7) with ESMTP id lB6Ejex1011380; Thu, 6 Dec 2007 09:46:30 -0500 (EST) From: Black_David@emc.com Received: from CORPUSMX20A.corp.emc.com ([128.221.62.13]) by corpussmtp3.corp.emc.com with Microsoft SMTPSVC(6.0.3790.1830); Thu, 6 Dec 2007 09:46:21 -0500 X-MimeOLE: Produced By Microsoft Exchange V6.5 Content-class: urn:content-classes:message MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable Subject: RE: [Ips] Immediate Commands in iSCSI Date: Thu, 6 Dec 2007 09:46:20 -0500 Message-ID: In-Reply-To: X-MS-Has-Attach: X-MS-TNEF-Correlator: Thread-Topic: [Ips] Immediate Commands in iSCSI thread-index: Acg36Cpsntt9u3UfRoKYVE5Pd8a5fwAK3Zuw References: <5DDAB7BA7BDB58439DD0EED0B8E9A3AEFD9D2E@ausx3mpc102.aus.amer.dell.com><5DDAB7BA7BDB58439DD0EED0B8E9A3AEFD9F00@ausx3mpc102.aus.amer.dell.com> To: X-OriginalArrivalTime: 06 Dec 2007 14:46:21.0991 (UTC) FILETIME=[C4A91370:01C83816] X-PMX-Version: 4.7.1.128075, Antispam-Engine: 2.5.1.298604, Antispam-Data: 2007.8.30.51425 X-PerlMx-Spam: Gauge=, SPAM=0%, Reason='EMC_BODY_1+ -3, EMC_FROM_0+ -3, NO_REAL_NAME 0, __C230066_P5 0, __CT 0, __CTE 0, __CT_TEXT_PLAIN 0, __HAS_MSGID 0, __IMS_MSGID 0, __MIME_TEXT_ONLY 0, __MIME_VERSION 0, __SANE_MSGID 0' X-Tablus-Inspected: yes X-Tablus-Classifications: public X-Tablus-Action: allow X-Spam-Score: -4.0 (----) X-Scan-Signature: e5ba305d0e64821bf3d8bc5d3bb07228 Cc: ips@ietf.org, Black_David@emc.com X-BeenThere: ips@ietf.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: IP Storage List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: ips-bounces@ietf.org Jijo, > A related question - from the above discussions, i assume that > there is no, application-level, end-to-end flow control from > the initiator-to-target, as it relates to queued commands. As > TCP flow control layers already allowed the command-bytes in > to the host, we have the memory buffers at the TCP layer, > and later commands are just ignored, with ICR return code 0x06. > And initiator re-submits, target re-rejects, and this repeats > this is not good for internet, as it relates to congestion. > When tagged queueing/no of outstanding commands support added > to SCSI, iSCSI was not in anybody's mind(IMHO).. please > dis-agree with me here. Happy to do so, as that last statement is incorrect, because the initial assumption is wrong. Tagged command queuing is in SAM-2, which is normatively referenced by iSCSI (RFC 3720). The mechanism that you're looking for is TASK SET FULL status (28h), which is widely supported. The recent discussion on this list was specific to iSCSI commands, which are intended for exceptional cases where the outstanding queue of undelivered commands in iSCSI needs to be bypassed (see Section 3.2.2.1 of RFC 3720). Thanks, --David ---------------------------------------------------- David L. Black, Distinguished Engineer EMC Corporation, 176 South St., Hopkinton, MA 01748 +1 (508) 293-7953 FAX: +1 (508) 293-7786 black_david@emc.com Mobile: +1 (978) 394-7754 ---------------------------------------------------- _______________________________________________ Ips mailing list Ips@ietf.org https://www1.ietf.org/mailman/listinfo/ips From ips-bounces@ietf.org Thu Dec 06 09:53:43 2007 Return-path: Received: from [127.0.0.1] (helo=stiedprmman1.va.neustar.com) by megatron.ietf.org with esmtp (Exim 4.43) id 1J0I6f-0003pu-8Q; Thu, 06 Dec 2007 09:53:41 -0500 Received: from ips by megatron.ietf.org with local (Exim 4.43) id 1J0I6e-0003pY-3b for ips-confirm+ok@megatron.ietf.org; Thu, 06 Dec 2007 09:53:40 -0500 Received: from [10.90.34.44] (helo=chiedprmail1.ietf.org) by megatron.ietf.org with esmtp (Exim 4.43) id 1J0I6d-0003pN-Os for ips@ietf.org; Thu, 06 Dec 2007 09:53:39 -0500 Received: from mexforward.lss.emc.com ([128.222.32.20]) by chiedprmail1.ietf.org with esmtp (Exim 4.43) id 1J0I6d-000268-BG for ips@ietf.org; Thu, 06 Dec 2007 09:53:39 -0500 Received: from hop04-l1d11-si01.isus.emc.com (HOP04-L1D11-SI01.isus.emc.com [10.254.111.54]) by mexforward.lss.emc.com (Switch-3.2.5/Switch-3.1.7) with ESMTP id lB6ErbtH003732; Thu, 6 Dec 2007 09:53:37 -0500 (EST) Received: from mailhub.lss.emc.com (uraeus.lss.emc.com [10.254.144.14]) by hop04-l1d11-si01.isus.emc.com (Tablus Interceptor); Thu, 6 Dec 2007 09:53:37 -0500 Received: from corpussmtp3.corp.emc.com (corpussmtp3.corp.emc.com [10.254.64.53]) by mailhub.lss.emc.com (Switch-3.2.5/Switch-3.1.7) with ESMTP id lB6Er9GP022605; Thu, 6 Dec 2007 09:53:35 -0500 (EST) From: Black_David@emc.com Received: from CORPUSMX20A.corp.emc.com ([128.221.62.13]) by corpussmtp3.corp.emc.com with Microsoft SMTPSVC(6.0.3790.1830); Thu, 6 Dec 2007 09:53:15 -0500 X-MimeOLE: Produced By Microsoft Exchange V6.5 Content-class: urn:content-classes:message MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable Subject: RE: [Ips] Immediate Commands in iSCSI Date: Thu, 6 Dec 2007 09:53:13 -0500 Message-ID: In-Reply-To: X-MS-Has-Attach: X-MS-TNEF-Correlator: Thread-Topic: [Ips] Immediate Commands in iSCSI thread-index: Acg36Cpsntt9u3UfRoKYVE5Pd8a5fwAK3ZuwAAD9J9A= X-Priority: 1 Priority: Urgent Importance: high References: <5DDAB7BA7BDB58439DD0EED0B8E9A3AEFD9D2E@ausx3mpc102.aus.amer.dell.com><5DDAB7BA7BDB58439DD0EED0B8E9A3AEFD9F00@ausx3mpc102.aus.amer.dell.com> To: , X-OriginalArrivalTime: 06 Dec 2007 14:53:15.0632 (UTC) FILETIME=[BB35AF00:01C83817] X-PMX-Version: 4.7.1.128075, Antispam-Engine: 2.5.1.298604, Antispam-Data: 2007.8.30.51425 X-PerlMx-Spam: Gauge=, SPAM=0%, Reason='EMC_BODY_1+ -3, EMC_FROM_0+ -3, PRIORITY_NO_NAME 0.716, NO_REAL_NAME 0, __C230066_P5 0, __CT 0, __CTE 0, __CT_TEXT_PLAIN 0, __HAS_MSGID 0, __HAS_X_PRIORITY 0, __IMS_MSGID 0, __MIME_TEXT_ONLY 0, __MIME_VERSION 0, __SANE_MSGID 0' X-Tablus-Inspected: yes X-Tablus-Classifications: public X-Tablus-Action: allow X-Spam-Score: 0.0 (/) X-Scan-Signature: 39bd8f8cbb76cae18b7e23f7cf6b2b9f Cc: ips@ietf.org X-BeenThere: ips@ietf.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: IP Storage List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: ips-bounces@ietf.org Crucial editing mistake - see below. Sorry, --David > Jijo, >=20 > > A related question - from the above discussions, i assume that > > there is no, application-level, end-to-end flow control from > > the initiator-to-target, as it relates to queued commands. As > > TCP flow control layers already allowed the command-bytes in > > to the host, we have the memory buffers at the TCP layer, > > and later commands are just ignored, with ICR return code 0x06. > > And initiator re-submits, target re-rejects, and this repeats > > this is not good for internet, as it relates to congestion. > > When tagged queueing/no of outstanding commands support added > > to SCSI, iSCSI was not in anybody's mind(IMHO).. please > > dis-agree with me here. >=20 > Happy to do so, as that last statement is incorrect, because > the initial assumption is wrong. Tagged command queuing is in > SAM-2, which is normatively referenced by iSCSI (RFC 3720). The > mechanism that you're looking for is TASK SET FULL status (28h), > which is widely supported. The recent discussion on this list > was specific to iSCSI commands, which are intended for exceptional "iSCSI commands" --> "immediate iSCSI commands" > cases where the outstanding queue of undelivered commands > in iSCSI needs to be bypassed (see Section 3.2.2.1 of RFC 3720). >=20 > Thanks, > --David > ---------------------------------------------------- > David L. Black, Distinguished Engineer > EMC Corporation, 176 South St., Hopkinton, MA 01748 > +1 (508) 293-7953 FAX: +1 (508) 293-7786 > black_david@emc.com Mobile: +1 (978) 394-7754 > ---------------------------------------------------- >=20 _______________________________________________ Ips mailing list Ips@ietf.org https://www1.ietf.org/mailman/listinfo/ips From ips-bounces@ietf.org Thu Dec 06 10:23:46 2007 Return-path: Received: from [127.0.0.1] (helo=stiedprmman1.va.neustar.com) by megatron.ietf.org with esmtp (Exim 4.43) id 1J0IZl-0008Uf-Mz; Thu, 06 Dec 2007 10:23:45 -0500 Received: from ips by megatron.ietf.org with local (Exim 4.43) id 1J0IZk-0008UW-Fw for ips-confirm+ok@megatron.ietf.org; Thu, 06 Dec 2007 10:23:44 -0500 Received: from [10.91.34.44] (helo=ietf-mx.ietf.org) by megatron.ietf.org with esmtp (Exim 4.43) id 1J0IZk-0008UN-6K for ips@ietf.org; Thu, 06 Dec 2007 10:23:44 -0500 Received: from noteserv1.attotech.com ([208.69.85.41]) by ietf-mx.ietf.org with esmtp (Exim 4.43) id 1J0IZf-0006QB-9c for ips@ietf.org; Thu, 06 Dec 2007 10:23:44 -0500 To: ips@ietf.org Subject: Re: [Ips] Immediate Commands in iSCSI MIME-Version: 1.0 X-Mailer: Lotus Notes Release 7.0.2 September 26, 2006 Message-ID: From: dcuddihy@attotech.com Date: Thu, 6 Dec 2007 10:23:38 -0500 X-MIMETrack: Serialize by Router on NOTESERV1/SERV/ATTO(Release 7.0.2FP1|January 10, 2007) at 12/06/2007 10:23:39 AM, Serialize complete at 12/06/2007 10:23:39 AM X-Spam-Score: 0.0 (/) X-Scan-Signature: 88b11fc64c1bfdb4425294ef5374ca07 X-BeenThere: ips@ietf.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: IP Storage List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Content-Type: multipart/mixed; boundary="===============0269587727==" Errors-To: ips-bounces@ietf.org This is a multipart message in MIME format. --===============0269587727== Content-Type: multipart/alternative; boundary="=_alternative 00549008852573A9_=" This is a multipart message in MIME format. --=_alternative 00549008852573A9_= Content-Type: text/plain; charset="US-ASCII" Jijo, There is an ISCSI level flow control from the target back to the initiator via the MaxCmdSn field. The field can be found in the SCSI response, and it stops initiators from sending more than (MaxCmdSn-ExpCmdSn+1) commands on the session at any one time. See RFC 3720 10.4 (specifically 10.4.11) for more information. regards, david Java: The elegant simplicity of C++ and the blazing speed of smalltalk. David J Cuddihy Principal Engineer ATTO Technology, Inc. www.attotech.com Power Behind the Storage "Jijo Chacko" 12/06/2007 04:12 AM To "Ken Sandars" cc ips@ietf.org, jacob_cherian@dell.com, nam_nguyen@dell.com, julian_satran@il.ibm.com Subject Re: [Ips] Immediate Commands in iSCSI A related question - from the above discussions, i assume that there is no, application-level, end-to-end flow control from the initiator-to-target, as it relates to queued commands. As TCP flow control layers already allowed the command-bytes in to the host, we have the memory buffers at the TCP layer, and later commands are just ignored, with ICR return code 0x06. And initiator re-submits, target re-rejects, and this repeats this is not good for internet, as it relates to congestion. When tagged queueing/no of outstanding commands support added to SCSI, iSCSI was not in anybody's mind(IMHO).. please dis-agree with me here. And strongly feel that, we need to have some way of throttling the initiator to prevent from swamping the target, or target implementation code should have the logic to re-submit the un-honored commands from the TCP buffers(or local buffers), before returning an error code to the initiator. or there can be an iSCSI, pending-coomands cache at the target stack, to solve this problem. In essence, IMHO, we should do some fixes for target iSCSI implementation, to provide some staging buffers for already tcp-received commands. We should not be returning wrong error code to the initiator, if we already received the command at the TCP layer, and the target has the ability to apply it. We should be returning the error code only if target failed to apply it. IN the SCSI world, as the number of initiators are typically just 1, and the data is not travelling across WAN, target can return "Too many commands pending" errors.. as it has only local effect, having few meters. Please correct me, if i am wrong... thanks, Jijo Chacko Bangalore. _______________________________________________ Ips mailing list Ips@ietf.org https://www1.ietf.org/mailman/listinfo/ips _______________________________________________ Ips mailing list Ips@ietf.org https://www1.ietf.org/mailman/listinfo/ips --=_alternative 00549008852573A9_= Content-Type: text/html; charset="US-ASCII"
Jijo,

There is an ISCSI level flow control from the target back to the initiator via the MaxCmdSn field.  The field can be found in the SCSI response, and it stops initiators from sending more than (MaxCmdSn-ExpCmdSn+1) commands on the session at any one time.  See RFC 3720 10.4 (specifically 10.4.11) for more information.

regards,

david


Java: The elegant simplicity of C++
and the blazing speed of smalltalk.

David J Cuddihy
Principal Engineer
ATTO Technology, Inc.

www.attotech.com
Power Behind the Storage



"Jijo Chacko" <consultant.bangalore@gmail.com>

12/06/2007 04:12 AM

To
"Ken Sandars" <kensandars@hotmail.com>
cc
ips@ietf.org, jacob_cherian@dell.com, nam_nguyen@dell.com, julian_satran@il.ibm.com
Subject
Re: [Ips] Immediate Commands in iSCSI






A related question - from the above discussions, i assume that there is no, application-level, end-to-end flow control from the initiator-to-target, as it relates to queued commands. As TCP flow control layers already allowed the command-bytes in to the host,  we have the memory buffers at the TCP layer, and later commands are just ignored, with ICR return code 0x06. And initiator re-submits, target re-rejects,  and this repeats this is not good for internet, as it relates to congestion. When tagged queueing/no of outstanding commands support added to SCSI, iSCSI was not in anybody's mind(IMHO).. please dis-agree with me here.

And strongly feel that, we need to have some way of throttling the initiator to prevent from swamping the target, or  target implementation code should have the logic to re-submit the un-honored commands from the TCP buffers(or local buffers), before returning an error code to the initiator. or there can be an iSCSI, pending-coomands cache at the target stack, to solve this problem.

In essence, IMHO, we should do some fixes for target iSCSI implementation, to provide some staging buffers for already tcp-received commands. We should not be returning wrong error code to the initiator, if we already received the command at the TCP layer, and the target has the ability to apply it. We should be returning the error code only if target failed to apply it.

IN the SCSI world, as the number of initiators are typically just 1, and the data is not travelling across WAN, target can  return "Too many commands pending" errors.. as it has only local effect, having few meters.

Please correct me, if i am wrong...

thanks,
Jijo Chacko
Bangalore.


_______________________________________________
Ips mailing list
Ips@ietf.org
https://www1.ietf.org/mailman/listinfo/ips

_______________________________________________
Ips mailing list
Ips@ietf.org
https://www1.ietf.org/mailman/listinfo/ips
--=_alternative 00549008852573A9_=-- --===============0269587727== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline _______________________________________________ Ips mailing list Ips@ietf.org https://www1.ietf.org/mailman/listinfo/ips --===============0269587727==-- From ips-bounces@ietf.org Thu Dec 06 16:41:13 2007 Return-path: Received: from [127.0.0.1] (helo=stiedprmman1.va.neustar.com) by megatron.ietf.org with esmtp (Exim 4.43) id 1J0OT1-0001If-EN; Thu, 06 Dec 2007 16:41:11 -0500 Received: from ips by megatron.ietf.org with local (Exim 4.43) id 1J0OSz-0001IU-VJ for ips-confirm+ok@megatron.ietf.org; Thu, 06 Dec 2007 16:41:09 -0500 Received: from [10.90.34.44] (helo=chiedprmail1.ietf.org) by megatron.ietf.org with esmtp (Exim 4.43) id 1J0OSz-0001IM-Is for ips@ietf.org; Thu, 06 Dec 2007 16:41:09 -0500 Received: from ausc60pc101.us.dell.com ([143.166.85.206]) by chiedprmail1.ietf.org with esmtp (Exim 4.43) id 1J0OSy-0002I6-FB for ips@ietf.org; Thu, 06 Dec 2007 16:41:09 -0500 X-IronPort-AV: E=Sophos;i="4.23,263,1194242400"; d="bmp'146?scan'146,208,217,146";a="464384423" X-MimeOLE: Produced By Microsoft Exchange V6.5 Content-class: urn:content-classes:message MIME-Version: 1.0 Date: Thu, 6 Dec 2007 15:39:34 -0600 Message-ID: <5DDAB7BA7BDB58439DD0EED0B8E9A3AEFDA9A7@ausx3mpc102.aus.amer.dell.com> X-MS-Has-Attach: yes X-MS-TNEF-Correlator: Thread-Topic: Immediate Commands in iSCSI Thread-Index: Acg4UH34xoJ6UOSgQgWUf56OCn7/oQ== From: To: X-OriginalArrivalTime: 06 Dec 2007 21:41:06.0955 (UTC) FILETIME=[B5413DB0:01C83850] X-Spam-Score: 0.0 (/) X-Scan-Signature: 2e73e7ad830b2f0a95c4eabd630e061d Subject: [Ips] Immediate Commands in iSCSI X-BeenThere: ips@ietf.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: IP Storage List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Content-Type: multipart/mixed; boundary="===============2090863650==" Errors-To: ips-bounces@ietf.org This is a multi-part message in MIME format. --===============2090863650== Content-class: urn:content-classes:message Content-Type: multipart/related; type="multipart/alternative"; boundary="----_=_NextPart_001_01C83850.9F5907DA" This is a multi-part message in MIME format. ------_=_NextPart_001_01C83850.9F5907DA Content-Type: multipart/alternative; boundary="----_=_NextPart_002_01C83850.9F5907DA" ------_=_NextPart_002_01C83850.9F5907DA Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable Ken/Julo, We may have been talking past each other here :-) I agree that defining a status code for the Response field of the TMF Response PDU is the simplest option: 0x07 - Immediate Command Reject - Too many immediate commands 0x06 is already defined as "Function authorization failed." Or were you recommending changing the description for the 0x06 response code? Thanks, Jacob RE: Immediate Commands in iSCSI =20 <> <> <> <>=20 by Ken Sandars = Dec 04, 2007; 07:08pm :: Reply | Reply = to Author = | Show Only this Message Hi Jacob, A Reject PDU which indicates its reason as "Immediate Command Reject - too many immediate commands" seems pretty granular to me. :-) That's what the reject code 0x06 means (as Julo had recommended). The initiator may resubmit the PDU whenever it deems appropriate. Does that make sense? IMHO, a target implementation constrains the number of non-TMF commands and might throttle repeated TMF commands when they are in progress (and the clearing effects are taking a long time to resolve). Even with a large number of LU's it would be unusual to constrain the number of concurrent LU Reset operations. Have you got a trace showing this behaviour? HTH, Ken Subject: RE: [Ips] Immediate Commands in iSCSI Date: Mon, 3 Dec 2007 14:15:19 -0600 From: Jacob_Cherian@... =20 To: Julian_Satran@... =20 CC: ips@... ; Nam_Nguyen@... =20 Julian, No offense taken. The scenario that you brought up was considered since most configurations with iSCSI are with multiple hosts. However, the allocation of immediate commands to connection/sessions/hosts depends on the actual implementation. RFC 3720 already requires support for at least one per connection. Even we cannot agree on a key, having a more granular status for a rejected TMF that says the target is currently busy processing a TMF and cannot process another at this time (but will be able to sometime in the future), helps initiators to wait and retry, instead of assuming a failure and escalating the reset.=20 =20 Jacob =20 From: Julian Satran [mailto:Julian_Satran@... ] Sent: Monday, December 03, 2007 8:42 AM To: Cherian, Jacob Cc: ips@... ; Nguyen, Nam Subject: RE: [Ips] Immediate Commands in iSCSI =20 Jacob,=20 I do not want to appear offensive - but did you consider the "larger picture"? Assuming that the negotiation key or status is there and assuming that the target talks to 7 (seven) different initiators what would it tell each of them (it has 7 different sessions)? Should he reserve a quota for each of them - so that it can meet what it declared? What should a target do with commands that exceed the number?=20 In iSCSI as a rule we avoided setting or negotiating parameters that we don't need to set/negotiate and I think the number of accepted immediate commands is one of them.=20 Julo=20 RE: [Ips] Immediate Commands in iSCSI =20 Jacob_Cherian=20 to:=20 Julian Satran=20 12/03/07 04:06 PM =20 Cc:=20 ips, Nam_Nguyen =09 =20 The goal of the new status or key is to help the initiator to know the correct number of immediate commands that the target supports. The issue with overloading a status that does clearly define the error condition is that it does help in achieving interoperability between the myriad of initiators and targets that are out there. I was looking to see if there was another way that I had not considered.=20 =20 Jacob=20 =20 =20 From: Julian Satran [mailto:Julian_Satran@... ] Sent: Monday, December 03, 2007 1:23 AM To: Cherian, Jacob Cc: ips@... ; Nguyen, Nam Subject: Re: [Ips] Immediate Commands in iSCSI=20 =20 It looks to me that the Reject response with reason 0x06 should be enough for the initiator to handle the situation you describe.=20 If the TM are the cause then each of those has a response indicating termination. After receiving a reject an initiator should reissue new TMs only after getting back one or more responses. The initiator may even accurately guess the number of commands the targets is capable of handling per connection and per LU. The only case in which this handling may be awkward is over a WAN but if you are operating over a WAN and TM performance is your only worry you are in good shape :-)=20 Julo=20 =20 [Ips] Immediate Commands in iSCSI =20 =20 Jacob_Cherian=20 to:=20 ips=20 12/03/07 05:23 AM =20 =20 Cc:=20 Nam_Nguyen =09 =20 =20 RFC 3720 states:=20 The number of commands used for immediate delivery is not limited and their delivery for execution is not acknowledged through the numbering scheme. Immediate commands MAY be rejected by the iSCSI target layer due to a lack of resources. An iSCSI target MUST be able to handle at least one immediate task management command and one immediate non-task-management iSCSI command per connection at any time.=20 =20 With storage systems that have very large number of logical units, we run into an issue when we use LU Reset or Abort Task for reset recovery. I couldn't find a way to tell an initiator how many immediate commands a target can accept. The issue with this is that if we have to issue multiple task management commands and it is more than what the target can accept, it will reject the task management request. Since we don't know why it was rejected, we will have to escalate the reset. This can be very disruptive with nodes that have very large number of logical units. There may be two ways to resolve this:=20 =20 1) Define a new Text negotiation key for number of outstanding immediate commands supported. Unlike non-immediate (data) commands where we can send TASK SET FULL, there is no way for a target to report in the Task Management Function Response PDU that is unable to process the current request before it is out of resources. The default vaule for this new key can be 1, so if this key is not negotiated initiators can assume one per connection.=20 2) Define a new value for the Response field in the Task Management Function Response PDU for reporting that the target cannot process the request currently.=20 =20 Any suggestions on how to resolve this?=20 =20 Thanks,=20 Jacob=20 ------_=_NextPart_002_01C83850.9F5907DA Content-Type: text/html; charset="us-ascii" Content-Transfer-Encoding: quoted-printable Immediate Commands in iSCSI

Ken/Julo,

  We may have been talking past each other = here J  I agree that defining a status code for the Response field of the = TMF Response PDU is the simplest option:

0x07 - Immediate Command Reject - Too many immediate = commands

0x06 is already defined as “Function = authorization failed.” Or were you recommending changing the = description for the 0x06 response code?

Thanks,

Jacob

RE: = Immediate Commands in iSCSI 3D"Picture 3D"Picture 3D"Picture 3D"Picture

by = Ken = Sandars Dec 04, 2007; 07:08pm ::

Reply | Reply to = Author | Show Only this = Message

Hi Jacob,

A Reject PDU which indicates its reason as "Immediate Command = Reject - too many immediate commands" seems pretty granular to me. = :-) That's what the reject code 0x06 means (as Julo had recommended). = The initiator may resubmit the PDU whenever it deems appropriate. Does = that make sense?

IMHO, a target implementation constrains the number of non-TMF commands = and might throttle repeated TMF commands when they are in progress (and = the clearing effects are taking a long time to resolve). Even with a = large number of LU's it would be unusual to constrain the number of = concurrent LU Reset operations. Have you got a trace showing this = behaviour?


HTH,
Ken

Subject: RE: [Ips] Immediate Commands in iSCSI
Date: Mon, 3 Dec 2007 14:15:19 -0600
From: Jacob_Cherian@...
To: Julian_Satran@...
CC: ips@...; Nam_Nguyen@...

Julian,

    No offense taken. = The scenario that you brought up was considered since most = configurations with iSCSI are with multiple hosts. However, the = allocation of immediate commands to connection/sessions/hosts depends on = the actual implementation. RFC 3720 already requires support for at = least one per connection. Even we cannot agree on a key, having a more = granular status for a rejected TMF that says the target is currently = busy processing a TMF and cannot process another at this time (but will = be able to sometime in the future), helps initiators to wait and retry, = instead of assuming a failure and escalating the = reset.

 

 Jacob

 

From: Julian Satran = [mailto:Julian_Satran@...]
Sent: Monday, December 03, 2007 = 8:42 AM
To: Cherian, Jacob
Cc: ips@...; Nguyen, Nam
Subject: RE: [Ips] Immediate Commands in = iSCSI

 


Jacob,

I do not want = to appear offensive - but did you consider the "larger = picture"? Assuming that the negotiation key or status is there and = assuming that the target talks to 7 (seven) different initiators what = would it tell each of them (it has 7 different sessions)? Should he = reserve a quota for each of them - so that it can meet what it declared? = What should a target do with commands that exceed the = number?

In iSCSI as a = rule we avoided setting or negotiating parameters that we don't need to = set/negotiate and I think the number of accepted immediate commands is = one of them.

Julo

        RE: [Ips] Immediate Commands in = iSCSI

 

Jacob_Cherian =

to:

Julian Satran =

12/03/07 04:06 = PM

 

Cc:

ips, = Nam_Nguyen

       
 




The goal of the new status or key is to help the = initiator to know the correct number of immediate commands that the = target supports. The issue with overloading a status that does clearly = define the error condition is that it does help in achieving = interoperability between the myriad of initiators and targets that are = out there. I was looking to see if there was another way that I had not = considered.
 
Jacob
 
 
From: Julian Satran = [mailto:Julian_Satran@...]
Sent: Monday, December 03, 2007 1:23 AM
To: Cherian, Jacob
Cc: ips@...; Nguyen, = Nam
Subject: Re: [Ips] Immediate Commands in = iSCSI
 

It looks to  me that the Reject = response with reason 0x06 should be enough for the initiator to handle = the situation you describe.
If the TM are the cause then each of those = has a response indicating termination. After receiving a reject an = initiator should reissue new TMs only after getting back one or more = responses.  The initiator may even accurately guess the number of = commands the targets is capable of handling per connection and per LU. = The only case in which this handling may be awkward is over a WAN but if = you are operating over a WAN and TM performance is your only worry you = are in good shape :-)

Julo =

        =  

[Ips] Immediate Commands = in iSCSI


 

 

Jacob_Cherian =

to:

ips

12/03/07 05:23 = AM


 
 

Cc: =

Nam_Nguyen

       

 

 






RFC 3720 states:

The number of commands used for = immediate delivery is not limited and their delivery for execution is = not acknowledged through the numbering scheme. Immediate commands MAY be = rejected by the iSCSI target layer due to a lack of resources. An iSCSI = target MUST be able to handle at least one immediate task management = command and one immediate non-task-management iSCSI command per = connection at any time.

 

With storage systems that have very = large number of logical units, we run into an issue when we use LU Reset = or Abort Task for reset recovery. I couldn’t find a way to tell an = initiator how many immediate commands a target can accept. The issue = with this is that if we have to issue multiple task management commands = and it is more than what the target can accept, it will reject the task = management request. Since we don’t know why it was rejected, we = will have to escalate the reset. This can be very disruptive with nodes = that have very large number of logical units. There may be two ways to = resolve this:

 

1)      Define a new = Text negotiation key for number of outstanding immediate commands = supported. Unlike non-immediate (data) commands where we can send TASK = SET FULL, there is no way for a target to report in the Task Management = Function Response PDU that is unable to process the current request = before it is out of resources.  The default vaule for this new key = can be 1, so if this key is not negotiated initiators can assume one per = connection.

2)      Define a new = value for the Response field in the Task Management Function Response = PDU for reporting that the target cannot process the request = currently.

 

Any suggestions on how to resolve = this?

 

Thanks,

Jacob

------_=_NextPart_002_01C83850.9F5907DA-- ------_=_NextPart_001_01C83850.9F5907DA Content-Type: image/bmp; name="ole0.bmp" Content-Transfer-Encoding: base64 Content-Description: Picture (Device Independent Bitmap) Content-Location: No%20AttachName Qk3aAQAAAAAAAOoAAAAoAAAADwAAAA8AAAABAAgAAAAAAAAAAAAAAAAAAAAAAC0AAAAAAAAA3d3d AJycnADi4uIArKysANfX1wDMzMwAsbGxAHt7ewDc3NwAwcHBANHR0QCbm5sAkJCQAD8/PwDW1tYA oKCgAE9PTwBvb28A4ODgAMXFxQDV1dUAurq6AImJiQCOjo4Ac3NzAFhYWABoaGgATU1NAP///wDJ yckA2dnZAKOjowCYmJgAR0dHAN7e3gDDw8MAcnJyADw8PADT09MAgoKCAMjIyACtra0AkpKSANjY 2ACioqIAHBwcHBwfGBwcHBwcHBwcABwcHBwcBRwcHBwcHBwcHAAcHBwcHAIBHBwcHBwcHBwAHBwc HBwCHxwcHBwcHBwcABwcHBwcCAEcHBwcHBwcHAAcHBwcHAgqHBwcHBwcHBwAHBwcHBwUFxwcHBwc HBwcABwcHBwcFCobHBwcHBwcHAAcHBwcHBQDERslJRwcHBwAHBwcHBwUBR8nERshHBwcABwcHBwc FAUTBgEYGRwcHAAcHBwcHBQUBRMVFxEcHBwAHBwcHBwICAgUBQMHHBwcABwcHBwcHBwCCBQTHxwc HAAcHBwcHBwcHAgcFBMcHBwA ------_=_NextPart_001_01C83850.9F5907DA Content-Type: image/bmp; name="ole1.bmp" Content-Transfer-Encoding: base64 Content-Description: Picture (Device Independent Bitmap) Content-Location: No%20AttachName-1 Qk2uAQAAAAAAAB4BAAAoAAAADAAAAAwAAAABAAgAAAAAAAAAAAAAAAAAAAAAADoAAAAAAAAAr4FR AMSPWQDfuJMAt41hAMKkhQC3kWoArH5PAODBoQDEpoYAyKmKAOXHqQC+mXMAqX5RAMKKUQCshFkA roRZAOTLsgDMrpEAqX1QAK2DWAC5kGQA2Kt/ANmrfwCneUkAqn9RALOLYQDFp4gA0qBtANOgbQCx hFYA37iUAM6WXgCtfUsA5ceoAMuPUwDx4tMAqn5QALiUbwDiyK0A2Kx/AKh6SQCpekkAtYFNAMWM UQD///8ArHxKAOzWvgC/mHEA37mUAM6XXgCpfU8AsoBMAMirjgDNoHMAuIlYAMuUXADmz7UAtoJN ACwsLCwsLCwsLCwsLCwsGQwLLCwEDgQsLCwsLBQLDAw0GjQsLCwsNh0wIRAjJSwsLCwsLBMWMAou BSwsLCwsIDccFjAKEA4RLCwgKyI3HBYwCiYOLDkgFyArNxw1DCAZLCwsLCwsKwEgLCwsLCwsLCw5 ICAsLCwsLCwsLCwsICwsLCwsLCwsLCwsLDksLCwsLA== ------_=_NextPart_001_01C83850.9F5907DA Content-Type: image/bmp; name="ole2.bmp" Content-Transfer-Encoding: base64 Content-Description: Picture (Device Independent Bitmap) Content-Location: No%20AttachName-2 Qk2uAQAAAAAAAB4BAAAoAAAADAAAAAwAAAABAAgAAAAAAAAAAAAAAAAAAAAAADoAAAAAAAAAr4FR AMSPWQDfuJMAt41hAMKkhQC3kWoArH5PAODBoQDEpoYAyKmKAOXHqQC+mXMAqX5RAMKKUQCshFkA roRZAOTLsgDMrpEAqX1QAK2DWAC5kGQA2Kt/ANmrfwCneUkAqn9RALOLYQDFp4gA0qBtANOgbQCx hFYA37iUAM6WXgCtfUsA5ceoAMuPUwDx4tMAqn5QALiUbwDiyK0A2Kx/AKh6SQCpekkAtYFNAMWM UQD///8ArHxKAOzWvgC/mHEA37mUAM6XXgCpfU8AsoBMAMirjgDNoHMAuIlYAMuUXADmz7UAtoJN ACwsLCwsLCwsLCwsLCwsGQwLLCwEDgQsLCwsLBQLDAw0GjQsLCwsNh0wIRAjJSwsLCwsLBMWMAou BSwsLCwsIDccFjAKEA4RLCwgKyI3HBYwCiYOLDkgFyArNxw1DCAZLCwsLCwsKwEgLCwsLCwsLCw5 ICAsLCwsLCwsLCwsICwsLCwsLCwsLCwsLDksLCwsLA== ------_=_NextPart_001_01C83850.9F5907DA Content-Type: image/bmp; name="ole3.bmp" Content-Transfer-Encoding: base64 Content-Description: Picture (Device Independent Bitmap) Content-Location: No%20AttachName-3 Qk2uAQAAAAAAAB4BAAAoAAAADAAAAAwAAAABAAgAAAAAAAAAAAAAAAAAAAAAADoAAAAAAAAAr4FR AMSPWQDfuJMAt41hAMKkhQC3kWoArH5PAODBoQDEpoYAyKmKAOXHqQC+mXMAqX5RAMKKUQCshFkA roRZAOTLsgDMrpEAqX1QAK2DWAC5kGQA2Kt/ANmrfwCneUkAqn9RALOLYQDFp4gA0qBtANOgbQCx hFYA37iUAM6WXgCtfUsA5ceoAMuPUwDx4tMAqn5QALiUbwDiyK0A2Kx/AKh6SQCpekkAtYFNAMWM UQD///8ArHxKAOzWvgC/mHEA37mUAM6XXgCpfU8AsoBMAMirjgDNoHMAuIlYAMuUXADmz7UAtoJN ACwsLCwsLCwsLCwsLCwsGQwLLCwEDgQsLCwsLBQLDAw0GjQsLCwsNh0wIRAjJSwsLCwsLBMWMAou BSwsLCwsIDccFjAKEA4RLCwgKyI3HBYwCiYOLDkgFyArNxw1DCAZLCwsLCwsKwEgLCwsLCwsLCw5 ICAsLCwsLCwsLCwsICwsLCwsLCwsLCwsLDksLCwsLA== ------_=_NextPart_001_01C83850.9F5907DA-- --===============2090863650== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline _______________________________________________ Ips mailing list Ips@ietf.org https://www1.ietf.org/mailman/listinfo/ips --===============2090863650==-- From ips-bounces@ietf.org Thu Dec 06 16:54:46 2007 Return-path: Received: from [127.0.0.1] (helo=stiedprmman1.va.neustar.com) by megatron.ietf.org with esmtp (Exim 4.43) id 1J0Og9-0002DV-8R; Thu, 06 Dec 2007 16:54:45 -0500 Received: from ips by megatron.ietf.org with local (Exim 4.43) id 1J0Og7-0002A3-PA for ips-confirm+ok@megatron.ietf.org; Thu, 06 Dec 2007 16:54:43 -0500 Received: from [10.91.34.44] (helo=ietf-mx.ietf.org) by megatron.ietf.org with esmtp (Exim 4.43) id 1J0Og7-00029h-Eu for ips@ietf.org; Thu, 06 Dec 2007 16:54:43 -0500 Received: from mail0.lsil.com ([147.145.40.20]) by ietf-mx.ietf.org with esmtp (Exim 4.43) id 1J0Og4-00056v-7O for ips@ietf.org; Thu, 06 Dec 2007 16:54:43 -0500 Received: from milmhbs0.lsil.com (mhbs.lsil.com [147.145.1.30]) by mail0.lsil.com (8.12.11/8.12.11) with ESMTP id lB6Lq9Zr014608; Thu, 6 Dec 2007 13:52:09 -0800 (PST) Received: from ict-mailhub.ks.lsil.com (naict042u.ks.lsil.com [172.22.12.2]) by milmhbs0.lsil.com (8.12.11/8.12.11) with ESMTP id lB6LsYxX031351; Thu, 6 Dec 2007 13:54:35 -0800 Received: from aspry-powerbook.ks.lsil.com (aspry-powerbook.ks.lsil.com [172.22.32.7]) by ict-mailhub.ks.lsil.com (8.12.11/8.12.11) with ESMTP id lB6LsWA2001652; Thu, 6 Dec 2007 15:54:33 -0600 (CST) Message-Id: <9951E3B5-88B6-420B-8645-A2FD6CC2B158@lsi.com> From: Andy Spry To: ips@ietf.org In-Reply-To: <5DDAB7BA7BDB58439DD0EED0B8E9A3AEFDA9A7@ausx3mpc102.aus.amer.dell.com> Subject: Re: [Ips] Immediate Commands in iSCSI Mime-Version: 1.0 (Apple Message framework v915) Date: Thu, 6 Dec 2007 15:54:36 -0600 References: <5DDAB7BA7BDB58439DD0EED0B8E9A3AEFDA9A7@ausx3mpc102.aus.amer.dell.com> X-Mailer: Apple Mail (2.915) X-Scanned-By: MIMEDefang 2.39 X-Spam-Score: 0.2 (/) X-Scan-Signature: b92e72fc2b623ddd11e6d81413fb81b2 X-BeenThere: ips@ietf.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: IP Storage List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Content-Type: multipart/mixed; boundary="===============1738068149==" Errors-To: ips-bounces@ietf.org --===============1738068149== Content-Type: multipart/alternative; boundary=Apple-Mail-98--372455595 --Apple-Mail-98--372455595 Content-Type: text/plain; charset=WINDOWS-1252; format=flowed; delsp=yes Content-Transfer-Encoding: quoted-printable Jacob, I think that 0x06 code refers to the Reason code for an iSCSI Reject =20 PDU. The sequence would be I-> Sends TMF Request PDU as immediate command, exceeding the =20 capacity of the target for immediate commands. T-> Responds with a Reject PDU with Reason code 0x06 (too many =20 immediate commands). The TMF Request is blocked at the door and never makes it to the TMF =20 Request handler to generate a TMF Response. The payload of the Reject PDU is the header of the TMF Request PDU, so =20= the entire TMF request is returned to the initiator where it could be =20= re-issued. The hard part is determining when the load on the target =20 has been reduced so the command can be resent. Dr. Andrew J. Spry Storage Architect LSI Corporation andy.spry@lsi.com On Dec 6, 2007, at 3:39 PM, wrote: > Ken/Julo, > > We may have been talking past each other here J I agree that =20 > defining a status code for the Response field of the TMF Response =20 > PDU is the simplest option: > > > 0x07 - Immediate Command Reject - Too many immediate commands > > 0x06 is already defined as =93Function authorization failed.=94 Or = were =20 > you recommending changing the description for the 0x06 response code? > > Thanks, > > Jacob > > > RE: Immediate Commands in iSCSI > > by Ken Sandars Dec 04, 2007; 07:08pm :: > > Reply | Reply to Author | Show Only this Message > > Hi Jacob, > > A Reject PDU which indicates its reason as "Immediate Command Reject =20= > - too many immediate commands" seems pretty granular to me. :-) =20 > That's what the reject code 0x06 means (as Julo had recommended). =20 > The initiator may resubmit the PDU whenever it deems appropriate. =20 > Does that make sense? > > IMHO, a target implementation constrains the number of non-TMF =20 > commands and might throttle repeated TMF commands when they are in =20 > progress (and the clearing effects are taking a long time to =20 > resolve). Even with a large number of LU's it would be unusual to =20 > constrain the number of concurrent LU Reset operations. Have you got =20= > a trace showing this behaviour? > > > HTH, > Ken > > > Subject: RE: [Ips] Immediate Commands in iSCSI > Date: Mon, 3 Dec 2007 14:15:19 -0600 > From: Jacob_Cherian@... > To: Julian_Satran@... > CC: ips@...; Nam_Nguyen@... > > Julian, > > No offense taken. The scenario that you brought up was =20 > considered since most configurations with iSCSI are with multiple =20 > hosts. However, the allocation of immediate commands to connection/=20 > sessions/hosts depends on the actual implementation. RFC 3720 =20 > already requires support for at least one per connection. Even we =20 > cannot agree on a key, having a more granular status for a rejected =20= > TMF that says the target is currently busy processing a TMF and =20 > cannot process another at this time (but will be able to sometime in =20= > the future), helps initiators to wait and retry, instead of assuming =20= > a failure and escalating the reset. > > > Jacob > > > From: Julian Satran [mailto:Julian_Satran@...] > Sent: Monday, December 03, 2007 8:42 AM > To: Cherian, Jacob > Cc: ips@...; Nguyen, Nam > Subject: RE: [Ips] Immediate Commands in iSCSI > > > > Jacob, > > I do not want to appear offensive - but did you consider the "larger =20= > picture"? Assuming that the negotiation key or status is there and =20 > assuming that the target talks to 7 (seven) different initiators =20 > what would it tell each of them (it has 7 different sessions)? =20 > Should he reserve a quota for each of them - so that it can meet =20 > what it declared? What should a target do with commands that exceed =20= > the number? > > In iSCSI as a rule we avoided setting or negotiating parameters that =20= > we don't need to set/negotiate and I think the number of accepted =20 > immediate commands is one of them. > > Julo > > RE: [Ips] Immediate Commands in iSCSI > > > > Jacob_Cherian > > to: > > Julian Satran > > 12/03/07 04:06 PM > > > > Cc: > > ips, Nam_Nguyen > > > > > > > > > > > > The goal of the new status or key is to help the initiator to know =20 > the correct number of immediate commands that the target supports. =20 > The issue with overloading a status that does clearly define the =20 > error condition is that it does help in achieving interoperability =20 > between the myriad of initiators and targets that are out there. I =20 > was looking to see if there was another way that I had not considered. > > Jacob > > > From: Julian Satran [mailto:Julian_Satran@...] > Sent: Monday, December 03, 2007 1:23 AM > To: Cherian, Jacob > Cc: ips@...; Nguyen, Nam > Subject: Re: [Ips] Immediate Commands in iSCSI > > > It looks to me that the Reject response with reason 0x06 should be =20= > enough for the initiator to handle the situation you describe. > If the TM are the cause then each of those has a response indicating =20= > termination. After receiving a reject an initiator should reissue =20 > new TMs only after getting back one or more responses. The =20 > initiator may even accurately guess the number of commands the =20 > targets is capable of handling per connection and per LU. The only =20 > case in which this handling may be awkward is over a WAN but if you =20= > are operating over a WAN and TM performance is your only worry you =20 > are in good shape :-) > > Julo > > > [Ips] Immediate Commands in iSCSI > > > > > > > Jacob_Cherian > > to: > > ips > > 12/03/07 05:23 AM > > > > > > > Cc: > > Nam_Nguyen > > > > > > > > > > > > > > > > RFC 3720 states: > > The number of commands used for immediate delivery is not limited =20 > and their delivery for execution is not acknowledged through the =20 > numbering scheme. Immediate commands MAY be rejected by the iSCSI =20 > target layer due to a lack of resources. An iSCSI target MUST be =20 > able to handle at least one immediate task management command and =20 > one immediate non-task-management iSCSI command per connection at =20 > any time. > > > > With storage systems that have very large number of logical units, =20 > we run into an issue when we use LU Reset or Abort Task for reset =20 > recovery. I couldn=92t find a way to tell an initiator how many =20 > immediate commands a target can accept. The issue with this is that =20= > if we have to issue multiple task management commands and it is more =20= > than what the target can accept, it will reject the task management =20= > request. Since we don=92t know why it was rejected, we will have to =20= > escalate the reset. This can be very disruptive with nodes that have =20= > very large number of logical units. There may be two ways to resolve =20= > this: > > > > 1) Define a new Text negotiation key for number of outstanding =20= > immediate commands supported. Unlike non-immediate (data) commands =20 > where we can send TASK SET FULL, there is no way for a target to =20 > report in the Task Management Function Response PDU that is unable =20 > to process the current request before it is out of resources. The =20 > default vaule for this new key can be 1, so if this key is not =20 > negotiated initiators can assume one per connection. > > 2) Define a new value for the Response field in the Task =20 > Management Function Response PDU for reporting that the target =20 > cannot process the request currently. > > > > Any suggestions on how to resolve this? > > > > Thanks, > > Jacob > > > _______________________________________________ > Ips mailing list > Ips@ietf.org > https://www1.ietf.org/mailman/listinfo/ips --Apple-Mail-98--372455595 Content-Type: text/html; charset=WINDOWS-1252 Content-Transfer-Encoding: quoted-printable Jacob,

I think that 0x06 code = refers to the Reason code for an iSCSI Reject PDU.  

The sequence would = be 

I->  Sends TMF = Request PDU as immediate command, exceeding the capacity of the target = for immediate commands.

T-> Responds with a = Reject PDU with Reason code 0x06 (too many immediate commands). =  

The = TMF Request is blocked at the door and never makes it to the TMF Request = handler to generate a TMF Response.  

The payload of the Reject = PDU is the header of the TMF Request PDU, so the entire TMF request is = returned to the initiator where it could be re-issued.  The hard = part is determining when the load on the target has been reduced so the = command can be resent.  

Dr. Andrew J. = Spry
Storage Architect
LSI = Corporation


 =

On Dec 6, 2007, at 3:39 PM, <Jacob_Cherian@Dell.com> = wrote:

Ken/Julo,

  We may have been talking past = each other here J  I = agree that defining a status code for the Response field of = the TMF Response PDU is the simplest option:


0x07 = - Immediate Command Reject - Too many immediate = commands

0x06 is already = defined as =93Function authorization = failed.=94 Or were you recommending changing the description = for the 0x06 response code?

Thanks,

Jacob

RE: Immediate Commands in = iSCSI 3D"Picture= 3D"Picture 3D"Picture 3D"Picture

by Ken = Sandars Dec 04, 2007; 07:08pm ::

Reply | Reply to = Author | Show Only this = Message

Hi Jacob,

A Reject PDU which indicates its reason as = "Immediate Command Reject - too many immediate commands" seems pretty = granular to me. :-) That's what the reject code 0x06 means (as Julo had = recommended). The initiator may resubmit the PDU whenever it deems = appropriate. Does that make sense?

IMHO, a target = implementation constrains the number of non-TMF commands and might = throttle repeated TMF commands when they are in progress (and the = clearing effects are taking a long time to resolve). Even with a large = number of LU's it would be unusual to constrain the number of concurrent = LU Reset operations. Have you got a trace showing this = behaviour?


= HTH,
Ken


Subject: RE: [Ips] Immediate Commands in iSCSI
Date: Mon, 3 Dec 2007 = 14:15:19 -0600
From: Jacob_Cherian@...
To: Julian_Satran@...
CC: ips@...; Nam_Nguyen@...

Julian,

    No offense taken. = The scenario that you brought up was considered since most = configurations with iSCSI are with multiple hosts. However, the = allocation of immediate commands to connection/sessions/hosts depends on = the actual implementation. RFC 3720 already requires support for at = least one per connection. Even we cannot agree on a key, having a more = granular status for a rejected TMF that says the target is currently = busy processing a TMF and cannot process another at this time (but will = be able to sometime in the future), helps initiators to wait and retry, = instead of assuming a failure and escalating the = reset.

 

 Jacob

 

From: Julian Satran = [mailto:Julian_Satran@...]
= Sent: Monday, December 03, = 2007 8:42 AM
To: Cherian, Jacob
Cc: ips@...; Nguyen, Nam
= Subject: RE: [Ips] Immediate Commands in = iSCSI

 


Jacob,

I do not want to appear offensive - but did you consider = the "larger picture"? Assuming that the negotiation key or status is = there and assuming that the target talks to 7 (seven) different = initiators what would it tell each of them (it has 7 different = sessions)? Should he reserve a quota for each of them - so that it can = meet what it declared? What should a target do with commands that exceed = the number?

In iSCSI as a rule we = avoided setting or negotiating parameters that we don't need to = set/negotiate and I think the number of accepted immediate commands is = one of them.

Julo

        RE: [Ips] Immediate Commands in = iSCSI


 

Jacob_Cherian =

to:

Julian Satran=

12/03/07 04:06 PM


 

Cc: =

ips, = Nam_Nguyen




       
 




=
The goal of the new status or key is to help the = initiator to know the correct number of immediate commands that the = target supports. The issue with overloading a status that does clearly = define the error condition is that it does help in achieving = interoperability between the myriad of initiators and targets that are = out there. I was looking to see if there was another way that I had not = considered.
 
= Jacob
=  
=  
= From: Julian Satran = [mailto:Julian_Satran@...]
Sent: Monday, December 03, = 2007 1:23 AM
To: Cherian, Jacob
Cc: ips@...; Nguyen, = Nam
Subject: Re: [Ips] Immediate Commands in = iSCSI
 

It looks to  me = that the Reject response with reason 0x06 should be enough for the = initiator to handle the situation you describe.
If the TM are the cause then each of those has a response = indicating termination. After receiving a reject an initiator should = reissue new TMs only after getting back one or more responses.  The = initiator may even accurately guess the number of commands the targets = is capable of handling per connection and per LU. The only case in which = this handling may be awkward is over a WAN but if you are operating over = a WAN and TM performance is your only worry you are in good shape = :-)

Julo =

        =  

[Ips] Immediate = Commands in iSCSI



  =

 

Jacob_Cherian =

to:

ips

12/03/07 05:23 = AM



 
=  

Cc: =

Nam_Nguyen




       

  =

 





=
RFC 3720 states:
=
The number of commands used for immediate delivery is not limited = and their delivery for execution is not acknowledged through the = numbering scheme. Immediate commands MAY be rejected by the iSCSI target = layer due to a lack of resources. An iSCSI target MUST be able to handle = at least one immediate task management command and one immediate = non-task-management iSCSI command per connection at any = time.

 

With storage = systems that have very large number of logical units, we run into an = issue when we use LU Reset or Abort Task for reset recovery. I couldn=92t = find a way to tell an initiator how many immediate commands a target can = accept. The issue with this is that if we have to issue multiple task = management commands and it is more than what the target can accept, it = will reject the task management request. Since we don=92t know why it = was rejected, we will have to escalate the reset. This can be very = disruptive with nodes that have very large number of logical units. = There may be two ways to resolve this:

 

1)      Define a new Text = negotiation key for number of outstanding immediate commands supported. = Unlike non-immediate (data) commands where we can send TASK SET FULL, = there is no way for a target to report in the Task Management Function = Response PDU that is unable to process the current request before it is = out of resources.  The default vaule for this new key can be 1, so = if this key is not negotiated initiators can assume one per = connection.

2)   =    Define a new value for the Response field in the Task = Management Function Response PDU for reporting that the target cannot = process the request currently.
=
 
=
Any suggestions on how to resolve this?

 

Thanks,

Jacob =


=
_______________________________________________
Ips mailing = list
Ips@ietf.org
https://www1.ietf.org/= mailman/listinfo/ips

= --Apple-Mail-98--372455595-- --===============1738068149== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline _______________________________________________ Ips mailing list Ips@ietf.org https://www1.ietf.org/mailman/listinfo/ips --===============1738068149==--