From mailman-bounces@ietf.org Fri Oct 1 06:32:35 2004 Received: from ietf-mx.ietf.org (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id GAA15129 for ; Fri, 1 Oct 2004 06:32:35 -0400 (EDT) Received: from megatron.ietf.org ([132.151.6.71]) by ietf-mx.ietf.org with esmtp (Exim 4.33) id 1CDKqn-0003rR-5C for ips-web-archive@ietf.org; Fri, 01 Oct 2004 06:41:21 -0400 Received: from localhost.localdomain ([127.0.0.1] helo=megatron.ietf.org) by megatron.ietf.org with esmtp (Exim 4.32) id 1CDJgE-00026N-UG for ips-web-archive@ietf.org; Fri, 01 Oct 2004 05:26:22 -0400 MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Subject: ietf.org mailing list memberships reminder From: mailman-owner@ietf.org To: ips-web-archive@ietf.org X-No-Archive: yes Message-ID: Date: Fri, 01 Oct 2004 05:10:57 -0400 Precedence: bulk X-BeenThere: mailman@lists.ietf.org X-Mailman-Version: 2.1.5 List-Id: Mailman site list X-List-Administrivia: yes Sender: mailman-bounces@ietf.org Errors-To: mailman-bounces@ietf.org X-Spam-Score: 0.3 (/) X-Scan-Signature: 3e15cc4fdc61d7bce84032741d11c8e5 Content-Transfer-Encoding: 7bit This is a reminder, sent out once a month, about your ietf.org mailing list memberships. It includes your subscription info and how to use it to change it or unsubscribe from a list. You can visit the URLs to change your membership status or configuration, including unsubscribing, setting digest-style delivery or disabling delivery altogether (e.g., for a vacation), and so on. In addition to the URL interfaces, you can also use email to make such changes. For more info, send a message to the '-request' address of the list (for example, mailman-request@ietf.org) containing just the word 'help' in the message body, and an email message will be sent to you with instructions. ********************************************************************** NOTE WELL: Any submission to the IETF intended by the Contributor for publication as all or part of an IETF Internet-Draft or RFC and any statement made within the context of an IETF activity is considered an "IETF Contribution". Such statements include oral statements in IETF sessions, as well as written and electronic communications made at any time or place, which are addressed to: o the IETF plenary session, o any IETF working group or portion thereof, o the IESG, or any member thereof on behalf of the IESG, o the IAB or any member thereof on behalf of the IAB, o any IETF mailing list, including the IETF list itself, any working group or design team list, or any other list functioning under IETF auspices, o the RFC Editor or the Internet-Drafts function All IETF Contributions are subject to the rules of RFC 3667 and RFC 3668. Statements made outside of an IETF session, mailing list or other function, that are clearly not intended to be input to an IETF activity, group or function, are not IETF Contributions in the context of this notice. Please consult RFC 3667 for details. ******************************************************************************* If you have questions, problems, comments, etc, send them to mailman-owner@ietf.org. Thanks! Passwords for ips-web-archive@ietf.org: List Password // URL ---- -------- ips@ietf.org epceek https://www1.ietf.org/mailman/options/ips/ips-web-archive%40ietf.org From ips-bounces@ietf.org Fri Oct 1 15:01:56 2004 Received: from ietf-mx.ietf.org (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id PAA06970 for ; Fri, 1 Oct 2004 15:01:56 -0400 (EDT) Received: from megatron.ietf.org ([132.151.6.71]) by ietf-mx.ietf.org with esmtp (Exim 4.33) id 1CDSnk-0007vL-9z for ips-web-archive@ietf.org; Fri, 01 Oct 2004 15:10:47 -0400 Received: from localhost.localdomain ([127.0.0.1] helo=megatron.ietf.org) by megatron.ietf.org with esmtp (Exim 4.32) id 1CDSGV-0004Sz-Ew; Fri, 01 Oct 2004 14:36:23 -0400 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by megatron.ietf.org with esmtp (Exim 4.32) id 1CDRse-0006pr-8G for ips@megatron.ietf.org; Fri, 01 Oct 2004 14:11:44 -0400 Received: from ietf-mx.ietf.org (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id OAA02877 for ; Fri, 1 Oct 2004 14:11:40 -0400 (EDT) Received: from sccrmhc13.comcast.net ([204.127.202.64]) by ietf-mx.ietf.org with esmtp (Exim 4.33) id 1CDS16-0006J6-3u for ips@ietf.org; Fri, 01 Oct 2004 14:20:31 -0400 Received: from ivvt2dxrc11 (c-66-177-57-55.se.client2.attbi.com[66.177.57.55]) by comcast.net (sccrmhc13) with SMTP id <2004100118110901600c34vve> (Authid: esquicksall); Fri, 1 Oct 2004 18:11:09 +0000 Message-ID: <001101c4a7e2$06f0fa40$0303a8c0@ivvt2dxrc11> From: "Eddy Quicksall" To: "Hirobumi Yamaguchi" , References: <41357A00.5B98981A@jp.fujitsu.com> Subject: Re: [Ips] Format Error and Invalid PDU field Date: Fri, 1 Oct 2004 14:11:07 -0400 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-2022-jp" Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 6.00.2800.1437 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2800.1441 X-Spam-Score: 0.1 (/) X-Scan-Signature: f607d15ccc2bc4eaf3ade8ffa8af02a0 Content-Transfer-Encoding: 7bit X-BeenThere: ips@ietf.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: IP Storage List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Sender: ips-bounces@ietf.org Errors-To: ips-bounces@ietf.org X-Spam-Score: 0.1 (/) X-Scan-Signature: 52f7a77164458f8c7b36b66787c853da Content-Transfer-Encoding: 7bit I don't see an answer to this question. My take is that if the initiator does something wrong, it will probably not correct itself automatically. So, I would always just drop the connection. Eddy ----- Original Message ----- From: "Hirobumi Yamaguchi" To: Sent: Wednesday, September 01, 2004 3:28 AM Subject: [Ips] Format Error and Invalid PDU field > Hi, > > I would like to know about the difference between Format Error > and invalid PDU field. > > RFC3720 section 6.6 says: > > The following two explicit violations of PDU layout rules are format > errors: > > a) Illegal contents of any PDU header field except the Opcode > (legal values are specified in Section 10 iSCSI PDU Formats). > b) Inconsistent field contents (consistent field contents are > specified in Section 10 iSCSI PDU Formats). > > When a target or an initiator receives an iSCSI PDU with a format > error, it MUST immediately terminate all transport connections in the > session... > > > section 10.17.1 says: > > (Reason code 0x09 Invalid PDU field) > Note 2: A target should use this reason code for all invalid values > of PDU fields that are meant to describe a task, a response, or a > data transfer. Some examples are invalid TTT/ITT, buffer offset, LUN > qualifying a TTT, and an invalid sequence number in a SNACK. > > Please show me some examples about Format Error. > > When a target receives a TMF request PDU with `Function'=6(TARGET WARM RESET) > and `LUN or Reserved'=1(this must be zero as reserved), > should the target close the session or send a Reject PDU with `Reason code`=9 ? > > If the target should send the Reject PDU, should it send TMF response PDU > after the Reject PDU? > > Thanks, > Hirobumi Yamaguchi > > _______________________________________________ > Ips mailing list > Ips@ietf.org > https://www1.ietf.org/mailman/listinfo/ips _______________________________________________ Ips mailing list Ips@ietf.org https://www1.ietf.org/mailman/listinfo/ips From ips-bounces@ietf.org Sat Oct 2 12:03:37 2004 Received: from ietf-mx.ietf.org (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id MAA24232 for ; Sat, 2 Oct 2004 12:03:37 -0400 (EDT) Received: from megatron.ietf.org ([132.151.6.71]) by ietf-mx.ietf.org with esmtp (Exim 4.33) id 1CDmUw-0005l9-JI for ips-web-archive@ietf.org; Sat, 02 Oct 2004 12:12:39 -0400 Received: from localhost.localdomain ([127.0.0.1] helo=megatron.ietf.org) by megatron.ietf.org with esmtp (Exim 4.32) id 1CDmHr-0004Zk-UA; Sat, 02 Oct 2004 11:59:07 -0400 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by megatron.ietf.org with esmtp (Exim 4.32) id 1CDmAk-0008VL-W3 for ips@megatron.ietf.org; Sat, 02 Oct 2004 11:51:47 -0400 Received: from ietf-mx.ietf.org (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id LAA23800 for ; Sat, 2 Oct 2004 11:51:44 -0400 (EDT) Received: from rwcrmhc11.comcast.net ([204.127.198.35]) by ietf-mx.ietf.org with esmtp (Exim 4.33) id 1CDmJP-0005Um-Kl for ips@ietf.org; Sat, 02 Oct 2004 12:00:46 -0400 Received: from ivvt2dxrc11 (c-66-177-57-55.se.client2.attbi.com[66.177.57.55]) by comcast.net (rwcrmhc11) with SMTP id <2004100215511201300c3bl1e> (Authid: esquicksall); Sat, 2 Oct 2004 15:51:13 +0000 Message-ID: <000801c4a897$a4c5efe0$0303a8c0@ivvt2dxrc11> From: "Eddy Quicksall" To: Date: Sat, 2 Oct 2004 11:51:11 -0400 MIME-Version: 1.0 X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 6.00.2800.1437 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2800.1441 X-Spam-Score: 0.2 (/) X-Scan-Signature: 41c17b4b16d1eedaa8395c26e9a251c4 Subject: [Ips] setting StatSN in a data pdu X-BeenThere: ips@ietf.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: IP Storage List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Content-Type: multipart/mixed; boundary="===============0356721980==" Sender: ips-bounces@ietf.org Errors-To: ips-bounces@ietf.org X-Spam-Score: 0.3 (/) X-Scan-Signature: 32b73d73e8047ed17386f9799119ce43 This is a multi-part message in MIME format. --===============0356721980== Content-Type: multipart/alternative; boundary="----=_NextPart_000_0005_01C4A876.1CFFF400" This is a multi-part message in MIME format. ------=_NextPart_000_0005_01C4A876.1CFFF400 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable Section 3.5.1.5 says "In addition, StatSN is carried by the Data-In = PDUs." But 10.7.3 says "The fields StatSN, Status, and Residual Count only have = meaningful content if the S bit is set to 1" I believe section 3.5.1.5 should say "In addition, when the S bit is set = StatSN is carried by the Data-In PDUs." Eddy ------=_NextPart_000_0005_01C4A876.1CFFF400 Content-Type: text/html; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable
Section 3.5.1.5 says "In = addition, StatSN=20 is carried by the Data-In PDUs."
 
But 10.7.3 says "The = fields StatSN,=20 Status, and Residual Count only have meaningful content=20 if the S bit is set to 1"
 
I believe section 3.5.1.5 should say "In addition, = when the S=20 bit is set StatSN is carried by the Data-In = PDUs."
 
Eddy
------=_NextPart_000_0005_01C4A876.1CFFF400-- --===============0356721980== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline Content-Transfer-Encoding: 7bit _______________________________________________ Ips mailing list Ips@ietf.org https://www1.ietf.org/mailman/listinfo/ips --===============0356721980==-- From ips-bounces@ietf.org Mon Oct 4 00:35:22 2004 Received: from ietf-mx.ietf.org (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id AAA24623 for ; Mon, 4 Oct 2004 00:10:54 -0400 (EDT) Received: from megatron.ietf.org ([132.151.6.71]) by ietf-mx.ietf.org with esmtp (Exim 4.33) id 1CEKKa-0002zH-PM for ips-web-archive@ietf.org; Mon, 04 Oct 2004 00:20:15 -0400 Received: from localhost.localdomain ([127.0.0.1] helo=megatron.ietf.org) by megatron.ietf.org with esmtp (Exim 4.32) id 1CEJtp-0002TG-9N; Sun, 03 Oct 2004 23:52:33 -0400 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by megatron.ietf.org with esmtp (Exim 4.32) id 1CEJqN-0008GR-CO for ips@megatron.ietf.org; Sun, 03 Oct 2004 23:48:59 -0400 Received: from ietf-mx.ietf.org (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id XAA18496 for ; Sun, 3 Oct 2004 23:48:54 -0400 (EDT) Received: from taurus.voltaire.com ([212.143.27.73]) by ietf-mx.ietf.org with esmtp (Exim 4.33) id 1CEACR-0000l5-R4 for ips@ietf.org; Sun, 03 Oct 2004 13:31:08 -0400 X-MimeOLE: Produced By Microsoft Exchange V6.5.7226.0 Content-class: urn:content-classes:message MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable Subject: RE: [Ips] DA+iSER Flow Control, Limiting number of oustanding cmds at target Date: Sun, 3 Oct 2004 19:21:06 +0200 Message-ID: Thread-Topic: [Ips] DA+iSER Flow Control, Limiting number of oustanding cmds at target Thread-Index: AcSg54qkgeGBxftBTAC7Jc0YoSIhRwIgcOig From: "Alex Nezhinsky" To: "Mallikarjun C." , "Mike Ko" , "Julian Satran" X-Spam-Score: 0.0 (/) X-Scan-Signature: 4b800b1eab964a31702fa68f1ff0e955 Content-Transfer-Encoding: quoted-printable Cc: ips@ietf.org X-BeenThere: ips@ietf.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: IP Storage List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Sender: ips-bounces@ietf.org Errors-To: ips-bounces@ietf.org X-Spam-Score: 0.0 (/) X-Scan-Signature: d8ae4fd88fcaf47c1a71c804d04f413d Content-Transfer-Encoding: quoted-printable =20 Mallikarjun, Mike, Julo, Thanx for clarifiaction. Actually, I thought about the receive buffers as connection-level resources, which must be allocated beforehand, during Allocate_Connection_Resources, and posted subsequently.=20 I thought that treating these buffers as task-level resources opens a possibility that their allocation fails at a later stage and the entire connection may be closed. But of course, in this case the concerns that I tried to raise are insubstantial. Returning to the flow control issue -- a possible scheme for posting receive buffers may be the following one. Each time the initiator send a request PDU, it posts a receive buffer for the response (before sending). Each time the target sends a PDU with an increased value of MaxCmdSN, it posts enough recv buffers to accommodate with the longest incoming command-related PDU sequence for the advertized CmdSN delta. This addresses only the "normal path" PDUs. Immediate commands and NOP-outs from initiator, NOP-Ins and AsyncMsgs from the target are outside of the flow control offered by MaxCmdSN. What is a feasible provision for the out-of band PDUs like these? In the latest "Flow Ctrl" thread I saw remarks about a past discussion, where a suggestion to limit the number of simultaneously issued PDUs of these types had been dismissed. I could not find this discussion. Can you shortly reproduce the reasoning and explain how do you see a possible implementation?=20 Alexander Nezhinsky. > -----Original Message----- > From: Mallikarjun C. [mailto:cbm@rose.hp.com]=20 > Sent: Wednesday, September 22, 2004 11:00 PM > To: Alex Nezhinsky > Cc: ips@ietf.org > Subject: Re: [Ips] DA+iSER Flow Control,Limiting number of=20 > oustanding cmds at target >=20 > Alex, >=20 > > The only reason may be that the initiator does not know=20 > the upper bound > when it executes Allocate_Connection_Resources. >=20 > It appears that your concern is wrt sizing of per-task buffer=20 > resources on the invocation of Allocate_Connection_Resources. >=20 > Note that section 9.4 in DA says the following: >=20 > "Note > that the Datamover layer however does not allocate any=20 > Datamover-specific task-level resources upon invocation of=20 > this Primitive." >=20 > The task buffers are thus not related to=20 > Allocate_Connection_Resources.=20 > The buffer management is specific to implementations. DA=20 > only specifies the architectural primitives that are=20 > minimally required to allow the definition of a Datamover=20 > protocol, but not everything for an implementation. >=20 > Thanks. > -- > Mallikarjun >=20 > Mallikarjun Chadalapaka > Networked Storage Architecture > Network Storage Solutions > Hewlett-Packard MS 5668 > Roseville CA 95747 > cbm [at] rose.hp.com >=20 _______________________________________________ Ips mailing list Ips@ietf.org https://www1.ietf.org/mailman/listinfo/ips From ips-bounces@ietf.org Mon Oct 4 04:51:22 2004 Received: from ietf-mx.ietf.org (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id EAA13478 for ; Mon, 4 Oct 2004 04:51:22 -0400 (EDT) Received: from megatron.ietf.org ([132.151.6.71]) by ietf-mx.ietf.org with esmtp (Exim 4.33) id 1CEOi5-0002jf-GX for ips-web-archive@ietf.org; Mon, 04 Oct 2004 05:00:45 -0400 Received: from localhost.localdomain ([127.0.0.1] helo=megatron.ietf.org) by megatron.ietf.org with esmtp (Exim 4.32) id 1CEOXL-0003X1-2p; Mon, 04 Oct 2004 04:49:39 -0400 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by megatron.ietf.org with esmtp (Exim 4.32) id 1CEOK3-00027p-G6 for ips@megatron.ietf.org; Mon, 04 Oct 2004 04:35:55 -0400 Received: from ietf-mx.ietf.org (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id EAA11756 for ; Mon, 4 Oct 2004 04:35:52 -0400 (EDT) Received: from mtagate3.de.ibm.com ([195.212.29.152]) by ietf-mx.ietf.org with esmtp (Exim 4.33) id 1CEOT5-0008PT-OT for ips@ietf.org; Mon, 04 Oct 2004 04:45:16 -0400 Received: from d12nrmr1507.megacenter.de.ibm.com (d12nrmr1507.megacenter.de.ibm.com [9.149.167.1]) by mtagate3.de.ibm.com (8.12.10/8.12.10) with ESMTP id i948ZM0v055926; Mon, 4 Oct 2004 08:35:22 GMT Received: from d12ml102.megacenter.de.ibm.com (d12av02.megacenter.de.ibm.com [9.149.165.228]) by d12nrmr1507.megacenter.de.ibm.com (8.12.10/NCO/VER6.6) with ESMTP id i948ZDXw217288; Mon, 4 Oct 2004 10:35:14 +0200 In-Reply-To: To: "Alex Nezhinsky" Subject: RE: [Ips] DA+iSER Flow Control, Limiting number of oustanding cmds at target MIME-Version: 1.0 X-Mailer: Lotus Notes Build V70_M2_07222004 Beta 2NP July 22, 2004 From: Julian Satran Message-ID: Date: Mon, 4 Oct 2004 10:35:12 +0200 X-MIMETrack: Serialize by Router on D12ML102/12/M/IBM(Release 6.5.1| March 5, 2004) at 04/10/2004 10:35:21, Serialize complete at 04/10/2004 10:35:21 X-Spam-Score: 0.1 (/) X-Scan-Signature: cdeeb24e6b743a852c396a4af0e53c8f Cc: ips@ietf.org, Mike Ko , "Mallikarjun C." X-BeenThere: ips@ietf.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: IP Storage List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Content-Type: multipart/mixed; boundary="===============0836257291==" Sender: ips-bounces@ietf.org Errors-To: ips-bounces@ietf.org X-Spam-Score: 0.1 (/) X-Scan-Signature: 40161b1d86420e0807d771943d981d25 This is a multipart message in MIME format. --===============0836257291== Content-Type: multipart/alternative; boundary="=_alternative 002D6A59C2256F23_=" This is a multipart message in MIME format. --=_alternative 002D6A59C2256F23_= Content-Type: text/plain; charset="US-ASCII" Alex, The relevant text in RFC 3720 - 3.2.2.1 is: The number of commands used for immediate delivery is not limited and their delivery for execution is not acknowledged through the numbering scheme. Immediate commands MAY be rejected by the iSCSI target layer due to a lack of resources. An iSCSI target MUST be able to handle at least one immediate task management command and one immediate non-task-management iSCSI command per connection at any time. I hope this helps. Hag Sameach, Julo "Alex Nezhinsky" Sent by: ips-bounces@ietf.org 03/10/04 19:21 To "Mallikarjun C." , "Mike Ko" , Julian Satran/Haifa/IBM@IBMIL cc ips@ietf.org Subject RE: [Ips] DA+iSER Flow Control, Limiting number of oustanding cmds at target Mallikarjun, Mike, Julo, Thanx for clarifiaction. Actually, I thought about the receive buffers as connection-level resources, which must be allocated beforehand, during Allocate_Connection_Resources, and posted subsequently. I thought that treating these buffers as task-level resources opens a possibility that their allocation fails at a later stage and the entire connection may be closed. But of course, in this case the concerns that I tried to raise are insubstantial. Returning to the flow control issue -- a possible scheme for posting receive buffers may be the following one. Each time the initiator send a request PDU, it posts a receive buffer for the response (before sending). Each time the target sends a PDU with an increased value of MaxCmdSN, it posts enough recv buffers to accommodate with the longest incoming command-related PDU sequence for the advertized CmdSN delta. This addresses only the "normal path" PDUs. Immediate commands and NOP-outs from initiator, NOP-Ins and AsyncMsgs from the target are outside of the flow control offered by MaxCmdSN. What is a feasible provision for the out-of band PDUs like these? In the latest "Flow Ctrl" thread I saw remarks about a past discussion, where a suggestion to limit the number of simultaneously issued PDUs of these types had been dismissed. I could not find this discussion. Can you shortly reproduce the reasoning and explain how do you see a possible implementation? Alexander Nezhinsky. > -----Original Message----- > From: Mallikarjun C. [mailto:cbm@rose.hp.com] > Sent: Wednesday, September 22, 2004 11:00 PM > To: Alex Nezhinsky > Cc: ips@ietf.org > Subject: Re: [Ips] DA+iSER Flow Control,Limiting number of > oustanding cmds at target > > Alex, > > > The only reason may be that the initiator does not know > the upper bound > when it executes Allocate_Connection_Resources. > > It appears that your concern is wrt sizing of per-task buffer > resources on the invocation of Allocate_Connection_Resources. > > Note that section 9.4 in DA says the following: > > "Note > that the Datamover layer however does not allocate any > Datamover-specific task-level resources upon invocation of > this Primitive." > > The task buffers are thus not related to > Allocate_Connection_Resources. > The buffer management is specific to implementations. DA > only specifies the architectural primitives that are > minimally required to allow the definition of a Datamover > protocol, but not everything for an implementation. > > Thanks. > -- > Mallikarjun > > Mallikarjun Chadalapaka > Networked Storage Architecture > Network Storage Solutions > Hewlett-Packard MS 5668 > Roseville CA 95747 > cbm [at] rose.hp.com > _______________________________________________ Ips mailing list Ips@ietf.org https://www1.ietf.org/mailman/listinfo/ips --=_alternative 002D6A59C2256F23_= Content-Type: text/html; charset="US-ASCII"
Alex,

The relevant text in RFC 3720 - 3.2.2.1 is:

  The number of commands used for immediate delivery is not limited and
   their delivery for execution is not acknowledged through the
   numbering scheme.  Immediate commands MAY be rejected by the iSCSI
   target layer due to a lack of resources.  An iSCSI target MUST be
   able to handle at least one immediate task management command and one
   immediate non-task-management iSCSI command per connection at any
   time.

I hope this helps.

Hag Sameach,
Julo


"Alex Nezhinsky" <alexn@voltaire.com>
Sent by: ips-bounces@ietf.org

03/10/04 19:21

To
"Mallikarjun C." <cbm@rose.hp.com>, "Mike Ko" <mako@almaden.ibm.com>, Julian Satran/Haifa/IBM@IBMIL
cc
ips@ietf.org
Subject
RE: [Ips] DA+iSER Flow Control,        Limiting number of oustanding cmds at target





 
Mallikarjun, Mike, Julo,

Thanx for clarifiaction. Actually, I thought about the receive buffers
as connection-level resources, which must be allocated beforehand,
during Allocate_Connection_Resources, and  posted subsequently.
I thought that treating these buffers as task-level resources opens a
possibility that their allocation fails at a later stage and the entire
connection may be closed. But of course, in this case the concerns that
I tried to raise are insubstantial.

Returning to the flow control issue -- a possible scheme for posting
receive buffers may be the following one. Each time the initiator send a
request PDU, it posts a receive buffer for the response (before
sending). Each time the target sends a PDU with an increased value of
MaxCmdSN, it posts enough recv buffers to accommodate with the longest
incoming command-related PDU sequence for the advertized CmdSN delta.
This addresses only the "normal path" PDUs.
Immediate commands and NOP-outs from initiator, NOP-Ins and AsyncMsgs
from the target are outside of the flow control offered by MaxCmdSN.
What is a feasible provision for the out-of band PDUs like these? In the
latest "Flow Ctrl" thread I saw remarks about a past discussion, where a
suggestion to limit the number of simultaneously issued PDUs of these
types had been dismissed. I could not find this discussion. Can you
shortly reproduce the reasoning and explain how do you see a possible
implementation?

Alexander Nezhinsky.

> -----Original Message-----
> From: Mallikarjun C. [mailto:cbm@rose.hp.com]
> Sent: Wednesday, September 22, 2004 11:00 PM
> To: Alex Nezhinsky
> Cc: ips@ietf.org
> Subject: Re: [Ips] DA+iSER Flow Control,Limiting number of
> oustanding cmds at target
>
> Alex,
>
>  > The only reason may be that the initiator does not know
> the upper bound  > when it executes Allocate_Connection_Resources.
>
> It appears that your concern is wrt sizing of per-task buffer
> resources on the invocation of Allocate_Connection_Resources.
>
> Note that section 9.4 in DA says the following:
>
> "Note
> that the Datamover layer however does not allocate any
> Datamover-specific task-level resources upon invocation of
> this Primitive."
>
> The task buffers are thus not related to
> Allocate_Connection_Resources.
>   The buffer management is specific to implementations.  DA
> only specifies the architectural primitives that are
> minimally required to allow the definition of a Datamover
> protocol, but not everything for an implementation.
>
> Thanks.
> --
> Mallikarjun
>
> Mallikarjun Chadalapaka
> Networked Storage Architecture
> Network Storage Solutions
> Hewlett-Packard MS 5668
> Roseville CA 95747
> cbm [at] rose.hp.com
>

_______________________________________________
Ips mailing list
Ips@ietf.org
https://www1.ietf.org/mailman/listinfo/ips
--=_alternative 002D6A59C2256F23_=-- --===============0836257291== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline Content-Transfer-Encoding: 7bit _______________________________________________ Ips mailing list Ips@ietf.org https://www1.ietf.org/mailman/listinfo/ips --===============0836257291==-- From ips-bounces@ietf.org Mon Oct 4 06:22:49 2004 Received: from ietf-mx.ietf.org (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id GAA20297 for ; Mon, 4 Oct 2004 06:22:49 -0400 (EDT) Received: from megatron.ietf.org ([132.151.6.71]) by ietf-mx.ietf.org with esmtp (Exim 4.33) id 1CEQ8c-00053I-IQ for ips-web-archive@ietf.org; Mon, 04 Oct 2004 06:32:14 -0400 Received: from localhost.localdomain ([127.0.0.1] helo=megatron.ietf.org) by megatron.ietf.org with esmtp (Exim 4.32) id 1CEPyK-0000ve-7B; Mon, 04 Oct 2004 06:21:36 -0400 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by megatron.ietf.org with esmtp (Exim 4.32) id 1CEPvX-0000Lf-CK for ips@megatron.ietf.org; Mon, 04 Oct 2004 06:18:43 -0400 Received: from ietf-mx.ietf.org (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id GAA19986 for ; Mon, 4 Oct 2004 06:18:40 -0400 (EDT) Received: from system60.simplicato.com ([207.99.47.60] helo=host52a.simplicato.com) by ietf-mx.ietf.org with esmtp (Exim 4.33) id 1CEQ4Z-00040b-NP for ips@ietf.org; Mon, 04 Oct 2004 06:28:04 -0400 Received: from localhost (localhost.simplicato.com [127.0.0.1]) by host52a.simplicato.com (Postfix) with ESMTP id DA25011E094; Mon, 4 Oct 2004 06:18:38 -0400 (EDT) Received: from host52a.simplicato.com ([127.0.0.1]) by localhost (host52a.simplicato.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 15813-09; Mon, 4 Oct 2004 06:18:37 -0400 (EDT) Received: from mail.asomi.com (localhost.simplicato.com [127.0.0.1]) by host52a.simplicato.com (Postfix) with SMTP id AACF811E093; Mon, 4 Oct 2004 06:18:37 -0400 (EDT) Received: from 62.90.193.206 (proxying for 62.90.193.206) (SquirrelMail authenticated user cait@asomi.com) by mail52a.simplicato.com with HTTP; Mon, 4 Oct 2004 05:18:37 -0500 (CDT) Message-ID: <36270.62.90.193.206.1096885117.squirrel@mail52a.simplicato.com> In-Reply-To: References: Date: Mon, 4 Oct 2004 05:18:37 -0500 (CDT) Subject: RE: [Ips] DA+iSER Flow Control, Limiting number of oustanding cmds attarget From: "Caitlin Bestler" To: "Julian Satran" User-Agent: SquirrelMail/1.4.2 MIME-Version: 1.0 Content-Type: text/plain;charset=iso-8859-1 X-Priority: 3 Importance: Normal X-Virus-Scanned: by amavisd-new at simplicato.com Content-Transfer-Encoding: quoted-printable X-Spam-Score: 0.8 (/) X-Scan-Signature: 0ddefe323dd869ab027dbfff7eff0465 Content-Transfer-Encoding: quoted-printable Cc: ips@ietf.org, "Mallikarjun C." , Alex Nezhinsky , Mike Ko X-BeenThere: ips@ietf.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: IP Storage List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Sender: ips-bounces@ietf.org Errors-To: ips-bounces@ietf.org X-Spam-Score: 0.8 (/) X-Scan-Signature: 3e15cc4fdc61d7bce84032741d11c8e5 Content-Transfer-Encoding: quoted-printable Julian Satran said: > Alex, > > The relevant text in RFC 3720 - 3.2.2.1 is: > > The number of commands used for immediate delivery is not limited and > their delivery for execution is not acknowledged through the > numbering scheme. Immediate commands MAY be rejected by the iSCSI > target layer due to a lack of resources. An iSCSI target MUST be > able to handle at least one immediate task management command and on= e > immediate non-task-management iSCSI command per connection at any > time. > > I hope this helps. > > Hag Sameach, > Julo > The iSCSI target "MAY" be allowed to discard excess immediate commands, but that doesn't help the RDMA layer that needs a buffer for an untagged message, or it will be required to terminate the connection. Because iSER maps both the flow-controlled and non-flow-controlled messag= es to the same connection, lack of a buffer for an immediate command places non-immediate messages at risk. Obviously, creating an extra connection for infrequent commands is not the correct solution. But a correct solution does require that an iSER implementation has *something* to enable it to determine how many buffers to pre-post to the RDMAP layer. It already has the answer for non-immediate commands, which is the bulk of the traffic. It is also relatively easy to make some very good estimates on how many would be required for immediate commands. But interoperability of RFC compliant implementations should be based upon their RFC compliance, not on compatible guestimations. What I believe Alex is asking for, and I asked for earlier, is a basis fo= r making that estimate that is based on a specification, and not heuristic measurements of N leading products. -- Caitlin Bestler http://asomi.com/ _______________________________________________ Ips mailing list Ips@ietf.org https://www1.ietf.org/mailman/listinfo/ips From ips-bounces@ietf.org Mon Oct 4 07:46:55 2004 Received: from ietf-mx.ietf.org (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id HAA26747 for ; Mon, 4 Oct 2004 07:46:55 -0400 (EDT) Received: from megatron.ietf.org ([132.151.6.71]) by ietf-mx.ietf.org with esmtp (Exim 4.33) id 1CERRz-0000dY-Ed for ips-web-archive@ietf.org; Mon, 04 Oct 2004 07:56:19 -0400 Received: from localhost.localdomain ([127.0.0.1] helo=megatron.ietf.org) by megatron.ietf.org with esmtp (Exim 4.32) id 1CERF7-0008S8-LS; Mon, 04 Oct 2004 07:43:01 -0400 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by megatron.ietf.org with esmtp (Exim 4.32) id 1CERDA-00085G-Iz for ips@megatron.ietf.org; Mon, 04 Oct 2004 07:41:03 -0400 Received: from ietf-mx.ietf.org (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id HAA26288 for ; Mon, 4 Oct 2004 07:40:58 -0400 (EDT) Received: from mtagate3.de.ibm.com ([195.212.29.152]) by ietf-mx.ietf.org with esmtp (Exim 4.33) id 1CERMD-0008Su-RG for ips@ietf.org; Mon, 04 Oct 2004 07:50:22 -0400 Received: from d12nrmr1607.megacenter.de.ibm.com (d12nrmr1607.megacenter.de.ibm.com [9.149.167.49]) by mtagate3.de.ibm.com (8.12.10/8.12.10) with ESMTP id i94BeL0v040932; Mon, 4 Oct 2004 11:40:21 GMT Received: from d12ml102.megacenter.de.ibm.com (d12av02.megacenter.de.ibm.com [9.149.165.228]) by d12nrmr1607.megacenter.de.ibm.com (8.12.10/NCO/VER6.6) with ESMTP id i94BeKOM106488; Mon, 4 Oct 2004 13:40:21 +0200 In-Reply-To: <95AC37E4A7C3304D94DB6AF7F64FA1B501513CEB@cacexc08.americas.cpqcorp.net> To: "Krueger, Marjorie" Subject: RE: [Ips] Initiator CHAP authentication of target? MIME-Version: 1.0 X-Mailer: Lotus Notes Build V70_M2_07222004 Beta 2NP July 22, 2004 From: Julian Satran Message-ID: Date: Mon, 4 Oct 2004 13:40:19 +0200 X-MIMETrack: Serialize by Router on D12ML102/12/M/IBM(Release 6.5.1| March 5, 2004) at 04/10/2004 13:40:21, Serialize complete at 04/10/2004 13:40:21 X-Spam-Score: 0.6 (/) X-Scan-Signature: ce732c7d36989a1bd55104ba259c40a1 Cc: ips@ietf.org, William Studenmund X-BeenThere: ips@ietf.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: IP Storage List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Content-Type: multipart/mixed; boundary="===============1797078473==" Sender: ips-bounces@ietf.org Errors-To: ips-bounces@ietf.org X-Spam-Score: 0.6 (/) X-Scan-Signature: e5bfa71b340354e384155def5e70b13b This is a multipart message in MIME format. --===============1797078473== Content-Type: multipart/alternative; boundary="=_alternative 003E0EDFC2256F23_=" This is a multipart message in MIME format. --=_alternative 003E0EDFC2256F23_= Content-Type: text/plain; charset="US-ASCII" Marjorie, Maybe I am not understanding clearly the "substance" of your question but if you are questioning the validity of an initiator not using the data supplied to authenticate the target - that is "legal" in the sense that the protocol has no way of enforcing the check in CHAP (i.e., there is no returned value that will irrefutably prove that the initiator did check). I would not use this protocol in a money exchange transaction! The weaknesses of CHAP have been discussed at some length on the list. Regards, Julo "Krueger, Marjorie" Sent by: ips-bounces@ietf.org 01/10/04 01:28 To "William Studenmund" cc ips@ietf.org Subject RE: [Ips] Initiator CHAP authentication of target? It appears a certain initiator is ignoring CHAP_N returned by the target and just using the target node name for CHAP_N regardless of what the target sent in the login sequence. I think it seems like invalid behavior but I just wanted to check with this audience. Thanks Marjorie > -----Original Message----- > From: William Studenmund [mailto:wrstuden@wasabisystems.com] > Sent: Thursday, September 30, 2004 3:32 PM > To: Krueger, Marjorie > Cc: ips@ietf.org > Subject: Re: [Ips] Initiator CHAP authentication of target? > > On Sep 30, 2004, at 3:22 PM, Krueger, Marjorie wrote: > > > So if > > > >> or, if it requires target authentication, with: > >> > >> CHAP_N= CHAP_R= CHAP_I= CHAP_C= > >> > >> If the initiator authentication fails, the target MUST answer > >> with a > >> Login reject with "Authentication Failure" status. > >> Otherwise, if the > >> initiator required target authentication, the target > MUST either > >> answer with a Login reject with "Authentication > Failure" or reply > >> with: > >> > >> CHAP_N= CHAP_R= > > > > And the target replys with CHAP_N that is not defined and the > > initiator accepts that (accepts anything) is that valid > behavior? In > > otherwords, if the initiator ignores CHAP_N and only uses > the target > > node name and CHAP_R to authenticate, is that valid behavior? > > Shouldn't the target be able to force the use of CHAP_N? > > I'm confused. The initiator can't NOT use CHAP_N. Thus the > target shouldn't be able to force something that's required. :-) > > To quote the paragraph after the one above: > > Where N, (A,A1,A2), I, C, and R are (correspondingly) the Name, > Algorithm, Identifier, Challenge, and Response as defined in > [RFC1994], N is a text string, A,A1,A2, and I are > numbers, and C and > R are large-binary-values and their binary length (not > the length of > the character string that represents them in encoded > form) MUST not > exceed 1024 bytes. > > So an initiator that just uses the target's name for CHAP > purposes seems broken. > > Take care, > > Bill > _______________________________________________ Ips mailing list Ips@ietf.org https://www1.ietf.org/mailman/listinfo/ips --=_alternative 003E0EDFC2256F23_= Content-Type: text/html; charset="US-ASCII"
Marjorie,

Maybe I am not understanding clearly the "substance" of your question but if you are questioning the validity of an initiator not using the data supplied to authenticate the target - that is "legal" in the sense that the protocol has no way of enforcing the check in CHAP (i.e., there is no returned value that will irrefutably prove that the initiator did check).
I would not use this protocol in a money exchange transaction! The weaknesses of CHAP have been discussed at some length on the list.

Regards,
Julo


"Krueger, Marjorie" <marjorie.krueger@hp.com>
Sent by: ips-bounces@ietf.org

01/10/04 01:28

To
"William Studenmund" <wrstuden@wasabisystems.com>
cc
ips@ietf.org
Subject
RE: [Ips] Initiator CHAP authentication of target?





It appears a certain initiator is ignoring CHAP_N returned by the target
and just using the target node name for CHAP_N regardless of what the
target sent in the login sequence.  I think it seems like invalid
behavior but I just wanted to check with this audience.

Thanks
Marjorie

> -----Original Message-----
> From: William Studenmund [mailto:wrstuden@wasabisystems.com]
> Sent: Thursday, September 30, 2004 3:32 PM
> To: Krueger, Marjorie
> Cc: ips@ietf.org
> Subject: Re: [Ips] Initiator CHAP authentication of target?
>
> On Sep 30, 2004, at 3:22 PM, Krueger, Marjorie wrote:
>
> > So if
> >
> >>     or, if it requires target authentication, with:
> >>
> >>        CHAP_N=<N> CHAP_R=<R> CHAP_I=<I> CHAP_C=<C>
> >>
> >>     If the initiator authentication fails, the target MUST answer
> >> with a
> >>     Login reject with "Authentication Failure" status.
> >> Otherwise, if the
> >>     initiator required target authentication, the target
> MUST either
> >>     answer with a Login reject with "Authentication
> Failure" or reply
> >>     with:
> >>
> >>        CHAP_N=<N> CHAP_R=<R>
> >
> > And the target replys with CHAP_N that is not defined and the
> > initiator accepts that (accepts anything) is that valid
> behavior?  In
> > otherwords, if the initiator ignores CHAP_N and only uses
> the target
> > node name and CHAP_R to authenticate, is that valid behavior?  
> > Shouldn't the target be able to force the use of CHAP_N?
>
> I'm confused. The initiator can't NOT use CHAP_N. Thus the
> target shouldn't be able to force something that's required. :-)
>
> To quote the paragraph after the one above:
>
>     Where N, (A,A1,A2), I, C, and R are (correspondingly) the Name,
>     Algorithm, Identifier, Challenge, and Response as defined in
>     [RFC1994], N is a text string, A,A1,A2, and I are
> numbers, and C and
>     R are large-binary-values and their binary length (not
> the length of
>     the character string that represents them in encoded
> form) MUST not
>     exceed 1024 bytes.
>
> So an initiator that just uses the target's name for CHAP
> purposes seems broken.
>
> Take care,
>
> Bill
>

_______________________________________________
Ips mailing list
Ips@ietf.org
https://www1.ietf.org/mailman/listinfo/ips
--=_alternative 003E0EDFC2256F23_=-- --===============1797078473== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline Content-Transfer-Encoding: 7bit _______________________________________________ Ips mailing list Ips@ietf.org https://www1.ietf.org/mailman/listinfo/ips --===============1797078473==-- From ips-bounces@ietf.org Mon Oct 4 08:44:19 2004 Received: from ietf-mx.ietf.org (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id IAA01011 for ; Mon, 4 Oct 2004 08:44:19 -0400 (EDT) Received: from megatron.ietf.org ([132.151.6.71]) by ietf-mx.ietf.org with esmtp (Exim 4.33) id 1CESLY-0000xW-JP for ips-web-archive@ietf.org; Mon, 04 Oct 2004 08:53:44 -0400 Received: from localhost.localdomain ([127.0.0.1] helo=megatron.ietf.org) by megatron.ietf.org with esmtp (Exim 4.32) id 1CESBd-0002CL-FV; Mon, 04 Oct 2004 08:43:29 -0400 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by megatron.ietf.org with esmtp (Exim 4.32) id 1CESAY-00022s-NT for ips@megatron.ietf.org; Mon, 04 Oct 2004 08:42:23 -0400 Received: from ietf-mx.ietf.org (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id IAA00940 for ; Mon, 4 Oct 2004 08:42:20 -0400 (EDT) Received: from mtagate4.de.ibm.com ([195.212.29.153]) by ietf-mx.ietf.org with esmtp (Exim 4.33) id 1CESJb-0000XV-8z for ips@ietf.org; Mon, 04 Oct 2004 08:51:46 -0400 Received: from d12nrmr1607.megacenter.de.ibm.com (d12nrmr1607.megacenter.de.ibm.com [9.149.167.49]) by mtagate4.de.ibm.com (8.12.10/8.12.10) with ESMTP id i94CfjLi108004; Mon, 4 Oct 2004 12:41:45 GMT Received: from d12ml102.megacenter.de.ibm.com (d12av02.megacenter.de.ibm.com [9.149.165.228]) by d12nrmr1607.megacenter.de.ibm.com (8.12.10/NCO/VER6.6) with ESMTP id i94CfiOL111586; Mon, 4 Oct 2004 14:41:44 +0200 In-Reply-To: <36270.62.90.193.206.1096885117.squirrel@mail52a.simplicato.com> To: "Caitlin Bestler" Subject: RE: [Ips] DA+iSER Flow Control, Limiting number of oustanding cmds attarget MIME-Version: 1.0 X-Mailer: Lotus Notes Build V70_M2_07222004 Beta 2NP July 22, 2004 From: Julian Satran Message-ID: Date: Mon, 4 Oct 2004 14:41:43 +0200 X-MIMETrack: Serialize by Router on D12ML102/12/M/IBM(Release 6.5.1| March 5, 2004) at 04/10/2004 14:41:44, Serialize complete at 04/10/2004 14:41:44 X-Spam-Score: 1.0 (+) X-Scan-Signature: 7a0494a0224ca59418dd8f92694c1fdb Cc: ips@ietf.org, Mike Ko , Alex Nezhinsky , "Mallikarjun C." X-BeenThere: ips@ietf.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: IP Storage List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Content-Type: multipart/mixed; boundary="===============0590054901==" Sender: ips-bounces@ietf.org Errors-To: ips-bounces@ietf.org X-Spam-Score: 0.0 (/) X-Scan-Signature: d008c19e97860b8641c1851f84665a75 This is a multipart message in MIME format. --===============0590054901== Content-Type: multipart/alternative; boundary="=_alternative 00415B16C2256F23_=" This is a multipart message in MIME format. --=_alternative 00415B16C2256F23_= Content-Type: text/plain; charset="US-ASCII" This is an iSER specific requirement and an iSER specific key may help. I am afraid however that even this might not be foolproof as an initiator may fill any number of buffers with non task management functions and then claim that the target is not iSCSI compliant as it does not accept an immediate task management function command. Or even worse - as RDMA has no way of distinguishing immediate from non-immediate commands an initiator can clog the pipe with non-immediate commands before the target will feel abused. Dropping the session when an initiator has sent more than the allowed number of commands+2 seems perfectly reasonable. Julo "Caitlin Bestler" 04/10/04 12:18 To Julian Satran/Haifa/IBM@IBMIL cc "Alex Nezhinsky" , ips@ietf.org, "Mike Ko" , "Mallikarjun C." Subject RE: [Ips] DA+iSER Flow Control, Limiting number of oustanding cmds attarget Julian Satran said: > Alex, > > The relevant text in RFC 3720 - 3.2.2.1 is: > > The number of commands used for immediate delivery is not limited and > their delivery for execution is not acknowledged through the > numbering scheme. Immediate commands MAY be rejected by the iSCSI > target layer due to a lack of resources. An iSCSI target MUST be > able to handle at least one immediate task management command and one > immediate non-task-management iSCSI command per connection at any > time. > > I hope this helps. > > Hag Sameach, > Julo > The iSCSI target "MAY" be allowed to discard excess immediate commands, but that doesn't help the RDMA layer that needs a buffer for an untagged message, or it will be required to terminate the connection. Because iSER maps both the flow-controlled and non-flow-controlled messages to the same connection, lack of a buffer for an immediate command places non-immediate messages at risk. Obviously, creating an extra connection for infrequent commands is not the correct solution. But a correct solution does require that an iSER implementation has *something* to enable it to determine how many buffers to pre-post to the RDMAP layer. It already has the answer for non-immediate commands, which is the bulk of the traffic. It is also relatively easy to make some very good estimates on how many would be required for immediate commands. But interoperability of RFC compliant implementations should be based upon their RFC compliance, not on compatible guestimations. What I believe Alex is asking for, and I asked for earlier, is a basis for making that estimate that is based on a specification, and not heuristic measurements of N leading products. -- Caitlin Bestler http://asomi.com/ --=_alternative 00415B16C2256F23_= Content-Type: text/html; charset="US-ASCII"
This is an iSER specific requirement and an iSER specific key may help. I am afraid however that even this might not be foolproof as an initiator may fill any number of buffers with non task management functions and then claim that the target is not iSCSI compliant as it does not accept an immediate task management function command.
Or even worse - as RDMA has no way of distinguishing immediate from non-immediate commands an initiator can clog the pipe with non-immediate commands before the target will feel abused.
Dropping the session when an initiator has sent more than the allowed number of commands+2 seems perfectly reasonable.

Julo


"Caitlin Bestler" <cait@asomi.com>

04/10/04 12:18

To
Julian Satran/Haifa/IBM@IBMIL
cc
"Alex Nezhinsky" <alexn@voltaire.com>, ips@ietf.org, "Mike Ko" <mako@almaden.ibm.com>, "Mallikarjun C." <cbm@rose.hp.com>
Subject
RE: [Ips] DA+iSER Flow Control,      Limiting number of oustanding cmds attarget






Julian Satran said:
> Alex,
>
> The relevant text in RFC 3720 - 3.2.2.1 is:
>
>   The number of commands used for immediate delivery is not limited and
>    their delivery for execution is not acknowledged through the
>    numbering scheme.  Immediate commands MAY be rejected by the iSCSI
>    target layer due to a lack of resources.  An iSCSI target MUST be
>    able to handle at least one immediate task management command and one
>    immediate non-task-management iSCSI command per connection at any
>    time.
>
> I hope this helps.
>
> Hag Sameach,
> Julo
>


The iSCSI target "MAY" be allowed to discard excess immediate commands,
but that doesn't help the RDMA layer that needs a buffer for an untagged
message, or it will be required to terminate the connection.

Because iSER maps both the flow-controlled and non-flow-controlled messages
to the same connection, lack of a buffer for an immediate command places
non-immediate messages at risk.

Obviously, creating an extra connection for infrequent commands is not
the correct solution.  But a correct solution does require that an iSER
implementation has *something* to enable it to determine how many
buffers to pre-post to the RDMAP layer.  It already has the answer for
non-immediate commands, which is the bulk of the traffic.  It is also
relatively easy to make some very good estimates on how many would
be required for immediate commands.

But interoperability of RFC compliant implementations should be based
upon their RFC compliance, not on compatible guestimations.

What I believe Alex is asking for, and I asked for earlier, is a basis for
making that estimate that is based on a specification, and not heuristic
measurements of N leading products.

--
Caitlin Bestler
http://asomi.com/
--=_alternative 00415B16C2256F23_=-- --===============0590054901== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline Content-Transfer-Encoding: 7bit _______________________________________________ Ips mailing list Ips@ietf.org https://www1.ietf.org/mailman/listinfo/ips --===============0590054901==-- From ips-bounces@ietf.org Mon Oct 4 09:29:54 2004 Received: from ietf-mx.ietf.org (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id JAA04347 for ; Mon, 4 Oct 2004 09:29:53 -0400 (EDT) Received: from megatron.ietf.org ([132.151.6.71]) by ietf-mx.ietf.org with esmtp (Exim 4.33) id 1CET3f-0000hz-Ge for ips-web-archive@ietf.org; Mon, 04 Oct 2004 09:39:19 -0400 Received: from localhost.localdomain ([127.0.0.1] helo=megatron.ietf.org) by megatron.ietf.org with esmtp (Exim 4.32) id 1CESs4-00070J-HX; Mon, 04 Oct 2004 09:27:20 -0400 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by megatron.ietf.org with esmtp (Exim 4.32) id 1CESmN-0006Rd-4t for ips@megatron.ietf.org; Mon, 04 Oct 2004 09:21:27 -0400 Received: from ietf-mx.ietf.org (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id JAA03838 for ; Mon, 4 Oct 2004 09:21:24 -0400 (EDT) Received: from rwcrmhc12.comcast.net ([216.148.227.85]) by ietf-mx.ietf.org with esmtp (Exim 4.33) id 1CESvS-0007cE-8i for ips@ietf.org; Mon, 04 Oct 2004 09:30:50 -0400 Received: from ivvt2dxrc11 (c-66-177-57-55.se.client2.attbi.com[66.177.57.55]) by comcast.net (rwcrmhc12) with SMTP id <2004100413205401400rfilie> (Authid: esquicksall); Mon, 4 Oct 2004 13:20:55 +0000 Message-ID: <000801c4aa14$fc2f1b70$0303a8c0@ivvt2dxrc11> From: "Eddy Quicksall" To: Date: Mon, 4 Oct 2004 09:20:56 -0400 MIME-Version: 1.0 X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 6.00.2800.1437 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2800.1441 X-Spam-Score: 0.2 (/) X-Scan-Signature: 5ebbf074524e58e662bc8209a6235027 Subject: [Ips] resending Data-in with S bit set X-BeenThere: ips@ietf.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: IP Storage List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Content-Type: multipart/mixed; boundary="===============1432106145==" Sender: ips-bounces@ietf.org Errors-To: ips-bounces@ietf.org X-Spam-Score: 0.6 (/) X-Scan-Signature: dbb8771284c7a36189745aa720dc20ab This is a multi-part message in MIME format. --===============1432106145== Content-Type: multipart/alternative; boundary="----=_NextPart_000_0005_01C4A9F3.74534DA0" This is a multi-part message in MIME format. ------=_NextPart_000_0005_01C4A9F3.74534DA0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable Can you confirm that the intent of 10.16.6. is as follows? =20 If the Initiator has requested for a Status SNACK with BegRun=3D0 and = RunLength=3D0 (i.e, all unacknowledged status pdus) and the target has = some Data-In PDUs carrying *status* that have not been acknowledged, the = target will *not* resend those Data-In PDUs carrying the status. =20 The unacknowledged queue looks like =20 response(statSN=3D5) ---> response(statSN=3D6) -----> Data-In w/S = bit(statSN=3D7) ----> response(statSN=3D8) =20 The target will resend pdus with statSN 5, 6 and 8. The Data-In with = statSN 7 will not be resent even though it sends status. =20 Eddy ------=_NextPart_000_0005_01C4A9F3.74534DA0 Content-Type: text/html; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable

Can you confirm that the = intent of=20 10.16.6. is as follows?

 

If the Initiator has = requested for a=20 Status SNACK with BegRun=3D0 and RunLength=3D0 (i.e, all unacknowledged = status pdus)=20 and the target has some Data-In PDUs carrying *status* that have not = been=20 acknowledged, the target will *not* resend those Data-In PDUs carrying = the=20 status.

 

The unacknowledged queue = looks=20 like

 

   response(statSN=3D5) = --->=20 response(statSN=3D6) -----> Data-In w/S bit(statSN=3D7) ---->=20 response(statSN=3D8)

 

The target will resend pdus = with statSN=20 5, 6 and 8. The Data-In with statSN 7 will not be resent even though it = sends=20 status.

 

Eddy

 

------=_NextPart_000_0005_01C4A9F3.74534DA0-- --===============1432106145== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline Content-Transfer-Encoding: 7bit _______________________________________________ Ips mailing list Ips@ietf.org https://www1.ietf.org/mailman/listinfo/ips --===============1432106145==-- From ips-bounces@ietf.org Mon Oct 4 10:45:47 2004 Received: from ietf-mx.ietf.org (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id KAA10528 for ; Mon, 4 Oct 2004 10:45:47 -0400 (EDT) Received: from megatron.ietf.org ([132.151.6.71]) by ietf-mx.ietf.org with esmtp (Exim 4.33) id 1CEUF8-00059o-CV for ips-web-archive@ietf.org; Mon, 04 Oct 2004 10:55:14 -0400 Received: from localhost.localdomain ([127.0.0.1] helo=megatron.ietf.org) by megatron.ietf.org with esmtp (Exim 4.32) id 1CEU4S-0005i4-Qr; Mon, 04 Oct 2004 10:44:12 -0400 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by megatron.ietf.org with esmtp (Exim 4.32) id 1CEU3h-0005ZV-3l for ips@megatron.ietf.org; Mon, 04 Oct 2004 10:43:29 -0400 Received: from ietf-mx.ietf.org (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id KAA10401 for ; Mon, 4 Oct 2004 10:43:22 -0400 (EDT) Received: from mtagate4.de.ibm.com ([195.212.29.153]) by ietf-mx.ietf.org with esmtp (Exim 4.33) id 1CEUCl-0004iy-U6 for ips@ietf.org; Mon, 04 Oct 2004 10:52:49 -0400 Received: from d12nrmr1607.megacenter.de.ibm.com (d12nrmr1607.megacenter.de.ibm.com [9.149.167.49]) by mtagate4.de.ibm.com (8.12.10/8.12.10) with ESMTP id i94EgmLi141430; Mon, 4 Oct 2004 14:42:48 GMT Received: from d12ml102.megacenter.de.ibm.com (d12av02.megacenter.de.ibm.com [9.149.165.228]) by d12nrmr1607.megacenter.de.ibm.com (8.12.10/NCO/VER6.6) with ESMTP id i94EglOL100132; Mon, 4 Oct 2004 16:42:47 +0200 In-Reply-To: <000801c4aa14$fc2f1b70$0303a8c0@ivvt2dxrc11> To: "Eddy Quicksall" Subject: Re: [Ips] resending Data-in with S bit set MIME-Version: 1.0 X-Mailer: Lotus Notes Build V70_M2_07222004 Beta 2NP July 22, 2004 From: Julian Satran Message-ID: Date: Mon, 4 Oct 2004 16:42:46 +0200 X-MIMETrack: Serialize by Router on D12ML102/12/M/IBM(Release 6.5.1| March 5, 2004) at 04/10/2004 16:42:48, Serialize complete at 04/10/2004 16:42:48 X-Spam-Score: 0.5 (/) X-Scan-Signature: 6cca30437e2d04f45110f2ff8dc1b1d5 Cc: ips@ietf.org X-BeenThere: ips@ietf.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: IP Storage List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Content-Type: multipart/mixed; boundary="===============0995970037==" Sender: ips-bounces@ietf.org Errors-To: ips-bounces@ietf.org X-Spam-Score: 0.5 (/) X-Scan-Signature: cf3becbbd6d1a45acbe2ffd4ab88bdc2 This is a multipart message in MIME format. --===============0995970037== Content-Type: multipart/alternative; boundary="=_alternative 005091B8C2256F23_=" This is a multipart message in MIME format. --=_alternative 005091B8C2256F23_= Content-Type: text/plain; charset="US-ASCII" Eddy, This is correct. The data-in PDUs don't have to be sent. The initiator has to request the data-in PDUs explicitly and MUST do so before it acks the status (that implicitly acks all the data and deletes any record of the command at target). Julo ips-bounces@ietf.org wrote on 04/10/2004 15:20:56: > Can you confirm that the intent of 10.16.6. is as follows? > > If the Initiator has requested for a Status SNACK with BegRun=0 and > RunLength=0 (i.e, all unacknowledged status pdus) and the target has > some Data-In PDUs carrying *status* that have not been acknowledged, > the target will *not* resend those Data-In PDUs carrying the status. > > The unacknowledged queue looks like > > response(statSN=5) ---> response(statSN=6) -----> Data-In w/S > bit(statSN=7) ----> response(statSN=8) > > The target will resend pdus with statSN 5, 6 and 8. The Data-In with > statSN 7 will not be resent even though it sends status. > > Eddy > _______________________________________________ > Ips mailing list > Ips@ietf.org > https://www1.ietf.org/mailman/listinfo/ips --=_alternative 005091B8C2256F23_= Content-Type: text/html; charset="US-ASCII"
Eddy,

This is correct.
The data-in PDUs don't have to be sent.
The initiator has to request the data-in PDUs explicitly and MUST do so before it acks the status (that implicitly acks all the data and deletes any record of the command at target).

Julo

ips-bounces@ietf.org wrote on 04/10/2004 15:20:56:

> Can you confirm that the intent of 10.16.6. is as follows?
>  
> If the Initiator has requested for a Status SNACK with BegRun=0 and
> RunLength=0 (i.e, all unacknowledged status pdus) and the target has
> some Data-In PDUs carrying *status* that have not been acknowledged,
> the target will *not* resend those Data-In PDUs carrying the status.
>  
> The unacknowledged queue looks like
>  
>    response(statSN=5) ---> response(statSN=6) -----> Data-In w/S
> bit(statSN=7) ----> response(statSN=8)
>  
> The target will resend pdus with statSN 5, 6 and 8. The Data-In with
> statSN 7 will not be resent even though it sends status.
>  
> Eddy
>  _______________________________________________
> Ips mailing list
> Ips@ietf.org
> https://www1.ietf.org/mailman/listinfo/ips
 --=_alternative 005091B8C2256F23_=-- --===============0995970037== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline Content-Transfer-Encoding: 7bit _______________________________________________ Ips mailing list Ips@ietf.org https://www1.ietf.org/mailman/listinfo/ips --===============0995970037==-- From ips-bounces@ietf.org Mon Oct 4 12:00:01 2004 Received: from ietf-mx.ietf.org (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id MAA16065 for ; Mon, 4 Oct 2004 12:00:01 -0400 (EDT) Received: from megatron.ietf.org ([132.151.6.71]) by ietf-mx.ietf.org with esmtp (Exim 4.33) id 1CEVOz-0005UJ-0X for ips-web-archive@ietf.org; Mon, 04 Oct 2004 12:09:29 -0400 Received: from localhost.localdomain ([127.0.0.1] helo=megatron.ietf.org) by megatron.ietf.org with esmtp (Exim 4.32) id 1CEVAR-00076f-6b; Mon, 04 Oct 2004 11:54:27 -0400 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by megatron.ietf.org with esmtp (Exim 4.32) id 1CEV4i-0006Nw-5s for ips@megatron.ietf.org; Mon, 04 Oct 2004 11:48:32 -0400 Received: from ietf-mx.ietf.org (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id LAA15341 for ; Mon, 4 Oct 2004 11:48:29 -0400 (EDT) Received: from taurus.voltaire.com ([212.143.27.73]) by ietf-mx.ietf.org with esmtp (Exim 4.33) id 1CEVDm-0003Nd-Es for ips@ietf.org; Mon, 04 Oct 2004 11:57:57 -0400 X-MimeOLE: Produced By Microsoft Exchange V6.5.7226.0 Content-class: urn:content-classes:message MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable Subject: RE: [Ips] DA+iSER Flow Control, Limiting number of oustanding cmds attarget Date: Mon, 4 Oct 2004 17:47:55 +0200 Message-ID: Thread-Topic: [Ips] DA+iSER Flow Control, Limiting number of oustanding cmds attarget Thread-Index: AcSqD+2D/oEhy2gyTC6Ywuar/SeZTAAFXYFg From: "Alex Nezhinsky" To: "Julian Satran" , "Caitlin Bestler" X-Spam-Score: 1.0 (+) X-Scan-Signature: d2b46e3b2dfbff2088e0b72a54104985 Content-Transfer-Encoding: quoted-printable Cc: ips@ietf.org, Mike Ko , "Mallikarjun C." X-BeenThere: ips@ietf.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: IP Storage List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Sender: ips-bounces@ietf.org Errors-To: ips-bounces@ietf.org X-Spam-Score: 0.0 (/) X-Scan-Signature: 03169bfe4792634a390035a01a6c6d2f Content-Transfer-Encoding: quoted-printable > What I believe Alex is asking for, and I asked for earlier, is a basis for > making that estimate that is based on a specification, and not heuristic > measurements of N leading products. Absolutely.=20 Immediate commands is only one type of "out-of-band" PDUs. NOPs is another one which is relevant in the context of flow cotrol itself. If the value of MaxCmdSN changes at the target when no PDUs are about to be sent, iSCSI suggests sending a NOP-IN in order to communicate the new MaxCmdSN value. So the initiator should be ready to get NOPs at any time.=20 If response PDU is lost for whatever reason, the initiator will send NOP-OUT, this means that the target should post additional buffer for each command it receives, just in case. Text requests are limited to 1 outstanding, while task mgmt requests are not. =20 > Dropping the session when an initiator has sent more than the allowed number=20 > of commands+2 seems perfectly reasonable.=20 Why not use a simple flow control scheme, in which, for example, iSER header has=20 one additional field designating the number of recv buffers posted in addition to=20 those implied by MaxCmdSn. That is, it's not just commands+2, but F(commands)+X, where X is communicated in each iSER header and F is a function specific for initiator, target.=20 What the specification could also provide is, for example, a minimal value for X, similar to "MUST be able to handle at least one immediate task management command and one immediate non-task-management iSCSI command per connection at any time". Alexander Nezhinsky ________________________________ From: Julian Satran [mailto:Julian_Satran@il.ibm.com]=20 Sent: Monday, October 04, 2004 2:42 PM To: Caitlin Bestler Cc: Alex Nezhinsky; Mallikarjun C.; ips@ietf.org; Mike Ko; ips@ietf.org Subject: RE: [Ips] DA+iSER Flow Control, Limiting number of oustanding cmds attarget =09 =09 This is an iSER specific requirement and an iSER specific key may help. I am afraid however that even this might not be foolproof as an initiator may fill any number of buffers with non task management functions and then claim that the target is not iSCSI compliant as it does not accept an immediate task management function command.=20 Or even worse - as RDMA has no way of distinguishing immediate from non-immediate commands an initiator can clog the pipe with non-immediate commands before the target will feel abused.=20 Dropping the session when an initiator has sent more than the allowed number of commands+2 seems perfectly reasonable.=20 =09 Julo=20 =09 ________________________________ =09 "Caitlin Bestler" =20 04/10/04 12:18=20 =09 To Julian Satran/Haifa/IBM@IBMIL=20 cc "Alex Nezhinsky" , ips@ietf.org, "Mike Ko" , "Mallikarjun C." =20 Subject RE: [Ips] DA+iSER Flow Control, Limiting number of oustanding cmds attarget =09 Julian Satran said: > Alex, > > The relevant text in RFC 3720 - 3.2.2.1 is: > > The number of commands used for immediate delivery is not limited and > their delivery for execution is not acknowledged through the > numbering scheme. Immediate commands MAY be rejected by the iSCSI > target layer due to a lack of resources. An iSCSI target MUST be > able to handle at least one immediate task management command and one > immediate non-task-management iSCSI command per connection at any > time. > > I hope this helps. > > Hag Sameach, > Julo > =09 =09 The iSCSI target "MAY" be allowed to discard excess immediate commands, but that doesn't help the RDMA layer that needs a buffer for an untagged message, or it will be required to terminate the connection. =09 Because iSER maps both the flow-controlled and non-flow-controlled messages to the same connection, lack of a buffer for an immediate command places non-immediate messages at risk. =09 Obviously, creating an extra connection for infrequent commands is not the correct solution. But a correct solution does require that an iSER implementation has *something* to enable it to determine how many buffers to pre-post to the RDMAP layer. It already has the answer for non-immediate commands, which is the bulk of the traffic. It is also relatively easy to make some very good estimates on how many would be required for immediate commands. =09 But interoperability of RFC compliant implementations should be based upon their RFC compliance, not on compatible guestimations. =09 What I believe Alex is asking for, and I asked for earlier, is a basis for making that estimate that is based on a specification, and not heuristic measurements of N leading products. =09 -- Caitlin Bestler http://asomi.com/ =09 _______________________________________________ Ips mailing list Ips@ietf.org https://www1.ietf.org/mailman/listinfo/ips From ips-bounces@ietf.org Mon Oct 4 15:06:47 2004 Received: from ietf-mx.ietf.org (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id PAA04055 for ; Mon, 4 Oct 2004 15:06:47 -0400 (EDT) Received: from megatron.ietf.org ([132.151.6.71]) by ietf-mx.ietf.org with esmtp (Exim 4.33) id 1CEYJk-0005Sp-2j for ips-web-archive@ietf.org; Mon, 04 Oct 2004 15:16:16 -0400 Received: from localhost.localdomain ([127.0.0.1] helo=megatron.ietf.org) by megatron.ietf.org with esmtp (Exim 4.32) id 1CEY8Q-0003pd-0u; Mon, 04 Oct 2004 15:04:34 -0400 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by megatron.ietf.org with esmtp (Exim 4.32) id 1CEY1m-0006Jh-P5 for ips@megatron.ietf.org; Mon, 04 Oct 2004 14:57:43 -0400 Received: from ietf-mx.ietf.org (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id OAA02958 for ; Mon, 4 Oct 2004 14:57:40 -0400 (EDT) Received: from palrel12.hp.com ([156.153.255.237]) by ietf-mx.ietf.org with esmtp (Exim 4.33) id 1CEYAu-0003pQ-Oi for ips@ietf.org; Mon, 04 Oct 2004 15:07:09 -0400 Received: from cacexg13.americas.cpqcorp.net (cacexg13.americas.cpqcorp.net [16.92.1.76]) by palrel12.hp.com (Postfix) with ESMTP id 442E940ADE6; Mon, 4 Oct 2004 11:57:40 -0700 (PDT) Received: from cacexc08.americas.cpqcorp.net ([16.92.1.35]) by cacexg13.americas.cpqcorp.net with Microsoft SMTPSVC(6.0.3790.0); Mon, 4 Oct 2004 11:57:40 -0700 X-MimeOLE: Produced By Microsoft Exchange V6.5.7226.0 Content-class: urn:content-classes:message MIME-Version: 1.0 Subject: RE: [Ips] Initiator CHAP authentication of target? Date: Mon, 4 Oct 2004 11:57:38 -0700 Message-ID: <95AC37E4A7C3304D94DB6AF7F64FA1B50151400D@cacexc08.americas.cpqcorp.net> Thread-Topic: [Ips] Initiator CHAP authentication of target? Thread-Index: AcSqBvLvjpxemBPVQLCv8Vmh95IbtAAPHCmw From: "Krueger, Marjorie" To: "Julian Satran" X-OriginalArrivalTime: 04 Oct 2004 18:57:40.0095 (UTC) FILETIME=[058EA8F0:01C4AA44] X-Spam-Score: 0.7 (/) X-Scan-Signature: bd8a74b81c71f965ca7918b90d1c49c0 Cc: ips@ietf.org, William Studenmund X-BeenThere: ips@ietf.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: IP Storage List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Content-Type: multipart/mixed; boundary="===============0880284916==" Sender: ips-bounces@ietf.org Errors-To: ips-bounces@ietf.org X-Spam-Score: 0.7 (/) X-Scan-Signature: 79bb66f827e54e9d5c5c7f1f9d645608 This is a multi-part message in MIME format. --===============0880284916== Content-class: urn:content-classes:message Content-Type: multipart/alternative; boundary="----_=_NextPart_001_01C4AA44.04E0CA2E" This is a multi-part message in MIME format. ------_=_NextPart_001_01C4AA44.04E0CA2E Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable While it may not be "detectable" by the target, it certainly seems like "expected behavior" (ie, following the protocol) is that the initiator MUST use the CHAP_N and CHAP_R supplied in the login response - would you agree? When testing an initiator implementation, if one can prove that a certain initiator isn't using the values supplied, one can say that the initiator is not correctly implemented? =20 -Marjorie ________________________________ From: Julian Satran [mailto:Julian_Satran@il.ibm.com]=20 Sent: Monday, October 04, 2004 4:40 AM To: Krueger, Marjorie Cc: ips@ietf.org; William Studenmund Subject: RE: [Ips] Initiator CHAP authentication of target? =09 =09 Marjorie,=20 =09 Maybe I am not understanding clearly the "substance" of your question but if you are questioning the validity of an initiator not using the data supplied to authenticate the target - that is "legal" in the sense that the protocol has no way of enforcing the check in CHAP (i.e., there is no returned value that will irrefutably prove that the initiator did check).=20 I would not use this protocol in a money exchange transaction! The weaknesses of CHAP have been discussed at some length on the list.=20 =09 Regards,=20 Julo=20 =09 =09 =09 "Krueger, Marjorie" =20 Sent by: ips-bounces@ietf.org=20 01/10/04 01:28=20 To "William Studenmund" =20 cc ips@ietf.org=20 Subject RE: [Ips] Initiator CHAP authentication of target? =09 It appears a certain initiator is ignoring CHAP_N returned by the target and just using the target node name for CHAP_N regardless of what the target sent in the login sequence. I think it seems like invalid behavior but I just wanted to check with this audience. =09 Thanks Marjorie =09 > -----Original Message----- > From: William Studenmund [mailto:wrstuden@wasabisystems.com]=20 > Sent: Thursday, September 30, 2004 3:32 PM > To: Krueger, Marjorie > Cc: ips@ietf.org > Subject: Re: [Ips] Initiator CHAP authentication of target? >=20 > On Sep 30, 2004, at 3:22 PM, Krueger, Marjorie wrote: >=20 > > So if > > > >> or, if it requires target authentication, with: > >> > >> CHAP_N=3D CHAP_R=3D CHAP_I=3D CHAP_C=3D > >> > >> If the initiator authentication fails, the target MUST answer=20 > >> with a > >> Login reject with "Authentication Failure" status. > >> Otherwise, if the > >> initiator required target authentication, the target=20 > MUST either > >> answer with a Login reject with "Authentication=20 > Failure" or reply > >> with: > >> > >> CHAP_N=3D CHAP_R=3D > > > > And the target replys with CHAP_N that is not defined and the=20 > > initiator accepts that (accepts anything) is that valid=20 > behavior? In=20 > > otherwords, if the initiator ignores CHAP_N and only uses=20 > the target=20 > > node name and CHAP_R to authenticate, is that valid behavior? =20 > > Shouldn't the target be able to force the use of CHAP_N? >=20 > I'm confused. The initiator can't NOT use CHAP_N. Thus the=20 > target shouldn't be able to force something that's required. :-) >=20 > To quote the paragraph after the one above: >=20 > Where N, (A,A1,A2), I, C, and R are (correspondingly) the Name, > Algorithm, Identifier, Challenge, and Response as defined in > [RFC1994], N is a text string, A,A1,A2, and I are=20 > numbers, and C and > R are large-binary-values and their binary length (not=20 > the length of > the character string that represents them in encoded=20 > form) MUST not > exceed 1024 bytes. >=20 > So an initiator that just uses the target's name for CHAP=20 > purposes seems broken. >=20 > Take care, >=20 > Bill >=20 =09 _______________________________________________ Ips mailing list Ips@ietf.org https://www1.ietf.org/mailman/listinfo/ips =09 =09 ------_=_NextPart_001_01C4AA44.04E0CA2E Content-Type: text/html; charset="us-ascii" Content-Transfer-Encoding: quoted-printable
While it may not be "detectable" by the target, = it=20 certainly seems like "expected behavior" (ie, following the protocol) is = that=20 the initiator MUST use the CHAP_N and CHAP_R supplied in the login = response -=20 would you agree?  When testing an initiator implementation, if one = can=20 prove that a certain initiator isn't using the values supplied, one can = say that=20 the initiator is not correctly implemented?
 
-Marjorie

From: Julian Satran=20 [mailto:Julian_Satran@il.ibm.com]
Sent: Monday, October 04, = 2004=20 4:40 AM
To: Krueger, Marjorie
Cc: ips@ietf.org; = William=20 Studenmund
Subject: RE: [Ips] Initiator CHAP authentication = of=20 target?


Marjorie, =

Maybe I am not understanding clearly the = "substance" of=20 your question but if you are questioning the validity of an initiator = not=20 using the data supplied to authenticate the target - that is "legal" = in the=20 sense that the protocol has no way of enforcing the check in CHAP = (i.e., there=20 is no returned value that will irrefutably prove that the initiator = did=20 check).
I would not use = this protocol=20 in a money exchange transaction! The weaknesses of CHAP have been = discussed at=20 some length on the list.

Regards,
Julo=20


"Krueger, = Marjorie"=20 <marjorie.krueger@hp.com>
Sent by: ips-bounces@ietf.org=20

01/10/04 01:28

To
"William Studenmund"=20 <wrstuden@wasabisystems.com>=20
cc
ips@ietf.org=20
Subject
RE: [Ips] Initiator = CHAP=20 authentication of target?

=




It appears a certain initiator is ignoring CHAP_N = returned by the=20 target
and just using the target node name for CHAP_N regardless of = what=20 the
target sent in the login sequence.  I think it seems like=20 invalid
behavior but I just wanted to check with this=20 audience.

Thanks
Marjorie

> -----Original=20 Message-----
> From: William Studenmund=20 [mailto:wrstuden@wasabisystems.com]
> Sent: Thursday, September = 30,=20 2004 3:32 PM
> To: Krueger, Marjorie
> Cc: = ips@ietf.org
>=20 Subject: Re: [Ips] Initiator CHAP authentication of target?
> =
>=20 On Sep 30, 2004, at 3:22 PM, Krueger, Marjorie wrote:
>
> = > So=20 if
> >
> >>     or, if it requires = target=20 authentication, with:
> >>
> >>     =  =20  CHAP_N=3D<N> CHAP_R=3D<R> CHAP_I=3D<I>=20 CHAP_C=3D<C>
> >>
> >>     If = the=20 initiator authentication fails, the target MUST answer
> = >> with=20 a
> >>     Login reject with "Authentication = Failure"=20 status.
> >> Otherwise, if the
> >>   =  =20 initiator required target authentication, the target
> MUST=20 either
> >>     answer with a Login reject with=20 "Authentication
> Failure" or reply
> >>   =  =20 with:
> >>
> >>      =20  CHAP_N=3D<N> CHAP_R=3D<R>
> >
> > = And the=20 target replys with CHAP_N that is not defined and the
> > = initiator=20 accepts that (accepts anything) is that valid
> behavior? =  In=20
> > otherwords, if the initiator ignores CHAP_N and only = uses=20
> the target
> > node name and CHAP_R to = authenticate, is=20 that valid behavior?  
> > Shouldn't the target be able = to force=20 the use of CHAP_N?
>
> I'm confused. The initiator can't = NOT use=20 CHAP_N. Thus the
> target shouldn't be able to force something = that's=20 required. :-)
>
> To quote the paragraph after the one=20 above:
>
>     Where N, (A,A1,A2), I, C, and R = are=20 (correspondingly) the Name,
>     Algorithm, = Identifier,=20 Challenge, and Response as defined in
>     [RFC1994], = N is a=20 text string, A,A1,A2, and I are
> numbers, and C and
> =  =20   R are large-binary-values and their binary length (not
> = the=20 length of
>     the character string that represents = them in=20 encoded
> form) MUST not
>     exceed 1024=20 bytes.
>
> So an initiator that just uses the target's = name for=20 CHAP
> purposes seems broken.
>
> Take = care,
>=20
> Bill
>=20

_______________________________________________
Ips mailing = = list
Ips@ietf.org
https://www1.ietf.org/mailman/listinfo/ips

------_=_NextPart_001_01C4AA44.04E0CA2E-- --===============0880284916== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline Content-Transfer-Encoding: 7bit _______________________________________________ Ips mailing list Ips@ietf.org https://www1.ietf.org/mailman/listinfo/ips --===============0880284916==-- From ips-bounces@ietf.org Mon Oct 4 15:17:15 2004 Received: from ietf-mx.ietf.org (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id PAA05327 for ; Mon, 4 Oct 2004 15:17:15 -0400 (EDT) Received: from megatron.ietf.org ([132.151.6.71]) by ietf-mx.ietf.org with esmtp (Exim 4.33) id 1CEYTr-00077s-Q3 for ips-web-archive@ietf.org; Mon, 04 Oct 2004 15:26:44 -0400 Received: from localhost.localdomain ([127.0.0.1] helo=megatron.ietf.org) by megatron.ietf.org with esmtp (Exim 4.32) id 1CEY9Q-0004cr-Di; Mon, 04 Oct 2004 15:05:36 -0400 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by megatron.ietf.org with esmtp (Exim 4.32) id 1CEY4L-0008N7-FX for ips@megatron.ietf.org; Mon, 04 Oct 2004 15:00:21 -0400 Received: from ietf-mx.ietf.org (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id PAA03127 for ; Mon, 4 Oct 2004 15:00:19 -0400 (EDT) Received: from e4.ny.us.ibm.com ([32.97.182.104]) by ietf-mx.ietf.org with esmtp (Exim 4.33) id 1CEYDT-000466-RN for ips@ietf.org; Mon, 04 Oct 2004 15:09:48 -0400 Received: from northrelay02.pok.ibm.com (northrelay02.pok.ibm.com [9.56.224.150]) by e4.ny.us.ibm.com (8.12.10/8.12.9) with ESMTP id i94Ixgvx517298; Mon, 4 Oct 2004 14:59:42 -0400 Received: from d01ml604.pok.ibm.com (d01av02.pok.ibm.com [9.56.224.216]) by northrelay02.pok.ibm.com (8.12.10/NCO/VER6.6) with ESMTP id i94J0tNR108886; Mon, 4 Oct 2004 15:00:55 -0400 In-Reply-To: Importance: Normal MIME-Version: 1.0 To: "Alex Nezhinsky" Subject: RE: [Ips] DA+iSER Flow Control, Limiting number of oustanding cmds attarget X-Mailer: Lotus Notes Release 6.5.2 June 01, 2004 Message-ID: From: Mike Ko Date: Mon, 4 Oct 2004 11:59:38 -0700 X-MIMETrack: Serialize by Router on D01ML604/01/M/IBM(Release 6.51HF582 | September 27, 2004) at 10/04/2004 14:59:41, Serialize complete at 10/04/2004 14:59:41 Content-Type: text/plain; charset="US-ASCII" X-Spam-Score: 0.0 (/) X-Scan-Signature: ec7c6dab5a62df223002ae71b5179d41 Cc: Caitlin Bestler , ips@ietf.org, Julian Satran , "Mallikarjun C." X-BeenThere: ips@ietf.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: IP Storage List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Sender: ips-bounces@ietf.org Errors-To: ips-bounces@ietf.org X-Spam-Score: 0.0 (/) X-Scan-Signature: 311e798ce51dbeacf5cdfcc8e9fda21b Alex, This is the gist of the iSER "flow control" related discussion on the IPS forum in July/August of 2003. >From the initiator to the target, for control-type PDUs where the immediate flag is not set, the CmdSN and MaxCmdSN mechanism in iSCSI already provides the flow control needed. For unsolicited SCSI Data-out PDUs, FirstBurstLength can be used to estimate the amount of buffering required. What remains that are not covered by any existing flow control are PDUs where the immediate flag is set or SNACK Request (which is not needed in iSER). >From the target to the initiator, unsocited NOP-in are regulated by NOP-out. The unsolicited PDUs which are not covered by any existing flow control are Async Messages. One of the solution proposed is to set a declared limit on the number of such unsolicited requests. The next question is how does each side determine when is the limit replenished. One possible solution is as follows: For the initiator, the ExpCmdSN from the target will be used to advance the credit. After sending PDUs with the immediate flag set, when the initiator sends any iSCSI control-type PDU with a new CmdSN without the immediate flag, if the target responds with an iSCSI control-type PDU with ExpCmdSN set to CmdSN + 1, then credit at the initiator is restored to its full value. For the target, the ExpStatSN from the initiator will be used to advance the credit. When the initiator sends any PDU which contains the ExpStatSN that acknowledges the StatSN in the Async Message, the number of credits at the target used up by that Async Message will be restored. Mike To: "Julian Satran" , "Caitlin Bestler" cc: "Mallikarjun C." , "Mike Ko" , Subject: RE: [Ips] DA+iSER Flow Control, Limiting number of oustanding cmds attarget > What I believe Alex is asking for, and I asked for earlier, is a basis for > making that estimate that is based on a specification, and not heuristic > measurements of N leading products. Absolutely. Immediate commands is only one type of "out-of-band" PDUs. NOPs is another one which is relevant in the context of flow cotrol itself. If the value of MaxCmdSN changes at the target when no PDUs are about to be sent, iSCSI suggests sending a NOP-IN in order to communicate the new MaxCmdSN value. So the initiator should be ready to get NOPs at any time. If response PDU is lost for whatever reason, the initiator will send NOP-OUT, this means that the target should post additional buffer for each command it receives, just in case. Text requests are limited to 1 outstanding, while task mgmt requests are not. > Dropping the session when an initiator has sent more than the allowed number > of commands+2 seems perfectly reasonable. Why not use a simple flow control scheme, in which, for example, iSER header has one additional field designating the number of recv buffers posted in addition to those implied by MaxCmdSn. That is, it's not just commands+2, but F(commands)+X, where X is communicated in each iSER header and F is a function specific for initiator, target. What the specification could also provide is, for example, a minimal value for X, similar to "MUST be able to handle at least one immediate task management command and one immediate non-task-management iSCSI command per connection at any time". Alexander Nezhinsky ________________________________ From: Julian Satran [mailto:Julian_Satran@il.ibm.com] Sent: Monday, October 04, 2004 2:42 PM To: Caitlin Bestler Cc: Alex Nezhinsky; Mallikarjun C.; ips@ietf.org; Mike Ko; ips@ietf.org Subject: RE: [Ips] DA+iSER Flow Control, Limiting number of oustanding cmds attarget This is an iSER specific requirement and an iSER specific key may help. I am afraid however that even this might not be foolproof as an initiator may fill any number of buffers with non task management functions and then claim that the target is not iSCSI compliant as it does not accept an immediate task management function command. Or even worse - as RDMA has no way of distinguishing immediate from non-immediate commands an initiator can clog the pipe with non-immediate commands before the target will feel abused. Dropping the session when an initiator has sent more than the allowed number of commands+2 seems perfectly reasonable. Julo ________________________________ "Caitlin Bestler" 04/10/04 12:18 To Julian Satran/Haifa/IBM@IBMIL cc "Alex Nezhinsky" , ips@ietf.org, "Mike Ko" , "Mallikarjun C." Subject RE: [Ips] DA+iSER Flow Control, Limiting number of oustanding cmds attarget Julian Satran said: > Alex, > > The relevant text in RFC 3720 - 3.2.2.1 is: > > The number of commands used for immediate delivery is not limited and > their delivery for execution is not acknowledged through the > numbering scheme. Immediate commands MAY be rejected by the iSCSI > target layer due to a lack of resources. An iSCSI target MUST be > able to handle at least one immediate task management command and one > immediate non-task-management iSCSI command per connection at any > time. > > I hope this helps. > > Hag Sameach, > Julo > The iSCSI target "MAY" be allowed to discard excess immediate commands, but that doesn't help the RDMA layer that needs a buffer for an untagged message, or it will be required to terminate the connection. Because iSER maps both the flow-controlled and non-flow-controlled messages to the same connection, lack of a buffer for an immediate command places non-immediate messages at risk. Obviously, creating an extra connection for infrequent commands is not the correct solution. But a correct solution does require that an iSER implementation has *something* to enable it to determine how many buffers to pre-post to the RDMAP layer. It already has the answer for non-immediate commands, which is the bulk of the traffic. It is also relatively easy to make some very good estimates on how many would be required for immediate commands. But interoperability of RFC compliant implementations should be based upon their RFC compliance, not on compatible guestimations. What I believe Alex is asking for, and I asked for earlier, is a basis for making that estimate that is based on a specification, and not heuristic measurements of N leading products. -- Caitlin Bestler http://asomi.com/ _______________________________________________ Ips mailing list Ips@ietf.org https://www1.ietf.org/mailman/listinfo/ips From ips-bounces@ietf.org Mon Oct 4 15:49:13 2004 Received: from ietf-mx.ietf.org (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id PAA09087 for ; Mon, 4 Oct 2004 15:49:13 -0400 (EDT) Received: from megatron.ietf.org ([132.151.6.71]) by ietf-mx.ietf.org with esmtp (Exim 4.33) id 1CEYyo-0003XE-6C for ips-web-archive@ietf.org; Mon, 04 Oct 2004 15:58:42 -0400 Received: from localhost.localdomain ([127.0.0.1] helo=megatron.ietf.org) by megatron.ietf.org with esmtp (Exim 4.32) id 1CEYhs-0005dh-H9; Mon, 04 Oct 2004 15:41:12 -0400 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by megatron.ietf.org with esmtp (Exim 4.32) id 1CEYSB-0006s1-LZ for ips@megatron.ietf.org; Mon, 04 Oct 2004 15:25:01 -0400 Received: from ietf-mx.ietf.org (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id PAA06443 for ; Mon, 4 Oct 2004 15:24:57 -0400 (EDT) Received: from e34.co.us.ibm.com ([32.97.110.132]) by ietf-mx.ietf.org with esmtp (Exim 4.33) id 1CEYbJ-0008Px-J2 for ips@ietf.org; Mon, 04 Oct 2004 15:34:26 -0400 Received: from westrelay02.boulder.ibm.com (westrelay02.boulder.ibm.com [9.17.195.11]) by e34.co.us.ibm.com (8.12.10/8.12.9) with ESMTP id i94JNGsB061456; Mon, 4 Oct 2004 15:23:16 -0400 Received: from d03nm115.boulder.ibm.com (d03av02.boulder.ibm.com [9.17.195.168]) by westrelay02.boulder.ibm.com (8.12.10/NCO/VER6.6) with ESMTP id i94JN561188520; Mon, 4 Oct 2004 13:23:05 -0600 In-Reply-To: To: "Alex Nezhinsky" MIME-Version: 1.0 Subject: RE: [Ips] DA+iSER Flow Control, Limiting number of oustanding cmds attarget X-Mailer: Lotus Notes Release 6.0.2CF1 June 9, 2003 From: John Hufferd Message-ID: Date: Mon, 4 Oct 2004 12:23:03 -0700 X-MIMETrack: Serialize by Router on D03NM115/03/M/IBM(Release 6.51HF338 | June 21, 2004) at 10/04/2004 13:23:15, Serialize complete at 10/04/2004 13:23:15 X-Spam-Score: 1.4 (+) X-Scan-Signature: 578029c71870398e30dc7af6a1ae528d Cc: Caitlin Bestler , ips@ietf.org, Mike Ko , Julian Satran , "Mallikarjun C." X-BeenThere: ips@ietf.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: IP Storage List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Content-Type: multipart/mixed; boundary="===============0950110242==" Sender: ips-bounces@ietf.org Errors-To: ips-bounces@ietf.org X-Spam-Score: 0.5 (/) X-Scan-Signature: 4b6c13497db1e4e7b275c86cfcdad7bb This is a multipart message in MIME format. --===============0950110242== Content-Type: multipart/alternative; boundary="=_alternative 006A67CD88256F23_=" This is a multipart message in MIME format. --=_alternative 006A67CD88256F23_= Content-Type: text/plain; charset="US-ASCII" The important thing to understand is that you need to queue enough buffers so that the suddenly chatty Initiator can proceed with out an impact to latency. In most of the robust implementations, which will be controlling hundreds/thousands of logical connections, the approprate consideration is the fact that regardless of the number of logical connections, there can not be more data than the link is capable of handling. Therefore, we designed the concept of Shared Receive Queue. The purpose was to have a method where the number of buffers that were needed to handle the link was all that was required. Which of course meant that there needed to be an ongoing active buffer replenishment technique. Which is similar to what is done in other SCSI environments. When ever one attempts to work this problem with non shared receive queues, the problem gets out of hand very quickly. With or without some additional form of end to end credit, the Total number of buffers that is needed becomes the sum of the buffers needed for all logical connections, since at any time they could burst (or desire to burst) at full link rates. So the idea of having to add a few extra, for each connection seemed to be only useful for very limited implementations, and having flow control above what iSCSI offered seemed to be a needless impact on latency. Even then the sum of all the credits (meaning backed by buffers) would be a lot more than what is needed to match the link speed. So based on the experience of the iSER designers we did not like the latency impact or the total buffer impact of using NON Shared Receive Queues. I believe your questions were in the area of attempting to implement iSER without Shared Receive Queues, and I feel that approach is problematical in itself, and will cause a much larger impact on the buffer space needed within the Storage Controller. On the other hand, if you have an implementation that only has a very few Logical connections, there is probably not an important problem of queueing a number of extra buffers. Lets use a Tape Unit as an example, any specific drive needs to be able to have enough buffers to match the speed of the link with a single Initiator. If you are at distance, and attempting to respond appropriately in some manner with credits, you may very well find that you either cause needless latency impact, or the number of buffers needed to avoid the latency, matches what you need for the maximum line rate anyway. When you consider what is needed both in the primary storage focus which is 100s - 1000s of connections, to the needs for an "at distance" small number of connections, the iSER spec came up with the right balance and additional flow control seems inappropriate. . . John L. Hufferd Senior Technical Staff Member (STSM) IBM/System Group, San Jose CA Main Office: (408) 256-0403, Tie: 276-0403, eFax: (408) 904-4688 Alt Office: (408) 997-6136, Cell: (408) 499-9702 Internet Address: hufferd@us.ibm.com "Alex Nezhinsky" Sent by: ips-bounces@ietf.org 10/04/2004 08:47 AM To "Julian Satran" , "Caitlin Bestler" cc ips@ietf.org, Mike Ko , "Mallikarjun C." Subject RE: [Ips] DA+iSER Flow Control, Limiting number of oustanding cmds attarget > What I believe Alex is asking for, and I asked for earlier, is a basis for > making that estimate that is based on a specification, and not heuristic > measurements of N leading products. Absolutely. Immediate commands is only one type of "out-of-band" PDUs. NOPs is another one which is relevant in the context of flow cotrol itself. If the value of MaxCmdSN changes at the target when no PDUs are about to be sent, iSCSI suggests sending a NOP-IN in order to communicate the new MaxCmdSN value. So the initiator should be ready to get NOPs at any time. If response PDU is lost for whatever reason, the initiator will send NOP-OUT, this means that the target should post additional buffer for each command it receives, just in case. Text requests are limited to 1 outstanding, while task mgmt requests are not. > Dropping the session when an initiator has sent more than the allowed number > of commands+2 seems perfectly reasonable. Why not use a simple flow control scheme, in which, for example, iSER header has one additional field designating the number of recv buffers posted in addition to those implied by MaxCmdSn. That is, it's not just commands+2, but F(commands)+X, where X is communicated in each iSER header and F is a function specific for initiator, target. What the specification could also provide is, for example, a minimal value for X, similar to "MUST be able to handle at least one immediate task management command and one immediate non-task-management iSCSI command per connection at any time". Alexander Nezhinsky ________________________________ From: Julian Satran [mailto:Julian_Satran@il.ibm.com] Sent: Monday, October 04, 2004 2:42 PM To: Caitlin Bestler Cc: Alex Nezhinsky; Mallikarjun C.; ips@ietf.org; Mike Ko; ips@ietf.org Subject: RE: [Ips] DA+iSER Flow Control, Limiting number of oustanding cmds attarget This is an iSER specific requirement and an iSER specific key may help. I am afraid however that even this might not be foolproof as an initiator may fill any number of buffers with non task management functions and then claim that the target is not iSCSI compliant as it does not accept an immediate task management function command. Or even worse - as RDMA has no way of distinguishing immediate from non-immediate commands an initiator can clog the pipe with non-immediate commands before the target will feel abused. Dropping the session when an initiator has sent more than the allowed number of commands+2 seems perfectly reasonable. Julo ________________________________ "Caitlin Bestler" 04/10/04 12:18 To Julian Satran/Haifa/IBM@IBMIL cc "Alex Nezhinsky" , ips@ietf.org, "Mike Ko" , "Mallikarjun C." Subject RE: [Ips] DA+iSER Flow Control, Limiting number of oustanding cmds attarget Julian Satran said: > Alex, > > The relevant text in RFC 3720 - 3.2.2.1 is: > > The number of commands used for immediate delivery is not limited and > their delivery for execution is not acknowledged through the > numbering scheme. Immediate commands MAY be rejected by the iSCSI > target layer due to a lack of resources. An iSCSI target MUST be > able to handle at least one immediate task management command and one > immediate non-task-management iSCSI command per connection at any > time. > > I hope this helps. > > Hag Sameach, > Julo > The iSCSI target "MAY" be allowed to discard excess immediate commands, but that doesn't help the RDMA layer that needs a buffer for an untagged message, or it will be required to terminate the connection. Because iSER maps both the flow-controlled and non-flow-controlled messages to the same connection, lack of a buffer for an immediate command places non-immediate messages at risk. Obviously, creating an extra connection for infrequent commands is not the correct solution. But a correct solution does require that an iSER implementation has *something* to enable it to determine how many buffers to pre-post to the RDMAP layer. It already has the answer for non-immediate commands, which is the bulk of the traffic. It is also relatively easy to make some very good estimates on how many would be required for immediate commands. But interoperability of RFC compliant implementations should be based upon their RFC compliance, not on compatible guestimations. What I believe Alex is asking for, and I asked for earlier, is a basis for making that estimate that is based on a specification, and not heuristic measurements of N leading products. -- Caitlin Bestler http://asomi.com/ _______________________________________________ Ips mailing list Ips@ietf.org https://www1.ietf.org/mailman/listinfo/ips --=_alternative 006A67CD88256F23_= Content-Type: text/html; charset="US-ASCII"
The important thing to understand is that you need to queue enough buffers so that the suddenly chatty Initiator can proceed with out an impact to latency.   In most of the robust implementations, which will be controlling hundreds/thousands of logical connections, the approprate consideration is the fact that regardless of the number of logical connections, there can not be more data than the link is capable of handling.  Therefore, we designed the concept of Shared Receive Queue.  The purpose was to have a method where the number of buffers that were needed to handle the link was all that was required.  Which of course meant that there needed to be an ongoing active buffer replenishment technique.  Which is similar to what is done in other SCSI environments.  

When ever one attempts to work this problem with non shared receive queues, the problem gets out of hand very quickly.  With or without some additional form of end to end credit, the Total number of buffers that is needed becomes the sum of the buffers needed for all logical connections, since at any time they could burst (or desire to burst) at full link rates.  So the idea of having to add a few extra, for each connection seemed to be only useful for very limited implementations, and having flow control above what iSCSI offered seemed to be a needless impact on latency.  Even then the sum of all the credits (meaning backed by buffers) would be a lot more than what is needed to match the link speed.

So based on the experience of the iSER designers we did not like the latency impact or the total buffer impact of using NON Shared Receive Queues.  I believe your questions were in the area of attempting to implement iSER without Shared Receive Queues, and I feel that approach is problematical in itself, and will cause a much larger impact on the buffer space needed within the Storage Controller.  

On the other hand, if you have an implementation that only has a very few Logical connections, there is probably not an important problem of queueing a number of extra buffers.  Lets use a Tape Unit as an example, any specific drive needs to be able to have enough buffers to match the speed of the link with a single Initiator.  If you are at distance, and attempting to respond appropriately in some manner with credits, you may very well find that you either cause needless latency impact, or the number of buffers needed to avoid the latency, matches what you need for the maximum line rate anyway.  

When you consider what is needed both in the primary storage focus which is 100s - 1000s of connections, to the needs for an "at distance" small number of connections, the iSER spec came up with the right balance and additional flow control seems inappropriate.
.
.
John L. Hufferd
Senior Technical Staff Member (STSM)
IBM/System Group, San Jose CA
Main Office: (408) 256-0403, Tie: 276-0403, eFax: (408) 904-4688
Alt Office: (408) 997-6136, Cell: (408) 499-9702
Internet Address: hufferd@us.ibm.com


"Alex Nezhinsky" <alexn@voltaire.com>
Sent by: ips-bounces@ietf.org

10/04/2004 08:47 AM

To
"Julian Satran" <Julian_Satran@il.ibm.com>, "Caitlin Bestler" <cait@asomi.com>
cc
ips@ietf.org, Mike Ko <mako@almaden.ibm.com>, "Mallikarjun C." <cbm@rose.hp.com>
Subject
RE: [Ips] DA+iSER Flow Control,        Limiting number of oustanding cmds attarget





> What I believe Alex is asking for, and I asked for earlier, is a basis
for
> making that estimate that is based on a specification, and not
heuristic
> measurements of N leading products.
Absolutely.

Immediate commands is only one type of "out-of-band" PDUs.
NOPs is another one which is relevant in the context of flow cotrol
itself.
If the value of MaxCmdSN changes at the target when no PDUs are about to
be sent,
iSCSI suggests sending a NOP-IN in order to communicate the new MaxCmdSN
value.
So the initiator should be ready to get NOPs at any time.

If response PDU is lost for whatever reason, the initiator will send
NOP-OUT,
this means that the target should post additional buffer for each
command it receives,
just in case.

Text requests are limited to 1 outstanding, while task mgmt requests are
not.

> Dropping the session when an initiator has sent more than the allowed
number
> of commands+2 seems perfectly reasonable.
Why not use a simple flow control scheme, in which, for example, iSER
header has
one additional field designating the number of recv buffers posted in
addition to
those implied by MaxCmdSn. That is, it's not just commands+2, but
F(commands)+X,
where X is communicated in each iSER header and F is a function specific
for initiator,
target.
What the specification could also provide is, for example, a minimal
value for X,
similar to "MUST be able to handle at least one immediate task
management command and one
immediate non-task-management iSCSI command per connection at any time".

Alexander Nezhinsky
                ________________________________

                From: Julian Satran [mailto:Julian_Satran@il.ibm.com]
                Sent: Monday, October 04, 2004 2:42 PM
                To: Caitlin Bestler
                Cc: Alex Nezhinsky; Mallikarjun C.; ips@ietf.org; Mike Ko;
ips@ietf.org
                Subject: RE: [Ips] DA+iSER Flow Control, Limiting number of
oustanding cmds attarget
               
                                                 
                This is an iSER specific requirement and an iSER specific key
may help. I am afraid however that even this might not be foolproof as
an initiator may fill any number of buffers with non task management
functions and then claim that the target is not iSCSI compliant as it
does not accept an immediate task management function command.
                Or even worse - as RDMA has no way of distinguishing immediate
from non-immediate commands an initiator can clog the pipe with
non-immediate commands before the target will feel abused.
                Dropping the session when an initiator has sent more than the
allowed number of commands+2 seems perfectly reasonable.
               
                Julo
               
                ________________________________
               
                "Caitlin Bestler" <cait@asomi.com>

                04/10/04 12:18

                                 
                                 To
                                 Julian Satran/Haifa/IBM@IBMIL
                                 cc
                                 "Alex Nezhinsky" <alexn@voltaire.com>, ips@ietf.org,
"Mike Ko" <mako@almaden.ibm.com>, "Mallikarjun C." <cbm@rose.hp.com>
                                 Subject
                                 RE: [Ips] DA+iSER Flow Control,      Limiting number of
oustanding cmds attarget
               
                Julian Satran said:
                > Alex,
                >
                > The relevant text in RFC 3720 - 3.2.2.1 is:
                >
                >   The number of commands used for immediate delivery is not
limited and
                >    their delivery for execution is not acknowledged through
the
                >    numbering scheme.  Immediate commands MAY be rejected by
the iSCSI
                >    target layer due to a lack of resources.  An iSCSI target
MUST be
                >    able to handle at least one immediate task management
command and one
                >    immediate non-task-management iSCSI command per connection
at any
                >    time.
                >
                > I hope this helps.
                >
                > Hag Sameach,
                > Julo
                >
               
               
                The iSCSI target "MAY" be allowed to discard excess immediate
commands,
                but that doesn't help the RDMA layer that needs a buffer for an
untagged
                message, or it will be required to terminate the connection.
               
                Because iSER maps both the flow-controlled and
non-flow-controlled messages
                to the same connection, lack of a buffer for an immediate
command places
                non-immediate messages at risk.
               
                Obviously, creating an extra connection for infrequent commands
is not
                the correct solution.  But a correct solution does require that
an iSER
                implementation has *something* to enable it to determine how
many
                buffers to pre-post to the RDMAP layer.  It already has the
answer for
                non-immediate commands, which is the bulk of the traffic.  It is
also
                relatively easy to make some very good estimates on how many
would
                be required for immediate commands.
               
                But interoperability of RFC compliant implementations should be
based
                upon their RFC compliance, not on compatible guestimations.
               
                What I believe Alex is asking for, and I asked for earlier, is a
basis for
                making that estimate that is based on a specification, and not
heuristic
                measurements of N leading products.
               
                --
                Caitlin Bestler
                http://asomi.com/
               

_______________________________________________
Ips mailing list
Ips@ietf.org
https://www1.ietf.org/mailman/listinfo/ips
--=_alternative 006A67CD88256F23_=-- --===============0950110242== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline Content-Transfer-Encoding: 7bit _______________________________________________ Ips mailing list Ips@ietf.org https://www1.ietf.org/mailman/listinfo/ips --===============0950110242==-- From ips-bounces@ietf.org Mon Oct 4 17:21:25 2004 Received: from ietf-mx.ietf.org (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id RAA25813 for ; Mon, 4 Oct 2004 17:21:25 -0400 (EDT) Received: from megatron.ietf.org ([132.151.6.71]) by ietf-mx.ietf.org with esmtp (Exim 4.33) id 1CEaQ4-0001gZ-7u for ips-web-archive@ietf.org; Mon, 04 Oct 2004 17:30:56 -0400 Received: from localhost.localdomain ([127.0.0.1] helo=megatron.ietf.org) by megatron.ietf.org with esmtp (Exim 4.32) id 1CEZ3l-000051-9r; Mon, 04 Oct 2004 16:03:49 -0400 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by megatron.ietf.org with esmtp (Exim 4.32) id 1CEYvC-0003Kq-DO for ips@megatron.ietf.org; Mon, 04 Oct 2004 15:54:58 -0400 Received: from ietf-mx.ietf.org (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id PAA09382 for ; Mon, 4 Oct 2004 15:54:55 -0400 (EDT) Received: from e33.co.us.ibm.com ([32.97.110.131]) by ietf-mx.ietf.org with esmtp (Exim 4.33) id 1CEZ4K-0004GO-QQ for ips@ietf.org; Mon, 04 Oct 2004 16:04:25 -0400 Received: from westrelay02.boulder.ibm.com (westrelay02.boulder.ibm.com [9.17.195.11]) by e33.co.us.ibm.com (8.12.10/8.12.9) with ESMTP id i94JsN6A454826 for ; Mon, 4 Oct 2004 15:54:23 -0400 Received: from d03nm115.boulder.ibm.com (d03av04.boulder.ibm.com [9.17.195.170]) by westrelay02.boulder.ibm.com (8.12.10/NCO/VER6.6) with ESMTP id i94JsN61358402; Mon, 4 Oct 2004 13:54:23 -0600 In-Reply-To: <95AC37E4A7C3304D94DB6AF7F64FA1B50151400D@cacexc08.americas.cpqcorp.net> To: "Krueger, Marjorie" MIME-Version: 1.0 Subject: RE: [Ips] Initiator CHAP authentication of target? X-Mailer: Lotus Notes Release 6.0.2CF1 June 9, 2003 From: John Hufferd Message-ID: Date: Mon, 4 Oct 2004 12:54:21 -0700 X-MIMETrack: Serialize by Router on D03NM115/03/M/IBM(Release 6.51HF338 | June 21, 2004) at 10/04/2004 13:54:23, Serialize complete at 10/04/2004 13:54:23 X-Spam-Score: 0.6 (/) X-Scan-Signature: e9d8c60d9288f2c774f26bab15869505 Cc: ips@ietf.org, William Studenmund , Julian Satran X-BeenThere: ips@ietf.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: IP Storage List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Content-Type: multipart/mixed; boundary="===============0074957652==" Sender: ips-bounces@ietf.org Errors-To: ips-bounces@ietf.org X-Spam-Score: 0.6 (/) X-Scan-Signature: 9e1884bf469cda400682762c3e8d96c9 This is a multipart message in MIME format. --===============0074957652== Content-Type: multipart/alternative; boundary="=_alternative 006D44B588256F23_=" This is a multipart message in MIME format. --=_alternative 006D44B588256F23_= Content-Type: text/plain; charset="US-ASCII" Marjorie, As I read the CHAP Spec., the name is not used in the computation of the check/response. It is used as an index to quickly locate the approprate Secret. With that in mind, regardless of the Response to the Challenge, the name is not used in the computation (Hash); if the computation works out correctly, that is really all that is needed. And I do not see any security exposure based on a different name, except if the same Secret is used by more than one storage controller which is forbidden by the iSCSI spec. Therefore, it seems to me the only way one could tell if the other side is ignoring the Name that is sent, is to send an invalid name, and then complain that the link was not dropped, or given an invalid protocol message. Which I think is a strange way to operate. So technically, the name must be sent, but I do not see a requirement that it must be check in some special way that excludes ignoring it. I would have like us to have a default of the iSCSI Node name, if the CHAP_N keyword is not sent, but since we did not do that, I think that in most cases to ease the Administrative burden of setting yet another name, most installations will use the iSCSI Node Name . . John L. Hufferd Senior Technical Staff Member (STSM) IBM/System Group, San Jose CA Main Office: (408) 256-0403, Tie: 276-0403, eFax: (408) 904-4688 Alt Office: (408) 997-6136, Cell: (408) 499-9702 Internet Address: hufferd@us.ibm.com "Krueger, Marjorie" Sent by: ips-bounces@ietf.org 10/04/2004 11:57 AM To "Julian Satran" cc ips@ietf.org, William Studenmund Subject RE: [Ips] Initiator CHAP authentication of target? While it may not be "detectable" by the target, it certainly seems like "expected behavior" (ie, following the protocol) is that the initiator MUST use the CHAP_N and CHAP_R supplied in the login response - would you agree? When testing an initiator implementation, if one can prove that a certain initiator isn't using the values supplied, one can say that the initiator is not correctly implemented? -Marjorie From: Julian Satran [mailto:Julian_Satran@il.ibm.com] Sent: Monday, October 04, 2004 4:40 AM To: Krueger, Marjorie Cc: ips@ietf.org; William Studenmund Subject: RE: [Ips] Initiator CHAP authentication of target? Marjorie, Maybe I am not understanding clearly the "substance" of your question but if you are questioning the validity of an initiator not using the data supplied to authenticate the target - that is "legal" in the sense that the protocol has no way of enforcing the check in CHAP (i.e., there is no returned value that will irrefutably prove that the initiator did check). I would not use this protocol in a money exchange transaction! The weaknesses of CHAP have been discussed at some length on the list. Regards, Julo "Krueger, Marjorie" Sent by: ips-bounces@ietf.org 01/10/04 01:28 To "William Studenmund" cc ips@ietf.org Subject RE: [Ips] Initiator CHAP authentication of target? It appears a certain initiator is ignoring CHAP_N returned by the target and just using the target node name for CHAP_N regardless of what the target sent in the login sequence. I think it seems like invalid behavior but I just wanted to check with this audience. Thanks Marjorie > -----Original Message----- > From: William Studenmund [mailto:wrstuden@wasabisystems.com] > Sent: Thursday, September 30, 2004 3:32 PM > To: Krueger, Marjorie > Cc: ips@ietf.org > Subject: Re: [Ips] Initiator CHAP authentication of target? > > On Sep 30, 2004, at 3:22 PM, Krueger, Marjorie wrote: > > > So if > > > >> or, if it requires target authentication, with: > >> > >> CHAP_N= CHAP_R= CHAP_I= CHAP_C= > >> > >> If the initiator authentication fails, the target MUST answer > >> with a > >> Login reject with "Authentication Failure" status. > >> Otherwise, if the > >> initiator required target authentication, the target > MUST either > >> answer with a Login reject with "Authentication > Failure" or reply > >> with: > >> > >> CHAP_N= CHAP_R= > > > > And the target replys with CHAP_N that is not defined and the > > initiator accepts that (accepts anything) is that valid > behavior? In > > otherwords, if the initiator ignores CHAP_N and only uses > the target > > node name and CHAP_R to authenticate, is that valid behavior? > > Shouldn't the target be able to force the use of CHAP_N? > > I'm confused. The initiator can't NOT use CHAP_N. Thus the > target shouldn't be able to force something that's required. :-) > > To quote the paragraph after the one above: > > Where N, (A,A1,A2), I, C, and R are (correspondingly) the Name, > Algorithm, Identifier, Challenge, and Response as defined in > [RFC1994], N is a text string, A,A1,A2, and I are > numbers, and C and > R are large-binary-values and their binary length (not > the length of > the character string that represents them in encoded > form) MUST not > exceed 1024 bytes. > > So an initiator that just uses the target's name for CHAP > purposes seems broken. > > Take care, > > Bill > _______________________________________________ Ips mailing list Ips@ietf.org https://www1.ietf.org/mailman/listinfo/ips _______________________________________________ Ips mailing list Ips@ietf.org https://www1.ietf.org/mailman/listinfo/ips --=_alternative 006D44B588256F23_= Content-Type: text/html; charset="US-ASCII"
Marjorie,
As I read the CHAP Spec., the name is not used in the computation of the check/response.  It is used as an index to quickly locate the approprate Secret.  With that in mind, regardless of the Response to the Challenge, the name is not used in the computation (Hash); if the computation works out correctly, that is really all that is needed.  And I do not see any security exposure based on a different name, except if the same Secret is used by more than one storage controller which is forbidden by the iSCSI spec.  Therefore, it seems to me the only way one could tell if the other side is ignoring the Name that is sent, is to send an invalid name, and then complain that the link was not dropped, or given an invalid protocol message.  Which I think is a strange way to operate.  So technically, the name must be sent, but I do not see a requirement that it must be check in some special way that excludes ignoring it.  I would have like us to have a default of the iSCSI Node name, if the CHAP_N keyword is not sent, but since we did not do that, I think that in most cases to ease the Administrative burden of setting yet another name, most installations will use the iSCSI Node Name

.
.
John L. Hufferd
Senior Technical Staff Member (STSM)
IBM/System Group, San Jose CA
Main Office: (408) 256-0403, Tie: 276-0403, eFax: (408) 904-4688
Alt Office: (408) 997-6136, Cell: (408) 499-9702
Internet Address: hufferd@us.ibm.com


"Krueger, Marjorie" <marjorie.krueger@hp.com>
Sent by: ips-bounces@ietf.org

10/04/2004 11:57 AM

To
"Julian Satran" <Julian_Satran@il.ibm.com>
cc
ips@ietf.org, William Studenmund <wrstuden@wasabisystems.com>
Subject
RE: [Ips] Initiator CHAP authentication of target?





While it may not be "detectable" by the target, it certainly seems like "expected behavior" (ie, following the protocol) is that the initiator MUST use the CHAP_N and CHAP_R supplied in the login response - would you agree?  When testing an initiator implementation, if one can prove that a certain initiator isn't using the values supplied, one can say that the initiator is not correctly implemented?
 
-Marjorie

From: Julian Satran [mailto:Julian_Satran@il.ibm.com]
Sent: Monday, October 04, 2004 4:40 AM
To: Krueger, Marjorie
Cc: ips@ietf.org; William Studenmund
Subject: RE: [Ips] Initiator CHAP authentication of target?

Marjorie,

Maybe I am not understanding clearly the "substance" of your question but if you are questioning the validity of an initiator not using the data supplied to authenticate the target - that is "legal" in the sense that the protocol has no way of enforcing the check in CHAP (i.e., there is no returned value that will irrefutably prove that the initiator did check).
I would not use this protocol in a money exchange transaction! The weaknesses of CHAP have been discussed at some length on the list.

Regards,
Julo

"Krueger, Marjorie" <marjorie.krueger@hp.com>
Sent by: ips-bounces@ietf.org

01/10/04 01:28


To
"William Studenmund" <wrstuden@wasabisystems.com>
cc
ips@ietf.org
Subject
RE: [Ips] Initiator CHAP authentication of target?







It appears a certain initiator is ignoring CHAP_N returned by the target
and just using the target node name for CHAP_N regardless of what the
target sent in the login sequence.  I think it seems like invalid
behavior but I just wanted to check with this audience.

Thanks
Marjorie

> -----Original Message-----
> From: William Studenmund [mailto:wrstuden@wasabisystems.com]
> Sent: Thursday, September 30, 2004 3:32 PM
> To: Krueger, Marjorie
> Cc: ips@ietf.org
> Subject: Re: [Ips] Initiator CHAP authentication of target?
>
> On Sep 30, 2004, at 3:22 PM, Krueger, Marjorie wrote:
>
> > So if
> >
> >>     or, if it requires target authentication, with:
> >>
> >>        CHAP_N=<N> CHAP_R=<R> CHAP_I=<I> CHAP_C=<C>
> >>
> >>     If the initiator authentication fails, the target MUST answer
> >> with a
> >>     Login reject with "Authentication Failure" status.
> >> Otherwise, if the
> >>     initiator required target authentication, the target
> MUST either
> >>     answer with a Login reject with "Authentication
> Failure" or reply
> >>     with:
> >>
> >>        CHAP_N=<N> CHAP_R=<R>
> >
> > And the target replys with CHAP_N that is not defined and the
> > initiator accepts that (accepts anything) is that valid
> behavior?  In
> > otherwords, if the initiator ignores CHAP_N and only uses
> the target
> > node name and CHAP_R to authenticate, is that valid behavior?  
> > Shouldn't the target be able to force the use of CHAP_N?
>
> I'm confused. The initiator can't NOT use CHAP_N. Thus the
> target shouldn't be able to force something that's required. :-)
>
> To quote the paragraph after the one above:
>
>     Where N, (A,A1,A2), I, C, and R are (correspondingly) the Name,
>     Algorithm, Identifier, Challenge, and Response as defined in
>     [RFC1994], N is a text string, A,A1,A2, and I are
> numbers, and C and
>     R are large-binary-values and their binary length (not
> the length of
>     the character string that represents them in encoded
> form) MUST not
>     exceed 1024 bytes.
>
> So an initiator that just uses the target's name for CHAP
> purposes seems broken.
>
> Take care,
>
> Bill
>

_______________________________________________
Ips mailing list
Ips@ietf.org
https://www1.ietf.org/mailman/listinfo/ips
_______________________________________________
Ips mailing list
Ips@ietf.org
https://www1.ietf.org/mailman/listinfo/ips
--=_alternative 006D44B588256F23_=-- --===============0074957652== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline Content-Transfer-Encoding: 7bit _______________________________________________ Ips mailing list Ips@ietf.org https://www1.ietf.org/mailman/listinfo/ips --===============0074957652==-- From ips-bounces@ietf.org Mon Oct 4 17:21:40 2004 Received: from ietf-mx.ietf.org (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id RAA25863 for ; Mon, 4 Oct 2004 17:21:40 -0400 (EDT) Received: from megatron.ietf.org ([132.151.6.71]) by ietf-mx.ietf.org with esmtp (Exim 4.33) id 1CEaQI-0001hK-Mk for ips-web-archive@ietf.org; Mon, 04 Oct 2004 17:31:11 -0400 Received: from localhost.localdomain ([127.0.0.1] helo=megatron.ietf.org) by megatron.ietf.org with esmtp (Exim 4.32) id 1CEZ3m-00005c-3T; Mon, 04 Oct 2004 16:03:50 -0400 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by megatron.ietf.org with esmtp (Exim 4.32) id 1CEYvN-0003Pt-9V for ips@megatron.ietf.org; Mon, 04 Oct 2004 15:55:11 -0400 Received: from ietf-mx.ietf.org (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id PAA09396 for ; Mon, 4 Oct 2004 15:55:04 -0400 (EDT) Received: from sadr.equallogic.com ([66.155.203.134]) by ietf-mx.ietf.org with smtp (Exim 4.33) id 1CEZ4R-0004GS-3W for ips@ietf.org; Mon, 04 Oct 2004 16:04:34 -0400 Received: from sadr.equallogic.com (localhost.localdomain [127.0.0.1]) by sadr.equallogic.com (8.12.8/8.12.8) with ESMTP id i94JsTFn025318 for ; Mon, 4 Oct 2004 15:54:29 -0400 Received: from M30.equallogic.com (m30 [172.16.1.30]) by sadr.equallogic.com (8.12.8/8.12.8) with SMTP id i94JsSsw025313; Mon, 4 Oct 2004 15:54:28 -0400 Received: from pkoning.equallogic.com ([172.16.1.220]) by M30.equallogic.com with Microsoft SMTPSVC(5.0.2195.6713); Mon, 4 Oct 2004 15:54:29 -0400 MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Message-ID: <16737.43635.104900.308014@gargle.gargle.HOWL> Date: Mon, 4 Oct 2004 15:54:27 -0400 From: Paul Koning To: marjorie.krueger@hp.com Subject: RE: [Ips] Initiator CHAP authentication of target? References: <95AC37E4A7C3304D94DB6AF7F64FA1B50151400D@cacexc08.americas.cpqcorp.net> X-Mailer: VM 7.17 under 21.4 (patch 14) "Reasonable Discussion" XEmacs Lucid X-OriginalArrivalTime: 04 Oct 2004 19:54:29.0176 (UTC) FILETIME=[F5872380:01C4AA4B] X-Spam-Score: 0.0 (/) X-Scan-Signature: 7655788c23eb79e336f5f8ba8bce7906 Content-Transfer-Encoding: 7bit Cc: ips@ietf.org, wrstuden@wasabisystems.com, Julian_Satran@il.ibm.com X-BeenThere: ips@ietf.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: IP Storage List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Sender: ips-bounces@ietf.org Errors-To: ips-bounces@ietf.org X-Spam-Score: 0.0 (/) X-Scan-Signature: 0bc60ec82efc80c84b8d02f4b0e4de22 Content-Transfer-Encoding: 7bit >>>>> "Marjorie" == Marjorie Krueger writes: Marjorie> While it may not be "detectable" by the target, it Marjorie> certainly seems like "expected behavior" (ie, following the Marjorie> protocol) is that the initiator MUST use the CHAP_N and Marjorie> CHAP_R supplied in the login response - would you agree? Marjorie> When testing an initiator implementation, if one can prove Marjorie> that a certain initiator isn't using the values supplied, Marjorie> one can say that the initiator is not correctly Marjorie> implemented? I certainly would say that. If we hadn't wanted the initiator to look at CHAP_N, we would have left it out of the protocol. The CHAP exchange proves that you know the secret that goes with name CHAP_N. This is testable, too. If an exchange succeeds, and a repeat of that exchange with a different CHAP_N in the message also succeeds, then the initiator was ignoring CHAP_N. paul _______________________________________________ Ips mailing list Ips@ietf.org https://www1.ietf.org/mailman/listinfo/ips From ips-bounces@ietf.org Tue Oct 5 02:47:49 2004 Received: from ietf-mx.ietf.org (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id CAA24544 for ; Tue, 5 Oct 2004 02:47:49 -0400 (EDT) Received: from megatron.ietf.org ([132.151.6.71]) by ietf-mx.ietf.org with esmtp (Exim 4.33) id 1CEjGF-0000Ft-N2 for ips-web-archive@ietf.org; Tue, 05 Oct 2004 02:57:23 -0400 Received: from localhost.localdomain ([127.0.0.1] helo=megatron.ietf.org) by megatron.ietf.org with esmtp (Exim 4.32) id 1CEj5N-00041u-5m; Tue, 05 Oct 2004 02:46:09 -0400 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by megatron.ietf.org with esmtp (Exim 4.32) id 1CEj4f-0003th-M9 for ips@megatron.ietf.org; Tue, 05 Oct 2004 02:45:28 -0400 Received: from ietf-mx.ietf.org (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id CAA24311 for ; Tue, 5 Oct 2004 02:45:23 -0400 (EDT) Received: from system60.simplicato.com ([207.99.47.60] helo=host52a.simplicato.com) by ietf-mx.ietf.org with esmtp (Exim 4.33) id 1CEjDt-0000Cm-B4 for ips@ietf.org; Tue, 05 Oct 2004 02:54:58 -0400 Received: from localhost (localhost.simplicato.com [127.0.0.1]) by host52a.simplicato.com (Postfix) with ESMTP id 85FA611E054; Tue, 5 Oct 2004 02:45:22 -0400 (EDT) Received: from host52a.simplicato.com ([127.0.0.1]) by localhost (host52a.simplicato.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 62217-07; Tue, 5 Oct 2004 02:45:21 -0400 (EDT) Received: from mail.asomi.com (localhost.simplicato.com [127.0.0.1]) by host52a.simplicato.com (Postfix) with SMTP id 2C10211E077; Tue, 5 Oct 2004 02:45:21 -0400 (EDT) Received: from 62.90.193.206 (proxying for 62.90.193.206) (SquirrelMail authenticated user cait@asomi.com) by mail52a.simplicato.com with HTTP; Tue, 5 Oct 2004 01:45:21 -0500 (CDT) Message-ID: <52342.62.90.193.206.1096958721.squirrel@mail52a.simplicato.com> In-Reply-To: References: Date: Tue, 5 Oct 2004 01:45:21 -0500 (CDT) Subject: RE: [Ips] DA+iSER Flow Control, Limiting number of oustanding cmds attarget From: "Caitlin Bestler" To: "John Hufferd" , ips@ietf.org User-Agent: SquirrelMail/1.4.2 MIME-Version: 1.0 Content-Type: text/plain;charset=iso-8859-1 X-Priority: 3 Importance: Normal X-Virus-Scanned: by amavisd-new at simplicato.com Content-Transfer-Encoding: quoted-printable X-Spam-Score: 0.8 (/) X-Scan-Signature: b5d20af10c334b36874c0264b10f59f1 Content-Transfer-Encoding: quoted-printable Cc: Julian Satran , "Mallikarjun C." , Alex Nezhinsky , Mike Ko X-BeenThere: ips@ietf.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: IP Storage List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Sender: ips-bounces@ietf.org Errors-To: ips-bounces@ietf.org X-Spam-Score: 0.8 (/) X-Scan-Signature: cf3becbbd6d1a45acbe2ffd4ab88bdc2 Content-Transfer-Encoding: quoted-printable John Hufferd said: > The important thing to understand is that you need to queue > enough buffers so that the suddenly chatty Initiator can proceed > with out an impact to latency. In most of the robust > implementations, which will be controlling hundreds/thousands > of logical connections, the approprate consideration is the > fact that regardless of the number of logical connections, > there can not be more data than the link is capable of > handling. Therefore, we designed the concept of Shared > Receive Queue. The purpose was to have a method where the > number of buffers that were needed to handle the link was > all that was required. Which of course meant that there > needed to be an ongoing active buffer replenishment technique. > Which is similar to what is done in other SCSI environments. > > When ever one attempts to work this problem with non shared > receive queues, the problem gets out of hand very quickly. > With or without some additional form of end to end credit, > the Total number of buffers that is needed becomes the sum > of the buffers needed for all logical connections, since at > any time they could burst (or desire to burst) at full link > rates. So the idea of having to add a few extra, for each > connection seemed to be only useful for very limited > implementations, and having flow control above what > iSCSI offered seemed to be a needless impact on latency. > Even then the sum of all the credits (meaning backed by > buffers) would be a lot more than what is needed to match > the link speed. > There are two very distinct issues here. One is the memory allocation strategy that should be employed by a server that supports a large number of clients. The second is how to determine what is a legal untagged message at the RDMA layer. As Julian pointed out, the iSCSI layer can quickly discard excessive asynchronous messages. The challenge is enabling the iSER layer to provide buffers for them so that they can be captured at the RDMA layer prior to their being interpreted at the iSCSI layer. Shared Receive Queues are relevant here on two fronts. First, iSCSI flow control is really more session oriented than connection oriented. Pre-posting separate buffers to receive Cmd X on *each* connection is clearly over-buffering. This may be valid, because memory is cheap, but clearly it is not required to pre-commit duplicate buffers for each connection in a session. But Shared Receive Queues are typically used to cover multiple sessions, indeed frequently thousands of sessions. As noted in the RDDP Security spec, when using a shared resource across multiple sessions, the application must take security precautions to prevent the shared resource from being drained by a single malicious or malfunctioning client. Basically, you MUST ensure that a single client sending excess untagged messages will not drain the Shared Receive Queue and result in connections for *other* sessions failing for lack of a receive buffer. Which means that use of a Shared Receive Queue does *nothing* to eliminate the need to calculate a per-connection limit on the maximum number of untagged messages -- it only means that you do not need to pre-allocate that many buffers. But any ULP that is compliant with the RDDP Security requirements must supply implementations with enough information so that they can enable the per-connection limits on untagged messages. Those limits might be a configuration field for a QP using an SRQ, or they might be the size of the QP's Receive Queue. It doesn't matter what strategy any specific implementation uses, what matters is that there needs to be a method of setting that limit which has its justification in *specifications*, not in measurements. Otherwise, *each* connection must be enabled to receive buffers at line rate. And unless you can push the untagged messages to the iSER module *at line rate* you don't have a workable solution. Requiring the iSER module to filter untagged messages at line rate would clearly make user-mode implementations infeasible, and would prevent iSER from being implemented over general purpose RDMA verbs. *Either* of which would strike me as justifying a change in the DataMover architecture to avoid this problem. -- Caitlin Bestler http://asomi.com/ _______________________________________________ Ips mailing list Ips@ietf.org https://www1.ietf.org/mailman/listinfo/ips From ips-bounces@ietf.org Tue Oct 5 03:19:48 2004 Received: from ietf-mx.ietf.org (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id DAA27884 for ; Tue, 5 Oct 2004 03:19:48 -0400 (EDT) Received: from megatron.ietf.org ([132.151.6.71]) by ietf-mx.ietf.org with esmtp (Exim 4.33) id 1CEjlC-0006LS-Iv for ips-web-archive@ietf.org; Tue, 05 Oct 2004 03:29:22 -0400 Received: from localhost.localdomain ([127.0.0.1] helo=megatron.ietf.org) by megatron.ietf.org with esmtp (Exim 4.32) id 1CEjZp-00017I-89; Tue, 05 Oct 2004 03:17:37 -0400 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by megatron.ietf.org with esmtp (Exim 4.32) id 1CEjVy-0008Tx-Sg for ips@megatron.ietf.org; Tue, 05 Oct 2004 03:13:39 -0400 Received: from ietf-mx.ietf.org (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id DAA27189 for ; Tue, 5 Oct 2004 03:13:37 -0400 (EDT) Received: from [203.124.158.219] (helo=ganesha.intranet.calsoftinc.com) by ietf-mx.ietf.org with esmtp (Exim 4.33) id 1CEjfD-0005CQ-6J for ips@ietf.org; Tue, 05 Oct 2004 03:23:12 -0400 Received: from kush (unknown [172.16.0.55]) by ganesha.intranet.calsoftinc.com (Postfix) with ESMTP id 3EA9BDB57F for ; Tue, 5 Oct 2004 12:36:23 +0530 (IST) From: Faraz S To: ips@ietf.org Content-Type: text/plain Organization: Calsoft Pvt Ltd Message-Id: <1096960328.8961.13.camel@kush.(none)> Mime-Version: 1.0 X-Mailer: Ximian Evolution 1.4.6 Date: Tue, 05 Oct 2004 12:42:08 +0530 Content-Transfer-Encoding: 7bit X-Spam-Score: 0.0 (/) X-Scan-Signature: 8abaac9e10c826e8252866cbe6766464 Content-Transfer-Encoding: 7bit Subject: [Ips] Parallelism / Thread identification for a target. X-BeenThere: ips@ietf.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: IP Storage List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Sender: ips-bounces@ietf.org Errors-To: ips-bounces@ietf.org X-Spam-Score: 0.0 (/) X-Scan-Signature: 39bd8f8cbb76cae18b7e23f7cf6b2b9f Content-Transfer-Encoding: 7bit Hi Everybody; We need to implement a high performance (Terms of I/O bandwith) iSCSI target. We did think of a high level design in which we had come up with following threads (considering only 1 session with NR_CONNECTION number of connections per session). 1 Thread to Listen for incomming Login requests. 1 Thread per connection to recieve and queue iSCSI resquest. 1 Thread per connection to send iSCSI resquests. so numbers of threads come to 1 + 2(NR_CONNECTION) (which comes to 5 with connections=2) Can somebody justify the above kernel thread usage along the Lines. 1. Finally all send and recieves are going to be serialized over a physical interface. So how does the number of interfaces(pyhsical lines) we have affect the thread equation. 2. How can multiple sessions be added and will be thier effect on the thread equation. I may be completely wrong in the High level desing. So please can somebody give me a set of guidelines which describes the various subsystems and the parallel threads of iSCSI target. To be implemented on Linux. Regards Faraz. -- _______________________________________________ Ips mailing list Ips@ietf.org https://www1.ietf.org/mailman/listinfo/ips From ips-bounces@ietf.org Tue Oct 5 03:56:00 2004 Received: from ietf-mx.ietf.org (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id DAA00468 for ; Tue, 5 Oct 2004 03:56:00 -0400 (EDT) Received: from megatron.ietf.org ([132.151.6.71]) by ietf-mx.ietf.org with esmtp (Exim 4.33) id 1CEkKE-0003wg-W8 for ips-web-archive@ietf.org; Tue, 05 Oct 2004 04:05:35 -0400 Received: from localhost.localdomain ([127.0.0.1] helo=megatron.ietf.org) by megatron.ietf.org with esmtp (Exim 4.32) id 1CEk7O-0005i4-F4; Tue, 05 Oct 2004 03:52:18 -0400 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by megatron.ietf.org with esmtp (Exim 4.32) id 1CEk3A-0005Ka-CZ for ips@megatron.ietf.org; Tue, 05 Oct 2004 03:47:56 -0400 Received: from ietf-mx.ietf.org (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id DAA29874 for ; Tue, 5 Oct 2004 03:47:54 -0400 (EDT) Received: from e31.co.us.ibm.com ([32.97.110.129]) by ietf-mx.ietf.org with esmtp (Exim 4.33) id 1CEkC9-0002MI-PE for ips@ietf.org; Tue, 05 Oct 2004 03:57:30 -0400 Received: from westrelay04.boulder.ibm.com (westrelay04.boulder.ibm.com [9.17.193.32]) by e31.co.us.ibm.com (8.12.10/8.12.9) with ESMTP id i957kwaI316250; Tue, 5 Oct 2004 03:46:58 -0400 Received: from d03nm115.boulder.ibm.com (d03av02.boulder.ibm.com [9.17.195.168]) by westrelay04.boulder.ibm.com (8.12.10/NCO/VER6.6) with ESMTP id i957kvWA152384; Tue, 5 Oct 2004 01:46:58 -0600 In-Reply-To: <52342.62.90.193.206.1096958721.squirrel@mail52a.simplicato.com> To: "Caitlin Bestler" MIME-Version: 1.0 Subject: RE: [Ips] DA+iSER Flow Control, Limiting number of oustanding cmds attarget X-Mailer: Lotus Notes Release 6.0.2CF1 June 9, 2003 From: John Hufferd Message-ID: Date: Tue, 5 Oct 2004 00:46:53 -0700 X-MIMETrack: Serialize by Router on D03NM115/03/M/IBM(Release 6.51HF338 | June 21, 2004) at 10/05/2004 01:46:58, Serialize complete at 10/05/2004 01:46:58 X-Spam-Score: 0.0 (/) X-Scan-Signature: 65215b440f7ab00ca9514de4a7a89926 Cc: ips@ietf.org, Mike Ko , Julian Satran , Alex Nezhinsky , "Mallikarjun C." X-BeenThere: ips@ietf.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: IP Storage List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Content-Type: multipart/mixed; boundary="===============1095379714==" Sender: ips-bounces@ietf.org Errors-To: ips-bounces@ietf.org X-Spam-Score: 0.0 (/) X-Scan-Signature: 10dcc25e55b9b5f7d6ded516404bdc4c This is a multipart message in MIME format. --===============1095379714== Content-Type: multipart/alternative; boundary="=_alternative 002AAB9B88256F24_=" This is a multipart message in MIME format. --=_alternative 002AAB9B88256F24_= Content-Type: text/plain; charset="US-ASCII" This is not a general User State application it is a very specific Kernel Mode operation. This is Storage I/O, and the server you are talking about is a very specially constructed kernel application that follows the rules of SCSI, and the operations that run in a storage controller manages a large cache of memory that is divided up between the what is Going out and coming in. And this is both going out and coming in the communication link and going out and coming in from the SCSI LUN. iSCSI/iSER is just a part of the buffering and caching that is going on. There is various different algorithms to deal with Cache, both write through and write behind and algorithms to select the right buffer locations for new data (from either side) and algorithms to decide which Cache buffers need to be written to disk, etc. The Caches/Buffers are usually all managed via chains, and what comes in as an iSCSI buffer becomes a Cache buffer and then a LUN I/O buffer, then a free buffer, then perhaps placed again on the iSER buffer chain or on the LUN read chain, etc. You can almost talk about a Storage Controller as "Buffers are US". There can not be a problem for a storage controller to service the line rate of the communication link, or the storage controller will not operate correctly, and that type of operation is fundamental to a storage controller. Therefore to talk of this as if the storage controller is some general user App is just not correct. This type of buffer balancing is how storage controllers have always acted, and balancing buffers and caches in and out is not a problem. We do it all the time. The flow control of iSCSI is all that is needed when we are working with Shared RQs, and are managing the buffers ourselves. . . John L. Hufferd Senior Technical Staff Member (STSM) IBM/System Group, San Jose CA Main Office: (408) 256-0403, Tie: 276-0403, eFax: (408) 904-4688 Alt Office: (408) 997-6136, Cell: (408) 499-9702 Internet Address: hufferd@us.ibm.com "Caitlin Bestler" 10/04/2004 11:45 PM To John Hufferd/San Jose/IBM@IBMUS, ips@ietf.org cc "Alex Nezhinsky" , "Mike Ko" , "Julian Satran" , "Mallikarjun C." Subject RE: [Ips] DA+iSER Flow Control, Limiting number of oustanding cmds attarget John Hufferd said: > The important thing to understand is that you need to queue > enough buffers so that the suddenly chatty Initiator can proceed > with out an impact to latency. In most of the robust > implementations, which will be controlling hundreds/thousands > of logical connections, the approprate consideration is the > fact that regardless of the number of logical connections, > there can not be more data than the link is capable of > handling. Therefore, we designed the concept of Shared > Receive Queue. The purpose was to have a method where the > number of buffers that were needed to handle the link was > all that was required. Which of course meant that there > needed to be an ongoing active buffer replenishment technique. > Which is similar to what is done in other SCSI environments. > > When ever one attempts to work this problem with non shared > receive queues, the problem gets out of hand very quickly. > With or without some additional form of end to end credit, > the Total number of buffers that is needed becomes the sum > of the buffers needed for all logical connections, since at > any time they could burst (or desire to burst) at full link > rates. So the idea of having to add a few extra, for each > connection seemed to be only useful for very limited > implementations, and having flow control above what > iSCSI offered seemed to be a needless impact on latency. > Even then the sum of all the credits (meaning backed by > buffers) would be a lot more than what is needed to match > the link speed. > There are two very distinct issues here. One is the memory allocation strategy that should be employed by a server that supports a large number of clients. The second is how to determine what is a legal untagged message at the RDMA layer. As Julian pointed out, the iSCSI layer can quickly discard excessive asynchronous messages. The challenge is enabling the iSER layer to provide buffers for them so that they can be captured at the RDMA layer prior to their being interpreted at the iSCSI layer. Shared Receive Queues are relevant here on two fronts. First, iSCSI flow control is really more session oriented than connection oriented. Pre-posting separate buffers to receive Cmd X on *each* connection is clearly over-buffering. This may be valid, because memory is cheap, but clearly it is not required to pre-commit duplicate buffers for each connection in a session. But Shared Receive Queues are typically used to cover multiple sessions, indeed frequently thousands of sessions. As noted in the RDDP Security spec, when using a shared resource across multiple sessions, the application must take security precautions to prevent the shared resource from being drained by a single malicious or malfunctioning client. Basically, you MUST ensure that a single client sending excess untagged messages will not drain the Shared Receive Queue and result in connections for *other* sessions failing for lack of a receive buffer. Which means that use of a Shared Receive Queue does *nothing* to eliminate the need to calculate a per-connection limit on the maximum number of untagged messages -- it only means that you do not need to pre-allocate that many buffers. But any ULP that is compliant with the RDDP Security requirements must supply implementations with enough information so that they can enable the per-connection limits on untagged messages. Those limits might be a configuration field for a QP using an SRQ, or they might be the size of the QP's Receive Queue. It doesn't matter what strategy any specific implementation uses, what matters is that there needs to be a method of setting that limit which has its justification in *specifications*, not in measurements. Otherwise, *each* connection must be enabled to receive buffers at line rate. And unless you can push the untagged messages to the iSER module *at line rate* you don't have a workable solution. Requiring the iSER module to filter untagged messages at line rate would clearly make user-mode implementations infeasible, and would prevent iSER from being implemented over general purpose RDMA verbs. *Either* of which would strike me as justifying a change in the DataMover architecture to avoid this problem. -- Caitlin Bestler http://asomi.com/ --=_alternative 002AAB9B88256F24_= Content-Type: text/html; charset="US-ASCII"
This is not a general User State application it is a very specific Kernel Mode operation.  This is Storage I/O, and the server you are talking about is a very specially constructed kernel application that follows the rules of SCSI, and the operations that run in a storage controller manages a large cache of memory that is divided up between the what is Going out and coming in.  And this is both going out and coming in the communication link and going out and coming in from the SCSI LUN.  iSCSI/iSER is just a part of the buffering and caching that is going on.  There is various different algorithms to deal with Cache, both write through and write behind and algorithms to select the right buffer locations for new data (from either side) and algorithms to decide which Cache buffers need to be written to disk, etc.  The Caches/Buffers are usually all managed via chains, and what comes in as an iSCSI buffer becomes a Cache buffer and then a LUN I/O buffer, then a free buffer, then perhaps placed again on the iSER buffer chain or on the LUN read chain, etc.  You can almost talk about a Storage Controller as "Buffers are US".  There can not be a problem for a storage controller to service the line rate of the communication link, or the storage controller will not operate correctly, and that type of operation is fundamental to a storage controller.  Therefore to talk of this as if the storage controller is some general user App is just not correct.  This type of buffer balancing is how storage controllers have always acted, and balancing buffers and caches in and out is not a problem.  We do it all the time.  The flow control of iSCSI is all that is needed when we are working with Shared RQs, and are managing the buffers ourselves.
.
.
John L. Hufferd
Senior Technical Staff Member (STSM)
IBM/System Group, San Jose CA
Main Office: (408) 256-0403, Tie: 276-0403, eFax: (408) 904-4688
Alt Office: (408) 997-6136, Cell: (408) 499-9702
Internet Address: hufferd@us.ibm.com


"Caitlin Bestler" <cait@asomi.com>

10/04/2004 11:45 PM

To
John Hufferd/San Jose/IBM@IBMUS, ips@ietf.org
cc
"Alex Nezhinsky" <alexn@voltaire.com>, "Mike Ko" <mako@almaden.ibm.com>, "Julian Satran" <julian_satran@il.ibm.com>, "Mallikarjun C." <cbm@rose.hp.com>
Subject
RE: [Ips] DA+iSER Flow Control,     Limiting number of oustanding cmds attarget






John Hufferd said:
> The important thing to understand is that you need to queue
> enough buffers so that the suddenly chatty Initiator can proceed
> with out an impact to latency.   In most of the robust
> implementations, which will be controlling hundreds/thousands
> of logical connections, the approprate consideration is the
> fact that regardless of the number of logical connections,
> there can not be more data than the link is capable of
> handling.  Therefore, we designed the concept of Shared
> Receive Queue. The purpose was to have a method where the
> number of buffers that were needed to handle the link was
> all that was required.  Which of course meant that there
> needed to be an ongoing active buffer replenishment technique.
> Which is similar to what is done in other SCSI environments.
>
> When ever one attempts to work this problem with non shared
> receive queues, the problem gets out of hand very quickly.
> With or without some additional form of end to end credit,
> the Total number of buffers that is needed becomes the sum
> of the buffers needed for all logical connections, since at
> any time they could burst (or desire to burst) at full link
> rates.  So the idea of having to add a few extra, for each
> connection seemed to be only useful for very limited
> implementations, and having flow control above what
> iSCSI offered seemed to be a needless impact on latency.
> Even then the sum of all the credits (meaning backed by
> buffers) would be a lot more than what is needed to match
> the link speed.
>

There are two very distinct issues here.  One is the memory
allocation strategy that should be employed by a server that
supports a large number of clients.  The second is how to
determine what is a legal untagged message at the RDMA layer.

As Julian pointed out, the iSCSI layer can quickly discard
excessive asynchronous messages. The challenge is enabling
the iSER layer to provide buffers for them so that they
can be captured at the RDMA layer prior to their being
interpreted at the iSCSI layer.

Shared Receive Queues are relevant here on two fronts.
First, iSCSI flow control is really more session oriented
than connection oriented.  Pre-posting separate buffers to
receive Cmd X on *each* connection is clearly over-buffering.
This may be valid, because memory is cheap, but clearly it
is not required to pre-commit duplicate buffers for each
connection in a session.

But Shared Receive Queues are typically used to cover
multiple sessions, indeed frequently thousands of sessions.

As noted in the RDDP Security spec, when using a shared
resource across multiple sessions, the application must take
security precautions to prevent the shared resource from being
drained by a single malicious or malfunctioning client.

Basically, you MUST ensure that a single client sending
excess untagged messages will not drain the Shared Receive
Queue and result in connections for *other* sessions failing
for lack of a receive buffer.

Which means that use of a Shared Receive Queue does *nothing*
to eliminate the need to calculate a per-connection limit on
the maximum number of untagged messages -- it only means that
you do not need to pre-allocate that many buffers.

But any ULP that is compliant with the RDDP Security requirements
must supply implementations with enough information so that they
can enable the per-connection limits on untagged messages.  Those
limits might be a configuration field for a QP using an SRQ, or
they might be the size of the QP's Receive Queue.  It doesn't
matter what strategy any specific implementation uses, what
matters is that there needs to be a method of setting that
limit which has its justification in *specifications*, not
in measurements.

Otherwise, *each* connection must be enabled to receive buffers
at line rate.  And unless you can push the untagged messages to
the iSER module *at line rate* you don't have a workable solution.
Requiring the iSER module to filter untagged messages at line
rate would clearly make user-mode implementations infeasible,
and would prevent iSER from being implemented over general
purpose RDMA verbs.  *Either* of which would strike me as
justifying a change in the DataMover architecture to avoid
this problem.



--
Caitlin Bestler
http://asomi.com/
--=_alternative 002AAB9B88256F24_=-- --===============1095379714== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline Content-Transfer-Encoding: 7bit _______________________________________________ Ips mailing list Ips@ietf.org https://www1.ietf.org/mailman/listinfo/ips --===============1095379714==-- From ips-bounces@ietf.org Tue Oct 5 04:31:47 2004 Received: from ietf-mx.ietf.org (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id EAA02606 for ; Tue, 5 Oct 2004 04:31:46 -0400 (EDT) Received: from megatron.ietf.org ([132.151.6.71]) by ietf-mx.ietf.org with esmtp (Exim 4.33) id 1CEksp-0000sS-Df for ips-web-archive@ietf.org; Tue, 05 Oct 2004 04:41:22 -0400 Received: from localhost.localdomain ([127.0.0.1] helo=megatron.ietf.org) by megatron.ietf.org with esmtp (Exim 4.32) id 1CEkWl-0005vx-47; Tue, 05 Oct 2004 04:18:31 -0400 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by megatron.ietf.org with esmtp (Exim 4.32) id 1CEkPI-0004ya-V8 for ips@megatron.ietf.org; Tue, 05 Oct 2004 04:10:51 -0400 Received: from ietf-mx.ietf.org (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id EAA01350 for ; Tue, 5 Oct 2004 04:10:44 -0400 (EDT) Received: from system60.simplicato.com ([207.99.47.60] helo=host52a.simplicato.com) by ietf-mx.ietf.org with esmtp (Exim 4.33) id 1CEkYV-0006ND-9O for ips@ietf.org; Tue, 05 Oct 2004 04:20:19 -0400 Received: from localhost (localhost.simplicato.com [127.0.0.1]) by host52a.simplicato.com (Postfix) with ESMTP id 8183311E09C; Tue, 5 Oct 2004 04:10:43 -0400 (EDT) Received: from host52a.simplicato.com ([127.0.0.1]) by localhost (host52a.simplicato.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 80786-08; Tue, 5 Oct 2004 04:10:42 -0400 (EDT) Received: from mail.asomi.com (localhost.simplicato.com [127.0.0.1]) by host52a.simplicato.com (Postfix) with SMTP id 12BA811E096; Tue, 5 Oct 2004 04:10:42 -0400 (EDT) Received: from 62.90.193.206 (proxying for 62.90.193.206) (SquirrelMail authenticated user cait@asomi.com) by mail52a.simplicato.com with HTTP; Tue, 5 Oct 2004 03:10:42 -0500 (CDT) Message-ID: <9353.62.90.193.206.1096963842.squirrel@mail52a.simplicato.com> In-Reply-To: References: <52342.62.90.193.206.1096958721.squirrel@mail52a.simplicato.com> Date: Tue, 5 Oct 2004 03:10:42 -0500 (CDT) Subject: RE: [Ips] DA+iSER Flow Control, Limiting number of oustanding cmdsattarget From: "Caitlin Bestler" To: "John Hufferd" User-Agent: SquirrelMail/1.4.2 MIME-Version: 1.0 Content-Type: text/plain;charset=iso-8859-1 X-Priority: 3 Importance: Normal X-Virus-Scanned: by amavisd-new at simplicato.com Content-Transfer-Encoding: quoted-printable X-Spam-Score: 0.8 (/) X-Scan-Signature: b4a0a5f5992e2a4954405484e7717d8c Content-Transfer-Encoding: quoted-printable Cc: Caitlin Bestler , Mike Ko , ips@ietf.org, "Mallikarjun C." , Julian Satran , Alex Nezhinsky X-BeenThere: ips@ietf.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: IP Storage List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Sender: ips-bounces@ietf.org Errors-To: ips-bounces@ietf.org X-Spam-Score: 0.8 (/) X-Scan-Signature: 92df29fa99cf13e554b84c8374345c17 Content-Transfer-Encoding: quoted-printable John Hufferd said: > This is not a general User State application it is a very specific Kern= el > Mode operation. This is Storage I/O, and the server you are talking ab= out > is a very specially constructed kernel application that follows the rul= es > of SCSI, and the operations that run in a storage controller manages a > large cache of memory that is divided up between the what is Going out = and > coming in. And this is both going out and coming in the communication > link and going out and coming in from the SCSI LUN. iSCSI/iSER is just= a > part of the buffering and caching that is going on. There is various > different algorithms to deal with Cache, both write through and write > behind and algorithms to select the right buffer locations for new data > (from either side) and algorithms to decide which Cache buffers need to= be > written to disk, etc. The Caches/Buffers are usually all managed via > chains, and what comes in as an iSCSI buffer becomes a Cache buffer and > then a LUN I/O buffer, then a free buffer, then perhaps placed again on > the iSER buffer chain or on the LUN read chain, etc. You can almost ta= lk > about a Storage Controller as "Buffers are US". There can not be a > problem for a storage controller to service the line rate of the > communication link, or the storage controller will not operate correctl= y, > and that type of operation is fundamental to a storage controller. > Therefore to talk of this as if the storage controller is some general > user App is just not correct. This type of buffer balancing is how > storage controllers have always acted, and balancing buffers and caches= in > and out is not a problem. We do it all the time. The flow control of > iSCSI is all that is needed when we are working with Shared RQs, and ar= e > managing the buffers ourselves. > . > . I disagree. I believe it is fundamentally incorrect to propose a protocol as an IETF standard based upon such implementation assumptions. Especially when those assumptions are not stated. If that is the justification, then it the DataMover architecture at the minimimum must make it clear that the iSCSI layer MUST accept pushed iSCSI PDUs at line rate. That would make it a correct specification, in the sense of consistency. But I still do not think it would be wise. One of the goals of iSER should be to enable it to be implemented over RDMA RNICs using their *existing* local interfaces. If an intelligent NIC is going to support a special local interface specifically for iSCSI it might as well be iSCSI, not iSER. The goal is to reduce the amount of application-specific logic in the NIC, not to create a third interface in addition to those supporting generic RDMA and non-RDMA iSCSI. -- Caitlin Bestler http://asomi.com/ _______________________________________________ Ips mailing list Ips@ietf.org https://www1.ietf.org/mailman/listinfo/ips From ips-bounces@ietf.org Tue Oct 5 05:47:26 2004 Received: from ietf-mx.ietf.org (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id FAA08412 for ; Tue, 5 Oct 2004 05:47:26 -0400 (EDT) Received: from megatron.ietf.org ([132.151.6.71]) by ietf-mx.ietf.org with esmtp (Exim 4.33) id 1CEm47-00023Z-FO for ips-web-archive@ietf.org; Tue, 05 Oct 2004 05:57:03 -0400 Received: from localhost.localdomain ([127.0.0.1] helo=megatron.ietf.org) by megatron.ietf.org with esmtp (Exim 4.32) id 1CElqk-0000Xe-IS; Tue, 05 Oct 2004 05:43:14 -0400 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by megatron.ietf.org with esmtp (Exim 4.32) id 1CElmd-0008D8-9i for ips@megatron.ietf.org; Tue, 05 Oct 2004 05:38:59 -0400 Received: from ietf-mx.ietf.org (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id FAA07272 for ; Tue, 5 Oct 2004 05:38:57 -0400 (EDT) Received: from mtagate1.de.ibm.com ([195.212.29.150]) by ietf-mx.ietf.org with esmtp (Exim 4.33) id 1CElvs-0000pF-Az for ips@ietf.org; Tue, 05 Oct 2004 05:48:33 -0400 Received: from d12nrmr1507.megacenter.de.ibm.com (d12nrmr1507.megacenter.de.ibm.com [9.149.167.1]) by mtagate1.de.ibm.com (8.12.10/8.12.10) with ESMTP id i959cLfQ058426; Tue, 5 Oct 2004 09:38:21 GMT Received: from d12ml102.megacenter.de.ibm.com (d12av02.megacenter.de.ibm.com [9.149.165.228]) by d12nrmr1507.megacenter.de.ibm.com (8.12.10/NCO/VER6.6) with ESMTP id i959cKKO221580; Tue, 5 Oct 2004 11:38:20 +0200 In-Reply-To: <16737.43635.104900.308014@gargle.gargle.HOWL> To: Paul Koning Subject: RE: [Ips] Initiator CHAP authentication of target? MIME-Version: 1.0 X-Mailer: Lotus Notes Build V70_M2_07222004 Beta 2NP July 22, 2004 From: Julian Satran Message-ID: Date: Tue, 5 Oct 2004 11:38:19 +0200 X-MIMETrack: Serialize by Router on D12ML102/12/M/IBM(Release 6.5.1| March 5, 2004) at 05/10/2004 11:38:21, Serialize complete at 05/10/2004 11:38:21 X-Spam-Score: 0.5 (/) X-Scan-Signature: 6ffdee8af20de249c24731d8414917d3 Cc: ips@ietf.org, wrstuden@wasabisystems.com, marjorie.krueger@hp.com X-BeenThere: ips@ietf.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: IP Storage List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Content-Type: multipart/mixed; boundary="===============1472270193==" Sender: ips-bounces@ietf.org Errors-To: ips-bounces@ietf.org X-Spam-Score: 0.5 (/) X-Scan-Signature: 2ed806e2f53ff1a061ad4f97e00345ac This is a multipart message in MIME format. --===============1472270193== Content-Type: multipart/alternative; boundary="=_alternative 003005FDC2256F24_=" This is a multipart message in MIME format. --=_alternative 003005FDC2256F24_= Content-Type: text/plain; charset="US-ASCII" Paul, Please read John's answer. CHAP does not require (AFAIK) the use of CHAP_N. Julo Paul Koning 04/10/04 21:54 To marjorie.krueger@hp.com cc Julian Satran/Haifa/IBM@IBMIL, ips@ietf.org, wrstuden@wasabisystems.com Subject RE: [Ips] Initiator CHAP authentication of target? >>>>> "Marjorie" == Marjorie Krueger writes: Marjorie> While it may not be "detectable" by the target, it Marjorie> certainly seems like "expected behavior" (ie, following the Marjorie> protocol) is that the initiator MUST use the CHAP_N and Marjorie> CHAP_R supplied in the login response - would you agree? Marjorie> When testing an initiator implementation, if one can prove Marjorie> that a certain initiator isn't using the values supplied, Marjorie> one can say that the initiator is not correctly Marjorie> implemented? I certainly would say that. If we hadn't wanted the initiator to look at CHAP_N, we would have left it out of the protocol. The CHAP exchange proves that you know the secret that goes with name CHAP_N. This is testable, too. If an exchange succeeds, and a repeat of that exchange with a different CHAP_N in the message also succeeds, then the initiator was ignoring CHAP_N. paul --=_alternative 003005FDC2256F24_= Content-Type: text/html; charset="US-ASCII"
Paul,

Please read John's answer. CHAP does not require (AFAIK) the use of CHAP_N.

Julo


Paul Koning <pkoning@equallogic.com>

04/10/04 21:54

To
marjorie.krueger@hp.com
cc
Julian Satran/Haifa/IBM@IBMIL, ips@ietf.org, wrstuden@wasabisystems.com
Subject
RE: [Ips] Initiator CHAP authentication of target?





>>>>> "Marjorie" == Marjorie Krueger <Krueger> writes:

Marjorie> While it may not be "detectable" by the target, it
Marjorie> certainly seems like "expected behavior" (ie, following the
Marjorie> protocol) is that the initiator MUST use the CHAP_N and
Marjorie> CHAP_R supplied in the login response - would you agree?
Marjorie> When testing an initiator implementation, if one can prove
Marjorie> that a certain initiator isn't using the values supplied,
Marjorie> one can say that the initiator is not correctly
Marjorie> implemented?

I certainly would say that.  If we hadn't wanted the initiator to look
at CHAP_N, we would have left it out of the protocol.

The CHAP exchange proves that you know the secret that goes with name
CHAP_N.

This is testable, too.  If an exchange succeeds, and a repeat of that
exchange with a different CHAP_N in the message also succeeds, then
the initiator was ignoring CHAP_N.

                paul
--=_alternative 003005FDC2256F24_=-- --===============1472270193== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline Content-Transfer-Encoding: 7bit _______________________________________________ Ips mailing list Ips@ietf.org https://www1.ietf.org/mailman/listinfo/ips --===============1472270193==-- From ips-bounces@ietf.org Tue Oct 5 09:55:47 2004 Received: from ietf-mx.ietf.org (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id JAA29209 for ; Tue, 5 Oct 2004 09:55:47 -0400 (EDT) Received: from megatron.ietf.org ([132.151.6.71]) by ietf-mx.ietf.org with esmtp (Exim 4.33) id 1CEpwT-000098-IK for ips-web-archive@ietf.org; Tue, 05 Oct 2004 10:05:25 -0400 Received: from localhost.localdomain ([127.0.0.1] helo=megatron.ietf.org) by megatron.ietf.org with esmtp (Exim 4.32) id 1CEpkC-00053G-HN; Tue, 05 Oct 2004 09:52:44 -0400 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by megatron.ietf.org with esmtp (Exim 4.32) id 1CEpgO-0004Wq-48 for ips@megatron.ietf.org; Tue, 05 Oct 2004 09:48:48 -0400 Received: from ietf-mx.ietf.org (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id JAA28568 for ; Tue, 5 Oct 2004 09:48:46 -0400 (EDT) Received: from mononoke.wasabisystems.com ([66.173.145.228]) by ietf-mx.ietf.org with esmtp (Exim 4.33) id 1CEppg-0007cQ-9q for ips@ietf.org; Tue, 05 Oct 2004 09:58:24 -0400 Received: from [192.168.1.125] (unknown [66.173.145.226]) by mononoke.wasabisystems.com (Postfix) with ESMTP id 2AD5C870AA; Tue, 5 Oct 2004 09:48:43 -0400 (EDT) In-Reply-To: References: Mime-Version: 1.0 (Apple Message framework v619) Message-Id: <40ABE948-16D5-11D9-8100-000A95D14BEC@wasabisystems.com> From: William Studenmund Subject: Re: [Ips] Initiator CHAP authentication of target? Date: Tue, 5 Oct 2004 09:48:27 -0400 To: John Hufferd X-Pgp-Agent: GPGMail 1.0 (v30, 10.3) X-Mailer: Apple Mail (2.619) X-Spam-Score: 0.0 (/) X-Scan-Signature: 68ba2b07ef271dba6ee42a93832cfa4c Cc: ips@ietf.org, Julian Satran , "Krueger, Marjorie" X-BeenThere: ips@ietf.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: IP Storage List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Content-Type: multipart/mixed; boundary="===============1828808974==" Sender: ips-bounces@ietf.org Errors-To: ips-bounces@ietf.org X-Spam-Score: 0.0 (/) X-Scan-Signature: 6ba8aaf827dcb437101951262f69b3de --===============1828808974== Content-Type: multipart/signed; protocol="application/pgp-signature"; micalg=pgp-sha1; boundary="Apple-Mail-4-565314720" --Apple-Mail-4-565314720 Content-Type: multipart/signed; micalg=sha1; boundary=Apple-Mail-3-565314714; protocol="application/pkcs7-signature" --Apple-Mail-3-565314714 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: quoted-printable On Oct 4, 2004, at 3:54 PM, John Hufferd wrote: > Marjorie, > As I read the CHAP Spec., the name is not used in the computation of=20= > the check/response. =A0It is used as an index to quickly locate the=20 > approprate Secret. =A0With that in mind, regardless of the Response to=20= > the Challenge, the name is not used in the computation (Hash); if the=20= > computation works out correctly, that is really all that is needed.=20 > =A0And I do not see any security exposure based on a different name,=20= > except if the same Secret is used by more than one storage controller=20= > which is forbidden by the iSCSI spec. =A0Therefore, it seems to me the=20= > only way one could tell if the other side is ignoring the Name that is=20= > sent, is to send an invalid name, and then complain that the link was=20= > not dropped, or given an invalid protocol message. =A0Which I think is = a=20 > strange way to operate. =A0So technically, the name must be sent, but = I=20 > do not see a requirement that it must be check in some special way=20 > that excludes ignoring it. =A0I would have like us to have a default = of=20 > the iSCSI Node name, if the CHAP_N keyword is not sent, but since we=20= > did not do that, I think that in most cases to ease the Administrative=20= > burden of setting yet another name, most installations will use the=20 > iSCSI Node Name I think the issue that Marjorie is describing is a case where the=20 target sends a CHAP_N which is not the iSCSI node name, yet the=20 initiator uses the node name to look up the secret for calculating the=20= CHAP hash. Thus it doesn't get the same secret and fails to=20 authenticate the target. I agree that this behavior does not open up a security vulnerability=20 nor does it impact how the hash is calculated. I also agree that it's fine for a target (or specifically its=20 administrator) to choose to use the target name for the CHAP name, but=20= I think that decision should be on the target side, not the initiator=20 side. Take care, Bill= --Apple-Mail-3-565314714 Content-Type: application/pkcs7-signature; name=smime.p7s Content-Disposition: attachment; filename=smime.p7s Content-Transfer-Encoding: base64 MIAGCSqGSIb3DQEHAqCAMIACAQExCzAJBgUrDgMCGgUAMIAGCSqGSIb3DQEHAQAAoIIFfDCCBXgw ggNgoAMCAQICAUYwDQYJKoZIhvcNAQEEBQAwga0xCzAJBgNVBAYTAlVTMREwDwYDVQQIEwhOZXcg WW9yazERMA8GA1UEBxMITmV3IFlvcmsxHTAbBgNVBAoTFFdhc2FiaSBTeXN0ZW1zLCBJbmMuMQww CgYDVQQLEwNHSFExHzAdBgNVBAMTFldhc2FiaSBTeXN0ZW1zIFJvb3QgQ0ExKjAoBgkqhkiG9w0B CQEWG3dlYm1hc3RlckB3YXNhYmlzeXN0ZW1zLmNvbTAeFw0wMzA4MTExODU2NDZaFw0wODA2MjUx ODU2NDZaMIGXMQswCQYDVQQGEwJVUzETMBEGA1UECBMKQ2FsaWZvcm5pYTESMBAGA1UEBxMJU2Fu IERpZWdvMRcwFQYDVQQKEw5XYXNhYmkgU3lzdGVtczEbMBkGA1UEAxMSV2lsbGlhbSBTdHVkZW5t dW5kMSkwJwYJKoZIhvcNAQkBFhp3cnN0dWRlbkB3YXNhYmlzeXN0ZW1zLmNvbTCBnzANBgkqhkiG 9w0BAQEFAAOBjQAwgYkCgYEA9CZhg4GVWYTRGcJz8lU1ipzInlBuGqfZQs88Z1rTP7+c3AyhKwpE TyTmK4UhxslzKa93YLqifLxZKYt1jI6FNq40IQWCKBVywWBwmL5guCu0851f9ywH5Uj2wJdiDrOI sOJKNpwNqml8bFdTrFS2qvyCxTIWVMLYAzUBBcl2F7MCAwEAAaOCATkwggE1MAkGA1UdEwQCMAAw LAYJYIZIAYb4QgENBB8WHU9wZW5TU0wgR2VuZXJhdGVkIENlcnRpZmljYXRlMB0GA1UdDgQWBBTt mjHbSBlMV48RWOBETM3EV0Tm/jCB2gYDVR0jBIHSMIHPgBQrBOnN88CNCHPaV5wLS212tcrgBaGB s6SBsDCBrTELMAkGA1UEBhMCVVMxETAPBgNVBAgTCE5ldyBZb3JrMREwDwYDVQQHEwhOZXcgWW9y azEdMBsGA1UEChMUV2FzYWJpIFN5c3RlbXMsIEluYy4xDDAKBgNVBAsTA0dIUTEfMB0GA1UEAxMW V2FzYWJpIFN5c3RlbXMgUm9vdCBDQTEqMCgGCSqGSIb3DQEJARYbd2VibWFzdGVyQHdhc2FiaXN5 c3RlbXMuY29tggEAMA0GCSqGSIb3DQEBBAUAA4ICAQCpxoPEYunR7qiXyycEZd0UIatSt/WalMoH nuN9S421iS/sD1wSDqzHyM5YyUWn4z4o6VhT2kD9UV4lk5aXxxefEfGzFcR7Xyah7tO3ziuReFfx qUP5+DTlwDuUBc8Iow6NwY9GlRCKFSsEroLWLGODyYWgN1ojjKOtcXRiOZkYTP3VfDUV2/Xrhh9O l6+3Otf069+KWeOWedWeMsbe1rac96yPEjPogToWkBXVJEeEQktd0BLETt7GUuSZg2Fho/nslqZv nnnKZmrOfdFy9iXDL9T0Hp/OHf5B9RP+r00BJJFO8JzBPIL2IA3CBPApOQty5A00Jg1CGfDQxiV3 s8G0aRUbmFqg9r0OfhPYjJ8BHfw1HHWb7Cu91Sw+d5UnYracZhsR2Gc5H5CnfS6IZhc7ulfnSX8f I4BXR4vbeRUG76J3rkkbkO1haYHLk5w5RlICHsJOBSElG9ggLBWNIfUZwAzo84lfmDmJMP1YCOmi DFCbW1DgxhqVFpYEf5Pq2pVcCguopK2G9oZjH5fxq6ehisk9dN+7NEQ4OSqC43ehaStYOoQAJruN e6e5tCtRMx7bMI7eVPiHxE9/Y38Q+sc5oRN9mCyyAXMweuMRNHAHAWdFB7NhHD6gBa2lh91zIMdf iSVRlBXoMrBKJX6VecgnhvUp4G6QAnN7EMu6QHlc/jGCA0swggNHAgEBMIGzMIGtMQswCQYDVQQG EwJVUzERMA8GA1UECBMITmV3IFlvcmsxETAPBgNVBAcTCE5ldyBZb3JrMR0wGwYDVQQKExRXYXNh YmkgU3lzdGVtcywgSW5jLjEMMAoGA1UECxMDR0hRMR8wHQYDVQQDExZXYXNhYmkgU3lzdGVtcyBS b290IENBMSowKAYJKoZIhvcNAQkBFht3ZWJtYXN0ZXJAd2FzYWJpc3lzdGVtcy5jb20CAUYwCQYF Kw4DAhoFAKCCAe0wGAYJKoZIhvcNAQkDMQsGCSqGSIb3DQEHATAcBgkqhkiG9w0BCQUxDxcNMDQx MDA1MTM0ODI3WjAjBgkqhkiG9w0BCQQxFgQUbzaLe2Is2qDEOZyScAakvKfeCv0wgcQGCSsGAQQB gjcQBDGBtjCBszCBrTELMAkGA1UEBhMCVVMxETAPBgNVBAgTCE5ldyBZb3JrMREwDwYDVQQHEwhO ZXcgWW9yazEdMBsGA1UEChMUV2FzYWJpIFN5c3RlbXMsIEluYy4xDDAKBgNVBAsTA0dIUTEfMB0G A1UEAxMWV2FzYWJpIFN5c3RlbXMgUm9vdCBDQTEqMCgGCSqGSIb3DQEJARYbd2VibWFzdGVyQHdh c2FiaXN5c3RlbXMuY29tAgFGMIHGBgsqhkiG9w0BCRACCzGBtqCBszCBrTELMAkGA1UEBhMCVVMx ETAPBgNVBAgTCE5ldyBZb3JrMREwDwYDVQQHEwhOZXcgWW9yazEdMBsGA1UEChMUV2FzYWJpIFN5 c3RlbXMsIEluYy4xDDAKBgNVBAsTA0dIUTEfMB0GA1UEAxMWV2FzYWJpIFN5c3RlbXMgUm9vdCBD QTEqMCgGCSqGSIb3DQEJARYbd2VibWFzdGVyQHdhc2FiaXN5c3RlbXMuY29tAgFGMA0GCSqGSIb3 DQEBAQUABIGAa1eCCVfu0jvMnAE0WQRKKTRQ3/wbQ1HveKvcNEkbJ8otI4W454IRWuM8tgoo9wx+ SqjwUx4d/Dv5j82sIP0hXOQQg6DTD2qyhJxyiJ75WYNwagpVjX0v9ZTNaq4CCcElu5HUV0RW1bnR kH2XMnYsftwaRCeYO6FjwMPsPZjbG0wAAAAAAAA= --Apple-Mail-3-565314714-- --Apple-Mail-4-565314720 content-type: application/pgp-signature; x-mac-type=70674453; name=PGP.sig content-description: This is a digitally signed message part content-disposition: inline; filename=PGP.sig content-transfer-encoding: 7bit Content-Transfer-Encoding: 7bit Content-Transfer-Encoding: 7bit -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.2.3 (Darwin) iD8DBQFBYqY5DJT2Egh26K0RAmVPAJwKsr+tE2a2wnggt2FUEK2AMukA/ACghSOx gZu6GzPJzJ+WC2igTTn9YbQ= =lNPj -----END PGP SIGNATURE----- --Apple-Mail-4-565314720-- --===============1828808974== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline Content-Transfer-Encoding: 7bit _______________________________________________ Ips mailing list Ips@ietf.org https://www1.ietf.org/mailman/listinfo/ips --===============1828808974==-- From ips-bounces@ietf.org Tue Oct 5 11:09:30 2004 Received: from ietf-mx.ietf.org (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id LAA06185 for ; Tue, 5 Oct 2004 11:09:30 -0400 (EDT) Received: from megatron.ietf.org ([132.151.6.71]) by ietf-mx.ietf.org with esmtp (Exim 4.33) id 1CEr5o-0002pE-W4 for ips-web-archive@ietf.org; Tue, 05 Oct 2004 11:19:10 -0400 Received: from localhost.localdomain ([127.0.0.1] helo=megatron.ietf.org) by megatron.ietf.org with esmtp (Exim 4.32) id 1CEqv9-0006M7-TH; Tue, 05 Oct 2004 11:08:07 -0400 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by megatron.ietf.org with esmtp (Exim 4.32) id 1CEqsk-0005vF-Ep for ips@megatron.ietf.org; Tue, 05 Oct 2004 11:05:38 -0400 Received: from ietf-mx.ietf.org (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id LAA05955 for ; Tue, 5 Oct 2004 11:05:36 -0400 (EDT) Received: from taurus.voltaire.com ([212.143.27.73]) by ietf-mx.ietf.org with esmtp (Exim 4.33) id 1CEr23-0001kO-W3 for ips@ietf.org; Tue, 05 Oct 2004 11:15:16 -0400 X-MimeOLE: Produced By Microsoft Exchange V6.5.7226.0 Content-class: urn:content-classes:message MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable Subject: RE: [Ips] DA+iSER Flow Control, Limiting number of oustanding cmds attarget Date: Tue, 5 Oct 2004 17:04:56 +0200 Message-ID: Thread-Topic: [Ips] DA+iSER Flow Control, Limiting number of oustanding cmds attarget Thread-Index: AcSqRLy/YsfBk2oRSIyA/6tJBCwIzQAmgdCQ From: "Alex Nezhinsky" To: "Mike Ko" X-Spam-Score: 0.0 (/) X-Scan-Signature: 3e15cc4fdc61d7bce84032741d11c8e5 Content-Transfer-Encoding: quoted-printable Cc: Caitlin Bestler , ips@ietf.org, Julian Satran , "Mallikarjun C." X-BeenThere: ips@ietf.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: IP Storage List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Sender: ips-bounces@ietf.org Errors-To: ips-bounces@ietf.org X-Spam-Score: 0.0 (/) X-Scan-Signature: f607d15ccc2bc4eaf3ade8ffa8af02a0 Content-Transfer-Encoding: quoted-printable > -----Original Message----- > From: Mike Ko [mailto:mako@almaden.ibm.com]=20 > Sent: Monday, October 04, 2004 9:00 PM > To: Alex Nezhinsky > Cc: Julian Satran; Caitlin Bestler; Mallikarjun C.; ips@ietf.org > Subject: RE: [Ips] DA+iSER Flow Control, Limiting number of=20 > oustanding cmds attarget > From the target to the initiator, unsocited NOP-in are regulated by NOP-out.=20 I don't quite understand this statement. NOP-INs, which are unsolicited can't be regulated by their replies. Do you mean that there can be only one outstanding NOP-IN at a time? This would clearly solve the problem, but I could not find such limitation in iSCSI specification. For example, 10.4.11 says: "When MaxCmdSN changes at the target while the target has no pending PDUs to convey this information to the initiator, it MUST generate a NOP-IN to carry the new MaxCmdSN". This seems to imply a possibility of sending a few NOP-INs without waiting for a response NOP-OUT, because some resources may become free gradually. > The unsolicited PDUs which are not covered by any existing flow control are Async Messages. > One of the solution proposed is to set a declared limit on the number of such unsolicited requests. Is it going to be declared in iSER spec as a const., as a negotiable/declarative parameter? Is it going to be declared in iSER spec at all, or rather left to implementations? Does this limit cover all the PDU types involved or is declared per type? > The next question is how does each side determine when is the=20 > limit replenished. One possible solution is as follows: >=20 > For the initiator, the ExpCmdSN from the target will be used=20 > to advance the credit. After sending PDUs with the immediate=20 > flag set, when the initiator sends any iSCSI control-type PDU=20 > with a new CmdSN without the immediate flag, if the target=20 > responds with an iSCSI control-type PDU with ExpCmdSN set to=20 > CmdSN + 1, then credit at the initiator is restored to its full value. >=20 > For the target, the ExpStatSN from the initiator will be used=20 > to advance the credit. When the initiator sends any PDU=20 > which contains the ExpStatSN that acknowledges the StatSN in=20 > the Async Message, the number of credits at the target used=20 > up by that Async Message will be restored. Sure, this works. These schemes, though, will force iSER DataMover into a complex state tracking (just try to visualize the credit update condition and the internal state varaibles involved in it). If a separate credit counter is implemented, it only has to be updated from any incoming PDU's header and checked+decremented upon every relevant send opration. _______________________________________________ Ips mailing list Ips@ietf.org https://www1.ietf.org/mailman/listinfo/ips From ips-bounces@ietf.org Tue Oct 5 12:48:56 2004 Received: from ietf-mx.ietf.org (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id MAA13527 for ; Tue, 5 Oct 2004 12:48:56 -0400 (EDT) Received: from megatron.ietf.org ([132.151.6.71]) by ietf-mx.ietf.org with esmtp (Exim 4.33) id 1CEse5-00078Q-2x for ips-web-archive@ietf.org; Tue, 05 Oct 2004 12:58:37 -0400 Received: from localhost.localdomain ([127.0.0.1] helo=megatron.ietf.org) by megatron.ietf.org with esmtp (Exim 4.32) id 1CEsNp-0004Oz-0a; Tue, 05 Oct 2004 12:41:49 -0400 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by megatron.ietf.org with esmtp (Exim 4.32) id 1CEsJe-0003jt-VF for ips@megatron.ietf.org; Tue, 05 Oct 2004 12:37:31 -0400 Received: from ietf-mx.ietf.org (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id MAA12594 for ; Tue, 5 Oct 2004 12:37:27 -0400 (EDT) Received: from e3.ny.us.ibm.com ([32.97.182.103]) by ietf-mx.ietf.org with esmtp (Exim 4.33) id 1CEsSv-0004ri-4o for ips@ietf.org; Tue, 05 Oct 2004 12:47:08 -0400 Received: from northrelay02.pok.ibm.com (northrelay02.pok.ibm.com [9.56.224.150]) by e3.ny.us.ibm.com (8.12.10/8.12.9) with ESMTP id i95GarZI783560; Tue, 5 Oct 2004 12:36:54 -0400 Received: from d01ml604.pok.ibm.com (d01av04.pok.ibm.com [9.56.224.64]) by northrelay02.pok.ibm.com (8.12.10/NCO/VER6.6) with ESMTP id i95Gc7H6144322; Tue, 5 Oct 2004 12:38:08 -0400 In-Reply-To: Importance: Normal MIME-Version: 1.0 To: "Alex Nezhinsky" Subject: RE: [Ips] DA+iSER Flow Control, Limiting number of oustanding cmds attarget X-Mailer: Lotus Notes Release 6.5.2 June 01, 2004 Message-ID: From: Mike Ko Date: Tue, 5 Oct 2004 09:36:51 -0700 X-MIMETrack: Serialize by Router on D01ML604/01/M/IBM(Release 6.51HF582 | September 27, 2004) at 10/05/2004 12:36:53, Serialize complete at 10/05/2004 12:36:53 Content-Type: text/plain; charset="US-ASCII" X-Spam-Score: 0.0 (/) X-Scan-Signature: 14582b0692e7f70ce7111d04db3781c8 Cc: Caitlin Bestler , ips@ietf.org, "Mallikarjun C." , Julian Satran , Mike Ko X-BeenThere: ips@ietf.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: IP Storage List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Sender: ips-bounces@ietf.org Errors-To: ips-bounces@ietf.org X-Spam-Score: 0.0 (/) X-Scan-Signature: 6e922792024732fb1bb6f346e63517e4 Alex, My comments are in line preceded by [mk]. Mike To: "Mike Ko" cc: "Julian Satran" , "Caitlin Bestler" , "Mallikarjun C." , Subject: RE: [Ips] DA+iSER Flow Control, Limiting number of oustanding cmds attarget > -----Original Message----- > From: Mike Ko [mailto:mako@almaden.ibm.com] > Sent: Monday, October 04, 2004 9:00 PM > To: Alex Nezhinsky > Cc: Julian Satran; Caitlin Bestler; Mallikarjun C.; ips@ietf.org > Subject: RE: [Ips] DA+iSER Flow Control, Limiting number of > oustanding cmds attarget > From the target to the initiator, unsocited NOP-in are regulated by NOP-out. I don't quite understand this statement. NOP-INs, which are unsolicited can't be regulated by their replies. Do you mean that there can be only one outstanding NOP-IN at a time? This would clearly solve the problem, but I could not find such limitation in iSCSI specification. For example, 10.4.11 says: "When MaxCmdSN changes at the target while the target has no pending PDUs to convey this information to the initiator, it MUST generate a NOP-IN to carry the new MaxCmdSN". This seems to imply a possibility of sending a few NOP-INs without waiting for a response NOP-OUT, because some resources may become free gradually. [mk] More correctly, I should state that the number of unsolicited requests from the target are regulated by the declared limit. The credit replenishment rule (using ExpStatSN for example) applies in the general case. However, for each NOP-in sent by the target which expects a corresponding NOP-out (TTT is set to a valid value), then when the initiator responds with the NOP-out, the iSER layer can use this to replenish the receive buffer. [mk] > The unsolicited PDUs which are not covered by any existing flow control are Async Messages. > One of the solution proposed is to set a declared limit on the number of such unsolicited requests. Is it going to be declared in iSER spec as a const., as a negotiable/declarative parameter? Is it going to be declared in iSER spec at all, or rather left to implementations? Does this limit cover all the PDU types involved or is declared per type? [mk] The intent is to have the initiator and the target declare to each other during the iSCSI Login the maximum number of unsolicited requests (as specified in my earlier e-mail) that the other side can send. [mk] > The next question is how does each side determine when is the > limit replenished. One possible solution is as follows: > > For the initiator, the ExpCmdSN from the target will be used > to advance the credit. After sending PDUs with the immediate > flag set, when the initiator sends any iSCSI control-type PDU > with a new CmdSN without the immediate flag, if the target > responds with an iSCSI control-type PDU with ExpCmdSN set to > CmdSN + 1, then credit at the initiator is restored to its full value. > > For the target, the ExpStatSN from the initiator will be used > to advance the credit. When the initiator sends any PDU > which contains the ExpStatSN that acknowledges the StatSN in > the Async Message, the number of credits at the target used > up by that Async Message will be restored. Sure, this works. These schemes, though, will force iSER DataMover into a complex state tracking (just try to visualize the credit update condition and the internal state varaibles involved in it). [mk] I agree. That's why John Hufferd suggesting using the Shared Receive Queue instead. [mk] If a separate credit counter is implemented, it only has to be updated from any incoming PDU's header and checked+decremented upon every relevant send opration. [mk] Why would a separate credit counter create less "complex state tracking" problem than the one that relies on ExpCmdSN (or ExpStatSN)? [mk] _______________________________________________ Ips mailing list Ips@ietf.org https://www1.ietf.org/mailman/listinfo/ips From ips-bounces@ietf.org Tue Oct 5 12:49:57 2004 Received: from ietf-mx.ietf.org (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id MAA13596 for ; Tue, 5 Oct 2004 12:49:57 -0400 (EDT) Received: from megatron.ietf.org ([132.151.6.71]) by ietf-mx.ietf.org with esmtp (Exim 4.33) id 1CEsf3-00079U-Pz for ips-web-archive@ietf.org; Tue, 05 Oct 2004 12:59:38 -0400 Received: from localhost.localdomain ([127.0.0.1] helo=megatron.ietf.org) by megatron.ietf.org with esmtp (Exim 4.32) id 1CEsOF-0004VJ-6n; Tue, 05 Oct 2004 12:42:15 -0400 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by megatron.ietf.org with esmtp (Exim 4.32) id 1CEsMe-0004Ao-70 for ips@megatron.ietf.org; Tue, 05 Oct 2004 12:40:36 -0400 Received: from ietf-mx.ietf.org (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id MAA12746 for ; Tue, 5 Oct 2004 12:40:31 -0400 (EDT) Received: from e33.co.us.ibm.com ([32.97.110.131]) by ietf-mx.ietf.org with esmtp (Exim 4.33) id 1CEsVL-0005J8-U5 for ips@ietf.org; Tue, 05 Oct 2004 12:49:36 -0400 Received: from westrelay02.boulder.ibm.com (westrelay02.boulder.ibm.com [9.17.195.11]) by e33.co.us.ibm.com (8.12.10/8.12.9) with ESMTP id i95Gcq6A288636; Tue, 5 Oct 2004 12:38:52 -0400 Received: from d03nm115.boulder.ibm.com (d03av02.boulder.ibm.com [9.17.195.168]) by westrelay02.boulder.ibm.com (8.12.10/NCO/VER6.6) with ESMTP id i95GcjcO067124; Tue, 5 Oct 2004 10:38:51 -0600 In-Reply-To: <9353.62.90.193.206.1096963842.squirrel@mail52a.simplicato.com> To: "Caitlin Bestler" MIME-Version: 1.0 Subject: RE: [Ips] DA+iSER Flow Control, Limiting number of oustanding cmdsattarget X-Mailer: Lotus Notes Release 6.0.2CF1 June 9, 2003 From: John Hufferd Message-ID: Date: Tue, 5 Oct 2004 09:38:38 -0700 X-MIMETrack: Serialize by Router on D03NM115/03/M/IBM(Release 6.51HF338 | June 21, 2004) at 10/05/2004 10:38:51, Serialize complete at 10/05/2004 10:38:51 X-Spam-Score: 1.0 (+) X-Scan-Signature: 32029c790f79bd4a84a26bd2915c54b9 Cc: Caitlin Bestler , Mike Ko , ips@ietf.org, "Mallikarjun C." , Julian Satran , Alex Nezhinsky X-BeenThere: ips@ietf.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: IP Storage List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Content-Type: multipart/mixed; boundary="===============1709706597==" Sender: ips-bounces@ietf.org Errors-To: ips-bounces@ietf.org X-Spam-Score: 0.0 (/) X-Scan-Signature: e178fd6cb61ffb6940cd878e7fea8606 This is a multipart message in MIME format. --===============1709706597== Content-Type: multipart/alternative; boundary="=_alternative 005B55F488256F24_=" This is a multipart message in MIME format. --=_alternative 005B55F488256F24_= Content-Type: text/plain; charset="US-ASCII" Caitlin, I have no idea why you say that we need something special in the RNIC that is used for iSCSI/iSER. That is NOT the case, and I did not intend to suggest anything like that. The only issue you keep bringing up is with buffering, which is fully handled by the Storage Controllers and Shared RQs. And the point of all the implementations is how fast we can service the data coming on the link. I do not think we need to say anything like the Goal is to operate at link speed! The purpose of SRQs is to permit application such as iSER to be able to respond at link speed, while keeping the use of pre posted buffers to a minimum. . . John L. Hufferd Senior Technical Staff Member (STSM) IBM/System Group, San Jose CA Main Office: (408) 256-0403, Tie: 276-0403, eFax: (408) 904-4688 Alt Office: (408) 997-6136, Cell: (408) 499-9702 Internet Address: hufferd@us.ibm.com "Caitlin Bestler" 10/05/2004 01:10 AM To John Hufferd/San Jose/IBM@IBMUS cc "Caitlin Bestler" , "Alex Nezhinsky" , "Mallikarjun C." , ips@ietf.org, "Julian Satran" , "Mike Ko" Subject RE: [Ips] DA+iSER Flow Control, Limiting number of oustanding cmdsattarget John Hufferd said: > This is not a general User State application it is a very specific Kernel > Mode operation. This is Storage I/O, and the server you are talking about > is a very specially constructed kernel application that follows the rules > of SCSI, and the operations that run in a storage controller manages a > large cache of memory that is divided up between the what is Going out and > coming in. And this is both going out and coming in the communication > link and going out and coming in from the SCSI LUN. iSCSI/iSER is just a > part of the buffering and caching that is going on. There is various > different algorithms to deal with Cache, both write through and write > behind and algorithms to select the right buffer locations for new data > (from either side) and algorithms to decide which Cache buffers need to be > written to disk, etc. The Caches/Buffers are usually all managed via > chains, and what comes in as an iSCSI buffer becomes a Cache buffer and > then a LUN I/O buffer, then a free buffer, then perhaps placed again on > the iSER buffer chain or on the LUN read chain, etc. You can almost talk > about a Storage Controller as "Buffers are US". There can not be a > problem for a storage controller to service the line rate of the > communication link, or the storage controller will not operate correctly, > and that type of operation is fundamental to a storage controller. > Therefore to talk of this as if the storage controller is some general > user App is just not correct. This type of buffer balancing is how > storage controllers have always acted, and balancing buffers and caches in > and out is not a problem. We do it all the time. The flow control of > iSCSI is all that is needed when we are working with Shared RQs, and are > managing the buffers ourselves. > . > . I disagree. I believe it is fundamentally incorrect to propose a protocol as an IETF standard based upon such implementation assumptions. Especially when those assumptions are not stated. If that is the justification, then it the DataMover architecture at the minimimum must make it clear that the iSCSI layer MUST accept pushed iSCSI PDUs at line rate. That would make it a correct specification, in the sense of consistency. But I still do not think it would be wise. One of the goals of iSER should be to enable it to be implemented over RDMA RNICs using their *existing* local interfaces. If an intelligent NIC is going to support a special local interface specifically for iSCSI it might as well be iSCSI, not iSER. The goal is to reduce the amount of application-specific logic in the NIC, not to create a third interface in addition to those supporting generic RDMA and non-RDMA iSCSI. -- Caitlin Bestler http://asomi.com/ --=_alternative 005B55F488256F24_= Content-Type: text/html; charset="US-ASCII"
Caitlin,
I have no idea why you say that we need something special in the RNIC that is used for iSCSI/iSER.  That is NOT the case, and I did not intend to suggest anything like that.  The only issue you keep bringing up is with buffering, which is fully handled by the Storage Controllers and Shared RQs.  And the point of all the implementations is how fast we can service the data coming on the link.  I do not think we need to say anything like the Goal is to operate at link speed!  The purpose of SRQs is to permit application such as iSER to be able to respond at link speed, while keeping the use of pre posted buffers to a minimum.
.
.
John L. Hufferd
Senior Technical Staff Member (STSM)
IBM/System Group, San Jose CA
Main Office: (408) 256-0403, Tie: 276-0403, eFax: (408) 904-4688
Alt Office: (408) 997-6136, Cell: (408) 499-9702
Internet Address: hufferd@us.ibm.com


"Caitlin Bestler" <cait@asomi.com>

10/05/2004 01:10 AM

To
John Hufferd/San Jose/IBM@IBMUS
cc
"Caitlin Bestler" <cait@asomi.com>, "Alex Nezhinsky" <alexn@voltaire.com>, "Mallikarjun C." <cbm@rose.hp.com>, ips@ietf.org, "Julian Satran" <julian_satran@il.ibm.com>, "Mike Ko" <mako@almaden.ibm.com>
Subject
RE: [Ips] DA+iSER Flow Control,          Limiting number of oustanding cmdsattarget






John Hufferd said:
> This is not a general User State application it is a very specific Kernel
> Mode operation.  This is Storage I/O, and the server you are talking about
> is a very specially constructed kernel application that follows the rules
> of SCSI, and the operations that run in a storage controller manages a
> large cache of memory that is divided up between the what is Going out and
> coming in.  And this is both going out and coming in the communication
> link and going out and coming in from the SCSI LUN.  iSCSI/iSER is just a
> part of the buffering and caching that is going on.  There is various
> different algorithms to deal with Cache, both write through and write
> behind and algorithms to select the right buffer locations for new data
> (from either side) and algorithms to decide which Cache buffers need to be
> written to disk, etc.  The Caches/Buffers are usually all managed via
> chains, and what comes in as an iSCSI buffer becomes a Cache buffer and
> then a LUN I/O buffer, then a free buffer, then perhaps placed again on
> the iSER buffer chain or on the LUN read chain, etc.  You can almost talk
> about a Storage Controller as "Buffers are US".  There can not be a
> problem for a storage controller to service the line rate of the
> communication link, or the storage controller will not operate correctly,
> and that type of operation is fundamental to a storage controller.
> Therefore to talk of this as if the storage controller is some general
> user App is just not correct.  This type of buffer balancing is how
> storage controllers have always acted, and balancing buffers and caches in
> and out is not a problem.  We do it all the time.  The flow control of
> iSCSI is all that is needed when we are working with Shared RQs, and are
> managing the buffers ourselves.
> .
> .

I disagree.  I believe it is fundamentally incorrect to propose a
protocol as an IETF standard based upon such implementation assumptions.

Especially when those assumptions are not stated.

If that is the justification, then it the DataMover architecture
at the minimimum must make it clear that the iSCSI layer MUST
accept pushed iSCSI PDUs at line rate.

That would make it a correct specification, in the sense of
consistency.  But I still do not think it would be wise.
One of the goals of iSER should be to enable it to be
implemented over RDMA RNICs using their *existing* local
interfaces.  If an intelligent NIC is going to support a
special local interface specifically for iSCSI it might
as well be iSCSI, not iSER.  The goal is to reduce the
amount of application-specific logic in the NIC, not to
create a third interface in addition to those supporting
generic RDMA and non-RDMA iSCSI.




--
Caitlin Bestler
http://asomi.com/
--=_alternative 005B55F488256F24_=-- --===============1709706597== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline Content-Transfer-Encoding: 7bit _______________________________________________ Ips mailing list Ips@ietf.org https://www1.ietf.org/mailman/listinfo/ips --===============1709706597==-- From ips-bounces@ietf.org Tue Oct 5 13:23:09 2004 Received: from ietf-mx.ietf.org (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id NAA17684 for ; Tue, 5 Oct 2004 13:23:09 -0400 (EDT) Received: from megatron.ietf.org ([132.151.6.71]) by ietf-mx.ietf.org with esmtp (Exim 4.33) id 1CEtBC-00047a-DF for ips-web-archive@ietf.org; Tue, 05 Oct 2004 13:32:51 -0400 Received: from localhost.localdomain ([127.0.0.1] helo=megatron.ietf.org) by megatron.ietf.org with esmtp (Exim 4.32) id 1CEsue-0002It-SO; Tue, 05 Oct 2004 13:15:44 -0400 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by megatron.ietf.org with esmtp (Exim 4.32) id 1CEsn8-0000ZT-6o for ips@megatron.ietf.org; Tue, 05 Oct 2004 13:07:58 -0400 Received: from ietf-mx.ietf.org (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id NAA15957 for ; Tue, 5 Oct 2004 13:07:55 -0400 (EDT) Received: from system60.simplicato.com ([207.99.47.60] helo=host52a.simplicato.com) by ietf-mx.ietf.org with esmtp (Exim 4.33) id 1CEswS-0001g0-7n for ips@ietf.org; Tue, 05 Oct 2004 13:17:36 -0400 Received: from localhost (localhost.simplicato.com [127.0.0.1]) by host52a.simplicato.com (Postfix) with ESMTP id 72BD011E138; Tue, 5 Oct 2004 13:07:53 -0400 (EDT) Received: from host52a.simplicato.com ([127.0.0.1]) by localhost (host52a.simplicato.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 67413-06; Tue, 5 Oct 2004 13:07:52 -0400 (EDT) Received: from [192.168.1.2] (bzq-198-76.red.bezeqint.net [212.179.198.76]) by host52a.simplicato.com (Postfix) with ESMTP id 9C03211E136; Tue, 5 Oct 2004 13:07:50 -0400 (EDT) Mime-Version: 1.0 (Apple Message framework v619) In-Reply-To: References: Message-Id: <16CC602C-16F1-11D9-A237-000393814604@asomi.com> From: Caitlin Bestler Subject: Re: [Ips] DA+iSER Flow Control, Limiting number of oustanding cmdsattarget Date: Tue, 5 Oct 2004 19:07:50 +0200 To: ips@ietf.org, John Hufferd X-Mailer: Apple Mail (2.619) X-Virus-Scanned: by amavisd-new at simplicato.com X-Spam-Score: 1.1 (+) X-Scan-Signature: 88b11fc64c1bfdb4425294ef5374ca07 X-BeenThere: ips@ietf.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: IP Storage List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Content-Type: multipart/mixed; boundary="===============0889257719==" Sender: ips-bounces@ietf.org Errors-To: ips-bounces@ietf.org X-Spam-Score: 0.1 (/) X-Scan-Signature: f884eb1d4ec5a230688d7edc526ea665 --===============0889257719== Content-Type: multipart/alternative; boundary=Apple-Mail-2-577270372 --Apple-Mail-2-577270372 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: quoted-printable On Oct 5, 2004, at 6:38 PM, John Hufferd wrote: > > Caitlin, > I have no idea why you say that we need something special in the RNIC=20= > that is used for iSCSI/iSER. > That is NOT the case, and I did not intend to suggest anything like=20 > that. =A0The only issue you keep > bringing up is with buffering, which is fully handled by the Storage=20= > Controllers and Shared RQs. > =A0And the point of all the implementations is how fast we can service=20= > the data coming on the link. > I do not think we need to say anything like the Goal is to operate at=20= > link speed! =A0The purpose of > SRQs is to permit application such as iSER to be able to respond at=20 > link speed, while keeping > the use of pre posted buffers to a minimum. The de facto standard for the local interface to an RNIC requires=20 pre-posting of receive buffers for untagged messages, either specifically to the=20 endpoint (the receive queue) or to a shared receive queue supporting a group of=20= endpoints. The supply of available buffers is finite. Once a buffer is claimed,=20 it will be delivered to the local ULP (iSER in this case). The buffer can only be repopulated to the buffer pool after it is processed. If there are no session limits on an individual session understood at the RDMA layer, then one remote connection can drain the pool at wire speed. The iSER layer cannot return buffers to the buffer pool than it can accept them. If the only break on a runaway connection is the iSER layer (or even beyond that the iSCSI layer) then they must be able to accept iSCSI PDUs at line rate or a remote connection will be able to drain the Shared Receive Queue. SRQs have per-endpoint limits designed to short circuit such attacks. But you have to have a value to set, and it is a limit on the number of outstanding untagged messages. The RDMA layer has no way to distinguish between valuable messages that the iSCSI must process (or terminate the session) and those that the iSCSI layer is allowed to throw away. This is basic math. If a single connection can drain the SRQ at line rate (and that's the maximum number of *messages* possible at line rate) then the iSCSI layer must accept the iSCSI PDUs at line rate -- OR there has to be a method of stating that a given connection has send too many untagged messages. That's a credit limit. If you don't allow there to be a credit limit then the only credit limit is the *message* rate of the wire. Either that or you leave all sessions vulnerable to a bug in one client (or even one malicious attack). I do not believe requiring iSCSI to accept iSCSI PDUs at wire rate is feasible (it is also impossible on a PCI-X bus), and not addressing the security issue is unacceptable. The objection that has been repeatedly raised was that there was no basis for setting that credit limit based upon specifications. --Apple-Mail-2-577270372 Content-Type: text/enriched; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable On Oct 5, 2004, at 6:38 PM, John Hufferd wrote: Caitlin,=20 I have no idea why you say that we need something special in the RNIC that is used for iSCSI/iSER. That is NOT the case, and I did not intend to suggest anything like that. =A0The only issue you keep bringing up is with buffering, which is fully handled by the Storage Controllers and Shared RQs. =A0And the point of all the implementations is how fast we can service the data coming on the link. I do not think we need to say anything like the Goal is to operate at link speed! =A0The purpose of SRQs is to permit application such as iSER to be able to respond at link speed, while keeping the use of pre posted buffers to a minimum.=20 The de facto standard for the local interface to an RNIC requires pre-posting of receive buffers for untagged messages, either specifically to the endpoint (the receive queue) or to a shared receive queue supporting a group of endpoints. The supply of available buffers is finite. Once a buffer is claimed, it will be delivered to the local ULP (iSER in this case). The buffer can only be repopulated to the buffer pool after it is processed. If there are no session limits on an individual session understood at the RDMA layer, then one remote connection can drain the pool at wire speed. The iSER layer cannot return buffers to the buffer pool than it can accept them. If the only break on a runaway connection is the iSER layer (or even beyond that the iSCSI layer) then they must be able to accept iSCSI PDUs at line rate or a remote connection will be able to drain the Shared Receive Queue. SRQs have per-endpoint limits designed to short circuit such attacks. But you have to have a value to set, and it is a limit on the number of outstanding untagged messages. The RDMA layer has no way to distinguish between valuable messages that the iSCSI must process (or terminate the session) and those that the iSCSI layer is allowed to throw away. This is basic math. If a single connection can drain the SRQ at line rate (and that's the maximum number of *messages* possible at line rate) then the iSCSI layer must accept the iSCSI PDUs at line rate -- OR there has to be a method of stating that a given connection has send too many untagged messages. =20 That's a credit limit. If you don't allow there to be a credit limit then the only credit limit is the *message* rate of the wire. Either that or you leave all sessions vulnerable to a bug in one client (or even one malicious attack). I do not believe requiring iSCSI to accept iSCSI PDUs at wire rate is feasible (it is also impossible on a PCI-X bus), and not addressing the security issue is unacceptable. The objection that has been repeatedly raised was that there was no basis for setting that credit limit based upon specifications. --Apple-Mail-2-577270372-- --===============0889257719== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline Content-Transfer-Encoding: 7bit _______________________________________________ Ips mailing list Ips@ietf.org https://www1.ietf.org/mailman/listinfo/ips --===============0889257719==-- From ips-bounces@ietf.org Tue Oct 5 14:53:46 2004 Received: from ietf-mx.ietf.org (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id OAA25146 for ; Tue, 5 Oct 2004 14:53:46 -0400 (EDT) Received: from megatron.ietf.org ([132.151.6.71]) by ietf-mx.ietf.org with esmtp (Exim 4.33) id 1CEuat-0001bu-8G for ips-web-archive@ietf.org; Tue, 05 Oct 2004 15:03:27 -0400 Received: from localhost.localdomain ([127.0.0.1] helo=megatron.ietf.org) by megatron.ietf.org with esmtp (Exim 4.32) id 1CEuKQ-0001BH-DG; Tue, 05 Oct 2004 14:46:26 -0400 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by megatron.ietf.org with esmtp (Exim 4.32) id 1CEu9d-0006xQ-M2 for ips@megatron.ietf.org; Tue, 05 Oct 2004 14:35:17 -0400 Received: from ietf-mx.ietf.org (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id OAA23395 for ; Tue, 5 Oct 2004 14:35:16 -0400 (EDT) Received: from e32.co.us.ibm.com ([32.97.110.130]) by ietf-mx.ietf.org with esmtp (Exim 4.33) id 1CEuIu-0006nO-Vz for ips@ietf.org; Tue, 05 Oct 2004 14:44:56 -0400 Received: from westrelay04.boulder.ibm.com (westrelay04.boulder.ibm.com [9.17.193.32]) by e32.co.us.ibm.com (8.12.10/8.12.9) with ESMTP id i95IYOqY636296; Tue, 5 Oct 2004 14:34:29 -0400 Received: from d03nm115.boulder.ibm.com (d03av04.boulder.ibm.com [9.17.195.170]) by westrelay04.boulder.ibm.com (8.12.10/NCO/VER6.6) with ESMTP id i95IYNds149242; Tue, 5 Oct 2004 12:34:23 -0600 In-Reply-To: <16CC602C-16F1-11D9-A237-000393814604@asomi.com> To: ips@ietf.org, Caitlin Bestler MIME-Version: 1.0 Subject: Re: [Ips] DA+iSER Flow Control, Limiting number of oustanding cmdsattarget X-Mailer: Lotus Notes Release 6.0.2CF1 June 9, 2003 From: John Hufferd Message-ID: Date: Tue, 5 Oct 2004 11:34:19 -0700 X-MIMETrack: Serialize by Router on D03NM115/03/M/IBM(Release 6.51HF338 | June 21, 2004) at 10/05/2004 12:34:23, Serialize complete at 10/05/2004 12:34:23 X-Spam-Score: 1.0 (+) X-Scan-Signature: 453b1bfcf0292bffe4cab90ba115f503 X-BeenThere: ips@ietf.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: IP Storage List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Content-Type: multipart/mixed; boundary="===============1515724950==" Sender: ips-bounces@ietf.org Errors-To: ips-bounces@ietf.org X-Spam-Score: 0.0 (/) X-Scan-Signature: 24d000849df6f171c5ec1cca2ea21b82 This is a multipart message in MIME format. --===============1515724950== Content-Type: multipart/alternative; boundary="=_alternative 0065F22788256F24_=" This is a multipart message in MIME format. --=_alternative 0065F22788256F24_= Content-Type: text/plain; charset="US-ASCII" Caitlin, I am having difficulty following your issues. But I think it now has something to do with the fact that the code in the iSCSI part of the iSCSI/iSER process will not be able to determine that its partner is over subscribing its links. Since we are talking about a single Kernel application that is only divided into logical modules, which we decide to call iSER and the non iSER part of iSCSI. All the flow control of iSCSI is still in effect, and will react to a given buffer within nano seconds of its data being received via the SRQ buffers. The iSCSI code will then determine if the peer is doing something wrong (like exceed its iSCSI set limits). If the iSCSI set limits are exceeded, then the iSCSI part of the code will terminate the connection. If the iSCSI set limits have not been exceeded, that peer can burst on the link as much as it can, that is not a problem, and I do not like the idea of adding yet another set of flow controls above what iSCSI has. With a well run buffer chain management this is not a problem. iSCSI will see the problem almost immediately and make the needed corrections. Yes there is some design and implementation art to making sure that the SRQ buffer chain management, the general Cache buffer chain management, iSER and iSCSI work well together, but that is where the Intellectual Property of each vendor comes in. And each vendor will claim, as they always do, that they handle it better than every other vendor. But there is no real problem of having the iSCSI part of the code be the point where over subscription of the link is determined and handled. . . John L. Hufferd Senior Technical Staff Member (STSM) IBM/System Group, San Jose CA Main Office: (408) 256-0403, Tie: 276-0403, eFax: (408) 904-4688 Alt Office: (408) 997-6136, Cell: (408) 499-9702 Internet Address: hufferd@us.ibm.com Caitlin Bestler 10/05/2004 10:07 AM To ips@ietf.org, John Hufferd/San Jose/IBM@IBMUS cc Subject Re: [Ips] DA+iSER Flow Control, Limiting number of oustanding cmdsattarget On Oct 5, 2004, at 6:38 PM, John Hufferd wrote: > > Caitlin, > I have no idea why you say that we need something special in the RNIC > that is used for iSCSI/iSER. > That is NOT the case, and I did not intend to suggest anything like > that. The only issue you keep > bringing up is with buffering, which is fully handled by the Storage > Controllers and Shared RQs. > And the point of all the implementations is how fast we can service > the data coming on the link. > I do not think we need to say anything like the Goal is to operate at > link speed! The purpose of > SRQs is to permit application such as iSER to be able to respond at > link speed, while keeping > the use of pre posted buffers to a minimum. The de facto standard for the local interface to an RNIC requires pre-posting of receive buffers for untagged messages, either specifically to the endpoint (the receive queue) or to a shared receive queue supporting a group of endpoints. The supply of available buffers is finite. Once a buffer is claimed, it will be delivered to the local ULP (iSER in this case). The buffer can only be repopulated to the buffer pool after it is processed. If there are no session limits on an individual session understood at the RDMA layer, then one remote connection can drain the pool at wire speed. The iSER layer cannot return buffers to the buffer pool than it can accept them. If the only break on a runaway connection is the iSER layer (or even beyond that the iSCSI layer) then they must be able to accept iSCSI PDUs at line rate or a remote connection will be able to drain the Shared Receive Queue. SRQs have per-endpoint limits designed to short circuit such attacks. But you have to have a value to set, and it is a limit on the number of outstanding untagged messages. The RDMA layer has no way to distinguish between valuable messages that the iSCSI must process (or terminate the session) and those that the iSCSI layer is allowed to throw away. This is basic math. If a single connection can drain the SRQ at line rate (and that's the maximum number of *messages* possible at line rate) then the iSCSI layer must accept the iSCSI PDUs at line rate -- OR there has to be a method of stating that a given connection has send too many untagged messages. That's a credit limit. If you don't allow there to be a credit limit then the only credit limit is the *message* rate of the wire. Either that or you leave all sessions vulnerable to a bug in one client (or even one malicious attack). I do not believe requiring iSCSI to accept iSCSI PDUs at wire rate is feasible (it is also impossible on a PCI-X bus), and not addressing the security issue is unacceptable. The objection that has been repeatedly raised was that there was no basis for setting that credit limit based upon specifications. --=_alternative 0065F22788256F24_= Content-Type: text/html; charset="US-ASCII"
Caitlin,
I am having difficulty following your issues.  But I think it now has something to do with the fact that the code in the iSCSI part of the iSCSI/iSER process will not be able to determine that its partner is over subscribing its links.  

Since we are talking about a single Kernel application that is only divided into logical modules, which we decide to call iSER and the non iSER part of iSCSI.  All the flow control of iSCSI is still in effect, and will react to a given buffer within nano seconds of its data being received via the SRQ buffers.  The iSCSI code will then determine if the peer is doing something wrong (like exceed its iSCSI set limits).  If the iSCSI set limits are exceeded, then the iSCSI part of the code will terminate the connection.  If the iSCSI set limits have not been exceeded, that peer can burst on the link as much as it can, that is not a problem, and I do not like the idea of adding yet another set of flow controls above what iSCSI has.  

With a well run buffer chain management this is not a problem.  iSCSI will see the problem almost immediately and make the needed corrections.  Yes there is some design and implementation art to making sure that the SRQ buffer chain management, the general Cache buffer chain management, iSER and iSCSI work well together, but that is where the Intellectual Property of each vendor comes in.  And each vendor will claim, as they always do, that they handle it better than every other vendor.  But there is no real problem of having the iSCSI part of the code be the point where over subscription of the link is determined and handled.
.
.
John L. Hufferd
Senior Technical Staff Member (STSM)
IBM/System Group, San Jose CA
Main Office: (408) 256-0403, Tie: 276-0403, eFax: (408) 904-4688
Alt Office: (408) 997-6136, Cell: (408) 499-9702
Internet Address: hufferd@us.ibm.com


Caitlin Bestler <cait@asomi.com>

10/05/2004 10:07 AM

To
ips@ietf.org, John Hufferd/San Jose/IBM@IBMUS
cc
Subject
Re: [Ips] DA+iSER Flow Control,          Limiting number of oustanding cmdsattarget






On Oct 5, 2004, at 6:38 PM, John Hufferd wrote:

>
> Caitlin,
> I have no idea why you say that we need something special in the RNIC
> that is used for iSCSI/iSER.
> That is NOT the case, and I did not intend to suggest anything like
> that.  The only issue you keep
> bringing up is with buffering, which is fully handled by the Storage
> Controllers and Shared RQs.
>  And the point of all the implementations is how fast we can service
> the data coming on the link.
> I do not think we need to say anything like the Goal is to operate at
> link speed!  The purpose of
> SRQs is to permit application such as iSER to be able to respond at
> link speed, while keeping
> the use of pre posted buffers to a minimum.

The de facto standard for the local interface to an RNIC requires
pre-posting
of receive buffers for untagged messages, either specifically to the
endpoint
(the receive queue) or to a shared receive queue supporting a group of
endpoints.

The supply of available buffers is finite.  Once a buffer is claimed,
it will be
delivered to the local ULP (iSER in this case).  The buffer can only be
repopulated to the buffer pool after it is processed.

If there are no session limits on an individual session understood at
the RDMA layer, then one remote connection can drain the pool at
wire speed.

The iSER layer cannot return buffers to the buffer pool than it can
accept them.   If the only break on a runaway connection is the
iSER layer (or even beyond that the iSCSI layer) then they must
be able to accept iSCSI PDUs at line rate or a remote connection
will be able to drain the Shared Receive Queue.

SRQs have per-endpoint limits designed to short circuit such
attacks.  But you have to have a value to set, and it is a limit
on the number of outstanding untagged messages.  The RDMA
layer has no way to distinguish between valuable messages
that the iSCSI must process (or terminate the session) and
those that the iSCSI layer is allowed to throw away.

This is basic math.  If a single connection can drain the SRQ
at line rate (and that's the maximum number of *messages*
possible at line rate) then the iSCSI layer must accept the
iSCSI PDUs at line rate -- OR there has to be a method of
stating that a given connection has send too many untagged
messages.

That's a credit limit.  If you don't allow there to be a credit
limit then the only credit limit is the *message* rate of the wire.
Either that or you leave all sessions vulnerable to a bug
in one client (or even one malicious attack).  I do not believe
requiring iSCSI to accept iSCSI PDUs at wire rate is feasible
(it is also impossible on a PCI-X bus), and not addressing
the security issue is unacceptable.

The objection that has been repeatedly raised was that there
was no basis for setting that credit limit based upon specifications.
--=_alternative 0065F22788256F24_=-- --===============1515724950== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline Content-Transfer-Encoding: 7bit _______________________________________________ Ips mailing list Ips@ietf.org https://www1.ietf.org/mailman/listinfo/ips --===============1515724950==-- From ips-bounces@ietf.org Tue Oct 5 16:50:29 2004 Received: from ietf-mx.ietf.org (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id QAA10361 for ; Tue, 5 Oct 2004 16:50:29 -0400 (EDT) Received: from megatron.ietf.org ([132.151.6.71]) by ietf-mx.ietf.org with esmtp (Exim 4.33) id 1CEwPr-0002c4-Op for ips-web-archive@ietf.org; Tue, 05 Oct 2004 17:00:12 -0400 Received: from localhost.localdomain ([127.0.0.1] helo=megatron.ietf.org) by megatron.ietf.org with esmtp (Exim 4.32) id 1CEw7s-0003yb-1d; Tue, 05 Oct 2004 16:41:36 -0400 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by megatron.ietf.org with esmtp (Exim 4.32) id 1CEw0A-0001Ko-Cb for ips@megatron.ietf.org; Tue, 05 Oct 2004 16:33:38 -0400 Received: from ietf-mx.ietf.org (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id QAA08065 for ; Tue, 5 Oct 2004 16:33:31 -0400 (EDT) Received: from system60.simplicato.com ([207.99.47.60] helo=host52a.simplicato.com) by ietf-mx.ietf.org with esmtp (Exim 4.33) id 1CEw9R-0000wB-Hz for ips@ietf.org; Tue, 05 Oct 2004 16:43:13 -0400 Received: from localhost (localhost.simplicato.com [127.0.0.1]) by host52a.simplicato.com (Postfix) with ESMTP id A636211E0A8; Tue, 5 Oct 2004 16:33:27 -0400 (EDT) Received: from host52a.simplicato.com ([127.0.0.1]) by localhost (host52a.simplicato.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 81606-05; Tue, 5 Oct 2004 16:33:26 -0400 (EDT) Received: from [192.168.1.2] (bzq-198-76.red.bezeqint.net [212.179.198.76]) by host52a.simplicato.com (Postfix) with ESMTP id CF96911E0F8; Tue, 5 Oct 2004 16:33:24 -0400 (EDT) In-Reply-To: References: Mime-Version: 1.0 (Apple Message framework v619) Message-Id: From: Caitlin Bestler Subject: Re: [Ips] DA+iSER Flow Control, Limiting number of oustanding cmdsattarget Date: Tue, 5 Oct 2004 22:33:26 +0200 To: John Hufferd X-Mailer: Apple Mail (2.619) X-Virus-Scanned: by amavisd-new at simplicato.com X-Spam-Score: 0.1 (/) X-Scan-Signature: b1c41982e167b872076d0018e4e1dc3c Cc: ips@ietf.org X-BeenThere: ips@ietf.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: IP Storage List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Content-Type: multipart/mixed; boundary="===============1059834232==" Sender: ips-bounces@ietf.org Errors-To: ips-bounces@ietf.org X-Spam-Score: 0.1 (/) X-Scan-Signature: 21bf7a2f1643ae0bf20c1e010766eb78 --===============1059834232== Content-Type: multipart/alternative; boundary=Apple-Mail-2-589605908 --Apple-Mail-2-589605908 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: quoted-printable On Oct 5, 2004, at 8:34 PM, John Hufferd wrote: > > Caitlin, > I am having difficulty following your issues. =A0But I think it now = has=20 > something to do with the fact that > the code in the iSCSI part of the iSCSI/iSER process will not be able=20= > to determine that its partner > is over subscribing its links. =A0 > I am not thinking of the problem as a single software entity. I am focusing on being a provider of a Datamover Architecture service. I have no guarantee that the Consumer of my service is aware that they must reap completions fast enough to prevent the buffer pool from being drained. Further the Datamover specification could easily be read so that they would think it is *my* responsibility to apply back-pressure on the peer to prevent the packets from being sent. The *only* form of back pressure available over RDMA is dropping the connection. I do not want to drop connections without justification, such as violation of a credit limit. Dropping a connection when there has not been a violation or an error is *not* providing a reliable transport service. This is a particular concern with RDMA because it is possible to implement RDMA in a way where the TCP bottlenecks are not totally eliminated. So a competitor's product might conceal this problem with the "solution" of failing to eliminate the TCP bottleneck completely enough. Without a clear statement of responsibility for flow control being at the iSCSI layer my customer may conclude that the problem is *my* implementation. --Apple-Mail-2-589605908 Content-Type: text/enriched; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable On Oct 5, 2004, at 8:34 PM, John Hufferd wrote: Caitlin,=20 I am having difficulty following your issues. =A0But I think it now has something to do with the fact that the code in the iSCSI part of the iSCSI/iSER process will not be able to determine that its partner is over subscribing its links. =A0=20 I am not thinking of the problem as a single software entity. I am focusing on being a provider of a Datamover Architecture service. I have no guarantee that the Consumer of my service is aware that they must reap completions fast enough to prevent the buffer pool from being drained. Further the Datamover specification could easily be read so that they would think it is *my* responsibility to apply back-pressure on the peer to prevent the packets from being sent. The *only* form of back pressure available over RDMA is dropping the connection. I do not want to drop connections without justification, such as violation of a credit limit. Dropping a connection when there has not been a violation or an error is *not* providing a reliable transport service. This is a particular concern with RDMA because it is possible to implement RDMA in a way where the TCP bottlenecks are not totally eliminated. So a competitor's product might conceal this problem with the "solution" of failing to eliminate the TCP bottleneck completely enough. Without a clear statement of responsibility for flow control being at the iSCSI layer my customer may conclude that the problem is *my* implementation. --Apple-Mail-2-589605908-- --===============1059834232== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline Content-Transfer-Encoding: 7bit _______________________________________________ Ips mailing list Ips@ietf.org https://www1.ietf.org/mailman/listinfo/ips --===============1059834232==-- From ips-bounces@ietf.org Tue Oct 5 18:00:49 2004 Received: from ietf-mx.ietf.org (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id SAA15465 for ; Tue, 5 Oct 2004 18:00:49 -0400 (EDT) Received: from megatron.ietf.org ([132.151.6.71]) by ietf-mx.ietf.org with esmtp (Exim 4.33) id 1CExVw-0003fq-JD for ips-web-archive@ietf.org; Tue, 05 Oct 2004 18:10:32 -0400 Received: from localhost.localdomain ([127.0.0.1] helo=megatron.ietf.org) by megatron.ietf.org with esmtp (Exim 4.32) id 1CExHb-0007tk-Em; Tue, 05 Oct 2004 17:55:43 -0400 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by megatron.ietf.org with esmtp (Exim 4.32) id 1CExEw-0007Yx-PD for ips@megatron.ietf.org; Tue, 05 Oct 2004 17:52:58 -0400 Received: from ietf-mx.ietf.org (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id RAA15107 for ; Tue, 5 Oct 2004 17:52:56 -0400 (EDT) Received: from sccrmhc11.comcast.net ([204.127.202.55]) by ietf-mx.ietf.org with esmtp (Exim 4.33) id 1CExOH-00034S-Va for ips@ietf.org; Tue, 05 Oct 2004 18:02:40 -0400 Received: from ivvt2dxrc11 (209-30-141-131.ded.swbell.net[209.30.141.131]) by comcast.net (sccrmhc11) with SMTP id <2004100521522201100j8a3ee> (Authid: esquicksall); Tue, 5 Oct 2004 21:52:23 +0000 Message-ID: <00f501c4ab25$9e433200$6e01000a@ivvt2dxrc11> From: "Eddy Quicksall" To: References: <000801c4a897$a4c5efe0$0303a8c0@ivvt2dxrc11> Subject: Re: [Ips] setting StatSN in a data pdu Date: Tue, 5 Oct 2004 17:52:30 -0400 MIME-Version: 1.0 X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 6.00.2800.1437 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2800.1441 X-Spam-Score: 0.2 (/) X-Scan-Signature: 093efd19b5f651b2707595638f6c4003 X-BeenThere: ips@ietf.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: IP Storage List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Content-Type: multipart/mixed; boundary="===============0639675917==" Sender: ips-bounces@ietf.org Errors-To: ips-bounces@ietf.org X-Spam-Score: 0.3 (/) X-Scan-Signature: 200d029292fbb60d25b263122ced50fc This is a multi-part message in MIME format. --===============0639675917== Content-Type: multipart/alternative; boundary="----=_NextPart_000_00F0_01C4AB04.15C509B0" This is a multi-part message in MIME format. ------=_NextPart_000_00F0_01C4AB04.15C509B0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable Julian or Mallikarjun, I didn't see a response to this. Eddy ----- Original Message -----=20 From: Eddy Quicksall=20 To: ips@ietf.org=20 Sent: Saturday, October 02, 2004 11:51 AM Subject: [Ips] setting StatSN in a data pdu Section 3.5.1.5 says "In addition, StatSN is carried by the Data-In = PDUs." But 10.7.3 says "The fields StatSN, Status, and Residual Count only = have meaningful content if the S bit is set to 1" =20 I believe section 3.5.1.5 should say "In addition, when the S bit is = set StatSN is carried by the Data-In PDUs." =20 Eddy -------------------------------------------------------------------------= ----- _______________________________________________ Ips mailing list Ips@ietf.org https://www1.ietf.org/mailman/listinfo/ips ------=_NextPart_000_00F0_01C4AB04.15C509B0 Content-Type: text/html; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable
Julian or Mallikarjun,
 
I didn't see a response to this.
 
Eddy
----- Original Message -----
From:=20 Eddy = Quicksall=20
Sent: Saturday, October 02, = 2004 11:51=20 AM
Subject: [Ips] setting StatSN = in a data=20 pdu

Section 3.5.1.5 says "In = addition, StatSN=20 is carried by the Data-In PDUs."
 
But 10.7.3 says = "The fields=20 StatSN, Status, and Residual Count only have meaningful content=20 if the S bit is set to 1"
 
I believe section 3.5.1.5 should say "In = addition, when the=20 S bit is set StatSN is carried by the Data-In = PDUs."
 
Eddy

_______________________________________________
Ips mailing=20 = list
Ips@ietf.org
https://www1.ietf.org/mailman/listinfo/ips
------=_NextPart_000_00F0_01C4AB04.15C509B0-- --===============0639675917== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline Content-Transfer-Encoding: 7bit _______________________________________________ Ips mailing list Ips@ietf.org https://www1.ietf.org/mailman/listinfo/ips --===============0639675917==-- From ips-bounces@ietf.org Tue Oct 5 18:03:48 2004 Received: from ietf-mx.ietf.org (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id SAA15705 for ; Tue, 5 Oct 2004 18:03:48 -0400 (EDT) Received: from megatron.ietf.org ([132.151.6.71]) by ietf-mx.ietf.org with esmtp (Exim 4.33) id 1CExYp-0003lm-G0 for ips-web-archive@ietf.org; Tue, 05 Oct 2004 18:13:31 -0400 Received: from localhost.localdomain ([127.0.0.1] helo=megatron.ietf.org) by megatron.ietf.org with esmtp (Exim 4.32) id 1CExNe-0001ab-HF; Tue, 05 Oct 2004 18:01:58 -0400 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by megatron.ietf.org with esmtp (Exim 4.32) id 1CExKA-0008Hu-IU for ips@megatron.ietf.org; Tue, 05 Oct 2004 17:58:23 -0400 Received: from ietf-mx.ietf.org (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id RAA15315 for ; Tue, 5 Oct 2004 17:58:20 -0400 (EDT) Received: from rospat.qlogic.com ([198.70.192.130] helo=homer.qlogic.com) by ietf-mx.ietf.org with esmtp (Exim 4.33) id 1CExTX-0003Tg-By for ips@ietf.org; Tue, 05 Oct 2004 18:08:04 -0400 X-MimeOLE: Produced By Microsoft Exchange V6.0.6556.0 content-class: urn:content-classes:message MIME-Version: 1.0 Subject: RE: [Ips] resending Data-in with S bit set Date: Tue, 5 Oct 2004 14:54:31 -0700 Message-ID: <2A8D600C919D2D41A3DAA2D2FEC71714AF8F92@homer.qlogic.org> Thread-Topic: [Ips] resending Data-in with S bit set Thread-Index: AcSqIO16lXbx+mAaR4itSsH07JniRwBARykw From: "Dean Scoville" To: "Julian Satran" , "Eddy Quicksall" X-Spam-Score: 0.3 (/) X-Scan-Signature: d11a451997816a91a305dcb5ab1b85dd Cc: ips@ietf.org X-BeenThere: ips@ietf.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: IP Storage List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Content-Type: multipart/mixed; boundary="===============1618506709==" Sender: ips-bounces@ietf.org Errors-To: ips-bounces@ietf.org X-Spam-Score: 0.6 (/) X-Scan-Signature: 10dcc25e55b9b5f7d6ded516404bdc4c This is a multi-part message in MIME format. --===============1618506709== content-class: urn:content-classes:message Content-Type: multipart/alternative; boundary="----_=_NextPart_001_01C4AB25.E4BAEFD4" This is a multi-part message in MIME format. ------_=_NextPart_001_01C4AB25.E4BAEFD4 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable Julian, =20 Section 10.16 states the following about PDUs transmitted as a result of = a SNACK: =20 "The numbered-response(s) or R2T(s), requested by a SNACK, MUST be delivered as exact replicas of the ones that the target transmitted = originally except for the fields ExpCmdSN, MaxCmdSN, and ExpDataSN, which MUST carry the current values. R2T(s) requested by SNACK MUST also carry = the current values and except for resegmentation (see Section 10.16.3 = Resegmentation)." =20 Section 10.16.1 describes Status and Data/R2T SNACK types as: =20 0-Data/R2T SNACK - requesting retransmission of one or more Data-In = or R2T PDUs. 1-Status SNACK - requesting retransmission of one or more numbered = responses. =20 It seems to me that if the target receives a Status SNACK for status = sent on a Data-In=20 PDU due to phase collapse, the target should send an exact replica of = the Data-In PDU. Similarly, if the target receives a Data SNACK, it should retransmit the = Data-In PDU with the S bit set, since it is a retransmission. =20 I may have misunderstood your recent posting, but I thought it said the = target should=20 generate a totally new SCSI Response PDU in response to the Status SNACK = rather than retransmitting the original Data-In PDU. And that it should send a = Data-In PDU without the S-bit set in the case of a Data SNACK. Both of these = behaviors seem to=20 violate what is stated in Sections 10.16 and 10.16.1 of the iSCSI RFC, = as quoted above. =20 I would appreciate clarification on the target's expected response in = these situations. =20 thanks, =20 Dean Scoville -----Original Message----- From: ips-bounces@ietf.org [mailto:ips-bounces@ietf.org]On Behalf Of = Julian Satran Sent: Monday, October 04, 2004 7:43 AM To: Eddy Quicksall Cc: ips@ietf.org Subject: Re: [Ips] resending Data-in with S bit set Eddy,=20 This is correct.=20 The data-in PDUs don't have to be sent.=20 The initiator has to request the data-in PDUs explicitly and MUST do so = before it acks the status (that implicitly acks all the data and deletes = any record of the command at target).=20 Julo=20 ips-bounces@ietf.org wrote on 04/10/2004 15:20:56: > Can you confirm that the intent of 10.16.6. is as follows?=20 > =20 > If the Initiator has requested for a Status SNACK with BegRun=3D0 and=20 > RunLength=3D0 (i.e, all unacknowledged status pdus) and the target has > some Data-In PDUs carrying *status* that have not been acknowledged, > the target will *not* resend those Data-In PDUs carrying the status.=20 > =20 > The unacknowledged queue looks like=20 > =20 > response(statSN=3D5) ---> response(statSN=3D6) -----> Data-In w/S=20 > bit(statSN=3D7) ----> response(statSN=3D8)=20 > =20 > The target will resend pdus with statSN 5, 6 and 8. The Data-In with > statSN 7 will not be resent even though it sends status.=20 > =20 > Eddy=20 > _______________________________________________ > Ips mailing list > Ips@ietf.org > https://www1.ietf.org/mailman/listinfo/ips ------_=_NextPart_001_01C4AB25.E4BAEFD4 Content-Type: text/html; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable
Julian,
 
Section 10.16 states the following about PDUs = transmitted as a=20 result of a SNACK:
 
   "The numbered-response(s) or R2T(s), requested by = a SNACK,=20 MUST be
    delivered as exact = replicas of the=20 ones that the target transmitted = originally
    except for the fields = ExpCmdSN,=20 MaxCmdSN, and ExpDataSN, which MUST
    carry the current values. = R2T(s)=20 requested by SNACK MUST  also carry = the
    current values and except = for=20 resegmentation (see Section 10.16.3=20 Resegmentation)."
 
Section=20 10.16.1 describes Status and Data/R2T SNACK types=20 as:
 
    0-Data/R2T SNACK - = requesting=20 retransmission of one or more Data-In or R2T=20 PDUs.
    1-Status SNACK - = requesting=20 retransmission of one or more numbered=20 responses.
 = ;
It seems to me that if the target receives a = Status=20 SNACK for status sent on a Data-In
PDU due to=20 phase collapse, the = target should=20 send an exact replica of the Data-In = PDU.
Similarly, if the target=20 receives a Data SNACK, = it should=20 retransmit the Data-In PDU with
the S bit set, = since=20 it is a=20 retransmission.
 
I may have misunderstood your recent posting, = but I=20 thought it said the=20 target should=20
generate a totally new SCSI Response PDU in = response=20 to the=20 Status SNACK=20 rather
than retransmitting the original Data-In = PDU. And=20 that it should send a Data-In PDU
without the S-bit set in the case of a Data = SNACK. Both=20 of these behaviors seem to =
violate what is stated=20 in=20 Sections 10.16 and = 10.16.1 of the=20 iSCSI RFC, as quoted above.
 
I=20 would appreciate clarification on the target's expected response in = these=20 situations.
 
thanks,
 
Dean=20 Scoville
-----Original Message-----
From: = ips-bounces@ietf.org=20 [mailto:ips-bounces@ietf.org]On Behalf Of Julian = Satran
Sent:=20 Monday, October 04, 2004 7:43 AM
To: Eddy = Quicksall
Cc:=20 ips@ietf.org
Subject: Re: [Ips] resending Data-in with S bit = set


Eddy,=20

This is correct. =
The data-in PDUs don't have to be = sent.=20
The initiator has to request the = data-in PDUs=20 explicitly and MUST do so before it acks the status (that implicitly = acks all=20 the data and deletes any record of the command at target).=20

Julo

ips-bounces@ietf.org wrote on 04/10/2004 = 15:20:56:

> Can=20 you confirm that the intent of 10.16.6. is as follows? =
>  
> If = the Initiator=20 has requested for a Status SNACK with BegRun=3D0 and
> = RunLength=3D0 (i.e,=20 all unacknowledged status pdus) and the target has
> some = Data-In PDUs=20 carrying *status* that have not been acknowledged,
> the target = will=20 *not* resend those Data-In PDUs carrying the status. =
>  
> The=20 unacknowledged queue looks like
>=20  
>   =  response(statSN=3D5)=20 ---> response(statSN=3D6) -----> Data-In w/S
> = bit(statSN=3D7)=20 ----> response(statSN=3D8)
>=20  
> The target will resend = pdus with=20 statSN 5, 6 and 8. The Data-In with
> statSN 7 will not be = resent even=20 though it sends status.
>=20  
> Eddy =
> =  _______________________________________________
>=20 Ips mailing list
> Ips@ietf.org
>=20 = https://www1.ietf.org/mailman/listinfo/ips
------_=_NextPart_001_01C4AB25.E4BAEFD4-- --===============1618506709== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline Content-Transfer-Encoding: 7bit _______________________________________________ Ips mailing list Ips@ietf.org https://www1.ietf.org/mailman/listinfo/ips --===============1618506709==-- From ips-bounces@ietf.org Tue Oct 5 18:12:19 2004 Received: from ietf-mx.ietf.org (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id SAA16878 for ; Tue, 5 Oct 2004 18:12:19 -0400 (EDT) Received: from megatron.ietf.org ([132.151.6.71]) by ietf-mx.ietf.org with esmtp (Exim 4.33) id 1CExh5-00054R-2U for ips-web-archive@ietf.org; Tue, 05 Oct 2004 18:22:03 -0400 Received: from localhost.localdomain ([127.0.0.1] helo=megatron.ietf.org) by megatron.ietf.org with esmtp (Exim 4.32) id 1CExWR-00060C-B3; Tue, 05 Oct 2004 18:11:03 -0400 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by megatron.ietf.org with esmtp (Exim 4.32) id 1CExUM-0004af-SY for ips@megatron.ietf.org; Tue, 05 Oct 2004 18:08:55 -0400 Received: from ietf-mx.ietf.org (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id SAA16369 for ; Tue, 5 Oct 2004 18:08:51 -0400 (EDT) Received: from sccrmhc11.comcast.net ([204.127.202.55]) by ietf-mx.ietf.org with esmtp (Exim 4.33) id 1CExdj-0004F0-7s for ips@ietf.org; Tue, 05 Oct 2004 18:18:35 -0400 Received: from ivvt2dxrc11 (209-30-141-131.ded.swbell.net[209.30.141.131]) by comcast.net (sccrmhc11) with SMTP id <2004100522082201100j7p5te> (Authid: esquicksall); Tue, 5 Oct 2004 22:08:22 +0000 Message-ID: <010701c4ab27$db4f2d50$6e01000a@ivvt2dxrc11> From: "Eddy Quicksall" To: "Dean Scoville" , "Julian Satran" References: <2A8D600C919D2D41A3DAA2D2FEC71714AF8F92@homer.qlogic.org> Subject: Re: [Ips] resending Data-in with S bit set Date: Tue, 5 Oct 2004 18:08:33 -0400 MIME-Version: 1.0 X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 6.00.2800.1437 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2800.1441 X-Spam-Score: 0.4 (/) X-Scan-Signature: 58b614506802734014829a093beb6879 Cc: ips@ietf.org X-BeenThere: ips@ietf.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: IP Storage List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Content-Type: multipart/mixed; boundary="===============0249519802==" Sender: ips-bounces@ietf.org Errors-To: ips-bounces@ietf.org X-Spam-Score: 0.7 (/) X-Scan-Signature: 819069d28e3cfe534e22b502261ce83f This is a multi-part message in MIME format. --===============0249519802== Content-Type: multipart/alternative; boundary="----=_NextPart_000_0102_01C4AB06.53C61360" This is a multi-part message in MIME format. ------=_NextPart_000_0102_01C4AB06.53C61360 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable Maybe my question was misunderstood. To say it another way: Is it true that a Status SNACK only solicits Status PDUs and that it = does not solicit Data-In PDUs which have the S bit set? I interpret Julian's answer to this as "yes, that is true".=20 Eddy ----- Original Message -----=20 From: Dean Scoville=20 To: Julian Satran ; Eddy Quicksall=20 Cc: ips@ietf.org=20 Sent: Tuesday, October 05, 2004 5:54 PM Subject: RE: [Ips] resending Data-in with S bit set Julian, Section 10.16 states the following about PDUs transmitted as a result = of a SNACK: "The numbered-response(s) or R2T(s), requested by a SNACK, MUST be delivered as exact replicas of the ones that the target = transmitted originally except for the fields ExpCmdSN, MaxCmdSN, and ExpDataSN, which = MUST carry the current values. R2T(s) requested by SNACK MUST also = carry the current values and except for resegmentation (see Section 10.16.3 = Resegmentation)." Section 10.16.1 describes Status and Data/R2T SNACK types as: 0-Data/R2T SNACK - requesting retransmission of one or more = Data-In or R2T PDUs. 1-Status SNACK - requesting retransmission of one or more numbered = responses. It seems to me that if the target receives a Status SNACK for status = sent on a Data-In=20 PDU due to phase collapse, the target should send an exact replica of = the Data-In PDU. Similarly, if the target receives a Data SNACK, it should retransmit = the Data-In PDU with the S bit set, since it is a retransmission. I may have misunderstood your recent posting, but I thought it said = the target should=20 generate a totally new SCSI Response PDU in response to the Status = SNACK rather than retransmitting the original Data-In PDU. And that it should send = a Data-In PDU without the S-bit set in the case of a Data SNACK. Both of these = behaviors seem to=20 violate what is stated in Sections 10.16 and 10.16.1 of the iSCSI RFC, = as quoted above. I would appreciate clarification on the target's expected response in = these situations. thanks, Dean Scoville -----Original Message----- From: ips-bounces@ietf.org [mailto:ips-bounces@ietf.org]On Behalf Of = Julian Satran Sent: Monday, October 04, 2004 7:43 AM To: Eddy Quicksall Cc: ips@ietf.org Subject: Re: [Ips] resending Data-in with S bit set Eddy,=20 This is correct.=20 The data-in PDUs don't have to be sent.=20 The initiator has to request the data-in PDUs explicitly and MUST do = so before it acks the status (that implicitly acks all the data and = deletes any record of the command at target).=20 Julo=20 ips-bounces@ietf.org wrote on 04/10/2004 15:20:56: > Can you confirm that the intent of 10.16.6. is as follows?=20 > =20 > If the Initiator has requested for a Status SNACK with BegRun=3D0 = and=20 > RunLength=3D0 (i.e, all unacknowledged status pdus) and the target = has > some Data-In PDUs carrying *status* that have not been = acknowledged, > the target will *not* resend those Data-In PDUs carrying the = status.=20 > =20 > The unacknowledged queue looks like=20 > =20 > response(statSN=3D5) ---> response(statSN=3D6) -----> Data-In = w/S=20 > bit(statSN=3D7) ----> response(statSN=3D8)=20 > =20 > The target will resend pdus with statSN 5, 6 and 8. The Data-In = with > statSN 7 will not be resent even though it sends status.=20 > =20 > Eddy=20 > _______________________________________________ > Ips mailing list > Ips@ietf.org > https://www1.ietf.org/mailman/listinfo/ips ------=_NextPart_000_0102_01C4AB06.53C61360 Content-Type: text/html; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable
Maybe my question was misunderstood. To say it = another=20 way:
 
Is it true that a Status SNACK only = solicits=20 Status PDUs and that it does not solicit Data-In PDUs which = have the=20 S bit set?
 
I interpret Julian's answer to this as "yes, that is = true".=20
 
Eddy
----- Original Message -----
From:=20 Dean=20 Scoville
To: Julian Satran ; Eddy = Quicksall=20
Sent: Tuesday, October 05, 2004 = 5:54=20 PM
Subject: RE: [Ips] resending = Data-in with=20 S bit set

Julian,
 
Section 10.16 states the following about PDUs = transmitted as=20 a result of a SNACK:
 
   "The numbered-response(s) or R2T(s), requested = by a SNACK,=20 MUST be
    delivered as exact = replicas of the=20 ones that the target transmitted = originally
    except for the fields = ExpCmdSN,=20 MaxCmdSN, and ExpDataSN, which MUST
    carry the current = values. R2T(s)=20 requested by SNACK MUST  also carry = the
    current values and = except for=20 resegmentation (see Section 10.16.3=20 Resegmentation)."
 
Section 10.16.1 describes Status and Data/R2T SNACK = types=20 as:
 
    0-Data/R2T SNACK - = requesting=20 retransmission of one or more Data-In or R2T=20 PDUs.
    1-Status SNACK - = requesting=20 retransmission of one or more numbered=20 responses.
 
It seems to me that if the target receives = a Status=20 SNACK for status sent on a Data-In
PDU due=20 to phase collapse, the = target should=20 send an exact replica of the Data-In = PDU.
Similarly, if the target=20 receives a Data = SNACK, it should=20 retransmit the Data-In PDU with
the S bit set, = since it is a=20 = retransmission.
 
I may have misunderstood your recent = posting, but I=20 thought it said the=20 target should=20
generate a totally new SCSI Response PDU in = response=20 to the=20 Status SNACK=20 rather
than retransmitting the original Data-In=20 PDU. And that it should send a Data-In=20 PDU
without the S-bit set in the case of a Data = SNACK.=20 Both of these behaviors seem = to=20
violate what is stated=20 in=20 Sections 10.16 and = 10.16.1 of the=20 iSCSI RFC, as quoted above.
 
I=20 would appreciate clarification on the target's expected response = in these=20 situations.
 
thanks,
 
Dean=20 Scoville
-----Original Message-----
From: = ips-bounces@ietf.org=20 [mailto:ips-bounces@ietf.org]On Behalf Of Julian=20 Satran
Sent: Monday, October 04, 2004 7:43 = AM
To: Eddy=20 Quicksall
Cc: ips@ietf.org
Subject: Re: [Ips] = resending=20 Data-in with S bit set


Eddy,

This = is=20 correct.
The data-in = PDUs don't have=20 to be sent.
The = initiator has to=20 request the data-in PDUs explicitly and MUST do so before it acks = the status=20 (that implicitly acks all the data and deletes any record of the = command at=20 target).

Julo =

ips-bounces@ietf.org wrote on 04/10/2004=20 15:20:56:

> Can you confirm that the intent of 10.16.6. is = as=20 follows?
>   =
> If the Initiator has requested for a Status SNACK = with=20 BegRun=3D0 and
> RunLength=3D0 (i.e, all unacknowledged = status pdus) and=20 the target has
> some Data-In PDUs carrying *status* that have = not=20 been acknowledged,
> the target will *not* resend those = Data-In PDUs=20 carrying the status.
>=20  
> The unacknowledged = queue looks=20 like
>   =
>    response(statSN=3D5) ---> = response(statSN=3D6)=20 -----> Data-In w/S
> bit(statSN=3D7) ---->=20 response(statSN=3D8)
> =  =20
> The target will resend pdus with statSN = 5, 6 and=20 8. The Data-In with
> statSN 7 will not be resent even though = it sends=20 status.
>   =
> Eddy
>=20  _______________________________________________
> Ips = mailing=20 list
> Ips@ietf.org
>=20 = https://www1.ietf.org/mailman/listinfo/ips
<= /TT> ------=_NextPart_000_0102_01C4AB06.53C61360-- --===============0249519802== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline Content-Transfer-Encoding: 7bit _______________________________________________ Ips mailing list Ips@ietf.org https://www1.ietf.org/mailman/listinfo/ips --===============0249519802==-- From ips-bounces@ietf.org Tue Oct 5 18:29:28 2004 Received: from ietf-mx.ietf.org (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id SAA18450 for ; Tue, 5 Oct 2004 18:29:28 -0400 (EDT) Received: from megatron.ietf.org ([132.151.6.71]) by ietf-mx.ietf.org with esmtp (Exim 4.33) id 1CExxf-0007Lq-Ta for ips-web-archive@ietf.org; Tue, 05 Oct 2004 18:39:12 -0400 Received: from localhost.localdomain ([127.0.0.1] helo=megatron.ietf.org) by megatron.ietf.org with esmtp (Exim 4.32) id 1CExk6-0003Ou-4O; Tue, 05 Oct 2004 18:25:10 -0400 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by megatron.ietf.org with esmtp (Exim 4.32) id 1CExgI-0002gG-5W for ips@megatron.ietf.org; Tue, 05 Oct 2004 18:21:15 -0400 Received: from ietf-mx.ietf.org (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id SAA17921 for ; Tue, 5 Oct 2004 18:21:10 -0400 (EDT) Received: from e32.co.us.ibm.com ([32.97.110.130]) by ietf-mx.ietf.org with esmtp (Exim 4.33) id 1CExpd-0006JJ-Gx for ips@ietf.org; Tue, 05 Oct 2004 18:30:55 -0400 Received: from westrelay02.boulder.ibm.com (westrelay02.boulder.ibm.com [9.17.195.11]) by e32.co.us.ibm.com (8.12.10/8.12.9) with ESMTP id i95MKcqY474896; Tue, 5 Oct 2004 18:20:38 -0400 Received: from d03nm115.boulder.ibm.com (d03av04.boulder.ibm.com [9.17.195.170]) by westrelay02.boulder.ibm.com (8.12.10/NCO/VER6.6) with ESMTP id i95MKbcL412936; Tue, 5 Oct 2004 16:20:38 -0600 In-Reply-To: To: Caitlin Bestler MIME-Version: 1.0 Subject: Re: [Ips] DA+iSER Flow Control, Limiting number of oustanding cmdsattarget X-Mailer: Lotus Notes Release 6.0.2CF1 June 9, 2003 From: John Hufferd Message-ID: Date: Tue, 5 Oct 2004 15:20:32 -0700 X-MIMETrack: Serialize by Router on D03NM115/03/M/IBM(Release 6.51HF338 | June 21, 2004) at 10/05/2004 16:20:38, Serialize complete at 10/05/2004 16:20:38 X-Spam-Score: 0.0 (/) X-Scan-Signature: f2728948111f2edaaf8980b5b9de55af Cc: ips@ietf.org X-BeenThere: ips@ietf.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: IP Storage List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Content-Type: multipart/mixed; boundary="===============1964216304==" Sender: ips-bounces@ietf.org Errors-To: ips-bounces@ietf.org X-Spam-Score: 0.0 (/) X-Scan-Signature: 2a9ffb6f997442a3b543bcdaf483b990 This is a multipart message in MIME format. --===============1964216304== Content-Type: multipart/alternative; boundary="=_alternative 007AA7E988256F24_=" This is a multipart message in MIME format. --=_alternative 007AA7E988256F24_= Content-Type: text/plain; charset="US-ASCII" Caitlin, I guess the problem is that we are looking at a completely different model of what is going on with the iSCSI/iSER application. I do not think that we defined iSCSI/iSER with the thought that they were somehow disjoint models. In fact I believe that the name iSCSI/iSER kind of describes how we anticipated the protocol would be used. Having a Data Mover Architecture Service, was not something that we had in mind. iSCSI/iSER is the service. That is, it is a kernel mode Device Driver, which handles specific logical TCP/RNIC ports. When a SRQ is used, it is used by the iSCSI/iSER service, and was not envisioned to be managed by some Data Mover Service that would need to respond to multiple iSCSI Device Drivers. If for some reason a storage controller or OS vendor feels they need to have multiple iSCSI/iSER Device Drivers, the SRQ would apply to each specific one. That is, each iSCSI/iSER Device Driver would have its own SRQ. Therefore, the Architecture was defined with the idea that iSCSI & iSER were "joined at the hip" (sort of speak). If an implementation chooses to separate the iSER from the iSCSI, and if the remaining iSCSI Device Driver is not capable of seeing its buffers fast enough to stop the connection when it goes beyond the iSCSI flow control, then that is simply a bug in the implementation. Even with that type of an implementation the iSCSI Kernel mode Device Driver should be able to inspect the Buffers within nano seconds of the data arrival. Again, I do not see the problem. Now, just so you understand why I object to the issue of additional flow control on top of iSCSI, you need to reference what has to be done with Credit protocols like FC which need to have extra HW boundary boxes to "spoof" the credits in order to have the protocol go across very large distances. iSCSI was design to travel over large distances, without such "spoofing" techniques, and I object to additional Flow control, especially since it is not obvious that it is a hard requirement and also since I do not see the case that a Storage Controller needs "Beyond iSCSI" Flow Control. . . John L. Hufferd Senior Technical Staff Member (STSM) IBM/System Group, San Jose CA Main Office: (408) 256-0403, Tie: 276-0403, eFax: (408) 904-4688 Alt Office: (408) 997-6136, Cell: (408) 499-9702 Internet Address: hufferd@us.ibm.com Caitlin Bestler 10/05/2004 01:33 PM To John Hufferd/San Jose/IBM@IBMUS cc ips@ietf.org Subject Re: [Ips] DA+iSER Flow Control, Limiting number of oustanding cmdsattarget On Oct 5, 2004, at 8:34 PM, John Hufferd wrote: > > Caitlin, > I am having difficulty following your issues. But I think it now has > something to do with the fact that > the code in the iSCSI part of the iSCSI/iSER process will not be able > to determine that its partner > is over subscribing its links. > I am not thinking of the problem as a single software entity. I am focusing on being a provider of a Datamover Architecture service. I have no guarantee that the Consumer of my service is aware that they must reap completions fast enough to prevent the buffer pool from being drained. Further the Datamover specification could easily be read so that they would think it is *my* responsibility to apply back-pressure on the peer to prevent the packets from being sent. The *only* form of back pressure available over RDMA is dropping the connection. I do not want to drop connections without justification, such as violation of a credit limit. Dropping a connection when there has not been a violation or an error is *not* providing a reliable transport service. This is a particular concern with RDMA because it is possible to implement RDMA in a way where the TCP bottlenecks are not totally eliminated. So a competitor's product might conceal this problem with the "solution" of failing to eliminate the TCP bottleneck completely enough. Without a clear statement of responsibility for flow control being at the iSCSI layer my customer may conclude that the problem is *my* implementation. --=_alternative 007AA7E988256F24_= Content-Type: text/html; charset="US-ASCII"
Caitlin,
I guess the problem is that we are looking at a completely different model of what is going on with the iSCSI/iSER application.  I do not think that we defined iSCSI/iSER with the thought that they were somehow disjoint models.  In fact I believe that the name iSCSI/iSER kind of describes how we anticipated the protocol would be used.

Having a Data Mover Architecture Service, was not something that we had in mind.  iSCSI/iSER is the service.  That is, it is a kernel mode Device Driver, which handles specific logical TCP/RNIC ports.  When a SRQ is used, it is used by the iSCSI/iSER service, and was not envisioned to be managed by some Data Mover Service that would need to respond to multiple iSCSI Device Drivers.  If for some reason a storage controller or OS vendor feels they need to have multiple iSCSI/iSER Device Drivers, the SRQ would apply to each specific one.  That is, each iSCSI/iSER Device Driver would have its own SRQ.  

Therefore, the Architecture was defined with the idea that iSCSI & iSER were "joined at the hip" (sort of speak).  If an implementation chooses to separate the iSER from the iSCSI, and if the remaining iSCSI Device Driver is not capable of seeing its buffers fast enough to stop the connection when it goes beyond the iSCSI flow control, then that is simply a bug in the implementation.  Even with that type of an implementation the iSCSI Kernel mode Device Driver should be able to inspect the Buffers within nano seconds of the data arrival.  Again, I do not see the problem.

Now, just so you understand why I object to the issue of additional flow control on top of iSCSI, you need to reference what has to be done with Credit protocols like FC which need to have extra HW boundary boxes to "spoof" the credits in order to have the protocol go across very large distances.  iSCSI was design to travel over large distances, without such "spoofing" techniques, and I object to additional Flow control, especially since it is not obvious that it is a hard requirement and also since I do not see the case that a Storage Controller needs "Beyond iSCSI" Flow Control.  
.
.
John L. Hufferd
Senior Technical Staff Member (STSM)
IBM/System Group, San Jose CA
Main Office: (408) 256-0403, Tie: 276-0403, eFax: (408) 904-4688
Alt Office: (408) 997-6136, Cell: (408) 499-9702
Internet Address: hufferd@us.ibm.com


Caitlin Bestler <cait@asomi.com>

10/05/2004 01:33 PM

To
John Hufferd/San Jose/IBM@IBMUS
cc
ips@ietf.org
Subject
Re: [Ips] DA+iSER Flow Control, Limiting number of oustanding cmdsattarget






On Oct 5, 2004, at 8:34 PM, John Hufferd wrote:

>
> Caitlin,
> I am having difficulty following your issues.  But I think it now has
> something to do with the fact that
> the code in the iSCSI part of the iSCSI/iSER process will not be able
> to determine that its partner
> is over subscribing its links.  
>

I am not thinking of the problem as a single software entity.  I am
focusing on being a provider of a Datamover Architecture service.

I have no guarantee that the Consumer of my service is
aware that they must reap completions fast enough to
prevent the buffer pool from being drained.

Further the Datamover specification could easily be
read so that they would think it is *my* responsibility
to apply back-pressure on the peer to prevent the
packets from being sent.  The *only* form of back
pressure available over RDMA is dropping the
connection.  I do not want to drop connections
without justification, such as violation of a credit
limit.  Dropping a connection when there has not
been a violation or an error is *not* providing a
reliable transport service.

This is a particular concern with RDMA because
it is possible to implement RDMA in a way where
the TCP bottlenecks are not totally eliminated.
So a competitor's product might conceal this
problem with the "solution" of failing to eliminate
the TCP bottleneck completely enough.  Without
a clear statement of responsibility for flow control
being at the iSCSI layer my customer may conclude
that the problem is *my* implementation.
--=_alternative 007AA7E988256F24_=-- --===============1964216304== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline Content-Transfer-Encoding: 7bit _______________________________________________ Ips mailing list Ips@ietf.org https://www1.ietf.org/mailman/listinfo/ips --===============1964216304==-- From ips-bounces@ietf.org Tue Oct 5 18:33:49 2004 Received: from ietf-mx.ietf.org (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id SAA18936 for ; Tue, 5 Oct 2004 18:33:49 -0400 (EDT) Received: from megatron.ietf.org ([132.151.6.71]) by ietf-mx.ietf.org with esmtp (Exim 4.33) id 1CEy1t-0008F5-2y for ips-web-archive@ietf.org; Tue, 05 Oct 2004 18:43:33 -0400 Received: from localhost.localdomain ([127.0.0.1] helo=megatron.ietf.org) by megatron.ietf.org with esmtp (Exim 4.32) id 1CExqg-0004kt-RU; Tue, 05 Oct 2004 18:31:58 -0400 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by megatron.ietf.org with esmtp (Exim 4.32) id 1CExlN-0003f4-AA for ips@megatron.ietf.org; Tue, 05 Oct 2004 18:26:29 -0400 Received: from ietf-mx.ietf.org (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id SAA18260 for ; Tue, 5 Oct 2004 18:26:25 -0400 (EDT) Received: from rospat.qlogic.com ([198.70.192.130] helo=homer.qlogic.com) by ietf-mx.ietf.org with esmtp (Exim 4.33) id 1CExuf-0006uB-Es for ips@ietf.org; Tue, 05 Oct 2004 18:36:09 -0400 X-MimeOLE: Produced By Microsoft Exchange V6.0.6556.0 content-class: urn:content-classes:message MIME-Version: 1.0 Subject: RE: [Ips] resending Data-in with S bit set Date: Tue, 5 Oct 2004 15:22:33 -0700 Message-ID: <2A8D600C919D2D41A3DAA2D2FEC71714AF8F93@homer.qlogic.org> Thread-Topic: [Ips] resending Data-in with S bit set Thread-Index: AcSrJ2Qzo8ILLg/gROyMSgBnVB7PNQAAH7aA From: "Dean Scoville" To: "Eddy Quicksall" , "Julian Satran" X-Spam-Score: 0.3 (/) X-Scan-Signature: 4297a3b9ddbbc388d94c1425bc2288b8 Cc: ips@ietf.org X-BeenThere: ips@ietf.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: IP Storage List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Content-Type: multipart/mixed; boundary="===============1808585975==" Sender: ips-bounces@ietf.org Errors-To: ips-bounces@ietf.org X-Spam-Score: 0.6 (/) X-Scan-Signature: b271b9c4fc51b08326fa0949e61c0156 This is a multi-part message in MIME format. --===============1808585975== content-class: urn:content-classes:message Content-Type: multipart/alternative; boundary="----_=_NextPart_001_01C4AB29.CF9B3722" This is a multi-part message in MIME format. ------_=_NextPart_001_01C4AB29.CF9B3722 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable When you say only "Status" PDUs, do you mean only "SCSI Response" PDUs, and that the target should remain silent about any phase collapsed = Data-In PDUs?=20 That doesn't make sense to me. If the initiator has 100 commands = outstanding and issues a Status SNACK because a received StatSN indicates that some PDUs = are missing, it shouldn't have to issue Data SNACKs for all of the = outstanding commands to see if one of them perhaps was using phase collapse and happened to = have the missing StatusSN. Is that really what is intended? =20 The last sentence in Section 3.2 defines a numbered response as: =20 "Also, unless otherwise specified, status, response, or numbered = response have the same meaning." =20 And Section 6.1.4.2 states: =20 "Lost iSCSI numbered Response. It is recognized by either = identifying a data digest error on a Response PDU or a Data-In PDU carrying the status, or by = receiving a Response PDU with a higher StatSN than expected." =20 The above two RFC statements would lead me to believe that numbered = responses include=20 Data-In PDUs, and that the numbered responses referred to in Section = 10.16.1 in=20 reference to Status SNACKs should also include Data-In retransmissions. =20 Dean -----Original Message----- From: Eddy Quicksall [mailto:eddy_quicksall_iVivity_iSCSI@comcast.net] Sent: Tuesday, October 05, 2004 3:09 PM To: Dean Scoville; Julian Satran Cc: ips@ietf.org Subject: Re: [Ips] resending Data-in with S bit set Maybe my question was misunderstood. To say it another way: =20 Is it true that a Status SNACK only solicits Status PDUs and that it = does not solicit Data-In PDUs which have the S bit set? =20 I interpret Julian's answer to this as "yes, that is true".=20 =20 Eddy ----- Original Message -----=20 From: Dean Scoville =20 To: Julian Satran ; Eddy Quicksall = =20 Cc: ips@ietf.org=20 Sent: Tuesday, October 05, 2004 5:54 PM Subject: RE: [Ips] resending Data-in with S bit set Julian, =20 Section 10.16 states the following about PDUs transmitted as a result of = a SNACK: =20 "The numbered-response(s) or R2T(s), requested by a SNACK, MUST be delivered as exact replicas of the ones that the target transmitted = originally except for the fields ExpCmdSN, MaxCmdSN, and ExpDataSN, which MUST carry the current values. R2T(s) requested by SNACK MUST also carry = the current values and except for resegmentation (see Section 10.16.3 = Resegmentation)." =20 Section 10.16.1 describes Status and Data/R2T SNACK types as: =20 0-Data/R2T SNACK - requesting retransmission of one or more Data-In = or R2T PDUs. 1-Status SNACK - requesting retransmission of one or more numbered = responses. =20 It seems to me that if the target receives a Status SNACK for status = sent on a Data-In=20 PDU due to phase collapse, the target should send an exact replica of = the Data-In PDU. Similarly, if the target receives a Data SNACK, it should retransmit the = Data-In PDU with the S bit set, since it is a retransmission. =20 I may have misunderstood your recent posting, but I thought it said the = target should=20 generate a totally new SCSI Response PDU in response to the Status SNACK = rather than retransmitting the original Data-In PDU. And that it should send a = Data-In PDU without the S-bit set in the case of a Data SNACK. Both of these = behaviors seem to=20 violate what is stated in Sections 10.16 and 10.16.1 of the iSCSI RFC, = as quoted above. =20 I would appreciate clarification on the target's expected response in = these situations. =20 thanks, =20 Dean Scoville -----Original Message----- From: ips-bounces@ietf.org [mailto:ips-bounces@ietf.org]On Behalf Of = Julian Satran Sent: Monday, October 04, 2004 7:43 AM To: Eddy Quicksall Cc: ips@ietf.org Subject: Re: [Ips] resending Data-in with S bit set Eddy,=20 This is correct.=20 The data-in PDUs don't have to be sent.=20 The initiator has to request the data-in PDUs explicitly and MUST do so = before it acks the status (that implicitly acks all the data and deletes = any record of the command at target).=20 Julo=20 ips-bounces@ietf.org wrote on 04/10/2004 15:20:56: > Can you confirm that the intent of 10.16.6. is as follows?=20 > =20 > If the Initiator has requested for a Status SNACK with BegRun=3D0 and=20 > RunLength=3D0 (i.e, all unacknowledged status pdus) and the target has > some Data-In PDUs carrying *status* that have not been acknowledged, > the target will *not* resend those Data-In PDUs carrying the status.=20 > =20 > The unacknowledged queue looks like=20 > =20 > response(statSN=3D5) ---> response(statSN=3D6) -----> Data-In w/S=20 > bit(statSN=3D7) ----> response(statSN=3D8)=20 > =20 > The target will resend pdus with statSN 5, 6 and 8. The Data-In with > statSN 7 will not be resent even though it sends status.=20 > =20 > Eddy=20 > _______________________________________________ > Ips mailing list > Ips@ietf.org > https://www1.ietf.org/mailman/listinfo/ips ------_=_NextPart_001_01C4AB29.CF9B3722 Content-Type: text/html; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable
When=20 you say only "Status" PDUs, do you mean only "SCSI Response"=20 PDUs,
and=20 that the target should remain silent about any phase collapsed Data-In = PDUs?=20
That=20 doesn't make sense to me. If the initiator has 100 commands outstanding=20 and
issues a Status SNACK because a received = StatSN=20 indicates that some PDUs are
missing, it shouldn't have to=20 issue Data SNACKs for all of = the=20 outstanding commands
to see if=20 one of them perhaps was using phase collapse and happened to have=20 the
missing StatusSN. Is that really what is=20 intended?
 
The last sentence in Section 3.2 defines a = numbered=20 response as:
 
    "Also, unless otherwise = specified,=20 status, response, or numbered response have the same=20 meaning."
 
And Section 6.1.4.2=20 states:
 
    "Lost iSCSI numbered = Response. It is=20 recognized by either identifying a data = digest
     error on a Response = PDU or a=20 Data-In PDU carrying the status, or by receiving=20 a
     Response PDU with a = higher=20 StatSN than expected."
 
The above two RFC statements would lead me to = believe=20 that numbered responses include
Data-In PDUs, = and=20 that the numbered responses referred to in Section 10.16.1 in=20
reference to = Status SNACKs should also include Data-In=20 retransmissions.
 
Dean
-----Original Message-----
From: Eddy Quicksall=20 [mailto:eddy_quicksall_iVivity_iSCSI@comcast.net]
Sent: = Tuesday,=20 October 05, 2004 3:09 PM
To: Dean Scoville; Julian=20 Satran
Cc: ips@ietf.org
Subject: Re: [Ips] = resending=20 Data-in with S bit set

Maybe my question was misunderstood. To say it = another=20 way:
 
Is it true that a Status SNACK only = solicits=20 Status PDUs and that it does not solicit Data-In PDUs = which have=20 the S bit set?
 
I interpret Julian's answer to this as "yes, that = is true".=20
 
Eddy
----- Original Message -----
From:=20 Dean Scoville
To: Julian Satran ; Eddy = Quicksall=20
Sent: Tuesday, October 05, = 2004 5:54=20 PM
Subject: RE: [Ips] resending = Data-in=20 with S bit set

Julian,
 
Section 10.16 states the following about PDUs = transmitted=20 as a result of a SNACK:
 
   "The numbered-response(s) or R2T(s), requested = by a=20 SNACK, MUST be
    delivered as exact = replicas of=20 the ones that the target transmitted=20 originally
    except for the fields = ExpCmdSN,=20 MaxCmdSN, and ExpDataSN, which = MUST
    carry the current = values. R2T(s)=20 requested by SNACK MUST  also carry=20 the
    current values and = except for=20 resegmentation (see Section 10.16.3=20 Resegmentation)."
 
Section 10.16.1 describes Status and Data/R2T SNACK = types=20 as:
 
    0-Data/R2T SNACK - = requesting=20 retransmission of one or more Data-In or R2T=20 PDUs.
    1-Status SNACK - = requesting=20 retransmission of one or more numbered=20 responses.
 
It seems to me that if the target = receives a Status=20 SNACK for status sent on a Data-In =
PDU due to phase collapse, the target should send an = exact=20 replica of the Data-In PDU.
Similarly, if the target=20 receives a Data = SNACK, it=20 should retransmit the Data-In PDU = with
the S bit set, = since it is a=20 = retransmission.
 
I may have misunderstood your recent = posting, but I=20 thought it said the=20 target should=20
generate a totally new SCSI Response PDU = in=20 response to the=20 Status SNACK=20 rather
than retransmitting the original Data-In=20 PDU. And that it should send a Data-In=20 PDU
without the S-bit set in the case of a = Data SNACK.=20 Both of these behaviors = seem to=20
violate what is stated=20 in=20 Sections 10.16 and = 10.16.1 of=20 the iSCSI RFC, as quoted above.
 
I=20 would appreciate clarification on the target's expected = response in=20 these situations.
 
thanks,
 
Dean Scoville
-----Original Message-----
From: = ips-bounces@ietf.org=20 [mailto:ips-bounces@ietf.org]On Behalf Of Julian=20 Satran
Sent: Monday, October 04, 2004 7:43 = AM
To: Eddy=20 Quicksall
Cc: ips@ietf.org
Subject: Re: [Ips]=20 resending Data-in with S bit set


Eddy,

This is correct.
The=20 data-in PDUs don't have to be sent.
The initiator has to request the data-in PDUs explicitly = and MUST=20 do so before it acks the status (that implicitly acks all the data = and=20 deletes any record of the command at target).

Julo

ips-bounces@ietf.org wrote on 04/10/2004 = 15:20:56:

>=20 Can you confirm that the intent of 10.16.6. is as = follows?=20
>  
> If=20 the Initiator has requested for a Status SNACK with BegRun=3D0 and =
>=20 RunLength=3D0 (i.e, all unacknowledged status pdus) and the target = has
> some Data-In PDUs carrying *status* that have not been = acknowledged,
> the target will *not* resend those Data-In = PDUs=20 carrying the status.
>=20  
> The unacknowledged = queue=20 looks like
> =  =20
>    response(statSN=3D5) = --->=20 response(statSN=3D6) -----> Data-In w/S
> = bit(statSN=3D7) ---->=20 response(statSN=3D8)
>=20  
> The target will = resend pdus=20 with statSN 5, 6 and 8. The Data-In with
> statSN 7 will not = be=20 resent even though it sends status.
>=20  
> Eddy =
>=20  _______________________________________________
> Ips = mailing=20 list
> Ips@ietf.org
>=20 = https://www1.ietf.org/mailman/listinfo/ips
<= /BLOCKQUOTE> ------_=_NextPart_001_01C4AB29.CF9B3722-- --===============1808585975== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline Content-Transfer-Encoding: 7bit _______________________________________________ Ips mailing list Ips@ietf.org https://www1.ietf.org/mailman/listinfo/ips --===============1808585975==-- From ips-bounces@ietf.org Tue Oct 5 18:52:50 2004 Received: from ietf-mx.ietf.org (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id SAA20655 for ; Tue, 5 Oct 2004 18:52:49 -0400 (EDT) Received: from megatron.ietf.org ([132.151.6.71]) by ietf-mx.ietf.org with esmtp (Exim 4.33) id 1CEyKH-0002aa-TC for ips-web-archive@ietf.org; Tue, 05 Oct 2004 19:02:34 -0400 Received: from localhost.localdomain ([127.0.0.1] helo=megatron.ietf.org) by megatron.ietf.org with esmtp (Exim 4.32) id 1CEy6K-0007ow-Px; Tue, 05 Oct 2004 18:48:08 -0400 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by megatron.ietf.org with esmtp (Exim 4.32) id 1CEy2k-0006tR-G6 for ips@megatron.ietf.org; Tue, 05 Oct 2004 18:44:26 -0400 Received: from ietf-mx.ietf.org (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id SAA19819 for ; Tue, 5 Oct 2004 18:44:23 -0400 (EDT) Received: from sccrmhc13.comcast.net ([204.127.202.64]) by ietf-mx.ietf.org with esmtp (Exim 4.33) id 1CEyC7-0000mu-4p for ips@ietf.org; Tue, 05 Oct 2004 18:54:07 -0400 Received: from ivvt2dxrc11 (209-30-141-131.ded.swbell.net[209.30.141.131]) by comcast.net (sccrmhc13) with SMTP id <2004100522435301600b9cvne> (Authid: esquicksall); Tue, 5 Oct 2004 22:43:54 +0000 Message-ID: <013501c4ab2c$d4dc88f0$6e01000a@ivvt2dxrc11> From: "Eddy Quicksall" To: "Dean Scoville" , "Julian Satran" References: <2A8D600C919D2D41A3DAA2D2FEC71714AF8F93@homer.qlogic.org> Subject: Re: [Ips] resending Data-in with S bit set Date: Tue, 5 Oct 2004 18:44:09 -0400 MIME-Version: 1.0 X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 6.00.2800.1437 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2800.1441 X-Spam-Score: 0.7 (/) X-Scan-Signature: d9951f061208886fc6ada4b24aa6f98d Cc: ips@ietf.org X-BeenThere: ips@ietf.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: IP Storage List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Content-Type: multipart/mixed; boundary="===============1315112406==" Sender: ips-bounces@ietf.org Errors-To: ips-bounces@ietf.org X-Spam-Score: 0.7 (/) X-Scan-Signature: b148ead9c6581b10314b24a9438d3a5f This is a multi-part message in MIME format. --===============1315112406== Content-Type: multipart/alternative; boundary="----=_NextPart_000_0132_01C4AB0B.4CFC2740" This is a multi-part message in MIME format. ------=_NextPart_000_0132_01C4AB0B.4CFC2740 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable Yes, when I said "Status PDUs", I used shorthand and meant SCSI Response = PDUs". What you point out is exactly the reason I wanted to know the intent and = if I understood the spec correctly. I did not mean to say I agree or = disagree with the intent. Since the spec is finished, I think it is too late to argue for a = change. It looks like the initiator just has to be a little smart when = it needs all PDUs that carry status. Eddy ----- Original Message -----=20 From: Dean Scoville=20 To: Eddy Quicksall ; Julian Satran=20 Cc: ips@ietf.org=20 Sent: Tuesday, October 05, 2004 6:22 PM Subject: RE: [Ips] resending Data-in with S bit set When you say only "Status" PDUs, do you mean only "SCSI Response" = PDUs, and that the target should remain silent about any phase collapsed = Data-In PDUs?=20 That doesn't make sense to me. If the initiator has 100 commands = outstanding and issues a Status SNACK because a received StatSN indicates that some = PDUs are missing, it shouldn't have to issue Data SNACKs for all of the = outstanding commands to see if one of them perhaps was using phase collapse and happened to = have the missing StatusSN. Is that really what is intended? The last sentence in Section 3.2 defines a numbered response as: "Also, unless otherwise specified, status, response, or numbered = response have the same meaning." And Section 6.1.4.2 states: "Lost iSCSI numbered Response. It is recognized by either = identifying a data digest error on a Response PDU or a Data-In PDU carrying the status, or = by receiving a Response PDU with a higher StatSN than expected." The above two RFC statements would lead me to believe that numbered = responses include=20 Data-In PDUs, and that the numbered responses referred to in Section = 10.16.1 in=20 reference to Status SNACKs should also include Data-In = retransmissions. Dean -----Original Message----- From: Eddy Quicksall = [mailto:eddy_quicksall_iVivity_iSCSI@comcast.net] Sent: Tuesday, October 05, 2004 3:09 PM To: Dean Scoville; Julian Satran Cc: ips@ietf.org Subject: Re: [Ips] resending Data-in with S bit set Maybe my question was misunderstood. To say it another way: Is it true that a Status SNACK only solicits Status PDUs and that it = does not solicit Data-In PDUs which have the S bit set? I interpret Julian's answer to this as "yes, that is true".=20 Eddy ----- Original Message -----=20 From: Dean Scoville=20 To: Julian Satran ; Eddy Quicksall=20 Cc: ips@ietf.org=20 Sent: Tuesday, October 05, 2004 5:54 PM Subject: RE: [Ips] resending Data-in with S bit set Julian, Section 10.16 states the following about PDUs transmitted as a = result of a SNACK: "The numbered-response(s) or R2T(s), requested by a SNACK, MUST = be delivered as exact replicas of the ones that the target = transmitted originally except for the fields ExpCmdSN, MaxCmdSN, and ExpDataSN, which = MUST carry the current values. R2T(s) requested by SNACK MUST also = carry the current values and except for resegmentation (see Section = 10.16.3 Resegmentation)." Section 10.16.1 describes Status and Data/R2T SNACK types as: 0-Data/R2T SNACK - requesting retransmission of one or more = Data-In or R2T PDUs. 1-Status SNACK - requesting retransmission of one or more = numbered responses. It seems to me that if the target receives a Status SNACK for = status sent on a Data-In=20 PDU due to phase collapse, the target should send an exact replica = of the Data-In PDU. Similarly, if the target receives a Data SNACK, it should = retransmit the Data-In PDU with the S bit set, since it is a retransmission. I may have misunderstood your recent posting, but I thought it = said the target should=20 generate a totally new SCSI Response PDU in response to the Status = SNACK rather than retransmitting the original Data-In PDU. And that it should = send a Data-In PDU without the S-bit set in the case of a Data SNACK. Both of these = behaviors seem to=20 violate what is stated in Sections 10.16 and 10.16.1 of the iSCSI = RFC, as quoted above. I would appreciate clarification on the target's expected response = in these situations. thanks, Dean Scoville -----Original Message----- From: ips-bounces@ietf.org [mailto:ips-bounces@ietf.org]On = Behalf Of Julian Satran Sent: Monday, October 04, 2004 7:43 AM To: Eddy Quicksall Cc: ips@ietf.org Subject: Re: [Ips] resending Data-in with S bit set Eddy,=20 This is correct.=20 The data-in PDUs don't have to be sent.=20 The initiator has to request the data-in PDUs explicitly and = MUST do so before it acks the status (that implicitly acks all the data = and deletes any record of the command at target).=20 Julo=20 ips-bounces@ietf.org wrote on 04/10/2004 15:20:56: > Can you confirm that the intent of 10.16.6. is as follows?=20 > =20 > If the Initiator has requested for a Status SNACK with = BegRun=3D0 and=20 > RunLength=3D0 (i.e, all unacknowledged status pdus) and the = target has > some Data-In PDUs carrying *status* that have not been = acknowledged, > the target will *not* resend those Data-In PDUs carrying the = status.=20 > =20 > The unacknowledged queue looks like=20 > =20 > response(statSN=3D5) ---> response(statSN=3D6) -----> = Data-In w/S=20 > bit(statSN=3D7) ----> response(statSN=3D8)=20 > =20 > The target will resend pdus with statSN 5, 6 and 8. The = Data-In with > statSN 7 will not be resent even though it sends status.=20 > =20 > Eddy=20 > _______________________________________________ > Ips mailing list > Ips@ietf.org > https://www1.ietf.org/mailman/listinfo/ips -------------------------------------------------------------------------= ----- _______________________________________________ Ips mailing list Ips@ietf.org https://www1.ietf.org/mailman/listinfo/ips ------=_NextPart_000_0132_01C4AB0B.4CFC2740 Content-Type: text/html; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable
Yes, when I said "Status PDUs", I used shorthand and = meant=20 SCSI Response PDUs".
 
What you point out is exactly the reason I wanted to = know the=20 intent and if I understood the spec correctly. I did not mean to = say I=20 agree or disagree with the intent.
 
Since the spec is finished, I think it is too late = to argue=20 for a change. It looks like the initiator just has to be a little smart = when it=20 needs all PDUs that carry status.
 
Eddy
----- Original Message -----
From:=20 Dean=20 Scoville
To: Eddy = Quicksall ; Julian=20 Satran
Sent: Tuesday, October 05, 2004 = 6:22=20 PM
Subject: RE: [Ips] resending = Data-in with=20 S bit set

When=20 you say only "Status" PDUs, do you mean only "SCSI Response"=20 PDUs,
and=20 that the target should remain silent about any phase collapsed Data-In = PDUs?=20
That=20 doesn't make sense to me. If the initiator has 100 commands = outstanding=20 and
issues a Status SNACK because a received = StatSN=20 indicates that some PDUs are
missing, it shouldn't have to=20 issue Data SNACKs for all = of the=20 outstanding commands
to see if=20 one of them perhaps was using phase collapse and happened to have=20 the
missing StatusSN. Is that really what is=20 intended?
 
The last sentence in Section 3.2 defines a = numbered=20 response as:
 
    "Also, unless otherwise = specified,=20 status, response, or numbered response have the same=20 meaning."
 
And Section 6.1.4.2=20 states:
 
    "Lost iSCSI numbered = Response. It=20 is recognized by either identifying a data=20 digest
     error on a = Response PDU or a=20 Data-In PDU carrying the status, or by receiving=20 a
     Response PDU with = a higher=20 StatSN than expected."
 
The above two RFC statements would lead me = to believe=20 that numbered responses include
Data-In PDUs, = and=20 that the numbered responses referred to in Section 10.16.1 in=20
reference to = Status SNACKs should also include Data-In=20 retransmissions.
 
Dean
-----Original Message-----
From: Eddy Quicksall=20 [mailto:eddy_quicksall_iVivity_iSCSI@comcast.net]
Sent: = Tuesday,=20 October 05, 2004 3:09 PM
To: Dean Scoville; Julian=20 Satran
Cc: ips@ietf.org
Subject: Re: [Ips] = resending=20 Data-in with S bit set

Maybe my question was misunderstood. To say it = another=20 way:
 
Is it true that a Status SNACK only = solicits=20 Status PDUs and that it does not solicit Data-In PDUs = which have=20 the S bit set?
 
I interpret Julian's answer to this as "yes, = that is=20 true".
 
Eddy
----- Original Message ----- =
From:=20 Dean Scoville
To: Julian Satran ; Eddy = Quicksall=20
Sent: Tuesday, October 05, = 2004 5:54=20 PM
Subject: RE: [Ips] = resending Data-in=20 with S bit set

Julian,
 
Section 10.16 states the following about PDUs = transmitted=20 as a result of a SNACK:
 
   "The numbered-response(s) or R2T(s), = requested by a=20 SNACK, MUST be
    delivered as exact = replicas of=20 the ones that the target transmitted=20 originally
    except for the = fields=20 ExpCmdSN, MaxCmdSN, and ExpDataSN, which=20 MUST
    carry the current = values.=20 R2T(s) requested by SNACK MUST  also carry=20 the
    current values and = except for=20 resegmentation (see Section 10.16.3=20 Resegmentation)."
 
Section 10.16.1 describes Status and Data/R2T SNACK = types=20 as:
 
    0-Data/R2T SNACK - = requesting=20 retransmission of one or more Data-In or R2T=20 PDUs.
    1-Status SNACK - = requesting=20 retransmission of one or more numbered=20 responses.
 
It seems to me that if the target = receives a=20 Status SNACK for status sent on a = Data-In =
PDU due to = phase collapse, the target should send = an exact=20 replica of the Data-In PDU.
Similarly, if the target=20 receives a Data = SNACK, it=20 should retransmit the Data-In PDU = with
the S bit set, = since it is a=20 = retransmission.
 
I may have misunderstood your recent = posting, but=20 I thought it said the target should =
generate a totally new SCSI Response = PDU in=20 response to the=20 Status SNACK=20 rather
than retransmitting the original = Data-In=20 PDU. And that it should send a Data-In=20 PDU
without the S-bit set in the case of a = Data=20 SNACK. Both of these behaviors seem to =
violate what is stated=20 in=20 Sections 10.16 and = 10.16.1 of=20 the iSCSI RFC, as quoted above.
 
I would appreciate clarification on the target's = expected=20 response in these situations.
 
thanks,
 
Dean Scoville
-----Original Message-----
From: = ips-bounces@ietf.org=20 [mailto:ips-bounces@ietf.org]On Behalf Of Julian=20 Satran
Sent: Monday, October 04, 2004 7:43 = AM
To:=20 Eddy Quicksall
Cc: ips@ietf.org
Subject: Re: = [Ips]=20 resending Data-in with S bit set


Eddy,

This is correct.
The=20 data-in PDUs don't have to be sent.
The initiator has to request the data-in PDUs = explicitly and MUST=20 do so before it acks the status (that implicitly acks all the = data and=20 deletes any record of the command at target). =

Julo

ips-bounces@ietf.org wrote on 04/10/2004=20 15:20:56:

> Can you confirm that the intent of = 10.16.6. is as=20 follows?
> =  =20
> If the Initiator has requested for a = Status=20 SNACK with BegRun=3D0 and
> RunLength=3D0 (i.e, all = unacknowledged=20 status pdus) and the target has
> some Data-In PDUs = carrying=20 *status* that have not been acknowledged,
> the target = will *not*=20 resend those Data-In PDUs carrying the status. =
>  
> = The=20 unacknowledged queue looks like
>=20  
>  =20  response(statSN=3D5) ---> response(statSN=3D6) = -----> Data-In=20 w/S
> bit(statSN=3D7) ----> = response(statSN=3D8)=20
>  
>=20 The target will resend pdus with statSN 5, 6 and 8. The Data-In=20 with
> statSN 7 will not be resent even though it sends=20 status.
> =  =20
> Eddy
>=20  _______________________________________________
> = Ips=20 mailing list
> Ips@ietf.org
>=20 = https://www1.ietf.org/mailman/listinfo/ips
<= /BLOCKQUOTE>

_______________________________________________
Ips mailing=20 = list
Ips@ietf.org
https://www1.ietf.org/mailman/listinfo/ips
------=_NextPart_000_0132_01C4AB0B.4CFC2740-- --===============1315112406== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline Content-Transfer-Encoding: 7bit _______________________________________________ Ips mailing list Ips@ietf.org https://www1.ietf.org/mailman/listinfo/ips --===============1315112406==-- From ips-bounces@ietf.org Tue Oct 5 18:59:39 2004 Received: from ietf-mx.ietf.org (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id SAA21182 for ; Tue, 5 Oct 2004 18:59:39 -0400 (EDT) Received: from megatron.ietf.org ([132.151.6.71]) by ietf-mx.ietf.org with esmtp (Exim 4.33) id 1CEyQt-0003tZ-6W for ips-web-archive@ietf.org; Tue, 05 Oct 2004 19:09:23 -0400 Received: from localhost.localdomain ([127.0.0.1] helo=megatron.ietf.org) by megatron.ietf.org with esmtp (Exim 4.32) id 1CEyCe-0000gk-Bm; Tue, 05 Oct 2004 18:54:40 -0400 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by megatron.ietf.org with esmtp (Exim 4.32) id 1CEyB5-0000F1-KL for ips@megatron.ietf.org; Tue, 05 Oct 2004 18:53:03 -0400 Received: from ietf-mx.ietf.org (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id SAA20678 for ; Tue, 5 Oct 2004 18:53:00 -0400 (EDT) Received: from rospat.qlogic.com ([198.70.192.130] helo=homer.qlogic.com) by ietf-mx.ietf.org with esmtp (Exim 4.33) id 1CEyKS-0002a4-8W for ips@ietf.org; Tue, 05 Oct 2004 19:02:45 -0400 X-MimeOLE: Produced By Microsoft Exchange V6.0.6556.0 content-class: urn:content-classes:message MIME-Version: 1.0 Subject: RE: [Ips] resending Data-in with S bit set Date: Tue, 5 Oct 2004 15:49:12 -0700 Message-ID: <2A8D600C919D2D41A3DAA2D2FEC71714735434@homer.qlogic.org> Thread-Topic: [Ips] resending Data-in with S bit set Thread-Index: AcSrLFrvz64HW6i9RPqXpSDBb1qwmwAACzpQ From: "Dean Scoville" To: "Eddy Quicksall" , "Julian Satran" X-Spam-Score: 0.6 (/) X-Scan-Signature: 66ff42a56498684039efb58366a4b354 Cc: ips@ietf.org X-BeenThere: ips@ietf.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: IP Storage List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Content-Type: multipart/mixed; boundary="===============0266113936==" Sender: ips-bounces@ietf.org Errors-To: ips-bounces@ietf.org X-Spam-Score: 0.6 (/) X-Scan-Signature: 52b2f07c128fbe337bbf87ebd50cfd46 This is a multi-part message in MIME format. --===============0266113936== content-class: urn:content-classes:message Content-Type: multipart/alternative; boundary="----_=_NextPart_001_01C4AB2D.8871437E" This is a multi-part message in MIME format. ------_=_NextPart_001_01C4AB2D.8871437E Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable I'd say the RFC is ambiguous, because it sounds like it is now being = interpreted to say that=20 numbered responses include only SCSI Response PDUs, even though I = thought the statements in sections 3.2 and 6.1.4.2 indicated that numbered responses = include any PDUs carrying status... including Data-In PDUs with the S-bit set. Does that = mean the initiator shouldn't be able to SNACK for a Reject PDU or a NOP-PDU, since they = aren't SCSI Response PDUs either? =20 Dean -----Original Message----- From: Eddy Quicksall [mailto:eddy_quicksall_iVivity_iSCSI@comcast.net] Sent: Tuesday, October 05, 2004 3:44 PM To: Dean Scoville; Julian Satran Cc: ips@ietf.org Subject: Re: [Ips] resending Data-in with S bit set Yes, when I said "Status PDUs", I used shorthand and meant SCSI Response = PDUs". =20 What you point out is exactly the reason I wanted to know the intent and = if I understood the spec correctly. I did not mean to say I agree or = disagree with the intent. =20 Since the spec is finished, I think it is too late to argue for a = change. It looks like the initiator just has to be a little smart when = it needs all PDUs that carry status. =20 Eddy ----- Original Message -----=20 From: Dean Scoville =20 To: Eddy Quicksall ; = Julian Satran =20 Cc: ips@ietf.org=20 Sent: Tuesday, October 05, 2004 6:22 PM Subject: RE: [Ips] resending Data-in with S bit set When you say only "Status" PDUs, do you mean only "SCSI Response" PDUs, and that the target should remain silent about any phase collapsed = Data-In PDUs?=20 That doesn't make sense to me. If the initiator has 100 commands = outstanding and issues a Status SNACK because a received StatSN indicates that some PDUs = are missing, it shouldn't have to issue Data SNACKs for all of the = outstanding commands to see if one of them perhaps was using phase collapse and happened to = have the missing StatusSN. Is that really what is intended? =20 The last sentence in Section 3.2 defines a numbered response as: =20 "Also, unless otherwise specified, status, response, or numbered = response have the same meaning." =20 And Section 6.1.4.2 states: =20 "Lost iSCSI numbered Response. It is recognized by either = identifying a data digest error on a Response PDU or a Data-In PDU carrying the status, or by = receiving a Response PDU with a higher StatSN than expected." =20 The above two RFC statements would lead me to believe that numbered = responses include=20 Data-In PDUs, and that the numbered responses referred to in Section = 10.16.1 in=20 reference to Status SNACKs should also include Data-In retransmissions. =20 Dean -----Original Message----- From: Eddy Quicksall [mailto:eddy_quicksall_iVivity_iSCSI@comcast.net] Sent: Tuesday, October 05, 2004 3:09 PM To: Dean Scoville; Julian Satran Cc: ips@ietf.org Subject: Re: [Ips] resending Data-in with S bit set Maybe my question was misunderstood. To say it another way: =20 Is it true that a Status SNACK only solicits Status PDUs and that it = does not solicit Data-In PDUs which have the S bit set? =20 I interpret Julian's answer to this as "yes, that is true".=20 =20 Eddy ----- Original Message -----=20 From: Dean Scoville =20 To: Julian Satran ; Eddy = Quicksall=20 Cc: ips@ietf.org=20 Sent: Tuesday, October 05, 2004 5:54 PM Subject: RE: [Ips] resending Data-in with S bit set Julian, =20 Section 10.16 states the following about PDUs transmitted as a result of = a SNACK: =20 "The numbered-response(s) or R2T(s), requested by a SNACK, MUST be delivered as exact replicas of the ones that the target transmitted = originally except for the fields ExpCmdSN, MaxCmdSN, and ExpDataSN, which MUST carry the current values. R2T(s) requested by SNACK MUST also carry = the current values and except for resegmentation (see Section 10.16.3 = Resegmentation)." =20 Section 10.16.1 describes Status and Data/R2T SNACK types as: =20 0-Data/R2T SNACK - requesting retransmission of one or more Data-In = or R2T PDUs. 1-Status SNACK - requesting retransmission of one or more numbered = responses. =20 It seems to me that if the target receives a Status SNACK for status = sent on a Data-In=20 PDU due to phase collapse, the target should send an exact replica of = the Data-In PDU. Similarly, if the target receives a Data SNACK, it should retransmit the = Data-In PDU with the S bit set, since it is a retransmission. =20 I may have misunderstood your recent posting, but I thought it said the = target should=20 generate a totally new SCSI Response PDU in response to the Status SNACK = rather than retransmitting the original Data-In PDU. And that it should send a = Data-In PDU without the S-bit set in the case of a Data SNACK. Both of these = behaviors seem to=20 violate what is stated in Sections 10.16 and 10.16.1 of the iSCSI RFC, = as quoted above. =20 I would appreciate clarification on the target's expected response in = these situations. =20 thanks, =20 Dean Scoville -----Original Message----- From: ips-bounces@ietf.org [mailto:ips-bounces@ietf.org]On Behalf Of = Julian Satran Sent: Monday, October 04, 2004 7:43 AM To: Eddy Quicksall Cc: ips@ietf.org Subject: Re: [Ips] resending Data-in with S bit set Eddy,=20 This is correct.=20 The data-in PDUs don't have to be sent.=20 The initiator has to request the data-in PDUs explicitly and MUST do so = before it acks the status (that implicitly acks all the data and deletes = any record of the command at target).=20 Julo=20 ips-bounces@ietf.org wrote on 04/10/2004 15:20:56: > Can you confirm that the intent of 10.16.6. is as follows?=20 > =20 > If the Initiator has requested for a Status SNACK with BegRun=3D0 and=20 > RunLength=3D0 (i.e, all unacknowledged status pdus) and the target has > some Data-In PDUs carrying *status* that have not been acknowledged, > the target will *not* resend those Data-In PDUs carrying the status.=20 > =20 > The unacknowledged queue looks like=20 > =20 > response(statSN=3D5) ---> response(statSN=3D6) -----> Data-In w/S=20 > bit(statSN=3D7) ----> response(statSN=3D8)=20 > =20 > The target will resend pdus with statSN 5, 6 and 8. The Data-In with > statSN 7 will not be resent even though it sends status.=20 > =20 > Eddy=20 > _______________________________________________ > Ips mailing list > Ips@ietf.org > https://www1.ietf.org/mailman/listinfo/ips _____ =20 _______________________________________________ Ips mailing list Ips@ietf.org https://www1.ietf.org/mailman/listinfo/ips ------_=_NextPart_001_01C4AB2D.8871437E Content-Type: text/html; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable
I'd=20 say the RFC is ambiguous, because it sounds like it is now being=20 interpreted to say that
numbered responses include only SCSI Response PDUs, even though = I thought=20 the
statements in sections 3.2=20 and 6.1.4.2 indicated that numbered responses include any=20 PDUs
carrying status... including Data-In PDUs with the S-bit set. = Does that=20 mean the initiator
shouldn't be able to SNACK for a Reject PDU or a NOP-PDU, since = they=20 aren't SCSI Response
PDUs=20 either?
 
Dean
-----Original Message-----
From: Eddy Quicksall=20 [mailto:eddy_quicksall_iVivity_iSCSI@comcast.net]
Sent: = Tuesday,=20 October 05, 2004 3:44 PM
To: Dean Scoville; Julian=20 Satran
Cc: ips@ietf.org
Subject: Re: [Ips] = resending=20 Data-in with S bit set

Yes, when I said "Status PDUs", I used shorthand = and meant=20 SCSI Response PDUs".
 
What you point out is exactly the reason I wanted = to know=20 the intent and if I understood the spec correctly. I did not mean = to say=20 I agree or disagree with the intent.
 
Since the spec is finished, I think it is too late = to argue=20 for a change. It looks like the initiator just has to be a little = smart when=20 it needs all PDUs that carry status.
 
Eddy
----- Original Message -----
From:=20 Dean Scoville
To: Eddy = Quicksall ;=20 Julian Satran
Sent: Tuesday, October 05, = 2004 6:22=20 PM
Subject: RE: [Ips] resending = Data-in=20 with S bit set

When you say only "Status" PDUs, do you mean only "SCSI = Response"=20 PDUs,
and that the target should remain silent about any phase = collapsed=20 Data-In PDUs?
That doesn't make sense to me. If the initiator has 100 = commands=20 outstanding and
issues a Status SNACK because a received = StatSN=20 indicates that some PDUs are
missing, it shouldn't have to=20 issue Data SNACKs for = all of the=20 outstanding commands
to see if one of them perhaps was using phase = collapse and=20 happened to have the
missing StatusSN. Is that really what is=20 intended?
 
The last sentence in Section 3.2 defines = a numbered=20 response as:
 
    "Also, unless = otherwise=20 specified, status, response, or numbered response have the same=20 meaning."
 
And Section 6.1.4.2=20 states:
 
    "Lost iSCSI numbered = Response.=20 It is recognized by either identifying a data=20 digest
     error on a = Response PDU or=20 a Data-In PDU carrying the status, or by receiving=20 a
     Response PDU = with a higher=20 StatSN than expected."
 
The above two RFC statements would lead = me to=20 believe that numbered responses include =
Data-In PDUs, = and that the numbered responses referred = to in=20 Section 10.16.1 in
reference to = Status SNACKs should also include Data-In = retransmissions.
 
Dean
-----Original Message-----
From: Eddy Quicksall = [mailto:eddy_quicksall_iVivity_iSCSI@comcast.net]
Sent: = Tuesday,=20 October 05, 2004 3:09 PM
To: Dean Scoville; Julian=20 Satran
Cc: ips@ietf.org
Subject: Re: [Ips] = resending=20 Data-in with S bit set

Maybe my question was misunderstood. To say it = another=20 way:
 
Is it true that a Status SNACK = only solicits=20 Status PDUs and that it does not solicit Data-In PDUs = which=20 have the S bit set?
 
I interpret Julian's answer to this as "yes, = that is=20 true".
 
Eddy
----- Original Message ----- =
From:=20 Dean Scoville =
To: Julian Satran ; Eddy=20 Quicksall
Sent: Tuesday, October = 05, 2004=20 5:54 PM
Subject: RE: [Ips] = resending=20 Data-in with S bit set

Julian,
 
Section 10.16 states the following about PDUs = transmitted as a result of a SNACK:
 
   "The numbered-response(s) or R2T(s), = requested by a=20 SNACK, MUST be
    delivered as exact = replicas=20 of the ones that the target transmitted=20 originally
    except for the = fields=20 ExpCmdSN, MaxCmdSN, and ExpDataSN, which=20 MUST
    carry the current = values.=20 R2T(s) requested by SNACK MUST  also carry=20 the
    current values and = except=20 for resegmentation (see Section 10.16.3=20 Resegmentation)."
 
Section 10.16.1 describes Status and Data/R2T = SNACK types=20 as:
 
    0-Data/R2T SNACK - = requesting retransmission of one or more Data-In or R2T=20 PDUs.
    1-Status SNACK - = requesting=20 retransmission of one or more numbered=20 responses.
 
It seems to me that if the target = receives a=20 Status SNACK for status sent on a = Data-In =
PDU due to = phase collapse, the target should = send an exact=20 replica of the Data-In PDU.
Similarly, if the target=20 receives a Data = SNACK, it=20 should retransmit the Data-In PDU = with
the S bit set,=20 since it is a=20 = retransmission.
 
I may have misunderstood your recent = posting,=20 but I thought it said the target should=20
generate a totally new SCSI Response = PDU in=20 response to the=20 Status SNACK=20 rather
than retransmitting the original = Data-In=20 PDU. And that it should send a Data-In=20 PDU
without the S-bit set in the case of = a Data=20 SNACK. Both of these behaviors seem to =
violate what is stated=20 in=20 Sections 10.16 = and 10.16.1=20 of the iSCSI RFC, as quoted = above.
 
I would appreciate clarification on the target's = expected=20 response in these situations.
 
thanks,
 
Dean Scoville
-----Original Message-----
From: = ips-bounces@ietf.org=20 [mailto:ips-bounces@ietf.org]On Behalf Of Julian=20 Satran
Sent: Monday, October 04, 2004 7:43 = AM
To:=20 Eddy Quicksall
Cc: ips@ietf.org
Subject: = Re: [Ips]=20 resending Data-in with S bit set


Eddy,

This is correct.
The=20 data-in PDUs don't have to be sent.
The initiator has to request the data-in PDUs = explicitly and=20 MUST do so before it acks the status (that implicitly acks all = the=20 data and deletes any record of the command at target).=20

Julo =

ips-bounces@ietf.org wrote on 04/10/2004=20 15:20:56:

> Can you confirm that the intent of = 10.16.6. is=20 as follows?
> =  =20
> If the Initiator has requested for = a Status=20 SNACK with BegRun=3D0 and
> RunLength=3D0 (i.e, all = unacknowledged=20 status pdus) and the target has
> some Data-In PDUs = carrying=20 *status* that have not been acknowledged,
> the target = will=20 *not* resend those Data-In PDUs carrying the = status.=20
>  
>=20 The unacknowledged queue looks like
>  
>  =20  response(statSN=3D5) ---> response(statSN=3D6) = -----> Data-In=20 w/S
> bit(statSN=3D7) ----> = response(statSN=3D8)=20
>  
>=20 The target will resend pdus with statSN 5, 6 and 8. The = Data-In=20 with
> statSN 7 will not be resent even though it sends=20 status.
> =  =20
> Eddy
>=20  _______________________________________________
> = Ips=20 mailing list
> Ips@ietf.org
>=20 = https://www1.ietf.org/mailman/listinfo/ips
<= /BLOCKQUOTE>

_______________________________________________
Ips = mailing=20 = list
Ips@ietf.org
https://www1.ietf.org/mailman/listinfo/ips
------_=_NextPart_001_01C4AB2D.8871437E-- --===============0266113936== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline Content-Transfer-Encoding: 7bit _______________________________________________ Ips mailing list Ips@ietf.org https://www1.ietf.org/mailman/listinfo/ips --===============0266113936==-- From ips-bounces@ietf.org Tue Oct 5 19:08:22 2004 Received: from ietf-mx.ietf.org (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id TAA21966 for ; Tue, 5 Oct 2004 19:08:22 -0400 (EDT) Received: from megatron.ietf.org ([132.151.6.71]) by ietf-mx.ietf.org with esmtp (Exim 4.33) id 1CEyZK-0004TT-4x for ips-web-archive@ietf.org; Tue, 05 Oct 2004 19:18:07 -0400 Received: from localhost.localdomain ([127.0.0.1] helo=megatron.ietf.org) by megatron.ietf.org with esmtp (Exim 4.32) id 1CEyL5-00026Z-Nk; Tue, 05 Oct 2004 19:03:23 -0400 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by megatron.ietf.org with esmtp (Exim 4.32) id 1CEyGZ-0001L1-QK for ips@megatron.ietf.org; Tue, 05 Oct 2004 18:58:44 -0400 Received: from ietf-mx.ietf.org (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id SAA21064 for ; Tue, 5 Oct 2004 18:58:40 -0400 (EDT) Received: from mononoke.wasabisystems.com ([66.173.145.228]) by ietf-mx.ietf.org with esmtp (Exim 4.33) id 1CEyPw-0003Th-Vr for ips@ietf.org; Tue, 05 Oct 2004 19:08:25 -0400 Received: from [192.168.1.125] (unknown [66.173.145.226]) by mononoke.wasabisystems.com (Postfix) with ESMTP id D776F870AB; Tue, 5 Oct 2004 18:58:41 -0400 (EDT) In-Reply-To: <2A8D600C919D2D41A3DAA2D2FEC71714AF8F93@homer.qlogic.org> References: <2A8D600C919D2D41A3DAA2D2FEC71714AF8F93@homer.qlogic.org> Mime-Version: 1.0 (Apple Message framework v619) Message-Id: <15429145-1722-11D9-8100-000A95D14BEC@wasabisystems.com> From: William Studenmund Subject: Re: [Ips] resending Data-in with S bit set Date: Tue, 5 Oct 2004 18:58:27 -0400 To: "Dean Scoville" X-Pgp-Agent: GPGMail 1.0 (v30, 10.3) X-Mailer: Apple Mail (2.619) X-Spam-Score: 0.0 (/) X-Scan-Signature: b1c41982e167b872076d0018e4e1dc3c Cc: ips@ietf.org, Eddy Quicksall , Julian Satran X-BeenThere: ips@ietf.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: IP Storage List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Content-Type: multipart/mixed; boundary="===============1492195705==" Sender: ips-bounces@ietf.org Errors-To: ips-bounces@ietf.org X-Spam-Score: 0.0 (/) X-Scan-Signature: 0770535483960d190d4a0d020e7060bd --===============1492195705== Content-Type: multipart/signed; protocol="application/pgp-signature"; micalg=pgp-sha1; boundary="Apple-Mail-25-598313135" --Apple-Mail-25-598313135 Content-Type: multipart/signed; micalg=sha1; boundary=Apple-Mail-24-598313130; protocol="application/pkcs7-signature" --Apple-Mail-24-598313130 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: quoted-printable On Oct 5, 2004, at 6:22 PM, Dean Scoville wrote: > When you say only "Status" PDUs, do you mean only "SCSI Response" = PDUs, > and that the target should remain silent about any phase collapsed=20 > Data-In PDUs? > That doesn't make sense to me. If the initiator has 100 commands=20 > outstanding and > issues a Status SNACK because a received StatSN indicates that some=20 > PDUs are > missing, it shouldn't have to issue Data SNACKs for all of the=20 > outstanding commands > to see if one of them perhaps was using phase collapse and happened to=20= > have the > missing StatusSN. Is that really what is intended? To be honest, while I think that's what Julian said, I don't think=20 that's correct. I think we should treat a phase-collapsed PDU as both a=20= status PDU and a data PDU. The problem with requiring Data SNACK be the=20= only way to request a retransmit is that if the initiator was looking=20 for a specific StatSN, it has to retry every task to find it; the=20 request is indexed inefficiently. > The last sentence in Section 3.2 defines a numbered response as: > =A0 > =A0=A0=A0 "Also, unless otherwise specified, status, response, or = numbered=20 > response have the same meaning." > =A0 > And Section 6.1.4.2 states: > =A0 > =A0=A0=A0 "Lost iSCSI numbered Response. It is recognized by either=20 > identifying a data digest > =A0=A0=A0=A0 error on a Response PDU or a Data-In PDU carrying the = status, or=20 > by receiving a > =A0=A0=A0=A0 Response PDU with a higher StatSN than expected." > =A0 > The above two RFC statements would lead me to believe that numbered=20 > responses include > Data-In PDUs, and that the numbered responses referred to in Section=20= > 10.16.1 in > reference to Status SNACKs should also include Data-In=20 > retransmissions. Take care, Bill --Apple-Mail-24-598313130 Content-Type: application/pkcs7-signature; name=smime.p7s Content-Disposition: attachment; filename=smime.p7s Content-Transfer-Encoding: base64 MIAGCSqGSIb3DQEHAqCAMIACAQExCzAJBgUrDgMCGgUAMIAGCSqGSIb3DQEHAQAAoIIFfDCCBXgw ggNgoAMCAQICAUYwDQYJKoZIhvcNAQEEBQAwga0xCzAJBgNVBAYTAlVTMREwDwYDVQQIEwhOZXcg WW9yazERMA8GA1UEBxMITmV3IFlvcmsxHTAbBgNVBAoTFFdhc2FiaSBTeXN0ZW1zLCBJbmMuMQww CgYDVQQLEwNHSFExHzAdBgNVBAMTFldhc2FiaSBTeXN0ZW1zIFJvb3QgQ0ExKjAoBgkqhkiG9w0B CQEWG3dlYm1hc3RlckB3YXNhYmlzeXN0ZW1zLmNvbTAeFw0wMzA4MTExODU2NDZaFw0wODA2MjUx ODU2NDZaMIGXMQswCQYDVQQGEwJVUzETMBEGA1UECBMKQ2FsaWZvcm5pYTESMBAGA1UEBxMJU2Fu IERpZWdvMRcwFQYDVQQKEw5XYXNhYmkgU3lzdGVtczEbMBkGA1UEAxMSV2lsbGlhbSBTdHVkZW5t dW5kMSkwJwYJKoZIhvcNAQkBFhp3cnN0dWRlbkB3YXNhYmlzeXN0ZW1zLmNvbTCBnzANBgkqhkiG 9w0BAQEFAAOBjQAwgYkCgYEA9CZhg4GVWYTRGcJz8lU1ipzInlBuGqfZQs88Z1rTP7+c3AyhKwpE TyTmK4UhxslzKa93YLqifLxZKYt1jI6FNq40IQWCKBVywWBwmL5guCu0851f9ywH5Uj2wJdiDrOI sOJKNpwNqml8bFdTrFS2qvyCxTIWVMLYAzUBBcl2F7MCAwEAAaOCATkwggE1MAkGA1UdEwQCMAAw LAYJYIZIAYb4QgENBB8WHU9wZW5TU0wgR2VuZXJhdGVkIENlcnRpZmljYXRlMB0GA1UdDgQWBBTt mjHbSBlMV48RWOBETM3EV0Tm/jCB2gYDVR0jBIHSMIHPgBQrBOnN88CNCHPaV5wLS212tcrgBaGB s6SBsDCBrTELMAkGA1UEBhMCVVMxETAPBgNVBAgTCE5ldyBZb3JrMREwDwYDVQQHEwhOZXcgWW9y azEdMBsGA1UEChMUV2FzYWJpIFN5c3RlbXMsIEluYy4xDDAKBgNVBAsTA0dIUTEfMB0GA1UEAxMW V2FzYWJpIFN5c3RlbXMgUm9vdCBDQTEqMCgGCSqGSIb3DQEJARYbd2VibWFzdGVyQHdhc2FiaXN5 c3RlbXMuY29tggEAMA0GCSqGSIb3DQEBBAUAA4ICAQCpxoPEYunR7qiXyycEZd0UIatSt/WalMoH nuN9S421iS/sD1wSDqzHyM5YyUWn4z4o6VhT2kD9UV4lk5aXxxefEfGzFcR7Xyah7tO3ziuReFfx qUP5+DTlwDuUBc8Iow6NwY9GlRCKFSsEroLWLGODyYWgN1ojjKOtcXRiOZkYTP3VfDUV2/Xrhh9O l6+3Otf069+KWeOWedWeMsbe1rac96yPEjPogToWkBXVJEeEQktd0BLETt7GUuSZg2Fho/nslqZv nnnKZmrOfdFy9iXDL9T0Hp/OHf5B9RP+r00BJJFO8JzBPIL2IA3CBPApOQty5A00Jg1CGfDQxiV3 s8G0aRUbmFqg9r0OfhPYjJ8BHfw1HHWb7Cu91Sw+d5UnYracZhsR2Gc5H5CnfS6IZhc7ulfnSX8f I4BXR4vbeRUG76J3rkkbkO1haYHLk5w5RlICHsJOBSElG9ggLBWNIfUZwAzo84lfmDmJMP1YCOmi DFCbW1DgxhqVFpYEf5Pq2pVcCguopK2G9oZjH5fxq6ehisk9dN+7NEQ4OSqC43ehaStYOoQAJruN e6e5tCtRMx7bMI7eVPiHxE9/Y38Q+sc5oRN9mCyyAXMweuMRNHAHAWdFB7NhHD6gBa2lh91zIMdf iSVRlBXoMrBKJX6VecgnhvUp4G6QAnN7EMu6QHlc/jGCA0swggNHAgEBMIGzMIGtMQswCQYDVQQG EwJVUzERMA8GA1UECBMITmV3IFlvcmsxETAPBgNVBAcTCE5ldyBZb3JrMR0wGwYDVQQKExRXYXNh YmkgU3lzdGVtcywgSW5jLjEMMAoGA1UECxMDR0hRMR8wHQYDVQQDExZXYXNhYmkgU3lzdGVtcyBS b290IENBMSowKAYJKoZIhvcNAQkBFht3ZWJtYXN0ZXJAd2FzYWJpc3lzdGVtcy5jb20CAUYwCQYF Kw4DAhoFAKCCAe0wGAYJKoZIhvcNAQkDMQsGCSqGSIb3DQEHATAcBgkqhkiG9w0BCQUxDxcNMDQx MDA1MjI1ODI4WjAjBgkqhkiG9w0BCQQxFgQUuGbRrYMfZrTlLpG4x7p+xFkckH8wgcQGCSsGAQQB gjcQBDGBtjCBszCBrTELMAkGA1UEBhMCVVMxETAPBgNVBAgTCE5ldyBZb3JrMREwDwYDVQQHEwhO ZXcgWW9yazEdMBsGA1UEChMUV2FzYWJpIFN5c3RlbXMsIEluYy4xDDAKBgNVBAsTA0dIUTEfMB0G A1UEAxMWV2FzYWJpIFN5c3RlbXMgUm9vdCBDQTEqMCgGCSqGSIb3DQEJARYbd2VibWFzdGVyQHdh c2FiaXN5c3RlbXMuY29tAgFGMIHGBgsqhkiG9w0BCRACCzGBtqCBszCBrTELMAkGA1UEBhMCVVMx ETAPBgNVBAgTCE5ldyBZb3JrMREwDwYDVQQHEwhOZXcgWW9yazEdMBsGA1UEChMUV2FzYWJpIFN5 c3RlbXMsIEluYy4xDDAKBgNVBAsTA0dIUTEfMB0GA1UEAxMWV2FzYWJpIFN5c3RlbXMgUm9vdCBD QTEqMCgGCSqGSIb3DQEJARYbd2VibWFzdGVyQHdhc2FiaXN5c3RlbXMuY29tAgFGMA0GCSqGSIb3 DQEBAQUABIGALtwI7c8IFNq62Fz3qs+V/c6WzSC073GssKtV0B/OgM2vf1lT9E/H4/SR7cA0mY9L 9K9BPpuImFnuw9/bHBYdgJCb5qxZHRd1TBvy0BGdxa04W/Aw/5zek5Vciu724GiDgKgt7ju7zzah D5BnpVbGFeSd3uwM4GJrtlugbATb5XMAAAAAAAA= --Apple-Mail-24-598313130-- --Apple-Mail-25-598313135 content-type: application/pgp-signature; x-mac-type=70674453; name=PGP.sig content-description: This is a digitally signed message part content-disposition: inline; filename=PGP.sig content-transfer-encoding: 7bit Content-Transfer-Encoding: 7bit Content-Transfer-Encoding: 7bit -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.2.3 (Darwin) iD8DBQFBYycgDJT2Egh26K0RAr1fAJ9Y4mDMuQYjX0LzHvBXku2FdX9b/QCfTykf RRIMgWjrYIRmWiiamx9Z5Rc= =/t5x -----END PGP SIGNATURE----- --Apple-Mail-25-598313135-- --===============1492195705== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline Content-Transfer-Encoding: 7bit _______________________________________________ Ips mailing list Ips@ietf.org https://www1.ietf.org/mailman/listinfo/ips --===============1492195705==-- From ips-bounces@ietf.org Tue Oct 5 23:05:52 2004 Received: from ietf-mx.ietf.org (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id XAA11106 for ; Tue, 5 Oct 2004 23:05:52 -0400 (EDT) Received: from megatron.ietf.org ([132.151.6.71]) by ietf-mx.ietf.org with esmtp (Exim 4.33) id 1CF2HB-0006YR-OE for ips-web-archive@ietf.org; Tue, 05 Oct 2004 23:15:38 -0400 Received: from localhost.localdomain ([127.0.0.1] helo=megatron.ietf.org) by megatron.ietf.org with esmtp (Exim 4.32) id 1CF25N-00049Y-QE; Tue, 05 Oct 2004 23:03:25 -0400 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by megatron.ietf.org with esmtp (Exim 4.32) id 1CF21G-0003oH-Su for ips@megatron.ietf.org; Tue, 05 Oct 2004 22:59:11 -0400 Received: from ietf-mx.ietf.org (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id WAA10722 for ; Tue, 5 Oct 2004 22:59:08 -0400 (EDT) Received: from sccrmhc11.comcast.net ([204.127.202.55]) by ietf-mx.ietf.org with esmtp (Exim 4.33) id 1CF2Af-00057n-OD for ips@ietf.org; Tue, 05 Oct 2004 23:08:54 -0400 Received: from ivvt2dxrc11 (209-30-141-131.ded.swbell.net[209.30.141.131]) by comcast.net (sccrmhc11) with SMTP id <2004100602583701100j7rv3e> (Authid: esquicksall); Wed, 6 Oct 2004 02:58:37 +0000 Message-ID: <016901c4ab50$60d7df30$6e01000a@ivvt2dxrc11> From: "Eddy Quicksall" To: "Dean Scoville" , "Julian Satran" References: <2A8D600C919D2D41A3DAA2D2FEC71714AF8F93@homer.qlogic.org> Subject: Re: [Ips] resending Data-in with S bit set Date: Tue, 5 Oct 2004 22:58:36 -0400 MIME-Version: 1.0 X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 6.00.2800.1437 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2800.1441 X-Spam-Score: 0.7 (/) X-Scan-Signature: 27216fd639035830d9361a5ade4ff99c Cc: ips@ietf.org X-BeenThere: ips@ietf.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: IP Storage List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Content-Type: multipart/mixed; boundary="===============1063641593==" Sender: ips-bounces@ietf.org Errors-To: ips-bounces@ietf.org X-Spam-Score: 0.7 (/) X-Scan-Signature: cb592aae7f4601895f35714165597859 This is a multi-part message in MIME format. --===============1063641593== Content-Type: multipart/alternative; boundary="----=_NextPart_000_0166_01C4AB2E.D8B7B560" This is a multi-part message in MIME format. ------=_NextPart_000_0166_01C4AB2E.D8B7B560 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable I think Dean has a good point here. I missed the definition in 3.2 he = gives below and now it looks like the intent was to include the Data-In = PDUs with the S bit set as those sent in response to a Status SNACK. = Also, it makes the most sense as Dean has pointed out. Julian, can you please check with the other authors and see what the = original intent was? Eddy ----- Original Message -----=20 From: Dean Scoville=20 To: Eddy Quicksall ; Julian Satran=20 Cc: ips@ietf.org=20 Sent: Tuesday, October 05, 2004 6:22 PM Subject: RE: [Ips] resending Data-in with S bit set When you say only "Status" PDUs, do you mean only "SCSI Response" = PDUs, and that the target should remain silent about any phase collapsed = Data-In PDUs?=20 That doesn't make sense to me. If the initiator has 100 commands = outstanding and issues a Status SNACK because a received StatSN indicates that some = PDUs are missing, it shouldn't have to issue Data SNACKs for all of the = outstanding commands to see if one of them perhaps was using phase collapse and happened to = have the missing StatusSN. Is that really what is intended? The last sentence in Section 3.2 defines a numbered response as: "Also, unless otherwise specified, status, response, or numbered = response have the same meaning." And Section 6.1.4.2 states: "Lost iSCSI numbered Response. It is recognized by either = identifying a data digest error on a Response PDU or a Data-In PDU carrying the status, or = by receiving a Response PDU with a higher StatSN than expected." The above two RFC statements would lead me to believe that numbered = responses include=20 Data-In PDUs, and that the numbered responses referred to in Section = 10.16.1 in=20 reference to Status SNACKs should also include Data-In = retransmissions. Dean -----Original Message----- From: Eddy Quicksall = [mailto:eddy_quicksall_iVivity_iSCSI@comcast.net] Sent: Tuesday, October 05, 2004 3:09 PM To: Dean Scoville; Julian Satran Cc: ips@ietf.org Subject: Re: [Ips] resending Data-in with S bit set Maybe my question was misunderstood. To say it another way: Is it true that a Status SNACK only solicits Status PDUs and that it = does not solicit Data-In PDUs which have the S bit set? I interpret Julian's answer to this as "yes, that is true".=20 Eddy ----- Original Message -----=20 From: Dean Scoville=20 To: Julian Satran ; Eddy Quicksall=20 Cc: ips@ietf.org=20 Sent: Tuesday, October 05, 2004 5:54 PM Subject: RE: [Ips] resending Data-in with S bit set Julian, Section 10.16 states the following about PDUs transmitted as a = result of a SNACK: "The numbered-response(s) or R2T(s), requested by a SNACK, MUST = be delivered as exact replicas of the ones that the target = transmitted originally except for the fields ExpCmdSN, MaxCmdSN, and ExpDataSN, which = MUST carry the current values. R2T(s) requested by SNACK MUST also = carry the current values and except for resegmentation (see Section = 10.16.3 Resegmentation)." Section 10.16.1 describes Status and Data/R2T SNACK types as: 0-Data/R2T SNACK - requesting retransmission of one or more = Data-In or R2T PDUs. 1-Status SNACK - requesting retransmission of one or more = numbered responses. It seems to me that if the target receives a Status SNACK for = status sent on a Data-In=20 PDU due to phase collapse, the target should send an exact replica = of the Data-In PDU. Similarly, if the target receives a Data SNACK, it should = retransmit the Data-In PDU with the S bit set, since it is a retransmission. I may have misunderstood your recent posting, but I thought it = said the target should=20 generate a totally new SCSI Response PDU in response to the Status = SNACK rather than retransmitting the original Data-In PDU. And that it should = send a Data-In PDU without the S-bit set in the case of a Data SNACK. Both of these = behaviors seem to=20 violate what is stated in Sections 10.16 and 10.16.1 of the iSCSI = RFC, as quoted above. I would appreciate clarification on the target's expected response = in these situations. thanks, Dean Scoville -----Original Message----- From: ips-bounces@ietf.org [mailto:ips-bounces@ietf.org]On = Behalf Of Julian Satran Sent: Monday, October 04, 2004 7:43 AM To: Eddy Quicksall Cc: ips@ietf.org Subject: Re: [Ips] resending Data-in with S bit set Eddy,=20 This is correct.=20 The data-in PDUs don't have to be sent.=20 The initiator has to request the data-in PDUs explicitly and = MUST do so before it acks the status (that implicitly acks all the data = and deletes any record of the command at target).=20 Julo=20 ips-bounces@ietf.org wrote on 04/10/2004 15:20:56: > Can you confirm that the intent of 10.16.6. is as follows?=20 > =20 > If the Initiator has requested for a Status SNACK with = BegRun=3D0 and=20 > RunLength=3D0 (i.e, all unacknowledged status pdus) and the = target has > some Data-In PDUs carrying *status* that have not been = acknowledged, > the target will *not* resend those Data-In PDUs carrying the = status.=20 > =20 > The unacknowledged queue looks like=20 > =20 > response(statSN=3D5) ---> response(statSN=3D6) -----> = Data-In w/S=20 > bit(statSN=3D7) ----> response(statSN=3D8)=20 > =20 > The target will resend pdus with statSN 5, 6 and 8. The = Data-In with > statSN 7 will not be resent even though it sends status.=20 > =20 > Eddy=20 > _______________________________________________ > Ips mailing list > Ips@ietf.org > https://www1.ietf.org/mailman/listinfo/ips -------------------------------------------------------------------------= ----- _______________________________________________ Ips mailing list Ips@ietf.org https://www1.ietf.org/mailman/listinfo/ips ------=_NextPart_000_0166_01C4AB2E.D8B7B560 Content-Type: text/html; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable
I think Dean has a good point here. I missed the = definition in=20 3.2 he gives below and now it looks like the intent was to include the = Data-In=20 PDUs with the S bit set as those sent in response to a Status SNACK. = Also, it=20 makes the most sense as Dean has pointed out.
 
Julian, can you please check with the other authors = and see=20 what the original intent was?
 
Eddy
----- Original Message -----
From:=20 Dean=20 Scoville
To: Eddy = Quicksall ; Julian=20 Satran
Sent: Tuesday, October 05, 2004 = 6:22=20 PM
Subject: RE: [Ips] resending = Data-in with=20 S bit set

When=20 you say only "Status" PDUs, do you mean only "SCSI Response"=20 PDUs,
and=20 that the target should remain silent about any phase collapsed Data-In = PDUs?=20
That=20 doesn't make sense to me. If the initiator has 100 commands = outstanding=20 and
issues a Status SNACK because a received = StatSN=20 indicates that some PDUs are
missing, it shouldn't have to=20 issue Data SNACKs for all = of the=20 outstanding commands
to see if=20 one of them perhaps was using phase collapse and happened to have=20 the
missing StatusSN. Is that really what is=20 intended?
 
The last sentence in Section 3.2 defines a = numbered=20 response as:
 
    "Also, unless otherwise = specified,=20 status, response, or numbered response have the same=20 meaning."
 
And Section 6.1.4.2=20 states:
 
    "Lost iSCSI numbered = Response. It=20 is recognized by either identifying a data=20 digest
     error on a = Response PDU or a=20 Data-In PDU carrying the status, or by receiving=20 a
     Response PDU with = a higher=20 StatSN than expected."
 
The above two RFC statements would lead me = to believe=20 that numbered responses include
Data-In PDUs, = and=20 that the numbered responses referred to in Section 10.16.1 in=20
reference to = Status SNACKs should also include Data-In=20 retransmissions.
 
Dean
-----Original Message-----
From: Eddy Quicksall=20 [mailto:eddy_quicksall_iVivity_iSCSI@comcast.net]
Sent: = Tuesday,=20 October 05, 2004 3:09 PM
To: Dean Scoville; Julian=20 Satran
Cc: ips@ietf.org
Subject: Re: [Ips] = resending=20 Data-in with S bit set

Maybe my question was misunderstood. To say it = another=20 way:
 
Is it true that a Status SNACK only = solicits=20 Status PDUs and that it does not solicit Data-In PDUs = which have=20 the S bit set?
 
I interpret Julian's answer to this as "yes, = that is=20 true".
 
Eddy
----- Original Message ----- =
From:=20 Dean Scoville
To: Julian Satran ; Eddy = Quicksall=20
Sent: Tuesday, October 05, = 2004 5:54=20 PM
Subject: RE: [Ips] = resending Data-in=20 with S bit set

Julian,
 
Section 10.16 states the following about PDUs = transmitted=20 as a result of a SNACK:
 
   "The numbered-response(s) or R2T(s), = requested by a=20 SNACK, MUST be
    delivered as exact = replicas of=20 the ones that the target transmitted=20 originally
    except for the = fields=20 ExpCmdSN, MaxCmdSN, and ExpDataSN, which=20 MUST
    carry the current = values.=20 R2T(s) requested by SNACK MUST  also carry=20 the
    current values and = except for=20 resegmentation (see Section 10.16.3=20 Resegmentation)."
 
Section 10.16.1 describes Status and Data/R2T SNACK = types=20 as:
 
    0-Data/R2T SNACK - = requesting=20 retransmission of one or more Data-In or R2T=20 PDUs.
    1-Status SNACK - = requesting=20 retransmission of one or more numbered=20 responses.
 
It seems to me that if the target = receives a=20 Status SNACK for status sent on a = Data-In =
PDU due to = phase collapse, the target should send = an exact=20 replica of the Data-In PDU.
Similarly, if the target=20 receives a Data = SNACK, it=20 should retransmit the Data-In PDU = with
the S bit set, = since it is a=20 = retransmission.
 
I may have misunderstood your recent = posting, but=20 I thought it said the target should =
generate a totally new SCSI Response = PDU in=20 response to the=20 Status SNACK=20 rather
than retransmitting the original = Data-In=20 PDU. And that it should send a Data-In=20 PDU
without the S-bit set in the case of a = Data=20 SNACK. Both of these behaviors seem to =
violate what is stated=20 in=20 Sections 10.16 and = 10.16.1 of=20 the iSCSI RFC, as quoted above.
 
I would appreciate clarification on the target's = expected=20 response in these situations.
 
thanks,
 
Dean Scoville
-----Original Message-----
From: = ips-bounces@ietf.org=20 [mailto:ips-bounces@ietf.org]On Behalf Of Julian=20 Satran
Sent: Monday, October 04, 2004 7:43 = AM
To:=20 Eddy Quicksall
Cc: ips@ietf.org
Subject: Re: = [Ips]=20 resending Data-in with S bit set


Eddy,

This is correct.
The=20 data-in PDUs don't have to be sent.
The initiator has to request the data-in PDUs = explicitly and MUST=20 do so before it acks the status (that implicitly acks all the = data and=20 deletes any record of the command at target). =

Julo

ips-bounces@ietf.org wrote on 04/10/2004=20 15:20:56:

> Can you confirm that the intent of = 10.16.6. is as=20 follows?
> =  =20
> If the Initiator has requested for a = Status=20 SNACK with BegRun=3D0 and
> RunLength=3D0 (i.e, all = unacknowledged=20 status pdus) and the target has
> some Data-In PDUs = carrying=20 *status* that have not been acknowledged,
> the target = will *not*=20 resend those Data-In PDUs carrying the status. =
>  
> = The=20 unacknowledged queue looks like
>=20  
>  =20  response(statSN=3D5) ---> response(statSN=3D6) = -----> Data-In=20 w/S
> bit(statSN=3D7) ----> = response(statSN=3D8)=20
>  
>=20 The target will resend pdus with statSN 5, 6 and 8. The Data-In=20 with
> statSN 7 will not be resent even though it sends=20 status.
> =  =20
> Eddy
>=20  _______________________________________________
> = Ips=20 mailing list
> Ips@ietf.org
>=20 = https://www1.ietf.org/mailman/listinfo/ips
<= /BLOCKQUOTE>

_______________________________________________
Ips mailing=20 = list
Ips@ietf.org
https://www1.ietf.org/mailman/listinfo/ips
------=_NextPart_000_0166_01C4AB2E.D8B7B560-- --===============1063641593== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline Content-Transfer-Encoding: 7bit _______________________________________________ Ips mailing list Ips@ietf.org https://www1.ietf.org/mailman/listinfo/ips --===============1063641593==-- From ips-bounces@ietf.org Tue Oct 5 23:12:36 2004 Received: from ietf-mx.ietf.org (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id XAA11525 for ; Tue, 5 Oct 2004 23:12:36 -0400 (EDT) Received: from megatron.ietf.org ([132.151.6.71]) by ietf-mx.ietf.org with esmtp (Exim 4.33) id 1CF2Ni-0007RP-KW for ips-web-archive@ietf.org; Tue, 05 Oct 2004 23:22:23 -0400 Received: from localhost.localdomain ([127.0.0.1] helo=megatron.ietf.org) by megatron.ietf.org with esmtp (Exim 4.32) id 1CF2Bt-00057u-HE; Tue, 05 Oct 2004 23:10:09 -0400 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by megatron.ietf.org with esmtp (Exim 4.32) id 1CF25W-0004Bh-8u for ips@megatron.ietf.org; Tue, 05 Oct 2004 23:03:34 -0400 Received: from ietf-mx.ietf.org (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id XAA10982 for ; Tue, 5 Oct 2004 23:03:31 -0400 (EDT) Received: from system60.simplicato.com ([207.99.47.60] helo=host52a.simplicato.com) by ietf-mx.ietf.org with esmtp (Exim 4.33) id 1CF2Ev-00067m-5t for ips@ietf.org; Tue, 05 Oct 2004 23:13:17 -0400 Received: from localhost (localhost.simplicato.com [127.0.0.1]) by host52a.simplicato.com (Postfix) with ESMTP id DEB9411E054; Tue, 5 Oct 2004 23:03:25 -0400 (EDT) Received: from host52a.simplicato.com ([127.0.0.1]) by localhost (host52a.simplicato.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 09172-06; Tue, 5 Oct 2004 23:03:24 -0400 (EDT) Received: from [192.168.1.2] (bzq-198-76.red.bezeqint.net [212.179.198.76]) by host52a.simplicato.com (Postfix) with ESMTP id 1502911E026; Tue, 5 Oct 2004 23:03:22 -0400 (EDT) In-Reply-To: References: Mime-Version: 1.0 (Apple Message framework v619) Message-Id: <381D9C18-1744-11D9-90FD-000393814604@asomi.com> From: Caitlin Bestler Subject: Re: [Ips] DA+iSER Flow Control, Limiting number of oustanding cmdsattarget Date: Wed, 6 Oct 2004 05:02:55 +0200 To: John Hufferd X-Mailer: Apple Mail (2.619) X-Virus-Scanned: by amavisd-new at simplicato.com X-Spam-Score: 0.1 (/) X-Scan-Signature: 7e267523e0685e5aa2dbbdde4b659686 Cc: ips@ietf.org X-BeenThere: ips@ietf.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: IP Storage List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Content-Type: multipart/mixed; boundary="===============1932168219==" Sender: ips-bounces@ietf.org Errors-To: ips-bounces@ietf.org X-Spam-Score: 0.1 (/) X-Scan-Signature: 1c0c3d540ad9f95212b1c2a9a2cc2595 --===============1932168219== Content-Type: multipart/alternative; boundary=Apple-Mail-5-612974498 --Apple-Mail-5-612974498 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: quoted-printable On Oct 6, 2004, at 12:20 AM, John Hufferd wrote: > > Caitlin, > I guess the problem is that we are looking at a completely different=20= > model of what is going on with the > iSCSI/iSER application. =A0I do not think that we defined iSCSI/iSER=20= > with the thought that they were somehow > disjoint models. =A0In fact I believe that the name iSCSI/iSER kind of=20= > describes how we anticipated the protocol > would be used. > But that is how the DataMover Architecture is specified. It is a clean=20= layer. Responsibilities must be separated in specifications, even if=20 implementations choose to be integrated. Isn't that fundamental to the process of=20 setting standards? > > > Therefore, the Architecture was defined with the idea that iSCSI &=20 > iSER were "joined at the hip" (sort of speak). > If an implementation chooses to separate the iSER from the iSCSI, and=20= > if the remaining iSCSI Device Driver is not > capable of seeing its buffers fast enough to stop the connection when=20= > it goes beyond the iSCSI flow control, then > that is simply a bug in the implementation. =A0Even with that type of = an=20 > implementation the iSCSI Kernel mode Device > Driver should be able to inspect the Buffers within nano seconds of=20 > the data arrival. =A0Again, I do not see the problem. > The pre-existing iWARP interfaces are *not* "joined at the hip" with=20 their users. One of the goals is to use enable iSCSI to be implemented over the general purpose iWARP=20 service. Doing that requires The Upper Layer Protocol to provide for flow control of=20 untagged messages, and to protect against attacks of resources shared across session lines. The pre-existing interfaces are *not* designed to push deliverable=20 packets to the ULP on demand, but rather to generate completions of pre-posted receive=20 operations, which the ULP consumes at its own pace. Further, each connection has a credit=20 limit, violation of which will result at the minimum in an alarm being raised allowing for=20= prompt termination of a runaway or malicious client. If the iSCSI layer is *not* using the per-connection limit, and is=20 using a shared resource to service multiple clients that have no basis for shared trust it is=20 violating the RDMAP/RDDP security requirements for ULPs. So in order to comply with the Security *requirements* for using iWARP,=20= iSER MUST have sufficient information to set per-connection limits on untagged=20 messages. It is also basic theory. You cannot provide a reliable transport=20 service (in the absence of a lower layer error) unless you can block sends or push receives. =20 The DataMover architecture fails to define this, but does require it to be reliable. > Now, just so you understand why I object to the issue of additional=20 > flow control on top of iSCSI, > you need to reference what has to be done with Credit protocols like=20= > FC which need to have extra > HW boundary boxes to "spoof" the credits in order to have the protocol=20= > go across very large distances. > =A0iSCSI was design to travel over large distances, without such=20 > "spoofing" techniques, and I object t > additional Flow control, especially since it is not obvious that it=20= > is a hard requirement and also since I > do not see the case that a Storage Controller needs "Beyond iSCSI"=20 > Flow Control. =A0 Absolutely no new wire protocol is needed. The only thing that is=20 required is to *document* the an allowance for the packets that iSCSI currently does not flow control. iSCSI is relying on the TCP bottleneck to flow control these. When TCP buffer credits have been exhausted the sender is prevented from sending more data without breaking the connection. The DataMover Architecture needs to be defined so it works over transports that do not provide this bottleneck "feature". That includes iSER, and it would include other possible DataMover architectures. Stating that the the DataMover architecture user may be required to pre-supply buffers would solve the broad computer science problem. It would not solve the security vulnerability of enabling expedited handling of a resource drain caused by a single malfunctioning or malicious client. Failure to address this issue would make DA and iSER proposed standards that *fail* to comply with the security requirements of the protocol they claim to be utilizing. Obviously the IETF cannot hand-inspect applications to ensure they comply with ULP requirements, but at the minimum other IETF drafts should be compliant with IETF requirements. --Apple-Mail-5-612974498 Content-Type: text/enriched; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable On Oct 6, 2004, at 12:20 AM, John Hufferd wrote: Caitlin,=20 I guess the problem is that we are looking at a completely different model of what is going on with the iSCSI/iSER application. =A0I do not think that we defined iSCSI/iSER with the thought that they were somehow disjoint models. =A0In fact I believe that the name iSCSI/iSER kind of describes how we anticipated the protocol would be used. But that is how the DataMover Architecture is specified. It is a clean layer. Responsibilities must be separated in specifications, even if implementations choose to be integrated. Isn't that fundamental to the process of setting standards? Therefore, the Architecture was defined with the idea that iSCSI & iSER were "joined at the hip" (sort of speak). If an implementation chooses to separate the iSER from the iSCSI, and if the remaining iSCSI Device Driver is not capable of seeing its buffers fast enough to stop the connection when it goes beyond the iSCSI flow control, then that is simply a bug in the implementation. =A0Even with that type of an implementation the iSCSI Kernel mode Device Driver should be able to inspect the Buffers within nano seconds of the data arrival. =A0Again, I do not see the problem.=20 The pre-existing iWARP interfaces are *not* "joined at the hip" with their users. One of the goals is to use enable iSCSI to be implemented over the general purpose iWARP service. Doing that requires The Upper Layer Protocol to provide for flow control of untagged messages, and to protect against attacks of resources shared across session lines. The pre-existing interfaces are *not* designed to push deliverable packets to the ULP on demand, but rather to generate completions of pre-posted receive operations, which the ULP consumes at its own pace. Further, each connection has a credit limit, violation of which will result at the minimum in an alarm being raised allowing for prompt termination of a runaway or malicious client. If the iSCSI layer is *not* using the per-connection limit, and is using a shared resource to service multiple clients that have no basis for shared trust it is violating the RDMAP/RDDP security requirements for ULPs. So in order to comply with the Security *requirements* for using iWARP, iSER MUST have sufficient information to set per-connection limits on untagged messages. It is also basic theory. You cannot provide a reliable transport service (in the absence of a lower layer error) unless you can block sends or push receives.=20 The DataMover architecture fails to define this, but does require it to be reliable. Now, just so you understand why I object to the issue of additional flow control on top of iSCSI, you need to reference what has to be done with Credit protocols like FC which need to have extra HW boundary boxes to "spoof" the credits in order to have the protocol go across very large distances. =A0iSCSI was design to travel over large distances, without such "spoofing" techniques, and I object t additional Flow control, especially since it is not obvious that it is a hard requirement and also since I do not see the case that a Storage Controller needs "Beyond iSCSI" Flow Control. =A0=20 Absolutely no new wire protocol is needed. The only thing that is required is to *document* the an allowance for the packets that iSCSI currently does not flow control. iSCSI is relying on the TCP bottleneck to flow control these. When TCP buffer credits have been exhausted the sender is prevented from sending more data without breaking the connection. The DataMover Architecture needs to be defined so it works over transports that do not provide this bottleneck "feature". That includes iSER, and it would include other possible DataMover architectures. Stating that the the DataMover architecture user may be required to pre-supply buffers would solve the broad computer science problem. It would not solve the security vulnerability of enabling expedited handling of a resource drain caused by a single malfunctioning or malicious client. Failure to address this issue would make DA and iSER proposed standards that *fail* to comply with the security requirements of the protocol they claim to be utilizing. Obviously the IETF cannot hand-inspect applications to ensure they comply with ULP requirements, but at the minimum other IETF drafts should be compliant with IETF requirements. --Apple-Mail-5-612974498-- --===============1932168219== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline Content-Transfer-Encoding: 7bit _______________________________________________ Ips mailing list Ips@ietf.org https://www1.ietf.org/mailman/listinfo/ips --===============1932168219==-- From ips-bounces@ietf.org Wed Oct 6 03:55:53 2004 Received: from ietf-mx.ietf.org (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id DAA29251 for ; Wed, 6 Oct 2004 03:55:53 -0400 (EDT) Received: from megatron.ietf.org ([132.151.6.71]) by ietf-mx.ietf.org with esmtp (Exim 4.33) id 1CF6nt-0006Yt-OP for ips-web-archive@ietf.org; Wed, 06 Oct 2004 04:05:42 -0400 Received: from localhost.localdomain ([127.0.0.1] helo=megatron.ietf.org) by megatron.ietf.org with esmtp (Exim 4.32) id 1CF6cn-0006fH-JK; Wed, 06 Oct 2004 03:54:13 -0400 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by megatron.ietf.org with esmtp (Exim 4.32) id 1CF6VE-0005Jr-Np for ips@megatron.ietf.org; Wed, 06 Oct 2004 03:46:25 -0400 Received: from ietf-mx.ietf.org (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id DAA28575 for ; Wed, 6 Oct 2004 03:46:22 -0400 (EDT) Received: from e33.co.us.ibm.com ([32.97.110.131]) by ietf-mx.ietf.org with esmtp (Exim 4.33) id 1CF6ef-0005ZG-RO for ips@ietf.org; Wed, 06 Oct 2004 03:56:10 -0400 Received: from westrelay04.boulder.ibm.com (westrelay04.boulder.ibm.com [9.17.193.32]) by e33.co.us.ibm.com (8.12.10/8.12.9) with ESMTP id i967jo6A660056; Wed, 6 Oct 2004 03:45:50 -0400 Received: from d03nm115.boulder.ibm.com (d03av02.boulder.ibm.com [9.17.195.168]) by westrelay04.boulder.ibm.com (8.12.10/NCO/VER6.6) with ESMTP id i967jnds154098; Wed, 6 Oct 2004 01:45:49 -0600 In-Reply-To: <381D9C18-1744-11D9-90FD-000393814604@asomi.com> To: Caitlin Bestler MIME-Version: 1.0 Subject: Re: [Ips] DA+iSER Flow Control, Limiting number of oustanding cmdsattarget X-Mailer: Lotus Notes Release 6.0.2CF1 June 9, 2003 From: John Hufferd Message-ID: Date: Wed, 6 Oct 2004 00:45:43 -0700 X-MIMETrack: Serialize by Router on D03NM115/03/M/IBM(Release 6.51HF338 | June 21, 2004) at 10/06/2004 01:45:49, Serialize complete at 10/06/2004 01:45:49 X-Spam-Score: 0.0 (/) X-Scan-Signature: bc102ac530ba955ef81f1f75b8bebe44 Cc: ips@ietf.org X-BeenThere: ips@ietf.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: IP Storage List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Content-Type: multipart/mixed; boundary="===============1521432823==" Sender: ips-bounces@ietf.org Errors-To: ips-bounces@ietf.org X-Spam-Score: 0.0 (/) X-Scan-Signature: 4d9ae72af46718088458d214998cc683 This is a multipart message in MIME format. --===============1521432823== Content-Type: multipart/alternative; boundary="=_alternative 002A902688256F25_=" This is a multipart message in MIME format. --=_alternative 002A902688256F25_= Content-Type: text/plain; charset="US-ASCII" I guess I still do not see the security exposure for a kernel device driver that can inspect the buffers within nano seconds of the data arrival and determine if the packet is valid, and if an invalid set of buffers is received, terminating the connection. This puts the security issue right on the shoulders of the iSCSI device driver that can tell that the buffers are invalid. I do not see the issue. A run away connection that invalidly uses up all the bandwidth is just not a real problem if the buffers are inspected as they come in by the iSCSI device driver, and the connection dropped when an error is detected. . . John L. Hufferd Senior Technical Staff Member (STSM) IBM/System Group, San Jose CA Main Office: (408) 256-0403, Tie: 276-0403, eFax: (408) 904-4688 Alt Office: (408) 997-6136, Cell: (408) 499-9702 Internet Address: hufferd@us.ibm.com Caitlin Bestler 10/05/2004 08:02 PM To John Hufferd/San Jose/IBM@IBMUS cc ips@ietf.org Subject Re: [Ips] DA+iSER Flow Control, Limiting number of oustanding cmdsattarget On Oct 6, 2004, at 12:20 AM, John Hufferd wrote: > > Caitlin, > I guess the problem is that we are looking at a completely different > model of what is going on with the > iSCSI/iSER application. I do not think that we defined iSCSI/iSER > with the thought that they were somehow > disjoint models. In fact I believe that the name iSCSI/iSER kind of > describes how we anticipated the protocol > would be used. > But that is how the DataMover Architecture is specified. It is a clean layer. Responsibilities must be separated in specifications, even if implementations choose to be integrated. Isn't that fundamental to the process of setting standards? > > > Therefore, the Architecture was defined with the idea that iSCSI & > iSER were "joined at the hip" (sort of speak). > If an implementation chooses to separate the iSER from the iSCSI, and > if the remaining iSCSI Device Driver is not > capable of seeing its buffers fast enough to stop the connection when > it goes beyond the iSCSI flow control, then > that is simply a bug in the implementation. Even with that type of an > implementation the iSCSI Kernel mode Device > Driver should be able to inspect the Buffers within nano seconds of > the data arrival. Again, I do not see the problem. > The pre-existing iWARP interfaces are *not* "joined at the hip" with their users. One of the goals is to use enable iSCSI to be implemented over the general purpose iWARP service. Doing that requires The Upper Layer Protocol to provide for flow control of untagged messages, and to protect against attacks of resources shared across session lines. The pre-existing interfaces are *not* designed to push deliverable packets to the ULP on demand, but rather to generate completions of pre-posted receive operations, which the ULP consumes at its own pace. Further, each connection has a credit limit, violation of which will result at the minimum in an alarm being raised allowing for prompt termination of a runaway or malicious client. If the iSCSI layer is *not* using the per-connection limit, and is using a shared resource to service multiple clients that have no basis for shared trust it is violating the RDMAP/RDDP security requirements for ULPs. So in order to comply with the Security *requirements* for using iWARP, iSER MUST have sufficient information to set per-connection limits on untagged messages. It is also basic theory. You cannot provide a reliable transport service (in the absence of a lower layer error) unless you can block sends or push receives. The DataMover architecture fails to define this, but does require it to be reliable. > Now, just so you understand why I object to the issue of additional > flow control on top of iSCSI, > you need to reference what has to be done with Credit protocols like > FC which need to have extra > HW boundary boxes to "spoof" the credits in order to have the protocol > go across very large distances. > iSCSI was design to travel over large distances, without such > "spoofing" techniques, and I object t > additional Flow control, especially since it is not obvious that it > is a hard requirement and also since I > do not see the case that a Storage Controller needs "Beyond iSCSI" > Flow Control. Absolutely no new wire protocol is needed. The only thing that is required is to *document* the an allowance for the packets that iSCSI currently does not flow control. iSCSI is relying on the TCP bottleneck to flow control these. When TCP buffer credits have been exhausted the sender is prevented from sending more data without breaking the connection. The DataMover Architecture needs to be defined so it works over transports that do not provide this bottleneck "feature". That includes iSER, and it would include other possible DataMover architectures. Stating that the the DataMover architecture user may be required to pre-supply buffers would solve the broad computer science problem. It would not solve the security vulnerability of enabling expedited handling of a resource drain caused by a single malfunctioning or malicious client. Failure to address this issue would make DA and iSER proposed standards that *fail* to comply with the security requirements of the protocol they claim to be utilizing. Obviously the IETF cannot hand-inspect applications to ensure they comply with ULP requirements, but at the minimum other IETF drafts should be compliant with IETF requirements. --=_alternative 002A902688256F25_= Content-Type: text/html; charset="US-ASCII"
I guess I still do not see the security exposure for a kernel device driver that can inspect the buffers within nano seconds of the data arrival and determine if the packet is valid, and if an invalid set of buffers is received, terminating the connection.  This puts the security issue right on the shoulders of the iSCSI device driver that can tell that the buffers are invalid.  I do not see the issue.  A run away connection that invalidly uses up all the bandwidth is just not a real problem if the buffers are inspected as they come in by the iSCSI device driver, and the connection dropped when an error is detected.
.
.
John L. Hufferd
Senior Technical Staff Member (STSM)
IBM/System Group, San Jose CA
Main Office: (408) 256-0403, Tie: 276-0403, eFax: (408) 904-4688
Alt Office: (408) 997-6136, Cell: (408) 499-9702
Internet Address: hufferd@us.ibm.com


Caitlin Bestler <cait@asomi.com>

10/05/2004 08:02 PM

To
John Hufferd/San Jose/IBM@IBMUS
cc
ips@ietf.org
Subject
Re: [Ips] DA+iSER Flow Control, Limiting number of oustanding cmdsattarget






On Oct 6, 2004, at 12:20 AM, John Hufferd wrote:

>
> Caitlin,
> I guess the problem is that we are looking at a completely different
> model of what is going on with the
> iSCSI/iSER application.  I do not think that we defined iSCSI/iSER
> with the thought that they were somehow
> disjoint models.  In fact I believe that the name iSCSI/iSER kind of
> describes how we anticipated the protocol
> would be used.
>

But that is how the DataMover Architecture is specified.  It is a clean
layer.
Responsibilities must be separated in specifications, even if
implementations
choose to be integrated.  Isn't that fundamental to the process of
setting standards?
>
>
> Therefore, the Architecture was defined with the idea that iSCSI &
> iSER were "joined at the hip" (sort of speak).
> If an implementation chooses to separate the iSER from the iSCSI, and
> if the remaining iSCSI Device Driver is not
> capable of seeing its buffers fast enough to stop the connection when
> it goes beyond the iSCSI flow control, then
> that is simply a bug in the implementation.  Even with that type of an
> implementation the iSCSI Kernel mode Device
> Driver should be able to inspect the Buffers within nano seconds of
> the data arrival.  Again, I do not see the problem.
>

The pre-existing iWARP interfaces are *not* "joined at the hip" with
their users. One of the goals is
to use enable iSCSI to be implemented over the general purpose iWARP
service.  Doing that
requires The Upper Layer Protocol to provide for flow control of
untagged messages, and to
protect against attacks of resources shared across session lines.

The pre-existing interfaces are *not* designed to push deliverable
packets to the ULP on
demand, but rather to generate completions of pre-posted receive
operations, which the
ULP consumes at its own pace.  Further, each connection has a credit
limit, violation of
which will result at the minimum in an alarm being raised allowing for
prompt termination
of a runaway or malicious client.

If the iSCSI layer is *not* using the per-connection limit, and is
using a shared resource
to service multiple clients that have no basis for shared trust it is
violating the RDMAP/RDDP
security requirements for ULPs.

So in order to comply with the Security *requirements* for using iWARP,
iSER MUST have
sufficient information to set per-connection limits on untagged
messages.

It is also basic theory.  You cannot provide a reliable transport
service (in the absence
of a lower layer error) unless you can block sends or push receives.  
The DataMover
architecture fails to define this, but does require it to be reliable.


> Now, just so you understand why I object to the issue of additional
> flow control on top of iSCSI,
> you need to reference what has to be done with Credit protocols like
> FC which need to have extra
> HW boundary boxes to "spoof" the credits in order to have the protocol
> go across very large distances.
>  iSCSI was design to travel over large distances, without such
> "spoofing" techniques, and I object t
>  additional Flow control, especially since it is not obvious that it
> is a hard requirement and also since I
> do not see the case that a Storage Controller needs "Beyond iSCSI"
> Flow Control.  

Absolutely no new wire protocol is needed.  The only thing that is
required
is to *document* the an allowance for the packets that iSCSI currently
does not flow control.

iSCSI is relying on the TCP bottleneck to flow control these.  When TCP
buffer credits have been exhausted the sender is prevented from sending
more data without breaking the connection. The DataMover Architecture
needs to be defined so it works over transports that do not provide this
bottleneck "feature".  That includes iSER, and it would include other
possible DataMover architectures.

Stating that the the DataMover architecture user may be required
to pre-supply buffers would solve the broad computer science
problem.  It would not solve the security vulnerability of enabling
expedited handling of a resource drain caused by a single
malfunctioning or malicious client.

Failure to address this issue would make DA and iSER proposed
standards that *fail* to comply with the security requirements of
the protocol they claim to be utilizing.  Obviously the IETF cannot
hand-inspect applications to ensure they comply with ULP
requirements, but at the minimum other IETF drafts should
be compliant with IETF requirements.
--=_alternative 002A902688256F25_=-- --===============1521432823== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline Content-Transfer-Encoding: 7bit _______________________________________________ Ips mailing list Ips@ietf.org https://www1.ietf.org/mailman/listinfo/ips --===============1521432823==-- From ips-bounces@ietf.org Wed Oct 6 04:52:11 2004 Received: from ietf-mx.ietf.org (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id EAA03179 for ; Wed, 6 Oct 2004 04:52:10 -0400 (EDT) Received: from megatron.ietf.org ([132.151.6.71]) by ietf-mx.ietf.org with esmtp (Exim 4.33) id 1CF7gN-0001mH-Qy for ips-web-archive@ietf.org; Wed, 06 Oct 2004 05:02:00 -0400 Received: from localhost.localdomain ([127.0.0.1] helo=megatron.ietf.org) by megatron.ietf.org with esmtp (Exim 4.32) id 1CF7Lm-0006ay-Kp; Wed, 06 Oct 2004 04:40:42 -0400 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by megatron.ietf.org with esmtp (Exim 4.32) id 1CF7Dr-0005Bs-6h for ips@megatron.ietf.org; Wed, 06 Oct 2004 04:32:31 -0400 Received: from ietf-mx.ietf.org (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id EAA01563 for ; Wed, 6 Oct 2004 04:32:28 -0400 (EDT) Received: from mtagate2.de.ibm.com ([195.212.29.151]) by ietf-mx.ietf.org with esmtp (Exim 4.33) id 1CF7NI-0000Ne-Iz for ips@ietf.org; Wed, 06 Oct 2004 04:42:17 -0400 Received: from d12nrmr1507.megacenter.de.ibm.com (d12nrmr1507.megacenter.de.ibm.com [9.149.167.1]) by mtagate2.de.ibm.com (8.12.10/8.12.10) with ESMTP id i968VoFW147186; Wed, 6 Oct 2004 08:31:50 GMT Received: from d12ml102.megacenter.de.ibm.com (d12av02.megacenter.de.ibm.com [9.149.165.228]) by d12nrmr1507.megacenter.de.ibm.com (8.12.10/NCO/VER6.6) with ESMTP id i968VkiS210722; Wed, 6 Oct 2004 10:31:50 +0200 In-Reply-To: <2A8D600C919D2D41A3DAA2D2FEC71714735434@homer.qlogic.org> To: "Dean Scoville" Subject: RE: [Ips] resending Data-in with S bit set MIME-Version: 1.0 X-Mailer: Lotus Notes Build V70_M2_07222004 Beta 2NP July 22, 2004 From: Julian Satran Message-ID: Date: Wed, 6 Oct 2004 10:31:45 +0200 X-MIMETrack: Serialize by Router on D12ML102/12/M/IBM(Release 6.5.1| March 5, 2004) at 06/10/2004 10:31:49, Serialize complete at 06/10/2004 10:31:49 X-Spam-Score: 0.9 (/) X-Scan-Signature: af2aae76ea468dc53420d9ba52ca6045 Cc: ips@ietf.org, Eddy Quicksall X-BeenThere: ips@ietf.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: IP Storage List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Content-Type: multipart/mixed; boundary="===============0907518050==" Sender: ips-bounces@ietf.org Errors-To: ips-bounces@ietf.org X-Spam-Score: 0.9 (/) X-Scan-Signature: 64b72a8e61417554b4b727cb14e7034d This is a multipart message in MIME format. --===============0907518050== Content-Type: multipart/alternative; boundary="=_alternative 002D5604C2256F25_=" This is a multipart message in MIME format. --=_alternative 002D5604C2256F25_= Content-Type: text/plain; charset="US-ASCII" Dean & all, Sorry - the spec is not ambiguous and I was wrong in my previous answer (I just overlooked the StatSN field in Data-In). The target should (following 10.16) re-issue the exact copy of the Data-In PDU that included the status. Julo "Dean Scoville" 06/10/04 00:49 To "Eddy Quicksall" , Julian Satran/Haifa/IBM@IBMIL cc Subject RE: [Ips] resending Data-in with S bit set I'd say the RFC is ambiguous, because it sounds like it is now being interpreted to say that numbered responses include only SCSI Response PDUs, even though I thought the statements in sections 3.2 and 6.1.4.2 indicated that numbered responses include any PDUs carrying status... including Data-In PDUs with the S-bit set. Does that mean the initiator shouldn't be able to SNACK for a Reject PDU or a NOP-PDU, since they aren't SCSI Response PDUs either? Dean -----Original Message----- From: Eddy Quicksall [mailto:eddy_quicksall_iVivity_iSCSI@comcast.net] Sent: Tuesday, October 05, 2004 3:44 PM To: Dean Scoville; Julian Satran Cc: ips@ietf.org Subject: Re: [Ips] resending Data-in with S bit set Yes, when I said "Status PDUs", I used shorthand and meant SCSI Response PDUs". What you point out is exactly the reason I wanted to know the intent and if I understood the spec correctly. I did not mean to say I agree or disagree with the intent. Since the spec is finished, I think it is too late to argue for a change. It looks like the initiator just has to be a little smart when it needs all PDUs that carry status. Eddy ----- Original Message ----- From: Dean Scoville To: Eddy Quicksall ; Julian Satran Cc: ips@ietf.org Sent: Tuesday, October 05, 2004 6:22 PM Subject: RE: [Ips] resending Data-in with S bit set When you say only "Status" PDUs, do you mean only "SCSI Response" PDUs, and that the target should remain silent about any phase collapsed Data-In PDUs? That doesn't make sense to me. If the initiator has 100 commands outstanding and issues a Status SNACK because a received StatSN indicates that some PDUs are missing, it shouldn't have to issue Data SNACKs for all of the outstanding commands to see if one of them perhaps was using phase collapse and happened to have the missing StatusSN. Is that really what is intended? The last sentence in Section 3.2 defines a numbered response as: "Also, unless otherwise specified, status, response, or numbered response have the same meaning." And Section 6.1.4.2 states: "Lost iSCSI numbered Response. It is recognized by either identifying a data digest error on a Response PDU or a Data-In PDU carrying the status, or by receiving a Response PDU with a higher StatSN than expected." The above two RFC statements would lead me to believe that numbered responses include Data-In PDUs, and that the numbered responses referred to in Section 10.16.1 in reference to Status SNACKs should also include Data-In retransmissions. Dean -----Original Message----- From: Eddy Quicksall [mailto:eddy_quicksall_iVivity_iSCSI@comcast.net] Sent: Tuesday, October 05, 2004 3:09 PM To: Dean Scoville; Julian Satran Cc: ips@ietf.org Subject: Re: [Ips] resending Data-in with S bit set Maybe my question was misunderstood. To say it another way: Is it true that a Status SNACK only solicits Status PDUs and that it does not solicit Data-In PDUs which have the S bit set? I interpret Julian's answer to this as "yes, that is true". Eddy ----- Original Message ----- From: Dean Scoville To: Julian Satran ; Eddy Quicksall Cc: ips@ietf.org Sent: Tuesday, October 05, 2004 5:54 PM Subject: RE: [Ips] resending Data-in with S bit set Julian, Section 10.16 states the following about PDUs transmitted as a result of a SNACK: "The numbered-response(s) or R2T(s), requested by a SNACK, MUST be delivered as exact replicas of the ones that the target transmitted originally except for the fields ExpCmdSN, MaxCmdSN, and ExpDataSN, which MUST carry the current values. R2T(s) requested by SNACK MUST also carry the current values and except for resegmentation (see Section 10.16.3 Resegmentation)." Section 10.16.1 describes Status and Data/R2T SNACK types as: 0-Data/R2T SNACK - requesting retransmission of one or more Data-In or R2T PDUs. 1-Status SNACK - requesting retransmission of one or more numbered responses. It seems to me that if the target receives a Status SNACK for status sent on a Data-In PDU due to phase collapse, the target should send an exact replica of the Data-In PDU. Similarly, if the target receives a Data SNACK, it should retransmit the Data-In PDU with the S bit set, since it is a retransmission. I may have misunderstood your recent posting, but I thought it said the target should generate a totally new SCSI Response PDU in response to the Status SNACK rather than retransmitting the original Data-In PDU. And that it should send a Data-In PDU without the S-bit set in the case of a Data SNACK. Both of these behaviors seem to violate what is stated in Sections 10.16 and 10.16.1 of the iSCSI RFC, as quoted above. I would appreciate clarification on the target's expected response in these situations. thanks, Dean Scoville -----Original Message----- From: ips-bounces@ietf.org [mailto:ips-bounces@ietf.org]On Behalf Of Julian Satran Sent: Monday, October 04, 2004 7:43 AM To: Eddy Quicksall Cc: ips@ietf.org Subject: Re: [Ips] resending Data-in with S bit set Eddy, This is correct. The data-in PDUs don't have to be sent. The initiator has to request the data-in PDUs explicitly and MUST do so before it acks the status (that implicitly acks all the data and deletes any record of the command at target). Julo ips-bounces@ietf.org wrote on 04/10/2004 15:20:56: > Can you confirm that the intent of 10.16.6. is as follows? > > If the Initiator has requested for a Status SNACK with BegRun=0 and > RunLength=0 (i.e, all unacknowledged status pdus) and the target has > some Data-In PDUs carrying *status* that have not been acknowledged, > the target will *not* resend those Data-In PDUs carrying the status. > > The unacknowledged queue looks like > > response(statSN=5) ---> response(statSN=6) -----> Data-In w/S > bit(statSN=7) ----> response(statSN=8) > > The target will resend pdus with statSN 5, 6 and 8. The Data-In with > statSN 7 will not be resent even though it sends status. > > Eddy > _______________________________________________ > Ips mailing list > Ips@ietf.org > https://www1.ietf.org/mailman/listinfo/ips _______________________________________________ Ips mailing list Ips@ietf.org https://www1.ietf.org/mailman/listinfo/ips --=_alternative 002D5604C2256F25_= Content-Type: text/html; charset="US-ASCII"
Dean & all,


Sorry - the spec is not ambiguous and I was wrong in my previous answer (I just overlooked the StatSN field in Data-In).
The target should (following 10.16) re-issue the exact copy of the Data-In PDU that included the status.

Julo


"Dean Scoville" <dean.scoville@qlogic.com>

06/10/04 00:49

To
"Eddy Quicksall" <eddy_quicksall_iVivity_iSCSI@comcast.net>, Julian Satran/Haifa/IBM@IBMIL
cc
<ips@ietf.org>
Subject
RE: [Ips] resending Data-in with S bit set





I'd say the RFC is ambiguous, because it sounds like it is now being interpreted to say that
numbered responses include only SCSI Response PDUs, even though I thought the
statements in sections 3.2 and 6.1.4.2 indicated that numbered responses include any PDUs
carrying status... including Data-In PDUs with the S-bit set. Does that mean the initiator
shouldn't be able to SNACK for a Reject PDU or a NOP-PDU, since they aren't SCSI Response
PDUs either?
 
Dean
-----Original Message-----
From: Eddy Quicksall [mailto:eddy_quicksall_iVivity_iSCSI@comcast.net]
Sent: Tuesday, October 05, 2004 3:44 PM
To: Dean Scoville; Julian Satran
Cc: ips@ietf.org
Subject: Re: [Ips] resending Data-in with S bit set
Yes, when I said "Status PDUs", I used shorthand and meant SCSI Response PDUs".
 
What you point out is exactly the reason I wanted to know the intent and if I understood the spec correctly. I did not mean to say I agree or disagree with the intent.
 
Since the spec is finished, I think it is too late to argue for a change. It looks like the initiator just has to be a little smart when it needs all PDUs that carry status.
 
Eddy
----- Original Message -----
From: Dean Scoville
To: Eddy Quicksall ; Julian Satran
Cc: ips@ietf.org
Sent: Tuesday, October 05, 2004 6:22 PM
Subject: RE: [Ips] resending Data-in with S bit set

When you say only "Status" PDUs, do you mean only "SCSI Response" PDUs,
and that the target should remain silent about any phase collapsed Data-In PDUs?
That doesn't make sense to me. If the initiator has 100 commands outstanding and
issues a Status SNACK because a received StatSN indicates that some PDUs are
missing, it shouldn't have to issue Data SNACKs for all of the outstanding commands
to see if one of them perhaps was using phase collapse and happened to have the
missing StatusSN. Is that really what is intended?
 
The last sentence in Section 3.2 defines a numbered response as:
 
    "Also, unless otherwise specified, status, response, or numbered response have the same meaning."
 
And Section 6.1.4.2 states:
 
    "Lost iSCSI numbered Response. It is recognized by either identifying a data digest
     error on a Response PDU or a Data-In PDU carrying the status, or by receiving a
     Response PDU with a higher StatSN than expected."
 
The above two RFC statements would lead me to believe that numbered responses include
Data-In PDUs, and that the numbered responses referred to in Section 10.16.1 in
reference to Status SNACKs should also include Data-In retransmissions.
 
Dean
-----Original Message-----
From: Eddy Quicksall [mailto:eddy_quicksall_iVivity_iSCSI@comcast.net]
Sent: Tuesday, October 05, 2004 3:09 PM
To: Dean Scoville; Julian Satran
Cc: ips@ietf.org
Subject: Re: [Ips] resending Data-in with S bit set
Maybe my question was misunderstood. To say it another way:
 
Is it true that a Status SNACK only solicits Status PDUs and that it does not solicit Data-In PDUs which have the S bit set?
 
I interpret Julian's answer to this as "yes, that is true".
 
Eddy
----- Original Message -----
From: Dean Scoville
To: Julian Satran ; Eddy Quicksall
Cc: ips@ietf.org
Sent: Tuesday, October 05, 2004 5:54 PM
Subject: RE: [Ips] resending Data-in with S bit set

Julian,
 
Section 10.16 states the following about PDUs transmitted as a result of a SNACK:
 
   "The numbered-response(s) or R2T(s), requested by a SNACK, MUST be
    delivered as exact replicas of the ones that the target transmitted originally
    except for the fields ExpCmdSN, MaxCmdSN, and ExpDataSN, which MUST
    carry the current values. R2T(s) requested by SNACK MUST  also carry the
    current values and except for resegmentation (see Section 10.16.3 Resegmentation)."
 
Section 10.16.1 describes Status and Data/R2T SNACK types as:
 
    0-Data/R2T SNACK - requesting retransmission of one or more Data-In or R2T PDUs.
    1-Status SNACK - requesting retransmission of one or more numbered responses.
 
It seems to me that if the target receives a Status SNACK for status sent on a Data-In
PDU due to phase collapse, the target should send an exact replica of the Data-In PDU.
Similarly, if the target receives a Data SNACK, it should retransmit the Data-In PDU with
the S bit set, since it is a retransmission.
 
I may have misunderstood your recent posting, but I thought it said the target should
generate a totally new SCSI Response PDU in response to the Status SNACK rather
than retransmitting the original Data-In PDU. And that it should send a Data-In PDU
without the S-bit set in the case of a Data SNACK. Both of these behaviors seem to
violate what is stated in Sections 10.16 and 10.16.1 of the iSCSI RFC, as quoted above.
 
I would appreciate clarification on the target's expected response in these situations.
 
thanks,
 
Dean Scoville
-----Original Message-----
From: ips-bounces@ietf.org [mailto:ips-bounces@ietf.org]On Behalf Of Julian Satran
Sent: Monday, October 04, 2004 7:43 AM
To: Eddy Quicksall
Cc: ips@ietf.org
Subject: Re: [Ips] resending Data-in with S bit set

Eddy,

This is correct.
The data-in PDUs don't have to be sent.
The initiator has to request the data-in PDUs explicitly and MUST do so before it acks the status (that implicitly acks all the data and deletes any record of the command at target).

Julo

ips-bounces@ietf.org wrote on 04/10/2004 15:20:56:

> Can you confirm that the intent of 10.16.6. is as follows?
>  
> If the Initiator has requested for a Status SNACK with BegRun=0 and
> RunLength=0 (i.e, all unacknowledged status pdus) and the target has
> some Data-In PDUs carrying *status* that have not been acknowledged,
> the target will *not* resend those Data-In PDUs carrying the status.
>  
> The unacknowledged queue looks like
>  
>    response(statSN=5) ---> response(statSN=6) -----> Data-In w/S
> bit(statSN=7) ----> response(statSN=8)
>  
> The target will resend pdus with statSN 5, 6 and 8. The Data-In with
> statSN 7 will not be resent even though it sends status.
>  
> Eddy
>  _______________________________________________
> Ips mailing list
> Ips@ietf.org
> https://www1.ietf.org/mailman/listinfo/ips

_______________________________________________
Ips mailing list
Ips@ietf.org
https://www1.ietf.org/mailman/listinfo/ips

--=_alternative 002D5604C2256F25_=-- --===============0907518050== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline Content-Transfer-Encoding: 7bit _______________________________________________ Ips mailing list Ips@ietf.org https://www1.ietf.org/mailman/listinfo/ips --===============0907518050==-- From ips-bounces@ietf.org Wed Oct 6 04:58:44 2004 Received: from ietf-mx.ietf.org (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id EAA03779 for ; Wed, 6 Oct 2004 04:58:44 -0400 (EDT) Received: from megatron.ietf.org ([132.151.6.71]) by ietf-mx.ietf.org with esmtp (Exim 4.33) id 1CF7mk-00022b-4m for ips-web-archive@ietf.org; Wed, 06 Oct 2004 05:08:34 -0400 Received: from localhost.localdomain ([127.0.0.1] helo=megatron.ietf.org) by megatron.ietf.org with esmtp (Exim 4.32) id 1CF7ZD-0000Fu-ET; Wed, 06 Oct 2004 04:54:35 -0400 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by megatron.ietf.org with esmtp (Exim 4.32) id 1CF7Nu-0006ut-OK for ips@megatron.ietf.org; Wed, 06 Oct 2004 04:42:54 -0400 Received: from ietf-mx.ietf.org (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id EAA02461 for ; Wed, 6 Oct 2004 04:42:52 -0400 (EDT) Received: from system60.simplicato.com ([207.99.47.60] helo=host52a.simplicato.com) by ietf-mx.ietf.org with esmtp (Exim 4.33) id 1CF7XN-00012q-23 for ips@ietf.org; Wed, 06 Oct 2004 04:52:41 -0400 Received: from localhost (localhost.simplicato.com [127.0.0.1]) by host52a.simplicato.com (Postfix) with ESMTP id 8DE2211E04C; Wed, 6 Oct 2004 04:42:51 -0400 (EDT) Received: from host52a.simplicato.com ([127.0.0.1]) by localhost (host52a.simplicato.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 73421-01; Wed, 6 Oct 2004 04:42:49 -0400 (EDT) Received: from mail.asomi.com (localhost.simplicato.com [127.0.0.1]) by host52a.simplicato.com (Postfix) with SMTP id 2C48111E0AF; Wed, 6 Oct 2004 04:42:49 -0400 (EDT) Received: from 62.90.160.198 (proxying for 62.90.160.198) (SquirrelMail authenticated user cait@asomi.com) by mail52a.simplicato.com with HTTP; Wed, 6 Oct 2004 03:42:49 -0500 (CDT) Message-ID: <26668.62.90.160.198.1097052169.squirrel@mail52a.simplicato.com> In-Reply-To: References: <381D9C18-1744-11D9-90FD-000393814604@asomi.com> Date: Wed, 6 Oct 2004 03:42:49 -0500 (CDT) Subject: Re: [Ips] DA+iSER Flow Control, Limiting number of oustanding cmdsattarget From: "Caitlin Bestler" To: "John Hufferd" User-Agent: SquirrelMail/1.4.2 MIME-Version: 1.0 Content-Type: text/plain;charset=iso-8859-1 X-Priority: 3 Importance: Normal X-Virus-Scanned: by amavisd-new at simplicato.com Content-Transfer-Encoding: quoted-printable X-Spam-Score: 0.8 (/) X-Scan-Signature: 41c17b4b16d1eedaa8395c26e9a251c4 Content-Transfer-Encoding: quoted-printable Cc: Caitlin Bestler , ips@ietf.org X-BeenThere: ips@ietf.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: IP Storage List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Sender: ips-bounces@ietf.org Errors-To: ips-bounces@ietf.org X-Spam-Score: 0.8 (/) X-Scan-Signature: 25620135586de10c627e3628c432b04a Content-Transfer-Encoding: quoted-printable John Hufferd said: > I guess I still do not see the security exposure for a kernel device > driver that can inspect the buffers within nano seconds of the data > arrival and determine if the packet is valid, and if an invalid set of > buffers is received, terminating the connection. This puts the securit= y > issue right on the shoulders of the iSCSI device driver that can tell t= hat > the buffers are invalid. I do not see the issue. A run away connectio= n > that invalidly uses up all the bandwidth is just not a real problem if = the > buffers are inspected as they come in by the iSCSI device driver, and t= he > connection dropped when an error is detected. > . > . That division of responsibility should certainly be allowed. It would require the iSER layer to reap completions almost instantly, and to pass all completed iSCSI PDUs to the iSCSI layer almost instantly. The iSCSI layer would have to keep up, period. It could do so by tossing packets and terminating iSCSI sessions. The current draft does not provide language that would justify that division of responsibilities. That should be fixed. I also believe, although I really don't care that strongly about it, that a properly designed wire protocol should not be dependent on a specific implementation strategy. That means that there MUST be a method by which it is impossible for a single malfunctionign and/or malicious client to drain shared resources and thereby cause a failure affecting other clients. Requiring the iSCSI layer to at least accept packets at line rate, or to refrain from using shared resources, would indeed accomplish that. But it unnecessarily constrains the range of possible implementations. Specifying a cap on the number of "extra" untagged messages, either as a ULP constant or a session parameter,would enable the use of per-connection safety mechanisms associated with SRQs. That would enable a wide range of possible implementations to use SRQs to implement iSCSI/iSER. As previously noted, pre-allocating a fully committed buffer for each possible untagged message on each connection is not a prudent method of allocating memory for an iSCSI daemon. Enabling the widest possible range of implementations to use SRQs when implementing iSER is a desirable goal. Implementations which are indeed capable of having the iSCSI layer accept iSCSI PDUs at line rate would never have to use this information. At worst they are exchanging one extra piece of data during session setup. That is much to ask if it enables a wider range of possible implementations. One of the benefits of RDMA is that when it is used properly in can enable daemons to be implemented in user-space that previously could only have been attempted in the kernel. That is a desirable feature. iSER should not be designed on the assumption that it is unobtainable. -- Caitlin Bestler http://asomi.com/ _______________________________________________ Ips mailing list Ips@ietf.org https://www1.ietf.org/mailman/listinfo/ips From ips-bounces@ietf.org Wed Oct 6 19:32:55 2004 Received: from ietf-mx.ietf.org (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id TAA29345 for ; Wed, 6 Oct 2004 19:32:55 -0400 (EDT) Received: from megatron.ietf.org ([132.151.6.71]) by ietf-mx.ietf.org with esmtp (Exim 4.33) id 1CFLQr-0003cg-QG for ips-web-archive@ietf.org; Wed, 06 Oct 2004 19:42:54 -0400 Received: from localhost.localdomain ([127.0.0.1] helo=megatron.ietf.org) by megatron.ietf.org with esmtp (Exim 4.32) id 1CFLDQ-000315-Le; Wed, 06 Oct 2004 19:29:00 -0400 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by megatron.ietf.org with esmtp (Exim 4.32) id 1CFLCl-0002pF-KQ for ips@megatron.ietf.org; Wed, 06 Oct 2004 19:28:19 -0400 Received: from ietf-mx.ietf.org (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id TAA29103 for ; Wed, 6 Oct 2004 19:28:15 -0400 (EDT) Received: from e31.co.us.ibm.com ([32.97.110.129]) by ietf-mx.ietf.org with esmtp (Exim 4.33) id 1CFLML-0003Ee-G3 for ips@ietf.org; Wed, 06 Oct 2004 19:38:14 -0400 Received: from westrelay02.boulder.ibm.com (westrelay02.boulder.ibm.com [9.17.195.11]) by e31.co.us.ibm.com (8.12.10/8.12.9) with ESMTP id i96NRKaI431372; Wed, 6 Oct 2004 19:27:20 -0400 Received: from d03nm115.boulder.ibm.com (d03av02.boulder.ibm.com [9.17.195.168]) by westrelay02.boulder.ibm.com (8.12.10/NCO/VER6.6) with ESMTP id i96NRE1L384592; Wed, 6 Oct 2004 17:27:19 -0600 In-Reply-To: <26668.62.90.160.198.1097052169.squirrel@mail52a.simplicato.com> To: "Caitlin Bestler" MIME-Version: 1.0 Subject: Re: [Ips] DA+iSER Flow Control, Limiting number of oustanding cmdsattarget X-Mailer: Lotus Notes Release 6.0.2CF1 June 9, 2003 From: John Hufferd Message-ID: Date: Wed, 6 Oct 2004 16:27:05 -0700 X-MIMETrack: Serialize by Router on D03NM115/03/M/IBM(Release 6.51HF338 | June 21, 2004) at 10/06/2004 17:27:19, Serialize complete at 10/06/2004 17:27:19 X-Spam-Score: 0.9 (/) X-Scan-Signature: 46ad68ada464411807db2a0edd5648ae Cc: ips@ietf.org X-BeenThere: ips@ietf.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: IP Storage List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Content-Type: multipart/mixed; boundary="===============0783652438==" Sender: ips-bounces@ietf.org Errors-To: ips-bounces@ietf.org X-Spam-Score: 0.0 (/) X-Scan-Signature: bdfdd9dd835c9bb499f7c92933fef080 This is a multipart message in MIME format. --===============0783652438== Content-Type: multipart/alternative; boundary="=_alternative 0080BF4088256F25_=" This is a multipart message in MIME format. --=_alternative 0080BF4088256F25_= Content-Type: text/plain; charset="US-ASCII" Caitlin, Perhaps we are talking past each other. My point has been and still is that I would not like to see additional flow control used with iSCSI/iSER that would prevent the "at distance" operations from being as efficient as current iSCSI. Now having said that, I believe that the point that Mike Ko said in his previous note is that all we need to do is to cap the number of special responses that are beyond the normal expected iSCSI windowing process. I think that is not really needed for the correct use of SRQ, and I believe you concurred on that point. However, I believe that it may be of value to someone that wants to implement iSCSI/iSER without SRQ, or perhaps even in your hypothetical generic Data Mover Service. So my interpretation of what Mike was saying is as follows (which if correct I can accept): 1. Each iSCSI/iSER side (peer) will declare to the other (for this discussion lets call this Keyword=Value pair "MaxOutstandingUnexpectedPDUs") a value that represents the number of messages that can occur beyond what is handled by the normal iSCSI Flow Control. The values may be either numeric. or "None" where None means no limit specified. I would even suggest that "None" should be the default. 2. From the Target side, this value (that it would declare), is the Max number of Outstanding Immediate Commands, and/or Outstanding SNACK commands that it could receive (SNACK should not be an important consideration, but the spec says that even if SNACKs are not needed, if SNACKs are issued by the Initiator they are passed on, so this value would cover SNACKs also). 3. From the Initiator side, this value (that it would declare), is the Max number of outstanding Async Messages and/or NOPs that it can receive from the Target. There would be no need to have any additional flow control, and the NON SRQ Target implementations could ensure that the number of buffers that they queue will be at least sufficient to handle the iSCSI Command Window, plus the number of Declared "MaxOutstandingUnexpectedPDUs". And the initiator would queue the number of buffers that will be sufficient to handle the iSCSI Command Window responses, plus the number of Declared "MaxOutstandingUnexpectedPDUs". Now, as I said before, I believe it is best handled by an SRQ approach that does not limit the flow of messages "at Distance", and that where ever possible the peers should operate with the value of "NONE" (no limits) and that is why I think it should be the default. The reason that this approach is acceptable to me is because the use of Immediate commands (and SNACKs) or the arrival of Async Message PDUs from the Target, should be rare enough to not inhibit the normal iSCSI flow control. However, we should have a caution stated somewhere, that this might have an important Error Recovery impacts when dealing with ERL=2 and we are in an at distance situation. That is, when a link fails (which may happen more than we like "at distance") the ERL=2 mode says that the Task Management (Immediate) Commands must reassign the Tasks to a surviving connection. It is very likely that we may see hundreds of commands that are subject to this action and since we have only a limited amount of time to make this switch, it could be that the distance and limit on Outstanding Immediate commands could lengthen this recovery to a point that the recovery time out is exceeded. (This would not be a problem with the SRQ design that I talked about previously.) Caitlin, with your acceptance that the No Limit SRQ approach that I specified, and with the above proposal, which I believe may cover your additional issues, I think we maybe close to closure on this. However, some of the folks on this reflector have told me that they prefer a resolution that requires the "Graceful handling" of RQ overruns by the RNIC. There are several approaches to do that, and quite frankly I prefer that over the above. But perhaps we need to see if we at least can agree on the above and then discuss the other options. . . John L. Hufferd Senior Technical Staff Member (STSM) IBM/System Group, San Jose CA Main Office: (408) 256-0403, Tie: 276-0403, eFax: (408) 904-4688 Alt Office: (408) 997-6136, Cell: (408) 499-9702 Internet Address: hufferd@us.ibm.com "Caitlin Bestler" 10/06/2004 01:42 AM To John Hufferd/San Jose/IBM@IBMUS cc "Caitlin Bestler" , ips@ietf.org Subject Re: [Ips] DA+iSER Flow Control, Limiting number of oustanding cmdsattarget John Hufferd said: > I guess I still do not see the security exposure for a kernel device > driver that can inspect the buffers within nano seconds of the data > arrival and determine if the packet is valid, and if an invalid set of > buffers is received, terminating the connection. This puts the security > issue right on the shoulders of the iSCSI device driver that can tell that > the buffers are invalid. I do not see the issue. A run away connection > that invalidly uses up all the bandwidth is just not a real problem if the > buffers are inspected as they come in by the iSCSI device driver, and the > connection dropped when an error is detected. > . > . That division of responsibility should certainly be allowed. It would require the iSER layer to reap completions almost instantly, and to pass all completed iSCSI PDUs to the iSCSI layer almost instantly. The iSCSI layer would have to keep up, period. It could do so by tossing packets and terminating iSCSI sessions. The current draft does not provide language that would justify that division of responsibilities. That should be fixed. I also believe, although I really don't care that strongly about it, that a properly designed wire protocol should not be dependent on a specific implementation strategy. That means that there MUST be a method by which it is impossible for a single malfunctionign and/or malicious client to drain shared resources and thereby cause a failure affecting other clients. Requiring the iSCSI layer to at least accept packets at line rate, or to refrain from using shared resources, would indeed accomplish that. But it unnecessarily constrains the range of possible implementations. Specifying a cap on the number of "extra" untagged messages, either as a ULP constant or a session parameter,would enable the use of per-connection safety mechanisms associated with SRQs. That would enable a wide range of possible implementations to use SRQs to implement iSCSI/iSER. As previously noted, pre-allocating a fully committed buffer for each possible untagged message on each connection is not a prudent method of allocating memory for an iSCSI daemon. Enabling the widest possible range of implementations to use SRQs when implementing iSER is a desirable goal. Implementations which are indeed capable of having the iSCSI layer accept iSCSI PDUs at line rate would never have to use this information. At worst they are exchanging one extra piece of data during session setup. That is much to ask if it enables a wider range of possible implementations. One of the benefits of RDMA is that when it is used properly in can enable daemons to be implemented in user-space that previously could only have been attempted in the kernel. That is a desirable feature. iSER should not be designed on the assumption that it is unobtainable. -- Caitlin Bestler http://asomi.com/ --=_alternative 0080BF4088256F25_= Content-Type: text/html; charset="US-ASCII"
Caitlin,
Perhaps we are talking past each other.  My point has been and still is that I would not like to see additional flow control used with iSCSI/iSER that would prevent the "at distance" operations from being as efficient as current iSCSI.  Now having said that, I believe that the point that Mike Ko said in his previous note is that all we need to do is to cap the number of special responses that are beyond the normal expected iSCSI windowing process.  I think that is not really needed for the correct use of SRQ, and I believe you concurred on that point.  However, I believe that it may be of value to someone that wants to implement iSCSI/iSER without SRQ, or perhaps even in your hypothetical generic Data Mover Service.  So my interpretation of what Mike was saying is as follows (which if correct I can accept):

1. Each iSCSI/iSER side (peer) will declare to the other (for this discussion lets call this Keyword=Value pair "MaxOutstandingUnexpectedPDUs") a value that represents the number of messages that can occur beyond what is handled by the normal iSCSI Flow Control.  The values may be either numeric. or "None" where None means no limit specified.  I would even suggest that "None" should be the default.
2. From the Target side, this value (that it would declare), is the Max number of Outstanding Immediate Commands, and/or Outstanding SNACK commands that it could receive (SNACK should not be an important consideration, but the spec says that even if SNACKs are not needed, if SNACKs are issued by the Initiator they are passed on, so this value would cover SNACKs also).
3. From the Initiator side, this value (that it would declare), is the Max number of outstanding Async Messages and/or NOPs that it can receive from the Target.

There would be no need to have any additional flow control, and the NON SRQ Target implementations could ensure that the number of buffers that they queue will be at least sufficient to handle the iSCSI Command Window, plus the number of Declared "MaxOutstandingUnexpectedPDUs".   And the initiator would queue the number of buffers that will be sufficient to handle the iSCSI Command Window responses, plus the number of Declared "MaxOutstandingUnexpectedPDUs".

Now, as I said before, I believe it is best handled by an SRQ approach that does not limit the flow of messages "at Distance", and that where ever possible the peers should operate with the value of "NONE" (no limits) and that is why I think it should be the default.  

The reason that this approach is acceptable to me is because the use of Immediate commands (and SNACKs) or the arrival of Async Message PDUs from the Target, should be rare enough to not inhibit the normal iSCSI flow control.  However, we should have a caution stated somewhere, that this might have an important Error Recovery impacts when dealing with ERL=2 and we are in an at distance situation.  That is, when a link fails (which may happen more than we like "at distance") the ERL=2 mode says that the Task Management (Immediate) Commands must reassign the Tasks to a surviving connection.  It is very likely that we may see hundreds of commands that are subject to this action and since we have only a limited amount of time to make this switch,  it could be that the distance and limit on Outstanding Immediate commands could lengthen this recovery to a point that the recovery time out is exceeded.  (This would not be a problem with the SRQ design that I talked about previously.)

Caitlin, with your acceptance that the No Limit SRQ approach that I specified, and with the above proposal, which I believe may cover your additional issues, I think we maybe close to closure on this.

However, some of the folks on this reflector have told me that they prefer a resolution that requires the "Graceful handling" of RQ overruns by the RNIC. There are several approaches to do that, and quite frankly I prefer that over the above.  But perhaps we need to see if we at least can agree on the above and then discuss the other options.
.
.
John L. Hufferd
Senior Technical Staff Member (STSM)
IBM/System Group, San Jose CA
Main Office: (408) 256-0403, Tie: 276-0403, eFax: (408) 904-4688
Alt Office: (408) 997-6136, Cell: (408) 499-9702
Internet Address: hufferd@us.ibm.com


"Caitlin Bestler" <cait@asomi.com>

10/06/2004 01:42 AM

To
John Hufferd/San Jose/IBM@IBMUS
cc
"Caitlin Bestler" <cait@asomi.com>, ips@ietf.org
Subject
Re: [Ips] DA+iSER Flow Control,      Limiting number of oustanding cmdsattarget






John Hufferd said:
> I guess I still do not see the security exposure for a kernel device
> driver that can inspect the buffers within nano seconds of the data
> arrival and determine if the packet is valid, and if an invalid set of
> buffers is received, terminating the connection.  This puts the security
> issue right on the shoulders of the iSCSI device driver that can tell that
> the buffers are invalid.  I do not see the issue.  A run away connection
> that invalidly uses up all the bandwidth is just not a real problem if the
> buffers are inspected as they come in by the iSCSI device driver, and the
> connection dropped when an error is detected.
> .
> .

That division of responsibility should certainly be allowed.
It would require the iSER layer to reap completions almost
instantly, and to pass all completed iSCSI PDUs to the iSCSI
layer almost instantly. The iSCSI layer would have to keep up,
period.  It could do so by tossing packets and terminating
iSCSI sessions.

The current draft does not provide language that would justify
that division of responsibilities.  That should be fixed.

I also believe, although I really don't care that strongly
about it, that a properly designed wire protocol should not
be dependent on a specific implementation strategy.

That means that there MUST be a method by which it is impossible
for a single malfunctionign and/or malicious client to drain
shared resources and thereby cause a failure affecting other
clients.  Requiring the iSCSI layer to at least accept packets
at line rate, or to refrain from using shared resources, would
indeed accomplish that.  But it unnecessarily constrains the
range of possible implementations.

Specifying a cap on the number of "extra" untagged messages,
either as a ULP constant or a session parameter,would enable
the use of per-connection safety mechanisms associated with
SRQs.  That would enable a wide range of possible implementations
to use SRQs to implement iSCSI/iSER.

As previously noted, pre-allocating a fully committed buffer
for each possible untagged message on each connection is not
a prudent method of allocating memory for an iSCSI daemon.
Enabling the widest possible range of implementations to
use SRQs when implementing iSER is a desirable goal.

Implementations which are indeed capable of having the iSCSI
layer accept iSCSI PDUs at line rate would never have to use
this information.  At worst they are exchanging one extra piece
of data during session setup.  That is much to ask if it enables
a wider range of possible implementations.

One of the benefits of RDMA is that when it is used properly
in can enable daemons to be implemented in user-space that
previously could only have been attempted in the kernel.
That is a desirable feature.  iSER should not be designed
on the assumption that it is unobtainable.


--
Caitlin Bestler
http://asomi.com/
--=_alternative 0080BF4088256F25_=-- --===============0783652438== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline Content-Transfer-Encoding: 7bit _______________________________________________ Ips mailing list Ips@ietf.org https://www1.ietf.org/mailman/listinfo/ips --===============0783652438==-- From ips-bounces@ietf.org Wed Oct 6 21:09:44 2004 Received: from ietf-mx.ietf.org (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id VAA06373 for ; Wed, 6 Oct 2004 21:09:44 -0400 (EDT) Received: from megatron.ietf.org ([132.151.6.71]) by ietf-mx.ietf.org with esmtp (Exim 4.33) id 1CFMwX-0001B5-Us for ips-web-archive@ietf.org; Wed, 06 Oct 2004 21:19:42 -0400 Received: from localhost.localdomain ([127.0.0.1] helo=megatron.ietf.org) by megatron.ietf.org with esmtp (Exim 4.32) id 1CFMgU-00044c-Ip; Wed, 06 Oct 2004 21:03:06 -0400 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by megatron.ietf.org with esmtp (Exim 4.32) id 1CFMek-0003c5-NW for ips@megatron.ietf.org; Wed, 06 Oct 2004 21:01:18 -0400 Received: from ietf-mx.ietf.org (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id VAA05662 for ; Wed, 6 Oct 2004 21:01:16 -0400 (EDT) Received: from atlrel8.hp.com ([156.153.255.206]) by ietf-mx.ietf.org with esmtp (Exim 4.33) id 1CFMoL-0000c7-Nu for ips@ietf.org; Wed, 06 Oct 2004 21:11:14 -0400 Received: from rosemail.rose.hp.com (rosemail.rose.hp.com [15.43.209.160]) by atlrel8.hp.com (Postfix) with ESMTP id 6077CA5CA for ; Wed, 6 Oct 2004 21:01:16 -0400 (EDT) Received: from [127.0.0.1] (pal1nai160042.nsr.hp.com [15.244.160.42]) by rosemail.rose.hp.com (Postfix) with ESMTP id 55D937FFE for ; Wed, 6 Oct 2004 18:01:15 -0700 (PDT) Message-ID: <41649554.60700@rose.hp.com> Date: Wed, 06 Oct 2004 18:01:08 -0700 From: "Mallikarjun C." User-Agent: Mozilla Thunderbird 0.7.3 (Windows/20040803) X-Accept-Language: en-us, en MIME-Version: 1.0 To: ips@ietf.org References: In-Reply-To: Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit X-Spam-Score: 0.0 (/) X-Scan-Signature: e178fd6cb61ffb6940cd878e7fea8606 Content-Transfer-Encoding: 7bit Subject: [Ips] iSER flow control options (Was: Re: DA+iSER Flow Control, Limiting number of oustanding cmdsattarget) X-BeenThere: ips@ietf.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: IP Storage List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Sender: ips-bounces@ietf.org Errors-To: ips-bounces@ietf.org X-Spam-Score: 0.0 (/) X-Scan-Signature: 27ec2ff0f5c3b18b49c722f4f1748838 Content-Transfer-Encoding: 7bit I finally managed to catch up on this thread, let me attempt to summarize what I think are the options for iSER flow control. I would like to get more comments to assess the WG position, perhaps also from the co-chairs on this topic. I hope this long email acts as a helpful summary. Let me ignore S-RQ for a moment and consider a simple RQ. Let's even assume the target had adequately provisioned receive buffers (CmdSN window + 2 buffers, i.e. no overcommitting). In this simplistic case, if the initiator sends too many immediate commands, we know that the target would run out of receive buffers. Now the options that I see to deal with this scenario are (you may add more if I missed any): 1. Replenish buffers faster than the arrival of the unexpected messages (iSCSI layer may later discard the "too many immediate commands" if it deems so). 2. Require the RNIC to do graceful handling of lack of receive buffers on these exceptions. 3. Define a new iSCSI MaxOutstandingUnexpectedPDUs key-based limit scheme to deal with these exceptions (that John describes below). 4. Define an iSER end2end flow control protocol (such as the one iSER design team originally considered). 5. Mandate an S-RQ as the mandatory RNIC feature (so there's more flexibility for overprovisioning across multiple connections). 6. Allow the connection to be dropped because the buffers are overrun. #1 may be feasible (and every implementation tries to do it anyway), but basing the protocol on that assumption doesn't strike me as a "standards" way of specifying a protocol. #2 is quite nice from iSCSI & iSER's perspective. We should be able to cast any immediate/un-flow-controlled iSCSI PDU as an "exception" (which it is) message (in DDP parlance) and thus get around adding any more protocol restrictions in iSER. The major consideration here is the feasibility (from an RNIC vendor perspective) of realizing such a graceful handling. #3 seems like a workable option if we can cleary specify it. It however does add more protocol complexity to iSCSI, and makes some exception management cases harder to deal with (when the limit is not "None") as John points out below. #4 is clearly expensive although we know that it can work. This should be the last resort. #5 is a second-order choice in that it may still require help from other options. The main considerations here are: making normative references to the Verbs spec [ or defining the S-RQ text in abstract terms (i.e. stripping off references to WQE, CQE, RQ, CQ, RI etc.) - which again doesn't strike me as a "standards" way of defining a protocol ], the acceptance (from an RNIC vendor perspective) on requiring it. We still need to address a single connection case from the specification perspective (i.e. consider an embedded target supporting a single iSCSI/iSER connection). #6 is also feasible (I think Julian too hinted at this in an email). Clearly, an initiator sending too many immediate commands or a target sending too many Async Messages is suspect. Problem though is that this is a major functional difference between how an iSCSI system would behave, and an iSCSI/iSER system would behave (thus potentially exposing the dichotomy to layers further up in the stack). The other issue is that a genuine exception could be punished with an even bigger one - leading to termination of several I/Os. The way I see it, the current iSER spec depends on #2, #1, #5 and #6. Note that while #2 alone is necessary and sufficient for operation without connection drops, others work to minimize the probability of invoking graceful handling (#2) but together ultimately allow a connection failure if the graceful handling service cannot be maintained by the RNIC after several DDP exceptions. In the opinion of iSER authors, this integrated approach is quite reasonable. I realize that an argument can be made for #3 (which Caitlin & Alex did as I understand the exchanges so far). More comments from others would help the authors understand the WG position on this. -- Mallikarjun Mallikarjun Chadalapaka Networked Storage Architecture Network Storage Solutions Hewlett-Packard MS 5668 Roseville CA 95747 cbm [at] rose.hp.com John Hufferd wrote: > > Caitlin, > Perhaps we are talking past each other. My point has been and still is > that I would not like to see additional flow control used with > iSCSI/iSER that would prevent the "at distance" operations from being as > efficient as current iSCSI. Now having said that, I believe that the > point that Mike Ko said in his previous note is that all we need to do > is to cap the number of special responses that are beyond the normal > expected iSCSI windowing process. I think that is not really needed for > the correct use of SRQ, and I believe you concurred on that point. > However, I believe that it may be of value to someone that wants to > implement iSCSI/iSER without SRQ, or perhaps even in your hypothetical > generic Data Mover Service. So my interpretation of what Mike was > saying is as follows (which if correct I can accept): > > 1. Each iSCSI/iSER side (peer) will declare to the other (for this > discussion lets call this Keyword=Value pair > "MaxOutstandingUnexpectedPDUs") a value that represents the number of > messages that can occur beyond what is handled by the normal iSCSI Flow > Control. The values may be either numeric. or "None" where None means > no limit specified. I would even suggest that "None" should be the > default. > 2. From the Target side, this value (that it would declare), is the Max > number of Outstanding Immediate Commands, and/or Outstanding SNACK > commands that it could receive (SNACK should not be an important > consideration, but the spec says that even if SNACKs are not needed, if > SNACKs are issued by the Initiator they are passed on, so this value > would cover SNACKs also). > 3. From the Initiator side, this value (that it would declare), is the > Max number of outstanding Async Messages and/or NOPs that it can receive > from the Target. > > There would be no need to have any additional flow control, and the NON > SRQ Target implementations could ensure that the number of buffers that > they queue will be at least sufficient to handle the iSCSI Command > Window, plus the number of Declared "MaxOutstandingUnexpectedPDUs". > And the initiator would queue the number of buffers that will be > sufficient to handle the iSCSI Command Window responses, plus the number > of Declared "MaxOutstandingUnexpectedPDUs". > > Now, as I said before, I believe it is best handled by an SRQ approach > that does not limit the flow of messages "at Distance", and that where > ever possible the peers should operate with the value of "NONE" (no > limits) and that is why I think it should be the default. > > The reason that this approach is acceptable to me is because the use of > Immediate commands (and SNACKs) or the arrival of Async Message PDUs > from the Target, should be rare enough to not inhibit the normal iSCSI > flow control. However, we should have a caution stated somewhere, that > this might have an important Error Recovery impacts when dealing with > ERL=2 and we are in an at distance situation. That is, when a link > fails (which may happen more than we like "at distance") the ERL=2 mode > says that the Task Management (Immediate) Commands must reassign the > Tasks to a surviving connection. It is very likely that we may see > hundreds of commands that are subject to this action and since we have > only a limited amount of time to make this switch, it could be that the > distance and limit on Outstanding Immediate commands could lengthen this > recovery to a point that the recovery time out is exceeded. (This would > not be a problem with the SRQ design that I talked about previously.) > > Caitlin, with your acceptance that the No Limit SRQ approach that I > specified, and with the above proposal, which I believe may cover your > additional issues, I think we maybe close to closure on this. > > However, some of the folks on this reflector have told me that they > prefer a resolution that requires the "Graceful handling" of RQ overruns > by the RNIC. There are several approaches to do that, and quite frankly > I prefer that over the above. But perhaps we need to see if we at least > can agree on the above and then discuss the other options. > . > . > John L. Hufferd > Senior Technical Staff Member (STSM) > IBM/System Group, San Jose CA > Main Office: (408) 256-0403, Tie: 276-0403, eFax: (408) 904-4688 > Alt Office: (408) 997-6136, Cell: (408) 499-9702 > Internet Address: hufferd@us.ibm.com > > > *"Caitlin Bestler" * > > 10/06/2004 01:42 AM > > > To > John Hufferd/San Jose/IBM@IBMUS > cc > "Caitlin Bestler" , ips@ietf.org > Subject > Re: [Ips] DA+iSER Flow Control, Limiting number of oustanding > cmdsattarget > > > > > > > > > > John Hufferd said: > > I guess I still do not see the security exposure for a kernel device > > driver that can inspect the buffers within nano seconds of the data > > arrival and determine if the packet is valid, and if an invalid set of > > buffers is received, terminating the connection. This puts the security > > issue right on the shoulders of the iSCSI device driver that can tell > that > > the buffers are invalid. I do not see the issue. A run away connection > > that invalidly uses up all the bandwidth is just not a real problem > if the > > buffers are inspected as they come in by the iSCSI device driver, and the > > connection dropped when an error is detected. > > . > > . > > That division of responsibility should certainly be allowed. > It would require the iSER layer to reap completions almost > instantly, and to pass all completed iSCSI PDUs to the iSCSI > layer almost instantly. The iSCSI layer would have to keep up, > period. It could do so by tossing packets and terminating > iSCSI sessions. > > The current draft does not provide language that would justify > that division of responsibilities. That should be fixed. > > I also believe, although I really don't care that strongly > about it, that a properly designed wire protocol should not > be dependent on a specific implementation strategy. > > That means that there MUST be a method by which it is impossible > for a single malfunctionign and/or malicious client to drain > shared resources and thereby cause a failure affecting other > clients. Requiring the iSCSI layer to at least accept packets > at line rate, or to refrain from using shared resources, would > indeed accomplish that. But it unnecessarily constrains the > range of possible implementations. > > Specifying a cap on the number of "extra" untagged messages, > either as a ULP constant or a session parameter,would enable > the use of per-connection safety mechanisms associated with > SRQs. That would enable a wide range of possible implementations > to use SRQs to implement iSCSI/iSER. > > As previously noted, pre-allocating a fully committed buffer > for each possible untagged message on each connection is not > a prudent method of allocating memory for an iSCSI daemon. > Enabling the widest possible range of implementations to > use SRQs when implementing iSER is a desirable goal. > > Implementations which are indeed capable of having the iSCSI > layer accept iSCSI PDUs at line rate would never have to use > this information. At worst they are exchanging one extra piece > of data during session setup. That is much to ask if it enables > a wider range of possible implementations. > > One of the benefits of RDMA is that when it is used properly > in can enable daemons to be implemented in user-space that > previously could only have been attempted in the kernel. > That is a desirable feature. iSER should not be designed > on the assumption that it is unobtainable. > > > -- > Caitlin Bestler > http://asomi.com/ > > > ------------------------------------------------------------------------ > > _______________________________________________ > Ips mailing list > Ips@ietf.org > https://www1.ietf.org/mailman/listinfo/ips _______________________________________________ Ips mailing list Ips@ietf.org https://www1.ietf.org/mailman/listinfo/ips From ips-bounces@ietf.org Wed Oct 6 21:30:33 2004 Received: from ietf-mx.ietf.org (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id VAA08507 for ; Wed, 6 Oct 2004 21:30:33 -0400 (EDT) Received: from megatron.ietf.org ([132.151.6.71]) by ietf-mx.ietf.org with esmtp (Exim 4.33) id 1CFNGg-0002Ty-8o for ips-web-archive@ietf.org; Wed, 06 Oct 2004 21:40:31 -0400 Received: from localhost.localdomain ([127.0.0.1] helo=megatron.ietf.org) by megatron.ietf.org with esmtp (Exim 4.32) id 1CFMzs-0000Mb-R4; Wed, 06 Oct 2004 21:23:08 -0400 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by megatron.ietf.org with esmtp (Exim 4.32) id 1CFMuM-0007fG-Nn for ips@megatron.ietf.org; Wed, 06 Oct 2004 21:17:26 -0400 Received: from ietf-mx.ietf.org (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id VAA07084 for ; Wed, 6 Oct 2004 21:17:24 -0400 (EDT) Received: from palrel13.hp.com ([156.153.255.238]) by ietf-mx.ietf.org with esmtp (Exim 4.33) id 1CFN3v-0001d4-Ba for ips@ietf.org; Wed, 06 Oct 2004 21:27:22 -0400 Received: from rosemail.rose.hp.com (rosemail.rose.hp.com [15.43.209.160]) by palrel13.hp.com (Postfix) with ESMTP id 56A701C0CCE9 for ; Wed, 6 Oct 2004 18:17:22 -0700 (PDT) Received: from [127.0.0.1] (pal1nai160042.nsr.hp.com [15.244.160.42]) by rosemail.rose.hp.com (Postfix) with ESMTP id 0297E7FFE for ; Wed, 6 Oct 2004 18:17:21 -0700 (PDT) Message-ID: <4164991F.3000508@rose.hp.com> Date: Wed, 06 Oct 2004 18:17:19 -0700 From: "Mallikarjun C." User-Agent: Mozilla Thunderbird 0.7.3 (Windows/20040803) X-Accept-Language: en-us, en MIME-Version: 1.0 To: ips@ietf.org Subject: Re: [Ips] setting StatSN in a data pdu References: <000801c4a897$a4c5efe0$0303a8c0@ivvt2dxrc11> <00f501c4ab25$9e433200$6e01000a@ivvt2dxrc11> In-Reply-To: <00f501c4ab25$9e433200$6e01000a@ivvt2dxrc11> Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit X-Spam-Score: 0.0 (/) X-Scan-Signature: 3e15cc4fdc61d7bce84032741d11c8e5 Content-Transfer-Encoding: 7bit X-BeenThere: ips@ietf.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: IP Storage List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Sender: ips-bounces@ietf.org Errors-To: ips-bounces@ietf.org X-Spam-Score: 0.0 (/) X-Scan-Signature: f607d15ccc2bc4eaf3ade8ffa8af02a0 Content-Transfer-Encoding: 7bit Eddy, yes, what you suggest is more accurate. -- Mallikarjun Mallikarjun Chadalapaka Networked Storage Architecture Network Storage Solutions Hewlett-Packard MS 5668 Roseville CA 95747 cbm [at] rose.hp.com Eddy Quicksall wrote: > Julian or Mallikarjun, > > I didn't see a response to this. > > Eddy > > ----- Original Message ----- > *From:* Eddy Quicksall > > *To:* ips@ietf.org > *Sent:* Saturday, October 02, 2004 11:51 AM > *Subject:* [Ips] setting StatSN in a data pdu > > Section 3.5.1.5 says "In addition, StatSN is carried by the Data-In > PDUs." > > But 10.7.3 says "The fields StatSN, Status, and Residual Count only > have meaningful content if the S bit is set to 1" > > I believe section 3.5.1.5 should say "In addition, when the S bit is > set StatSN is carried by the Data-In PDUs." > > Eddy > > ------------------------------------------------------------------------ > > _______________________________________________ > Ips mailing list > Ips@ietf.org > https://www1.ietf.org/mailman/listinfo/ips > > > ------------------------------------------------------------------------ > > _______________________________________________ > Ips mailing list > Ips@ietf.org > https://www1.ietf.org/mailman/listinfo/ips _______________________________________________ Ips mailing list Ips@ietf.org https://www1.ietf.org/mailman/listinfo/ips From ips-bounces@ietf.org Wed Oct 6 21:38:28 2004 Received: from ietf-mx.ietf.org (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id VAA09279 for ; Wed, 6 Oct 2004 21:38:28 -0400 (EDT) Received: from megatron.ietf.org ([132.151.6.71]) by ietf-mx.ietf.org with esmtp (Exim 4.33) id 1CFNOM-00034y-U9 for ips-web-archive@ietf.org; Wed, 06 Oct 2004 21:48:27 -0400 Received: from localhost.localdomain ([127.0.0.1] helo=megatron.ietf.org) by megatron.ietf.org with esmtp (Exim 4.32) id 1CFNCY-0002Dn-QO; Wed, 06 Oct 2004 21:36:14 -0400 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by megatron.ietf.org with esmtp (Exim 4.32) id 1CFN3c-0000k2-7k for ips@megatron.ietf.org; Wed, 06 Oct 2004 21:27:00 -0400 Received: from ietf-mx.ietf.org (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id VAA07889 for ; Wed, 6 Oct 2004 21:26:58 -0400 (EDT) Received: from palrel13.hp.com ([156.153.255.238]) by ietf-mx.ietf.org with esmtp (Exim 4.33) id 1CFNDD-0002GK-LU for ips@ietf.org; Wed, 06 Oct 2004 21:36:56 -0400 Received: from rosemail.rose.hp.com (rosemail.rose.hp.com [15.43.209.160]) by palrel13.hp.com (Postfix) with ESMTP id E793F1C00531 for ; Wed, 6 Oct 2004 18:26:58 -0700 (PDT) Received: from [127.0.0.1] (pal1nai160042.nsr.hp.com [15.244.160.42]) by rosemail.rose.hp.com (Postfix) with ESMTP id 589487FFD for ; Wed, 6 Oct 2004 18:26:58 -0700 (PDT) Message-ID: <41649B60.60002@rose.hp.com> Date: Wed, 06 Oct 2004 18:26:56 -0700 From: "Mallikarjun C." User-Agent: Mozilla Thunderbird 0.7.3 (Windows/20040803) X-Accept-Language: en-us, en MIME-Version: 1.0 To: ips@ietf.org Subject: Re: [Ips] resending Data-in with S bit set References: In-Reply-To: Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit X-Spam-Score: 0.0 (/) X-Scan-Signature: 2b2ad76aced9b1d558e34a970a85c027 Content-Transfer-Encoding: 7bit X-BeenThere: ips@ietf.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: IP Storage List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Sender: ips-bounces@ietf.org Errors-To: ips-bounces@ietf.org X-Spam-Score: 0.0 (/) X-Scan-Signature: dd7e0c3fd18d19cffdd4de99a114001d Content-Transfer-Encoding: 7bit I agree with what Julian says below. The "exact replica" wording of 10.16 is quite unambiguous. -- Mallikarjun Mallikarjun Chadalapaka Networked Storage Architecture Network Storage Solutions Hewlett-Packard MS 5668 Roseville CA 95747 cbm [at] rose.hp.com Julian Satran wrote: > > Dean & all, > > > Sorry - the spec is not ambiguous and I was wrong in my previous answer > (I just overlooked the StatSN field in Data-In). > The target should (following 10.16) re-issue the exact copy of the > Data-In PDU that included the status. > > Julo > > > *"Dean Scoville" * > > 06/10/04 00:49 > > > To > "Eddy Quicksall" , Julian > Satran/Haifa/IBM@IBMIL > cc > > Subject > RE: [Ips] resending Data-in with S bit set > > > > > > > > > I'd say the RFC is ambiguous, because it sounds like it is now being > interpreted to say that > numbered responses include only SCSI Response PDUs, even though I > thought the > statements in sections 3.2 and 6.1.4.2 indicated that numbered responses > include any PDUs > carrying status... including Data-In PDUs with the S-bit set. Does that > mean the initiator > shouldn't be able to SNACK for a Reject PDU or a NOP-PDU, since they > aren't SCSI Response > PDUs either? > > Dean > -----Original Message-----* > From:* Eddy Quicksall [mailto:eddy_quicksall_iVivity_iSCSI@comcast.net]* > Sent:* Tuesday, October 05, 2004 3:44 PM* > To:* Dean Scoville; Julian Satran* > Cc:* ips@ietf.org* > Subject:* Re: [Ips] resending Data-in with S bit set > > Yes, when I said "Status PDUs", I used shorthand and meant SCSI Response > PDUs". > > What you point out is exactly the reason I wanted to know the intent and > if I understood the spec correctly. I did not mean to say I agree or > disagree with the intent. > > Since the spec is finished, I think it is too late to argue for a > change. It looks like the initiator just has to be a little smart when > it needs all PDUs that carry status. > > Eddy > ----- Original Message ----- > *From:* _Dean Scoville_ > *To:* _Eddy Quicksall_ > ; _Julian Satran_ > *Cc:* _ips@ietf.org_ > *Sent:* Tuesday, October 05, 2004 6:22 PM > *Subject:* RE: [Ips] resending Data-in with S bit set > > When you say only "Status" PDUs, do you mean only "SCSI Response" PDUs, > and that the target should remain silent about any phase collapsed > Data-In PDUs? > That doesn't make sense to me. If the initiator has 100 commands > outstanding and > issues a Status SNACK because a received StatSN indicates that some PDUs > are > missing, it shouldn't have to issue Data SNACKs for all of the > outstanding commands > to see if one of them perhaps was using phase collapse and happened to > have the > missing StatusSN. Is that really what is intended? > > The last sentence in Section 3.2 defines a numbered response as: > > "Also, unless otherwise specified, status, response, or numbered > response have the same meaning." > > And Section 6.1.4.2 states: > > "Lost iSCSI numbered Response. It is recognized by either > identifying a data digest > error on a Response PDU or a Data-In PDU carrying the status, or by > receiving a > Response PDU with a higher StatSN than expected." > > The above two RFC statements would lead me to believe that numbered > responses include > Data-In PDUs, and that the numbered responses referred to in Section > 10.16.1 in > reference to Status SNACKs should also include Data-In retransmissions. > > Dean > -----Original Message-----* > From:* Eddy Quicksall [mailto:eddy_quicksall_iVivity_iSCSI@comcast.net]* > Sent:* Tuesday, October 05, 2004 3:09 PM* > To:* Dean Scoville; Julian Satran* > Cc:* ips@ietf.org* > Subject:* Re: [Ips] resending Data-in with S bit set > > Maybe my question was misunderstood. To say it another way: > > Is it true that a Status SNACK _only_ solicits Status PDUs and that it > does _not_ solicit Data-In PDUs which have the S bit set? > > I interpret Julian's answer to this as "yes, that is true". > > Eddy > ----- Original Message ----- > *From:* _Dean Scoville_ > *To:* _Julian Satran_ ; _Eddy > Quicksall_ > *Cc:* _ips@ietf.org_ > *Sent:* Tuesday, October 05, 2004 5:54 PM > *Subject:* RE: [Ips] resending Data-in with S bit set > > Julian, > > Section 10.16 states the following about PDUs transmitted as a result of > a SNACK: > > "The numbered-response(s) or R2T(s), requested by a SNACK, MUST be > delivered as exact replicas of the ones that the target transmitted > originally > except for the fields ExpCmdSN, MaxCmdSN, and ExpDataSN, which MUST > carry the current values. R2T(s) requested by SNACK MUST also carry > the > current values and except for resegmentation (see Section 10.16.3 > Resegmentation)." > > Section 10.16.1 describes Status and Data/R2T SNACK types as: > > 0-Data/R2T SNACK - requesting retransmission of one or more Data-In > or R2T PDUs. > 1-Status SNACK - requesting retransmission of one or more numbered > responses. > > It seems to me that if the target receives a Status SNACK for status > sent on a Data-In > PDU due to phase collapse, the target should send an exact replica of > the Data-In PDU. > Similarly, if the target receives a Data SNACK, it should retransmit the > Data-In PDU with > the S bit set, since it is a retransmission. > > I may have misunderstood your recent posting, but I thought it said the > target should > generate a totally new SCSI Response PDU in response to the Status SNACK > rather > than retransmitting the original Data-In PDU. And that it should send a > Data-In PDU > without the S-bit set in the case of a Data SNACK. Both of these > behaviors seem to > violate what is stated in Sections 10.16 and 10.16.1 of the iSCSI RFC, > as quoted above. > > I would appreciate clarification on the target's expected response in > these situations. > > thanks, > > Dean Scoville > -----Original Message-----* > From:* ips-bounces@ietf.org [mailto:ips-bounces@ietf.org]*On Behalf Of > *Julian Satran* > Sent:* Monday, October 04, 2004 7:43 AM* > To:* Eddy Quicksall* > Cc:* ips@ietf.org* > Subject:* Re: [Ips] resending Data-in with S bit set > > > Eddy, > > This is correct. > The data-in PDUs don't have to be sent. > The initiator has to request the data-in PDUs explicitly and MUST do so > before it acks the status (that implicitly acks all the data and deletes > any record of the command at target). > > Julo > > ips-bounces@ietf.org wrote on 04/10/2004 15:20:56: > > > Can you confirm that the intent of 10.16.6. is as follows? > > > > If the Initiator has requested for a Status SNACK with BegRun=0 and > > RunLength=0 (i.e, all unacknowledged status pdus) and the target has > > some Data-In PDUs carrying *status* that have not been acknowledged, > > the target will *not* resend those Data-In PDUs carrying the status. > > > > The unacknowledged queue looks like > > > > response(statSN=5) ---> response(statSN=6) -----> Data-In w/S > > bit(statSN=7) ----> response(statSN=8) > > > > The target will resend pdus with statSN 5, 6 and 8. The Data-In with > > statSN 7 will not be resent even though it sends status. > > > > Eddy > > _______________________________________________ > > Ips mailing list > > Ips@ietf.org > > https://www1.ietf.org/mailman/listinfo/ips > > ------------------------------------------------------------------------ > > _______________________________________________ > Ips mailing list > Ips@ietf.org > https://www1.ietf.org/mailman/listinfo/ips > > > ------------------------------------------------------------------------ > > _______________________________________________ > Ips mailing list > Ips@ietf.org > https://www1.ietf.org/mailman/listinfo/ips _______________________________________________ Ips mailing list Ips@ietf.org https://www1.ietf.org/mailman/listinfo/ips From ips-bounces@ietf.org Thu Oct 7 06:07:46 2004 Received: from ietf-mx.ietf.org (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id GAA29947 for ; Thu, 7 Oct 2004 06:07:46 -0400 (EDT) Received: from megatron.ietf.org ([132.151.6.71]) by ietf-mx.ietf.org with esmtp (Exim 4.33) id 1CFVLJ-0000D9-Bp for ips-web-archive@ietf.org; Thu, 07 Oct 2004 06:17:49 -0400 Received: from localhost.localdomain ([127.0.0.1] helo=megatron.ietf.org) by megatron.ietf.org with esmtp (Exim 4.32) id 1CFV35-0000ES-Ap; Thu, 07 Oct 2004 05:58:59 -0400 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by megatron.ietf.org with esmtp (Exim 4.32) id 1CFUzl-0007h8-D6 for ips@megatron.ietf.org; Thu, 07 Oct 2004 05:55:33 -0400 Received: from ietf-mx.ietf.org (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id FAA28809 for ; Thu, 7 Oct 2004 05:55:30 -0400 (EDT) Received: from system60.simplicato.com ([207.99.47.60] helo=host52a.simplicato.com) by ietf-mx.ietf.org with esmtp (Exim 4.33) id 1CFV9R-0007sf-BQ for ips@ietf.org; Thu, 07 Oct 2004 06:05:33 -0400 Received: from localhost (localhost.simplicato.com [127.0.0.1]) by host52a.simplicato.com (Postfix) with ESMTP id 1ADEE11E05D; Thu, 7 Oct 2004 05:55:30 -0400 (EDT) Received: from host52a.simplicato.com ([127.0.0.1]) by localhost (host52a.simplicato.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 75155-05; Thu, 7 Oct 2004 05:55:29 -0400 (EDT) Received: from [192.168.1.2] (bzq-198-76.red.bezeqint.net [212.179.198.76]) by host52a.simplicato.com (Postfix) with ESMTP id 46CC811E04C; Thu, 7 Oct 2004 05:55:27 -0400 (EDT) In-Reply-To: <41649554.60700@rose.hp.com> References: <41649554.60700@rose.hp.com> Mime-Version: 1.0 (Apple Message framework v619) Content-Type: text/plain; charset=US-ASCII; format=flowed Message-Id: <034358AE-1847-11D9-8836-000393814604@asomi.com> Content-Transfer-Encoding: 7bit From: Caitlin Bestler Subject: Re: [Ips] iSER flow control options (Was: Re: DA+iSER Flow Control, Limiting number of oustanding cmdsattarget) Date: Thu, 7 Oct 2004 11:55:26 +0200 To: "Mallikarjun C." X-Mailer: Apple Mail (2.619) X-Virus-Scanned: by amavisd-new at simplicato.com X-Spam-Score: 0.1 (/) X-Scan-Signature: f2728948111f2edaaf8980b5b9de55af Content-Transfer-Encoding: 7bit Cc: ips@ietf.org X-BeenThere: ips@ietf.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: IP Storage List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Sender: ips-bounces@ietf.org Errors-To: ips-bounces@ietf.org X-Spam-Score: 0.1 (/) X-Scan-Signature: 6a45e05c1e4343200aa6b327df2c43fc Content-Transfer-Encoding: 7bit On Oct 7, 2004, at 3:01 AM, Mallikarjun C. wrote: > I finally managed to catch up on this thread, let me attempt to > summarize what I think are > the options for iSER flow control. I would like to get more comments > to assess the WG > position, perhaps also from the co-chairs on this topic. I hope this > long email acts as a > helpful summary. > > Let me ignore S-RQ for a moment and consider a simple RQ. Let's even > assume the target > had adequately provisioned receive buffers (CmdSN window + 2 buffers, > i.e. no overcommitting). > In this simplistic case, if the initiator sends too many immediate > commands, we know that the > target would run out of receive buffers. Now the options that I see > to deal with this scenario > are (you may add more if I missed any): > > 1. Replenish buffers faster than the arrival of the unexpected > messages (iSCSI layer may later > discard the "too many immediate commands" if it deems so). > 2. Require the RNIC to do graceful handling of lack of receive buffers > on these exceptions. > 3. Define a new iSCSI MaxOutstandingUnexpectedPDUs key-based limit > scheme to deal with > these exceptions (that John describes below). > 4. Define an iSER end2end flow control protocol (such as the one iSER > design team originally > considered). > 5. Mandate an S-RQ as the mandatory RNIC feature (so there's more > flexibility for overprovisioning > across multiple connections). > 6. Allow the connection to be dropped because the buffers are overrun. > > > #1 may be feasible (and every implementation tries to do it anyway), > but basing the protocol on that > assumption doesn't strike me as a "standards" way of specifying a > protocol. I think the operative word here is "try". What I am particularly concerned about is that for both TCP and SCP transport level flow control is based on the availability of receive buffering and network congestion. A TCP connection or SCTP association in use for RDDP effectively has no buffer limits, assuming it can forward DDP Segments to the RDDP layer at wire speed. RDDP is responsible for placing the buffers when presented, or terminating the stream. It has no back-pressure mechanism. However, many implementations will not tailor the calculation of the buffer window for RDMA, but rather use the general purpose algorithm. That can result in false blocking of placable traffic, but this failure will not be apparent when users are used to TCP bottlenecking. Existing iSCSI implementations in particular could easily be relying on TCP bottlenecks. When they test on an implementation that does not tailor the TCP window on RDMA connections to "no system buffering, hence no buffer limits" they will not detect that the problem is still there. Later they will test on an implementation that *does* fully eliminate the TCP bottleneck, and promptly declare that "this RNIC breaks my iSCSI" and demand the vendor "fix it". > > #2 is quite nice from iSCSI & iSER's perspective. We should be able > to cast any > immediate/un-flow-controlled iSCSI PDU as an "exception" (which it is) > message > (in DDP parlance) and thus get around adding any more protocol > restrictions in iSER. > The major consideration here is the feasibility (from an RNIC vendor > perspective) of > realizing such a graceful handling. That is moving the "take it or flush it" interface up from the LLP to RDDP interface to the RDMAP to ULP interface. It is a good idea, one that local interfaces *should* offer. But we all know that it isn't part of the de facto standard. It is one of many defects in that interface, but I would still oppose adopting any ULP that cannot be implemented on top of commonly deployed verbs. > > #3 seems like a workable option if we can cleary specify it. It > however does add > more protocol complexity to iSCSI, and makes some exception management > cases > harder to deal with (when the limit is not "None") as John points out > below. There is also "3a" which is to make a default limit that is specified in the protocol itself. I believe the goal should be that no application that is generating these messages for their intended purposes should ever have to check the limit. The receiver SHOULD (or perhaps even MUST) set it high enough so that *only* a malfunctioning or malicious client could exceed it. The flow control can be fairly simple to implement however, anytime the flow control limit would have been hit, it is for an unanticipated message that the target iSCSI side would be free to throw away anyway. The sender could simply do so immediately. But I do think it would be better to define the credit limit such that it would be impossible to violate it when generating messages legitimately. The goal here is to enable the receiving side to identify a runaway malfunctioning or malicious client that is attacking shared resources *before* it results in a legitimate message from another client being dropped. Even without SRQs, it is not that difficult for the receiving iSER layer to allow sufficient extra receive buffers to deal with all *legitimate* iSCSI packets. It is also certainly preferable for the receive side to have some extra buffers available rather than requiring non-malfunctioning/non-abusive senders to check their credit limit before sending any of the "unaccounted" iSCSI PDUs. These messages were excluded from the normal iSCSI flow control precisely because they were not anticipated to be high volume. So there are two ways to determine when a connection is sending illegitimate iSCSI packets: place a credit limit that legitimate applications will not violate (even without thinking about it) or have the iSER layer examine packets at wire speed. > > #4 is clearly expensive although we know that it can work. This > should be the last resort. > > #5 is a second-order choice in that it may still require help from > other options. The main > considerations here are: making normative references to the Verbs spec > [ or defining the > S-RQ text in abstract terms (i.e. stripping off references to WQE, > CQE, RQ, CQ, RI etc.) > which again doesn't strike me as a "standards" way of defining a > protocol ], the acceptance > (from an RNIC vendor perspective) on requiring it. We still need to > address a single > connection case from the specification perspective (i.e. consider an > embedded target > supporting a single iSCSI/iSER connection). An SRQ does not eliminate the vulnerability. In fact it is necessary for it to exist. A runaway client connected to a simple RQ can only tear down its own connection. It does not impact other clients. With an SRQ, you either have a per-connection limit/alarm or you are dependent on wire-speed draining of completions by the ULP (the iSER layer). That case is essentially option 1, enabling the per-connection limit/alarm requires having a number to set it with which leads back to option 3 (or 3a). An argument for 3, as opposed to 3a, is to specifically allow "None". But the iSER layer should only uses "None" when it is committing to drain completions at wire speed *or* that the pool of buffers used for untagged messages is not shared with any other iSCSI session. > > #6 is also feasible (I think Julian too hinted at this in an email). > Clearly, an initiator sending too many immediate commands or a target > sending too many Async Messages is suspect. Problem though is that > this is a major functional difference between how an iSCSI system > would behave, and an iSCSI/iSER system would behave (thus potentially > exposing the dichotomy to layers further up in the stack). The other > issue is that a genuine exception could be punished with an even > bigger one - leading to termination of several I/Os. > > The way I see it, the current iSER spec depends on #2, #1, #5 and #6. > Note that while #2 > alone is necessary and sufficient for operation without connection > drops, others work to > minimize the probability of invoking graceful handling (#2) but > together ultimately allow > a connection failure if the graceful handling service cannot be > maintained by the RNIC > after several DDP exceptions. In the opinion of iSER authors, this > integrated approach is quite reasonable. > Option 2 is not a real solution. It is simply attempting to shove the excess messages into buffers that are accounted for (either "system buffers" or by abusing the network itself as a buffer forcing false retransmits). But ultimately it is just hoping that the problem goes away because it does nothing to ensure that only the connection causing the problem will have the corrective action taken against it. > I realize that an argument can be made for #3 (which Caitlin & Alex > did as I understand the > exchanges so far). More comments from others would help the authors > understand the WG position on this. > I believe the RDDP WG position on this is clear. In my opinion option, only options 1 and 3 comply with the RDDP Security requirements for use of shared receive resources. _______________________________________________ Ips mailing list Ips@ietf.org https://www1.ietf.org/mailman/listinfo/ips From ips-bounces@ietf.org Thu Oct 7 08:19:28 2004 Received: from ietf-mx.ietf.org (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id IAA09945 for ; Thu, 7 Oct 2004 08:19:28 -0400 (EDT) Received: from megatron.ietf.org ([132.151.6.71]) by ietf-mx.ietf.org with esmtp (Exim 4.33) id 1CFXOl-0008Q5-Tr for ips-web-archive@ietf.org; Thu, 07 Oct 2004 08:29:32 -0400 Received: from localhost.localdomain ([127.0.0.1] helo=megatron.ietf.org) by megatron.ietf.org with esmtp (Exim 4.32) id 1CFXBm-0005DV-Ki; Thu, 07 Oct 2004 08:16:06 -0400 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by megatron.ietf.org with esmtp (Exim 4.32) id 1CFX4W-0003DK-WD for ips@megatron.ietf.org; Thu, 07 Oct 2004 08:08:37 -0400 Received: from ietf-mx.ietf.org (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id IAA08987 for ; Thu, 7 Oct 2004 08:08:35 -0400 (EDT) Received: from magic.adaptec.com ([216.52.22.17]) by ietf-mx.ietf.org with esmtp (Exim 4.33) id 1CFXED-0007f6-JY for ips@ietf.org; Thu, 07 Oct 2004 08:18:38 -0400 Received: from redfish.adaptec.com (redfish.adaptec.com [162.62.50.11]) by magic.adaptec.com (8.11.6/8.11.6) with ESMTP id i97C82W30306 for ; Thu, 7 Oct 2004 05:08:02 -0700 Received: from aime2k02.adaptec.com (aime2k02.adaptec.com [10.25.8.42]) by redfish.adaptec.com (8.11.6/8.11.6) with ESMTP id i97C82x00610 for ; Thu, 7 Oct 2004 05:08:02 -0700 X-MimeOLE: Produced By Microsoft Exchange V6.0.6487.1 Content-Class: urn:content-classes:message MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable Subject: RE: [Ips] iSER flow control options (Was: Re: DA+iSER Flow Control, Limiting number of oustanding cmdsattarget) Date: Thu, 7 Oct 2004 05:08:00 -0700 Message-ID: Thread-Topic: [Ips] iSER flow control options (Was: Re: DA+iSER Flow Control, Limiting number of oustanding cmdsattarget) Thread-Index: AcSsVbeFd+jTGK+dT+O9F75yrt14iwADv1AA From: "Carrier, John" To: X-Spam-Score: 0.0 (/) X-Scan-Signature: 9e5c23589e6cce06555030c0194c9e2b Content-Transfer-Encoding: quoted-printable X-BeenThere: ips@ietf.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: IP Storage List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Sender: ips-bounces@ietf.org Errors-To: ips-bounces@ietf.org X-Spam-Score: 0.0 (/) X-Scan-Signature: fcb459c204557d9509ce9c1b55d771f1 Content-Transfer-Encoding: quoted-printable I have nothing really to add to the conversation, other than I agree = with Caitlin's analysis. =20 The iSER design team relied on the RNIC to handle RQ overruns = gracefully. But for SRQs, the verbs were modified to provide a = mechanism for the RNIC to report to the ULP when a connection = over-consumed SRQ resources. Since verbs are currently out of the scope = of the IETF, this leaves iSER vulnerable as Caitlin has described. I appreciate JohnH's concerns about the effects credit-based flow = control could have on iSCSI at a distance, but that argument just = highlights how iSCSI relies on system buffers consumed by the transport = for its flow control. An RNIC isn't designed for this level of resource = availability.=20 --jc > -----Original Message----- > From: ips-bounces@ietf.org [mailto:ips-bounces@ietf.org]On Behalf Of > Caitlin Bestler > Sent: Thursday, October 07, 2004 2:55 AM > To: Mallikarjun C. > Cc: ips@ietf.org > Subject: Re: [Ips] iSER flow control options (Was: Re: DA+iSER Flow > Control,Limiting number of oustanding cmdsattarget) >=20 >=20 >=20 > On Oct 7, 2004, at 3:01 AM, Mallikarjun C. wrote: >=20 > > I finally managed to catch up on this thread, let me attempt to=20 > > summarize what I think are > > the options for iSER flow control. I would like to get=20 > more comments=20 > > to assess the WG > > position, perhaps also from the co-chairs on this topic. I=20 > hope this=20 > > long email acts as a > > helpful summary. > > > > Let me ignore S-RQ for a moment and consider a simple RQ. =20 > Let's even=20 > > assume the target > > had adequately provisioned receive buffers (CmdSN window +=20 > 2 buffers,=20 > > i.e. no overcommitting). > > In this simplistic case, if the initiator sends too many immediate=20 > > commands, we know that the > > target would run out of receive buffers. Now the options=20 > that I see=20 > > to deal with this scenario > > are (you may add more if I missed any): > > > > 1. Replenish buffers faster than the arrival of the unexpected=20 > > messages (iSCSI layer may later > > discard the "too many immediate commands" if it deems so). > > 2. Require the RNIC to do graceful handling of lack of=20 > receive buffers=20 > > on these exceptions. > > 3. Define a new iSCSI MaxOutstandingUnexpectedPDUs key-based limit=20 > > scheme to deal with > > these exceptions (that John describes below). > > 4. Define an iSER end2end flow control protocol (such as=20 > the one iSER=20 > > design team originally > > considered). > > 5. Mandate an S-RQ as the mandatory RNIC feature (so there's more=20 > > flexibility for overprovisioning > > across multiple connections). > > 6. Allow the connection to be dropped because the buffers=20 > are overrun. > > > > >=20 > > #1 may be feasible (and every implementation tries to do it=20 > anyway),=20 > > but basing the protocol on that > > assumption doesn't strike me as a "standards" way of specifying a=20 > > protocol. >=20 > I think the operative word here is "try". What I am particularly=20 > concerned about is that for both TCP > and SCP transport level flow control is based on the availability of=20 > receive buffering and network > congestion. >=20 > A TCP connection or SCTP association in use for RDDP=20 > effectively has no=20 > buffer limits, assuming > it can forward DDP Segments to the RDDP layer at wire speed. RDDP is=20 > responsible for placing > the buffers when presented, or terminating the stream. It has no=20 > back-pressure mechanism. >=20 > However, many implementations will not tailor the calculation of the=20 > buffer window for RDMA, > but rather use the general purpose algorithm. That can=20 > result in false=20 > blocking of placable > traffic, but this failure will not be apparent when users are used to=20 > TCP bottlenecking. >=20 > Existing iSCSI implementations in particular could easily be=20 > relying on=20 > TCP bottlenecks. > When they test on an implementation that does not tailor the=20 > TCP window=20 > on RDMA connections > to "no system buffering, hence no buffer limits" they will not detect=20 > that the problem is still there. > Later they will test on an implementation that *does* fully eliminate=20 > the TCP bottleneck, and > promptly declare that "this RNIC breaks my iSCSI" and demand=20 > the vendor=20 > "fix it". >=20 > > > > #2 is quite nice from iSCSI & iSER's perspective. We=20 > should be able=20 > > to cast any > > immediate/un-flow-controlled iSCSI PDU as an "exception"=20 > (which it is)=20 > > message > > (in DDP parlance) and thus get around adding any more protocol=20 > > restrictions in iSER. > > The major consideration here is the feasibility (from an=20 > RNIC vendor=20 > > perspective) of > > realizing such a graceful handling. >=20 > That is moving the "take it or flush it" interface up from the LLP to=20 > RDDP interface to > the RDMAP to ULP interface. It is a good idea, one that local=20 > interfaces *should* > offer. But we all know that it isn't part of the de facto=20 > standard. It=20 > is one of many > defects in that interface, but I would still oppose adopting any ULP=20 > that cannot > be implemented on top of commonly deployed verbs. >=20 > > > > #3 seems like a workable option if we can cleary specify it. It=20 > > however does add > > more protocol complexity to iSCSI, and makes some exception=20 > management=20 > > cases > > harder to deal with (when the limit is not "None") as John=20 > points out=20 > > below. >=20 > There is also "3a" which is to make a default limit that is specified=20 > in the protocol itself. >=20 > I believe the goal should be that no application that is generating=20 > these messages for > their intended purposes should ever have to check the limit. The=20 > receiver SHOULD > (or perhaps even MUST) set it high enough so that *only* a=20 > malfunctioning or malicious > client could exceed it. >=20 > The flow control can be fairly simple to implement however,=20 > anytime the=20 > flow control > limit would have been hit, it is for an unanticipated message=20 > that the=20 > target iSCSI side > would be free to throw away anyway. The sender could simply do so=20 > immediately. > But I do think it would be better to define the credit limit=20 > such that=20 > it would be impossible > to violate it when generating messages legitimately. >=20 > The goal here is to enable the receiving side to identify a runaway=20 > malfunctioning or > malicious client that is attacking shared resources *before*=20 > it results=20 > in a legitimate > message from another client being dropped. >=20 > Even without SRQs, it is not that difficult for the receiving iSER=20 > layer to allow sufficient > extra receive buffers to deal with all *legitimate* iSCSI=20 > packets. It=20 > is also certainly > preferable for the receive side to have some extra buffers available=20 > rather than > requiring non-malfunctioning/non-abusive senders to check=20 > their credit=20 > limit > before sending any of the "unaccounted" iSCSI PDUs. These=20 > messages were > excluded from the normal iSCSI flow control precisely because=20 > they were=20 > not > anticipated to be high volume. >=20 > So there are two ways to determine when a connection is sending=20 > illegitimate > iSCSI packets: place a credit limit that legitimate=20 > applications will=20 > not violate > (even without thinking about it) or have the iSER layer=20 > examine packets=20 > at > wire speed. >=20 > > > > #4 is clearly expensive although we know that it can work. This=20 > > should be the last resort. > > > > #5 is a second-order choice in that it may still require help from=20 > > other options. The main > > considerations here are: making normative references to the=20 > Verbs spec=20 > > [ or defining the > > S-RQ text in abstract terms (i.e. stripping off references to WQE,=20 > > CQE, RQ, CQ, RI etc.) > > which again doesn't strike me as a "standards" way of defining a=20 > > protocol ], the acceptance > > (from an RNIC vendor perspective) on requiring it. We=20 > still need to =20 > > address a single > > connection case from the specification perspective (i.e.=20 > consider an=20 > > embedded target > > supporting a single iSCSI/iSER connection). >=20 > An SRQ does not eliminate the vulnerability. In fact it is necessary=20 > for it to exist. A runaway > client connected to a simple RQ can only tear down its own=20 > connection. =20 > It does not impact > other clients. >=20 > With an SRQ, you either have a per-connection limit/alarm or you are=20 > dependent on wire-speed > draining of completions by the ULP (the iSER layer). That case is=20 > essentially option 1, > enabling the per-connection limit/alarm requires having a=20 > number to set=20 > it with which > leads back to option 3 (or 3a). >=20 > An argument for 3, as opposed to 3a, is to specifically allow "None".=20 > But the iSER layer > should only uses "None" when it is committing to drain completions at=20 > wire speed *or* > that the pool of buffers used for untagged messages is not=20 > shared with=20 > any other iSCSI > session. >=20 > > > > #6 is also feasible (I think Julian too hinted at this in=20 > an email).=20 > > Clearly, an initiator sending too many immediate commands=20 > or a target=20 > > sending too many Async Messages is suspect. Problem though is that=20 > > this is a major functional difference between how an iSCSI system=20 > > would behave, and an iSCSI/iSER system would behave (thus=20 > potentially=20 > > exposing the dichotomy to layers further up in the stack). =20 > The other=20 > > issue is that a genuine exception could be punished with an even=20 > > bigger one - leading to termination of several I/Os. > > > > The way I see it, the current iSER spec depends on #2, #1,=20 > #5 and #6.=20 > > Note that while #2 > > alone is necessary and sufficient for operation without connection=20 > > drops, others work to > > minimize the probability of invoking graceful handling (#2) but=20 > > together ultimately allow > > a connection failure if the graceful handling service cannot be=20 > > maintained by the RNIC > > after several DDP exceptions. In the opinion of iSER authors, this=20 > > integrated approach is quite reasonable. > > > Option 2 is not a real solution. It is simply attempting to=20 > shove the=20 > excess messages into > buffers that are accounted for (either "system buffers" or by abusing=20 > the network itself as a > buffer forcing false retransmits). But ultimately it is just hoping=20 > that the problem goes away > because it does nothing to ensure that only the connection=20 > causing the=20 > problem will > have the corrective action taken against it. >=20 > > I realize that an argument can be made for #3 (which Caitlin & Alex=20 > > did as I understand the > > exchanges so far). More comments from others would help=20 > the authors=20 > > understand the WG position on this. > > >=20 > I believe the RDDP WG position on this is clear. In my=20 > opinion option,=20 > only options 1 and 3 > comply with the RDDP Security requirements for use of shared receive=20 > resources. >=20 >=20 > _______________________________________________ > Ips mailing list > Ips@ietf.org > https://www1.ietf.org/mailman/listinfo/ips >=20 _______________________________________________ Ips mailing list Ips@ietf.org https://www1.ietf.org/mailman/listinfo/ips From ips-bounces@ietf.org Thu Oct 7 11:38:22 2004 Received: from ietf-mx.ietf.org (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id LAA27545 for ; Thu, 7 Oct 2004 11:38:22 -0400 (EDT) Received: from megatron.ietf.org ([132.151.6.71]) by ietf-mx.ietf.org with esmtp (Exim 4.33) id 1CFaVI-0004Lb-1u for ips-web-archive@ietf.org; Thu, 07 Oct 2004 11:48:29 -0400 Received: from localhost.localdomain ([127.0.0.1] helo=megatron.ietf.org) by megatron.ietf.org with esmtp (Exim 4.32) id 1CFa1j-0007x7-KG; Thu, 07 Oct 2004 11:17:55 -0400 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by megatron.ietf.org with esmtp (Exim 4.32) id 1CFZtt-0005eI-EQ for ips@megatron.ietf.org; Thu, 07 Oct 2004 11:09:49 -0400 Received: from ietf-mx.ietf.org (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id LAA25022 for ; Thu, 7 Oct 2004 11:09:46 -0400 (EDT) Received: from mononoke.wasabisystems.com ([66.173.145.228]) by ietf-mx.ietf.org with esmtp (Exim 4.33) id 1CFa3c-0002v1-6q for ips@ietf.org; Thu, 07 Oct 2004 11:19:52 -0400 Received: from [192.168.1.125] (unknown [66.173.145.226]) by mononoke.wasabisystems.com (Postfix) with ESMTP id ED4A8870A2; Thu, 7 Oct 2004 11:09:44 -0400 (EDT) In-Reply-To: <41649B60.60002@rose.hp.com> References: <41649B60.60002@rose.hp.com> Mime-Version: 1.0 (Apple Message framework v619) Message-Id: From: William Studenmund Subject: Re: [Ips] resending Data-in with S bit set Date: Thu, 7 Oct 2004 11:09:30 -0400 To: "Mallikarjun C." X-Pgp-Agent: GPGMail 1.0 (v30, 10.3) X-Mailer: Apple Mail (2.619) X-Spam-Score: 0.0 (/) X-Scan-Signature: 14582b0692e7f70ce7111d04db3781c8 Cc: ips@ietf.org X-BeenThere: ips@ietf.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: IP Storage List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Content-Type: multipart/mixed; boundary="===============1728761075==" Sender: ips-bounces@ietf.org Errors-To: ips-bounces@ietf.org X-Spam-Score: 0.0 (/) X-Scan-Signature: 1676547e4f33b5e63227e9c02bd359e3 --===============1728761075== Content-Type: multipart/signed; protocol="application/pgp-signature"; micalg=pgp-sha1; boundary="Apple-Mail-25-742975501" --Apple-Mail-25-742975501 Content-Type: multipart/signed; micalg=sha1; boundary=Apple-Mail-24-742975496; protocol="application/pkcs7-signature" --Apple-Mail-24-742975496 Content-Type: text/plain; charset=US-ASCII; format=flowed Content-Transfer-Encoding: 7bit On Oct 6, 2004, at 9:26 PM, Mallikarjun C. wrote: > I agree with what Julian says below. The "exact replica" wording of > 10.16 is quite unambiguous. Just to play devil's advocate for a bit, what happens if the MaxRecvDataSegmentLength the initiator now advertises is so small that the original PDU may not be sent? Do we resend it anyway? Resend part? Note: the scenario is that the task completed, the initiator changed its advertised MaxRecvDataSegmentLength, then it sent a SNACK to retransmit status. Take care, Bill --Apple-Mail-24-742975496 Content-Type: application/pkcs7-signature; name=smime.p7s Content-Disposition: attachment; filename=smime.p7s Content-Transfer-Encoding: base64 MIAGCSqGSIb3DQEHAqCAMIACAQExCzAJBgUrDgMCGgUAMIAGCSqGSIb3DQEHAQAAoIIFfDCCBXgw ggNgoAMCAQICAUYwDQYJKoZIhvcNAQEEBQAwga0xCzAJBgNVBAYTAlVTMREwDwYDVQQIEwhOZXcg WW9yazERMA8GA1UEBxMITmV3IFlvcmsxHTAbBgNVBAoTFFdhc2FiaSBTeXN0ZW1zLCBJbmMuMQww CgYDVQQLEwNHSFExHzAdBgNVBAMTFldhc2FiaSBTeXN0ZW1zIFJvb3QgQ0ExKjAoBgkqhkiG9w0B CQEWG3dlYm1hc3RlckB3YXNhYmlzeXN0ZW1zLmNvbTAeFw0wMzA4MTExODU2NDZaFw0wODA2MjUx ODU2NDZaMIGXMQswCQYDVQQGEwJVUzETMBEGA1UECBMKQ2FsaWZvcm5pYTESMBAGA1UEBxMJU2Fu IERpZWdvMRcwFQYDVQQKEw5XYXNhYmkgU3lzdGVtczEbMBkGA1UEAxMSV2lsbGlhbSBTdHVkZW5t dW5kMSkwJwYJKoZIhvcNAQkBFhp3cnN0dWRlbkB3YXNhYmlzeXN0ZW1zLmNvbTCBnzANBgkqhkiG 9w0BAQEFAAOBjQAwgYkCgYEA9CZhg4GVWYTRGcJz8lU1ipzInlBuGqfZQs88Z1rTP7+c3AyhKwpE TyTmK4UhxslzKa93YLqifLxZKYt1jI6FNq40IQWCKBVywWBwmL5guCu0851f9ywH5Uj2wJdiDrOI sOJKNpwNqml8bFdTrFS2qvyCxTIWVMLYAzUBBcl2F7MCAwEAAaOCATkwggE1MAkGA1UdEwQCMAAw LAYJYIZIAYb4QgENBB8WHU9wZW5TU0wgR2VuZXJhdGVkIENlcnRpZmljYXRlMB0GA1UdDgQWBBTt mjHbSBlMV48RWOBETM3EV0Tm/jCB2gYDVR0jBIHSMIHPgBQrBOnN88CNCHPaV5wLS212tcrgBaGB s6SBsDCBrTELMAkGA1UEBhMCVVMxETAPBgNVBAgTCE5ldyBZb3JrMREwDwYDVQQHEwhOZXcgWW9y azEdMBsGA1UEChMUV2FzYWJpIFN5c3RlbXMsIEluYy4xDDAKBgNVBAsTA0dIUTEfMB0GA1UEAxMW V2FzYWJpIFN5c3RlbXMgUm9vdCBDQTEqMCgGCSqGSIb3DQEJARYbd2VibWFzdGVyQHdhc2FiaXN5 c3RlbXMuY29tggEAMA0GCSqGSIb3DQEBBAUAA4ICAQCpxoPEYunR7qiXyycEZd0UIatSt/WalMoH nuN9S421iS/sD1wSDqzHyM5YyUWn4z4o6VhT2kD9UV4lk5aXxxefEfGzFcR7Xyah7tO3ziuReFfx qUP5+DTlwDuUBc8Iow6NwY9GlRCKFSsEroLWLGODyYWgN1ojjKOtcXRiOZkYTP3VfDUV2/Xrhh9O l6+3Otf069+KWeOWedWeMsbe1rac96yPEjPogToWkBXVJEeEQktd0BLETt7GUuSZg2Fho/nslqZv nnnKZmrOfdFy9iXDL9T0Hp/OHf5B9RP+r00BJJFO8JzBPIL2IA3CBPApOQty5A00Jg1CGfDQxiV3 s8G0aRUbmFqg9r0OfhPYjJ8BHfw1HHWb7Cu91Sw+d5UnYracZhsR2Gc5H5CnfS6IZhc7ulfnSX8f I4BXR4vbeRUG76J3rkkbkO1haYHLk5w5RlICHsJOBSElG9ggLBWNIfUZwAzo84lfmDmJMP1YCOmi DFCbW1DgxhqVFpYEf5Pq2pVcCguopK2G9oZjH5fxq6ehisk9dN+7NEQ4OSqC43ehaStYOoQAJruN e6e5tCtRMx7bMI7eVPiHxE9/Y38Q+sc5oRN9mCyyAXMweuMRNHAHAWdFB7NhHD6gBa2lh91zIMdf iSVRlBXoMrBKJX6VecgnhvUp4G6QAnN7EMu6QHlc/jGCA0swggNHAgEBMIGzMIGtMQswCQYDVQQG EwJVUzERMA8GA1UECBMITmV3IFlvcmsxETAPBgNVBAcTCE5ldyBZb3JrMR0wGwYDVQQKExRXYXNh YmkgU3lzdGVtcywgSW5jLjEMMAoGA1UECxMDR0hRMR8wHQYDVQQDExZXYXNhYmkgU3lzdGVtcyBS b290IENBMSowKAYJKoZIhvcNAQkBFht3ZWJtYXN0ZXJAd2FzYWJpc3lzdGVtcy5jb20CAUYwCQYF Kw4DAhoFAKCCAe0wGAYJKoZIhvcNAQkDMQsGCSqGSIb3DQEHATAcBgkqhkiG9w0BCQUxDxcNMDQx MDA3MTUwOTMwWjAjBgkqhkiG9w0BCQQxFgQUyCY0Hyc4Nq09KG8FK/A5Fkvrjr4wgcQGCSsGAQQB gjcQBDGBtjCBszCBrTELMAkGA1UEBhMCVVMxETAPBgNVBAgTCE5ldyBZb3JrMREwDwYDVQQHEwhO ZXcgWW9yazEdMBsGA1UEChMUV2FzYWJpIFN5c3RlbXMsIEluYy4xDDAKBgNVBAsTA0dIUTEfMB0G A1UEAxMWV2FzYWJpIFN5c3RlbXMgUm9vdCBDQTEqMCgGCSqGSIb3DQEJARYbd2VibWFzdGVyQHdh c2FiaXN5c3RlbXMuY29tAgFGMIHGBgsqhkiG9w0BCRACCzGBtqCBszCBrTELMAkGA1UEBhMCVVMx ETAPBgNVBAgTCE5ldyBZb3JrMREwDwYDVQQHEwhOZXcgWW9yazEdMBsGA1UEChMUV2FzYWJpIFN5 c3RlbXMsIEluYy4xDDAKBgNVBAsTA0dIUTEfMB0GA1UEAxMWV2FzYWJpIFN5c3RlbXMgUm9vdCBD QTEqMCgGCSqGSIb3DQEJARYbd2VibWFzdGVyQHdhc2FiaXN5c3RlbXMuY29tAgFGMA0GCSqGSIb3 DQEBAQUABIGApef2yzFwRfALqk7693yAYxYM+Qq2NyaClkB3DFsIcqlgacN7qD+BSBGpEV0maE+p qfWiAWSTcTNQqBMPPCNwnaA9kzd7p0BNM1bUOwg4bbXdLwKWeMCT+RnSKjWipn1/f6jsxuIzgwCo koNVHrtUMH1rxdgEos0E3kIjJTZYK5sAAAAAAAA= --Apple-Mail-24-742975496-- --Apple-Mail-25-742975501 content-type: application/pgp-signature; x-mac-type=70674453; name=PGP.sig content-description: This is a digitally signed message part content-disposition: inline; filename=PGP.sig content-transfer-encoding: 7bit Content-Transfer-Encoding: 7bit Content-Transfer-Encoding: 7bit -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.2.3 (Darwin) iD8DBQFBZVw2DJT2Egh26K0RAtyCAKCHTOuhW0RiyiAB+SBHQCEBlp4WPgCeIvYR 3DFxnrGUO7YLRDaqCmYMy6o= =6k26 -----END PGP SIGNATURE----- --Apple-Mail-25-742975501-- --===============1728761075== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline Content-Transfer-Encoding: 7bit _______________________________________________ Ips mailing list Ips@ietf.org https://www1.ietf.org/mailman/listinfo/ips --===============1728761075==-- From ips-bounces@ietf.org Thu Oct 7 13:28:07 2004 Received: from ietf-mx.ietf.org (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id NAA06752 for ; Thu, 7 Oct 2004 13:28:07 -0400 (EDT) Received: from megatron.ietf.org ([132.151.6.71]) by ietf-mx.ietf.org with esmtp (Exim 4.33) id 1CFcDW-0003BV-1h for ips-web-archive@ietf.org; Thu, 07 Oct 2004 13:38:15 -0400 Received: from localhost.localdomain ([127.0.0.1] helo=megatron.ietf.org) by megatron.ietf.org with esmtp (Exim 4.32) id 1CFbrM-0004Qx-1V; Thu, 07 Oct 2004 13:15:20 -0400 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by megatron.ietf.org with esmtp (Exim 4.32) id 1CFbmK-0003OO-Fu for ips@megatron.ietf.org; Thu, 07 Oct 2004 13:10:13 -0400 Received: from ietf-mx.ietf.org (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id NAA05703 for ; Thu, 7 Oct 2004 13:10:04 -0400 (EDT) Received: from palrel12.hp.com ([156.153.255.237]) by ietf-mx.ietf.org with esmtp (Exim 4.33) id 1CFbvu-00026t-Aj for ips@ietf.org; Thu, 07 Oct 2004 13:20:12 -0400 Received: from rosemail.rose.hp.com (rosemail.rose.hp.com [15.43.209.160]) by palrel12.hp.com (Postfix) with ESMTP id DC38C410D20; Thu, 7 Oct 2004 10:09:55 -0700 (PDT) Received: from [127.0.0.1] (palnai12-1853.corp.hp.com [15.244.199.61]) by rosemail.rose.hp.com (Postfix) with ESMTP id 718F17FFB; Thu, 7 Oct 2004 10:09:55 -0700 (PDT) Message-ID: <4165785D.5050802@rose.hp.com> Date: Thu, 07 Oct 2004 10:09:49 -0700 From: "Mallikarjun C." User-Agent: Mozilla Thunderbird 0.7.3 (Windows/20040803) X-Accept-Language: en-us, en MIME-Version: 1.0 To: William Studenmund Subject: Re: [Ips] resending Data-in with S bit set References: <41649B60.60002@rose.hp.com> In-Reply-To: Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit X-Spam-Score: 0.0 (/) X-Scan-Signature: 0bc60ec82efc80c84b8d02f4b0e4de22 Content-Transfer-Encoding: 7bit Cc: ips@ietf.org X-BeenThere: ips@ietf.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: IP Storage List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Sender: ips-bounces@ietf.org Errors-To: ips-bounces@ietf.org X-Spam-Score: 0.0 (/) X-Scan-Signature: b19722fc8d3865b147c75ae2495625f2 Content-Transfer-Encoding: 7bit Bill, AFAICT, you are describing the same scenario currently covered by the R-Data SNACK (10.16.3). I'd like to know more details if that's not the case. M William Studenmund wrote: > On Oct 6, 2004, at 9:26 PM, Mallikarjun C. wrote: > >> I agree with what Julian says below. The "exact replica" wording of >> 10.16 is quite unambiguous. > > > Just to play devil's advocate for a bit, what happens if the > MaxRecvDataSegmentLength the initiator now advertises is so small that > the original PDU may not be sent? Do we resend it anyway? Resend part? > > Note: the scenario is that the task completed, the initiator changed its > advertised MaxRecvDataSegmentLength, then it sent a SNACK to retransmit > status. > > Take care, > > Bill _______________________________________________ Ips mailing list Ips@ietf.org https://www1.ietf.org/mailman/listinfo/ips From ips-bounces@ietf.org Thu Oct 7 22:25:31 2004 Received: from ietf-mx.ietf.org (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id WAA27691 for ; Thu, 7 Oct 2004 22:25:31 -0400 (EDT) Received: from megatron.ietf.org ([132.151.6.71]) by ietf-mx.ietf.org with esmtp (Exim 4.33) id 1CFkbf-00088v-9p for ips-web-archive@ietf.org; Thu, 07 Oct 2004 22:35:43 -0400 Received: from localhost.localdomain ([127.0.0.1] helo=megatron.ietf.org) by megatron.ietf.org with esmtp (Exim 4.32) id 1CFkLl-0000wI-MO; Thu, 07 Oct 2004 22:19:17 -0400 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by megatron.ietf.org with esmtp (Exim 4.32) id 1CFkHn-0008PP-6u for ips@megatron.ietf.org; Thu, 07 Oct 2004 22:15:11 -0400 Received: from ietf-mx.ietf.org (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id WAA27117 for ; Thu, 7 Oct 2004 22:15:07 -0400 (EDT) Received: from mononoke.wasabisystems.com ([66.173.145.228]) by ietf-mx.ietf.org with esmtp (Exim 4.33) id 1CFkRb-0007bn-5V for ips@ietf.org; Thu, 07 Oct 2004 22:25:19 -0400 Received: from [192.168.1.125] (unknown [66.173.145.226]) by mononoke.wasabisystems.com (Postfix) with ESMTP id 48856870CA; Thu, 7 Oct 2004 22:15:08 -0400 (EDT) In-Reply-To: <4165785D.5050802@rose.hp.com> References: <41649B60.60002@rose.hp.com> <4165785D.5050802@rose.hp.com> Mime-Version: 1.0 (Apple Message framework v619) Message-Id: From: William Studenmund Subject: Re: [Ips] resending Data-in with S bit set Date: Thu, 7 Oct 2004 22:14:55 -0400 To: "Mallikarjun C." X-Pgp-Agent: GPGMail 1.0 (v30, 10.3) X-Mailer: Apple Mail (2.619) X-Spam-Score: 0.0 (/) X-Scan-Signature: 944ecb6e61f753561f559a497458fb4f Cc: ips@ietf.org X-BeenThere: ips@ietf.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: IP Storage List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Content-Type: multipart/mixed; boundary="===============1670550760==" Sender: ips-bounces@ietf.org Errors-To: ips-bounces@ietf.org X-Spam-Score: 0.0 (/) X-Scan-Signature: 8fbbaa16f9fd29df280814cb95ae2290 --===============1670550760== Content-Type: multipart/signed; protocol="application/pgp-signature"; micalg=pgp-sha1; boundary="Apple-Mail-46-782901255" --Apple-Mail-46-782901255 Content-Type: multipart/signed; micalg=sha1; boundary=Apple-Mail-45-782901248; protocol="application/pkcs7-signature" --Apple-Mail-45-782901248 Content-Type: text/plain; charset=US-ASCII; format=flowed Content-Transfer-Encoding: 7bit On Oct 7, 2004, at 1:09 PM, Mallikarjun C. wrote: > Bill, > > AFAICT, you are describing the same scenario currently covered by the > R-Data SNACK (10.16.3). I'd like to know more details if that's not > the case. I agree 10.16.3 covers it, now that I re-read it; I'd missed the status considerations in R-Data SNACK. However the case I had in mind is a slightly-different side of the scenario, where we have resegmentation but the initiator sends a Status SNACK. For an R-Data SNACK, I get that the status now has to be transmitted in a SCSI Response PDU; no phase collapsed read here. And if multiple R-Data SNACKs are sent for a task, the SCSI Response has to be sent with the latest SNACK Tag. What I don't see discussed is what you do if you (the target) get a Status SNACK after resegmentation (no R-Data SNACK). Do you send a SCSI Response for the task? I think that'd be the best thing. Then the initiator can discover that it's missing data and send an R-Data SNACK. Obviously since it lost the status, it didn't get the data, so it shouldn't acknowledge the status. :-) Take care, Bill --Apple-Mail-45-782901248 Content-Type: application/pkcs7-signature; name=smime.p7s Content-Disposition: attachment; filename=smime.p7s Content-Transfer-Encoding: base64 MIAGCSqGSIb3DQEHAqCAMIACAQExCzAJBgUrDgMCGgUAMIAGCSqGSIb3DQEHAQAAoIIFfDCCBXgw ggNgoAMCAQICAUYwDQYJKoZIhvcNAQEEBQAwga0xCzAJBgNVBAYTAlVTMREwDwYDVQQIEwhOZXcg WW9yazERMA8GA1UEBxMITmV3IFlvcmsxHTAbBgNVBAoTFFdhc2FiaSBTeXN0ZW1zLCBJbmMuMQww CgYDVQQLEwNHSFExHzAdBgNVBAMTFldhc2FiaSBTeXN0ZW1zIFJvb3QgQ0ExKjAoBgkqhkiG9w0B CQEWG3dlYm1hc3RlckB3YXNhYmlzeXN0ZW1zLmNvbTAeFw0wMzA4MTExODU2NDZaFw0wODA2MjUx ODU2NDZaMIGXMQswCQYDVQQGEwJVUzETMBEGA1UECBMKQ2FsaWZvcm5pYTESMBAGA1UEBxMJU2Fu IERpZWdvMRcwFQYDVQQKEw5XYXNhYmkgU3lzdGVtczEbMBkGA1UEAxMSV2lsbGlhbSBTdHVkZW5t dW5kMSkwJwYJKoZIhvcNAQkBFhp3cnN0dWRlbkB3YXNhYmlzeXN0ZW1zLmNvbTCBnzANBgkqhkiG 9w0BAQEFAAOBjQAwgYkCgYEA9CZhg4GVWYTRGcJz8lU1ipzInlBuGqfZQs88Z1rTP7+c3AyhKwpE TyTmK4UhxslzKa93YLqifLxZKYt1jI6FNq40IQWCKBVywWBwmL5guCu0851f9ywH5Uj2wJdiDrOI sOJKNpwNqml8bFdTrFS2qvyCxTIWVMLYAzUBBcl2F7MCAwEAAaOCATkwggE1MAkGA1UdEwQCMAAw LAYJYIZIAYb4QgENBB8WHU9wZW5TU0wgR2VuZXJhdGVkIENlcnRpZmljYXRlMB0GA1UdDgQWBBTt mjHbSBlMV48RWOBETM3EV0Tm/jCB2gYDVR0jBIHSMIHPgBQrBOnN88CNCHPaV5wLS212tcrgBaGB s6SBsDCBrTELMAkGA1UEBhMCVVMxETAPBgNVBAgTCE5ldyBZb3JrMREwDwYDVQQHEwhOZXcgWW9y azEdMBsGA1UEChMUV2FzYWJpIFN5c3RlbXMsIEluYy4xDDAKBgNVBAsTA0dIUTEfMB0GA1UEAxMW V2FzYWJpIFN5c3RlbXMgUm9vdCBDQTEqMCgGCSqGSIb3DQEJARYbd2VibWFzdGVyQHdhc2FiaXN5 c3RlbXMuY29tggEAMA0GCSqGSIb3DQEBBAUAA4ICAQCpxoPEYunR7qiXyycEZd0UIatSt/WalMoH nuN9S421iS/sD1wSDqzHyM5YyUWn4z4o6VhT2kD9UV4lk5aXxxefEfGzFcR7Xyah7tO3ziuReFfx qUP5+DTlwDuUBc8Iow6NwY9GlRCKFSsEroLWLGODyYWgN1ojjKOtcXRiOZkYTP3VfDUV2/Xrhh9O l6+3Otf069+KWeOWedWeMsbe1rac96yPEjPogToWkBXVJEeEQktd0BLETt7GUuSZg2Fho/nslqZv nnnKZmrOfdFy9iXDL9T0Hp/OHf5B9RP+r00BJJFO8JzBPIL2IA3CBPApOQty5A00Jg1CGfDQxiV3 s8G0aRUbmFqg9r0OfhPYjJ8BHfw1HHWb7Cu91Sw+d5UnYracZhsR2Gc5H5CnfS6IZhc7ulfnSX8f I4BXR4vbeRUG76J3rkkbkO1haYHLk5w5RlICHsJOBSElG9ggLBWNIfUZwAzo84lfmDmJMP1YCOmi DFCbW1DgxhqVFpYEf5Pq2pVcCguopK2G9oZjH5fxq6ehisk9dN+7NEQ4OSqC43ehaStYOoQAJruN e6e5tCtRMx7bMI7eVPiHxE9/Y38Q+sc5oRN9mCyyAXMweuMRNHAHAWdFB7NhHD6gBa2lh91zIMdf iSVRlBXoMrBKJX6VecgnhvUp4G6QAnN7EMu6QHlc/jGCA0swggNHAgEBMIGzMIGtMQswCQYDVQQG EwJVUzERMA8GA1UECBMITmV3IFlvcmsxETAPBgNVBAcTCE5ldyBZb3JrMR0wGwYDVQQKExRXYXNh YmkgU3lzdGVtcywgSW5jLjEMMAoGA1UECxMDR0hRMR8wHQYDVQQDExZXYXNhYmkgU3lzdGVtcyBS b290IENBMSowKAYJKoZIhvcNAQkBFht3ZWJtYXN0ZXJAd2FzYWJpc3lzdGVtcy5jb20CAUYwCQYF Kw4DAhoFAKCCAe0wGAYJKoZIhvcNAQkDMQsGCSqGSIb3DQEHATAcBgkqhkiG9w0BCQUxDxcNMDQx MDA4MDIxNDU2WjAjBgkqhkiG9w0BCQQxFgQUhArRx1G6hOZ5HG3YfhsSxMNwz8swgcQGCSsGAQQB gjcQBDGBtjCBszCBrTELMAkGA1UEBhMCVVMxETAPBgNVBAgTCE5ldyBZb3JrMREwDwYDVQQHEwhO ZXcgWW9yazEdMBsGA1UEChMUV2FzYWJpIFN5c3RlbXMsIEluYy4xDDAKBgNVBAsTA0dIUTEfMB0G A1UEAxMWV2FzYWJpIFN5c3RlbXMgUm9vdCBDQTEqMCgGCSqGSIb3DQEJARYbd2VibWFzdGVyQHdh c2FiaXN5c3RlbXMuY29tAgFGMIHGBgsqhkiG9w0BCRACCzGBtqCBszCBrTELMAkGA1UEBhMCVVMx ETAPBgNVBAgTCE5ldyBZb3JrMREwDwYDVQQHEwhOZXcgWW9yazEdMBsGA1UEChMUV2FzYWJpIFN5 c3RlbXMsIEluYy4xDDAKBgNVBAsTA0dIUTEfMB0GA1UEAxMWV2FzYWJpIFN5c3RlbXMgUm9vdCBD QTEqMCgGCSqGSIb3DQEJARYbd2VibWFzdGVyQHdhc2FiaXN5c3RlbXMuY29tAgFGMA0GCSqGSIb3 DQEBAQUABIGAFhJMRACjPMwKc6B4m0SZpmMIks8P1FGDLdhVlWcHPDAYR4KR8dIUwsEnV4TDtOYH rAPpyl2cyVqPpCXp6JL3DyEUEbht9ztmSOMDZIlmmvuYfvT/F7ytTC8IWjTpTmaLWRVuDXZVbMdc w2I38iEO6i/Fyrv9JPrNX7URD/1PjXMAAAAAAAA= --Apple-Mail-45-782901248-- --Apple-Mail-46-782901255 content-type: application/pgp-signature; x-mac-type=70674453; name=PGP.sig content-description: This is a digitally signed message part content-disposition: inline; filename=PGP.sig content-transfer-encoding: 7bit Content-Transfer-Encoding: 7bit Content-Transfer-Encoding: 7bit -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.2.3 (Darwin) iD8DBQFBZfgrDJT2Egh26K0RAhApAJ41m37CUG9U0POArPyDk8MW4YJkUACgmWjG yhM479lc9xGzI8pXa/eRdl8= =cibh -----END PGP SIGNATURE----- --Apple-Mail-46-782901255-- --===============1670550760== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline Content-Transfer-Encoding: 7bit _______________________________________________ Ips mailing list Ips@ietf.org https://www1.ietf.org/mailman/listinfo/ips --===============1670550760==-- From ips-bounces@ietf.org Fri Oct 8 13:55:07 2004 Received: from ietf-mx.ietf.org (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id NAA15793 for ; Fri, 8 Oct 2004 13:55:07 -0400 (EDT) Received: from megatron.ietf.org ([132.151.6.71]) by ietf-mx.ietf.org with esmtp (Exim 4.33) id 1CFz7O-0004Pf-01 for ips-web-archive@ietf.org; Fri, 08 Oct 2004 14:05:26 -0400 Received: from localhost.localdomain ([127.0.0.1] helo=megatron.ietf.org) by megatron.ietf.org with esmtp (Exim 4.32) id 1CFyu0-00025j-Jn; Fri, 08 Oct 2004 13:51:36 -0400 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by megatron.ietf.org with esmtp (Exim 4.32) id 1CFyt0-0001mi-47 for ips@megatron.ietf.org; Fri, 08 Oct 2004 13:50:34 -0400 Received: from ietf-mx.ietf.org (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id NAA15343 for ; Fri, 8 Oct 2004 13:50:34 -0400 (EDT) Received: from atlrel8.hp.com ([156.153.255.206]) by ietf-mx.ietf.org with esmtp (Exim 4.33) id 1CFz2w-0004Hl-KX for ips@ietf.org; Fri, 08 Oct 2004 14:00:52 -0400 Received: from rosemail.rose.hp.com (rosemail.rose.hp.com [15.43.209.160]) by atlrel8.hp.com (Postfix) with ESMTP id DD20F5965 for ; Fri, 8 Oct 2004 13:50:17 -0400 (EDT) Received: from [127.0.0.1] (dhcp-roseor-beb159.rose.hp.com [15.23.141.159]) by rosemail.rose.hp.com (Postfix) with ESMTP id 96C2A8056 for ; Fri, 8 Oct 2004 10:50:17 -0700 (PDT) Message-ID: <4166D357.40708@rose.hp.com> Date: Fri, 08 Oct 2004 10:50:15 -0700 From: "Mallikarjun C." User-Agent: Mozilla Thunderbird 0.7.3 (Windows/20040803) X-Accept-Language: en-us, en MIME-Version: 1.0 To: ips@ietf.org Subject: Re: [Ips] resending Data-in with S bit set References: <41649B60.60002@rose.hp.com> <4165785D.5050802@rose.hp.com> In-Reply-To: Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit X-Spam-Score: 0.0 (/) X-Scan-Signature: 0a7aa2e6e558383d84476dc338324fab Content-Transfer-Encoding: 7bit X-BeenThere: ips@ietf.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: IP Storage List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Sender: ips-bounces@ietf.org Errors-To: ips-bounces@ietf.org X-Spam-Score: 0.0 (/) X-Scan-Signature: 244a2fd369eaf00ce6820a760a3de2e8 Content-Transfer-Encoding: 7bit >what you do if you (the target) get a > Status SNACK after resegmentation (no R-Data SNACK). Do you send a SCSI > Response for the task? 10.16.3 says the following in part: "....if it has already sent a response containing another value in the SNACK Tag field or had the status included in the last Data-In PDU, it must send a new SCSI Response PDU. .... If an initiator attempts to recover a lost SCSI Response (with a Status SNACK, see Section 10.16.1 Type) when more than one response has been sent, the target will send the SCSI Response with the latest content known to the target, including the last SNACK Tag for the command." Does that text address the scenario you've in mind? Mallikarjun William Studenmund wrote: > On Oct 7, 2004, at 1:09 PM, Mallikarjun C. wrote: > >> Bill, >> >> AFAICT, you are describing the same scenario currently covered by the >> R-Data SNACK (10.16.3). I'd like to know more details if that's not >> the case. > > > I agree 10.16.3 covers it, now that I re-read it; I'd missed the status > considerations in R-Data SNACK. However the case I had in mind is a > slightly-different side of the scenario, where we have resegmentation > but the initiator sends a Status SNACK. > > For an R-Data SNACK, I get that the status now has to be transmitted in > a SCSI Response PDU; no phase collapsed read here. And if multiple > R-Data SNACKs are sent for a task, the SCSI Response has to be sent with > the latest SNACK Tag. > > What I don't see discussed is what you do if you (the target) get a > Status SNACK after resegmentation (no R-Data SNACK). Do you send a SCSI > Response for the task? I think that'd be the best thing. Then the > initiator can discover that it's missing data and send an R-Data SNACK. > Obviously since it lost the status, it didn't get the data, so it > shouldn't acknowledge the status. :-) > > Take care, > > Bill > _______________________________________________ Ips mailing list Ips@ietf.org https://www1.ietf.org/mailman/listinfo/ips From ips-bounces@ietf.org Fri Oct 8 14:38:06 2004 Received: from ietf-mx.ietf.org (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id OAA19883 for ; Fri, 8 Oct 2004 14:38:06 -0400 (EDT) Received: from megatron.ietf.org ([132.151.6.71]) by ietf-mx.ietf.org with esmtp (Exim 4.33) id 1CFzmy-0005WE-BL for ips-web-archive@ietf.org; Fri, 08 Oct 2004 14:48:25 -0400 Received: from localhost.localdomain ([127.0.0.1] helo=megatron.ietf.org) by megatron.ietf.org with esmtp (Exim 4.32) id 1CFzVK-0004ni-Eh; Fri, 08 Oct 2004 14:30:10 -0400 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by megatron.ietf.org with esmtp (Exim 4.32) id 1CFzIp-0001tl-N1 for ips@megatron.ietf.org; Fri, 08 Oct 2004 14:17:15 -0400 Received: from ietf-mx.ietf.org (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id OAA17897 for ; Fri, 8 Oct 2004 14:17:16 -0400 (EDT) Received: from mononoke.wasabisystems.com ([66.173.145.228]) by ietf-mx.ietf.org with esmtp (Exim 4.33) id 1CFzSp-0004y6-0F for ips@ietf.org; Fri, 08 Oct 2004 14:27:35 -0400 Received: from [192.168.1.125] (unknown [66.173.145.226]) by mononoke.wasabisystems.com (Postfix) with ESMTP id 0D8A4870B6; Fri, 8 Oct 2004 14:17:15 -0400 (EDT) In-Reply-To: <4166D357.40708@rose.hp.com> References: <41649B60.60002@rose.hp.com> <4165785D.5050802@rose.hp.com> <4166D357.40708@rose.hp.com> Mime-Version: 1.0 (Apple Message framework v619) Message-Id: <4489F857-1956-11D9-91D4-000A95D14BEC@wasabisystems.com> From: William Studenmund Subject: Re: [Ips] resending Data-in with S bit set Date: Fri, 8 Oct 2004 14:17:03 -0400 To: "Mallikarjun C." X-Pgp-Agent: GPGMail 1.0 (v30, 10.3) X-Mailer: Apple Mail (2.619) X-Spam-Score: 0.0 (/) X-Scan-Signature: b2809b6f39decc6de467dcf252f42af1 Cc: ips@ietf.org X-BeenThere: ips@ietf.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: IP Storage List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Content-Type: multipart/mixed; boundary="===============0723716773==" Sender: ips-bounces@ietf.org Errors-To: ips-bounces@ietf.org X-Spam-Score: 0.0 (/) X-Scan-Signature: 21bf7a2f1643ae0bf20c1e010766eb78 --===============0723716773== Content-Type: multipart/signed; protocol="application/pgp-signature"; micalg=pgp-sha1; boundary="Apple-Mail-54-840628611" --Apple-Mail-54-840628611 Content-Type: multipart/signed; micalg=sha1; boundary=Apple-Mail-53-840628606; protocol="application/pkcs7-signature" --Apple-Mail-53-840628606 Content-Type: text/plain; charset=US-ASCII; format=flowed Content-Transfer-Encoding: 7bit On Oct 8, 2004, at 1:50 PM, Mallikarjun C. wrote: > >what you do if you (the target) get a > > Status SNACK after resegmentation (no R-Data SNACK). Do you send a > SCSI > > Response for the task? > > > > 10.16.3 says the following in part: > > "....if it has already sent a response containing another value > in the SNACK Tag field or had the status included in the last Data-In > PDU, it must send a new SCSI Response PDU. > .... > If an > initiator attempts to recover a lost SCSI Response (with a > Status SNACK, see Section 10.16.1 Type) when more than one response > has been sent, the target will send the SCSI Response with the latest > content known to the target, including the last SNACK Tag for the > command." > > > Does that text address the scenario you've in mind? Almost. I'm thinking about the case where we have sent only one response (thus not "more than one"). We then get a Status SNACK, and we can't retransmit the status as it was phase-collapsed in with a Data-In that is too big to send now. Since we haven't sent more than one status, I don't think the text speaks directly to this case. I think the spirit of the text, send a SCSI Command Response PDU, is the right thing to do. I'm just not sure if the text tells me that's what I should do. One feeling I get is that the text in this section was written with the assumption that the first SNACK that will be sent for a task after resegmentation will be an R-Data SNACK. For that sequence, I think the text works well. This case is where the first SNACK for a task is a Status SNACK, as the initiator didn't know what task corresponded to a given StatSN value. i.e. the initiator noticed a hole in StatSN clearing, and went to inquire about the hole. This case isn't covered so well. Thank you for continuing to help me with this question. Take care, Bill --Apple-Mail-53-840628606 Content-Type: application/pkcs7-signature; name=smime.p7s Content-Disposition: attachment; filename=smime.p7s Content-Transfer-Encoding: base64 MIAGCSqGSIb3DQEHAqCAMIACAQExCzAJBgUrDgMCGgUAMIAGCSqGSIb3DQEHAQAAoIIFfDCCBXgw ggNgoAMCAQICAUYwDQYJKoZIhvcNAQEEBQAwga0xCzAJBgNVBAYTAlVTMREwDwYDVQQIEwhOZXcg WW9yazERMA8GA1UEBxMITmV3IFlvcmsxHTAbBgNVBAoTFFdhc2FiaSBTeXN0ZW1zLCBJbmMuMQww CgYDVQQLEwNHSFExHzAdBgNVBAMTFldhc2FiaSBTeXN0ZW1zIFJvb3QgQ0ExKjAoBgkqhkiG9w0B CQEWG3dlYm1hc3RlckB3YXNhYmlzeXN0ZW1zLmNvbTAeFw0wMzA4MTExODU2NDZaFw0wODA2MjUx ODU2NDZaMIGXMQswCQYDVQQGEwJVUzETMBEGA1UECBMKQ2FsaWZvcm5pYTESMBAGA1UEBxMJU2Fu IERpZWdvMRcwFQYDVQQKEw5XYXNhYmkgU3lzdGVtczEbMBkGA1UEAxMSV2lsbGlhbSBTdHVkZW5t dW5kMSkwJwYJKoZIhvcNAQkBFhp3cnN0dWRlbkB3YXNhYmlzeXN0ZW1zLmNvbTCBnzANBgkqhkiG 9w0BAQEFAAOBjQAwgYkCgYEA9CZhg4GVWYTRGcJz8lU1ipzInlBuGqfZQs88Z1rTP7+c3AyhKwpE TyTmK4UhxslzKa93YLqifLxZKYt1jI6FNq40IQWCKBVywWBwmL5guCu0851f9ywH5Uj2wJdiDrOI sOJKNpwNqml8bFdTrFS2qvyCxTIWVMLYAzUBBcl2F7MCAwEAAaOCATkwggE1MAkGA1UdEwQCMAAw LAYJYIZIAYb4QgENBB8WHU9wZW5TU0wgR2VuZXJhdGVkIENlcnRpZmljYXRlMB0GA1UdDgQWBBTt mjHbSBlMV48RWOBETM3EV0Tm/jCB2gYDVR0jBIHSMIHPgBQrBOnN88CNCHPaV5wLS212tcrgBaGB s6SBsDCBrTELMAkGA1UEBhMCVVMxETAPBgNVBAgTCE5ldyBZb3JrMREwDwYDVQQHEwhOZXcgWW9y azEdMBsGA1UEChMUV2FzYWJpIFN5c3RlbXMsIEluYy4xDDAKBgNVBAsTA0dIUTEfMB0GA1UEAxMW V2FzYWJpIFN5c3RlbXMgUm9vdCBDQTEqMCgGCSqGSIb3DQEJARYbd2VibWFzdGVyQHdhc2FiaXN5 c3RlbXMuY29tggEAMA0GCSqGSIb3DQEBBAUAA4ICAQCpxoPEYunR7qiXyycEZd0UIatSt/WalMoH nuN9S421iS/sD1wSDqzHyM5YyUWn4z4o6VhT2kD9UV4lk5aXxxefEfGzFcR7Xyah7tO3ziuReFfx qUP5+DTlwDuUBc8Iow6NwY9GlRCKFSsEroLWLGODyYWgN1ojjKOtcXRiOZkYTP3VfDUV2/Xrhh9O l6+3Otf069+KWeOWedWeMsbe1rac96yPEjPogToWkBXVJEeEQktd0BLETt7GUuSZg2Fho/nslqZv nnnKZmrOfdFy9iXDL9T0Hp/OHf5B9RP+r00BJJFO8JzBPIL2IA3CBPApOQty5A00Jg1CGfDQxiV3 s8G0aRUbmFqg9r0OfhPYjJ8BHfw1HHWb7Cu91Sw+d5UnYracZhsR2Gc5H5CnfS6IZhc7ulfnSX8f I4BXR4vbeRUG76J3rkkbkO1haYHLk5w5RlICHsJOBSElG9ggLBWNIfUZwAzo84lfmDmJMP1YCOmi DFCbW1DgxhqVFpYEf5Pq2pVcCguopK2G9oZjH5fxq6ehisk9dN+7NEQ4OSqC43ehaStYOoQAJruN e6e5tCtRMx7bMI7eVPiHxE9/Y38Q+sc5oRN9mCyyAXMweuMRNHAHAWdFB7NhHD6gBa2lh91zIMdf iSVRlBXoMrBKJX6VecgnhvUp4G6QAnN7EMu6QHlc/jGCA0swggNHAgEBMIGzMIGtMQswCQYDVQQG EwJVUzERMA8GA1UECBMITmV3IFlvcmsxETAPBgNVBAcTCE5ldyBZb3JrMR0wGwYDVQQKExRXYXNh YmkgU3lzdGVtcywgSW5jLjEMMAoGA1UECxMDR0hRMR8wHQYDVQQDExZXYXNhYmkgU3lzdGVtcyBS b290IENBMSowKAYJKoZIhvcNAQkBFht3ZWJtYXN0ZXJAd2FzYWJpc3lzdGVtcy5jb20CAUYwCQYF Kw4DAhoFAKCCAe0wGAYJKoZIhvcNAQkDMQsGCSqGSIb3DQEHATAcBgkqhkiG9w0BCQUxDxcNMDQx MDA4MTgxNzA0WjAjBgkqhkiG9w0BCQQxFgQUrdRDcCZny2KHDsypSJnh2yEGosQwgcQGCSsGAQQB gjcQBDGBtjCBszCBrTELMAkGA1UEBhMCVVMxETAPBgNVBAgTCE5ldyBZb3JrMREwDwYDVQQHEwhO ZXcgWW9yazEdMBsGA1UEChMUV2FzYWJpIFN5c3RlbXMsIEluYy4xDDAKBgNVBAsTA0dIUTEfMB0G A1UEAxMWV2FzYWJpIFN5c3RlbXMgUm9vdCBDQTEqMCgGCSqGSIb3DQEJARYbd2VibWFzdGVyQHdh c2FiaXN5c3RlbXMuY29tAgFGMIHGBgsqhkiG9w0BCRACCzGBtqCBszCBrTELMAkGA1UEBhMCVVMx ETAPBgNVBAgTCE5ldyBZb3JrMREwDwYDVQQHEwhOZXcgWW9yazEdMBsGA1UEChMUV2FzYWJpIFN5 c3RlbXMsIEluYy4xDDAKBgNVBAsTA0dIUTEfMB0GA1UEAxMWV2FzYWJpIFN5c3RlbXMgUm9vdCBD QTEqMCgGCSqGSIb3DQEJARYbd2VibWFzdGVyQHdhc2FiaXN5c3RlbXMuY29tAgFGMA0GCSqGSIb3 DQEBAQUABIGAlCTSVvXszKG68pez1o3DTMgejyqe/AX1+csnLX/JGscDpnD8Ckzcd1wmNcwtWG4i beeyj/tZJ1/hg3NCfoNvdTjzQJVJEf9Wj88uT4ARTJRiUmY/JTzaLLZoMEyOvaXxkCacsChFVD1b TQQvXHpL8Xzff0faWXZHk2+vQWqochUAAAAAAAA= --Apple-Mail-53-840628606-- --Apple-Mail-54-840628611 content-type: application/pgp-signature; x-mac-type=70674453; name=PGP.sig content-description: This is a digitally signed message part content-disposition: inline; filename=PGP.sig content-transfer-encoding: 7bit Content-Transfer-Encoding: 7bit Content-Transfer-Encoding: 7bit -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.2.3 (Darwin) iD8DBQFBZtmqDJT2Egh26K0RAolqAJ0SghHIMHrY6j/SDe5Agbabgz9XmwCcDXpI i31UZ+3weAIF5zc182NKreo= =xJ/m -----END PGP SIGNATURE----- --Apple-Mail-54-840628611-- --===============0723716773== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline Content-Transfer-Encoding: 7bit _______________________________________________ Ips mailing list Ips@ietf.org https://www1.ietf.org/mailman/listinfo/ips --===============0723716773==-- From ips-bounces@ietf.org Fri Oct 8 15:03:48 2004 Received: from ietf-mx.ietf.org (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id PAA24108 for ; Fri, 8 Oct 2004 15:03:48 -0400 (EDT) Received: from megatron.ietf.org ([132.151.6.71]) by ietf-mx.ietf.org with esmtp (Exim 4.33) id 1CG0Br-0006LX-UP for ips-web-archive@ietf.org; Fri, 08 Oct 2004 15:14:08 -0400 Received: from localhost.localdomain ([127.0.0.1] helo=megatron.ietf.org) by megatron.ietf.org with esmtp (Exim 4.32) id 1CFzur-0003CY-Av; Fri, 08 Oct 2004 14:56:33 -0400 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by megatron.ietf.org with esmtp (Exim 4.32) id 1CFzp9-0001PH-EK for ips@megatron.ietf.org; Fri, 08 Oct 2004 14:50:41 -0400 Received: from ietf-mx.ietf.org (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id OAA22143 for ; Fri, 8 Oct 2004 14:50:40 -0400 (EDT) Received: from palrel13.hp.com ([156.153.255.238]) by ietf-mx.ietf.org with esmtp (Exim 4.33) id 1CFzz9-0005ts-3f for ips@ietf.org; Fri, 08 Oct 2004 15:00:59 -0400 Received: from rosemail.rose.hp.com (rosemail.rose.hp.com [15.43.209.160]) by palrel13.hp.com (Postfix) with ESMTP id 2A3FA1C0D33E; Fri, 8 Oct 2004 11:50:39 -0700 (PDT) Received: from [127.0.0.1] (dhcp-roseor-beb159.rose.hp.com [15.23.141.159]) by rosemail.rose.hp.com (Postfix) with ESMTP id 0A0C68056; Fri, 8 Oct 2004 11:50:39 -0700 (PDT) Message-ID: <4166E17D.5000209@rose.hp.com> Date: Fri, 08 Oct 2004 11:50:37 -0700 From: "Mallikarjun C." User-Agent: Mozilla Thunderbird 0.7.3 (Windows/20040803) X-Accept-Language: en-us, en MIME-Version: 1.0 To: Caitlin Bestler Subject: Re: [Ips] iSER flow control options References: <41649554.60700@rose.hp.com> <034358AE-1847-11D9-8836-000393814604@asomi.com> In-Reply-To: <034358AE-1847-11D9-8836-000393814604@asomi.com> Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit X-Spam-Score: 0.0 (/) X-Scan-Signature: 731ea0e9f5725b67e634db1918f3b951 Content-Transfer-Encoding: 7bit Cc: ips@ietf.org X-BeenThere: ips@ietf.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: IP Storage List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Sender: ips-bounces@ietf.org Errors-To: ips-bounces@ietf.org X-Spam-Score: 0.0 (/) X-Scan-Signature: 872695ea777a517bf5717e5acc69f8be Content-Transfer-Encoding: 7bit Caitlin, #1: I do not see any major disagreement here. #2: You say "It is a good idea, one that local interfaces *should* offer. But we all know that it isn't part of the de facto standard." I don't understand this. I presume you're referring to Verbs in the second setence, but #2 has nothing to do with Verbs as far as I can tell. This can be done totally under the covers by the RNIC. The latest DDP spec does allow (or even encourage) this behavior. #3: I think you are agreeing with this option. #3a: >There is also "3a" which is to make a default limit that is specified in the protocol itself. I wish we know the "right" limit, but we don't. So I don't see any qualitative difference from current iSER's approach. >The > receiver SHOULD > (or perhaps even MUST) set it high enough so that *only* a > malfunctioning or malicious > client could exceed it. IMHO, to meet this requirement, the number of posted "unexpected buffers" would have to be so high that it becomes very expensive and thus leads implementers to wanting #2 anyway. Besides, in a qualitative sense, let me point out that this option has the exact *same* "best effort" flow control nature associated with today's iSER spec approach (i.e. neither is "positive" flow control, without help from #2), except this will turn out be more demanding on buffers. I'm not sure why you don't prefer the current approach but recommend this one. #4: No one wants it. #5: I don't see any major disagreement. We both agree that this is not a self-sufficient option for different reasons. #6: I presume agreement. > Option 2 is not a real solution. It is simply attempting to shove the > excess messages into > buffers that are accounted for (either "system buffers" or by abusing > the network itself as a > buffer forcing false retransmits). I disagree. This is exactly the same thing host-based overcommitting TCP/IP implementations do. If the stack runs out of mbufs, the frame is dropped on the floor and recovered later. Killing a TCP connection under the same exception (buffer overflow) scenario when it runs on an RNIC is ridiculous, that's why the RDDP WG allowed the DDP spec to change to accommodate graceful handling. >But ultimately it is just hoping > that the problem goes away > because it does nothing to ensure that only the connection causing the > problem will > have the corrective action taken against it. Disagree again, even more. iSCSI/iSER layers of course *do not* just sit around hoping that the problem goes away. They do inspect the PDUs, confirm the legitimacy (or otherwise), replenish buffers or kill the connection - all at the application level in an aggressive fashion. It's just that RNIC clients don't need premature "help" from the RNIC in killing the connection. That's what #2 is all about. > I believe the RDDP WG position on this is clear. In my opinion option, > only options 1 and 3 > comply with the RDDP Security requirements for use of shared receive > resources. I really would like us to view S-RQ as a "dampener" in invoking #2 (graceful handling), not as a mandatory feature that iSER requires - because it doesn't (though I as an implementer would require S-RQ in my implementation to realize the dampening). The net is that I see the real major disagreement is only wrt #2 - *not* on #2's desirability, but wrt iSER relying on #2. Thanks. -- Mallikarjun Mallikarjun Chadalapaka Networked Storage Architecture Network Storage Solutions Hewlett-Packard MS 5668 Roseville CA 95747 cbm [at] rose.hp.com Caitlin Bestler wrote: > > On Oct 7, 2004, at 3:01 AM, Mallikarjun C. wrote: > >> I finally managed to catch up on this thread, let me attempt to >> summarize what I think are >> the options for iSER flow control. I would like to get more comments >> to assess the WG >> position, perhaps also from the co-chairs on this topic. I hope this >> long email acts as a >> helpful summary. >> >> Let me ignore S-RQ for a moment and consider a simple RQ. Let's even >> assume the target >> had adequately provisioned receive buffers (CmdSN window + 2 buffers, >> i.e. no overcommitting). >> In this simplistic case, if the initiator sends too many immediate >> commands, we know that the >> target would run out of receive buffers. Now the options that I see >> to deal with this scenario >> are (you may add more if I missed any): >> >> 1. Replenish buffers faster than the arrival of the unexpected >> messages (iSCSI layer may later >> discard the "too many immediate commands" if it deems so). >> 2. Require the RNIC to do graceful handling of lack of receive buffers >> on these exceptions. >> 3. Define a new iSCSI MaxOutstandingUnexpectedPDUs key-based limit >> scheme to deal with >> these exceptions (that John describes below). >> 4. Define an iSER end2end flow control protocol (such as the one iSER >> design team originally >> considered). >> 5. Mandate an S-RQ as the mandatory RNIC feature (so there's more >> flexibility for overprovisioning >> across multiple connections). >> 6. Allow the connection to be dropped because the buffers are overrun. >> >> > >> #1 may be feasible (and every implementation tries to do it anyway), >> but basing the protocol on that >> assumption doesn't strike me as a "standards" way of specifying a >> protocol. > > > I think the operative word here is "try". What I am particularly > concerned about is that for both TCP > and SCP transport level flow control is based on the availability of > receive buffering and network > congestion. > > A TCP connection or SCTP association in use for RDDP effectively has no > buffer limits, assuming > it can forward DDP Segments to the RDDP layer at wire speed. RDDP is > responsible for placing > the buffers when presented, or terminating the stream. It has no > back-pressure mechanism. > > However, many implementations will not tailor the calculation of the > buffer window for RDMA, > but rather use the general purpose algorithm. That can result in false > blocking of placable > traffic, but this failure will not be apparent when users are used to > TCP bottlenecking. > > Existing iSCSI implementations in particular could easily be relying on > TCP bottlenecks. > When they test on an implementation that does not tailor the TCP window > on RDMA connections > to "no system buffering, hence no buffer limits" they will not detect > that the problem is still there. > Later they will test on an implementation that *does* fully eliminate > the TCP bottleneck, and > promptly declare that "this RNIC breaks my iSCSI" and demand the vendor > "fix it". > >> >> #2 is quite nice from iSCSI & iSER's perspective. We should be able >> to cast any >> immediate/un-flow-controlled iSCSI PDU as an "exception" (which it is) >> message >> (in DDP parlance) and thus get around adding any more protocol >> restrictions in iSER. >> The major consideration here is the feasibility (from an RNIC vendor >> perspective) of >> realizing such a graceful handling. > > > That is moving the "take it or flush it" interface up from the LLP to > RDDP interface to > the RDMAP to ULP interface. It is a good idea, one that local > interfaces *should* > offer. But we all know that it isn't part of the de facto standard. It > is one of many > defects in that interface, but I would still oppose adopting any ULP > that cannot > be implemented on top of commonly deployed verbs. > >> >> #3 seems like a workable option if we can cleary specify it. It >> however does add >> more protocol complexity to iSCSI, and makes some exception management >> cases >> harder to deal with (when the limit is not "None") as John points out >> below. > > > There is also "3a" which is to make a default limit that is specified in > the protocol itself. > > I believe the goal should be that no application that is generating > these messages for > their intended purposes should ever have to check the limit. The > receiver SHOULD > (or perhaps even MUST) set it high enough so that *only* a > malfunctioning or malicious > client could exceed it. > > The flow control can be fairly simple to implement however, anytime the > flow control > limit would have been hit, it is for an unanticipated message that the > target iSCSI side > would be free to throw away anyway. The sender could simply do so > immediately. > But I do think it would be better to define the credit limit such that > it would be impossible > to violate it when generating messages legitimately. > > The goal here is to enable the receiving side to identify a runaway > malfunctioning or > malicious client that is attacking shared resources *before* it results > in a legitimate > message from another client being dropped. > > Even without SRQs, it is not that difficult for the receiving iSER layer > to allow sufficient > extra receive buffers to deal with all *legitimate* iSCSI packets. It > is also certainly > preferable for the receive side to have some extra buffers available > rather than > requiring non-malfunctioning/non-abusive senders to check their credit > limit > before sending any of the "unaccounted" iSCSI PDUs. These messages were > excluded from the normal iSCSI flow control precisely because they were not > anticipated to be high volume. > > So there are two ways to determine when a connection is sending > illegitimate > iSCSI packets: place a credit limit that legitimate applications will > not violate > (even without thinking about it) or have the iSER layer examine packets at > wire speed. > >> >> #4 is clearly expensive although we know that it can work. This >> should be the last resort. >> >> #5 is a second-order choice in that it may still require help from >> other options. The main >> considerations here are: making normative references to the Verbs spec >> [ or defining the >> S-RQ text in abstract terms (i.e. stripping off references to WQE, >> CQE, RQ, CQ, RI etc.) >> which again doesn't strike me as a "standards" way of defining a >> protocol ], the acceptance >> (from an RNIC vendor perspective) on requiring it. We still need to >> address a single >> connection case from the specification perspective (i.e. consider an >> embedded target >> supporting a single iSCSI/iSER connection). > > > An SRQ does not eliminate the vulnerability. In fact it is necessary > for it to exist. A runaway > client connected to a simple RQ can only tear down its own connection. > It does not impact > other clients. > > With an SRQ, you either have a per-connection limit/alarm or you are > dependent on wire-speed > draining of completions by the ULP (the iSER layer). That case is > essentially option 1, > enabling the per-connection limit/alarm requires having a number to set > it with which > leads back to option 3 (or 3a). > > An argument for 3, as opposed to 3a, is to specifically allow "None". > But the iSER layer > should only uses "None" when it is committing to drain completions at > wire speed *or* > that the pool of buffers used for untagged messages is not shared with > any other iSCSI > session. > >> >> #6 is also feasible (I think Julian too hinted at this in an email). >> Clearly, an initiator sending too many immediate commands or a target >> sending too many Async Messages is suspect. Problem though is that >> this is a major functional difference between how an iSCSI system >> would behave, and an iSCSI/iSER system would behave (thus potentially >> exposing the dichotomy to layers further up in the stack). The other >> issue is that a genuine exception could be punished with an even >> bigger one - leading to termination of several I/Os. >> >> The way I see it, the current iSER spec depends on #2, #1, #5 and #6. >> Note that while #2 >> alone is necessary and sufficient for operation without connection >> drops, others work to >> minimize the probability of invoking graceful handling (#2) but >> together ultimately allow >> a connection failure if the graceful handling service cannot be >> maintained by the RNIC >> after several DDP exceptions. In the opinion of iSER authors, this >> integrated approach is quite reasonable. >> > Option 2 is not a real solution. It is simply attempting to shove the > excess messages into > buffers that are accounted for (either "system buffers" or by abusing > the network itself as a > buffer forcing false retransmits). But ultimately it is just hoping > that the problem goes away > because it does nothing to ensure that only the connection causing the > problem will > have the corrective action taken against it. > >> I realize that an argument can be made for #3 (which Caitlin & Alex >> did as I understand the >> exchanges so far). More comments from others would help the authors >> understand the WG position on this. >> > > I believe the RDDP WG position on this is clear. In my opinion option, > only options 1 and 3 > comply with the RDDP Security requirements for use of shared receive > resources. _______________________________________________ Ips mailing list Ips@ietf.org https://www1.ietf.org/mailman/listinfo/ips From ips-bounces@ietf.org Fri Oct 8 15:21:12 2004 Received: from ietf-mx.ietf.org (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id PAA26897 for ; Fri, 8 Oct 2004 15:21:12 -0400 (EDT) Received: from megatron.ietf.org ([132.151.6.71]) by ietf-mx.ietf.org with esmtp (Exim 4.33) id 1CG0Si-0006mX-6S for ips-web-archive@ietf.org; Fri, 08 Oct 2004 15:31:32 -0400 Received: from localhost.localdomain ([127.0.0.1] helo=megatron.ietf.org) by megatron.ietf.org with esmtp (Exim 4.32) id 1CG09T-0001QH-3Y; Fri, 08 Oct 2004 15:11:39 -0400 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by megatron.ietf.org with esmtp (Exim 4.32) id 1CG06a-00061r-HV for ips@megatron.ietf.org; Fri, 08 Oct 2004 15:08:40 -0400 Received: from ietf-mx.ietf.org (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id PAA25111 for ; Fri, 8 Oct 2004 15:08:38 -0400 (EDT) Received: from atlrel7.hp.com ([156.153.255.213]) by ietf-mx.ietf.org with esmtp (Exim 4.33) id 1CG0GV-0006VH-4P for ips@ietf.org; Fri, 08 Oct 2004 15:18:58 -0400 Received: from rosemail.rose.hp.com (rosemail.rose.hp.com [15.43.209.160]) by atlrel7.hp.com (Postfix) with ESMTP id 78877A1; Fri, 8 Oct 2004 15:08:35 -0400 (EDT) Received: from [127.0.0.1] (dhcp-roseor-beb159.rose.hp.com [15.23.141.159]) by rosemail.rose.hp.com (Postfix) with ESMTP id 2D6878056; Fri, 8 Oct 2004 12:08:35 -0700 (PDT) Message-ID: <4166E5B0.1090506@rose.hp.com> Date: Fri, 08 Oct 2004 12:08:32 -0700 From: "Mallikarjun C." User-Agent: Mozilla Thunderbird 0.7.3 (Windows/20040803) X-Accept-Language: en-us, en MIME-Version: 1.0 To: William Studenmund Subject: Re: [Ips] resending Data-in with S bit set References: <41649B60.60002@rose.hp.com> <4165785D.5050802@rose.hp.com> <4166D357.40708@rose.hp.com> <4489F857-1956-11D9-91D4-000A95D14BEC@wasabisystems.com> In-Reply-To: <4489F857-1956-11D9-91D4-000A95D14BEC@wasabisystems.com> Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit X-Spam-Score: 0.0 (/) X-Scan-Signature: 1a1bf7677bfe77d8af1ebe0e91045c5b Content-Transfer-Encoding: 7bit Cc: ips@ietf.org X-BeenThere: ips@ietf.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: IP Storage List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Sender: ips-bounces@ietf.org Errors-To: ips-bounces@ietf.org X-Spam-Score: 0.0 (/) X-Scan-Signature: 944ecb6e61f753561f559a497458fb4f Content-Transfer-Encoding: 7bit First off, I agree with the required target behavior you describe. That's the intended behavior. Secondly, you say: >i.e. the initiator noticed a hole in StatSN > clearing, and went to inquire about the hole. This case isn't covered so > well. I can agree with this too, in that the text is not terribly explicit about your scenario. Finally, regarding the scenario you describe: "We then get a Status SNACK, and we can't retransmit the status as it was phase-collapsed in with a Data-In that is too big to send now." I continue to think that the first part of the quoted text addresses your scenario (this is only for the record, since you figured out the right way to handle the scenario anyway): "....or had the status included in the last Data-In PDU, it must send a new SCSI Response PDU." The "must send a new SCSI Response PDU" requirement is not really for the reason you mention ("too big to send now"), but because of the following reasons: 10.4.4 says: "After issuing a R-Data SNACK the initiator must discard any SCSI status unless contained in an SCSI Response PDU carrying the same SNACK Tag as the last issued R-Data SNACK for the SCSI command on the current connection." & a phase-collapsed SCSI Data-in PDU doesn't have room for a SNACK Tag. Hope that helps. -- Mallikarjun Mallikarjun Chadalapaka Networked Storage Architecture Network Storage Solutions Hewlett-Packard MS 5668 Roseville CA 95747 cbm [at] rose.hp.com William Studenmund wrote: > On Oct 8, 2004, at 1:50 PM, Mallikarjun C. wrote: > >> >what you do if you (the target) get a >> > Status SNACK after resegmentation (no R-Data SNACK). Do you send a SCSI >> > Response for the task? >> >> >> >> 10.16.3 says the following in part: >> >> "....if it has already sent a response containing another value >> in the SNACK Tag field or had the status included in the last Data-In >> PDU, it must send a new SCSI Response PDU. >> .... >> If an >> initiator attempts to recover a lost SCSI Response (with a >> Status SNACK, see Section 10.16.1 Type) when more than one response >> has been sent, the target will send the SCSI Response with the latest >> content known to the target, including the last SNACK Tag for the >> command." >> >> >> Does that text address the scenario you've in mind? > > > Almost. I'm thinking about the case where we have sent only one response > (thus not "more than one"). We then get a Status SNACK, and we can't > retransmit the status as it was phase-collapsed in with a Data-In that > is too big to send now. > > Since we haven't sent more than one status, I don't think the text > speaks directly to this case. I think the spirit of the text, send a > SCSI Command Response PDU, is the right thing to do. I'm just not sure > if the text tells me that's what I should do. > > One feeling I get is that the text in this section was written with the > assumption that the first SNACK that will be sent for a task after > resegmentation will be an R-Data SNACK. For that sequence, I think the > text works well. This case is where the first SNACK for a task is a > Status SNACK, as the initiator didn't know what task corresponded to a > given StatSN value. i.e. the initiator noticed a hole in StatSN > clearing, and went to inquire about the hole. This case isn't covered so > well. > > Thank you for continuing to help me with this question. > > Take care, > > Bill _______________________________________________ Ips mailing list Ips@ietf.org https://www1.ietf.org/mailman/listinfo/ips From ips-bounces@ietf.org Fri Oct 8 17:42:50 2004 Received: from ietf-mx.ietf.org (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id RAA20314 for ; Fri, 8 Oct 2004 17:42:50 -0400 (EDT) Received: from megatron.ietf.org ([132.151.6.71]) by ietf-mx.ietf.org with esmtp (Exim 4.33) id 1CG2fn-0005dG-BI for ips-web-archive@ietf.org; Fri, 08 Oct 2004 17:53:12 -0400 Received: from localhost.localdomain ([127.0.0.1] helo=megatron.ietf.org) by megatron.ietf.org with esmtp (Exim 4.32) id 1CG2Lt-0001wT-UI; Fri, 08 Oct 2004 17:32:37 -0400 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by megatron.ietf.org with esmtp (Exim 4.32) id 1CG25X-00039W-Fc for ips@megatron.ietf.org; Fri, 08 Oct 2004 17:15:45 -0400 Received: from ietf-mx.ietf.org (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id RAA17455 for ; Fri, 8 Oct 2004 17:15:41 -0400 (EDT) Received: from mononoke.wasabisystems.com ([66.173.145.228]) by ietf-mx.ietf.org with esmtp (Exim 4.33) id 1CG2FV-0004pl-Ho for ips@ietf.org; Fri, 08 Oct 2004 17:26:02 -0400 Received: from [192.168.1.125] (unknown [66.173.145.226]) by mononoke.wasabisystems.com (Postfix) with ESMTP id 9AEFB870B6; Fri, 8 Oct 2004 17:15:40 -0400 (EDT) In-Reply-To: <4166E5B0.1090506@rose.hp.com> References: <41649B60.60002@rose.hp.com> <4165785D.5050802@rose.hp.com> <4166D357.40708@rose.hp.com> <4489F857-1956-11D9-91D4-000A95D14BEC@wasabisystems.com> <4166E5B0.1090506@rose.hp.com> Mime-Version: 1.0 (Apple Message framework v619) Message-Id: <30FE007A-196F-11D9-91D4-000A95D14BEC@wasabisystems.com> From: William Studenmund Subject: Re: [Ips] resending Data-in with S bit set Date: Fri, 8 Oct 2004 17:15:27 -0400 To: "Mallikarjun C." X-Pgp-Agent: GPGMail 1.0 (v30, 10.3) X-Mailer: Apple Mail (2.619) X-Spam-Score: 0.0 (/) X-Scan-Signature: 42e3ed3f10a1d8bef690f09da16f507a Cc: ips@ietf.org X-BeenThere: ips@ietf.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: IP Storage List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Content-Type: multipart/mixed; boundary="===============0388221540==" Sender: ips-bounces@ietf.org Errors-To: ips-bounces@ietf.org X-Spam-Score: 0.0 (/) X-Scan-Signature: 25eb6223a37c19d53ede858176b14339 --===============0388221540== Content-Type: multipart/signed; protocol="application/pgp-signature"; micalg=pgp-sha1; boundary="Apple-Mail-58-851333237" --Apple-Mail-58-851333237 Content-Type: multipart/signed; micalg=sha1; boundary=Apple-Mail-57-851333231; protocol="application/pkcs7-signature" --Apple-Mail-57-851333231 Content-Type: text/plain; charset=US-ASCII; format=flowed Content-Transfer-Encoding: 7bit On Oct 8, 2004, at 3:08 PM, Mallikarjun C. wrote: > First off, I agree with the required target behavior you describe. > That's the intended behavior. > > Secondly, you say: > >i.e. the initiator noticed a hole in StatSN > > clearing, and went to inquire about the hole. This case isn't > covered so > > well. > > I can agree with this too, in that the text is not terribly explicit > about your scenario. > > Finally, regarding the scenario you describe: > > "We then get a Status SNACK, and we can't retransmit the status as it > was phase-collapsed in with a Data-In that is too big to send now." > > I continue to think that the first part of the quoted text addresses > your scenario (this is only for the record, since you figured out the > right way to handle the scenario anyway): > > "....or had the status included in the last Data-In > PDU, it must send a new SCSI Response PDU." > > The "must send a new SCSI Response PDU" requirement is not really for > the reason you mention ("too big to send now"), but because of the > following reasons: > > 10.4.4 says: > "After issuing a R-Data SNACK the initiator must discard any SCSI > status unless contained in an SCSI Response PDU carrying the same > SNACK Tag as the last issued R-Data SNACK for the SCSI command on the > current connection." > > & a phase-collapsed SCSI Data-in PDU doesn't have room for a SNACK Tag. Oh, that makes a LOT of sense. However I'm still a bit puzzled as this scenario is "before" an R-Data SNACK has been sent. Thus the text doesn't seem to directly apply. I mean I think the text is fine once the R-Data has gotten there, just not before. I think it'd be fine to, after resegmentation but before an R-Data SNACK has been received, just resend a Data-In PDU with phase collapsed status iff it is still ok to send. Consider the case where the read was for say 2k of data and the resegmenting was from 8k to 4k. Just send the PDU again as it was. If the Data-In PDU is too big to send (i.e. the initiator really should send an R-Data SNACK but hasn't realized it yet), I think the spec isn't clear, but that we have agreed that the target should send a SCSI Response. Since there has been no R-Data SNACK, there is no SNACK tag, and so 10.4.4 indicates that the SNACK tag field should be zero. What I expect to then happen is that the initiator will now have an ITT for the StatSN, notice it has not received any or all of the data, then send an R-Data SNACK for that data. Then all of the text in 10.16.3 kicks in. Take care, Bill --Apple-Mail-57-851333231 Content-Type: application/pkcs7-signature; name=smime.p7s Content-Disposition: attachment; filename=smime.p7s Content-Transfer-Encoding: base64 MIAGCSqGSIb3DQEHAqCAMIACAQExCzAJBgUrDgMCGgUAMIAGCSqGSIb3DQEHAQAAoIIFfDCCBXgw ggNgoAMCAQICAUYwDQYJKoZIhvcNAQEEBQAwga0xCzAJBgNVBAYTAlVTMREwDwYDVQQIEwhOZXcg WW9yazERMA8GA1UEBxMITmV3IFlvcmsxHTAbBgNVBAoTFFdhc2FiaSBTeXN0ZW1zLCBJbmMuMQww CgYDVQQLEwNHSFExHzAdBgNVBAMTFldhc2FiaSBTeXN0ZW1zIFJvb3QgQ0ExKjAoBgkqhkiG9w0B CQEWG3dlYm1hc3RlckB3YXNhYmlzeXN0ZW1zLmNvbTAeFw0wMzA4MTExODU2NDZaFw0wODA2MjUx ODU2NDZaMIGXMQswCQYDVQQGEwJVUzETMBEGA1UECBMKQ2FsaWZvcm5pYTESMBAGA1UEBxMJU2Fu IERpZWdvMRcwFQYDVQQKEw5XYXNhYmkgU3lzdGVtczEbMBkGA1UEAxMSV2lsbGlhbSBTdHVkZW5t dW5kMSkwJwYJKoZIhvcNAQkBFhp3cnN0dWRlbkB3YXNhYmlzeXN0ZW1zLmNvbTCBnzANBgkqhkiG 9w0BAQEFAAOBjQAwgYkCgYEA9CZhg4GVWYTRGcJz8lU1ipzInlBuGqfZQs88Z1rTP7+c3AyhKwpE TyTmK4UhxslzKa93YLqifLxZKYt1jI6FNq40IQWCKBVywWBwmL5guCu0851f9ywH5Uj2wJdiDrOI sOJKNpwNqml8bFdTrFS2qvyCxTIWVMLYAzUBBcl2F7MCAwEAAaOCATkwggE1MAkGA1UdEwQCMAAw LAYJYIZIAYb4QgENBB8WHU9wZW5TU0wgR2VuZXJhdGVkIENlcnRpZmljYXRlMB0GA1UdDgQWBBTt mjHbSBlMV48RWOBETM3EV0Tm/jCB2gYDVR0jBIHSMIHPgBQrBOnN88CNCHPaV5wLS212tcrgBaGB s6SBsDCBrTELMAkGA1UEBhMCVVMxETAPBgNVBAgTCE5ldyBZb3JrMREwDwYDVQQHEwhOZXcgWW9y azEdMBsGA1UEChMUV2FzYWJpIFN5c3RlbXMsIEluYy4xDDAKBgNVBAsTA0dIUTEfMB0GA1UEAxMW V2FzYWJpIFN5c3RlbXMgUm9vdCBDQTEqMCgGCSqGSIb3DQEJARYbd2VibWFzdGVyQHdhc2FiaXN5 c3RlbXMuY29tggEAMA0GCSqGSIb3DQEBBAUAA4ICAQCpxoPEYunR7qiXyycEZd0UIatSt/WalMoH nuN9S421iS/sD1wSDqzHyM5YyUWn4z4o6VhT2kD9UV4lk5aXxxefEfGzFcR7Xyah7tO3ziuReFfx qUP5+DTlwDuUBc8Iow6NwY9GlRCKFSsEroLWLGODyYWgN1ojjKOtcXRiOZkYTP3VfDUV2/Xrhh9O l6+3Otf069+KWeOWedWeMsbe1rac96yPEjPogToWkBXVJEeEQktd0BLETt7GUuSZg2Fho/nslqZv nnnKZmrOfdFy9iXDL9T0Hp/OHf5B9RP+r00BJJFO8JzBPIL2IA3CBPApOQty5A00Jg1CGfDQxiV3 s8G0aRUbmFqg9r0OfhPYjJ8BHfw1HHWb7Cu91Sw+d5UnYracZhsR2Gc5H5CnfS6IZhc7ulfnSX8f I4BXR4vbeRUG76J3rkkbkO1haYHLk5w5RlICHsJOBSElG9ggLBWNIfUZwAzo84lfmDmJMP1YCOmi DFCbW1DgxhqVFpYEf5Pq2pVcCguopK2G9oZjH5fxq6ehisk9dN+7NEQ4OSqC43ehaStYOoQAJruN e6e5tCtRMx7bMI7eVPiHxE9/Y38Q+sc5oRN9mCyyAXMweuMRNHAHAWdFB7NhHD6gBa2lh91zIMdf iSVRlBXoMrBKJX6VecgnhvUp4G6QAnN7EMu6QHlc/jGCA0swggNHAgEBMIGzMIGtMQswCQYDVQQG EwJVUzERMA8GA1UECBMITmV3IFlvcmsxETAPBgNVBAcTCE5ldyBZb3JrMR0wGwYDVQQKExRXYXNh YmkgU3lzdGVtcywgSW5jLjEMMAoGA1UECxMDR0hRMR8wHQYDVQQDExZXYXNhYmkgU3lzdGVtcyBS b290IENBMSowKAYJKoZIhvcNAQkBFht3ZWJtYXN0ZXJAd2FzYWJpc3lzdGVtcy5jb20CAUYwCQYF Kw4DAhoFAKCCAe0wGAYJKoZIhvcNAQkDMQsGCSqGSIb3DQEHATAcBgkqhkiG9w0BCQUxDxcNMDQx MDA4MjExNTI4WjAjBgkqhkiG9w0BCQQxFgQUknyQPOmijrUPtb8vOjeQNAR7TJ4wgcQGCSsGAQQB gjcQBDGBtjCBszCBrTELMAkGA1UEBhMCVVMxETAPBgNVBAgTCE5ldyBZb3JrMREwDwYDVQQHEwhO ZXcgWW9yazEdMBsGA1UEChMUV2FzYWJpIFN5c3RlbXMsIEluYy4xDDAKBgNVBAsTA0dIUTEfMB0G A1UEAxMWV2FzYWJpIFN5c3RlbXMgUm9vdCBDQTEqMCgGCSqGSIb3DQEJARYbd2VibWFzdGVyQHdh c2FiaXN5c3RlbXMuY29tAgFGMIHGBgsqhkiG9w0BCRACCzGBtqCBszCBrTELMAkGA1UEBhMCVVMx ETAPBgNVBAgTCE5ldyBZb3JrMREwDwYDVQQHEwhOZXcgWW9yazEdMBsGA1UEChMUV2FzYWJpIFN5 c3RlbXMsIEluYy4xDDAKBgNVBAsTA0dIUTEfMB0GA1UEAxMWV2FzYWJpIFN5c3RlbXMgUm9vdCBD QTEqMCgGCSqGSIb3DQEJARYbd2VibWFzdGVyQHdhc2FiaXN5c3RlbXMuY29tAgFGMA0GCSqGSIb3 DQEBAQUABIGATvdlp6W6b9YlZs6SosysYYnyXJKvkJm26iujr4wZHWAZS84RKLJXy2by5nmsODVF sRMftTGSBFYNzF3/ywrriNq5Gd3bN4qBgSYz/TzDCYPrH8kw5QcS1ShUSFwem00/1OkI4FsnxJ/Y WkO9Mvwfz+oLqFC1xfmYcZ/B1Oo3ixsAAAAAAAA= --Apple-Mail-57-851333231-- --Apple-Mail-58-851333237 content-type: application/pgp-signature; x-mac-type=70674453; name=PGP.sig content-description: This is a digitally signed message part content-disposition: inline; filename=PGP.sig content-transfer-encoding: 7bit Content-Transfer-Encoding: 7bit Content-Transfer-Encoding: 7bit -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.2.3 (Darwin) iD8DBQFBZwN7DJT2Egh26K0RArq4AKCYNYw3ZeRISt/RBmQeIDycCVJwEwCgmHs7 QjpeH3MkXT+GYA1KaEN+qS4= =C84L -----END PGP SIGNATURE----- --Apple-Mail-58-851333237-- --===============0388221540== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline Content-Transfer-Encoding: 7bit _______________________________________________ Ips mailing list Ips@ietf.org https://www1.ietf.org/mailman/listinfo/ips --===============0388221540==-- From ips-bounces@ietf.org Fri Oct 8 22:26:11 2004 Received: from ietf-mx.ietf.org (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id WAA20759 for ; Fri, 8 Oct 2004 22:26:11 -0400 (EDT) Received: from megatron.ietf.org ([132.151.6.71]) by ietf-mx.ietf.org with esmtp (Exim 4.33) id 1CG763-0002rG-HH for ips-web-archive@ietf.org; Fri, 08 Oct 2004 22:36:35 -0400 Received: from localhost.localdomain ([127.0.0.1] helo=megatron.ietf.org) by megatron.ietf.org with esmtp (Exim 4.32) id 1CG6pP-0001yH-8e; Fri, 08 Oct 2004 22:19:23 -0400 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by megatron.ietf.org with esmtp (Exim 4.32) id 1CG6lW-0001Ek-6A for ips@megatron.ietf.org; Fri, 08 Oct 2004 22:15:24 -0400 Received: from ietf-mx.ietf.org (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id WAA15705 for ; Fri, 8 Oct 2004 22:15:20 -0400 (EDT) From: elizabeth.rodriguez@dothill.com Received: from artecon.dothill.com ([155.254.128.254] helo=dothill.com) by ietf-mx.ietf.org with esmtp (Exim 4.33) id 1CG6vX-0002ge-RQ for ips@ietf.org; Fri, 08 Oct 2004 22:25:44 -0400 Received: from exchange.artecon.com (exchange [206.6.182.75]) by dothill.com (8.12.9+Sun/8.12.5) with ESMTP id i9929hLl011949; Fri, 8 Oct 2004 19:09:44 -0700 (PDT) Received: by EXCHANGE with Internet Mail Service (5.5.2653.19) id <4M1460MH>; Fri, 8 Oct 2004 19:14:56 -0700 Message-ID: To: ips@ietf.org Date: Fri, 8 Oct 2004 19:18:26 -0700 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2653.19) Content-Type: text/plain X-Spam-Score: 0.3 (/) X-Scan-Signature: 41c17b4b16d1eedaa8395c26e9a251c4 Cc: T11_5@t11.org, t11_3@t11.org, mankin@psg.com, bwijnen@lucent.com, roger.cummings@veritas.com, kzm@cisco.com, black_david@emc.com Subject: [Ips] Change to FC Mgmt MIB X-BeenThere: ips@ietf.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: IP Storage List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Sender: ips-bounces@ietf.org Errors-To: ips-bounces@ietf.org X-Spam-Score: 0.3 (/) X-Scan-Signature: 25620135586de10c627e3628c432b04a All, An ambiguity has been found in the text of the FC Mgmt MIB that I believe is not a substantial change, but is an important clarification. Keith McCloghrie describes issue in the following email, along with a proposed addition to resolve the issue. I agree with Keith that this change is important and should be made. The FC Mgmt MIB has completed IETF last call, so this is a late comment against the MIB. The draft is scheduled to be discussed next week, so I am asking everyone to review the change and if anyone has any comments on the change to please either discuss it on the IPS mailing list or let David or I know as soon as possible, but no later than Wed, Oct 13th. In absence of comment, or upon consensus of the working group, the change will be made as an RFC Editors note. Thank you, Elizabeth IPS co-chair -----Original Message----- From: Keith McCloghrie [mailto:kzm@cisco.com] Sent: Friday, October 08, 2004 5:56 PM To: elizabeth.rodriguez@dothill.com Cc: mankin@psg.com; black_david@emc.com; roger.cummings@veritas.com; kzm@cisco.com Subject: Re: Last Call: 'Fibre Channel Management MIB' to Proposed Standard Elizabeth, I'm sending this email to document the fact that, during T11.5's "Ad Hoc Management" meeting this week (at which you and I were both present), we found a minor but potential ambiguity in draft-ietf-ips-fcmgmt-mib-05.txt. Specifically, this definition: fcmInstanceFabricId OBJECT-TYPE SYNTAX FcNameIdOrZero MAX-ACCESS read-only STATUS current DESCRIPTION "The globally unique Fabric Identifier which identifies the fabric to which the Fibre Channel entity/entities managed by this management instance are connected, or, of which they are a part. This is typically the Node WWN of the principal switch of a Fibre Channel fabric. The zero-length string indicates that the fabric identifier is unknown (or not applicable)." ::= { fcmInstanceEntry 9 } is written as if the "entity/entities" can be connected to at most one Fabric. While that was true at the time it was written, the recent emergence of Virtual Fabrics in T11 will likely mean that multiple Virtual Fabrics can be multiplexed onto a single physical port, in which case a management instance will need to be connected to multiple of them (otherwise, I can't concieve of how the port can be managed!!). My suggestion on how to resolve the ambiguity would be to add one sentence to the above: In the event the Fibre Channel entity/entities managed by this management instance are connected to multiple fabrics, then this object records the first (known) one. In the future, it may be possible and appropriate to define further MIB objects, but such are not needed at this time. All that is needed at the present time is to remove the future potential of an ambiguity. Keith. _______________________________________________ Ips mailing list Ips@ietf.org https://www1.ietf.org/mailman/listinfo/ips From ips-bounces@ietf.org Sat Oct 9 11:12:55 2004 Received: from ietf-mx.ietf.org (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id LAA19412 for ; Sat, 9 Oct 2004 11:12:55 -0400 (EDT) Received: from megatron.ietf.org ([132.151.6.71]) by ietf-mx.ietf.org with esmtp (Exim 4.33) id 1CGJ4A-00086Q-Ly for ips-web-archive@ietf.org; Sat, 09 Oct 2004 11:23:26 -0400 Received: from localhost.localdomain ([127.0.0.1] helo=megatron.ietf.org) by megatron.ietf.org with esmtp (Exim 4.32) id 1CGIqZ-0001Om-OS; Sat, 09 Oct 2004 11:09:23 -0400 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by megatron.ietf.org with esmtp (Exim 4.32) id 1CGIoE-0000rJ-Vt for ips@megatron.ietf.org; Sat, 09 Oct 2004 11:07:03 -0400 Received: from ietf-mx.ietf.org (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id LAA19086 for ; Sat, 9 Oct 2004 11:06:57 -0400 (EDT) Received: from system60.simplicato.com ([207.99.47.60] helo=host52a.simplicato.com) by ietf-mx.ietf.org with esmtp (Exim 4.33) id 1CGIyN-000803-RU for ips@ietf.org; Sat, 09 Oct 2004 11:17:28 -0400 Received: from localhost (localhost.simplicato.com [127.0.0.1]) by host52a.simplicato.com (Postfix) with ESMTP id BB68911E0F8; Sat, 9 Oct 2004 11:06:54 -0400 (EDT) Received: from host52a.simplicato.com ([127.0.0.1]) by localhost (host52a.simplicato.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 33980-06; Sat, 9 Oct 2004 11:06:53 -0400 (EDT) Received: from [192.168.1.2] (bzq-198-76.red.bezeqint.net [212.179.198.76]) by host52a.simplicato.com (Postfix) with ESMTP id 931A711E0AA; Sat, 9 Oct 2004 11:06:52 -0400 (EDT) In-Reply-To: <4166E17D.5000209@rose.hp.com> References: <41649554.60700@rose.hp.com> <034358AE-1847-11D9-8836-000393814604@asomi.com> <4166E17D.5000209@rose.hp.com> Mime-Version: 1.0 (Apple Message framework v619) Content-Type: text/plain; charset=US-ASCII; format=flowed Message-Id: Content-Transfer-Encoding: 7bit From: Caitlin Bestler Subject: Re: [Ips] iSER flow control options Date: Sat, 9 Oct 2004 17:06:50 +0200 To: "Mallikarjun C." X-Mailer: Apple Mail (2.619) X-Virus-Scanned: by amavisd-new at simplicato.com X-Spam-Score: 0.1 (/) X-Scan-Signature: 6e922792024732fb1bb6f346e63517e4 Content-Transfer-Encoding: 7bit Cc: ips@ietf.org X-BeenThere: ips@ietf.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: IP Storage List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Sender: ips-bounces@ietf.org Errors-To: ips-bounces@ietf.org X-Spam-Score: 0.1 (/) X-Scan-Signature: 9a2be21919e71dc6faef12b370c4ecf5 Content-Transfer-Encoding: 7bit On Oct 8, 2004, at 8:50 PM, Mallikarjun C. wrote: > Caitlin, > > #1: I do not see any major disagreement here. > > #2: You say "It is a good idea, one that local interfaces *should* > offer. But we all know that it isn't part of the de facto standard." > I don't understand this. I presume you're referring to Verbs in the > second setence, > but #2 has nothing to do with Verbs as far as I can tell. This can be > done totally > under the covers by the RNIC. The latest DDP spec does allow (or even > encourage) > this behavior. I was envisioning a synchronous exception callback from the RI to a privileged application that said "ALERT imminent connection failure, provide a buffer NOW or else!" What you were thinking of is a non-standard interface between RDDP and the LLP. The RDDP spec *allows* an implementation to offer such a mechanism as a *fault recovery* feature. It is *not* something that any application should *rely* upon, and it is *not* something that can be required of the RDDP layer. First, it is absolutely impossible when RDDP is implemented cleanly on top of the LLP. Even in hardware implementations this can be the case, and will indeed be common if the RNIC also supplies TOE services. > > #3: I think you are agreeing with this option. > > #3a: > >There is also "3a" which is to make a default limit that is specified > in the protocol itself. > > I wish we know the "right" limit, but we don't. So I don't see any > qualitative difference from current iSER's approach. > It would be a published standard that implementers could use as justification. This is a minimal solution that is vital for creating a level playing field. But the explicitly negotiated limit has advantages for the cases John identified. > > > > > Option 2 is not a real solution. It is simply attempting to shove > the > > excess messages into buffers that are accounted for (either "system > > buffers" or by abusing the network itself as a buffer forcing false > retransmits). > > I disagree. This is exactly the same thing host-based overcommitting > TCP/IP implementations do. > If the stack runs out of mbufs, the frame is dropped on the floor and > recovered later. Killing a TCP > connection under the same exception (buffer overflow) scenario when it > runs on an RNIC is ridiculous, > that's why the RDDP WG allowed the DDP spec to change to accommodate > graceful handling. > As noted earlier, the RDDP WG also said that the RDDP Consumer could not rely upon the existence of such a mechanism. It only agreed to remove the requirement that RDDP MUST terminate the connection in response. Eliminating the requirement that the Highway Patrol shoot everyone going over the speed limit is quite different from repealing the speed limit. If you send more untagged buffers than you are allowed to your connection MAY be terminated. You MUST NOT complain about it. If you advertise X credits you are expected to do so in good faith, and are expected to have those buffers available when needed. You are not supposed to over-advertise and then rely on the network to sweep your mistake under the rug. That's like a bank saying failure to give a depositor back their money is all right because they can always get Welfare. Statistically under-allocating advertised buffers is fine and proper, but whether you are working at the LLP or RDMAP layers the expectation is that the network is not going to pay for your mistaken estimates. > > > The net is that I see the real major disagreement is only wrt #2 - > *not* on #2's desirability, but wrt iSER relying on #2. > Exactly, #2 is an implementation strategy that should not be forbidden, but neither should it be relied upon. _______________________________________________ Ips mailing list Ips@ietf.org https://www1.ietf.org/mailman/listinfo/ips From ips-bounces@ietf.org Mon Oct 11 18:26:50 2004 Received: from ietf-mx.ietf.org (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id SAA05552 for ; Mon, 11 Oct 2004 18:26:49 -0400 (EDT) Received: from megatron.ietf.org ([132.151.6.71]) by ietf-mx.ietf.org with esmtp (Exim 4.33) id 1CH8nf-0006f9-4y for ips-web-archive@ietf.org; Mon, 11 Oct 2004 18:37:51 -0400 Received: from localhost.localdomain ([127.0.0.1] helo=megatron.ietf.org) by megatron.ietf.org with esmtp (Exim 4.32) id 1CH8Q5-00076w-Kv; Mon, 11 Oct 2004 18:13:29 -0400 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by megatron.ietf.org with esmtp (Exim 4.32) id 1CH8KJ-0003Bw-Dw for ips@megatron.ietf.org; Mon, 11 Oct 2004 18:07:31 -0400 Received: from ietf-mx.ietf.org (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id SAA03951 for ; Mon, 11 Oct 2004 18:07:28 -0400 (EDT) Received: from palrel12.hp.com ([156.153.255.237]) by ietf-mx.ietf.org with esmtp (Exim 4.33) id 1CH8Uv-0006OQ-5v for ips@ietf.org; Mon, 11 Oct 2004 18:18:29 -0400 Received: from rosemail.rose.hp.com (rosemail.rose.hp.com [15.43.209.160]) by palrel12.hp.com (Postfix) with ESMTP id 5236641145A; Mon, 11 Oct 2004 15:07:28 -0700 (PDT) Received: from [127.0.0.1] (dhcp-roseor-bdj092.rose.hp.com [15.23.139.92]) by rosemail.rose.hp.com (Postfix) with ESMTP id 1F02F82CD; Mon, 11 Oct 2004 15:07:28 -0700 (PDT) Message-ID: <416B0413.8080208@rose.hp.com> Date: Mon, 11 Oct 2004 15:07:15 -0700 From: "Mallikarjun C." User-Agent: Mozilla Thunderbird 0.7.3 (Windows/20040803) X-Accept-Language: en-us, en MIME-Version: 1.0 To: William Studenmund Subject: Re: [Ips] resending Data-in with S bit set References: <41649B60.60002@rose.hp.com> <4165785D.5050802@rose.hp.com> <4166D357.40708@rose.hp.com> <4489F857-1956-11D9-91D4-000A95D14BEC@wasabisystems.com> <4166E5B0.1090506@rose.hp.com> <30FE007A-196F-11D9-91D4-000A95D14BEC@wasabisystems.com> In-Reply-To: <30FE007A-196F-11D9-91D4-000A95D14BEC@wasabisystems.com> Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit X-Spam-Score: 0.0 (/) X-Scan-Signature: 7fa173a723009a6ca8ce575a65a5d813 Content-Transfer-Encoding: 7bit Cc: ips@ietf.org X-BeenThere: ips@ietf.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: IP Storage List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Sender: ips-bounces@ietf.org Errors-To: ips-bounces@ietf.org X-Spam-Score: 0.0 (/) X-Scan-Signature: 37af5f8fbf6f013c5b771388e24b09e7 Content-Transfer-Encoding: 7bit I think I may be finally understanding your scenario, but let's confirm that.... A read SCSI task is in progress and the initiator changes MaxRecvDataSegementLength. The task continues. Subsequently, the initiator notices it lost a status. Here's what I think should happen (I think the same as what you described, except for the ExpDataSN part). - As always, initiator starts with a status SNACK. - Regardless of the original transmission (phase collapse or not, old PDU still fits with the new size or not, single/multiple data PDUs), target retransmits the status in an explicit Response PDU with ExpDataSN set to something like 0xffffffff. - Initiator checks ExpDataSN with its records. They shouldn't match, leading to an R-Data SNACK. - Rest of the semantics described in 10.16.3 apply. My reasons are two: - Target always does the same thing in this scenario (PDU size change followed by a status SNACK first) - i.e. doesn't phase collapse even if it still can. - Initiator operates based on ExpDataSN as always, consistent with the spec intent to not have to do scoreboarding/counting. OTOH, the RFC doesn't set aside 0xffffffff as a reserved value for ExpDataSN. So I guess this is a hack for now. Alternatively (or if target had already sent 2**32-1 PDUs for that task), what you described works as well (if the initiator is counting bytes). -- Mallikarjun Mallikarjun Chadalapaka Networked Storage Architecture Network Storage Solutions Hewlett-Packard MS 5668 Roseville CA 95747 cbm [at] rose.hp.com William Studenmund wrote: > On Oct 8, 2004, at 3:08 PM, Mallikarjun C. wrote: > >> First off, I agree with the required target behavior you describe. >> That's the intended behavior. >> >> Secondly, you say: >> >i.e. the initiator noticed a hole in StatSN >> > clearing, and went to inquire about the hole. This case isn't >> covered so >> > well. >> >> I can agree with this too, in that the text is not terribly explicit >> about your scenario. >> >> Finally, regarding the scenario you describe: >> >> "We then get a Status SNACK, and we can't retransmit the status as it >> was phase-collapsed in with a Data-In that is too big to send now." >> >> I continue to think that the first part of the quoted text addresses >> your scenario (this is only for the record, since you figured out the >> right way to handle the scenario anyway): >> >> "....or had the status included in the last Data-In >> PDU, it must send a new SCSI Response PDU." >> >> The "must send a new SCSI Response PDU" requirement is not really for >> the reason you mention ("too big to send now"), but because of the >> following reasons: >> >> 10.4.4 says: >> "After issuing a R-Data SNACK the initiator must discard any SCSI >> status unless contained in an SCSI Response PDU carrying the same >> SNACK Tag as the last issued R-Data SNACK for the SCSI command on the >> current connection." >> >> & a phase-collapsed SCSI Data-in PDU doesn't have room for a SNACK Tag. > > > Oh, that makes a LOT of sense. > > However I'm still a bit puzzled as this scenario is "before" an R-Data > SNACK has been sent. Thus the text doesn't seem to directly apply. I > mean I think the text is fine once the R-Data has gotten there, just not > before. > > I think it'd be fine to, after resegmentation but before an R-Data SNACK > has been received, just resend a Data-In PDU with phase collapsed status > iff it is still ok to send. Consider the case where the read was for say > 2k of data and the resegmenting was from 8k to 4k. Just send the PDU > again as it was. > > If the Data-In PDU is too big to send (i.e. the initiator really should > send an R-Data SNACK but hasn't realized it yet), I think the spec isn't > clear, but that we have agreed that the target should send a SCSI > Response. Since there has been no R-Data SNACK, there is no SNACK tag, > and so 10.4.4 indicates that the SNACK tag field should be zero. > > What I expect to then happen is that the initiator will now have an ITT > for the StatSN, notice it has not received any or all of the data, then > send an R-Data SNACK for that data. Then all of the text in 10.16.3 > kicks in. > > Take care, > > Bill _______________________________________________ Ips mailing list Ips@ietf.org https://www1.ietf.org/mailman/listinfo/ips From ips-bounces@ietf.org Mon Oct 11 21:11:32 2004 Received: from ietf-mx.ietf.org (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id VAA16916 for ; Mon, 11 Oct 2004 21:11:32 -0400 (EDT) Received: from megatron.ietf.org ([132.151.6.71]) by ietf-mx.ietf.org with esmtp (Exim 4.33) id 1CHBN4-0001CV-Eu for ips-web-archive@ietf.org; Mon, 11 Oct 2004 21:22:34 -0400 Received: from localhost.localdomain ([127.0.0.1] helo=megatron.ietf.org) by megatron.ietf.org with esmtp (Exim 4.32) id 1CHB8f-0001Il-9X; Mon, 11 Oct 2004 21:07:41 -0400 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by megatron.ietf.org with esmtp (Exim 4.32) id 1CHB2g-0007al-7g for ips@megatron.ietf.org; Mon, 11 Oct 2004 21:01:30 -0400 Received: from ietf-mx.ietf.org (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id VAA16161 for ; Mon, 11 Oct 2004 21:01:27 -0400 (EDT) Received: from mononoke.wasabisystems.com ([66.173.145.228]) by ietf-mx.ietf.org with esmtp (Exim 4.33) id 1CHBDJ-00011t-9z for ips@ietf.org; Mon, 11 Oct 2004 21:12:29 -0400 Received: from [127.0.0.1] (localhost [127.0.0.1]) by mononoke.wasabisystems.com (Postfix) with ESMTP id 81DA7870A0; Mon, 11 Oct 2004 21:01:26 -0400 (EDT) In-Reply-To: <416B0413.8080208@rose.hp.com> References: <41649B60.60002@rose.hp.com> <4165785D.5050802@rose.hp.com> <4166D357.40708@rose.hp.com> <4489F857-1956-11D9-91D4-000A95D14BEC@wasabisystems.com> <4166E5B0.1090506@rose.hp.com> <30FE007A-196F-11D9-91D4-000A95D14BEC@wasabisystems.com> <416B0413.8080208@rose.hp.com> Mime-Version: 1.0 (Apple Message framework v619) Message-Id: <3DB9AEB1-1BEA-11D9-8B4B-000A95D14BEC@wasabisystems.com> From: William Studenmund Subject: Re: [Ips] resending Data-in with S bit set Date: Mon, 11 Oct 2004 21:01:19 -0400 To: "Mallikarjun C." X-Pgp-Agent: GPGMail 1.0 (v30, 10.3) X-Mailer: Apple Mail (2.619) X-Spam-Score: 0.0 (/) X-Scan-Signature: d2b46e3b2dfbff2088e0b72a54104985 Cc: ips@ietf.org X-BeenThere: ips@ietf.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: IP Storage List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Content-Type: multipart/mixed; boundary="===============2032647652==" Sender: ips-bounces@ietf.org Errors-To: ips-bounces@ietf.org X-Spam-Score: 0.0 (/) X-Scan-Signature: 995b2e24d23b953c94bac5288c432399 --===============2032647652== Content-Type: multipart/signed; protocol="application/pgp-signature"; micalg=pgp-sha1; boundary="Apple-Mail-24--1023398626" --Apple-Mail-24--1023398626 Content-Type: multipart/signed; micalg=sha1; boundary=Apple-Mail-23--1023398629; protocol="application/pkcs7-signature" --Apple-Mail-23--1023398629 Content-Type: text/plain; charset=US-ASCII; format=flowed Content-Transfer-Encoding: 7bit On Oct 11, 2004, at 6:07 PM, Mallikarjun C. wrote: > I think I may be finally understanding your scenario, but let's > confirm that.... Ok! > A read SCSI task is in progress and the initiator changes > MaxRecvDataSegementLength. Specifically, for what I'm thinking of, all of the data have already been sent before the MaxRecvDataSegmentLength > The task continues. Subsequently, the initiator notices it lost a > status. Here's what I think should happen (I think the same as what > you described, except for the ExpDataSN part). > > - As always, initiator starts with a status SNACK. > - Regardless of the original transmission (phase collapse or not, old > PDU still fits with the new size or not, single/multiple data PDUs), > target retransmits the status in an explicit Response PDU with > ExpDataSN set to something like 0xffffffff. > - Initiator checks ExpDataSN with its records. They shouldn't match, > leading to an R-Data SNACK. > - Rest of the semantics described in 10.16.3 apply. > > My reasons are two: > > - Target always does the same thing in this scenario (PDU size change > followed by a status SNACK first) - i.e. doesn't phase collapse even > if it still can. How about this idea. If the target can still phase collapse (the PDU would still fit), it MAY, but due to the fact that there was a resegmentation, the target MAY instead treat the PDU as if it could not fit. That is to say it is an implementation decision if the target just remembers "resegmented" or if it looks at the PDU. > - Initiator operates based on ExpDataSN as always, consistent with the > spec intent to not have to do scoreboarding/counting. How does what I suggest break ExpDataSN? I was about to say I like what you suggested, then I pondered a bit. I don't think we need the magic ExpDataSN value. If the PDU is too big to send, then the target can just generate a SCSI Response PDU with the correct ExpDataSN and a SNACK Tag of 0. Given that the initiator didn't notice the status value in the phase-collapsed read PDU, it should also have not noticed the data reception in it, so there should be a disconnect between ExpDataSN on the target (sent in the SCSI Response) and in the initiator. So a SCSI Response sent in place of the collapsed Data-In should trigger the initiator to send an R-Data SNACK. Shouldn't it? > OTOH, the RFC doesn't set aside 0xffffffff as a reserved value for > ExpDataSN. So I guess this is a hack for now. Alternatively (or if > target had already sent 2**32-1 PDUs for that task), what you > described works as well (if the initiator is counting bytes). I don't see how you have to count bytes. I'm not trying to sound belligerent, and I'm sorry if it comes across that way. But I think just sending the SCSI Response would work. I'm not seeing how the # of bytes'd be needed. The one thing that would be impacted would be a broken initiator that got the data out of this phase-collapsed-read Data-In PDU but not the status. I think it deserves what it gets... Take care, Bill --Apple-Mail-23--1023398629 Content-Type: application/pkcs7-signature; name=smime.p7s Content-Disposition: attachment; filename=smime.p7s Content-Transfer-Encoding: base64 MIAGCSqGSIb3DQEHAqCAMIACAQExCzAJBgUrDgMCGgUAMIAGCSqGSIb3DQEHAQAAoIIFfDCCBXgw ggNgoAMCAQICAUYwDQYJKoZIhvcNAQEEBQAwga0xCzAJBgNVBAYTAlVTMREwDwYDVQQIEwhOZXcg WW9yazERMA8GA1UEBxMITmV3IFlvcmsxHTAbBgNVBAoTFFdhc2FiaSBTeXN0ZW1zLCBJbmMuMQww CgYDVQQLEwNHSFExHzAdBgNVBAMTFldhc2FiaSBTeXN0ZW1zIFJvb3QgQ0ExKjAoBgkqhkiG9w0B CQEWG3dlYm1hc3RlckB3YXNhYmlzeXN0ZW1zLmNvbTAeFw0wMzA4MTExODU2NDZaFw0wODA2MjUx ODU2NDZaMIGXMQswCQYDVQQGEwJVUzETMBEGA1UECBMKQ2FsaWZvcm5pYTESMBAGA1UEBxMJU2Fu IERpZWdvMRcwFQYDVQQKEw5XYXNhYmkgU3lzdGVtczEbMBkGA1UEAxMSV2lsbGlhbSBTdHVkZW5t dW5kMSkwJwYJKoZIhvcNAQkBFhp3cnN0dWRlbkB3YXNhYmlzeXN0ZW1zLmNvbTCBnzANBgkqhkiG 9w0BAQEFAAOBjQAwgYkCgYEA9CZhg4GVWYTRGcJz8lU1ipzInlBuGqfZQs88Z1rTP7+c3AyhKwpE TyTmK4UhxslzKa93YLqifLxZKYt1jI6FNq40IQWCKBVywWBwmL5guCu0851f9ywH5Uj2wJdiDrOI sOJKNpwNqml8bFdTrFS2qvyCxTIWVMLYAzUBBcl2F7MCAwEAAaOCATkwggE1MAkGA1UdEwQCMAAw LAYJYIZIAYb4QgENBB8WHU9wZW5TU0wgR2VuZXJhdGVkIENlcnRpZmljYXRlMB0GA1UdDgQWBBTt mjHbSBlMV48RWOBETM3EV0Tm/jCB2gYDVR0jBIHSMIHPgBQrBOnN88CNCHPaV5wLS212tcrgBaGB s6SBsDCBrTELMAkGA1UEBhMCVVMxETAPBgNVBAgTCE5ldyBZb3JrMREwDwYDVQQHEwhOZXcgWW9y azEdMBsGA1UEChMUV2FzYWJpIFN5c3RlbXMsIEluYy4xDDAKBgNVBAsTA0dIUTEfMB0GA1UEAxMW V2FzYWJpIFN5c3RlbXMgUm9vdCBDQTEqMCgGCSqGSIb3DQEJARYbd2VibWFzdGVyQHdhc2FiaXN5 c3RlbXMuY29tggEAMA0GCSqGSIb3DQEBBAUAA4ICAQCpxoPEYunR7qiXyycEZd0UIatSt/WalMoH nuN9S421iS/sD1wSDqzHyM5YyUWn4z4o6VhT2kD9UV4lk5aXxxefEfGzFcR7Xyah7tO3ziuReFfx qUP5+DTlwDuUBc8Iow6NwY9GlRCKFSsEroLWLGODyYWgN1ojjKOtcXRiOZkYTP3VfDUV2/Xrhh9O l6+3Otf069+KWeOWedWeMsbe1rac96yPEjPogToWkBXVJEeEQktd0BLETt7GUuSZg2Fho/nslqZv nnnKZmrOfdFy9iXDL9T0Hp/OHf5B9RP+r00BJJFO8JzBPIL2IA3CBPApOQty5A00Jg1CGfDQxiV3 s8G0aRUbmFqg9r0OfhPYjJ8BHfw1HHWb7Cu91Sw+d5UnYracZhsR2Gc5H5CnfS6IZhc7ulfnSX8f I4BXR4vbeRUG76J3rkkbkO1haYHLk5w5RlICHsJOBSElG9ggLBWNIfUZwAzo84lfmDmJMP1YCOmi DFCbW1DgxhqVFpYEf5Pq2pVcCguopK2G9oZjH5fxq6ehisk9dN+7NEQ4OSqC43ehaStYOoQAJruN e6e5tCtRMx7bMI7eVPiHxE9/Y38Q+sc5oRN9mCyyAXMweuMRNHAHAWdFB7NhHD6gBa2lh91zIMdf iSVRlBXoMrBKJX6VecgnhvUp4G6QAnN7EMu6QHlc/jGCA0swggNHAgEBMIGzMIGtMQswCQYDVQQG EwJVUzERMA8GA1UECBMITmV3IFlvcmsxETAPBgNVBAcTCE5ldyBZb3JrMR0wGwYDVQQKExRXYXNh YmkgU3lzdGVtcywgSW5jLjEMMAoGA1UECxMDR0hRMR8wHQYDVQQDExZXYXNhYmkgU3lzdGVtcyBS b290IENBMSowKAYJKoZIhvcNAQkBFht3ZWJtYXN0ZXJAd2FzYWJpc3lzdGVtcy5jb20CAUYwCQYF Kw4DAhoFAKCCAe0wGAYJKoZIhvcNAQkDMQsGCSqGSIb3DQEHATAcBgkqhkiG9w0BCQUxDxcNMDQx MDEyMDEwMTIwWjAjBgkqhkiG9w0BCQQxFgQUo0dxSuOInZH+TXAjhXQfIDeMDskwgcQGCSsGAQQB gjcQBDGBtjCBszCBrTELMAkGA1UEBhMCVVMxETAPBgNVBAgTCE5ldyBZb3JrMREwDwYDVQQHEwhO ZXcgWW9yazEdMBsGA1UEChMUV2FzYWJpIFN5c3RlbXMsIEluYy4xDDAKBgNVBAsTA0dIUTEfMB0G A1UEAxMWV2FzYWJpIFN5c3RlbXMgUm9vdCBDQTEqMCgGCSqGSIb3DQEJARYbd2VibWFzdGVyQHdh c2FiaXN5c3RlbXMuY29tAgFGMIHGBgsqhkiG9w0BCRACCzGBtqCBszCBrTELMAkGA1UEBhMCVVMx ETAPBgNVBAgTCE5ldyBZb3JrMREwDwYDVQQHEwhOZXcgWW9yazEdMBsGA1UEChMUV2FzYWJpIFN5 c3RlbXMsIEluYy4xDDAKBgNVBAsTA0dIUTEfMB0GA1UEAxMWV2FzYWJpIFN5c3RlbXMgUm9vdCBD QTEqMCgGCSqGSIb3DQEJARYbd2VibWFzdGVyQHdhc2FiaXN5c3RlbXMuY29tAgFGMA0GCSqGSIb3 DQEBAQUABIGAZLoWWW13i53EoxsDMJum0nq+eKhM+1nSbMa6Qn+zAVHHDhPFJUsPd3goURLPbI5o gAXYWVi8lZAtt9rq0EZp/ZbAcCtDSQlcZOKpIDx9JqhCiBGXmHkhp02K6tWf9ukXm2RTCzDU1A+J NtzaqctG7ouzRdz76DNBGeWGGhdFhRMAAAAAAAA= --Apple-Mail-23--1023398629-- --Apple-Mail-24--1023398626 content-type: application/pgp-signature; x-mac-type=70674453; name=PGP.sig content-description: This is a digitally signed message part content-disposition: inline; filename=PGP.sig content-transfer-encoding: 7bit Content-Transfer-Encoding: 7bit Content-Transfer-Encoding: 7bit -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.2.3 (Darwin) iD8DBQFBayzlDJT2Egh26K0RAuEeAJ9rncqW2Bfb3oiLjG7FSBkMcl4o3QCggCz9 1saoKMxkHVMhaX3SDFo2hu4= =djPZ -----END PGP SIGNATURE----- --Apple-Mail-24--1023398626-- --===============2032647652== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline Content-Transfer-Encoding: 7bit _______________________________________________ Ips mailing list Ips@ietf.org https://www1.ietf.org/mailman/listinfo/ips --===============2032647652==-- From ips-bounces@ietf.org Tue Oct 12 08:15:07 2004 Received: from ietf-mx.ietf.org (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id IAA13030 for ; Tue, 12 Oct 2004 08:15:07 -0400 (EDT) Received: from megatron.ietf.org ([132.151.6.71]) by ietf-mx.ietf.org with esmtp (Exim 4.33) id 1CHLjJ-0003jX-5M for ips-web-archive@ietf.org; Tue, 12 Oct 2004 08:26:14 -0400 Received: from localhost.localdomain ([127.0.0.1] helo=megatron.ietf.org) by megatron.ietf.org with esmtp (Exim 4.32) id 1CHLOB-0002rH-LP; Tue, 12 Oct 2004 08:04:23 -0400 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by megatron.ietf.org with esmtp (Exim 4.32) id 1CHLFs-00009R-6f for ips@megatron.ietf.org; Tue, 12 Oct 2004 07:55:48 -0400 Received: from ietf-mx.ietf.org (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id HAA10899 for ; Tue, 12 Oct 2004 07:55:47 -0400 (EDT) Received: from taurus.voltaire.com ([212.143.27.73]) by ietf-mx.ietf.org with esmtp (Exim 4.33) id 1CHLQb-0003EW-3g for ips@ietf.org; Tue, 12 Oct 2004 08:06:53 -0400 X-MimeOLE: Produced By Microsoft Exchange V6.5.7226.0 Content-class: urn:content-classes:message MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable Subject: Re: [Ips] iSER flow control options Date: Tue, 12 Oct 2004 13:55:13 +0200 Message-ID: Thread-Topic: Re: [Ips] iSER flow control options Thread-Index: AcSvwACCETuLN/9BSHq++UrGsePKpAAiuJTQAAG+u+A= From: "Alex Nezhinsky" To: X-Spam-Score: 0.0 (/) X-Scan-Signature: c3a18ef96977fc9bcc21a621cbf1174b Content-Transfer-Encoding: quoted-printable X-BeenThere: ips@ietf.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: IP Storage List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Sender: ips-bounces@ietf.org Errors-To: ips-bounces@ietf.org X-Spam-Score: 0.0 (/) X-Scan-Signature: 32b73d73e8047ed17386f9799119ce43 Content-Transfer-Encoding: quoted-printable Hi, all Reading through the last messages in the "flow control" thread was quite helpful, but I still have a few questions. [Caitlin Bestler said> Shared Receive Queues are relevant here on two fronts. First, iSCSI flow control is really more session oriented than connection oriented. Pre-posting separate buffers to receive Cmd X on *each* connection is clearly over-buffering. This may be valid, because memory is cheap, but clearly it is not required to pre-commit duplicate buffers for each connection in a session. One possible solution is as follows: For the initiator, the ExpCmdSN from the target will be used to advance the credit... From what has been said previously on similar issues, I can see two replies to the above problem (if it is a problem): 1) Use S-RQ and all such problems are gone 2) At least in target, DataMover does not have to be "that" separated from the iSCSI layer. Well, I'll try to present our implementation issues, they may be of value also as a "yet another use case" for iSER. We are building a storage gateway between IB and FC. The current incarnation is based on iSER.=20 We have an operational (buffer management is the major remaining issue) iSER DataMover linux kernel module, where iSER is built on top of IB rather than iWARP. It is built as a DataMover "service provider", capable to serve concurrently a number of iSCSI entities incl. initiator and/or target-type. Such capability is useful as it enables us to have the same code for both initiator and target platforms, streamlines debugging and can enable easy extention to a router-like environment where the same box acts both as initiator and target through different interfaces. We are in process of integration with the open-source linux-iscsi initiator project, which is to act as an iSCSI client of the iSER DataMover.=20 I got into such level of detalization to stress that we already did not opt for intimate integration between the iSER/iSCSI layers, but rather tried to keep a reasonable level of separation between them.=20 Being conscious about the ever-changing environment of RDMA transports, we build iSER DataMover on top of DAPL, as DAT is more general than any specific implementation of IB, iWARP etc. So we did know about the S-RQ design option, it's only that it is still not fully supported by the specs issued by the DAT collaborative. Though it surely will be supported in the (hopefully near) future, I guess this is an example of implementation which can't be based on S-RQ, at least for a time being. I'd note also that nothing either in the DataMover Architecture or iSER specs has warned us that the design should be based upon S-RQ, just as noted by [Mallikarjun C. said> I really would like us to view S-RQ as a "dampener" in invoking #2 (graceful handling), not as a mandatory feature that iSER requires - because it doesn't (though I as an implementer would require S-RQ in my implementation to realize the dampening).; Tue, 12 Oct 2004 14:06:41 -0400 (EDT) Received: from megatron.ietf.org ([132.151.6.71]) by ietf-mx.ietf.org with esmtp (Exim 4.33) id 1CHRDb-0003iZ-Jz for ips-web-archive@ietf.org; Tue, 12 Oct 2004 14:17:52 -0400 Received: from localhost.localdomain ([127.0.0.1] helo=megatron.ietf.org) by megatron.ietf.org with esmtp (Exim 4.32) id 1CHQtC-0005wu-Ca; Tue, 12 Oct 2004 13:56:46 -0400 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by megatron.ietf.org with esmtp (Exim 4.32) id 1CHQnu-0004KP-R9 for ips@megatron.ietf.org; Tue, 12 Oct 2004 13:51:20 -0400 Received: from ietf-mx.ietf.org (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id NAA13804 for ; Tue, 12 Oct 2004 13:51:18 -0400 (EDT) Received: from e32.co.us.ibm.com ([32.97.110.130]) by ietf-mx.ietf.org with esmtp (Exim 4.33) id 1CHQyh-0003Lu-Ew for ips@ietf.org; Tue, 12 Oct 2004 14:02:28 -0400 Received: from westrelay04.boulder.ibm.com (westrelay04.boulder.ibm.com [9.17.193.32]) by e32.co.us.ibm.com (8.12.10/8.12.9) with ESMTP id i9CHokqY703128; Tue, 12 Oct 2004 13:50:46 -0400 Received: from d03nmar1.almaden.ibm.com (d03av02.boulder.ibm.com [9.17.195.168]) by westrelay04.boulder.ibm.com (8.12.10/NCO/VER6.6) with ESMTP id i9CHoiX7023108; Tue, 12 Oct 2004 11:50:44 -0600 In-Reply-To: Importance: Normal MIME-Version: 1.0 To: "Alex Nezhinsky" Subject: Re: [Ips] iSER flow control options X-Mailer: Lotus Notes Release 6.5.2 June 01, 2004 Message-ID: From: Mike Ko Date: Tue, 12 Oct 2004 10:50:38 -0700 X-MIMETrack: Serialize by Router on D03NMAR1/03/M/IBM(Build V70_M2_07222004 Beta 2NP|July 22, 2004) at 10/12/2004 10:50:43, Serialize complete at 10/12/2004 10:50:43 Content-Type: text/plain; charset="US-ASCII" X-Spam-Score: 0.0 (/) X-Scan-Signature: 03169bfe4792634a390035a01a6c6d2f Cc: ips@ietf.org X-BeenThere: ips@ietf.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: IP Storage List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Sender: ips-bounces@ietf.org Errors-To: ips-bounces@ietf.org X-Spam-Score: 0.0 (/) X-Scan-Signature: 3fbd9b434023f8abfcb1532abaec7a21 Alex, In response to your statement that "such credit update scheme implies that iSER DataMover manages the session context (connection-session bindings, and the state variables incl. CmdSN, ExpCmdSN, MaxCmdSN), bear in mind that it is the iSCSI layer that manages CmdSN, ExpCmdSN, MaxCmdSN, etc., not the iSER layer. On the target side, for cases where the immediate flag is not set, the iSER layer must provision sufficient number of receive buffers to support the MaxCmdSN advertised by the iSCSI layer at the target. For multiple connections per session, how the iSER layer at each connection handle the number of receive buffers required per connection is implementation dependent. An SRQ can be used, but not required. Similarly, on the initiator side, the iSER layer can determine the number of receive buffers required since most control-type PDUs sent by the target are responses to requests sent by the initiator side. Therefore the flow control issue being debated here refers to those PDUs which are not regulated by the CmdSN flow. On the various schemes summarized by Mallikarjun, I agree with the general sentiment that option #3 (declared limit) should work, though it might not be the first choice for some. However, by setting the declared limit to a high enough value (or "None"), the limit essentailly goes away and implementations that can support option #1 (iSER/iSCSI layer being able to replenish the buffers as fast as the packet arrival rate) or #2 (RNIC being able to support graceful handling) would not be impeded by the declared limit. The key is to have the limit being declared instead of a stated value in the protocol (option #3a) to allow the implementation the flexibility, and to use existing iSCSI pacing mechanisms to advance the "credits" in order to avoid introducing yet another credit scheme into the iSER layer. The use of the declared limit has been discussed last year and I have summarized the main points in my note of October 4 and 5. To recap, from the initiator to the target, control-type PDUs with the immediate flag set and SNACK request are not regulated by the CmdSN mechanism. (SNACKs are not used in iSER.) From the target to the initiator, Async Messages and unsolicited NOP-in are sent not as a response to a request from the initiator. These are the unregulated flows which fall under the declared limit. I propose that we consider the following scheme to advance the credits set by the declared limit. ExpCmdSN andExpStatSN are used as pacing mechanisms. For the initiator, the ExpCmdSN from the target will be used to advance the credit. After sending PDUs with the immediate flag set, when the initiator sends any iSCSI control-type PDU with a new CmdSN without the immediate flag, if the target responds with an iSCSI control-type PDU with ExpCmdSN set to CmdSN + 1, then credit at the initiator is restored to its full value. For the target, the ExpStatSN from the initiator will be used to advance the credit. For unsolicited PDUs sent by the target, if the StatSN of that PDU is acknowledged by ExpStatSN, then the credit is restored. Comments? Mike Sent by: ips-bounces@ietf.org To: cc: Subject: Re: [Ips] iSER flow control options Hi, all Reading through the last messages in the "flow control" thread was quite helpful, but I still have a few questions. [Caitlin Bestler said> Shared Receive Queues are relevant here on two fronts. First, iSCSI flow control is really more session oriented than connection oriented. Pre-posting separate buffers to receive Cmd X on *each* connection is clearly over-buffering. This may be valid, because memory is cheap, but clearly it is not required to pre-commit duplicate buffers for each connection in a session. One possible solution is as follows: For the initiator, the ExpCmdSN from the target will be used to advance the credit... From what has been said previously on similar issues, I can see two replies to the above problem (if it is a problem): 1) Use S-RQ and all such problems are gone 2) At least in target, DataMover does not have to be "that" separated from the iSCSI layer. Well, I'll try to present our implementation issues, they may be of value also as a "yet another use case" for iSER. We are building a storage gateway between IB and FC. The current incarnation is based on iSER. We have an operational (buffer management is the major remaining issue) iSER DataMover linux kernel module, where iSER is built on top of IB rather than iWARP. It is built as a DataMover "service provider", capable to serve concurrently a number of iSCSI entities incl. initiator and/or target-type. Such capability is useful as it enables us to have the same code for both initiator and target platforms, streamlines debugging and can enable easy extention to a router-like environment where the same box acts both as initiator and target through different interfaces. We are in process of integration with the open-source linux-iscsi initiator project, which is to act as an iSCSI client of the iSER DataMover. I got into such level of detalization to stress that we already did not opt for intimate integration between the iSER/iSCSI layers, but rather tried to keep a reasonable level of separation between them. Being conscious about the ever-changing environment of RDMA transports, we build iSER DataMover on top of DAPL, as DAT is more general than any specific implementation of IB, iWARP etc. So we did know about the S-RQ design option, it's only that it is still not fully supported by the specs issued by the DAT collaborative. Though it surely will be supported in the (hopefully near) future, I guess this is an example of implementation which can't be based on S-RQ, at least for a time being. I'd note also that nothing either in the DataMover Architecture or iSER specs has warned us that the design should be based upon S-RQ, just as noted by [Mallikarjun C. said> I really would like us to view S-RQ as a "dampener" in invoking #2 (graceful handling), not as a mandatory feature that iSER requires - because it doesn't (though I as an implementer would require S-RQ in my implementation to realize the dampening).; Tue, 12 Oct 2004 15:05:18 -0400 (EDT) Received: from megatron.ietf.org ([132.151.6.71]) by ietf-mx.ietf.org with esmtp (Exim 4.33) id 1CHS8H-0004uG-48 for ips-web-archive@ietf.org; Tue, 12 Oct 2004 15:16:28 -0400 Received: from localhost.localdomain ([127.0.0.1] helo=megatron.ietf.org) by megatron.ietf.org with esmtp (Exim 4.32) id 1CHRua-00073X-MT; Tue, 12 Oct 2004 15:02:16 -0400 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by megatron.ietf.org with esmtp (Exim 4.32) id 1CHRqZ-0002mb-Vu for ips@megatron.ietf.org; Tue, 12 Oct 2004 14:58:08 -0400 Received: from ietf-mx.ietf.org (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id OAA19150 for ; Tue, 12 Oct 2004 14:58:07 -0400 (EDT) Received: from e32.co.us.ibm.com ([32.97.110.130]) by ietf-mx.ietf.org with esmtp (Exim 4.33) id 1CHS1M-0004ng-N5 for ips@ietf.org; Tue, 12 Oct 2004 15:09:17 -0400 Received: from westrelay01.boulder.ibm.com (westrelay01.boulder.ibm.com [9.17.195.10]) by e32.co.us.ibm.com (8.12.10/8.12.9) with ESMTP id i9CIvYqY572520 for ; Tue, 12 Oct 2004 14:57:34 -0400 Received: from d03av03.boulder.ibm.com (d03av03.boulder.ibm.com [9.17.195.169]) by westrelay01.boulder.ibm.com (8.12.10/NCO/VER6.6) with ESMTP id i9CIvYTu156094 for ; Tue, 12 Oct 2004 12:57:34 -0600 Received: from d03av03.boulder.ibm.com (loopback [127.0.0.1]) by d03av03.boulder.ibm.com (8.12.11/8.12.11) with ESMTP id i9CIvXo5015977 for ; Tue, 12 Oct 2004 12:57:34 -0600 Received: from d03nm115.boulder.ibm.com (d03nm115.boulder.ibm.com [9.17.195.141]) by d03av03.boulder.ibm.com (8.12.11/8.12.11) with ESMTP id i9CIvXMd015963; Tue, 12 Oct 2004 12:57:33 -0600 In-Reply-To: To: Mike Ko MIME-Version: 1.0 Subject: Re: [Ips] iSER flow control options X-Mailer: Lotus Notes Release 6.0.2CF1 June 9, 2003 From: John Hufferd Message-ID: Date: Tue, 12 Oct 2004 11:57:28 -0700 X-MIMETrack: Serialize by Router on D03NM115/03/M/IBM(Release 6.51HF338 | June 21, 2004) at 10/12/2004 12:57:32, Serialize complete at 10/12/2004 12:57:32 X-Spam-Score: 0.0 (/) X-Scan-Signature: 94902b99ee6852833c9a2b680a1de4d3 Cc: ips@ietf.org, Alex Nezhinsky X-BeenThere: ips@ietf.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: IP Storage List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Content-Type: multipart/mixed; boundary="===============1787275284==" Sender: ips-bounces@ietf.org Errors-To: ips-bounces@ietf.org X-Spam-Score: 0.0 (/) X-Scan-Signature: 4de5d7f989d6c039c8b887f1940f36ab This is a multipart message in MIME format. --===============1787275284== Content-Type: multipart/alternative; boundary="=_alternative 0068100388256F2B_=" This is a multipart message in MIME format. --=_alternative 0068100388256F2B_= Content-Type: text/plain; charset="US-ASCII" I agree with both Mallikarjun, and Mike. That is, my company will probably require vendors that OEM their RNIC to us, to provide "Graceful handling", and I expect that we and others will also exploit the value retained with non restricted iSCSI flow control and therefore will require the use of SRQs. However, as I said in my Oct 6, 2004 message I believe that there can be an additional declared parameter (the Keyword=Value pair "MaxOutstandingUnexpectedPDUs") which can take on values of 1-?????, including "None" (which means non restricted -- to be used in unrestricted iSCSI flow control). This Keyword=Value pair gives non shared RQ implementations, and InfiniBand implementations, which do not yet have SRQs in their HW, a reasonable way to be compatible with each other, yet not inhibit the creative implementation that can use the unrestricted iSCSI flow control. (Note: SRQ is now in the IB specifications.) . . John L. Hufferd Senior Technical Staff Member (STSM) IBM/System Group, San Jose CA Main Office: (408) 256-0403, Tie: 276-0403, eFax: (408) 904-4688 Alt Office: (408) 997-6136, Cell: (408) 499-9702 Internet Address: hufferd@us.ibm.com Mike Ko Sent by: ips-bounces@ietf.org 10/12/2004 10:50 AM To "Alex Nezhinsky" cc ips@ietf.org Subject Re: [Ips] iSER flow control options Alex, In response to your statement that "such credit update scheme implies that iSER DataMover manages the session context (connection-session bindings, and the state variables incl. CmdSN, ExpCmdSN, MaxCmdSN), bear in mind that it is the iSCSI layer that manages CmdSN, ExpCmdSN, MaxCmdSN, etc., not the iSER layer. On the target side, for cases where the immediate flag is not set, the iSER layer must provision sufficient number of receive buffers to support the MaxCmdSN advertised by the iSCSI layer at the target. For multiple connections per session, how the iSER layer at each connection handle the number of receive buffers required per connection is implementation dependent. An SRQ can be used, but not required. Similarly, on the initiator side, the iSER layer can determine the number of receive buffers required since most control-type PDUs sent by the target are responses to requests sent by the initiator side. Therefore the flow control issue being debated here refers to those PDUs which are not regulated by the CmdSN flow. On the various schemes summarized by Mallikarjun, I agree with the general sentiment that option #3 (declared limit) should work, though it might not be the first choice for some. However, by setting the declared limit to a high enough value (or "None"), the limit essentailly goes away and implementations that can support option #1 (iSER/iSCSI layer being able to replenish the buffers as fast as the packet arrival rate) or #2 (RNIC being able to support graceful handling) would not be impeded by the declared limit. The key is to have the limit being declared instead of a stated value in the protocol (option #3a) to allow the implementation the flexibility, and to use existing iSCSI pacing mechanisms to advance the "credits" in order to avoid introducing yet another credit scheme into the iSER layer. The use of the declared limit has been discussed last year and I have summarized the main points in my note of October 4 and 5. To recap, from the initiator to the target, control-type PDUs with the immediate flag set and SNACK request are not regulated by the CmdSN mechanism. (SNACKs are not used in iSER.) From the target to the initiator, Async Messages and unsolicited NOP-in are sent not as a response to a request from the initiator. These are the unregulated flows which fall under the declared limit. I propose that we consider the following scheme to advance the credits set by the declared limit. ExpCmdSN andExpStatSN are used as pacing mechanisms. For the initiator, the ExpCmdSN from the target will be used to advance the credit. After sending PDUs with the immediate flag set, when the initiator sends any iSCSI control-type PDU with a new CmdSN without the immediate flag, if the target responds with an iSCSI control-type PDU with ExpCmdSN set to CmdSN + 1, then credit at the initiator is restored to its full value. For the target, the ExpStatSN from the initiator will be used to advance the credit. For unsolicited PDUs sent by the target, if the StatSN of that PDU is acknowledged by ExpStatSN, then the credit is restored. Comments? Mike Sent by: ips-bounces@ietf.org To: cc: Subject: Re: [Ips] iSER flow control options Hi, all Reading through the last messages in the "flow control" thread was quite helpful, but I still have a few questions. [Caitlin Bestler said> Shared Receive Queues are relevant here on two fronts. First, iSCSI flow control is really more session oriented than connection oriented. Pre-posting separate buffers to receive Cmd X on *each* connection is clearly over-buffering. This may be valid, because memory is cheap, but clearly it is not required to pre-commit duplicate buffers for each connection in a session. One possible solution is as follows: For the initiator, the ExpCmdSN from the target will be used to advance the credit... From what has been said previously on similar issues, I can see two replies to the above problem (if it is a problem): 1) Use S-RQ and all such problems are gone 2) At least in target, DataMover does not have to be "that" separated from the iSCSI layer. Well, I'll try to present our implementation issues, they may be of value also as a "yet another use case" for iSER. We are building a storage gateway between IB and FC. The current incarnation is based on iSER. We have an operational (buffer management is the major remaining issue) iSER DataMover linux kernel module, where iSER is built on top of IB rather than iWARP. It is built as a DataMover "service provider", capable to serve concurrently a number of iSCSI entities incl. initiator and/or target-type. Such capability is useful as it enables us to have the same code for both initiator and target platforms, streamlines debugging and can enable easy extention to a router-like environment where the same box acts both as initiator and target through different interfaces. We are in process of integration with the open-source linux-iscsi initiator project, which is to act as an iSCSI client of the iSER DataMover. I got into such level of detalization to stress that we already did not opt for intimate integration between the iSER/iSCSI layers, but rather tried to keep a reasonable level of separation between them. Being conscious about the ever-changing environment of RDMA transports, we build iSER DataMover on top of DAPL, as DAT is more general than any specific implementation of IB, iWARP etc. So we did know about the S-RQ design option, it's only that it is still not fully supported by the specs issued by the DAT collaborative. Though it surely will be supported in the (hopefully near) future, I guess this is an example of implementation which can't be based on S-RQ, at least for a time being. I'd note also that nothing either in the DataMover Architecture or iSER specs has warned us that the design should be based upon S-RQ, just as noted by [Mallikarjun C. said> I really would like us to view S-RQ as a "dampener" in invoking #2 (graceful handling), not as a mandatory feature that iSER requires - because it doesn't (though I as an implementer would require S-RQ in my implementation to realize the dampening).I agree with both Mallikarjun, and Mike.  That is, my company will probably require vendors that OEM their RNIC to us, to provide "Graceful handling", and I expect that we and others will also exploit the value retained with non restricted iSCSI flow control and therefore will require the use of SRQs.  However, as I said in my Oct 6, 2004 message I believe that there can be an additional declared parameter (the Keyword=Value pair "MaxOutstandingUnexpectedPDUs") which can take on values of 1-?????, including "None" (which means non restricted  -- to be used in unrestricted iSCSI flow control).  This Keyword=Value pair gives non shared RQ implementations, and InfiniBand implementations, which do not yet have SRQs in their HW,  a reasonable way to be compatible with each other, yet not inhibit the creative implementation that can use the unrestricted iSCSI flow control. (Note: SRQ is now in the IB specifications.)
.
.
John L. Hufferd
Senior Technical Staff Member (STSM)
IBM/System Group, San Jose CA
Main Office: (408) 256-0403, Tie: 276-0403, eFax: (408) 904-4688
Alt Office: (408) 997-6136, Cell: (408) 499-9702
Internet Address: hufferd@us.ibm.com


Mike Ko <mako@almaden.ibm.com>
Sent by: ips-bounces@ietf.org

10/12/2004 10:50 AM

To
"Alex Nezhinsky" <alexn@voltaire.com>
cc
ips@ietf.org
Subject
Re: [Ips] iSER flow control options





Alex,

In response to your statement that "such credit update scheme implies that
iSER DataMover manages the session context (connection-session bindings,
and the state variables incl. CmdSN, ExpCmdSN, MaxCmdSN), bear in mind
that it is the iSCSI layer that manages CmdSN, ExpCmdSN, MaxCmdSN, etc.,
not the iSER layer.  On the target side, for cases where the immediate
flag is not set, the iSER layer must provision sufficient number of
receive buffers to support the MaxCmdSN advertised by the iSCSI layer at
the target.  For multiple connections per session, how the iSER layer at
each connection handle the number of receive buffers required per
connection is implementation dependent.  An SRQ can be used, but not
required.  Similarly, on the initiator side, the iSER layer can determine
the number of receive buffers required since most control-type PDUs sent
by the target are responses to requests sent by the initiator side.
Therefore the flow control issue being debated here refers to those PDUs
which are not regulated by the CmdSN flow.

On the various schemes summarized by Mallikarjun, I agree with the general
sentiment that option #3 (declared limit) should work, though it might not
be the first choice for some.  However, by setting the declared limit to a
high enough value (or "None"), the limit essentailly goes away and
implementations that can support option #1 (iSER/iSCSI layer being able to
replenish the buffers as fast as the packet arrival rate) or #2 (RNIC
being able to support graceful handling) would not be impeded by the
declared limit.  The key is to have the limit being declared instead of a
stated value in the protocol (option #3a) to allow the implementation the
flexibility, and to use existing iSCSI pacing mechanisms to advance the
"credits" in order to avoid introducing yet another credit scheme into the
iSER layer.

The use of the declared limit has been discussed last year and I have
summarized the main points in my note of October 4 and 5.  To recap, from
the initiator to the target, control-type PDUs with the immediate flag set
and SNACK request are not regulated by the CmdSN mechanism.  (SNACKs are
not used in iSER.)  From the target to the initiator, Async Messages and
unsolicited NOP-in are sent not as a response to a request from the
initiator.  These are the unregulated flows which fall under the declared
limit.  I propose that we consider the following scheme to advance the
credits set by the declared limit.  ExpCmdSN andExpStatSN are used as
pacing mechanisms.

For the initiator, the ExpCmdSN from the target will be used to advance
the credit.  After sending PDUs with the immediate flag set, when the
initiator sends any iSCSI control-type PDU with a new CmdSN without the
immediate flag, if the target responds with an iSCSI control-type PDU with
ExpCmdSN set to CmdSN + 1, then credit at the initiator is restored to its
full value.

For the target, the ExpStatSN from the initiator will be used to advance
the credit.  For unsolicited PDUs sent by the target, if the StatSN of
that PDU is acknowledged by ExpStatSN, then the credit is restored.  

Comments?

Mike
Sent by:        ips-bounces@ietf.org
To:     <ips@ietf.org>
cc:      
Subject:        Re: [Ips] iSER flow control options



Hi, all
Reading through the last messages in the "flow control" thread was quite
helpful, but I still have a few questions.

[Caitlin Bestler said> Shared Receive Queues are relevant here on two
fronts. First, iSCSI flow control is really more session oriented than
connection oriented. Pre-posting separate buffers to receive Cmd X on
*each* connection is clearly over-buffering. This may be valid, because
memory is cheap, but clearly it is not required to pre-commit duplicate
buffers for each connection in a session. <Caitlin Bestler]

This has alerted me with respect to the suggested credit update scheme:
[Mike Ko said> One possible solution is as follows: For the initiator,
the ExpCmdSN from the target will be used to advance the credit... <Mike
Ko]

Such credit update scheme implies that iSER DataMover manages the
session context (connection-session bindings, and the state variables
incl. CmdSN,ExpCmdSN,MaxCmdSN). Attempting not to overcommit buffers
when posting based on MaxCmdSN brings up the same requirement.
This information is routinely maintained by the iSCSI layer. There is no
API defined between the iSCSI and DataMover, which can tell a DataMover
about the session context information. If I understand it right, iSER as
a DataMover was intended to operate solely on connections without
bothering about sessions. Not only recovering session-scope information
from the passing PDUs is actually a duplication of the job already done
by iSCSI. iSER is not supposed to see the Login phase (where the session
ids and handles are exchanged) at all. Of course, connection-session
binding may be passed "somehow" (e.g. as faked keys of
Notice_Key_Values), but I guess that this wasn't the original intention.

Did I overlook anything?

>From what has been said previously on similar issues, I can see two
replies to the above problem (if it is a problem):
1) Use S-RQ and all such problems are gone
2) At least in target, DataMover does not have to be "that" separated
from the iSCSI layer.

Well, I'll try to present our implementation issues, they may be of
value also as a "yet another use case" for iSER.

We are building a storage gateway between IB and FC. The current
incarnation is based on iSER.
We have an operational (buffer management is the major remaining issue)
iSER DataMover linux kernel module, where iSER is built on top of IB
rather than iWARP. It is built as a DataMover "service provider",
capable to serve concurrently a number of iSCSI entities incl. initiator
and/or target-type. Such capability is useful as it enables us to have
the same code for both initiator and target platforms, streamlines
debugging and can enable easy extention to a router-like environment
where the same box acts both as initiator and target through different
interfaces.
We are in process of integration with the open-source linux-iscsi
initiator project, which is to act as an iSCSI client of the iSER
DataMover.
I got into such level of detalization to stress that we already did not
opt for intimate integration between the iSER/iSCSI layers, but rather
tried to keep a reasonable level of separation between them.

Being conscious about the ever-changing environment of RDMA transports,
we build iSER DataMover on top of DAPL, as DAT is more general than any
specific implementation of IB, iWARP etc. So we did know about the S-RQ
design option, it's only that it is still not fully supported by the
specs issued by the DAT collaborative. Though it surely will be
supported in the (hopefully near) future, I guess this is an example of
implementation which can't be based on S-RQ, at least for a time being.
I'd note also that nothing either in the DataMover Architecture or iSER
specs has warned us that the design should be based upon S-RQ, just as
noted by  [Mallikarjun C. said> I really would like us to view S-RQ as a
"dampener" in invoking #2 (graceful handling), not as a mandatory
feature that iSER requires - because it doesn't (though I as an
implementer would require S-RQ in my implementation to realize the
dampening).<Mallikarjun C.]

So it is currently a non S-RQ based implementation. As such, we are
running into a problem, how to manage the buffer credits (along the
lines of the latest discussion, plus see my question above).
Another, more subtle one is that the resulting design seems to differ
substantially from what would be necessary with a S-RQ implementation.
The fact that using a S-RQ can't be extended easily from a non S-RQ
implementation strikes me as non-elegant, to say the least.

I'd appreciate your comments and suggestions.

Alexander Nezhinsky

_______________________________________________
Ips mailing list
Ips@ietf.org
https://www1.ietf.org/mailman/listinfo/ips


_______________________________________________
Ips mailing list
Ips@ietf.org
https://www1.ietf.org/mailman/listinfo/ips
--=_alternative 0068100388256F2B_=-- --===============1787275284== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline Content-Transfer-Encoding: 7bit _______________________________________________ Ips mailing list Ips@ietf.org https://www1.ietf.org/mailman/listinfo/ips --===============1787275284==-- From ips-bounces@ietf.org Tue Oct 12 19:07:06 2004 Received: from ietf-mx.ietf.org (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id TAA26042 for ; Tue, 12 Oct 2004 19:07:06 -0400 (EDT) Received: from megatron.ietf.org ([132.151.6.71]) by ietf-mx.ietf.org with esmtp (Exim 4.33) id 1CHVuO-000698-8y for ips-web-archive@ietf.org; Tue, 12 Oct 2004 19:18:20 -0400 Received: from localhost.localdomain ([127.0.0.1] helo=megatron.ietf.org) by megatron.ietf.org with esmtp (Exim 4.32) id 1CHVi5-0001Kb-LR; Tue, 12 Oct 2004 19:05:37 -0400 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by megatron.ietf.org with esmtp (Exim 4.32) id 1CHVdX-0007zx-LM for ips@megatron.ietf.org; Tue, 12 Oct 2004 19:00:55 -0400 Received: from ietf-mx.ietf.org (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id TAA25467 for ; Tue, 12 Oct 2004 19:00:53 -0400 (EDT) Received: from atlrel6.hp.com ([156.153.255.205]) by ietf-mx.ietf.org with esmtp (Exim 4.33) id 1CHVoC-00061j-W9 for ips@ietf.org; Tue, 12 Oct 2004 19:12:07 -0400 Received: from rosemail.rose.hp.com (rosemail.rose.hp.com [15.43.209.160]) by atlrel6.hp.com (Postfix) with ESMTP id 555EF225D for ; Tue, 12 Oct 2004 19:00:43 -0400 (EDT) Received: from [127.0.0.1] (dhcp-roseor-bdj092.rose.hp.com [15.23.139.92]) by rosemail.rose.hp.com (Postfix) with ESMTP id 081B27FF4 for ; Tue, 12 Oct 2004 16:00:43 -0700 (PDT) Message-ID: <416C6214.9080408@rose.hp.com> Date: Tue, 12 Oct 2004 16:00:36 -0700 From: "Mallikarjun C." User-Agent: Mozilla Thunderbird 0.7.3 (Windows/20040803) X-Accept-Language: en-us, en MIME-Version: 1.0 To: ips@ietf.org Subject: Re: [Ips] resending Data-in with S bit set References: <41649B60.60002@rose.hp.com> <4165785D.5050802@rose.hp.com> <4166D357.40708@rose.hp.com> <4489F857-1956-11D9-91D4-000A95D14BEC@wasabisystems.com> <4166E5B0.1090506@rose.hp.com> <30FE007A-196F-11D9-91D4-000A95D14BEC@wasabisystems.com> <416B0413.8080208@rose.hp.com> <3DB9AEB1-1BEA-11D9-8B4B-000A95D14BEC@wasabisystems.com> In-Reply-To: <3DB9AEB1-1BEA-11D9-8B4B-000A95D14BEC@wasabisystems.com> Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit X-Spam-Score: 0.0 (/) X-Scan-Signature: 2beba50d0fcdeee5f091c59f204d4365 Content-Transfer-Encoding: 7bit X-BeenThere: ips@ietf.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: IP Storage List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Sender: ips-bounces@ietf.org Errors-To: ips-bounces@ietf.org X-Spam-Score: 0.0 (/) X-Scan-Signature: dbb8771284c7a36189745aa720dc20ab Content-Transfer-Encoding: 7bit Bill, >there should be a > disconnect between ExpDataSN on the target (sent in the SCSI Response) > and in the initiator. Yes, that should work for the scenario you described (and in other cases that I can now think of). In general, during the time we designed the iSCSI resegmentation approach, the operating assumption as I recall was to not rely on absolute sequence numbers from the previous epoch (i.e. before the PDU size change) after the change. The R-Data SNACK simply causes a reversion to the last ack point which is consistent with not using absolute SNs from the past. The thinking then was that between possibly multiple PDU size changes during the life of a task and and sharing of the DataSN/R2TSN space for bidirectional commands, there's enough room for implementations to get the absolute sequence numbers wrong in their SNACK requests. Besides, we (at least I) didn't feel like we have exhaustively enumerated all potential scenarios. As far as I recall, we have not defined a quiescing process to change the PDU size either. I'd of course like to hear from other participants of those discussions wrt this recollection. At least this is my rationale for suggesting a big hammer approach that doesn't rely on the ExpDataSN field in the status PDU (BTW, I don't recall a discussion on ExpDataSN in the past). But I do realize that in my approach, under this scenario, the ExpDataSN field in the Response PDU is always suspect - so a SNACK is always required or the I/O would fail. Thanks. -- Mallikarjun Mallikarjun Chadalapaka Networked Storage Architecture Network Storage Solutions Hewlett-Packard MS 5668 Roseville CA 95747 cbm [at] rose.hp.com William Studenmund wrote: > On Oct 11, 2004, at 6:07 PM, Mallikarjun C. wrote: > >> I think I may be finally understanding your scenario, but let's >> confirm that.... > > > Ok! > >> A read SCSI task is in progress and the initiator changes >> MaxRecvDataSegementLength. > > > Specifically, for what I'm thinking of, all of the data have already > been sent before the MaxRecvDataSegmentLength > >> The task continues. Subsequently, the initiator notices it lost a >> status. Here's what I think should happen (I think the same as what >> you described, except for the ExpDataSN part). >> >> - As always, initiator starts with a status SNACK. >> - Regardless of the original transmission (phase collapse or not, old >> PDU still fits with the new size or not, single/multiple data PDUs), >> target retransmits the status in an explicit Response PDU with >> ExpDataSN set to something like 0xffffffff. >> - Initiator checks ExpDataSN with its records. They shouldn't match, >> leading to an R-Data SNACK. >> - Rest of the semantics described in 10.16.3 apply. >> >> My reasons are two: >> >> - Target always does the same thing in this scenario (PDU size change >> followed by a status SNACK first) - i.e. doesn't phase collapse even >> if it still can. > > > How about this idea. If the target can still phase collapse (the PDU > would still fit), it MAY, but due to the fact that there was a > resegmentation, the target MAY instead treat the PDU as if it could not > fit. That is to say it is an implementation decision if the target just > remembers "resegmented" or if it looks at the PDU. > >> - Initiator operates based on ExpDataSN as always, consistent with the >> spec intent to not have to do scoreboarding/counting. > > > How does what I suggest break ExpDataSN? > > I was about to say I like what you suggested, then I pondered a bit. I > don't think we need the magic ExpDataSN value. If the PDU is too big to > send, then the target can just generate a SCSI Response PDU with the > correct ExpDataSN and a SNACK Tag of 0. Given that the initiator didn't > notice the status value in the phase-collapsed read PDU, it should also > have not noticed the data reception in it, so there should be a > disconnect between ExpDataSN on the target (sent in the SCSI Response) > and in the initiator. So a SCSI Response sent in place of the collapsed > Data-In should trigger the initiator to send an R-Data SNACK. Shouldn't it? > >> OTOH, the RFC doesn't set aside 0xffffffff as a reserved value for >> ExpDataSN. So I guess this is a hack for now. Alternatively (or if >> target had already sent 2**32-1 PDUs for that task), what you >> described works as well (if the initiator is counting bytes). > > > I don't see how you have to count bytes. I'm not trying to sound > belligerent, and I'm sorry if it comes across that way. But I think just > sending the SCSI Response would work. I'm not seeing how the # of > bytes'd be needed. > > The one thing that would be impacted would be a broken initiator that > got the data out of this phase-collapsed-read Data-In PDU but not the > status. I think it deserves what it gets... > > Take care, > > Bill _______________________________________________ Ips mailing list Ips@ietf.org https://www1.ietf.org/mailman/listinfo/ips From ips-bounces@ietf.org Tue Oct 12 20:38:58 2004 Received: from ietf-mx.ietf.org (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id UAA02979 for ; Tue, 12 Oct 2004 20:38:58 -0400 (EDT) Received: from megatron.ietf.org ([132.151.6.71]) by ietf-mx.ietf.org with esmtp (Exim 4.33) id 1CHXLG-0007q8-UJ for ips-web-archive@ietf.org; Tue, 12 Oct 2004 20:50:11 -0400 Received: from localhost.localdomain ([127.0.0.1] helo=megatron.ietf.org) by megatron.ietf.org with esmtp (Exim 4.32) id 1CHX8G-0004hT-Qr; Tue, 12 Oct 2004 20:36:44 -0400 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by megatron.ietf.org with esmtp (Exim 4.32) id 1CHX0P-00022a-9d for ips@megatron.ietf.org; Tue, 12 Oct 2004 20:28:37 -0400 Received: from ietf-mx.ietf.org (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id UAA02072 for ; Tue, 12 Oct 2004 20:28:35 -0400 (EDT) From: pat_thaler@agilent.com Received: from msgbas1x.cos.agilent.com ([192.25.240.36]) by ietf-mx.ietf.org with esmtp (Exim 4.33) id 1CHXB9-0007dG-Jn for ips@ietf.org; Tue, 12 Oct 2004 20:39:49 -0400 Received: from enccos5.cos.agilent.com (enccos5.cos.agilent.com [130.29.152.94]) by msgbas1x.cos.agilent.com (Postfix) with SMTP id 3775127954 for ; Tue, 12 Oct 2004 18:28:29 -0600 (MDT) Received: from relcos1.cos.agilent.com (130.29.152.239) by enccos5.cos.agilent.com (Sigaba Gateway v3.83) with ESMTP id 2186332; Tue, 12 Oct 2004 18:29:03 -0600 Received: from wcosvs02.cos.agilent.com (wcosvs02.cos.agilent.com [130.29.152.188]) by relcos1.cos.agilent.com (Postfix) with ESMTP id 07AB8C0 for ; Tue, 12 Oct 2004 18:28:29 -0600 (MDT) Received: from wcosbh22.cos.agilent.com ([130.29.152.178]) by wcosvs02.cos.agilent.com with InterScan Messaging Security Suite; Tue, 12 Oct 2004 18:28:28 -0600 Received: from wcosmb05.cos.agilent.com ([130.29.152.72]) by wcosbh22.cos.agilent.com with Microsoft SMTPSVC(5.0.2195.6713); Tue, 12 Oct 2004 18:28:28 -0600 X-MimeOLE: Produced By Microsoft Exchange V6.0.6603.0 Content-class: urn:content-classes:message MIME-Version: 1.0 Subject: RE: [Ips] iSER flow control options Date: Tue, 12 Oct 2004 18:28:27 -0600 Message-ID: <9418406DEC3B8A4BBDB1B87291D851AF0E7552@wcosmb05.cos.agilent.com> Thread-Topic: [Ips] iSER flow control options Thread-Index: AcSwjpXilaozRruwSsKlSBGlGnUKxwAInx2g To: , X-OriginalArrivalTime: 13 Oct 2004 00:28:28.0126 (UTC) FILETIME=[8F35D7E0:01C4B0BB] X-Spam-Score: 0.5 (/) X-Scan-Signature: 86a4d7ebdc337882c86d755b1c60a122 Cc: ips@ietf.org, alexn@voltaire.com X-BeenThere: ips@ietf.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: IP Storage List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Content-Type: multipart/mixed; boundary="===============1390949142==" Sender: ips-bounces@ietf.org Errors-To: ips-bounces@ietf.org X-Spam-Score: 0.5 (/) X-Scan-Signature: 6d231d878f8d68d85cf12b60d23450ce This is a multi-part message in MIME format. --===============1390949142== Content-class: urn:content-classes:message Content-Type: multipart/alternative; boundary="----_=_NextPart_001_01C4B0BB.8E9669D7" This is a multi-part message in MIME format. ------_=_NextPart_001_01C4B0BB.8E9669D7 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable John, =20 This is an interesting idea, but I have some concerns about it. It isn't = clear that the sender has a way to reliably know when an unexpected PDU = is no longer outstanding and there may be lock-up conditions possible = (both sides have the maximum number of outstanding unexpected PDUs, the = initiator has no credit (e.g. maxCmdSN) to send an expected PDU and the = target has no expected PDU to send. =20 End of outstanding status: This is easy for some unexpected PDU types - anything from the target = that advances StatSN can be considered not-outstanding when ExpSN = acknowledges it. I'll use "retired" to mean that a PDU is no longer = outstanding. =20 It is less clear how to do it for some other types.=20 A unidirectional NOP-Out or a NOP-Out that responds to a NOP-In has no = response from the target so there is no way to tell when it is retired. A unidirectional NOP-In doesn't get a response and doesn't advance = StatSN so there is no means to tell when it is retired.=20 SNACK Request - it causes a retransmission of something, but when the = initiator receives the requested PDUs, I don't think it can always know = whether they were sent in response to the SNACK vs. being the delayed = original PDU or a copy resent due to a time out at the target. Immediate Commands - they may be rejected by the target because of = insufficient resources. There is a reject code for too many immediate = commands, but there isn't any way to tie it to the rejected command and = reading the text, I'm not sure whether it is sent once for each rejected = immediate command or once for multiple immediate commands. There may = also be cases where the target doesn't even have the resources to send = the reject. Therefore, immediate commands may not get a response that = allows them to be retired. Reject - I'm not sure if all rejects should be considered unexpected. = Possibly the ones that are sent in place of a response (e.g. the reject = that is sent in response to a text negotiation or a command that can't = be initiated) can be considered expected. Probably it is best to take = the simple course and count all of them as unexpected. In any case, it = appears that they don't advance StatSN (I think - the standard isn't = terribly clear here) so they aren't acknowledged and there is no way to = retire them. =20 Reject and StatSN aside - 10.4.7 says "StatSN is incremented by one for = every response/status ...."=20 10.17.3 says that StatSN carries the usual value=20 Since reject is not a response or a status, it appears that it is not = incremented for rejects. But asynchronous messages advance StatSN even = though they aren't a response/status so I'm not sure if this was what we = intended or if it was an ommission in the definition of reject. =20 Regards, Pat =20 =20 -----Original Message----- From: ips-bounces@ietf.org [mailto:ips-bounces@ietf.org]On Behalf Of = John Hufferd Sent: Tuesday, 12 October, 2004 11:57 AM To: Mike Ko Cc: ips@ietf.org; Alex Nezhinsky Subject: Re: [Ips] iSER flow control options I agree with both Mallikarjun, and Mike. That is, my company will = probably require vendors that OEM their RNIC to us, to provide "Graceful = handling", and I expect that we and others will also exploit the value = retained with non restricted iSCSI flow control and therefore will = require the use of SRQs. However, as I said in my Oct 6, 2004 message I = believe that there can be an additional declared parameter (the = Keyword=3DValue pair "MaxOutstandingUnexpectedPDUs") which can take on = values of 1-?????, including "None" (which means non restricted -- to = be used in unrestricted iSCSI flow control). This Keyword=3DValue pair = gives non shared RQ implementations, and InfiniBand implementations, = which do not yet have SRQs in their HW, a reasonable way to be = compatible with each other, yet not inhibit the creative implementation = that can use the unrestricted iSCSI flow control. (Note: SRQ is now in = the IB specifications.)=20 . . John L. Hufferd Senior Technical Staff Member (STSM) IBM/System Group, San Jose CA Main Office: (408) 256-0403, Tie: 276-0403, eFax: (408) 904-4688 Alt Office: (408) 997-6136, Cell: (408) 499-9702 Internet Address: hufferd@us.ibm.com=20 Mike Ko =20 Sent by: ips-bounces@ietf.org=20 10/12/2004 10:50 AM=20 To "Alex Nezhinsky" =20 cc ips@ietf.org=20 Subject Re: [Ips] iSER flow control options =09 Alex, In response to your statement that "such credit update scheme implies = that=20 iSER DataMover manages the session context (connection-session bindings, = and the state variables incl. CmdSN, ExpCmdSN, MaxCmdSN), bear in mind=20 that it is the iSCSI layer that manages CmdSN, ExpCmdSN, MaxCmdSN, etc., = not the iSER layer. On the target side, for cases where the immediate=20 flag is not set, the iSER layer must provision sufficient number of=20 receive buffers to support the MaxCmdSN advertised by the iSCSI layer at = the target. For multiple connections per session, how the iSER layer at = each connection handle the number of receive buffers required per=20 connection is implementation dependent. An SRQ can be used, but not=20 required. Similarly, on the initiator side, the iSER layer can = determine=20 the number of receive buffers required since most control-type PDUs sent = by the target are responses to requests sent by the initiator side.=20 Therefore the flow control issue being debated here refers to those PDUs = which are not regulated by the CmdSN flow. On the various schemes summarized by Mallikarjun, I agree with the = general=20 sentiment that option #3 (declared limit) should work, though it might = not=20 be the first choice for some. However, by setting the declared limit to = a=20 high enough value (or "None"), the limit essentailly goes away and=20 implementations that can support option #1 (iSER/iSCSI layer being able = to=20 replenish the buffers as fast as the packet arrival rate) or #2 (RNIC=20 being able to support graceful handling) would not be impeded by the=20 declared limit. The key is to have the limit being declared instead of = a=20 stated value in the protocol (option #3a) to allow the implementation = the=20 flexibility, and to use existing iSCSI pacing mechanisms to advance the=20 "credits" in order to avoid introducing yet another credit scheme into = the=20 iSER layer. The use of the declared limit has been discussed last year and I have=20 summarized the main points in my note of October 4 and 5. To recap, = from=20 the initiator to the target, control-type PDUs with the immediate flag = set=20 and SNACK request are not regulated by the CmdSN mechanism. (SNACKs are = not used in iSER.) From the target to the initiator, Async Messages and = unsolicited NOP-in are sent not as a response to a request from the=20 initiator. These are the unregulated flows which fall under the = declared=20 limit. I propose that we consider the following scheme to advance the=20 credits set by the declared limit. ExpCmdSN andExpStatSN are used as=20 pacing mechanisms. For the initiator, the ExpCmdSN from the target will be used to advance=20 the credit. After sending PDUs with the immediate flag set, when the=20 initiator sends any iSCSI control-type PDU with a new CmdSN without the immediate flag, if the target responds with an iSCSI control-type PDU = with=20 ExpCmdSN set to CmdSN + 1, then credit at the initiator is restored to = its=20 full value.=20 For the target, the ExpStatSN from the initiator will be used to advance = the credit. For unsolicited PDUs sent by the target, if the StatSN of=20 that PDU is acknowledged by ExpStatSN, then the credit is restored. =20 Comments? Mike Sent by: ips-bounces@ietf.org To: cc: =20 Subject: Re: [Ips] iSER flow control options Hi, all Reading through the last messages in the "flow control" thread was quite helpful, but I still have a few questions. [Caitlin Bestler said> Shared Receive Queues are relevant here on two fronts. First, iSCSI flow control is really more session oriented than connection oriented. Pre-posting separate buffers to receive Cmd X on *each* connection is clearly over-buffering. This may be valid, because memory is cheap, but clearly it is not required to pre-commit duplicate buffers for each connection in a session. One possible solution is as follows: For the initiator, the ExpCmdSN from the target will be used to advance the credit... From what has been said previously on similar issues, I can see two replies to the above problem (if it is a problem): 1) Use S-RQ and all such problems are gone 2) At least in target, DataMover does not have to be "that" separated from the iSCSI layer. Well, I'll try to present our implementation issues, they may be of value also as a "yet another use case" for iSER. We are building a storage gateway between IB and FC. The current incarnation is based on iSER. We have an operational (buffer management is the major remaining issue) iSER DataMover linux kernel module, where iSER is built on top of IB rather than iWARP. It is built as a DataMover "service provider", capable to serve concurrently a number of iSCSI entities incl. initiator and/or target-type. Such capability is useful as it enables us to have the same code for both initiator and target platforms, streamlines debugging and can enable easy extention to a router-like environment where the same box acts both as initiator and target through different interfaces. We are in process of integration with the open-source linux-iscsi initiator project, which is to act as an iSCSI client of the iSER DataMover. I got into such level of detalization to stress that we already did not opt for intimate integration between the iSER/iSCSI layers, but rather tried to keep a reasonable level of separation between them. Being conscious about the ever-changing environment of RDMA transports, we build iSER DataMover on top of DAPL, as DAT is more general than any specific implementation of IB, iWARP etc. So we did know about the S-RQ design option, it's only that it is still not fully supported by the specs issued by the DAT collaborative. Though it surely will be supported in the (hopefully near) future, I guess this is an example of implementation which can't be based on S-RQ, at least for a time being. I'd note also that nothing either in the DataMover Architecture or iSER specs has warned us that the design should be based upon S-RQ, just as noted by [Mallikarjun C. said> I really would like us to view S-RQ as a "dampener" in invoking #2 (graceful handling), not as a mandatory feature that iSER requires - because it doesn't (though I as an implementer would require S-RQ in my implementation to realize the dampening).

John,
 
This = is an=20 interesting idea, but I have some concerns about it. It isn't clear that = the=20 sender has a way to reliably know when an unexpected PDU is no longer=20 outstanding and there may be lock-up conditions possible (both sides = have the=20 maximum number of outstanding unexpected PDUs, the initiator has no = credit (e.g.=20 maxCmdSN) to send an expected PDU and the target has no expected PDU to=20 send.
 
End of = outstanding=20 status:
This = is easy for=20 some unexpected PDU types - anything from the target that advances = StatSN can be=20 considered not-outstanding when ExpSN acknowledges it. I'll use = "retired" to=20 mean that a PDU is no longer outstanding.
 
It is = less clear how=20 to do it for some other types.
A = unidirectional=20 NOP-Out or a NOP-Out that responds to a NOP-In has no response from = the=20 target so there is no way to tell when it is = retired.
A unidirectional NOP-In doesn't get a response = and doesn't=20 advance StatSN so there is no means to tell when it is retired.=20
SNACK = Request - it=20 causes a retransmission of something, but when the initiator receives = the=20 requested PDUs, I don't think it can always know whether they were sent = in=20 response to the SNACK vs. being the delayed original PDU or a copy = resent due to=20 a time out at the target.
Immediate Commands -=20 they may be rejected by the target because of insufficient resources. = There is a=20 reject code for too many immediate commands, but there isn't any way to = tie it=20 to the rejected command and reading the text, I'm not sure whether it is = sent=20 once for each rejected immediate command or once for multiple immediate=20 commands. There may also be cases where the target doesn't even have the = resources to send the reject. Therefore, immediate commands may not get = a=20 response that allows them to be retired.
Reject = - I'm not=20 sure if all rejects should be considered unexpected. Possibly the = ones that=20 are sent in place of a response (e.g. the reject that is sent in = response to a=20 text negotiation or a command that can't be initiated) can be considered = expected. Probably it is best to take the simple course and count all of = them as=20 unexpected. In any case, it appears that they don't advance StatSN = (I think=20 - the standard isn't terribly clear here) so they aren't acknowledged = and there=20 is no way to retire them.
 
Reject = and StatSN=20 aside - 10.4.7 says "StatSN is incremented by one for every = response/status=20 ...."
10.17.3 says that=20 StatSN carries the usual value 
Since = reject is not=20 a response or a status, it appears that it is not incremented for = rejects. But=20 asynchronous messages advance StatSN even though they aren't a = response/status=20 so I'm not sure if this was what we intended or if it was an ommission = in the=20 definition of reject.
 
Regards,
Pat
 
 
-----Original Message-----
From: = ips-bounces@ietf.org=20 [mailto:ips-bounces@ietf.org]On Behalf Of John = Hufferd
Sent:=20 Tuesday, 12 October, 2004 11:57 AM
To: Mike Ko
Cc: = ips@ietf.org; Alex Nezhinsky
Subject: Re: [Ips] iSER flow = control=20 options


I = agree with both=20 Mallikarjun, and Mike.  That is, my company will probably require = vendors=20 that OEM their RNIC to us, to provide "Graceful handling", and I = expect that=20 we and others will also exploit the value retained with non restricted = iSCSI=20 flow control and therefore will require the use of SRQs. =  However, as I=20 said in my Oct 6, 2004 message I believe that there can be an = additional=20 declared parameter (the Keyword=3DValue pair = "MaxOutstandingUnexpectedPDUs")=20 which can take on values of 1-?????, including "None" (which means non = restricted  -- to be used in unrestricted iSCSI flow control). =  This=20 Keyword=3DValue pair gives non shared RQ implementations, and = InfiniBand=20 implementations, which do not yet have SRQs in their HW,  a = reasonable=20 way to be compatible with each other, yet not inhibit the creative=20 implementation that can use the unrestricted iSCSI flow control. = (Note: SRQ is=20 now in the IB specifications.)
.
.
John L. Hufferd
Senior Technical Staff Member=20 (STSM)
IBM/System Group, San Jose CA
Main Office: (408) = 256-0403, Tie:=20 276-0403, eFax: (408) 904-4688
Alt Office: (408) 997-6136, Cell: = (408)=20 499-9702
Internet Address: hufferd@us.ibm.com


Mike Ko=20 <mako@almaden.ibm.com>
Sent by: ips-bounces@ietf.org=20

10/12/2004 10:50 AM =

To
"Alex = Nezhinsky"=20 <alexn@voltaire.com>=20
cc
ips@ietf.org=20
Subject
Re: [Ips] = iSER flow=20 control options

=




Alex,

In response to your statement that "such = credit update=20 scheme implies that
iSER DataMover manages the session context=20 (connection-session bindings,
and the state variables incl. CmdSN, = ExpCmdSN, MaxCmdSN), bear in mind
that it is the iSCSI layer that = manages=20 CmdSN, ExpCmdSN, MaxCmdSN, etc.,
not the iSER layer.  On the = target=20 side, for cases where the immediate
flag is not set, the iSER = layer must=20 provision sufficient number of
receive buffers to support the = MaxCmdSN=20 advertised by the iSCSI layer at
the target.  For multiple=20 connections per session, how the iSER layer at
each connection = handle the=20 number of receive buffers required per
connection is = implementation=20 dependent.  An SRQ can be used, but not
required. =  Similarly, on=20 the initiator side, the iSER layer can determine
the number of = receive=20 buffers required since most control-type PDUs sent
by the target = are=20 responses to requests sent by the initiator side.
Therefore the = flow=20 control issue being debated here refers to those PDUs
which are = not=20 regulated by the CmdSN flow.

On the various schemes summarized = by=20 Mallikarjun, I agree with the general
sentiment that option #3 = (declared=20 limit) should work, though it might not
be the first choice for = some.=20  However, by setting the declared limit to a
high enough = value (or=20 "None"), the limit essentailly goes away and
implementations that = can=20 support option #1 (iSER/iSCSI layer being able to
replenish the = buffers as=20 fast as the packet arrival rate) or #2 (RNIC
being able to support = graceful handling) would not be impeded by the
declared limit. =  The=20 key is to have the limit being declared instead of a
stated value = in the=20 protocol (option #3a) to allow the implementation the
flexibility, = and to=20 use existing iSCSI pacing mechanisms to advance the
"credits" in = order to=20 avoid introducing yet another credit scheme into the
iSER=20 layer.

The use of the declared limit has been discussed last = year and I=20 have
summarized the main points in my note of October 4 and 5. =  To=20 recap, from
the initiator to the target, control-type PDUs with = the=20 immediate flag set
and SNACK request are not regulated by the = CmdSN=20 mechanism.  (SNACKs are
not used in iSER.)  From the = target to=20 the initiator, Async Messages and
unsolicited NOP-in are sent not = as a=20 response to a request from the
initiator.  These are the = unregulated=20 flows which fall under the declared
limit.  I propose that we = consider the following scheme to advance the
credits set by the = declared=20 limit.  ExpCmdSN andExpStatSN are used as
pacing=20 mechanisms.

For the initiator, the ExpCmdSN from the target = will be=20 used to advance
the credit.  After sending PDUs with the = immediate=20 flag set, when the
initiator sends any iSCSI control-type PDU with = a new=20 CmdSN without the
immediate flag, if the target responds with an = iSCSI=20 control-type PDU with
ExpCmdSN set to CmdSN + 1, then credit at = the=20 initiator is restored to its
full value.

For the target, = the=20 ExpStatSN from the initiator will be used to advance
the credit. =  For=20 unsolicited PDUs sent by the target, if the StatSN of
that PDU is=20 acknowledged by ExpStatSN, then the credit is restored.=20  

Comments?

Mike
Sent by:      =20  ips-bounces@ietf.org
To:     = <ips@ietf.org>
cc:=20      
Subject:        Re: [Ips] = iSER=20 flow control options



Hi, all
Reading through the = last=20 messages in the "flow control" thread was quite
helpful, but I = still have a=20 few questions.

[Caitlin Bestler said> Shared Receive Queues = are=20 relevant here on two
fronts. First, iSCSI flow control is really = more=20 session oriented than
connection oriented. Pre-posting separate = buffers to=20 receive Cmd X on
*each* connection is clearly over-buffering. This = may be=20 valid, because
memory is cheap, but clearly it is not required to=20 pre-commit duplicate
buffers for each connection in a session. = <Caitlin=20 Bestler]

This has alerted me with respect to the suggested = credit=20 update scheme:
[Mike Ko said> One possible solution is as = follows: For=20 the initiator,
the ExpCmdSN from the target will be used to advance = the=20 credit... <Mike
Ko]

Such credit update scheme implies = that iSER=20 DataMover manages the
session context (connection-session bindings, = and the=20 state variables
incl. CmdSN,ExpCmdSN,MaxCmdSN). Attempting not to=20 overcommit buffers
when posting based on MaxCmdSN brings up the = same=20 requirement.
This information is routinely maintained by the iSCSI = layer.=20 There is no
API defined between the iSCSI and DataMover, which can = tell a=20 DataMover
about the session context information. If I understand it = right,=20 iSER as
a DataMover was intended to operate solely on connections=20 without
bothering about sessions. Not only recovering session-scope = information
from the passing PDUs is actually a duplication of the = job=20 already done
by iSCSI. iSER is not supposed to see the Login phase = (where=20 the session
ids and handles are exchanged) at all. Of course,=20 connection-session
binding may be passed "somehow" (e.g. as faked = keys=20 of
Notice_Key_Values), but I guess that this wasn't the original=20 intention.

Did I overlook anything?

>From what has = been said=20 previously on similar issues, I can see two
replies to the above = problem=20 (if it is a problem):
1) Use S-RQ and all such problems are = gone
2) At=20 least in target, DataMover does not have to be "that" = separated
from the=20 iSCSI layer.

Well, I'll try to present our implementation = issues, they=20 may be of
value also as a "yet another use case" for = iSER.

We are=20 building a storage gateway between IB and FC. The = current
incarnation is=20 based on iSER.
We have an operational (buffer management is the = major=20 remaining issue)
iSER DataMover linux kernel module, where iSER is = built on=20 top of IB
rather than iWARP. It is built as a DataMover "service=20 provider",
capable to serve concurrently a number of iSCSI entities = incl.=20 initiator
and/or target-type. Such capability is useful as it = enables us to=20 have
the same code for both initiator and target platforms,=20 streamlines
debugging and can enable easy extention to a = router-like=20 environment
where the same box acts both as initiator and target = through=20 different
interfaces.
We are in process of integration with the=20 open-source linux-iscsi
initiator project, which is to act as an = iSCSI=20 client of the iSER
DataMover.
I got into such level of = detalization to=20 stress that we already did not
opt for intimate integration between = the=20 iSER/iSCSI layers, but rather
tried to keep a reasonable level of=20 separation between them.

Being conscious about the = ever-changing=20 environment of RDMA transports,
we build iSER DataMover on top of = DAPL, as=20 DAT is more general than any
specific implementation of IB, iWARP = etc. So=20 we did know about the S-RQ
design option, it's only that it is = still not=20 fully supported by the
specs issued by the DAT collaborative. = Though it=20 surely will be
supported in the (hopefully near) future, I guess = this is an=20 example of
implementation which can't be based on S-RQ, at least = for a time=20 being.
I'd note also that nothing either in the DataMover = Architecture or=20 iSER
specs has warned us that the design should be based upon S-RQ, = just=20 as
noted by  [Mallikarjun C. said> I really would like us = to view=20 S-RQ as a
"dampener" in invoking #2 (graceful handling), not as a=20 mandatory
feature that iSER requires - because it doesn't (though I = as=20 an
implementer would require S-RQ in my implementation to realize=20 the
dampening).<Mallikarjun C.]

So it is currently a non = S-RQ=20 based implementation. As such, we are
running into a problem, how = to manage=20 the buffer credits (along the
lines of the latest discussion, plus = see my=20 question above).
Another, more subtle one is that the resulting = design=20 seems to differ
substantially from what would be necessary with a = S-RQ=20 implementation.
The fact that using a S-RQ can't be extended easily = from a=20 non S-RQ
implementation strikes me as non-elegant, to say the=20 least.

I'd appreciate your comments and = suggestions.

Alexander=20 = Nezhinsky

_______________________________________________
Ips=20 mailing=20 = list
Ips@ietf.org
https://www1.ietf.org/mailman/listinfo/ips

_______________________________________________
Ips=20 mailing=20 = list
Ips@ietf.org
https://www1.ietf.org/mailman/listinfo/ips

------_=_NextPart_001_01C4B0BB.8E9669D7-- --===============1390949142== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline Content-Transfer-Encoding: 7bit _______________________________________________ Ips mailing list Ips@ietf.org https://www1.ietf.org/mailman/listinfo/ips --===============1390949142==-- From ips-bounces@ietf.org Wed Oct 13 00:34:47 2004 Received: from ietf-mx.ietf.org (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id AAA19253 for ; Wed, 13 Oct 2004 00:34:47 -0400 (EDT) Received: from megatron.ietf.org ([132.151.6.71]) by ietf-mx.ietf.org with esmtp (Exim 4.33) id 1CHb1Y-0003bK-1F for ips-web-archive@ietf.org; Wed, 13 Oct 2004 00:46:04 -0400 Received: from localhost.localdomain ([127.0.0.1] helo=megatron.ietf.org) by megatron.ietf.org with esmtp (Exim 4.32) id 1CHamL-0004mu-6Y; Wed, 13 Oct 2004 00:30:21 -0400 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by megatron.ietf.org with esmtp (Exim 4.32) id 1CHalY-0004Xw-79 for ips@megatron.ietf.org; Wed, 13 Oct 2004 00:29:33 -0400 Received: from ietf-mx.ietf.org (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id AAA18910 for ; Wed, 13 Oct 2004 00:29:27 -0400 (EDT) Received: from e2.ny.us.ibm.com ([32.97.182.102]) by ietf-mx.ietf.org with esmtp (Exim 4.33) id 1CHawN-0003XV-JX for ips@ietf.org; Wed, 13 Oct 2004 00:40:44 -0400 Received: from northrelay04.pok.ibm.com (northrelay04.pok.ibm.com [9.56.224.206]) by e2.ny.us.ibm.com (8.12.10/8.12.9) with ESMTP id i9D4StVR170506; Wed, 13 Oct 2004 00:28:55 -0400 Received: from d01ml604.pok.ibm.com (d01av02.pok.ibm.com [9.56.224.216]) by northrelay04.pok.ibm.com (8.12.10/NCO/VER6.6) with ESMTP id i9D4U9Lk099352; Wed, 13 Oct 2004 00:30:10 -0400 In-Reply-To: <9418406DEC3B8A4BBDB1B87291D851AF0E7552@wcosmb05.cos.agilent.com> Importance: Normal MIME-Version: 1.0 To: Subject: RE: [Ips] iSER flow control options X-Mailer: Lotus Notes Release 6.5.2 June 01, 2004 Message-ID: From: Mike Ko Date: Tue, 12 Oct 2004 21:28:51 -0700 X-MIMETrack: Serialize by Router on D01ML604/01/M/IBM(Release 6.51HF642 | October 12, 2004) at 10/13/2004 00:28:54, Serialize complete at 10/13/2004 00:28:54 Content-Type: text/plain; charset="US-ASCII" X-Spam-Score: 0.2 (/) X-Scan-Signature: 32a65c0bf5eb4ec26489239c7cdd0636 Cc: ips@ietf.org X-BeenThere: ips@ietf.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: IP Storage List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Sender: ips-bounces@ietf.org Errors-To: ips-bounces@ietf.org X-Spam-Score: 0.2 (/) X-Scan-Signature: e654cfa5e44bd623be3eb2c720858b05 Pat, My comments are embedded. Mike To: John Hufferd/San Jose/IBM@IBMUS, cc: , Subject: RE: [Ips] iSER flow control options John, This is an interesting idea, but I have some concerns about it. It isn't clear that the sender has a way to reliably know when an unexpected PDU is no longer outstanding and there may be lock-up conditions possible (both sides have the maximum number of outstanding unexpected PDUs, the initiator has no credit (e.g. maxCmdSN) to send an expected PDU and the target has no expected PDU to send. End of outstanding status: This is easy for some unexpected PDU types - anything from the target that advances StatSN can be considered not-outstanding when ExpSN acknowledges it. I'll use "retired" to mean that a PDU is no longer outstanding. It is less clear how to do it for some other types. A unidirectional NOP-Out or a NOP-Out that responds to a NOP-In has no response from the target so there is no way to tell when it is retired. [mk] A NOP-out from the initiator carries CmdSN. This is paced by the normal MaxCmdSN mechanism in iSCSI. The initiator knows how many control-type PDUs it can send where the Immediate flag is not set. Besides, when the target advances ExpCmdSN, the initiator is notified that the target has acknowledged all commands up to but not including the ExpCmdSN. [mk] A unidirectional NOP-In doesn't get a response and doesn't advance StatSN so there is no means to tell when it is retired. [mk] Since StatSN is not advanced for a unidirectional NOP-in, the target has to wait for the next request-response exchange which advances ExpStatSN in order to ascertain that the NOP-in has been retired. If the target is running out of credit, it can always send the NOP-in as a Ping, and use the NOP-out as an indicator that the NOP-in has been retired. [mk] SNACK Request - it causes a retransmission of something, but when the initiator receives the requested PDUs, I don't think it can always know whether they were sent in response to the SNACK vs. being the delayed original PDU or a copy resent due to a time out at the target. [mk] SNACKs are not needed in iSER. See section 7.3.11 of the iSER draft for the rationale. If they are used anyway, one way to ascertain that the SNACKs have been processed is to wait until the SCSI response is returned for the command where SNACKs are used. [mk] Immediate Commands - they may be rejected by the target because of insufficient resources. There is a reject code for too many immediate commands, but there isn't any way to tie it to the rejected command and reading the text, I'm not sure whether it is sent once for each rejected immediate command or once for multiple immediate commands. There may also be cases where the target doesn't even have the resources to send the reject. Therefore, immediate commands may not get a response that allows them to be retired. [mk] After sending PDUs with the immediate flag set, when the initiator sends any iSCSI control-type PDU with a new CmdSN without the immediate flag, if the target responds with an iSCSI control-type PDU with ExpCmdSN set to CmdSN + 1, then credit at the initiator is restored to its full value. [mk] Reject - I'm not sure if all rejects should be considered unexpected. Possibly the ones that are sent in place of a response (e.g. the reject that is sent in response to a text negotiation or a command that can't be initiated) can be considered expected. Probably it is best to take the simple course and count all of them as unexpected. In any case, it appears that they don't advance StatSN (I think - the standard isn't terribly clear here) so they aren't acknowledged and there is no way to retire them. Reject and StatSN aside - 10.4.7 says "StatSN is incremented by one for every response/status ...." 10.17.3 says that StatSN carries the usual value Since reject is not a response or a status, it appears that it is not incremented for rejects. [mk] Section 10.17.3 of RFC3720 states that "StatSN is advanced after a Reject". So when the initiator advances ExpStatSN, the target will know that Reject has been retired. [mk] But asynchronous messages advance StatSN even though they aren't a response/status so I'm not sure if this was what we intended or if it was an ommission in the definition of reject. Regards, Pat -----Original Message----- From: ips-bounces@ietf.org [mailto:ips-bounces@ietf.org]On Behalf Of John Hufferd Sent: Tuesday, 12 October, 2004 11:57 AM To: Mike Ko Cc: ips@ietf.org; Alex Nezhinsky Subject: Re: [Ips] iSER flow control options I agree with both Mallikarjun, and Mike. That is, my company will probably require vendors that OEM their RNIC to us, to provide "Graceful handling", and I expect that we and others will also exploit the value retained with non restricted iSCSI flow control and therefore will require the use of SRQs. However, as I said in my Oct 6, 2004 message I believe that there can be an additional declared parameter (the Keyword=Value pair "MaxOutstandingUnexpectedPDUs") which can take on values of 1-?????, including "None" (which means non restricted -- to be used in unrestricted iSCSI flow control). This Keyword=Value pair gives non shared RQ implementations, and InfiniBand implementations, which do not yet have SRQs in their HW, a reasonable way to be compatible with each other, yet not inhibit the creative implementation that can use the unrestricted iSCSI flow control. (Note: SRQ is now in the IB specifications.) . . John L. Hufferd Senior Technical Staff Member (STSM) IBM/System Group, San Jose CA Main Office: (408) 256-0403, Tie: 276-0403, eFax: (408) 904-4688 Alt Office: (408) 997-6136, Cell: (408) 499-9702 Internet Address: hufferd@us.ibm.com Mike Ko Sent by: ips-bounces@ietf.org 10/12/2004 10:50 AM To "Alex Nezhinsky" cc ips@ietf.org Subject Re: [Ips] iSER flow control options Alex, In response to your statement that "such credit update scheme implies that iSER DataMover manages the session context (connection-session bindings, and the state variables incl. CmdSN, ExpCmdSN, MaxCmdSN), bear in mind that it is the iSCSI layer that manages CmdSN, ExpCmdSN, MaxCmdSN, etc., not the iSER layer. On the target side, for cases where the immediate flag is not set, the iSER layer must provision sufficient number of receive buffers to support the MaxCmdSN advertised by the iSCSI layer at the target. For multiple connections per session, how the iSER layer at each connection handle the number of receive buffers required per connection is implementation dependent. An SRQ can be used, but not required. Similarly, on the initiator side, the iSER layer can determine the number of receive buffers required since most control-type PDUs sent by the target are responses to requests sent by the initiator side. Therefore the flow control issue being debated here refers to those PDUs which are not regulated by the CmdSN flow. On the various schemes summarized by Mallikarjun, I agree with the general sentiment that option #3 (declared limit) should work, though it might not be the first choice for some. However, by setting the declared limit to a high enough value (or "None"), the limit essentailly goes away and implementations that can support option #1 (iSER/iSCSI layer being able to replenish the buffers as fast as the packet arrival rate) or #2 (RNIC being able to support graceful handling) would not be impeded by the declared limit. The key is to have the limit being declared instead of a stated value in the protocol (option #3a) to allow the implementation the flexibility, and to use existing iSCSI pacing mechanisms to advance the "credits" in order to avoid introducing yet another credit scheme into the iSER layer. The use of the declared limit has been discussed last year and I have summarized the main points in my note of October 4 and 5. To recap, from the initiator to the target, control-type PDUs with the immediate flag set and SNACK request are not regulated by the CmdSN mechanism. (SNACKs are not used in iSER.) From the target to the initiator, Async Messages and unsolicited NOP-in are sent not as a response to a request from the initiator. These are the unregulated flows which fall under the declared limit. I propose that we consider the following scheme to advance the credits set by the declared limit. ExpCmdSN andExpStatSN are used as pacing mechanisms. For the initiator, the ExpCmdSN from the target will be used to advance the credit. After sending PDUs with the immediate flag set, when the initiator sends any iSCSI control-type PDU with a new CmdSN without the immediate flag, if the target responds with an iSCSI control-type PDU with ExpCmdSN set to CmdSN + 1, then credit at the initiator is restored to its full value. For the target, the ExpStatSN from the initiator will be used to advance the credit. For unsolicited PDUs sent by the target, if the StatSN of that PDU is acknowledged by ExpStatSN, then the credit is restored. Comments? Mike Sent by: ips-bounces@ietf.org To: cc: Subject: Re: [Ips] iSER flow control options Hi, all Reading through the last messages in the "flow control" thread was quite helpful, but I still have a few questions. [Caitlin Bestler said> Shared Receive Queues are relevant here on two fronts. First, iSCSI flow control is really more session oriented than connection oriented. Pre-posting separate buffers to receive Cmd X on *each* connection is clearly over-buffering. This may be valid, because memory is cheap, but clearly it is not required to pre-commit duplicate buffers for each connection in a session. One possible solution is as follows: For the initiator, the ExpCmdSN from the target will be used to advance the credit... From what has been said previously on similar issues, I can see two replies to the above problem (if it is a problem): 1) Use S-RQ and all such problems are gone 2) At least in target, DataMover does not have to be "that" separated from the iSCSI layer. Well, I'll try to present our implementation issues, they may be of value also as a "yet another use case" for iSER. We are building a storage gateway between IB and FC. The current incarnation is based on iSER. We have an operational (buffer management is the major remaining issue) iSER DataMover linux kernel module, where iSER is built on top of IB rather than iWARP. It is built as a DataMover "service provider", capable to serve concurrently a number of iSCSI entities incl. initiator and/or target-type. Such capability is useful as it enables us to have the same code for both initiator and target platforms, streamlines debugging and can enable easy extention to a router-like environment where the same box acts both as initiator and target through different interfaces. We are in process of integration with the open-source linux-iscsi initiator project, which is to act as an iSCSI client of the iSER DataMover. I got into such level of detalization to stress that we already did not opt for intimate integration between the iSER/iSCSI layers, but rather tried to keep a reasonable level of separation between them. Being conscious about the ever-changing environment of RDMA transports, we build iSER DataMover on top of DAPL, as DAT is more general than any specific implementation of IB, iWARP etc. So we did know about the S-RQ design option, it's only that it is still not fully supported by the specs issued by the DAT collaborative. Though it surely will be supported in the (hopefully near) future, I guess this is an example of implementation which can't be based on S-RQ, at least for a time being. I'd note also that nothing either in the DataMover Architecture or iSER specs has warned us that the design should be based upon S-RQ, just as noted by [Mallikarjun C. said> I really would like us to view S-RQ as a "dampener" in invoking #2 (graceful handling), not as a mandatory feature that iSER requires - because it doesn't (though I as an implementer would require S-RQ in my implementation to realize the dampening).; Wed, 13 Oct 2004 08:58:07 -0400 (EDT) Received: from megatron.ietf.org ([132.151.6.71]) by ietf-mx.ietf.org with esmtp (Exim 4.33) id 1CHisf-0004VX-Vn for ips-web-archive@ietf.org; Wed, 13 Oct 2004 09:09:26 -0400 Received: from localhost.localdomain ([127.0.0.1] helo=megatron.ietf.org) by megatron.ietf.org with esmtp (Exim 4.32) id 1CHifR-0007PT-Jr; Wed, 13 Oct 2004 08:55:45 -0400 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by megatron.ietf.org with esmtp (Exim 4.32) id 1CHiZQ-0005qZ-5L for ips@megatron.ietf.org; Wed, 13 Oct 2004 08:49:32 -0400 Received: from ietf-mx.ietf.org (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id IAA23089 for ; Wed, 13 Oct 2004 08:49:27 -0400 (EDT) Received: from sccrmhc13.comcast.net ([204.127.202.64]) by ietf-mx.ietf.org with esmtp (Exim 4.33) id 1CHikH-0004Ju-OG for ips@ietf.org; Wed, 13 Oct 2004 09:00:46 -0400 Received: from ivvt2dxrc11 (c-66-177-57-55.se.client2.attbi.com[66.177.57.55]) by comcast.net (sccrmhc13) with SMTP id <2004101312485401600c7amae> (Authid: esquicksall); Wed, 13 Oct 2004 12:48:54 +0000 Message-ID: <000301c4b122$ff67be90$0303a8c0@ivvt2dxrc11> From: "Eddy Quicksall" To: "Mallikarjun C." , References: <41649B60.60002@rose.hp.com> <4165785D.5050802@rose.hp.com><4166D357.40708@rose.hp.com><4489F857-1956-11D9-91D4-000A95D14BEC@wasabisystems.com><4166E5B0.1090506@rose.hp.com><30FE007A-196F-11D9-91D4-000A95D14BEC@wasabisystems.com><416B0413.8080208@rose.hp.com><3DB9AEB1-1BEA-11D9-8B4B-000A95D14BEC@wasabisystems.com> <416C6214.9080408@rose.hp.com> Subject: Re: [Ips] resending Data-in with S bit set Date: Wed, 13 Oct 2004 08:48:53 -0400 MIME-Version: 1.0 Content-Type: text/plain; format=flowed; charset="iso-8859-1"; reply-type=response Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 6.00.2900.2180 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2900.2180 X-Spam-Score: 0.1 (/) X-Scan-Signature: 29dc808194f5fb921c09d0040806d6eb Content-Transfer-Encoding: 7bit X-BeenThere: ips@ietf.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: IP Storage List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Sender: ips-bounces@ietf.org Errors-To: ips-bounces@ietf.org X-Spam-Score: 0.1 (/) X-Scan-Signature: 3971661e40967acfc35f708dd5f33760 Content-Transfer-Encoding: 7bit Is there any way the initiator can loose status other than from a CRC error? In which case if markers are not used, there is no way to recover other than reinstating the connection. Is that correct? Eddy ----- Original Message ----- From: "Mallikarjun C." To: Sent: Tuesday, October 12, 2004 7:00 PM Subject: Re: [Ips] resending Data-in with S bit set > Bill, > > >there should be a > > disconnect between ExpDataSN on the target (sent in the SCSI Response) > > and in the initiator. > > Yes, that should work for the scenario you described (and in other cases > that I can now think of). > > In general, during the time we designed the iSCSI resegmentation approach, > the operating assumption as I recall was to not rely on absolute sequence > numbers from the previous epoch (i.e. before the PDU size change) after > the change. The R-Data SNACK simply causes a reversion to the last ack > point which is consistent with not using absolute SNs from the past. > > The thinking then was that between possibly multiple PDU size changes > during the life of a task and and sharing of the DataSN/R2TSN space for > bidirectional commands, there's enough room for implementations to get the > absolute sequence numbers wrong in their SNACK requests. Besides, we (at > least I) didn't feel like we have exhaustively enumerated all potential > scenarios. As far as I recall, we have not defined a quiescing process to > change the PDU size either. > > I'd of course like to hear from other participants of those discussions > wrt this recollection. > > At least this is my rationale for suggesting a big hammer approach that > doesn't rely on the ExpDataSN field in the status PDU (BTW, I don't recall > a discussion on ExpDataSN in the past). But I do realize that in my > approach, under this scenario, the ExpDataSN field in the Response PDU is > always suspect - so a SNACK is always required or the I/O would fail. > > Thanks. > -- > Mallikarjun > > Mallikarjun Chadalapaka > Networked Storage Architecture > Network Storage Solutions > Hewlett-Packard MS 5668 > Roseville CA 95747 > cbm [at] rose.hp.com > > > > > > > > William Studenmund wrote: > >> On Oct 11, 2004, at 6:07 PM, Mallikarjun C. wrote: >> >>> I think I may be finally understanding your scenario, but let's confirm >>> that.... >> >> >> Ok! >> >>> A read SCSI task is in progress and the initiator changes >>> MaxRecvDataSegementLength. >> >> >> Specifically, for what I'm thinking of, all of the data have already been >> sent before the MaxRecvDataSegmentLength >> >>> The task continues. Subsequently, the initiator notices it lost a >>> status. Here's what I think should happen (I think the same as what you >>> described, except for the ExpDataSN part). >>> >>> - As always, initiator starts with a status SNACK. >>> - Regardless of the original transmission (phase collapse or not, old >>> PDU still fits with the new size or not, single/multiple data PDUs), >>> target retransmits the status in an explicit Response PDU with ExpDataSN >>> set to something like 0xffffffff. >>> - Initiator checks ExpDataSN with its records. They shouldn't match, >>> leading to an R-Data SNACK. >>> - Rest of the semantics described in 10.16.3 apply. >>> >>> My reasons are two: >>> >>> - Target always does the same thing in this scenario (PDU size change >>> followed by a status SNACK first) - i.e. doesn't phase collapse even if >>> it still can. >> >> >> How about this idea. If the target can still phase collapse (the PDU >> would still fit), it MAY, but due to the fact that there was a >> resegmentation, the target MAY instead treat the PDU as if it could not >> fit. That is to say it is an implementation decision if the target just >> remembers "resegmented" or if it looks at the PDU. >> >>> - Initiator operates based on ExpDataSN as always, consistent with the >>> spec intent to not have to do scoreboarding/counting. >> >> >> How does what I suggest break ExpDataSN? >> >> I was about to say I like what you suggested, then I pondered a bit. I >> don't think we need the magic ExpDataSN value. If the PDU is too big to >> send, then the target can just generate a SCSI Response PDU with the >> correct ExpDataSN and a SNACK Tag of 0. Given that the initiator didn't >> notice the status value in the phase-collapsed read PDU, it should also >> have not noticed the data reception in it, so there should be a >> disconnect between ExpDataSN on the target (sent in the SCSI Response) >> and in the initiator. So a SCSI Response sent in place of the collapsed >> Data-In should trigger the initiator to send an R-Data SNACK. Shouldn't >> it? >> >>> OTOH, the RFC doesn't set aside 0xffffffff as a reserved value for >>> ExpDataSN. So I guess this is a hack for now. Alternatively (or if >>> target had already sent 2**32-1 PDUs for that task), what you described >>> works as well (if the initiator is counting bytes). >> >> >> I don't see how you have to count bytes. I'm not trying to sound >> belligerent, and I'm sorry if it comes across that way. But I think just >> sending the SCSI Response would work. I'm not seeing how the # of bytes'd >> be needed. >> >> The one thing that would be impacted would be a broken initiator that got >> the data out of this phase-collapsed-read Data-In PDU but not the status. >> I think it deserves what it gets... >> >> Take care, >> >> Bill > > > > > _______________________________________________ > Ips mailing list > Ips@ietf.org > https://www1.ietf.org/mailman/listinfo/ips _______________________________________________ Ips mailing list Ips@ietf.org https://www1.ietf.org/mailman/listinfo/ips From ips-bounces@ietf.org Wed Oct 13 10:11:27 2004 Received: from ietf-mx.ietf.org (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id KAA29472 for ; Wed, 13 Oct 2004 10:11:27 -0400 (EDT) Received: from megatron.ietf.org ([132.151.6.71]) by ietf-mx.ietf.org with esmtp (Exim 4.33) id 1CHk1e-0005sY-1x for ips-web-archive@ietf.org; Wed, 13 Oct 2004 10:22:47 -0400 Received: from localhost.localdomain ([127.0.0.1] helo=megatron.ietf.org) by megatron.ietf.org with esmtp (Exim 4.32) id 1CHjja-0004DN-A5; Wed, 13 Oct 2004 10:04:06 -0400 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by megatron.ietf.org with esmtp (Exim 4.32) id 1CHjgS-0001aJ-D7 for ips@megatron.ietf.org; Wed, 13 Oct 2004 10:00:52 -0400 Received: from ietf-mx.ietf.org (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id KAA28401 for ; Wed, 13 Oct 2004 10:00:46 -0400 (EDT) Received: from mononoke.wasabisystems.com ([66.173.145.228]) by ietf-mx.ietf.org with esmtp (Exim 4.33) id 1CHjrI-0005hi-Hu for ips@ietf.org; Wed, 13 Oct 2004 10:12:05 -0400 Received: from [192.168.1.125] (unknown [66.173.145.226]) by mononoke.wasabisystems.com (Postfix) with ESMTP id C666587095; Wed, 13 Oct 2004 10:00:40 -0400 (EDT) In-Reply-To: <416C6214.9080408@rose.hp.com> References: <41649B60.60002@rose.hp.com> <4165785D.5050802@rose.hp.com> <4166D357.40708@rose.hp.com> <4489F857-1956-11D9-91D4-000A95D14BEC@wasabisystems.com> <4166E5B0.1090506@rose.hp.com> <30FE007A-196F-11D9-91D4-000A95D14BEC@wasabisystems.com> <416B0413.8080208@rose.hp.com> <3DB9AEB1-1BEA-11D9-8B4B-000A95D14BEC@wasabisystems.com> <416C6214.9080408@rose.hp.com> Mime-Version: 1.0 (Apple Message framework v619) Message-Id: <3FE9186D-1D20-11D9-859D-000A95D14BEC@wasabisystems.com> From: William Studenmund Subject: Re: [Ips] resending Data-in with S bit set Date: Wed, 13 Oct 2004 10:00:25 -0400 To: "Mallikarjun C." X-Pgp-Agent: GPGMail 1.0 (v30, 10.3) X-Mailer: Apple Mail (2.619) X-Spam-Score: 0.0 (/) X-Scan-Signature: 87a3f533bb300b99e2a18357f3c1563d Cc: ips@ietf.org X-BeenThere: ips@ietf.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: IP Storage List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Content-Type: multipart/mixed; boundary="===============0455571497==" Sender: ips-bounces@ietf.org Errors-To: ips-bounces@ietf.org X-Spam-Score: 0.0 (/) X-Scan-Signature: e472ca43d56132790a46d9eefd95f0a5 --===============0455571497== Content-Type: multipart/signed; protocol="application/pgp-signature"; micalg=pgp-sha1; boundary="Apple-Mail-2--890250972" --Apple-Mail-2--890250972 Content-Type: multipart/signed; micalg=sha1; boundary=Apple-Mail-1--890250977; protocol="application/pkcs7-signature" --Apple-Mail-1--890250977 Content-Type: text/plain; charset=US-ASCII; format=flowed Content-Transfer-Encoding: 7bit On Oct 12, 2004, at 7:00 PM, Mallikarjun C. wrote: > Bill, > > >there should be a > > disconnect between ExpDataSN on the target (sent in the SCSI > Response) > > and in the initiator. > > Yes, that should work for the scenario you described (and in other > cases that I can now think of). > > In general, during the time we designed the iSCSI resegmentation > approach, the operating assumption as I recall was to not rely on > absolute sequence numbers from the previous epoch (i.e. before the PDU > size change) after the change. The R-Data SNACK simply causes a > reversion to the last ack point which is consistent with not using > absolute SNs from the past. > > The thinking then was that between possibly multiple PDU size changes > during the life of a task and and sharing of the DataSN/R2TSN space > for bidirectional commands, there's enough room for implementations to > get the absolute sequence numbers wrong in their SNACK requests. > Besides, we (at least I) didn't feel like we have exhaustively > enumerated all potential scenarios. As far as I recall, we have not > defined a quiescing process to change the PDU size either. I definitely agree that once there's been a resegmentation during the data sending, that something like R-Data SNACK is what we need. And as a target implementer, I think R-Data SNACK is the right thing to do; trying to figure out what was where & all just seems like a mess. Plus, it could be you can't resend any of the old PDUs.... > I'd of course like to hear from other participants of those > discussions wrt this recollection. > > At least this is my rationale for suggesting a big hammer approach > that doesn't rely on the ExpDataSN field in the status PDU (BTW, I > don't recall a discussion on ExpDataSN in the past). But I do realize > that in my approach, under this scenario, the ExpDataSN field in the > Response PDU is always suspect - so a SNACK is always required or the > I/O would fail. Here's another way to look at my suggestion. It is what you're suggesting, except that the magic DataSN number's different. Rather than 0xffffffff, I chose the DataSN we put in the unacknowledged Data-IN. My idea was that since the initiator didn't get the StatSN we're talking about, it didn't get the Data-IN PDU. So the _one_ DataSN number we're certain the initiator didn't get was the one in that packet. :-) And since the only thing the initiator will be able to do once it gets the SCSI Response with the DataSN it won't agree with is to send an R-Data SNACK, we will then have a SNACK tag and the sequence number spaces are safe. Although to be honest, we could go with the 0xffffffff magic number you mentioned. We'd need to make some sort of supplemental RFC describing it, but we could. The reason I think it'd be fine is that MaxRecvDataSegmentLength can't go below 512. So we would need to go through around 512 error recovery actions for each original PDU in the transfer and have had a maximally-sized transfer to hit 0xffffffff for DataSN/R2T. Take care, Bill --Apple-Mail-1--890250977 Content-Type: application/pkcs7-signature; name=smime.p7s Content-Disposition: attachment; filename=smime.p7s Content-Transfer-Encoding: base64 MIAGCSqGSIb3DQEHAqCAMIACAQExCzAJBgUrDgMCGgUAMIAGCSqGSIb3DQEHAQAAoIIFfDCCBXgw ggNgoAMCAQICAUYwDQYJKoZIhvcNAQEEBQAwga0xCzAJBgNVBAYTAlVTMREwDwYDVQQIEwhOZXcg WW9yazERMA8GA1UEBxMITmV3IFlvcmsxHTAbBgNVBAoTFFdhc2FiaSBTeXN0ZW1zLCBJbmMuMQww CgYDVQQLEwNHSFExHzAdBgNVBAMTFldhc2FiaSBTeXN0ZW1zIFJvb3QgQ0ExKjAoBgkqhkiG9w0B CQEWG3dlYm1hc3RlckB3YXNhYmlzeXN0ZW1zLmNvbTAeFw0wMzA4MTExODU2NDZaFw0wODA2MjUx ODU2NDZaMIGXMQswCQYDVQQGEwJVUzETMBEGA1UECBMKQ2FsaWZvcm5pYTESMBAGA1UEBxMJU2Fu IERpZWdvMRcwFQYDVQQKEw5XYXNhYmkgU3lzdGVtczEbMBkGA1UEAxMSV2lsbGlhbSBTdHVkZW5t dW5kMSkwJwYJKoZIhvcNAQkBFhp3cnN0dWRlbkB3YXNhYmlzeXN0ZW1zLmNvbTCBnzANBgkqhkiG 9w0BAQEFAAOBjQAwgYkCgYEA9CZhg4GVWYTRGcJz8lU1ipzInlBuGqfZQs88Z1rTP7+c3AyhKwpE TyTmK4UhxslzKa93YLqifLxZKYt1jI6FNq40IQWCKBVywWBwmL5guCu0851f9ywH5Uj2wJdiDrOI sOJKNpwNqml8bFdTrFS2qvyCxTIWVMLYAzUBBcl2F7MCAwEAAaOCATkwggE1MAkGA1UdEwQCMAAw LAYJYIZIAYb4QgENBB8WHU9wZW5TU0wgR2VuZXJhdGVkIENlcnRpZmljYXRlMB0GA1UdDgQWBBTt mjHbSBlMV48RWOBETM3EV0Tm/jCB2gYDVR0jBIHSMIHPgBQrBOnN88CNCHPaV5wLS212tcrgBaGB s6SBsDCBrTELMAkGA1UEBhMCVVMxETAPBgNVBAgTCE5ldyBZb3JrMREwDwYDVQQHEwhOZXcgWW9y azEdMBsGA1UEChMUV2FzYWJpIFN5c3RlbXMsIEluYy4xDDAKBgNVBAsTA0dIUTEfMB0GA1UEAxMW V2FzYWJpIFN5c3RlbXMgUm9vdCBDQTEqMCgGCSqGSIb3DQEJARYbd2VibWFzdGVyQHdhc2FiaXN5 c3RlbXMuY29tggEAMA0GCSqGSIb3DQEBBAUAA4ICAQCpxoPEYunR7qiXyycEZd0UIatSt/WalMoH nuN9S421iS/sD1wSDqzHyM5YyUWn4z4o6VhT2kD9UV4lk5aXxxefEfGzFcR7Xyah7tO3ziuReFfx qUP5+DTlwDuUBc8Iow6NwY9GlRCKFSsEroLWLGODyYWgN1ojjKOtcXRiOZkYTP3VfDUV2/Xrhh9O l6+3Otf069+KWeOWedWeMsbe1rac96yPEjPogToWkBXVJEeEQktd0BLETt7GUuSZg2Fho/nslqZv nnnKZmrOfdFy9iXDL9T0Hp/OHf5B9RP+r00BJJFO8JzBPIL2IA3CBPApOQty5A00Jg1CGfDQxiV3 s8G0aRUbmFqg9r0OfhPYjJ8BHfw1HHWb7Cu91Sw+d5UnYracZhsR2Gc5H5CnfS6IZhc7ulfnSX8f I4BXR4vbeRUG76J3rkkbkO1haYHLk5w5RlICHsJOBSElG9ggLBWNIfUZwAzo84lfmDmJMP1YCOmi DFCbW1DgxhqVFpYEf5Pq2pVcCguopK2G9oZjH5fxq6ehisk9dN+7NEQ4OSqC43ehaStYOoQAJruN e6e5tCtRMx7bMI7eVPiHxE9/Y38Q+sc5oRN9mCyyAXMweuMRNHAHAWdFB7NhHD6gBa2lh91zIMdf iSVRlBXoMrBKJX6VecgnhvUp4G6QAnN7EMu6QHlc/jGCA0swggNHAgEBMIGzMIGtMQswCQYDVQQG EwJVUzERMA8GA1UECBMITmV3IFlvcmsxETAPBgNVBAcTCE5ldyBZb3JrMR0wGwYDVQQKExRXYXNh YmkgU3lzdGVtcywgSW5jLjEMMAoGA1UECxMDR0hRMR8wHQYDVQQDExZXYXNhYmkgU3lzdGVtcyBS b290IENBMSowKAYJKoZIhvcNAQkBFht3ZWJtYXN0ZXJAd2FzYWJpc3lzdGVtcy5jb20CAUYwCQYF Kw4DAhoFAKCCAe0wGAYJKoZIhvcNAQkDMQsGCSqGSIb3DQEHATAcBgkqhkiG9w0BCQUxDxcNMDQx MDEzMTQwMDI2WjAjBgkqhkiG9w0BCQQxFgQUWNR4R0csMO9NOCpHdHE6qoo029AwgcQGCSsGAQQB gjcQBDGBtjCBszCBrTELMAkGA1UEBhMCVVMxETAPBgNVBAgTCE5ldyBZb3JrMREwDwYDVQQHEwhO ZXcgWW9yazEdMBsGA1UEChMUV2FzYWJpIFN5c3RlbXMsIEluYy4xDDAKBgNVBAsTA0dIUTEfMB0G A1UEAxMWV2FzYWJpIFN5c3RlbXMgUm9vdCBDQTEqMCgGCSqGSIb3DQEJARYbd2VibWFzdGVyQHdh c2FiaXN5c3RlbXMuY29tAgFGMIHGBgsqhkiG9w0BCRACCzGBtqCBszCBrTELMAkGA1UEBhMCVVMx ETAPBgNVBAgTCE5ldyBZb3JrMREwDwYDVQQHEwhOZXcgWW9yazEdMBsGA1UEChMUV2FzYWJpIFN5 c3RlbXMsIEluYy4xDDAKBgNVBAsTA0dIUTEfMB0GA1UEAxMWV2FzYWJpIFN5c3RlbXMgUm9vdCBD QTEqMCgGCSqGSIb3DQEJARYbd2VibWFzdGVyQHdhc2FiaXN5c3RlbXMuY29tAgFGMA0GCSqGSIb3 DQEBAQUABIGAG0UVZUQs2oh3W0A7NMb/HOO+ZTuFdHnND08W9O3BIXuaQx3UD+0n1kcFsdNZDLoZ 7o3DXH/VGlGT5eyBkDNqN7ERpNk0DZ1WaanjjIG83bVl9pP+RFJr7iNlU6THazXYMj3+sD0Fa4Dq F0WaEDB2O3a5vLlUs3y6vkgQlN71BPIAAAAAAAA= --Apple-Mail-1--890250977-- --Apple-Mail-2--890250972 content-type: application/pgp-signature; x-mac-type=70674453; name=PGP.sig content-description: This is a digitally signed message part content-disposition: inline; filename=PGP.sig content-transfer-encoding: 7bit Content-Transfer-Encoding: 7bit Content-Transfer-Encoding: 7bit -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.2.3 (Darwin) iD8DBQFBbTUHDJT2Egh26K0RAqQgAJ9fR79kxKSCRpqtGfywu2jWA9/7IgCfXUIs neHkvxz/++Fz0USyP2If7lI= =SlSW -----END PGP SIGNATURE----- --Apple-Mail-2--890250972-- --===============0455571497== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline Content-Transfer-Encoding: 7bit _______________________________________________ Ips mailing list Ips@ietf.org https://www1.ietf.org/mailman/listinfo/ips --===============0455571497==-- From ips-bounces@ietf.org Thu Oct 14 02:13:11 2004 Received: from ietf-mx.ietf.org (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id CAA07449 for ; Thu, 14 Oct 2004 02:13:11 -0400 (EDT) Received: from megatron.ietf.org ([132.151.6.71]) by ietf-mx.ietf.org with esmtp (Exim 4.33) id 1CHz2X-0001xA-8b for ips-web-archive@ietf.org; Thu, 14 Oct 2004 02:24:41 -0400 Received: from localhost.localdomain ([127.0.0.1] helo=megatron.ietf.org) by megatron.ietf.org with esmtp (Exim 4.32) id 1CHypS-0001Lt-S8; Thu, 14 Oct 2004 02:11:10 -0400 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by megatron.ietf.org with esmtp (Exim 4.32) id 1CHygk-0004KR-Ft for ips@megatron.ietf.org; Thu, 14 Oct 2004 02:02:10 -0400 Received: from ietf-mx.ietf.org (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id CAA26313 for ; Thu, 14 Oct 2004 02:02:09 -0400 (EDT) Received: from web50406.mail.yahoo.com ([206.190.38.71]) by ietf-mx.ietf.org with smtp (Exim 4.33) id 1CHyrp-0001o0-Os for ips@ietf.org; Thu, 14 Oct 2004 02:13:39 -0400 Message-ID: <20041014060135.46915.qmail@web50406.mail.yahoo.com> Received: from [61.11.17.228] by web50406.mail.yahoo.com via HTTP; Wed, 13 Oct 2004 23:01:35 PDT Date: Wed, 13 Oct 2004 23:01:35 -0700 (PDT) From: ulka vaze To: ips@ietf.org MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-Spam-Score: 0.0 (/) X-Scan-Signature: 0bc60ec82efc80c84b8d02f4b0e4de22 Subject: [Ips] query about report luns command with microsoft iscsi initiator X-BeenThere: ips@ietf.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: IP Storage List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Sender: ips-bounces@ietf.org Errors-To: ips-bounces@ietf.org X-Spam-Score: 0.0 (/) X-Scan-Signature: b19722fc8d3865b147c75ae2495625f2 Hi all, I am using Microsoft initiator ver 1.02 for my target server. which is on MAC .After Login completion it send Report LUNs command.In this PDU it sends 16 byte as expected data transfer length. i am getting CDB as below 0xa0 00 00 00 00 00 00 00 00 10 00 00 00 00 00 00 I am not passing this cdb to scsi layer as I would be generating the response at my iscsi layer only. So my question is- * Do i have to send report LUNs response as per the format specified in SPC-2 in the data segment. * From above CDB allocation length is coming 16 bytes so i'll give lun list and first lun address.is it correct? *in iscsi response pdu if check condition is not there then will data segment contains only response data from scsi? anticipating reply. thanks ulka _______________________________ Do you Yahoo!? Declare Yourself - Register online to vote today! http://vote.yahoo.com _______________________________________________ Ips mailing list Ips@ietf.org https://www1.ietf.org/mailman/listinfo/ips From ips-bounces@ietf.org Thu Oct 14 13:31:36 2004 Received: from ietf-mx.ietf.org (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id NAA09867 for ; Thu, 14 Oct 2004 13:31:36 -0400 (EDT) Received: from megatron.ietf.org ([132.151.6.71]) by ietf-mx.ietf.org with esmtp (Exim 4.33) id 1CI9dC-0007TT-24 for ips-web-archive@ietf.org; Thu, 14 Oct 2004 13:43:14 -0400 Received: from localhost.localdomain ([127.0.0.1] helo=megatron.ietf.org) by megatron.ietf.org with esmtp (Exim 4.32) id 1CI9Jt-0004Ek-78; Thu, 14 Oct 2004 13:23:17 -0400 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by megatron.ietf.org with esmtp (Exim 4.32) id 1CI9Ew-0003Hz-Ii for ips@megatron.ietf.org; Thu, 14 Oct 2004 13:18:10 -0400 Received: from ietf-mx.ietf.org (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id NAA08959 for ; Thu, 14 Oct 2004 13:18:06 -0400 (EDT) Received: from mail1.microsoft.com ([131.107.3.125]) by ietf-mx.ietf.org with esmtp (Exim 4.33) id 1CI9Q8-0007B3-F4 for ips@ietf.org; Thu, 14 Oct 2004 13:29:44 -0400 Received: from mailout1.microsoft.com ([157.54.1.117]) by mail1.microsoft.com with Microsoft SMTPSVC(6.0.3790.196); Thu, 14 Oct 2004 10:17:02 -0700 Received: from red-hub-01.redmond.corp.microsoft.com ([157.54.7.71]) by mailout1.microsoft.com with Microsoft SMTPSVC(6.0.3790.0); Thu, 14 Oct 2004 10:17:02 -0700 Received: from win-imc-02.wingroup.windeploy.ntdev.microsoft.com ([157.54.0.84]) by red-hub-01.redmond.corp.microsoft.com with Microsoft SMTPSVC(6.0.3790.211); Thu, 14 Oct 2004 10:17:40 -0700 Received: from WIN-MSG-10.wingroup.windeploy.ntdev.microsoft.com ([157.54.12.81]) by win-imc-02.wingroup.windeploy.ntdev.microsoft.com with Microsoft SMTPSVC(6.0.3790.1069); Thu, 14 Oct 2004 10:17:36 -0700 X-MimeOLE: Produced By Microsoft Exchange V6.5.7226.0 Content-class: urn:content-classes:message MIME-Version: 1.0 Content-Type: text/plain; charset="US-ASCII" Content-Transfer-Encoding: quoted-printable Subject: RE: [Ips] query about report luns command with microsoft iscsi initiator Date: Thu, 14 Oct 2004 10:17:36 -0700 Message-ID: Thread-Topic: [Ips] query about report luns command with microsoft iscsi initiator thread-index: AcSxtTnAocI6ShJPQAimBqcTN2HkAQAW3VxA From: "Lakshmi Ramasubramanian" To: "ulka vaze" , X-OriginalArrivalTime: 14 Oct 2004 17:17:36.0795 (UTC) FILETIME=[B37146B0:01C4B211] X-Spam-Score: 0.0 (/) X-Scan-Signature: 3e15cc4fdc61d7bce84032741d11c8e5 Content-Transfer-Encoding: quoted-printable X-BeenThere: ips@ietf.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: IP Storage List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Sender: ips-bounces@ietf.org Errors-To: ips-bounces@ietf.org X-Spam-Score: 0.0 (/) X-Scan-Signature: f607d15ccc2bc4eaf3ade8ffa8af02a0 Content-Transfer-Encoding: quoted-printable Ulka, You are right - MS Initiator will send SCSI Report LUNs command with buffer size 16 bytes. Target should return data in the format given in SPC-2. Target should give the first lun address. If you have more than one LUN to report, you should set the actual buffer size needed in the first 4 bytes of the response (Format as given in SPC-2). Initiator will retry ReportLUNs with a bigger buffer if more than one LUN is available. Hope that answers your questions. -lakshmi -----Original Message----- From: ips-bounces@ietf.org [mailto:ips-bounces@ietf.org] On Behalf Of ulka vaze Sent: Wednesday, October 13, 2004 11:02 PM To: ips@ietf.org Subject: [Ips] query about report luns command with microsoft iscsi initiator Hi all, I am using Microsoft initiator ver 1.02 for my target server. which is on MAC .After Login completion it send Report LUNs command.In this PDU it sends 16 byte as expected data transfer length. i am getting CDB as below 0xa0 00 00 00 00 00 00 00 00 10 00 00 00 00 00 00 I am not passing this cdb to scsi layer as I would be generating the response at my iscsi layer only. So my question is- * Do i have to send report LUNs response as per the format specified in SPC-2 in the data segment. * From above CDB allocation length is coming 16 bytes=20 so i'll give lun list and first lun address.is it correct? *in iscsi response pdu if check condition is not there then will data segment contains only response data from scsi? anticipating reply. thanks ulka =09 _______________________________ Do you Yahoo!? Declare Yourself - Register online to vote today! http://vote.yahoo.com _______________________________________________ Ips mailing list Ips@ietf.org https://www1.ietf.org/mailman/listinfo/ips _______________________________________________ Ips mailing list Ips@ietf.org https://www1.ietf.org/mailman/listinfo/ips From ips-bounces@ietf.org Thu Oct 14 14:22:52 2004 Received: from ietf-mx.ietf.org (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id OAA13182 for ; Thu, 14 Oct 2004 14:22:51 -0400 (EDT) Received: from megatron.ietf.org ([132.151.6.71]) by ietf-mx.ietf.org with esmtp (Exim 4.33) id 1CIAQm-0008R7-Gr for ips-web-archive@ietf.org; Thu, 14 Oct 2004 14:34:28 -0400 Received: from localhost.localdomain ([127.0.0.1] helo=megatron.ietf.org) by megatron.ietf.org with esmtp (Exim 4.32) id 1CIADX-00086Z-Fr; Thu, 14 Oct 2004 14:20:47 -0400 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by megatron.ietf.org with esmtp (Exim 4.32) id 1CIAB2-0007eS-Ob for ips@megatron.ietf.org; Thu, 14 Oct 2004 14:18:12 -0400 Received: from ietf-mx.ietf.org (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id OAA12938 for ; Thu, 14 Oct 2004 14:18:11 -0400 (EDT) Received: from atlrel8.hp.com ([156.153.255.206]) by ietf-mx.ietf.org with esmtp (Exim 4.33) id 1CIAMF-0008Lf-BA for ips@ietf.org; Thu, 14 Oct 2004 14:29:47 -0400 Received: from rosemail.rose.hp.com (rosemail.rose.hp.com [15.43.209.160]) by atlrel8.hp.com (Postfix) with ESMTP id BB845CC7C for ; Thu, 14 Oct 2004 14:18:10 -0400 (EDT) Received: from [127.0.0.1] (dhcp-roseor-beb159.rose.hp.com [15.23.141.159]) by rosemail.rose.hp.com (Postfix) with ESMTP id 6C872826C for ; Thu, 14 Oct 2004 11:18:10 -0700 (PDT) Message-ID: <416EC2D5.4010509@rose.hp.com> Date: Thu, 14 Oct 2004 11:17:57 -0700 From: "Mallikarjun C." User-Agent: Mozilla Thunderbird 0.7.3 (Windows/20040803) X-Accept-Language: en-us, en MIME-Version: 1.0 To: ips@ietf.org Subject: Re: [Ips] resending Data-in with S bit set References: <41649B60.60002@rose.hp.com> <4165785D.5050802@rose.hp.com><4166D357.40708@rose.hp.com><4489F857-1956-11D9-91D4-000A95D14BEC@wasabisystems.com><4166E5B0.1090506@rose.hp.com><30FE007A-196F-11D9-91D4-000A95D14BEC@wasabisystems.com><416B0413.8080208@rose.hp.com><3DB9AEB1-1BEA-11D9-8B4B-000A95D14BEC@wasabisystems.com> <416C6214.9080408@rose.hp.com> <000301c4b122$ff67be90$0303a8c0@ivvt2dxrc11> In-Reply-To: <000301c4b122$ff67be90$0303a8c0@ivvt2dxrc11> Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit X-Spam-Score: 0.0 (/) X-Scan-Signature: 963faf56c3a5b6715f0b71b66181e01a Content-Transfer-Encoding: 7bit X-BeenThere: ips@ietf.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: IP Storage List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Sender: ips-bounces@ietf.org Errors-To: ips-bounces@ietf.org X-Spam-Score: 0.0 (/) X-Scan-Signature: 43317e64100dd4d87214c51822b582d1 Content-Transfer-Encoding: 7bit I think there are 3 cases here. Status can be lost if it's phase-collapsed and there's a data digest failure on the last Data-in PDU. A SNACK on the Data-in PDU should work here. If there's no phase collapse, a data digest failure on the sense & response data segment of Response PDU would lead to loss of status. A SNACK on the Response PDU should work here. If there's no phase collapse, a header digest failure on the Response PDU would also lead to loss of status. Markers would be generally required here, but the RFC doesn't mandate it other than saying PDU boundaries should be "reliably ascertained by other means" - other than based on length fields. (section 6.7). Waiting for a sequence error should work here. -- Mallikarjun Mallikarjun Chadalapaka Networked Storage Architecture Network Storage Solutions Hewlett-Packard MS 5668 Roseville CA 95747 cbm [at] rose.hp.com Eddy Quicksall wrote: > Is there any way the initiator can loose status other than from a CRC > error? In which case if markers are not used, there is no way to recover > other than reinstating the connection. Is that correct? > > Eddy > > ----- Original Message ----- From: "Mallikarjun C." > To: > Sent: Tuesday, October 12, 2004 7:00 PM > Subject: Re: [Ips] resending Data-in with S bit set > > >> Bill, >> >> >there should be a >> > disconnect between ExpDataSN on the target (sent in the SCSI Response) >> > and in the initiator. >> >> Yes, that should work for the scenario you described (and in other >> cases that I can now think of). >> >> In general, during the time we designed the iSCSI resegmentation >> approach, the operating assumption as I recall was to not rely on >> absolute sequence numbers from the previous epoch (i.e. before the PDU >> size change) after the change. The R-Data SNACK simply causes a >> reversion to the last ack point which is consistent with not using >> absolute SNs from the past. >> >> The thinking then was that between possibly multiple PDU size changes >> during the life of a task and and sharing of the DataSN/R2TSN space >> for bidirectional commands, there's enough room for implementations to >> get the absolute sequence numbers wrong in their SNACK requests. >> Besides, we (at least I) didn't feel like we have exhaustively >> enumerated all potential scenarios. As far as I recall, we have not >> defined a quiescing process to change the PDU size either. >> >> I'd of course like to hear from other participants of those >> discussions wrt this recollection. >> >> At least this is my rationale for suggesting a big hammer approach >> that doesn't rely on the ExpDataSN field in the status PDU (BTW, I >> don't recall a discussion on ExpDataSN in the past). But I do realize >> that in my approach, under this scenario, the ExpDataSN field in the >> Response PDU is always suspect - so a SNACK is always required or the >> I/O would fail. >> >> Thanks. >> -- >> Mallikarjun >> >> Mallikarjun Chadalapaka >> Networked Storage Architecture >> Network Storage Solutions >> Hewlett-Packard MS 5668 >> Roseville CA 95747 >> cbm [at] rose.hp.com >> >> >> >> >> >> >> >> William Studenmund wrote: >> >>> On Oct 11, 2004, at 6:07 PM, Mallikarjun C. wrote: >>> >>>> I think I may be finally understanding your scenario, but let's >>>> confirm that.... >>> >>> >>> >>> Ok! >>> >>>> A read SCSI task is in progress and the initiator changes >>>> MaxRecvDataSegementLength. >>> >>> >>> >>> Specifically, for what I'm thinking of, all of the data have already >>> been sent before the MaxRecvDataSegmentLength >>> >>>> The task continues. Subsequently, the initiator notices it lost a >>>> status. Here's what I think should happen (I think the same as what >>>> you described, except for the ExpDataSN part). >>>> >>>> - As always, initiator starts with a status SNACK. >>>> - Regardless of the original transmission (phase collapse or not, >>>> old PDU still fits with the new size or not, single/multiple data >>>> PDUs), target retransmits the status in an explicit Response PDU >>>> with ExpDataSN set to something like 0xffffffff. >>>> - Initiator checks ExpDataSN with its records. They shouldn't >>>> match, leading to an R-Data SNACK. >>>> - Rest of the semantics described in 10.16.3 apply. >>>> >>>> My reasons are two: >>>> >>>> - Target always does the same thing in this scenario (PDU size >>>> change followed by a status SNACK first) - i.e. doesn't phase >>>> collapse even if it still can. >>> >>> >>> >>> How about this idea. If the target can still phase collapse (the PDU >>> would still fit), it MAY, but due to the fact that there was a >>> resegmentation, the target MAY instead treat the PDU as if it could >>> not fit. That is to say it is an implementation decision if the >>> target just remembers "resegmented" or if it looks at the PDU. >>> >>>> - Initiator operates based on ExpDataSN as always, consistent with >>>> the spec intent to not have to do scoreboarding/counting. >>> >>> >>> >>> How does what I suggest break ExpDataSN? >>> >>> I was about to say I like what you suggested, then I pondered a bit. >>> I don't think we need the magic ExpDataSN value. If the PDU is too >>> big to send, then the target can just generate a SCSI Response PDU >>> with the correct ExpDataSN and a SNACK Tag of 0. Given that the >>> initiator didn't notice the status value in the phase-collapsed read >>> PDU, it should also have not noticed the data reception in it, so >>> there should be a disconnect between ExpDataSN on the target (sent in >>> the SCSI Response) and in the initiator. So a SCSI Response sent in >>> place of the collapsed Data-In should trigger the initiator to send >>> an R-Data SNACK. Shouldn't it? >>> >>>> OTOH, the RFC doesn't set aside 0xffffffff as a reserved value for >>>> ExpDataSN. So I guess this is a hack for now. Alternatively (or if >>>> target had already sent 2**32-1 PDUs for that task), what you >>>> described works as well (if the initiator is counting bytes). >>> >>> >>> >>> I don't see how you have to count bytes. I'm not trying to sound >>> belligerent, and I'm sorry if it comes across that way. But I think >>> just sending the SCSI Response would work. I'm not seeing how the # >>> of bytes'd be needed. >>> >>> The one thing that would be impacted would be a broken initiator that >>> got the data out of this phase-collapsed-read Data-In PDU but not the >>> status. I think it deserves what it gets... >>> >>> Take care, >>> >>> Bill >> >> >> >> >> >> _______________________________________________ >> Ips mailing list >> Ips@ietf.org >> https://www1.ietf.org/mailman/listinfo/ips _______________________________________________ Ips mailing list Ips@ietf.org https://www1.ietf.org/mailman/listinfo/ips From ips-bounces@ietf.org Thu Oct 14 14:46:03 2004 Received: from ietf-mx.ietf.org (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id OAA14745 for ; Thu, 14 Oct 2004 14:46:02 -0400 (EDT) Received: from megatron.ietf.org ([132.151.6.71]) by ietf-mx.ietf.org with esmtp (Exim 4.33) id 1CIAnD-0000Ok-SS for ips-web-archive@ietf.org; Thu, 14 Oct 2004 14:57:40 -0400 Received: from localhost.localdomain ([127.0.0.1] helo=megatron.ietf.org) by megatron.ietf.org with esmtp (Exim 4.32) id 1CIAYT-0003O3-08; Thu, 14 Oct 2004 14:42:25 -0400 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by megatron.ietf.org with esmtp (Exim 4.32) id 1CIARy-0002BD-79 for ips@megatron.ietf.org; Thu, 14 Oct 2004 14:35:42 -0400 Received: from ietf-mx.ietf.org (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id OAA14128 for ; Thu, 14 Oct 2004 14:35:40 -0400 (EDT) Received: from atlrel6.hp.com ([156.153.255.205]) by ietf-mx.ietf.org with esmtp (Exim 4.33) id 1CIAdB-0000EP-Ah for ips@ietf.org; Thu, 14 Oct 2004 14:47:17 -0400 Received: from rosemail.rose.hp.com (rosemail.rose.hp.com [15.43.209.160]) by atlrel6.hp.com (Postfix) with ESMTP id C4F3897B9 for ; Thu, 14 Oct 2004 14:35:40 -0400 (EDT) Received: from [127.0.0.1] (dhcp-roseor-beb159.rose.hp.com [15.23.141.159]) by rosemail.rose.hp.com (Postfix) with ESMTP id 754D2800A for ; Thu, 14 Oct 2004 11:35:40 -0700 (PDT) Message-ID: <416EC6FA.1070608@rose.hp.com> Date: Thu, 14 Oct 2004 11:35:38 -0700 From: "Mallikarjun C." User-Agent: Mozilla Thunderbird 0.7.3 (Windows/20040803) X-Accept-Language: en-us, en MIME-Version: 1.0 To: ips@ietf.org Subject: Re: [Ips] resending Data-in with S bit set References: <41649B60.60002@rose.hp.com> <4165785D.5050802@rose.hp.com> <4166D357.40708@rose.hp.com> <4489F857-1956-11D9-91D4-000A95D14BEC@wasabisystems.com> <4166E5B0.1090506@rose.hp.com> <30FE007A-196F-11D9-91D4-000A95D14BEC@wasabisystems.com> <416B0413.8080208@rose.hp.com> <3DB9AEB1-1BEA-11D9-8B4B-000A95D14BEC@wasabisystems.com> <416C6214.9080408@rose.hp.com> <3FE9186D-1D20-11D9-859D-000A95D14BEC@wasabisystems.com> In-Reply-To: <3FE9186D-1D20-11D9-859D-000A95D14BEC@wasabisystems.com> Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit X-Spam-Score: 0.0 (/) X-Scan-Signature: 34d35111647d654d033d58d318c0d21a Content-Transfer-Encoding: 7bit X-BeenThere: ips@ietf.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: IP Storage List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Sender: ips-bounces@ietf.org Errors-To: ips-bounces@ietf.org X-Spam-Score: 0.0 (/) X-Scan-Signature: e1b0e72ff1bbd457ceef31828f216a86 Content-Transfer-Encoding: 7bit >I think R-Data SNACK is the right thing to do; > trying to figure out what was where & all just seems like a mess. This is basically the approach taken in 10.16.3. > Here's another way to look at my suggestion. ...So the _one_ DataSN number we're > certain the initiator didn't get was the one in that packet. As I already said, I don't see an obvious problem with your suggestion. It's just the "certain" part (which assumes both sides getting the absolute DataSNs right) that we were not too sure of. -- Mallikarjun Mallikarjun Chadalapaka Networked Storage Architecture Network Storage Solutions Hewlett-Packard MS 5668 Roseville CA 95747 cbm [at] rose.hp.com William Studenmund wrote: > On Oct 12, 2004, at 7:00 PM, Mallikarjun C. wrote: > >> Bill, >> >> >there should be a >> > disconnect between ExpDataSN on the target (sent in the SCSI Response) >> > and in the initiator. >> >> Yes, that should work for the scenario you described (and in other >> cases that I can now think of). >> >> In general, during the time we designed the iSCSI resegmentation >> approach, the operating assumption as I recall was to not rely on >> absolute sequence numbers from the previous epoch (i.e. before the PDU >> size change) after the change. The R-Data SNACK simply causes a >> reversion to the last ack point which is consistent with not using >> absolute SNs from the past. >> >> The thinking then was that between possibly multiple PDU size changes >> during the life of a task and and sharing of the DataSN/R2TSN space >> for bidirectional commands, there's enough room for implementations to >> get the absolute sequence numbers wrong in their SNACK requests. >> Besides, we (at least I) didn't feel like we have exhaustively >> enumerated all potential scenarios. As far as I recall, we have not >> defined a quiescing process to change the PDU size either. > > > I definitely agree that once there's been a resegmentation during the > data sending, that something like R-Data SNACK is what we need. And as a > target implementer, I think R-Data SNACK is the right thing to do; > trying to figure out what was where & all just seems like a mess. Plus, > it could be you can't resend any of the old PDUs.... > >> I'd of course like to hear from other participants of those >> discussions wrt this recollection. >> >> At least this is my rationale for suggesting a big hammer approach >> that doesn't rely on the ExpDataSN field in the status PDU (BTW, I >> don't recall a discussion on ExpDataSN in the past). But I do realize >> that in my approach, under this scenario, the ExpDataSN field in the >> Response PDU is always suspect - so a SNACK is always required or the >> I/O would fail. > > > Here's another way to look at my suggestion. It is what you're > suggesting, except that the magic DataSN number's different. Rather than > 0xffffffff, I chose the DataSN we put in the unacknowledged Data-IN. My > idea was that since the initiator didn't get the StatSN we're talking > about, it didn't get the Data-IN PDU. So the _one_ DataSN number we're > certain the initiator didn't get was the one in that packet. :-) And > since the only thing the initiator will be able to do once it gets the > SCSI Response with the DataSN it won't agree with is to send an R-Data > SNACK, we will then have a SNACK tag and the sequence number spaces are > safe. > > Although to be honest, we could go with the 0xffffffff magic number you > mentioned. We'd need to make some sort of supplemental RFC describing > it, but we could. The reason I think it'd be fine is that > MaxRecvDataSegmentLength can't go below 512. So we would need to go > through around 512 error recovery actions for each original PDU in the > transfer and have had a maximally-sized transfer to hit 0xffffffff for > DataSN/R2T. > > Take care, > > Bill _______________________________________________ Ips mailing list Ips@ietf.org https://www1.ietf.org/mailman/listinfo/ips From ips-bounces@ietf.org Thu Oct 14 18:25:11 2004 Received: from ietf-mx.ietf.org (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id SAA14690 for ; Thu, 14 Oct 2004 18:25:11 -0400 (EDT) Received: from megatron.ietf.org ([132.151.6.71]) by ietf-mx.ietf.org with esmtp (Exim 4.33) id 1CIEDL-0007lC-W0 for ips-web-archive@ietf.org; Thu, 14 Oct 2004 18:36:52 -0400 Received: from localhost.localdomain ([127.0.0.1] helo=megatron.ietf.org) by megatron.ietf.org with esmtp (Exim 4.32) id 1CIDw7-0000kg-Vt; Thu, 14 Oct 2004 18:19:03 -0400 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by megatron.ietf.org with esmtp (Exim 4.32) id 1CIDvN-0000El-NK for ips@megatron.ietf.org; Thu, 14 Oct 2004 18:18:17 -0400 Received: from ietf-mx.ietf.org (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id SAA13698 for ; Thu, 14 Oct 2004 18:18:14 -0400 (EDT) From: pat_thaler@agilent.com Received: from msgbas2x.cos.agilent.com ([192.25.240.37]) by ietf-mx.ietf.org with esmtp (Exim 4.33) id 1CIE6R-0007Z7-GC for ips@ietf.org; Thu, 14 Oct 2004 18:29:54 -0400 Received: from enccos5.cos.agilent.com (enccos5.cos.agilent.com [130.29.152.94]) by msgbas2x.cos.agilent.com (Postfix) with SMTP id 7F9B896B0 for ; Thu, 14 Oct 2004 16:18:04 -0600 (MDT) Received: from relcos2.cos.agilent.com (130.29.152.237) by enccos5.cos.agilent.com (Sigaba Gateway v3.83) with ESMTP id 2401333; Thu, 14 Oct 2004 16:18:38 -0600 Received: from wcosvs02.cos.agilent.com (wcosvs02.cos.agilent.com [130.29.152.188]) by relcos2.cos.agilent.com (Postfix) with ESMTP id 2375F14F for ; Thu, 14 Oct 2004 16:18:04 -0600 (MDT) Received: from wcosbh23.cos.agilent.com ([130.29.152.145]) by wcosvs02.cos.agilent.com with InterScan Messaging Security Suite; Thu, 14 Oct 2004 16:18:03 -0600 Received: from wcosmb05.cos.agilent.com ([130.29.152.72]) by wcosbh23.cos.agilent.com with Microsoft SMTPSVC(5.0.2195.6713); Thu, 14 Oct 2004 16:18:02 -0600 X-MimeOLE: Produced By Microsoft Exchange V6.0.6603.0 Content-class: urn:content-classes:message MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable Date: Thu, 14 Oct 2004 16:18:02 -0600 Message-ID: <9418406DEC3B8A4BBDB1B87291D851AF0E755E@wcosmb05.cos.agilent.com> Thread-Topic: [Ips] iSER flow control options Thread-Index: AcSwhmlK82mwaWJ6QJi061TXG45+ngBtDQfg To: X-OriginalArrivalTime: 14 Oct 2004 22:18:02.0839 (UTC) FILETIME=[ABCD4E70:01C4B23B] X-Spam-Score: 0.3 (/) X-Scan-Signature: 7a6398bf8aaeabc7a7bb696b6b0a2aad Content-Transfer-Encoding: quoted-printable Subject: [Ips] IPS and RDDP meeting times in November X-BeenThere: ips@ietf.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: IP Storage List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Sender: ips-bounces@ietf.org Errors-To: ips-bounces@ietf.org X-Spam-Score: 0.3 (/) X-Scan-Signature: 856eb5f76e7a34990d1d457d8e8e5b7f Content-Transfer-Encoding: quoted-printable David, It is getting close to the IETF meeting and once again it is a challenge = for us storage types as T10 is the same week. Any progress on getting = IPS and RDDP scheduled for early in the week (Monday) so we can = multiplex? By the way, it looks like the same thing occurs again in March 05. Also = March 2005 is still showing as TBD location for IETF. Is there a = location yet? Regards, Pat _______________________________________________ Ips mailing list Ips@ietf.org https://www1.ietf.org/mailman/listinfo/ips From ips-bounces@ietf.org Thu Oct 14 19:30:36 2004 Received: from ietf-mx.ietf.org (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id TAA19115 for ; Thu, 14 Oct 2004 19:30:36 -0400 (EDT) Received: from megatron.ietf.org ([132.151.6.71]) by ietf-mx.ietf.org with esmtp (Exim 4.33) id 1CIFEf-0000Sl-Is for ips-web-archive@ietf.org; Thu, 14 Oct 2004 19:42:17 -0400 Received: from localhost.localdomain ([127.0.0.1] helo=megatron.ietf.org) by megatron.ietf.org with esmtp (Exim 4.32) id 1CIF2T-0003s2-6O; Thu, 14 Oct 2004 19:29:41 -0400 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by megatron.ietf.org with esmtp (Exim 4.32) id 1CIF0w-0003gL-AE for ips@megatron.ietf.org; Thu, 14 Oct 2004 19:28:06 -0400 Received: from ietf-mx.ietf.org (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id TAA19034 for ; Thu, 14 Oct 2004 19:28:03 -0400 (EDT) Message-Id: <200410142328.TAA19034@ietf.org> Received: from psg.com ([147.28.0.62] ident=mailnull) by ietf-mx.ietf.org with esmtp (Exim 4.33) id 1CIFC9-0000QN-4j for ips@ietf.org; Thu, 14 Oct 2004 19:39:44 -0400 Received: from localhost ([127.0.0.1] helo=psg.com) by psg.com with esmtp (Exim 4.41 (FreeBSD)) id 1CIF0o-000Gy5-Jt; Thu, 14 Oct 2004 23:27:58 +0000 To: pat_thaler@agilent.com Subject: Re: [Ips] IPS and RDDP meeting times in November In-Reply-To: Message from pat_thaler@agilent.com of "Thu, 14 Oct 2004 16:18:02 MDT." <9418406DEC3B8A4BBDB1B87291D851AF0E755E@wcosmb05.cos.agilent.com> Date: Thu, 14 Oct 2004 16:27:58 -0700 From: Allison Mankin X-Spam-Score: 0.0 (/) X-Scan-Signature: 79899194edc4f33a41f49410777972f8 Cc: ips@ietf.org, mankin@psg.com, black_david@emc.com, elizabeth.rodriguez@dothill.com X-BeenThere: ips@ietf.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: IP Storage List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Sender: ips-bounces@ietf.org Errors-To: ips-bounces@ietf.org X-Spam-Score: 0.8 (/) X-Scan-Signature: 9182cfff02fae4f1b6e9349e01d62f32 Pat, IPS folks, Your Chairs asked me to ensure that the WG can meet on Monday, and I've committed to making sure it stays there. About the problem with T10 meeting dates and IETF meeting dates, recurring in March 2005, one problem is that the T10 and T11 meetings haven't found their way onto past conflicts calendars... (sorry) There's a conflicts calendar under construction now, for as far out as we can go, so could people send me the important dates to avoid from the IPS point of view? Other meetings for the meeting planners to be aware of are welcome as well. There are no assurances of avoiding all the conflicts, but it helps to start with enough info. Allison _______________________________________________ Ips mailing list Ips@ietf.org https://www1.ietf.org/mailman/listinfo/ips From ips-bounces@ietf.org Fri Oct 15 00:40:08 2004 Received: from ietf-mx.ietf.org (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id AAA07314 for ; Fri, 15 Oct 2004 00:40:08 -0400 (EDT) Received: from megatron.ietf.org ([132.151.6.71]) by ietf-mx.ietf.org with esmtp (Exim 4.33) id 1CIK4G-0005O8-0L for ips-web-archive@ietf.org; Fri, 15 Oct 2004 00:51:52 -0400 Received: from localhost.localdomain ([127.0.0.1] helo=megatron.ietf.org) by megatron.ietf.org with esmtp (Exim 4.32) id 1CIJrd-0006Er-QZ; Fri, 15 Oct 2004 00:38:49 -0400 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by megatron.ietf.org with esmtp (Exim 4.32) id 1CIJqA-00066n-Q4 for ips@megatron.ietf.org; Fri, 15 Oct 2004 00:37:18 -0400 Received: from ietf-mx.ietf.org (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id AAA07241 for ; Fri, 15 Oct 2004 00:37:14 -0400 (EDT) Received: from e1.ny.us.ibm.com ([32.97.182.101]) by ietf-mx.ietf.org with esmtp (Exim 4.33) id 1CIK1R-0005L8-OQ for ips@ietf.org; Fri, 15 Oct 2004 00:48:58 -0400 Received: from northrelay02.pok.ibm.com (northrelay02.pok.ibm.com [9.56.224.150]) by e1.ny.us.ibm.com (8.12.10/NS PXFA) with ESMTP id i9F4ajrZ359412 for ; Fri, 15 Oct 2004 00:36:45 -0400 Received: from d01ml604.pok.ibm.com (d01av04.pok.ibm.com [9.56.224.64]) by northrelay02.pok.ibm.com (8.12.10/NCO/VER6.6) with ESMTP id i9F4c140038040 for ; Fri, 15 Oct 2004 00:38:01 -0400 Importance: Normal MIME-Version: 1.0 To: X-Mailer: Lotus Notes Release 6.5.2 June 01, 2004 Message-ID: From: Mike Ko Date: Thu, 14 Oct 2004 21:36:43 -0700 X-MIMETrack: Serialize by Router on D01ML604/01/M/IBM(Release 6.51HF643 | October 13, 2004) at 10/15/2004 00:36:44, Serialize complete at 10/15/2004 00:36:44 Content-Type: text/plain; charset="US-ASCII" X-Spam-Score: 0.0 (/) X-Scan-Signature: 2086112c730e13d5955355df27e3074b Subject: [Ips] New iSCSI key X-BeenThere: ips@ietf.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: IP Storage List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Sender: ips-bounces@ietf.org Errors-To: ips-bounces@ietf.org X-Spam-Score: 0.0 (/) X-Scan-Signature: 22bbb45ef41b733eb2d03ee71ece8243 If we do decide on adding a new iSCSI key to limit the number of "unexpected" or "unanticipated" PDUs, I offer the following text to be considered for adding to section 6 of the iSER draft. Currently the key is tentatively called "MaxOutstandingUnexpectedPDUs". Unless there are other suggestions, this name will be used. Use: LO (leading only), Declarative Senders: Initiator and Target Scope: SW (session-wide) Irrelevant when: RDMAExtensions=No MaxOutstandingUnexpectedPDUs= Default is None This key is used by the initiator and the target to declare the maximum number of outstanding "unexpected" control-type PDUs that it can receive. It is intended to allow the receiving side to determine the amount of buffer resoruces needed beyond the normal flow control mechanism available in iSCSI. An initiator or target that cannot function properly without any limit on MaxOutstandingUnexpectedPDU (which is the default) should choose a value such that it would not impose an unnecessary constraint on the iSCSI layer under normal circumstances. The proper value to use is implementation dependent. If the sending side fails to adhere to the declared limit as set by the receiving side, it is up to the receiving side to determine ways of handling the overrun, including dropping the connection. For control-type PDUs sent by an initiator to a target, where the immediate flag is not set, the command numbering mechanism regulates the maximum number of PDUs that the target can expect. The exceptions include the SNACK PDU, the SCSI Data-out PDU, and any PDU with the immediate flag set. 1. The SNACK PDU is not needed in iSER (see section 7.3.11). But if one is requested for transmission by the iSCSI layer at the initiator, it will be sent by the iSER layer as a control-type PDU. Since it is "unexpected", the number that can be sent by the initiator falls under the MaxOutstandingUnexpectedPDU declared by the target. The PDU is outstanding until the target responds with a control-type PDU with ExpCmdSN set to X + 1 where X is the CmdSN of the referenced command. 2. For unsolicited SCSI Data-out PDU, the target can use FirstBurstLength to determine the amount of buffering required. Solicited data are normally handled using the R2T mechanism (see section 7.3.4). However, if a SCSI Data-out PDU containing solicited data is requested for transmission by the iSCSI layer at the initiator, the iSER layer may send it as unsolciited data. Since it is "unexpected", the number that can be sent by the initiator falls under the MaxOutstandingUnexpectedPDU declared by the target. The PDU is outstanding until the target responds with a control-type PDU with ExpCmdSN set to X + 1 where X is the CmdSN of the referenced command. 3. A PDU with the immediate flag set is considered "unexpected", and the number that can be sent by the initiator falls under the MaxOutstandingUnexpectedPDU declared by the target. The PDU is outstanding until the target responds with a control-type PDU with ExpCmdSN set to X + 1 where X is the CmdSN of the PDU with the immediate flag set. Control-type PDUs sent by a target as a response to a request from an initiator are expected by the initiator and buffer resources required can be provisioned. The exceptions include the SCSI Data-in PDU, the R2T PDU, the Asynchronous Message PDU, the Reject PDU, and the NOP-in PDU. 1. A SCSI Data-in PDU is a data-type PDU and is transformed into RDMA Write operations (see section 7.3.5). 2. An R2T PDU is a data-type PDU and is transformed into RDMA Read operations (see section 7.3.6). 3. An Asynchronous Message PDU is considered "unexpected", and the number that can be sent by the target falls under the MaxOutstandingUnexpectedPDU declared by the initiator. The PDU is outstanding until the initiator sends a control-type PDU with ExpStatSN set to X + 1 where X is the StatSN of the Asychronous Message PDU. 4. A Reject PDU is considered "unexpected", and the number that can be sent by the target falls under the MaxOutstandingUnexpectedPDU declared by the initiator. The PDU is outstanding until the initiator sends a control-type PDU with ExpStatSN set to X + 1 where X is the StatSN of the Reject PDU. 5. A NOP-in PDU sent as a response by the target to a NOP-out PDU from the initiator is expected by the initiator. Otherwise, a NOP-out PDU is considered "unexpected", and the number that can be sent by the target falls under the MaxOutstandingUnexpectedPDU declared by the initiator. The PDU is outstanding until the initiator sends a control-type PDU with ExpStatSN set to X + 1 where X is the StatSN of the NOP-in PDU. Mike _______________________________________________ Ips mailing list Ips@ietf.org https://www1.ietf.org/mailman/listinfo/ips From ips-bounces@ietf.org Fri Oct 15 02:23:03 2004 Received: from ietf-mx.ietf.org (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id CAA26693 for ; Fri, 15 Oct 2004 02:23:03 -0400 (EDT) Received: from megatron.ietf.org ([132.151.6.71]) by ietf-mx.ietf.org with esmtp (Exim 4.33) id 1CILfq-0006xr-Nv for ips-web-archive@ietf.org; Fri, 15 Oct 2004 02:34:47 -0400 Received: from localhost.localdomain ([127.0.0.1] helo=megatron.ietf.org) by megatron.ietf.org with esmtp (Exim 4.32) id 1CILSy-0007md-Vo; Fri, 15 Oct 2004 02:21:29 -0400 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by megatron.ietf.org with esmtp (Exim 4.32) id 1CILRc-0007Jb-Jx for ips@megatron.ietf.org; Fri, 15 Oct 2004 02:20:04 -0400 Received: from ietf-mx.ietf.org (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id CAA26395 for ; Fri, 15 Oct 2004 02:20:02 -0400 (EDT) Received: from astound-64-85-224-245.ca.astound.net ([64.85.224.245] helo=master.linux-ide.org) by ietf-mx.ietf.org with esmtp (Exim 4.33) id 1CILcv-0006v1-A9 for ips@ietf.org; Fri, 15 Oct 2004 02:31:45 -0400 Received: from localhost (andre@localhost) by master.linux-ide.org (8.11.0/8.11.0/SuSE Linux 8.11.0-0.4) with ESMTP id i9F67QM15727; Thu, 14 Oct 2004 23:07:27 -0700 Date: Thu, 14 Oct 2004 23:07:26 -0700 (PDT) From: Andre Hedrick X-Sender: andre@master.linux-ide.org To: Mike Ko Subject: Re: [Ips] New iSCSI key In-Reply-To: Message-ID: MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-Spam-Score: 0.0 (/) X-Scan-Signature: 87a3f533bb300b99e2a18357f3c1563d Cc: ips@ietf.org X-BeenThere: ips@ietf.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: IP Storage List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Sender: ips-bounces@ietf.org Errors-To: ips-bounces@ietf.org X-Spam-Score: 0.0 (/) X-Scan-Signature: 29dc808194f5fb921c09d0040806d6eb Mike, Not to be cheeky, but would it be possible for everyone to support and send all the current keys first before adding another? Cheers, Andre On Thu, 14 Oct 2004, Mike Ko wrote: > If we do decide on adding a new iSCSI key to limit the number of > "unexpected" > or "unanticipated" PDUs, I offer the following text to be considered for > adding > to section 6 of the iSER draft. > > Currently the key is tentatively called "MaxOutstandingUnexpectedPDUs". > Unless > there are other suggestions, this name will be used. > > Use: LO (leading only), Declarative > Senders: Initiator and Target > Scope: SW (session-wide) > Irrelevant when: RDMAExtensions=No > MaxOutstandingUnexpectedPDUs= > Default is None > > This key is used by the initiator and the target to declare the maximum > number > of outstanding "unexpected" control-type PDUs that it can receive. It is > intended to allow the receiving side to determine the amount of buffer > resoruces needed beyond the normal flow control mechanism available in > iSCSI. > An initiator or target that cannot function properly without any limit on > MaxOutstandingUnexpectedPDU (which is the default) should choose a value > such > that it would not impose an unnecessary constraint on the iSCSI layer > under > normal circumstances. The proper value to use is implementation > dependent. If > the sending side fails to adhere to the declared limit as set by the > receiving > side, it is up to the receiving side to determine ways of handling the > overrun, > including dropping the connection. > > For control-type PDUs sent by an initiator to a target, where the > immediate > flag is not set, the command numbering mechanism regulates the maximum > number > of PDUs that the target can expect. The exceptions include the SNACK PDU, > the > SCSI Data-out PDU, and any PDU with the immediate flag set. > > 1. The SNACK PDU is not needed in iSER (see section 7.3.11). But if one > is > requested for transmission by the iSCSI layer at the initiator, it will be > sent > by the iSER layer as a control-type PDU. Since it is "unexpected", the > number > that can be sent by the initiator falls under the > MaxOutstandingUnexpectedPDU > declared by the target. The PDU is outstanding until the target responds > with > a control-type PDU with ExpCmdSN set to X + 1 where X is the CmdSN of the > referenced command. > > 2. For unsolicited SCSI Data-out PDU, the target can use FirstBurstLength > to > determine the amount of buffering required. Solicited data are normally > handled using the R2T mechanism (see section 7.3.4). However, if a SCSI > Data-out PDU containing solicited data is requested for transmission by > the > iSCSI layer at the initiator, the iSER layer may send it as unsolciited > data. > Since it is "unexpected", the number that can be sent by the initiator > falls > under the MaxOutstandingUnexpectedPDU declared by the target. The PDU is > outstanding until the target responds with a control-type PDU with > ExpCmdSN set > to X + 1 where X is the CmdSN of the referenced command. > > 3. A PDU with the immediate flag set is considered "unexpected", and the > number > that can be sent by the initiator falls under the > MaxOutstandingUnexpectedPDU > declared by the target. The PDU is outstanding until the target responds > with > a control-type PDU with ExpCmdSN set to X + 1 where X is the CmdSN of the > PDU > with the immediate flag set. > > Control-type PDUs sent by a target as a response to a request from an > initiator > are expected by the initiator and buffer resources required can be > provisioned. > The exceptions include the SCSI Data-in PDU, the R2T PDU, the Asynchronous > Message PDU, the Reject PDU, and the NOP-in PDU. > > 1. A SCSI Data-in PDU is a data-type PDU and is transformed into RDMA > Write > operations (see section 7.3.5). > > 2. An R2T PDU is a data-type PDU and is transformed into RDMA Read > operations > (see section 7.3.6). > > 3. An Asynchronous Message PDU is considered "unexpected", and the number > that > can be sent by the target falls under the MaxOutstandingUnexpectedPDU > declared > by the initiator. The PDU is outstanding until the initiator sends a > control-type PDU with ExpStatSN set to X + 1 where X is the StatSN of the > Asychronous Message PDU. > > 4. A Reject PDU is considered "unexpected", and the number that can be > sent by > the target falls under the MaxOutstandingUnexpectedPDU declared by the > initiator. The PDU is outstanding until the initiator sends a > control-type PDU > with ExpStatSN set to X + 1 where X is the StatSN of the Reject PDU. > > 5. A NOP-in PDU sent as a response by the target to a NOP-out PDU from the > initiator is expected by the initiator. Otherwise, a NOP-out PDU is > considered > "unexpected", and the number that can be sent by the target falls under > the > MaxOutstandingUnexpectedPDU declared by the initiator. The PDU is > outstanding > until the initiator sends a control-type PDU with ExpStatSN set to X + 1 > where > X is the StatSN of the NOP-in PDU. > > Mike > > _______________________________________________ > Ips mailing list > Ips@ietf.org > https://www1.ietf.org/mailman/listinfo/ips > _______________________________________________ Ips mailing list Ips@ietf.org https://www1.ietf.org/mailman/listinfo/ips From ips-bounces@ietf.org Fri Oct 15 02:44:38 2004 Received: from ietf-mx.ietf.org (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id CAA28624 for ; Fri, 15 Oct 2004 02:44:38 -0400 (EDT) Received: from megatron.ietf.org ([132.151.6.71]) by ietf-mx.ietf.org with esmtp (Exim 4.33) id 1CIM0j-0007J9-Hq for ips-web-archive@ietf.org; Fri, 15 Oct 2004 02:56:21 -0400 Received: from localhost.localdomain ([127.0.0.1] helo=megatron.ietf.org) by megatron.ietf.org with esmtp (Exim 4.32) id 1CILl6-0004SF-Ku; Fri, 15 Oct 2004 02:40:12 -0400 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by megatron.ietf.org with esmtp (Exim 4.32) id 1CILjR-0004Db-N2 for ips@megatron.ietf.org; Fri, 15 Oct 2004 02:38:29 -0400 Received: from ietf-mx.ietf.org (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id CAA28206 for ; Fri, 15 Oct 2004 02:38:28 -0400 (EDT) Received: from e3.ny.us.ibm.com ([32.97.182.103]) by ietf-mx.ietf.org with esmtp (Exim 4.33) id 1CILuk-0007Ck-WF for ips@ietf.org; Fri, 15 Oct 2004 02:50:11 -0400 Received: from northrelay04.pok.ibm.com (northrelay04.pok.ibm.com [9.56.224.206]) by e3.ny.us.ibm.com (8.12.10/8.12.9) with ESMTP id i9F6brPJ479086; Fri, 15 Oct 2004 02:37:54 -0400 Received: from d01ml604.pok.ibm.com (d01av02.pok.ibm.com [9.56.224.216]) by northrelay04.pok.ibm.com (8.12.10/NCO/VER6.6) with ESMTP id i9F6d85x101220; Fri, 15 Oct 2004 02:39:08 -0400 In-Reply-To: Importance: Normal MIME-Version: 1.0 To: Andre Hedrick Subject: Re: [Ips] New iSCSI key X-Mailer: Lotus Notes Release 6.5.2 June 01, 2004 Message-ID: From: Mike Ko Date: Thu, 14 Oct 2004 23:37:51 -0700 X-MIMETrack: Serialize by Router on D01ML604/01/M/IBM(Release 6.51HF643 | October 13, 2004) at 10/15/2004 02:37:53, Serialize complete at 10/15/2004 02:37:53 Content-Type: text/plain; charset="US-ASCII" X-Spam-Score: 0.0 (/) X-Scan-Signature: 848ed35f2a4fc0638fa89629cb640f48 Cc: ips@ietf.org X-BeenThere: ips@ietf.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: IP Storage List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Sender: ips-bounces@ietf.org Errors-To: ips-bounces@ietf.org X-Spam-Score: 0.0 (/) X-Scan-Signature: 0e9ebc0cbd700a87c0637ad0e2c91610 Andre, I am not sure I follow your comment. Can you be more specific? Mike To: Mike Ko cc: ips@ietf.org Subject: Re: [Ips] New iSCSI key Mike, Not to be cheeky, but would it be possible for everyone to support and send all the current keys first before adding another? Cheers, Andre On Thu, 14 Oct 2004, Mike Ko wrote: > If we do decide on adding a new iSCSI key to limit the number of > "unexpected" > or "unanticipated" PDUs, I offer the following text to be considered for > adding > to section 6 of the iSER draft. > > Currently the key is tentatively called "MaxOutstandingUnexpectedPDUs". > Unless > there are other suggestions, this name will be used. > > Use: LO (leading only), Declarative > Senders: Initiator and Target > Scope: SW (session-wide) > Irrelevant when: RDMAExtensions=No > MaxOutstandingUnexpectedPDUs= > Default is None > > This key is used by the initiator and the target to declare the maximum > number > of outstanding "unexpected" control-type PDUs that it can receive. It is > intended to allow the receiving side to determine the amount of buffer > resoruces needed beyond the normal flow control mechanism available in > iSCSI. > An initiator or target that cannot function properly without any limit on > MaxOutstandingUnexpectedPDU (which is the default) should choose a value > such > that it would not impose an unnecessary constraint on the iSCSI layer > under > normal circumstances. The proper value to use is implementation > dependent. If > the sending side fails to adhere to the declared limit as set by the > receiving > side, it is up to the receiving side to determine ways of handling the > overrun, > including dropping the connection. > > For control-type PDUs sent by an initiator to a target, where the > immediate > flag is not set, the command numbering mechanism regulates the maximum > number > of PDUs that the target can expect. The exceptions include the SNACK PDU, > the > SCSI Data-out PDU, and any PDU with the immediate flag set. > > 1. The SNACK PDU is not needed in iSER (see section 7.3.11). But if one > is > requested for transmission by the iSCSI layer at the initiator, it will be > sent > by the iSER layer as a control-type PDU. Since it is "unexpected", the > number > that can be sent by the initiator falls under the > MaxOutstandingUnexpectedPDU > declared by the target. The PDU is outstanding until the target responds > with > a control-type PDU with ExpCmdSN set to X + 1 where X is the CmdSN of the > referenced command. > > 2. For unsolicited SCSI Data-out PDU, the target can use FirstBurstLength > to > determine the amount of buffering required. Solicited data are normally > handled using the R2T mechanism (see section 7.3.4). However, if a SCSI > Data-out PDU containing solicited data is requested for transmission by > the > iSCSI layer at the initiator, the iSER layer may send it as unsolciited > data. > Since it is "unexpected", the number that can be sent by the initiator > falls > under the MaxOutstandingUnexpectedPDU declared by the target. The PDU is > outstanding until the target responds with a control-type PDU with > ExpCmdSN set > to X + 1 where X is the CmdSN of the referenced command. > > 3. A PDU with the immediate flag set is considered "unexpected", and the > number > that can be sent by the initiator falls under the > MaxOutstandingUnexpectedPDU > declared by the target. The PDU is outstanding until the target responds > with > a control-type PDU with ExpCmdSN set to X + 1 where X is the CmdSN of the > PDU > with the immediate flag set. > > Control-type PDUs sent by a target as a response to a request from an > initiator > are expected by the initiator and buffer resources required can be > provisioned. > The exceptions include the SCSI Data-in PDU, the R2T PDU, the Asynchronous > Message PDU, the Reject PDU, and the NOP-in PDU. > > 1. A SCSI Data-in PDU is a data-type PDU and is transformed into RDMA > Write > operations (see section 7.3.5). > > 2. An R2T PDU is a data-type PDU and is transformed into RDMA Read > operations > (see section 7.3.6). > > 3. An Asynchronous Message PDU is considered "unexpected", and the number > that > can be sent by the target falls under the MaxOutstandingUnexpectedPDU > declared > by the initiator. The PDU is outstanding until the initiator sends a > control-type PDU with ExpStatSN set to X + 1 where X is the StatSN of the > Asychronous Message PDU. > > 4. A Reject PDU is considered "unexpected", and the number that can be > sent by > the target falls under the MaxOutstandingUnexpectedPDU declared by the > initiator. The PDU is outstanding until the initiator sends a > control-type PDU > with ExpStatSN set to X + 1 where X is the StatSN of the Reject PDU. > > 5. A NOP-in PDU sent as a response by the target to a NOP-out PDU from the > initiator is expected by the initiator. Otherwise, a NOP-out PDU is > considered > "unexpected", and the number that can be sent by the target falls under > the > MaxOutstandingUnexpectedPDU declared by the initiator. The PDU is > outstanding > until the initiator sends a control-type PDU with ExpStatSN set to X + 1 > where > X is the StatSN of the NOP-in PDU. > > Mike > > _______________________________________________ > Ips mailing list > Ips@ietf.org > https://www1.ietf.org/mailman/listinfo/ips > _______________________________________________ Ips mailing list Ips@ietf.org https://www1.ietf.org/mailman/listinfo/ips From ips-bounces@ietf.org Fri Oct 15 03:21:05 2004 Received: from ietf-mx.ietf.org (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id DAA01393 for ; Fri, 15 Oct 2004 03:21:05 -0400 (EDT) Received: from megatron.ietf.org ([132.151.6.71]) by ietf-mx.ietf.org with esmtp (Exim 4.33) id 1CIMa0-000818-UT for ips-web-archive@ietf.org; Fri, 15 Oct 2004 03:32:49 -0400 Received: from localhost.localdomain ([127.0.0.1] helo=megatron.ietf.org) by megatron.ietf.org with esmtp (Exim 4.32) id 1CIMMv-0005RZ-4p; Fri, 15 Oct 2004 03:19:17 -0400 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by megatron.ietf.org with esmtp (Exim 4.32) id 1CIMIB-0004XT-Pd for ips@megatron.ietf.org; Fri, 15 Oct 2004 03:14:23 -0400 Received: from ietf-mx.ietf.org (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id DAA00883 for ; Fri, 15 Oct 2004 03:14:21 -0400 (EDT) Received: from e31.co.us.ibm.com ([32.97.110.129]) by ietf-mx.ietf.org with esmtp (Exim 4.33) id 1CIMTV-0007sE-1a for ips@ietf.org; Fri, 15 Oct 2004 03:26:05 -0400 Received: from westrelay02.boulder.ibm.com (westrelay02.boulder.ibm.com [9.17.195.11]) by e31.co.us.ibm.com (8.12.10/8.12.9) with ESMTP id i9F7DpLv331426 for ; Fri, 15 Oct 2004 03:13:51 -0400 Received: from d03av04.boulder.ibm.com (d03av04.boulder.ibm.com [9.17.195.170]) by westrelay02.boulder.ibm.com (8.12.10/NCO/VER6.6) with ESMTP id i9F7Dp2O267714 for ; Fri, 15 Oct 2004 01:13:51 -0600 Received: from d03av04.boulder.ibm.com (loopback [127.0.0.1]) by d03av04.boulder.ibm.com (8.12.11/8.12.11) with ESMTP id i9F7DpxO004112 for ; Fri, 15 Oct 2004 01:13:51 -0600 Received: from d03nm115.boulder.ibm.com (d03nm115.boulder.ibm.com [9.17.195.141]) by d03av04.boulder.ibm.com (8.12.11/8.12.11) with ESMTP id i9F7DpkU004108 for ; Fri, 15 Oct 2004 01:13:51 -0600 In-Reply-To: To: Mike Ko MIME-Version: 1.0 Subject: Re: [Ips] New iSCSI key X-Mailer: Lotus Notes Release 6.0.2CF1 June 9, 2003 From: John Hufferd Message-ID: Date: Fri, 15 Oct 2004 00:13:38 -0700 X-MIMETrack: Serialize by Router on D03NM115/03/M/IBM(Release 6.51HF338 | June 21, 2004) at 10/15/2004 01:13:51, Serialize complete at 10/15/2004 01:13:51 X-Spam-Score: 0.0 (/) X-Scan-Signature: b5216aa5b0df24d46eaed76d4f65aa31 Cc: ips@ietf.org X-BeenThere: ips@ietf.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: IP Storage List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Content-Type: multipart/mixed; boundary="===============1852256545==" Sender: ips-bounces@ietf.org Errors-To: ips-bounces@ietf.org X-Spam-Score: 0.0 (/) X-Scan-Signature: 563af5038a5e1dade28c8affc0fff375 This is a multipart message in MIME format. --===============1852256545== Content-Type: multipart/alternative; boundary="=_alternative 00279FBD88256F2E_=" This is a multipart message in MIME format. --=_alternative 00279FBD88256F2E_= Content-Type: text/plain; charset="US-ASCII" Mike, First I think there is a Typo near the end of your note when you talk about NOP-Out PDU where you might have meant Nop-In. That is, where you say "...Otherwise, a NOP-out PDU is considered "unexpected",...." perhaps you should say Nop-In. Second I think we need to discuss the unsolicited Data-Out PDU. Please expand on the concept of the use of Data-Out PDU for solicited Data in iSER. Third we need to discuss or expand on considering the Reject PDU an unsolicited PDU. Perhaps you meant Rejects for PDUs that are not for the Command itself. . . John L. Hufferd Senior Technical Staff Member (STSM) IBM/System Group, San Jose CA Main Office: (408) 256-0403, Tie: 276-0403, eFax: (408) 904-4688 Alt Office: (408) 997-6136, Cell: (408) 499-9702 Internet Address: hufferd@us.ibm.com Mike Ko Sent by: ips-bounces@ietf.org 10/14/2004 09:36 PM To cc Subject [Ips] New iSCSI key If we do decide on adding a new iSCSI key to limit the number of "unexpected" or "unanticipated" PDUs, I offer the following text to be considered for adding to section 6 of the iSER draft. Currently the key is tentatively called "MaxOutstandingUnexpectedPDUs". Unless there are other suggestions, this name will be used. Use: LO (leading only), Declarative Senders: Initiator and Target Scope: SW (session-wide) Irrelevant when: RDMAExtensions=No MaxOutstandingUnexpectedPDUs= Default is None This key is used by the initiator and the target to declare the maximum number of outstanding "unexpected" control-type PDUs that it can receive. It is intended to allow the receiving side to determine the amount of buffer resoruces needed beyond the normal flow control mechanism available in iSCSI. An initiator or target that cannot function properly without any limit on MaxOutstandingUnexpectedPDU (which is the default) should choose a value such that it would not impose an unnecessary constraint on the iSCSI layer under normal circumstances. The proper value to use is implementation dependent. If the sending side fails to adhere to the declared limit as set by the receiving side, it is up to the receiving side to determine ways of handling the overrun, including dropping the connection. For control-type PDUs sent by an initiator to a target, where the immediate flag is not set, the command numbering mechanism regulates the maximum number of PDUs that the target can expect. The exceptions include the SNACK PDU, the SCSI Data-out PDU, and any PDU with the immediate flag set. 1. The SNACK PDU is not needed in iSER (see section 7.3.11). But if one is requested for transmission by the iSCSI layer at the initiator, it will be sent by the iSER layer as a control-type PDU. Since it is "unexpected", the number that can be sent by the initiator falls under the MaxOutstandingUnexpectedPDU declared by the target. The PDU is outstanding until the target responds with a control-type PDU with ExpCmdSN set to X + 1 where X is the CmdSN of the referenced command. 2. For unsolicited SCSI Data-out PDU, the target can use FirstBurstLength to determine the amount of buffering required. Solicited data are normally handled using the R2T mechanism (see section 7.3.4). However, if a SCSI Data-out PDU containing solicited data is requested for transmission by the iSCSI layer at the initiator, the iSER layer may send it as unsolciited data. Since it is "unexpected", the number that can be sent by the initiator falls under the MaxOutstandingUnexpectedPDU declared by the target. The PDU is outstanding until the target responds with a control-type PDU with ExpCmdSN set to X + 1 where X is the CmdSN of the referenced command. 3. A PDU with the immediate flag set is considered "unexpected", and the number that can be sent by the initiator falls under the MaxOutstandingUnexpectedPDU declared by the target. The PDU is outstanding until the target responds with a control-type PDU with ExpCmdSN set to X + 1 where X is the CmdSN of the PDU with the immediate flag set. Control-type PDUs sent by a target as a response to a request from an initiator are expected by the initiator and buffer resources required can be provisioned. The exceptions include the SCSI Data-in PDU, the R2T PDU, the Asynchronous Message PDU, the Reject PDU, and the NOP-in PDU. 1. A SCSI Data-in PDU is a data-type PDU and is transformed into RDMA Write operations (see section 7.3.5). 2. An R2T PDU is a data-type PDU and is transformed into RDMA Read operations (see section 7.3.6). 3. An Asynchronous Message PDU is considered "unexpected", and the number that can be sent by the target falls under the MaxOutstandingUnexpectedPDU declared by the initiator. The PDU is outstanding until the initiator sends a control-type PDU with ExpStatSN set to X + 1 where X is the StatSN of the Asychronous Message PDU. 4. A Reject PDU is considered "unexpected", and the number that can be sent by the target falls under the MaxOutstandingUnexpectedPDU declared by the initiator. The PDU is outstanding until the initiator sends a control-type PDU with ExpStatSN set to X + 1 where X is the StatSN of the Reject PDU. 5. A NOP-in PDU sent as a response by the target to a NOP-out PDU from the initiator is expected by the initiator. Otherwise, a NOP-out PDU is considered "unexpected", and the number that can be sent by the target falls under the MaxOutstandingUnexpectedPDU declared by the initiator. The PDU is outstanding until the initiator sends a control-type PDU with ExpStatSN set to X + 1 where X is the StatSN of the NOP-in PDU. Mike _______________________________________________ Ips mailing list Ips@ietf.org https://www1.ietf.org/mailman/listinfo/ips --=_alternative 00279FBD88256F2E_= Content-Type: text/html; charset="US-ASCII"
Mike,
First I think there is a Typo near the end of your note when you talk about NOP-Out PDU where you might have meant Nop-In.  That is, where you say "...Otherwise, a NOP-out PDU is
considered "unexpected",...." perhaps you should say Nop-In.

Second I think we need to discuss the unsolicited Data-Out PDU.  Please expand on the concept of the use of Data-Out PDU for solicited Data in iSER.

Third we need to discuss or expand on considering the Reject PDU an unsolicited PDU.   Perhaps you meant Rejects for PDUs that are not for the Command itself.
.
.
John L. Hufferd
Senior Technical Staff Member (STSM)
IBM/System Group, San Jose CA
Main Office: (408) 256-0403, Tie: 276-0403, eFax: (408) 904-4688
Alt Office: (408) 997-6136, Cell: (408) 499-9702
Internet Address: hufferd@us.ibm.com


Mike Ko <mako@almaden.ibm.com>
Sent by: ips-bounces@ietf.org

10/14/2004 09:36 PM

To
<ips@ietf.org>
cc
Subject
[Ips] New iSCSI key





If we do decide on adding a new iSCSI key to limit the number of
"unexpected"
or "unanticipated" PDUs, I offer the following text to be considered for
adding
to section 6 of the iSER draft.

Currently the key is tentatively called "MaxOutstandingUnexpectedPDUs".
Unless
there are other suggestions, this name will be used.

Use: LO (leading only), Declarative
Senders: Initiator and Target
Scope: SW (session-wide)
Irrelevant when: RDMAExtensions=No
MaxOutstandingUnexpectedPDUs=<numerical-value-from-0-to-(2**64-1) | None>
Default is None

This key is used by the initiator and the target to declare the maximum
number
of outstanding "unexpected" control-type PDUs that it can receive.  It is
intended to allow the receiving side to determine the amount of buffer
resoruces needed beyond the normal flow control mechanism available in
iSCSI.
An initiator or target that cannot function properly without any limit on
MaxOutstandingUnexpectedPDU (which is the default) should choose a value
such
that it would not impose an unnecessary constraint on the iSCSI layer
under
normal circumstances.  The proper value to use is implementation
dependent.  If
the sending side fails to adhere to the declared limit as set by the
receiving
side, it is up to the receiving side to determine ways of handling the
overrun,
including dropping the connection.

For control-type PDUs sent by an initiator to a target, where the
immediate
flag is not set, the command numbering mechanism regulates the maximum
number
of PDUs that the target can expect.  The exceptions include the SNACK PDU,
the
SCSI Data-out PDU, and any PDU with the immediate flag set.

1. The SNACK PDU is not needed in iSER (see section 7.3.11).  But if one
is
requested for transmission by the iSCSI layer at the initiator, it will be
sent
by the iSER layer as a control-type PDU.  Since it is "unexpected", the
number
that can be sent by the initiator falls under the
MaxOutstandingUnexpectedPDU
declared by the target.  The PDU is outstanding until the target responds
with
a control-type PDU with ExpCmdSN set to X + 1 where X is the CmdSN of the
referenced command.

2. For unsolicited SCSI Data-out PDU, the target can use FirstBurstLength
to
determine the amount of buffering required.  Solicited data are normally
handled using the R2T mechanism (see section 7.3.4).  However, if a SCSI
Data-out PDU containing solicited data is requested for transmission by
the
iSCSI layer at the initiator, the iSER layer may send it as unsolciited
data.
Since it is "unexpected", the number that can be sent by the initiator
falls
under the MaxOutstandingUnexpectedPDU declared by the target.  The PDU is
outstanding until the target responds with a control-type PDU with
ExpCmdSN set
to X + 1 where X is the CmdSN of the referenced command.

3. A PDU with the immediate flag set is considered "unexpected", and the
number
that can be sent by the initiator falls under the
MaxOutstandingUnexpectedPDU
declared by the target.  The PDU is outstanding until the target responds
with
a control-type PDU with ExpCmdSN set to X + 1 where X is the CmdSN of the
PDU
with the immediate flag set.

Control-type PDUs sent by a target as a response to a request from an
initiator
are expected by the initiator and buffer resources required can be
provisioned.
The exceptions include the SCSI Data-in PDU, the R2T PDU, the Asynchronous
Message PDU, the Reject PDU, and the NOP-in PDU.

1. A SCSI Data-in PDU is a data-type PDU and is transformed into RDMA
Write
operations (see section 7.3.5).

2. An R2T PDU is a data-type PDU and is transformed into RDMA Read
operations
(see section 7.3.6).

3. An Asynchronous Message PDU is considered "unexpected", and the number
that
can be sent by the target falls under the MaxOutstandingUnexpectedPDU
declared
by the initiator.  The PDU is outstanding until the initiator sends a
control-type PDU with ExpStatSN set to X + 1 where X is the StatSN of the
Asychronous Message PDU.

4. A Reject PDU is considered "unexpected", and the number that can be
sent by
the target falls under the MaxOutstandingUnexpectedPDU declared by the
initiator.  The PDU is outstanding until the initiator sends a
control-type PDU
with ExpStatSN set to X + 1 where X is the StatSN of the Reject PDU.

5. A NOP-in PDU sent as a response by the target to a NOP-out PDU from the
initiator is expected by the initiator.  Otherwise, a NOP-out PDU is
considered
"unexpected", and the number that can be sent by the target falls under
the
MaxOutstandingUnexpectedPDU declared by the initiator.  The PDU is
outstanding
until the initiator sends a control-type PDU with ExpStatSN set to X + 1
where
X is the StatSN of the NOP-in PDU.

Mike

_______________________________________________
Ips mailing list
Ips@ietf.org
https://www1.ietf.org/mailman/listinfo/ips
--=_alternative 00279FBD88256F2E_=-- --===============1852256545== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline Content-Transfer-Encoding: 7bit _______________________________________________ Ips mailing list Ips@ietf.org https://www1.ietf.org/mailman/listinfo/ips --===============1852256545==-- From ips-bounces@ietf.org Fri Oct 15 05:42:40 2004 Received: from ietf-mx.ietf.org (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id FAA10573 for ; Fri, 15 Oct 2004 05:42:40 -0400 (EDT) Received: from megatron.ietf.org ([132.151.6.71]) by ietf-mx.ietf.org with esmtp (Exim 4.33) id 1CIOn4-0002JL-Cq for ips-web-archive@ietf.org; Fri, 15 Oct 2004 05:54:26 -0400 Received: from localhost.localdomain ([127.0.0.1] helo=megatron.ietf.org) by megatron.ietf.org with esmtp (Exim 4.32) id 1CIOZ9-0008Nm-Vz; Fri, 15 Oct 2004 05:40:03 -0400 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by megatron.ietf.org with esmtp (Exim 4.32) id 1CIOXr-00085Q-9m for ips@megatron.ietf.org; Fri, 15 Oct 2004 05:38:43 -0400 Received: from ietf-mx.ietf.org (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id FAA09631 for ; Fri, 15 Oct 2004 05:38:40 -0400 (EDT) Received: from system60.simplicato.com ([207.99.47.60] helo=host52a.simplicato.com) by ietf-mx.ietf.org with esmtp (Exim 4.33) id 1CIOjB-00026b-G5 for ips@ietf.org; Fri, 15 Oct 2004 05:50:26 -0400 Received: from localhost (localhost.simplicato.com [127.0.0.1]) by host52a.simplicato.com (Postfix) with ESMTP id B1D7911E0DD; Fri, 15 Oct 2004 05:38:36 -0400 (EDT) Received: from host52a.simplicato.com ([127.0.0.1]) by localhost (host52a.simplicato.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 58699-02; Fri, 15 Oct 2004 05:38:35 -0400 (EDT) Received: from [192.168.1.6] (c-24-15-57-63.client.comcast.net [24.15.57.63]) by host52a.simplicato.com (Postfix) with ESMTP id A99D811E0B2; Fri, 15 Oct 2004 05:38:35 -0400 (EDT) In-Reply-To: References: Mime-Version: 1.0 (Apple Message framework v619) Content-Type: text/plain; charset=US-ASCII; format=flowed Message-Id: Content-Transfer-Encoding: 7bit From: Caitlin Bestler Subject: Re: [Ips] New iSCSI key Date: Fri, 15 Oct 2004 04:38:30 -0500 To: Mike Ko X-Mailer: Apple Mail (2.619) X-Virus-Scanned: by amavisd-new at simplicato.com X-Spam-Score: 1.4 (+) X-Scan-Signature: 538aad3a3c4f01d8b6a6477ca4248793 Content-Transfer-Encoding: 7bit Cc: ips@ietf.org X-BeenThere: ips@ietf.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: IP Storage List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Sender: ips-bounces@ietf.org Errors-To: ips-bounces@ietf.org X-Spam-Score: 1.4 (+) X-Scan-Signature: f4c2cf0bccc868e4cc88dace71fb3f44 Content-Transfer-Encoding: 7bit On Oct 14, 2004, at 11:36 PM, Mike Ko wrote: > If we do decide on adding a new iSCSI key to limit the number of > "unexpected" or "unanticipated" PDUs, I offer the following text to > be considered for adding to section 6 of the iSER draft. > > Currently the key is tentatively called "MaxOutstandingUnexpectedPDUs". > Unless there are other suggestions, this name will be used. > > Use: LO (leading only), Declarative > Senders: Initiator and Target > Scope: SW (session-wide) > Irrelevant when: RDMAExtensions=No > MaxOutstandingUnexpectedPDUs= None> > Default is None > > Having "None" be the default makes sense in the immediate context: that is if your peer does not inform you of a limit there is none. However "None" should not be the default assumed by the user of a DataMover service. The IETF RDDP WG has decided that an SRQ user must not be forced to take per-connection overrun alerts on its own as the only available remedy to prevent an SRQ-drain denial-of-service attack. The mechanisms that would enable the hardware or Privileged Resource Manager to guard against these attacks require a finite number. That immediately leads to the question as to how this limit is communicated between the DataMover service and the user of that service. Possibilities include: - DataMover publishes a value, user lives with it. - User publishes a value, DataMover lives with it. - User requests a value, DataMover indicates what it can/is willing to provide. Of course this is a conceptual interface only. There is no obligation to implement the DataMover as a distinct layer that is separate from the iSCSI code. My other question is whether there is already language that adequately defines when an unexpected PDU is no longer "outstanding". I have been assuming that when any later numbered PDU has been acked that it is no longer outstanding. Is there already language that covers that adequately? _______________________________________________ Ips mailing list Ips@ietf.org https://www1.ietf.org/mailman/listinfo/ips From ips-bounces@ietf.org Fri Oct 15 06:00:04 2004 Received: from ietf-mx.ietf.org (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id GAA12346 for ; Fri, 15 Oct 2004 06:00:04 -0400 (EDT) Received: from megatron.ietf.org ([132.151.6.71]) by ietf-mx.ietf.org with esmtp (Exim 4.33) id 1CIP3r-0002kw-BQ for ips-web-archive@ietf.org; Fri, 15 Oct 2004 06:11:50 -0400 Received: from localhost.localdomain ([127.0.0.1] helo=megatron.ietf.org) by megatron.ietf.org with esmtp (Exim 4.32) id 1CIOl6-0001kT-Ba; Fri, 15 Oct 2004 05:52:24 -0400 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by megatron.ietf.org with esmtp (Exim 4.32) id 1CIOkQ-0001aq-RN for ips@megatron.ietf.org; Fri, 15 Oct 2004 05:51:43 -0400 Received: from ietf-mx.ietf.org (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id FAA11516 for ; Fri, 15 Oct 2004 05:51:40 -0400 (EDT) Received: from system60.simplicato.com ([207.99.47.60] helo=host52a.simplicato.com) by ietf-mx.ietf.org with esmtp (Exim 4.33) id 1CIOvm-0002YZ-35 for ips@ietf.org; Fri, 15 Oct 2004 06:03:26 -0400 Received: from localhost (localhost.simplicato.com [127.0.0.1]) by host52a.simplicato.com (Postfix) with ESMTP id E8B4211E097; Fri, 15 Oct 2004 05:51:39 -0400 (EDT) Received: from host52a.simplicato.com ([127.0.0.1]) by localhost (host52a.simplicato.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 60815-06; Fri, 15 Oct 2004 05:51:39 -0400 (EDT) Received: from [192.168.1.6] (c-24-15-57-63.client.comcast.net [24.15.57.63]) by host52a.simplicato.com (Postfix) with ESMTP id E96BA11E0A0; Fri, 15 Oct 2004 05:51:38 -0400 (EDT) In-Reply-To: References: Mime-Version: 1.0 (Apple Message framework v619) Content-Type: text/plain; charset=US-ASCII; format=flowed Message-Id: Content-Transfer-Encoding: 7bit From: Caitlin Bestler Subject: Re: [Ips] New iSCSI key Date: Fri, 15 Oct 2004 04:51:35 -0500 To: Andre Hedrick X-Mailer: Apple Mail (2.619) X-Virus-Scanned: by amavisd-new at simplicato.com X-Spam-Score: 0.1 (/) X-Scan-Signature: 7aafa0432175920a4b3e118e16c5cb64 Content-Transfer-Encoding: 7bit Cc: ips@ietf.org, Mike Ko X-BeenThere: ips@ietf.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: IP Storage List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Sender: ips-bounces@ietf.org Errors-To: ips-bounces@ietf.org X-Spam-Score: 0.1 (/) X-Scan-Signature: 0a7aa2e6e558383d84476dc338324fab Content-Transfer-Encoding: 7bit On Oct 15, 2004, at 1:07 AM, Andre Hedrick wrote: > > Mike, > > Not to be cheeky, but would it be possible for everyone to support and > send all the current keys first before adding another? > > If you are not implementing iSER this key is irrelevant. If you have implemented iSER along the lines that the drafters envisioned then you simply use the "None" default. Otherwise you are merely publishing the allowance you already calculated in determining how to configure and populate your SRQ. A given implementation probably already has a maximum number of outstanding unanticipated PDUs. For one thing it should have understood that the receiving iSCSI peer was already at liberty to discard most of them, and likely would when the number got large. In any event, if you know that number and the allowance published by your peer is greater than you do not have to do *anything*. Otherwise you simply need to keep track of the number of unanticipated buffers you have outstanding, and refrain from sending more. Since that is already done for anticipated buffers it would not seem to be that difficult to implement. Keep in mind that your peer SHOULD provide enough extra credits that a typical implementation will not be required to do any actual counting. It might make sense to create an official "safe harbor". Any implementation that knows it would never generate more than X outstanding unanticipated buffers will NEVER have to implement any flow control logic for unanticipated buffers. An implementation is also at liberty to simply ignore the limit, but it does so at the risk that sending too many unanticipated messages will result in the connection being broken -- which is EXACTLY what the state of iSER would be if this key is not added. So adding this key does not have to represent more complex logic for anyone, but it does allow implementations to have better control over avoiding overruns caused by unanticipated messages when sent over transports that lack TCP's bottleneck "feature". _______________________________________________ Ips mailing list Ips@ietf.org https://www1.ietf.org/mailman/listinfo/ips From ips-bounces@ietf.org Fri Oct 15 07:14:03 2004 Received: from ietf-mx.ietf.org (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id HAA17917 for ; Fri, 15 Oct 2004 07:14:03 -0400 (EDT) Received: from megatron.ietf.org ([132.151.6.71]) by ietf-mx.ietf.org with esmtp (Exim 4.33) id 1CIQDW-0004M9-H3 for ips-web-archive@ietf.org; Fri, 15 Oct 2004 07:25:50 -0400 Received: from localhost.localdomain ([127.0.0.1] helo=megatron.ietf.org) by megatron.ietf.org with esmtp (Exim 4.32) id 1CIQ0J-0000Jm-Kg; Fri, 15 Oct 2004 07:12:11 -0400 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by megatron.ietf.org with esmtp (Exim 4.32) id 1CIPzf-0000BT-0y for ips@megatron.ietf.org; Fri, 15 Oct 2004 07:11:31 -0400 Received: from ietf-mx.ietf.org (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id HAA17750 for ; Fri, 15 Oct 2004 07:11:27 -0400 (EDT) Received: from system60.simplicato.com ([207.99.47.60] helo=host52a.simplicato.com) by ietf-mx.ietf.org with esmtp (Exim 4.33) id 1CIQB1-0004Ig-4w for ips@ietf.org; Fri, 15 Oct 2004 07:23:15 -0400 Received: from localhost (localhost.simplicato.com [127.0.0.1]) by host52a.simplicato.com (Postfix) with ESMTP id 40D6311E0DF for ; Fri, 15 Oct 2004 07:11:28 -0400 (EDT) Received: from host52a.simplicato.com ([127.0.0.1]) by localhost (host52a.simplicato.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 74717-05 for ; Fri, 15 Oct 2004 07:11:27 -0400 (EDT) Received: from [192.168.1.6] (c-24-15-57-63.client.comcast.net [24.15.57.63]) by host52a.simplicato.com (Postfix) with ESMTP id 72DD511E0DD for ; Fri, 15 Oct 2004 07:11:27 -0400 (EDT) Mime-Version: 1.0 (Apple Message framework v619) In-Reply-To: References: Content-Type: text/plain; charset=US-ASCII; format=flowed Message-Id: Content-Transfer-Encoding: 7bit From: Caitlin Bestler Subject: Re: [Ips] New iSCSI key Date: Fri, 15 Oct 2004 06:11:23 -0500 To: ips@ietf.org X-Mailer: Apple Mail (2.619) X-Virus-Scanned: by amavisd-new at simplicato.com X-Spam-Score: 1.4 (+) X-Scan-Signature: 8abaac9e10c826e8252866cbe6766464 Content-Transfer-Encoding: 7bit X-BeenThere: ips@ietf.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: IP Storage List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Sender: ips-bounces@ietf.org Errors-To: ips-bounces@ietf.org X-Spam-Score: 1.4 (+) X-Scan-Signature: 39bd8f8cbb76cae18b7e23f7cf6b2b9f Content-Transfer-Encoding: 7bit On Oct 15, 2004, at 4:38 AM, Caitlin Bestler wrote: > > > That immediately leads to the question as to how this limit is > communicated between the DataMover service and the user > of that service. Possibilities include: > - DataMover publishes a value, user lives with it. > - User publishes a value, DataMover lives with it. > - User requests a value, DataMover indicates what > it can/is willing to provide. > > Of course this is a conceptual interface only. There is > no obligation to implement the DataMover as a distinct > layer that is separate from the iSCSI code. > > Also, a DataMover may offer a blocking or balking interface for submitting iSCSI PDUs. If that is the nature of the local interface, then any publishing of the limit is optional, and the semantics are that *any* iSCSI PDU that the Datamover considers premature may result in blocking of the caller or in the put being temporarily disallowed (i.e., the caller gets a "try again later" error). That is essentially the semantics offered by a TCP transport. It is hard to anticipate when you will hit the transport layer buffering limits. When you do you block, or you are told to try again later. _______________________________________________ Ips mailing list Ips@ietf.org https://www1.ietf.org/mailman/listinfo/ips From ips-bounces@ietf.org Fri Oct 15 12:47:44 2004 Received: from ietf-mx.ietf.org (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id MAA19404 for ; Fri, 15 Oct 2004 12:47:44 -0400 (EDT) Received: from megatron.ietf.org ([132.151.6.71]) by ietf-mx.ietf.org with esmtp (Exim 4.33) id 1CIVQT-0003yI-OV for ips-web-archive@ietf.org; Fri, 15 Oct 2004 12:59:35 -0400 Received: from localhost.localdomain ([127.0.0.1] helo=megatron.ietf.org) by megatron.ietf.org with esmtp (Exim 4.32) id 1CIV7o-00049a-Cu; Fri, 15 Oct 2004 12:40:16 -0400 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by megatron.ietf.org with esmtp (Exim 4.32) id 1CIV3L-0003Fe-Qd for ips@megatron.ietf.org; Fri, 15 Oct 2004 12:35:40 -0400 Received: from ietf-mx.ietf.org (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id MAA18084 for ; Fri, 15 Oct 2004 12:35:36 -0400 (EDT) Received: from e2.ny.us.ibm.com ([32.97.182.102]) by ietf-mx.ietf.org with esmtp (Exim 4.33) id 1CIVEk-0003hl-Db for ips@ietf.org; Fri, 15 Oct 2004 12:47:26 -0400 Received: from northrelay02.pok.ibm.com (northrelay02.pok.ibm.com [9.56.224.150]) by e2.ny.us.ibm.com (8.12.10/8.12.9) with ESMTP id i9FGZ1BF015936; Fri, 15 Oct 2004 12:35:01 -0400 Received: from d01ml604.pok.ibm.com (d01av04.pok.ibm.com [9.56.224.64]) by northrelay02.pok.ibm.com (8.12.10/NCO/VER6.6) with ESMTP id i9FGaHiM055312; Fri, 15 Oct 2004 12:36:17 -0400 In-Reply-To: Importance: Normal MIME-Version: 1.0 To: Caitlin Bestler Subject: Re: [Ips] New iSCSI key X-Mailer: Lotus Notes Release 6.5.2 June 01, 2004 Message-ID: From: Mike Ko Date: Fri, 15 Oct 2004 09:34:58 -0700 X-MIMETrack: Serialize by Router on D01ML604/01/M/IBM(Release 6.51HF643 | October 13, 2004) at 10/15/2004 12:35:00, Serialize complete at 10/15/2004 12:35:00 Content-Type: text/plain; charset="US-ASCII" X-Spam-Score: 1.3 (+) X-Scan-Signature: 3002fc2e661cd7f114cb6bae92fe88f1 Cc: ips@ietf.org X-BeenThere: ips@ietf.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: IP Storage List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Sender: ips-bounces@ietf.org Errors-To: ips-bounces@ietf.org X-Spam-Score: 1.3 (+) X-Scan-Signature: f66b12316365a3fe519e75911daf28a8 Caitlin, My comments are embedded. Mike To: Mike Ko cc: Subject: Re: [Ips] New iSCSI key On Oct 14, 2004, at 11:36 PM, Mike Ko wrote: > If we do decide on adding a new iSCSI key to limit the number of > "unexpected" or "unanticipated" PDUs, I offer the following text to > be considered for adding to section 6 of the iSER draft. > > Currently the key is tentatively called "MaxOutstandingUnexpectedPDUs". > Unless there are other suggestions, this name will be used. > > Use: LO (leading only), Declarative > Senders: Initiator and Target > Scope: SW (session-wide) > Irrelevant when: RDMAExtensions=No > MaxOutstandingUnexpectedPDUs= None> > Default is None > > Having "None" be the default makes sense in the immediate context: that is if your peer does not inform you of a limit there is none. However "None" should not be the default assumed by the user of a DataMover service. The IETF RDDP WG has decided that an SRQ user must not be forced to take per-connection overrun alerts on its own as the only available remedy to prevent an SRQ-drain denial-of-service attack. The mechanisms that would enable the hardware or Privileged Resource Manager to guard against these attacks require a finite number. [mk[ My first thought was that the default value should apply for most implementations so that the key need not be declared during the login process. For implementations that support graceful handling or where the iSER/iSCSI layer can replenish the buffers as fast as the packet arrival rate, this is indeed the case. For other implementations, the value to use could be highly dependent on the implementation, making a generally acceptable default difficult to define. But I could be wrong, and I welcome any suggestion as to what this default value should be. ]mk] That immediately leads to the question as to how this limit is communicated between the DataMover service and the user of that service. Possibilities include: - DataMover publishes a value, user lives with it. - User publishes a value, DataMover lives with it. - User requests a value, DataMover indicates what it can/is willing to provide. Of course this is a conceptual interface only. There is no obligation to implement the DataMover as a distinct layer that is separate from the iSCSI code. [mk[ Will respond in your follow-up note. ]mk] My other question is whether there is already language that adequately defines when an unexpected PDU is no longer "outstanding". I have been assuming that when any later numbered PDU has been acked that it is no longer outstanding. Is there already language that covers that adequately? [mk[ There were discussions on the IPS forum that mention the use of additional explicit credit scheme (Alex Nezhinsky) and uncertainty as to when an "unexpected" PDU is no longer outstanding, or "retired" (Pat Thaler). The text was written to address these concerns. I don't have any problem adding references to appropriate sections in RFC3720 that describe when an unexpected PDU is no longer outstanding. ]mk] _______________________________________________ Ips mailing list Ips@ietf.org https://www1.ietf.org/mailman/listinfo/ips From ips-bounces@ietf.org Fri Oct 15 12:49:04 2004 Received: from ietf-mx.ietf.org (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id MAA19698 for ; Fri, 15 Oct 2004 12:49:04 -0400 (EDT) Received: from megatron.ietf.org ([132.151.6.71]) by ietf-mx.ietf.org with esmtp (Exim 4.33) id 1CIVRm-000432-GI for ips-web-archive@ietf.org; Fri, 15 Oct 2004 13:00:54 -0400 Received: from localhost.localdomain ([127.0.0.1] helo=megatron.ietf.org) by megatron.ietf.org with esmtp (Exim 4.32) id 1CIVDk-0005We-DJ; Fri, 15 Oct 2004 12:46:24 -0400 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by megatron.ietf.org with esmtp (Exim 4.32) id 1CIV72-00040s-Ft for ips@megatron.ietf.org; Fri, 15 Oct 2004 12:39:28 -0400 Received: from ietf-mx.ietf.org (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id MAA18308 for ; Fri, 15 Oct 2004 12:39:25 -0400 (EDT) Received: from e1.ny.us.ibm.com ([32.97.182.101]) by ietf-mx.ietf.org with esmtp (Exim 4.33) id 1CIVIQ-0003m8-FD for ips@ietf.org; Fri, 15 Oct 2004 12:51:15 -0400 Received: from northrelay02.pok.ibm.com (northrelay02.pok.ibm.com [9.56.224.150]) by e1.ny.us.ibm.com (8.12.10/NS PXFA) with ESMTP id i9FGcsrZ154964; Fri, 15 Oct 2004 12:38:54 -0400 Received: from d01ml604.pok.ibm.com (d01av02.pok.ibm.com [9.56.224.216]) by northrelay02.pok.ibm.com (8.12.10/NCO/VER6.6) with ESMTP id i9FGeAiM060460; Fri, 15 Oct 2004 12:40:10 -0400 In-Reply-To: Importance: Normal MIME-Version: 1.0 To: Caitlin Bestler Subject: Re: [Ips] New iSCSI key X-Mailer: Lotus Notes Release 6.5.2 June 01, 2004 Message-ID: From: Mike Ko Date: Fri, 15 Oct 2004 09:38:52 -0700 X-MIMETrack: Serialize by Router on D01ML604/01/M/IBM(Release 6.51HF643 | October 13, 2004) at 10/15/2004 12:38:54, Serialize complete at 10/15/2004 12:38:54 Content-Type: text/plain; charset="US-ASCII" X-Spam-Score: 1.3 (+) X-Scan-Signature: 4d87d2aa806f79fed918a62e834505ca Cc: ips@ietf.org X-BeenThere: ips@ietf.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: IP Storage List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Sender: ips-bounces@ietf.org Errors-To: ips-bounces@ietf.org X-Spam-Score: 1.3 (+) X-Scan-Signature: 5a9a1bd6c2d06a21d748b7d0070ddcb8 Caitlin, It seems to me how the iSER layer and the iSCSI layer exchange information on the value of MaxOutstandingUnexpectedPDU is implementation dependent and need not be specified in the iSER draft. Mike Sent by: ips-bounces@ietf.org To: ips@ietf.org cc: Subject: Re: [Ips] New iSCSI key On Oct 15, 2004, at 4:38 AM, Caitlin Bestler wrote: > > > That immediately leads to the question as to how this limit is > communicated between the DataMover service and the user > of that service. Possibilities include: > - DataMover publishes a value, user lives with it. > - User publishes a value, DataMover lives with it. > - User requests a value, DataMover indicates what > it can/is willing to provide. > > Of course this is a conceptual interface only. There is > no obligation to implement the DataMover as a distinct > layer that is separate from the iSCSI code. > > Also, a DataMover may offer a blocking or balking interface for submitting iSCSI PDUs. If that is the nature of the local interface, then any publishing of the limit is optional, and the semantics are that *any* iSCSI PDU that the Datamover considers premature may result in blocking of the caller or in the put being temporarily disallowed (i.e., the caller gets a "try again later" error). That is essentially the semantics offered by a TCP transport. It is hard to anticipate when you will hit the transport layer buffering limits. When you do you block, or you are told to try again later. _______________________________________________ Ips mailing list Ips@ietf.org https://www1.ietf.org/mailman/listinfo/ips _______________________________________________ Ips mailing list Ips@ietf.org https://www1.ietf.org/mailman/listinfo/ips From ips-bounces@ietf.org Fri Oct 15 13:06:40 2004 Received: from ietf-mx.ietf.org (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id NAA20890 for ; Fri, 15 Oct 2004 13:06:40 -0400 (EDT) Received: from megatron.ietf.org ([132.151.6.71]) by ietf-mx.ietf.org with esmtp (Exim 4.33) id 1CIVio-0004SA-HK for ips-web-archive@ietf.org; Fri, 15 Oct 2004 13:18:30 -0400 Received: from localhost.localdomain ([127.0.0.1] helo=megatron.ietf.org) by megatron.ietf.org with esmtp (Exim 4.32) id 1CIVVr-0002AU-IM; Fri, 15 Oct 2004 13:05:07 -0400 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by megatron.ietf.org with esmtp (Exim 4.32) id 1CIVTC-0001PI-5T for ips@megatron.ietf.org; Fri, 15 Oct 2004 13:02:23 -0400 Received: from ietf-mx.ietf.org (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id NAA20620 for ; Fri, 15 Oct 2004 13:02:18 -0400 (EDT) Received: from e2.ny.us.ibm.com ([32.97.182.102]) by ietf-mx.ietf.org with esmtp (Exim 4.33) id 1CIVea-0004LX-3K for ips@ietf.org; Fri, 15 Oct 2004 13:14:09 -0400 Received: from northrelay04.pok.ibm.com (northrelay04.pok.ibm.com [9.56.224.206]) by e2.ny.us.ibm.com (8.12.10/8.12.9) with ESMTP id i9FH1kBF442622 for ; Fri, 15 Oct 2004 13:01:46 -0400 Received: from d01ml604.pok.ibm.com (d01av02.pok.ibm.com [9.56.224.216]) by northrelay04.pok.ibm.com (8.12.10/NCO/VER6.6) with ESMTP id i9FH30R2044568 for ; Fri, 15 Oct 2004 13:03:00 -0400 In-Reply-To: Importance: Normal MIME-Version: 1.0 To: John Hufferd Subject: Re: [Ips] New iSCSI key X-Mailer: Lotus Notes Release 6.5.2 June 01, 2004 Message-ID: From: Mike Ko Date: Fri, 15 Oct 2004 10:01:43 -0700 X-MIMETrack: Serialize by Router on D01ML604/01/M/IBM(Release 6.51HF643 | October 13, 2004) at 10/15/2004 13:01:45, Serialize complete at 10/15/2004 13:01:45 Content-Type: text/plain; charset="US-ASCII" X-Spam-Score: 0.0 (/) X-Scan-Signature: 612a16ba5c5f570bfc42b3ac5606ac53 Cc: ips@ietf.org X-BeenThere: ips@ietf.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: IP Storage List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Sender: ips-bounces@ietf.org Errors-To: ips-bounces@ietf.org X-Spam-Score: 0.0 (/) X-Scan-Signature: 17e5edc4dfd335965c1d21372171c01c John, My comments are embedded. Mike To: Mike Ko cc: ips@ietf.org Subject: Re: [Ips] New iSCSI key Mike, First I think there is a Typo near the end of your note when you talk about NOP-Out PDU where you might have meant Nop-In. That is, where you say "...Otherwise, a NOP-out PDU is considered "unexpected",...." perhaps you should say Nop-In. [mk[ Good catch. I will fix that. ]mk] Second I think we need to discuss the unsolicited Data-Out PDU. Please expand on the concept of the use of Data-Out PDU for solicited Data in iSER. [mk[ SCSI Data-out PDU should not be used for solicited data even though it is supported in the iSER draft. Since this is not a typical usage, the burden is left to the iSER/iSCSI layer to keep track of the number of "unexpected" PDU that it can send. What specifically do you want to expand on this topic? ]mk] Third we need to discuss or expand on considering the Reject PDU an unsolicited PDU. Perhaps you meant Rejects for PDUs that are not for the Command itself. [mk[ I will replace the phrase "A Reject PDU is considered "unexpected"" with the following: For a task which has never been active before the Reject PDU is sent, the target should not send any further responses because the command itself is discarded (see section 6.3 of RFC3720). Therefore, this type of Reject PDU is expecetd. All other Reject PDUs are "unexpected". ]mk] . . John L. Hufferd Senior Technical Staff Member (STSM) IBM/System Group, San Jose CA Main Office: (408) 256-0403, Tie: 276-0403, eFax: (408) 904-4688 Alt Office: (408) 997-6136, Cell: (408) 499-9702 Internet Address: hufferd@us.ibm.com Mike Ko Sent by: ips-bounces@ietf.org 10/14/2004 09:36 PM To cc Subject [Ips] New iSCSI key If we do decide on adding a new iSCSI key to limit the number of "unexpected" or "unanticipated" PDUs, I offer the following text to be considered for adding to section 6 of the iSER draft. Currently the key is tentatively called "MaxOutstandingUnexpectedPDUs". Unless there are other suggestions, this name will be used. Use: LO (leading only), Declarative Senders: Initiator and Target Scope: SW (session-wide) Irrelevant when: RDMAExtensions=No MaxOutstandingUnexpectedPDUs= Default is None This key is used by the initiator and the target to declare the maximum number of outstanding "unexpected" control-type PDUs that it can receive. It is intended to allow the receiving side to determine the amount of buffer resoruces needed beyond the normal flow control mechanism available in iSCSI. An initiator or target that cannot function properly without any limit on MaxOutstandingUnexpectedPDU (which is the default) should choose a value such that it would not impose an unnecessary constraint on the iSCSI layer under normal circumstances. The proper value to use is implementation dependent. If the sending side fails to adhere to the declared limit as set by the receiving side, it is up to the receiving side to determine ways of handling the overrun, including dropping the connection. For control-type PDUs sent by an initiator to a target, where the immediate flag is not set, the command numbering mechanism regulates the maximum number of PDUs that the target can expect. The exceptions include the SNACK PDU, the SCSI Data-out PDU, and any PDU with the immediate flag set. 1. The SNACK PDU is not needed in iSER (see section 7.3.11). But if one is requested for transmission by the iSCSI layer at the initiator, it will be sent by the iSER layer as a control-type PDU. Since it is "unexpected", the number that can be sent by the initiator falls under the MaxOutstandingUnexpectedPDU declared by the target. The PDU is outstanding until the target responds with a control-type PDU with ExpCmdSN set to X + 1 where X is the CmdSN of the referenced command. 2. For unsolicited SCSI Data-out PDU, the target can use FirstBurstLength to determine the amount of buffering required. Solicited data are normally handled using the R2T mechanism (see section 7.3.4). However, if a SCSI Data-out PDU containing solicited data is requested for transmission by the iSCSI layer at the initiator, the iSER layer may send it as unsolciited data. Since it is "unexpected", the number that can be sent by the initiator falls under the MaxOutstandingUnexpectedPDU declared by the target. The PDU is outstanding until the target responds with a control-type PDU with ExpCmdSN set to X + 1 where X is the CmdSN of the referenced command. 3. A PDU with the immediate flag set is considered "unexpected", and the number that can be sent by the initiator falls under the MaxOutstandingUnexpectedPDU declared by the target. The PDU is outstanding until the target responds with a control-type PDU with ExpCmdSN set to X + 1 where X is the CmdSN of the PDU with the immediate flag set. Control-type PDUs sent by a target as a response to a request from an initiator are expected by the initiator and buffer resources required can be provisioned. The exceptions include the SCSI Data-in PDU, the R2T PDU, the Asynchronous Message PDU, the Reject PDU, and the NOP-in PDU. 1. A SCSI Data-in PDU is a data-type PDU and is transformed into RDMA Write operations (see section 7.3.5). 2. An R2T PDU is a data-type PDU and is transformed into RDMA Read operations (see section 7.3.6). 3. An Asynchronous Message PDU is considered "unexpected", and the number that can be sent by the target falls under the MaxOutstandingUnexpectedPDU declared by the initiator. The PDU is outstanding until the initiator sends a control-type PDU with ExpStatSN set to X + 1 where X is the StatSN of the Asychronous Message PDU. 4. A Reject PDU is considered "unexpected", and the number that can be sent by the target falls under the MaxOutstandingUnexpectedPDU declared by the initiator. The PDU is outstanding until the initiator sends a control-type PDU with ExpStatSN set to X + 1 where X is the StatSN of the Reject PDU. 5. A NOP-in PDU sent as a response by the target to a NOP-out PDU from the initiator is expected by the initiator. Otherwise, a NOP-out PDU is considered "unexpected", and the number that can be sent by the target falls under the MaxOutstandingUnexpectedPDU declared by the initiator. The PDU is outstanding until the initiator sends a control-type PDU with ExpStatSN set to X + 1 where X is the StatSN of the NOP-in PDU. Mike _______________________________________________ Ips mailing list Ips@ietf.org https://www1.ietf.org/mailman/listinfo/ips _______________________________________________ Ips mailing list Ips@ietf.org https://www1.ietf.org/mailman/listinfo/ips From ips-bounces@ietf.org Fri Oct 15 13:29:12 2004 Received: from ietf-mx.ietf.org (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id NAA22708 for ; Fri, 15 Oct 2004 13:29:12 -0400 (EDT) Received: from megatron.ietf.org ([132.151.6.71]) by ietf-mx.ietf.org with esmtp (Exim 4.33) id 1CIW4d-0004xd-4f for ips-web-archive@ietf.org; Fri, 15 Oct 2004 13:41:03 -0400 Received: from localhost.localdomain ([127.0.0.1] helo=megatron.ietf.org) by megatron.ietf.org with esmtp (Exim 4.32) id 1CIVqL-0005WF-Ix; Fri, 15 Oct 2004 13:26:17 -0400 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by megatron.ietf.org with esmtp (Exim 4.32) id 1CIVod-0005Bb-JR for ips@megatron.ietf.org; Fri, 15 Oct 2004 13:24:32 -0400 Received: from ietf-mx.ietf.org (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id NAA22254 for ; Fri, 15 Oct 2004 13:24:28 -0400 (EDT) Received: from e35.co.us.ibm.com ([32.97.110.133]) by ietf-mx.ietf.org with esmtp (Exim 4.33) id 1CIW01-0004qn-SA for ips@ietf.org; Fri, 15 Oct 2004 13:36:19 -0400 Received: from westrelay04.boulder.ibm.com (westrelay04.boulder.ibm.com [9.17.193.32]) by e35.co.us.ibm.com (8.12.10/8.12.9) with ESMTP id i9FHNwNX540856 for ; Fri, 15 Oct 2004 13:23:58 -0400 Received: from d03av04.boulder.ibm.com (d03av04.boulder.ibm.com [9.17.195.170]) by westrelay04.boulder.ibm.com (8.12.10/NCO/VER6.6) with ESMTP id i9FHNvOX152888 for ; Fri, 15 Oct 2004 11:23:58 -0600 Received: from d03av04.boulder.ibm.com (loopback [127.0.0.1]) by d03av04.boulder.ibm.com (8.12.11/8.12.11) with ESMTP id i9FHNvdX005131 for ; Fri, 15 Oct 2004 11:23:57 -0600 Received: from d03nm115.boulder.ibm.com (d03nm115.boulder.ibm.com [9.17.195.141]) by d03av04.boulder.ibm.com (8.12.11/8.12.11) with ESMTP id i9FHNvTV005125 for ; Fri, 15 Oct 2004 11:23:57 -0600 In-Reply-To: To: Mike Ko MIME-Version: 1.0 Subject: Re: [Ips] New iSCSI key X-Mailer: Lotus Notes Release 6.0.2CF1 June 9, 2003 From: John Hufferd Message-ID: Date: Fri, 15 Oct 2004 10:23:42 -0700 X-MIMETrack: Serialize by Router on D03NM115/03/M/IBM(Release 6.51HF338 | June 21, 2004) at 10/15/2004 11:23:57, Serialize complete at 10/15/2004 11:23:57 X-Spam-Score: 4.3 (++++) X-Scan-Signature: 1b82b4ba484bbe86cdae6d5f8b2d2ccb Cc: ips@ietf.org X-BeenThere: ips@ietf.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: IP Storage List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Content-Type: multipart/mixed; boundary="===============1354441740==" Sender: ips-bounces@ietf.org Errors-To: ips-bounces@ietf.org X-Spam-Score: 4.3 (++++) X-Scan-Signature: b4be0d55bab88df9d21005ced9551e26 This is a multipart message in MIME format. --===============1354441740== Content-Type: multipart/alternative; boundary="=_alternative 005F7A6588256F2E_=" This is a multipart message in MIME format. --=_alternative 005F7A6588256F2E_= Content-Type: text/plain; charset="US-ASCII" Mike, I think you covered my concerns, but perhaps we should actually use words in the draft similar to what you have used here for the unsolicited Data-Out PDUs. I just thought that the words you proposed for the draft were a little bit confusing, since we never have talk about how an iSCSI layer requests Data-Out PDUs, etc. I think your words in this note were more to the point. . . John L. Hufferd Senior Technical Staff Member (STSM) IBM/System Group, San Jose CA Main Office: (408) 256-0403, Tie: 276-0403, eFax: (408) 904-4688 Alt Office: (408) 997-6136, Cell: (408) 499-9702 Internet Address: hufferd@us.ibm.com Mike Ko/Almaden/IBM 10/15/2004 10:01 AM To John Hufferd/San Jose/IBM@IBMUS cc ips@ietf.org Subject Re: [Ips] New iSCSI key John, My comments are embedded. Mike To: Mike Ko cc: ips@ietf.org Subject: Re: [Ips] New iSCSI key Mike, First I think there is a Typo near the end of your note when you talk about NOP-Out PDU where you might have meant Nop-In. That is, where you say "...Otherwise, a NOP-out PDU is considered "unexpected",...." perhaps you should say Nop-In. [mk[ Good catch. I will fix that. ]mk] Second I think we need to discuss the unsolicited Data-Out PDU. Please expand on the concept of the use of Data-Out PDU for solicited Data in iSER. [mk[ SCSI Data-out PDU should not be used for solicited data even though it is supported in the iSER draft. Since this is not a typical usage, the burden is left to the iSER/iSCSI layer to keep track of the number of "unexpected" PDU that it can send. What specifically do you want to expand on this topic? ]mk] Third we need to discuss or expand on considering the Reject PDU an unsolicited PDU. Perhaps you meant Rejects for PDUs that are not for the Command itself. [mk[ I will replace the phrase "A Reject PDU is considered "unexpected"" with the following: For a task which has never been active before the Reject PDU is sent, the target should not send any further responses because the command itself is discarded (see section 6.3 of RFC3720). Therefore, this type of Reject PDU is expecetd. All other Reject PDUs are "unexpected". ]mk] . . John L. Hufferd Senior Technical Staff Member (STSM) IBM/System Group, San Jose CA Main Office: (408) 256-0403, Tie: 276-0403, eFax: (408) 904-4688 Alt Office: (408) 997-6136, Cell: (408) 499-9702 Internet Address: hufferd@us.ibm.com Mike Ko Sent by: ips-bounces@ietf.org 10/14/2004 09:36 PM To cc Subject [Ips] New iSCSI key If we do decide on adding a new iSCSI key to limit the number of "unexpected" or "unanticipated" PDUs, I offer the following text to be considered for adding to section 6 of the iSER draft. Currently the key is tentatively called "MaxOutstandingUnexpectedPDUs". Unless there are other suggestions, this name will be used. Use: LO (leading only), Declarative Senders: Initiator and Target Scope: SW (session-wide) Irrelevant when: RDMAExtensions=No MaxOutstandingUnexpectedPDUs= Default is None This key is used by the initiator and the target to declare the maximum number of outstanding "unexpected" control-type PDUs that it can receive. It is intended to allow the receiving side to determine the amount of buffer resoruces needed beyond the normal flow control mechanism available in iSCSI. An initiator or target that cannot function properly without any limit on MaxOutstandingUnexpectedPDU (which is the default) should choose a value such that it would not impose an unnecessary constraint on the iSCSI layer under normal circumstances. The proper value to use is implementation dependent. If the sending side fails to adhere to the declared limit as set by the receiving side, it is up to the receiving side to determine ways of handling the overrun, including dropping the connection. For control-type PDUs sent by an initiator to a target, where the immediate flag is not set, the command numbering mechanism regulates the maximum number of PDUs that the target can expect. The exceptions include the SNACK PDU, the SCSI Data-out PDU, and any PDU with the immediate flag set. 1. The SNACK PDU is not needed in iSER (see section 7.3.11). But if one is requested for transmission by the iSCSI layer at the initiator, it will be sent by the iSER layer as a control-type PDU. Since it is "unexpected", the number that can be sent by the initiator falls under the MaxOutstandingUnexpectedPDU declared by the target. The PDU is outstanding until the target responds with a control-type PDU with ExpCmdSN set to X + 1 where X is the CmdSN of the referenced command. 2. For unsolicited SCSI Data-out PDU, the target can use FirstBurstLength to determine the amount of buffering required. Solicited data are normally handled using the R2T mechanism (see section 7.3.4). However, if a SCSI Data-out PDU containing solicited data is requested for transmission by the iSCSI layer at the initiator, the iSER layer may send it as unsolciited data. Since it is "unexpected", the number that can be sent by the initiator falls under the MaxOutstandingUnexpectedPDU declared by the target. The PDU is outstanding until the target responds with a control-type PDU with ExpCmdSN set to X + 1 where X is the CmdSN of the referenced command. 3. A PDU with the immediate flag set is considered "unexpected", and the number that can be sent by the initiator falls under the MaxOutstandingUnexpectedPDU declared by the target. The PDU is outstanding until the target responds with a control-type PDU with ExpCmdSN set to X + 1 where X is the CmdSN of the PDU with the immediate flag set. Control-type PDUs sent by a target as a response to a request from an initiator are expected by the initiator and buffer resources required can be provisioned. The exceptions include the SCSI Data-in PDU, the R2T PDU, the Asynchronous Message PDU, the Reject PDU, and the NOP-in PDU. 1. A SCSI Data-in PDU is a data-type PDU and is transformed into RDMA Write operations (see section 7.3.5). 2. An R2T PDU is a data-type PDU and is transformed into RDMA Read operations (see section 7.3.6). 3. An Asynchronous Message PDU is considered "unexpected", and the number that can be sent by the target falls under the MaxOutstandingUnexpectedPDU declared by the initiator. The PDU is outstanding until the initiator sends a control-type PDU with ExpStatSN set to X + 1 where X is the StatSN of the Asychronous Message PDU. 4. A Reject PDU is considered "unexpected", and the number that can be sent by the target falls under the MaxOutstandingUnexpectedPDU declared by the initiator. The PDU is outstanding until the initiator sends a control-type PDU with ExpStatSN set to X + 1 where X is the StatSN of the Reject PDU. 5. A NOP-in PDU sent as a response by the target to a NOP-out PDU from the initiator is expected by the initiator. Otherwise, a NOP-out PDU is considered "unexpected", and the number that can be sent by the target falls under the MaxOutstandingUnexpectedPDU declared by the initiator. The PDU is outstanding until the initiator sends a control-type PDU with ExpStatSN set to X + 1 where X is the StatSN of the NOP-in PDU. Mike _______________________________________________ Ips mailing list Ips@ietf.org https://www1.ietf.org/mailman/listinfo/ips --=_alternative 005F7A6588256F2E_= Content-Type: text/html; charset="US-ASCII"
Mike,
I think you covered my concerns, but perhaps we should actually use words in the draft similar to what you have used here for the unsolicited Data-Out PDUs.  I just thought that the words you proposed for the draft were a little bit confusing, since we never have talk about how an iSCSI layer requests Data-Out PDUs, etc.  I think your words in this note were more to the point.
.
.
John L. Hufferd
Senior Technical Staff Member (STSM)
IBM/System Group, San Jose CA
Main Office: (408) 256-0403, Tie: 276-0403, eFax: (408) 904-4688
Alt Office: (408) 997-6136, Cell: (408) 499-9702
Internet Address: hufferd@us.ibm.com


Mike Ko/Almaden/IBM

10/15/2004 10:01 AM

To
John Hufferd/San Jose/IBM@IBMUS
cc
ips@ietf.org
Subject
Re: [Ips] New iSCSI keyLink




John,

My comments are embedded.

Mike

To:        Mike Ko <mako@almaden.ibm.com>
cc:        ips@ietf.org
Subject:        Re: [Ips] New iSCSI key




Mike,
First I think there is a Typo near the end of your note when you talk about NOP-Out PDU where you might have meant Nop-In.  That is, where you say "...Otherwise, a NOP-out PDU is
considered "unexpected",...." perhaps you should say Nop-In.

[mk[
Good catch.  I will fix that.
]mk]

Second I think we need to discuss the unsolicited Data-Out PDU.  Please expand on the concept of the use of Data-Out PDU for solicited Data in iSER.

[mk[
SCSI Data-out PDU should not be used for solicited data even though it is supported in the iSER draft.  Since this is not a typical usage, the burden is left to the iSER/iSCSI layer to keep track of the number of "unexpected" PDU that it can send.  What specifically do you want to expand on this topic?
]mk]

Third we need to discuss or expand on considering the Reject PDU an unsolicited PDU.   Perhaps you meant Rejects for PDUs that are not for the Command itself.

[mk[
I will replace the phrase "A Reject PDU is considered "unexpected"" with the following:
For a task which has never been active before the Reject PDU is sent, the target should not send any further responses because the command itself is discarded (see section 6.3 of RFC3720).  Therefore, this type of Reject PDU is expecetd.  All other Reject PDUs are "unexpected".
]mk]
.
.
John L. Hufferd
Senior Technical Staff Member (STSM)
IBM/System Group, San Jose CA
Main Office: (408) 256-0403, Tie: 276-0403, eFax: (408) 904-4688
Alt Office: (408) 997-6136, Cell: (408) 499-9702
Internet Address: hufferd@us.ibm.com


Mike Ko <mako@almaden.ibm.com>  
Sent by: ips-bounces@ietf.org

10/14/2004 09:36 PM
To
<ips@ietf.org>
cc
Subject
[Ips] New iSCSI key






If we do decide on adding a new iSCSI key to limit the number of
"unexpected"
or "unanticipated" PDUs, I offer the following text to be considered for
adding
to section 6 of the iSER draft.

Currently the key is tentatively called "MaxOutstandingUnexpectedPDUs".
Unless
there are other suggestions, this name will be used.

Use: LO (leading only), Declarative
Senders: Initiator and Target
Scope: SW (session-wide)
Irrelevant when: RDMAExtensions=No
MaxOutstandingUnexpectedPDUs=<numerical-value-from-0-to-(2**64-1) | None>
Default is None

This key is used by the initiator and the target to declare the maximum
number
of outstanding "unexpected" control-type PDUs that it can receive.  It is
intended to allow the receiving side to determine the amount of buffer
resoruces needed beyond the normal flow control mechanism available in
iSCSI.
An initiator or target that cannot function properly without any limit on
MaxOutstandingUnexpectedPDU (which is the default) should choose a value
such
that it would not impose an unnecessary constraint on the iSCSI layer
under
normal circumstances.  The proper value to use is implementation
dependent.  If
the sending side fails to adhere to the declared limit as set by the
receiving
side, it is up to the receiving side to determine ways of handling the
overrun,
including dropping the connection.

For control-type PDUs sent by an initiator to a target, where the
immediate
flag is not set, the command numbering mechanism regulates the maximum
number
of PDUs that the target can expect.  The exceptions include the SNACK PDU,
the
SCSI Data-out PDU, and any PDU with the immediate flag set.

1. The SNACK PDU is not needed in iSER (see section 7.3.11).  But if one
is
requested for transmission by the iSCSI layer at the initiator, it will be
sent
by the iSER layer as a control-type PDU.  Since it is "unexpected", the
number
that can be sent by the initiator falls under the
MaxOutstandingUnexpectedPDU
declared by the target.  The PDU is outstanding until the target responds
with
a control-type PDU with ExpCmdSN set to X + 1 where X is the CmdSN of the
referenced command.

2. For unsolicited SCSI Data-out PDU, the target can use FirstBurstLength
to
determine the amount of buffering required.  Solicited data are normally
handled using the R2T mechanism (see section 7.3.4).  However, if a SCSI
Data-out PDU containing solicited data is requested for transmission by
the
iSCSI layer at the initiator, the iSER layer may send it as unsolciited
data.
Since it is "unexpected", the number that can be sent by the initiator
falls
under the MaxOutstandingUnexpectedPDU declared by the target.  The PDU is
outstanding until the target responds with a control-type PDU with
ExpCmdSN set
to X + 1 where X is the CmdSN of the referenced command.

3. A PDU with the immediate flag set is considered "unexpected", and the
number
that can be sent by the initiator falls under the
MaxOutstandingUnexpectedPDU
declared by the target.  The PDU is outstanding until the target responds
with
a control-type PDU with ExpCmdSN set to X + 1 where X is the CmdSN of the
PDU
with the immediate flag set.

Control-type PDUs sent by a target as a response to a request from an
initiator
are expected by the initiator and buffer resources required can be
provisioned.
The exceptions include the SCSI Data-in PDU, the R2T PDU, the Asynchronous
Message PDU, the Reject PDU, and the NOP-in PDU.

1. A SCSI Data-in PDU is a data-type PDU and is transformed into RDMA
Write
operations (see section 7.3.5).

2. An R2T PDU is a data-type PDU and is transformed into RDMA Read
operations
(see section 7.3.6).

3. An Asynchronous Message PDU is considered "unexpected", and the number
that
can be sent by the target falls under the MaxOutstandingUnexpectedPDU
declared
by the initiator.  The PDU is outstanding until the initiator sends a
control-type PDU with ExpStatSN set to X + 1 where X is the StatSN of the
Asychronous Message PDU.

4. A Reject PDU is considered "unexpected", and the number that can be
sent by
the target falls under the MaxOutstandingUnexpectedPDU declared by the
initiator.  The PDU is outstanding until the initiator sends a
control-type PDU
with ExpStatSN set to X + 1 where X is the StatSN of the Reject PDU.

5. A NOP-in PDU sent as a response by the target to a NOP-out PDU from the
initiator is expected by the initiator.  Otherwise, a NOP-out PDU is
considered
"unexpected", and the number that can be sent by the target falls under
the
MaxOutstandingUnexpectedPDU declared by the initiator.  The PDU is
outstanding
until the initiator sends a control-type PDU with ExpStatSN set to X + 1
where
X is the StatSN of the NOP-in PDU.

Mike

_______________________________________________
Ips mailing list
Ips@ietf.org
https://www1.ietf.org/mailman/listinfo/ips
 


--=_alternative 005F7A6588256F2E_=-- --===============1354441740== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline Content-Transfer-Encoding: 7bit _______________________________________________ Ips mailing list Ips@ietf.org https://www1.ietf.org/mailman/listinfo/ips --===============1354441740==-- From ips-bounces@ietf.org Fri Oct 15 14:11:49 2004 Received: from ietf-mx.ietf.org (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id OAA25708 for ; Fri, 15 Oct 2004 14:11:49 -0400 (EDT) Received: from megatron.ietf.org ([132.151.6.71]) by ietf-mx.ietf.org with esmtp (Exim 4.33) id 1CIWjq-0005rQ-Re for ips-web-archive@ietf.org; Fri, 15 Oct 2004 14:23:39 -0400 Received: from localhost.localdomain ([127.0.0.1] helo=megatron.ietf.org) by megatron.ietf.org with esmtp (Exim 4.32) id 1CIWT8-0004B9-ML; Fri, 15 Oct 2004 14:06:22 -0400 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by megatron.ietf.org with esmtp (Exim 4.32) id 1CIWLT-0002Rq-UF for ips@megatron.ietf.org; Fri, 15 Oct 2004 13:58:29 -0400 Received: from ietf-mx.ietf.org (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id NAA24797 for ; Fri, 15 Oct 2004 13:58:26 -0400 (EDT) Received: from e1.ny.us.ibm.com ([32.97.182.101]) by ietf-mx.ietf.org with esmtp (Exim 4.33) id 1CIWWt-0005aK-1W for ips@ietf.org; Fri, 15 Oct 2004 14:10:15 -0400 Received: from northrelay04.pok.ibm.com (northrelay04.pok.ibm.com [9.56.224.206]) by e1.ny.us.ibm.com (8.12.10/NS PXFA) with ESMTP id i9FHvorZ456816 for ; Fri, 15 Oct 2004 13:57:50 -0400 Received: from d01ml604.pok.ibm.com (d01av02.pok.ibm.com [9.56.224.216]) by northrelay04.pok.ibm.com (8.12.10/NCO/VER6.6) with ESMTP id i9FHx4R2094124 for ; Fri, 15 Oct 2004 13:59:04 -0400 In-Reply-To: Importance: Normal MIME-Version: 1.0 To: John Hufferd Subject: Re: [Ips] New iSCSI key X-Mailer: Lotus Notes Release 6.5.2 June 01, 2004 Message-ID: From: Mike Ko Date: Fri, 15 Oct 2004 10:57:46 -0700 X-MIMETrack: Serialize by Router on D01ML604/01/M/IBM(Release 6.51HF643 | October 13, 2004) at 10/15/2004 13:57:48, Serialize complete at 10/15/2004 13:57:48 Content-Type: text/plain; charset="US-ASCII" X-Spam-Score: 0.0 (/) X-Scan-Signature: 223e3c753032a50d5dc4443c921c3fcd Cc: ips@ietf.org X-BeenThere: ips@ietf.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: IP Storage List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Sender: ips-bounces@ietf.org Errors-To: ips-bounces@ietf.org X-Spam-Score: 0.0 (/) X-Scan-Signature: 71f780ffdd80c541d3e75aa5f2710d3d John, Would the following replacement text for the sentence "Solicited data are normally handled ..." better? "The R2T mechanism is normally used to handle solicited data and the SCSI Data-out PDU should not be used for this purpose (see section 7.3.4)." Mike From: John Hufferd on 10/15/2004 10:22 AM To: Mike Ko/Almaden/IBM cc: ips@ietf.org Subject: Re: [Ips] New iSCSI key Mike, I think you covered my concerns, but perhaps we should actually use words in the draft similar to what you have used here for the unsolicited Data-Out PDUs. I just thought that the words you proposed for the draft were a little bit confusing, since we never have talk about how an iSCSI layer requests Data-Out PDUs, etc. I think your words in this note were more to the point. . . John L. Hufferd Senior Technical Staff Member (STSM) IBM/System Group, San Jose CA Main Office: (408) 256-0403, Tie: 276-0403, eFax: (408) 904-4688 Alt Office: (408) 997-6136, Cell: (408) 499-9702 Internet Address: hufferd@us.ibm.com Mike Ko/Almaden/IBM 10/15/2004 10:01 AM To John Hufferd/San Jose/IBM@IBMUS cc ips@ietf.org Subject Re: [Ips] New iSCSI key John, My comments are embedded. Mike To: Mike Ko cc: ips@ietf.org Subject: Re: [Ips] New iSCSI key Mike, First I think there is a Typo near the end of your note when you talk about NOP-Out PDU where you might have meant Nop-In. That is, where you say "...Otherwise, a NOP-out PDU is considered "unexpected",...." perhaps you should say Nop-In. [mk[ Good catch. I will fix that. ]mk] Second I think we need to discuss the unsolicited Data-Out PDU. Please expand on the concept of the use of Data-Out PDU for solicited Data in iSER. [mk[ SCSI Data-out PDU should not be used for solicited data even though it is supported in the iSER draft. Since this is not a typical usage, the burden is left to the iSER/iSCSI layer to keep track of the number of "unexpected" PDU that it can send. What specifically do you want to expand on this topic? ]mk] Third we need to discuss or expand on considering the Reject PDU an unsolicited PDU. Perhaps you meant Rejects for PDUs that are not for the Command itself. [mk[ I will replace the phrase "A Reject PDU is considered "unexpected"" with the following: For a task which has never been active before the Reject PDU is sent, the target should not send any further responses because the command itself is discarded (see section 6.3 of RFC3720). Therefore, this type of Reject PDU is expecetd. All other Reject PDUs are "unexpected". ]mk] . . John L. Hufferd Senior Technical Staff Member (STSM) IBM/System Group, San Jose CA Main Office: (408) 256-0403, Tie: 276-0403, eFax: (408) 904-4688 Alt Office: (408) 997-6136, Cell: (408) 499-9702 Internet Address: hufferd@us.ibm.com Mike Ko Sent by: ips-bounces@ietf.org 10/14/2004 09:36 PM To cc Subject [Ips] New iSCSI key If we do decide on adding a new iSCSI key to limit the number of "unexpected" or "unanticipated" PDUs, I offer the following text to be considered for adding to section 6 of the iSER draft. Currently the key is tentatively called "MaxOutstandingUnexpectedPDUs". Unless there are other suggestions, this name will be used. Use: LO (leading only), Declarative Senders: Initiator and Target Scope: SW (session-wide) Irrelevant when: RDMAExtensions=No MaxOutstandingUnexpectedPDUs= Default is None This key is used by the initiator and the target to declare the maximum number of outstanding "unexpected" control-type PDUs that it can receive. It is intended to allow the receiving side to determine the amount of buffer resoruces needed beyond the normal flow control mechanism available in iSCSI. An initiator or target that cannot function properly without any limit on MaxOutstandingUnexpectedPDU (which is the default) should choose a value such that it would not impose an unnecessary constraint on the iSCSI layer under normal circumstances. The proper value to use is implementation dependent. If the sending side fails to adhere to the declared limit as set by the receiving side, it is up to the receiving side to determine ways of handling the overrun, including dropping the connection. For control-type PDUs sent by an initiator to a target, where the immediate flag is not set, the command numbering mechanism regulates the maximum number of PDUs that the target can expect. The exceptions include the SNACK PDU, the SCSI Data-out PDU, and any PDU with the immediate flag set. 1. The SNACK PDU is not needed in iSER (see section 7.3.11). But if one is requested for transmission by the iSCSI layer at the initiator, it will be sent by the iSER layer as a control-type PDU. Since it is "unexpected", the number that can be sent by the initiator falls under the MaxOutstandingUnexpectedPDU declared by the target. The PDU is outstanding until the target responds with a control-type PDU with ExpCmdSN set to X + 1 where X is the CmdSN of the referenced command. 2. For unsolicited SCSI Data-out PDU, the target can use FirstBurstLength to determine the amount of buffering required. Solicited data are normally handled using the R2T mechanism (see section 7.3.4). However, if a SCSI Data-out PDU containing solicited data is requested for transmission by the iSCSI layer at the initiator, the iSER layer may send it as unsolciited data. Since it is "unexpected", the number that can be sent by the initiator falls under the MaxOutstandingUnexpectedPDU declared by the target. The PDU is outstanding until the target responds with a control-type PDU with ExpCmdSN set to X + 1 where X is the CmdSN of the referenced command. 3. A PDU with the immediate flag set is considered "unexpected", and the number that can be sent by the initiator falls under the MaxOutstandingUnexpectedPDU declared by the target. The PDU is outstanding until the target responds with a control-type PDU with ExpCmdSN set to X + 1 where X is the CmdSN of the PDU with the immediate flag set. Control-type PDUs sent by a target as a response to a request from an initiator are expected by the initiator and buffer resources required can be provisioned. The exceptions include the SCSI Data-in PDU, the R2T PDU, the Asynchronous Message PDU, the Reject PDU, and the NOP-in PDU. 1. A SCSI Data-in PDU is a data-type PDU and is transformed into RDMA Write operations (see section 7.3.5). 2. An R2T PDU is a data-type PDU and is transformed into RDMA Read operations (see section 7.3.6). 3. An Asynchronous Message PDU is considered "unexpected", and the number that can be sent by the target falls under the MaxOutstandingUnexpectedPDU declared by the initiator. The PDU is outstanding until the initiator sends a control-type PDU with ExpStatSN set to X + 1 where X is the StatSN of the Asychronous Message PDU. 4. A Reject PDU is considered "unexpected", and the number that can be sent by the target falls under the MaxOutstandingUnexpectedPDU declared by the initiator. The PDU is outstanding until the initiator sends a control-type PDU with ExpStatSN set to X + 1 where X is the StatSN of the Reject PDU. 5. A NOP-in PDU sent as a response by the target to a NOP-out PDU from the initiator is expected by the initiator. Otherwise, a NOP-out PDU is considered "unexpected", and the number that can be sent by the target falls under the MaxOutstandingUnexpectedPDU declared by the initiator. The PDU is outstanding until the initiator sends a control-type PDU with ExpStatSN set to X + 1 where X is the StatSN of the NOP-in PDU. Mike _______________________________________________ Ips mailing list Ips@ietf.org https://www1.ietf.org/mailman/listinfo/ips _______________________________________________ Ips mailing list Ips@ietf.org https://www1.ietf.org/mailman/listinfo/ips From ips-bounces@ietf.org Fri Oct 15 17:30:49 2004 Received: from ietf-mx.ietf.org (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id RAA24481 for ; Fri, 15 Oct 2004 17:30:49 -0400 (EDT) Received: from megatron.ietf.org ([132.151.6.71]) by ietf-mx.ietf.org with esmtp (Exim 4.33) id 1CIZqT-0005ys-QL for ips-web-archive@ietf.org; Fri, 15 Oct 2004 17:42:42 -0400 Received: from localhost.localdomain ([127.0.0.1] helo=megatron.ietf.org) by megatron.ietf.org with esmtp (Exim 4.32) id 1CIZTQ-000759-Nd; Fri, 15 Oct 2004 17:18:52 -0400 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by megatron.ietf.org with esmtp (Exim 4.32) id 1CIYcc-0006Ob-6j for ips@megatron.ietf.org; Fri, 15 Oct 2004 16:24:18 -0400 Received: from ietf-mx.ietf.org (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id QAA09520 for ; Fri, 15 Oct 2004 16:24:15 -0400 (EDT) Received: from e5.ny.us.ibm.com ([32.97.182.105]) by ietf-mx.ietf.org with esmtp (Exim 4.33) id 1CIYo0-0001Sw-7x for ips@ietf.org; Fri, 15 Oct 2004 16:36:06 -0400 Received: from northrelay02.pok.ibm.com (northrelay02.pok.ibm.com [9.56.224.150]) by e5.ny.us.ibm.com (8.12.10/8.12.9) with ESMTP id i9FKNdpA325500 for ; Fri, 15 Oct 2004 16:23:39 -0400 Received: from d01ml604.pok.ibm.com (d01av04.pok.ibm.com [9.56.224.64]) by northrelay02.pok.ibm.com (8.12.10/NCO/VER6.6) with ESMTP id i9FKOtiM059864 for ; Fri, 15 Oct 2004 16:24:55 -0400 In-Reply-To: Importance: Normal MIME-Version: 1.0 To: John Hufferd Subject: Re: [Ips] New iSCSI key X-Mailer: Lotus Notes Release 6.5.2 June 01, 2004 Message-ID: From: Mike Ko Date: Fri, 15 Oct 2004 13:23:36 -0700 X-MIMETrack: Serialize by Router on D01ML604/01/M/IBM(Release 6.51HF643 | October 13, 2004) at 10/15/2004 16:23:38, Serialize complete at 10/15/2004 16:23:38 Content-Type: text/plain; charset="US-ASCII" X-Spam-Score: 0.0 (/) X-Scan-Signature: c0aa019322dfce838bd8604f5a841b57 Cc: ips@ietf.org X-BeenThere: ips@ietf.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: IP Storage List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Sender: ips-bounces@ietf.org Errors-To: ips-bounces@ietf.org X-Spam-Score: 0.0 (/) X-Scan-Signature: 3be09dac38eaa50f02d21c7fcee1128c John, Here is the replacement paragraph for the SCSI Data-out PDU: 2. Unsolicited data in a SCSI write operation are sent by the iSCSI layer at the initiator using SCSI Data-out PDUs. The iSER layer at the target can use FirstBurstLength to determine the amount of buffering required. For solicited data, an R2T PDU sent by the iSCSI layer at the target is transformed by the iSER layer at the target into an RDMA Read operation (see section 7.3.6). Therefore the iSCSI layer at the initiator should not be issuing SCSI Data-out PDUs for solicited data. If a SCSI Data-out PDU containing solicited data is requested for transmission by the iSCSI layer at the initiator anyway, the iSER layer may send it as a control-type PDU (see section 7.3.4). Since it is "unexpected", the number that can be sent by the initiator falls under the MaxOutstandingUnexpectedPDU declared by the target. The PDU is outstanding until the target responds with a control-type PDU with ExpCmdSN set to X + 1 where X is the CmdSN of the referenced command. Mike Sent by: ips-bounces@ietf.org To: John Hufferd/San Jose/IBM@IBMUS cc: ips@ietf.org Subject: Re: [Ips] New iSCSI key John, Would the following replacement text for the sentence "Solicited data are normally handled ..." better? "The R2T mechanism is normally used to handle solicited data and the SCSI Data-out PDU should not be used for this purpose (see section 7.3.4)." Mike From: John Hufferd on 10/15/2004 10:22 AM To: Mike Ko/Almaden/IBM cc: ips@ietf.org Subject: Re: [Ips] New iSCSI key Mike, I think you covered my concerns, but perhaps we should actually use words in the draft similar to what you have used here for the unsolicited Data-Out PDUs. I just thought that the words you proposed for the draft were a little bit confusing, since we never have talk about how an iSCSI layer requests Data-Out PDUs, etc. I think your words in this note were more to the point. . . John L. Hufferd Senior Technical Staff Member (STSM) IBM/System Group, San Jose CA Main Office: (408) 256-0403, Tie: 276-0403, eFax: (408) 904-4688 Alt Office: (408) 997-6136, Cell: (408) 499-9702 Internet Address: hufferd@us.ibm.com Mike Ko/Almaden/IBM 10/15/2004 10:01 AM To John Hufferd/San Jose/IBM@IBMUS cc ips@ietf.org Subject Re: [Ips] New iSCSI key John, My comments are embedded. Mike To: Mike Ko cc: ips@ietf.org Subject: Re: [Ips] New iSCSI key Mike, First I think there is a Typo near the end of your note when you talk about NOP-Out PDU where you might have meant Nop-In. That is, where you say "...Otherwise, a NOP-out PDU is considered "unexpected",...." perhaps you should say Nop-In. [mk[ Good catch. I will fix that. ]mk] Second I think we need to discuss the unsolicited Data-Out PDU. Please expand on the concept of the use of Data-Out PDU for solicited Data in iSER. [mk[ SCSI Data-out PDU should not be used for solicited data even though it is supported in the iSER draft. Since this is not a typical usage, the burden is left to the iSER/iSCSI layer to keep track of the number of "unexpected" PDU that it can send. What specifically do you want to expand on this topic? ]mk] Third we need to discuss or expand on considering the Reject PDU an unsolicited PDU. Perhaps you meant Rejects for PDUs that are not for the Command itself. [mk[ I will replace the phrase "A Reject PDU is considered "unexpected"" with the following: For a task which has never been active before the Reject PDU is sent, the target should not send any further responses because the command itself is discarded (see section 6.3 of RFC3720). Therefore, this type of Reject PDU is expecetd. All other Reject PDUs are "unexpected". ]mk] . . John L. Hufferd Senior Technical Staff Member (STSM) IBM/System Group, San Jose CA Main Office: (408) 256-0403, Tie: 276-0403, eFax: (408) 904-4688 Alt Office: (408) 997-6136, Cell: (408) 499-9702 Internet Address: hufferd@us.ibm.com Mike Ko Sent by: ips-bounces@ietf.org 10/14/2004 09:36 PM To cc Subject [Ips] New iSCSI key If we do decide on adding a new iSCSI key to limit the number of "unexpected" or "unanticipated" PDUs, I offer the following text to be considered for adding to section 6 of the iSER draft. Currently the key is tentatively called "MaxOutstandingUnexpectedPDUs". Unless there are other suggestions, this name will be used. Use: LO (leading only), Declarative Senders: Initiator and Target Scope: SW (session-wide) Irrelevant when: RDMAExtensions=No MaxOutstandingUnexpectedPDUs= Default is None This key is used by the initiator and the target to declare the maximum number of outstanding "unexpected" control-type PDUs that it can receive. It is intended to allow the receiving side to determine the amount of buffer resoruces needed beyond the normal flow control mechanism available in iSCSI. An initiator or target that cannot function properly without any limit on MaxOutstandingUnexpectedPDU (which is the default) should choose a value such that it would not impose an unnecessary constraint on the iSCSI layer under normal circumstances. The proper value to use is implementation dependent. If the sending side fails to adhere to the declared limit as set by the receiving side, it is up to the receiving side to determine ways of handling the overrun, including dropping the connection. For control-type PDUs sent by an initiator to a target, where the immediate flag is not set, the command numbering mechanism regulates the maximum number of PDUs that the target can expect. The exceptions include the SNACK PDU, the SCSI Data-out PDU, and any PDU with the immediate flag set. 1. The SNACK PDU is not needed in iSER (see section 7.3.11). But if one is requested for transmission by the iSCSI layer at the initiator, it will be sent by the iSER layer as a control-type PDU. Since it is "unexpected", the number that can be sent by the initiator falls under the MaxOutstandingUnexpectedPDU declared by the target. The PDU is outstanding until the target responds with a control-type PDU with ExpCmdSN set to X + 1 where X is the CmdSN of the referenced command. 2. For unsolicited SCSI Data-out PDU, the target can use FirstBurstLength to determine the amount of buffering required. Solicited data are normally handled using the R2T mechanism (see section 7.3.4). However, if a SCSI Data-out PDU containing solicited data is requested for transmission by the iSCSI layer at the initiator, the iSER layer may send it as unsolciited data. Since it is "unexpected", the number that can be sent by the initiator falls under the MaxOutstandingUnexpectedPDU declared by the target. The PDU is outstanding until the target responds with a control-type PDU with ExpCmdSN set to X + 1 where X is the CmdSN of the referenced command. 3. A PDU with the immediate flag set is considered "unexpected", and the number that can be sent by the initiator falls under the MaxOutstandingUnexpectedPDU declared by the target. The PDU is outstanding until the target responds with a control-type PDU with ExpCmdSN set to X + 1 where X is the CmdSN of the PDU with the immediate flag set. Control-type PDUs sent by a target as a response to a request from an initiator are expected by the initiator and buffer resources required can be provisioned. The exceptions include the SCSI Data-in PDU, the R2T PDU, the Asynchronous Message PDU, the Reject PDU, and the NOP-in PDU. 1. A SCSI Data-in PDU is a data-type PDU and is transformed into RDMA Write operations (see section 7.3.5). 2. An R2T PDU is a data-type PDU and is transformed into RDMA Read operations (see section 7.3.6). 3. An Asynchronous Message PDU is considered "unexpected", and the number that can be sent by the target falls under the MaxOutstandingUnexpectedPDU declared by the initiator. The PDU is outstanding until the initiator sends a control-type PDU with ExpStatSN set to X + 1 where X is the StatSN of the Asychronous Message PDU. 4. A Reject PDU is considered "unexpected", and the number that can be sent by the target falls under the MaxOutstandingUnexpectedPDU declared by the initiator. The PDU is outstanding until the initiator sends a control-type PDU with ExpStatSN set to X + 1 where X is the StatSN of the Reject PDU. 5. A NOP-in PDU sent as a response by the target to a NOP-out PDU from the initiator is expected by the initiator. Otherwise, a NOP-out PDU is considered "unexpected", and the number that can be sent by the target falls under the MaxOutstandingUnexpectedPDU declared by the initiator. The PDU is outstanding until the initiator sends a control-type PDU with ExpStatSN set to X + 1 where X is the StatSN of the NOP-in PDU. Mike _______________________________________________ Ips mailing list Ips@ietf.org https://www1.ietf.org/mailman/listinfo/ips _______________________________________________ Ips mailing list Ips@ietf.org https://www1.ietf.org/mailman/listinfo/ips _______________________________________________ Ips mailing list Ips@ietf.org https://www1.ietf.org/mailman/listinfo/ips From ips-bounces@ietf.org Fri Oct 15 19:32:39 2004 Received: from ietf-mx.ietf.org (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id TAA04903 for ; Fri, 15 Oct 2004 19:32:39 -0400 (EDT) Received: from megatron.ietf.org ([132.151.6.71]) by ietf-mx.ietf.org with esmtp (Exim 4.33) id 1CIbkO-0000LP-Kn for ips-web-archive@ietf.org; Fri, 15 Oct 2004 19:44:33 -0400 Received: from localhost.localdomain ([127.0.0.1] helo=megatron.ietf.org) by megatron.ietf.org with esmtp (Exim 4.32) id 1CIbWe-00014j-P7; Fri, 15 Oct 2004 19:30:20 -0400 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by megatron.ietf.org with esmtp (Exim 4.32) id 1CIbMp-00061u-6R for ips@megatron.ietf.org; Fri, 15 Oct 2004 19:20:11 -0400 Received: from ietf-mx.ietf.org (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id TAA03569 for ; Fri, 15 Oct 2004 19:20:07 -0400 (EDT) Received: from e31.co.us.ibm.com ([32.97.110.129]) by ietf-mx.ietf.org with esmtp (Exim 4.33) id 1CIbYG-0008Qg-J5 for ips@ietf.org; Fri, 15 Oct 2004 19:32:01 -0400 Received: from westrelay04.boulder.ibm.com (westrelay04.boulder.ibm.com [9.17.193.32]) by e31.co.us.ibm.com (8.12.10/8.12.9) with ESMTP id i9FNJbLv250272 for ; Fri, 15 Oct 2004 19:19:37 -0400 Received: from d03av04.boulder.ibm.com (d03av04.boulder.ibm.com [9.17.195.170]) by westrelay04.boulder.ibm.com (8.12.10/NCO/VER6.6) with ESMTP id i9FNJbOX158480 for ; Fri, 15 Oct 2004 17:19:37 -0600 Received: from d03av04.boulder.ibm.com (loopback [127.0.0.1]) by d03av04.boulder.ibm.com (8.12.11/8.12.11) with ESMTP id i9FNJadS022857 for ; Fri, 15 Oct 2004 17:19:36 -0600 Received: from d03nm115.boulder.ibm.com (d03nm115.boulder.ibm.com [9.17.195.141]) by d03av04.boulder.ibm.com (8.12.11/8.12.11) with ESMTP id i9FNJadu022845 for ; Fri, 15 Oct 2004 17:19:36 -0600 In-Reply-To: To: Mike Ko MIME-Version: 1.0 Subject: Re: [Ips] New iSCSI key X-Mailer: Lotus Notes Release 6.0.2CF1 June 9, 2003 From: John Hufferd Message-ID: Date: Fri, 15 Oct 2004 16:19:21 -0700 X-MIMETrack: Serialize by Router on D03NM115/03/M/IBM(Release 6.51HF338 | June 21, 2004) at 10/15/2004 17:19:35, Serialize complete at 10/15/2004 17:19:35 X-Spam-Score: 0.0 (/) X-Scan-Signature: 97b8643c8d9cc60b17ea5996f3e435cb Cc: ips@ietf.org X-BeenThere: ips@ietf.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: IP Storage List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Content-Type: multipart/mixed; boundary="===============1241093618==" Sender: ips-bounces@ietf.org Errors-To: ips-bounces@ietf.org X-Spam-Score: 0.0 (/) X-Scan-Signature: 5bf916ef84139eb2787672b9fb107b6e This is a multipart message in MIME format. --===============1241093618== Content-Type: multipart/alternative; boundary="=_alternative 0080085188256F2E_=" This is a multipart message in MIME format. --=_alternative 0080085188256F2E_= Content-Type: text/plain; charset="US-ASCII" Mike, I think we are getting closer, but since if Data-Outs ever gets solicited by an R2T, even though it shouldn't, it is by definition expected. So since the only other case for Data-Out PDUs is for unsolicited Data that can be anticipated, as you pointed out, via the FirstBurstLength, I just do not see Data-Out PDUs being in the same class as what we have been discussing, and think we should not even factor it into the unexpected classification. In other words just drop point 2. However, the text you have at the beginning of that paragraph, "Unsolicited data in a SCSI write operation are sent by the iSCSI layer at the initiator using SCSI Data-out PDUs. The iSER layer at the target can use FirstBurstLength to determine the amount of buffering required." should some how be placed somewhere else in the draft. . . John L. Hufferd Senior Technical Staff Member (STSM) IBM/System Group, San Jose CA Main Office: (408) 256-0403, Tie: 276-0403, eFax: (408) 904-4688 Alt Office: (408) 997-6136, Cell: (408) 499-9702 Internet Address: hufferd@us.ibm.com Mike Ko/Almaden/IBM 10/15/2004 01:23 PM To John Hufferd/San Jose/IBM@IBMUS cc ips@ietf.org Subject Re: [Ips] New iSCSI key John, Here is the replacement paragraph for the SCSI Data-out PDU: 2. Unsolicited data in a SCSI write operation are sent by the iSCSI layer at the initiator using SCSI Data-out PDUs. The iSER layer at the target can use FirstBurstLength to determine the amount of buffering required. For solicited data, an R2T PDU sent by the iSCSI layer at the target is transformed by the iSER layer at the target into an RDMA Read operation (see section 7.3.6). Therefore the iSCSI layer at the initiator should not be issuing SCSI Data-out PDUs for solicited data. If a SCSI Data-out PDU containing solicited data is requested for transmission by the iSCSI layer at the initiator anyway, the iSER layer may send it as a control-type PDU (see section 7.3.4). Since it is "unexpected", the number that can be sent by the initiator falls under the MaxOutstandingUnexpectedPDU declared by the target. The PDU is outstanding until the target responds with a control-type PDU with ExpCmdSN set to X + 1 where X is the CmdSN of the referenced command. Mike Sent by: ips-bounces@ietf.org To: John Hufferd/San Jose/IBM@IBMUS cc: ips@ietf.org Subject: Re: [Ips] New iSCSI key John, Would the following replacement text for the sentence "Solicited data are normally handled ..." better? "The R2T mechanism is normally used to handle solicited data and the SCSI Data-out PDU should not be used for this purpose (see section 7.3.4)." Mike From: John Hufferd on 10/15/2004 10:22 AM To: Mike Ko/Almaden/IBM cc: ips@ietf.org Subject: Re: [Ips] New iSCSI key Mike, I think you covered my concerns, but perhaps we should actually use words in the draft similar to what you have used here for the unsolicited Data-Out PDUs. I just thought that the words you proposed for the draft were a little bit confusing, since we never have talk about how an iSCSI layer requests Data-Out PDUs, etc. I think your words in this note were more to the point. . . John L. Hufferd Senior Technical Staff Member (STSM) IBM/System Group, San Jose CA Main Office: (408) 256-0403, Tie: 276-0403, eFax: (408) 904-4688 Alt Office: (408) 997-6136, Cell: (408) 499-9702 Internet Address: hufferd@us.ibm.com Mike Ko/Almaden/IBM 10/15/2004 10:01 AM To John Hufferd/San Jose/IBM@IBMUS cc ips@ietf.org Subject Re: [Ips] New iSCSI key John, My comments are embedded. Mike To: Mike Ko cc: ips@ietf.org Subject: Re: [Ips] New iSCSI key Mike, First I think there is a Typo near the end of your note when you talk about NOP-Out PDU where you might have meant Nop-In. That is, where you say "...Otherwise, a NOP-out PDU is considered "unexpected",...." perhaps you should say Nop-In. [mk[ Good catch. I will fix that. ]mk] Second I think we need to discuss the unsolicited Data-Out PDU. Please expand on the concept of the use of Data-Out PDU for solicited Data in iSER. [mk[ SCSI Data-out PDU should not be used for solicited data even though it is supported in the iSER draft. Since this is not a typical usage, the burden is left to the iSER/iSCSI layer to keep track of the number of "unexpected" PDU that it can send. What specifically do you want to expand on this topic? ]mk] Third we need to discuss or expand on considering the Reject PDU an unsolicited PDU. Perhaps you meant Rejects for PDUs that are not for the Command itself. [mk[ I will replace the phrase "A Reject PDU is considered "unexpected"" with the following: For a task which has never been active before the Reject PDU is sent, the target should not send any further responses because the command itself is discarded (see section 6.3 of RFC3720). Therefore, this type of Reject PDU is expecetd. All other Reject PDUs are "unexpected". ]mk] . . John L. Hufferd Senior Technical Staff Member (STSM) IBM/System Group, San Jose CA Main Office: (408) 256-0403, Tie: 276-0403, eFax: (408) 904-4688 Alt Office: (408) 997-6136, Cell: (408) 499-9702 Internet Address: hufferd@us.ibm.com Mike Ko Sent by: ips-bounces@ietf.org 10/14/2004 09:36 PM To cc Subject [Ips] New iSCSI key If we do decide on adding a new iSCSI key to limit the number of "unexpected" or "unanticipated" PDUs, I offer the following text to be considered for adding to section 6 of the iSER draft. Currently the key is tentatively called "MaxOutstandingUnexpectedPDUs". Unless there are other suggestions, this name will be used. Use: LO (leading only), Declarative Senders: Initiator and Target Scope: SW (session-wide) Irrelevant when: RDMAExtensions=No MaxOutstandingUnexpectedPDUs= Default is None This key is used by the initiator and the target to declare the maximum number of outstanding "unexpected" control-type PDUs that it can receive. It is intended to allow the receiving side to determine the amount of buffer resoruces needed beyond the normal flow control mechanism available in iSCSI. An initiator or target that cannot function properly without any limit on MaxOutstandingUnexpectedPDU (which is the default) should choose a value such that it would not impose an unnecessary constraint on the iSCSI layer under normal circumstances. The proper value to use is implementation dependent. If the sending side fails to adhere to the declared limit as set by the receiving side, it is up to the receiving side to determine ways of handling the overrun, including dropping the connection. For control-type PDUs sent by an initiator to a target, where the immediate flag is not set, the command numbering mechanism regulates the maximum number of PDUs that the target can expect. The exceptions include the SNACK PDU, the SCSI Data-out PDU, and any PDU with the immediate flag set. 1. The SNACK PDU is not needed in iSER (see section 7.3.11). But if one is requested for transmission by the iSCSI layer at the initiator, it will be sent by the iSER layer as a control-type PDU. Since it is "unexpected", the number that can be sent by the initiator falls under the MaxOutstandingUnexpectedPDU declared by the target. The PDU is outstanding until the target responds with a control-type PDU with ExpCmdSN set to X + 1 where X is the CmdSN of the referenced command. 2. For unsolicited SCSI Data-out PDU, the target can use FirstBurstLength to determine the amount of buffering required. Solicited data are normally handled using the R2T mechanism (see section 7.3.4). However, if a SCSI Data-out PDU containing solicited data is requested for transmission by the iSCSI layer at the initiator, the iSER layer may send it as unsolciited data. Since it is "unexpected", the number that can be sent by the initiator falls under the MaxOutstandingUnexpectedPDU declared by the target. The PDU is outstanding until the target responds with a control-type PDU with ExpCmdSN set to X + 1 where X is the CmdSN of the referenced command. 3. A PDU with the immediate flag set is considered "unexpected", and the number that can be sent by the initiator falls under the MaxOutstandingUnexpectedPDU declared by the target. The PDU is outstanding until the target responds with a control-type PDU with ExpCmdSN set to X + 1 where X is the CmdSN of the PDU with the immediate flag set. Control-type PDUs sent by a target as a response to a request from an initiator are expected by the initiator and buffer resources required can be provisioned. The exceptions include the SCSI Data-in PDU, the R2T PDU, the Asynchronous Message PDU, the Reject PDU, and the NOP-in PDU. 1. A SCSI Data-in PDU is a data-type PDU and is transformed into RDMA Write operations (see section 7.3.5). 2. An R2T PDU is a data-type PDU and is transformed into RDMA Read operations (see section 7.3.6). 3. An Asynchronous Message PDU is considered "unexpected", and the number that can be sent by the target falls under the MaxOutstandingUnexpectedPDU declared by the initiator. The PDU is outstanding until the initiator sends a control-type PDU with ExpStatSN set to X + 1 where X is the StatSN of the Asychronous Message PDU. 4. A Reject PDU is considered "unexpected", and the number that can be sent by the target falls under the MaxOutstandingUnexpectedPDU declared by the initiator. The PDU is outstanding until the initiator sends a control-type PDU with ExpStatSN set to X + 1 where X is the StatSN of the Reject PDU. 5. A NOP-in PDU sent as a response by the target to a NOP-out PDU from the initiator is expected by the initiator. Otherwise, a NOP-out PDU is considered "unexpected", and the number that can be sent by the target falls under the MaxOutstandingUnexpectedPDU declared by the initiator. The PDU is outstanding until the initiator sends a control-type PDU with ExpStatSN set to X + 1 where X is the StatSN of the NOP-in PDU. Mike _______________________________________________ Ips mailing list Ips@ietf.org https://www1.ietf.org/mailman/listinfo/ips _______________________________________________ Ips mailing list Ips@ietf.org https://www1.ietf.org/mailman/listinfo/ips --=_alternative 0080085188256F2E_= Content-Type: text/html; charset="US-ASCII"
Mike,
I think we are getting closer, but since if Data-Outs ever gets solicited by an R2T, even though it shouldn't, it is by definition expected.  So since the only other case for Data-Out PDUs is for unsolicited Data that can be anticipated, as you pointed out, via the FirstBurstLength, I just do not see Data-Out PDUs being in the same class as what we have been discussing, and think we should not even factor it into the unexpected classification.  In other words just drop point 2.  However, the text you have at the beginning of that paragraph, "Unsolicited data in a SCSI write operation are sent by the iSCSI layer at the initiator using SCSI Data-out PDUs.  The iSER layer at the target can use
FirstBurstLength to determine the amount of buffering required."  should some how be placed somewhere else in the draft.
.
.
John L. Hufferd
Senior Technical Staff Member (STSM)
IBM/System Group, San Jose CA
Main Office: (408) 256-0403, Tie: 276-0403, eFax: (408) 904-4688
Alt Office: (408) 997-6136, Cell: (408) 499-9702
Internet Address: hufferd@us.ibm.com


Mike Ko/Almaden/IBM

10/15/2004 01:23 PM

To
John Hufferd/San Jose/IBM@IBMUS
cc
ips@ietf.org
Subject
Re: [Ips] New iSCSI keyLink




John,
Here is the replacement paragraph for the SCSI Data-out PDU:

2. Unsolicited data in a SCSI write operation are sent by the iSCSI layer at
the initiator using SCSI Data-out PDUs.  The iSER layer at the target can use
FirstBurstLength to determine the amount of buffering required.  For solicited
data, an R2T PDU sent by the iSCSI layer at the target is transformed by the
iSER layer at the target into an RDMA Read operation (see section 7.3.6).
Therefore the iSCSI layer at the initiator should not be issuing SCSI Data-out
PDUs for solicited data.  If a SCSI Data-out PDU containing solicited data is
requested for transmission by the iSCSI layer at the initiator anyway, the iSER
layer may send it as a control-type PDU (see section 7.3.4).  Since it is
"unexpected", the number that can be sent by the initiator falls under the
MaxOutstandingUnexpectedPDU declared by the target.  The PDU is outstanding
until the target responds with a control-type PDU with ExpCmdSN set to X + 1
where X is the CmdSN of the referenced command.

Mike

Sent by:        ips-bounces@ietf.org

To:        John Hufferd/San Jose/IBM@IBMUS
cc:        ips@ietf.org
Subject:        Re: [Ips] New iSCSI key



John,
Would the following replacement text for the sentence "Solicited data are
normally handled ..." better?
"The R2T mechanism is normally used to handle solicited data and the SCSI
Data-out PDU should not be used for this purpose (see section 7.3.4)."
Mike
From:   John Hufferd on 10/15/2004 10:22 AM
To:     Mike Ko/Almaden/IBM
cc:     ips@ietf.org
Subject:        Re: [Ips] New iSCSI key
Mike,
I think you covered my concerns, but perhaps we should actually use words
in the draft similar to what you have used here for the unsolicited
Data-Out PDUs.  I just thought that the words you proposed for the draft
were a little bit confusing, since we never have talk about how an iSCSI
layer requests Data-Out PDUs, etc.  I think your words in this note were
more to the point.
.
.
John L. Hufferd
Senior Technical Staff Member (STSM)
IBM/System Group, San Jose CA
Main Office: (408) 256-0403, Tie: 276-0403, eFax: (408) 904-4688
Alt Office: (408) 997-6136, Cell: (408) 499-9702
Internet Address: hufferd@us.ibm.com


Mike Ko/Almaden/IBM
10/15/2004 10:01 AM
To
John Hufferd/San Jose/IBM@IBMUS
cc
ips@ietf.org
Subject
Re: [Ips] New iSCSI key




John,
My comments are embedded.
Mike
To:     Mike Ko <mako@almaden.ibm.com>
cc:     ips@ietf.org
Subject:        Re: [Ips] New iSCSI key



Mike,
First I think there is a Typo near the end of your note when you talk
about NOP-Out PDU where you might have meant Nop-In.  That is, where you
say "...Otherwise, a NOP-out PDU is
considered "unexpected",...." perhaps you should say Nop-In.
[mk[
Good catch.  I will fix that.
]mk]
Second I think we need to discuss the unsolicited Data-Out PDU.  Please
expand on the concept of the use of Data-Out PDU for solicited Data in
iSER.
[mk[
SCSI Data-out PDU should not be used for solicited data even though it is
supported in the iSER draft.  Since this is not a typical usage, the
burden is left to the iSER/iSCSI layer to keep track of the number of
"unexpected" PDU that it can send.  What specifically do you want to
expand on this topic?
]mk]
Third we need to discuss or expand on considering the Reject PDU an
unsolicited PDU.   Perhaps you meant Rejects for PDUs that are not for the
Command itself.
[mk[
I will replace the phrase "A Reject PDU is considered "unexpected"" with
the following:
For a task which has never been active before the Reject PDU is sent, the
target should not send any further responses because the command itself is
discarded (see section 6.3 of RFC3720).  Therefore, this type of Reject
PDU is expecetd.  All other Reject PDUs are "unexpected".
]mk]
.
.
John L. Hufferd
Senior Technical Staff Member (STSM)
IBM/System Group, San Jose CA
Main Office: (408) 256-0403, Tie: 276-0403, eFax: (408) 904-4688
Alt Office: (408) 997-6136, Cell: (408) 499-9702
Internet Address: hufferd@us.ibm.com


Mike Ko <mako@almaden.ibm.com>
Sent by: ips-bounces@ietf.org
10/14/2004 09:36 PM
To
<ips@ietf.org>
cc
Subject
[Ips] New iSCSI key






If we do decide on adding a new iSCSI key to limit the number of
"unexpected"
or "unanticipated" PDUs, I offer the following text to be considered for
adding
to section 6 of the iSER draft.
Currently the key is tentatively called "MaxOutstandingUnexpectedPDUs".
Unless
there are other suggestions, this name will be used.
Use: LO (leading only), Declarative
Senders: Initiator and Target
Scope: SW (session-wide)
Irrelevant when: RDMAExtensions=No
MaxOutstandingUnexpectedPDUs=<numerical-value-from-0-to-(2**64-1) | None>
Default is None
This key is used by the initiator and the target to declare the maximum
number
of outstanding "unexpected" control-type PDUs that it can receive.  It is
intended to allow the receiving side to determine the amount of buffer
resoruces needed beyond the normal flow control mechanism available in
iSCSI.
An initiator or target that cannot function properly without any limit on
MaxOutstandingUnexpectedPDU (which is the default) should choose a value
such
that it would not impose an unnecessary constraint on the iSCSI layer
under
normal circumstances.  The proper value to use is implementation
dependent.  If
the sending side fails to adhere to the declared limit as set by the
receiving
side, it is up to the receiving side to determine ways of handling the
overrun,
including dropping the connection.
For control-type PDUs sent by an initiator to a target, where the
immediate
flag is not set, the command numbering mechanism regulates the maximum
number
of PDUs that the target can expect.  The exceptions include the SNACK PDU,
the
SCSI Data-out PDU, and any PDU with the immediate flag set.
1. The SNACK PDU is not needed in iSER (see section 7.3.11).  But if one
is
requested for transmission by the iSCSI layer at the initiator, it will be
sent
by the iSER layer as a control-type PDU.  Since it is "unexpected", the
number
that can be sent by the initiator falls under the
MaxOutstandingUnexpectedPDU
declared by the target.  The PDU is outstanding until the target responds
with
a control-type PDU with ExpCmdSN set to X + 1 where X is the CmdSN of the
referenced command.
2. For unsolicited SCSI Data-out PDU, the target can use FirstBurstLength
to
determine the amount of buffering required.  Solicited data are normally
handled using the R2T mechanism (see section 7.3.4).  However, if a SCSI
Data-out PDU containing solicited data is requested for transmission by
the
iSCSI layer at the initiator, the iSER layer may send it as unsolciited
data.
Since it is "unexpected", the number that can be sent by the initiator
falls
under the MaxOutstandingUnexpectedPDU declared by the target.  The PDU is
outstanding until the target responds with a control-type PDU with
ExpCmdSN set
to X + 1 where X is the CmdSN of the referenced command.
3. A PDU with the immediate flag set is considered "unexpected", and the
number
that can be sent by the initiator falls under the
MaxOutstandingUnexpectedPDU
declared by the target.  The PDU is outstanding until the target responds
with
a control-type PDU with ExpCmdSN set to X + 1 where X is the CmdSN of the
PDU
with the immediate flag set.
Control-type PDUs sent by a target as a response to a request from an
initiator
are expected by the initiator and buffer resources required can be
provisioned.
The exceptions include the SCSI Data-in PDU, the R2T PDU, the Asynchronous
Message PDU, the Reject PDU, and the NOP-in PDU.
1. A SCSI Data-in PDU is a data-type PDU and is transformed into RDMA
Write
operations (see section 7.3.5).
2. An R2T PDU is a data-type PDU and is transformed into RDMA Read
operations
(see section 7.3.6).
3. An Asynchronous Message PDU is considered "unexpected", and the number
that
can be sent by the target falls under the MaxOutstandingUnexpectedPDU
declared
by the initiator.  The PDU is outstanding until the initiator sends a
control-type PDU with ExpStatSN set to X + 1 where X is the StatSN of the
Asychronous Message PDU.
4. A Reject PDU is considered "unexpected", and the number that can be
sent by
the target falls under the MaxOutstandingUnexpectedPDU declared by the
initiator.  The PDU is outstanding until the initiator sends a
control-type PDU
with ExpStatSN set to X + 1 where X is the StatSN of the Reject PDU.
5. A NOP-in PDU sent as a response by the target to a NOP-out PDU from the
initiator is expected by the initiator.  Otherwise, a NOP-out PDU is
considered
"unexpected", and the number that can be sent by the target falls under
the
MaxOutstandingUnexpectedPDU declared by the initiator.  The PDU is
outstanding
until the initiator sends a control-type PDU with ExpStatSN set to X + 1
where
X is the StatSN of the NOP-in PDU.
Mike
_______________________________________________
Ips mailing list
Ips@ietf.org
https://www1.ietf.org/mailman/listinfo/ips




_______________________________________________
Ips mailing list
Ips@ietf.org
https://www1.ietf.org/mailman/listinfo/ips

--=_alternative 0080085188256F2E_=-- --===============1241093618== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline Content-Transfer-Encoding: 7bit _______________________________________________ Ips mailing list Ips@ietf.org https://www1.ietf.org/mailman/listinfo/ips --===============1241093618==-- From ips-bounces@ietf.org Fri Oct 15 20:38:09 2004 Received: from ietf-mx.ietf.org (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id UAA09911 for ; Fri, 15 Oct 2004 20:38:09 -0400 (EDT) Received: from megatron.ietf.org ([132.151.6.71]) by ietf-mx.ietf.org with esmtp (Exim 4.33) id 1CIclk-0001pd-G2 for ips-web-archive@ietf.org; Fri, 15 Oct 2004 20:50:03 -0400 Received: from localhost.localdomain ([127.0.0.1] helo=megatron.ietf.org) by megatron.ietf.org with esmtp (Exim 4.32) id 1CIcYY-0003Yo-PP; Fri, 15 Oct 2004 20:36:22 -0400 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by megatron.ietf.org with esmtp (Exim 4.32) id 1CIcWx-0002s9-K7 for ips@megatron.ietf.org; Fri, 15 Oct 2004 20:34:43 -0400 Received: from ietf-mx.ietf.org (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id UAA09674 for ; Fri, 15 Oct 2004 20:34:41 -0400 (EDT) Received: from e1.ny.us.ibm.com ([32.97.182.101]) by ietf-mx.ietf.org with esmtp (Exim 4.33) id 1CIciP-0001kc-Rx for ips@ietf.org; Fri, 15 Oct 2004 20:46:34 -0400 Received: from northrelay04.pok.ibm.com (northrelay04.pok.ibm.com [9.56.224.206]) by e1.ny.us.ibm.com (8.12.10/NS PXFA) with ESMTP id i9G0YArZ245936 for ; Fri, 15 Oct 2004 20:34:10 -0400 Received: from d01ml604.pok.ibm.com (d01av04.pok.ibm.com [9.56.224.64]) by northrelay04.pok.ibm.com (8.12.10/NCO/VER6.6) with ESMTP id i9G0ZPR2099340 for ; Fri, 15 Oct 2004 20:35:25 -0400 In-Reply-To: Importance: Normal MIME-Version: 1.0 To: John Hufferd Subject: Re: [Ips] New iSCSI key X-Mailer: Lotus Notes Release 6.5.2 June 01, 2004 Message-ID: From: Mike Ko Date: Fri, 15 Oct 2004 17:34:07 -0700 X-MIMETrack: Serialize by Router on D01ML604/01/M/IBM(Release 6.51HF643 | October 13, 2004) at 10/15/2004 20:34:10, Serialize complete at 10/15/2004 20:34:10 Content-Type: text/plain; charset="US-ASCII" X-Spam-Score: 0.0 (/) X-Scan-Signature: 3dc828214e948ff35b815af10e94a823 Cc: ips@ietf.org X-BeenThere: ips@ietf.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: IP Storage List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Sender: ips-bounces@ietf.org Errors-To: ips-bounces@ietf.org X-Spam-Score: 0.0 (/) X-Scan-Signature: d437399464e10b52abe9a34ed7e712d0 John, The initiator should never see an R2T since an R2T is transformed into an RDMA Read Request by the target and is never sent as a control-type PDU. Neither the DA nor the iSER spec ever mentioned sending an R2T as a control-type PDU. That being the case, the initiator should never be sending SCSI Data-out PDUs that are "solicited" by an R2T. Conversely, a target will not be "expecting" control-type PDUs containing "solicited" data since it never sends an R2T as is. But since a SCSI Data-out PDU containing legitimate unsolicited data is sent as a control-type PDU, we allow the iSER layer to send *all* SCSI Data-out PDUs as control-type PDUs, even when they contain "solicited" data. However in view of MaxOutstandingUnexpectedPDU, what happens if the limit is smaller than the amount of "solicited" data that the iSCSI layer wants to send? Using ExpCmdSN to determine if the PDUs are outstanding or not might not work since the target might not advance ExpCmdSN frequent enough for all the "solicited" data to be transferred. We would need a different mechanism to replenish the credit. But sending "solicited" data as control-type PDUs is an error since it is not "solicited" by an R2T. Should we make life complicated in order to accomodate an error case? For SCSI Data-in and R2T, neither the DA nor the iSER spec mentioned allowing the target to send them as control-type PDUs. On SCSI Data-out PDUs, the DA spec mentioned the following: Section 7.1 "the iSCSI layer does not request a SCSI Data-Out PDU carrying solicited data for transmission across the Datamover Interface per this architecture" Section 7.1.1 "a SCSI Data-Out PDU for solicited data is never requested for transmission across the Datamover Interface at the initiator" Section 11.4.5.1 "the solicited SCSI Data-out arriving on the target is not notified to the iSCSI layer using the Control_Notify Primitive because the solicited SCSI Data-out was not sent by the initiator iSCSI layer as control-type PDUs" So to be consistent with the DA spec, I propose that we also disallow the sending of "solicitied" data as control-type PDUs. If we accept this correction, then the SCSI Data-out handling for MaxOutstandingUnexpectedPDUs can be simplified. Also the last paragraph in section 7.3.4 will be updated to remove the reference to allowing "solicited" data being sent as control-type PDUs. Comments? Mike From: John Hufferd on 10/15/2004 04:18 PM To: Mike Ko/Almaden/IBM cc: ips@ietf.org Subject: Re: [Ips] New iSCSI key Mike, I think we are getting closer, but since if Data-Outs ever gets solicited by an R2T, even though it shouldn't, it is by definition expected. So since the only other case for Data-Out PDUs is for unsolicited Data that can be anticipated, as you pointed out, via the FirstBurstLength, I just do not see Data-Out PDUs being in the same class as what we have been discussing, and think we should not even factor it into the unexpected classification. In other words just drop point 2. However, the text you have at the beginning of that paragraph, "Unsolicited data in a SCSI write operation are sent by the iSCSI layer at the initiator using SCSI Data-out PDUs. The iSER layer at the target can use FirstBurstLength to determine the amount of buffering required." should some how be placed somewhere else in the draft. . . John L. Hufferd Senior Technical Staff Member (STSM) IBM/System Group, San Jose CA Main Office: (408) 256-0403, Tie: 276-0403, eFax: (408) 904-4688 Alt Office: (408) 997-6136, Cell: (408) 499-9702 Internet Address: hufferd@us.ibm.com Mike Ko/Almaden/IBM 10/15/2004 01:23 PM To John Hufferd/San Jose/IBM@IBMUS cc ips@ietf.org Subject Re: [Ips] New iSCSI key John, Here is the replacement paragraph for the SCSI Data-out PDU: 2. Unsolicited data in a SCSI write operation are sent by the iSCSI layer at the initiator using SCSI Data-out PDUs. The iSER layer at the target can use FirstBurstLength to determine the amount of buffering required. For solicited data, an R2T PDU sent by the iSCSI layer at the target is transformed by the iSER layer at the target into an RDMA Read operation (see section 7.3.6). Therefore the iSCSI layer at the initiator should not be issuing SCSI Data-out PDUs for solicited data. If a SCSI Data-out PDU containing solicited data is requested for transmission by the iSCSI layer at the initiator anyway, the iSER layer may send it as a control-type PDU (see section 7.3.4). Since it is "unexpected", the number that can be sent by the initiator falls under the MaxOutstandingUnexpectedPDU declared by the target. The PDU is outstanding until the target responds with a control-type PDU with ExpCmdSN set to X + 1 where X is the CmdSN of the referenced command. Mike Sent by: ips-bounces@ietf.org To: John Hufferd/San Jose/IBM@IBMUS cc: ips@ietf.org Subject: Re: [Ips] New iSCSI key John, Would the following replacement text for the sentence "Solicited data are normally handled ..." better? "The R2T mechanism is normally used to handle solicited data and the SCSI Data-out PDU should not be used for this purpose (see section 7.3.4)." Mike From: John Hufferd on 10/15/2004 10:22 AM To: Mike Ko/Almaden/IBM cc: ips@ietf.org Subject: Re: [Ips] New iSCSI key Mike, I think you covered my concerns, but perhaps we should actually use words in the draft similar to what you have used here for the unsolicited Data-Out PDUs. I just thought that the words you proposed for the draft were a little bit confusing, since we never have talk about how an iSCSI layer requests Data-Out PDUs, etc. I think your words in this note were more to the point. . . John L. Hufferd Senior Technical Staff Member (STSM) IBM/System Group, San Jose CA Main Office: (408) 256-0403, Tie: 276-0403, eFax: (408) 904-4688 Alt Office: (408) 997-6136, Cell: (408) 499-9702 Internet Address: hufferd@us.ibm.com Mike Ko/Almaden/IBM 10/15/2004 10:01 AM To John Hufferd/San Jose/IBM@IBMUS cc ips@ietf.org Subject Re: [Ips] New iSCSI key John, My comments are embedded. Mike To: Mike Ko cc: ips@ietf.org Subject: Re: [Ips] New iSCSI key Mike, First I think there is a Typo near the end of your note when you talk about NOP-Out PDU where you might have meant Nop-In. That is, where you say "...Otherwise, a NOP-out PDU is considered "unexpected",...." perhaps you should say Nop-In. [mk[ Good catch. I will fix that. ]mk] Second I think we need to discuss the unsolicited Data-Out PDU. Please expand on the concept of the use of Data-Out PDU for solicited Data in iSER. [mk[ SCSI Data-out PDU should not be used for solicited data even though it is supported in the iSER draft. Since this is not a typical usage, the burden is left to the iSER/iSCSI layer to keep track of the number of "unexpected" PDU that it can send. What specifically do you want to expand on this topic? ]mk] Third we need to discuss or expand on considering the Reject PDU an unsolicited PDU. Perhaps you meant Rejects for PDUs that are not for the Command itself. [mk[ I will replace the phrase "A Reject PDU is considered "unexpected"" with the following: For a task which has never been active before the Reject PDU is sent, the target should not send any further responses because the command itself is discarded (see section 6.3 of RFC3720). Therefore, this type of Reject PDU is expecetd. All other Reject PDUs are "unexpected". ]mk] . . John L. Hufferd Senior Technical Staff Member (STSM) IBM/System Group, San Jose CA Main Office: (408) 256-0403, Tie: 276-0403, eFax: (408) 904-4688 Alt Office: (408) 997-6136, Cell: (408) 499-9702 Internet Address: hufferd@us.ibm.com Mike Ko Sent by: ips-bounces@ietf.org 10/14/2004 09:36 PM To cc Subject [Ips] New iSCSI key If we do decide on adding a new iSCSI key to limit the number of "unexpected" or "unanticipated" PDUs, I offer the following text to be considered for adding to section 6 of the iSER draft. Currently the key is tentatively called "MaxOutstandingUnexpectedPDUs". Unless there are other suggestions, this name will be used. Use: LO (leading only), Declarative Senders: Initiator and Target Scope: SW (session-wide) Irrelevant when: RDMAExtensions=No MaxOutstandingUnexpectedPDUs= Default is None This key is used by the initiator and the target to declare the maximum number of outstanding "unexpected" control-type PDUs that it can receive. It is intended to allow the receiving side to determine the amount of buffer resoruces needed beyond the normal flow control mechanism available in iSCSI. An initiator or target that cannot function properly without any limit on MaxOutstandingUnexpectedPDU (which is the default) should choose a value such that it would not impose an unnecessary constraint on the iSCSI layer under normal circumstances. The proper value to use is implementation dependent. If the sending side fails to adhere to the declared limit as set by the receiving side, it is up to the receiving side to determine ways of handling the overrun, including dropping the connection. For control-type PDUs sent by an initiator to a target, where the immediate flag is not set, the command numbering mechanism regulates the maximum number of PDUs that the target can expect. The exceptions include the SNACK PDU, the SCSI Data-out PDU, and any PDU with the immediate flag set. 1. The SNACK PDU is not needed in iSER (see section 7.3.11). But if one is requested for transmission by the iSCSI layer at the initiator, it will be sent by the iSER layer as a control-type PDU. Since it is "unexpected", the number that can be sent by the initiator falls under the MaxOutstandingUnexpectedPDU declared by the target. The PDU is outstanding until the target responds with a control-type PDU with ExpCmdSN set to X + 1 where X is the CmdSN of the referenced command. 2. For unsolicited SCSI Data-out PDU, the target can use FirstBurstLength to determine the amount of buffering required. Solicited data are normally handled using the R2T mechanism (see section 7.3.4). However, if a SCSI Data-out PDU containing solicited data is requested for transmission by the iSCSI layer at the initiator, the iSER layer may send it as unsolciited data. Since it is "unexpected", the number that can be sent by the initiator falls under the MaxOutstandingUnexpectedPDU declared by the target. The PDU is outstanding until the target responds with a control-type PDU with ExpCmdSN set to X + 1 where X is the CmdSN of the referenced command. 3. A PDU with the immediate flag set is considered "unexpected", and the number that can be sent by the initiator falls under the MaxOutstandingUnexpectedPDU declared by the target. The PDU is outstanding until the target responds with a control-type PDU with ExpCmdSN set to X + 1 where X is the CmdSN of the PDU with the immediate flag set. Control-type PDUs sent by a target as a response to a request from an initiator are expected by the initiator and buffer resources required can be provisioned. The exceptions include the SCSI Data-in PDU, the R2T PDU, the Asynchronous Message PDU, the Reject PDU, and the NOP-in PDU. 1. A SCSI Data-in PDU is a data-type PDU and is transformed into RDMA Write operations (see section 7.3.5). 2. An R2T PDU is a data-type PDU and is transformed into RDMA Read operations (see section 7.3.6). 3. An Asynchronous Message PDU is considered "unexpected", and the number that can be sent by the target falls under the MaxOutstandingUnexpectedPDU declared by the initiator. The PDU is outstanding until the initiator sends a control-type PDU with ExpStatSN set to X + 1 where X is the StatSN of the Asychronous Message PDU. 4. A Reject PDU is considered "unexpected", and the number that can be sent by the target falls under the MaxOutstandingUnexpectedPDU declared by the initiator. The PDU is outstanding until the initiator sends a control-type PDU with ExpStatSN set to X + 1 where X is the StatSN of the Reject PDU. 5. A NOP-in PDU sent as a response by the target to a NOP-out PDU from the initiator is expected by the initiator. Otherwise, a NOP-out PDU is considered "unexpected", and the number that can be sent by the target falls under the MaxOutstandingUnexpectedPDU declared by the initiator. The PDU is outstanding until the initiator sends a control-type PDU with ExpStatSN set to X + 1 where X is the StatSN of the NOP-in PDU. Mike _______________________________________________ Ips mailing list Ips@ietf.org https://www1.ietf.org/mailman/listinfo/ips _______________________________________________ Ips mailing list Ips@ietf.org https://www1.ietf.org/mailman/listinfo/ips _______________________________________________ Ips mailing list Ips@ietf.org https://www1.ietf.org/mailman/listinfo/ips From ips-bounces@ietf.org Sat Oct 16 02:34:25 2004 Received: from ietf-mx.ietf.org (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id CAA27126 for ; Sat, 16 Oct 2004 02:34:24 -0400 (EDT) Received: from megatron.ietf.org ([132.151.6.71]) by ietf-mx.ietf.org with esmtp (Exim 4.33) id 1CIiKb-0007w3-7e for ips-web-archive@ietf.org; Sat, 16 Oct 2004 02:46:21 -0400 Received: from localhost.localdomain ([127.0.0.1] helo=megatron.ietf.org) by megatron.ietf.org with esmtp (Exim 4.32) id 1CIi5M-0000q6-Gk; Sat, 16 Oct 2004 02:30:36 -0400 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by megatron.ietf.org with esmtp (Exim 4.32) id 1CIi3g-0008BX-8L for ips@megatron.ietf.org; Sat, 16 Oct 2004 02:28:52 -0400 Received: from ietf-mx.ietf.org (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id CAA26726 for ; Sat, 16 Oct 2004 02:28:47 -0400 (EDT) Received: from mtagate1.de.ibm.com ([195.212.29.150]) by ietf-mx.ietf.org with esmtp (Exim 4.33) id 1CIiF6-0007qP-Me for ips@ietf.org; Sat, 16 Oct 2004 02:40:44 -0400 Received: from d12nrmr1607.megacenter.de.ibm.com (d12nrmr1607.megacenter.de.ibm.com [9.149.167.49]) by mtagate1.de.ibm.com (8.12.10/8.12.10) with ESMTP id i9G6S1fQ088090; Sat, 16 Oct 2004 06:28:01 GMT Received: from d12ml102.megacenter.de.ibm.com (d12av02.megacenter.de.ibm.com [9.149.165.228]) by d12nrmr1607.megacenter.de.ibm.com (8.12.10/NCO/VER6.6) with ESMTP id i9G6S090058708; Sat, 16 Oct 2004 08:28:00 +0200 In-Reply-To: <200410142328.TAA19034@ietf.org> To: Allison Mankin Subject: Re: [Ips] IPS and RDDP meeting times in November MIME-Version: 1.0 X-Mailer: Lotus Notes Build V70_M2_07222004 Beta 2NP July 22, 2004 From: Julian Satran Message-ID: Date: Sat, 16 Oct 2004 08:27:57 +0200 X-MIMETrack: Serialize by Router on D12ML102/12/M/IBM(Release 6.5.1| March 5, 2004) at 16/10/2004 08:28:00, Serialize complete at 16/10/2004 08:28:00 X-Spam-Score: 0.5 (/) X-Scan-Signature: 1676547e4f33b5e63227e9c02bd359e3 Cc: pat_thaler@agilent.com, ips@ietf.org, mankin@psg.com, black_david@emc.com, elizabeth.rodriguez@dothill.com X-BeenThere: ips@ietf.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: IP Storage List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Content-Type: multipart/mixed; boundary="===============1152167864==" Sender: ips-bounces@ietf.org Errors-To: ips-bounces@ietf.org X-Spam-Score: 0.5 (/) X-Scan-Signature: f49c97ce49302a02285a2d36a99eef8c This is a multipart message in MIME format. --===============1152167864== Content-Type: multipart/alternative; boundary="=_alternative 001E99C8C2256F2F_=" This is a multipart message in MIME format. --=_alternative 001E99C8C2256F2F_= Content-Type: text/plain; charset="US-ASCII" Supercomputing is another one the IETF as a whole may not want to overlap with (this year it does). Julo Allison Mankin Sent by: ips-bounces@ietf.org 15/10/04 01:27 To pat_thaler@agilent.com cc ips@ietf.org, mankin@psg.com, black_david@emc.com, elizabeth.rodriguez@dothill.com Subject Re: [Ips] IPS and RDDP meeting times in November Pat, IPS folks, Your Chairs asked me to ensure that the WG can meet on Monday, and I've committed to making sure it stays there. About the problem with T10 meeting dates and IETF meeting dates, recurring in March 2005, one problem is that the T10 and T11 meetings haven't found their way onto past conflicts calendars... (sorry) There's a conflicts calendar under construction now, for as far out as we can go, so could people send me the important dates to avoid from the IPS point of view? Other meetings for the meeting planners to be aware of are welcome as well. There are no assurances of avoiding all the conflicts, but it helps to start with enough info. Allison _______________________________________________ Ips mailing list Ips@ietf.org https://www1.ietf.org/mailman/listinfo/ips --=_alternative 001E99C8C2256F2F_= Content-Type: text/html; charset="US-ASCII"
Supercomputing is another one the IETF as a whole may not want to overlap with (this year it does). Julo


Allison Mankin <mankin@psg.com>
Sent by: ips-bounces@ietf.org

15/10/04 01:27

To
pat_thaler@agilent.com
cc
ips@ietf.org, mankin@psg.com, black_david@emc.com, elizabeth.rodriguez@dothill.com
Subject
Re: [Ips] IPS and RDDP meeting times in November





Pat, IPS folks,

Your Chairs asked me to ensure that the WG can meet on Monday,
and I've committed to making sure it stays there.

About the problem with T10 meeting dates and IETF meeting dates,
recurring in March 2005, one problem is that the T10 and T11
meetings haven't found their way onto past conflicts calendars...

(sorry)

There's a conflicts calendar under construction now, for as far
out as we can go, so could people send me the important dates
to avoid from the IPS point of view?  Other meetings for the
meeting planners to be aware of are welcome as well.

There are no assurances of avoiding all the conflicts, but it
helps to start with enough info.

Allison

_______________________________________________
Ips mailing list
Ips@ietf.org
https://www1.ietf.org/mailman/listinfo/ips
--=_alternative 001E99C8C2256F2F_=-- --===============1152167864== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline Content-Transfer-Encoding: 7bit _______________________________________________ Ips mailing list Ips@ietf.org https://www1.ietf.org/mailman/listinfo/ips --===============1152167864==-- From ips-bounces@ietf.org Sat Oct 16 03:20:01 2004 Received: from ietf-mx.ietf.org (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id DAA28874 for ; Sat, 16 Oct 2004 03:20:01 -0400 (EDT) Received: from megatron.ietf.org ([132.151.6.71]) by ietf-mx.ietf.org with esmtp (Exim 4.33) id 1CIj2j-00006b-U9 for ips-web-archive@ietf.org; Sat, 16 Oct 2004 03:31:58 -0400 Received: from localhost.localdomain ([127.0.0.1] helo=megatron.ietf.org) by megatron.ietf.org with esmtp (Exim 4.32) id 1CIipf-0003Jp-BR; Sat, 16 Oct 2004 03:18:27 -0400 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by megatron.ietf.org with esmtp (Exim 4.32) id 1CIino-0002x5-6E for ips@megatron.ietf.org; Sat, 16 Oct 2004 03:16:32 -0400 Received: from ietf-mx.ietf.org (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id DAA28784 for ; Sat, 16 Oct 2004 03:16:30 -0400 (EDT) Received: from e34.co.us.ibm.com ([32.97.110.132]) by ietf-mx.ietf.org with esmtp (Exim 4.33) id 1CIizJ-0008VA-Hb for ips@ietf.org; Sat, 16 Oct 2004 03:28:27 -0400 Received: from westrelay04.boulder.ibm.com (westrelay04.boulder.ibm.com [9.17.193.32]) by e34.co.us.ibm.com (8.12.10/8.12.9) with ESMTP id i9G7FwJ8517996 for ; Sat, 16 Oct 2004 03:15:58 -0400 Received: from d03av02.boulder.ibm.com (d03av02.boulder.ibm.com [9.17.195.168]) by westrelay04.boulder.ibm.com (8.12.10/NCO/VER6.6) with ESMTP id i9G7FwOX113390 for ; Sat, 16 Oct 2004 01:15:58 -0600 Received: from d03av02.boulder.ibm.com (loopback [127.0.0.1]) by d03av02.boulder.ibm.com (8.12.11/8.12.11) with ESMTP id i9G7FwGY006902 for ; Sat, 16 Oct 2004 01:15:58 -0600 Received: from d03nm115.boulder.ibm.com (d03nm115.boulder.ibm.com [9.17.195.141]) by d03av02.boulder.ibm.com (8.12.11/8.12.11) with ESMTP id i9G7FwJC006899 for ; Sat, 16 Oct 2004 01:15:58 -0600 In-Reply-To: To: Mike Ko MIME-Version: 1.0 Subject: Re: [Ips] New iSCSI key X-Mailer: Lotus Notes Release 6.0.2CF1 June 9, 2003 From: John Hufferd Message-ID: Date: Sat, 16 Oct 2004 00:15:41 -0700 X-MIMETrack: Serialize by Router on D03NM115/03/M/IBM(Release 6.51HF338 | June 21, 2004) at 10/16/2004 01:15:57, Serialize complete at 10/16/2004 01:15:57 X-Spam-Score: 0.5 (/) X-Scan-Signature: 52005c17383d4e3f609075a067e11971 Cc: ips@ietf.org X-BeenThere: ips@ietf.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: IP Storage List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Content-Type: multipart/mixed; boundary="===============0869235917==" Sender: ips-bounces@ietf.org Errors-To: ips-bounces@ietf.org X-Spam-Score: 0.5 (/) X-Scan-Signature: 31c5f48314ee2074d17118e5d97ef478 This is a multipart message in MIME format. --===============0869235917== Content-Type: multipart/alternative; boundary="=_alternative 0014D66F88256F2F_=" This is a multipart message in MIME format. --=_alternative 0014D66F88256F2F_= Content-Type: text/plain; charset="US-ASCII" Comments below. . . John L. Hufferd Senior Technical Staff Member (STSM) IBM/System Group, San Jose CA Main Office: (408) 256-0403, Tie: 276-0403, eFax: (408) 904-4688 Alt Office: (408) 997-6136, Cell: (408) 499-9702 Internet Address: hufferd@us.ibm.com Mike Ko/Almaden/IBM 10/15/2004 05:34 PM To John Hufferd/San Jose/IBM@IBMUS cc ips@ietf.org Subject Re: [Ips] New iSCSI key John, The initiator should never see an R2T since an R2T is transformed into an RDMA Read Request by the target and is never sent as a control-type PDU. Neither the DA nor the iSER spec ever mentioned sending an R2T as a control-type PDU. That being the case, the initiator should never be sending SCSI Data-out PDUs that are "solicited" by an R2T. Conversely, a target will not be "expecting" control-type PDUs containing "solicited" data since it never sends an R2T as is. But since a SCSI Data-out PDU containing legitimate unsolicited data is sent as a control-type PDU, we allow the iSER layer to send *all* SCSI Data-out PDUs as control-type PDUs, even when they contain "solicited" data. That makes no since. It can not be solicited data if there is no way to solicit it. However in view of MaxOutstandingUnexpectedPDU, what happens if the limit is smaller than the amount of "solicited" data that the iSCSI layer wants to send? Again this makes no sense, because there is no way to solicit the data. Using ExpCmdSN to determine if the PDUs are outstanding or not might not work since the target might not advance ExpCmdSN frequent enough for all the "solicited" data to be transferred. If there is no way to solicit the data then there should be no fear about getting solicited data. We would need a different mechanism to replenish the credit. But sending "solicited" data as control-type PDUs is an error since it is not "solicited" by an R2T. Agreed Should we make life complicated in order to accommodate an error case? No! For SCSI Data-in and R2T, neither the DA nor the iSER spec mentioned allowing the target to send them as control-type PDUs. On SCSI Data-out PDUs, the DA spec mentioned the following: Section 7.1 "the iSCSI layer does not request a SCSI Data-Out PDU carrying solicited data for transmission across the Datamover Interface per this architecture" Section 7.1.1 "a SCSI Data-Out PDU for solicited data is never requested for transmission across the Datamover Interface at the initiator" Section 11.4.5.1 "the solicited SCSI Data-out arriving on the target is not notified to the iSCSI layer using the Control_Notify Primitive because the solicited SCSI Data-out was not sent by the initiator iSCSI layer as control-type PDUs" So to be consistent with the DA spec, I propose that we also disallow the sending of "solicitied" data as control-type PDUs. If we accept this correction, then the SCSI Data-out handling for MaxOutstandingUnexpectedPDUs can be simplified. Also the last paragraph in section 7.3.4 will be updated to remove the reference to allowing "solicited" data being sent as control-type PDUs. I believe the current wordage is simply a piece of left over text from an early approach, which we failed to remove. So yes, I agree that we should disallow the sending of "solicited" data as a control-type PDU. Comments? Mike From: John Hufferd on 10/15/2004 04:18 PM To: Mike Ko/Almaden/IBM cc: ips@ietf.org Subject: Re: [Ips] New iSCSI key Mike, I think we are getting closer, but since if Data-Outs ever gets solicited by an R2T, even though it shouldn't, it is by definition expected. So since the only other case for Data-Out PDUs is for unsolicited Data that can be anticipated, as you pointed out, via the FirstBurstLength, I just do not see Data-Out PDUs being in the same class as what we have been discussing, and think we should not even factor it into the unexpected classification. In other words just drop point 2. However, the text you have at the beginning of that paragraph, "Unsolicited data in a SCSI write operation are sent by the iSCSI layer at the initiator using SCSI Data-out PDUs. The iSER layer at the target can use FirstBurstLength to determine the amount of buffering required." should some how be placed somewhere else in the draft. . . John L. Hufferd Senior Technical Staff Member (STSM) IBM/System Group, San Jose CA Main Office: (408) 256-0403, Tie: 276-0403, eFax: (408) 904-4688 Alt Office: (408) 997-6136, Cell: (408) 499-9702 Internet Address: hufferd@us.ibm.com Mike Ko/Almaden/IBM 10/15/2004 01:23 PM To John Hufferd/San Jose/IBM@IBMUS cc ips@ietf.org Subject Re: [Ips] New iSCSI key John, Here is the replacement paragraph for the SCSI Data-out PDU: 2. Unsolicited data in a SCSI write operation are sent by the iSCSI layer at the initiator using SCSI Data-out PDUs. The iSER layer at the target can use FirstBurstLength to determine the amount of buffering required. For solicited data, an R2T PDU sent by the iSCSI layer at the target is transformed by the iSER layer at the target into an RDMA Read operation (see section 7.3.6). Therefore the iSCSI layer at the initiator should not be issuing SCSI Data-out PDUs for solicited data. If a SCSI Data-out PDU containing solicited data is requested for transmission by the iSCSI layer at the initiator anyway, the iSER layer may send it as a control-type PDU (see section 7.3.4). Since it is "unexpected", the number that can be sent by the initiator falls under the MaxOutstandingUnexpectedPDU declared by the target. The PDU is outstanding until the target responds with a control-type PDU with ExpCmdSN set to X + 1 where X is the CmdSN of the referenced command. Mike Sent by: ips-bounces@ietf.org To: John Hufferd/San Jose/IBM@IBMUS cc: ips@ietf.org Subject: Re: [Ips] New iSCSI key John, Would the following replacement text for the sentence "Solicited data are normally handled ..." better? "The R2T mechanism is normally used to handle solicited data and the SCSI Data-out PDU should not be used for this purpose (see section 7.3.4)." Mike From: John Hufferd on 10/15/2004 10:22 AM To: Mike Ko/Almaden/IBM cc: ips@ietf.org Subject: Re: [Ips] New iSCSI key Mike, I think you covered my concerns, but perhaps we should actually use words in the draft similar to what you have used here for the unsolicited Data-Out PDUs. I just thought that the words you proposed for the draft were a little bit confusing, since we never have talk about how an iSCSI layer requests Data-Out PDUs, etc. I think your words in this note were more to the point. . . John L. Hufferd Senior Technical Staff Member (STSM) IBM/System Group, San Jose CA Main Office: (408) 256-0403, Tie: 276-0403, eFax: (408) 904-4688 Alt Office: (408) 997-6136, Cell: (408) 499-9702 Internet Address: hufferd@us.ibm.com Mike Ko/Almaden/IBM 10/15/2004 10:01 AM To John Hufferd/San Jose/IBM@IBMUS cc ips@ietf.org Subject Re: [Ips] New iSCSI key John, My comments are embedded. Mike To: Mike Ko cc: ips@ietf.org Subject: Re: [Ips] New iSCSI key Mike, First I think there is a Typo near the end of your note when you talk about NOP-Out PDU where you might have meant Nop-In. That is, where you say "...Otherwise, a NOP-out PDU is considered "unexpected",...." perhaps you should say Nop-In. [mk[ Good catch. I will fix that. ]mk] Second I think we need to discuss the unsolicited Data-Out PDU. Please expand on the concept of the use of Data-Out PDU for solicited Data in iSER. [mk[ SCSI Data-out PDU should not be used for solicited data even though it is supported in the iSER draft. Since this is not a typical usage, the burden is left to the iSER/iSCSI layer to keep track of the number of "unexpected" PDU that it can send. What specifically do you want to expand on this topic? ]mk] Third we need to discuss or expand on considering the Reject PDU an unsolicited PDU. Perhaps you meant Rejects for PDUs that are not for the Command itself. [mk[ I will replace the phrase "A Reject PDU is considered "unexpected"" with the following: For a task which has never been active before the Reject PDU is sent, the target should not send any further responses because the command itself is discarded (see section 6.3 of RFC3720). Therefore, this type of Reject PDU is expecetd. All other Reject PDUs are "unexpected". ]mk] . . John L. Hufferd Senior Technical Staff Member (STSM) IBM/System Group, San Jose CA Main Office: (408) 256-0403, Tie: 276-0403, eFax: (408) 904-4688 Alt Office: (408) 997-6136, Cell: (408) 499-9702 Internet Address: hufferd@us.ibm.com Mike Ko Sent by: ips-bounces@ietf.org 10/14/2004 09:36 PM To cc Subject [Ips] New iSCSI key If we do decide on adding a new iSCSI key to limit the number of "unexpected" or "unanticipated" PDUs, I offer the following text to be considered for adding to section 6 of the iSER draft. Currently the key is tentatively called "MaxOutstandingUnexpectedPDUs". Unless there are other suggestions, this name will be used. Use: LO (leading only), Declarative Senders: Initiator and Target Scope: SW (session-wide) Irrelevant when: RDMAExtensions=No MaxOutstandingUnexpectedPDUs= Default is None This key is used by the initiator and the target to declare the maximum number of outstanding "unexpected" control-type PDUs that it can receive. It is intended to allow the receiving side to determine the amount of buffer resoruces needed beyond the normal flow control mechanism available in iSCSI. An initiator or target that cannot function properly without any limit on MaxOutstandingUnexpectedPDU (which is the default) should choose a value such that it would not impose an unnecessary constraint on the iSCSI layer under normal circumstances. The proper value to use is implementation dependent. If the sending side fails to adhere to the declared limit as set by the receiving side, it is up to the receiving side to determine ways of handling the overrun, including dropping the connection. For control-type PDUs sent by an initiator to a target, where the immediate flag is not set, the command numbering mechanism regulates the maximum number of PDUs that the target can expect. The exceptions include the SNACK PDU, the SCSI Data-out PDU, and any PDU with the immediate flag set. 1. The SNACK PDU is not needed in iSER (see section 7.3.11). But if one is requested for transmission by the iSCSI layer at the initiator, it will be sent by the iSER layer as a control-type PDU. Since it is "unexpected", the number that can be sent by the initiator falls under the MaxOutstandingUnexpectedPDU declared by the target. The PDU is outstanding until the target responds with a control-type PDU with ExpCmdSN set to X + 1 where X is the CmdSN of the referenced command. 2. For unsolicited SCSI Data-out PDU, the target can use FirstBurstLength to determine the amount of buffering required. Solicited data are normally handled using the R2T mechanism (see section 7.3.4). However, if a SCSI Data-out PDU containing solicited data is requested for transmission by the iSCSI layer at the initiator, the iSER layer may send it as unsolciited data. Since it is "unexpected", the number that can be sent by the initiator falls under the MaxOutstandingUnexpectedPDU declared by the target. The PDU is outstanding until the target responds with a control-type PDU with ExpCmdSN set to X + 1 where X is the CmdSN of the referenced command. 3. A PDU with the immediate flag set is considered "unexpected", and the number that can be sent by the initiator falls under the MaxOutstandingUnexpectedPDU declared by the target. The PDU is outstanding until the target responds with a control-type PDU with ExpCmdSN set to X + 1 where X is the CmdSN of the PDU with the immediate flag set. Control-type PDUs sent by a target as a response to a request from an initiator are expected by the initiator and buffer resources required can be provisioned. The exceptions include the SCSI Data-in PDU, the R2T PDU, the Asynchronous Message PDU, the Reject PDU, and the NOP-in PDU. 1. A SCSI Data-in PDU is a data-type PDU and is transformed into RDMA Write operations (see section 7.3.5). 2. An R2T PDU is a data-type PDU and is transformed into RDMA Read operations (see section 7.3.6). 3. An Asynchronous Message PDU is considered "unexpected", and the number that can be sent by the target falls under the MaxOutstandingUnexpectedPDU declared by the initiator. The PDU is outstanding until the initiator sends a control-type PDU with ExpStatSN set to X + 1 where X is the StatSN of the Asychronous Message PDU. 4. A Reject PDU is considered "unexpected", and the number that can be sent by the target falls under the MaxOutstandingUnexpectedPDU declared by the initiator. The PDU is outstanding until the initiator sends a control-type PDU with ExpStatSN set to X + 1 where X is the StatSN of the Reject PDU. 5. A NOP-in PDU sent as a response by the target to a NOP-out PDU from the initiator is expected by the initiator. Otherwise, a NOP-out PDU is considered "unexpected", and the number that can be sent by the target falls under the MaxOutstandingUnexpectedPDU declared by the initiator. The PDU is outstanding until the initiator sends a control-type PDU with ExpStatSN set to X + 1 where X is the StatSN of the NOP-in PDU. Mike _______________________________________________ Ips mailing list Ips@ietf.org https://www1.ietf.org/mailman/listinfo/ips _______________________________________________ Ips mailing list Ips@ietf.org https://www1.ietf.org/mailman/listinfo/ips --=_alternative 0014D66F88256F2F_= Content-Type: text/html; charset="US-ASCII"
Comments below.
.
.
John L. Hufferd
Senior Technical Staff Member (STSM)
IBM/System Group, San Jose CA
Main Office: (408) 256-0403, Tie: 276-0403, eFax: (408) 904-4688
Alt Office: (408) 997-6136, Cell: (408) 499-9702
Internet Address: hufferd@us.ibm.com


Mike Ko/Almaden/IBM

10/15/2004 05:34 PM

To
John Hufferd/San Jose/IBM@IBMUS
cc
ips@ietf.org
Subject
Re: [Ips] New iSCSI keyLink




John,

The initiator should never see an R2T since an R2T is transformed into an RDMA Read Request by the target and is never sent as a control-type PDU.  Neither the DA nor the iSER spec ever mentioned sending an R2T as a control-type PDU.  That being the case, the initiator should never be sending SCSI Data-out PDUs that are "solicited" by an R2T.  Conversely, a target will not be "expecting" control-type PDUs containing "solicited" data since it never sends an R2T as is.  But since a SCSI Data-out PDU containing legitimate unsolicited data is sent as a control-type PDU, we allow the iSER layer to send *all* SCSI Data-out PDUs as control-type PDUs, even when they contain "solicited" data.  
<Huff> That makes no since.  It can not be solicited data if there is no way to solicit it. <Huff>

However in view of MaxOutstandingUnexpectedPDU, what happens if the limit is smaller than the amount of "solicited" data that the iSCSI layer wants to send?  
<Huff> Again this makes no sense, because there is no way to solicit the data.  <Huff>

Using ExpCmdSN to determine if the PDUs are outstanding or not might not work since the target might not advance ExpCmdSN frequent enough for all the "solicited" data to be transferred.  
<Huff> If there is no way to solicit the data then there should be no fear about getting solicited data. <Huff>

We would need a different mechanism to replenish the credit.  But sending "solicited" data as control-type PDUs is an error since it is not "solicited" by an R2T.  
<Huff> Agreed <Huff>

Should we make life complicated in order to accommodate an error case?
<Huff> No! <Huff>

For SCSI Data-in and R2T, neither the DA nor the iSER spec mentioned allowing the target to send them as control-type PDUs.  On SCSI Data-out PDUs, the DA spec mentioned the following:

Section 7.1 "the iSCSI layer does not request a SCSI Data-Out PDU carrying solicited data for transmission across the Datamover Interface per this architecture"

Section 7.1.1 "a SCSI Data-Out PDU for solicited data is never requested for transmission across the Datamover Interface at the initiator"

Section 11.4.5.1 "the solicited SCSI Data-out arriving on the target is not notified to the iSCSI layer using the
Control_Notify Primitive because the solicited SCSI Data-out was not sent by the initiator iSCSI layer as control-type PDUs"

So to be consistent with the DA spec, I propose that we also disallow the sending of "solicitied" data as control-type PDUs.  If we accept this correction, then the SCSI Data-out handling for MaxOutstandingUnexpectedPDUs can be simplified.  Also the last paragraph in section 7.3.4 will be updated to remove the reference to allowing "solicited" data being sent as control-type PDUs.
<Huff> I believe the current wordage is simply a piece of left over text from an early approach, which we failed to remove.  So yes, I agree that we should disallow the sending of "solicited" data as a control-type PDU.

Comments?

Mike

From:        John Hufferd on 10/15/2004 04:18 PM

To:        Mike Ko/Almaden/IBM
cc:        ips@ietf.org
Subject:        Re: [Ips] New iSCSI key  Link

Mike,
I think we are getting closer, but since if Data-Outs ever gets solicited by an R2T, even though it shouldn't, it is by definition expected.  So since the only other case for Data-Out PDUs is for unsolicited Data that can be anticipated, as you pointed out, via the FirstBurstLength, I just do not see Data-Out PDUs being in the same class as what we have been discussing, and think we should not even factor it into the unexpected classification.  In other words just drop point 2.  However, the text you have at the beginning of that paragraph, "Unsolicited data in a SCSI write operation are sent by the iSCSI layer at the initiator using SCSI Data-out PDUs.  The iSER layer at the target can use
FirstBurstLength to determine the amount of buffering required."  should some how be placed somewhere else in the draft.
.
.
John L. Hufferd
Senior Technical Staff Member (STSM)
IBM/System Group, San Jose CA
Main Office: (408) 256-0403, Tie: 276-0403, eFax: (408) 904-4688
Alt Office: (408) 997-6136, Cell: (408) 499-9702
Internet Address: hufferd@us.ibm.com


Mike Ko/Almaden/IBM

10/15/2004 01:23 PM

To
John Hufferd/San Jose/IBM@IBMUS
cc
ips@ietf.org
Subject
Re: [Ips] New iSCSI keyLink




John,
Here is the replacement paragraph for the SCSI Data-out PDU:

2. Unsolicited data in a SCSI write operation are sent by the iSCSI layer at
the initiator using SCSI Data-out PDUs.  The iSER layer at the target can use
FirstBurstLength to determine the amount of buffering required.  For solicited
data, an R2T PDU sent by the iSCSI layer at the target is transformed by the
iSER layer at the target into an RDMA Read operation (see section 7.3.6).
Therefore the iSCSI layer at the initiator should not be issuing SCSI Data-out
PDUs for solicited data.  If a SCSI Data-out PDU containing solicited data is
requested for transmission by the iSCSI layer at the initiator anyway, the iSER
layer may send it as a control-type PDU (see section 7.3.4).  Since it is
"unexpected", the number that can be sent by the initiator falls under the
MaxOutstandingUnexpectedPDU declared by the target.  The PDU is outstanding
until the target responds with a control-type PDU with ExpCmdSN set to X + 1
where X is the CmdSN of the referenced command.

Mike

Sent by:        ips-bounces@ietf.org

To:        John Hufferd/San Jose/IBM@IBMUS
cc:        ips@ietf.org
Subject:        Re: [Ips] New iSCSI key



John,
Would the following replacement text for the sentence "Solicited data are
normally handled ..." better?
"The R2T mechanism is normally used to handle solicited data and the SCSI
Data-out PDU should not be used for this purpose (see section 7.3.4)."
Mike
From:   John Hufferd on 10/15/2004 10:22 AM
To:     Mike Ko/Almaden/IBM
cc:     ips@ietf.org
Subject:        Re: [Ips] New iSCSI key
Mike,
I think you covered my concerns, but perhaps we should actually use words
in the draft similar to what you have used here for the unsolicited
Data-Out PDUs.  I just thought that the words you proposed for the draft
were a little bit confusing, since we never have talk about how an iSCSI
layer requests Data-Out PDUs, etc.  I think your words in this note were
more to the point.
.
.
John L. Hufferd
Senior Technical Staff Member (STSM)
IBM/System Group, San Jose CA
Main Office: (408) 256-0403, Tie: 276-0403, eFax: (408) 904-4688
Alt Office: (408) 997-6136, Cell: (408) 499-9702
Internet Address: hufferd@us.ibm.com


Mike Ko/Almaden/IBM
10/15/2004 10:01 AM
To
John Hufferd/San Jose/IBM@IBMUS
cc
ips@ietf.org
Subject
Re: [Ips] New iSCSI key




John,
My comments are embedded.
Mike
To:     Mike Ko <mako@almaden.ibm.com>
cc:     ips@ietf.org
Subject:        Re: [Ips] New iSCSI key



Mike,
First I think there is a Typo near the end of your note when you talk
about NOP-Out PDU where you might have meant Nop-In.  That is, where you
say "...Otherwise, a NOP-out PDU is
considered "unexpected",...." perhaps you should say Nop-In.
[mk[
Good catch.  I will fix that.
]mk]
Second I think we need to discuss the unsolicited Data-Out PDU.  Please
expand on the concept of the use of Data-Out PDU for solicited Data in
iSER.
[mk[
SCSI Data-out PDU should not be used for solicited data even though it is
supported in the iSER draft.  Since this is not a typical usage, the
burden is left to the iSER/iSCSI layer to keep track of the number of
"unexpected" PDU that it can send.  What specifically do you want to
expand on this topic?
]mk]
Third we need to discuss or expand on considering the Reject PDU an
unsolicited PDU.   Perhaps you meant Rejects for PDUs that are not for the
Command itself.
[mk[
I will replace the phrase "A Reject PDU is considered "unexpected"" with
the following:
For a task which has never been active before the Reject PDU is sent, the
target should not send any further responses because the command itself is
discarded (see section 6.3 of RFC3720).  Therefore, this type of Reject
PDU is expecetd.  All other Reject PDUs are "unexpected".
]mk]
.
.
John L. Hufferd
Senior Technical Staff Member (STSM)
IBM/System Group, San Jose CA
Main Office: (408) 256-0403, Tie: 276-0403, eFax: (408) 904-4688
Alt Office: (408) 997-6136, Cell: (408) 499-9702
Internet Address: hufferd@us.ibm.com


Mike Ko <mako@almaden.ibm.com>
Sent by: ips-bounces@ietf.org
10/14/2004 09:36 PM
To
<ips@ietf.org>
cc
Subject
[Ips] New iSCSI key






If we do decide on adding a new iSCSI key to limit the number of
"unexpected"
or "unanticipated" PDUs, I offer the following text to be considered for
adding
to section 6 of the iSER draft.
Currently the key is tentatively called "MaxOutstandingUnexpectedPDUs".
Unless
there are other suggestions, this name will be used.
Use: LO (leading only), Declarative
Senders: Initiator and Target
Scope: SW (session-wide)
Irrelevant when: RDMAExtensions=No
MaxOutstandingUnexpectedPDUs=<numerical-value-from-0-to-(2**64-1) | None>
Default is None
This key is used by the initiator and the target to declare the maximum
number
of outstanding "unexpected" control-type PDUs that it can receive.  It is
intended to allow the receiving side to determine the amount of buffer
resoruces needed beyond the normal flow control mechanism available in
iSCSI.
An initiator or target that cannot function properly without any limit on
MaxOutstandingUnexpectedPDU (which is the default) should choose a value
such
that it would not impose an unnecessary constraint on the iSCSI layer
under
normal circumstances.  The proper value to use is implementation
dependent.  If
the sending side fails to adhere to the declared limit as set by the
receiving
side, it is up to the receiving side to determine ways of handling the
overrun,
including dropping the connection.
For control-type PDUs sent by an initiator to a target, where the
immediate
flag is not set, the command numbering mechanism regulates the maximum
number
of PDUs that the target can expect.  The exceptions include the SNACK PDU,
the
SCSI Data-out PDU, and any PDU with the immediate flag set.
1. The SNACK PDU is not needed in iSER (see section 7.3.11).  But if one
is
requested for transmission by the iSCSI layer at the initiator, it will be
sent
by the iSER layer as a control-type PDU.  Since it is "unexpected", the
number
that can be sent by the initiator falls under the
MaxOutstandingUnexpectedPDU
declared by the target.  The PDU is outstanding until the target responds
with
a control-type PDU with ExpCmdSN set to X + 1 where X is the CmdSN of the
referenced command.
2. For unsolicited SCSI Data-out PDU, the target can use FirstBurstLength
to
determine the amount of buffering required.  Solicited data are normally
handled using the R2T mechanism (see section 7.3.4).  However, if a SCSI
Data-out PDU containing solicited data is requested for transmission by
the
iSCSI layer at the initiator, the iSER layer may send it as unsolciited
data.
Since it is "unexpected", the number that can be sent by the initiator
falls
under the MaxOutstandingUnexpectedPDU declared by the target.  The PDU is
outstanding until the target responds with a control-type PDU with
ExpCmdSN set
to X + 1 where X is the CmdSN of the referenced command.
3. A PDU with the immediate flag set is considered "unexpected", and the
number
that can be sent by the initiator falls under the
MaxOutstandingUnexpectedPDU
declared by the target.  The PDU is outstanding until the target responds
with
a control-type PDU with ExpCmdSN set to X + 1 where X is the CmdSN of the
PDU
with the immediate flag set.
Control-type PDUs sent by a target as a response to a request from an
initiator
are expected by the initiator and buffer resources required can be
provisioned.
The exceptions include the SCSI Data-in PDU, the R2T PDU, the Asynchronous
Message PDU, the Reject PDU, and the NOP-in PDU.
1. A SCSI Data-in PDU is a data-type PDU and is transformed into RDMA
Write
operations (see section 7.3.5).
2. An R2T PDU is a data-type PDU and is transformed into RDMA Read
operations
(see section 7.3.6).
3. An Asynchronous Message PDU is considered "unexpected", and the number
that
can be sent by the target falls under the MaxOutstandingUnexpectedPDU
declared
by the initiator.  The PDU is outstanding until the initiator sends a
control-type PDU with ExpStatSN set to X + 1 where X is the StatSN of the
Asychronous Message PDU.
4. A Reject PDU is considered "unexpected", and the number that can be
sent by
the target falls under the MaxOutstandingUnexpectedPDU declared by the
initiator.  The PDU is outstanding until the initiator sends a
control-type PDU
with ExpStatSN set to X + 1 where X is the StatSN of the Reject PDU.
5. A NOP-in PDU sent as a response by the target to a NOP-out PDU from the
initiator is expected by the initiator.  Otherwise, a NOP-out PDU is
considered
"unexpected", and the number that can be sent by the target falls under
the
MaxOutstandingUnexpectedPDU declared by the initiator.  The PDU is
outstanding
until the initiator sends a control-type PDU with ExpStatSN set to X + 1
where
X is the StatSN of the NOP-in PDU.
Mike
_______________________________________________
Ips mailing list
Ips@ietf.org
https://www1.ietf.org/mailman/listinfo/ips




_______________________________________________
Ips mailing list
Ips@ietf.org
https://www1.ietf.org/mailman/listinfo/ips


--=_alternative 0014D66F88256F2F_=-- --===============0869235917== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline Content-Transfer-Encoding: 7bit _______________________________________________ Ips mailing list Ips@ietf.org https://www1.ietf.org/mailman/listinfo/ips --===============0869235917==-- From ips-bounces@ietf.org Mon Oct 18 15:11:27 2004 Received: from ietf-mx.ietf.org (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id PAA14235 for ; Mon, 18 Oct 2004 15:11:27 -0400 (EDT) Received: from megatron.ietf.org ([132.151.6.71]) by ietf-mx.ietf.org with esmtp (Exim 4.33) id 1CJd6p-0007Tl-I2 for ips-web-archive@ietf.org; Mon, 18 Oct 2004 15:23:56 -0400 Received: from localhost.localdomain ([127.0.0.1] helo=megatron.ietf.org) by megatron.ietf.org with esmtp (Exim 4.32) id 1CJcUf-0006su-60; Mon, 18 Oct 2004 14:44:29 -0400 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by megatron.ietf.org with esmtp (Exim 4.32) id 1CJc0L-0001ia-UL for ips@megatron.ietf.org; Mon, 18 Oct 2004 14:13:09 -0400 Received: from ietf-mx.ietf.org (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id OAA08474 for ; Mon, 18 Oct 2004 14:13:03 -0400 (EDT) Received: from palrel13.hp.com ([156.153.255.238]) by ietf-mx.ietf.org with esmtp (Exim 4.33) id 1CJcCH-0006AN-SL for ips@ietf.org; Mon, 18 Oct 2004 14:25:31 -0400 Received: from rosemail.rose.hp.com (rosemail.rose.hp.com [15.43.209.160]) by palrel13.hp.com (Postfix) with ESMTP id 4130A1C11CA4 for ; Mon, 18 Oct 2004 11:13:01 -0700 (PDT) Received: from [127.0.0.1] (dhcp-roseor-beb159.rose.hp.com [15.23.141.159]) by rosemail.rose.hp.com (Postfix) with ESMTP id 218CB809C for ; Mon, 18 Oct 2004 11:13:01 -0700 (PDT) Message-ID: <417407AA.2070309@rose.hp.com> Date: Mon, 18 Oct 2004 11:12:58 -0700 From: "Mallikarjun C." User-Agent: Mozilla Thunderbird 0.7.3 (Windows/20040803) X-Accept-Language: en-us, en MIME-Version: 1.0 To: ips@ietf.org Subject: Re: [Ips] New iSCSI key References: In-Reply-To: Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit X-Spam-Score: 0.0 (/) X-Scan-Signature: 64b72a8e61417554b4b727cb14e7034d Content-Transfer-Encoding: 7bit X-BeenThere: ips@ietf.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: IP Storage List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Sender: ips-bounces@ietf.org Errors-To: ips-bounces@ietf.org X-Spam-Score: 0.0 (/) X-Scan-Signature: 233de1b21593fc0f4cdf285406dca0da Content-Transfer-Encoding: 7bit I'm all for developing this option (this thread discusses #3 in the original list of options) but I really would like more feedback on other options as well. Developing this option is a good thing since doing so allows us to appreciate the (non)complexity of #3 compared to other candidates. With that caveat, a few comments on this thread. I agree that the current 7.3.4 discussion of the "sending solicited PDUs for data that was never solicited" case is just confusing. I recommend deleting the last two sentences in the last para of 7.3.4 (starting from "However..."). In the current architecture, if the TTT happens to be non-0xffffffff, it's an initiator code bug that will be suitably responded to by the target. If we do this, then I presume the #2 item in Mike's I->T list drops out. A general suggestion about all items is: factor out the repeating sentence to the top of the list ("Since it is "unexpected",....."). Recommend deleting #1 & #2 in the T->I list that simply explain data-type PDUs, but describe them as "exceptions" to the control-type PDU approach. I think this is confusing and besides, covered elsewhere. #4 on the T->I list discusses Reject. I presume this is for Rejects on non-command PDUs since Rejects on commands are "expected" in a way (at least the buffer room is available on the initiator). If you agree with this, suggest noting this "non-command" nature explicitly. I assume #5 is changing per John's comments. Thanks. -- Mallikarjun Mallikarjun Chadalapaka Networked Storage Architecture Network Storage Solutions Hewlett-Packard MS 5668 Roseville CA 95747 cbm [at] rose.hp.com John Hufferd wrote: > > Comments below. > . > . > John L. Hufferd > Senior Technical Staff Member (STSM) > IBM/System Group, San Jose CA > Main Office: (408) 256-0403, Tie: 276-0403, eFax: (408) 904-4688 > Alt Office: (408) 997-6136, Cell: (408) 499-9702 > Internet Address: hufferd@us.ibm.com > > > *Mike Ko/Almaden/IBM* > > 10/15/2004 05:34 PM > > > To > John Hufferd/San Jose/IBM@IBMUS > cc > ips@ietf.org > Subject > Re: [Ips] New iSCSI keyLink > > > > > > > > > John, > > The initiator should never see an R2T since an R2T is transformed into > an RDMA Read Request by the target and is never sent as a control-type > PDU. Neither the DA nor the iSER spec ever mentioned sending an R2T as > a control-type PDU. That being the case, the initiator should never be > sending SCSI Data-out PDUs that are "solicited" by an R2T. Conversely, > a target will not be "expecting" control-type PDUs containing > "solicited" data since it never sends an R2T as is. But since a SCSI > Data-out PDU containing legitimate unsolicited data is sent as a > control-type PDU, we allow the iSER layer to send *all* SCSI Data-out > PDUs as control-type PDUs, even when they contain "solicited" data. > That makes no since. It can not be solicited data if there is no > way to solicit it. > > However in view of MaxOutstandingUnexpectedPDU, what happens if the > limit is smaller than the amount of "solicited" data that the iSCSI > layer wants to send? > Again this makes no sense, because there is no way to solicit the > data. > > Using ExpCmdSN to determine if the PDUs are outstanding or not might not > work since the target might not advance ExpCmdSN frequent enough for all > the "solicited" data to be transferred. > If there is no way to solicit the data then there should be no > fear about getting solicited data. > > We would need a different mechanism to replenish the credit. But > sending "solicited" data as control-type PDUs is an error since it is > not "solicited" by an R2T. > Agreed > > Should we make life complicated in order to accommodate an error case? > No! > > For SCSI Data-in and R2T, neither the DA nor the iSER spec mentioned > allowing the target to send them as control-type PDUs. On SCSI Data-out > PDUs, the DA spec mentioned the following: > > Section 7.1 "the iSCSI layer does not request a SCSI Data-Out PDU > carrying solicited data for transmission across the Datamover Interface > per this architecture" > > Section 7.1.1 "a SCSI Data-Out PDU for solicited data is never requested > for transmission across the Datamover Interface at the initiator" > > Section 11.4.5.1 "the solicited SCSI Data-out arriving on the target is > not notified to the iSCSI layer using the > Control_Notify Primitive because the solicited SCSI Data-out was not > sent by the initiator iSCSI layer as control-type PDUs" > > So to be consistent with the DA spec, I propose that we also disallow > the sending of "solicitied" data as control-type PDUs. If we accept > this correction, then the SCSI Data-out handling for > MaxOutstandingUnexpectedPDUs can be simplified. Also the last paragraph > in section 7.3.4 will be updated to remove the reference to allowing > "solicited" data being sent as control-type PDUs. > I believe the current wordage is simply a piece of left over text > from an early approach, which we failed to remove. So yes, I agree that > we should disallow the sending of "solicited" data as a control-type PDU. > > Comments? > > Mike > > From: John Hufferd on 10/15/2004 04:18 PM > > To: Mike Ko/Almaden/IBM > cc: ips@ietf.org > Subject: Re: [Ips] New iSCSI key Link > > > > Mike, > I think we are getting closer, but since if Data-Outs ever gets > solicited by an R2T, even though it shouldn't, it is by definition > expected. So since the only other case for Data-Out PDUs is for > unsolicited Data that can be anticipated, as you pointed out, via the > FirstBurstLength, I just do not see Data-Out PDUs being in the same > class as what we have been discussing, and think we should not even > factor it into the unexpected classification. In other words just drop > point 2. However, the text you have at the beginning of that paragraph, > "Unsolicited data in a SCSI write operation are sent by the iSCSI layer > at the initiator using SCSI Data-out PDUs. The iSER layer at the target > can use > FirstBurstLength to determine the amount of buffering required." should > some how be placed somewhere else in the draft. > . > . > John L. Hufferd > Senior Technical Staff Member (STSM) > IBM/System Group, San Jose CA > Main Office: (408) 256-0403, Tie: 276-0403, eFax: (408) 904-4688 > Alt Office: (408) 997-6136, Cell: (408) 499-9702 > Internet Address: hufferd@us.ibm.com > > > *Mike Ko/Almaden/IBM* > > 10/15/2004 01:23 PM > > > To > John Hufferd/San Jose/IBM@IBMUS > cc > ips@ietf.org > Subject > Re: [Ips] New iSCSI keyLink > > > > > > > > > John, > Here is the replacement paragraph for the SCSI Data-out PDU: > > 2. Unsolicited data in a SCSI write operation are sent by the iSCSI > layer at > the initiator using SCSI Data-out PDUs. The iSER layer at the target > can use > FirstBurstLength to determine the amount of buffering required. For > solicited > data, an R2T PDU sent by the iSCSI layer at the target is transformed by > the > iSER layer at the target into an RDMA Read operation (see section 7.3.6). > Therefore the iSCSI layer at the initiator should not be issuing SCSI > Data-out > PDUs for solicited data. If a SCSI Data-out PDU containing solicited > data is > requested for transmission by the iSCSI layer at the initiator anyway, > the iSER > layer may send it as a control-type PDU (see section 7.3.4). Since it is > "unexpected", the number that can be sent by the initiator falls under the > MaxOutstandingUnexpectedPDU declared by the target. The PDU is outstanding > until the target responds with a control-type PDU with ExpCmdSN set to X > + 1 > where X is the CmdSN of the referenced command. > > Mike > > Sent by: ips-bounces@ietf.org > > To: John Hufferd/San Jose/IBM@IBMUS > cc: ips@ietf.org > Subject: Re: [Ips] New iSCSI key > > > > John, > > Would the following replacement text for the sentence "Solicited data are > normally handled ..." better? > "The R2T mechanism is normally used to handle solicited data and the SCSI > Data-out PDU should not be used for this purpose (see section 7.3.4)." > > Mike > From: John Hufferd on 10/15/2004 10:22 AM > > To: Mike Ko/Almaden/IBM > cc: ips@ietf.org > Subject: Re: [Ips] New iSCSI key > > Mike, > I think you covered my concerns, but perhaps we should actually use words > in the draft similar to what you have used here for the unsolicited > Data-Out PDUs. I just thought that the words you proposed for the draft > were a little bit confusing, since we never have talk about how an iSCSI > layer requests Data-Out PDUs, etc. I think your words in this note were > more to the point. > . > . > John L. Hufferd > Senior Technical Staff Member (STSM) > IBM/System Group, San Jose CA > Main Office: (408) 256-0403, Tie: 276-0403, eFax: (408) 904-4688 > Alt Office: (408) 997-6136, Cell: (408) 499-9702 > Internet Address: hufferd@us.ibm.com > > > > Mike Ko/Almaden/IBM > 10/15/2004 10:01 AM > > To > John Hufferd/San Jose/IBM@IBMUS > cc > ips@ietf.org > Subject > Re: [Ips] New iSCSI key > > > > > > John, > > My comments are embedded. > > Mike > To: Mike Ko > cc: ips@ietf.org > Subject: Re: [Ips] New iSCSI key > > > > > Mike, > First I think there is a Typo near the end of your note when you talk > about NOP-Out PDU where you might have meant Nop-In. That is, where you > say "...Otherwise, a NOP-out PDU is > considered "unexpected",...." perhaps you should say Nop-In. > > [mk[ > Good catch. I will fix that. > ]mk] > > Second I think we need to discuss the unsolicited Data-Out PDU. Please > expand on the concept of the use of Data-Out PDU for solicited Data in > iSER. > > [mk[ > SCSI Data-out PDU should not be used for solicited data even though it is > supported in the iSER draft. Since this is not a typical usage, the > burden is left to the iSER/iSCSI layer to keep track of the number of > "unexpected" PDU that it can send. What specifically do you want to > expand on this topic? > ]mk] > > Third we need to discuss or expand on considering the Reject PDU an > unsolicited PDU. Perhaps you meant Rejects for PDUs that are not for the > Command itself. > > [mk[ > I will replace the phrase "A Reject PDU is considered "unexpected"" with > the following: > For a task which has never been active before the Reject PDU is sent, the > target should not send any further responses because the command itself is > discarded (see section 6.3 of RFC3720). Therefore, this type of Reject > PDU is expecetd. All other Reject PDUs are "unexpected". > ]mk] > . > . > John L. Hufferd > Senior Technical Staff Member (STSM) > IBM/System Group, San Jose CA > Main Office: (408) 256-0403, Tie: 276-0403, eFax: (408) 904-4688 > Alt Office: (408) 997-6136, Cell: (408) 499-9702 > Internet Address: hufferd@us.ibm.com > > > > Mike Ko > Sent by: ips-bounces@ietf.org > > 10/14/2004 09:36 PM > > To > > cc > > Subject > [Ips] New iSCSI key > > > > > > > > If we do decide on adding a new iSCSI key to limit the number of > "unexpected" > or "unanticipated" PDUs, I offer the following text to be considered for > adding > to section 6 of the iSER draft. > > Currently the key is tentatively called "MaxOutstandingUnexpectedPDUs". > Unless > there are other suggestions, this name will be used. > > Use: LO (leading only), Declarative > Senders: Initiator and Target > Scope: SW (session-wide) > Irrelevant when: RDMAExtensions=No > MaxOutstandingUnexpectedPDUs= > Default is None > > This key is used by the initiator and the target to declare the maximum > number > of outstanding "unexpected" control-type PDUs that it can receive. It is > intended to allow the receiving side to determine the amount of buffer > resoruces needed beyond the normal flow control mechanism available in > iSCSI. > An initiator or target that cannot function properly without any limit on > MaxOutstandingUnexpectedPDU (which is the default) should choose a value > such > that it would not impose an unnecessary constraint on the iSCSI layer > under > normal circumstances. The proper value to use is implementation > dependent. If > the sending side fails to adhere to the declared limit as set by the > receiving > side, it is up to the receiving side to determine ways of handling the > overrun, > including dropping the connection. > > For control-type PDUs sent by an initiator to a target, where the > immediate > flag is not set, the command numbering mechanism regulates the maximum > number > of PDUs that the target can expect. The exceptions include the SNACK PDU, > > the > SCSI Data-out PDU, and any PDU with the immediate flag set. > > 1. The SNACK PDU is not needed in iSER (see section 7.3.11). But if one > is > requested for transmission by the iSCSI layer at the initiator, it will be > > sent > by the iSER layer as a control-type PDU. Since it is "unexpected", the > number > that can be sent by the initiator falls under the > MaxOutstandingUnexpectedPDU > declared by the target. The PDU is outstanding until the target responds > with > a control-type PDU with ExpCmdSN set to X + 1 where X is the CmdSN of the > referenced command. > > 2. For unsolicited SCSI Data-out PDU, the target can use FirstBurstLength > to > determine the amount of buffering required. Solicited data are normally > handled using the R2T mechanism (see section 7.3.4). However, if a SCSI > Data-out PDU containing solicited data is requested for transmission by > the > iSCSI layer at the initiator, the iSER layer may send it as unsolciited > data. > Since it is "unexpected", the number that can be sent by the initiator > falls > under the MaxOutstandingUnexpectedPDU declared by the target. The PDU is > outstanding until the target responds with a control-type PDU with > ExpCmdSN set > to X + 1 where X is the CmdSN of the referenced command. > > 3. A PDU with the immediate flag set is considered "unexpected", and the > number > that can be sent by the initiator falls under the > MaxOutstandingUnexpectedPDU > declared by the target. The PDU is outstanding until the target responds > with > a control-type PDU with ExpCmdSN set to X + 1 where X is the CmdSN of the > PDU > with the immediate flag set. > > Control-type PDUs sent by a target as a response to a request from an > initiator > are expected by the initiator and buffer resources required can be > provisioned. > The exceptions include the SCSI Data-in PDU, the R2T PDU, the Asynchronous > Message PDU, the Reject PDU, and the NOP-in PDU. > > 1. A SCSI Data-in PDU is a data-type PDU and is transformed into RDMA > Write > operations (see section 7.3.5). > > 2. An R2T PDU is a data-type PDU and is transformed into RDMA Read > operations > (see section 7.3.6). > > 3. An Asynchronous Message PDU is considered "unexpected", and the number > that > can be sent by the target falls under the MaxOutstandingUnexpectedPDU > declared > by the initiator. The PDU is outstanding until the initiator sends a > control-type PDU with ExpStatSN set to X + 1 where X is the StatSN of the > Asychronous Message PDU. > > 4. A Reject PDU is considered "unexpected", and the number that can be > sent by > the target falls under the MaxOutstandingUnexpectedPDU declared by the > initiator. The PDU is outstanding until the initiator sends a > control-type PDU > with ExpStatSN set to X + 1 where X is the StatSN of the Reject PDU. > > 5. A NOP-in PDU sent as a response by the target to a NOP-out PDU from the > initiator is expected by the initiator. Otherwise, a NOP-out PDU is > considered > "unexpected", and the number that can be sent by the target falls under > the > MaxOutstandingUnexpectedPDU declared by the initiator. The PDU is > outstanding > until the initiator sends a control-type PDU with ExpStatSN set to X + 1 > where > X is the StatSN of the NOP-in PDU. > > Mike > > _______________________________________________ > Ips mailing list > Ips@ietf.org > https://www1.ietf.org/mailman/listinfo/ips > > > > > > _______________________________________________ > Ips mailing list > Ips@ietf.org > https://www1.ietf.org/mailman/listinfo/ips > > > > ------------------------------------------------------------------------ > > _______________________________________________ > Ips mailing list > Ips@ietf.org > https://www1.ietf.org/mailman/listinfo/ips _______________________________________________ Ips mailing list Ips@ietf.org https://www1.ietf.org/mailman/listinfo/ips From ips-bounces@ietf.org Mon Oct 18 15:54:38 2004 Received: from ietf-mx.ietf.org (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id PAA17722 for ; Mon, 18 Oct 2004 15:54:38 -0400 (EDT) Received: from megatron.ietf.org ([132.151.6.71]) by ietf-mx.ietf.org with esmtp (Exim 4.33) id 1CJdma-0008RO-Rx for ips-web-archive@ietf.org; Mon, 18 Oct 2004 16:07:08 -0400 Received: from localhost.localdomain ([127.0.0.1] helo=megatron.ietf.org) by megatron.ietf.org with esmtp (Exim 4.32) id 1CJd3l-0006ii-1F; Mon, 18 Oct 2004 15:20:45 -0400 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by megatron.ietf.org with esmtp (Exim 4.32) id 1CJcjP-0003qg-Sm for ips@megatron.ietf.org; Mon, 18 Oct 2004 14:59:43 -0400 Received: from ietf-mx.ietf.org (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id OAA12770 for ; Mon, 18 Oct 2004 14:59:37 -0400 (EDT) Received: from sccrmhc11.comcast.net ([204.127.202.55]) by ietf-mx.ietf.org with esmtp (Exim 4.33) id 1CJcvN-0007CU-Mc for ips@ietf.org; Mon, 18 Oct 2004 15:12:06 -0400 Received: from ivvt2dxrc11 (unknown[64.238.111.98]) by comcast.net (sccrmhc11) with SMTP id <20041018185907011006arl1e> (Authid: esquicksall); Mon, 18 Oct 2004 18:59:07 +0000 Message-ID: <000601c4b544$8bface90$ea01a8c0@ivvt2dxrc11> From: "Eddy Quicksall" To: "Julian Satran" References: Subject: Re: [Ips] PDUs allowed on discovery sessions Date: Mon, 18 Oct 2004 14:59:07 -0400 MIME-Version: 1.0 X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 6.00.2900.2180 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2900.2180 X-Spam-Score: 0.6 (/) X-Scan-Signature: fca741f5016e6ff607eaed2fd431d10d Cc: ips@ietf.org X-BeenThere: ips@ietf.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: IP Storage List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Content-Type: multipart/mixed; boundary="===============1537936942==" Sender: ips-bounces@ietf.org Errors-To: ips-bounces@ietf.org X-Spam-Score: 0.6 (/) X-Scan-Signature: b058151374d77ee76edaac850f7449fb This is a multi-part message in MIME format. --===============1537936942== Content-Type: multipart/alternative; boundary="----=_NextPart_000_0003_01C4B523.04827D80" This is a multi-part message in MIME format. ------=_NextPart_000_0003_01C4B523.04827D80 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable Given what you said below, in a long lived discovery, the initiator and = target can still ping each other to verify that the connection is still = active; except that the target will always "answer" the ping with a = reject. Is that correct? Eddy ----- Original Message -----=20 From: Julian Satran=20 To: Dean Scoville=20 Cc: ips@ietf.org=20 Sent: Wednesday, September 01, 2004 1:05 PM Subject: RE: [Ips] PDUs allowed on discovery sessions Dean - that is the least it should do. I think that a target = implementation is also free to close the session if it is "abused" = (avoid a DOS attack).=20 Julo=20 "Dean Scoville" =20 01/09/04 17:14=20 To Julian Satran/Haifa/IBM@IBMIL =20 cc =20 Subject RE: [Ips] PDUs allowed on discovery sessions=20 =20 =20 Julian,=20 Thanks for the correction. Just so I understand, when the target=20 rejects other PDUs... does it do so by sending a Reject PDU?=20 =20 =20 Thank you.=20 Dean=20 =20 -----Original Message----- From: Julian Satran [mailto:Julian_Satran@il.ibm.com] Sent: Tuesday, August 31, 2004 11:59 PM To: Dean Scoville Cc: ips@ietf.org Subject: Re: [Ips] PDUs allowed on discovery sessions Dean,=20 ips-bounces@ietf.org wrote on 31/08/2004 19:39:50: > Section 3.3: >=20 > b) Discovery-session - a session only opened for target=20 > discovery. The target MUST ONLY accept text requests with the > SendTargets key and a logout request with the reason "close=20 > the session". All other requests MUST be rejected. >=20 > Section 12.21: >=20 > A discovery session indicates to the Target that the only purpose = of > this Session is discovery. The only requests a target accepts in > this type of session are a text request with the SendTargets key = and a > logout request with reason "close the session". >=20 > I originally interpreted the above statements to mean that PDU types > such as NOPs > could not be sent on discovery sessions. After reading more = closely,the above > statements only limit what the target is allowed to received (not = PDUs the=20 > target can send--such as Asynchronous Messages), and if NOPs aren't = considered > requests, then the initiator can send them as well. >=20 Your interpretation is too liberal.=20 NOPs are forbidden (there was a long discussion about allowing or not = long lived discovery sessions as a means of "discovering" changes and = the consensus was not to support them. You may argue that a target may = send NOPs (and the text does not forbid it) but it MUST reject incoming = NOPs.=20 AM was left in as it is required for "logout signals" and if you are = sneaky you can use it to signal changes as well.=20 All other PDUs are expressly forbidden.=20 > Is it correct to say that the following PDUs are allowed/disallowed > on discovery sessions?=20 >=20 > PDUs allowed on discovery sessions:=20 > - Asynchronous Message > - SendTargets Text Request/Response > - Login Request/Response > - Logout Request/Response with reason "close the session" > - Reject > - NOP-Out/NOP-In >=20 > PDUs not allowed on discovery sessions: =20 > - SCSI Command/Response > - Task Management Function Request/Response > - SCSI Data-Out/Data-In > - non-SendTargets Text Request/Response > - Ready To Transfer > - Logout Request/Response with reason other than "close the = session" > - SNACK Request >=20 >=20 > thanks, >=20 > Dean Scoville >=20 > _______________________________________________ > Ips mailing list > Ips@ietf.org > https://www1.ietf.org/mailman/listinfo/ips=20 -------------------------------------------------------------------------= ----- _______________________________________________ Ips mailing list Ips@ietf.org https://www1.ietf.org/mailman/listinfo/ips ------=_NextPart_000_0003_01C4B523.04827D80 Content-Type: text/html; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable

Given what you said below, in a long lived = discovery, the=20 initiator and target can still ping each other to verify that the = connection is=20 still active; except that the target will always "answer" the ping with = a=20 reject.
 
Is that correct?
 
Eddy
----- Original Message -----
From:=20 Julian Satran
Sent: Wednesday, September 01, = 2004 1:05=20 PM
Subject: RE: [Ips] PDUs allowed = on=20 discovery sessions


Dean - that is the least it should do. I = think that a=20 target implementation is also free to close the session if it is = "abused"=20 (avoid a DOS attack).

Julo=20


"Dean = Scoville" <dean.scoville@qlogic.com>= =20

01/09/04 17:14

To
Julian Satran/Haifa/IBM@IBMIL =
cc
<ips@ietf.org>=20
Subject
RE: [Ips] PDUs allowed = on=20 discovery sessions

=




<= FONT=20 face=3D"Courier New" size=3D2>Julian,
Thanks for the correction. Just so I understand, when the = target=20
rejects other PDUs... does it = do so by=20 sending a Reject PDU?
  =
 
Thank = you.=20
Dean
 =20
-----Original = Message-----
From: Julian=20 Satran [mailto:Julian_Satran@il.ibm.com]
Sent: Tuesday, = August 31,=20 2004 11:59 PM
To: Dean Scoville
Cc:=20 ips@ietf.org
Subject: Re: [Ips] PDUs allowed on discovery=20 sessions


Dean,

ips-bounces@ietf.org wrote = on=20 31/08/2004 19:39:50:

> Section 3.3:
>
>   =  b)=20 Discovery-session - a session only opened for target
>   =  =20   discovery. The target MUST ONLY accept text requests with = the
>=20       SendTargets key and a logout request with the = reason=20 "close
>       the session". All other requests = MUST be=20 rejected.
>
> Section 12.21:
>
>   =  A=20 discovery session indicates to the Target that the only purpose = of
>=20    this Session is discovery. The only requests a target = accepts=20 in
>    this type of session are a text request with = the=20 SendTargets key and a
>    logout request with reason = "close=20 the session".
>
> I originally interpreted the above = statements=20 to mean that PDU types
> such as NOPs
> could not be sent = on=20 discovery sessions. After reading more closely,the above
> = statements=20 only limit what the target is allowed to received (not PDUs the =
>=20 target can send--such as Asynchronous Messages), and if NOPs aren't=20 considered
> requests, then the initiator can send them as = well.
>=20

Your = interpretation=20 is too liberal.
NOPs are=20 forbidden (there was a long discussion about allowing or not long = lived=20 discovery sessions as a means of "discovering" changes and the = consensus was=20 not to support them. You may argue that a target may send NOPs (and = the text=20 does not forbid it) but it MUST reject incoming NOPs.=20

AM was left in as it is required for = "logout=20 signals" and if you are sneaky you can use it to signal changes as=20 well.

All = other PDUs=20 are expressly forbidden.

> Is it correct to say that the following PDUs = are=20 allowed/disallowed
> on discovery sessions?
>
> = PDUs=20 allowed on discovery sessions:
>    - Asynchronous=20 Message
>    - SendTargets Text = Request/Response
>=20    - Login Request/Response
>    - Logout=20 Request/Response with reason "close the session"
>   =  -=20 Reject
>    - NOP-Out/NOP-In
>
> PDUs not = allowed=20 on discovery sessions:  
>    - SCSI=20 Command/Response
>    - Task Management Function=20 Request/Response
>    - SCSI Data-Out/Data-In
> =  =20  - non-SendTargets Text Request/Response
>    - = Ready To=20 Transfer
>    - Logout Request/Response with reason = other than=20 "close the session"
>    - SNACK Request
> =
>=20
> thanks,
>
> Dean Scoville
>
>=20 _______________________________________________
> Ips mailing=20 list
> Ips@ietf.org
>=20 https://www1.ietf.org/mailman/listinfo/ips

_______________________________________________
Ips mailing=20 = list
Ips@ietf.org
https://www1.ietf.org/mailman/listinfo/ips
------=_NextPart_000_0003_01C4B523.04827D80-- --===============1537936942== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline Content-Transfer-Encoding: 7bit _______________________________________________ Ips mailing list Ips@ietf.org https://www1.ietf.org/mailman/listinfo/ips --===============1537936942==-- From ips-bounces@ietf.org Mon Oct 18 17:54:53 2004 Received: from ietf-mx.ietf.org (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id RAA27719 for ; Mon, 18 Oct 2004 17:54:53 -0400 (EDT) Received: from megatron.ietf.org ([132.151.6.71]) by ietf-mx.ietf.org with esmtp (Exim 4.33) id 1CJff1-0002fO-Jf for ips-web-archive@ietf.org; Mon, 18 Oct 2004 18:07:24 -0400 Received: from localhost.localdomain ([127.0.0.1] helo=megatron.ietf.org) by megatron.ietf.org with esmtp (Exim 4.32) id 1CJewo-0006Rf-3d; Mon, 18 Oct 2004 17:21:42 -0400 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by megatron.ietf.org with esmtp (Exim 4.32) id 1CJeeD-0002jC-GT for ips@megatron.ietf.org; Mon, 18 Oct 2004 17:02:29 -0400 Received: from ietf-mx.ietf.org (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id RAA23078 for ; Mon, 18 Oct 2004 17:02:21 -0400 (EDT) Received: from sadr.equallogic.com ([66.155.203.134]) by ietf-mx.ietf.org with smtp (Exim 4.33) id 1CJeqB-0001SM-1u for ips@ietf.org; Mon, 18 Oct 2004 17:14:52 -0400 Received: from sadr.equallogic.com (localhost.localdomain [127.0.0.1]) by sadr.equallogic.com (8.12.8/8.12.8) with ESMTP id i9IL1l2D029844 for ; Mon, 18 Oct 2004 17:01:47 -0400 Received: from M30.equallogic.com (m30 [172.16.1.30]) by sadr.equallogic.com (8.12.8/8.12.8) with SMTP id i9IL1lqn029839; Mon, 18 Oct 2004 17:01:47 -0400 Received: from PKONING.equallogic.com ([172.16.1.105]) by M30.equallogic.com with Microsoft SMTPSVC(5.0.2195.6713); Mon, 18 Oct 2004 17:01:46 -0400 MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Message-ID: <16756.12089.569000.413737@gargle.gargle.HOWL> Date: Mon, 18 Oct 2004 17:01:45 -0400 From: Paul Koning To: eddy_quicksall_iVivity_iSCSI@comcast.net Subject: Re: [Ips] PDUs allowed on discovery sessions References: <000601c4b544$8bface90$ea01a8c0@ivvt2dxrc11> X-Mailer: VM 7.07 under 21.4 (patch 10) "Military Intelligence (Windows)" XEmacs Lucid X-OriginalArrivalTime: 18 Oct 2004 21:01:46.0840 (UTC) FILETIME=[ADF24580:01C4B555] X-Spam-Score: 0.0 (/) X-Scan-Signature: 8b30eb7682a596edff707698f4a80f7d Content-Transfer-Encoding: 7bit Cc: ips@ietf.org, Julian_Satran@il.ibm.com X-BeenThere: ips@ietf.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: IP Storage List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Sender: ips-bounces@ietf.org Errors-To: ips-bounces@ietf.org X-Spam-Score: 0.0 (/) X-Scan-Signature: 21c69d3cfc2dd19218717dbe1d974352 Content-Transfer-Encoding: 7bit >>>>> "Eddy" == Eddy Quicksall writes: Eddy> Given what you said below, in a long lived discovery, the Eddy> initiator and target can still ping each other to verify that Eddy> the connection is still active; except that the target will Eddy> always "answer" the ping with a reject. Is that correct? That doesn't sound like what Julian said, and I wouldn't have said it. If the protocol prohibits the sending of PDU type X, it is wrong to assume that a node will always respond to such illegal PDUs in the way you wanted. If you want a fixed well-defined answer to a PDU, that PDU would be defined as allowed -- not as prohibited -- with a fixed action on receipt. If you're committing a protocol violation, don't be surprised to see your connection summarily deleted. paul >> Eddy ----- Original Message ----- From: Julian Satran To: Dean >> Scoville Cc: ips@ietf.org Sent: Wednesday, September 01, 2004 >> 1:05 PM Subject: RE: [Ips] PDUs allowed on discovery sessions >> Dean - that is the least it should do. I think that a target >> implementation is also free to close the session if it is >> "abused" (avoid a DOS attack). >> Julo >> "Dean Scoville" 01/09/04 17:14 To >> Julian Satran/Haifa/IBM@IBMIL cc Subject RE: >> [Ips] PDUs allowed on discovery sessions >>> Julian, Thanks for the correction. Just so I understand, when >>> the target rejects other PDUs... does it do so by sending a >>> Reject PDU? >>> Thank you. Dean _______________________________________________ Ips mailing list Ips@ietf.org https://www1.ietf.org/mailman/listinfo/ips From ips-bounces@ietf.org Mon Oct 18 18:52:12 2004 Received: from ietf-mx.ietf.org (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id SAA02769 for ; Mon, 18 Oct 2004 18:52:12 -0400 (EDT) Received: from megatron.ietf.org ([132.151.6.71]) by ietf-mx.ietf.org with esmtp (Exim 4.33) id 1CJgYV-0003nu-Oj for ips-web-archive@ietf.org; Mon, 18 Oct 2004 19:04:44 -0400 Received: from localhost.localdomain ([127.0.0.1] helo=megatron.ietf.org) by megatron.ietf.org with esmtp (Exim 4.32) id 1CJfvD-0007sU-R5; Mon, 18 Oct 2004 18:24:07 -0400 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by megatron.ietf.org with esmtp (Exim 4.32) id 1CJfeh-0000WJ-Is for ips@megatron.ietf.org; Mon, 18 Oct 2004 18:07:03 -0400 Received: from ietf-mx.ietf.org (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id SAA28949 for ; Mon, 18 Oct 2004 18:06:56 -0400 (EDT) Received: from e5.ny.us.ibm.com ([32.97.182.105]) by ietf-mx.ietf.org with esmtp (Exim 4.33) id 1CJfqg-0002xA-Eg for ips@ietf.org; Mon, 18 Oct 2004 18:19:27 -0400 Received: from d01relay02.pok.ibm.com (d01relay02.pok.ibm.com [9.56.227.234]) by e5.ny.us.ibm.com (8.12.10/8.12.9) with ESMTP id i9IM6PpA230922; Mon, 18 Oct 2004 18:06:25 -0400 Received: from d01ml604.pok.ibm.com (d01av04.pok.ibm.com [9.56.224.64]) by d01relay02.pok.ibm.com (8.12.10/NCO/VER6.6) with ESMTP id i9IM6OR4249068; Mon, 18 Oct 2004 18:06:24 -0400 In-Reply-To: <417407AA.2070309@rose.hp.com> Importance: Normal MIME-Version: 1.0 To: "Mallikarjun C." Subject: Re: [Ips] New iSCSI key X-Mailer: Lotus Notes Release 6.5.2 June 01, 2004 Message-ID: From: Mike Ko Date: Mon, 18 Oct 2004 15:06:23 -0700 X-MIMETrack: Serialize by Router on D01ML604/01/M/IBM(Release 6.51HF645 | October 15, 2004) at 10/18/2004 18:06:24, Serialize complete at 10/18/2004 18:06:24 Content-Type: text/plain; charset="US-ASCII" X-Spam-Score: 0.0 (/) X-Scan-Signature: 3fbd9b434023f8abfcb1532abaec7a21 Cc: ips@ietf.org X-BeenThere: ips@ietf.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: IP Storage List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Sender: ips-bounces@ietf.org Errors-To: ips-bounces@ietf.org X-Spam-Score: 0.0 (/) X-Scan-Signature: 20f22c03b5c66958bff5ef54fcda6e48 Mallikarjun, Here is the revised write up for the new key based on feedback from you and John. Note that there are also minor changes not covered by the feedback: 1. The "unexpected" PDU is outstanding until ExpStatSN is *at least* X + 1 where X is the StatSN of the "unexpected" PDU (was "until ExpStatSN is X + 1"). 2. A SNACK PDU is outstanding until the target sends a SCSI Response PDU for the referenced command. Start of key description: ----------- Use: LO (leading only), Declarative Senders: Initiator and Target Scope: SW (session-wide) Irrelevant when: RDMAExtensions=No MaxOutstandingUnexpectedPDUs= Default is None This key is used by the initiator and the target to declare the maximum number of outstanding "unexpected" control-type PDUs that it can receive. It is intended to allow the receiving side to determine the amount of buffer resources needed beyond the normal flow control mechanism available in iSCSI. An initiator or target that cannot function properly without any limit on MaxOutstandingUnexpectedPDUs (which is the default) should choose a value such that it would not impose an unnecessary constraint on the iSCSI layer under normal circumstances. The proper value to use is implementation dependent. If the sending side fails to adhere to the declared limit as set by the receiving side, it is up to the receiving side to determine ways of handling the overrun, including dropping the connection. For control-type PDUs sent by an initiator to a target, where the immediate flag is not set, the command numbering mechanism regulates the maximum number of PDUs that the target can expect. For iSCSI Data-out PDUs which are used by the iSCSI layer at the initiator to send unsolicited data, the iSER layer at the target can use FirstBurstLength to determine the amount of buffering required. For the "unexpected" PDUs, the MaxOutstandingUnexpectedPDUs key declared by the target is used to limit the number of oustanding "unexpected" control-type PDUs which can be sent by the initiator. The "unexpected" PDUs include the following: 1. A PDU with the immediate flag set. The PDU is outstanding until the target responds with a control-type PDU with ExpCmdSN set to at least X + 1 where X is the CmdSN of the PDU with the immediate flag set. 2. A SNACK PDU. Note that SNACK PDUs are not needed in iSER (see section 7.3.11). A SNACK PDU which is sent anyway by the initiator is outstanding until the target responds with a SCSI Response PDU for the referenced command. Control-type PDUs sent by a target as a response to a request from an initiator are expected by the initiator and buffer resources required can be provisioned. Note that the SCSI Data-in PDU and the R2T PDU are data-type PDUs (see section 7.1). For the "unexpected" control-type PDUs, the MaxOutstandingUnexpectedPDUs key declared by the initiator is used to limit the number of outstanding "unexpected" control-type PDUs which can be sent by the target. An "unexpected" PDU is outstanding until the initiator sends a control-type PDU with ExpStatSN set to at least X + 1 where X is the StatSN of the "unexpected" PDU. The "unexpected" PDUs include the following: 1. An Asynchronous Message PDU 2. A Reject PDU except when it is sent before a task is active since the command itself is discarded and the target should not send any further responses (see section 6.3 of RFC3720). 3. A NOP-in PDU except when it is sent as a response to a NOP-out PDU. --------------------- end of key description. For section 7.3.4, the last paragraph will be replaced by the following text: "Note that for solicited data, the SCSI Data-out PDUs are not used since R2T PDUs are not delivered to the iSCSI layer at the initiator; instead R2T PDUs are transformed by the iSER layer at the target into RDMA Read operations (see section 7.3.6)." Mike Sent by: ips-bounces@ietf.org To: ips@ietf.org cc: Subject: Re: [Ips] New iSCSI key I'm all for developing this option (this thread discusses #3 in the original list of options) but I really would like more feedback on other options as well. Developing this option is a good thing since doing so allows us to appreciate the (non)complexity of #3 compared to other candidates. With that caveat, a few comments on this thread. I agree that the current 7.3.4 discussion of the "sending solicited PDUs for data that was never solicited" case is just confusing. I recommend deleting the last two sentences in the last para of 7.3.4 (starting from "However..."). In the current architecture, if the TTT happens to be non-0xffffffff, it's an initiator code bug that will be suitably responded to by the target. If we do this, then I presume the #2 item in Mike's I->T list drops out. A general suggestion about all items is: factor out the repeating sentence to the top of the list ("Since it is "unexpected",....."). Recommend deleting #1 & #2 in the T->I list that simply explain data-type PDUs, but describe them as "exceptions" to the control-type PDU approach. I think this is confusing and besides, covered elsewhere. #4 on the T->I list discusses Reject. I presume this is for Rejects on non-command PDUs since Rejects on commands are "expected" in a way (at least the buffer room is available on the initiator). If you agree with this, suggest noting this "non-command" nature explicitly. I assume #5 is changing per John's comments. Thanks. -- Mallikarjun Mallikarjun Chadalapaka Networked Storage Architecture Network Storage Solutions Hewlett-Packard MS 5668 Roseville CA 95747 cbm [at] rose.hp.com _______________________________________________ Ips mailing list Ips@ietf.org https://www1.ietf.org/mailman/listinfo/ips From ips-bounces@ietf.org Mon Oct 18 19:59:55 2004 Received: from ietf-mx.ietf.org (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id TAA07684 for ; Mon, 18 Oct 2004 19:59:55 -0400 (EDT) Received: from megatron.ietf.org ([132.151.6.71]) by ietf-mx.ietf.org with esmtp (Exim 4.33) id 1CJhc2-0005N8-CB for ips-web-archive@ietf.org; Mon, 18 Oct 2004 20:12:26 -0400 Received: from localhost.localdomain ([127.0.0.1] helo=megatron.ietf.org) by megatron.ietf.org with esmtp (Exim 4.32) id 1CJhDM-0005AH-KF; Mon, 18 Oct 2004 19:46:56 -0400 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by megatron.ietf.org with esmtp (Exim 4.32) id 1CJh5R-0002w4-TL for ips@megatron.ietf.org; Mon, 18 Oct 2004 19:38:45 -0400 Received: from ietf-mx.ietf.org (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id TAA06366 for ; Mon, 18 Oct 2004 19:38:37 -0400 (EDT) Received: from system60.simplicato.com ([207.99.47.60] helo=host52a.simplicato.com) by ietf-mx.ietf.org with esmtp (Exim 4.33) id 1CJhHR-0004uc-FI for ips@ietf.org; Mon, 18 Oct 2004 19:51:10 -0400 Received: from localhost (localhost.simplicato.com [127.0.0.1]) by host52a.simplicato.com (Postfix) with ESMTP id 85B3911E1A9; Mon, 18 Oct 2004 19:38:30 -0400 (EDT) Received: from host52a.simplicato.com ([127.0.0.1]) by localhost (host52a.simplicato.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 88265-01; Mon, 18 Oct 2004 19:38:29 -0400 (EDT) Received: from [192.168.0.2] (64-144-5-25.client.dsl.net [64.144.5.25]) by host52a.simplicato.com (Postfix) with ESMTP id 8C83C11E19E; Mon, 18 Oct 2004 19:38:29 -0400 (EDT) Mime-Version: 1.0 (Apple Message framework v619) In-Reply-To: References: Content-Type: text/plain; charset=US-ASCII; format=flowed Message-Id: Content-Transfer-Encoding: 7bit From: Caitlin Bestler Subject: Re: [Ips] New iSCSI key Date: Mon, 18 Oct 2004 18:38:33 -0500 To: ips@ietf.org, Mike Ko X-Mailer: Apple Mail (2.619) X-Virus-Scanned: by amavisd-new at simplicato.com X-Spam-Score: 0.0 (/) X-Scan-Signature: de4f315c9369b71d7dd5909b42224370 Content-Transfer-Encoding: 7bit X-BeenThere: ips@ietf.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: IP Storage List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Sender: ips-bounces@ietf.org Errors-To: ips-bounces@ietf.org X-Spam-Score: 0.0 (/) X-Scan-Signature: 798b2e660f1819ae38035ac1d8d5e3ab Content-Transfer-Encoding: 7bit On Oct 18, 2004, at 5:06 PM, Mike Ko wrote: > If the sending side fails to adhere to the declared limit as set by > the > receiving side, it is up to the receiving side to determine ways of > handling the > overrun, including dropping the connection. > It may be worth noting that the sending side MAY enforce the limit by blocking or refusing to send a PDU that would have exceeded the limit. This would be the equivalent of the local back-pressure applied by TCP when transport buffering is exhausted. Or then, it might belong more in the DataMover Architecture draft. _______________________________________________ Ips mailing list Ips@ietf.org https://www1.ietf.org/mailman/listinfo/ips From ips-bounces@ietf.org Mon Oct 18 20:54:07 2004 Received: from ietf-mx.ietf.org (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id UAA12267 for ; Mon, 18 Oct 2004 20:54:07 -0400 (EDT) Received: from megatron.ietf.org ([132.151.6.71]) by ietf-mx.ietf.org with esmtp (Exim 4.33) id 1CJiSR-0006Vb-UI for ips-web-archive@ietf.org; Mon, 18 Oct 2004 21:06:39 -0400 Received: from localhost.localdomain ([127.0.0.1] helo=megatron.ietf.org) by megatron.ietf.org with esmtp (Exim 4.32) id 1CJhvn-0006hh-Pp; Mon, 18 Oct 2004 20:32:51 -0400 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by megatron.ietf.org with esmtp (Exim 4.32) id 1CJhoZ-0005Wt-V6 for ips@megatron.ietf.org; Mon, 18 Oct 2004 20:25:24 -0400 Received: from ietf-mx.ietf.org (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id UAA09838 for ; Mon, 18 Oct 2004 20:25:17 -0400 (EDT) Received: from e2.ny.us.ibm.com ([32.97.182.102]) by ietf-mx.ietf.org with esmtp (Exim 4.33) id 1CJi0X-0005sd-NI for ips@ietf.org; Mon, 18 Oct 2004 20:37:49 -0400 Received: from d01relay02.pok.ibm.com (d01relay02.pok.ibm.com [9.56.227.234]) by e2.ny.us.ibm.com (8.12.10/8.12.9) with ESMTP id i9J0OUBF242560; Mon, 18 Oct 2004 20:24:30 -0400 Received: from d01ml604.pok.ibm.com (d01av02.pok.ibm.com [9.56.224.216]) by d01relay02.pok.ibm.com (8.12.10/NCO/VER6.6) with ESMTP id i9J0OQR4109610; Mon, 18 Oct 2004 20:24:30 -0400 In-Reply-To: Importance: Normal MIME-Version: 1.0 To: Caitlin Bestler Subject: Re: [Ips] New iSCSI key X-Mailer: Lotus Notes Release 6.5.2 June 01, 2004 Message-ID: From: Mike Ko Date: Mon, 18 Oct 2004 17:24:25 -0700 X-MIMETrack: Serialize by Router on D01ML604/01/M/IBM(Release 6.51HF645 | October 15, 2004) at 10/18/2004 20:24:29, Serialize complete at 10/18/2004 20:24:29 Content-Type: text/plain; charset="US-ASCII" X-Spam-Score: 0.0 (/) X-Scan-Signature: 8abaac9e10c826e8252866cbe6766464 Cc: ips@ietf.org X-BeenThere: ips@ietf.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: IP Storage List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Sender: ips-bounces@ietf.org Errors-To: ips-bounces@ietf.org X-Spam-Score: 0.0 (/) X-Scan-Signature: 39bd8f8cbb76cae18b7e23f7cf6b2b9f Caitlin, It seems to me that we can treat this new key for "unexpected" PDUs similar to the way MaxCmdSN is described in iSCSI. Rather than specifying all the possible ways the sender can do to meet the requirement, we can just leave it up to implementation. Mike To: ips@ietf.org, Mike Ko cc: Subject: Re: [Ips] New iSCSI key On Oct 18, 2004, at 5:06 PM, Mike Ko wrote: > If the sending side fails to adhere to the declared limit as set by > the > receiving side, it is up to the receiving side to determine ways of > handling the > overrun, including dropping the connection. > It may be worth noting that the sending side MAY enforce the limit by blocking or refusing to send a PDU that would have exceeded the limit. This would be the equivalent of the local back-pressure applied by TCP when transport buffering is exhausted. Or then, it might belong more in the DataMover Architecture draft. _______________________________________________ Ips mailing list Ips@ietf.org https://www1.ietf.org/mailman/listinfo/ips From ips-bounces@ietf.org Mon Oct 18 23:44:11 2004 Received: from ietf-mx.ietf.org (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id XAA26424 for ; Mon, 18 Oct 2004 23:44:10 -0400 (EDT) Received: from megatron.ietf.org ([132.151.6.71]) by ietf-mx.ietf.org with esmtp (Exim 4.33) id 1CJl75-0001i0-VJ for ips-web-archive@ietf.org; Mon, 18 Oct 2004 23:56:44 -0400 Received: from localhost.localdomain ([127.0.0.1] helo=megatron.ietf.org) by megatron.ietf.org with esmtp (Exim 4.32) id 1CJkSd-0001ZI-TK; Mon, 18 Oct 2004 23:14:55 -0400 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by megatron.ietf.org with esmtp (Exim 4.32) id 1CJk6J-0006Id-0z for ips@megatron.ietf.org; Mon, 18 Oct 2004 22:51:51 -0400 Received: from ietf-mx.ietf.org (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id WAA22515 for ; Mon, 18 Oct 2004 22:50:53 -0400 (EDT) Received: from mtagate2.de.ibm.com ([195.212.29.151]) by ietf-mx.ietf.org with esmtp (Exim 4.33) id 1CJkHT-0000ms-Jm for ips@ietf.org; Mon, 18 Oct 2004 23:03:27 -0400 Received: from d12nrmr1607.megacenter.de.ibm.com (d12nrmr1607.megacenter.de.ibm.com [9.149.167.49]) by mtagate2.de.ibm.com (8.12.10/8.12.10) with ESMTP id i9J2oGxD025182; Tue, 19 Oct 2004 02:50:16 GMT Received: from d12ml102.megacenter.de.ibm.com (d12av02.megacenter.de.ibm.com [9.149.165.228]) by d12nrmr1607.megacenter.de.ibm.com (8.12.10/NCO/VER6.6) with ESMTP id i9J2oGuU184738; Tue, 19 Oct 2004 04:50:16 +0200 In-Reply-To: <16756.12089.569000.413737@gargle.gargle.HOWL> To: Paul Koning Subject: Re: [Ips] PDUs allowed on discovery sessions MIME-Version: 1.0 X-Mailer: Lotus Notes Build V70_M2_07222004 Beta 2NP July 22, 2004 From: Julian Satran Message-ID: Date: Mon, 18 Oct 2004 20:50:14 -0600 X-MIMETrack: Serialize by Router on D12ML102/12/M/IBM(Release 6.5.1| March 5, 2004) at 19/10/2004 04:50:15, Serialize complete at 19/10/2004 04:50:15 X-Spam-Score: 0.5 (/) X-Scan-Signature: 225414c974e0d6437992164e91287a51 Cc: ips@ietf.org, eddy_quicksall_iVivity_iSCSI@comcast.net X-BeenThere: ips@ietf.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: IP Storage List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Content-Type: multipart/mixed; boundary="===============2146341523==" Sender: ips-bounces@ietf.org Errors-To: ips-bounces@ietf.org X-Spam-Score: 0.5 (/) X-Scan-Signature: 48472a944c87678fcfe8db15ffecdfff This is a multipart message in MIME format. --===============2146341523== Content-Type: multipart/alternative; boundary="=_alternative 000E3A9786256F32_=" This is a multipart message in MIME format. --=_alternative 000E3A9786256F32_= Content-Type: text/plain; charset="US-ASCII" Paul is correct. A target that is felt abused may abort the connection. I think that the consensus of the group was that we don't want to encourage long-lived discovery although we should not outright forbid them (that would require more checks). Julo Paul Koning 18/10/04 16:01 To eddy_quicksall_iVivity_iSCSI@comcast.net cc Julian Satran/Haifa/IBM@IBMIL, ips@ietf.org Subject Re: [Ips] PDUs allowed on discovery sessions >>>>> "Eddy" == Eddy Quicksall writes: Eddy> Given what you said below, in a long lived discovery, the Eddy> initiator and target can still ping each other to verify that Eddy> the connection is still active; except that the target will Eddy> always "answer" the ping with a reject. Is that correct? That doesn't sound like what Julian said, and I wouldn't have said it. If the protocol prohibits the sending of PDU type X, it is wrong to assume that a node will always respond to such illegal PDUs in the way you wanted. If you want a fixed well-defined answer to a PDU, that PDU would be defined as allowed -- not as prohibited -- with a fixed action on receipt. If you're committing a protocol violation, don't be surprised to see your connection summarily deleted. paul >> Eddy ----- Original Message ----- From: Julian Satran To: Dean >> Scoville Cc: ips@ietf.org Sent: Wednesday, September 01, 2004 >> 1:05 PM Subject: RE: [Ips] PDUs allowed on discovery sessions >> Dean - that is the least it should do. I think that a target >> implementation is also free to close the session if it is >> "abused" (avoid a DOS attack). >> Julo >> "Dean Scoville" 01/09/04 17:14 To >> Julian Satran/Haifa/IBM@IBMIL cc Subject RE: >> [Ips] PDUs allowed on discovery sessions >>> Julian, Thanks for the correction. Just so I understand, when >>> the target rejects other PDUs... does it do so by sending a >>> Reject PDU? >>> Thank you. Dean --=_alternative 000E3A9786256F32_= Content-Type: text/html; charset="US-ASCII"
Paul is correct. A target that is felt abused may abort the connection.
I think that the consensus of the group was that we don't want to encourage long-lived discovery although we should not outright forbid them (that would require more checks).

Julo


Paul Koning <pkoning@equallogic.com>

18/10/04 16:01

To
eddy_quicksall_iVivity_iSCSI@comcast.net
cc
Julian Satran/Haifa/IBM@IBMIL, ips@ietf.org
Subject
Re: [Ips] PDUs allowed on discovery sessions





>>>>> "Eddy" == Eddy Quicksall <eddy_quicksall_iVivity_iSCSI@comcast.net> writes:

Eddy> Given what you said below, in a long lived discovery, the
Eddy> initiator and target can still ping each other to verify that
Eddy> the connection is still active; except that the target will
Eddy> always "answer" the ping with a reject.  Is that correct?

That doesn't sound like what Julian said, and I wouldn't have said
it.  If the protocol prohibits the sending of  PDU type X, it is wrong
to assume that a node will always respond to such illegal PDUs in the
way you wanted.  If you want a fixed well-defined answer to a PDU,
that PDU would be defined as allowed -- not as prohibited -- with a
fixed action on receipt.

If you're committing a protocol violation, don't be surprised to see
your connection summarily deleted.

    paul

>> Eddy ----- Original Message ----- From: Julian Satran To: Dean
>> Scoville Cc: ips@ietf.org Sent: Wednesday, September 01, 2004
>> 1:05 PM Subject: RE: [Ips] PDUs allowed on discovery sessions

>> Dean - that is the least it should do. I think that a target
>> implementation is also free to close the session if it is
>> "abused" (avoid a DOS attack).

>> Julo

>> "Dean Scoville" <dean.scoville@qlogic.com> 01/09/04 17:14 To
>> Julian Satran/Haifa/IBM@IBMIL cc <ips@ietf.org> Subject RE:
>> [Ips] PDUs allowed on discovery sessions

>>> Julian, Thanks for the correction. Just so I understand, when
>>> the target rejects other PDUs... does it do so by sending a
>>> Reject PDU?
   
>>> Thank you.  Dean
--=_alternative 000E3A9786256F32_=-- --===============2146341523== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline Content-Transfer-Encoding: 7bit _______________________________________________ Ips mailing list Ips@ietf.org https://www1.ietf.org/mailman/listinfo/ips --===============2146341523==-- From ips-bounces@ietf.org Thu Oct 21 21:18:51 2004 Received: from ietf-mx.ietf.org (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id VAA21202 for ; Thu, 21 Oct 2004 21:18:51 -0400 (EDT) Received: from megatron.ietf.org ([132.151.6.71]) by ietf-mx.ietf.org with esmtp (Exim 4.33) id 1CKoHj-0000BK-0n for ips-web-archive@ietf.org; Thu, 21 Oct 2004 21:32:03 -0400 Received: from localhost.localdomain ([127.0.0.1] helo=megatron.ietf.org) by megatron.ietf.org with esmtp (Exim 4.32) id 1CKlTf-0003HW-Tz; Thu, 21 Oct 2004 18:32:11 -0400 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by megatron.ietf.org with esmtp (Exim 4.32) id 1CKkFI-0005aL-41 for ips@megatron.ietf.org; Thu, 21 Oct 2004 17:13:16 -0400 Received: from ietf-mx.ietf.org (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id RAA28644 for ; Thu, 21 Oct 2004 17:13:13 -0400 (EDT) Received: from rwcrmhc11.comcast.net ([204.127.198.35]) by ietf-mx.ietf.org with esmtp (Exim 4.33) id 1CKkRy-0001CP-Ls for ips@ietf.org; Thu, 21 Oct 2004 17:26:23 -0400 Received: from ivvt2dxrc11 (unknown[64.238.111.98]) by comcast.net (rwcrmhc11) with SMTP id <200410212112420130091835e> (Authid: esquicksall); Thu, 21 Oct 2004 21:12:43 +0000 Message-ID: <002801c4b7b2$b47d9fd0$ea01a8c0@ivvt2dxrc11> From: "Eddy Quicksall" To: "Paul Koning" , "Julian Satran" References: Subject: Re: [Ips] PDUs allowed on discovery sessions Date: Thu, 21 Oct 2004 17:12:41 -0400 MIME-Version: 1.0 X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 6.00.2900.2180 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2900.2180 X-Spam-Score: 0.9 (/) X-Scan-Signature: 8a4bcf8f67063cac573319207fe3db35 Cc: ips@ietf.org X-BeenThere: ips@ietf.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: IP Storage List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Content-Type: multipart/mixed; boundary="===============0186250605==" Sender: ips-bounces@ietf.org Errors-To: ips-bounces@ietf.org X-Spam-Score: 0.9 (/) X-Scan-Signature: 142a000676f5977e1797396caab8b611 This is a multi-part message in MIME format. --===============0186250605== Content-Type: multipart/alternative; boundary="----=_NextPart_000_0025_01C4B791.2C748070" This is a multi-part message in MIME format. ------=_NextPart_000_0025_01C4B791.2C748070 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable I asked this because the protocol does not prohibit the initiator from = sending a NOP-out. And you actually implied it in your response: "You = may argue that a target may send NOPs (and the text does not forbid it) = but it MUST reject incoming NOPs." [I assume you meant outgoing] Can you point out where the protocol prohibits it? In fact, I think the = protocol requires the initiator to respond with a NOP-Out: Upon receipt of a NOP-In with the Target Transfer Tag set to a valid value (not the reserved 0xffffffff), the initiator MUST respond with a NOP-Out. In this case, the NOP-Out Target Transfer Tag MUST contain a copy of the NOP-In Target Transfer Tag. If the target can't send a NOP-in to see if the initiator is still = alive, how should it determine this? If there is an initiator that has a long lived discovery session (and I = think you said they do exist), then where does the protocol say how = long the target should wait before it drops the connection?=20 Eddy ----- Original Message -----=20 From: Julian Satran=20 To: Paul Koning=20 Cc: eddy_quicksall_iVivity_iSCSI@comcast.net ; ips@ietf.org=20 Sent: Monday, October 18, 2004 10:50 PM Subject: Re: [Ips] PDUs allowed on discovery sessions Paul is correct. A target that is felt abused may abort the = connection.=20 I think that the consensus of the group was that we don't want to = encourage long-lived discovery although we should not outright forbid = them (that would require more checks).=20 Julo Paul Koning =20 18/10/04 16:01=20 To eddy_quicksall_iVivity_iSCSI@comcast.net =20 cc Julian Satran/Haifa/IBM@IBMIL, ips@ietf.org =20 Subject Re: [Ips] PDUs allowed on discovery sessions=20 =20 =20 >>>>> "Eddy" =3D=3D Eddy Quicksall = writes: Eddy> Given what you said below, in a long lived discovery, the Eddy> initiator and target can still ping each other to verify that Eddy> the connection is still active; except that the target will Eddy> always "answer" the ping with a reject. Is that correct? That doesn't sound like what Julian said, and I wouldn't have said it. If the protocol prohibits the sending of PDU type X, it is wrong to assume that a node will always respond to such illegal PDUs in the way you wanted. If you want a fixed well-defined answer to a PDU, that PDU would be defined as allowed -- not as prohibited -- with a fixed action on receipt. If you're committing a protocol violation, don't be surprised to see your connection summarily deleted. paul >> Eddy ----- Original Message ----- From: Julian Satran To: Dean >> Scoville Cc: ips@ietf.org Sent: Wednesday, September 01, 2004 >> 1:05 PM Subject: RE: [Ips] PDUs allowed on discovery sessions >> Dean - that is the least it should do. I think that a target >> implementation is also free to close the session if it is >> "abused" (avoid a DOS attack). >> Julo >> "Dean Scoville" 01/09/04 17:14 To >> Julian Satran/Haifa/IBM@IBMIL cc Subject RE: >> [Ips] PDUs allowed on discovery sessions >>> Julian, Thanks for the correction. Just so I understand, when >>> the target rejects other PDUs... does it do so by sending a >>> Reject PDU? =20 >>> Thank you. Dean ------=_NextPart_000_0025_01C4B791.2C748070 Content-Type: text/html; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable
I asked this because the protocol does not prohibit = the=20 initiator from sending a NOP-out. And you actually implied it = in your=20 response: "You may argue that a target may = send NOPs=20 (and the text does not forbid it) but it MUST reject incoming NOPs." [I = assume=20 you meant outgoing]
 
Can you point out where the = protocol=20 prohibits it? In fact, I think the protocol requires the initiator to = respond=20 with a NOP-Out:
 

   Upon receipt of a NOP-In = with the=20 Target Transfer Tag set to a valid

   value (not the reserved=20 0xffffffff), the initiator MUST respond with

   a NOP-Out.  In this case, the NOP-Out = Target=20 Transfer Tag MUST

   contain a copy of the = NOP-In=20 Target Transfer Tag.

 

If the target can't send a NOP-in to see = if the=20 initiator is still alive, how should it determine this?
 
If there is an initiator that has a long lived=20 discovery session (and I think  you said they do exist), then = where=20 does the protocol say how long the target should wait before it drops = the=20 connection?
 
Eddy
 
 
----- Original Message -----
From:=20 Julian Satran
Cc: eddy_quicksall_i= Vivity_iSCSI@comcast.net=20 ; ips@ietf.org
Sent: Monday, October 18, 2004 = 10:50=20 PM
Subject: Re: [Ips] PDUs allowed = on=20 discovery sessions


Paul is correct. A = target that=20 is felt abused may abort the connection.
I think that the consensus of the group was that we don't = want to=20 encourage long-lived discovery although we should not outright forbid = them=20 (that would require more checks).

Julo


Paul Koning = <pkoning@equallogic.com>= =20

18/10/04 16:01

To
eddy_quicksall_i= Vivity_iSCSI@comcast.net=20
cc
Julian Satran/Haifa/IBM@IBMIL,=20 ips@ietf.org=20
Subject
Re: [Ips] PDUs allowed = on=20 discovery sessions

=




>>>>> "Eddy" =3D=3D Eddy Quicksall <eddy_quicksall_i= Vivity_iSCSI@comcast.net>=20 writes:

Eddy> Given what you said below, in a long lived = discovery,=20 the
Eddy> initiator and target can still ping each other to = verify=20 that
Eddy> the connection is still active; except that the = target=20 will
Eddy> always "answer" the ping with a reject.  Is that = correct?

That doesn't sound like what Julian said, and I = wouldn't have=20 said
it.  If the protocol prohibits the sending of  PDU = type X,=20 it is wrong
to assume that a node will always respond to such = illegal PDUs=20 in the
way you wanted.  If you want a fixed well-defined = answer to a=20 PDU,
that PDU would be defined as allowed -- not as prohibited -- = with=20 a
fixed action on receipt.

If you're committing a protocol=20 violation, don't be surprised to see
your connection summarily=20 deleted.

    paul

>> Eddy ----- Original = Message=20 ----- From: Julian Satran To: Dean
>> Scoville Cc: = ips@ietf.org Sent:=20 Wednesday, September 01, 2004
>> 1:05 PM Subject: RE: [Ips] = PDUs=20 allowed on discovery sessions

>> Dean - that is the least = it=20 should do. I think that a target
>> implementation is also = free to=20 close the session if it is
>> "abused" (avoid a DOS=20 attack).

>> Julo

>> "Dean Scoville"=20 <dean.scoville@qlogic.com> 01/09/04 17:14 To
>> Julian=20 Satran/Haifa/IBM@IBMIL cc <ips@ietf.org> Subject RE:
>> = [Ips]=20 PDUs allowed on discovery sessions

>>> Julian, Thanks = for the=20 correction. Just so I understand, when
>>> the target = rejects=20 other PDUs... does it do so by sending a
>>> Reject = PDU?
 =20  
>>> Thank you.=20  Dean


------=_NextPart_000_0025_01C4B791.2C748070-- --===============0186250605== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline Content-Transfer-Encoding: 7bit _______________________________________________ Ips mailing list Ips@ietf.org https://www1.ietf.org/mailman/listinfo/ips --===============0186250605==-- From ips-bounces@ietf.org Thu Oct 21 21:45:19 2004 Received: from ietf-mx.ietf.org (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id VAA26685 for ; Thu, 21 Oct 2004 21:45:19 -0400 (EDT) Received: from megatron.ietf.org ([132.151.6.71]) by ietf-mx.ietf.org with esmtp (Exim 4.33) id 1CKohK-0001ed-LO for ips-web-archive@ietf.org; Thu, 21 Oct 2004 21:58:31 -0400 Received: from localhost.localdomain ([127.0.0.1] helo=megatron.ietf.org) by megatron.ietf.org with esmtp (Exim 4.32) id 1CKo8Z-0002dU-PQ; Thu, 21 Oct 2004 21:22:35 -0400 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by megatron.ietf.org with esmtp (Exim 4.32) id 1CKlXS-00036o-Ow for ips@megatron.ietf.org; Thu, 21 Oct 2004 18:36:06 -0400 Received: from ietf-mx.ietf.org (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id SAA17430 for ; Thu, 21 Oct 2004 18:36:03 -0400 (EDT) Received: from rwcrmhc11.comcast.net ([204.127.198.35]) by ietf-mx.ietf.org with esmtp (Exim 4.33) id 1CKlk9-0006O2-8V for ips@ietf.org; Thu, 21 Oct 2004 18:49:14 -0400 Received: from ivvt2dxrc11 (unknown[64.238.111.98]) by comcast.net (rwcrmhc11) with SMTP id <200410212235330130091882e> (Authid: esquicksall); Thu, 21 Oct 2004 22:35:33 +0000 Message-ID: <003901c4b7be$473bff00$ea01a8c0@ivvt2dxrc11> From: "Eddy Quicksall" To: Date: Thu, 21 Oct 2004 18:35:32 -0400 MIME-Version: 1.0 X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 6.00.2900.2180 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2900.2180 X-Spam-Score: 0.1 (/) X-Scan-Signature: ea4ac80f790299f943f0a53be7e1a21a Subject: [Ips] use of the I bit in a nop-out X-BeenThere: ips@ietf.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: IP Storage List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Content-Type: multipart/mixed; boundary="===============2023823691==" Sender: ips-bounces@ietf.org Errors-To: ips-bounces@ietf.org X-Spam-Score: 0.5 (/) X-Scan-Signature: 4d87d2aa806f79fed918a62e834505ca This is a multi-part message in MIME format. --===============2023823691== Content-Type: multipart/alternative; boundary="----=_NextPart_000_0036_01C4B79C.BF516420" This is a multi-part message in MIME format. ------=_NextPart_000_0036_01C4B79C.BF516420 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable Can anyone point out a good use for the I bit in a NOP-out? Eddy ------=_NextPart_000_0036_01C4B79C.BF516420 Content-Type: text/html; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable
Can anyone point out a good use for the I bit in a=20 NOP-out?
 
Eddy
------=_NextPart_000_0036_01C4B79C.BF516420-- --===============2023823691== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline Content-Transfer-Encoding: 7bit _______________________________________________ Ips mailing list Ips@ietf.org https://www1.ietf.org/mailman/listinfo/ips --===============2023823691==-- From ips-bounces@ietf.org Thu Oct 21 21:47:16 2004 Received: from ietf-mx.ietf.org (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id VAA26907 for ; Thu, 21 Oct 2004 21:47:16 -0400 (EDT) Received: from megatron.ietf.org ([132.151.6.71]) by ietf-mx.ietf.org with esmtp (Exim 4.33) id 1CKojE-0001iW-Cj for ips-web-archive@ietf.org; Thu, 21 Oct 2004 22:00:28 -0400 Received: from localhost.localdomain ([127.0.0.1] helo=megatron.ietf.org) by megatron.ietf.org with esmtp (Exim 4.32) id 1CKoBh-0005ia-IU; Thu, 21 Oct 2004 21:25:49 -0400 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by megatron.ietf.org with esmtp (Exim 4.32) id 1CKlug-0006Xb-0r for ips@megatron.ietf.org; Thu, 21 Oct 2004 19:00:06 -0400 Received: from ietf-mx.ietf.org (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id TAA21796 for ; Thu, 21 Oct 2004 19:00:02 -0400 (EDT) Received: from rwcrmhc12.comcast.net ([216.148.227.85]) by ietf-mx.ietf.org with esmtp (Exim 4.33) id 1CKm7M-0007p0-V3 for ips@ietf.org; Thu, 21 Oct 2004 19:13:13 -0400 Received: from ivvt2dxrc11 (unknown[64.238.111.98]) by comcast.net (rwcrmhc12) with SMTP id <2004102122593001400474jhe> (Authid: esquicksall); Thu, 21 Oct 2004 22:59:31 +0000 Message-ID: <004101c4b7c1$9fc893b0$ea01a8c0@ivvt2dxrc11> From: "Eddy Quicksall" To: "Paul Koning" , "Julian Satran" References: Subject: Re: [Ips] PDUs allowed on discovery sessions Date: Thu, 21 Oct 2004 18:59:29 -0400 MIME-Version: 1.0 X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 6.00.2900.2180 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2900.2180 X-Spam-Score: 0.9 (/) X-Scan-Signature: bcd240e64c427d3d3617cfc704e7fd7f Cc: ips@ietf.org X-BeenThere: ips@ietf.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: IP Storage List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Content-Type: multipart/mixed; boundary="===============1917610091==" Sender: ips-bounces@ietf.org Errors-To: ips-bounces@ietf.org X-Spam-Score: 0.6 (/) X-Scan-Signature: 6907f330301e69261fa73bed91449a20 This is a multi-part message in MIME format. --===============1917610091== Content-Type: multipart/alternative; boundary="----=_NextPart_000_003E_01C4B7A0.17E10610" This is a multi-part message in MIME format. ------=_NextPart_000_003E_01C4B7A0.17E10610 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable How long would you say I should keep the connection active? Eddy ----- Original Message -----=20 From: Julian Satran=20 To: Paul Koning=20 Cc: eddy_quicksall_iVivity_iSCSI@comcast.net ; ips@ietf.org=20 Sent: Monday, October 18, 2004 10:50 PM Subject: Re: [Ips] PDUs allowed on discovery sessions Paul is correct. A target that is felt abused may abort the = connection.=20 I think that the consensus of the group was that we don't want to = encourage long-lived discovery although we should not outright forbid = them (that would require more checks).=20 Julo=20 Paul Koning =20 18/10/04 16:01=20 To eddy_quicksall_iVivity_iSCSI@comcast.net =20 cc Julian Satran/Haifa/IBM@IBMIL, ips@ietf.org =20 Subject Re: [Ips] PDUs allowed on discovery sessions=20 =20 =20 >>>>> "Eddy" =3D=3D Eddy Quicksall = writes: Eddy> Given what you said below, in a long lived discovery, the Eddy> initiator and target can still ping each other to verify that Eddy> the connection is still active; except that the target will Eddy> always "answer" the ping with a reject. Is that correct? That doesn't sound like what Julian said, and I wouldn't have said it. If the protocol prohibits the sending of PDU type X, it is wrong to assume that a node will always respond to such illegal PDUs in the way you wanted. If you want a fixed well-defined answer to a PDU, that PDU would be defined as allowed -- not as prohibited -- with a fixed action on receipt. If you're committing a protocol violation, don't be surprised to see your connection summarily deleted. paul >> Eddy ----- Original Message ----- From: Julian Satran To: Dean >> Scoville Cc: ips@ietf.org Sent: Wednesday, September 01, 2004 >> 1:05 PM Subject: RE: [Ips] PDUs allowed on discovery sessions >> Dean - that is the least it should do. I think that a target >> implementation is also free to close the session if it is >> "abused" (avoid a DOS attack). >> Julo >> "Dean Scoville" 01/09/04 17:14 To >> Julian Satran/Haifa/IBM@IBMIL cc Subject RE: >> [Ips] PDUs allowed on discovery sessions >>> Julian, Thanks for the correction. Just so I understand, when >>> the target rejects other PDUs... does it do so by sending a >>> Reject PDU? =20 >>> Thank you. Dean ------=_NextPart_000_003E_01C4B7A0.17E10610 Content-Type: text/html; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable
How long would you say I should keep the connection=20 active?
 
Eddy
----- Original Message -----
From:=20 Julian Satran
Cc: eddy_quicksall_i= Vivity_iSCSI@comcast.net=20 ; ips@ietf.org
Sent: Monday, October 18, 2004 = 10:50=20 PM
Subject: Re: [Ips] PDUs allowed = on=20 discovery sessions


Paul is correct. A = target that=20 is felt abused may abort the connection.
I think that the consensus of the group was that we don't = want to=20 encourage long-lived discovery although we should not outright forbid = them=20 (that would require more checks).

Julo


Paul Koning = <pkoning@equallogic.com>= =20

18/10/04 16:01

To
eddy_quicksall_i= Vivity_iSCSI@comcast.net=20
cc
Julian Satran/Haifa/IBM@IBMIL,=20 ips@ietf.org=20
Subject
Re: [Ips] PDUs allowed = on=20 discovery sessions

=




>>>>> "Eddy" =3D=3D Eddy Quicksall <eddy_quicksall_i= Vivity_iSCSI@comcast.net>=20 writes:

Eddy> Given what you said below, in a long lived = discovery,=20 the
Eddy> initiator and target can still ping each other to = verify=20 that
Eddy> the connection is still active; except that the = target=20 will
Eddy> always "answer" the ping with a reject.  Is that = correct?

That doesn't sound like what Julian said, and I = wouldn't have=20 said
it.  If the protocol prohibits the sending of  PDU = type X,=20 it is wrong
to assume that a node will always respond to such = illegal PDUs=20 in the
way you wanted.  If you want a fixed well-defined = answer to a=20 PDU,
that PDU would be defined as allowed -- not as prohibited -- = with=20 a
fixed action on receipt.

If you're committing a protocol=20 violation, don't be surprised to see
your connection summarily=20 deleted.

    paul

>> Eddy ----- Original = Message=20 ----- From: Julian Satran To: Dean
>> Scoville Cc: = ips@ietf.org Sent:=20 Wednesday, September 01, 2004
>> 1:05 PM Subject: RE: [Ips] = PDUs=20 allowed on discovery sessions

>> Dean - that is the least = it=20 should do. I think that a target
>> implementation is also = free to=20 close the session if it is
>> "abused" (avoid a DOS=20 attack).

>> Julo

>> "Dean Scoville"=20 <dean.scoville@qlogic.com> 01/09/04 17:14 To
>> Julian=20 Satran/Haifa/IBM@IBMIL cc <ips@ietf.org> Subject RE:
>> = [Ips]=20 PDUs allowed on discovery sessions

>>> Julian, Thanks = for the=20 correction. Just so I understand, when
>>> the target = rejects=20 other PDUs... does it do so by sending a
>>> Reject = PDU?
 =20  
>>> Thank you.=20  Dean


------=_NextPart_000_003E_01C4B7A0.17E10610-- --===============1917610091== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline Content-Transfer-Encoding: 7bit _______________________________________________ Ips mailing list Ips@ietf.org https://www1.ietf.org/mailman/listinfo/ips --===============1917610091==-- From ips-bounces@ietf.org Thu Oct 21 21:49:58 2004 Received: from ietf-mx.ietf.org (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id VAA27486 for ; Thu, 21 Oct 2004 21:49:58 -0400 (EDT) Received: from megatron.ietf.org ([132.151.6.71]) by ietf-mx.ietf.org with esmtp (Exim 4.33) id 1CKolm-0001qG-DU for ips-web-archive@ietf.org; Thu, 21 Oct 2004 22:03:10 -0400 Received: from localhost.localdomain ([127.0.0.1] helo=megatron.ietf.org) by megatron.ietf.org with esmtp (Exim 4.32) id 1CKoF6-00051L-AR; Thu, 21 Oct 2004 21:29:20 -0400 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by megatron.ietf.org with esmtp (Exim 4.32) id 1CKmZU-0007aZ-5T for ips@megatron.ietf.org; Thu, 21 Oct 2004 19:42:16 -0400 Received: from ietf-mx.ietf.org (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id TAA02102 for ; Thu, 21 Oct 2004 19:42:09 -0400 (EDT) Received: from sadr.equallogic.com ([66.155.203.134]) by ietf-mx.ietf.org with smtp (Exim 4.33) id 1CKmlv-0002hI-58 for ips@ietf.org; Thu, 21 Oct 2004 19:55:21 -0400 Received: from sadr.equallogic.com (localhost.localdomain [127.0.0.1]) by sadr.equallogic.com (8.12.8/8.12.8) with ESMTP id i9LNfP2D001285 for ; Thu, 21 Oct 2004 19:41:25 -0400 Received: from M30.equallogic.com (m30 [172.16.1.30]) by sadr.equallogic.com (8.12.8/8.12.8) with SMTP id i9LNfPqn001280; Thu, 21 Oct 2004 19:41:25 -0400 Received: from PKONING.equallogic.com ([172.16.2.49]) by M30.equallogic.com with Microsoft SMTPSVC(5.0.2195.6713); Thu, 21 Oct 2004 19:41:24 -0400 MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Message-ID: <16760.18708.39000.743110@gargle.gargle.HOWL> Date: Thu, 21 Oct 2004 19:41:08 -0400 From: Paul Koning To: eddy_quicksall_iVivity_iSCSI@comcast.net Subject: Re: [Ips] PDUs allowed on discovery sessions References: <004101c4b7c1$9fc893b0$ea01a8c0@ivvt2dxrc11> X-Mailer: VM 7.07 under 21.4 (patch 10) "Military Intelligence (Windows)" XEmacs Lucid X-OriginalArrivalTime: 21 Oct 2004 23:41:24.0844 (UTC) FILETIME=[7A1EF2C0:01C4B7C7] X-Spam-Score: 0.0 (/) X-Scan-Signature: cf4fa59384e76e63313391b70cd0dd25 Content-Transfer-Encoding: 7bit Cc: ips@ietf.org, Julian_Satran@il.ibm.com X-BeenThere: ips@ietf.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: IP Storage List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Sender: ips-bounces@ietf.org Errors-To: ips-bounces@ietf.org X-Spam-Score: 0.0 (/) X-Scan-Signature: 7d33c50f3756db14428398e2bdedd581 Content-Transfer-Encoding: 7bit >>>>> "Eddy" == Eddy Quicksall writes: Eddy> How long would you say I should keep the connection active? The common practice seems to be to open the connection, ask for what you want to know, and close it. That seems like a good choice. If you're told (out of band -- perhaps through a management UI) that something might have changed, repeat the operation. If you need to be notified of change as it happens, use iSNS, it's meant for that job. Yes, you could try to keep the discovery session open and poll for news every little while. But that isn't the approach I would pick. paul _______________________________________________ Ips mailing list Ips@ietf.org https://www1.ietf.org/mailman/listinfo/ips From ips-bounces@ietf.org Fri Oct 22 08:25:49 2004 Received: from ietf-mx.ietf.org (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id IAA27862 for ; Fri, 22 Oct 2004 08:25:48 -0400 (EDT) Received: from megatron.ietf.org ([132.151.6.71]) by ietf-mx.ietf.org with esmtp (Exim 4.33) id 1CKyhE-0004wY-82 for ips-web-archive@ietf.org; Fri, 22 Oct 2004 08:39:04 -0400 Received: from localhost.localdomain ([127.0.0.1] helo=megatron.ietf.org) by megatron.ietf.org with esmtp (Exim 4.32) id 1CKyQ0-00026r-IE; Fri, 22 Oct 2004 08:21:16 -0400 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by megatron.ietf.org with esmtp (Exim 4.32) id 1CKyOh-0001iJ-60 for ips@megatron.ietf.org; Fri, 22 Oct 2004 08:19:55 -0400 Received: from ietf-mx.ietf.org (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id IAA27336 for ; Fri, 22 Oct 2004 08:19:52 -0400 (EDT) Received: from rwcrmhc11.comcast.net ([204.127.198.35]) by ietf-mx.ietf.org with esmtp (Exim 4.33) id 1CKybS-0004pO-S6 for ips@ietf.org; Fri, 22 Oct 2004 08:33:08 -0400 Received: from ivvt2dxrc11 (unknown[64.238.111.98]) by comcast.net (rwcrmhc11) with SMTP id <200410221219190130091bioe> (Authid: esquicksall); Fri, 22 Oct 2004 12:19:20 +0000 Message-ID: <000501c4b831$4f141140$ea01a8c0@ivvt2dxrc11> From: "Eddy Quicksall" To: "Paul Koning" References: <004101c4b7c1$9fc893b0$ea01a8c0@ivvt2dxrc11> <16760.18708.39000.743110@gargle.gargle.HOWL> Subject: Re: [Ips] PDUs allowed on discovery sessions Date: Fri, 22 Oct 2004 08:18:57 -0400 MIME-Version: 1.0 Content-Type: text/plain; format=flowed; charset="iso-8859-1"; reply-type=original Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 6.00.2900.2180 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2900.2180 X-Spam-Score: 0.0 (/) X-Scan-Signature: 69a74e02bbee44ab4f8eafdbcedd94a1 Content-Transfer-Encoding: 7bit Cc: ips@ietf.org, Julian_Satran@il.ibm.com X-BeenThere: ips@ietf.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: IP Storage List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Sender: ips-bounces@ietf.org Errors-To: ips-bounces@ietf.org X-Spam-Score: 0.0 (/) X-Scan-Signature: 9ed51c9d1356100bce94f1ae4ec616a9 Content-Transfer-Encoding: 7bit I'm a target. Can you please give me a reference in the spec that says: 1) an initiator can't keep a discover session open 2) if it does, how long does the target wait before dropping the connection Eddy ----- Original Message ----- From: "Paul Koning" To: Cc: ; Sent: Thursday, October 21, 2004 7:41 PM Subject: Re: [Ips] PDUs allowed on discovery sessions >>>>>> "Eddy" == Eddy Quicksall >>>>>> writes: > > Eddy> How long would you say I should keep the connection active? > > The common practice seems to be to open the connection, ask for what > you want to know, and close it. That seems like a good choice. > > If you're told (out of band -- perhaps through a management UI) that > something might have changed, repeat the operation. If you need to be > notified of change as it happens, use iSNS, it's meant for that job. > > Yes, you could try to keep the discovery session open and poll for > news every little while. But that isn't the approach I would pick. > > paul > > > _______________________________________________ > Ips mailing list > Ips@ietf.org > https://www1.ietf.org/mailman/listinfo/ips _______________________________________________ Ips mailing list Ips@ietf.org https://www1.ietf.org/mailman/listinfo/ips From ips-bounces@ietf.org Fri Oct 22 10:09:39 2004 Received: from ietf-mx.ietf.org (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id KAA06321 for ; Fri, 22 Oct 2004 10:09:39 -0400 (EDT) Received: from megatron.ietf.org ([132.151.6.71]) by ietf-mx.ietf.org with esmtp (Exim 4.33) id 1CL0Jj-00077h-F7 for ips-web-archive@ietf.org; Fri, 22 Oct 2004 10:22:56 -0400 Received: from localhost.localdomain ([127.0.0.1] helo=megatron.ietf.org) by megatron.ietf.org with esmtp (Exim 4.32) id 1CL05Z-0000TP-5G; Fri, 22 Oct 2004 10:08:17 -0400 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by megatron.ietf.org with esmtp (Exim 4.32) id 1CKzsQ-0001lr-Bq for ips@megatron.ietf.org; Fri, 22 Oct 2004 09:54:42 -0400 Received: from ietf-mx.ietf.org (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id JAA04632 for ; Fri, 22 Oct 2004 09:54:39 -0400 (EDT) Received: from sadr.equallogic.com ([66.155.203.134]) by ietf-mx.ietf.org with smtp (Exim 4.33) id 1CL054-0006jA-2d for ips@ietf.org; Fri, 22 Oct 2004 10:07:56 -0400 Received: from sadr.equallogic.com (localhost.localdomain [127.0.0.1]) by sadr.equallogic.com (8.12.8/8.12.8) with ESMTP id i9MDru2D018269 for ; Fri, 22 Oct 2004 09:53:57 -0400 Received: from M30.equallogic.com (m30 [172.16.1.30]) by sadr.equallogic.com (8.12.8/8.12.8) with SMTP id i9MDruqn018264; Fri, 22 Oct 2004 09:53:56 -0400 Received: from PKONING.equallogic.com ([172.16.1.105]) by M30.equallogic.com with Microsoft SMTPSVC(5.0.2195.6713); Fri, 22 Oct 2004 09:53:56 -0400 MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Message-ID: <16761.4339.144000.509412@gargle.gargle.HOWL> Date: Fri, 22 Oct 2004 09:53:55 -0400 From: Paul Koning To: eddy_quicksall_iVivity_iSCSI@comcast.net Subject: Re: [Ips] PDUs allowed on discovery sessions References: <004101c4b7c1$9fc893b0$ea01a8c0@ivvt2dxrc11> <16760.18708.39000.743110@gargle.gargle.HOWL> <000501c4b831$4f141140$ea01a8c0@ivvt2dxrc11> X-Mailer: VM 7.07 under 21.4 (patch 10) "Military Intelligence (Windows)" XEmacs Lucid X-OriginalArrivalTime: 22 Oct 2004 13:53:56.0701 (UTC) FILETIME=[9300E8D0:01C4B83E] X-Spam-Score: 0.0 (/) X-Scan-Signature: de4f315c9369b71d7dd5909b42224370 Content-Transfer-Encoding: 7bit Cc: ips@ietf.org, Julian_Satran@il.ibm.com X-BeenThere: ips@ietf.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: IP Storage List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Sender: ips-bounces@ietf.org Errors-To: ips-bounces@ietf.org X-Spam-Score: 0.0 (/) X-Scan-Signature: 798b2e660f1819ae38035ac1d8d5e3ab Content-Transfer-Encoding: 7bit >>>>> "Eddy" == Eddy Quicksall writes: Eddy> I'm a target. Can you please give me a reference in the spec Eddy> that says: 1) an initiator can't keep a discover session open Eddy> 2) if it does, how long does the target wait before dropping Eddy> the connection I never implied #1, because the spec says that an initiator CAN keep a discovery session open. What I said is that this is not common practice and not a particularly useful thing to do, and I would not do it if I were implementing an iniatiator. As for #2, the spec does not say, and I see no reason why it should. That's a matter of local choice. paul _______________________________________________ Ips mailing list Ips@ietf.org https://www1.ietf.org/mailman/listinfo/ips From ips-bounces@ietf.org Fri Oct 22 10:26:53 2004 Received: from ietf-mx.ietf.org (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id KAA08487 for ; Fri, 22 Oct 2004 10:26:53 -0400 (EDT) Received: from megatron.ietf.org ([132.151.6.71]) by ietf-mx.ietf.org with esmtp (Exim 4.33) id 1CL0aP-0007SF-2K for ips-web-archive@ietf.org; Fri, 22 Oct 2004 10:40:10 -0400 Received: from localhost.localdomain ([127.0.0.1] helo=megatron.ietf.org) by megatron.ietf.org with esmtp (Exim 4.32) id 1CL09j-0002y6-Hl; Fri, 22 Oct 2004 10:12:35 -0400 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by megatron.ietf.org with esmtp (Exim 4.32) id 1CL05g-0000VB-Bm for ips@megatron.ietf.org; Fri, 22 Oct 2004 10:08:24 -0400 Received: from ietf-mx.ietf.org (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id KAA06080 for ; Fri, 22 Oct 2004 10:08:21 -0400 (EDT) Received: from sadr.equallogic.com ([66.155.203.134]) by ietf-mx.ietf.org with smtp (Exim 4.33) id 1CL0IT-00072G-Q0 for ips@ietf.org; Fri, 22 Oct 2004 10:21:38 -0400 Received: from sadr.equallogic.com (localhost.localdomain [127.0.0.1]) by sadr.equallogic.com (8.12.8/8.12.8) with ESMTP id i9ME7o2D019334 for ; Fri, 22 Oct 2004 10:07:50 -0400 Received: from M30.equallogic.com (m30 [172.16.1.30]) by sadr.equallogic.com (8.12.8/8.12.8) with SMTP id i9ME7oqn019329; Fri, 22 Oct 2004 10:07:50 -0400 Received: from PKONING.equallogic.com ([172.16.1.105]) by M30.equallogic.com with Microsoft SMTPSVC(5.0.2195.6713); Fri, 22 Oct 2004 10:07:50 -0400 MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Message-ID: <16761.5172.607000.452486@gargle.gargle.HOWL> Date: Fri, 22 Oct 2004 10:07:48 -0400 From: Paul Koning To: eddy_quicksall_iVivity_iSCSI@comcast.net Subject: Re: [Ips] PDUs allowed on discovery sessions References: <002801c4b7b2$b47d9fd0$ea01a8c0@ivvt2dxrc11> X-Mailer: VM 7.07 under 21.4 (patch 10) "Military Intelligence (Windows)" XEmacs Lucid X-OriginalArrivalTime: 22 Oct 2004 14:07:50.0483 (UTC) FILETIME=[83F9F230:01C4B840] X-Spam-Score: 0.0 (/) X-Scan-Signature: 08170828343bcf1325e4a0fb4584481c Content-Transfer-Encoding: 7bit Cc: ips@ietf.org, Julian_Satran@il.ibm.com X-BeenThere: ips@ietf.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: IP Storage List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Sender: ips-bounces@ietf.org Errors-To: ips-bounces@ietf.org X-Spam-Score: 0.0 (/) X-Scan-Signature: 79899194edc4f33a41f49410777972f8 Content-Transfer-Encoding: 7bit >>>>> "Eddy" == Eddy Quicksall writes: Eddy> I asked this because the protocol does not prohibit the Eddy> initiator from sending a NOP-out. And you actually implied it Eddy> in your response: "You may argue that a target may send NOPs Eddy> (and the text does not forbid it) but it MUST reject incoming Eddy> NOPs." [I assume you meant outgoing] Can you point out where Eddy> the protocol prohibits it? In fact, I think the protocol Eddy> requires the initiator to respond with a NOP-Out... Section 3.3 says only that "Send Targets" and "Logout, close session" are legal in discovery sections, and nothing else is. paul _______________________________________________ Ips mailing list Ips@ietf.org https://www1.ietf.org/mailman/listinfo/ips From ips-bounces@ietf.org Fri Oct 22 11:00:23 2004 Received: from ietf-mx.ietf.org (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id LAA11687 for ; Fri, 22 Oct 2004 11:00:23 -0400 (EDT) Received: from megatron.ietf.org ([132.151.6.71]) by ietf-mx.ietf.org with esmtp (Exim 4.33) id 1CL16q-0008Lz-Ei for ips-web-archive@ietf.org; Fri, 22 Oct 2004 11:13:41 -0400 Received: from localhost.localdomain ([127.0.0.1] helo=megatron.ietf.org) by megatron.ietf.org with esmtp (Exim 4.32) id 1CL0of-0004Mm-Az; Fri, 22 Oct 2004 10:54:53 -0400 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by megatron.ietf.org with esmtp (Exim 4.32) id 1CL0kQ-00036i-6H for ips@megatron.ietf.org; Fri, 22 Oct 2004 10:50:30 -0400 Received: from ietf-mx.ietf.org (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id KAA11173 for ; Fri, 22 Oct 2004 10:50:26 -0400 (EDT) Received: from sccrmhc11.comcast.net ([204.127.202.55]) by ietf-mx.ietf.org with esmtp (Exim 4.33) id 1CL0xD-0008C2-6q for ips@ietf.org; Fri, 22 Oct 2004 11:03:44 -0400 Received: from ivvt2dxrc11 (unknown[64.238.111.98]) by comcast.net (sccrmhc11) with SMTP id <20041022144955011006abc0e> (Authid: esquicksall); Fri, 22 Oct 2004 14:49:55 +0000 Message-ID: <001401c4b846$58c5b530$ea01a8c0@ivvt2dxrc11> From: "Eddy Quicksall" To: "Paul Koning" References: <004101c4b7c1$9fc893b0$ea01a8c0@ivvt2dxrc11><16760.18708.39000.743110@gargle.gargle.HOWL><000501c4b831$4f141140$ea01a8c0@ivvt2dxrc11> <16761.4339.144000.509412@gargle.gargle.HOWL> Subject: Re: [Ips] PDUs allowed on discovery sessions Date: Fri, 22 Oct 2004 10:49:33 -0400 MIME-Version: 1.0 Content-Type: text/plain; format=flowed; charset="iso-8859-1"; reply-type=original Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 6.00.2900.2180 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2900.2180 X-Spam-Score: 0.0 (/) X-Scan-Signature: 5a9a1bd6c2d06a21d748b7d0070ddcb8 Content-Transfer-Encoding: 7bit Cc: ips@ietf.org, Julian_Satran@il.ibm.com X-BeenThere: ips@ietf.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: IP Storage List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Sender: ips-bounces@ietf.org Errors-To: ips-bounces@ietf.org X-Spam-Score: 0.0 (/) X-Scan-Signature: 082a9cbf4d599f360ac7f815372a6a15 Content-Transfer-Encoding: 7bit > I never implied #1, because the spec says that an initiator CAN keep a > discovery session open. For a long discovery session, what procedure would you suggest that the target use to determine if the initiator is still active? e.g., for a normal session, section 10.18 says: A NOP-Out may be used by an initiator as a "ping request" to verify that a connection/session is still active and all its components are operational. and 10.19 says: NOP-In is either sent by a target ..., as a "ping" to an initiator, or ... BTW, I have no problem if some words were added to the spec that say something like: An initiator MUST not leave a discover session open for longer than Time2Retain seconds. Or--- Allow NOP-out in discovery sessions. Eddy ----- Original Message ----- From: "Paul Koning" To: Cc: ; Sent: Friday, October 22, 2004 9:53 AM Subject: Re: [Ips] PDUs allowed on discovery sessions >>>>>> "Eddy" == Eddy Quicksall >>>>>> writes: > > Eddy> I'm a target. Can you please give me a reference in the spec > Eddy> that says: 1) an initiator can't keep a discover session open > Eddy> 2) if it does, how long does the target wait before dropping > Eddy> the connection > > I never implied #1, because the spec says that an initiator CAN keep a > discovery session open. What I said is that this is not common > practice and not a particularly useful thing to do, and I would not do > it if I were implementing an iniatiator. > > As for #2, the spec does not say, and I see no reason why it should. > That's a matter of local choice. > > paul > _______________________________________________ Ips mailing list Ips@ietf.org https://www1.ietf.org/mailman/listinfo/ips From ips-bounces@ietf.org Fri Oct 22 11:45:56 2004 Received: from ietf-mx.ietf.org (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id LAA15260 for ; Fri, 22 Oct 2004 11:45:56 -0400 (EDT) Received: from megatron.ietf.org ([132.151.6.71]) by ietf-mx.ietf.org with esmtp (Exim 4.33) id 1CL1ow-0000nc-8y for ips-web-archive@ietf.org; Fri, 22 Oct 2004 11:59:14 -0400 Received: from localhost.localdomain ([127.0.0.1] helo=megatron.ietf.org) by megatron.ietf.org with esmtp (Exim 4.32) id 1CL1Z7-0003gd-8n; Fri, 22 Oct 2004 11:42:53 -0400 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by megatron.ietf.org with esmtp (Exim 4.32) id 1CL1Sw-0002e4-Kc for ips@megatron.ietf.org; Fri, 22 Oct 2004 11:36:30 -0400 Received: from ietf-mx.ietf.org (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id LAA14530 for ; Fri, 22 Oct 2004 11:36:26 -0400 (EDT) Received: from rospat.qlogic.com ([198.70.192.130] helo=homer.qlogic.com) by ietf-mx.ietf.org with esmtp (Exim 4.33) id 1CL1fk-0000am-P4 for ips@ietf.org; Fri, 22 Oct 2004 11:49:45 -0400 X-MimeOLE: Produced By Microsoft Exchange V6.0.6556.0 content-class: urn:content-classes:message Subject: RE: [Ips] use of the I bit in a nop-out MIME-Version: 1.0 Date: Fri, 22 Oct 2004 08:32:16 -0700 Message-ID: <2A8D600C919D2D41A3DAA2D2FEC717147354A3@homer.qlogic.org> Thread-Topic: [Ips] use of the I bit in a nop-out Thread-Index: AcS32GX2ZuIeX3ePRKG1mRb/dOCkbgAc1T1A From: "Dean Scoville" To: "Eddy Quicksall" , X-Spam-Score: 0.6 (/) X-Scan-Signature: 3002fc2e661cd7f114cb6bae92fe88f1 X-BeenThere: ips@ietf.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: IP Storage List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Content-Type: multipart/mixed; boundary="===============0661244425==" Sender: ips-bounces@ietf.org Errors-To: ips-bounces@ietf.org X-Spam-Score: 0.9 (/) X-Scan-Signature: 944ecb6e61f753561f559a497458fb4f This is a multi-part message in MIME format. --===============0661244425== content-class: urn:content-classes:message Content-Type: multipart/alternative; boundary="----_=_NextPart_001_01C4B84C.4FB02C2C" This is a multi-part message in MIME format. ------_=_NextPart_001_01C4B84C.4FB02C2C Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable According to section 10.18.1, the I-bit MUST be set to 1 whenever = ITT=3D0xffffffff in the NOP-Out PDU. The I-bit keeps the CmdSN from advancing when the initiator either = replies to a NOP-In ping sent by the target or when it sends a NOP-Out to update the target's = counters (without requiring a reply). =20 Dean -----Original Message----- From: ips-bounces@ietf.org [mailto:ips-bounces@ietf.org]On Behalf Of = Eddy Quicksall Sent: Thursday, October 21, 2004 3:36 PM To: ips@ietf.org Subject: [Ips] use of the I bit in a nop-out Can anyone point out a good use for the I bit in a NOP-out? =20 Eddy ------_=_NextPart_001_01C4B84C.4FB02C2C Content-Type: text/html; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable
According to section 10.18.1, the I-bit MUST be set to 1=20 whenever ITT=3D0xffffffff in the NOP-Out PDU.
The=20 I-bit keeps the CmdSN from advancing when the initiator either replies = to a=20 NOP-In ping
sent=20 by the target or when it sends a NOP-Out to update the target's counters = (without requiring
a=20 reply).
 
Dean
-----Original Message-----
From: = ips-bounces@ietf.org=20 [mailto:ips-bounces@ietf.org]On Behalf Of Eddy=20 Quicksall
Sent: Thursday, October 21, 2004 3:36 = PM
To:=20 ips@ietf.org
Subject: [Ips] use of the I bit in a=20 nop-out

Can anyone point out a good use for the I bit in a = NOP-out?
 
Eddy
------_=_NextPart_001_01C4B84C.4FB02C2C-- --===============0661244425== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline Content-Transfer-Encoding: 7bit _______________________________________________ Ips mailing list Ips@ietf.org https://www1.ietf.org/mailman/listinfo/ips --===============0661244425==-- From ips-bounces@ietf.org Fri Oct 22 13:45:23 2004 Received: from ietf-mx.ietf.org (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id NAA23867 for ; Fri, 22 Oct 2004 13:45:23 -0400 (EDT) Received: from megatron.ietf.org ([132.151.6.71]) by ietf-mx.ietf.org with esmtp (Exim 4.33) id 1CL3gX-00032J-ME for ips-web-archive@ietf.org; Fri, 22 Oct 2004 13:58:41 -0400 Received: from localhost.localdomain ([127.0.0.1] helo=megatron.ietf.org) by megatron.ietf.org with esmtp (Exim 4.32) id 1CL3Pv-0001SD-1X; Fri, 22 Oct 2004 13:41:31 -0400 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by megatron.ietf.org with esmtp (Exim 4.32) id 1CL3IM-00081l-Fb for ips@megatron.ietf.org; Fri, 22 Oct 2004 13:33:42 -0400 Received: from ietf-mx.ietf.org (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id NAA23278 for ; Fri, 22 Oct 2004 13:33:37 -0400 (EDT) Received: from rwcrmhc13.comcast.net ([204.127.198.39]) by ietf-mx.ietf.org with esmtp (Exim 4.33) id 1CL3VB-0002qB-DX for ips@ietf.org; Fri, 22 Oct 2004 13:46:57 -0400 Received: from ivvt2dxrc11 (unknown[64.238.111.98]) by comcast.net (rwcrmhc13) with SMTP id <2004102217330701500mivq1e> (Authid: esquicksall); Fri, 22 Oct 2004 17:33:08 +0000 Message-ID: <001001c4b85d$24f5e2e0$ea01a8c0@ivvt2dxrc11> From: "Eddy Quicksall" To: "Dean Scoville" , References: <2A8D600C919D2D41A3DAA2D2FEC717147354A3@homer.qlogic.org> Subject: Re: [Ips] use of the I bit in a nop-out Date: Fri, 22 Oct 2004 13:32:44 -0400 MIME-Version: 1.0 X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 6.00.2900.2180 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2900.2180 X-Spam-Score: 0.6 (/) X-Scan-Signature: 8fbbaa16f9fd29df280814cb95ae2290 X-BeenThere: ips@ietf.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: IP Storage List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Content-Type: multipart/mixed; boundary="===============1308650629==" Sender: ips-bounces@ietf.org Errors-To: ips-bounces@ietf.org X-Spam-Score: 0.9 (/) X-Scan-Signature: d16ce744298aacf98517bc7c108bd198 This is a multi-part message in MIME format. --===============1308650629== Content-Type: multipart/alternative; boundary="----=_NextPart_000_000D_01C4B83B.9CDA7400" This is a multi-part message in MIME format. ------=_NextPart_000_000D_01C4B83B.9CDA7400 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable Yes, that is a requirement. I failed to say it but what I should have = asked was for an initiator generated ping with the ITT !=3D 0xFFFFFFFF.=20 Actually, you sparked a thought... it allows the ping code to be in a = separate thread that does not have access to CmdSN numbering. Eddy ----- Original Message -----=20 From: Dean Scoville=20 To: Eddy Quicksall ; ips@ietf.org=20 Sent: Friday, October 22, 2004 11:32 AM Subject: RE: [Ips] use of the I bit in a nop-out According to section 10.18.1, the I-bit MUST be set to 1 whenever = ITT=3D0xffffffff in the NOP-Out PDU. The I-bit keeps the CmdSN from advancing when the initiator either = replies to a NOP-In ping sent by the target or when it sends a NOP-Out to update the target's = counters (without requiring a reply). Dean -----Original Message----- From: ips-bounces@ietf.org [mailto:ips-bounces@ietf.org]On Behalf Of = Eddy Quicksall Sent: Thursday, October 21, 2004 3:36 PM To: ips@ietf.org Subject: [Ips] use of the I bit in a nop-out Can anyone point out a good use for the I bit in a NOP-out? Eddy ------=_NextPart_000_000D_01C4B83B.9CDA7400 Content-Type: text/html; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable
Yes, that is a requirement.  I failed to = say it but=20 what I should have asked was for an initiator generated ping with the = ITT !=3D=20 0xFFFFFFFF.
 
Actually, you sparked a thought... it allows the = ping code to=20 be in a separate thread that does not have access to CmdSN=20 numbering.
 
Eddy
----- Original Message -----
From:=20 Dean=20 Scoville
To: Eddy = Quicksall ; ips@ietf.org =
Sent: Friday, October 22, 2004 = 11:32=20 AM
Subject: RE: [Ips] use of the I = bit in a=20 nop-out

According to section 10.18.1, the I-bit MUST be set to 1=20 whenever ITT=3D0xffffffff in the NOP-Out PDU.
The=20 I-bit keeps the CmdSN from advancing when the initiator either replies = to a=20 NOP-In ping
sent=20 by the target or when it sends a NOP-Out to update the target's = counters=20 (without requiring
a=20 reply).
 
Dean
-----Original Message-----
From: ips-bounces@ietf.org=20 [mailto:ips-bounces@ietf.org]On Behalf Of Eddy=20 Quicksall
Sent: Thursday, October 21, 2004 3:36 = PM
To:=20 ips@ietf.org
Subject: [Ips] use of the I bit in a=20 nop-out

Can anyone point out a good use for the I bit in = a=20 NOP-out?
 
Eddy
------=_NextPart_000_000D_01C4B83B.9CDA7400-- --===============1308650629== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline Content-Transfer-Encoding: 7bit _______________________________________________ Ips mailing list Ips@ietf.org https://www1.ietf.org/mailman/listinfo/ips --===============1308650629==-- From ips-bounces@ietf.org Fri Oct 22 14:04:06 2004 Received: from ietf-mx.ietf.org (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id OAA25254 for ; Fri, 22 Oct 2004 14:04:06 -0400 (EDT) Received: from megatron.ietf.org ([132.151.6.71]) by ietf-mx.ietf.org with esmtp (Exim 4.33) id 1CL3ye-0003OP-J8 for ips-web-archive@ietf.org; Fri, 22 Oct 2004 14:17:24 -0400 Received: from localhost.localdomain ([127.0.0.1] helo=megatron.ietf.org) by megatron.ietf.org with esmtp (Exim 4.32) id 1CL3ga-0006JL-KX; Fri, 22 Oct 2004 13:58:44 -0400 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by megatron.ietf.org with esmtp (Exim 4.32) id 1CL3ah-000527-EF for ips@megatron.ietf.org; Fri, 22 Oct 2004 13:52:39 -0400 Received: from ietf-mx.ietf.org (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id NAA24506 for ; Fri, 22 Oct 2004 13:52:36 -0400 (EDT) Received: from mtagate2.de.ibm.com ([195.212.29.151]) by ietf-mx.ietf.org with esmtp (Exim 4.33) id 1CL3nW-0003Ak-6f for ips@ietf.org; Fri, 22 Oct 2004 14:05:54 -0400 Received: from d12nrmr1507.megacenter.de.ibm.com (d12nrmr1507.megacenter.de.ibm.com [9.149.167.1]) by mtagate2.de.ibm.com (8.12.10/8.12.10) with ESMTP id i9MHq0xD143884; Fri, 22 Oct 2004 17:52:00 GMT Received: from d12ml102.megacenter.de.ibm.com (d12av02.megacenter.de.ibm.com [9.149.165.228]) by d12nrmr1507.megacenter.de.ibm.com (8.12.10/NCO/VER6.6) with ESMTP id i9MHpxtm218894; Fri, 22 Oct 2004 19:51:59 +0200 In-Reply-To: <001401c4b846$58c5b530$ea01a8c0@ivvt2dxrc11> To: "Eddy Quicksall" Subject: Re: [Ips] PDUs allowed on discovery sessions MIME-Version: 1.0 X-Mailer: Lotus Notes Build V70_M2_07222004 Beta 2NP July 22, 2004 From: Julian Satran Message-ID: Date: Fri, 22 Oct 2004 12:51:59 -0500 X-MIMETrack: Serialize by Router on D12ML102/12/M/IBM(Release 6.5.1| March 5, 2004) at 22/10/2004 19:51:59, Serialize complete at 22/10/2004 19:51:59 X-Spam-Score: 0.5 (/) X-Scan-Signature: a0534e6179a1e260079328e8b03c7901 Cc: ips@ietf.org, Paul Koning X-BeenThere: ips@ietf.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: IP Storage List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Content-Type: multipart/mixed; boundary="===============1769711243==" Sender: ips-bounces@ietf.org Errors-To: ips-bounces@ietf.org X-Spam-Score: 0.5 (/) X-Scan-Signature: bcd240e64c427d3d3617cfc704e7fd7f This is a multipart message in MIME format. --===============1769711243== Content-Type: multipart/alternative; boundary="=_alternative 0060423385256F35_=" This is a multipart message in MIME format. --=_alternative 0060423385256F35_= Content-Type: text/plain; charset="US-ASCII" Eddy, The protocol does not forbid you to keep a session open. It just doesn't allow you to do anything to make it really work :-) (i.e., usefull to sense changes). As to all your other questions (how long, what is abuse) it is all up to the implementer. Julo "Eddy Quicksall" 22/10/04 10:49 To "Paul Koning" cc , Julian Satran/Haifa/IBM@IBMIL Subject Re: [Ips] PDUs allowed on discovery sessions > I never implied #1, because the spec says that an initiator CAN keep a > discovery session open. For a long discovery session, what procedure would you suggest that the target use to determine if the initiator is still active? e.g., for a normal session, section 10.18 says: A NOP-Out may be used by an initiator as a "ping request" to verify that a connection/session is still active and all its components are operational. and 10.19 says: NOP-In is either sent by a target ..., as a "ping" to an initiator, or ... BTW, I have no problem if some words were added to the spec that say something like: An initiator MUST not leave a discover session open for longer than Time2Retain seconds. Or--- Allow NOP-out in discovery sessions. Eddy ----- Original Message ----- From: "Paul Koning" To: Cc: ; Sent: Friday, October 22, 2004 9:53 AM Subject: Re: [Ips] PDUs allowed on discovery sessions >>>>>> "Eddy" == Eddy Quicksall >>>>>> writes: > > Eddy> I'm a target. Can you please give me a reference in the spec > Eddy> that says: 1) an initiator can't keep a discover session open > Eddy> 2) if it does, how long does the target wait before dropping > Eddy> the connection > > I never implied #1, because the spec says that an initiator CAN keep a > discovery session open. What I said is that this is not common > practice and not a particularly useful thing to do, and I would not do > it if I were implementing an iniatiator. > > As for #2, the spec does not say, and I see no reason why it should. > That's a matter of local choice. > > paul > --=_alternative 0060423385256F35_= Content-Type: text/html; charset="US-ASCII"
Eddy,

The protocol does not forbid you to keep a session open.
It just doesn't allow you to do anything to make it really work :-) (i.e., usefull to sense changes).
As to all your other questions (how long, what is abuse) it is all up to the implementer.


Julo


"Eddy Quicksall" <eddy_quicksall_iVivity_iSCSI@comcast.net>

22/10/04 10:49

To
"Paul Koning" <pkoning@equallogic.com>
cc
<ips@ietf.org>, Julian Satran/Haifa/IBM@IBMIL
Subject
Re: [Ips] PDUs allowed on discovery sessions





> I never implied #1, because the spec says that an initiator CAN keep a
> discovery session open.

For a long discovery session, what procedure would you suggest that the
target use to determine if the initiator is still active?

e.g., for a normal session, section 10.18 says:
  A NOP-Out may be used by an initiator as a "ping request" to verify

  that a connection/session is still active and all its components are

  operational.

and 10.19 says:
  NOP-In is either sent by a target ..., as a  "ping" to an initiator, or
...


BTW, I have no problem if some words were added to the spec that say
something like:

An initiator MUST not leave a discover session open for longer than
Time2Retain seconds.

Or---

Allow NOP-out in discovery sessions.

Eddy

----- Original Message -----
From: "Paul Koning" <pkoning@equallogic.com>
To: <eddy_quicksall_iVivity_iSCSI@comcast.net>
Cc: <ips@ietf.org>; <Julian_Satran@il.ibm.com>
Sent: Friday, October 22, 2004 9:53 AM
Subject: Re: [Ips] PDUs allowed on discovery sessions


>>>>>> "Eddy" == Eddy Quicksall <eddy_quicksall_iVivity_iSCSI@comcast.net>
>>>>>> writes:
>
> Eddy> I'm a target. Can you please give me a reference in the spec
> Eddy> that says: 1) an initiator can't keep a discover session open
> Eddy> 2) if it does, how long does the target wait before dropping
> Eddy> the connection
>
> I never implied #1, because the spec says that an initiator CAN keep a
> discovery session open.  What I said is that this is not common
> practice and not a particularly useful thing to do, and I would not do
> it if I were implementing an iniatiator.
>
> As for #2, the spec does not say, and I see no reason why it should.
> That's a matter of local choice.
>
>       paul
>


--=_alternative 0060423385256F35_=-- --===============1769711243== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline Content-Transfer-Encoding: 7bit _______________________________________________ Ips mailing list Ips@ietf.org https://www1.ietf.org/mailman/listinfo/ips --===============1769711243==-- From ips-bounces@ietf.org Fri Oct 22 14:49:38 2004 Received: from ietf-mx.ietf.org (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id OAA28745 for ; Fri, 22 Oct 2004 14:49:38 -0400 (EDT) Received: from megatron.ietf.org ([132.151.6.71]) by ietf-mx.ietf.org with esmtp (Exim 4.33) id 1CL4gj-0004Ec-M0 for ips-web-archive@ietf.org; Fri, 22 Oct 2004 15:02:57 -0400 Received: from localhost.localdomain ([127.0.0.1] helo=megatron.ietf.org) by megatron.ietf.org with esmtp (Exim 4.32) id 1CL4QD-0002xy-30; Fri, 22 Oct 2004 14:45:53 -0400 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by megatron.ietf.org with esmtp (Exim 4.32) id 1CL4Ez-0008OZ-MK for ips@megatron.ietf.org; Fri, 22 Oct 2004 14:34:17 -0400 Received: from ietf-mx.ietf.org (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id OAA27485 for ; Fri, 22 Oct 2004 14:34:14 -0400 (EDT) Received: from sadr.equallogic.com ([66.155.203.134]) by ietf-mx.ietf.org with smtp (Exim 4.33) id 1CL4Rf-0003ur-4C for ips@ietf.org; Fri, 22 Oct 2004 14:47:33 -0400 Received: from sadr.equallogic.com (localhost.localdomain [127.0.0.1]) by sadr.equallogic.com (8.12.8/8.12.8) with ESMTP id i9MIXW2D007846 for ; Fri, 22 Oct 2004 14:33:33 -0400 Received: from M30.equallogic.com (m30 [172.16.1.30]) by sadr.equallogic.com (8.12.8/8.12.8) with SMTP id i9MIXWqn007841; Fri, 22 Oct 2004 14:33:32 -0400 Received: from PKONING.equallogic.com ([172.16.1.105]) by M30.equallogic.com with Microsoft SMTPSVC(5.0.2195.6713); Fri, 22 Oct 2004 14:33:32 -0400 MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Message-ID: <16761.21115.122000.858059@gargle.gargle.HOWL> Date: Fri, 22 Oct 2004 14:33:31 -0400 From: Paul Koning To: eddy_quicksall_iVivity_iSCSI@comcast.net Subject: Re: [Ips] PDUs allowed on discovery sessions References: <004101c4b7c1$9fc893b0$ea01a8c0@ivvt2dxrc11> <16760.18708.39000.743110@gargle.gargle.HOWL> <000501c4b831$4f141140$ea01a8c0@ivvt2dxrc11> <16761.4339.144000.509412@gargle.gargle.HOWL> <001401c4b846$58c5b530$ea01a8c0@ivvt2dxrc11> X-Mailer: VM 7.07 under 21.4 (patch 10) "Military Intelligence (Windows)" XEmacs Lucid X-OriginalArrivalTime: 22 Oct 2004 18:33:32.0352 (UTC) FILETIME=[A2121C00:01C4B865] X-Spam-Score: 0.0 (/) X-Scan-Signature: b19722fc8d3865b147c75ae2495625f2 Content-Transfer-Encoding: 7bit Cc: ips@ietf.org, Julian_Satran@il.ibm.com X-BeenThere: ips@ietf.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: IP Storage List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Sender: ips-bounces@ietf.org Errors-To: ips-bounces@ietf.org X-Spam-Score: 0.0 (/) X-Scan-Signature: 69a74e02bbee44ab4f8eafdbcedd94a1 Content-Transfer-Encoding: 7bit >>>>> "Eddy" == Eddy Quicksall writes: >> I never implied #1, because the spec says that an initiator CAN >> keep a discovery session open. Eddy> For a long discovery session, what procedure would you suggest Eddy> that the target use to determine if the initiator is still Eddy> active? Why do anything? It seems perfectly ok to keep it open for a bit if that's the initiators's choice, and then to disconnect it when the resources could be better used elsewhere. Or just always after some number of seconds. I don't know what useful information I'd get out of a NOP that might change my decisionmaking in the case of discovery sessions. Eddy> BTW, I have no problem if some words were added to the spec Eddy> that say something like: Eddy> An initiator MUST not leave a discover session open for longer Eddy> than Time2Retain seconds. I suppose we could do that, but would it make the standard better? It's not clear to me. Eddy> Allow NOP-out in discovery sessions. Same comment. Yes, that would be possible -- but I don't understand how it adds value. paul _______________________________________________ Ips mailing list Ips@ietf.org https://www1.ietf.org/mailman/listinfo/ips From ips-bounces@ietf.org Fri Oct 22 18:20:49 2004 Received: from ietf-mx.ietf.org (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id SAA02457 for ; Fri, 22 Oct 2004 18:20:49 -0400 (EDT) Received: from megatron.ietf.org ([132.151.6.71]) by ietf-mx.ietf.org with esmtp (Exim 4.33) id 1CL7z8-0005fQ-S5 for ips-web-archive@ietf.org; Fri, 22 Oct 2004 18:34:11 -0400 Received: from localhost.localdomain ([127.0.0.1] helo=megatron.ietf.org) by megatron.ietf.org with esmtp (Exim 4.32) id 1CL6wr-0001Sz-M4; Fri, 22 Oct 2004 17:27:45 -0400 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by megatron.ietf.org with esmtp (Exim 4.32) id 1CL6XL-0003dF-8V for ips@megatron.ietf.org; Fri, 22 Oct 2004 17:01:23 -0400 Received: from ietf-mx.ietf.org (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id RAA15580 for ; Fri, 22 Oct 2004 17:01:18 -0400 (EDT) Received: from rwcrmhc12.comcast.net ([216.148.227.85]) by ietf-mx.ietf.org with esmtp (Exim 4.33) id 1CL6kA-0000Nl-Aw for ips@ietf.org; Fri, 22 Oct 2004 17:14:39 -0400 Received: from ivvt2dxrc11 (unknown[64.238.111.98]) by comcast.net (rwcrmhc12) with SMTP id <20041022210046014003s387e> (Authid: esquicksall); Fri, 22 Oct 2004 21:00:47 +0000 Message-ID: <001701c4b87a$2713dab0$3f01a8c0@ivvt2dxrc11> From: "Eddy Quicksall" To: "Paul Koning" References: <004101c4b7c1$9fc893b0$ea01a8c0@ivvt2dxrc11><16760.18708.39000.743110@gargle.gargle.HOWL><000501c4b831$4f141140$ea01a8c0@ivvt2dxrc11><16761.4339.144000.509412@gargle.gargle.HOWL><001401c4b846$58c5b530$ea01a8c0@ivvt2dxrc11> <16761.21115.122000.858059@gargle.gargle.HOWL> Subject: Re: [Ips] PDUs allowed on discovery sessions Date: Fri, 22 Oct 2004 17:00:23 -0400 MIME-Version: 1.0 Content-Type: text/plain; format=flowed; charset="iso-8859-1"; reply-type=original Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 6.00.2900.2180 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2900.2180 X-Spam-Score: 0.0 (/) X-Scan-Signature: 7aafa0432175920a4b3e118e16c5cb64 Content-Transfer-Encoding: 7bit Cc: ips@ietf.org, Julian_Satran@il.ibm.com X-BeenThere: ips@ietf.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: IP Storage List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Sender: ips-bounces@ietf.org Errors-To: ips-bounces@ietf.org X-Spam-Score: 0.0 (/) X-Scan-Signature: 0a7aa2e6e558383d84476dc338324fab Content-Transfer-Encoding: 7bit In a normal session the standard has given a method (the ping) so the initiator and/or target can determine if the partner is still active. I see no reason why a discovery session should be an exception to this. Yes, it would make the standard better if this was made clear. i.e. if it said that long discovery sessions are illegal and how long the initiator should wait to close the connection or even better, to allow the NOP ping. It seems to me that this was simply an oversight in the text of the spec. Eddy ----- Original Message ----- From: "Paul Koning" To: Cc: ; Sent: Friday, October 22, 2004 2:33 PM Subject: Re: [Ips] PDUs allowed on discovery sessions >>>>>> "Eddy" == Eddy Quicksall >>>>>> writes: > > >> I never implied #1, because the spec says that an initiator CAN > >> keep a discovery session open. > > Eddy> For a long discovery session, what procedure would you suggest > Eddy> that the target use to determine if the initiator is still > Eddy> active? > > Why do anything? It seems perfectly ok to keep it open for a bit if > that's the initiators's choice, and then to disconnect it when the > resources could be better used elsewhere. Or just always after some > number of seconds. > > I don't know what useful information I'd get out of a NOP that might > change my decisionmaking in the case of discovery sessions. > > Eddy> BTW, I have no problem if some words were added to the spec > Eddy> that say something like: > > Eddy> An initiator MUST not leave a discover session open for longer > Eddy> than Time2Retain seconds. > > I suppose we could do that, but would it make the standard better? > It's not clear to me. > > Eddy> Allow NOP-out in discovery sessions. > > Same comment. Yes, that would be possible -- but I don't understand > how it adds value. > > paul > _______________________________________________ Ips mailing list Ips@ietf.org https://www1.ietf.org/mailman/listinfo/ips From ips-bounces@ietf.org Fri Oct 22 19:49:47 2004 Received: from ietf-mx.ietf.org (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id TAA14685 for ; Fri, 22 Oct 2004 19:49:47 -0400 (EDT) Received: from megatron.ietf.org ([132.151.6.71]) by ietf-mx.ietf.org with esmtp (Exim 4.33) id 1CL9NG-0000zq-AE for ips-web-archive@ietf.org; Fri, 22 Oct 2004 20:03:10 -0400 Received: from localhost.localdomain ([127.0.0.1] helo=megatron.ietf.org) by megatron.ietf.org with esmtp (Exim 4.32) id 1CL98V-0004IQ-GX; Fri, 22 Oct 2004 19:47:55 -0400 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by megatron.ietf.org with esmtp (Exim 4.32) id 1CL90l-00086T-5p for ips@megatron.ietf.org; Fri, 22 Oct 2004 19:39:57 -0400 Received: from ietf-mx.ietf.org (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id TAA14118 for ; Fri, 22 Oct 2004 19:39:53 -0400 (EDT) From: elizabeth.rodriguez@dothill.com Received: from artecon.dothill.com ([155.254.128.254] helo=dothill.com) by ietf-mx.ietf.org with esmtp (Exim 4.33) id 1CL9Df-0000nn-Rk for ips@ietf.org; Fri, 22 Oct 2004 19:53:16 -0400 Received: from exchange.artecon.com (exchange [206.6.182.75]) by dothill.com (8.12.9+Sun/8.12.5) with ESMTP id i9MNXwLl009268; Fri, 22 Oct 2004 16:33:59 -0700 (PDT) Received: by EXCHANGE with Internet Mail Service (5.5.2653.19) id <4M1493PQ>; Fri, 22 Oct 2004 16:38:30 -0700 Message-ID: To: ips@ietf.org Subject: [Ips] MIB Doctor review: ips-fcip-mib-5-reviewed.txt Date: Fri, 22 Oct 2004 16:42:51 -0700 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2653.19) Content-Type: text/plain X-Spam-Score: 0.3 (/) X-Scan-Signature: ba0d4c5f57f7c289496fce758bbf4798 Cc: T11_5@t11.org, t11_3@t11.org X-BeenThere: ips@ietf.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: IP Storage List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Sender: ips-bounces@ietf.org Errors-To: ips-bounces@ietf.org X-Spam-Score: 0.3 (/) X-Scan-Signature: f8184d7d4d1b986353eb58ea3e887935 All, A new FCIP MIB will soon be published (it has been submitted to the ID office. Below is a list of issues/changes requested from the expert review of the MIB along with the resolution to the comments. If anyone sees any issues with the new MIB, please let David or I know or post to the IPS mailing list ASAP. Thanks to Anil and Ravi for getting this updated. Elizabeth -----Original Message----- From: anilr@accton.com [mailto:anilr@accton.com] Sent: Friday, October 22, 2004 1:51 PM To: Elizabeth Rodriguez Cc: black_david@emc.com; kzm@cisco.com; ravin@lightsand.com Subject: Re: [Ips] MIB Doctor review: ips-fcip-mib-5-reviewed.txt Elizabeth, Keith, Mike: Ravi and I have submitted an updated draft-ietf-ips-fcip-mib-06.txt (attached below), addressing the comments from MIB Doctor (Mike M) and Elizabeth. Please find my responses marked with Anil> within the text below. Elizabeth wrote: One thing that has changed since you first published this MIB is the boilerplate and guidelines for the MIB. I am including a sample boilerplate that some have been using, for your reference. Anil> Updated. MIB Doctor (Mike MacFaden) wrote: This is full review. A. Documentation Guidelines o Correct MIB boilerplate (3.1) -> present o Narrative Section (3.2) -> present, needs work o Definitions Section (3.3) -> present o Intellectual Property Section (3.4) -> present, does not match sectio 5, rfc3668. Anil> Fixed. o References section (3.5) -> present, missing iana url for iftype Anil> Added. o Security section (3.6) -> present o IANA Considerations Section (3.7) -> warning, has "mib-2 8889" instead of "XXX" Anil> Fixed. o Copyright Notices section (3.8) -> present, - Copyright found in MODULE-IDENTITY? -> NO Anil> Added. o Spelling? -> 2 words (occurred, conections) Anil> Corrected. o Definitions section detail Here are the diagnostics from certain mib syntax checkers: Diagnostics from Version 0.4.2 smilint version 0.4.2 of smilint -m -s -l 6 --> no errors detected. Bill Fenner's automated check here: http://www.icir.org/fenner/mibs/htmllint/FCIP-MGMT-MIB.html Comments/Nits: 1. Narrative section should use complete token: "described in [BB2]," should be "[FCBB2]" Anil> Fixed. 2. Description for fcipEntityTcpConPort needs to remove duplicate: "entity entity" Anil> Fixed. 3. Description for fcipLinkFcipBB2LkaTimeOuts mispells "occured" Anil> Fixed. 4. Description for fcipTcpConnGroup mispells "conections" Anil> Fixed. 5. Narrative section could use an ascii art picture depicting FC Device/FC Entity/FCIP Link, Data Engine relationship to make the relationships easier to grasp. Anil> Added. Primary reference is always in base FCIP draft. 6. Narrative section 2.5: interpreted the prose to mean at any point in time either the Dynamic Route Table is populated or the Static Route Table is populated. I am not clear as to how to determine which table is being used. Anil> Added wording to indicate that both can co-exist, and Static Route Table takes precedence in case of conflicts. 7. Narative "3. Relationship to other MIBs" states: Objects accessible from other MIBs applicable to FCIP devices have not been included in this MIB. I would add to this introduction something like : The following lists all other applicable MIB modules/objects that should be present with FCIP-MGMT-MIB> Anil> Added. 8. IANA ifType 'fcipLink' not in this document yet, will need to be before last call. Anil> Added. 9. Section 3.2 relation to Interfaces. This section should describe what should go into the ifStackTable for ifType 'fcipLink' or ifXTable. Suggest you set ifUpper/ifLower for this ifIndex to 0. That is the simple answer and will not lead to an explosion of entries in ifStackTable. FC-MGMT-MIB is a good example for how to document interaction with ifTable/ifXTable/ifStackTable. My suspicion is that there is no need to have this ifEntry at all given the descriptions presented. Anil> Given that fcipLink interfaces may be viewed as independant of the physical interfaces, there does not appear to be a lot of value in specifying ifStackTable entries. Also looked at FC-MGMT-MIB for guidance, no mention of ifStackTable. We feel that entries in ifTable suffice to describe these interfaces, and additional ifStackTable entries would serve little purpose. 10. Section 3.4 shouldn't need such a statement. Typically one is narrowing/clarifying the definition of ifTable objects for the particular interface. Any "conflict" should be revised in the working group. "(RFC 2863 takes precedence over these guidelines in the event of any conflict.)" Anil> Statement removed. 11. Definition of "ifSpeed" is ambiguous. If you do a bind(INADDR_ANY) then which speed should you use? Anil> ifSpeed of an FCIP Link is the same as that of the physical interface. At best, it would be able to use the full bandwidth of the interface.There is no way to gauge the available speed at any given instant. 12. ifPromiscuousMode This will normally be 'false' so under what conditions would it be true? Anil> It will never be true. Accordingly, removed the word "normally". 13. Suggest: ifConnectorPresent This will normally be 'false'. you plainly state this is set to 'false' Anil> Done. 14. Section 3.3 seems rather shallow -- not sure how this mib module generally affects this one. Anil> Only point we were making here is that on an FCIP device, there is bound to be an FC interface(s), and that we expect the FC portion of the management is fully handled by the FC Mgmt MIB. 15. FcEntityMode -> does not give guidance when to use other(1), assume that might be used if the value is not yet set? Anil> At this time, there is no other known interoperable method. Anil> Removed the value 'other'. 16. Can the items in the FCIP group be changed at any time or is there some state a FCIP Device should be in before one changes fcipDynIpConfType, fcipEntitySACKOption? Anil> There is no state an FCIP Device needs to be in before such changes. 17. RFC 1323, 2474 not listed in Normative reference section. Anil> Added. 18. fcipEntityName should use SnmpAdminString not DisplayString Anil> Done. 19. I assume if I configure a fcipEntityInstanceTable and set it to active(1), then the tcpConnTable(if ipv4) will be visible to show the TCP state (eg LISTEN)? Might want to mention what setting fcipLinkStatus to active will do. Also describe what setting it to destroy(6) will do. RMON2 mib provides good examples of documenting what your RowStatus object does. Same issue for fcipLinkStatus At a minimum, specify what needs to be set in order to make a row active, and define the defaults for those optional ones. fcipStaticRouteStatus too -- wasn't sure what is considered valid agent behavior to create rows in this table if dynamic routing was active. Anil> Updated descriptions of fcipEntityRowStatus, fcipLinkStatus, and fcipStaticRouteStatus to specify behavior for active(1) and destroy(6) values. 20. fcipLinkLocalFcipEntityMode -- should the be read-only? Anil> Yes, it's read-only. 21. fcipTcpConnTable might benefit from a TimeStamp object containing the creating time of the tcp session. Anil> We could not agree upon sufficient need to add the above object. 22. fcipDiscoveryDomainName is SYNTAX DisplayString --> should be SnmpAdminString? Anil> Yes, changed. 23. fcipLinkErrorsTable doesn't list any discontinutity object. Assuming default. Anil> Could not find sufficient justification for such an object. Anil> However, note that links have corresponding interfaces which do have continuity objects. 24. Copyright should be updated to 2004 at next revision Anil> Fixed. 26. Section 6 lists objects but does not describe why viewing the identification or topology information is considered senstive. No mention of writable objects that if abused could lead to disruption are described. See section 3.6 of [1]. Anil> Added two writable objects to section. As for the readable objects already specified, we feel that specifying that these are identification or topology information tells you why they are considered sensitive (which is essentially what the FC Mgmt MIB does). MIB has been updated and verified to be free of compile errors. Change log added to reflect changes since the -05 version. Regards, Anil R. and Ravi N. (See attached file: draft-ietf-ips-fcip-mib-06.txt) _______________________________________________ Ips mailing list Ips@ietf.org https://www1.ietf.org/mailman/listinfo/ips From ips-bounces@ietf.org Fri Oct 22 22:39:27 2004 Received: from ietf-mx.ietf.org (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id WAA05826 for ; Fri, 22 Oct 2004 22:39:26 -0400 (EDT) Received: from megatron.ietf.org ([132.151.6.71]) by ietf-mx.ietf.org with esmtp (Exim 4.33) id 1CLC1S-0003eb-Li for ips-web-archive@ietf.org; Fri, 22 Oct 2004 22:52:51 -0400 Received: from localhost.localdomain ([127.0.0.1] helo=megatron.ietf.org) by megatron.ietf.org with esmtp (Exim 4.32) id 1CLBjw-00078a-HJ; Fri, 22 Oct 2004 22:34:44 -0400 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by megatron.ietf.org with esmtp (Exim 4.32) id 1CLBfp-0005Er-Ng; Fri, 22 Oct 2004 22:30:29 -0400 Received: from ietf-mx.ietf.org (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id WAA05312; Fri, 22 Oct 2004 22:30:28 -0400 (EDT) Received: from mtagate4.de.ibm.com ([195.212.29.153]) by ietf-mx.ietf.org with esmtp (Exim 4.33) id 1CLBsl-0003Vz-NS; Fri, 22 Oct 2004 22:43:52 -0400 Received: from d12nrmr1607.megacenter.de.ibm.com (d12nrmr1607.megacenter.de.ibm.com [9.149.167.49]) by mtagate4.de.ibm.com (8.12.10/8.12.10) with ESMTP id i9N2TpxX098244; Sat, 23 Oct 2004 02:29:51 GMT Received: from d12ml102.megacenter.de.ibm.com (d12av02.megacenter.de.ibm.com [9.149.165.228]) by d12nrmr1607.megacenter.de.ibm.com (8.12.10/NCO/VER6.6) with ESMTP id i9N2ToWP186010; Sat, 23 Oct 2004 04:29:50 +0200 In-Reply-To: <001701c4b87a$2713dab0$3f01a8c0@ivvt2dxrc11> To: "Eddy Quicksall" Subject: Re: [Ips] PDUs allowed on discovery sessions MIME-Version: 1.0 X-Mailer: Lotus Notes Build V70_M2_07222004 Beta 2NP July 22, 2004 From: Julian Satran Message-ID: Date: Fri, 22 Oct 2004 21:29:48 -0500 X-MIMETrack: Serialize by Router on D12ML102/12/M/IBM(Release 6.5.1| March 5, 2004) at 23/10/2004 04:29:50, Serialize complete at 23/10/2004 04:29:50 X-Spam-Score: 0.5 (/) X-Scan-Signature: ff0adf256e4dd459cc25215cfa732ac1 Cc: ips@ietf.org, ips-bounces@ietf.org, Paul Koning X-BeenThere: ips@ietf.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: IP Storage List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Content-Type: multipart/mixed; boundary="===============0217480525==" Sender: ips-bounces@ietf.org Errors-To: ips-bounces@ietf.org X-Spam-Score: 0.0 (/) X-Scan-Signature: b045c2b078f76b9f842d469de8a32de3 This is a multipart message in MIME format. --===============0217480525== Content-Type: multipart/alternative; boundary="=_alternative 000A9BE285256F36_=" This is a multipart message in MIME format. --=_alternative 000A9BE285256F36_= Content-Type: text/plain; charset="US-ASCII" Eddy, I can assure you that it was no oversight. Saying that long sessions are illegal would have required us to specify what is "long". As this is very much application and implementation dependent we just made illegal using the means by which long idle sessions are maintained. The standard does not say either at what interval, and if at all, NOPs mus be used in regular sessions. Julo "Eddy Quicksall" Sent by: ips-bounces@ietf.org 22/10/04 17:00 To "Paul Koning" cc ips@ietf.org, Julian Satran/Haifa/IBM@IBMIL Subject Re: [Ips] PDUs allowed on discovery sessions In a normal session the standard has given a method (the ping) so the initiator and/or target can determine if the partner is still active. I see no reason why a discovery session should be an exception to this. Yes, it would make the standard better if this was made clear. i.e. if it said that long discovery sessions are illegal and how long the initiator should wait to close the connection or even better, to allow the NOP ping. It seems to me that this was simply an oversight in the text of the spec. Eddy ----- Original Message ----- From: "Paul Koning" To: Cc: ; Sent: Friday, October 22, 2004 2:33 PM Subject: Re: [Ips] PDUs allowed on discovery sessions >>>>>> "Eddy" == Eddy Quicksall >>>>>> writes: > > >> I never implied #1, because the spec says that an initiator CAN > >> keep a discovery session open. > > Eddy> For a long discovery session, what procedure would you suggest > Eddy> that the target use to determine if the initiator is still > Eddy> active? > > Why do anything? It seems perfectly ok to keep it open for a bit if > that's the initiators's choice, and then to disconnect it when the > resources could be better used elsewhere. Or just always after some > number of seconds. > > I don't know what useful information I'd get out of a NOP that might > change my decisionmaking in the case of discovery sessions. > > Eddy> BTW, I have no problem if some words were added to the spec > Eddy> that say something like: > > Eddy> An initiator MUST not leave a discover session open for longer > Eddy> than Time2Retain seconds. > > I suppose we could do that, but would it make the standard better? > It's not clear to me. > > Eddy> Allow NOP-out in discovery sessions. > > Same comment. Yes, that would be possible -- but I don't understand > how it adds value. > > paul > _______________________________________________ Ips mailing list Ips@ietf.org https://www1.ietf.org/mailman/listinfo/ips --=_alternative 000A9BE285256F36_= Content-Type: text/html; charset="US-ASCII"
Eddy,

I can assure you that it was no oversight. Saying that long sessions are illegal would have required us to specify what is "long".
As this is very much application and implementation dependent we just made illegal using the means by which long idle sessions are maintained.
The standard does not say either at what interval, and if at all, NOPs mus be used in regular sessions.


Julo


"Eddy Quicksall" <eddy_quicksall_iVivity_iSCSI@comcast.net>
Sent by: ips-bounces@ietf.org

22/10/04 17:00

To
"Paul Koning" <pkoning@equallogic.com>
cc
ips@ietf.org, Julian Satran/Haifa/IBM@IBMIL
Subject
Re: [Ips] PDUs allowed on discovery sessions





In a normal session the standard has given a method (the ping) so the
initiator and/or target can determine if the partner is still active. I see
no reason why a discovery session should be an exception to this.

Yes, it would make the standard better if this was made clear. i.e. if it
said that long discovery sessions are illegal and how long the initiator
should wait to close the connection or even better, to allow the NOP ping.
It seems to me that this was simply an oversight in the text of the spec.

Eddy

----- Original Message -----
From: "Paul Koning" <pkoning@equallogic.com>
To: <eddy_quicksall_iVivity_iSCSI@comcast.net>
Cc: <ips@ietf.org>; <Julian_Satran@il.ibm.com>
Sent: Friday, October 22, 2004 2:33 PM
Subject: Re: [Ips] PDUs allowed on discovery sessions


>>>>>> "Eddy" == Eddy Quicksall <eddy_quicksall_iVivity_iSCSI@comcast.net>
>>>>>> writes:
>
> >> I never implied #1, because the spec says that an initiator CAN
> >> keep a discovery session open.
>
> Eddy> For a long discovery session, what procedure would you suggest
> Eddy> that the target use to determine if the initiator is still
> Eddy> active?
>
> Why do anything?  It seems perfectly ok to keep it open for a bit if
> that's the initiators's choice, and then to disconnect it when the
> resources could be better used elsewhere.  Or just always after some
> number of seconds.
>
> I don't know what useful information I'd get out of a NOP that might
> change my decisionmaking in the case of discovery sessions.
>
> Eddy> BTW, I have no problem if some words were added to the spec
> Eddy> that say something like:
>
> Eddy> An initiator MUST not leave a discover session open for longer
> Eddy> than Time2Retain seconds.
>
> I suppose we could do that, but would it make the standard better?
> It's not clear to me.
>
> Eddy> Allow NOP-out in discovery sessions.
>
> Same comment.  Yes, that would be possible -- but I don't understand
> how it adds value.
>
>    paul
>


_______________________________________________
Ips mailing list
Ips@ietf.org
https://www1.ietf.org/mailman/listinfo/ips
--=_alternative 000A9BE285256F36_=-- --===============0217480525== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline Content-Transfer-Encoding: 7bit _______________________________________________ Ips mailing list Ips@ietf.org https://www1.ietf.org/mailman/listinfo/ips --===============0217480525==-- From ips-bounces@ietf.org Mon Oct 25 20:01:23 2004 Received: from ietf-mx.ietf.org (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id UAA21048 for ; Mon, 25 Oct 2004 20:01:23 -0400 (EDT) Received: from megatron.ietf.org ([132.151.6.71]) by ietf-mx.ietf.org with esmtp (Exim 4.33) id 1CMEzl-0004zI-2j for ips-web-archive@ietf.org; Mon, 25 Oct 2004 20:15:25 -0400 Received: from localhost.localdomain ([127.0.0.1] helo=megatron.ietf.org) by megatron.ietf.org with esmtp (Exim 4.32) id 1CMEUm-00028i-80; Mon, 25 Oct 2004 19:43:24 -0400 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by megatron.ietf.org with esmtp (Exim 4.32) id 1CMDRs-0006JS-3N for ips@megatron.ietf.org; Mon, 25 Oct 2004 18:36:20 -0400 Received: from ietf-mx.ietf.org (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id SAA02822 for ; Mon, 25 Oct 2004 18:36:16 -0400 (EDT) Received: from email.crossroads.com ([65.68.235.6]) by ietf-mx.ietf.org with smtp (Exim 4.33) id 1CMDfO-0007dC-HL for ips@ietf.org; Mon, 25 Oct 2004 18:50:18 -0400 Received: from mailnode1.COMMSTOR.Crossroads.com ([192.168.1.249]) by email.crossroads.com with Microsoft SMTPSVC(5.0.2195.6713); Mon, 25 Oct 2004 17:35:43 -0500 X-MimeOLE: Produced By Microsoft Exchange V6.0.6487.1 content-class: urn:content-classes:message MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable Date: Mon, 25 Oct 2004 17:35:43 -0500 Message-ID: <43F5E8AE07F17C47BC16F7A159E9E7006C39@mailnode1.commstor.crossroads.com> Thread-Topic: format errors Thread-Index: AcS64vaLEybcz/pETZuEJwZrIgmZXw== From: "Buck Landry" To: X-OriginalArrivalTime: 25 Oct 2004 22:35:43.0537 (UTC) FILETIME=[F6923A10:01C4BAE2] X-Spam-Score: 0.0 (/) X-Scan-Signature: 7aafa0432175920a4b3e118e16c5cb64 Content-Transfer-Encoding: quoted-printable Subject: [Ips] format errors X-BeenThere: ips@ietf.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: IP Storage List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Sender: ips-bounces@ietf.org Errors-To: ips-bounces@ietf.org X-Spam-Score: 0.0 (/) X-Scan-Signature: 0a7aa2e6e558383d84476dc338324fab Content-Transfer-Encoding: quoted-printable Hi folks. I'm confused about exactly what an initiator could send that = would be considered a "format error" -- as defined in rfc 3720 6.6, = something that would force a target to "immediately terminate all = transport connections in the session". I assume this means w/out = sending a Reject or Asynchronous Message first. For example, from 6.6 a "format error" can be: >>> a) Illegal contents of any PDU header field except the Opcode (legal values are specified in Section 10 iSCSI PDU Formats). <<< But in 10. I see: >>> Receipt of reserved code values in defined fields MUST be reported as a protocol error. <<< .. and "reporting" a protocol error sounds like issuing Reject, which = isn't the same as immediately terminating all transport connections. So = what exactly is "Illegal contents", then? In 6.6 we also have: >>> b) Inconsistent field contents (consistent field contents are specified in Section 10 iSCSI PDU Formats). <<< .. but the only reference to 'consistent' I see is in 10.7.4, >>> If the Target Transfer Tag is provided, then the LUN field MUST hold a valid value and be consistent with whatever was specified with the command <<< ... and this specific case seems to be covered by Reject w/"Invalid PDU = field" in 10.17.1... >>> Note 2: (...) Some examples are invalid TTT/ITT, buffer offset, LUN qualifying a TTT, (...) <<< Also, 3.2.4.3 says=20 >>> Using inconsistent field values is considered a protocol error. <<< ... does it actually mean to say "format error"? Because if it was = *merely* a protocol error, we could just Reject the PDU w/"Protocol = Error" and be on our merry way, right? I apologize for my confusion in advance; and thanks for your = consideration, --buck _______________________________________________ Ips mailing list Ips@ietf.org https://www1.ietf.org/mailman/listinfo/ips From ips-bounces@ietf.org Thu Oct 28 13:45:17 2004 Received: from ietf-mx.ietf.org (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id NAA00012 for ; Thu, 28 Oct 2004 13:45:17 -0400 (EDT) Received: from megatron.ietf.org ([132.151.6.71]) by ietf-mx.ietf.org with esmtp (Exim 4.33) id 1CNEYx-0007Q3-F1 for ips-web-archive@ietf.org; Thu, 28 Oct 2004 13:59:51 -0400 Received: from localhost.localdomain ([127.0.0.1] helo=megatron.ietf.org) by megatron.ietf.org with esmtp (Exim 4.32) id 1CNCkI-0002Nf-DW; Thu, 28 Oct 2004 12:03:26 -0400 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by megatron.ietf.org with esmtp (Exim 4.32) id 1CNBjP-0005FM-IA; Thu, 28 Oct 2004 10:58:27 -0400 Received: from CNRI.Reston.VA.US (localhost [127.0.0.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id KAA21026; Thu, 28 Oct 2004 10:58:26 -0400 (EDT) Message-Id: <200410281458.KAA21026@ietf.org> Mime-Version: 1.0 Content-Type: Multipart/Mixed; Boundary="NextPart" To: i-d-announce@ietf.org From: Internet-Drafts@ietf.org Date: Thu, 28 Oct 2004 10:58:26 -0400 Cc: ips@ietf.org Subject: [Ips] I-D ACTION:draft-ietf-ips-fcip-mib-06.txt X-BeenThere: ips@ietf.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: IP Storage List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Sender: ips-bounces@ietf.org Errors-To: ips-bounces@ietf.org X-Spam-Score: 0.4 (/) X-Scan-Signature: 14582b0692e7f70ce7111d04db3781c8 --NextPart A New Internet-Draft is available from the on-line Internet-Drafts directories. This draft is a work item of the IP Storage Working Group of the IETF. Title : Definition of Managed Objects for FCIP Author(s) : R. Natarajan, A. Rijhsinghani Filename : draft-ietf-ips-fcip-mib-06.txt Pages : 32 Date : 2004-10-27 This memo defines a portion of the Management Information Base (MIB) for use with network management protocols in TCP/IP based internets. In particular it defines objects for managing FCIP entities, as defined in [FCIP] and used in FC fabrics as described in [FCBB2]. A URL for this Internet-Draft is: http://www.ietf.org/internet-drafts/draft-ietf-ips-fcip-mib-06.txt To remove yourself from the I-D Announcement list, send a message to i-d-announce-request@ietf.org with the word unsubscribe in the body of the message. You can also visit https://www1.ietf.org/mailman/listinfo/I-D-announce to change your subscription settings. Internet-Drafts are also available by anonymous FTP. Login with the username "anonymous" and a password of your e-mail address. After logging in, type "cd internet-drafts" and then "get draft-ietf-ips-fcip-mib-06.txt". A list of Internet-Drafts directories can be found in http://www.ietf.org/shadow.html or ftp://ftp.ietf.org/ietf/1shadow-sites.txt Internet-Drafts can also be obtained by e-mail. Send a message to: mailserv@ietf.org. In the body type: "FILE /internet-drafts/draft-ietf-ips-fcip-mib-06.txt". NOTE: The mail server at ietf.org can return the document in MIME-encoded form by using the "mpack" utility. To use this feature, insert the command "ENCODING mime" before the "FILE" command. To decode the response(s), you will need "munpack" or a MIME-compliant mail reader. Different MIME-compliant mail readers exhibit different behavior, especially when dealing with "multipart" MIME messages (i.e. documents which have been split up into multiple messages), so check your local documentation on how to manipulate these messages. Below is the data which will enable a MIME compliant mail reader implementation to automatically retrieve the ASCII version of the Internet-Draft. --NextPart Content-Type: Multipart/Alternative; Boundary="OtherAccess" --OtherAccess Content-Type: Message/External-body; access-type="mail-server"; server="mailserv@ietf.org" Content-Type: text/plain Content-ID: <2004-10-28110055.I-D@ietf.org> ENCODING mime FILE /internet-drafts/draft-ietf-ips-fcip-mib-06.txt --OtherAccess Content-Type: Message/External-body; name="draft-ietf-ips-fcip-mib-06.txt"; site="ftp.ietf.org"; access-type="anon-ftp"; directory="internet-drafts" Content-Type: text/plain Content-ID: <2004-10-28110055.I-D@ietf.org> --OtherAccess-- --NextPart Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline Content-Transfer-Encoding: 7bit _______________________________________________ Ips mailing list Ips@ietf.org https://www1.ietf.org/mailman/listinfo/ips --NextPart-- From ips-bounces@ietf.org Thu Oct 28 19:48:59 2004 Received: from ietf-mx.ietf.org (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id TAA16750 for ; Thu, 28 Oct 2004 19:48:59 -0400 (EDT) Received: from megatron.ietf.org ([132.151.6.71]) by ietf-mx.ietf.org with esmtp (Exim 4.33) id 1CNKF2-0005XO-AY for ips-web-archive@ietf.org; Thu, 28 Oct 2004 20:03:40 -0400 Received: from localhost.localdomain ([127.0.0.1] helo=megatron.ietf.org) by megatron.ietf.org with esmtp (Exim 4.32) id 1CNICQ-0001NH-7u; Thu, 28 Oct 2004 17:52:50 -0400 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by megatron.ietf.org with esmtp (Exim 4.32) id 1CNFrR-0007cN-BX for ips@megatron.ietf.org; Thu, 28 Oct 2004 15:23:01 -0400 Received: from ietf-mx.ietf.org (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id PAA21347 for ; Thu, 28 Oct 2004 15:23:00 -0400 (EDT) Received: from palrel11.hp.com ([156.153.255.246]) by ietf-mx.ietf.org with esmtp (Exim 4.33) id 1CNG5Z-0005qG-8t for ips@ietf.org; Thu, 28 Oct 2004 15:37:37 -0400 Received: from rosemail.rose.hp.com (rosemail.rose.hp.com [15.43.209.160]) by palrel11.hp.com (Postfix) with ESMTP id 14CCC46B62; Thu, 28 Oct 2004 12:22:59 -0700 (PDT) Received: from [127.0.0.1] (dhcp-roseor-beb159.rose.hp.com [15.23.141.159]) by rosemail.rose.hp.com (Postfix) with ESMTP id 002E47FF4; Thu, 28 Oct 2004 12:22:58 -0700 (PDT) Message-ID: <41814708.2050802@rose.hp.com> Date: Thu, 28 Oct 2004 12:22:48 -0700 From: "Mallikarjun C." User-Agent: Mozilla Thunderbird 0.7.3 (Windows/20040803) X-Accept-Language: en-us, en MIME-Version: 1.0 To: Buck Landry Subject: Re: [Ips] format errors References: <43F5E8AE07F17C47BC16F7A159E9E7006C39@mailnode1.commstor.crossroads.com> In-Reply-To: <43F5E8AE07F17C47BC16F7A159E9E7006C39@mailnode1.commstor.crossroads.com> Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit X-Spam-Score: 0.0 (/) X-Scan-Signature: 9a2be21919e71dc6faef12b370c4ecf5 Content-Transfer-Encoding: 7bit Cc: ips@ietf.org X-BeenThere: ips@ietf.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: IP Storage List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Sender: ips-bounces@ietf.org Errors-To: ips-bounces@ietf.org X-Spam-Score: 0.0 (/) X-Scan-Signature: 7fa173a723009a6ca8ce575a65a5d813 Content-Transfer-Encoding: 7bit Hi Buck, Let me attempt to categorize the possible PDU contents as we saw them when the RFC was written. 1. Legal values - PDU header field values defined and sanctioned by the RFC as usable. - E.g., opcode values. One should process them per the RFC. 2. Reserved values - PDU header field values defined by the RFC but explicitly set aside for future use. - E.g., reserved opcode values, one should reject these with a protocol error. 3. Illegal values - PDU header field values undefined by the RFC - One should consider these as format errors. 4. Incorrect values - values of a PDU header field variable that the RFC does not put restrictions on (e.g., ITT, LUN for a TTT), but currently is known by initiator/target to be incorrect/invalid/non-existent. - Basically the cases covered by the "Invalid PDU field" Reject. 5. Inconsistent values - Cases where one PDU header field value disagrees with a different PDU header field value. - E.g., in a SCSI command PDU, ExpectedDataTransferLength != 0 and/or BidirectionalReadExpectedTransferLength != 0, but R = W = 0. One should consider these as format errors. Now, coming to the sentence in 3.2.4.3 that says "Using inconsistent field values is considered a protocol error", it is unfortunately a bit loose in that it unintentionally uses the keyword "inconsistent" claimed in the format error description (6.6). However notice that the text that precedes is actually decribing a category#4 error - i.e. an incorrect value. So the text is basically describing the right semantics - some protocol error had happened. This section doesn't tell you what to do with such an error (and that's a good thing), 10.17.1 provides that information - respond with an "Invalid PDU field" Reject. Hope that helps. -- Mallikarjun Mallikarjun Chadalapaka Networked Storage Architecture Network Storage Solutions Hewlett-Packard MS 5668 Roseville CA 95747 cbm [at] rose.hp.com Buck Landry wrote: > Hi folks. I'm confused about exactly what an initiator could send that would be considered a "format error" -- as defined in rfc 3720 6.6, something that would force a target to "immediately terminate all transport connections in the session". I assume this means w/out sending a Reject or Asynchronous Message first. > > For example, from 6.6 a "format error" can be: > > a) Illegal contents of any PDU header field except the Opcode > (legal values are specified in Section 10 iSCSI PDU Formats). > <<< > > But in 10. I see: > > Receipt of reserved code values in defined > fields MUST be reported as a protocol error. > <<< > > .. and "reporting" a protocol error sounds like issuing Reject, which isn't the same as immediately terminating all transport connections. So what exactly is "Illegal contents", then? > > In 6.6 we also have: > > b) Inconsistent field contents (consistent field contents are > specified in Section 10 iSCSI PDU Formats). > <<< > > .. but the only reference to 'consistent' I see is in 10.7.4, > > If the Target Transfer Tag is > provided, then the LUN field MUST hold a valid value and be > consistent with whatever was specified with the command > <<< > > ... and this specific case seems to be covered by Reject w/"Invalid PDU field" in 10.17.1... > > Note 2: (...) Some examples are invalid TTT/ITT, buffer offset, LUN > qualifying a TTT, (...) > <<< > > Also, 3.2.4.3 says > > Using inconsistent field values is considered a protocol error. > <<< > > ... does it actually mean to say "format error"? Because if it was *merely* a protocol error, we could just Reject the PDU w/"Protocol Error" and be on our merry way, right? > > I apologize for my confusion in advance; and thanks for your consideration, > --buck > > _______________________________________________ > Ips mailing list > Ips@ietf.org > https://www1.ietf.org/mailman/listinfo/ips _______________________________________________ Ips mailing list Ips@ietf.org https://www1.ietf.org/mailman/listinfo/ips From ips-bounces@ietf.org Thu Oct 28 22:17:22 2004 Received: from ietf-mx.ietf.org (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id WAA22581 for ; Thu, 28 Oct 2004 22:17:22 -0400 (EDT) Received: from megatron.ietf.org ([132.151.6.71]) by ietf-mx.ietf.org with esmtp (Exim 4.33) id 1CNMYd-0008CS-Ig for ips-web-archive@ietf.org; Thu, 28 Oct 2004 22:32:03 -0400 Received: from localhost.localdomain ([127.0.0.1] helo=megatron.ietf.org) by megatron.ietf.org with esmtp (Exim 4.32) id 1CNJtK-0005oL-4C; Thu, 28 Oct 2004 19:41:14 -0400 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by megatron.ietf.org with esmtp (Exim 4.32) id 1CNHW9-0002Tj-FO for ips@megatron.ietf.org; Thu, 28 Oct 2004 17:09:09 -0400 Received: from ietf-mx.ietf.org (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id RAA13680 for ; Thu, 28 Oct 2004 17:09:07 -0400 (EDT) Received: from mail75.messagelabs.com ([216.82.255.83]) by ietf-mx.ietf.org with smtp (Exim 4.33) id 1CNHkH-0004D8-SR for ips@ietf.org; Thu, 28 Oct 2004 17:23:46 -0400 X-VirusChecked: Checked X-Env-Sender: dmitzel@Brocade.COM X-Msg-Ref: server-10.tower-75.messagelabs.com!1098997454!14907818!1 X-StarScan-Version: 5.4.2; banners=-,-,- X-Originating-IP: [66.243.153.112] Received: (qmail 15389 invoked from network); 28 Oct 2004 21:04:14 -0000 Received: from f112.brocade.com (HELO discus.brocade.com) (66.243.153.112) by server-10.tower-75.messagelabs.com with SMTP; 28 Oct 2004 21:04:14 -0000 Received: from hq-ex-4.corp.brocade.com (hq-ex-4 [192.168.38.93]) by discus.brocade.com (Postfix) with ESMTP id 0B6CDF3531; Thu, 28 Oct 2004 14:02:34 -0700 (PDT) X-MimeOLE: Produced By Microsoft Exchange V6.0.6487.1 content-class: urn:content-classes:message MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable Subject: RE: [Ips] I-D ACTION:draft-ietf-ips-fcip-mib-06.txt Date: Thu, 28 Oct 2004 14:04:14 -0700 Message-ID: Thread-Topic: [Ips] I-D ACTION:draft-ietf-ips-fcip-mib-06.txt Thread-Index: AcS9FiHz19L8OOV1T5GdC28L2/PTwQAGh5mw From: "Danny Mitzel" To: , X-Spam-Score: 0.0 (/) X-Scan-Signature: 93238566e09e6e262849b4f805833007 Content-Transfer-Encoding: quoted-printable Cc: Danny Mitzel X-BeenThere: ips@ietf.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: IP Storage List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Sender: ips-bounces@ietf.org Errors-To: ips-bounces@ietf.org X-Spam-Score: 0.0 (/) X-Scan-Signature: 9466e0365fc95844abaf7c3f15a05c7d Content-Transfer-Encoding: quoted-printable I'm curious if anyone has managed to download and view this entire MIB document? I tried a download from the ietf.org I-D repository and the document appears to be missing nearly the entire MIB content. I get to somewhere on page 9: -- ****************************************************************** -- The FCIP group=20 -- -- This group defines the global scalar objects applicable to FCIP -- devices only and then the very next line is page 30: 9. Normative References: [FCIP] Rajagopal, M., et al "Fibre Channel Over TCP/IP (FCIP)", , August 2002. I'm also curious if anyone printing this or an earlier draft of the MIB has managed to get a hardcopy with correct paging? =20 formatting doesn't seem to follow the I-D/RFC recommendations on lines per page and embedded formfeed. cheers,danny _______________________________________________ Ips mailing list Ips@ietf.org https://www1.ietf.org/mailman/listinfo/ips From ips-bounces@ietf.org Fri Oct 29 00:27:58 2004 Received: from ietf-mx.ietf.org (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id AAA07241 for ; Fri, 29 Oct 2004 00:27:58 -0400 (EDT) Received: from megatron.ietf.org ([132.151.6.71]) by ietf-mx.ietf.org with esmtp (Exim 4.33) id 1CNOb3-0004JB-Ke for ips-web-archive@ietf.org; Fri, 29 Oct 2004 00:42:42 -0400 Received: from localhost.localdomain ([127.0.0.1] helo=megatron.ietf.org) by megatron.ietf.org with esmtp (Exim 4.32) id 1CNO7U-0005kN-2N; Fri, 29 Oct 2004 00:12:08 -0400 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by megatron.ietf.org with esmtp (Exim 4.32) id 1CMpi3-0005rz-Mm for ips@megatron.ietf.org; Wed, 27 Oct 2004 11:27:35 -0400 Received: from ietf-mx.ietf.org (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id LAA15133 for ; Wed, 27 Oct 2004 11:27:32 -0400 (EDT) Received: from mx2.netapp.com ([216.240.18.37]) by ietf-mx.ietf.org with esmtp (Exim 4.33) id 1CMpvw-00089h-9u for ips@ietf.org; Wed, 27 Oct 2004 11:41:57 -0400 Received: from smtp2.corp.netapp.com (10.57.159.114) by mx2.netapp.com with ESMTP; 27 Oct 2004 08:27:03 -0700 X-Ironport-AV: i="3.86,107,1096873200"; d="scan'217,208"; a="33255834:sNHT24660616" Received: from svlexc02.hq.netapp.com (svlexc02.corp.netapp.com [10.57.157.136]) by smtp2.corp.netapp.com (8.13.1/8.13.1/NTAP-1.6) with ESMTP id i9RFR2Sh002696 for ; Wed, 27 Oct 2004 08:27:02 -0700 (PDT) Received: from RTPEXC02.hq.netapp.com ([10.60.4.11]) by svlexc02.hq.netapp.com with Microsoft SMTPSVC(5.0.2195.6713); Wed, 27 Oct 2004 08:27:02 -0700 X-MimeOLE: Produced By Microsoft Exchange V6.0.6249.0 content-class: urn:content-classes:message MIME-Version: 1.0 Date: Wed, 27 Oct 2004 11:27:02 -0400 Message-ID: Thread-Topic: iSCSI: Recovering from DataDigest error on SCSI_CMD PDU with ImmediateData Thread-Index: AcS8OWkaNtHIGKt5ReWn2n2f4L0XLA== From: "Pittman, Joseph" To: X-OriginalArrivalTime: 27 Oct 2004 15:27:02.0804 (UTC) FILETIME=[68A55540:01C4BC39] X-Spam-Score: 0.8 (/) X-Scan-Signature: 410b68b37343617c6913e76d02180b14 X-Mailman-Approved-At: Fri, 29 Oct 2004 00:12:05 -0400 Subject: [Ips] iSCSI: Recovering from DataDigest error on SCSI_CMD PDU with ImmediateData X-BeenThere: ips@ietf.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: IP Storage List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Content-Type: multipart/mixed; boundary="===============0829215713==" Sender: ips-bounces@ietf.org Errors-To: ips-bounces@ietf.org X-Spam-Score: 0.8 (/) X-Scan-Signature: f1405b5eaa25d745f8c52e3273d3af78 This is a multi-part message in MIME format. --===============0829215713== content-class: urn:content-classes:message Content-Type: multipart/alternative; boundary="----_=_NextPart_001_01C4BC39.68457859" This is a multi-part message in MIME format. ------_=_NextPart_001_01C4BC39.68457859 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable On August 3-4 2004, I posed a question to this mailing list about the legitimacy of the following behavior on the part of an iSCSI target: =20 - Target receives SCSI_CMD PDU with Immediate Data attached - Target temporarily has no room to record immediate data, and so discards it - Target gets around to processing SCSI Write command - Target uses R2T to request (re-)transmission of the FirstBurst of data =20 The answer I received from the iSCSI spec authors (Thanks!) was that this behavior is acceptable (especially in an error situation), and a compliant initiator MUST provide the data (again) in response to the R2T in this situation. =20 Now, I have a question about a different error condition involving ImmediateData -- a DataDigest error. =20 Consider a hypothetical iSCSI target with the following pipelined architecture for handling inbound data: =20 +----------------------+ | iSCSI Session Layer | - Receives PDU headers +----------------------+ - Performs iSCSI session management - Drives SCSI data transfer =20 +----------------------+ | PDU Processing Layer | - Forwards PDU headers immediately upon receipt, +----------------------+ along with HeaderDigest valid indication - Holds data segments in layer-private memory - Verifies DataDigest values, and reports detected errors, only when data is forwarded up to the session layer =20 +----------------------+ | TCP/IP, NIC, etc. | +----------------------+ =20 Consider the handling by this architecture of a data digest error on a SCSI_CMD PDU with ImmediateData. =20 1) The session layer would begin processing the SCSI command (and possibly some others later in the command sequence). =20 2) The session layer would NOT know about the data digest error yet, since that specific error is not recognized until the data is later retrieved as part of the data transfer phase =20 3) So the session layer would NOT send a Reject PDU =20 4) When the session layer goes to retrieve the immediate data which is being held in the PDU processing Layer, then the session layer would learn of the data digest error =20 5) PROPOSED RECOVERY ACTION: The session layer would discard the immediate data, and use R2T to request retransmission by the initiator. =20 Practically speaking, I think the proposed recovery action would be guaranteed to work. Because, from the initiator's point of view, the target would be behaving exactly as described in the August 3/4 discussion mentioned above ("drop ImmData due to resource constraints"). =20 I am, however, aware of the following statement in the spec surrounding the handling of data digest errors: =20 6.7 Digest Errors =20 When a target receives any iSCSI PDU with a payload digest error, it MUST answer with a Reject PDU with a reason code of Data-Digest-Error and discard the PDU. =20 =20 Two questions: =20 1) Can anyone think of a situation where the PROPOSED RECOVERY ACTION would not sufficiently handle the error? =20 2) Would a target behaving in this way be in definite violation of the spec? Or are there perhaps some other statements in the spec which would render this behavior legitimate? =20 An authoritative ruling from some of the specification authors and experts would be GREATLY appreciated. =20 Thanks very much for any help. -- Joe Pittman jpittman@netapp.com =20 =20 ------_=_NextPart_001_01C4BC39.68457859 Content-Type: text/html; charset="us-ascii" Content-Transfer-Encoding: quoted-printable Message
On August 3-4 2004, I posed a = question to=20 this mailing list about the
legitimacy of the following behavior on = the part=20 of an iSCSI target:
 
- Target receives SCSI_CMD PDU = with=20 Immediate Data attached
- Target temporarily has no room to record = immediate=20 data, and
  so discards it
- Target gets around to processing = SCSI=20 Write command
- Target uses R2T to request (re-)transmission of the=20 FirstBurst
  of data
 
The answer I received from the = iSCSI spec=20 authors (Thanks!) was that
this behavior is acceptable (especially in = an=20 error situation), and a
compliant initiator MUST provide the data (again) in = response=20 to
the R2T in this situation.
 

Now, I have a question = about a=20 different error condition involving
ImmediateData -- a DataDigest=20 error.
 
Consider a=20 hypothetical iSCSI target with the following = pipelined
architecture for handling inbound data:
 
  = +----------------------+
  |=20 iSCSI Session Layer  |    - Receives PDU = headers
 =20 +----------------------+    - Performs iSCSI session=20 management
          = ;            =        =20 - Drives SCSI data transfer
 
  = +----------------------+
  |=20 PDU Processing Layer |    - Forwards PDU headers = immediately upon=20 receipt,
  = +----------------------+      along=20 with HeaderDigest valid=20 indication
          = ;            =        =20 - Holds data segments in layer-private=20 memory
          &nb= sp;           &nbs= p;      =20 - Verifies DataDigest values, and=20 reports
          &n= bsp;           &nb= sp;        =20 detected errors, only when data is=20 forwarded
          =             &= nbsp;        =20 up to the session layer
 
  = +----------------------+
  |=20 TCP/IP, NIC, etc.    |
 =20 +----------------------+
 

Consider the handling by = this=20 architecture of a data digest error
on a SCSI_CMD PDU with=20 ImmediateData.
 
1) The session layer would = begin processing=20 the SCSI command (and
   possibly some others later in the = command=20 sequence).
 
2) The session layer would NOT = know about=20 the data digest error yet,
   since that specific error is = not=20 recognized until the data is
   later retrieved as part of = the data=20 transfer phase
 
3) So the session layer would = NOT send a=20 Reject PDU
 
4) When the session layer goes = to retrieve=20 the immediate data which
   is being held in the PDU = processing=20 Layer, then the session layer
   would learn of the data = digest=20 error
 
5) PROPOSED RECOVERY ACTION: = The session=20 layer would discard the
   immediate data, and use R2T to = request=20 retransmission by the
   initiator.
 

Practically speaking, I = think the=20 proposed recovery action would be
guaranteed to work.  Because, = from the=20 initiator's point of view,
the target would be behaving exactly as = described=20 in the August 3/4
discussion mentioned above ("drop ImmData due to = resource=20 constraints").
 
I am, however, aware of the = following=20 statement in the spec surrounding
the handling of data digest=20 errors:
 
  6.7 Digest = Errors
 
  When a target receives = any iSCSI PDU=20 with a payload digest error, it
  MUST answer with a Reject PDU = with a=20 reason code of
  Data-Digest-Error and discard the = PDU.
 
 
Two questions:
 
1) Can anyone think of a = situation where=20 the PROPOSED RECOVERY ACTION
   would not sufficiently = handle the=20 error?
 
2) Would a target behaving in = this way be=20 in definite violation of the
   spec?  Or are there = perhaps=20 some other statements in the spec which
   would render = this=20 behavior legitimate?
 
   An authoritative = ruling from=20 some of the specification authors and
   experts would be = GREATLY=20 appreciated.
 

Thanks very much for any=20 help.
--
Joe Pittman
jpittman@netapp.com
 
=00 ------_=_NextPart_001_01C4BC39.68457859-- --===============0829215713== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline Content-Transfer-Encoding: 7bit _______________________________________________ Ips mailing list Ips@ietf.org https://www1.ietf.org/mailman/listinfo/ips --===============0829215713==-- From ips-bounces@ietf.org Fri Oct 29 00:59:18 2004 Received: from ietf-mx.ietf.org (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id AAA09018 for ; Fri, 29 Oct 2004 00:59:18 -0400 (EDT) Received: from megatron.ietf.org ([132.151.6.71]) by ietf-mx.ietf.org with esmtp (Exim 4.33) id 1CNP5N-0004rp-F3 for ips-web-archive@ietf.org; Fri, 29 Oct 2004 01:14:02 -0400 Received: from localhost.localdomain ([127.0.0.1] helo=megatron.ietf.org) by megatron.ietf.org with esmtp (Exim 4.32) id 1CNOkl-000881-GO; Fri, 29 Oct 2004 00:52:43 -0400 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by megatron.ietf.org with esmtp (Exim 4.32) id 1CNORS-0003dV-7X for ips@megatron.ietf.org; Fri, 29 Oct 2004 00:32:47 -0400 Received: from ietf-mx.ietf.org (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id AAA07667 for ; Fri, 29 Oct 2004 00:32:45 -0400 (EDT) From: elizabeth.rodriguez@dothill.com Received: from artecon.dothill.com ([155.254.128.254] helo=dothill.com) by ietf-mx.ietf.org with esmtp (Exim 4.33) id 1CNOfg-0004Q9-Vx for ips@ietf.org; Fri, 29 Oct 2004 00:47:29 -0400 Received: from exchange.artecon.com (exchange [206.6.182.75]) by dothill.com (8.12.9+Sun/8.12.5) with ESMTP id i9T4QsLl015307; Thu, 28 Oct 2004 21:26:54 -0700 (PDT) Received: by EXCHANGE with Internet Mail Service (5.5.2653.19) id <4M140TGL>; Thu, 28 Oct 2004 21:31:10 -0700 Message-ID: To: dmitzel@Brocade.COM, ips@ietf.org, ravin@lightsand.com Subject: RE: [Ips] I-D ACTION:draft-ietf-ips-fcip-mib-06.txt Date: Thu, 28 Oct 2004 21:35:58 -0700 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2653.19) Content-Type: text/plain X-Spam-Score: 0.3 (/) X-Scan-Signature: 0ddefe323dd869ab027dbfff7eff0465 Cc: mrm@kazeon.com, bwijnen@lucent.com, anilr@accton.com, ravin@lightsand.com, mankin@psg.com X-BeenThere: ips@ietf.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: IP Storage List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Sender: ips-bounces@ietf.org Errors-To: ips-bounces@ietf.org X-Spam-Score: 0.3 (/) X-Scan-Signature: 3e15cc4fdc61d7bce84032741d11c8e5 Hi all, I have checked the posted draft, and yes, as reported by a couple of people, the draft has somehow lost about 20 pages. I am checking with the ID administrator, to see if we can get a complete draft posted. I will update the list as soon as I hear something. Thanks, Elizabeth -----Original Message----- From: Danny Mitzel [mailto:dmitzel@Brocade.COM] Sent: Thursday, October 28, 2004 2:04 PM To: ips@ietf.org; ravin@lightsand.com Cc: Danny Mitzel Subject: RE: [Ips] I-D ACTION:draft-ietf-ips-fcip-mib-06.txt I'm curious if anyone has managed to download and view this entire MIB document? I tried a download from the ietf.org I-D repository and the document appears to be missing nearly the entire MIB content. I get to somewhere on page 9: -- ****************************************************************** -- The FCIP group -- -- This group defines the global scalar objects applicable to FCIP -- devices only and then the very next line is page 30: 9. Normative References: [FCIP] Rajagopal, M., et al "Fibre Channel Over TCP/IP (FCIP)", , August 2002. I'm also curious if anyone printing this or an earlier draft of the MIB has managed to get a hardcopy with correct paging? formatting doesn't seem to follow the I-D/RFC recommendations on lines per page and embedded formfeed. cheers,danny _______________________________________________ Ips mailing list Ips@ietf.org https://www1.ietf.org/mailman/listinfo/ips _______________________________________________ Ips mailing list Ips@ietf.org https://www1.ietf.org/mailman/listinfo/ips From ips-bounces@ietf.org Fri Oct 29 12:38:27 2004 Received: from ietf-mx.ietf.org (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id MAA13805 for ; Fri, 29 Oct 2004 12:38:27 -0400 (EDT) Received: from megatron.ietf.org ([132.151.6.71]) by ietf-mx.ietf.org with esmtp (Exim 4.33) id 1CNa05-0002vF-PN for ips-web-archive@ietf.org; Fri, 29 Oct 2004 12:53:18 -0400 Received: from localhost.localdomain ([127.0.0.1] helo=megatron.ietf.org) by megatron.ietf.org with esmtp (Exim 4.32) id 1CNZgw-0002g3-BU; Fri, 29 Oct 2004 12:33:30 -0400 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by megatron.ietf.org with esmtp (Exim 4.32) id 1CNZG7-0001Jt-Gm for ips@megatron.ietf.org; Fri, 29 Oct 2004 12:05:50 -0400 Received: from ietf-mx.ietf.org (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id MAA11599 for ; Fri, 29 Oct 2004 12:05:44 -0400 (EDT) From: elizabeth.rodriguez@dothill.com Received: from artecon.dothill.com ([155.254.128.254] helo=dothill.com) by ietf-mx.ietf.org with esmtp (Exim 4.33) id 1CNZUQ-0002FV-Bj for ips@ietf.org; Fri, 29 Oct 2004 12:20:34 -0400 Received: from exchange.artecon.com (exchange [206.6.182.75]) by dothill.com (8.12.9+Sun/8.12.5) with ESMTP id i9TFxSLl026522; Fri, 29 Oct 2004 08:59:28 -0700 (PDT) Received: by EXCHANGE with Internet Mail Service (5.5.2653.19) id <4M140WC1>; Fri, 29 Oct 2004 09:03:44 -0700 Message-ID: To: elizabeth.rodriguez@dothill.com, dmitzel@Brocade.COM, ips@ietf.org, ravin@lightsand.com Subject: RE: [Ips] I-D ACTION:draft-ietf-ips-fcip-mib-06.txt Date: Fri, 29 Oct 2004 09:08:29 -0700 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2653.19) Content-Type: text/plain X-Spam-Score: 0.3 (/) X-Scan-Signature: c0bedb65cce30976f0bf60a0a39edea4 Cc: david_peterson@cnt.com, anilr@accton.com, ravin@lightsand.com, bwijnen@lucent.com, mrm@kazeon.com, mankin@psg.com, black_david@emc.com X-BeenThere: ips@ietf.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: IP Storage List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Sender: ips-bounces@ietf.org Errors-To: ips-bounces@ietf.org X-Spam-Score: 0.3 (/) X-Scan-Signature: d0bdc596f8dd1c226c458f0b4df27a88 Hi all, The draft has been fixed. Please try to download it again. Thanks, Elizabeth -----Original Message----- From: Elizabeth Rodriguez Sent: Thursday, October 28, 2004 9:33 PM To: 'Danny Mitzel'; ips@ietf.org; ravin@lightsand.com Cc: 'mankin@psg.com'; 'OPS ADs'; 'Mike MacFaden (E-mail)'; 'anilr@accton.com'; 'ravin@lightsand.com' Subject: RE: [Ips] I-D ACTION:draft-ietf-ips-fcip-mib-06.txt Hi all, I have checked the posted draft, and yes, as reported by a couple of people, the draft has somehow lost about 20 pages. I am checking with the ID administrator, to see if we can get a complete draft posted. I will update the list as soon as I hear something. Thanks, Elizabeth -----Original Message----- From: Danny Mitzel [mailto:dmitzel@Brocade.COM] Sent: Thursday, October 28, 2004 2:04 PM To: ips@ietf.org; ravin@lightsand.com Cc: Danny Mitzel Subject: RE: [Ips] I-D ACTION:draft-ietf-ips-fcip-mib-06.txt I'm curious if anyone has managed to download and view this entire MIB document? I tried a download from the ietf.org I-D repository and the document appears to be missing nearly the entire MIB content. I get to somewhere on page 9: -- ****************************************************************** -- The FCIP group -- -- This group defines the global scalar objects applicable to FCIP -- devices only and then the very next line is page 30: 9. Normative References: [FCIP] Rajagopal, M., et al "Fibre Channel Over TCP/IP (FCIP)", , August 2002. I'm also curious if anyone printing this or an earlier draft of the MIB has managed to get a hardcopy with correct paging? formatting doesn't seem to follow the I-D/RFC recommendations on lines per page and embedded formfeed. cheers,danny _______________________________________________ Ips mailing list Ips@ietf.org https://www1.ietf.org/mailman/listinfo/ips _______________________________________________ Ips mailing list Ips@ietf.org https://www1.ietf.org/mailman/listinfo/ips From ips-bounces@ietf.org Sat Oct 30 04:14:13 2004 Received: from ietf-mx.ietf.org (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id EAA07018 for ; Sat, 30 Oct 2004 04:14:13 -0400 (EDT) Received: from megatron.ietf.org ([132.151.6.71]) by ietf-mx.ietf.org with esmtp (Exim 4.33) id 1CNobm-0005Pi-U6 for ips-web-archive@ietf.org; Sat, 30 Oct 2004 04:29:11 -0400 Received: from localhost.localdomain ([127.0.0.1] helo=megatron.ietf.org) by megatron.ietf.org with esmtp (Exim 4.32) id 1CNo8o-0007K8-GM; Sat, 30 Oct 2004 03:59:14 -0400 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by megatron.ietf.org with esmtp (Exim 4.32) id 1CNnxN-0002Wz-Ot; Sat, 30 Oct 2004 03:47:25 -0400 Received: from ietf-mx.ietf.org (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id DAA05948; Sat, 30 Oct 2004 03:47:22 -0400 (EDT) Received: from mtagate1.de.ibm.com ([195.212.29.150]) by ietf-mx.ietf.org with esmtp (Exim 4.33) id 1CNoBn-0004sc-SD; Sat, 30 Oct 2004 04:02:20 -0400 Received: from d12nrmr1507.megacenter.de.ibm.com (d12nrmr1507.megacenter.de.ibm.com [9.149.167.1]) by mtagate1.de.ibm.com (8.12.10/8.12.10) with ESMTP id i9U7kqEA154150; Sat, 30 Oct 2004 07:46:52 GMT Received: from d12ml102.megacenter.de.ibm.com (d12av02.megacenter.de.ibm.com [9.149.165.228]) by d12nrmr1507.megacenter.de.ibm.com (8.12.10/NCO/VER6.6) with ESMTP id i9U7kpZn220094; Sat, 30 Oct 2004 09:46:51 +0200 In-Reply-To: To: "Pittman, Joseph" Subject: Re: [Ips] iSCSI: Recovering from DataDigest error on SCSI_CMD PDU with ImmediateData MIME-Version: 1.0 X-Mailer: Lotus Notes Build V70_M2_07222004 Beta 2NP July 22, 2004 From: Julian Satran Message-ID: Date: Sat, 30 Oct 2004 09:46:49 +0200 X-MIMETrack: Serialize by Router on D12ML102/12/M/IBM(Release 6.5.1| March 5, 2004) at 30/10/2004 09:46:51, Serialize complete at 30/10/2004 09:46:51 X-Spam-Score: 0.6 (/) X-Scan-Signature: 9e5c23589e6cce06555030c0194c9e2b Cc: ips@ietf.org, ips-bounces@ietf.org X-BeenThere: ips@ietf.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: IP Storage List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Content-Type: multipart/mixed; boundary="===============1721085240==" Sender: ips-bounces@ietf.org Errors-To: ips-bounces@ietf.org X-Spam-Score: 0.6 (/) X-Scan-Signature: 90e8b0e368115979782f8b3d811b226b This is a multipart message in MIME format. --===============1721085240== Content-Type: multipart/alternative; boundary="=_alternative 002A07A5C2256F3D_=" This is a multipart message in MIME format. --=_alternative 002A07A5C2256F3D_= Content-Type: text/plain; charset="US-ASCII" We discussed this exact topic in 2002 and decided that the spec should mandate reject WHEN the DIGEST ERROR IT IS SEEN (as it is a good practice to notify that there must be something bad on the path (digest errors do not happen by chance). However in you architecture the recovery action is legitimate - as the initiator has no way of distinguishing if the target experienced a digest error or a buffer failure and the target has not seen yet the error . It can be argued that from a RAS point of view it is not the best practice - you will want to keep track of the digest errors as those are indicative of a problem on the path. You can handle those also within the management interface. I hope that helps, Julo "Pittman, Joseph" Sent by: ips-bounces@ietf.org 27/10/04 17:27 To cc Subject [Ips] iSCSI: Recovering from DataDigest error on SCSI_CMD PDU with ImmediateData On August 3-4 2004, I posed a question to this mailing list about the legitimacy of the following behavior on the part of an iSCSI target: - Target receives SCSI_CMD PDU with Immediate Data attached - Target temporarily has no room to record immediate data, and so discards it - Target gets around to processing SCSI Write command - Target uses R2T to request (re-)transmission of the FirstBurst of data The answer I received from the iSCSI spec authors (Thanks!) was that this behavior is acceptable (especially in an error situation), and a compliant initiator MUST provide the data (again) in response to the R2T in this situation. Now, I have a question about a different error condition involving ImmediateData -- a DataDigest error. Consider a hypothetical iSCSI target with the following pipelined architecture for handling inbound data: +----------------------+ | iSCSI Session Layer | - Receives PDU headers +----------------------+ - Performs iSCSI session management - Drives SCSI data transfer +----------------------+ | PDU Processing Layer | - Forwards PDU headers immediately upon receipt, +----------------------+ along with HeaderDigest valid indication - Holds data segments in layer-private memory - Verifies DataDigest values, and reports detected errors, only when data is forwarded up to the session layer +----------------------+ | TCP/IP, NIC, etc. | +----------------------+ Consider the handling by this architecture of a data digest error on a SCSI_CMD PDU with ImmediateData. 1) The session layer would begin processing the SCSI command (and possibly some others later in the command sequence). 2) The session layer would NOT know about the data digest error yet, since that specific error is not recognized until the data is later retrieved as part of the data transfer phase 3) So the session layer would NOT send a Reject PDU 4) When the session layer goes to retrieve the immediate data which is being held in the PDU processing Layer, then the session layer would learn of the data digest error 5) PROPOSED RECOVERY ACTION: The session layer would discard the immediate data, and use R2T to request retransmission by the initiator. Practically speaking, I think the proposed recovery action would be guaranteed to work. Because, from the initiator's point of view, the target would be behaving exactly as described in the August 3/4 discussion mentioned above ("drop ImmData due to resource constraints"). I am, however, aware of the following statement in the spec surrounding the handling of data digest errors: 6.7 Digest Errors When a target receives any iSCSI PDU with a payload digest error, it MUST answer with a Reject PDU with a reason code of Data-Digest-Error and discard the PDU. Two questions: 1) Can anyone think of a situation where the PROPOSED RECOVERY ACTION would not sufficiently handle the error? 2) Would a target behaving in this way be in definite violation of the spec? Or are there perhaps some other statements in the spec which would render this behavior legitimate? An authoritative ruling from some of the specification authors and experts would be GREATLY appreciated. Thanks very much for any help. -- Joe Pittman jpittman@netapp.com _______________________________________________ Ips mailing list Ips@ietf.org https://www1.ietf.org/mailman/listinfo/ips --=_alternative 002A07A5C2256F3D_= Content-Type: text/html; charset="US-ASCII"
We discussed this exact topic in 2002 and decided that the spec should mandate reject WHEN the DIGEST ERROR IT IS SEEN (as it is a good practice to notify that there must be something bad on the path (digest errors do not happen by chance). However in you architecture the recovery action is legitimate - as the initiator has no way of distinguishing if the target experienced a digest error or a buffer failure and  the target has not seen yet the error . It can be argued that from a RAS point of view it is not the best practice - you will want to keep track of the digest errors as those are indicative of a problem on the path. You can handle those also within the management interface.

I hope that helps,

Julo


"Pittman, Joseph" <Joseph.Pittman@netapp.com>
Sent by: ips-bounces@ietf.org

27/10/04 17:27

To
<ips@ietf.org>
cc
Subject
[Ips] iSCSI: Recovering from DataDigest error on SCSI_CMD PDU with        ImmediateData





On August 3-4 2004, I posed a question to this mailing list about the
legitimacy of the following behavior on the part of an iSCSI target:
 
- Target receives SCSI_CMD PDU with Immediate Data attached
- Target temporarily has no room to record immediate data, and
 so discards it
- Target gets around to processing SCSI Write command
- Target uses R2T to request (re-)transmission of the FirstBurst
 of data
 
The answer I received from the iSCSI spec authors (Thanks!) was that
this behavior is acceptable (especially in an error situation), and a
compliant initiator MUST provide the data (again) in response to
the R2T in this situation.
 

Now, I have a question about a different error condition involving
ImmediateData -- a DataDigest error.
 
Consider a hypothetical iSCSI target with the following pipelined
architecture for handling inbound data:
 
  +----------------------+
 | iSCSI Session Layer  |    - Receives PDU headers
 +----------------------+    - Performs iSCSI session management
                             - Drives SCSI data transfer
 
  +----------------------+
 | PDU Processing Layer |    - Forwards PDU headers immediately upon receipt,
 +----------------------+      along with HeaderDigest valid indication
                             - Holds data segments in layer-private memory
                             - Verifies DataDigest values, and reports
                               detected errors, only when data is forwarded
                               up to the session layer
 
  +----------------------+
 | TCP/IP, NIC, etc.    |
 +----------------------+
 

Consider the handling by this architecture of a data digest error
on a SCSI_CMD PDU with ImmediateData.
 
1) The session layer would begin processing the SCSI command (and
  possibly some others later in the command sequence).
 
2) The session layer would NOT know about the data digest error yet,
  since that specific error is not recognized until the data is
  later retrieved as part of the data transfer phase
 
3) So the session layer would NOT send a Reject PDU
 
4) When the session layer goes to retrieve the immediate data which
  is being held in the PDU processing Layer, then the session layer
  would learn of the data digest error
 
5) PROPOSED RECOVERY ACTION: The session layer would discard the
  immediate data, and use R2T to request retransmission by the
  initiator.
 

Practically speaking, I think the proposed recovery action would be
guaranteed to work.  Because, from the initiator's point of view,
the target would be behaving exactly as described in the August 3/4
discussion mentioned above ("drop ImmData due to resource constraints").
 
I am, however, aware of the following statement in the spec surrounding
the handling of data digest errors:
 
  6.7 Digest Errors
 
  When a target receives any iSCSI PDU with a payload digest error, it
 MUST answer with a Reject PDU with a reason code of
 Data-Digest-Error and discard the PDU.
 
 
Two questions:
 
1) Can anyone think of a situation where the PROPOSED RECOVERY ACTION
  would not sufficiently handle the error?
 
2) Would a target behaving in this way be in definite violation of the
  spec?  Or are there perhaps some other statements in the spec which
  would render this behavior legitimate?
 
   An authoritative ruling from some of the specification authors and
  experts would be GREATLY appreciated.
 

Thanks very much for any help.
--
Joe Pittman
jpittman@netapp.com
 _______________________________________________
Ips mailing list
Ips@ietf.org
https://www1.ietf.org/mailman/listinfo/ips
--=_alternative 002A07A5C2256F3D_=-- --===============1721085240== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline Content-Transfer-Encoding: 7bit _______________________________________________ Ips mailing list Ips@ietf.org https://www1.ietf.org/mailman/listinfo/ips --===============1721085240==--