From david.black@emc.com Thu May 3 15:29:17 2012 Return-Path: X-Original-To: storm@ietfa.amsl.com Delivered-To: storm@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 426E221F8705 for ; Thu, 3 May 2012 15:29:17 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -110.462 X-Spam-Level: X-Spam-Status: No, score=-110.462 tagged_above=-999 required=5 tests=[AWL=0.137, BAYES_00=-2.599, RCVD_IN_DNSWL_HI=-8, USER_IN_WHITELIST=-100] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id HqskZmhgOPQY for ; Thu, 3 May 2012 15:29:16 -0700 (PDT) Received: from mexforward.lss.emc.com (mexforward.lss.emc.com [128.222.32.20]) by ietfa.amsl.com (Postfix) with ESMTP id 8615921F86F5 for ; Thu, 3 May 2012 15:29:16 -0700 (PDT) Received: from hop04-l1d11-si03.isus.emc.com (HOP04-L1D11-SI03.isus.emc.com [10.254.111.23]) by mexforward.lss.emc.com (Switch-3.4.3/Switch-3.4.3) with ESMTP id q43MTE9r002342 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO) for ; Thu, 3 May 2012 18:29:14 -0400 Received: from mailhub.lss.emc.com (mailhub.lss.emc.com [10.254.222.130]) by hop04-l1d11-si03.isus.emc.com (RSA Interceptor) for ; Thu, 3 May 2012 18:29:02 -0400 Received: from mxhub12.corp.emc.com (mxhub12.corp.emc.com [10.254.92.107]) by mailhub.lss.emc.com (Switch-3.4.3/Switch-3.4.3) with ESMTP id q43MT1H4014244 for ; Thu, 3 May 2012 18:29:01 -0400 Received: from mx15a.corp.emc.com ([169.254.1.107]) by mxhub12.corp.emc.com ([10.254.92.107]) with mapi; Thu, 3 May 2012 18:29:01 -0400 From: To: Date: Thu, 3 May 2012 18:29:00 -0400 Thread-Topic: iSER approach - update Thread-Index: Ac0pfCLPSE0AecLXRPSgPUVAZl/VUw== Message-ID: <8D3D17ACE214DC429325B2B98F3AE71203D1120D@MX15A.corp.emc.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: acceptlanguage: en-US Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-EMM-MHVC: 1 Subject: [storm] iSER approach - update X-BeenThere: storm@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Storage Maintenance WG List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 03 May 2012 22:29:17 -0000 Attempting to summarize where we are ... the changes from my earlier attemp= t are: - Dropped the MUST for error reporting by the receiver. - Added a more general "SHOULD NOT use" to iii).=20 - Added caching discussion and need for yet more security considerations text to v). - Added a warning about relying on SendSE and SendInvSE at receiver - new i= tem vi). Are we getting closer? Here's the new version: i) Some iSER implementations have not followed RFC 5046's strict requiremen= ts for use of SendSE, SendInv and SendInvSE; they use Send instead. ii) For interoperability, iSER implementations SHOULD accept and correctly process SendSE, SendInv and SendInvSE messages. iii) SendSE, SendInv and SendInvSE are to be regarded as optimizations or enhancements to the basic Send message, and their support may vary by RCaP protocol and specific implementation. In general, these messages SHOULD NOT be used, unless the RCaP requires support for them in all implementations. If these messages are used, the implementation SHOULD be capable of reverting to use of Send in order to work with a receiver that does not support these message (and we need to note that attempted use of these messages with a peer that doesn't support them may result in a fatal error that closes the RCaP connection). A specific instance that needs to be noted is that these messages SHOULD NOT be used with the InfiniBand RCaP because InfiniBand does not require support for them in all cases. iv) New iSER implementations SHOULD use Send (and not these three additiona= l messages) unless there are compelling reasons for doing otherwise (latter is implicit in use of "SHOULD", but worth saying explicitly, IMHO). v) Some new text will be needed (especially Security Considerations) to make it clear that STag invalidation is the initiator's responsibility for security reasons, and the initiator cannot rely on the target using an Invalidate version of Send - the initiator MUST check and do its own invalidation as required. The initiator MAY choose to cache=20 mappings (i.e., reuse STags) across I/Os for efficiency, but that has security consequences (exposes initiator memory to remote access for longer) that have to be discussed in the new Security Considerations text. vi) Similarly, iSER implementations SHOULD NOT rely on events triggered by SendSE and SendInvSE, as these messages may not be used. In contrast to invalidation, there are no security consequences in this aspect. Thanks, --David ---------------------------------------------------- David L. Black, Distinguished Engineer EMC Corporation, 176 South St., Hopkinton, MA=A0 01748 +1 (508) 293-7953=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 FAX: +1 (508) 293-778= 6 david.black@emc.com=A0=A0=A0=A0=A0=A0=A0 Mobile: +1 (978) 394-7754 ---------------------------------------------------- From mkosjc@gmail.com Fri May 4 10:05:24 2012 Return-Path: X-Original-To: storm@ietfa.amsl.com Delivered-To: storm@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id DA0E621E802A for ; Fri, 4 May 2012 10:05:24 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -3.598 X-Spam-Level: X-Spam-Status: No, score=-3.598 tagged_above=-999 required=5 tests=[AWL=-0.000, BAYES_00=-2.599, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_LOW=-1] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 1YoAG6mbOEDr for ; Fri, 4 May 2012 10:05:24 -0700 (PDT) Received: from mail-qa0-f49.google.com (mail-qa0-f49.google.com [209.85.216.49]) by ietfa.amsl.com (Postfix) with ESMTP id DA69221E801A for ; Fri, 4 May 2012 10:05:23 -0700 (PDT) Received: by qabj40 with SMTP id j40so1539554qab.15 for ; Fri, 04 May 2012 10:05:23 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=14Sv7SqKpuqsttSgxblZPHpfV2d9GabwvhHS1UqXfSA=; b=BceZWfDF3ItC80yYY9RVg6Y5Ji/6QkvOqqQGvCNp10LPPVZj6Cfso6InaBSsazaN35 Ywz/E76323c0ABH/Y2Patd2Zzj/5Va0rAK1dYQuWDcQaWy4UN6RiEu12NULleaw3cJN/ XtANELE6SbuajqXLRxr8F0FnK+9wmJJgfWmI78ncMDJPQLjvWdOY2W4Gg4DlEa58nJrg YsTxzc/DqaU5L+DIUoCt4/fQ4VhVNb+6Mlr1rvCY8Afu9OS6AmMgoIZ3hRomrIz+g5C5 E8/mtVh3v79RSSecXdRjzjz6PD3F9DbzsXXeaWyhni3fYI1B2wgY++8uom9ZxHEyj3s0 +yJQ== MIME-Version: 1.0 Received: by 10.224.217.138 with SMTP id hm10mr11142641qab.76.1336151123373; Fri, 04 May 2012 10:05:23 -0700 (PDT) Received: by 10.229.133.194 with HTTP; Fri, 4 May 2012 10:05:23 -0700 (PDT) In-Reply-To: <8D3D17ACE214DC429325B2B98F3AE71203D1120D@MX15A.corp.emc.com> References: <8D3D17ACE214DC429325B2B98F3AE71203D1120D@MX15A.corp.emc.com> Date: Fri, 4 May 2012 10:05:23 -0700 Message-ID: From: Michael Ko To: david.black@emc.com Content-Type: multipart/alternative; boundary=20cf3005dc0e132e2704bf38eda4 Cc: storm@ietf.org Subject: Re: [storm] iSER approach - update X-BeenThere: storm@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Storage Maintenance WG List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 04 May 2012 17:05:25 -0000 --20cf3005dc0e132e2704bf38eda4 Content-Type: text/plain; charset=ISO-8859-1 David, I agree with your summary. Mike On Thu, May 3, 2012 at 3:29 PM, wrote: > Attempting to summarize where we are ... the changes from my earlier > attempt > are: > - Dropped the MUST for error reporting by the receiver. > - Added a more general "SHOULD NOT use" to iii). > - Added caching discussion and need for yet more security considerations > text to v). > - Added a warning about relying on SendSE and SendInvSE at receiver - new > item > vi). > > Are we getting closer? Here's the new version: > > i) Some iSER implementations have not followed RFC 5046's strict > requirements > for use of SendSE, SendInv and SendInvSE; they use Send instead. > ii) For interoperability, iSER implementations SHOULD accept and correctly > process SendSE, SendInv and SendInvSE messages. > iii) SendSE, SendInv and SendInvSE are to be regarded as optimizations or > enhancements to the basic Send message, and their support may vary > by > RCaP protocol and specific implementation. In general, these > messages > SHOULD NOT be used, unless the RCaP requires support for them in all > implementations. If these messages are used, the implementation > SHOULD > be capable of reverting to use of Send in order to work with a > receiver > that does not support these message (and we need to note that > attempted > use of these messages with a peer that doesn't support them may > result > in a fatal error that closes the RCaP connection). A specific > instance > that needs to be noted is that these messages SHOULD NOT be used > with > the InfiniBand RCaP because InfiniBand does not require support for > them > in all cases. > iv) New iSER implementations SHOULD use Send (and not these three > additional > messages) unless there are compelling reasons for doing otherwise > (latter is implicit in use of "SHOULD", but worth saying explicitly, > IMHO). > v) Some new text will be needed (especially Security Considerations) to > make it clear that STag invalidation is the initiator's > responsibility > for security reasons, and the initiator cannot rely on the target > using an Invalidate version of Send - the initiator MUST check and > do its own invalidation as required. The initiator MAY choose to > cache > mappings (i.e., reuse STags) across I/Os for efficiency, but that > has > security consequences (exposes initiator memory to remote access for > longer) that have to be discussed in the new Security > Considerations text. > vi) Similarly, iSER implementations SHOULD NOT rely on events triggered by > SendSE and SendInvSE, as these messages may not be used. In > contrast > to invalidation, there are no security consequences in this aspect. > > Thanks, > --David > ---------------------------------------------------- > David L. Black, Distinguished Engineer > EMC Corporation, 176 South St., Hopkinton, MA 01748 > +1 (508) 293-7953 FAX: +1 (508) 293-7786 > david.black@emc.com Mobile: +1 (978) 394-7754 > ---------------------------------------------------- > > > _______________________________________________ > storm mailing list > storm@ietf.org > https://www.ietf.org/mailman/listinfo/storm > --20cf3005dc0e132e2704bf38eda4 Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable David,

I agree with your summary.

Mike

On Thu, May 3, 2012 at 3:29 PM, <david.black@emc.com&g= t; wrote:
Attempting to summarize where we are ... the= changes from my earlier attempt
are:
- Dropped the MUST for error reporting by the receiver.
- Added a more general "SHOULD NOT use" to iii).
- Added caching discussion and need for yet more security considerations =A0 =A0 =A0 =A0text to v).
- Added a warning about relying on SendSE and SendInvSE at receiver - new i= tem
=A0 =A0 =A0 =A0vi).

Are we getting closer? =A0Here's the new version:

i) Some iSER implementations have not followed RFC 5046's strict requir= ements
=A0 =A0 =A0 =A0for use of SendSE, SendInv and SendInvSE; they use Send ins= tead.
ii) For interoperability, iSER implementations SHOULD accept and correctly<= br> =A0 =A0 =A0 =A0process SendSE, SendInv and SendInvSE messages.
iii) SendSE, SendInv and SendInvSE are to be regarded as optimizations or =A0 =A0 =A0 =A0enhancements to the basic Send message, and their support m= ay vary by
=A0 =A0 =A0 =A0RCaP protocol and specific implementation. =A0In general, t= hese messages
=A0 =A0 =A0 =A0SHOULD NOT be used, unless the RCaP requires support for th= em in all
=A0 =A0 =A0 =A0implementations. =A0If these messages are used, the impleme= ntation SHOULD
=A0 =A0 =A0 =A0be capable of reverting to use of Send in order to work wit= h a receiver
=A0 =A0 =A0 =A0that does not support these message (and we need to note th= at attempted
=A0 =A0 =A0 =A0use of these messages with a peer that doesn't support = them may result
=A0 =A0 =A0 =A0in a fatal error that closes the RCaP connection). =A0A spe= cific instance
=A0 =A0 =A0 =A0that needs to be noted is that these messages SHOULD NOT be= used with
=A0 =A0 =A0 =A0the InfiniBand RCaP because InfiniBand does not require sup= port for them
=A0 =A0 =A0 =A0in all cases.
iv) New iSER implementations SHOULD use Send (and not these three additiona= l
=A0 =A0 =A0 =A0messages) unless there are compelling reasons for doing oth= erwise
=A0 =A0 =A0 =A0(latter is implicit in use of "SHOULD", but worth= saying explicitly,
=A0 =A0 =A0 =A0IMHO).
v) Some new text will be needed (especially Security Considerations) to
=A0 =A0 =A0 =A0make it clear that STag invalidation is the initiator's= responsibility
=A0 =A0 =A0 =A0for security reasons, and the initiator cannot rely on the = target
=A0 =A0 =A0 =A0using an Invalidate version of Send - the initiator MUST ch= eck and
=A0 =A0 =A0 =A0do its own invalidation as required. =A0The initiator MAY c= hoose to cache
=A0 =A0 =A0 =A0mappings (i.e., reuse STags) across I/Os for efficiency, bu= t that has
=A0 =A0 =A0 =A0security consequences (exposes initiator memory to remote a= ccess for
=A0 =A0 =A0 =A0longer) that have to be discussed in the new Security Consi= derations text.
vi) Similarly, iSER implementations SHOULD NOT rely on events triggered by<= br> =A0 =A0 =A0 =A0SendSE and SendInvSE, as these messages may not be used. = =A0In contrast
=A0 =A0 =A0 =A0to invalidation, there are no security consequences in this= aspect.

Thanks,
--David
----------------------------------------------------
David L. Black, Distinguished Engineer
EMC Corporation, 176 South St., Hopkinton, MA=A0 01748
+1 (508)= 293-7953=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 FAX: +1 (508) 293-7786
david.black@emc.com=A0=A0=A0=A0= =A0=A0=A0 Mobile: +1 (978) 394-7754
----------------------------------------------------


_______________________________________________
storm mailing list
storm@ietf.org
h= ttps://www.ietf.org/mailman/listinfo/storm

--20cf3005dc0e132e2704bf38eda4-- From david.black@emc.com Tue May 15 10:54:58 2012 Return-Path: X-Original-To: storm@ietfa.amsl.com Delivered-To: storm@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id BCAB221F882B for ; Tue, 15 May 2012 10:54:58 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -110.474 X-Spam-Level: X-Spam-Status: No, score=-110.474 tagged_above=-999 required=5 tests=[AWL=0.125, BAYES_00=-2.599, RCVD_IN_DNSWL_HI=-8, USER_IN_WHITELIST=-100] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 0nDggGCIGZDG for ; Tue, 15 May 2012 10:54:58 -0700 (PDT) Received: from mexforward.lss.emc.com (mexforward.lss.emc.com [128.222.32.20]) by ietfa.amsl.com (Postfix) with ESMTP id E1C9B21F8827 for ; Tue, 15 May 2012 10:54:57 -0700 (PDT) Received: from hop04-l1d11-si02.isus.emc.com (HOP04-L1D11-SI02.isus.emc.com [10.254.111.55]) by mexforward.lss.emc.com (Switch-3.4.3/Switch-3.4.3) with ESMTP id q4FHsuVF023463 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO) for ; Tue, 15 May 2012 13:54:57 -0400 Received: from mailhub.lss.emc.com (mailhubhoprd03.lss.emc.com [10.254.221.145]) by hop04-l1d11-si02.isus.emc.com (RSA Interceptor) for ; Tue, 15 May 2012 13:54:49 -0400 Received: from mxhub25.corp.emc.com (mxhub25.corp.emc.com [10.254.110.181]) by mailhub.lss.emc.com (Switch-3.4.3/Switch-3.4.3) with ESMTP id q4FHsl2l029834 for ; Tue, 15 May 2012 13:54:47 -0400 Received: from mx15a.corp.emc.com ([169.254.1.236]) by mxhub25.corp.emc.com ([10.254.110.181]) with mapi; Tue, 15 May 2012 13:54:47 -0400 From: To: , Importance: high X-Priority: 1 Date: Tue, 15 May 2012 13:54:45 -0400 Thread-Topic: iSER approach - update: Next Steps Thread-Index: Ac0pfCLPSE0AecLXRPSgPUVAZl/VUwJRrRVg Message-ID: <8D3D17ACE214DC429325B2B98F3AE712057151F0@MX15A.corp.emc.com> References: <8D3D17ACE214DC429325B2B98F3AE71203D1120D@MX15A.corp.emc.com> In-Reply-To: <8D3D17ACE214DC429325B2B98F3AE71203D1120D@MX15A.corp.emc.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: acceptlanguage: en-US Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-EMM-MHVC: 1 Subject: [storm] iSER approach - update: Next Steps X-BeenThere: storm@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Storage Maintenance WG List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 15 May 2012 17:54:58 -0000 Having seen no dissension, I believe (with WG chair hat on) that we have WG rough consensus for this updated approach to resolving the iSER issues around SendSE, SendInv and SendInvSE. Mike (Ko) - please update the draft accordingly, *BUT*, I want to see the Security Considerations and related caching text for the following item proposed to the list and worked out on the list before going into the next version of t he draft (I'd be happy to review a first draft of this text via private email if that would help): > v) Some new text will be needed (especially Security Considerations) to > make it clear that STag invalidation is the initiator's responsibility > for security reasons, and the initiator cannot rely on the target > using an Invalidate version of Send - the initiator MUST check and > do its own invalidation as required. The initiator MAY choose to cache > mappings (i.e., reuse STags) across I/Os for efficiency, but that has > security consequences (exposes initiator memory to remote access for > longer) that have to be discussed in the new Security Considerations tex= t. In contrast, I believe the other five items are routine updates that can just be made in the next version of the draft. Thanks, --David > -----Original Message----- > From: storm-bounces@ietf.org [mailto:storm-bounces@ietf.org] On Behalf Of > david.black@emc.com > Sent: Thursday, May 03, 2012 6:29 PM > To: storm@ietf.org > Subject: [storm] iSER approach - update >=20 > Attempting to summarize where we are ... the changes from my earlier atte= mpt > are: > - Dropped the MUST for error reporting by the receiver. > - Added a more general "SHOULD NOT use" to iii). > - Added caching discussion and need for yet more security considerations > text to v). > - Added a warning about relying on SendSE and SendInvSE at receiver - new= item > vi). >=20 > Are we getting closer? Here's the new version: >=20 > i) Some iSER implementations have not followed RFC 5046's strict requirem= ents > for use of SendSE, SendInv and SendInvSE; they use Send instead. > ii) For interoperability, iSER implementations SHOULD accept and correctl= y > process SendSE, SendInv and SendInvSE messages. > iii) SendSE, SendInv and SendInvSE are to be regarded as optimizations or > enhancements to the basic Send message, and their support may vary by > RCaP protocol and specific implementation. In general, these messages > SHOULD NOT be used, unless the RCaP requires support for them in all > implementations. If these messages are used, the implementation SHOULD > be capable of reverting to use of Send in order to work with a receiver > that does not support these message (and we need to note that attempted > use of these messages with a peer that doesn't support them may result > in a fatal error that closes the RCaP connection). A specific instance > that needs to be noted is that these messages SHOULD NOT be used with > the InfiniBand RCaP because InfiniBand does not require support for them > in all cases. > iv) New iSER implementations SHOULD use Send (and not these three additio= nal > messages) unless there are compelling reasons for doing otherwise > (latter is implicit in use of "SHOULD", but worth saying explicitly, > IMHO). > v) Some new text will be needed (especially Security Considerations) to > make it clear that STag invalidation is the initiator's responsibility > for security reasons, and the initiator cannot rely on the target > using an Invalidate version of Send - the initiator MUST check and > do its own invalidation as required. The initiator MAY choose to cache > mappings (i.e., reuse STags) across I/Os for efficiency, but that has > security consequences (exposes initiator memory to remote access for > longer) that have to be discussed in the new Security Considerations tex= t. > vi) Similarly, iSER implementations SHOULD NOT rely on events triggered b= y > SendSE and SendInvSE, as these messages may not be used. In contrast > to invalidation, there are no security consequences in this aspect. >=20 > Thanks, > --David > ---------------------------------------------------- > David L. Black, Distinguished Engineer > EMC Corporation, 176 South St., Hopkinton, MA=A0 01748 > +1 (508) 293-7953=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 FAX: +1 (508) 293-7= 786 > david.black@emc.com=A0=A0=A0=A0=A0=A0=A0 Mobile: +1 (978) 394-7754 > ---------------------------------------------------- >=20 >=20 > _______________________________________________ > storm mailing list > storm@ietf.org > https://www.ietf.org/mailman/listinfo/storm From mkosjc@gmail.com Wed May 16 20:47:24 2012 Return-Path: X-Original-To: storm@ietfa.amsl.com Delivered-To: storm@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 8113B11E8073 for ; Wed, 16 May 2012 20:47:24 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -3.598 X-Spam-Level: X-Spam-Status: No, score=-3.598 tagged_above=-999 required=5 tests=[BAYES_00=-2.599, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_LOW=-1] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 8Ti+6KfMKPvs for ; Wed, 16 May 2012 20:47:23 -0700 (PDT) Received: from mail-vb0-f44.google.com (mail-vb0-f44.google.com [209.85.212.44]) by ietfa.amsl.com (Postfix) with ESMTP id 1075421F864A for ; Wed, 16 May 2012 20:47:22 -0700 (PDT) Received: by vbbez10 with SMTP id ez10so1613086vbb.31 for ; Wed, 16 May 2012 20:47:22 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=mgwg2zV9Kh9c6ngruH7AEFtJkl8CBxMLQyhSqbsf5D4=; b=huljDeiMJNRXY+UGU4l0h5WLpzfbDrSC7oAqeBHlHPPYrvVFNuyDQJmIKCnRBotxJm h7nBiXaW4pi1eYMBDGcXmHL1rU3Qdy3BoW2fi/oMnPOLMfMJjx2Dazv1r+HKKMbCRVtq RqGMPZBCfPmw7kvwf62tycu+jiL+Gfql/jV5Dg4w3Qx5N55QrCyIpQtIh291YGWgw078 armd8NZg4NpJuF3rNNoyM8YdVl/nzVaB/GxnkqUb8nVqs+U8dvvQowbTrtTvmNDGTJ5X sfX96MNmvHzyYW5JZJrR8KDcVa+WJsClbeQB3vDuGL/EgwfD2U1IHLXQExuuCHana0Xm QdBw== MIME-Version: 1.0 Received: by 10.220.241.135 with SMTP id le7mr4004298vcb.63.1337226441954; Wed, 16 May 2012 20:47:21 -0700 (PDT) Received: by 10.220.169.15 with HTTP; Wed, 16 May 2012 20:47:21 -0700 (PDT) In-Reply-To: <8D3D17ACE214DC429325B2B98F3AE712057151F0@MX15A.corp.emc.com> References: <8D3D17ACE214DC429325B2B98F3AE71203D1120D@MX15A.corp.emc.com> <8D3D17ACE214DC429325B2B98F3AE712057151F0@MX15A.corp.emc.com> Date: Wed, 16 May 2012 20:47:21 -0700 Message-ID: From: Michael Ko To: david.black@emc.com Content-Type: multipart/alternative; boundary=14dae9cdca7f0e95f504c0334bc3 Cc: storm@ietf.org Subject: Re: [storm] iSER approach - update: Next Steps X-BeenThere: storm@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Storage Maintenance WG List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 17 May 2012 03:47:24 -0000 --14dae9cdca7f0e95f504c0334bc3 Content-Type: text/plain; charset=ISO-8859-1 I am submitting the proposed changes pertaining to item (v) in David's note that David and I (mostly David) worked out to be reviewed by the group before they go into the new iSER draft. These are the changes to sec. 2.4.1: OLD The iSER layer at the initiator is required to invalildate the Advertised STag upon a normal completion of the associated task. The Send with Invalidate Message, if supported by the RCaP layer (e.g., iWARP), can be used for automatic invalidation when it is used to carry the SCSI Response PDU. There are two exceptions to this automatic invalidation - bidirectional commands, and abnormal completion of a command. The iSER Layer at the initiator is required to explicitly invalidate the STag in these cases, in addition to sanity checking the automatic invalidation even when that does happen. NEW The iSER layer at the initiator SHOULD invalidate the Advertised STag upon a normal completion of the associated task. The Send with Invalidate Message, if supported by the RCaP layer (e.g., iWARP), can be used for automatic invalidation when it is used to carry the SCSI Response PDU. There are two exceptions to this automatic invalidation - bidirectional commands, and abnormal completion of a command. The iSER Layer at the initiator SHOULD explicitly invalidate the STag in these two cases. That iSER layer MUST check that STag invalidation has occurred whenever receipt of a Send with Invalidate message is the expected means of causing an STag to be invalidated, and MUST perform the STag invalidation if the STag has not already been invalidated (e.g., because a Send message was used instead of Send with Invalidate). If the Advertised STag is not invalidated as recommended in the foregoing paragraph (e.g., in order to cache the STag for future reuse), the I/O Buffer remains exposed to the network for access by the RCaP. Such an I/O Buffer is capable of being read or written by the RCaP outside the scope of the iSCSI operation for which it was originally established, which has both robustness and security considerations. The robustness considerations are that the system containing the iSER initiator may react poorly to an unexpected modification of its memory. For the security considerations, see Section 11. END Here is a new Security Considerations paragraph for Section 11: A valid STag exposes I/O Buffer resources to the network for access via the RCaP. The security considerations referred to in the above paragraphs provide means of controlling that access in order to prevent undesired disclosure or modification of data in the I/O Buffer. These considerations are of heightened importance for implementations that do not invalidate the STag after completion of the associated task (ISCSI I/O operation) because the period of exposure is correspondingly longer. For this reason, STag invalidation after completion of the associated task is RECOMMENDED in Section 2.4.1 Mike On Tue, May 15, 2012 at 10:54 AM, wrote: > Having seen no dissension, I believe (with WG chair hat on) that we have > WG rough consensus for this updated approach to resolving the iSER issues > around SendSE, SendInv and SendInvSE. > > Mike (Ko) - please update the draft accordingly, *BUT*, I want to see the > Security Considerations and related caching text for the following item > proposed to the list and worked out on the list before going into the next > version of t he draft (I'd be happy to review a first draft of this text > via private email if that would help): > > > v) Some new text will be needed (especially Security Considerations) to > > make it clear that STag invalidation is the initiator's > responsibility > > for security reasons, and the initiator cannot rely on the target > > using an Invalidate version of Send - the initiator MUST check and > > do its own invalidation as required. The initiator MAY choose to > cache > > mappings (i.e., reuse STags) across I/Os for efficiency, but that > has > > security consequences (exposes initiator memory to remote access > for > > longer) that have to be discussed in the new Security > Considerations text. > > In contrast, I believe the other five items are routine updates that can > just be made in the next version of the draft. > > Thanks, > --David > > > -----Original Message----- > > From: storm-bounces@ietf.org [mailto:storm-bounces@ietf.org] On Behalf > Of > > david.black@emc.com > > Sent: Thursday, May 03, 2012 6:29 PM > > To: storm@ietf.org > > Subject: [storm] iSER approach - update > > > > Attempting to summarize where we are ... the changes from my earlier > attempt > > are: > > - Dropped the MUST for error reporting by the receiver. > > - Added a more general "SHOULD NOT use" to iii). > > - Added caching discussion and need for yet more security considerations > > text to v). > > - Added a warning about relying on SendSE and SendInvSE at receiver - > new item > > vi). > > > > Are we getting closer? Here's the new version: > > > > i) Some iSER implementations have not followed RFC 5046's strict > requirements > > for use of SendSE, SendInv and SendInvSE; they use Send instead. > > ii) For interoperability, iSER implementations SHOULD accept and > correctly > > process SendSE, SendInv and SendInvSE messages. > > iii) SendSE, SendInv and SendInvSE are to be regarded as optimizations or > > enhancements to the basic Send message, and their support may vary > by > > RCaP protocol and specific implementation. In general, these > messages > > SHOULD NOT be used, unless the RCaP requires support for them in > all > > implementations. If these messages are used, the implementation > SHOULD > > be capable of reverting to use of Send in order to work with a > receiver > > that does not support these message (and we need to note that > attempted > > use of these messages with a peer that doesn't support them may > result > > in a fatal error that closes the RCaP connection). A specific > instance > > that needs to be noted is that these messages SHOULD NOT be used > with > > the InfiniBand RCaP because InfiniBand does not require support > for them > > in all cases. > > iv) New iSER implementations SHOULD use Send (and not these three > additional > > messages) unless there are compelling reasons for doing otherwise > > (latter is implicit in use of "SHOULD", but worth saying > explicitly, > > IMHO). > > v) Some new text will be needed (especially Security Considerations) to > > make it clear that STag invalidation is the initiator's > responsibility > > for security reasons, and the initiator cannot rely on the target > > using an Invalidate version of Send - the initiator MUST check and > > do its own invalidation as required. The initiator MAY choose to > cache > > mappings (i.e., reuse STags) across I/Os for efficiency, but that > has > > security consequences (exposes initiator memory to remote access > for > > longer) that have to be discussed in the new Security > Considerations text. > > vi) Similarly, iSER implementations SHOULD NOT rely on events triggered > by > > SendSE and SendInvSE, as these messages may not be used. In > contrast > > to invalidation, there are no security consequences in this aspect. > > > > Thanks, > > --David > > ---------------------------------------------------- > > David L. Black, Distinguished Engineer > > EMC Corporation, 176 South St., Hopkinton, MA 01748 > > +1 (508) 293-7953 FAX: +1 (508) 293-7786 > > david.black@emc.com Mobile: +1 (978) 394-7754 > > ---------------------------------------------------- > --14dae9cdca7f0e95f504c0334bc3 Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable I am submitting the proposed changes pertaining to item (v) in David's = note that David and I (mostly David) worked out to be reviewed by the group= before they go into the new iSER draft.

These are the changes to se= c. 2.4.1:

OLD

The iSER layer at the initiator is required to invalildate t= he Advertised STag upon a normal completion of the associated task.=A0 The = Send with Invalidate Message, if supported by the RCaP layer (e.g., iWARP),= can be used for automatic invalidation when it is used to carry the SCSI R= esponse PDU.=A0 There are two exceptions to this automatic invalidation - b= idirectional commands, and abnormal completion of a command.=A0 The iSER La= yer at the initiator is required to explicitly invalidate the STag in these= cases, in addition to sanity checking the automatic invalidation even when= that does happen.

NEW

The iSER layer at the initiator SHOULD invalidate the Advert= ised STag upon a normal completion of the associated task.=A0 The Send with= Invalidate Message, if supported by the RCaP layer (e.g., iWARP), can be u= sed for automatic invalidation when it is used to carry the SCSI Response P= DU.=A0 There are two exceptions to this automatic invalidation - bidirectio= nal commands, and abnormal completion of a command.=A0 The iSER Layer at th= e initiator SHOULD explicitly invalidate the STag in these two cases.=A0 Th= at iSER layer MUST check that STag invalidation has occurred whenever recei= pt of a Send with Invalidate message is the expected means of causing an ST= ag to be invalidated, and MUST perform the STag invalidation if the STag ha= s not already been invalidated (e.g., because a Send message was used inste= ad of Send with Invalidate).=A0

If the Advertised STag is not invalidated as recommended in the foregoi= ng paragraph (e.g., in order to cache the STag for future reuse), the I/O B= uffer remains exposed to the network for access by the RCaP.=A0 Such an I/O= Buffer is capable of being read or written by the RCaP outside the scope o= f the iSCSI operation for which it was originally established, which has bo= th robustness and security considerations.=A0 The robustness considerations= are that the system containing the iSER initiator may react poorly to an u= nexpected modification of its memory.=A0 For the security considerations, s= ee Section 11.

END

Here is a new Security Considerations paragraph for Section = 11:

A valid STag exposes I/O Buffer resources to the network for acc= ess via the RCaP.=A0 The security considerations referred to in the above p= aragraphs provide means of controlling that access in order to prevent unde= sired disclosure or modification of data in the I/O Buffer.=A0 These consid= erations are of heightened importance for implementations that do not inval= idate the STag after completion of the associated task (ISCSI I/O operation= ) because the period of exposure is correspondingly longer.=A0 For this rea= son, STag invalidation after completion of the associated task is RECOMMEND= ED in Section 2.4.1

Mike

On Tue, May 15, 2012 at 10:54 AM= , <david.black@emc.com> wrote:
Having seen no dissension, I believe (with WG chair hat on) that we have WG rough consensus for this updated approach to resolving the iSER issues around SendSE, SendInv and SendInvSE.

Mike (Ko) - please update the draft accordingly, *BUT*, I want to see the Security Considerations and related caching text for the following item
proposed to the list and worked out on the list before going into the next<= br> version of t he draft (I'd be happy to review a first draft of this tex= t
via private email if that would help):

> v) Some new text will be needed (especially Security Considerations) t= o
> =A0 =A0 =A0 make it clear that STag invalidation is the initiator'= s responsibility
> =A0 =A0 =A0 for security reasons, and the initiator cannot rely on the= target
> =A0 =A0 =A0 using an Invalidate version of Send - the initiator MUST c= heck and
> =A0 =A0 =A0 do its own invalidation as required. =A0The initiator MAY = choose to cache
> =A0 =A0 =A0 mappings (i.e., reuse STags) across I/Os for efficiency, b= ut that has
> =A0 =A0 =A0 security consequences (exposes initiator memory to remote = access for
> =A0 =A0 =A0 longer) that have to be discussed in the new Security Cons= iderations text.

In contrast, I believe the other five items are routine updates that can just be made in the next version of the draft.

Thanks,
--David

> -----Original Message-----
> From: storm-bounces@ietf.org= [mailto:storm-bounces@ietf.o= rg] On Behalf Of
> david.black@emc.com
> Sent: Thursday, May 03, 2012 6:29 PM
> To: storm@ietf.org
> Subject: [storm] iSER approach - update
>
> Attempting to summarize where we are ... the changes from my earlier a= ttempt
> are:
> - Dropped the MUST for error reporting by the receiver.
> - Added a more general "SHOULD NOT use" to iii).
> - Added caching discussion and need for yet more security consideratio= ns
> =A0 =A0 =A0 text to v).
> - Added a warning about relying on SendSE and SendInvSE at receiver - = new item
> =A0 =A0 =A0 vi).
>
> Are we getting closer? =A0Here's the new version:
>
> i) Some iSER implementations have not followed RFC 5046's strict r= equirements
> =A0 =A0 =A0 for use of SendSE, SendInv and SendInvSE; they use Send in= stead.
> ii) For interoperability, iSER implementations SHOULD accept and corre= ctly
> =A0 =A0 =A0 process SendSE, SendInv and SendInvSE messages.
> iii) SendSE, SendInv and SendInvSE are to be regarded as optimizations= or
> =A0 =A0 =A0 enhancements to the basic Send message, and their support = may vary by
> =A0 =A0 =A0 RCaP protocol and specific implementation. =A0In general, = these messages
> =A0 =A0 =A0 SHOULD NOT be used, unless the RCaP requires support for t= hem in all
> =A0 =A0 =A0 implementations. =A0If these messages are used, the implem= entation SHOULD
> =A0 =A0 =A0 be capable of reverting to use of Send in order to work wi= th a receiver
> =A0 =A0 =A0 that does not support these message (and we need to note t= hat attempted
> =A0 =A0 =A0 use of these messages with a peer that doesn't support= them may result
> =A0 =A0 =A0 in a fatal error that closes the RCaP connection). =A0A sp= ecific instance
> =A0 =A0 =A0 that needs to be noted is that these messages SHOULD NOT b= e used with
> =A0 =A0 =A0 the InfiniBand RCaP because InfiniBand does not require su= pport for them
> =A0 =A0 =A0 in all cases.
> iv) New iSER implementations SHOULD use Send (and not these three addi= tional
> =A0 =A0 =A0 messages) unless there are compelling reasons for doing ot= herwise
> =A0 =A0 =A0 (latter is implicit in use of "SHOULD", but wort= h saying explicitly,
> =A0 =A0 =A0 IMHO).
> v) Some new text will be needed (especially Security Considerations) t= o
> =A0 =A0 =A0 make it clear that STag invalidation is the initiator'= s responsibility
> =A0 =A0 =A0 for security reasons, and the initiator cannot rely on the= target
> =A0 =A0 =A0 using an Invalidate version of Send - the initiator MUST c= heck and
> =A0 =A0 =A0 do its own invalidation as required. =A0The initiator MAY = choose to cache
> =A0 =A0 =A0 mappings (i.e., reuse STags) across I/Os for efficiency, b= ut that has
> =A0 =A0 =A0 security consequences (exposes initiator memory to remote = access for
> =A0 =A0 =A0 longer) that have to be discussed in the new Security Cons= iderations text.
> vi) Similarly, iSER implementations SHOULD NOT rely on events triggere= d by
> =A0 =A0 =A0 SendSE and SendInvSE, as these messages may not be used. = =A0In contrast
> =A0 =A0 =A0 to invalidation, there are no security consequences in thi= s aspect.
>
> Thanks,
> --David
> ----------------------------------------------------
> David L. Black, Distinguished Engineer
> EMC Corporation, 176 South St., Hopkinton, MA=A0 01748
> +1 = (508) 293-7953=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 FAX: +1 (508) 293-7786 > david.black@emc.com=A0=A0= =A0=A0=A0=A0=A0 Mobile: +1 (978) 394-7754
> ----------------------------------------------------

--14dae9cdca7f0e95f504c0334bc3-- From david.black@emc.com Thu May 17 06:51:41 2012 Return-Path: X-Original-To: storm@ietfa.amsl.com Delivered-To: storm@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 49B5C21F8650 for ; Thu, 17 May 2012 06:51:41 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -110.48 X-Spam-Level: X-Spam-Status: No, score=-110.48 tagged_above=-999 required=5 tests=[AWL=0.118, BAYES_00=-2.599, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_HI=-8, USER_IN_WHITELIST=-100] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id byH6nDLRF93v for ; Thu, 17 May 2012 06:51:34 -0700 (PDT) Received: from mexforward.lss.emc.com (mexforward.lss.emc.com [128.222.32.20]) by ietfa.amsl.com (Postfix) with ESMTP id CDEE621F86D4 for ; Thu, 17 May 2012 06:51:29 -0700 (PDT) Received: from hop04-l1d11-si04.isus.emc.com (HOP04-L1D11-SI04.isus.emc.com [10.254.111.24]) by mexforward.lss.emc.com (Switch-3.4.3/Switch-3.4.3) with ESMTP id q4HDpSTs019673 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO) for ; Thu, 17 May 2012 09:51:28 -0400 Received: from mailhub.lss.emc.com (mailhubhoprd02.lss.emc.com [10.254.221.253]) by hop04-l1d11-si04.isus.emc.com (RSA Interceptor) for ; Thu, 17 May 2012 09:51:17 -0400 Received: from mxhub10.corp.emc.com (mxhub10.corp.emc.com [10.254.92.105]) by mailhub.lss.emc.com (Switch-3.4.3/Switch-3.4.3) with ESMTP id q4HDp8kP021028 for ; Thu, 17 May 2012 09:51:17 -0400 Received: from mx15a.corp.emc.com ([169.254.1.236]) by mxhub10.corp.emc.com ([10.254.92.105]) with mapi; Thu, 17 May 2012 09:51:13 -0400 From: To: Importance: high X-Priority: 1 Date: Thu, 17 May 2012 09:51:12 -0400 Thread-Topic: iSER - new security text: Please review Thread-Index: Ac0z39UGaXBix7OwSYKHJ7L+Ie/AigAU1IPA Message-ID: <8D3D17ACE214DC429325B2B98F3AE712057153EB@MX15A.corp.emc.com> References: <8D3D17ACE214DC429325B2B98F3AE71203D1120D@MX15A.corp.emc.com> <8D3D17ACE214DC429325B2B98F3AE712057151F0@MX15A.corp.emc.com> In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: acceptlanguage: en-US Content-Type: multipart/alternative; boundary="_000_8D3D17ACE214DC429325B2B98F3AE712057153EBMX15Acorpemccom_" MIME-Version: 1.0 X-EMM-MHVC: 1 Subject: [storm] iSER - new security text: Please review X-BeenThere: storm@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Storage Maintenance WG List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 17 May 2012 13:51:41 -0000 --_000_8D3D17ACE214DC429325B2B98F3AE712057153EBMX15Acorpemccom_ Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable Any review and comments on this new text need to happen ASAP, as the revise= d iSER draft will probably be submitted and an RFC publication request for = it sent to the IESG over this coming weekend. Now that we *finally* have a= n approach to this issue that appears to work, I really do want this draft = moving onward ... Also, I will be on vacation with effectively no email access May 25-June 11= . After I get back, we're going to need to take up the RDMAP extensions dr= aft. Thanks, --David (as storm WG co-chair) From: Michael Ko [mailto:mkosjc@gmail.com] Sent: Wednesday, May 16, 2012 11:47 PM To: Black, David Cc: storm@ietf.org Subject: Re: [storm] iSER approach - update: Next Steps I am submitting the proposed changes pertaining to item (v) in David's note= that David and I (mostly David) worked out to be reviewed by the group bef= ore they go into the new iSER draft. These are the changes to sec. 2.4.1: OLD The iSER layer at the initiator is required to invalildate the Advertised S= Tag upon a normal completion of the associated task. The Send with Invalid= ate Message, if supported by the RCaP layer (e.g., iWARP), can be used for = automatic invalidation when it is used to carry the SCSI Response PDU. The= re are two exceptions to this automatic invalidation - bidirectional comman= ds, and abnormal completion of a command. The iSER Layer at the initiator = is required to explicitly invalidate the STag in these cases, in addition t= o sanity checking the automatic invalidation even when that does happen. NEW The iSER layer at the initiator SHOULD invalidate the Advertised STag upon = a normal completion of the associated task. The Send with Invalidate Messa= ge, if supported by the RCaP layer (e.g., iWARP), can be used for automatic= invalidation when it is used to carry the SCSI Response PDU. There are tw= o exceptions to this automatic invalidation - bidirectional commands, and a= bnormal completion of a command. The iSER Layer at the initiator SHOULD ex= plicitly invalidate the STag in these two cases. That iSER layer MUST chec= k that STag invalidation has occurred whenever receipt of a Send with Inval= idate message is the expected means of causing an STag to be invalidated, a= nd MUST perform the STag invalidation if the STag has not already been inva= lidated (e.g., because a Send message was used instead of Send with Invalid= ate). If the Advertised STag is not invalidated as recommended in the foregoing p= aragraph (e.g., in order to cache the STag for future reuse), the I/O Buffe= r remains exposed to the network for access by the RCaP. Such an I/O Buffe= r is capable of being read or written by the RCaP outside the scope of the = iSCSI operation for which it was originally established, which has both rob= ustness and security considerations. The robustness considerations are tha= t the system containing the iSER initiator may react poorly to an unexpecte= d modification of its memory. For the security considerations, see Section= 11. END Here is a new Security Considerations paragraph for Section 11: A valid STag exposes I/O Buffer resources to the network for access via the= RCaP. The security considerations referred to in the above paragraphs pro= vide means of controlling that access in order to prevent undesired disclos= ure or modification of data in the I/O Buffer. These considerations are of= heightened importance for implementations that do not invalidate the STag = after completion of the associated task (ISCSI I/O operation) because the p= eriod of exposure is correspondingly longer. For this reason, STag invalid= ation after completion of the associated task is RECOMMENDED in Section 2.4= .1 Mike On Tue, May 15, 2012 at 10:54 AM, > wrote: Having seen no dissension, I believe (with WG chair hat on) that we have WG rough consensus for this updated approach to resolving the iSER issues around SendSE, SendInv and SendInvSE. Mike (Ko) - please update the draft accordingly, *BUT*, I want to see the Security Considerations and related caching text for the following item proposed to the list and worked out on the list before going into the next version of t he draft (I'd be happy to review a first draft of this text via private email if that would help): > v) Some new text will be needed (especially Security Considerations) to > make it clear that STag invalidation is the initiator's responsibil= ity > for security reasons, and the initiator cannot rely on the target > using an Invalidate version of Send - the initiator MUST check and > do its own invalidation as required. The initiator MAY choose to c= ache > mappings (i.e., reuse STags) across I/Os for efficiency, but that h= as > security consequences (exposes initiator memory to remote access fo= r > longer) that have to be discussed in the new Security Consideration= s text. In contrast, I believe the other five items are routine updates that can just be made in the next version of the draft. Thanks, --David > -----Original Message----- > From: storm-bounces@ietf.org [mailto:storm= -bounces@ietf.org] On Behalf Of > david.black@emc.com > Sent: Thursday, May 03, 2012 6:29 PM > To: storm@ietf.org > Subject: [storm] iSER approach - update > > Attempting to summarize where we are ... the changes from my earlier atte= mpt > are: > - Dropped the MUST for error reporting by the receiver. > - Added a more general "SHOULD NOT use" to iii). > - Added caching discussion and need for yet more security considerations > text to v). > - Added a warning about relying on SendSE and SendInvSE at receiver - new= item > vi). > > Are we getting closer? Here's the new version: > > i) Some iSER implementations have not followed RFC 5046's strict requirem= ents > for use of SendSE, SendInv and SendInvSE; they use Send instead. > ii) For interoperability, iSER implementations SHOULD accept and correctl= y > process SendSE, SendInv and SendInvSE messages. > iii) SendSE, SendInv and SendInvSE are to be regarded as optimizations or > enhancements to the basic Send message, and their support may vary = by > RCaP protocol and specific implementation. In general, these messa= ges > SHOULD NOT be used, unless the RCaP requires support for them in al= l > implementations. If these messages are used, the implementation SH= OULD > be capable of reverting to use of Send in order to work with a rece= iver > that does not support these message (and we need to note that attem= pted > use of these messages with a peer that doesn't support them may res= ult > in a fatal error that closes the RCaP connection). A specific inst= ance > that needs to be noted is that these messages SHOULD NOT be used wi= th > the InfiniBand RCaP because InfiniBand does not require support for= them > in all cases. > iv) New iSER implementations SHOULD use Send (and not these three additio= nal > messages) unless there are compelling reasons for doing otherwise > (latter is implicit in use of "SHOULD", but worth saying explicitly= , > IMHO). > v) Some new text will be needed (especially Security Considerations) to > make it clear that STag invalidation is the initiator's responsibil= ity > for security reasons, and the initiator cannot rely on the target > using an Invalidate version of Send - the initiator MUST check and > do its own invalidation as required. The initiator MAY choose to c= ache > mappings (i.e., reuse STags) across I/Os for efficiency, but that h= as > security consequences (exposes initiator memory to remote access fo= r > longer) that have to be discussed in the new Security Consideration= s text. > vi) Similarly, iSER implementations SHOULD NOT rely on events triggered b= y > SendSE and SendInvSE, as these messages may not be used. In contra= st > to invalidation, there are no security consequences in this aspect. > > Thanks, > --David > ---------------------------------------------------- > David L. Black, Distinguished Engineer > EMC Corporation, 176 South St., Hopkinton, MA 01748 > +1 (508) 293-7953 FAX: +1 (5= 08) 293-7786 > david.black@emc.com Mobile: +1 (978) 3= 94-7754 > ---------------------------------------------------- --_000_8D3D17ACE214DC429325B2B98F3AE712057153EBMX15Acorpemccom_ Content-Type: text/html; charset="us-ascii" Content-Transfer-Encoding: quoted-printable

Any review and comme= nts on this new text need to happen ASAP, as the revised iSER draft will pr= obably be submitted and an RFC publication request for it sent to the IESG = over this coming weekend.  Now that we *finally* have an approa= ch to this issue that appears to work, I really do want this draft moving o= nward ...

 

Also, I will be on vacation with effectively no email acc= ess May 25-June 11.  After I get back, we’re going to need to ta= ke up the RDMAP extensions draft.

 

Thanks,
--David (as storm= WG co-chair)

 

From: Michael Ko [mai= lto:mkosjc@gmail.com]
Sent: Wednesday, May 16, 2012 11:47 PM
= To: Black, David
Cc: storm@ietf.org
Subject: Re:= [storm] iSER approach - update: Next Steps

 

I am submitting the proposed changes pertaining to ite= m (v) in David's note that David and I (mostly David) worked out to be revi= ewed by the group before they go into the new iSER draft.

These are = the changes to sec. 2.4.1:

OLD

The iSER layer at the initiat= or is required to invalildate the Advertised STag upon a normal completion = of the associated task.  The Send with Invalidate Message, if supporte= d by the RCaP layer (e.g., iWARP), can be used for automatic invalidation w= hen it is used to carry the SCSI Response PDU.  There are two exceptio= ns to this automatic invalidation - bidirectional commands, and abnormal co= mpletion of a command.  The iSER Layer at the initiator is required to= explicitly invalidate the STag in these cases, in addition to sanity check= ing the automatic invalidation even when that does happen.

NEW
The iSER layer at the initiator SHOULD invalidate the Advertised STag upo= n a normal completion of the associated task.  The Send with Invalidat= e Message, if supported by the RCaP layer (e.g., iWARP), can be used for au= tomatic invalidation when it is used to carry the SCSI Response PDU.  = There are two exceptions to this automatic invalidation - bidirectional com= mands, and abnormal completion of a command.  The iSER Layer at the in= itiator SHOULD explicitly invalidate the STag in these two cases.  Tha= t iSER layer MUST check that STag invalidation has occurred whenever receip= t of a Send with Invalidate message is the expected means of causing an STa= g to be invalidated, and MUST perform the STag invalidation if the STag has= not already been invalidated (e.g., because a Send message was used instea= d of Send with Invalidate). 

If the Advertised STag is not inv= alidated as recommended in the foregoing paragraph (e.g., in order to cache= the STag for future reuse), the I/O Buffer remains exposed to the network = for access by the RCaP.  Such an I/O Buffer is capable of being read o= r written by the RCaP outside the scope of the iSCSI operation for which it= was originally established, which has both robustness and security conside= rations.  The robustness considerations are that the system containing= the iSER initiator may react poorly to an unexpected modification of its m= emory.  For the security considerations, see Section 11.

END
Here is a new Security Considerations paragraph for Section 11:
A valid STag exposes I/O Buffer resources to the network for access via th= e RCaP.  The security considerations referred to in the above paragrap= hs provide means of controlling that access in order to prevent undesired d= isclosure or modification of data in the I/O Buffer.  These considerat= ions are of heightened importance for implementations that do not invalidat= e the STag after completion of the associated task (ISCSI I/O operation) be= cause the period of exposure is correspondingly longer.  For this reas= on, STag invalidation after completion of the associated task is RECOMMENDE= D in Section 2.4.1

Mike

On T= ue, May 15, 2012 at 10:54 AM, <david.black@emc.com> wrote:

Having seen no dissension, I believe (with WG chair hat on) th= at we have
WG rough consensus for this updated approach to resolving the= iSER issues
around SendSE, SendInv and SendInvSE.

Mike (Ko) - pl= ease update the draft accordingly, *BUT*, I want to see the
Security Con= siderations and related caching text for the following item
proposed to = the list and worked out on the list before going into the next
version o= f t he draft (I'd be happy to review a first draft of this text
via priv= ate email if that would help):

> v) Some new text will be needed = (especially Security Considerations) to
>       make i= t clear that STag invalidation is the initiator's responsibility
> &n= bsp;     for security reasons, and the initiator cannot rely on t= he target
>       using an Invalidate version of Send = - the initiator MUST check and
>       do its own inva= lidation as required.  The initiator MAY choose to cache
>  = ;     mappings (i.e., reuse STags) across I/Os for efficiency, bu= t that has
>       security consequences (exposes init= iator memory to remote access for
>       longer) that= have to be discussed in the new Security Considerations text.

In co= ntrast, I believe the other five items are routine updates that can
just= be made in the next version of the draft.

Thanks,
--David
> -----Original Message-----
> From: storm-bounces@ietf.org [mailto:storm-bounces@ietf.org] On Behalf Of
> david.black@emc.com
> Sent: Thursday= , May 03, 2012 6:29 PM
> To: storm@= ietf.org
> Subject: [storm] iSER approach - update
>
>= ; Attempting to summarize where we are ... the changes from my earlier atte= mpt
> are:
> - Dropped the MUST for error reporting by the rece= iver.
> - Added a more general "SHOULD NOT use" to iii).> - Added caching discussion and need for yet more security considerati= ons
>       text to v).
> - Added a warning abou= t relying on SendSE and SendInvSE at receiver - new item
>   &nb= sp;   vi).
>
> Are we getting closer?  Here's the new= version:
>
> i) Some iSER implementations have not followed RF= C 5046's strict requirements
>       for use of SendSE= , SendInv and SendInvSE; they use Send instead.
> ii) For interoperab= ility, iSER implementations SHOULD accept and correctly
>   &nbs= p;   process SendSE, SendInv and SendInvSE messages.
> iii) Send= SE, SendInv and SendInvSE are to be regarded as optimizations or
> &n= bsp;     enhancements to the basic Send message, and their suppor= t may vary by
>       RCaP protocol and specific imple= mentation.  In general, these messages
>       SH= OULD NOT be used, unless the RCaP requires support for them in all
> =       implementations.  If these messages are used, the= implementation SHOULD
>       be capable of reverting= to use of Send in order to work with a receiver
>     &nbs= p; that does not support these message (and we need to note that attempted<= br>>       use of these messages with a peer that doesn't= support them may result
>       in a fatal error that= closes the RCaP connection).  A specific instance
>   &nbs= p;   that needs to be noted is that these messages SHOULD NOT be used = with
>       the InfiniBand RCaP because InfiniBand do= es not require support for them
>       in all cases.<= br>> iv) New iSER implementations SHOULD use Send (and not these three a= dditional
>       messages) unless there are compellin= g reasons for doing otherwise
>       (latter is impli= cit in use of "SHOULD", but worth saying explicitly,
> &nbs= p;     IMHO).
> v) Some new text will be needed (especially= Security Considerations) to
>       make it clear tha= t STag invalidation is the initiator's responsibility
>    =   for security reasons, and the initiator cannot rely on the target>       using an Invalidate version of Send - the initi= ator MUST check and
>       do its own invalidation as= required.  The initiator MAY choose to cache
>     &n= bsp; mappings (i.e., reuse STags) across I/Os for efficiency, but that has<= br>>       security consequences (exposes initiator memor= y to remote access for
>       longer) that have to be= discussed in the new Security Considerations text.
> vi) Similarly, = iSER implementations SHOULD NOT rely on events triggered by
>   =     SendSE and SendInvSE, as these messages may not be used. &nbs= p;In contrast
>       to invalidation, there are no se= curity consequences in this aspect.
>
> Thanks,
> --David=
> ----------------------------------------------------
> David= L. Black, Distinguished Engineer
> EMC Corporation, 176 South St., H= opkinton, MA  01748
> +1 (508) 293-7953        &nbs= p;    FAX: +1 (5= 08) 293-7786
> david.black= @emc.com        Mobile: +1 (978) 394-7754
> -------------= ---------------------------------------

 

= --_000_8D3D17ACE214DC429325B2B98F3AE712057153EBMX15Acorpemccom_-- From david.black@emc.com Thu May 17 07:55:36 2012 Return-Path: X-Original-To: storm@ietfa.amsl.com Delivered-To: storm@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 803C421F852E for ; Thu, 17 May 2012 07:55:36 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -110.483 X-Spam-Level: X-Spam-Status: No, score=-110.483 tagged_above=-999 required=5 tests=[AWL=0.116, BAYES_00=-2.599, RCVD_IN_DNSWL_HI=-8, USER_IN_WHITELIST=-100] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id KhvNQTIAP2c1 for ; Thu, 17 May 2012 07:55:35 -0700 (PDT) Received: from mexforward.lss.emc.com (mexforward.lss.emc.com [128.222.32.20]) by ietfa.amsl.com (Postfix) with ESMTP id 739F721F85A7 for ; Thu, 17 May 2012 07:55:35 -0700 (PDT) Received: from hop04-l1d11-si01.isus.emc.com (HOP04-L1D11-SI01.isus.emc.com [10.254.111.54]) by mexforward.lss.emc.com (Switch-3.4.3/Switch-3.4.3) with ESMTP id q4HEtTLn027693 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO) for ; Thu, 17 May 2012 10:55:29 -0400 Received: from mailhub.lss.emc.com (mailhub.lss.emc.com [10.254.221.145]) by hop04-l1d11-si01.isus.emc.com (RSA Interceptor) for ; Thu, 17 May 2012 10:55:14 -0400 Received: from mxhub06.corp.emc.com (mxhub06.corp.emc.com [128.222.70.203]) by mailhub.lss.emc.com (Switch-3.4.3/Switch-3.4.3) with ESMTP id q4HEtEdQ006623 for ; Thu, 17 May 2012 10:55:14 -0400 Received: from mx15a.corp.emc.com ([169.254.1.236]) by mxhub06.corp.emc.com ([128.222.70.203]) with mapi; Thu, 17 May 2012 10:55:14 -0400 From: To: Date: Thu, 17 May 2012 10:55:12 -0400 Thread-Topic: Possible Vancouver meeting of storm WG Thread-Index: Ac00PQ+99CKphywNRweumW7si+UWOg== Message-ID: <8D3D17ACE214DC429325B2B98F3AE71205715405@MX15A.corp.emc.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: acceptlanguage: en-US Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-EMM-MHVC: 1 Subject: [storm] Possible Vancouver meeting of storm WG X-BeenThere: storm@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Storage Maintenance WG List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 17 May 2012 14:55:36 -0000 I've just put in an initial request for 1.5 hour meeting slot in Vancouver = (week of July 29th). As was the case with Paris, the final decision on whether to hold or cancel= this meeting will be made about a month in advance (end of June) based on = whether we have anything to work on that needs meeting time. Thanks, --David ---------------------------------------------------- David L. Black, Distinguished Engineer EMC Corporation, 176 South St., Hopkinton, MA=A0 01748 +1 (508) 293-7953=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 FAX: +1 (508) 293-778= 6 david.black@emc.com=A0=A0=A0=A0=A0=A0=A0 Mobile: +1 (978) 394-7754 ---------------------------------------------------- From david.black@emc.com Fri May 18 16:07:17 2012 Return-Path: X-Original-To: storm@ietfa.amsl.com Delivered-To: storm@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 8771321F85ED for ; Fri, 18 May 2012 16:07:17 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -110.598 X-Spam-Level: X-Spam-Status: No, score=-110.598 tagged_above=-999 required=5 tests=[BAYES_00=-2.599, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_HI=-8, USER_IN_WHITELIST=-100] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 0vpo8nZ9Q4tc for ; Fri, 18 May 2012 16:07:13 -0700 (PDT) Received: from mexforward.lss.emc.com (mexforward.lss.emc.com [128.222.32.20]) by ietfa.amsl.com (Postfix) with ESMTP id F3CE021F85C6 for ; Fri, 18 May 2012 16:07:12 -0700 (PDT) Received: from hop04-l1d11-si04.isus.emc.com (HOP04-L1D11-SI04.isus.emc.com [10.254.111.24]) by mexforward.lss.emc.com (Switch-3.4.3/Switch-3.4.3) with ESMTP id q4IN7A6A025243 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO) for ; Fri, 18 May 2012 19:07:10 -0400 Received: from mailhub.lss.emc.com (mailhub.lss.emc.com [10.254.222.129]) by hop04-l1d11-si04.isus.emc.com (RSA Interceptor) for ; Fri, 18 May 2012 19:06:53 -0400 Received: from mxhub26.corp.emc.com (mxhub26.corp.emc.com [10.254.110.182]) by mailhub.lss.emc.com (Switch-3.4.3/Switch-3.4.3) with ESMTP id q4IN6rZK019180 for ; Fri, 18 May 2012 19:06:53 -0400 Received: from mx15a.corp.emc.com ([169.254.1.236]) by mxhub26.corp.emc.com ([10.254.110.182]) with mapi; Fri, 18 May 2012 19:06:52 -0400 From: To: , Date: Fri, 18 May 2012 19:06:50 -0400 Thread-Topic: iSER - new security text: Please review Thread-Index: Ac0z39UGaXBix7OwSYKHJ7L+Ie/AigAU1IPAAEXNSAA= Message-ID: <8D3D17ACE214DC429325B2B98F3AE712057155B4@MX15A.corp.emc.com> References: <8D3D17ACE214DC429325B2B98F3AE71203D1120D@MX15A.corp.emc.com> <8D3D17ACE214DC429325B2B98F3AE712057151F0@MX15A.corp.emc.com> <8D3D17ACE214DC429325B2B98F3AE712057153EB@MX15A.corp.emc.com> In-Reply-To: <8D3D17ACE214DC429325B2B98F3AE712057153EB@MX15A.corp.emc.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: acceptlanguage: en-US Content-Type: multipart/alternative; boundary="_000_8D3D17ACE214DC429325B2B98F3AE712057155B4MX15Acorpemccom_" MIME-Version: 1.0 X-EMM-MHVC: 1 Subject: Re: [storm] iSER - new security text: Please review X-BeenThere: storm@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Storage Maintenance WG List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 18 May 2012 23:07:17 -0000 --_000_8D3D17ACE214DC429325B2B98F3AE712057155B4MX15Acorpemccom_ Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable Having seen no comments, Mike should incorporate this text into the revised= iSER draft (along with changes for the other items) and submit the result.= If I find anything that needs attention, I'll put it into an RFC Editor N= ote as part of the RFC publication request. Mike - please make sure to update the summary of changes from RFC 5046 in A= ppendix A of the new draft. Thanks, --David From: storm-bounces@ietf.org [mailto:storm-bounces@ietf.org] On Behalf Of d= avid.black@emc.com Sent: Thursday, May 17, 2012 9:51 AM To: storm@ietf.org Subject: [storm] iSER - new security text: Please review Importance: High Any review and comments on this new text need to happen ASAP, as the revise= d iSER draft will probably be submitted and an RFC publication request for = it sent to the IESG over this coming weekend. Now that we *finally* have a= n approach to this issue that appears to work, I really do want this draft = moving onward ... Also, I will be on vacation with effectively no email access May 25-June 11= . After I get back, we're going to need to take up the RDMAP extensions dr= aft. Thanks, --David (as storm WG co-chair) From: Michael Ko [mailto:mkosjc@gmail.com] Sent: Wednesday, May 16, 2012 11:47 PM To: Black, David Cc: storm@ietf.org Subject: Re: [storm] iSER approach - update: Next Steps I am submitting the proposed changes pertaining to item (v) in David's note= that David and I (mostly David) worked out to be reviewed by the group bef= ore they go into the new iSER draft. These are the changes to sec. 2.4.1: OLD The iSER layer at the initiator is required to invalildate the Advertised S= Tag upon a normal completion of the associated task. The Send with Invalid= ate Message, if supported by the RCaP layer (e.g., iWARP), can be used for = automatic invalidation when it is used to carry the SCSI Response PDU. The= re are two exceptions to this automatic invalidation - bidirectional comman= ds, and abnormal completion of a command. The iSER Layer at the initiator = is required to explicitly invalidate the STag in these cases, in addition t= o sanity checking the automatic invalidation even when that does happen. NEW The iSER layer at the initiator SHOULD invalidate the Advertised STag upon = a normal completion of the associated task. The Send with Invalidate Messa= ge, if supported by the RCaP layer (e.g., iWARP), can be used for automatic= invalidation when it is used to carry the SCSI Response PDU. There are tw= o exceptions to this automatic invalidation - bidirectional commands, and a= bnormal completion of a command. The iSER Layer at the initiator SHOULD ex= plicitly invalidate the STag in these two cases. That iSER layer MUST chec= k that STag invalidation has occurred whenever receipt of a Send with Inval= idate message is the expected means of causing an STag to be invalidated, a= nd MUST perform the STag invalidation if the STag has not already been inva= lidated (e.g., because a Send message was used instead of Send with Invalid= ate). If the Advertised STag is not invalidated as recommended in the foregoing p= aragraph (e.g., in order to cache the STag for future reuse), the I/O Buffe= r remains exposed to the network for access by the RCaP. Such an I/O Buffe= r is capable of being read or written by the RCaP outside the scope of the = iSCSI operation for which it was originally established, which has both rob= ustness and security considerations. The robustness considerations are tha= t the system containing the iSER initiator may react poorly to an unexpecte= d modification of its memory. For the security considerations, see Section= 11. END Here is a new Security Considerations paragraph for Section 11: A valid STag exposes I/O Buffer resources to the network for access via the= RCaP. The security considerations referred to in the above paragraphs pro= vide means of controlling that access in order to prevent undesired disclos= ure or modification of data in the I/O Buffer. These considerations are of= heightened importance for implementations that do not invalidate the STag = after completion of the associated task (ISCSI I/O operation) because the p= eriod of exposure is correspondingly longer. For this reason, STag invalid= ation after completion of the associated task is RECOMMENDED in Section 2.4= .1 Mike On Tue, May 15, 2012 at 10:54 AM, > wrote: Having seen no dissension, I believe (with WG chair hat on) that we have WG rough consensus for this updated approach to resolving the iSER issues around SendSE, SendInv and SendInvSE. Mike (Ko) - please update the draft accordingly, *BUT*, I want to see the Security Considerations and related caching text for the following item proposed to the list and worked out on the list before going into the next version of t he draft (I'd be happy to review a first draft of this text via private email if that would help): > v) Some new text will be needed (especially Security Considerations) to > make it clear that STag invalidation is the initiator's responsibil= ity > for security reasons, and the initiator cannot rely on the target > using an Invalidate version of Send - the initiator MUST check and > do its own invalidation as required. The initiator MAY choose to c= ache > mappings (i.e., reuse STags) across I/Os for efficiency, but that h= as > security consequences (exposes initiator memory to remote access fo= r > longer) that have to be discussed in the new Security Consideration= s text. In contrast, I believe the other five items are routine updates that can just be made in the next version of the draft. Thanks, --David > -----Original Message----- > From: storm-bounces@ietf.org [mailto:storm= -bounces@ietf.org] On Behalf Of > david.black@emc.com > Sent: Thursday, May 03, 2012 6:29 PM > To: storm@ietf.org > Subject: [storm] iSER approach - update > > Attempting to summarize where we are ... the changes from my earlier atte= mpt > are: > - Dropped the MUST for error reporting by the receiver. > - Added a more general "SHOULD NOT use" to iii). > - Added caching discussion and need for yet more security considerations > text to v). > - Added a warning about relying on SendSE and SendInvSE at receiver - new= item > vi). > > Are we getting closer? Here's the new version: > > i) Some iSER implementations have not followed RFC 5046's strict requirem= ents > for use of SendSE, SendInv and SendInvSE; they use Send instead. > ii) For interoperability, iSER implementations SHOULD accept and correctl= y > process SendSE, SendInv and SendInvSE messages. > iii) SendSE, SendInv and SendInvSE are to be regarded as optimizations or > enhancements to the basic Send message, and their support may vary = by > RCaP protocol and specific implementation. In general, these messa= ges > SHOULD NOT be used, unless the RCaP requires support for them in al= l > implementations. If these messages are used, the implementation SH= OULD > be capable of reverting to use of Send in order to work with a rece= iver > that does not support these message (and we need to note that attem= pted > use of these messages with a peer that doesn't support them may res= ult > in a fatal error that closes the RCaP connection). A specific inst= ance > that needs to be noted is that these messages SHOULD NOT be used wi= th > the InfiniBand RCaP because InfiniBand does not require support for= them > in all cases. > iv) New iSER implementations SHOULD use Send (and not these three additio= nal > messages) unless there are compelling reasons for doing otherwise > (latter is implicit in use of "SHOULD", but worth saying explicitly= , > IMHO). > v) Some new text will be needed (especially Security Considerations) to > make it clear that STag invalidation is the initiator's responsibil= ity > for security reasons, and the initiator cannot rely on the target > using an Invalidate version of Send - the initiator MUST check and > do its own invalidation as required. The initiator MAY choose to c= ache > mappings (i.e., reuse STags) across I/Os for efficiency, but that h= as > security consequences (exposes initiator memory to remote access fo= r > longer) that have to be discussed in the new Security Consideration= s text. > vi) Similarly, iSER implementations SHOULD NOT rely on events triggered b= y > SendSE and SendInvSE, as these messages may not be used. In contra= st > to invalidation, there are no security consequences in this aspect. > > Thanks, > --David > ---------------------------------------------------- > David L. Black, Distinguished Engineer > EMC Corporation, 176 South St., Hopkinton, MA 01748 > +1 (508) 293-7953 FAX: +1 (5= 08) 293-7786 > david.black@emc.com Mobile: +1 (978) 3= 94-7754 > ---------------------------------------------------- --_000_8D3D17ACE214DC429325B2B98F3AE712057155B4MX15Acorpemccom_ Content-Type: text/html; charset="us-ascii" Content-Transfer-Encoding: quoted-printable

Having seen no comme= nts, Mike should incorporate this text into the revised iSER draft (along w= ith changes for the other items) and submit the result.  If I find any= thing that needs attention, I’ll put it into an RFC Editor Note as pa= rt of the RFC publication request.

 

Mike - please make sure to updat= e the summary of changes from RFC 5046 in Appendix A of the new draft.=

 

Thanks,
--David

 

From: storm-bounces@ietf.org [mailto:storm-bounces@ietf.org] On Beha= lf Of david.black@emc.com
Sent: Thursday, May 17, 2012 9:51 A= M
To: storm@ietf.org
Subject: [storm] iSER - new securi= ty text: Please review
Importance: High

 

Any revi= ew and comments on this new text need to happen ASAP, as the revised iSER d= raft will probably be submitted and an RFC publication request for it sent = to the IESG over this coming weekend.  Now that we *finally* ha= ve an approach to this issue that appears to work, I really do want this dr= aft moving onward ...

 

Also, I will be on vacation with effectively = no email access May 25-June 11.  After I get back, we’re going t= o need to take up the RDMAP extensions draft.

 

Thanks,
--Da= vid (as storm WG co-chair)

<= o:p> 

From: Mi= chael Ko [mailto:mkosjc@gmail.com]
Sent: Wednesday, May 16, 2012= 11:47 PM
To: Black, David
Cc: storm@ietf.org
Sub= ject: Re: [storm] iSER approach - update: Next Steps<= /p>

 

I am submitting the proposed changes pert= aining to item (v) in David's note that David and I (mostly David) worked o= ut to be reviewed by the group before they go into the new iSER draft.
<= br>These are the changes to sec. 2.4.1:

OLD

The iSER layer a= t the initiator is required to invalildate the Advertised STag upon a norma= l completion of the associated task.  The Send with Invalidate Message= , if supported by the RCaP layer (e.g., iWARP), can be used for automatic i= nvalidation when it is used to carry the SCSI Response PDU.  There are= two exceptions to this automatic invalidation - bidirectional commands, an= d abnormal completion of a command.  The iSER Layer at the initiator i= s required to explicitly invalidate the STag in these cases, in addition to= sanity checking the automatic invalidation even when that does happen.
=
NEW

The iSER layer at the initiator SHOULD invalidate the Advert= ised STag upon a normal completion of the associated task.  The Send w= ith Invalidate Message, if supported by the RCaP layer (e.g., iWARP), can b= e used for automatic invalidation when it is used to carry the SCSI Respons= e PDU.  There are two exceptions to this automatic invalidation - bidi= rectional commands, and abnormal completion of a command.  The iSER La= yer at the initiator SHOULD explicitly invalidate the STag in these two cas= es.  That iSER layer MUST check that STag invalidation has occurred wh= enever receipt of a Send with Invalidate message is the expected means of c= ausing an STag to be invalidated, and MUST perform the STag invalidation if= the STag has not already been invalidated (e.g., because a Send message wa= s used instead of Send with Invalidate). 

If the Advertised ST= ag is not invalidated as recommended in the foregoing paragraph (e.g., in o= rder to cache the STag for future reuse), the I/O Buffer remains exposed to= the network for access by the RCaP.  Such an I/O Buffer is capable of= being read or written by the RCaP outside the scope of the iSCSI operation= for which it was originally established, which has both robustness and sec= urity considerations.  The robustness considerations are that the syst= em containing the iSER initiator may react poorly to an unexpected modifica= tion of its memory.  For the security considerations, see Section 11.<= br>
END

Here is a new Security Considerations paragraph for Secti= on 11:

A valid STag exposes I/O Buffer resources to the network for = access via the RCaP.  The security considerations referred to in the a= bove paragraphs provide means of controlling that access in order to preven= t undesired disclosure or modification of data in the I/O Buffer.  The= se considerations are of heightened importance for implementations that do = not invalidate the STag after completion of the associated task (ISCSI I/O = operation) because the period of exposure is correspondingly longer.  = For this reason, STag invalidation after completion of the associated task = is RECOMMENDED in Section 2.4.1

Mike

On Tue, May 15, 2012 at 10:54 AM, <david.black@emc.com> wrote:<= /p>

Having seen no dissension, I believe (with WG chair= hat on) that we have
WG rough consensus for this updated approach to re= solving the iSER issues
around SendSE, SendInv and SendInvSE.

Mik= e (Ko) - please update the draft accordingly, *BUT*, I want to see the
S= ecurity Considerations and related caching text for the following item
p= roposed to the list and worked out on the list before going into the nextversion of t he draft (I'd be happy to review a first draft of this text<= br>via private email if that would help):

> v) Some new text will= be needed (especially Security Considerations) to
>     &n= bsp; make it clear that STag invalidation is the initiator's responsibility=
>       for security reasons, and the initiator canno= t rely on the target
>       using an Invalidate versi= on of Send - the initiator MUST check and
>       do i= ts own invalidation as required.  The initiator MAY choose to cache>       mappings (i.e., reuse STags) across I/Os for eff= iciency, but that has
>       security consequences (e= xposes initiator memory to remote access for
>       l= onger) that have to be discussed in the new Security Considerations text.
In contrast, I believe the other five items are routine updates that = can
just be made in the next version of the draft.

Thanks,
--D= avid

> -----Original Message-----
> From: storm-bounces@ietf.org [mailto:storm-bounces@ietf.org] On Behalf Of
>= david.black@emc.com
> Sen= t: Thursday, May 03, 2012 6:29 PM
> To: storm@ietf.org
> Subject: [storm] iSER approach - update
= >
> Attempting to summarize where we are ... the changes from my e= arlier attempt
> are:
> - Dropped the MUST for error reporting = by the receiver.
> - Added a more general "SHOULD NOT use" = to iii).
> - Added caching discussion and need for yet more security = considerations
>       text to v).
> - Added a w= arning about relying on SendSE and SendInvSE at receiver - new item
>=       vi).
>
> Are we getting closer?  Her= e's the new version:
>
> i) Some iSER implementations have not = followed RFC 5046's strict requirements
>       for us= e of SendSE, SendInv and SendInvSE; they use Send instead.
> ii) For = interoperability, iSER implementations SHOULD accept and correctly
> =       process SendSE, SendInv and SendInvSE messages.
>= ; iii) SendSE, SendInv and SendInvSE are to be regarded as optimizations or=
>       enhancements to the basic Send message, and t= heir support may vary by
>       RCaP protocol and spe= cific implementation.  In general, these messages
>    = ;   SHOULD NOT be used, unless the RCaP requires support for them in a= ll
>       implementations.  If these messages ar= e used, the implementation SHOULD
>       be capable o= f reverting to use of Send in order to work with a receiver
>   =     that does not support these message (and we need to note that= attempted
>       use of these messages with a peer t= hat doesn't support them may result
>       in a fatal= error that closes the RCaP connection).  A specific instance
> =       that needs to be noted is that these messages SHOULD N= OT be used with
>       the InfiniBand RCaP because In= finiBand does not require support for them
>       in = all cases.
> iv) New iSER implementations SHOULD use Send (and not th= ese three additional
>       messages) unless there ar= e compelling reasons for doing otherwise
>       (latt= er is implicit in use of "SHOULD", but worth saying explicitly,>       IMHO).
> v) Some new text will be needed = (especially Security Considerations) to
>       make i= t clear that STag invalidation is the initiator's responsibility
> &n= bsp;     for security reasons, and the initiator cannot rely on t= he target
>       using an Invalidate version of Send = - the initiator MUST check and
>       do its own inva= lidation as required.  The initiator MAY choose to cache
>  = ;     mappings (i.e., reuse STags) across I/Os for efficiency, bu= t that has
>       security consequences (exposes init= iator memory to remote access for
>       longer) that= have to be discussed in the new Security Considerations text.
> vi) = Similarly, iSER implementations SHOULD NOT rely on events triggered by
&= gt;       SendSE and SendInvSE, as these messages may not be= used.  In contrast
>       to invalidation, ther= e are no security consequences in this aspect.
>
> Thanks,
&= gt; --David
> ----------------------------------------------------> David L. Black, Distinguished Engineer
> EMC Corporation, 176 S= outh St., Hopkinton, MA  01748
> +1 (508) 293-7953       = ;      FAX: +1 (508) 293-7786
> = david.black@emc.com        Mobile: <= a href=3D"tel:%2B1%20%28978%29%20394-7754">+1 (978) 394-7754
> --= --------------------------------------------------

 

= --_000_8D3D17ACE214DC429325B2B98F3AE712057155B4MX15Acorpemccom_-- From internet-drafts@ietf.org Fri May 18 17:54:51 2012 Return-Path: X-Original-To: storm@ietfa.amsl.com Delivered-To: storm@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 7353721F8496; Fri, 18 May 2012 17:54:51 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -102.242 X-Spam-Level: X-Spam-Status: No, score=-102.242 tagged_above=-999 required=5 tests=[AWL=0.357, BAYES_00=-2.599, USER_IN_WHITELIST=-100] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id aUifH3Wy3g4Y; Fri, 18 May 2012 17:54:50 -0700 (PDT) Received: from ietfa.amsl.com (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id E0AC521F8495; Fri, 18 May 2012 17:54:50 -0700 (PDT) MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable From: internet-drafts@ietf.org To: i-d-announce@ietf.org X-Test-IDTracker: no X-IETF-IDTracker: 4.02 Message-ID: <20120519005450.13747.52463.idtracker@ietfa.amsl.com> Date: Fri, 18 May 2012 17:54:50 -0700 Cc: storm@ietf.org Subject: [storm] I-D Action: draft-ietf-storm-iser-09.txt X-BeenThere: storm@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Storage Maintenance WG List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 19 May 2012 00:54:51 -0000 A New Internet-Draft is available from the on-line Internet-Drafts director= ies. This draft is a work item of the STORage Maintenance Working Group of = the IETF. Title : iSCSI Extensions for RDMA Specification Author(s) : Michael Ko Alexander Nezhinsky Filename : draft-ietf-storm-iser-09.txt Pages : 96 Date : 2012-05-18 iSCSI Extensions for RDMA provides the RDMA data transfer capability to iSCSI by layering iSCSI on top of an RDMA-Capable Protocol. An RDMA-Capable Protocol provides RDMA Read and Write services, which enable data to be transferred directly into SCSI I/O Buffers without intermediate data copies. This document describes the extensions to the iSCSI protocol to support RDMA services as provided by an RDMA- Capable Protocol. This document obsoletes RFC 5046. A URL for this Internet-Draft is: http://www.ietf.org/internet-drafts/draft-ietf-storm-iser-09.txt Internet-Drafts are also available by anonymous FTP at: ftp://ftp.ietf.org/internet-drafts/ This Internet-Draft can be retrieved at: ftp://ftp.ietf.org/internet-drafts/draft-ietf-storm-iser-09.txt The IETF datatracker page for this Internet-Draft is: https://datatracker.ietf.org/doc/draft-ietf-storm-iser/ From internet-drafts@ietf.org Fri May 18 18:09:36 2012 Return-Path: X-Original-To: storm@ietfa.amsl.com Delivered-To: storm@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 4196E21F863F; Fri, 18 May 2012 18:09:36 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -102.259 X-Spam-Level: X-Spam-Status: No, score=-102.259 tagged_above=-999 required=5 tests=[AWL=0.340, BAYES_00=-2.599, USER_IN_WHITELIST=-100] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id WXajs3cK4MPX; Fri, 18 May 2012 18:09:35 -0700 (PDT) Received: from ietfa.amsl.com (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id C8F0E21F8634; Fri, 18 May 2012 18:09:35 -0700 (PDT) MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable From: internet-drafts@ietf.org To: i-d-announce@ietf.org X-Test-IDTracker: no X-IETF-IDTracker: 4.02 Message-ID: <20120519010935.17736.90244.idtracker@ietfa.amsl.com> Date: Fri, 18 May 2012 18:09:35 -0700 Cc: storm@ietf.org Subject: [storm] I-D Action: draft-ietf-storm-iser-10.txt X-BeenThere: storm@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Storage Maintenance WG List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 19 May 2012 01:09:36 -0000 A New Internet-Draft is available from the on-line Internet-Drafts director= ies. This draft is a work item of the STORage Maintenance Working Group of = the IETF. Title : iSCSI Extensions for RDMA Specification Author(s) : Michael Ko Alexander Nezhinsky Filename : draft-ietf-storm-iser-10.txt Pages : 96 Date : 2012-05-18 iSCSI Extensions for RDMA provides the RDMA data transfer capability to iSCSI by layering iSCSI on top of an RDMA-Capable Protocol. An RDMA-Capable Protocol provides RDMA Read and Write services, which enable data to be transferred directly into SCSI I/O Buffers without intermediate data copies. This document describes the extensions to the iSCSI protocol to support RDMA services as provided by an RDMA- Capable Protocol. This document obsoletes RFC 5046. A URL for this Internet-Draft is: http://www.ietf.org/internet-drafts/draft-ietf-storm-iser-10.txt Internet-Drafts are also available by anonymous FTP at: ftp://ftp.ietf.org/internet-drafts/ This Internet-Draft can be retrieved at: ftp://ftp.ietf.org/internet-drafts/draft-ietf-storm-iser-10.txt The IETF datatracker page for this Internet-Draft is: https://datatracker.ietf.org/doc/draft-ietf-storm-iser/ From mkosjc@gmail.com Fri May 18 18:12:03 2012 Return-Path: X-Original-To: storm@ietfa.amsl.com Delivered-To: storm@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id CFAAF21F8631 for ; Fri, 18 May 2012 18:12:03 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -3.598 X-Spam-Level: X-Spam-Status: No, score=-3.598 tagged_above=-999 required=5 tests=[BAYES_00=-2.599, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_LOW=-1] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id YZM+Xq7qw6pe for ; Fri, 18 May 2012 18:12:03 -0700 (PDT) Received: from mail-qa0-f44.google.com (mail-qa0-f44.google.com [209.85.216.44]) by ietfa.amsl.com (Postfix) with ESMTP id E309F21F85B7 for ; Fri, 18 May 2012 18:12:02 -0700 (PDT) Received: by qadz3 with SMTP id z3so558948qad.10 for ; Fri, 18 May 2012 18:12:02 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type; bh=8DqOLIc+X1l1XRsyIr7c54ovXP1f2UnpCtoa9utVaJ8=; b=m3ey3Q0DxZr+ZF5+ytXGWVdcVUQP6qYTRxvQ2iID30uWLHiNaxKn3awrWmsIGspwP/ IJxK0v8ZGqwd1SJXLTQKLX6KDWS9Lugb63pc8AFEkGkj8z9LgQWoaCfg2TBF6nnfs5As KKY1ok15wxQho8V+Q1ygMrtCq7AFNitVQ9c7O/INTjprfYP0m0W4lj56qJQecQ3tzlR4 WRQ2eVn3DGGZ5KOFRCVpu3n9SsXsEelMFXkRm3ttdPu4gbe1qYqng4d038S1bpRgWssM 9OjqxlJFBQOOEiUE7pQ6pPKXiuTDgYhNPWPmDvpyMbqmre213J4uKDpvXx5En7UWasRM EHgQ== MIME-Version: 1.0 Received: by 10.224.31.202 with SMTP id z10mr26183997qac.79.1337389922380; Fri, 18 May 2012 18:12:02 -0700 (PDT) Received: by 10.229.144.201 with HTTP; Fri, 18 May 2012 18:12:02 -0700 (PDT) In-Reply-To: <20120519010935.17736.90244.idtracker@ietfa.amsl.com> References: <20120519010935.17736.90244.idtracker@ietfa.amsl.com> Date: Fri, 18 May 2012 18:12:02 -0700 Message-ID: From: Michael Ko To: storm@ietf.org Content-Type: multipart/alternative; boundary=20cf3074b14c3fe9be04c0595b90 Subject: [storm] Fwd: I-D Action: draft-ietf-storm-iser-10.txt X-BeenThere: storm@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Storage Maintenance WG List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 19 May 2012 01:12:03 -0000 --20cf3074b14c3fe9be04c0595b90 Content-Type: text/plain; charset=ISO-8859-1 Here is the latest iSER draft. The changes are as follows: 1. Updated section 2.4.1 to include discussions on STag invalidation. 2. Updated section 2.4.2 to include discussions on the use of Send message types. 3. Updated section 11 on STag invalidation. 4. Updated section 14 items 9 and 11. Mike ---------- Forwarded message ---------- From: Date: Fri, May 18, 2012 at 6:09 PM Subject: [storm] I-D Action: draft-ietf-storm-iser-10.txt To: i-d-announce@ietf.org Cc: storm@ietf.org A New Internet-Draft is available from the on-line Internet-Drafts directories. This draft is a work item of the STORage Maintenance Working Group of the IETF. Title : iSCSI Extensions for RDMA Specification Author(s) : Michael Ko Alexander Nezhinsky Filename : draft-ietf-storm-iser-10.txt Pages : 96 Date : 2012-05-18 iSCSI Extensions for RDMA provides the RDMA data transfer capability to iSCSI by layering iSCSI on top of an RDMA-Capable Protocol. An RDMA-Capable Protocol provides RDMA Read and Write services, which enable data to be transferred directly into SCSI I/O Buffers without intermediate data copies. This document describes the extensions to the iSCSI protocol to support RDMA services as provided by an RDMA- Capable Protocol. This document obsoletes RFC 5046. A URL for this Internet-Draft is: http://www.ietf.org/internet-drafts/draft-ietf-storm-iser-10.txt Internet-Drafts are also available by anonymous FTP at: ftp://ftp.ietf.org/internet-drafts/ This Internet-Draft can be retrieved at: ftp://ftp.ietf.org/internet-drafts/draft-ietf-storm-iser-10.txt The IETF datatracker page for this Internet-Draft is: https://datatracker.ietf.org/doc/draft-ietf-storm-iser/ _______________________________________________ storm mailing list storm@ietf.org https://www.ietf.org/mailman/listinfo/storm --20cf3074b14c3fe9be04c0595b90 Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Here is the latest iSER draft.=A0 The changes are as follows:

1. Upd= ated section 2.4.1 to include discussions on STag invalidation.
2. Updat= ed section 2.4.2 to include discussions on the use of Send message types. 3. Updated section 11 on STag invalidation.
4. Updated section 14 items = 9 and 11.

Mike

---------- Forwarde= d message ----------
From: <internet-drafts@ie= tf.org>
Date: Fri, May 18, 2012 at 6:09 PM
Subject: [storm] I-D Action: draft-ie= tf-storm-iser-10.txt
To: i-d-an= nounce@ietf.org
Cc: storm@ietf.org=



A New Internet-Draft is available from the on-line Internet-Drafts director= ies. This draft is a work item of the STORage Maintenance Working Group of = the IETF.

=A0 =A0 =A0 =A0Title =A0 =A0 =A0 =A0 =A0 : iSCSI Extensions for RDMA Speci= fication
=A0 =A0 =A0 =A0Author(s) =A0 =A0 =A0 : Michael Ko
=A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0Alexander Nezhinsky
=A0 =A0 =A0 =A0Filename =A0 =A0 =A0 =A0: draft-ietf-storm-iser-10.txt
=A0 =A0 =A0 =A0Pages =A0 =A0 =A0 =A0 =A0 : 96
=A0 =A0 =A0 =A0Date =A0 =A0 =A0 =A0 =A0 =A0: 2012-05-18

=A0 iSCSI Extensions for RDMA provides the RDMA data transfer capability =A0 to iSCSI by layering iSCSI on top of an RDMA-Capable Protocol. =A0An =A0 RDMA-Capable Protocol provides RDMA Read and Write services, which
=A0 enable data to be transferred directly into SCSI I/O Buffers without =A0 intermediate data copies. =A0This document describes the extensions to=
=A0 the iSCSI protocol to support RDMA services as provided by an RDMA- =A0 Capable Protocol.

=A0 This document obsoletes RFC 5046.


A URL for this Internet-Draft is:
http://www.ietf.org/internet-drafts/draft-ietf-storm-is= er-10.txt

Internet-Drafts are also available by anonymous FTP at:
ftp://ftp= .ietf.org/internet-drafts/

This Internet-Draft can be retrieved at:
ftp://ftp.ietf.org/internet-drafts/draft-ietf-storm-iser= -10.txt

The IETF datatracker page for this Internet-Draft is:
https://datatracker.ietf.org/doc/draft-ietf-storm-iser/

_______________________________________________
storm mailing list
storm@ietf.org
h= ttps://www.ietf.org/mailman/listinfo/storm

--20cf3074b14c3fe9be04c0595b90-- From internet-drafts@ietf.org Sun May 20 19:04:05 2012 Return-Path: X-Original-To: storm@ietfa.amsl.com Delivered-To: storm@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 3EA7121F853C; Sun, 20 May 2012 19:04:05 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -102.599 X-Spam-Level: X-Spam-Status: No, score=-102.599 tagged_above=-999 required=5 tests=[BAYES_00=-2.599, USER_IN_WHITELIST=-100] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id nCaN99yKwkRv; Sun, 20 May 2012 19:04:04 -0700 (PDT) Received: from ietfa.amsl.com (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id C8DEC21F8518; Sun, 20 May 2012 19:04:04 -0700 (PDT) MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable From: internet-drafts@ietf.org To: i-d-announce@ietf.org X-Test-IDTracker: no X-IETF-IDTracker: 4.02 Message-ID: <20120521020404.24199.82431.idtracker@ietfa.amsl.com> Date: Sun, 20 May 2012 19:04:04 -0700 Cc: storm@ietf.org Subject: [storm] I-D Action: draft-ietf-storm-iser-11.txt X-BeenThere: storm@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Storage Maintenance WG List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 21 May 2012 02:04:05 -0000 A New Internet-Draft is available from the on-line Internet-Drafts director= ies. This draft is a work item of the STORage Maintenance Working Group of = the IETF. Title : iSCSI Extensions for RDMA Specification Author(s) : Michael Ko Alexander Nezhinsky Filename : draft-ietf-storm-iser-11.txt Pages : 95 Date : 2012-05-20 iSCSI Extensions for RDMA provides the RDMA data transfer capability to iSCSI by layering iSCSI on top of an RDMA-Capable Protocol. An RDMA-Capable Protocol provides RDMA Read and Write services, which enable data to be transferred directly into SCSI I/O Buffers without intermediate data copies. This document describes the extensions to the iSCSI protocol to support RDMA services as provided by an RDMA- Capable Protocol. This document obsoletes RFC 5046. A URL for this Internet-Draft is: http://www.ietf.org/internet-drafts/draft-ietf-storm-iser-11.txt Internet-Drafts are also available by anonymous FTP at: ftp://ftp.ietf.org/internet-drafts/ This Internet-Draft can be retrieved at: ftp://ftp.ietf.org/internet-drafts/draft-ietf-storm-iser-11.txt The IETF datatracker page for this Internet-Draft is: https://datatracker.ietf.org/doc/draft-ietf-storm-iser/ From david.black@emc.com Sun May 20 23:22:57 2012 Return-Path: X-Original-To: storm@ietfa.amsl.com Delivered-To: storm@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 5BC349E8007 for ; Sun, 20 May 2012 23:22:57 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -110.599 X-Spam-Level: X-Spam-Status: No, score=-110.599 tagged_above=-999 required=5 tests=[AWL=0.000, BAYES_00=-2.599, RCVD_IN_DNSWL_HI=-8, USER_IN_WHITELIST=-100] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id FWOjUCCzKq0j for ; Sun, 20 May 2012 23:22:56 -0700 (PDT) Received: from mexforward.lss.emc.com (mexforward.lss.emc.com [128.222.32.20]) by ietfa.amsl.com (Postfix) with ESMTP id C83029E8006 for ; Sun, 20 May 2012 23:22:55 -0700 (PDT) Received: from hop04-l1d11-si01.isus.emc.com (HOP04-L1D11-SI01.isus.emc.com [10.254.111.54]) by mexforward.lss.emc.com (Switch-3.4.3/Switch-3.4.3) with ESMTP id q4L6MkM3007862 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO) for ; Mon, 21 May 2012 02:22:49 -0400 Received: from mailhub.lss.emc.com (mailhubhoprd03.lss.emc.com [10.254.221.145]) by hop04-l1d11-si01.isus.emc.com (RSA Interceptor) for ; Mon, 21 May 2012 02:22:35 -0400 Received: from mxhub28.corp.emc.com (mxhub28.corp.emc.com [10.254.110.184]) by mailhub.lss.emc.com (Switch-3.4.3/Switch-3.4.3) with ESMTP id q4L6MYk2015574 for ; Mon, 21 May 2012 02:22:35 -0400 Received: from mx15a.corp.emc.com ([169.254.1.236]) by mxhub28.corp.emc.com ([10.254.110.184]) with mapi; Mon, 21 May 2012 02:22:34 -0400 From: To: Date: Mon, 21 May 2012 02:22:32 -0400 Thread-Topic: I-D Action: draft-ietf-storm-iser-11.txt Thread-Index: Ac029hbOlhWQl8LTQreOoJ2S5rplTwAI+Rvw Message-ID: <8D3D17ACE214DC429325B2B98F3AE712057155EE@MX15A.corp.emc.com> References: <20120521020404.24199.82431.idtracker@ietfa.amsl.com> In-Reply-To: <20120521020404.24199.82431.idtracker@ietfa.amsl.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: acceptlanguage: en-US Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-EMM-MHVC: 1 Subject: Re: [storm] I-D Action: draft-ietf-storm-iser-11.txt X-BeenThere: storm@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Storage Maintenance WG List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 21 May 2012 06:22:57 -0000 The final reviews required before RFC publication can be requested found a = few minor glitches in -10, which -11 corrects. Thanks, --David > -----Original Message----- > From: i-d-announce-bounces@ietf.org [mailto:i-d-announce-bounces@ietf.org= ] On > Behalf Of internet-drafts@ietf.org > Sent: Sunday, May 20, 2012 10:04 PM > To: i-d-announce@ietf.org > Cc: storm@ietf.org > Subject: I-D Action: draft-ietf-storm-iser-11.txt >=20 >=20 > A New Internet-Draft is available from the on-line Internet-Drafts > directories. This draft is a work item of the STORage Maintenance Working > Group of the IETF. >=20 > Title : iSCSI Extensions for RDMA Specification > Author(s) : Michael Ko > Alexander Nezhinsky > Filename : draft-ietf-storm-iser-11.txt > Pages : 95 > Date : 2012-05-20 >=20 > iSCSI Extensions for RDMA provides the RDMA data transfer capability > to iSCSI by layering iSCSI on top of an RDMA-Capable Protocol. An > RDMA-Capable Protocol provides RDMA Read and Write services, which > enable data to be transferred directly into SCSI I/O Buffers without > intermediate data copies. This document describes the extensions to > the iSCSI protocol to support RDMA services as provided by an RDMA- > Capable Protocol. >=20 > This document obsoletes RFC 5046. >=20 >=20 > A URL for this Internet-Draft is: > http://www.ietf.org/internet-drafts/draft-ietf-storm-iser-11.txt >=20 > Internet-Drafts are also available by anonymous FTP at: > ftp://ftp.ietf.org/internet-drafts/ >=20 > This Internet-Draft can be retrieved at: > ftp://ftp.ietf.org/internet-drafts/draft-ietf-storm-iser-11.txt >=20 > The IETF datatracker page for this Internet-Draft is: > https://datatracker.ietf.org/doc/draft-ietf-storm-iser/ >=20 > _______________________________________________ > I-D-Announce mailing list > I-D-Announce@ietf.org > https://www.ietf.org/mailman/listinfo/i-d-announce > Internet-Draft directories: http://www.ietf.org/shadow.html > or ftp://ftp.ietf.org/ietf/1shadow-sites.txt From david.black@emc.com Sun May 20 23:33:02 2012 Return-Path: X-Original-To: storm@ietfa.amsl.com Delivered-To: storm@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 695CB21F853E for ; Sun, 20 May 2012 23:33:02 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -110.599 X-Spam-Level: X-Spam-Status: No, score=-110.599 tagged_above=-999 required=5 tests=[AWL=0.000, BAYES_00=-2.599, RCVD_IN_DNSWL_HI=-8, USER_IN_WHITELIST=-100] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 1bzAmT3pSFve for ; Sun, 20 May 2012 23:33:00 -0700 (PDT) Received: from mexforward.lss.emc.com (mexforward.lss.emc.com [128.222.32.20]) by ietfa.amsl.com (Postfix) with ESMTP id 015E121F8473 for ; Sun, 20 May 2012 23:32:59 -0700 (PDT) Received: from hop04-l1d11-si03.isus.emc.com (HOP04-L1D11-SI03.isus.emc.com [10.254.111.23]) by mexforward.lss.emc.com (Switch-3.4.3/Switch-3.4.3) with ESMTP id q4L6Wwjb016361 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO) for ; Mon, 21 May 2012 02:32:58 -0400 Received: from mailhub.lss.emc.com (mailhubhoprd04.lss.emc.com [10.254.222.226]) by hop04-l1d11-si03.isus.emc.com (RSA Interceptor) for ; Mon, 21 May 2012 02:32:47 -0400 Received: from mxhub33.corp.emc.com (mxhub33.corp.emc.com [10.254.93.81]) by mailhub.lss.emc.com (Switch-3.4.3/Switch-3.4.3) with ESMTP id q4L6WiZ8027246 for ; Mon, 21 May 2012 02:32:46 -0400 Received: from mx15a.corp.emc.com ([169.254.1.236]) by mxhub33.corp.emc.com ([::1]) with mapi; Mon, 21 May 2012 02:27:27 -0400 From: To: Date: Mon, 21 May 2012 02:27:25 -0400 Thread-Topic: RFC publication request: iSER draft Thread-Index: Ac03GsnZy5p26zG6RL6aaESmLx8GcQ== Message-ID: <8D3D17ACE214DC429325B2B98F3AE712057155F0@MX15A.corp.emc.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: acceptlanguage: en-US Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-EMM-MHVC: 1 Subject: [storm] RFC publication request: iSER draft X-BeenThere: storm@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Storage Maintenance WG List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 21 May 2012 06:33:02 -0000 I just submitted the request to publish the iSER draft: iSCSI Extensions for RDMA Specification draft-ietf-storm-iser-11.txt as a Proposed Standard RFC. The writeup used to request this publication is available from the datatracker, see: http://datatracker.ietf.org/doc/draft-ietf-storm-iser/ Thanks, --David ---------------------------------------------------- David L. Black, Distinguished Engineer EMC Corporation, 176 South St., Hopkinton, MA=A0 01748 +1 (508) 293-7953=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 FAX: +1 (508) 293-778= 6 david.black@emc.com=A0=A0=A0=A0=A0=A0=A0 Mobile: +1 (978) 394-7754 ---------------------------------------------------- From nezhinsky@gmail.com Mon May 21 06:15:35 2012 Return-Path: X-Original-To: storm@ietfa.amsl.com Delivered-To: storm@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id E1D7221F85F9 for ; Mon, 21 May 2012 06:15:35 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.184 X-Spam-Level: X-Spam-Status: No, score=-1.184 tagged_above=-999 required=5 tests=[BAYES_40=-0.185, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_LOW=-1] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id RNg4f3HBykMc for ; Mon, 21 May 2012 06:15:35 -0700 (PDT) Received: from mail-yw0-f44.google.com (mail-yw0-f44.google.com [209.85.213.44]) by ietfa.amsl.com (Postfix) with ESMTP id 0865821F85F7 for ; Mon, 21 May 2012 06:15:34 -0700 (PDT) Received: by yhq56 with SMTP id 56so5150541yhq.31 for ; Mon, 21 May 2012 06:15:34 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:date:message-id:subject:from:to:cc:content-type; bh=XYiwemVcOWbda/sC0ZJb9jhLIXvpZ6zb8XvncqdLoJ4=; b=Zm0HEZ0J7AzJRDpR10XArMbUP/GanEVJwFLAYiNay5/nr41ETWsTCue8YKp6PB/xhT Gj3uJcAbtc6h44f+OHlZryBCqLSL1rjBJ7O+kGOcoOnyWXR2NR8avssRLy9eClV8ZtDT 1457oT7cKKBMa4aNs3FxfrtOQ5ZYiCLvQB1AMAlmNM4fUzn29gBt86aztdS7yAHq8e/S fWgjJaZU20R3y6GLwVfn5Ca+4oqAwtk/AXg/DOTmtdKqtGoPNW1UoZgipfxs8ySircai DhBZ0k5dmn+YEUcKWvNn7GUZnK8Rq7RiroZhIPL3Jsi4ISL7wXpp/b3y1AecDqSHn1gk Zjdw== MIME-Version: 1.0 Received: by 10.42.10.73 with SMTP id p9mr8640159icp.43.1337606134026; Mon, 21 May 2012 06:15:34 -0700 (PDT) Received: by 10.231.211.193 with HTTP; Mon, 21 May 2012 06:15:34 -0700 (PDT) Date: Mon, 21 May 2012 16:15:34 +0300 Message-ID: From: Alexander Nezhinsky To: "storm@ietf.org" , "david.black@emc.com" , mkosjc@gmail.com Content-Type: multipart/alternative; boundary=20cf30244ca377d08a04c08bb24a Cc: Or Gerlitz , Mike Christie Subject: [storm] iSER - problem with unsolicited NOP-IN right after final Login Response X-BeenThere: storm@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Storage Maintenance WG List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 21 May 2012 13:15:36 -0000 --20cf30244ca377d08a04c08bb24a Content-Type: text/plain; charset=UTF-8 Hi I understand that it is a bad timing for sending this kind of mail, now that iSER draft was submitted, but actually we still have a small problem. It is related to the final Login Response handling and the transition to Full-Featured phase on the initiator side in Infiniband setups. When the target receives the final Login Request it send the final Login Response and from its perspective the connection is now in Full Featured Phase (assuming that it agreed to transition in the Login Response being sent). This means that the target is ready to accept SCSI commands, Text Requests etc. sent by the initiator. It also means that the target is eligible to send some unsolicited PDUs, notably unsolicited NOP-INs. With IB sending NOP-IN periodically is the easiest (an almost only feasible) way to determine closed connections reliably, because this kind of error is delivered to user only in response to a previously initiated TX operation. This leaves the initiator in a dubious position. It posts its RX buffers for that connection only when the final Login Response arrives. But during that time (after the target had sent the Last Login Response but before the Full Featured phase related RX-buffers are posted on the initiator side) the target may send the first NOP-IN as it considers the connection in Full Featured phase already and NumOfUnsolicited PDUs accounting for NOP-INs has been negotiated to a non-zero value. If the initiator works with a single RX-buffer posted during the entire login phase (which is a logical thing to do judging by the login exchange protocol) then an error occurs, as no buffers are posted when the NOP-IN arrives and the connection is shut down. Posting a single extra buffer before sending the last Login Request only alleviates the problem. Although this often solves it in practical terms (as the target most probably sends the next NOP-IN only after some timeout period measuring seconds or hundreds of milliseconds), it does not solves it in terms of protocol completeness, as the target MAY theoretically send more than one NOP-IN until FF buffers are posted. This issue was encountered recently in linux iscsi/iser initiator and the above solution has been applied to solve it against the existing target implementation (STGT), but the initiator remains exposed to this kind of errors. The solution is actually quite simple (theoretically) - if we bring back the requirement for iSER Hello exchange then the iSER assisted Full Featured phase does not commence until HelloReply PDU arrives at the target and the initiator has a definitive point in time when it can safely post its RX buffers - after the final LoginResponse returns but before it sends iSER Hello PDU. In practical terms it means that iSER Hello support requirement should be brought back to spec, which is a hassle. Should we decide on this now? Alexander P.S. : Thanx to Mike Christie and Or Gerlitz, the maintainers of linux iSCSI and iSER initiator for raising the issue. --20cf30244ca377d08a04c08bb24a Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable
Hi

I understand that it is a bad timing for se= nding this kind of mail, now that iSER draft was submitted,
but actually= we still have a small problem.

It is related to the=C2=A0final Login Response handling and the transi= tion to Full-Featured phase on the initiator side in
Infiniband setups. =

When the=C2=A0target receives=C2=A0the final Login Request=C2=A0it = send the final Login Response and from its perspective
the connection is now in Full Featured Phase (assuming that it agreed to tr= ansition in the Login Response being sent).

This means that the targ= et is ready to accept SCSI commands, Text Requests etc. sent by the initiat= or.
It also means that the target is eligible to send some unsolicited PDUs,=C2= =A0notably unsolicited NOP-INs.

With IB sending NOP-IN periodically = is the easiest (an almost only feasible) way to determine closed connection= s
reliably, because this kind of error is delivered to user only in response = to a previously initiated TX operation.

This leaves the initiator in= a dubious position. It posts its RX buffers for that connection only when = the final
Login Response arrives. But during that time (after the target had sent the= Last Login Response but before
the Full Featured phase related RX-buffe= rs are posted on the initiator side) the target may send
the first NOP-= IN as it considers the connection in Full Featured phase already and NumOfU= nsolicited PDUs
accounting for NOP-INs has been negotiated to a non-zero value.

If = the initiator works with a single RX-buffer posted during the entire login = phase (which is a logical thing to do
judging by the login exchange pro= tocol) then an error occurs, as no buffers are posted when the NOP-IN arriv= es
and the connection is shut down.

Posting a single extra buffer befor= e sending the last Login Request only alleviates the problem. Although this=
often solves it in practical terms (as the target most probably sends t= he next NOP-IN only after some timeout
period measuring seconds or hundreds of milliseconds), it does not solves i= t in terms of protocol completeness,
as the target MAY theoretically sen= d more than one NOP-IN until FF buffers are posted.

This issue was e= ncountered recently in linux iscsi/iser initiator and the above solution ha= s been applied to solve
it against the existing target implementation (STGT), but the initiator rem= ains exposed to this kind of errors.

The solution is actually quite = simple (theoretically) - if we bring back the requirement for iSER Hello ex= change
then the iSER assisted Full Featured phase does not commence until HelloRep= ly PDU arrives at the target
and the initiator has a definitive point i= n time when it can safely post its RX buffers - after the final LoginRespon= se
returns but before it sends iSER Hello PDU.

In practical terms it me= ans that iSER Hello support requirement should be brought back to spec, whi= ch is a hassle.

Should we decide on this now?

Alexander

P.S. : Thanx to Mike Christie and Or Gerlitz, the maintainers of linux = iSCSI and iSER initiator for raising the issue.


--20cf30244ca377d08a04c08bb24a-- From mkosjc@gmail.com Mon May 21 07:22:39 2012 Return-Path: X-Original-To: storm@ietfa.amsl.com Delivered-To: storm@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 0376421F85B5 for ; Mon, 21 May 2012 07:22:39 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -3.598 X-Spam-Level: X-Spam-Status: No, score=-3.598 tagged_above=-999 required=5 tests=[BAYES_00=-2.599, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_LOW=-1] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id f47zcQSqL069 for ; Mon, 21 May 2012 07:22:36 -0700 (PDT) Received: from mail-qc0-f172.google.com (mail-qc0-f172.google.com [209.85.216.172]) by ietfa.amsl.com (Postfix) with ESMTP id 101F021F8567 for ; Mon, 21 May 2012 07:22:35 -0700 (PDT) Received: by qcsq13 with SMTP id q13so3981951qcs.31 for ; Mon, 21 May 2012 07:22:35 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=DRGJxtf/F8PMXQbis0fbkQXSHfQC3e6TCkzUE6PHW7M=; b=E3CmPnVZ08wCH/n1c8L/f9ubfixk+yjPNk6kIP81NXMNUwtBdHR1nHUexMUCVDkcQs lmQjo3ZsF3UkdWZXN+/b2EW6iT/u7VsepwsVaXHLzhELjJBblSUDpK6a+HQ0wSdDiDdF 3O54yXubmhlb+qwH3a/OCbWT7tIBGuWLl/s4SDOGi3D4PWhfbyyA/UVLmYvcQIYeuTH4 NloXAf2wIvr/WgJZA7iaIdGV17sthKaqZ914ScMyMu5hS3/FRH1EdlbVIiqFAAMN/y41 1fB7lv/zsPEGJPZYGcQWjX11oJPjxnJI85J57xURZHq/hM449zyDj8gc7DvQemC2dONG mZiQ== MIME-Version: 1.0 Received: by 10.224.184.211 with SMTP id cl19mr38401157qab.52.1337610155476; Mon, 21 May 2012 07:22:35 -0700 (PDT) Received: by 10.229.144.201 with HTTP; Mon, 21 May 2012 07:22:35 -0700 (PDT) In-Reply-To: References: Date: Mon, 21 May 2012 07:22:35 -0700 Message-ID: From: Michael Ko To: Alexander Nezhinsky Content-Type: multipart/alternative; boundary=20cf302d4e3c2a478404c08ca2ca Cc: Mike Christie , Or Gerlitz , "storm@ietf.org" Subject: Re: [storm] iSER - problem with unsolicited NOP-IN right after final Login Response X-BeenThere: storm@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Storage Maintenance WG List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 21 May 2012 14:22:39 -0000 --20cf302d4e3c2a478404c08ca2ca Content-Type: text/plain; charset=ISO-8859-1 Alex, The iSER Hello support has never been removed in the latest spec. Only its use is made optional. So during login negotiation, just negotiate iSERHelloRequired to Yes. Mike On Mon, May 21, 2012 at 6:15 AM, Alexander Nezhinsky wrote: > Hi > > I understand that it is a bad timing for sending this kind of mail, now > that iSER draft was submitted, > but actually we still have a small problem. > > It is related to the final Login Response handling and the transition to > Full-Featured phase on the initiator side in > Infiniband setups. > > When the target receives the final Login Request it send the final Login > Response and from its perspective > the connection is now in Full Featured Phase (assuming that it agreed to > transition in the Login Response being sent). > > This means that the target is ready to accept SCSI commands, Text Requests > etc. sent by the initiator. > It also means that the target is eligible to send some unsolicited > PDUs, notably unsolicited NOP-INs. > > With IB sending NOP-IN periodically is the easiest (an almost only > feasible) way to determine closed connections > reliably, because this kind of error is delivered to user only in response > to a previously initiated TX operation. > > This leaves the initiator in a dubious position. It posts its RX buffers > for that connection only when the final > Login Response arrives. But during that time (after the target had sent > the Last Login Response but before > the Full Featured phase related RX-buffers are posted on the initiator > side) the target may send > the first NOP-IN as it considers the connection in Full Featured phase > already and NumOfUnsolicited PDUs > accounting for NOP-INs has been negotiated to a non-zero value. > > If the initiator works with a single RX-buffer posted during the entire > login phase (which is a logical thing to do > judging by the login exchange protocol) then an error occurs, as no > buffers are posted when the NOP-IN arrives > and the connection is shut down. > > Posting a single extra buffer before sending the last Login Request only > alleviates the problem. Although this > often solves it in practical terms (as the target most probably sends the > next NOP-IN only after some timeout > period measuring seconds or hundreds of milliseconds), it does not solves > it in terms of protocol completeness, > as the target MAY theoretically send more than one NOP-IN until FF buffers > are posted. > > This issue was encountered recently in linux iscsi/iser initiator and the > above solution has been applied to solve > it against the existing target implementation (STGT), but the initiator > remains exposed to this kind of errors. > > The solution is actually quite simple (theoretically) - if we bring back > the requirement for iSER Hello exchange > then the iSER assisted Full Featured phase does not commence until > HelloReply PDU arrives at the target > and the initiator has a definitive point in time when it can safely post > its RX buffers - after the final LoginResponse > returns but before it sends iSER Hello PDU. > > In practical terms it means that iSER Hello support requirement should be > brought back to spec, which is a hassle. > > Should we decide on this now? > > Alexander > > P.S. : Thanx to Mike Christie and Or Gerlitz, the maintainers of linux > iSCSI and iSER initiator for raising the issue. > > > --20cf302d4e3c2a478404c08ca2ca Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Alex,

The iSER Hello support has never been removed in the latest sp= ec.=A0 Only its use is made optional.=A0 So during login negotiation, just = negotiate iSERHelloRequired to Yes.

Mike

On Mon, May 21, 2012 at 6:15 AM, Alexander Nezhinsky <= nezhinsky@gmail.co= m> wrote:
Hi

I understand that it is a bad timing for se= nding this kind of mail, now that iSER draft was submitted,
but actually= we still have a small problem.

It is related to the=A0final Login Response handling and the transitio= n to Full-Featured phase on the initiator side in
Infiniband setups.
When the=A0target receives=A0the final Login Request=A0it send the fin= al Login Response and from its perspective
the connection is now in Full Featured Phase (assuming that it agreed to tr= ansition in the Login Response being sent).

This means that the targ= et is ready to accept SCSI commands, Text Requests etc. sent by the initiat= or.
It also means that the target is eligible to send some unsolicited PDUs,=A0= notably unsolicited NOP-INs.

With IB sending NOP-IN periodically is = the easiest (an almost only feasible) way to determine closed connections reliably, because this kind of error is delivered to user only in response = to a previously initiated TX operation.

This leaves the initiator in= a dubious position. It posts its RX buffers for that connection only when = the final
Login Response arrives. But during that time (after the target had sent the= Last Login Response but before
the Full Featured phase related RX-buffe= rs are posted on the initiator side) the target may send
the first NOP-= IN as it considers the connection in Full Featured phase already and NumOfU= nsolicited PDUs
accounting for NOP-INs has been negotiated to a non-zero value.

If = the initiator works with a single RX-buffer posted during the entire login = phase (which is a logical thing to do
judging by the login exchange pro= tocol) then an error occurs, as no buffers are posted when the NOP-IN arriv= es
and the connection is shut down.

Posting a single extra buffer befor= e sending the last Login Request only alleviates the problem. Although this=
often solves it in practical terms (as the target most probably sends t= he next NOP-IN only after some timeout
period measuring seconds or hundreds of milliseconds), it does not solves i= t in terms of protocol completeness,
as the target MAY theoretically sen= d more than one NOP-IN until FF buffers are posted.

This issue was e= ncountered recently in linux iscsi/iser initiator and the above solution ha= s been applied to solve
it against the existing target implementation (STGT), but the initiator rem= ains exposed to this kind of errors.

The solution is actually quite = simple (theoretically) - if we bring back the requirement for iSER Hello ex= change
then the iSER assisted Full Featured phase does not commence until HelloRep= ly PDU arrives at the target
and the initiator has a definitive point i= n time when it can safely post its RX buffers - after the final LoginRespon= se
returns but before it sends iSER Hello PDU.

In practical terms it me= ans that iSER Hello support requirement should be brought back to spec, whi= ch is a hassle.

Should we decide on this now?=

Alexander

P.S. : Thanx to Mike Christie and Or Gerlitz, the maintai= ners of linux iSCSI and iSER initiator for raising the issue.



--20cf302d4e3c2a478404c08ca2ca-- From david.black@emc.com Thu May 24 00:03:11 2012 Return-Path: X-Original-To: storm@ietfa.amsl.com Delivered-To: storm@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id B782011E80A6 for ; Thu, 24 May 2012 00:03:11 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -110.598 X-Spam-Level: X-Spam-Status: No, score=-110.598 tagged_above=-999 required=5 tests=[AWL=-0.000, BAYES_00=-2.599, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_HI=-8, USER_IN_WHITELIST=-100] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id HYfodSX-EppY for ; Thu, 24 May 2012 00:03:08 -0700 (PDT) Received: from mexforward.lss.emc.com (mexforward.lss.emc.com [128.222.32.20]) by ietfa.amsl.com (Postfix) with ESMTP id B2AF111E80A3 for ; Thu, 24 May 2012 00:03:07 -0700 (PDT) Received: from hop04-l1d11-si04.isus.emc.com (HOP04-L1D11-SI04.isus.emc.com [10.254.111.24]) by mexforward.lss.emc.com (Switch-3.4.3/Switch-3.4.3) with ESMTP id q4O72spH005886 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Thu, 24 May 2012 03:02:54 -0400 Received: from mailhub.lss.emc.com (mailhub.lss.emc.com [10.254.222.226]) by hop04-l1d11-si04.isus.emc.com (RSA Interceptor); Thu, 24 May 2012 03:02:42 -0400 Received: from mxhub05.corp.emc.com (mxhub05.corp.emc.com [128.222.70.202]) by mailhub.lss.emc.com (Switch-3.4.3/Switch-3.4.3) with ESMTP id q4O72gHB003995; Thu, 24 May 2012 03:02:42 -0400 Received: from mx15a.corp.emc.com ([169.254.1.236]) by mxhub05.corp.emc.com ([128.222.70.202]) with mapi; Thu, 24 May 2012 03:02:41 -0400 From: To: , Importance: high X-Priority: 1 Date: Thu, 24 May 2012 03:02:41 -0400 Thread-Topic: iSER - problem with unsolicited NOP-IN right after final Login Response Thread-Index: Ac03XVA1mYxqyKC/Qsy890Mqfd96yQCBTsDg Message-ID: <8D3D17ACE214DC429325B2B98F3AE71205813C92@MX15A.corp.emc.com> References: In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: acceptlanguage: en-US Content-Type: multipart/alternative; boundary="_000_8D3D17ACE214DC429325B2B98F3AE71205813C92MX15Acorpemccom_" MIME-Version: 1.0 X-EMM-MHVC: 1 Cc: ogerlitz@mellanox.com, michaelc@cs.wisc.edu, storm@ietf.org Subject: Re: [storm] iSER - problem with unsolicited NOP-IN right after final Login Response X-BeenThere: storm@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Storage Maintenance WG List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 24 May 2012 07:03:11 -0000 --_000_8D3D17ACE214DC429325B2B98F3AE71205813C92MX15Acorpemccom_ Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable Mike (Ko) and Alexander, Mike is of course correct that iSER Hello usage can be forced by negotiatin= g iSERHelloRequired to "Yes". However, existing implementations are likely= to reply with iSERHelloRequired=3DNotUnderstood, so we do need to specify = what should be done in order to interoperate with an implementation that re= fuses to deal with the iSER Hello exchange. I think the situation that Alexander described should be documented in a ne= w section 5.1.4 of the iSER draft. My general rule of thumb on this sort o= f surprise found by implementers in the "running code" is that it indicates= that something is missing in the spec. I believe that Alexander has descr= ibed the solution - more below. The new section 5.1.4 (suggested section title: Omission of the iSER Hello = Exchange) should describe default omission of the exchange, use of iSERHell= oRequired key to omit the iSER Hello exchange, and the consequences of targ= et use of unsolicited PDUs after login when the exchange is omitted, includ= ing IB's use of NOP-IN (as a keep-alive measure, right?) The crucial requirements points that I take away from Alexander's descripti= on are that if the iSER Hello exchange is omitted, then: 1) The target MAY send *one* unsolicited PDU immediately after sending the = Login Response. 2) The target MUST wait at least 200ms (use some other number if 200ms isn'= t a good choice) or until it receives a full feature mode PDU from the initiator before send= ing a second unsolicited PDU in order to ensure that initiator has sufficient time to allocate the full feature buffer resources for the connection= . 3) The initiator SHOULD allocate at least one additional buffer for use dur= ing login (so that at least two buffers are in use during login) in order to receive an unsoli= cited PDU that may follow login completion. Failure to allocate this second buffer m= ay cause connection termination if no buffer is available when an unsolicited = PDU arrives. Both Mike and I are on vacation, so it may be a few weeks until we can agre= e on the new text and get a -12 version of the draft with that new text sub= mitted. In the interim, I've asked our AD (Martin Stiemerling) to hold off= on further processing of the iSER draft until a -12 version with this new = text is submitted. I'd prefer to work this text out now rather than deal w= ith it as an IETF Last Call comment - as the problem turned up in actual im= plementations, I think it's worth the extra month that it's likely to take = to get correct text on how to avoid the problem into the draft. I'd suggest that Mike Ko post an initial draft of the text for the new sect= ion 5.1.4 to the list when he resurfaces ... Thanks, --David From: Michael Ko [mailto:mkosjc@gmail.com] Sent: Monday, May 21, 2012 10:23 AM To: Alexander Nezhinsky Cc: storm@ietf.org; Black, David; Or Gerlitz; Mike Christie Subject: Re: iSER - problem with unsolicited NOP-IN right after final Login= Response Alex, The iSER Hello support has never been removed in the latest spec. Only its= use is made optional. So during login negotiation, just negotiate iSERHel= loRequired to Yes. Mike On Mon, May 21, 2012 at 6:15 AM, Alexander Nezhinsky > wrote: Hi I understand that it is a bad timing for sending this kind of mail, now tha= t iSER draft was submitted, but actually we still have a small problem. It is related to the final Login Response handling and the transition to Fu= ll-Featured phase on the initiator side in Infiniband setups. When the target receives the final Login Request it send the final Login Re= sponse and from its perspective the connection is now in Full Featured Phase (assuming that it agreed to tr= ansition in the Login Response being sent). This means that the target is ready to accept SCSI commands, Text Requests = etc. sent by the initiator. It also means that the target is eligible to send some unsolicited PDUs, no= tably unsolicited NOP-INs. With IB sending NOP-IN periodically is the easiest (an almost only feasible= ) way to determine closed connections reliably, because this kind of error is delivered to user only in response = to a previously initiated TX operation. This leaves the initiator in a dubious position. It posts its RX buffers fo= r that connection only when the final Login Response arrives. But during that time (after the target had sent the= Last Login Response but before the Full Featured phase related RX-buffers are posted on the initiator side= ) the target may send the first NOP-IN as it considers the connection in Full Featured phase alre= ady and NumOfUnsolicited PDUs accounting for NOP-INs has been negotiated to a non-zero value. If the initiator works with a single RX-buffer posted during the entire log= in phase (which is a logical thing to do judging by the login exchange protocol) then an error occurs, as no buffers= are posted when the NOP-IN arrives and the connection is shut down. Posting a single extra buffer before sending the last Login Request only al= leviates the problem. Although this often solves it in practical terms (as the target most probably sends the n= ext NOP-IN only after some timeout period measuring seconds or hundreds of milliseconds), it does not solves i= t in terms of protocol completeness, as the target MAY theoretically send more than one NOP-IN until FF buffers = are posted. This issue was encountered recently in linux iscsi/iser initiator and the a= bove solution has been applied to solve it against the existing target implementation (STGT), but the initiator rem= ains exposed to this kind of errors. The solution is actually quite simple (theoretically) - if we bring back th= e requirement for iSER Hello exchange then the iSER assisted Full Featured phase does not commence until HelloRep= ly PDU arrives at the target and the initiator has a definitive point in time when it can safely post it= s RX buffers - after the final LoginResponse returns but before it sends iSER Hello PDU. In practical terms it means that iSER Hello support requirement should be b= rought back to spec, which is a hassle. Should we decide on this now? Alexander P.S. : Thanx to Mike Christie and Or Gerlitz, the maintainers of linux iSCS= I and iSER initiator for raising the issue. --_000_8D3D17ACE214DC429325B2B98F3AE71205813C92MX15Acorpemccom_ Content-Type: text/html; charset="us-ascii" Content-Transfer-Encoding: quoted-printable

Mike (Ko) and Alexan= der,

 

Mike is of course correct that iSER Hello usage can be forced = by negotiating iSERHelloRequired to “Yes”.  However, exist= ing implementations are likely to reply with iSERHelloRequired=3DNotUnderst= ood, so we do need to specify what should be done in order to interoperate = with an implementation that refuses to deal with the iSER Hello exchange.

 

I think the situation that Alexander described should be documente= d in a new section 5.1.4 of the iSER draft.  My general rule of thumb = on this sort of surprise found by implementers in the “running code&#= 8221; is that it indicates that something is missing in the spec.  I b= elieve that Alexander has described the solution - more below.

 

= The = new section 5.1.4 (suggested section title: Omission of the iSER Hello Exch= ange) should describe default omission of the exchange, use of iSERHelloReq= uired key to omit the iSER Hello exchange, and the consequences of target u= se of unsolicited PDUs after login when the exchange is omitted, including = IB’s use of NOP-IN (as a keep-alive measure, right?)

 

The cruc= ial requirements points that I take away from Alexander’s description= are that if the iSER Hello exchange is omitted, then:

 

1) The targe= t MAY send *one* unsolicited PDU immediately after sending the Login= Response.

 

2) The target MUST wait at least 200ms (use some other n= umber if 200ms isn’t a good choice)

or until it receives a full feature mode PD= U from the initiator before sending a

second unsolicited PDU in order to ensure that = initiator has sufficient

  =     time to allocate the full feature buffer resources for t= he connection.

3) The initiator SHOUL= D allocate at least one additional buffer for use during login (so that

at least two = buffers are in use during login) in order to receive an unsolicited PDU

that may foll= ow login completion.  Failure to allocate this second buffer may<= /o:p>

cause connectio= n termination if no buffer is available when an unsolicited PDU arrives.

 

Both Mike and I are on vacation, so it may be a few weeks until we ca= n agree on the new text and get a -12 version of the draft with that new te= xt submitted.  In the interim, I’ve asked our AD (Martin Stiemer= ling) to hold off on further processing of the iSER draft until a -12 versi= on with this new text is submitted.  I’d prefer to work this tex= t out now rather than deal with it as an IETF Last Call comment - as the pr= oblem turned up in actual implementations, I think it’s worth the ext= ra month that it’s likely to take to get correct text on how to avoid= the problem into the draft.

&nb= sp;

I’d suggest that Mike Ko post an= initial draft of the text for the new section 5.1.4 to the list when he re= surfaces ...

 =

Thanks,
--David

=

 

From: Michael Ko [mailto:mkosjc@gmail.com]
Sent: Mond= ay, May 21, 2012 10:23 AM
To: Alexander Nezhinsky
Cc: s= torm@ietf.org; Black, David; Or Gerlitz; Mike Christie
Subject: R= e: iSER - problem with unsolicited NOP-IN right after final Login Response<= o:p>

 

=

Alex,

The iSER H= ello support has never been removed in the latest spec.  Only its use = is made optional.  So during login negotiation, just negotiate iSERHel= loRequired to Yes.

Mike

On M= on, May 21, 2012 at 6:15 AM, Alexander Nezhinsky <nezhinsky@gmail.com> wrote:<= /o:p>

Hi
I understand that it is a bad timing for sending this kind of mail, n= ow that iSER draft was submitted,
but actually we still have a small pro= blem.

It is related to the final Login Response handling and the tra= nsition to Full-Featured phase on the initiator side in
Infiniband setup= s.

When the target receives the final Login Request = it send the final Login Response and from its perspective
the connection= is now in Full Featured Phase (assuming that it agreed to transition in th= e Login Response being sent).

This means that the target is ready to= accept SCSI commands, Text Requests etc. sent by the initiator.
It als= o means that the target is eligible to send some unsolicited PDUs, not= ably unsolicited NOP-INs.

With IB sending NOP-IN periodically is the= easiest (an almost only feasible) way to determine closed connections
r= eliably, because this kind of error is delivered to user only in response t= o a previously initiated TX operation.

This leaves the initiator in = a dubious position. It posts its RX buffers for that connection only when t= he final
Login Response arrives. But during that time (after the target = had sent the Last Login Response but before
the Full Featured phase rela= ted RX-buffers are posted on the initiator side) the target may send
th= e first NOP-IN as it considers the connection in Full Featured phase alread= y and NumOfUnsolicited PDUs
accounting for NOP-INs has been negotiated = to a non-zero value.

If the initiator works with a single RX-buffer= posted during the entire login phase (which is a logical thing to do
j= udging by the login exchange protocol) then an error occurs, as no buffers = are posted when the NOP-IN arrives
and the connection is shut down.
<= br>Posting a single extra buffer before sending the last Login Request only= alleviates the problem. Although this
often solves it in practical term= s (as the target most probably sends the next NOP-IN only after some timeou= t
period measuring seconds or hundreds of milliseconds), it does not sol= ves it in terms of protocol completeness,
as the target MAY theoreticall= y send more than one NOP-IN until FF buffers are posted.

This issue = was encountered recently in linux iscsi/iser initiator and the above soluti= on has been applied to solve
it against the existing target implementati= on (STGT), but the initiator remains exposed to this kind of errors.
The solution is actually quite simple (theoretically) - if we bring back t= he requirement for iSER Hello exchange
then the iSER assisted Full Featu= red phase does not commence until HelloReply PDU arrives at the target
= and the initiator has a definitive point in time when it can safely post it= s RX buffers - after the final LoginResponse
returns but before it send= s iSER Hello PDU.

In practical terms it means that iSER Hello suppor= t requirement should be brought back to spec, which is a hassle.

Sho= uld we decide on this now?

Alexander

P.S. : Thanx to Mike Christie and = Or Gerlitz, the maintainers of linux iSCSI and iSER initiator for raising t= he issue.

 

= --_000_8D3D17ACE214DC429325B2B98F3AE71205813C92MX15Acorpemccom_-- From cbm@chadalapaka.com Fri May 25 18:02:01 2012 Return-Path: X-Original-To: storm@ietfa.amsl.com Delivered-To: storm@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 0F37F21F86D4 for ; Fri, 25 May 2012 18:02:01 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -6.598 X-Spam-Level: X-Spam-Status: No, score=-6.598 tagged_above=-999 required=5 tests=[BAYES_00=-2.599, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_MED=-4] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id FuNapTCGDUf2 for ; Fri, 25 May 2012 18:01:56 -0700 (PDT) Received: from tx2outboundpool.messaging.microsoft.com (tx2ehsobe001.messaging.microsoft.com [65.55.88.11]) by ietfa.amsl.com (Postfix) with ESMTP id 625FF21F86EE for ; Fri, 25 May 2012 18:01:56 -0700 (PDT) Received: from mail161-tx2-R.bigfish.com (10.9.14.250) by TX2EHSOBE003.bigfish.com (10.9.40.23) with Microsoft SMTP Server id 14.1.225.23; Sat, 26 May 2012 01:01:43 +0000 Received: from mail161-tx2 (localhost [127.0.0.1]) by mail161-tx2-R.bigfish.com (Postfix) with ESMTP id 13C65E0206; Sat, 26 May 2012 01:01:43 +0000 (UTC) X-Forefront-Antispam-Report: CIP:157.56.240.117; KIP:(null); UIP:(null); IPV:NLI; H:BL2PRD0610HT004.namprd06.prod.outlook.com; RD:none; EFVD:NLI X-SpamScore: -25 X-BigFish: PS-25(zz9371Ic85fh98dK4015Izz1202hzz1033IL8275bh8275dhz2fh2a8h668h839hd25hf0ah) Received-SPF: pass (mail161-tx2: domain of chadalapaka.com designates 157.56.240.117 as permitted sender) client-ip=157.56.240.117; envelope-from=cbm@chadalapaka.com; helo=BL2PRD0610HT004.namprd06.prod.outlook.com ; .outlook.com ; Received: from mail161-tx2 (localhost.localdomain [127.0.0.1]) by mail161-tx2 (MessageSwitch) id 1337994099778995_22651; Sat, 26 May 2012 01:01:39 +0000 (UTC) Received: from TX2EHSMHS026.bigfish.com (unknown [10.9.14.240]) by mail161-tx2.bigfish.com (Postfix) with ESMTP id B0D7AC0046; Sat, 26 May 2012 01:01:39 +0000 (UTC) Received: from BL2PRD0610HT004.namprd06.prod.outlook.com (157.56.240.117) by TX2EHSMHS026.bigfish.com (10.9.99.126) with Microsoft SMTP Server (TLS) id 14.1.225.23; Sat, 26 May 2012 01:01:39 +0000 Received: from BL2PRD0610MB361.namprd06.prod.outlook.com ([169.254.10.84]) by BL2PRD0610HT004.namprd06.prod.outlook.com ([10.255.101.39]) with mapi id 14.16.0152.000; Sat, 26 May 2012 01:01:51 +0000 From: Mallikarjun Chadalapaka To: "david.black@emc.com" , "mkosjc@gmail.com" , "nezhinsky@gmail.com" Thread-Topic: [storm] iSER - problem with unsolicited NOP-IN right after final Login Response Thread-Index: AQHNOXtOIIaCinNGykWqG66+HjtgdZbbOw+g Date: Sat, 26 May 2012 01:01:50 +0000 Message-ID: References: <8D3D17ACE214DC429325B2B98F3AE71205813C92@MX15A.corp.emc.com> In-Reply-To: <8D3D17ACE214DC429325B2B98F3AE71205813C92@MX15A.corp.emc.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [131.107.174.123] Content-Type: multipart/alternative; boundary="_000_E160851FCED17643AE5F53B5D4D0783A092D0294BL2PRD0610MB361_" MIME-Version: 1.0 X-OriginatorOrg: chadalapaka.com Cc: "ogerlitz@mellanox.com" , "michaelc@cs.wisc.edu" , "storm@ietf.org" Subject: Re: [storm] iSER - problem with unsolicited NOP-IN right after final Login Response X-BeenThere: storm@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Storage Maintenance WG List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 26 May 2012 01:02:01 -0000 --_000_E160851FCED17643AE5F53B5D4D0783A092D0294BL2PRD0610MB361_ Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable I am curious to understand a bit more on why this is a protocol issue per s= e. Seems like one way to address this problem is via an implementation approac= h with the initiator posting in advance the negotiated number of unsolicite= d PDU buffers, at the same time it makes the (final) negotiation offer. As = the in-bound unsolicited PDUs can technically arrive any time after the off= er, due to standard network latency mechanics that Alexander summarized. Ha= s that approach been considered? Mallikarjun From: storm-bounces@ietf.org [mailto:storm-bounces@ietf.org] On Behalf Of d= avid.black@emc.com Sent: Thursday, May 24, 2012 12:03 AM To: mkosjc@gmail.com; nezhinsky@gmail.com Cc: ogerlitz@mellanox.com; michaelc@cs.wisc.edu; storm@ietf.org Subject: Re: [storm] iSER - problem with unsolicited NOP-IN right after fin= al Login Response Importance: High Mike (Ko) and Alexander, Mike is of course correct that iSER Hello usage can be forced by negotiatin= g iSERHelloRequired to "Yes". However, existing implementations are likely= to reply with iSERHelloRequired=3DNotUnderstood, so we do need to specify = what should be done in order to interoperate with an implementation that re= fuses to deal with the iSER Hello exchange. I think the situation that Alexander described should be documented in a ne= w section 5.1.4 of the iSER draft. My general rule of thumb on this sort o= f surprise found by implementers in the "running code" is that it indicates= that something is missing in the spec. I believe that Alexander has descr= ibed the solution - more below. The new section 5.1.4 (suggested section title: Omission of the iSER Hello = Exchange) should describe default omission of the exchange, use of iSERHell= oRequired key to omit the iSER Hello exchange, and the consequences of targ= et use of unsolicited PDUs after login when the exchange is omitted, includ= ing IB's use of NOP-IN (as a keep-alive measure, right?) The crucial requirements points that I take away from Alexander's descripti= on are that if the iSER Hello exchange is omitted, then: 1) The target MAY send *one* unsolicited PDU immediately after sending the = Login Response. 2) The target MUST wait at least 200ms (use some other number if 200ms isn'= t a good choice) or until it receives a full feature mode PDU from the initiator before send= ing a second unsolicited PDU in order to ensure that initiator has sufficient time to allocate the full feature buffer resources for the connection= . 3) The initiator SHOULD allocate at least one additional buffer for use dur= ing login (so that at least two buffers are in use during login) in order to receive an unsoli= cited PDU that may follow login completion. Failure to allocate this second buffer m= ay cause connection termination if no buffer is available when an unsolicited = PDU arrives. Both Mike and I are on vacation, so it may be a few weeks until we can agre= e on the new text and get a -12 version of the draft with that new text sub= mitted. In the interim, I've asked our AD (Martin Stiemerling) to hold off= on further processing of the iSER draft until a -12 version with this new = text is submitted. I'd prefer to work this text out now rather than deal w= ith it as an IETF Last Call comment - as the problem turned up in actual im= plementations, I think it's worth the extra month that it's likely to take = to get correct text on how to avoid the problem into the draft. I'd suggest that Mike Ko post an initial draft of the text for the new sect= ion 5.1.4 to the list when he resurfaces ... Thanks, --David From: Michael Ko [mailto:mkosjc@gmail.com] Sent: Monday, May 21, 2012 10:23 AM To: Alexander Nezhinsky Cc: storm@ietf.org; Black, David; Or Gerlitz; Mike C= hristie Subject: Re: iSER - problem with unsolicited NOP-IN right after final Login= Response Alex, The iSER Hello support has never been removed in the latest spec. Only its= use is made optional. So during login negotiation, just negotiate iSERHel= loRequired to Yes. Mike On Mon, May 21, 2012 at 6:15 AM, Alexander Nezhinsky > wrote: Hi I understand that it is a bad timing for sending this kind of mail, now tha= t iSER draft was submitted, but actually we still have a small problem. It is related to the final Login Response handling and the transition to Fu= ll-Featured phase on the initiator side in Infiniband setups. When the target receives the final Login Request it send the final Login Re= sponse and from its perspective the connection is now in Full Featured Phase (assuming that it agreed to tr= ansition in the Login Response being sent). This means that the target is ready to accept SCSI commands, Text Requests = etc. sent by the initiator. It also means that the target is eligible to send some unsolicited PDUs, no= tably unsolicited NOP-INs. With IB sending NOP-IN periodically is the easiest (an almost only feasible= ) way to determine closed connections reliably, because this kind of error is delivered to user only in response = to a previously initiated TX operation. This leaves the initiator in a dubious position. It posts its RX buffers fo= r that connection only when the final Login Response arrives. But during that time (after the target had sent the= Last Login Response but before the Full Featured phase related RX-buffers are posted on the initiator side= ) the target may send the first NOP-IN as it considers the connection in Full Featured phase alre= ady and NumOfUnsolicited PDUs accounting for NOP-INs has been negotiated to a non-zero value. If the initiator works with a single RX-buffer posted during the entire log= in phase (which is a logical thing to do judging by the login exchange protocol) then an error occurs, as no buffers= are posted when the NOP-IN arrives and the connection is shut down. Posting a single extra buffer before sending the last Login Request only al= leviates the problem. Although this often solves it in practical terms (as the target most probably sends the n= ext NOP-IN only after some timeout period measuring seconds or hundreds of milliseconds), it does not solves i= t in terms of protocol completeness, as the target MAY theoretically send more than one NOP-IN until FF buffers = are posted. This issue was encountered recently in linux iscsi/iser initiator and the a= bove solution has been applied to solve it against the existing target implementation (STGT), but the initiator rem= ains exposed to this kind of errors. The solution is actually quite simple (theoretically) - if we bring back th= e requirement for iSER Hello exchange then the iSER assisted Full Featured phase does not commence until HelloRep= ly PDU arrives at the target and the initiator has a definitive point in time when it can safely post it= s RX buffers - after the final LoginResponse returns but before it sends iSER Hello PDU. In practical terms it means that iSER Hello support requirement should be b= rought back to spec, which is a hassle. Should we decide on this now? Alexander P.S. : Thanx to Mike Christie and Or Gerlitz, the maintainers of linux iSCS= I and iSER initiator for raising the issue. --_000_E160851FCED17643AE5F53B5D4D0783A092D0294BL2PRD0610MB361_ Content-Type: text/html; charset="us-ascii" Content-Transfer-Encoding: quoted-printable

I am curious to understan= d a bit more on why this is a protocol issue per se.

 <= /p>

Seems like one way to add= ress this problem is via an implementation approach with the initiator post= ing in advance the negotiated number of unsolicited PDU buffers, at the same time it makes the (final) negotiation offer. As the i= n-bound unsolicited PDUs can technically arrive any time after the offer, d= ue to standard network latency mechanics that Alexander summarized. Has tha= t approach been considered?

 <= /p>

Mallikarjun

 <= /p>

 <= /p>

 <= /p>

 <= /p>

From: storm-bo= unces@ietf.org [mailto:storm-bounces@ietf.org] On Behalf Of david.black@emc.com
Sent: Thursday, May 24, 2012 12:03 AM
To: mkosjc@gmail.com; nezhinsky@gmail.com
Cc: ogerlitz@mellanox.com; michaelc@cs.wisc.edu; storm@ietf.org
Subject: Re: [storm] iSER - problem with unsolicited NOP-IN right af= ter final Login Response
Importance: High

 

Mike (Ko) and Alexander,

 

Mike is of course correct that iSER Hello usag= e can be forced by negotiating iSERHelloRequired to “Yes”. = ; However, existing implementations are likely to reply with iSERHelloRequi= red=3DNotUnderstood, so we do need to specify what should be done in order to interoperate with= an implementation that refuses to deal with the iSER Hello exchange.<= /o:p>

 

I think the situation that Alexander described= should be documented in a new section 5.1.4 of the iSER draft.  My ge= neral rule of thumb on this sort of surprise found by implementers in the “running code” is that it indicates tha= t something is missing in the spec.  I believe that Alexander has desc= ribed the solution - more below.

 

The new section 5.1.4 (suggested section title= : Omission of the iSER Hello Exchange) should describe default omission of = the exchange, use of iSERHelloRequired key to omit the iSER Hello exchange, and the consequences of target use of unsoli= cited PDUs after login when the exchange is omitted, including IB’s u= se of NOP-IN (as a keep-alive measure, right?)

 

The crucial requirements points that I take aw= ay from Alexander’s description are that if the iSER Hello exchange i= s omitted, then:

 

1) The target MAY send *one* unsolicite= d PDU immediately after sending the Login Response.

 

2) The target MUST wait at least 200ms (use so= me other number if 200ms isn’t a good choice)

or until it receive= s a full feature mode PDU from the initiator before sending a

second unsolicited = PDU in order to ensure that initiator has sufficient

      time to allocat= e the full feature buffer resources for the connection.

3) The initiator SHOULD allocate at least one = additional buffer for use during login (so that

at least two buffer= s are in use during login) in order to receive an unsolicited PDU

that may follow log= in completion.  Failure to allocate this second buffer may<= /span>

cause connection te= rmination if no buffer is available when an unsolicited PDU arrives.

 

Both Mike and I are on vacation, so it may be = a few weeks until we can agree on the new text and get a -12 version of the= draft with that new text submitted.  In the interim, I’ve asked our AD (Martin Stiemerling) to hold off on further proces= sing of the iSER draft until a -12 version with this new text is submitted.=   I’d prefer to work this text out now rather than deal with it = as an IETF Last Call comment - as the problem turned up in actual implementations, I think it’s worth the extra month tha= t it’s likely to take to get correct text on how to avoid the problem= into the draft.

 

I’d suggest that Mike Ko post an initial= draft of the text for the new section 5.1.4 to the list when he resurfaces= ...

 

Thanks,
--David

 

From: Michael = Ko [mailto:mkosjc@gmail.com] =
Sent: Monday, May 21, 2012 10:23 AM
To: Alexander Nezhinsky
Cc: storm@ietf.org; Black, Dav= id; Or Gerlitz; Mike Christie
Subject: Re: iSER - problem with unsolicited NOP-IN right after fina= l Login Response

 

Alex,

The iSER Hello support has never been removed in the latest spec.  Onl= y its use is made optional.  So during login negotiation, just negotia= te iSERHelloRequired to Yes.

Mike

On Mon, May 21, 2012 at 6:15 AM, Alexander Nezhinsky= <nezhinsky@gma= il.com> wrote:

Hi

I understand that it is a bad timing for sending this kind of mail, now tha= t iSER draft was submitted,
but actually we still have a small problem.

It is related to the&= nbsp;final Login Response handling and the transition to Full-Featured phas= e on the initiator side in
Infiniband setups.

When the target receives the final Login Request it send the= final Login Response and from its perspective
the connection is now in Full Featured Phase (assuming that it agreed to tr= ansition in the Login Response being sent).

This means that the target is ready to accept SCSI commands, Text Requests = etc. sent by the initiator.
It also means that the target is eligible to send some unsolicited PDUs,&nb= sp;notably unsolicited NOP-INs.

With IB sending NOP-IN periodically is the easiest (an almost only feasible= ) way to determine closed connections
reliably, because this kind of error is delivered to user only in response = to a previously initiated TX operation.

This leaves the initiator in a dubious position. It posts its RX buffers fo= r that connection only when the final
Login Response arrives. But during that time (after the target had sent the= Last Login Response but before
the Full Featured phase related RX-buffers are posted on the initiator side= ) the target may send
the first NOP-IN as it considers the connection in Full Featured phase alre= ady and NumOfUnsolicited PDUs
accounting for NOP-INs has been negotiated to a non-zero value.

If the initiator works with a single RX-buffer posted during the entire log= in phase (which is a logical thing to do
judging by the login exchange protocol) then an error occurs, as no buffers= are posted when the NOP-IN arrives
and the connection is shut down.

Posting a single extra buffer before sending the last Login Request only al= leviates the problem. Although this
often solves it in practical terms (as the target most probably sends the n= ext NOP-IN only after some timeout
period measuring seconds or hundreds of milliseconds), it does not solves i= t in terms of protocol completeness,
as the target MAY theoretically send more than one NOP-IN until FF buffers = are posted.

This issue was encountered recently in linux iscsi/iser initiator and the a= bove solution has been applied to solve
it against the existing target implementation (STGT), but the initiator rem= ains exposed to this kind of errors.

The solution is actually quite simple (theoretically) - if we bring back th= e requirement for iSER Hello exchange
then the iSER assisted Full Featured phase does not commence until HelloRep= ly PDU arrives at the target
and the initiator has a definitive point in time when it can safely post it= s RX buffers - after the final LoginResponse
returns but before it sends iSER Hello PDU.

In practical terms it means that iSER Hello support requirement should be b= rought back to spec, which is a hassle.

Should we decide on this now?

Alexander

P.S. : Thanx to Mike Christie and Or Gerlitz, the maintainers of linux iSCS= I and iSER initiator for raising the issue.

 

--_000_E160851FCED17643AE5F53B5D4D0783A092D0294BL2PRD0610MB361_--