From david.black@emc.com Mon Jan 2 14:09:25 2012 Return-Path: X-Original-To: storm@ietfa.amsl.com Delivered-To: storm@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id EF70F21F84B9 for ; Mon, 2 Jan 2012 14:09:25 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -106.586 X-Spam-Level: X-Spam-Status: No, score=-106.586 tagged_above=-999 required=5 tests=[AWL=0.013, BAYES_00=-2.599, RCVD_IN_DNSWL_MED=-4, USER_IN_WHITELIST=-100] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id UHEbgITf4Mai for ; Mon, 2 Jan 2012 14:09:25 -0800 (PST) Received: from mexforward.lss.emc.com (mexforward.lss.emc.com [128.222.32.20]) by ietfa.amsl.com (Postfix) with ESMTP id F33F521F84A7 for ; Mon, 2 Jan 2012 14:09:24 -0800 (PST) Received: from hop04-l1d11-si01.isus.emc.com (HOP04-L1D11-SI01.isus.emc.com [10.254.111.54]) by mexforward.lss.emc.com (Switch-3.4.3/Switch-3.4.3) with ESMTP id q02M9KxN011725 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO) for ; Mon, 2 Jan 2012 17:09:21 -0500 Received: from mailhub.lss.emc.com (mailhub.lss.emc.com [10.254.221.145]) by hop04-l1d11-si01.isus.emc.com (RSA Interceptor) for ; Mon, 2 Jan 2012 17:09:10 -0500 Received: from mxhub20.corp.emc.com (mxhub20.corp.emc.com [10.254.93.49]) by mailhub.lss.emc.com (Switch-3.4.3/Switch-3.4.3) with ESMTP id q02M98uo024645 for ; Mon, 2 Jan 2012 17:09:10 -0500 Received: from mx14a.corp.emc.com ([169.254.1.216]) by mxhub20.corp.emc.com ([10.254.93.49]) with mapi; Mon, 2 Jan 2012 17:09:08 -0500 From: To: Date: Mon, 2 Jan 2012 17:09:07 -0500 Thread-Topic: iSCSI drafts - last minute hiccup Thread-Index: AczJmyXGqkSlO2AtSXS10m6kJ5RXug== Message-ID: <7C4DFCE962635144B8FAE8CA11D0BF1E05A5CC9BB7@MX14A.corp.emc.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: acceptlanguage: en-US Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-EMM-MHVC: 1 Subject: [storm] iSCSI drafts - last minute hiccup X-BeenThere: storm@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Storage Maintenance WG List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 02 Jan 2012 22:09:26 -0000 I was about to submit the RFC Publication Requests to the IESG for both the= consolidated iSCSI draft and the iSCSI new features (SAM) draft, when idni= ts turned up some problems with uncited references in the consolidated iSCS= I draft. A new version of the iSCSI draft will be required that corrects these probl= ems, and should appear in the near future after which the publication reque= sts for both drafts will be submitted promptly. FYI, --David ---------------------------------------------------- David L. Black, Distinguished Engineer EMC Corporation, 176 South St., Hopkinton, MA=A0 01748 +1 (508) 293-7953=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 FAX: +1 (508) 293-778= 6 david.black@emc.com=A0=A0=A0=A0=A0=A0=A0 Mobile: +1 (978) 394-7754 ---------------------------------------------------- From david.black@emc.com Thu Jan 5 13:00:31 2012 Return-Path: X-Original-To: storm@ietfa.amsl.com Delivered-To: storm@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 98F9821F889D for ; Thu, 5 Jan 2012 13:00:31 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -106.591 X-Spam-Level: X-Spam-Status: No, score=-106.591 tagged_above=-999 required=5 tests=[AWL=0.008, BAYES_00=-2.599, RCVD_IN_DNSWL_MED=-4, USER_IN_WHITELIST=-100] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id DYlj-a-DD4uP for ; Thu, 5 Jan 2012 13:00:31 -0800 (PST) Received: from mexforward.lss.emc.com (mexforward.lss.emc.com [128.222.32.20]) by ietfa.amsl.com (Postfix) with ESMTP id E3D2C21F888F for ; Thu, 5 Jan 2012 13:00:30 -0800 (PST) Received: from hop04-l1d11-si03.isus.emc.com (HOP04-L1D11-SI03.isus.emc.com [10.254.111.23]) by mexforward.lss.emc.com (Switch-3.4.3/Switch-3.4.3) with ESMTP id q05L0Qjg016586 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO) for ; Thu, 5 Jan 2012 16:00:27 -0500 Received: from mailhub.lss.emc.com (mailhub.lss.emc.com [10.254.221.251]) by hop04-l1d11-si03.isus.emc.com (RSA Interceptor) for ; Thu, 5 Jan 2012 16:00:21 -0500 Received: from mxhub05.corp.emc.com (mxhub05.corp.emc.com [128.222.70.202]) by mailhub.lss.emc.com (Switch-3.4.3/Switch-3.4.3) with ESMTP id q05L0IdC032196 for ; Thu, 5 Jan 2012 16:00:18 -0500 Received: from mx14a.corp.emc.com ([169.254.1.216]) by mxhub05.corp.emc.com ([128.222.70.202]) with mapi; Thu, 5 Jan 2012 16:00:17 -0500 From: To: Importance: high X-Priority: 1 Date: Thu, 5 Jan 2012 16:00:16 -0500 Thread-Topic: RDDP Registries Draft: Experimental values & RDMAP opcodes Thread-Index: AczL7QaS6DcqU/89QrWJkKQ28N+CYA== Message-ID: <7C4DFCE962635144B8FAE8CA11D0BF1E05A5E7C342@MX14A.corp.emc.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: acceptlanguage: en-US Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-EMM-MHVC: 1 Subject: [storm] RDDP Registries Draft: Experimental values & RDMAP opcodes X-BeenThere: storm@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Storage Maintenance WG List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 05 Jan 2012 21:00:31 -0000 IESG evaluation of the RDDP registries draft (draft-ietf-storm-rddp-registr= ies-01.txt) has resulted in a need for a new version primarily to address some editoria= l issues. There is also a suggestion to set aside an experimentation/testing value fo= r RDMAP operation codes (0xF) and SCTP Function Codes for DDP Session Control (0xFF= FF). I wanted to make this suggestion visible for WG comment before making this change to= the draft. IMHO (WG chair hat off), setting aside an SCTP Function code for DDP Sessio= n Control for experimentation and testing seems like a fine idea. OTOH, I'm concerned about overall usage of RDMAP opcodes, and want to discu= ss how to plan for the future in this space. The current situation is that the RDMAP = opcode is a 4-bit field (16 possible codes. RDMAP currently uses 8 opcodes, the RDMAP extension draft proposes to use 4= more for immediate data and atomic operations, so setting aside an experimentation/t= esting value would use 13 of the 16 available opcodes. I'm a bit concerned that there w= ould only be three opcodes remaining. The questions to the WG are: - Is anyone else concerned? - If so, what should we do about this? I observer that there are 2 unused (reserved) bits adjacent to the MSB of R= DMAP opcode field that may provide some opportunity for expansion. An example of what'= s possible is to plan to rework the RDMAP extensions draft to use only 1 or 2 opcodes= (instead of the current 4 opcodes) by putting Request/Response and possibly Atomic/I= mmediate into those two bits (that'd leave 5 or 6 available opcodes and set the prec= edent that those 2 bits can be used as an op-sub-code). OTOH, going all the way to ex= panding the RDMAP opcode field to 6 bits probably involves bumping the RV (RDMA Version= ) field to 2, which does not seem like a good idea until we actually run out of opcode= s, as we'd probably have to design a version negotiation mechanism. Comments? What do other people think? Thanks, --David ---------------------------------------------------- David L. Black, Distinguished Engineer EMC Corporation, 176 South St., Hopkinton, MA=A0 01748 +1 (508) 293-7953=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 FAX: +1 (508) 293-778= 6 david.black@emc.com=A0=A0=A0=A0=A0=A0=A0 Mobile: +1 (978) 394-7754 ---------------------------------------------------- From internet-drafts@ietf.org Mon Jan 9 17:44:26 2012 Return-Path: X-Original-To: storm@ietfa.amsl.com Delivered-To: storm@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id E5ADC21F844B; Mon, 9 Jan 2012 17:44:25 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -102.566 X-Spam-Level: X-Spam-Status: No, score=-102.566 tagged_above=-999 required=5 tests=[AWL=0.033, BAYES_00=-2.599, USER_IN_WHITELIST=-100] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id zcoBii9m1+eg; Mon, 9 Jan 2012 17:44:25 -0800 (PST) Received: from ietfa.amsl.com (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id D97C411E8080; Mon, 9 Jan 2012 17:44:15 -0800 (PST) MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable From: internet-drafts@ietf.org To: i-d-announce@ietf.org X-Test-IDTracker: no X-IETF-IDTracker: 3.64p1 Message-ID: <20120110014415.32585.82495.idtracker@ietfa.amsl.com> Date: Mon, 09 Jan 2012 17:44:15 -0800 Cc: storm@ietf.org Subject: [storm] I-D Action: draft-ietf-storm-rdmap-ext-02.txt X-BeenThere: storm@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Storage Maintenance WG List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 10 Jan 2012 01:44:26 -0000 A New Internet-Draft is available from the on-line Internet-Drafts director= ies. This draft is a work item of the STORage Maintenance Working Group of = the IETF. Title : RDMA Protocol Extensions Author(s) : Hemal Shah Felix Marti Wael Noureddine Asgeir Eiriksson Robert Sharp Filename : draft-ietf-storm-rdmap-ext-02.txt Pages : 31 Date : 2012-01-09 This document specifies extensions to the IETF Remote Direct Memory Access Protocol (RDMAP [RFC5040]). RDMAP provides read and write services directly to applications and enables data to be transferred directly into Upper Layer Protocol (ULP) Buffers without intermediate data copies. The extensions specified in this document provide the following capabilities and/or improvements: Atomic Operations and Immediate Data. A URL for this Internet-Draft is: http://www.ietf.org/internet-drafts/draft-ietf-storm-rdmap-ext-02.txt Internet-Drafts are also available by anonymous FTP at: ftp://ftp.ietf.org/internet-drafts/ This Internet-Draft can be retrieved at: ftp://ftp.ietf.org/internet-drafts/draft-ietf-storm-rdmap-ext-02.txt From Internet-Drafts@ietf.org Tue Jan 10 07:45:02 2012 Return-Path: X-Original-To: storm@ietfa.amsl.com Delivered-To: storm@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 0262521F8717; Tue, 10 Jan 2012 07:45:02 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -102.557 X-Spam-Level: X-Spam-Status: No, score=-102.557 tagged_above=-999 required=5 tests=[AWL=0.042, BAYES_00=-2.599, USER_IN_WHITELIST=-100] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id lM2gkq1nZ5Li; Tue, 10 Jan 2012 07:45:01 -0800 (PST) Received: from ietfa.amsl.com (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 79A4C21F85F8; Tue, 10 Jan 2012 07:45:01 -0800 (PST) MIME-Version: 1.0 Content-Type: Multipart/Mixed; Boundary="NextPart" From: Internet-Drafts@ietf.org To: i-d-announce@ietf.org X-Test-IDTracker: no X-IETF-IDTracker: 3.64p1 Message-ID: <20120110154501.10708.66244.idtracker@ietfa.amsl.com> Date: Tue, 10 Jan 2012 07:45:01 -0800 Cc: storm@ietf.org Subject: [storm] I-D ACTION:draft-ietf-storm-iscsi-cons-05.txt X-BeenThere: storm@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Storage Maintenance WG List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 10 Jan 2012 15:45:02 -0000 --NextPart A new Internet-Draft is available from the on-line Internet-Drafts directories. This draft is a work item of the STORage Maintenance Working Group of the IETF. Title : iSCSI Protocol (Consolidated) Author(s) : M. Chadalapaka, et al Filename : draft-ietf-storm-iscsi-cons-05.txt Pages : 341 Date : 2012-01-10 This document describes a transport protocol for SCSI that works on top of TCP. The iSCSI protocol aims to be fully compliant with the standardized SCSI Architecture Model (SAM-2). RFC 3720 defined the original iSCSI protocol. RFC 3721 discusses iSCSI Naming examples and discovery techniques. Subsequently, RFC 3980 added an additional naming format to iSCSI protocol. RFC 4850 followed up by adding a new public extension key to iSCSI. RFC 5048 offered a number of clarifications and a few improvements and corrections to the original iSCSI protocol. This document obsoletes RFCs 3720, 3980, 4850 and 5048 by consolidating them into a single document and making additional updates to the consolidated specification. This document also updates RFC 3721 and RFC 3723. The text in this document thus supersedes the text in all the noted RFCs wherever there is a difference in semantics. A URL for this Internet-Draft is: http://www.ietf.org/internet-drafts/draft-ietf-storm-iscsi-cons-05.txt Internet-Drafts are also available by anonymous FTP at: ftp://ftp.ietf.org/internet-drafts/ Below is the data which will enable a MIME compliant mail reader implementation to automatically retrieve the ASCII version of the Internet-Draft. --NextPart Content-Type: Message/External-body; name="draft-ietf-storm-iscsi-cons-05.txt"; site="ftp.ietf.org"; access-type="anon-ftp"; directory="internet-drafts" Content-Type: text/plain Content-ID: <2012-01-10073223.I-D@ietf.org> --NextPart-- From david.black@emc.com Tue Jan 10 17:54:39 2012 Return-Path: X-Original-To: storm@ietfa.amsl.com Delivered-To: storm@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id D5B8611E80BE for ; Tue, 10 Jan 2012 17:54:39 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -106.592 X-Spam-Level: X-Spam-Status: No, score=-106.592 tagged_above=-999 required=5 tests=[AWL=0.007, BAYES_00=-2.599, RCVD_IN_DNSWL_MED=-4, USER_IN_WHITELIST=-100] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id XtwxyAIbE60h for ; Tue, 10 Jan 2012 17:54:39 -0800 (PST) Received: from mexforward.lss.emc.com (mexforward.lss.emc.com [128.222.32.20]) by ietfa.amsl.com (Postfix) with ESMTP id 3C3C011E80BD for ; Tue, 10 Jan 2012 17:54:38 -0800 (PST) Received: from hop04-l1d11-si04.isus.emc.com (HOP04-L1D11-SI04.isus.emc.com [10.254.111.24]) by mexforward.lss.emc.com (Switch-3.4.3/Switch-3.4.3) with ESMTP id q0B1sc9m031650 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO) for ; Tue, 10 Jan 2012 20:54:38 -0500 Received: from mailhub.lss.emc.com (mailhub.lss.emc.com [10.254.222.130]) by hop04-l1d11-si04.isus.emc.com (RSA Interceptor) for ; Tue, 10 Jan 2012 20:54:28 -0500 Received: from mxhub14.corp.emc.com (mxhub14.corp.emc.com [128.222.70.235]) by mailhub.lss.emc.com (Switch-3.4.3/Switch-3.4.3) with ESMTP id q0B1sRSK030313 for ; Tue, 10 Jan 2012 20:54:28 -0500 Received: from mx14a.corp.emc.com ([169.254.1.99]) by mxhub14.corp.emc.com ([128.222.70.235]) with mapi; Tue, 10 Jan 2012 20:54:28 -0500 From: To: Importance: high X-Priority: 1 Date: Tue, 10 Jan 2012 20:54:26 -0500 Thread-Topic: RDMAP opcodes - proposal Thread-Index: AczQA/KxuaL1Lor0Q2y6Gk0CrCJm4Q== Message-ID: <7C4DFCE962635144B8FAE8CA11D0BF1E05A7B80D5B@MX14A.corp.emc.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: acceptlanguage: en-US Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-EMM-MHVC: 1 Subject: [storm] RDMAP opcodes - proposal X-BeenThere: storm@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Storage Maintenance WG List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 11 Jan 2012 01:54:40 -0000 After thinking about the RDMAP opcode situation over the weekend, I have a = proposed course of action. Please comment, as I would like to move on this= fairly quickly in order to get both the MPA Peer Connect and RDMAP registr= ies drafts approved by the IESG. There are 8 currently unused RDMAP opcodes. The RDMAP extensions draft wil= l use 4 of them, and setting aside 1 more for test/experimentation leaves 3= . I'm not comfortable committing to a course of action that consumes more = than half of the unused opcodes without taking a good hard look at the alte= rnatives and consequences. Hence I propose to do the following: (1) Defer allocation of text/experimentation RDMAP opcode to RDMAP extensio= ns draft, which is already going to allocate RDMAP opcodes. (2) As part of discussion of the RDMAP extensions draft, consider the 2 res= erved (and currently unused) bits that are adjacent to the RDMAP opcode fie= ld. Although any use of those bits requires modifying RFC 5040 (RDMAP), th= at's a reasonable thing for the RDMAP extensions draft to do *if* it won't = cause serious implementation problems. (3) As part of the discussion of the reserved bits, the following questions= should be addressed: - Does the RDMAP extensions draft need 4 opcodes? For example, would 2 opc= odes plus use of one of the reserved bits as a direction bit (Request/Response) for those two opcodes be a better approach? - Should 4 or 8 of the currently unused 4-bit RDMAP opcodes be turned into = 6-bit opcodes by including the two reserved bits? Implementation concerns matter to this discussion, as they may constrain pr= actical use of those 2 reserved bits. By deferring allocation of the test/= experimentation opcode, we keep our options open for this discussion. Does this sound like a reasonable plan? Thanks, --David ---------------------------------------------------- David L. Black, Distinguished Engineer EMC Corporation, 176 South St., Hopkinton, MA=A0 01748 +1 (508) 293-7953=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 FAX: +1 (508) 293-778= 6 david.black@emc.com=A0=A0=A0=A0=A0=A0=A0 Mobile: +1 (978) 394-7754 ---------------------------------------------------- From david.black@emc.com Thu Jan 12 12:12:21 2012 Return-Path: X-Original-To: storm@ietfa.amsl.com Delivered-To: storm@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 9A35F21F865F for ; Thu, 12 Jan 2012 12:12:21 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -106.593 X-Spam-Level: X-Spam-Status: No, score=-106.593 tagged_above=-999 required=5 tests=[AWL=0.006, BAYES_00=-2.599, RCVD_IN_DNSWL_MED=-4, USER_IN_WHITELIST=-100] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id YIqfVPa7Mrvf for ; Thu, 12 Jan 2012 12:12:20 -0800 (PST) Received: from mexforward.lss.emc.com (mexforward.lss.emc.com [128.222.32.20]) by ietfa.amsl.com (Postfix) with ESMTP id D682821F8537 for ; Thu, 12 Jan 2012 12:12:19 -0800 (PST) Received: from hop04-l1d11-si01.isus.emc.com (HOP04-L1D11-SI01.isus.emc.com [10.254.111.54]) by mexforward.lss.emc.com (Switch-3.4.3/Switch-3.4.3) with ESMTP id q0CKCCBm002427 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO) for ; Thu, 12 Jan 2012 15:12:18 -0500 Received: from mailhub.lss.emc.com (mailhubhoprd01.lss.emc.com [10.254.221.251]) by hop04-l1d11-si01.isus.emc.com (RSA Interceptor) for ; Thu, 12 Jan 2012 15:12:01 -0500 Received: from mxhub13.corp.emc.com (mxhub13.corp.emc.com [128.222.70.234]) by mailhub.lss.emc.com (Switch-3.4.3/Switch-3.4.3) with ESMTP id q0CKBxlp026824 for ; Thu, 12 Jan 2012 15:11:59 -0500 Received: from mx14a.corp.emc.com ([169.254.1.99]) by mxhub13.corp.emc.com ([128.222.70.234]) with mapi; Thu, 12 Jan 2012 15:11:58 -0500 From: To: Importance: high X-Priority: 1 Date: Thu, 12 Jan 2012 15:11:56 -0500 Thread-Topic: RFC publication requests: iSCSI drafts (finally!) Thread-Index: AczRZm8gjmtIbBKRTbeJSo/HS25EKA== Message-ID: <7C4DFCE962635144B8FAE8CA11D0BF1E05A7B81346@MX14A.corp.emc.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: yes X-MS-TNEF-Correlator: acceptlanguage: en-US Content-Type: multipart/mixed; boundary="_003_7C4DFCE962635144B8FAE8CA11D0BF1E05A7B81346MX14Acorpemcc_" MIME-Version: 1.0 X-EMM-MHVC: 1 Subject: [storm] RFC publication requests: iSCSI drafts (finally!) X-BeenThere: storm@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Storage Maintenance WG List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 12 Jan 2012 20:12:21 -0000 --_003_7C4DFCE962635144B8FAE8CA11D0BF1E05A7B81346MX14Acorpemcc_ Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable Everyone, It's taken a while, but I've just submitted the RFC publication requests fo= r the two iSCSI drafts, the consolidated draft and the SAM (new features) d= raft. The PROTO writeups for both drafts are attached. There are a lot of people to thank for helping us get to this point, includ= ing the draft editors (Mallikarjun and Fred), the external reviewers from t= he implementation groups, and everyone in the WG who contributed to these d= rafts. This is a significant milestone - the journey's not over, but congratulatio= ns to the WG and thank you to everyone who helped us get this far! --David ---------------------------------------------------- David L. Black, Distinguished Engineer EMC Corporation, 176 South St., Hopkinton, MA=A0 01748 +1 (508) 293-7953=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 FAX: +1 (508) 293-778= 6 david.black@emc.com=A0=A0=A0=A0=A0=A0=A0 Mobile: +1 (978) 394-7754 ---------------------------------------------------- --_003_7C4DFCE962635144B8FAE8CA11D0BF1E05A7B81346MX14Acorpemcc_ Content-Type: text/plain; name="iSCSI SAM Proto Writeup.txt" Content-Description: iSCSI SAM Proto Writeup.txt Content-Disposition: attachment; filename="iSCSI SAM Proto Writeup.txt"; size=8161; creation-date="Wed, 28 Dec 2011 15:51:00 GMT"; modification-date="Thu, 12 Jan 2012 18:39:57 GMT" Content-Transfer-Encoding: base64 UFJPVE8gd3JpdGV1cDoNCg0KICBJbnRlcm5ldCBTbWFsbCBDb21wdXRlciBTeXN0ZW1zIEludGVy ZmFjZSAoaVNDU0kpIFNDU0kgRmVhdHVyZXMNCiAgICAgICAgICAgICAgICAgICAgICAgICAgICAg ICBVcGRhdGUNCiAgICAgICAgICAgICAgICAgZHJhZnQtaWV0Zi1zdG9ybS1pc2NzaS1zYW0tMDUu dHh0DQoNClJlcXVlc3RlZCBQdWJsaWNhdGlvbiBTdGF0dXM6IFByb3Bvc2VkIFN0YW5kYXJkDQpQ Uk9UTyBzaGVwaGVyZDogRGF2aWQgTC4gQmxhY2sgKFNUT1JNIFdHIENvLUNoYWlyKQ0KLS0tLS0t LS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0t LS0tLS0tLS0tDQoNCiAgKDEuYSkgV2hvIGlzIHRoZSBEb2N1bWVudCBTaGVwaGVyZCBmb3IgdGhp cyBkb2N1bWVudD8gSGFzIHRoZQ0KICAgICAgICBEb2N1bWVudCBTaGVwaGVyZCBwZXJzb25hbGx5 IHJldmlld2VkIHRoaXMgdmVyc2lvbiBvZiB0aGUgDQogICAgICAgIGRvY3VtZW50IGFuZCwgaW4g cGFydGljdWxhciwgZG9lcyBoZSBvciBzaGUgYmVsaWV2ZSB0aGlzIA0KICAgICAgICB2ZXJzaW9u IGlzIHJlYWR5IGZvciBmb3J3YXJkaW5nIHRvIHRoZSBJRVNHIGZvciBwdWJsaWNhdGlvbj8NCg0K RGF2aWQgTC4gQmxhY2sgKGRhdmlkLmJsYWNrQGVtYy5jb20pIGlzIHRoZSBEb2N1bWVudCBTaGVw aGVyZC4gIFRoZQ0KRG9jdW1lbnQgU2hlcGhlcmQgaGFzIHJldmlld2VkIHRoaXMgdmVyc2lvbiBv ZiB0aGUgZG9jdW1lbnQNCmFuZCBiZWxpZXZlcyB0aGF0IGl0IGlzIHJlYWR5IGZvciBmb3J3YXJk aW5nIHRvIHRoZSBJRVNHIGZvciBwdWJsaWNhdGlvbi4NCg0KICAoMS5iKSBIYXMgdGhlIGRvY3Vt ZW50IGhhZCBhZGVxdWF0ZSByZXZpZXcgYm90aCBmcm9tIGtleSBXRyBtZW1iZXJzIA0KICAgICAg ICBhbmQgZnJvbSBrZXkgbm9uLVdHIG1lbWJlcnM/IERvZXMgdGhlIERvY3VtZW50IFNoZXBoZXJk IGhhdmUgDQogICAgICAgIGFueSBjb25jZXJucyBhYm91dCB0aGUgZGVwdGggb3IgYnJlYWR0aCBv ZiB0aGUgcmV2aWV3cyB0aGF0IA0KICAgICAgICBoYXZlIGJlZW4gcGVyZm9ybWVkPw0KDQpUaGUg ZG9jdW1lbnQgaGFzIGhhZCBzdWZmaWNpZW50IHJldmlldyBmcm9tIGtleSBXRyBtZW1iZXJzLCBm cm9tIGltcGxlbWVudGVycw0Kd2hvIHdvcmsgb24gaW1wb3J0YW50IGlTQ1NJIGltcGxlbWVudGF0 aW9ucyAoYm90aCBpbml0aWF0b3IgYW5kIHRhcmdldA0KaW1wbGVtZW50YXRpb25zKSBvdXRzaWRl IHRoZSBXRyBhbmQgZnJvbSBrZXkgbWVtYmVycyBvZiBJTkNJVFMgVGVjaG5pY2FsDQpDb21taXR0 ZWUgVDEwLCB0aGUgb3JnYW5pemF0aW9uIHJlc3BvbnNpYmxlIGZvciBTQ1NJIHN0YW5kYXJkcy4N Cg0KICAoMS5jKSBEb2VzIHRoZSBEb2N1bWVudCBTaGVwaGVyZCBoYXZlIGNvbmNlcm5zIHRoYXQg dGhlIGRvY3VtZW50IA0KICAgICAgICBuZWVkcyBtb3JlIHJldmlldyBmcm9tIGEgcGFydGljdWxh ciBvciBicm9hZGVyIHBlcnNwZWN0aXZlLCANCiAgICAgICAgZS5nLiwgc2VjdXJpdHksIG9wZXJh dGlvbmFsIGNvbXBsZXhpdHksIHNvbWVvbmUgZmFtaWxpYXIgd2l0aCANCiAgICAgICAgQUFBLCBp bnRlcm5hdGlvbmFsaXphdGlvbiBvciBYTUw/DQoNCk5vLg0KDQogICgxLmQpIERvZXMgdGhlIERv Y3VtZW50IFNoZXBoZXJkIGhhdmUgYW55IHNwZWNpZmljIGNvbmNlcm5zIG9yIA0KICAgICAgICBp c3N1ZXMgd2l0aCB0aGlzIGRvY3VtZW50IHRoYXQgdGhlIFJlc3BvbnNpYmxlIEFyZWEgRGlyZWN0 b3INCiAgICAgICAgYW5kL29yIHRoZSBJRVNHIHNob3VsZCBiZSBhd2FyZSBvZj8gRm9yIGV4YW1w bGUsIHBlcmhhcHMgaGUgDQogICAgICAgIG9yIHNoZSBpcyB1bmNvbWZvcnRhYmxlIHdpdGggY2Vy dGFpbiBwYXJ0cyBvZiB0aGUgZG9jdW1lbnQsIG9yIA0KICAgICAgICBoYXMgY29uY2VybnMgd2hl dGhlciB0aGVyZSByZWFsbHkgaXMgYSBuZWVkIGZvciBpdC4gSW4gYW55IA0KICAgICAgICBldmVu dCwgaWYgdGhlIFdHIGhhcyBkaXNjdXNzZWQgdGhvc2UgaXNzdWVzIGFuZCBoYXMgaW5kaWNhdGVk IA0KICAgICAgICB0aGF0IGl0IHN0aWxsIHdpc2hlcyB0byBhZHZhbmNlIHRoZSBkb2N1bWVudCwg ZGV0YWlsIHRob3NlIA0KICAgICAgICBjb25jZXJucyBoZXJlLiBIYXMgYW4gSVBSIGRpc2Nsb3N1 cmUgcmVsYXRlZCB0byB0aGlzIGRvY3VtZW50IA0KICAgICAgICBiZWVuIGZpbGVkPyBJZiBzbywg cGxlYXNlIGluY2x1ZGUgYSByZWZlcmVuY2UgdG8gdGhlIA0KICAgICAgICBkaXNjbG9zdXJlIGFu ZCBzdW1tYXJpemUgdGhlIFdHIGRpc2N1c3Npb24gYW5kIGNvbmNsdXNpb24gb24gDQogICAgICAg IHRoaXMgaXNzdWUuDQoNCk5vLg0KDQogICgxLmUpIEhvdyBzb2xpZCBpcyB0aGUgV0cgY29uc2Vu c3VzIGJlaGluZCB0aGlzIGRvY3VtZW50PyBEb2VzIGl0IA0KICAgICAgICByZXByZXNlbnQgdGhl IHN0cm9uZyBjb25jdXJyZW5jZSBvZiBhIGZldyBpbmRpdmlkdWFscywgd2l0aCANCiAgICAgICAg b3RoZXJzIGJlaW5nIHNpbGVudCwgb3IgZG9lcyB0aGUgV0cgYXMgYSB3aG9sZSB1bmRlcnN0YW5k IGFuZCANCiAgICAgICAgYWdyZWUgd2l0aCBpdD8gICANCg0KVGhlIFdHIGNvbnNlbnN1cyBiZWhp bmQgdGhpcyBkb2N1bWVudCBpcyBzb2xpZDsgdGhlIFdHIGFzIGEgd2hvbGUNCnVuZGVyc3RhbmRz IHRoaXMgZG9jdW1lbnQgYW5kIGFncmVlcyB3aXRoIHRoZSBuZWVkIGZvciB0aGVzZSBtaW5vcg0K dXBkYXRlcy4NCg0KICAoMS5mKSBIYXMgYW55b25lIHRocmVhdGVuZWQgYW4gYXBwZWFsIG9yIG90 aGVyd2lzZSBpbmRpY2F0ZWQgZXh0cmVtZSANCiAgICAgICAgZGlzY29udGVudD8gSWYgc28sIHBs ZWFzZSBzdW1tYXJpc2UgdGhlIGFyZWFzIG9mIGNvbmZsaWN0IGluIA0KICAgICAgICBzZXBhcmF0 ZSBlbWFpbCBtZXNzYWdlcyB0byB0aGUgUmVzcG9uc2libGUgQXJlYSBEaXJlY3Rvci4gKEl0IA0K ICAgICAgICBzaG91bGQgYmUgaW4gYSBzZXBhcmF0ZSBlbWFpbCBiZWNhdXNlIHRoaXMgcXVlc3Rp b25uYWlyZSBpcyANCiAgICAgICAgZW50ZXJlZCBpbnRvIHRoZSBJRCBUcmFja2VyLikNCg0KTm8u DQoNCiAgKDEuZykgSGFzIHRoZSBEb2N1bWVudCBTaGVwaGVyZCBwZXJzb25hbGx5IHZlcmlmaWVk IHRoYXQgdGhlIA0KICAgICAgICBkb2N1bWVudCBzYXRpc2ZpZXMgYWxsIElEIG5pdHM/IChTZWUg dGhlIEludGVybmV0LURyYWZ0cyBDaGVja2xpc3QgDQogICAgICAgIGFuZCBodHRwOi8vdG9vbHMu aWV0Zi5vcmcvdG9vbHMvaWRuaXRzLykuIEJvaWxlcnBsYXRlIGNoZWNrcyBhcmUgDQogICAgICAg IG5vdCBlbm91Z2g7IHRoaXMgY2hlY2sgbmVlZHMgdG8gYmUgdGhvcm91Z2guDQoNClllcy4gaWRu aXRzIDIuMTIuMTIgYSkgY291bGRuJ3QgZmluZCB0aGUgSW50cm9kdWN0aW9uIHNlY3Rpb24gZHVl IHRvIGluZGVudGluZw0KdGhhdCBpZG5pdHMgYXBwYXJlbnRseSBkaWRuJ3QgZXhwZWN0LCBiKSBm bGFnZ2VkIG9jY3VycmVuY2VzIG9mICJSRkN4eHgiIGluDQppbnN0cnVjdGlvbnMgdG8gdGhlIFJG QyBFZGl0b3IgYXMgcG9zc2libGUgZG93bnJlZnMgKHRoZXkgYXJlbid0KSwgYW5kIGZsYWdnZWQN CmFsbCByZWZlcmVuY2VzIHRvIFNDU0kgc3RhbmRhcmRzIGFzIHBvc3NpYmxlIGRvd3JlZnMgKHRo ZXkgYWxzbyBhcmVuJ3QpLg0KDQogICAgICAgIEhhcyB0aGUgZG9jdW1lbnQgDQogICAgICAgIG1l dCBhbGwgZm9ybWFsIHJldmlldyBjcml0ZXJpYSBpdCBuZWVkcyB0bywgc3VjaCBhcyB0aGUgTUlC IA0KICAgICAgICBEb2N0b3IsIG1lZGlhIHR5cGUgYW5kIFVSSSB0eXBlIHJldmlld3M/DQoNCk4v QS4NCg0KICAoMS5oKSBIYXMgdGhlIGRvY3VtZW50IHNwbGl0IGl0cyByZWZlcmVuY2VzIGludG8g bm9ybWF0aXZlIGFuZCANCiAgICAgICAgaW5mb3JtYXRpdmU/DQoNClllcy4NCg0KICAgICAgICBB cmUgdGhlcmUgbm9ybWF0aXZlIHJlZmVyZW5jZXMgdG8gZG9jdW1lbnRzIHRoYXQgDQogICAgICAg IGFyZSBub3QgcmVhZHkgZm9yIGFkdmFuY2VtZW50IG9yIGFyZSBvdGhlcndpc2UgaW4gYW4gdW5j bGVhciANCiAgICAgICAgc3RhdGU/IElmIHN1Y2ggbm9ybWF0aXZlIHJlZmVyZW5jZXMgZXhpc3Qs IHdoYXQgaXMgdGhlIA0KICAgICAgICBzdHJhdGVneSBmb3IgdGhlaXIgY29tcGxldGlvbj8gQXJl IHRoZXJlIG5vcm1hdGl2ZSByZWZlcmVuY2VzIA0KICAgICAgICB0aGF0IGFyZSBkb3dud2FyZCBy ZWZlcmVuY2VzLCBhcyBkZXNjcmliZWQgaW4gW1JGQzM5NjddPyBJZiANCiAgICAgICAgc28sIGxp c3QgdGhlc2UgZG93bndhcmQgcmVmZXJlbmNlcyB0byBzdXBwb3J0IHRoZSBBcmVhIA0KICAgICAg ICBEaXJlY3RvciBpbiB0aGUgTGFzdCBDYWxsIHByb2NlZHVyZSBmb3IgdGhlbSBbUkZDMzk2N10u DQoNClRoaXMgZG9jdW1lbnQgbm9ybWF0aXZlbHkgcmVmZXJlbmNlcyBkcmFmdC1pZXRmLXN0b3Jt LWlzY3NpLWNvbnMteHgNCihjdXJyZW50IHZlcnNpb24gaXMgLTA1KSwgZm9yIHdoaWNoIGEgcHVi bGljYXRpb24gcmVxdWVzdCBpcyBiZWluZw0Kc3VibWl0dGVkIGF0IHRoZSBzYW1lIHRpbWUgYXMg dGhlIHB1YmxpY2F0aW9uIHJlcXVlc3QgZm9yIHRoaXMgZG9jdWVtbnQuDQoNCkluIGFkZGl0aW9u LCB0aGlzIGRvY3VtZW50IG5vcm1hdGl2ZWx5IHJlZmVyZW5jZXMgZm91ciBTQ1NJIHN0YW5kYXJk cw0KdGhhdCBhcmUgZGV2ZWxvcGVkIGJ5IElOQ0lUUyBUZWNobmljYWwgQ29tbWl0dGVlIFQxMCAo d3d3LnQxMC5vcmcpLA0KbmFtZWx5IFNBTTIsIFNBTTQsIFNBTTUgYW5kIFNQQzQuICBBcyBjb21w bGV0ZWQgc3RhbmRhcmRzLCBTQU0yIGFuZCBTQU00DQphcmUgbm90IHB1YmxpY2x5IGF2YWlsYWJs ZSBkb2N1bWVudHMsIGJlY2F1c2UgVDEwJ3MgcGFyZW50IHN0YW5kYXJkcw0Kb3JnYW5pemF0aW9u cyBmdW5kIHRoZWlyIG9wZXJhdGlvbnMgaW4gcGFydCBieSBjaGFyZ2luZyBmb3IgY29waWVzIG9m DQpzdGFuZGFyZHMuICBUaGUgZG9jdW1lbnQgc2hlcGhlcmQsIERhdmlkIEJsYWNrLCBpcyBhbHNv IHRoZSBvZmZpY2lhbA0KVDEwIExpYWlzb24gdG8gdGhlIElFVEYgYW5kIGluIHRoYXQgcm9sZSwg aGUgaGFzIGJlZW4gYXV0aG9yaXplZCBieSBUMTANCnRvIHByb3ZpZGUgY29waWVzIG9mIHRoZXNl IHN0YW5kYXJkcyB0byBJRVRGIHBhcnRpY2lwYW50cyBmb3IgdGhlaXIgcGVyc29uYWwNCnVzZSBp biBJRVRGIGFjdGl2aXRpZXMuIElmIGNvcGllcyBvZiB0aGVzZSBzdGFuZGFyYWRzIGFyZSBkZXNp cmVkLCBwbGVhc2UNCmNvbnRhY3QgdGhlIGRvY3VtZW50IHNoZXBoZXJkLCBEYXZpZCBCbGFjayAo ZGF2aWQuYmxhY2tAZW1jLmNvbSksIGRpcmVjdGx5Lg0KDQogICgxLmkpIEhhcyB0aGUgRG9jdW1l bnQgU2hlcGhlcmQgdmVyaWZpZWQgdGhhdCB0aGUgZG9jdW1lbnQgSUFOQSANCiAgICAgICAgY29u c2lkZXJhdGlvbiBzZWN0aW9uIGV4aXN0cyBhbmQgaXMgY29uc2lzdGVudCB3aXRoIHRoZSBib2R5 IA0KICAgICAgICBvZiB0aGUgZG9jdW1lbnQ/IElmIHRoZSBkb2N1bWVudCBzcGVjaWZpZXMgcHJv dG9jb2wgDQogICAgICAgIGV4dGVuc2lvbnMsIGFyZSByZXNlcnZhdGlvbnMgcmVxdWVzdGVkIGlu IGFwcHJvcHJpYXRlIElBTkEgDQogICAgICAgIHJlZ2lzdHJpZXM/IEFyZSB0aGUgSUFOQSByZWdp c3RyaWVzIGNsZWFybHkgaWRlbnRpZmllZD8gSWYgDQogICAgICAgIHRoZSBkb2N1bWVudCBjcmVh dGVzIGEgbmV3IHJlZ2lzdHJ5LCBkb2VzIGl0IGRlZmluZSB0aGUgDQogICAgICAgIHByb3Bvc2Vk IGluaXRpYWwgY29udGVudHMgb2YgdGhlIHJlZ2lzdHJ5IGFuZCBhbiBhbGxvY2F0aW9uIA0KICAg ICAgICBwcm9jZWR1cmUgZm9yIGZ1dHVyZSByZWdpc3RyYXRpb25zPyBEb2VzIGl0IHN1Z2dlc3Qg YSANCiAgICAgICAgcmVhc29uYWJsZSBuYW1lIGZvciB0aGUgbmV3IHJlZ2lzdHJ5PyBTZWUgW1JG QzUyMjZdLiBJZiB0aGUgDQogICAgICAgIGRvY3VtZW50IGRlc2NyaWJlcyBhbiBFeHBlcnQgUmV2 aWV3IHByb2Nlc3MgaGFzIFNoZXBoZXJkIA0KICAgICAgICBjb25mZXJyZWQgd2l0aCB0aGUgUmVz cG9uc2libGUgQXJlYSBEaXJlY3RvciBzbyB0aGF0IHRoZSBJRVNHIA0KICAgICAgICBjYW4gYXBw b2ludCB0aGUgbmVlZGVkIEV4cGVydCBkdXJpbmcgdGhlIElFU0cgRXZhbHVhdGlvbj8NCg0KVGhl IElBTkEgQ29uc2lkZXJhdGlvbnMgc2VjdGlvbiBoYXMgYmVlbiBjaGVja2VkIC0gdGhlIHRleHQg ZGVzY3JpYmluZw0KdGhlIGFkZGl0aW9ucyB0byBleGlzdGluZyByZWdpc3RyaWVzIGFuZCBjcmVh dGlvbiBvZiBvbmUgbmV3IHJlZ2lzdHJ5DQppcyBjbGVhci4gIFRoZSBuZXcgcmVnaXN0cnkgZG9l cyBub3QgdXNlIHRoZSBFeHBlcnQgUmV2aWV3IHByb2Nlc3MuDQoNCiAgKDEuaikgSGFzIHRoZSBE b2N1bWVudCBTaGVwaGVyZCB2ZXJpZmllZCB0aGF0IHNlY3Rpb25zIG9mIHRoZSANCiAgICAgICAg ZG9jdW1lbnQgdGhhdCBhcmUgd3JpdHRlbiBpbiBhIGZvcm1hbCBsYW5ndWFnZSwgc3VjaCBhcyBY TUwgDQogICAgICAgIGNvZGUsIEJORiBydWxlcywgTUlCIGRlZmluaXRpb25zLCBldGMuLCB2YWxp ZGF0ZSBjb3JyZWN0bHkgaW4gDQogICAgICAgIGFuIGF1dG9tYXRlZCBjaGVja2VyPw0KDQpOL0Eu DQoNCiAgKDEuaykgVGhlIElFU0cgYXBwcm92YWwgYW5ub3VuY2VtZW50IGluY2x1ZGVzIGEgRG9j dW1lbnQgDQogICAgICAgIEFubm91bmNlbWVudCBXcml0ZS1VcC4gUGxlYXNlIHByb3ZpZGUgc3Vj aCBhIERvY3VtZW50IA0KICAgICAgICBBbm5vdW5jZW1lbnQgV3JpdGUtVXA/IFJlY2VudCBleGFt cGxlcyBjYW4gYmUgZm91bmQgaW4gdGhlDQogICAgICAgICJBY3Rpb24iIGFubm91bmNlbWVudHMg Zm9yIGFwcHJvdmVkIGRvY3VtZW50cy4gVGhlIGFwcHJvdmFsIA0KICAgICAgICBhbm5vdW5jZW1l bnQgY29udGFpbnMgdGhlIGZvbGxvd2luZyBzZWN0aW9uczogDQoNCiAgICAgVGVjaG5pY2FsIFN1 bW1hcnkNCg0KICAgVGhlIEludGVybmV0IFNtYWxsIENvbXB1dGVyIFN5c3RlbXMgSW50ZXJmYWNl IChpU0NTSSkgaXMgYSBTQ1NJDQogICB0cmFuc3BvcnQgcHJvdG9jb2wgdGhhdCBtYXBzIHRoZSBT Q1NJIGZhbWlseSBvZiBwcm90b2NvbHMgb250bw0KICAgVENQL0lQLiBUaGUgYmFzZSBpU0NTSSBw cm90b2NvbCBpcyBiYXNlZCBvbiB0aGUgU0FNLTIgKFNDU0kNCiAgIEFyY2hpdGVjdHVyZSBNb2Rl bCAtIDIpIFNDU0kgc3RhbmRhcmQuICBUaGlzIGRvY3VtZW50IHNwZWNpZmllcw0KICAgZW5oYW5j ZW1lbnRzIHRvIHRoZSBpU0NTSSBwcm90b2NvbCB0byBzdXBwb3J0IGNlcnRhaW4gYWRkaXRpb25h bA0KICAgU0NTSSBmZWF0dXJlcyB0aGF0IGhhdmUgYmVlbiBkZWZpbmVkIGluIHN1YnNlcXVlbnQg dmVyc2lvbnMgb2YNCiAgIHRoZSBTQ1NJIEFyY2hpdGVjdHVyZSBNb2RlbC4NCg0KICAgICBXb3Jr aW5nIEdyb3VwIFN1bW1hcnkgDQoNCiAgIFRoZXJlIHdhcyB2ZXJ5IGxpdHRsZSBkaXNzZW50IGlu IHRoZSBXRyBvdmVyIHRoZSBmdW5jdGlvbmFsaXR5IGluIHRoaXMNCiAgIGRvY3VtZW50LiAgU2ln bmlmaWNhbnQgV0cgZGlzY3Vzc2lvbiB3YXMgZGV2b3RlZCB0byBjb3JyZWN0bHkgc3BlY2lmeWlu Zw0KICAgU0NTSS1yZWxhdGVkIGlkZW50aWZpZXJzIHVzZWQgYnkgdGhpcyBkcmFmdC4gIFJvYiBF bGxpb3R0IGFuZCBSYWxwaA0KICAgV2ViZXIgKGtleSBtZW1iZXJzIG9mIHRoZSBUMTAgU0NTSSBz dGFuZGFyZHMgb3JnYW5pemF0aW9uKSBwcm92aWRlZA0KICAgc2lnbmlmaWNhbnQgYXNzaXN0YW5j ZSBpbiB3b3JraW5nIHRocm91Z2ggdGhlIGlkZW50aWZpZXIgaXNzdWVzLg0KDQogICAgIERvY3Vt ZW50IFF1YWxpdHkgDQoNCiAgIGlTQ1NJIGltcGxlbWVudGVycyBmcm9tIERlbGwsIEVNQywgTWlj cm9zb2Z0LCBOZXRBcHAsIFJlZEhhdCBhbmQgVk13YXJlDQogICBoYXZlIHJldmlld2VkIHRoaXMg ZG9jdW1lbnQgZm9yIHF1YWxpdHkgYW5kIGNvbnNpc3RlbmN5IHdpdGggZXhpc3RpbmcNCiAgIGlt cGxlbWVudGF0aW9ucy4gIFRoZSByZXZpZXdzIGluZGljYXRlIHRoYXQgdGhlIGVuaGFuY2VtZW50 cyBhcmUgY2xlYXJseQ0KICAgc3BlY2lmaWVkLCBhbmQgYXJlIG5vdCBleHBlY3RlZCB0byBiZSBz aWduaWZpY2FudGx5IGRpc3J1cHRpdmUgdG8gYWRkIHRvDQogICBleGlzdGluZyBpbXBsZW1lbnRh dGlvbnMuDQoNCg== --_003_7C4DFCE962635144B8FAE8CA11D0BF1E05A7B81346MX14Acorpemcc_ Content-Type: text/plain; name="iSCSI Cons Proto Writeup.txt" Content-Description: iSCSI Cons Proto Writeup.txt Content-Disposition: attachment; filename="iSCSI Cons Proto Writeup.txt"; size=8699; creation-date="Wed, 28 Dec 2011 15:45:56 GMT"; modification-date="Thu, 12 Jan 2012 18:38:18 GMT" Content-Transfer-Encoding: base64 UFJPVE8gd3JpdGV1cDogDQogICAgICAgICAgICAgICAgICAgICAgaVNDU0kgUHJvdG9jb2wgKENv bnNvbGlkYXRlZCkNCiAgICAgICAgICAgICAgICAgICAgZHJhZnQtaWV0Zi1zdG9ybS1pc2NzaS1j b25zLTA1LnR4dA0KDQpSZXF1ZXN0ZWQgUHVibGljYXRpb24gU3RhdHVzOiBQcm9wb3NlZCBTdGFu ZGFyZA0KUFJPVE8gc2hlcGhlcmQ6IERhdmlkIEwuIEJsYWNrIChTVE9STSBXRyBDby1DaGFpcikN Ci0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0t LS0tLS0tLS0tLS0tLS0tLQ0KDQogICgxLmEpIFdobyBpcyB0aGUgRG9jdW1lbnQgU2hlcGhlcmQg Zm9yIHRoaXMgZG9jdW1lbnQ/IEhhcyB0aGUNCiAgICAgICAgRG9jdW1lbnQgU2hlcGhlcmQgcGVy c29uYWxseSByZXZpZXdlZCB0aGlzIHZlcnNpb24gb2YgdGhlIA0KICAgICAgICBkb2N1bWVudCBh bmQsIGluIHBhcnRpY3VsYXIsIGRvZXMgaGUgb3Igc2hlIGJlbGlldmUgdGhpcyANCiAgICAgICAg dmVyc2lvbiBpcyByZWFkeSBmb3IgZm9yd2FyZGluZyB0byB0aGUgSUVTRyBmb3IgcHVibGljYXRp b24/DQoNCkRhdmlkIEwuIEJsYWNrIChkYXZpZC5ibGFja0BlbWMuY29tKSBpcyB0aGUgRG9jdW1l bnQgU2hlcGhlcmQgYW5kIGEgY28tYXV0aG9yDQpvZiB0aGUgZHJhZnQuICBUaGUgRG9jdW1lbnQg U2hlcGhlcmQgaGFzIHJldmlld2VkIHRoaXMgdmVyc2lvbiBvZiB0aGUgZG9jdW1lbnQNCmFuZCBi ZWxpZXZlcyB0aGF0IGl0IGlzIHJlYWR5IGZvciBmb3J3YXJkaW5nIHRvIHRoZSBJRVNHIGZvciBw dWJsaWNhdGlvbi4NCg0KICAoMS5iKSBIYXMgdGhlIGRvY3VtZW50IGhhZCBhZGVxdWF0ZSByZXZp ZXcgYm90aCBmcm9tIGtleSBXRyBtZW1iZXJzIA0KICAgICAgICBhbmQgZnJvbSBrZXkgbm9uLVdH IG1lbWJlcnM/IERvZXMgdGhlIERvY3VtZW50IFNoZXBoZXJkIGhhdmUgDQogICAgICAgIGFueSBj b25jZXJucyBhYm91dCB0aGUgZGVwdGggb3IgYnJlYWR0aCBvZiB0aGUgcmV2aWV3cyB0aGF0IA0K ICAgICAgICBoYXZlIGJlZW4gcGVyZm9ybWVkPw0KDQpUaGUgZG9jdW1lbnQgaGFzIGhhZCBzdWZm aWNpZW50IHJldmlldyBmcm9tIGtleSBXRyBtZW1iZXJzLCBmcm9tIGltcGxlbWVudGVycw0Kd2hv IHdvcmsgb24gaW1wb3J0YW50IGlTQ1NJIGltcGxlbWVudGF0aW9ucyAoYm90aCBpbml0aWF0b3Ig YW5kIHRhcmdldCkgb3V0c2lkZQ0KdGhlIFdHIGFuZCBmcm9tIGtleSBtZW1iZXJzIG9mIElOQ0lU UyBUZWNobmljYWwgQ29tbWl0dGVlIFQxMCwgdGhlIG9yZ2FuaXphdGlvbg0KcmVzcG9uc2libGUg Zm9yIFNDU0kgc3RhbmRhcmRzLg0KDQogICgxLmMpIERvZXMgdGhlIERvY3VtZW50IFNoZXBoZXJk IGhhdmUgY29uY2VybnMgdGhhdCB0aGUgZG9jdW1lbnQgDQogICAgICAgIG5lZWRzIG1vcmUgcmV2 aWV3IGZyb20gYSBwYXJ0aWN1bGFyIG9yIGJyb2FkZXIgcGVyc3BlY3RpdmUsIA0KICAgICAgICBl LmcuLCBzZWN1cml0eSwgb3BlcmF0aW9uYWwgY29tcGxleGl0eSwgc29tZW9uZSBmYW1pbGlhciB3 aXRoIA0KICAgICAgICBBQUEsIGludGVybmF0aW9uYWxpemF0aW9uIG9yIFhNTD8NCg0KWWVzIC0g dGhlIGRvY3VtZW50IHVwZGF0ZXMgdGhlIGlTQ1NJIHByb2ZpbGUgb2YgSVBzZWMgdG8gaW5jbHVk ZSBJUHNlYyB2Mw0KKDQzMDAtc2VyaWVzIFJGQ3MpLiAgVGhlIHNlY3VyaXR5IGRpcmVjdG9yYXRl IGhhcyBhbHJlYWR5IGRlc2lnbmF0ZWQgYQ0Kd2VsbC1xdWFsaWZpZWQgcmV2aWV3ZXIgZm9yIHRo aXMgZHJhZnQsIFBhdWwgSG9mZm1hbiwgY28tY2hhaXIgb2YgdGhlDQppcHNlY21lIFdHLiAgVGhl IGRvY3VtZW50IHNoZXBoZXJkIHdpbGwgd29yayB3aXRoIHRoZSBTZWNEaXIgcmV2aWV3ZXINCnRv IGVuc3VyZSB0aGF0IHRoaXMgY29uY2VybiBpcyBhZGRyZXNzZWQgaW4gdGhlIFNlYy1EaXIgcmV2 aWV3Lg0KDQogICgxLmQpIERvZXMgdGhlIERvY3VtZW50IFNoZXBoZXJkIGhhdmUgYW55IHNwZWNp ZmljIGNvbmNlcm5zIG9yIA0KICAgICAgICBpc3N1ZXMgd2l0aCB0aGlzIGRvY3VtZW50IHRoYXQg dGhlIFJlc3BvbnNpYmxlIEFyZWEgRGlyZWN0b3INCiAgICAgICAgYW5kL29yIHRoZSBJRVNHIHNo b3VsZCBiZSBhd2FyZSBvZj8gRm9yIGV4YW1wbGUsIHBlcmhhcHMgaGUgDQogICAgICAgIG9yIHNo ZSBpcyB1bmNvbWZvcnRhYmxlIHdpdGggY2VydGFpbiBwYXJ0cyBvZiB0aGUgZG9jdW1lbnQsIG9y IA0KICAgICAgICBoYXMgY29uY2VybnMgd2hldGhlciB0aGVyZSByZWFsbHkgaXMgYSBuZWVkIGZv ciBpdC4gSW4gYW55IA0KICAgICAgICBldmVudCwgaWYgdGhlIFdHIGhhcyBkaXNjdXNzZWQgdGhv c2UgaXNzdWVzIGFuZCBoYXMgaW5kaWNhdGVkIA0KICAgICAgICB0aGF0IGl0IHN0aWxsIHdpc2hl cyB0byBhZHZhbmNlIHRoZSBkb2N1bWVudCwgZGV0YWlsIHRob3NlIA0KICAgICAgICBjb25jZXJu cyBoZXJlLiBIYXMgYW4gSVBSIGRpc2Nsb3N1cmUgcmVsYXRlZCB0byB0aGlzIGRvY3VtZW50IA0K ICAgICAgICBiZWVuIGZpbGVkPyBJZiBzbywgcGxlYXNlIGluY2x1ZGUgYSByZWZlcmVuY2UgdG8g dGhlIA0KICAgICAgICBkaXNjbG9zdXJlIGFuZCBzdW1tYXJpemUgdGhlIFdHIGRpc2N1c3Npb24g YW5kIGNvbmNsdXNpb24gb24gDQogICAgICAgIHRoaXMgaXNzdWUuDQoNCk5vLg0KDQogICgxLmUp IEhvdyBzb2xpZCBpcyB0aGUgV0cgY29uc2Vuc3VzIGJlaGluZCB0aGlzIGRvY3VtZW50PyBEb2Vz IGl0IA0KICAgICAgICByZXByZXNlbnQgdGhlIHN0cm9uZyBjb25jdXJyZW5jZSBvZiBhIGZldyBp bmRpdmlkdWFscywgd2l0aCANCiAgICAgICAgb3RoZXJzIGJlaW5nIHNpbGVudCwgb3IgZG9lcyB0 aGUgV0cgYXMgYSB3aG9sZSB1bmRlcnN0YW5kIGFuZCANCiAgICAgICAgYWdyZWUgd2l0aCBpdD8g ICANCg0KVGhlIFdHIGNvbnNlbnN1cyBiZWhpbmQgdGhpcyBkb2N1bWVudCBpcyBzb2xpZDsgdGhl IFdHIGFzIGEgd2hvbGUNCnVuZGVyc3RhbmRzIHRoaXMgZG9jdW1lbnQgYW5kIGFncmVlcyB3aXRo IHRoZSBuZWVkIGZvciBhIHNpbmdsZQ0KY29uc29saWRhdGVkIGlTQ1NJIHNwZWMgYW5kIHRoZSBt aW5vciB1cGRhdGVzIGNvbnRhaW5lZCBpbiB0aGlzIGRyYWZ0Lg0KDQogICgxLmYpIEhhcyBhbnlv bmUgdGhyZWF0ZW5lZCBhbiBhcHBlYWwgb3Igb3RoZXJ3aXNlIGluZGljYXRlZCBleHRyZW1lIA0K ICAgICAgICBkaXNjb250ZW50PyBJZiBzbywgcGxlYXNlIHN1bW1hcmlzZSB0aGUgYXJlYXMgb2Yg Y29uZmxpY3QgaW4gDQogICAgICAgIHNlcGFyYXRlIGVtYWlsIG1lc3NhZ2VzIHRvIHRoZSBSZXNw b25zaWJsZSBBcmVhIERpcmVjdG9yLiAoSXQgDQogICAgICAgIHNob3VsZCBiZSBpbiBhIHNlcGFy YXRlIGVtYWlsIGJlY2F1c2UgdGhpcyBxdWVzdGlvbm5haXJlIGlzIA0KICAgICAgICBlbnRlcmVk IGludG8gdGhlIElEIFRyYWNrZXIuKQ0KDQpOby4NCg0KICAoMS5nKSBIYXMgdGhlIERvY3VtZW50 IFNoZXBoZXJkIHBlcnNvbmFsbHkgdmVyaWZpZWQgdGhhdCB0aGUgDQogICAgICAgIGRvY3VtZW50 IHNhdGlzZmllcyBhbGwgSUQgbml0cz8gKFNlZSB0aGUgSW50ZXJuZXQtRHJhZnRzIENoZWNrbGlz dCANCiAgICAgICAgYW5kIGh0dHA6Ly90b29scy5pZXRmLm9yZy90b29scy9pZG5pdHMvKS4gQm9p bGVycGxhdGUgY2hlY2tzIGFyZSANCiAgICAgICAgbm90IGVub3VnaDsgdGhpcyBjaGVjayBuZWVk cyB0byBiZSB0aG9yb3VnaC4NCg0KWWVzLiAgaWRuaXRzIGdlbmVyYXRlcyBhIG51bWJlciBvZiBj b21tZW50cyB0aGF0IGRvIG5vdCByZXByZXNlbnQNCmFjdHVhbCBwcm9ibGVtcyB3aXRoIHRoZSBk cmFmdC4NCg0KICAgICAgICBIYXMgdGhlIGRvY3VtZW50IA0KICAgICAgICBtZXQgYWxsIGZvcm1h bCByZXZpZXcgY3JpdGVyaWEgaXQgbmVlZHMgdG8sIHN1Y2ggYXMgdGhlIE1JQiANCiAgICAgICAg RG9jdG9yLCBtZWRpYSB0eXBlIGFuZCBVUkkgdHlwZSByZXZpZXdzPw0KDQpOL0EuDQoNCiAgKDEu aCkgSGFzIHRoZSBkb2N1bWVudCBzcGxpdCBpdHMgcmVmZXJlbmNlcyBpbnRvIG5vcm1hdGl2ZSBh bmQgDQogICAgICAgIGluZm9ybWF0aXZlPw0KDQpZZXMuDQoNCiAgICAgICAgQXJlIHRoZXJlIG5v cm1hdGl2ZSByZWZlcmVuY2VzIHRvIGRvY3VtZW50cyB0aGF0IA0KICAgICAgICBhcmUgbm90IHJl YWR5IGZvciBhZHZhbmNlbWVudCBvciBhcmUgb3RoZXJ3aXNlIGluIGFuIHVuY2xlYXIgDQogICAg ICAgIHN0YXRlPyBJZiBzdWNoIG5vcm1hdGl2ZSByZWZlcmVuY2VzIGV4aXN0LCB3aGF0IGlzIHRo ZSANCiAgICAgICAgc3RyYXRlZ3kgZm9yIHRoZWlyIGNvbXBsZXRpb24/IEFyZSB0aGVyZSBub3Jt YXRpdmUgcmVmZXJlbmNlcyANCiAgICAgICAgdGhhdCBhcmUgZG93bndhcmQgcmVmZXJlbmNlcywg YXMgZGVzY3JpYmVkIGluIFtSRkMzOTY3XT8gSWYgDQogICAgICAgIHNvLCBsaXN0IHRoZXNlIGRv d253YXJkIHJlZmVyZW5jZXMgdG8gc3VwcG9ydCB0aGUgQXJlYSANCiAgICAgICAgRGlyZWN0b3Ig aW4gdGhlIExhc3QgQ2FsbCBwcm9jZWR1cmUgZm9yIHRoZW0gW1JGQzM5NjddLg0KDQpUaGlzIGRv Y3VtZW50IG5vcm1hdGl2ZWx5IHJlZmVyZW5jZXMgZHJhZnQtaWV0Zi1zdG9ybS1pc2NzaS1zYW0t MDUsIGZvcg0Kd2hpY2ggYW4gUkZDIHB1YmxpY2F0aW9uIHJlcXVlc3QgaXMgYmVpbmcgc3VibWl0 dGVkIGFsb25nIHdpdGggdGhlDQpwdWJsaWNhdGlvbiByZXF1ZXN0IGZvciB0aGlzIGRvY3VtZW50 Lg0KDQpJbiBhZGRpdGlvbiwgdGhpcyBkb2N1bWVudCBub3JtYXRpdmVseSByZWZlcmVuY2VzIGZp dmUgU0NTSSBzdGFuZGFyZHMNCnRoYXQgaGF2ZSBiZWVuIGRldmVsb3BlZCBieSBJTkNJVFMgVGVj aG5pY2FsIENvbW1pdHRlZSBUMTAgKHd3dy50MTAub3JnKSwNCm5hbWVseSBTQU0yLCBTQU0zLCBT QU00LCBTQkMgYW5kIFNQQzMuICBBcyBjb21wbGV0ZWQgc3RhbmRhcmRzLCB0aGVzZQ0KYXJlIG5v dCBwdWJsaWNseSBhdmFpbGFibGUgZG9jdW1lbnRzLCBiZWNhdXNlIFQxMCdzIHBhcmVudCBzdGFu ZGFyZHMNCm9yZ2FuaXphdGlvbnMgZnVuZCB0aGVpciBvcGVyYXRpb25zIGluIHBhcnQgYnkgY2hh cmdpbmcgZm9yIGNvcGllcyBvZg0Kc3RhbmRhcmRzLiAgVGhlIGRvY3VtZW50IHNoZXBoZXJkLCBE YXZpZCBCbGFjaywgaXMgYWxzbyB0aGUgb2ZmaWNpYWwNClQxMCBMaWFpc29uIHRvIHRoZSBJRVRG IGFuZCBpbiB0aGF0IHJvbGUsIGhlIGhhcyBiZWVuIGF1dGhvcml6ZWQgYnkgVDEwDQp0byBwcm92 aWRlIGNvcGllcyBvZiB0aGVzZSBzdGFuZGFyZHMgdG8gSUVURiBwYXJ0aWNpcGFudHMgZm9yIHRo ZWlyIHBlcnNvbmFsDQp1c2UgaW4gSUVURiBhY3Rpdml0aWVzLiBJZiBjb3BpZXMgb2YgdGhlc2Ug c3RhbmRhcmFkcyBhcmUgZGVzaXJlZCwgcGxlYXNlDQpjb250YWN0IHRoZSBkb2N1bWVudCBzaGVw aGVyZCwgRGF2aWQgQmxhY2sgKGRhdmlkLmJsYWNrQGVtYy5jb20pLCBkaXJlY3RseS4NCg0KICAo MS5pKSBIYXMgdGhlIERvY3VtZW50IFNoZXBoZXJkIHZlcmlmaWVkIHRoYXQgdGhlIGRvY3VtZW50 IElBTkEgDQogICAgICAgIGNvbnNpZGVyYXRpb24gc2VjdGlvbiBleGlzdHMgYW5kIGlzIGNvbnNp c3RlbnQgd2l0aCB0aGUgYm9keSANCiAgICAgICAgb2YgdGhlIGRvY3VtZW50PyBJZiB0aGUgZG9j dW1lbnQgc3BlY2lmaWVzIHByb3RvY29sIA0KICAgICAgICBleHRlbnNpb25zLCBhcmUgcmVzZXJ2 YXRpb25zIHJlcXVlc3RlZCBpbiBhcHByb3ByaWF0ZSBJQU5BIA0KICAgICAgICByZWdpc3RyaWVz PyBBcmUgdGhlIElBTkEgcmVnaXN0cmllcyBjbGVhcmx5IGlkZW50aWZpZWQ/IElmIA0KICAgICAg ICB0aGUgZG9jdW1lbnQgY3JlYXRlcyBhIG5ldyByZWdpc3RyeSwgZG9lcyBpdCBkZWZpbmUgdGhl IA0KICAgICAgICBwcm9wb3NlZCBpbml0aWFsIGNvbnRlbnRzIG9mIHRoZSByZWdpc3RyeSBhbmQg YW4gYWxsb2NhdGlvbiANCiAgICAgICAgcHJvY2VkdXJlIGZvciBmdXR1cmUgcmVnaXN0cmF0aW9u cz8gRG9lcyBpdCBzdWdnZXN0IGEgDQogICAgICAgIHJlYXNvbmFibGUgbmFtZSBmb3IgdGhlIG5l dyByZWdpc3RyeT8gU2VlIFtSRkM1MjI2XS4gSWYgdGhlIA0KICAgICAgICBkb2N1bWVudCBkZXNj cmliZXMgYW4gRXhwZXJ0IFJldmlldyBwcm9jZXNzIGhhcyBTaGVwaGVyZCANCiAgICAgICAgY29u ZmVycmVkIHdpdGggdGhlIFJlc3BvbnNpYmxlIEFyZWEgRGlyZWN0b3Igc28gdGhhdCB0aGUgSUVT RyANCiAgICAgICAgY2FuIGFwcG9pbnQgdGhlIG5lZWRlZCBFeHBlcnQgZHVyaW5nIHRoZSBJRVNH IEV2YWx1YXRpb24/DQoNClRoZSBJQU5BIENvbnNpZGVyYXRpb25zIHNlY3Rpb24gaGFzIGJlZW4g Y2hlY2tlZCAtIHRoZSBJQU5BIENvbnNpZGVyYXRpb25zDQphcmUgY2xlYXIgYW5kIGRvIG5vdCBy ZXF1aXJlIGFueSBhY3Rpb25zIGJ5IElBTkEuDQoNCiAgKDEuaikgSGFzIHRoZSBEb2N1bWVudCBT aGVwaGVyZCB2ZXJpZmllZCB0aGF0IHNlY3Rpb25zIG9mIHRoZSANCiAgICAgICAgZG9jdW1lbnQg dGhhdCBhcmUgd3JpdHRlbiBpbiBhIGZvcm1hbCBsYW5ndWFnZSwgc3VjaCBhcyBYTUwgDQogICAg ICAgIGNvZGUsIEJORiBydWxlcywgTUlCIGRlZmluaXRpb25zLCBldGMuLCB2YWxpZGF0ZSBjb3Jy ZWN0bHkgaW4gDQogICAgICAgIGFuIGF1dG9tYXRlZCBjaGVja2VyPw0KDQpOL0EsDQoNCiAgKDEu aykgVGhlIElFU0cgYXBwcm92YWwgYW5ub3VuY2VtZW50IGluY2x1ZGVzIGEgRG9jdW1lbnQgDQog ICAgICAgIEFubm91bmNlbWVudCBXcml0ZS1VcC4gUGxlYXNlIHByb3ZpZGUgc3VjaCBhIERvY3Vt ZW50IA0KICAgICAgICBBbm5vdW5jZW1lbnQgV3JpdGUtVXA/IFJlY2VudCBleGFtcGxlcyBjYW4g YmUgZm91bmQgaW4gdGhlDQogICAgICAgICJBY3Rpb24iIGFubm91bmNlbWVudHMgZm9yIGFwcHJv dmVkIGRvY3VtZW50cy4gVGhlIGFwcHJvdmFsIA0KICAgICAgICBhbm5vdW5jZW1lbnQgY29udGFp bnMgdGhlIGZvbGxvd2luZyBzZWN0aW9uczogDQoNCiAgICAgVGVjaG5pY2FsIFN1bW1hcnkNCg0K ICBUaGlzIGRvY3VtZW50IGRlc2NyaWJlcyBhIHRyYW5zcG9ydCBwcm90b2NvbCBmb3IgU0NTSSB0 aGF0IHdvcmtzDQogIG9uIHRvcCBvZiBUQ1AuIFRoZSBpU0NTSSBwcm90b2NvbCBhaW1zIHRvIGJl IGZ1bGx5IGNvbXBsaWFudCB3aXRoDQogIHRoZSBzdGFuZGFyZGl6ZWQgU0NTSSBBcmNoaXRlY3R1 cmUgTW9kZWwgKFNBTS0yKS4gUkZDIDM3MjANCiAgZGVmaW5lZCB0aGUgb3JpZ2luYWwgaVNDU0kg cHJvdG9jb2wuIFJGQyAzNzIxIGRpc2N1c3NlcyBpU0NTSQ0KICBOYW1pbmcgZXhhbXBsZXMgYW5k IGRpc2NvdmVyeSB0ZWNobmlxdWVzLiBTdWJzZXF1ZW50bHksIFJGQyAzOTgwDQogIGFkZGVkIGFu IGFkZGl0aW9uYWwgbmFtaW5nIGZvcm1hdCB0byBpU0NTSSBwcm90b2NvbC4gUkZDIDQ4NTANCiAg Zm9sbG93ZWQgdXAgYnkgYWRkaW5nIGEgbmV3IHB1YmxpYyBleHRlbnNpb24ga2V5IHRvIGlTQ1NJ LiBSRkMNCiAgNTA0OCBvZmZlcmVkIGEgbnVtYmVyIG9mIGNsYXJpZmljYXRpb25zIGFuZCBhIGZl dyBpbXByb3ZlbWVudHMgYW5kDQogIGNvcnJlY3Rpb25zIHRvIHRoZSBvcmlnaW5hbCBpU0NTSSBw cm90b2NvbC4NCg0KICBUaGlzIGRvY3VtZW50IG9ic29sZXRlcyBSRkNzIDM3MjAsIDM5ODAsIDQ4 NTAgYW5kIDUwNDggYnkNCiAgY29uc29saWRhdGluZyB0aGVtIGludG8gYSBzaW5nbGUgZG9jdW1l bnQgYW5kIG1ha2luZyBhZGRpdGlvbmFsDQogIHVwZGF0ZXMgdG8gdGhlIGNvbnNvbGlkYXRlZCBz cGVjaWZpY2F0aW9uLiBUaGlzIGRvY3VtZW50IGFsc28NCiAgdXBkYXRlcyBSRkMgMzcyMSBhbmQg UkZDIDM3MjMuIFRoZSB0ZXh0IGluIHRoaXMgZG9jdW1lbnQgdGh1cw0KICBzdXBlcnNlZGVzIHRo ZSB0ZXh0IGluIGFsbCB0aGUgbm90ZWQgUkZDcyB3aGVyZXZlciB0aGVyZSBpcyBhDQogIGRpZmZl cmVuY2UgaW4gc2VtYW50aWNzLg0KDQogICAgIFdvcmtpbmcgR3JvdXAgU3VtbWFyeSANCg0KICAg VGhlcmUgd2FzIHZlcnkgbGl0dGxlIGRpc3NlbnQgaW4gdGhlIFdHIG92ZXIgdGhlIGZ1bmN0aW9u YWxpdHkgaW4gdGhpcw0KICAgZG9jdW1lbnQuICBTaWduaWZpY2FudCBXRyBkaXNjdXNzaW9uIHdh cyBkZXZvdGVkIHRvIGNvcnJlY3RseSBzcGVjaWZ5aW5nDQogICBTQ1NJLXJlbGF0ZWQgaWRlbnRp ZmllcnMgdXNlZCBieSB0aGlzIGRyYWZ0LiAgUm9iIEVsbGlvdHQgYW5kIFJhbHBoDQogICBXZWJl ciAoa2V5IG1lbWJlcnMgb2YgdGhlIFQxMCBTQ1NJIHN0YW5kYXJkcyBvcmdhbml6YXRpb24pIHBy b3ZpZGVkDQogICBzaWduaWZpY2FudCBhc3Npc3RhbmNlIGluIHdvcmtpbmcgdGhyb3VnaCB0aGUg aWRlbnRpZmllciBpc3N1ZXMuDQoNCiAgICAgRG9jdW1lbnQgUXVhbGl0eSANCg0KICAgaVNDU0kg aW1wbGVtZW50ZXJzIGZyb20gRGVsbCwgRU1DLCBNaWNyb3NvZnQsIE5ldEFwcCwgUmVkSGF0IGFu ZCBWTXdhcmUNCiAgIGhhdmUgcmV2aWV3ZWQgdGhpcyBkb2N1bWVudCBmb3IgcXVhbGl0eSBhbmQg Y29uc2lzdGVuY3kgd2l0aCBleGlzdGluZw0KICAgaW1wbGVtZW50YXRpb25zLiAgVGhlIHJldmll d3MgaW5kaWNhdGUgdGhhdCB0aGUgY2hhbmdlcyBhcmUgY2xlYXJseQ0KICAgc3BlY2lmaWVkLCBh bmQgYXJlIG5vdCBleHBlY3RlZCB0byBiZSBzaWduaWZpY2FudGx5IGRpc3J1cHRpdmUgdG8gYWRk IHRvDQogICBleGlzdGluZyBpbXBsZW1lbnRhdGlvbnMuDQo= --_003_7C4DFCE962635144B8FAE8CA11D0BF1E05A7B81346MX14Acorpemcc_-- From hemal@broadcom.com Thu Jan 12 14:00:02 2012 Return-Path: X-Original-To: storm@ietfa.amsl.com Delivered-To: storm@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 4779321F85E0 for ; Thu, 12 Jan 2012 14:00:02 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -6.599 X-Spam-Level: X-Spam-Status: No, score=-6.599 tagged_above=-999 required=5 tests=[AWL=0.000, BAYES_00=-2.599, RCVD_IN_DNSWL_MED=-4] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id V8Cep8mRd3x4 for ; Thu, 12 Jan 2012 14:00:01 -0800 (PST) Received: from mms2.broadcom.com (mms2.broadcom.com [216.31.210.18]) by ietfa.amsl.com (Postfix) with ESMTP id 83BB421F85A4 for ; Thu, 12 Jan 2012 14:00:01 -0800 (PST) Received: from [10.9.208.26] by mms2.broadcom.com with ESMTP (Broadcom SMTP Relay (Email Firewall v6.3.2)); Thu, 12 Jan 2012 14:08:17 -0800 X-Server-Uuid: D3C04415-6FA8-4F2C-93C1-920E106A2031 Received: from IRVEXCHMB11.corp.ad.broadcom.com ( [fe80::9cdd:3a57:8694:f610]) by IRVEXCHCAS05.corp.ad.broadcom.com ( [::1]) with mapi id 14.01.0355.002; Thu, 12 Jan 2012 13:59:45 -0800 From: "Hemal Shah" To: "david.black@emc.com" , "storm@ietf.org" Thread-Topic: RDMAP opcodes - proposal Thread-Index: AczQA/KxuaL1Lor0Q2y6Gk0CrCJm4QBbYXTA Date: Thu, 12 Jan 2012 21:59:44 +0000 Message-ID: <2D98DD3F898B6B4DA287BF3BA07DAE9301F11E@IRVEXCHMB11.corp.ad.broadcom.com> References: <7C4DFCE962635144B8FAE8CA11D0BF1E05A7B80D5B@MX14A.corp.emc.com> In-Reply-To: <7C4DFCE962635144B8FAE8CA11D0BF1E05A7B80D5B@MX14A.corp.emc.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [10.12.148.247] MIME-Version: 1.0 X-WSS-ID: 6311865B3GG18363057-01-01 Content-Type: text/plain; charset=iso-8859-1 Content-Transfer-Encoding: quoted-printable Subject: Re: [storm] RDMAP opcodes - proposal X-BeenThere: storm@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Storage Maintenance WG List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 12 Jan 2012 22:00:02 -0000 David, I'm not in agreement with your proposal below. Even though we have 4 opcode= s remaining, I'm not comfortable with making protocol changes in the antici= pation of running out of opcodes in the future. I would rather make protoco= l changes when we run out of opcodes. Regarding (1), I do not think we should defer the allocation of RDMA opcode= s for RDMA protocol extensions draft. I see RDMA protocol extensions as enh= ancements to RFC5040. RDMA opcodes for RDMAP extensions should be viewed as= an integral part of RDMAP. Regarding (2), the use of reserved bits for RDMAP extensions draft will not= make RDMA protocol extensions draft compatible with RFC5040. It will cause= implementation problems. For example, an RFC5040 compliant implementation = (that will ignore reserved bits on receive) can misinterpret new opcodes de= fined in RDMA protocol extensions draft as one of the valid opcodes defined= in RFC5040. Regarding (3), RDMA protocol extensions draft needs 4 opcodes. Atomic Reque= st/Response having separate opcodes for request and response is consistent = with RDMA messages and opcodes definition in RFC5040. We have separate mess= age opcodes for RDMA read request and RDMA read response in RFC5040. I woul= d like to keep the opcodes for request/response separate and consistent in = both RFC5040 and RDMA protocol extensions draft. When we run out of opcodes in the future, we can revise the protocol versio= n and increase the size of RDMA opcode field as needed. Doing it now is not= necessary and can introduce implementation problems. I also think we should go ahead and request allocation of RDMAP opcodes for= the RDMAP extensions. Regards, Hemal =20 -----Original Message----- From: storm-bounces@ietf.org [mailto:storm-bounces@ietf.org] On Behalf Of d= avid.black@emc.com Sent: Tuesday, January 10, 2012 5:54 PM To: storm@ietf.org Subject: [storm] RDMAP opcodes - proposal Importance: High After thinking about the RDMAP opcode situation over the weekend, I have a = proposed course of action. Please comment, as I would like to move on this= fairly quickly in order to get both the MPA Peer Connect and RDMAP registr= ies drafts approved by the IESG. There are 8 currently unused RDMAP opcodes. The RDMAP extensions draft wil= l use 4 of them, and setting aside 1 more for test/experimentation leaves 3= . I'm not comfortable committing to a course of action that consumes more = than half of the unused opcodes without taking a good hard look at the alte= rnatives and consequences. Hence I propose to do the following: (1) Defer allocation of text/experimentation RDMAP opcode to RDMAP extensio= ns draft, which is already going to allocate RDMAP opcodes. (2) As part of discussion of the RDMAP extensions draft, consider the 2 res= erved (and currently unused) bits that are adjacent to the RDMAP opcode fie= ld. Although any use of those bits requires modifying RFC 5040 (RDMAP), th= at's a reasonable thing for the RDMAP extensions draft to do *if* it won't = cause serious implementation problems. (3) As part of the discussion of the reserved bits, the following questions= should be addressed: - Does the RDMAP extensions draft need 4 opcodes? For example, would 2 opc= odes plus use of one of the reserved bits as a direction bit (Request/Response) for those two opcodes be a better approach? - Should 4 or 8 of the currently unused 4-bit RDMAP opcodes be turned into = 6-bit opcodes by including the two reserved bits? Implementation concerns matter to this discussion, as they may constrain pr= actical use of those 2 reserved bits. By deferring allocation of the test/= experimentation opcode, we keep our options open for this discussion. Does this sound like a reasonable plan? Thanks, --David ---------------------------------------------------- David L. Black, Distinguished Engineer EMC Corporation, 176 South St., Hopkinton, MA=A0 01748 +1 (508) 293-7953=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 FAX: +1 (508) 293-778= 6 david.black@emc.com=A0=A0=A0=A0=A0=A0=A0 Mobile: +1 (978) 394-7754 ---------------------------------------------------- _______________________________________________ storm mailing list storm@ietf.org https://www.ietf.org/mailman/listinfo/storm From david.black@emc.com Thu Jan 12 14:50:40 2012 Return-Path: X-Original-To: storm@ietfa.amsl.com Delivered-To: storm@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 1B64721F855D for ; Thu, 12 Jan 2012 14:50:40 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -106.593 X-Spam-Level: X-Spam-Status: No, score=-106.593 tagged_above=-999 required=5 tests=[AWL=0.006, BAYES_00=-2.599, RCVD_IN_DNSWL_MED=-4, USER_IN_WHITELIST=-100] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id TZyS+jKTZ4sL for ; Thu, 12 Jan 2012 14:50:39 -0800 (PST) Received: from mexforward.lss.emc.com (mexforward.lss.emc.com [128.222.32.20]) by ietfa.amsl.com (Postfix) with ESMTP id 252C421F855B for ; Thu, 12 Jan 2012 14:50:37 -0800 (PST) Received: from hop04-l1d11-si02.isus.emc.com (HOP04-L1D11-SI02.isus.emc.com [10.254.111.55]) by mexforward.lss.emc.com (Switch-3.4.3/Switch-3.4.3) with ESMTP id q0CMoYpB029500 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Thu, 12 Jan 2012 17:50:35 -0500 Received: from mailhub.lss.emc.com (mailhub.lss.emc.com [10.254.222.226]) by hop04-l1d11-si02.isus.emc.com (RSA Interceptor); Thu, 12 Jan 2012 17:50:14 -0500 Received: from mxhub08.corp.emc.com (mxhub08.corp.emc.com [128.222.70.205]) by mailhub.lss.emc.com (Switch-3.4.3/Switch-3.4.3) with ESMTP id q0CMoDKb025349; Thu, 12 Jan 2012 17:50:14 -0500 Received: from mx14a.corp.emc.com ([169.254.1.99]) by mxhub08.corp.emc.com ([128.222.70.205]) with mapi; Thu, 12 Jan 2012 17:50:14 -0500 From: To: , Date: Thu, 12 Jan 2012 17:50:11 -0500 Thread-Topic: RDMAP opcodes - proposal Thread-Index: AczQA/KxuaL1Lor0Q2y6Gk0CrCJm4QBbYXTAAAE1dLA= Message-ID: <7C4DFCE962635144B8FAE8CA11D0BF1E05A7B813D4@MX14A.corp.emc.com> References: <7C4DFCE962635144B8FAE8CA11D0BF1E05A7B80D5B@MX14A.corp.emc.com> <2D98DD3F898B6B4DA287BF3BA07DAE9301F11E@IRVEXCHMB11.corp.ad.broadcom.com> In-Reply-To: <2D98DD3F898B6B4DA287BF3BA07DAE9301F11E@IRVEXCHMB11.corp.ad.broadcom.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: acceptlanguage: en-US Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-EMM-MHVC: 1 Subject: Re: [storm] RDMAP opcodes - proposal X-BeenThere: storm@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Storage Maintenance WG List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 12 Jan 2012 22:50:40 -0000 Hemal, Thanks for the response - your viewpoints are helpful, especially the imple= mentation-based comment that extending the opcode field beyond the 4-bits s= pecified in RFC 5040 would be problematic. I do have one clarification: > Regarding (1), I do not think we should defer the allocation of RDMA opco= des for RDMA protocol > extensions draft. > I also think we should go ahead and request allocation of RDMAP opcodes f= or the RDMAP extensions. That proposed course of action, adding RDMAP extensions opcodes to the RDDP= registries draft, will result in a normative reference to the RDMAP extens= ions draft, and that reference will delay RFC publication of not only the R= DDP registries draft, but also the MPA peer connect draft until the RDMAP e= xtensions draft catches up with them. I don't think imposing that delay is a good idea, as the RDMAP extensions d= raft can allocate and register its own opcodes 0x8-0xB - the IANA Considera= tions text in Section 10 of the -02 version of that draft will do that. In= practice, these four opcodes are implicitly reserved for the RDMAP extensi= on operations, as there are no other proposals to use those opcodes, and an= y such proposal would need to come through the storm WG. > Regarding (2), the use of reserved bits for RDMAP extensions draft will n= ot make RDMA protocol > extensions draft compatible with RFC5040. It will cause implementation pr= oblems. > Regarding (3), RDMA protocol extensions draft needs 4 opcodes. Atomic Req= uest/Response having separate > opcodes for request and response is consistent with RDMA messages and opc= odes definition in RFC5040. > When we run out of opcodes in the future, we can revise the protocol vers= ion and increase the size of > RDMA opcode field as needed. Doing it now is not necessary and can introd= uce implementation problems. This would lead to a plan to use the RDDP registries draft to register opco= de 0xF for test and experimentation now, and to use the RDMAP extensions dr= aft to register 0x8-0xB for atomic and immediate operations when it is appr= oved, leaving 3 unassigned RDMAP opcodes, 0xC-0xE, for possible future use. I'll wait until early next week to see if anyone else has comments before s= ubmitting a revised version of the RDDP registries draft that allocates RDM= AP opcode 0xF for test and experimentation, but I would like to get the rev= ised RDDP registries draft submitted soon, as this opcode issue is the last= item that is blocking IESG approval of the MPA peer connect draft. Thanks, --David > -----Original Message----- > From: Hemal Shah [mailto:hemal@broadcom.com] > Sent: Thursday, January 12, 2012 5:00 PM > To: Black, David; storm@ietf.org > Subject: RE: RDMAP opcodes - proposal >=20 > David, >=20 > I'm not in agreement with your proposal below. Even though we have 4 opco= des remaining, I'm not > comfortable with making protocol changes in the anticipation of running o= ut of opcodes in the future. > I would rather make protocol changes when we run out of opcodes. >=20 > Regarding (1), I do not think we should defer the allocation of RDMA opco= des for RDMA protocol > extensions draft. I see RDMA protocol extensions as enhancements to RFC50= 40. RDMA opcodes for RDMAP > extensions should be viewed as an integral part of RDMAP. >=20 > Regarding (2), the use of reserved bits for RDMAP extensions draft will n= ot make RDMA protocol > extensions draft compatible with RFC5040. It will cause implementation pr= oblems. For example, an > RFC5040 compliant implementation (that will ignore reserved bits on recei= ve) can misinterpret new > opcodes defined in RDMA protocol extensions draft as one of the valid opc= odes defined in RFC5040. >=20 > Regarding (3), RDMA protocol extensions draft needs 4 opcodes. Atomic Req= uest/Response having separate > opcodes for request and response is consistent with RDMA messages and opc= odes definition in RFC5040. > We have separate message opcodes for RDMA read request and RDMA read resp= onse in RFC5040. I would like > to keep the opcodes for request/response separate and consistent in both = RFC5040 and RDMA protocol > extensions draft. >=20 > When we run out of opcodes in the future, we can revise the protocol vers= ion and increase the size of > RDMA opcode field as needed. Doing it now is not necessary and can introd= uce implementation problems. >=20 > I also think we should go ahead and request allocation of RDMAP opcodes f= or the RDMAP extensions. >=20 > Regards, >=20 > Hemal >=20 > -----Original Message----- > From: storm-bounces@ietf.org [mailto:storm-bounces@ietf.org] On Behalf Of= david.black@emc.com > Sent: Tuesday, January 10, 2012 5:54 PM > To: storm@ietf.org > Subject: [storm] RDMAP opcodes - proposal > Importance: High >=20 > After thinking about the RDMAP opcode situation over the weekend, I have = a proposed course of action. > Please comment, as I would like to move on this fairly quickly in order t= o get both the MPA Peer > Connect and RDMAP registries drafts approved by the IESG. >=20 > There are 8 currently unused RDMAP opcodes. The RDMAP extensions draft w= ill use 4 of them, and > setting aside 1 more for test/experimentation leaves 3. I'm not comforta= ble committing to a course of > action that consumes more than half of the unused opcodes without taking = a good hard look at the > alternatives and consequences. Hence I propose to do the following: >=20 > (1) Defer allocation of text/experimentation RDMAP opcode to RDMAP extens= ions draft, which is already > going to allocate RDMAP opcodes. >=20 > (2) As part of discussion of the RDMAP extensions draft, consider the 2 r= eserved (and currently > unused) bits that are adjacent to the RDMAP opcode field. Although any u= se of those bits requires > modifying RFC 5040 (RDMAP), that's a reasonable thing for the RDMAP exten= sions draft to do *if* it > won't cause serious implementation problems. >=20 > (3) As part of the discussion of the reserved bits, the following questio= ns should be addressed: > - Does the RDMAP extensions draft need 4 opcodes? For example, would 2 o= pcodes > plus use of one of the reserved bits as a direction bit (Request/Respons= e) > for those two opcodes be a better approach? > - Should 4 or 8 of the currently unused 4-bit RDMAP opcodes be turned int= o 6-bit > opcodes by including the two reserved bits? >=20 > Implementation concerns matter to this discussion, as they may constrain = practical use of those 2 > reserved bits. By deferring allocation of the test/experimentation opcod= e, we keep our options open > for this discussion. >=20 > Does this sound like a reasonable plan? >=20 > Thanks, > --David > ---------------------------------------------------- > David L. Black, Distinguished Engineer > EMC Corporation, 176 South St., Hopkinton, MA=A0 01748 > +1 (508) 293-7953=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 FAX: +1 (508) 293-7= 786 > david.black@emc.com=A0=A0=A0=A0=A0=A0=A0 Mobile: +1 (978) 394-7754 > ---------------------------------------------------- >=20 >=20 > _______________________________________________ > storm mailing list > storm@ietf.org > https://www.ietf.org/mailman/listinfo/storm >=20 >=20 From hemal@broadcom.com Fri Jan 13 10:04:44 2012 Return-Path: X-Original-To: storm@ietfa.amsl.com Delivered-To: storm@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 32E2A21F85F8 for ; Fri, 13 Jan 2012 10:04:44 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -6.599 X-Spam-Level: X-Spam-Status: No, score=-6.599 tagged_above=-999 required=5 tests=[AWL=0.000, BAYES_00=-2.599, RCVD_IN_DNSWL_MED=-4] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id u29vUF0fIGJi for ; Fri, 13 Jan 2012 10:04:42 -0800 (PST) Received: from mms1.broadcom.com (mms1.broadcom.com [216.31.210.17]) by ietfa.amsl.com (Postfix) with ESMTP id A4FCD21F85EC for ; Fri, 13 Jan 2012 10:04:42 -0800 (PST) Received: from [10.9.208.27] by mms1.broadcom.com with ESMTP (Broadcom SMTP Relay (Email Firewall v6.3.2)); Fri, 13 Jan 2012 10:11:22 -0800 X-Server-Uuid: 02CED230-5797-4B57-9875-D5D2FEE4708A Received: from IRVEXCHMB11.corp.ad.broadcom.com ( [fe80::9cdd:3a57:8694:f610]) by irvexchmb05.corp.ad.broadcom.com ( [::1]) with mapi id 14.01.0289.001; Fri, 13 Jan 2012 10:02:15 -0800 From: "Hemal Shah" To: "david.black@emc.com" , "storm@ietf.org" Thread-Topic: RDMAP opcodes - proposal Thread-Index: AczQA/KxuaL1Lor0Q2y6Gk0CrCJm4QBbYXTAAAE1dLAAKaIS0A== Date: Fri, 13 Jan 2012 18:02:14 +0000 Message-ID: <2D98DD3F898B6B4DA287BF3BA07DAE9301F969@IRVEXCHMB11.corp.ad.broadcom.com> References: <7C4DFCE962635144B8FAE8CA11D0BF1E05A7B80D5B@MX14A.corp.emc.com> <2D98DD3F898B6B4DA287BF3BA07DAE9301F11E@IRVEXCHMB11.corp.ad.broadcom.com> <7C4DFCE962635144B8FAE8CA11D0BF1E05A7B813D4@MX14A.corp.emc.com> In-Reply-To: <7C4DFCE962635144B8FAE8CA11D0BF1E05A7B813D4@MX14A.corp.emc.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [10.12.148.247] MIME-Version: 1.0 X-WSS-ID: 630EAC403DS15434891-01-01 Content-Type: text/plain; charset=iso-8859-1 Content-Transfer-Encoding: quoted-printable Subject: Re: [storm] RDMAP opcodes - proposal X-BeenThere: storm@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Storage Maintenance WG List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 13 Jan 2012 18:04:44 -0000 David, Thanks for the clarification! I'm in agreement with you for not delaying RD= DP registries draft for the RFC publication of RDMAP extensions draft. The = RDMAP extensions draft -02 already has text in IANA considerations section = requesting its own RDMAP op codes. I'm also fine with reserving 0xF opcode in RDDP registries draft for test a= nd experimentation. Regards, Hemal=20 -----Original Message----- From: david.black@emc.com [mailto:david.black@emc.com]=20 Sent: Thursday, January 12, 2012 2:50 PM To: Hemal Shah; storm@ietf.org Subject: RE: RDMAP opcodes - proposal Hemal, Thanks for the response - your viewpoints are helpful, especially the imple= mentation-based comment that extending the opcode field beyond the 4-bits s= pecified in RFC 5040 would be problematic. I do have one clarification: > Regarding (1), I do not think we should defer the allocation of RDMA opco= des for RDMA protocol > extensions draft. > I also think we should go ahead and request allocation of RDMAP opcodes f= or the RDMAP extensions. That proposed course of action, adding RDMAP extensions opcodes to the RDDP= registries draft, will result in a normative reference to the RDMAP extens= ions draft, and that reference will delay RFC publication of not only the R= DDP registries draft, but also the MPA peer connect draft until the RDMAP e= xtensions draft catches up with them. I don't think imposing that delay is a good idea, as the RDMAP extensions d= raft can allocate and register its own opcodes 0x8-0xB - the IANA Considera= tions text in Section 10 of the -02 version of that draft will do that. In= practice, these four opcodes are implicitly reserved for the RDMAP extensi= on operations, as there are no other proposals to use those opcodes, and an= y such proposal would need to come through the storm WG. > Regarding (2), the use of reserved bits for RDMAP extensions draft will n= ot make RDMA protocol > extensions draft compatible with RFC5040. It will cause implementation pr= oblems. > Regarding (3), RDMA protocol extensions draft needs 4 opcodes. Atomic Req= uest/Response having separate > opcodes for request and response is consistent with RDMA messages and opc= odes definition in RFC5040. > When we run out of opcodes in the future, we can revise the protocol vers= ion and increase the size of > RDMA opcode field as needed. Doing it now is not necessary and can introd= uce implementation problems. This would lead to a plan to use the RDDP registries draft to register opco= de 0xF for test and experimentation now, and to use the RDMAP extensions dr= aft to register 0x8-0xB for atomic and immediate operations when it is appr= oved, leaving 3 unassigned RDMAP opcodes, 0xC-0xE, for possible future use. I'll wait until early next week to see if anyone else has comments before s= ubmitting a revised version of the RDDP registries draft that allocates RDM= AP opcode 0xF for test and experimentation, but I would like to get the rev= ised RDDP registries draft submitted soon, as this opcode issue is the last= item that is blocking IESG approval of the MPA peer connect draft. Thanks, --David > -----Original Message----- > From: Hemal Shah [mailto:hemal@broadcom.com] > Sent: Thursday, January 12, 2012 5:00 PM > To: Black, David; storm@ietf.org > Subject: RE: RDMAP opcodes - proposal >=20 > David, >=20 > I'm not in agreement with your proposal below. Even though we have 4 opco= des remaining, I'm not > comfortable with making protocol changes in the anticipation of running o= ut of opcodes in the future. > I would rather make protocol changes when we run out of opcodes. >=20 > Regarding (1), I do not think we should defer the allocation of RDMA opco= des for RDMA protocol > extensions draft. I see RDMA protocol extensions as enhancements to RFC50= 40. RDMA opcodes for RDMAP > extensions should be viewed as an integral part of RDMAP. >=20 > Regarding (2), the use of reserved bits for RDMAP extensions draft will n= ot make RDMA protocol > extensions draft compatible with RFC5040. It will cause implementation pr= oblems. For example, an > RFC5040 compliant implementation (that will ignore reserved bits on recei= ve) can misinterpret new > opcodes defined in RDMA protocol extensions draft as one of the valid opc= odes defined in RFC5040. >=20 > Regarding (3), RDMA protocol extensions draft needs 4 opcodes. Atomic Req= uest/Response having separate > opcodes for request and response is consistent with RDMA messages and opc= odes definition in RFC5040. > We have separate message opcodes for RDMA read request and RDMA read resp= onse in RFC5040. I would like > to keep the opcodes for request/response separate and consistent in both = RFC5040 and RDMA protocol > extensions draft. >=20 > When we run out of opcodes in the future, we can revise the protocol vers= ion and increase the size of > RDMA opcode field as needed. Doing it now is not necessary and can introd= uce implementation problems. >=20 > I also think we should go ahead and request allocation of RDMAP opcodes f= or the RDMAP extensions. >=20 > Regards, >=20 > Hemal >=20 > -----Original Message----- > From: storm-bounces@ietf.org [mailto:storm-bounces@ietf.org] On Behalf Of= david.black@emc.com > Sent: Tuesday, January 10, 2012 5:54 PM > To: storm@ietf.org > Subject: [storm] RDMAP opcodes - proposal > Importance: High >=20 > After thinking about the RDMAP opcode situation over the weekend, I have = a proposed course of action. > Please comment, as I would like to move on this fairly quickly in order t= o get both the MPA Peer > Connect and RDMAP registries drafts approved by the IESG. >=20 > There are 8 currently unused RDMAP opcodes. The RDMAP extensions draft w= ill use 4 of them, and > setting aside 1 more for test/experimentation leaves 3. I'm not comforta= ble committing to a course of > action that consumes more than half of the unused opcodes without taking = a good hard look at the > alternatives and consequences. Hence I propose to do the following: >=20 > (1) Defer allocation of text/experimentation RDMAP opcode to RDMAP extens= ions draft, which is already > going to allocate RDMAP opcodes. >=20 > (2) As part of discussion of the RDMAP extensions draft, consider the 2 r= eserved (and currently > unused) bits that are adjacent to the RDMAP opcode field. Although any u= se of those bits requires > modifying RFC 5040 (RDMAP), that's a reasonable thing for the RDMAP exten= sions draft to do *if* it > won't cause serious implementation problems. >=20 > (3) As part of the discussion of the reserved bits, the following questio= ns should be addressed: > - Does the RDMAP extensions draft need 4 opcodes? For example, would 2 o= pcodes > plus use of one of the reserved bits as a direction bit (Request/Respons= e) > for those two opcodes be a better approach? > - Should 4 or 8 of the currently unused 4-bit RDMAP opcodes be turned int= o 6-bit > opcodes by including the two reserved bits? >=20 > Implementation concerns matter to this discussion, as they may constrain = practical use of those 2 > reserved bits. By deferring allocation of the test/experimentation opcod= e, we keep our options open > for this discussion. >=20 > Does this sound like a reasonable plan? >=20 > Thanks, > --David > ---------------------------------------------------- > David L. Black, Distinguished Engineer > EMC Corporation, 176 South St., Hopkinton, MA=A0 01748 > +1 (508) 293-7953=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 FAX: +1 (508) 293-7= 786 > david.black@emc.com=A0=A0=A0=A0=A0=A0=A0 Mobile: +1 (978) 394-7754 > ---------------------------------------------------- >=20 >=20 > _______________________________________________ > storm mailing list > storm@ietf.org > https://www.ietf.org/mailman/listinfo/storm >=20 >=20 From internet-drafts@ietf.org Sun Jan 15 05:40:26 2012 Return-Path: X-Original-To: storm@ietfa.amsl.com Delivered-To: storm@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 75D7E21F84B9; Sun, 15 Jan 2012 05:40:26 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -102.544 X-Spam-Level: X-Spam-Status: No, score=-102.544 tagged_above=-999 required=5 tests=[AWL=0.055, BAYES_00=-2.599, USER_IN_WHITELIST=-100] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 1snRzhIeovAK; Sun, 15 Jan 2012 05:40:25 -0800 (PST) Received: from ietfa.amsl.com (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 1FC1821F849B; Sun, 15 Jan 2012 05:40:11 -0800 (PST) MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable From: internet-drafts@ietf.org To: i-d-announce@ietf.org X-Test-IDTracker: no X-IETF-IDTracker: 3.64p1 Message-ID: <20120115134011.27535.67315.idtracker@ietfa.amsl.com> Date: Sun, 15 Jan 2012 05:40:11 -0800 Cc: storm@ietf.org Subject: [storm] I-D Action: draft-ietf-storm-iser-08.txt X-BeenThere: storm@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Storage Maintenance WG List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 15 Jan 2012 13:40:26 -0000 A New Internet-Draft is available from the on-line Internet-Drafts director= ies. This draft is a work item of the STORage Maintenance Working Group of = the IETF. Title : iSCSI Extensions for RDMA Specification Author(s) : Michael Ko Alexander Nezhinsky Filename : draft-ietf-storm-iser-08.txt Pages : 95 Date : 2012-01-15 iSCSI Extensions for RDMA provides the RDMA data transfer capability to iSCSI by layering iSCSI on top of an RDMA-Capable Protocol. An RDMA-Capable Protocol provides RDMA Read and Write services, which enable data to be transferred directly into SCSI I/O Buffers without intermediate data copies. This document describes the extensions to the iSCSI protocol to support RDMA services as provided by an RDMA- Capable Protocol. This document obsoletes RFC 5046. A URL for this Internet-Draft is: http://www.ietf.org/internet-drafts/draft-ietf-storm-iser-08.txt Internet-Drafts are also available by anonymous FTP at: ftp://ftp.ietf.org/internet-drafts/ This Internet-Draft can be retrieved at: ftp://ftp.ietf.org/internet-drafts/draft-ietf-storm-iser-08.txt From Michael@huaweisymantec.com Sun Jan 15 05:43:12 2012 Return-Path: X-Original-To: storm@ietfa.amsl.com Delivered-To: storm@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id A732B21F84BF for ; Sun, 15 Jan 2012 05:43:12 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.598 X-Spam-Level: X-Spam-Status: No, score=-2.598 tagged_above=-999 required=5 tests=[BAYES_00=-2.599, HTML_MESSAGE=0.001] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id vX5cE712tThA for ; Sun, 15 Jan 2012 05:43:11 -0800 (PST) Received: from mta2.huaweisymantec.com (mta2.huaweisymantec.com [218.17.155.15]) by ietfa.amsl.com (Postfix) with ESMTP id 6950321F84BD for ; Sun, 15 Jan 2012 05:43:11 -0800 (PST) MIME-version: 1.0 Content-type: multipart/alternative; boundary="Boundary_(ID_120QrbP67QUlX/y++DOWIA)" Received: from hstml01-in.huaweisymantec.com ([172.26.3.42]) by hstga02-in.huaweisymantec.com (Sun Java(tm) System Messaging Server 6.3-8.03 (built Apr 24 2009; 32bit)) with ESMTP id <0LXU009QSE3TCX20@hstga02-in.huaweisymantec.com> for storm@ietf.org; Sun, 15 Jan 2012 21:43:06 +0800 (CST) Received: from m90003900a ([10.47.134.92]) by hstml01-in.huaweisymantec.com (Sun Java(tm) System Messaging Server 6.3-8.03 (built Apr 24 2009; 32bit)) with ESMTPA id <0LXU0045ME3PUK30@hstml01-in.huaweisymantec.com> for storm@ietf.org; Sun, 15 Jan 2012 21:43:05 +0800 (CST) Message-id: From: Michael Ko To: storm@ietf.org References: <20120115134011.27535.67315.idtracker@ietfa.amsl.com> Date: Sun, 15 Jan 2012 05:43:00 -0800 X-Priority: 3 X-MSMail-priority: Normal X-Mailer: Microsoft Outlook Express 6.00.2900.5931 X-MIMEOLE: Produced By Microsoft MimeOLE V6.00.2900.6109 Subject: Re: [storm] I-D Action: draft-ietf-storm-iser-08.txt X-BeenThere: storm@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Storage Maintenance WG List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 15 Jan 2012 13:43:12 -0000 This is a multi-part message in MIME format. --Boundary_(ID_120QrbP67QUlX/y++DOWIA) Content-type: text/plain; charset=iso-8859-1 Content-transfer-encoding: 7BIT This version contains all the updates as discussed in the latest exchanges with Hemal Shah and David Black. Mike ----- Original Message ----- From: internet-drafts@ietf.org To: i-d-announce@ietf.org Cc: storm@ietf.org Sent: Sunday, January 15, 2012 5:40 AM Subject: [storm] I-D Action: draft-ietf-storm-iser-08.txt A New Internet-Draft is available from the on-line Internet-Drafts directories. This draft is a work item of the STORage Maintenance Working Group of the IETF. Title : iSCSI Extensions for RDMA Specification Author(s) : Michael Ko Alexander Nezhinsky Filename : draft-ietf-storm-iser-08.txt Pages : 95 Date : 2012-01-15 iSCSI Extensions for RDMA provides the RDMA data transfer capability to iSCSI by layering iSCSI on top of an RDMA-Capable Protocol. An RDMA-Capable Protocol provides RDMA Read and Write services, which enable data to be transferred directly into SCSI I/O Buffers without intermediate data copies. This document describes the extensions to the iSCSI protocol to support RDMA services as provided by an RDMA- Capable Protocol. This document obsoletes RFC 5046. A URL for this Internet-Draft is: http://www.ietf.org/internet-drafts/draft-ietf-storm-iser-08.txt Internet-Drafts are also available by anonymous FTP at: ftp://ftp.ietf.org/internet-drafts/ This Internet-Draft can be retrieved at: ftp://ftp.ietf.org/internet-drafts/draft-ietf-storm-iser-08.txt _______________________________________________ storm mailing list storm@ietf.org https://www.ietf.org/mailman/listinfo/storm --Boundary_(ID_120QrbP67QUlX/y++DOWIA) Content-type: text/html; charset=iso-8859-1 Content-transfer-encoding: 7BIT
This version contains all the updates as discussed in the latest exchanges with Hemal Shah and David Black.
 
Mike
----- Original Message -----
Sent: Sunday, January 15, 2012 5:40 AM
Subject: [storm] I-D Action: draft-ietf-storm-iser-08.txt


A New Internet-Draft is available from the on-line Internet-Drafts directories. This draft is a work item of the STORage Maintenance Working Group of the IETF.

Title           : iSCSI Extensions for RDMA Specification
Author(s)       : Michael Ko
                          Alexander Nezhinsky
Filename        : draft-ietf-storm-iser-08.txt
Pages           : 95
Date            : 2012-01-15

   iSCSI Extensions for RDMA provides the RDMA data transfer capability
   to iSCSI by layering iSCSI on top of an RDMA-Capable Protocol.  An
   RDMA-Capable Protocol provides RDMA Read and Write services, which
   enable data to be transferred directly into SCSI I/O Buffers without
   intermediate data copies.  This document describes the extensions to
   the iSCSI protocol to support RDMA services as provided by an RDMA-
   Capable Protocol.

   This document obsoletes RFC 5046.


A URL for this Internet-Draft is:
http://www.ietf.org/internet-drafts/draft-ietf-storm-iser-08.txt

Internet-Drafts are also available by anonymous FTP at:
ftp://ftp.ietf.org/internet-drafts/

This Internet-Draft can be retrieved at:
ftp://ftp.ietf.org/internet-drafts/draft-ietf-storm-iser-08.txt

_______________________________________________
storm mailing list
storm@ietf.org
https://www.ietf.org/mailman/listinfo/storm
--Boundary_(ID_120QrbP67QUlX/y++DOWIA)-- From david.black@emc.com Mon Jan 16 14:08:49 2012 Return-Path: X-Original-To: storm@ietfa.amsl.com Delivered-To: storm@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 2175321F8690 for ; Mon, 16 Jan 2012 14:08:49 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -106.579 X-Spam-Level: X-Spam-Status: No, score=-106.579 tagged_above=-999 required=5 tests=[AWL=0.018, BAYES_00=-2.599, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_MED=-4, USER_IN_WHITELIST=-100] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id KiBcnxh+P6z4 for ; Mon, 16 Jan 2012 14:08:47 -0800 (PST) Received: from mexforward.lss.emc.com (mexforward.lss.emc.com [128.222.32.20]) by ietfa.amsl.com (Postfix) with ESMTP id 6C62421F868C for ; Mon, 16 Jan 2012 14:08:44 -0800 (PST) Received: from hop04-l1d11-si01.isus.emc.com (HOP04-L1D11-SI01.isus.emc.com [10.254.111.54]) by mexforward.lss.emc.com (Switch-3.4.3/Switch-3.4.3) with ESMTP id q0GM86kV009751 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Mon, 16 Jan 2012 17:08:13 -0500 Received: from mailhub.lss.emc.com (mailhub.lss.emc.com [10.254.221.145]) by hop04-l1d11-si01.isus.emc.com (RSA Interceptor); Mon, 16 Jan 2012 17:07:55 -0500 Received: from mxhub11.corp.emc.com (mxhub11.corp.emc.com [10.254.92.106]) by mailhub.lss.emc.com (Switch-3.4.3/Switch-3.4.3) with ESMTP id q0GM4KKg029637; Mon, 16 Jan 2012 17:07:55 -0500 Received: from mx14a.corp.emc.com ([169.254.1.99]) by mxhub11.corp.emc.com ([10.254.92.106]) with mapi; Mon, 16 Jan 2012 17:06:24 -0500 From: To: , Importance: high X-Priority: 1 Date: Mon, 16 Jan 2012 17:06:23 -0500 Thread-Topic: iSER - one last issue Thread-Index: AczTi8dTUnGPifc7Qzm0rCKaRKGZdgBDXxGA Message-ID: <7C4DFCE962635144B8FAE8CA11D0BF1E05A7CF06E5@MX14A.corp.emc.com> References: <20120115134011.27535.67315.idtracker@ietfa.amsl.com> In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: acceptlanguage: en-US Content-Type: multipart/alternative; boundary="_000_7C4DFCE962635144B8FAE8CA11D0BF1E05A7CF06E5MX14Acorpemcc_" MIME-Version: 1.0 X-EMM-MHVC: 1 Subject: [storm] iSER - one last issue X-BeenThere: storm@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Storage Maintenance WG List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 16 Jan 2012 22:08:49 -0000 --_000_7C4DFCE962635144B8FAE8CA11D0BF1E05A7CF06E5MX14Acorpemcc_ Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable Mike, Thanks for getting the -08 version of the iSER draft posted. I think that draft addresses all of the open issues, but I have a question for the WG about how to express the SendSE and SendInvSE requirements from RFC 5046. The -08 version of the iSER draft expresses requirements for the SendSE and SendInvSE messages (this is primarily for iSER over RDDP/iWARP) as: The SendSE Message should be used if supported by the RCaP layer (e.g., iWARP). My reading of RFC 5046 is that its requirements are tighter - to accurately reflect RFC 5046, I would replace "should" with "MUST" in the above text, at least for iWARP. In the alternative, if the "should"s remain, an explanatory item needs to be added to the Appendix A list of changes from RFC 5046. What do others think the right course of action is here, use "MUST" or explain weakening of requirement to "should" ? Thanks, --David From: storm-bounces@ietf.org [mailto:storm-bounces@ietf.org] On Behalf Of M= ichael Ko Sent: Sunday, January 15, 2012 8:43 AM To: storm@ietf.org Subject: Re: [storm] I-D Action: draft-ietf-storm-iser-08.txt This version contains all the updates as discussed in the latest exchanges = with Hemal Shah and David Black. Mike ----- Original Message ----- From: internet-drafts@ietf.org To: i-d-announce@ietf.org Cc: storm@ietf.org Sent: Sunday, January 15, 2012 5:40 AM Subject: [storm] I-D Action: draft-ietf-storm-iser-08.txt A New Internet-Draft is available from the on-line Internet-Drafts director= ies. This draft is a work item of the STORage Maintenance Working Group of = the IETF. Title : iSCSI Extensions for RDMA Specification Author(s) : Michael Ko Alexander Nezhinsky Filename : draft-ietf-storm-iser-08.txt Pages : 95 Date : 2012-01-15 iSCSI Extensions for RDMA provides the RDMA data transfer capability to iSCSI by layering iSCSI on top of an RDMA-Capable Protocol. An RDMA-Capable Protocol provides RDMA Read and Write services, which enable data to be transferred directly into SCSI I/O Buffers without intermediate data copies. This document describes the extensions to the iSCSI protocol to support RDMA services as provided by an RDMA- Capable Protocol. This document obsoletes RFC 5046. A URL for this Internet-Draft is: http://www.ietf.org/internet-drafts/draft-ietf-storm-iser-08.txt Internet-Drafts are also available by anonymous FTP at: ftp://ftp.ietf.org/internet-drafts/ This Internet-Draft can be retrieved at: ftp://ftp.ietf.org/internet-drafts/draft-ietf-storm-iser-08.txt _______________________________________________ storm mailing list storm@ietf.org https://www.ietf.org/mailman/listinfo/storm --_000_7C4DFCE962635144B8FAE8CA11D0BF1E05A7CF06E5MX14Acorpemcc_ Content-Type: text/html; charset="us-ascii" Content-Transfer-Encoding: quoted-printable

= Mike= ,

 

Thanks for getting the -08 version of the iSER draft posted. = ; I think that

draft addresses all of= the open issues, but I have a question for the WG

about how to express the SendSE and SendInvSE requirements fr= om RFC 5046.

 =

The -08 version of the iSER draft expresses requiremen= ts for the SendSE

and SendInvSE messa= ges (this is primarily for iSER over RDDP/iWARP) as:

<= p class=3DMsoNormal> 

The SendSE Message should be used if

supported by the RCaP layer (e.g., = iWARP).

 

My reading of RFC 5046 is that i= ts requirements are tighter - to accurately

reflect RFC 5046, I would replace “should” with “= ;MUST” in the above text,

<= span style=3D'font-size:10.0pt;font-family:"Courier New";color:black'>at le= ast for iWARP.

 

In the alternative, if the “should”s rem= ain, an explanatory item

needs to be = added to the Appendix A list of changes from RFC 5046.

 

What do othe= rs think the right course of action is here, use “MUST” or=

explain weakening of requirement to ̶= 0;should” ?

 

Thanks,
--David

<= /div>

 

From: storm-bounces@ietf.org [mailto:storm-bounces@ietf.org]= On Behalf Of Michael Ko
Sent: Sunday, January 15, 2012 8:= 43 AM
To: storm@ietf.org
Subject: Re: [storm] I-D Actio= n: draft-ietf-storm-iser-08.txt

 

This version contains all the updates as discussed in the= latest exchanges with Hemal Shah and David Black.

 

Mike

----- Original Message -----

Sent: Sunday, January 15, 2012 5:= 40 AM

Subject:<= span style=3D'font-size:10.0pt;font-family:"Arial","sans-serif"'> [storm] I= -D Action: draft-ietf-storm-iser-08.txt

 

A New Internet-Draft is available from the on-line Internet-Drafts directo= ries. This draft is a work item of the STORage Maintenance Working Group of= the IETF.

Title        &nbs= p;  : iSCSI Extensions for RDMA Specification
Author(s)  =      : Michael Ko
     &nbs= p;            &= nbsp;       Alexander Nezhinsky
Filename&n= bsp;       : draft-ietf-storm-iser-08.txt
= Pages           : 95
D= ate            : 201= 2-01-15

   iSCSI Extensions for RDMA provides the RDMA dat= a transfer capability
   to iSCSI by layering iSCSI on top of = an RDMA-Capable Protocol.  An
   RDMA-Capable Protocol pr= ovides RDMA Read and Write services, which
   enable data to b= e transferred directly into SCSI I/O Buffers without
   interm= ediate data copies.  This document describes the extensions to
&nbs= p;  the iSCSI protocol to support RDMA services as provided by an RDMA= -
   Capable Protocol.

   This document obsol= etes RFC 5046.


A URL for this Internet-Draft is:
http://www= .ietf.org/internet-drafts/draft-ietf-storm-iser-08.txt

Internet-= Drafts are also available by anonymous FTP at:
ftp://ftp.ietf.org/internet-drafts/

This = Internet-Draft can be retrieved at:
ftp://ftp.ietf.org/internet-drafts/= draft-ietf-storm-iser-08.txt

___________________________________= ____________
storm mailing list
sto= rm@ietf.org
= https://www.ietf.org/mailman/listinfo/storm

<= /body>= --_000_7C4DFCE962635144B8FAE8CA11D0BF1E05A7CF06E5MX14Acorpemcc_-- From internet-drafts@ietf.org Tue Jan 17 12:32:22 2012 Return-Path: X-Original-To: storm@ietfa.amsl.com Delivered-To: storm@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id F0C3221F86C5; Tue, 17 Jan 2012 12:32:21 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -102.574 X-Spam-Level: X-Spam-Status: No, score=-102.574 tagged_above=-999 required=5 tests=[AWL=0.025, BAYES_00=-2.599, USER_IN_WHITELIST=-100] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id PdZ7zVJ7sGWH; Tue, 17 Jan 2012 12:32:21 -0800 (PST) Received: from ietfa.amsl.com (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 62ED121F8639; Tue, 17 Jan 2012 12:32:21 -0800 (PST) MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable From: internet-drafts@ietf.org To: i-d-announce@ietf.org X-Test-IDTracker: no X-IETF-IDTracker: 3.64p1 Message-ID: <20120117203221.14940.19682.idtracker@ietfa.amsl.com> Date: Tue, 17 Jan 2012 12:32:21 -0800 Cc: storm@ietf.org Subject: [storm] I-D Action: draft-ietf-storm-rddp-registries-02.txt X-BeenThere: storm@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Storage Maintenance WG List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 17 Jan 2012 20:32:22 -0000 A New Internet-Draft is available from the on-line Internet-Drafts director= ies. This draft is a work item of the STORage Maintenance Working Group of = the IETF. Title : IANA Registries for the RDDP (Remote Direct Data Placeme= nt) Protocols Author(s) : Michael Ko David L. Black Filename : draft-ietf-storm-rddp-registries-02.txt Pages : 11 Date : 2012-01-17 The original RFCs that specified the RDDP protocol suite did not create IANA registries for RDDP error codes, operation codes and function codes. Extensions to the RDDP protocols now require these registries to be created. This memo creates the RDDP registries, populates them with values defined in the original RDDP RFCs, and provides guidance to IANA for future assignment of code points within these registries. A URL for this Internet-Draft is: http://www.ietf.org/internet-drafts/draft-ietf-storm-rddp-registries-02.txt Internet-Drafts are also available by anonymous FTP at: ftp://ftp.ietf.org/internet-drafts/ This Internet-Draft can be retrieved at: ftp://ftp.ietf.org/internet-drafts/draft-ietf-storm-rddp-registries-02.txt From david.black@emc.com Tue Jan 17 12:35:56 2012 Return-Path: X-Original-To: storm@ietfa.amsl.com Delivered-To: storm@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id A6D8921F871E for ; Tue, 17 Jan 2012 12:35:56 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -108.58 X-Spam-Level: X-Spam-Status: No, score=-108.58 tagged_above=-999 required=5 tests=[AWL=2.019, BAYES_00=-2.599, RCVD_IN_DNSWL_HI=-8, USER_IN_WHITELIST=-100] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id S-3b2wg4KnYE for ; Tue, 17 Jan 2012 12:35:56 -0800 (PST) Received: from mexforward.lss.emc.com (mexforward.lss.emc.com [128.222.32.20]) by ietfa.amsl.com (Postfix) with ESMTP id 1312521F871A for ; Tue, 17 Jan 2012 12:35:55 -0800 (PST) Received: from hop04-l1d11-si01.isus.emc.com (HOP04-L1D11-SI01.isus.emc.com [10.254.111.54]) by mexforward.lss.emc.com (Switch-3.4.3/Switch-3.4.3) with ESMTP id q0HKZnrM008647 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO) for ; Tue, 17 Jan 2012 15:35:53 -0500 Received: from mailhub.lss.emc.com (mailhub.lss.emc.com [10.254.222.226]) by hop04-l1d11-si01.isus.emc.com (RSA Interceptor) for ; Tue, 17 Jan 2012 15:35:34 -0500 Received: from mxhub08.corp.emc.com (mxhub08.corp.emc.com [128.222.70.205]) by mailhub.lss.emc.com (Switch-3.4.3/Switch-3.4.3) with ESMTP id q0HKZYFn003718 for ; Tue, 17 Jan 2012 15:35:34 -0500 Received: from mx14a.corp.emc.com ([169.254.1.99]) by mxhub08.corp.emc.com ([128.222.70.205]) with mapi; Tue, 17 Jan 2012 15:35:34 -0500 From: To: Date: Tue, 17 Jan 2012 15:35:32 -0500 Thread-Topic: [storm] I-D Action: draft-ietf-storm-rddp-registries-02.txt Thread-Index: AczVVzxphbfcb9T3SdSNAJRrocjazAAAAsGA Message-ID: <7C4DFCE962635144B8FAE8CA11D0BF1E05A7CF094B@MX14A.corp.emc.com> References: <20120117203221.14940.19682.idtracker@ietfa.amsl.com> In-Reply-To: <20120117203221.14940.19682.idtracker@ietfa.amsl.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: acceptlanguage: en-US Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-EMM-MHVC: 1 Subject: Re: [storm] I-D Action: draft-ietf-storm-rddp-registries-02.txt X-BeenThere: storm@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Storage Maintenance WG List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 17 Jan 2012 20:35:56 -0000 This version of the RDDP registries draft should resolve all of the IESG re= view comments and should result in IESG approval of both this draft and the MPA peer connect = draft. It allocates experimental codepoints both for RDMAP operation codes and SCTP for DDP fun= ction codes. FYI, --David > -----Original Message----- > From: storm-bounces@ietf.org [mailto:storm-bounces@ietf.org] On Behalf Of= internet-drafts@ietf.org > Sent: Tuesday, January 17, 2012 3:32 PM > To: i-d-announce@ietf.org > Cc: storm@ietf.org > Subject: [storm] I-D Action: draft-ietf-storm-rddp-registries-02.txt >=20 >=20 > A New Internet-Draft is available from the on-line Internet-Drafts direct= ories. This draft is a work > item of the STORage Maintenance Working Group of the IETF. >=20 > Title : IANA Registries for the RDDP (Remote Direct Data Place= ment) Protocols > Author(s) : Michael Ko > David L. Black > Filename : draft-ietf-storm-rddp-registries-02.txt > Pages : 11 > Date : 2012-01-17 >=20 > The original RFCs that specified the RDDP protocol suite did not > create IANA registries for RDDP error codes, operation codes and > function codes. Extensions to the RDDP protocols now require > these registries to be created. This memo creates the RDDP > registries, populates them with values defined in the original > RDDP RFCs, and provides guidance to IANA for future assignment > of code points within these registries. >=20 >=20 > A URL for this Internet-Draft is: > http://www.ietf.org/internet-drafts/draft-ietf-storm-rddp-registries-02.t= xt >=20 > Internet-Drafts are also available by anonymous FTP at: > ftp://ftp.ietf.org/internet-drafts/ >=20 > This Internet-Draft can be retrieved at: > ftp://ftp.ietf.org/internet-drafts/draft-ietf-storm-rddp-registries-02.tx= t >=20 > _______________________________________________ > storm mailing list > storm@ietf.org > https://www.ietf.org/mailman/listinfo/storm From david.black@emc.com Tue Jan 17 13:24:13 2012 Return-Path: X-Original-To: storm@ietfa.amsl.com Delivered-To: storm@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 3577E11E80BD for ; Tue, 17 Jan 2012 13:24:13 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -108.683 X-Spam-Level: X-Spam-Status: No, score=-108.683 tagged_above=-999 required=5 tests=[AWL=1.916, BAYES_00=-2.599, RCVD_IN_DNSWL_HI=-8, USER_IN_WHITELIST=-100] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id S68l5eB3ZR8E for ; Tue, 17 Jan 2012 13:24:12 -0800 (PST) Received: from mexforward.lss.emc.com (mexforward.lss.emc.com [128.222.32.20]) by ietfa.amsl.com (Postfix) with ESMTP id 0B29611E80A2 for ; Tue, 17 Jan 2012 13:24:11 -0800 (PST) Received: from hop04-l1d11-si01.isus.emc.com (HOP04-L1D11-SI01.isus.emc.com [10.254.111.54]) by mexforward.lss.emc.com (Switch-3.4.3/Switch-3.4.3) with ESMTP id q0HLOBWK005704 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO) for ; Tue, 17 Jan 2012 16:24:11 -0500 Received: from mailhub.lss.emc.com (mailhub.lss.emc.com [10.254.222.130]) by hop04-l1d11-si01.isus.emc.com (RSA Interceptor) for ; Tue, 17 Jan 2012 16:24:03 -0500 Received: from mxhub24.corp.emc.com (mxhub24.corp.emc.com [128.222.70.136]) by mailhub.lss.emc.com (Switch-3.4.3/Switch-3.4.3) with ESMTP id q0HLO2VA006928 for ; Tue, 17 Jan 2012 16:24:02 -0500 Received: from mx14a.corp.emc.com ([169.254.1.99]) by mxhub24.corp.emc.com ([128.222.70.136]) with mapi; Tue, 17 Jan 2012 16:24:02 -0500 From: To: Date: Tue, 17 Jan 2012 16:23:59 -0500 Thread-Topic: iSCSI MIB draft - WG LC complete, RFC publication requested Thread-Index: AczVXlOUhlcH7PdlQ/2c4Z4fL92idA== Message-ID: <7C4DFCE962635144B8FAE8CA11D0BF1E05A7CF0987@MX14A.corp.emc.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: yes X-MS-TNEF-Correlator: acceptlanguage: en-US Content-Type: multipart/mixed; boundary="_002_7C4DFCE962635144B8FAE8CA11D0BF1E05A7CF0987MX14Acorpemcc_" MIME-Version: 1.0 X-EMM-MHVC: 1 Subject: [storm] iSCSI MIB draft - WG LC complete, RFC publication requested X-BeenThere: storm@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Storage Maintenance WG List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 17 Jan 2012 21:24:13 -0000 --_002_7C4DFCE962635144B8FAE8CA11D0BF1E05A7CF0987MX14Acorpemcc_ Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable Working Group Last Call has completed on the iSCSI MIB draft without commen= ts. The RFC publication request has been submitted, and the PROTO writeup is at= tached. Thanks, --David ---------------------------------------------------- David L. Black, Distinguished Engineer EMC Corporation, 176 South St., Hopkinton, MA=A0 01748 +1 (508) 293-7953=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 FAX: +1 (508) 293-778= 6 david.black@emc.com=A0=A0=A0=A0=A0=A0=A0 Mobile: +1 (978) 394-7754 ---------------------------------------------------- --_002_7C4DFCE962635144B8FAE8CA11D0BF1E05A7CF0987MX14Acorpemcc_ Content-Type: text/plain; name="iSCSI MIB Proto Writeup.txt" Content-Description: iSCSI MIB Proto Writeup.txt Content-Disposition: attachment; filename="iSCSI MIB Proto Writeup.txt"; size=6371; creation-date="Tue, 17 Jan 2012 20:46:16 GMT"; modification-date="Tue, 17 Jan 2012 21:15:09 GMT" Content-Transfer-Encoding: base64 UFJPVE8gd3JpdGV1cDoNCg0KICAgICBEZWZpbml0aW9ucyBvZiBNYW5hZ2VkIE9iamVjdHMgZm9y IEludGVybmV0IFNtYWxsIENvbXB1dGVyIFN5c3RlbQ0KICAgICAgICAgICAgICAgICAgICAgICAg ICAgICBJbnRlcmZhY2UgKGlTQ1NJKQ0KICAgICAgICAgICAgICAgICAgICAgZHJhZnQtaWV0Zi1z dG9ybS1pc2NzaW1pYi0wMS50eHQNCg0KUmVxdWVzdGVkIFB1YmxpY2F0aW9uIFN0YXR1czogUHJv cG9zZWQgU3RhbmRhcmQNClBST1RPIHNoZXBoZXJkOiBEYXZpZCBMLiBCbGFjayAoU1RPUk0gV0cg Q28tQ2hhaXIpDQotLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0t LS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0NCg0KICAoMS5hKSBXaG8gaXMgdGhlIERvY3VtZW50 IFNoZXBoZXJkIGZvciB0aGlzIGRvY3VtZW50PyBIYXMgdGhlDQogICAgICAgIERvY3VtZW50IFNo ZXBoZXJkIHBlcnNvbmFsbHkgcmV2aWV3ZWQgdGhpcyB2ZXJzaW9uIG9mIHRoZSANCiAgICAgICAg ZG9jdW1lbnQgYW5kLCBpbiBwYXJ0aWN1bGFyLCBkb2VzIGhlIG9yIHNoZSBiZWxpZXZlIHRoaXMg DQogICAgICAgIHZlcnNpb24gaXMgcmVhZHkgZm9yIGZvcndhcmRpbmcgdG8gdGhlIElFU0cgZm9y IHB1YmxpY2F0aW9uPw0KDQpEYXZpZCBMLiBCbGFjayAoZGF2aWQuYmxhY2tAZW1jLmNvbSkgaXMg dGhlIERvY3VtZW50IFNoZXBoZXJkLg0KVGhlIERvY3VtZW50IFNoZXBoZXJkIGhhcyByZXZpZXdl ZCB0aGlzIHZlcnNpb24gb2YgdGhlIGRvY3VtZW50DQphbmQgYmVsaWV2ZXMgdGhhdCBpdCBpcyBy ZWFkeSBmb3IgZm9yd2FyZGluZyB0byB0aGUgSUVTRyBmb3IgcHVibGljYXRpb24uDQoNCiAgKDEu YikgSGFzIHRoZSBkb2N1bWVudCBoYWQgYWRlcXVhdGUgcmV2aWV3IGJvdGggZnJvbSBrZXkgV0cg bWVtYmVycyANCiAgICAgICAgYW5kIGZyb20ga2V5IG5vbi1XRyBtZW1iZXJzPyBEb2VzIHRoZSBE b2N1bWVudCBTaGVwaGVyZCBoYXZlIA0KICAgICAgICBhbnkgY29uY2VybnMgYWJvdXQgdGhlIGRl cHRoIG9yIGJyZWFkdGggb2YgdGhlIHJldmlld3MgdGhhdCANCiAgICAgICAgaGF2ZSBiZWVuIHBl cmZvcm1lZD8NCg0KVGhlIGRvY3VtZW50IGhhcyBoYWQgc3VmZmljaWVudCByZXZpZXcgZnJvbSBr ZXkgV0cgbWVtYmVycy4gIFRoaXMgaXMgYQ0KbWlub3IgdXBkYXRlIG9mIHRoZSBpU0NTSSBNSUIg YW5kIGJlbmVmaXRzIGZyb20gV0cgcmV2aWV3IG9mIHRoZQ0KaVNDU0kgZHJhZnRzIHRvIHdoaWNo IGl0IGNvcnJlc3BvbmRzLg0KDQogICgxLmMpIERvZXMgdGhlIERvY3VtZW50IFNoZXBoZXJkIGhh dmUgY29uY2VybnMgdGhhdCB0aGUgZG9jdW1lbnQgDQogICAgICAgIG5lZWRzIG1vcmUgcmV2aWV3 IGZyb20gYSBwYXJ0aWN1bGFyIG9yIGJyb2FkZXIgcGVyc3BlY3RpdmUsIA0KICAgICAgICBlLmcu LCBzZWN1cml0eSwgb3BlcmF0aW9uYWwgY29tcGxleGl0eSwgc29tZW9uZSBmYW1pbGlhciB3aXRo IA0KICAgICAgICBBQUEsIGludGVybmF0aW9uYWxpemF0aW9uIG9yIFhNTD8NCg0KTm8uDQoNCiAg KDEuZCkgRG9lcyB0aGUgRG9jdW1lbnQgU2hlcGhlcmQgaGF2ZSBhbnkgc3BlY2lmaWMgY29uY2Vy bnMgb3IgDQogICAgICAgIGlzc3VlcyB3aXRoIHRoaXMgZG9jdW1lbnQgdGhhdCB0aGUgUmVzcG9u c2libGUgQXJlYSBEaXJlY3Rvcg0KICAgICAgICBhbmQvb3IgdGhlIElFU0cgc2hvdWxkIGJlIGF3 YXJlIG9mPyBGb3IgZXhhbXBsZSwgcGVyaGFwcyBoZSANCiAgICAgICAgb3Igc2hlIGlzIHVuY29t Zm9ydGFibGUgd2l0aCBjZXJ0YWluIHBhcnRzIG9mIHRoZSBkb2N1bWVudCwgb3IgDQogICAgICAg IGhhcyBjb25jZXJucyB3aGV0aGVyIHRoZXJlIHJlYWxseSBpcyBhIG5lZWQgZm9yIGl0LiBJbiBh bnkgDQogICAgICAgIGV2ZW50LCBpZiB0aGUgV0cgaGFzIGRpc2N1c3NlZCB0aG9zZSBpc3N1ZXMg YW5kIGhhcyBpbmRpY2F0ZWQgDQogICAgICAgIHRoYXQgaXQgc3RpbGwgd2lzaGVzIHRvIGFkdmFu Y2UgdGhlIGRvY3VtZW50LCBkZXRhaWwgdGhvc2UgDQogICAgICAgIGNvbmNlcm5zIGhlcmUuIEhh cyBhbiBJUFIgZGlzY2xvc3VyZSByZWxhdGVkIHRvIHRoaXMgZG9jdW1lbnQgDQogICAgICAgIGJl ZW4gZmlsZWQ/IElmIHNvLCBwbGVhc2UgaW5jbHVkZSBhIHJlZmVyZW5jZSB0byB0aGUgDQogICAg ICAgIGRpc2Nsb3N1cmUgYW5kIHN1bW1hcml6ZSB0aGUgV0cgZGlzY3Vzc2lvbiBhbmQgY29uY2x1 c2lvbiBvbiANCiAgICAgICAgdGhpcyBpc3N1ZS4NCg0KTm8uDQoNCiAgKDEuZSkgSG93IHNvbGlk IGlzIHRoZSBXRyBjb25zZW5zdXMgYmVoaW5kIHRoaXMgZG9jdW1lbnQ/IERvZXMgaXQgDQogICAg ICAgIHJlcHJlc2VudCB0aGUgc3Ryb25nIGNvbmN1cnJlbmNlIG9mIGEgZmV3IGluZGl2aWR1YWxz LCB3aXRoIA0KICAgICAgICBvdGhlcnMgYmVpbmcgc2lsZW50LCBvciBkb2VzIHRoZSBXRyBhcyBh IHdob2xlIHVuZGVyc3RhbmQgYW5kIA0KICAgICAgICBhZ3JlZSB3aXRoIGl0PyAgIA0KDQpUaGUg V0cgaXMgbGFyZ2VseSBzaWxlbnQsIGJ1dCB0aGUgRG9jdW1lbnQgU2hlcGhlcmQgYmVsaWV2ZXMg dGhhdCB0aGUNCm5lZWQgZm9yIHRoaXMgTUlCIHVwZGF0ZSBpcyBjbGVhcmx5IHVuZGVyc3Rvb2Qg YnkgdGhlIFdHIGFzIGEgd2hvbGUsDQphbmQgbm8gb2JqZWN0aW9ucyBoYXZlIGJlZW4gcmFpc2Vk Lg0KDQogICgxLmYpIEhhcyBhbnlvbmUgdGhyZWF0ZW5lZCBhbiBhcHBlYWwgb3Igb3RoZXJ3aXNl IGluZGljYXRlZCBleHRyZW1lIA0KICAgICAgICBkaXNjb250ZW50PyBJZiBzbywgcGxlYXNlIHN1 bW1hcmlzZSB0aGUgYXJlYXMgb2YgY29uZmxpY3QgaW4gDQogICAgICAgIHNlcGFyYXRlIGVtYWls IG1lc3NhZ2VzIHRvIHRoZSBSZXNwb25zaWJsZSBBcmVhIERpcmVjdG9yLiAoSXQgDQogICAgICAg IHNob3VsZCBiZSBpbiBhIHNlcGFyYXRlIGVtYWlsIGJlY2F1c2UgdGhpcyBxdWVzdGlvbm5haXJl IGlzIA0KICAgICAgICBlbnRlcmVkIGludG8gdGhlIElEIFRyYWNrZXIuKQ0KDQpOby4NCg0KICAo MS5nKSBIYXMgdGhlIERvY3VtZW50IFNoZXBoZXJkIHBlcnNvbmFsbHkgdmVyaWZpZWQgdGhhdCB0 aGUgDQogICAgICAgIGRvY3VtZW50IHNhdGlzZmllcyBhbGwgSUQgbml0cz8gKFNlZSB0aGUgSW50 ZXJuZXQtRHJhZnRzIENoZWNrbGlzdCANCiAgICAgICAgYW5kIGh0dHA6Ly90b29scy5pZXRmLm9y Zy90b29scy9pZG5pdHMvKS4gQm9pbGVycGxhdGUgY2hlY2tzIGFyZSANCiAgICAgICAgbm90IGVu b3VnaDsgdGhpcyBjaGVjayBuZWVkcyB0byBiZSB0aG9yb3VnaC4NCg0KWWVzLg0KDQogICAgICAg IEhhcyB0aGUgZG9jdW1lbnQgDQogICAgICAgIG1ldCBhbGwgZm9ybWFsIHJldmlldyBjcml0ZXJp YSBpdCBuZWVkcyB0bywgc3VjaCBhcyB0aGUgTUlCIA0KICAgICAgICBEb2N0b3IsIG1lZGlhIHR5 cGUgYW5kIFVSSSB0eXBlIHJldmlld3M/DQoNCkEgTUlCIERvY3RvciByZXZpZXcgaXMgcmVxdWly ZWQuDQoNCiAgKDEuaCkgSGFzIHRoZSBkb2N1bWVudCBzcGxpdCBpdHMgcmVmZXJlbmNlcyBpbnRv IG5vcm1hdGl2ZSBhbmQgDQogICAgICAgIGluZm9ybWF0aXZlPw0KDQpZZXMuDQoNCiAgICAgICAg QXJlIHRoZXJlIG5vcm1hdGl2ZSByZWZlcmVuY2VzIHRvIGRvY3VtZW50cyB0aGF0IA0KICAgICAg ICBhcmUgbm90IHJlYWR5IGZvciBhZHZhbmNlbWVudCBvciBhcmUgb3RoZXJ3aXNlIGluIGFuIHVu Y2xlYXIgDQogICAgICAgIHN0YXRlPyBJZiBzdWNoIG5vcm1hdGl2ZSByZWZlcmVuY2VzIGV4aXN0 LCB3aGF0IGlzIHRoZSANCiAgICAgICAgc3RyYXRlZ3kgZm9yIHRoZWlyIGNvbXBsZXRpb24/IEFy ZSB0aGVyZSBub3JtYXRpdmUgcmVmZXJlbmNlcyANCiAgICAgICAgdGhhdCBhcmUgZG93bndhcmQg cmVmZXJlbmNlcywgYXMgZGVzY3JpYmVkIGluIFtSRkMzOTY3XT8gSWYgDQogICAgICAgIHNvLCBs aXN0IHRoZXNlIGRvd253YXJkIHJlZmVyZW5jZXMgdG8gc3VwcG9ydCB0aGUgQXJlYSANCiAgICAg ICAgRGlyZWN0b3IgaW4gdGhlIExhc3QgQ2FsbCBwcm9jZWR1cmUgZm9yIHRoZW0gW1JGQzM5Njdd Lg0KDQpSRkMgcHVibGljYXRpb24gcmVxdWVzdHMgaGF2ZSBhbHJlYWR5IGJlZW4gc3VibWl0dGVk IGZvciB0aGUgdHdvDQpJbnRlcm5ldC1EcmFmdHMgdGhhdCBhcmUgbm9ybWF0aXZlIHJlZmVyZW5j ZXMuDQoNCiAgKDEuaSkgSGFzIHRoZSBEb2N1bWVudCBTaGVwaGVyZCB2ZXJpZmllZCB0aGF0IHRo ZSBkb2N1bWVudCBJQU5BIA0KICAgICAgICBjb25zaWRlcmF0aW9uIHNlY3Rpb24gZXhpc3RzIGFu ZCBpcyBjb25zaXN0ZW50IHdpdGggdGhlIGJvZHkgDQogICAgICAgIG9mIHRoZSBkb2N1bWVudD8g SWYgdGhlIGRvY3VtZW50IHNwZWNpZmllcyBwcm90b2NvbCANCiAgICAgICAgZXh0ZW5zaW9ucywg YXJlIHJlc2VydmF0aW9ucyByZXF1ZXN0ZWQgaW4gYXBwcm9wcmlhdGUgSUFOQSANCiAgICAgICAg cmVnaXN0cmllcz8gQXJlIHRoZSBJQU5BIHJlZ2lzdHJpZXMgY2xlYXJseSBpZGVudGlmaWVkPyBJ ZiANCiAgICAgICAgdGhlIGRvY3VtZW50IGNyZWF0ZXMgYSBuZXcgcmVnaXN0cnksIGRvZXMgaXQg ZGVmaW5lIHRoZSANCiAgICAgICAgcHJvcG9zZWQgaW5pdGlhbCBjb250ZW50cyBvZiB0aGUgcmVn aXN0cnkgYW5kIGFuIGFsbG9jYXRpb24gDQogICAgICAgIHByb2NlZHVyZSBmb3IgZnV0dXJlIHJl Z2lzdHJhdGlvbnM/IERvZXMgaXQgc3VnZ2VzdCBhIA0KICAgICAgICByZWFzb25hYmxlIG5hbWUg Zm9yIHRoZSBuZXcgcmVnaXN0cnk/IFNlZSBbUkZDNTIyNl0uIElmIHRoZSANCiAgICAgICAgZG9j dW1lbnQgZGVzY3JpYmVzIGFuIEV4cGVydCBSZXZpZXcgcHJvY2VzcyBoYXMgU2hlcGhlcmQgDQog ICAgICAgIGNvbmZlcnJlZCB3aXRoIHRoZSBSZXNwb25zaWJsZSBBcmVhIERpcmVjdG9yIHNvIHRo YXQgdGhlIElFU0cgDQogICAgICAgIGNhbiBhcHBvaW50IHRoZSBuZWVkZWQgRXhwZXJ0IGR1cmlu ZyB0aGUgSUVTRyBFdmFsdWF0aW9uPw0KDQpUaGUgSUFOQSBjb25zaWRlcmF0aW9ucyBzZWN0aW9u IGhhcyBiZWVuIGNoZWNrZWQuICBObyBJQU5BIGFjdGlvbiBpcw0KcmVxdWlyZWQgYmVjYXVzZSB0 aGUgZXhpc3RpbmcgTUlCIG9iamVjdCBpZGVudGlmaWVyIHZhbHVlIGlzIHJldXNlZC4NCg0KICAo MS5qKSBIYXMgdGhlIERvY3VtZW50IFNoZXBoZXJkIHZlcmlmaWVkIHRoYXQgc2VjdGlvbnMgb2Yg dGhlIA0KICAgICAgICBkb2N1bWVudCB0aGF0IGFyZSB3cml0dGVuIGluIGEgZm9ybWFsIGxhbmd1 YWdlLCBzdWNoIGFzIFhNTCANCiAgICAgICAgY29kZSwgQk5GIHJ1bGVzLCBNSUIgZGVmaW5pdGlv bnMsIGV0Yy4sIHZhbGlkYXRlIGNvcnJlY3RseSBpbiANCiAgICAgICAgYW4gYXV0b21hdGVkIGNo ZWNrZXI/DQoNClllcywgcmVseWluZyBvbiBNSUIgY2hlY2tzIHJ1biBieSB0aGUgYXV0aG9ycy4N Cg0KICAoMS5rKSBUaGUgSUVTRyBhcHByb3ZhbCBhbm5vdW5jZW1lbnQgaW5jbHVkZXMgYSBEb2N1 bWVudCANCiAgICAgICAgQW5ub3VuY2VtZW50IFdyaXRlLVVwLiBQbGVhc2UgcHJvdmlkZSBzdWNo IGEgRG9jdW1lbnQgDQogICAgICAgIEFubm91bmNlbWVudCBXcml0ZS1VcD8gUmVjZW50IGV4YW1w bGVzIGNhbiBiZSBmb3VuZCBpbiB0aGUNCiAgICAgICAgIkFjdGlvbiIgYW5ub3VuY2VtZW50cyBm b3IgYXBwcm92ZWQgZG9jdW1lbnRzLiBUaGUgYXBwcm92YWwgDQogICAgICAgIGFubm91bmNlbWVu dCBjb250YWlucyB0aGUgZm9sbG93aW5nIHNlY3Rpb25zOiANCg0KICAgICBUZWNobmljYWwgU3Vt bWFyeQ0KDQogICBUaGlzIGRvY3VtZW50IGRlZmluZXMgYSBwb3J0aW9uIG9mIHRoZSBNYW5hZ2Vt ZW50IEluZm9ybWF0aW9uIEJhc2UNCiAgIChNSUIpIGZvciB1c2Ugd2l0aCBuZXR3b3JrIG1hbmFn ZW1lbnQgcHJvdG9jb2xzLiBJbiBwYXJ0aWN1bGFyLCBpdA0KICAgZGVmaW5lcyBvYmplY3RzIGZv ciBtYW5hZ2luZyBhIGNsaWVudCB1c2luZyB0aGUgSW50ZXJuZXQgU21hbGwNCiAgIENvbXB1dGVy IFN5c3RlbSBJbnRlcmZhY2UgKGlTQ1NJKSBwcm90b2NvbCAoU0NTSSBvdmVyIFRDUCkuDQoNCiAg IFRoaXMgZG9jdW1lbnQgbWFrZXMgbWlub3IgdXBkYXRlcyB0byB0aGUgaVNDU0kgTUlCIG9yaWdp bmFsbHkNCiAgIGRlZmluZWQgaW4gUkZDIDQ1NDQgdG8gbWF0Y2ggdGhlIGN1cnJlbnQgU0NTSSBz cGVjaWZpY2F0aW9ucy4NCg0KICAgICBXb3JraW5nIEdyb3VwIFN1bW1hcnkgDQoNCiAgIE5vdGhp bmcgZXhjZXB0aW9uYWwgdG8gbm90ZS4NCg0KICAgICBEb2N1bWVudCBRdWFsaXR5IA0KDQogICBU aGVyZSBhcmUgbXVsdGlwbGUgaW1wbGVtZW50YXRpb25zIG9mIHRoZSBpU0NTSSBNSUIgKFJGQyA0 NTQ0KQ0KICAgdGhhdCBpcyB1cGRhdGVkIGJ5IHRoaXMgZG9jdW1lbnQuDQo= --_002_7C4DFCE962635144B8FAE8CA11D0BF1E05A7CF0987MX14Acorpemcc_-- From Michael@huaweisymantec.com Wed Jan 18 11:22:05 2012 Return-Path: X-Original-To: storm@ietfa.amsl.com Delivered-To: storm@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 5956F1F0C3E for ; Wed, 18 Jan 2012 11:22:04 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.598 X-Spam-Level: X-Spam-Status: No, score=-2.598 tagged_above=-999 required=5 tests=[BAYES_00=-2.599, HTML_MESSAGE=0.001] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id KckRL0LrQtVi for ; Wed, 18 Jan 2012 11:22:03 -0800 (PST) Received: from mta2.huaweisymantec.com (mta2.huaweisymantec.com [218.17.155.15]) by ietfa.amsl.com (Postfix) with ESMTP id DA9D51F0C3C for ; Wed, 18 Jan 2012 11:22:02 -0800 (PST) MIME-version: 1.0 Content-type: multipart/alternative; boundary="Boundary_(ID_DPDvtPsijfw7Gj+gJx9P6A)" Received: from hstml02-in.huaweisymantec.com ([172.26.3.42]) by hstga02-in.huaweisymantec.com (Sun Java(tm) System Messaging Server 6.3-8.03 (built Apr 24 2009; 32bit)) with ESMTP id <0LY000GNEDSLXY70@hstga02-in.huaweisymantec.com> for storm@ietf.org; Thu, 19 Jan 2012 03:21:58 +0800 (CST) Received: from m90003900a ([69.199.248.19]) by hstml02-in.huaweisymantec.com (Sun Java(tm) System Messaging Server 6.3-8.03 (built Apr 24 2009; 32bit)) with ESMTPA id <0LY00059FDSD8B10@hstml02-in.huaweisymantec.com> for storm@ietf.org; Thu, 19 Jan 2012 03:21:57 +0800 (CST) Message-id: <6315937B35624B2E9C599A392DDB8EA1@china.huawei.com> From: Michael Ko To: david.black@emc.com, storm@ietf.org References: <20120115134011.27535.67315.idtracker@ietfa.amsl.com> <7C4DFCE962635144B8FAE8CA11D0BF1E05A7CF06E5@MX14A.corp.emc.com> Date: Wed, 18 Jan 2012 11:21:47 -0800 X-Priority: 3 X-MSMail-priority: Normal X-Mailer: Microsoft Outlook Express 6.00.2900.5931 X-MIMEOLE: Produced By Microsoft MimeOLE V6.00.2900.6109 Subject: Re: [storm] iSER - one last issue X-BeenThere: storm@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Storage Maintenance WG List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 18 Jan 2012 19:22:05 -0000 This is a multi-part message in MIME format. --Boundary_(ID_DPDvtPsijfw7Gj+gJx9P6A) Content-type: text/plain; charset=iso-8859-1 Content-transfer-encoding: 7BIT David, Here is my rationale for using the lower case "should". The intent of RFC5046 is that the Send Message type must be used instead of RDMA Reads or Writes. The Solicited Event feature of the Send Message is provided as a convenience. The receiver must still do the right thing in handling the Send Message regardless of whether the SE feature is used. In other words, the sender is responsible for using the right format for the message (Send vs RDMA) but the receiver must not rely on the sender to determine how to handle the received message. The same rationale goes for the Invalidate feature. Mike ----- Original Message ----- From: david.black@emc.com To: Michael@huaweisymantec.com ; storm@ietf.org Sent: Monday, January 16, 2012 2:06 PM Subject: iSER - one last issue Mike, Thanks for getting the -08 version of the iSER draft posted. I think that draft addresses all of the open issues, but I have a question for the WG about how to express the SendSE and SendInvSE requirements from RFC 5046. The -08 version of the iSER draft expresses requirements for the SendSE and SendInvSE messages (this is primarily for iSER over RDDP/iWARP) as: The SendSE Message should be used if supported by the RCaP layer (e.g., iWARP). My reading of RFC 5046 is that its requirements are tighter - to accurately reflect RFC 5046, I would replace "should" with "MUST" in the above text, at least for iWARP. In the alternative, if the "should"s remain, an explanatory item needs to be added to the Appendix A list of changes from RFC 5046. What do others think the right course of action is here, use "MUST" or explain weakening of requirement to "should" ? Thanks, --David From: storm-bounces@ietf.org [mailto:storm-bounces@ietf.org] On Behalf Of Michael Ko Sent: Sunday, January 15, 2012 8:43 AM To: storm@ietf.org Subject: Re: [storm] I-D Action: draft-ietf-storm-iser-08.txt This version contains all the updates as discussed in the latest exchanges with Hemal Shah and David Black. Mike ----- Original Message ----- From: internet-drafts@ietf.org To: i-d-announce@ietf.org Cc: storm@ietf.org Sent: Sunday, January 15, 2012 5:40 AM Subject: [storm] I-D Action: draft-ietf-storm-iser-08.txt A New Internet-Draft is available from the on-line Internet-Drafts directories. This draft is a work item of the STORage Maintenance Working Group of the IETF. Title : iSCSI Extensions for RDMA Specification Author(s) : Michael Ko Alexander Nezhinsky Filename : draft-ietf-storm-iser-08.txt Pages : 95 Date : 2012-01-15 iSCSI Extensions for RDMA provides the RDMA data transfer capability to iSCSI by layering iSCSI on top of an RDMA-Capable Protocol. An RDMA-Capable Protocol provides RDMA Read and Write services, which enable data to be transferred directly into SCSI I/O Buffers without intermediate data copies. This document describes the extensions to the iSCSI protocol to support RDMA services as provided by an RDMA- Capable Protocol. This document obsoletes RFC 5046. A URL for this Internet-Draft is: http://www.ietf.org/internet-drafts/draft-ietf-storm-iser-08.txt Internet-Drafts are also available by anonymous FTP at: ftp://ftp.ietf.org/internet-drafts/ This Internet-Draft can be retrieved at: ftp://ftp.ietf.org/internet-drafts/draft-ietf-storm-iser-08.txt _______________________________________________ storm mailing list storm@ietf.org https://www.ietf.org/mailman/listinfo/storm --Boundary_(ID_DPDvtPsijfw7Gj+gJx9P6A) Content-type: text/html; charset=iso-8859-1 Content-transfer-encoding: quoted-printable
David,
 
Here is my rationale for using the lower case=20 "should".
 
The intent of RFC5046 is that the Send Message type = must be=20 used instead of RDMA Reads or Writes.  The Solicited Event = feature of=20 the Send Message is provided as a convenience.  The receiver must = still do=20 the right thing in handling the Send Message regardless of whether the = SE=20 feature is used.  In other words, the sender is responsible for = using the=20 right format for the message (Send vs RDMA) but the receiver must = not rely=20 on the sender to determine how to handle the received message.  The = same=20 rationale goes for the Invalidate feature. 
 
Mike
----- Original Message -----=20
From: david.black@emc.com
To: Michael@huaweisymantec.com= ; storm@ietf.org =
Sent: Monday, January 16, 2012 2:06 PM
Subject: iSER - one last issue

Mike,

 

Thanks for=20 getting the -08 version of the iSER draft posted.  I think=20 that

draft=20 addresses all of the open issues, but I have a question for the=20 WG

about how to=20 express the SendSE and SendInvSE requirements from RFC=20 5046.

 

The = -08=20 version of the iSER draft expresses requirements for the=20 SendSE

and = SendInvSE=20 messages (this is primarily for iSER over RDDP/iWARP) = as:

 

The = SendSE=20 Message should be used if

supported by=20 the RCaP layer (e.g., iWARP).

 

My = reading of=20 RFC 5046 is that its requirements are tighter - to=20 accurately

reflect RFC=20 5046, I would replace =93should=94 with =93MUST=94 in the above=20 text,

at = least for=20 iWARP.

 

In = the=20 alternative, if the =93should=94s remain, an explanatory = item

needs to be=20 added to the Appendix A list of changes from RFC = 5046.

 

What = do others=20 think the right course of action is here, use =93MUST=94 = or

explain=20 weakening of requirement to =93should=94 ?

 

Thanks,
--David

 

From:=20 storm-bounces@ietf.org [mailto:storm-bounces@ietf.org] On Behalf Of=20 Michael Ko
Sent: Sunday, January 15, 2012 8:43 = AM
To:=20 storm@ietf.org
Subject: Re: [storm] I-D Action:=20 draft-ietf-storm-iser-08.txt

 

This version = contains all the=20 updates as discussed in the latest exchanges with Hemal Shah and David=20 Black.

 

Mike

----- = Original=20 Message -----

From: internet-drafts@ietf.org=20

To: i-d-announce@ietf.org=20

Cc: storm@ietf.org =

Sent: Sunday, = January 15,=20 2012 5:40 AM

Subject: [storm] = I-D Action:=20 draft-ietf-storm-iser-08.txt

 


A New Internet-Draft is available from the = on-line=20 Internet-Drafts directories. This draft is a work item of the STORage=20 Maintenance Working Group of the=20 IETF.

Title         &= nbsp; :=20 iSCSI Extensions for RDMA=20 Specification
Author(s)       : Michael = Ko
           &= nbsp;           &n= bsp; =20 Alexander = Nezhinsky
Filename        :=20 draft-ietf-storm-iser-08.txt
Pages      =     =20 : = 95
Date          &nb= sp; :=20 2012-01-15

   iSCSI Extensions for RDMA provides the = RDMA data=20 transfer capability
   to iSCSI by layering iSCSI on top of = an=20 RDMA-Capable Protocol.  An
   RDMA-Capable Protocol = provides=20 RDMA Read and Write services, which
   enable data to be=20 transferred directly into SCSI I/O Buffers without
   = intermediate=20 data copies.  This document describes the extensions = to
   the=20 iSCSI protocol to support RDMA services as provided by an = RDMA-
  =20 Capable Protocol.

   This document obsoletes RFC=20 5046.


A URL for this Internet-Draft is:
http://www.ietf.org/internet-drafts/draft-ietf-storm-iser-08.txt
=
Internet-Drafts=20 are also available by anonymous FTP at:
ftp://ftp.ietf.org/internet-= drafts/

This=20 Internet-Draft can be retrieved at:
= ftp://ftp.ietf.org/internet-drafts/draft-ietf-storm-iser-08.txt
_______________________________________________
storm=20 mailing list
storm@ietf.org
https://www.ietf.org= /mailman/listinfo/storm

--Boundary_(ID_DPDvtPsijfw7Gj+gJx9P6A)-- From david.black@emc.com Wed Jan 18 15:11:11 2012 Return-Path: X-Original-To: storm@ietfa.amsl.com Delivered-To: storm@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 0877711E80B9 for ; Wed, 18 Jan 2012 15:11:11 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -108.775 X-Spam-Level: X-Spam-Status: No, score=-108.775 tagged_above=-999 required=5 tests=[AWL=1.823, BAYES_00=-2.599, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_HI=-8, USER_IN_WHITELIST=-100] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 7jFZHm7ta4Bo for ; Wed, 18 Jan 2012 15:11:07 -0800 (PST) Received: from mexforward.lss.emc.com (mexforward.lss.emc.com [128.222.32.20]) by ietfa.amsl.com (Postfix) with ESMTP id DB71211E80A2 for ; Wed, 18 Jan 2012 15:10:00 -0800 (PST) Received: from hop04-l1d11-si01.isus.emc.com (HOP04-L1D11-SI01.isus.emc.com [10.254.111.54]) by mexforward.lss.emc.com (Switch-3.4.3/Switch-3.4.3) with ESMTP id q0IN9smP013830 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Wed, 18 Jan 2012 18:09:55 -0500 Received: from mailhub.lss.emc.com (mailhubhoprd04.lss.emc.com [10.254.222.226]) by hop04-l1d11-si01.isus.emc.com (RSA Interceptor); Wed, 18 Jan 2012 18:09:39 -0500 Received: from mxhub27.corp.emc.com (mxhub27.corp.emc.com [10.254.110.183]) by mailhub.lss.emc.com (Switch-3.4.3/Switch-3.4.3) with ESMTP id q0IN9cKu007457; Wed, 18 Jan 2012 18:09:38 -0500 Received: from mx14a.corp.emc.com ([169.254.1.99]) by mxhub27.corp.emc.com ([10.254.110.183]) with mapi; Wed, 18 Jan 2012 18:09:38 -0500 From: To: , Date: Wed, 18 Jan 2012 18:09:35 -0500 Thread-Topic: iSER - one last issue Thread-Index: AczWFoVrW51B5C1cRpS/cSPbLNSzJgAHkZHw Message-ID: <7C4DFCE962635144B8FAE8CA11D0BF1E05A7CF0CF7@MX14A.corp.emc.com> References: <20120115134011.27535.67315.idtracker@ietfa.amsl.com> <7C4DFCE962635144B8FAE8CA11D0BF1E05A7CF06E5@MX14A.corp.emc.com> <6315937B35624B2E9C599A392DDB8EA1@china.huawei.com> In-Reply-To: <6315937B35624B2E9C599A392DDB8EA1@china.huawei.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: acceptlanguage: en-US Content-Type: multipart/alternative; boundary="_000_7C4DFCE962635144B8FAE8CA11D0BF1E05A7CF0CF7MX14Acorpemcc_" MIME-Version: 1.0 X-EMM-MHVC: 1 Subject: Re: [storm] iSER - one last issue X-BeenThere: storm@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Storage Maintenance WG List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 18 Jan 2012 23:11:11 -0000 --_000_7C4DFCE962635144B8FAE8CA11D0BF1E05A7CF0CF7MX14Acorpemcc_ Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable Mike, Thank you for the explanation, but I don't believe you've correctly stated the intent of RFC 5046. Here are some examples from RFC 5046's text: 1.5. SCSI Read Overview The iSER Message is transferred to the target using a SendSE Message. The iSER layer at the target uses a SendInvSE Message to transfer the SCSI Response PDU back to the iSER layer at the initiator. Similar language occurs in 1.6 SCSI Write Overview. 5.2.1. Normal Connection Termination at the Initiator The iSER layer at the initiator MUST use a SendSE Message to send the Logout Request PDU to the target. Similar language occurs in 5.2.2 for target side normal connection terminat= ion, and more importantly in 7.3.1 SCSI Command: The iSER layer at the initiator MUST send the SCSI command in a SendSE Message to the target. None of the quoted text permits use of Send instead of SendSE or SendInv instead of SendInvSE. What you propose to do effectively changes "MUST" to "should" (lower case, weaker than "SHOULD"), and that sure looks like a change to RFC 5046. Are there good implementation-based reasons to weaken these requirements no= w? Does anyone else have a viewpoint on this topic? Thanks, --David From: Michael Ko [mailto:Michael@huaweisymantec.com] Sent: Wednesday, January 18, 2012 2:22 PM To: Black, David; storm@ietf.org Subject: Re: iSER - one last issue David, Here is my rationale for using the lower case "should". The intent of RFC5046 is that the Send Message type must be used instead of= RDMA Reads or Writes. The Solicited Event feature of the Send Message is = provided as a convenience. The receiver must still do the right thing in h= andling the Send Message regardless of whether the SE feature is used. In = other words, the sender is responsible for using the right format for the m= essage (Send vs RDMA) but the receiver must not rely on the sender to deter= mine how to handle the received message. The same rationale goes for the I= nvalidate feature. Mike ----- Original Message ----- From: david.black@emc.com To: Michael@huaweisymantec.com ; storm@i= etf.org Sent: Monday, January 16, 2012 2:06 PM Subject: iSER - one last issue Mike, Thanks for getting the -08 version of the iSER draft posted. I think that draft addresses all of the open issues, but I have a question for the WG about how to express the SendSE and SendInvSE requirements from RFC 5046. The -08 version of the iSER draft expresses requirements for the SendSE and SendInvSE messages (this is primarily for iSER over RDDP/iWARP) as: The SendSE Message should be used if supported by the RCaP layer (e.g., iWARP). My reading of RFC 5046 is that its requirements are tighter - to accurately reflect RFC 5046, I would replace "should" with "MUST" in the above text, at least for iWARP. In the alternative, if the "should"s remain, an explanatory item needs to be added to the Appendix A list of changes from RFC 5046. What do others think the right course of action is here, use "MUST" or explain weakening of requirement to "should" ? Thanks, --David From: storm-bounces@ietf.org [mailto:storm-bounces@ietf.org] On Behalf Of M= ichael Ko Sent: Sunday, January 15, 2012 8:43 AM To: storm@ietf.org Subject: Re: [storm] I-D Action: draft-ietf-storm-iser-08.txt This version contains all the updates as discussed in the latest exchanges = with Hemal Shah and David Black. Mike ----- Original Message ----- From: internet-drafts@ietf.org To: i-d-announce@ietf.org Cc: storm@ietf.org Sent: Sunday, January 15, 2012 5:40 AM Subject: [storm] I-D Action: draft-ietf-storm-iser-08.txt A New Internet-Draft is available from the on-line Internet-Drafts director= ies. This draft is a work item of the STORage Maintenance Working Group of = the IETF. Title : iSCSI Extensions for RDMA Specification Author(s) : Michael Ko Alexander Nezhinsky Filename : draft-ietf-storm-iser-08.txt Pages : 95 Date : 2012-01-15 iSCSI Extensions for RDMA provides the RDMA data transfer capability to iSCSI by layering iSCSI on top of an RDMA-Capable Protocol. An RDMA-Capable Protocol provides RDMA Read and Write services, which enable data to be transferred directly into SCSI I/O Buffers without intermediate data copies. This document describes the extensions to the iSCSI protocol to support RDMA services as provided by an RDMA- Capable Protocol. This document obsoletes RFC 5046. A URL for this Internet-Draft is: http://www.ietf.org/internet-drafts/draft-ietf-storm-iser-08.txt Internet-Drafts are also available by anonymous FTP at: ftp://ftp.ietf.org/internet-drafts/ This Internet-Draft can be retrieved at: ftp://ftp.ietf.org/internet-drafts/draft-ietf-storm-iser-08.txt _______________________________________________ storm mailing list storm@ietf.org https://www.ietf.org/mailman/listinfo/storm --_000_7C4DFCE962635144B8FAE8CA11D0BF1E05A7CF0CF7MX14Acorpemcc_ Content-Type: text/html; charset="us-ascii" Content-Transfer-Encoding: quoted-printable

= Mike= ,

 

Thank you for the explanation, but I don’t believe you̵= 7;ve correctly stated

the intent of R= FC 5046.  Here are some examples from RFC 5046’s text:

 

1.5.  SC= SI Read Overview

 

   The iSER Message is transferred= to the target using a SendSE Message.

 &nbs= p; The iSER layer at the target uses a SendInvSE Message to transfer the

   SCSI Response PDU back to the iSER lay= er at the initiator.

 

Similar language occurs in 1.6 SCSI Write Overview.=

 

5.2= .1.  Normal Connection Termination at the Initiator<= /p>

 

&nbs= p;  The iSER layer at the initiator MUST

   use a SendSE Message to send the Lo= gout Request PDU to the target.

 

Similar language occurs in 5.2.2 for target side normal connection termina= tion,

and more impo= rtantly in 7.3.1 SCSI Command:

 

=    The iSER layer at the initiator MUST send the SCSI command in = a

<= span style=3D'font-size:10.0pt;font-family:"Courier New"'>   Send= SE Message to the target.

 

Non= e of the quoted text permits use of Send instead of SendSE or SendInv<= /o:p>

instead of SendInvSE.

 

What you propose to do effectivel= y changes “MUST” to “should” (lower case,

weaker than “SHOULD&#= 8221;), and that sure looks like a change to RFC 5046.

 

Are there good implementation-based reasons to weak= en these requirements now?

 = ;

Does anyone else have a viewpoint o= n this topic?

 

Thanks,
--David

 

From: Michael Ko [mailto:Michael@huaweisymantec.com]
Sent:= Wednesday, January 18, 2012 2:22 PM
To: Black, David; storm@ietf= .org
Subject: Re: iSER - one last issue

 

David,

 

Here is my rationale for using the lower case= "should".

&= nbsp;

The intent of RFC5046 is that the Send Message type must be used = instead of RDMA Reads or Writes.  The Solicited Event feature of = the Send Message is provided as a convenience.  The receiver must stil= l do the right thing in handling the Send Message regardless of whether the= SE feature is used.  In other words, the sender is responsible for us= ing the right format for the message (Send vs RDMA) but the receiver m= ust not rely on the sender to determine how to handle the received message.=   The same rationale goes for the Invalidate feature. 

 

Mike

----- Original Message -----

Sent: Monday, January 16, 2012 2:06 PM

Subject: iSER - one last issue

 

 

<= p class=3DMsoNormal>Thanks for getting the -08 version of the iSER draft posted= .  I think that

draft addresses= all of the open issues, but I have a question for the WG=

about how to express the SendSE and SendInvSE requirem= ents from RFC 5046.

 

The -08 version of the iSER draft expresses r= equirements for the SendSE

and SendIn= vSE messages (this is primarily for iSER over RDDP/iWARP) as:

 

The SendSE Message should be used if

supported by the RCaP laye= r (e.g., iWARP).

 

My reading of RFC 5046 = is that its requirements are tighter - to accurately

<= p class=3DMsoNormal>reflect RFC 5046, I would replace “should” with= “MUST” in the above text,

 

In the alternative, if the “should̶= 1;s remain, an explanatory item

<= span style=3D'font-size:10.0pt;font-family:"Courier New";color:black'>needs= to be added to the Appendix A list of changes from RFC 5046.

 

<= span style=3D'font-size:10.0pt;font-family:"Courier New";color:black'>What = do others think the right course of action is here, use “MUST” = or

explain weakening of requirement t= o “should” ?

 <= /o:p>

Thanks,
--David

 

From: storm-bounces@ietf.org [mailto:storm-bounces@i= etf.org] On Behalf Of Michael Ko
Sent: Sunday, January 15,= 2012 8:43 AM
To: storm@ietf.org
Subject: Re: [storm] I= -D Action: draft-ietf-storm-iser-08.txt

 

This version contains all the updates as discussed = in the latest exchanges with Hemal Shah and David Black.<= /p>

 

Mike

----- Original Message -----

From: internet-drafts@ie= tf.org

Sent: Sunday, January 15, 201= 2 5:40 AM

Subject: [storm]= I-D Action: draft-ietf-storm-iser-08.txt

=

 

<= br>A New Internet-Draft is available from the on-line Internet-Drafts direc= tories. This draft is a work item of the STORage Maintenance Working Group = of the IETF.

Title        &n= bsp;  : iSCSI Extensions for RDMA Specification
Author(s) &nbs= p;     : Michael Ko
     &n= bsp;            = ;        Alexander Nezhinsky
Filename=         : draft-ietf-storm-iser-08.txtPages           : 95Date            : 2= 012-01-15

   iSCSI Extensions for RDMA provides the RDMA d= ata transfer capability
   to iSCSI by layering iSCSI on top o= f an RDMA-Capable Protocol.  An
   RDMA-Capable Protocol = provides RDMA Read and Write services, which
   enable data to= be transferred directly into SCSI I/O Buffers without
   inte= rmediate data copies.  This document describes the extensions to
&n= bsp;  the iSCSI protocol to support RDMA services as provided by an RD= MA-
   Capable Protocol.

   This document obs= oletes RFC 5046.


A URL for this Internet-Draft is:
http://w= ww.ietf.org/internet-drafts/draft-ietf-storm-iser-08.txt

Interne= t-Drafts are also available by anonymous FTP at:
ftp://ftp.ietf.org/internet-drafts/

Thi= s Internet-Draft can be retrieved at:
ftp://ftp.ietf.org/internet-draft= s/draft-ietf-storm-iser-08.txt

_________________________________= ______________
storm mailing list
s= torm@ietf.org
https://www.ietf.org/mailman/listinfo/storm

= --_000_7C4DFCE962635144B8FAE8CA11D0BF1E05A7CF0CF7MX14Acorpemcc_-- From hemal@broadcom.com Wed Jan 18 15:58:27 2012 Return-Path: X-Original-To: storm@ietfa.amsl.com Delivered-To: storm@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 1308211E8089 for ; Wed, 18 Jan 2012 15:58:27 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -6.598 X-Spam-Level: X-Spam-Status: No, score=-6.598 tagged_above=-999 required=5 tests=[AWL=-0.000, BAYES_00=-2.599, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_MED=-4] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id FFhpAQkephEP for ; Wed, 18 Jan 2012 15:58:23 -0800 (PST) Received: from mms2.broadcom.com (mms2.broadcom.com [216.31.210.18]) by ietfa.amsl.com (Postfix) with ESMTP id DB1BB11E8075 for ; Wed, 18 Jan 2012 15:58:22 -0800 (PST) Received: from [10.9.208.27] by mms2.broadcom.com with ESMTP (Broadcom SMTP Relay (Email Firewall v6.3.2)); Wed, 18 Jan 2012 16:04:10 -0800 X-Server-Uuid: D3C04415-6FA8-4F2C-93C1-920E106A2031 Received: from IRVEXCHMB11.corp.ad.broadcom.com ( [fe80::9cdd:3a57:8694:f610]) by irvexchmb05.corp.ad.broadcom.com ( [::1]) with mapi id 14.01.0289.001; Wed, 18 Jan 2012 15:55:30 -0800 From: "Hemal Shah" To: "david.black@emc.com" , "Michael@huaweisymantec.com" , "storm@ietf.org" Thread-Topic: iSER - one last issue Thread-Index: AQHM1jZ9kWFrhlJkoESTJ1G4SNHEtpYSxIMQ Date: Wed, 18 Jan 2012 23:55:30 +0000 Message-ID: <2D98DD3F898B6B4DA287BF3BA07DAE93022E4D@IRVEXCHMB11.corp.ad.broadcom.com> References: <20120115134011.27535.67315.idtracker@ietfa.amsl.com> <7C4DFCE962635144B8FAE8CA11D0BF1E05A7CF06E5@MX14A.corp.emc.com> <6315937B35624B2E9C599A392DDB8EA1@china.huawei.com> <7C4DFCE962635144B8FAE8CA11D0BF1E05A7CF0CF7@MX14A.corp.emc.com> In-Reply-To: <7C4DFCE962635144B8FAE8CA11D0BF1E05A7CF0CF7@MX14A.corp.emc.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [10.12.148.247] MIME-Version: 1.0 X-WSS-ID: 630982703GG20447112-01-01 Content-Type: multipart/alternative; boundary=_000_2D98DD3F898B6B4DA287BF3BA07DAE93022E4DIRVEXCHMB11corpad_ Subject: Re: [storm] iSER - one last issue X-BeenThere: storm@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Storage Maintenance WG List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 18 Jan 2012 23:58:27 -0000 --_000_2D98DD3F898B6B4DA287BF3BA07DAE93022E4DIRVEXCHMB11corpad_ Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable David and Mike, I believe that the requirements should not be weakened and we should stick = to the requirement in RFC5046. RFC5046 required the use of specific Send Message Type for valid reasons. In the case of requiring SendSE message use, the intent was to give a hint = to the remote side to generate an event upon processing of SendSE message. = For example, the SendSE is specifically carrying SCSI information that need= s attention on the remote side e.g. SCSI command, Logout... In the case of requiring SendInvSE Message, the intent was to invalidate th= e STag used in the data transfer as well as inform the remote side to gener= ate an event. For example, mandating the use of SendInvSE for the SCSI resp= onse PDU for a SCSI Read limits the exposure of STag used in SCSI Read data= transfer and avoids the explicit invalidation of the STag at the initiator= . Plus, it allows the target to generate an event on the initiator upon the= completion of SCSI Read command. By relaxing RFC5046 requirements in the latest iSER draft, we will not only= impact all iWARP based iSER implementations that rely on the use of specif= ic Send Message Type for SCSI data transfers but also change the intended b= ehavior of initiators and targets. For iWARP, I strongly suggest that the iSER draft does not relax the requir= ements specified in RFC5046. I hope that helps. Hemal From: storm-bounces@ietf.org [mailto:storm-bounces@ietf.org] On Behalf Of d= avid.black@emc.com Sent: Wednesday, January 18, 2012 3:10 PM To: Michael@huaweisymantec.com; storm@ietf.org Subject: Re: [storm] iSER - one last issue Mike, Thank you for the explanation, but I don't believe you've correctly stated the intent of RFC 5046. Here are some examples from RFC 5046's text: 1.5. SCSI Read Overview The iSER Message is transferred to the target using a SendSE Message. The iSER layer at the target uses a SendInvSE Message to transfer the SCSI Response PDU back to the iSER layer at the initiator. Similar language occurs in 1.6 SCSI Write Overview. 5.2.1. Normal Connection Termination at the Initiator The iSER layer at the initiator MUST use a SendSE Message to send the Logout Request PDU to the target. Similar language occurs in 5.2.2 for target side normal connection terminat= ion, and more importantly in 7.3.1 SCSI Command: The iSER layer at the initiator MUST send the SCSI command in a SendSE Message to the target. None of the quoted text permits use of Send instead of SendSE or SendInv instead of SendInvSE. What you propose to do effectively changes "MUST" to "should" (lower case, weaker than "SHOULD"), and that sure looks like a change to RFC 5046. Are there good implementation-based reasons to weaken these requirements no= w? Does anyone else have a viewpoint on this topic? Thanks, --David From: Michael Ko [mailto:Michael@huaweisymantec.com] Sent: Wednesday, January 18, 2012 2:22 PM To: Black, David; storm@ietf.org Subject: Re: iSER - one last issue David, Here is my rationale for using the lower case "should". The intent of RFC5046 is that the Send Message type must be used instead of= RDMA Reads or Writes. The Solicited Event feature of the Send Message is = provided as a convenience. The receiver must still do the right thing in h= andling the Send Message regardless of whether the SE feature is used. In = other words, the sender is responsible for using the right format for the m= essage (Send vs RDMA) but the receiver must not rely on the sender to deter= mine how to handle the received message. The same rationale goes for the I= nvalidate feature. Mike ----- Original Message ----- From: david.black@emc.com To: Michael@huaweisymantec.com ; storm@i= etf.org Sent: Monday, January 16, 2012 2:06 PM Subject: iSER - one last issue Mike, Thanks for getting the -08 version of the iSER draft posted. I think that draft addresses all of the open issues, but I have a question for the WG about how to express the SendSE and SendInvSE requirements from RFC 5046. The -08 version of the iSER draft expresses requirements for the SendSE and SendInvSE messages (this is primarily for iSER over RDDP/iWARP) as: The SendSE Message should be used if supported by the RCaP layer (e.g., iWARP). My reading of RFC 5046 is that its requirements are tighter - to accurately reflect RFC 5046, I would replace "should" with "MUST" in the above text, at least for iWARP. In the alternative, if the "should"s remain, an explanatory item needs to be added to the Appendix A list of changes from RFC 5046. What do others think the right course of action is here, use "MUST" or explain weakening of requirement to "should" ? Thanks, --David From: storm-bounces@ietf.org [mailto:storm-bounces@ietf.org] On Behalf Of M= ichael Ko Sent: Sunday, January 15, 2012 8:43 AM To: storm@ietf.org Subject: Re: [storm] I-D Action: draft-ietf-storm-iser-08.txt This version contains all the updates as discussed in the latest exchanges = with Hemal Shah and David Black. Mike ----- Original Message ----- From: internet-drafts@ietf.org To: i-d-announce@ietf.org Cc: storm@ietf.org Sent: Sunday, January 15, 2012 5:40 AM Subject: [storm] I-D Action: draft-ietf-storm-iser-08.txt A New Internet-Draft is available from the on-line Internet-Drafts director= ies. This draft is a work item of the STORage Maintenance Working Group of = the IETF. Title : iSCSI Extensions for RDMA Specification Author(s) : Michael Ko Alexander Nezhinsky Filename : draft-ietf-storm-iser-08.txt Pages : 95 Date : 2012-01-15 iSCSI Extensions for RDMA provides the RDMA data transfer capability to iSCSI by layering iSCSI on top of an RDMA-Capable Protocol. An RDMA-Capable Protocol provides RDMA Read and Write services, which enable data to be transferred directly into SCSI I/O Buffers without intermediate data copies. This document describes the extensions to the iSCSI protocol to support RDMA services as provided by an RDMA- Capable Protocol. This document obsoletes RFC 5046. A URL for this Internet-Draft is: http://www.ietf.org/internet-drafts/draft-ietf-storm-iser-08.txt Internet-Drafts are also available by anonymous FTP at: ftp://ftp.ietf.org/internet-drafts/ This Internet-Draft can be retrieved at: ftp://ftp.ietf.org/internet-drafts/draft-ietf-storm-iser-08.txt _______________________________________________ storm mailing list storm@ietf.org https://www.ietf.org/mailman/listinfo/storm --_000_2D98DD3F898B6B4DA287BF3BA07DAE93022E4DIRVEXCHMB11corpad_ Content-Type: text/html; charset=us-ascii Content-Transfer-Encoding: quoted-printable

Dav= id and Mike,

 

I b= elieve that the requirements should not be weakened and we should stick to = the requirement in RFC5046.

 

RFC= 5046 required the use of specific Send Message Type for valid reasons.=

 

In = the case of requiring SendSE message use, the intent was to give a hint to = the remote side to generate an event upon processing of SendSE message. For= example, the SendSE is specifically carrying SCSI information that needs attention on the remote side e.g. SCS= I command, Logout…

 

In = the case of requiring SendInvSE Message, the intent was to invalidate the S= Tag used in the data transfer as well as inform the remote side to generate= an event. For example, mandating the use of SendInvSE for the SCSI response PDU for a SCSI Read limits the expo= sure of STag used in SCSI Read data transfer and avoids the explicit invali= dation of the STag at the initiator. Plus, it allows the target to generate= an event on the initiator upon the completion of SCSI Read command.

 

By = relaxing RFC5046 requirements in the latest iSER draft, we will not only im= pact all iWARP based iSER implementations that rely on the use of specific = Send Message Type for SCSI data transfers but also change the intended behavior of initiators and targets.

 

For= iWARP, I strongly suggest that the iSER draft does not relax the requireme= nts specified in RFC5046.

 

I h= ope that helps.

 

Hem= al    

 

From: storm-bo= unces@ietf.org [mailto:storm-bounces@ietf.org] On Behalf Of david.black@emc.com
Sent: Wednesday, January 18, 2012 3:10 PM
To: Michael@huaweisymantec.com; storm@ietf.org
Subject: Re: [storm] iSER - one last issue

 

Mike,

 

Thank you for the explanation, but I don’= ;t believe you’ve correctly stated

the intent of RFC 5046.  Here are some ex= amples from RFC 5046’s text:

 

1.5.  SCSI Read Overview

 

   The iSER Messag= e is transferred to the target using a SendSE Message.

 

   The iSER layer = at the target uses a SendInvSE Message to transfer the

   SCSI Response PDU back to the iSER layer at t= he initiator.

 

Similar language occurs in 1.6 SCSI Write Overview.

 

5.2.1.  Normal Connecti= on Termination at the Initiator

 

   The iSER layer = at the initiator MUST

   use a SendSE Me= ssage to send the Logout Request PDU to the target.

 

Similar language occurs in 5= .2.2 for target side normal connection termination,

and more importantly in 7.3.= 1 SCSI Command:

 

   The iSER layer = at the initiator MUST send the SCSI command in a

   SendSE Message = to the target.

 

None of the quoted text perm= its use of Send instead of SendSE or SendInv

instead of SendInvSE.

 

What you propose to do effec= tively changes “MUST” to “should” (lower case,=

weaker than “SHOULD= 221;), and that sure looks like a change to RFC 5046.

 

Are there good implementatio= n-based reasons to weaken these requirements now?

 

Does anyone else have a viewpoint on this topi= c?

 

Thanks,
--David

 

From: Michael = Ko [mailto:Michael@huaweisymantec.com]
Sent: Wednesday, January 18, 2012 2:22 PM
To: Black, David; storm@ietf.org
Subject: Re: iSER - one last issue

 

David,<= /o:p>

 

Here is my rational= e for using the lower case "should".

 

The intent of RFC50= 46 is that the Send Message type must be used instead of RDMA Reads or Writ= es.  The Solicited Event feature of the Send Message is provided = as a convenience.  The receiver must still do the right thing in handling the Send Message regardless of whether the SE = feature is used.  In other words, the sender is responsible for using = the right format for the message (Send vs RDMA) but the receiver must = not rely on the sender to determine how to handle the received message.  The same rationale goes for the Invalid= ate feature. 

 

Mike

 

Mike,

 

Thanks for getting the -08 version of the iSER= draft posted.  I think that

draft addresses all of the open issues, but I = have a question for the WG

about how to express the SendSE and SendInvSE = requirements from RFC 5046.

 

The -08 version of the iSER draft expresses re= quirements for the SendSE

and SendInvSE messages (this is primarily for = iSER over RDDP/iWARP) as:

 

The SendSE Message = should be used if

supported by the RC= aP layer (e.g., iWARP).

 

My reading of RFC 5046 is that its requirement= s are tighter - to accurately

reflect RFC 5046, I would replace “shoul= d” with “MUST” in the above text,

at least for iWARP.

 

In the alternative, if the “should”= ;s remain, an explanatory item

needs to be added to the Appendix A list of ch= anges from RFC 5046.

 

What do others think the right course of actio= n is here, use “MUST” or

explain weakening of requirement to “sho= uld” ?

 

Thanks,
--David

 

From: storm-bo= unces@ietf.org [mailto:storm-bounces@ietf.org] On Behalf Of Michael Ko
Sent: Sunday, January 15, 2012 8:43 AM
To: storm@ietf.org
Subject: Re: [storm] I-D Action: draft-ietf-storm-iser-08.txt

 

This version contai= ns all the updates as discussed in the latest exchanges with Hemal Shah and= David Black.

 

Mike

 


A New Internet-Draft is available from the on-line Internet-Drafts director= ies. This draft is a work item of the STORage Maintenance Working Group of = the IETF.

Title           : iSCSI E= xtensions for RDMA Specification
Author(s)       : Michael Ko
            &nb= sp;            = Alexander Nezhinsky
Filename        : draft-ietf-storm-iser-= 08.txt
Pages           : 95
Date            : 20= 12-01-15

   iSCSI Extensions for RDMA provides the RDMA data transfer capa= bility
   to iSCSI by layering iSCSI on top of an RDMA-Capable Protocol.=   An
   RDMA-Capable Protocol provides RDMA Read and Write services, w= hich
   enable data to be transferred directly into SCSI I/O Buffers w= ithout
   intermediate data copies.  This document describes the ex= tensions to
   the iSCSI protocol to support RDMA services as provided by an = RDMA-
   Capable Protocol.

   This document obsoletes RFC 5046.


A URL for this Internet-Draft is:
http://www.ietf.org/internet-drafts/draft-ietf-storm-iser-08.txt

Internet-Drafts are also available by anonymous FTP at:
ftp://ftp.ietf.org/internet= -drafts/

This Internet-Draft can be retrieved at:
ftp://ftp.ietf.org/internet-drafts/draft-ietf-storm-iser-08.txt

_______________________________________________
storm mailing list
storm@ietf.org
https://www.ietf.or= g/mailman/listinfo/storm

--_000_2D98DD3F898B6B4DA287BF3BA07DAE93022E4DIRVEXCHMB11corpad_-- From ttalpey@microsoft.com Thu Jan 19 05:51:37 2012 Return-Path: X-Original-To: storm@ietfa.amsl.com Delivered-To: storm@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id BBDC721F8487 for ; Thu, 19 Jan 2012 05:51:36 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -103.598 X-Spam-Level: X-Spam-Status: No, score=-103.598 tagged_above=-999 required=5 tests=[BAYES_00=-2.599, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_LOW=-1, USER_IN_WHITELIST=-100] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id q1QLGVHn-1Lf for ; Thu, 19 Jan 2012 05:51:32 -0800 (PST) Received: from ch1outboundpool.messaging.microsoft.com (ch1ehsobe005.messaging.microsoft.com [216.32.181.185]) by ietfa.amsl.com (Postfix) with ESMTP id C312021F8472 for ; Thu, 19 Jan 2012 05:51:31 -0800 (PST) Received: from mail21-ch1-R.bigfish.com (10.43.68.250) by CH1EHSOBE001.bigfish.com (10.43.70.51) with Microsoft SMTP Server id 14.1.225.23; Thu, 19 Jan 2012 13:51:30 +0000 Received: from mail21-ch1 (localhost [127.0.0.1]) by mail21-ch1-R.bigfish.com (Postfix) with ESMTP id 25CFF44057E; Thu, 19 Jan 2012 13:51:30 +0000 (UTC) X-SpamScore: -48 X-BigFish: VS-48(zz9371I936eKc85fh148cM542M1418Mc1dM4015Lzz1202hzz1033IL8275bh8275dhz2fhc1bhc31hc1ahc1bhc31hc1ah2a8h668h839h) X-Forefront-Antispam-Report: CIP:131.107.125.8; KIP:(null); UIP:(null); IPV:NLI; H:TK5EX14MLTC101.redmond.corp.microsoft.com; RD:none; EFVD:NLI X-FB-SS: 13, Received-SPF: pass (mail21-ch1: domain of microsoft.com designates 131.107.125.8 as permitted sender) client-ip=131.107.125.8; envelope-from=ttalpey@microsoft.com; helo=TK5EX14MLTC101.redmond.corp.microsoft.com ; icrosoft.com ; Received: from mail21-ch1 (localhost.localdomain [127.0.0.1]) by mail21-ch1 (MessageSwitch) id 1326981089517377_635; Thu, 19 Jan 2012 13:51:29 +0000 (UTC) Received: from CH1EHSMHS028.bigfish.com (snatpool1.int.messaging.microsoft.com [10.43.68.248]) by mail21-ch1.bigfish.com (Postfix) with ESMTP id 799D58006D; Thu, 19 Jan 2012 13:51:29 +0000 (UTC) Received: from TK5EX14MLTC101.redmond.corp.microsoft.com (131.107.125.8) by CH1EHSMHS028.bigfish.com (10.43.70.28) with Microsoft SMTP Server (TLS) id 14.1.225.23; Thu, 19 Jan 2012 13:51:28 +0000 Received: from TK5EX14MBXC118.redmond.corp.microsoft.com ([169.254.9.75]) by TK5EX14MLTC101.redmond.corp.microsoft.com ([157.54.79.178]) with mapi id 14.02.0247.005; Thu, 19 Jan 2012 05:51:27 -0800 From: Tom Talpey To: Hemal Shah , "david.black@emc.com" , "Michael@huaweisymantec.com" , "storm@ietf.org" Thread-Topic: iSER - one last issue Thread-Index: AQHM1jZ8MWLUJLTWDk+GYyBugVM345YTUwAAgABXYlA= Date: Thu, 19 Jan 2012 13:51:27 +0000 Message-ID: References: <20120115134011.27535.67315.idtracker@ietfa.amsl.com> <7C4DFCE962635144B8FAE8CA11D0BF1E05A7CF06E5@MX14A.corp.emc.com> <6315937B35624B2E9C599A392DDB8EA1@china.huawei.com> <7C4DFCE962635144B8FAE8CA11D0BF1E05A7CF0CF7@MX14A.corp.emc.com> <2D98DD3F898B6B4DA287BF3BA07DAE93022E4D@IRVEXCHMB11.corp.ad.broadcom.com> In-Reply-To: <2D98DD3F898B6B4DA287BF3BA07DAE93022E4D@IRVEXCHMB11.corp.ad.broadcom.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [157.54.51.33] Content-Type: multipart/alternative; boundary="_000_F83812DF4B59B9499C1BC978336D917461CDE2B8TK5EX14MBXC118r_" MIME-Version: 1.0 X-OriginatorOrg: microsoft.com Subject: Re: [storm] iSER - one last issue X-BeenThere: storm@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Storage Maintenance WG List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 19 Jan 2012 13:51:37 -0000 --_000_F83812DF4B59B9499C1BC978336D917461CDE2B8TK5EX14MBXC118r_ Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable If using SendSE is so important here, when MUST a plain-old Send be used? T= here is no mention of any distinction in RFC5046 section 1.4.2 nor in this = draft's section 2.4.2. Even with a MUST on the SendSE, the behavior is indeterminate, since it wou= ld still be contingent on support for SendSE at the *sender*. This critical= exception voids any normative requirement, even over iWARP. The strongest = statement that can be made is SHOULD. Tom. From: storm-bounces@ietf.org [mailto:storm-bounces@ietf.org] On Behalf Of H= emal Shah Sent: Wednesday, January 18, 2012 6:56 PM To: david.black@emc.com; Michael@huaweisymantec.com; storm@ietf.org Subject: Re: [storm] iSER - one last issue David and Mike, I believe that the requirements should not be weakened and we should stick = to the requirement in RFC5046. RFC5046 required the use of specific Send Message Type for valid reasons. In the case of requiring SendSE message use, the intent was to give a hint = to the remote side to generate an event upon processing of SendSE message. = For example, the SendSE is specifically carrying SCSI information that need= s attention on the remote side e.g. SCSI command, Logout... In the case of requiring SendInvSE Message, the intent was to invalidate th= e STag used in the data transfer as well as inform the remote side to gener= ate an event. For example, mandating the use of SendInvSE for the SCSI resp= onse PDU for a SCSI Read limits the exposure of STag used in SCSI Read data= transfer and avoids the explicit invalidation of the STag at the initiator= . Plus, it allows the target to generate an event on the initiator upon the= completion of SCSI Read command. By relaxing RFC5046 requirements in the latest iSER draft, we will not only= impact all iWARP based iSER implementations that rely on the use of specif= ic Send Message Type for SCSI data transfers but also change the intended b= ehavior of initiators and targets. For iWARP, I strongly suggest that the iSER draft does not relax the requir= ements specified in RFC5046. I hope that helps. Hemal From: storm-bounces@ietf.org [mailto:storm-b= ounces@ietf.org] On Behalf Of david= .black@emc.com Sent: Wednesday, January 18, 2012 3:10 PM To: Michael@huaweisymantec.com; storm@ie= tf.org Subject: Re: [storm] iSER - one last issue Mike, Thank you for the explanation, but I don't believe you've correctly stated the intent of RFC 5046. Here are some examples from RFC 5046's text: 1.5. SCSI Read Overview The iSER Message is transferred to the target using a SendSE Message. The iSER layer at the target uses a SendInvSE Message to transfer the SCSI Response PDU back to the iSER layer at the initiator. Similar language occurs in 1.6 SCSI Write Overview. 5.2.1. Normal Connection Termination at the Initiator The iSER layer at the initiator MUST use a SendSE Message to send the Logout Request PDU to the target. Similar language occurs in 5.2.2 for target side normal connection terminat= ion, and more importantly in 7.3.1 SCSI Command: The iSER layer at the initiator MUST send the SCSI command in a SendSE Message to the target. None of the quoted text permits use of Send instead of SendSE or SendInv instead of SendInvSE. What you propose to do effectively changes "MUST" to "should" (lower case, weaker than "SHOULD"), and that sure looks like a change to RFC 5046. Are there good implementation-based reasons to weaken these requirements no= w? Does anyone else have a viewpoint on this topic? Thanks, --David From: Michael Ko [mailto:Michael@huaweisymantec.com] Sent: Wednesday, January 18, 2012 2:22 PM To: Black, David; storm@ietf.org Subject: Re: iSER - one last issue David, Here is my rationale for using the lower case "should". The intent of RFC5046 is that the Send Message type must be used instead of= RDMA Reads or Writes. The Solicited Event feature of the Send Message is = provided as a convenience. The receiver must still do the right thing in h= andling the Send Message regardless of whether the SE feature is used. In = other words, the sender is responsible for using the right format for the m= essage (Send vs RDMA) but the receiver must not rely on the sender to deter= mine how to handle the received message. The same rationale goes for the I= nvalidate feature. Mike ----- Original Message ----- From: david.black@emc.com To: Michael@huaweisymantec.com ; storm@i= etf.org Sent: Monday, January 16, 2012 2:06 PM Subject: iSER - one last issue Mike, Thanks for getting the -08 version of the iSER draft posted. I think that draft addresses all of the open issues, but I have a question for the WG about how to express the SendSE and SendInvSE requirements from RFC 5046. The -08 version of the iSER draft expresses requirements for the SendSE and SendInvSE messages (this is primarily for iSER over RDDP/iWARP) as: The SendSE Message should be used if supported by the RCaP layer (e.g., iWARP). My reading of RFC 5046 is that its requirements are tighter - to accurately reflect RFC 5046, I would replace "should" with "MUST" in the above text, at least for iWARP. In the alternative, if the "should"s remain, an explanatory item needs to be added to the Appendix A list of changes from RFC 5046. What do others think the right course of action is here, use "MUST" or explain weakening of requirement to "should" ? Thanks, --David From: storm-bounces@ietf.org [mailto:storm-b= ounces@ietf.org] On Behalf Of Micha= el Ko Sent: Sunday, January 15, 2012 8:43 AM To: storm@ietf.org Subject: Re: [storm] I-D Action: draft-ietf-storm-iser-08.txt This version contains all the updates as discussed in the latest exchanges = with Hemal Shah and David Black. Mike ----- Original Message ----- From: internet-drafts@ietf.org To: i-d-announce@ietf.org Cc: storm@ietf.org Sent: Sunday, January 15, 2012 5:40 AM Subject: [storm] I-D Action: draft-ietf-storm-iser-08.txt A New Internet-Draft is available from the on-line Internet-Drafts director= ies. This draft is a work item of the STORage Maintenance Working Group of = the IETF. Title : iSCSI Extensions for RDMA Specification Author(s) : Michael Ko Alexander Nezhinsky Filename : draft-ietf-storm-iser-08.txt Pages : 95 Date : 2012-01-15 iSCSI Extensions for RDMA provides the RDMA data transfer capability to iSCSI by layering iSCSI on top of an RDMA-Capable Protocol. An RDMA-Capable Protocol provides RDMA Read and Write services, which enable data to be transferred directly into SCSI I/O Buffers without intermediate data copies. This document describes the extensions to the iSCSI protocol to support RDMA services as provided by an RDMA- Capable Protocol. This document obsoletes RFC 5046. A URL for this Internet-Draft is: http://www.ietf.org/internet-drafts/draft-ietf-storm-iser-08.txt Internet-Drafts are also available by anonymous FTP at: ftp://ftp.ietf.org/internet-drafts/ This Internet-Draft can be retrieved at: ftp://ftp.ietf.org/internet-drafts/draft-ietf-storm-iser-08.txt _______________________________________________ storm mailing list storm@ietf.org https://www.ietf.org/mailman/listinfo/storm --_000_F83812DF4B59B9499C1BC978336D917461CDE2B8TK5EX14MBXC118r_ Content-Type: text/html; charset="us-ascii" Content-Transfer-Encoding: quoted-printable

If using SendSE is so imp= ortant here, when MUST a plain-old Send be used? There is no mention of any= distinction in RFC5046 section 1.4.2 nor in this draft’s section 2.4.2.

 <= /p>

Even with a MUST on the S= endSE, the behavior is indeterminate, since it would still be contingent on= support for SendSE at the *sender*. This critical exception voids any normative requirement, even over iWARP. The strongest = statement that can be made is SHOULD.

 <= /p>

Tom.

 <= /p>

From: storm-bo= unces@ietf.org [mailto:storm-bounces@ietf.org] On Behalf Of Hemal Shah
Sent: Wednesday, January 18, 2012 6:56 PM
To: david.black@emc.com; Michael@huaweisymantec.com; storm@ietf.org<= br> Subject: Re: [storm] iSER - one last issue

 

Dav= id and Mike,

 

I b= elieve that the requirements should not be weakened and we should stick to = the requirement in RFC5046.

 

RFC= 5046 required the use of specific Send Message Type for valid reasons.=

 

In = the case of requiring SendSE message use, the intent was to give a hint to = the remote side to generate an event upon processing of SendSE message. For= example, the SendSE is specifically carrying SCSI information that needs attention on the remote side e.g. SCS= I command, Logout…

 

In = the case of requiring SendInvSE Message, the intent was to invalidate the S= Tag used in the data transfer as well as inform the remote side to generate= an event. For example, mandating the use of SendInvSE for the SCSI response PDU for a SCSI Read limits the expo= sure of STag used in SCSI Read data transfer and avoids the explicit invali= dation of the STag at the initiator. Plus, it allows the target to generate= an event on the initiator upon the completion of SCSI Read command.

 

By = relaxing RFC5046 requirements in the latest iSER draft, we will not only im= pact all iWARP based iSER implementations that rely on the use of specific = Send Message Type for SCSI data transfers but also change the intended behavior of initiators and targets.

 

For= iWARP, I strongly suggest that the iSER draft does not relax the requireme= nts specified in RFC5046.

 

I h= ope that helps.

 

Hem= al    

 

From: storm-bounces@ietf.org [mailto:storm-bounces@ietf.org] On Behalf Of david.black@emc.com
Sent: Wednesday, January 18, 2012 3:10 PM
To: Michael@huaweisyma= ntec.com; storm@ietf.org
Subject: Re: [storm] iSER - one last issue

 

Mike,

 

Thank you for the explanation, but I don’= ;t believe you’ve correctly stated

the intent of RFC 5046.  Here are some ex= amples from RFC 5046’s text:

 

1.5.  SCSI Read Overview

 

   The iSER Messag= e is transferred to the target using a SendSE Message.

 

   The iSER layer = at the target uses a SendInvSE Message to transfer the

   SCSI Response PDU back to the iSER layer at t= he initiator.

 

Similar language occurs in 1.6 SCSI Write Overview.

 

5.2.1.  Normal Connecti= on Termination at the Initiator

 

   The iSER layer = at the initiator MUST

   use a SendSE Me= ssage to send the Logout Request PDU to the target.

 

Similar language occurs in 5= .2.2 for target side normal connection termination,

and more importantly in 7.3.= 1 SCSI Command:

 

   The iSER layer = at the initiator MUST send the SCSI command in a

   SendSE Message = to the target.

 

None of the quoted text perm= its use of Send instead of SendSE or SendInv

instead of SendInvSE.

 

What you propose to do effec= tively changes “MUST” to “should” (lower case,=

weaker than “SHOULD= 221;), and that sure looks like a change to RFC 5046.

 

Are there good implementatio= n-based reasons to weaken these requirements now?

 

Does anyone else have a viewpoint on this topi= c?

 

Thanks,
--David

 

From: Michael = Ko [mailto:Michael@huaw= eisymantec.com]
Sent: Wednesday, January 18, 2012 2:22 PM
To: Black, David; storm@ietf.org
Subject: Re: iSER - one last issue

 

David,<= /o:p>

 

Here is my rational= e for using the lower case "should".

 

The intent of RFC50= 46 is that the Send Message type must be used instead of RDMA Reads or Writ= es.  The Solicited Event feature of the Send Message is provided = as a convenience.  The receiver must still do the right thing in handling the Send Message regardless of whether the SE = feature is used.  In other words, the sender is responsible for using = the right format for the message (Send vs RDMA) but the receiver must = not rely on the sender to determine how to handle the received message.  The same rationale goes for the Invalid= ate feature. 

 

Mike

 

Mike,

 

Thanks for getting the -08 version of the iSER= draft posted.  I think that

draft addresses all of the open issues, but I = have a question for the WG

about how to express the SendSE and SendInvSE = requirements from RFC 5046.

 

The -08 version of the iSER draft expresses re= quirements for the SendSE

and SendInvSE messages (this is primarily for = iSER over RDDP/iWARP) as:

 

The SendSE Message = should be used if

supported by the RC= aP layer (e.g., iWARP).

 

My reading of RFC 5046 is that its requirement= s are tighter - to accurately

reflect RFC 5046, I would replace “shoul= d” with “MUST” in the above text,

at least for iWARP.

 

In the alternative, if the “should”= ;s remain, an explanatory item

needs to be added to the Appendix A list of ch= anges from RFC 5046.

 

What do others think the right course of actio= n is here, use “MUST” or

explain weakening of requirement to “sho= uld” ?

 

Thanks,
--David

 

From: storm-bounces@ietf.org [mailto:storm-bounces@ietf.org] On Behalf Of Michael Ko
Sent: Sunday, January 15, 2012 8:43 AM
To: storm@ietf.org
Subject: Re: [storm] I-D Action: draft-ietf-storm-iser-08.txt

 

This version contai= ns all the updates as discussed in the latest exchanges with Hemal Shah and= David Black.

 

Mike

 


A New Internet-Draft is available from the on-line Internet-Drafts director= ies. This draft is a work item of the STORage Maintenance Working Group of = the IETF.

Title           : iSCSI E= xtensions for RDMA Specification
Author(s)       : Michael Ko
            &nb= sp;            = Alexander Nezhinsky
Filename        : draft-ietf-storm-iser-= 08.txt
Pages           : 95
Date            : 20= 12-01-15

   iSCSI Extensions for RDMA provides the RDMA data transfer capa= bility
   to iSCSI by layering iSCSI on top of an RDMA-Capable Protocol.=   An
   RDMA-Capable Protocol provides RDMA Read and Write services, w= hich
   enable data to be transferred directly into SCSI I/O Buffers w= ithout
   intermediate data copies.  This document describes the ex= tensions to
   the iSCSI protocol to support RDMA services as provided by an = RDMA-
   Capable Protocol.

   This document obsoletes RFC 5046.


A URL for this Internet-Draft is:
http://www.ietf.org/internet-drafts/draft-ietf-storm-iser-08.txt

Internet-Drafts are also available by anonymous FTP at:
ftp://ftp.ietf.org/internet= -drafts/

This Internet-Draft can be retrieved at:
ftp://ftp.ietf.org/internet-drafts/draft-ietf-storm-iser-08.txt

_______________________________________________
storm mailing list
storm@ietf.org
https://www.ietf.or= g/mailman/listinfo/storm

--_000_F83812DF4B59B9499C1BC978336D917461CDE2B8TK5EX14MBXC118r_-- From hemal@broadcom.com Thu Jan 19 07:38:52 2012 Return-Path: X-Original-To: storm@ietfa.amsl.com Delivered-To: storm@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 3883421F85F1 for ; Thu, 19 Jan 2012 07:38:52 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -6.598 X-Spam-Level: X-Spam-Status: No, score=-6.598 tagged_above=-999 required=5 tests=[AWL=-0.000, BAYES_00=-2.599, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_MED=-4] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id NKFyO-o2rH5y for ; Thu, 19 Jan 2012 07:38:47 -0800 (PST) Received: from MMS3.broadcom.com (mms3.broadcom.com [216.31.210.19]) by ietfa.amsl.com (Postfix) with ESMTP id 17E6F21F85DA for ; Thu, 19 Jan 2012 07:38:47 -0800 (PST) Received: from [10.9.208.26] by MMS3.broadcom.com with ESMTP (Broadcom SMTP Relay (Email Firewall v6.3.2)); Thu, 19 Jan 2012 07:46:37 -0800 X-Server-Uuid: B55A25B1-5D7D-41F8-BC53-C57E7AD3C201 Received: from IRVEXCHMB11.corp.ad.broadcom.com ( [fe80::9cdd:3a57:8694:f610]) by IRVEXCHCAS05.corp.ad.broadcom.com ( [::1]) with mapi id 14.01.0355.002; Thu, 19 Jan 2012 07:38:28 -0800 From: "Hemal Shah" To: "Tom Talpey" , "david.black@emc.com" , "Michael@huaweisymantec.com" , "storm@ietf.org" Thread-Topic: iSER - one last issue Thread-Index: AQHM1jZ9kWFrhlJkoESTJ1G4SNHEtpYSxIMQgAF4DYD//5ImkA== Date: Thu, 19 Jan 2012 15:38:27 +0000 Message-ID: <2D98DD3F898B6B4DA287BF3BA07DAE93023279@IRVEXCHMB11.corp.ad.broadcom.com> References: <20120115134011.27535.67315.idtracker@ietfa.amsl.com> <7C4DFCE962635144B8FAE8CA11D0BF1E05A7CF06E5@MX14A.corp.emc.com> <6315937B35624B2E9C599A392DDB8EA1@china.huawei.com> <7C4DFCE962635144B8FAE8CA11D0BF1E05A7CF0CF7@MX14A.corp.emc.com> <2D98DD3F898B6B4DA287BF3BA07DAE93022E4D@IRVEXCHMB11.corp.ad.broadcom.com> In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [10.9.253.114] MIME-Version: 1.0 X-WSS-ID: 6306E55750415533243-01-01 Content-Type: multipart/alternative; boundary=_000_2D98DD3F898B6B4DA287BF3BA07DAE93023279IRVEXCHMB11corpad_ Subject: Re: [storm] iSER - one last issue X-BeenThere: storm@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Storage Maintenance WG List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 19 Jan 2012 15:38:52 -0000 --_000_2D98DD3F898B6B4DA287BF3BA07DAE93023279IRVEXCHMB11corpad_ Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable Tom, RFC5046 is very clear about the use of Send Message Type for iSCSI control-= type PDU. For specific iSCSI PDUs, Section 7 mandates the use of Send, Send= SE, or SendSEInv. Section 7 of RFC5046 clearly specifies the use of specifi= c Send Message Type for each iSCSI control-type PDUs (Login, Logout, Text r= equest, Text response...). Section 7.3.4 mandates the use of plain-old Send= also as needed. For example, see text below from 7.3.4. For unsolicited data, if the F bit is set to 0 in a SCSI Data-out PDU, the iSER layer at the initiator MUST use a Send Message to send the SCSI Data-out PDU. Regarding your comment in the second paragraph below, RFC5040 expectation i= s all the RDMAP messages specified in RFC5040 are supported (if that is not= the case then we cannot count on any of the other RDMAP messages to be sup= ported). So, an implementation that does not support SendSE at the sender i= s not compliant with RFC5040. So, your second point does not apply. I hope that addresses your comments. Hemal From: Tom Talpey [mailto:ttalpey@microsoft.com] Sent: Thursday, January 19, 2012 5:51 AM To: Hemal Shah; david.black@emc.com; Michael@huaweisymantec.com; storm@ietf= .org Subject: RE: iSER - one last issue If using SendSE is so important here, when MUST a plain-old Send be used? T= here is no mention of any distinction in RFC5046 section 1.4.2 nor in this = draft's section 2.4.2. Even with a MUST on the SendSE, the behavior is indeterminate, since it wou= ld still be contingent on support for SendSE at the *sender*. This critical= exception voids any normative requirement, even over iWARP. The strongest = statement that can be made is SHOULD. Tom. From: storm-bounces@ietf.org [mailto:storm-bounces@ietf.org] On Behalf Of H= emal Shah Sent: Wednesday, January 18, 2012 6:56 PM To: david.black@emc.com; Michael@huaweisymantec.com; storm@ietf.org Subject: Re: [storm] iSER - one last issue David and Mike, I believe that the requirements should not be weakened and we should stick = to the requirement in RFC5046. RFC5046 required the use of specific Send Message Type for valid reasons. In the case of requiring SendSE message use, the intent was to give a hint = to the remote side to generate an event upon processing of SendSE message. = For example, the SendSE is specifically carrying SCSI information that need= s attention on the remote side e.g. SCSI command, Logout... In the case of requiring SendInvSE Message, the intent was to invalidate th= e STag used in the data transfer as well as inform the remote side to gener= ate an event. For example, mandating the use of SendInvSE for the SCSI resp= onse PDU for a SCSI Read limits the exposure of STag used in SCSI Read data= transfer and avoids the explicit invalidation of the STag at the initiator= . Plus, it allows the target to generate an event on the initiator upon the= completion of SCSI Read command. By relaxing RFC5046 requirements in the latest iSER draft, we will not only= impact all iWARP based iSER implementations that rely on the use of specif= ic Send Message Type for SCSI data transfers but also change the intended b= ehavior of initiators and targets. For iWARP, I strongly suggest that the iSER draft does not relax the requir= ements specified in RFC5046. I hope that helps. Hemal From: storm-bounces@ietf.org [mailto:storm-b= ounces@ietf.org] On Behalf Of david= .black@emc.com Sent: Wednesday, January 18, 2012 3:10 PM To: Michael@huaweisymantec.com; storm@ie= tf.org Subject: Re: [storm] iSER - one last issue Mike, Thank you for the explanation, but I don't believe you've correctly stated the intent of RFC 5046. Here are some examples from RFC 5046's text: 1.5. SCSI Read Overview The iSER Message is transferred to the target using a SendSE Message. The iSER layer at the target uses a SendInvSE Message to transfer the SCSI Response PDU back to the iSER layer at the initiator. Similar language occurs in 1.6 SCSI Write Overview. 5.2.1. Normal Connection Termination at the Initiator The iSER layer at the initiator MUST use a SendSE Message to send the Logout Request PDU to the target. Similar language occurs in 5.2.2 for target side normal connection terminat= ion, and more importantly in 7.3.1 SCSI Command: The iSER layer at the initiator MUST send the SCSI command in a SendSE Message to the target. None of the quoted text permits use of Send instead of SendSE or SendInv instead of SendInvSE. What you propose to do effectively changes "MUST" to "should" (lower case, weaker than "SHOULD"), and that sure looks like a change to RFC 5046. Are there good implementation-based reasons to weaken these requirements no= w? Does anyone else have a viewpoint on this topic? Thanks, --David From: Michael Ko [mailto:Michael@huaweisymantec.com] Sent: Wednesday, January 18, 2012 2:22 PM To: Black, David; storm@ietf.org Subject: Re: iSER - one last issue David, Here is my rationale for using the lower case "should". The intent of RFC5046 is that the Send Message type must be used instead of= RDMA Reads or Writes. The Solicited Event feature of the Send Message is = provided as a convenience. The receiver must still do the right thing in h= andling the Send Message regardless of whether the SE feature is used. In = other words, the sender is responsible for using the right format for the m= essage (Send vs RDMA) but the receiver must not rely on the sender to deter= mine how to handle the received message. The same rationale goes for the I= nvalidate feature. Mike ----- Original Message ----- From: david.black@emc.com To: Michael@huaweisymantec.com ; storm@i= etf.org Sent: Monday, January 16, 2012 2:06 PM Subject: iSER - one last issue Mike, Thanks for getting the -08 version of the iSER draft posted. I think that draft addresses all of the open issues, but I have a question for the WG about how to express the SendSE and SendInvSE requirements from RFC 5046. The -08 version of the iSER draft expresses requirements for the SendSE and SendInvSE messages (this is primarily for iSER over RDDP/iWARP) as: The SendSE Message should be used if supported by the RCaP layer (e.g., iWARP). My reading of RFC 5046 is that its requirements are tighter - to accurately reflect RFC 5046, I would replace "should" with "MUST" in the above text, at least for iWARP. In the alternative, if the "should"s remain, an explanatory item needs to be added to the Appendix A list of changes from RFC 5046. What do others think the right course of action is here, use "MUST" or explain weakening of requirement to "should" ? Thanks, --David From: storm-bounces@ietf.org [mailto:storm-b= ounces@ietf.org] On Behalf Of Micha= el Ko Sent: Sunday, January 15, 2012 8:43 AM To: storm@ietf.org Subject: Re: [storm] I-D Action: draft-ietf-storm-iser-08.txt This version contains all the updates as discussed in the latest exchanges = with Hemal Shah and David Black. Mike ----- Original Message ----- From: internet-drafts@ietf.org To: i-d-announce@ietf.org Cc: storm@ietf.org Sent: Sunday, January 15, 2012 5:40 AM Subject: [storm] I-D Action: draft-ietf-storm-iser-08.txt A New Internet-Draft is available from the on-line Internet-Drafts director= ies. This draft is a work item of the STORage Maintenance Working Group of = the IETF. Title : iSCSI Extensions for RDMA Specification Author(s) : Michael Ko Alexander Nezhinsky Filename : draft-ietf-storm-iser-08.txt Pages : 95 Date : 2012-01-15 iSCSI Extensions for RDMA provides the RDMA data transfer capability to iSCSI by layering iSCSI on top of an RDMA-Capable Protocol. An RDMA-Capable Protocol provides RDMA Read and Write services, which enable data to be transferred directly into SCSI I/O Buffers without intermediate data copies. This document describes the extensions to the iSCSI protocol to support RDMA services as provided by an RDMA- Capable Protocol. This document obsoletes RFC 5046. A URL for this Internet-Draft is: http://www.ietf.org/internet-drafts/draft-ietf-storm-iser-08.txt Internet-Drafts are also available by anonymous FTP at: ftp://ftp.ietf.org/internet-drafts/ This Internet-Draft can be retrieved at: ftp://ftp.ietf.org/internet-drafts/draft-ietf-storm-iser-08.txt _______________________________________________ storm mailing list storm@ietf.org https://www.ietf.org/mailman/listinfo/storm --_000_2D98DD3F898B6B4DA287BF3BA07DAE93023279IRVEXCHMB11corpad_ Content-Type: text/html; charset=us-ascii Content-Transfer-Encoding: quoted-printable

Tom= ,

 

RFC= 5046 is very clear about the use of Send Message Type for iSCSI control-typ= e PDU. For specific iSCSI PDUs, Section 7 mandates the use of Send, SendSE,= or SendSEInv. Section 7 of RFC5046 clearly specifies the use of specific Send Message Type for each iSCSI con= trol-type PDUs (Login, Logout, Text request, Text response…). Section= 7.3.4 mandates the use of plain-old Send also as needed. For example, see = text below from 7.3.4.

 

For unsolicited data, if the= F bit is set to 0 in a SCSI Data-out

PDU, the iSER layer at the i= nitiator MUST use a Send Message to send

the SCSI Data-out PDU.

 

Reg= arding your comment in the second paragraph below, RFC5040 expectation is a= ll the RDMAP messages specified in RFC5040 are supported (if that is not th= e case then we cannot count on any of the other RDMAP messages to be supported). So, an implementation that does= not support SendSE at the sender is not compliant with RFC5040. So, your s= econd point does not apply.

 

I h= ope that addresses your comments.

 

Hem= al

 

From: Tom Talp= ey [mailto:ttalpey@microsoft.com]
Sent: Thursday, January 19, 2012 5:51 AM
To: Hemal Shah; david.black@emc.com; Michael@huaweisymantec.com; sto= rm@ietf.org
Subject: RE: iSER - one last issue

 

If using SendSE is so imp= ortant here, when MUST a plain-old Send be used? There is no mention of any= distinction in RFC5046 section 1.4.2 nor in this draft’s section 2.4.2.

 <= /p>

Even with a MUST on the S= endSE, the behavior is indeterminate, since it would still be contingent on= support for SendSE at the *sender*. This critical exception voids any normative requirement, even over iWARP. The strongest = statement that can be made is SHOULD.

 <= /p>

Tom.

 <= /p>

From: storm-bo= unces@ietf.org [mailto:storm-bounces@ietf.org] On Behalf Of Hemal Shah
Sent: Wednesday, January 18, 2012 6:56 PM
To: david.black@emc.com; Michael@huaweisymantec.com; storm@ietf.org<= br> Subject: Re: [storm] iSER - one last issue

 

Dav= id and Mike,

 

I b= elieve that the requirements should not be weakened and we should stick to = the requirement in RFC5046.

 

RFC= 5046 required the use of specific Send Message Type for valid reasons.=

 

In = the case of requiring SendSE message use, the intent was to give a hint to = the remote side to generate an event upon processing of SendSE message. For= example, the SendSE is specifically carrying SCSI information that needs attention on the remote side e.g. SCS= I command, Logout…

 

In = the case of requiring SendInvSE Message, the intent was to invalidate the S= Tag used in the data transfer as well as inform the remote side to generate= an event. For example, mandating the use of SendInvSE for the SCSI response PDU for a SCSI Read limits the expo= sure of STag used in SCSI Read data transfer and avoids the explicit invali= dation of the STag at the initiator. Plus, it allows the target to generate= an event on the initiator upon the completion of SCSI Read command.

 

By = relaxing RFC5046 requirements in the latest iSER draft, we will not only im= pact all iWARP based iSER implementations that rely on the use of specific = Send Message Type for SCSI data transfers but also change the intended behavior of initiators and targets.

 

For= iWARP, I strongly suggest that the iSER draft does not relax the requireme= nts specified in RFC5046.

 

I h= ope that helps.

 

Hem= al    

 

From: storm-bounces@ietf.org [mailto:storm-bounces@ietf.org] On Behalf Of david.black@emc.com
Sent: Wednesday, January 18, 2012 3:10 PM
To: Michael@huaweisyma= ntec.com; storm@ietf.org
Subject: Re: [storm] iSER - one last issue

 

Mike,

 

Thank you for the explanation, but I don’= ;t believe you’ve correctly stated

the intent of RFC 5046.  Here are some ex= amples from RFC 5046’s text:

 

1.5.  SCSI Read Overview

 

   The iSER Messag= e is transferred to the target using a SendSE Message.

 

   The iSER layer = at the target uses a SendInvSE Message to transfer the

   SCSI Response PDU back to the iSER layer at t= he initiator.

 

Similar language occurs in 1.6 SCSI Write Overview.

 

5.2.1.  Normal Connecti= on Termination at the Initiator

 

   The iSER layer = at the initiator MUST

   use a SendSE Me= ssage to send the Logout Request PDU to the target.

 

Similar language occurs in 5= .2.2 for target side normal connection termination,

and more importantly in 7.3.= 1 SCSI Command:

 

   The iSER layer = at the initiator MUST send the SCSI command in a

   SendSE Message = to the target.

 

None of the quoted text perm= its use of Send instead of SendSE or SendInv

instead of SendInvSE.

 

What you propose to do effec= tively changes “MUST” to “should” (lower case,=

weaker than “SHOULD= 221;), and that sure looks like a change to RFC 5046.

 

Are there good implementatio= n-based reasons to weaken these requirements now?

 

Does anyone else have a viewpoint on this topi= c?

 

Thanks,
--David

 

From: Michael = Ko [mailto:Michael@huaw= eisymantec.com]
Sent: Wednesday, January 18, 2012 2:22 PM
To: Black, David; storm@ietf.org
Subject: Re: iSER - one last issue

 

David,<= /o:p>

 

Here is my rational= e for using the lower case "should".

 

The intent of RFC50= 46 is that the Send Message type must be used instead of RDMA Reads or Writ= es.  The Solicited Event feature of the Send Message is provided = as a convenience.  The receiver must still do the right thing in handling the Send Message regardless of whether the SE = feature is used.  In other words, the sender is responsible for using = the right format for the message (Send vs RDMA) but the receiver must = not rely on the sender to determine how to handle the received message.  The same rationale goes for the Invalid= ate feature. 

 

Mike

 

Mike,

 

Thanks for getting the -08 version of the iSER= draft posted.  I think that

draft addresses all of the open issues, but I = have a question for the WG

about how to express the SendSE and SendInvSE = requirements from RFC 5046.

 

The -08 version of the iSER draft expresses re= quirements for the SendSE

and SendInvSE messages (this is primarily for = iSER over RDDP/iWARP) as:

 

The SendSE Message = should be used if

supported by the RC= aP layer (e.g., iWARP).

 

My reading of RFC 5046 is that its requirement= s are tighter - to accurately

reflect RFC 5046, I would replace “shoul= d” with “MUST” in the above text,

at least for iWARP.

 

In the alternative, if the “should”= ;s remain, an explanatory item

needs to be added to the Appendix A list of ch= anges from RFC 5046.

 

What do others think the right course of actio= n is here, use “MUST” or

explain weakening of requirement to “sho= uld” ?

 

Thanks,
--David

 

From: storm-bounces@ietf.org [mailto:storm-bounces@ietf.org] On Behalf Of Michael Ko
Sent: Sunday, January 15, 2012 8:43 AM
To: storm@ietf.org
Subject: Re: [storm] I-D Action: draft-ietf-storm-iser-08.txt

 

This version contai= ns all the updates as discussed in the latest exchanges with Hemal Shah and= David Black.

 

Mike

 


A New Internet-Draft is available from the on-line Internet-Drafts director= ies. This draft is a work item of the STORage Maintenance Working Group of = the IETF.

Title           : iSCSI E= xtensions for RDMA Specification
Author(s)       : Michael Ko
            &nb= sp;            = Alexander Nezhinsky
Filename        : draft-ietf-storm-iser-= 08.txt
Pages           : 95
Date            : 20= 12-01-15

   iSCSI Extensions for RDMA provides the RDMA data transfer capa= bility
   to iSCSI by layering iSCSI on top of an RDMA-Capable Protocol.=   An
   RDMA-Capable Protocol provides RDMA Read and Write services, w= hich
   enable data to be transferred directly into SCSI I/O Buffers w= ithout
   intermediate data copies.  This document describes the ex= tensions to
   the iSCSI protocol to support RDMA services as provided by an = RDMA-
   Capable Protocol.

   This document obsoletes RFC 5046.


A URL for this Internet-Draft is:
http://www.ietf.org/internet-drafts/draft-ietf-storm-iser-08.txt

Internet-Drafts are also available by anonymous FTP at:
ftp://ftp.ietf.org/internet= -drafts/

This Internet-Draft can be retrieved at:
ftp://ftp.ietf.org/internet-drafts/draft-ietf-storm-iser-08.txt

_______________________________________________
storm mailing list
storm@ietf.org
https://www.ietf.or= g/mailman/listinfo/storm

--_000_2D98DD3F898B6B4DA287BF3BA07DAE93023279IRVEXCHMB11corpad_-- From ttalpey@microsoft.com Thu Jan 19 08:09:44 2012 Return-Path: X-Original-To: storm@ietfa.amsl.com Delivered-To: storm@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 559D721F863F for ; Thu, 19 Jan 2012 08:09:44 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -103.598 X-Spam-Level: X-Spam-Status: No, score=-103.598 tagged_above=-999 required=5 tests=[BAYES_00=-2.599, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_LOW=-1, USER_IN_WHITELIST=-100] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id ZsXxcN0FZEIc for ; Thu, 19 Jan 2012 08:09:38 -0800 (PST) Received: from DB3EHSOBE001.bigfish.com (db3ehsobe001.messaging.microsoft.com [213.199.154.139]) by ietfa.amsl.com (Postfix) with ESMTP id 8110D21F8637 for ; Thu, 19 Jan 2012 08:09:37 -0800 (PST) Received: from mail32-db3-R.bigfish.com (10.3.81.240) by DB3EHSOBE001.bigfish.com (10.3.84.21) with Microsoft SMTP Server id 14.1.225.23; Thu, 19 Jan 2012 16:09:35 +0000 Received: from mail32-db3 (localhost [127.0.0.1]) by mail32-db3-R.bigfish.com (Postfix) with ESMTP id 78759180539; Thu, 19 Jan 2012 16:09:35 +0000 (UTC) X-SpamScore: -48 X-BigFish: VS-48(zz9371I936eKc85fh148cM542M1418Mc1dM4015Lzz1202hzz1033IL8275bh8275dhz2fhc1bhc31hc1ahc1bhc31hc1ah2a8h668h839h) X-Forefront-Antispam-Report: CIP:131.107.125.8; KIP:(null); UIP:(null); IPV:NLI; H:TK5EX14HUBC107.redmond.corp.microsoft.com; RD:none; EFVD:NLI X-FB-SS: 13, Received-SPF: pass (mail32-db3: domain of microsoft.com designates 131.107.125.8 as permitted sender) client-ip=131.107.125.8; envelope-from=ttalpey@microsoft.com; helo=TK5EX14HUBC107.redmond.corp.microsoft.com ; icrosoft.com ; Received: from mail32-db3 (localhost.localdomain [127.0.0.1]) by mail32-db3 (MessageSwitch) id 1326989374601008_27863; Thu, 19 Jan 2012 16:09:34 +0000 (UTC) Received: from DB3EHSMHS009.bigfish.com (unknown [10.3.81.254]) by mail32-db3.bigfish.com (Postfix) with ESMTP id 81DF3120061; Thu, 19 Jan 2012 16:09:34 +0000 (UTC) Received: from TK5EX14HUBC107.redmond.corp.microsoft.com (131.107.125.8) by DB3EHSMHS009.bigfish.com (10.3.87.109) with Microsoft SMTP Server (TLS) id 14.1.225.23; Thu, 19 Jan 2012 16:09:33 +0000 Received: from TK5EX14MBXC118.redmond.corp.microsoft.com ([169.254.9.75]) by TK5EX14HUBC107.redmond.corp.microsoft.com ([157.54.80.67]) with mapi id 14.02.0247.005; Thu, 19 Jan 2012 08:09:21 -0800 From: Tom Talpey To: Hemal Shah , "david.black@emc.com" , "Michael@huaweisymantec.com" , "storm@ietf.org" Thread-Topic: iSER - one last issue Thread-Index: AQHM1jZ8MWLUJLTWDk+GYyBugVM345YTUwAAgABXYlCAALATgP//fm+A Date: Thu, 19 Jan 2012 16:09:21 +0000 Message-ID: References: <20120115134011.27535.67315.idtracker@ietfa.amsl.com> <7C4DFCE962635144B8FAE8CA11D0BF1E05A7CF06E5@MX14A.corp.emc.com> <6315937B35624B2E9C599A392DDB8EA1@china.huawei.com> <7C4DFCE962635144B8FAE8CA11D0BF1E05A7CF0CF7@MX14A.corp.emc.com> <2D98DD3F898B6B4DA287BF3BA07DAE93022E4D@IRVEXCHMB11.corp.ad.broadcom.com> <2D98DD3F898B6B4DA287BF3BA07DAE93023279@IRVEXCHMB11.corp.ad.broadcom.com> In-Reply-To: <2D98DD3F898B6B4DA287BF3BA07DAE93023279@IRVEXCHMB11.corp.ad.broadcom.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [157.54.51.33] Content-Type: multipart/alternative; boundary="_000_F83812DF4B59B9499C1BC978336D917461CDE441TK5EX14MBXC118r_" MIME-Version: 1.0 X-OriginatorOrg: microsoft.com Subject: Re: [storm] iSER - one last issue X-BeenThere: storm@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Storage Maintenance WG List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 19 Jan 2012 16:09:44 -0000 --_000_F83812DF4B59B9499C1BC978336D917461CDE441TK5EX14MBXC118r_ Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable Thanks for pointing out that sentence in RFC5046 7.3.4. I believe it is the= only such stipulation. I'm ok with stating the requirement as RFC5046-over-RFC5040 compliance. But= this new draft opens the door to alternative behaviors, therefore it needs= to be stated more clearly. What's the requirement when SendSE is not suppo= rted? And, how does the receiver know what to expect? In other words, if it's proposed that the statement be: "When the RCaP layer is as specified in [RFC5040] and [RFC5= 041], the SendSE message MUST be used." ... then what MUST be used when the RCaP is not? That statement needs to be= made, too. From: Hemal Shah [mailto:hemal@broadcom.com] Sent: Thursday, January 19, 2012 10:38 AM To: Tom Talpey; david.black@emc.com; Michael@huaweisymantec.com; storm@ietf= .org Subject: RE: iSER - one last issue Tom, RFC5046 is very clear about the use of Send Message Type for iSCSI control-= type PDU. For specific iSCSI PDUs, Section 7 mandates the use of Send, Send= SE, or SendSEInv. Section 7 of RFC5046 clearly specifies the use of specifi= c Send Message Type for each iSCSI control-type PDUs (Login, Logout, Text r= equest, Text response...). Section 7.3.4 mandates the use of plain-old Send= also as needed. For example, see text below from 7.3.4. For unsolicited data, if the F bit is set to 0 in a SCSI Data-out PDU, the iSER layer at the initiator MUST use a Send Message to send the SCSI Data-out PDU. Regarding your comment in the second paragraph below, RFC5040 expectation i= s all the RDMAP messages specified in RFC5040 are supported (if that is not= the case then we cannot count on any of the other RDMAP messages to be sup= ported). So, an implementation that does not support SendSE at the sender i= s not compliant with RFC5040. So, your second point does not apply. I hope that addresses your comments. Hemal From: Tom Talpey [mailto:ttalpey@microsoft.com] Sent: Thursday, January 19, 2012 5:51 AM To: Hemal Shah; david.black@emc.com; Michael@hu= aweisymantec.com; storm@ietf.org Subject: RE: iSER - one last issue If using SendSE is so important here, when MUST a plain-old Send be used? T= here is no mention of any distinction in RFC5046 section 1.4.2 nor in this = draft's section 2.4.2. Even with a MUST on the SendSE, the behavior is indeterminate, since it wou= ld still be contingent on support for SendSE at the *sender*. This critical= exception voids any normative requirement, even over iWARP. The strongest = statement that can be made is SHOULD. Tom. From: storm-bounces@ietf.org [mailto:storm-b= ounces@ietf.org] On Behalf Of Hemal= Shah Sent: Wednesday, January 18, 2012 6:56 PM To: david.black@emc.com; Michael@huaweisymantec= .com; storm@ietf.org Subject: Re: [storm] iSER - one last issue David and Mike, I believe that the requirements should not be weakened and we should stick = to the requirement in RFC5046. RFC5046 required the use of specific Send Message Type for valid reasons. In the case of requiring SendSE message use, the intent was to give a hint = to the remote side to generate an event upon processing of SendSE message. = For example, the SendSE is specifically carrying SCSI information that need= s attention on the remote side e.g. SCSI command, Logout... In the case of requiring SendInvSE Message, the intent was to invalidate th= e STag used in the data transfer as well as inform the remote side to gener= ate an event. For example, mandating the use of SendInvSE for the SCSI resp= onse PDU for a SCSI Read limits the exposure of STag used in SCSI Read data= transfer and avoids the explicit invalidation of the STag at the initiator= . Plus, it allows the target to generate an event on the initiator upon the= completion of SCSI Read command. By relaxing RFC5046 requirements in the latest iSER draft, we will not only= impact all iWARP based iSER implementations that rely on the use of specif= ic Send Message Type for SCSI data transfers but also change the intended b= ehavior of initiators and targets. For iWARP, I strongly suggest that the iSER draft does not relax the requir= ements specified in RFC5046. I hope that helps. Hemal From: storm-bounces@ietf.org [mailto:storm-b= ounces@ietf.org] On Behalf Of david= .black@emc.com Sent: Wednesday, January 18, 2012 3:10 PM To: Michael@huaweisymantec.com; storm@ie= tf.org Subject: Re: [storm] iSER - one last issue Mike, Thank you for the explanation, but I don't believe you've correctly stated the intent of RFC 5046. Here are some examples from RFC 5046's text: 1.5. SCSI Read Overview The iSER Message is transferred to the target using a SendSE Message. The iSER layer at the target uses a SendInvSE Message to transfer the SCSI Response PDU back to the iSER layer at the initiator. Similar language occurs in 1.6 SCSI Write Overview. 5.2.1. Normal Connection Termination at the Initiator The iSER layer at the initiator MUST use a SendSE Message to send the Logout Request PDU to the target. Similar language occurs in 5.2.2 for target side normal connection terminat= ion, and more importantly in 7.3.1 SCSI Command: The iSER layer at the initiator MUST send the SCSI command in a SendSE Message to the target. None of the quoted text permits use of Send instead of SendSE or SendInv instead of SendInvSE. What you propose to do effectively changes "MUST" to "should" (lower case, weaker than "SHOULD"), and that sure looks like a change to RFC 5046. Are there good implementation-based reasons to weaken these requirements no= w? Does anyone else have a viewpoint on this topic? Thanks, --David From: Michael Ko [mailto:Michael@huaweisymantec.com] Sent: Wednesday, January 18, 2012 2:22 PM To: Black, David; storm@ietf.org Subject: Re: iSER - one last issue David, Here is my rationale for using the lower case "should". The intent of RFC5046 is that the Send Message type must be used instead of= RDMA Reads or Writes. The Solicited Event feature of the Send Message is = provided as a convenience. The receiver must still do the right thing in h= andling the Send Message regardless of whether the SE feature is used. In = other words, the sender is responsible for using the right format for the m= essage (Send vs RDMA) but the receiver must not rely on the sender to deter= mine how to handle the received message. The same rationale goes for the I= nvalidate feature. Mike ----- Original Message ----- From: david.black@emc.com To: Michael@huaweisymantec.com ; storm@i= etf.org Sent: Monday, January 16, 2012 2:06 PM Subject: iSER - one last issue Mike, Thanks for getting the -08 version of the iSER draft posted. I think that draft addresses all of the open issues, but I have a question for the WG about how to express the SendSE and SendInvSE requirements from RFC 5046. The -08 version of the iSER draft expresses requirements for the SendSE and SendInvSE messages (this is primarily for iSER over RDDP/iWARP) as: The SendSE Message should be used if supported by the RCaP layer (e.g., iWARP). My reading of RFC 5046 is that its requirements are tighter - to accurately reflect RFC 5046, I would replace "should" with "MUST" in the above text, at least for iWARP. In the alternative, if the "should"s remain, an explanatory item needs to be added to the Appendix A list of changes from RFC 5046. What do others think the right course of action is here, use "MUST" or explain weakening of requirement to "should" ? Thanks, --David From: storm-bounces@ietf.org [mailto:storm-b= ounces@ietf.org] On Behalf Of Micha= el Ko Sent: Sunday, January 15, 2012 8:43 AM To: storm@ietf.org Subject: Re: [storm] I-D Action: draft-ietf-storm-iser-08.txt This version contains all the updates as discussed in the latest exchanges = with Hemal Shah and David Black. Mike ----- Original Message ----- From: internet-drafts@ietf.org To: i-d-announce@ietf.org Cc: storm@ietf.org Sent: Sunday, January 15, 2012 5:40 AM Subject: [storm] I-D Action: draft-ietf-storm-iser-08.txt A New Internet-Draft is available from the on-line Internet-Drafts director= ies. This draft is a work item of the STORage Maintenance Working Group of = the IETF. Title : iSCSI Extensions for RDMA Specification Author(s) : Michael Ko Alexander Nezhinsky Filename : draft-ietf-storm-iser-08.txt Pages : 95 Date : 2012-01-15 iSCSI Extensions for RDMA provides the RDMA data transfer capability to iSCSI by layering iSCSI on top of an RDMA-Capable Protocol. An RDMA-Capable Protocol provides RDMA Read and Write services, which enable data to be transferred directly into SCSI I/O Buffers without intermediate data copies. This document describes the extensions to the iSCSI protocol to support RDMA services as provided by an RDMA- Capable Protocol. This document obsoletes RFC 5046. A URL for this Internet-Draft is: http://www.ietf.org/internet-drafts/draft-ietf-storm-iser-08.txt Internet-Drafts are also available by anonymous FTP at: ftp://ftp.ietf.org/internet-drafts/ This Internet-Draft can be retrieved at: ftp://ftp.ietf.org/internet-drafts/draft-ietf-storm-iser-08.txt _______________________________________________ storm mailing list storm@ietf.org https://www.ietf.org/mailman/listinfo/storm --_000_F83812DF4B59B9499C1BC978336D917461CDE441TK5EX14MBXC118r_ Content-Type: text/html; charset="us-ascii" Content-Transfer-Encoding: quoted-printable

Thanks for pointing out t= hat sentence in RFC5046 7.3.4. I believe it is the only such stipulation.

 <= /p>

I’m ok with stating= the requirement as RFC5046-over-RFC5040 compliance. But this new draft ope= ns the door to alternative behaviors, therefore it needs to be stated more clearly. What’s the requirement when SendSE is not suppo= rted? And, how does the receiver know what to expect?

 <= /p>

In other words, if itR= 17;s proposed that the statement be:

    &= nbsp;           “Wh= en the RCaP layer is as specified in [RFC5040] and [RFC5041], the SendSE me= ssage MUST be used.”

 <= /p>

… then what MUST be= used when the RCaP is not? That statement needs to be made, too.

 <= /p>

 <= /p>

From: Hemal Sh= ah [mailto:hemal@broadcom.com]
Sent: Thursday, January 19, 2012 10:38 AM
To: Tom Talpey; david.black@emc.com; Michael@huaweisymantec.com; sto= rm@ietf.org
Subject: RE: iSER - one last issue

 

Tom= ,

 

RFC= 5046 is very clear about the use of Send Message Type for iSCSI control-typ= e PDU. For specific iSCSI PDUs, Section 7 mandates the use of Send, SendSE,= or SendSEInv. Section 7 of RFC5046 clearly specifies the use of specific Send Message Type for each iSCSI con= trol-type PDUs (Login, Logout, Text request, Text response…). Section= 7.3.4 mandates the use of plain-old Send also as needed. For example, see = text below from 7.3.4.

 

For unsolicited data, if the= F bit is set to 0 in a SCSI Data-out

PDU, the iSER layer at the i= nitiator MUST use a Send Message to send

the SCSI Data-out PDU.

 

Reg= arding your comment in the second paragraph below, RFC5040 expectation is a= ll the RDMAP messages specified in RFC5040 are supported (if that is not th= e case then we cannot count on any of the other RDMAP messages to be supported). So, an implementation that does= not support SendSE at the sender is not compliant with RFC5040. So, your s= econd point does not apply.

 

I h= ope that addresses your comments.

 

Hem= al

 

From: Tom Talp= ey [mailto:ttalpey@microsoft= .com]
Sent: Thursday, January 19, 2012 5:51 AM
To: Hemal Shah; david.black@e= mc.com; Michael@huaweisymantec.com; storm@ietf.org
Subject: RE: iSER - one last issue

 

If using SendSE is so imp= ortant here, when MUST a plain-old Send be used? There is no mention of any= distinction in RFC5046 section 1.4.2 nor in this draft’s section 2.4.2.

 <= /p>

Even with a MUST on the S= endSE, the behavior is indeterminate, since it would still be contingent on= support for SendSE at the *sender*. This critical exception voids any normative requirement, even over iWARP. The strongest = statement that can be made is SHOULD.

 <= /p>

Tom.

 <= /p>

From: storm-bounces@ietf.org [mailto:storm-bounces@ietf.org] On Behalf Of Hemal Shah
Sent: Wednesday, January 18, 2012 6:56 PM
To: david.black@emc.com; = Michael@huaweisymantec.com; storm@iet= f.org
Subject: Re: [storm] iSER - one last issue

 

Dav= id and Mike,

 

I b= elieve that the requirements should not be weakened and we should stick to = the requirement in RFC5046.

 

RFC= 5046 required the use of specific Send Message Type for valid reasons.=

 

In = the case of requiring SendSE message use, the intent was to give a hint to = the remote side to generate an event upon processing of SendSE message. For= example, the SendSE is specifically carrying SCSI information that needs attention on the remote side e.g. SCS= I command, Logout…

 

In = the case of requiring SendInvSE Message, the intent was to invalidate the S= Tag used in the data transfer as well as inform the remote side to generate= an event. For example, mandating the use of SendInvSE for the SCSI response PDU for a SCSI Read limits the expo= sure of STag used in SCSI Read data transfer and avoids the explicit invali= dation of the STag at the initiator. Plus, it allows the target to generate= an event on the initiator upon the completion of SCSI Read command.

 

By = relaxing RFC5046 requirements in the latest iSER draft, we will not only im= pact all iWARP based iSER implementations that rely on the use of specific = Send Message Type for SCSI data transfers but also change the intended behavior of initiators and targets.

 

For= iWARP, I strongly suggest that the iSER draft does not relax the requireme= nts specified in RFC5046.

 

I h= ope that helps.

 

Hem= al    

 

From: storm-bounces@ietf.org [mailto:storm-bounces@ietf.org] On Behalf Of david.black@emc.com
Sent: Wednesday, January 18, 2012 3:10 PM
To: Michael@huaweisyma= ntec.com; storm@ietf.org
Subject: Re: [storm] iSER - one last issue

 

Mike,

 

Thank you for the explanation, but I don’= ;t believe you’ve correctly stated

the intent of RFC 5046.  Here are some ex= amples from RFC 5046’s text:

 

1.5.  SCSI Read Overview

 

   The iSER Messag= e is transferred to the target using a SendSE Message.

 

   The iSER layer = at the target uses a SendInvSE Message to transfer the

   SCSI Response PDU back to the iSER layer at t= he initiator.

 

Similar language occurs in 1.6 SCSI Write Overview.

 

5.2.1.  Normal Connecti= on Termination at the Initiator

 

   The iSER layer = at the initiator MUST

   use a SendSE Me= ssage to send the Logout Request PDU to the target.

 

Similar language occurs in 5= .2.2 for target side normal connection termination,

and more importantly in 7.3.= 1 SCSI Command:

 

   The iSER layer = at the initiator MUST send the SCSI command in a

   SendSE Message = to the target.

 

None of the quoted text perm= its use of Send instead of SendSE or SendInv

instead of SendInvSE.

 

What you propose to do effec= tively changes “MUST” to “should” (lower case,=

weaker than “SHOULD= 221;), and that sure looks like a change to RFC 5046.

 

Are there good implementatio= n-based reasons to weaken these requirements now?

 

Does anyone else have a viewpoint on this topi= c?

 

Thanks,
--David

 

From: Michael = Ko [mailto:Michael@huaw= eisymantec.com]
Sent: Wednesday, January 18, 2012 2:22 PM
To: Black, David; storm@ietf.org
Subject: Re: iSER - one last issue

 

David,<= /o:p>

 

Here is my rational= e for using the lower case "should".

 

The intent of RFC50= 46 is that the Send Message type must be used instead of RDMA Reads or Writ= es.  The Solicited Event feature of the Send Message is provided = as a convenience.  The receiver must still do the right thing in handling the Send Message regardless of whether the SE = feature is used.  In other words, the sender is responsible for using = the right format for the message (Send vs RDMA) but the receiver must = not rely on the sender to determine how to handle the received message.  The same rationale goes for the Invalid= ate feature. 

 

Mike

 

Mike,

 

Thanks for getting the -08 version of the iSER= draft posted.  I think that

draft addresses all of the open issues, but I = have a question for the WG

about how to express the SendSE and SendInvSE = requirements from RFC 5046.

 

The -08 version of the iSER draft expresses re= quirements for the SendSE

and SendInvSE messages (this is primarily for = iSER over RDDP/iWARP) as:

 

The SendSE Message = should be used if

supported by the RC= aP layer (e.g., iWARP).

 

My reading of RFC 5046 is that its requirement= s are tighter - to accurately

reflect RFC 5046, I would replace “shoul= d” with “MUST” in the above text,

at least for iWARP.

 

In the alternative, if the “should”= ;s remain, an explanatory item

needs to be added to the Appendix A list of ch= anges from RFC 5046.

 

What do others think the right course of actio= n is here, use “MUST” or

explain weakening of requirement to “sho= uld” ?

 

Thanks,
--David

 

From: storm-bounces@ietf.org [mailto:storm-bounces@ietf.org] On Behalf Of Michael Ko
Sent: Sunday, January 15, 2012 8:43 AM
To: storm@ietf.org
Subject: Re: [storm] I-D Action: draft-ietf-storm-iser-08.txt

 

This version contai= ns all the updates as discussed in the latest exchanges with Hemal Shah and= David Black.

 

Mike

 


A New Internet-Draft is available from the on-line Internet-Drafts director= ies. This draft is a work item of the STORage Maintenance Working Group of = the IETF.

Title           : iSCSI E= xtensions for RDMA Specification
Author(s)       : Michael Ko
            &nb= sp;            = Alexander Nezhinsky
Filename        : draft-ietf-storm-iser-= 08.txt
Pages           : 95
Date            : 20= 12-01-15

   iSCSI Extensions for RDMA provides the RDMA data transfer capa= bility
   to iSCSI by layering iSCSI on top of an RDMA-Capable Protocol.=   An
   RDMA-Capable Protocol provides RDMA Read and Write services, w= hich
   enable data to be transferred directly into SCSI I/O Buffers w= ithout
   intermediate data copies.  This document describes the ex= tensions to
   the iSCSI protocol to support RDMA services as provided by an = RDMA-
   Capable Protocol.

   This document obsoletes RFC 5046.


A URL for this Internet-Draft is:
http://www.ietf.org/internet-drafts/draft-ietf-storm-iser-08.txt

Internet-Drafts are also available by anonymous FTP at:
ftp://ftp.ietf.org/internet= -drafts/

This Internet-Draft can be retrieved at:
ftp://ftp.ietf.org/internet-drafts/draft-ietf-storm-iser-08.txt

_______________________________________________
storm mailing list
storm@ietf.org
https://www.ietf.or= g/mailman/listinfo/storm

--_000_F83812DF4B59B9499C1BC978336D917461CDE441TK5EX14MBXC118r_-- From hemal@broadcom.com Thu Jan 19 08:47:20 2012 Return-Path: X-Original-To: storm@ietfa.amsl.com Delivered-To: storm@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 2F95621F8605 for ; Thu, 19 Jan 2012 08:47:20 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -6.598 X-Spam-Level: X-Spam-Status: No, score=-6.598 tagged_above=-999 required=5 tests=[AWL=-0.000, BAYES_00=-2.599, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_MED=-4] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id b4pavPDDwKsO for ; Thu, 19 Jan 2012 08:47:14 -0800 (PST) Received: from MMS3.broadcom.com (mms3.broadcom.com [216.31.210.19]) by ietfa.amsl.com (Postfix) with ESMTP id E070A21F8604 for ; Thu, 19 Jan 2012 08:47:13 -0800 (PST) Received: from [10.9.208.27] by MMS3.broadcom.com with ESMTP (Broadcom SMTP Relay (Email Firewall v6.3.2)); Thu, 19 Jan 2012 08:52:54 -0800 X-Server-Uuid: B55A25B1-5D7D-41F8-BC53-C57E7AD3C201 Received: from IRVEXCHMB11.corp.ad.broadcom.com ( [fe80::9cdd:3a57:8694:f610]) by irvexchmb05.corp.ad.broadcom.com ( [::1]) with mapi id 14.01.0289.001; Thu, 19 Jan 2012 08:44:45 -0800 From: "Hemal Shah" To: "Tom Talpey" , "david.black@emc.com" , "Michael@huaweisymantec.com" , "storm@ietf.org" Thread-Topic: iSER - one last issue Thread-Index: AQHM1jZ9kWFrhlJkoESTJ1G4SNHEtpYSxIMQgAF4DYD//5ImkIAAlGGA//+Au2A= Date: Thu, 19 Jan 2012 16:44:44 +0000 Message-ID: <2D98DD3F898B6B4DA287BF3BA07DAE930234C6@IRVEXCHMB11.corp.ad.broadcom.com> References: <20120115134011.27535.67315.idtracker@ietfa.amsl.com> <7C4DFCE962635144B8FAE8CA11D0BF1E05A7CF06E5@MX14A.corp.emc.com> <6315937B35624B2E9C599A392DDB8EA1@china.huawei.com> <7C4DFCE962635144B8FAE8CA11D0BF1E05A7CF0CF7@MX14A.corp.emc.com> <2D98DD3F898B6B4DA287BF3BA07DAE93022E4D@IRVEXCHMB11.corp.ad.broadcom.com> <2D98DD3F898B6B4DA287BF3BA07DAE93023279@IRVEXCHMB11.corp.ad.broadcom.com> In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [10.9.253.114] MIME-Version: 1.0 X-WSS-ID: 630695EC50415552183-01-01 Content-Type: multipart/alternative; boundary=_000_2D98DD3F898B6B4DA287BF3BA07DAE930234C6IRVEXCHMB11corpad_ Subject: Re: [storm] iSER - one last issue X-BeenThere: storm@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Storage Maintenance WG List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 19 Jan 2012 16:47:20 -0000 --_000_2D98DD3F898B6B4DA287BF3BA07DAE930234C6IRVEXCHMB11corpad_ Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable Tom, You are welcome! You are right about 7.3.4 as the only section mandating Se= nd message. I think your concern about specifying the requirement for the other RCaP im= plementation is valid. I suggest we should word the requirements as below: "If the RCaP layer is as specified in [RFC5040] and [RFC5041], the SendSE m= essage MUST be used. For any other RCaP layer, the Send message SHOULD be u= sed. " Hemal From: Tom Talpey [mailto:ttalpey@microsoft.com] Sent: Thursday, January 19, 2012 8:09 AM To: Hemal Shah; david.black@emc.com; Michael@huaweisymantec.com; storm@ietf= .org Subject: RE: iSER - one last issue Thanks for pointing out that sentence in RFC5046 7.3.4. I believe it is the= only such stipulation. I'm ok with stating the requirement as RFC5046-over-RFC5040 compliance. But= this new draft opens the door to alternative behaviors, therefore it needs= to be stated more clearly. What's the requirement when SendSE is not suppo= rted? And, how does the receiver know what to expect? In other words, if it's proposed that the statement be: "When the RCaP layer is as specified in [RFC5040] and [RFC5= 041], the SendSE message MUST be used." ... then what MUST be used when the RCaP is not? That statement needs to be= made, too. From: Hemal Shah [mailto:hemal@broadcom.com] Sent: Thursday, January 19, 2012 10:38 AM To: Tom Talpey; david.black@emc.com; Michael@huaweisymantec.com; storm@ietf= .org Subject: RE: iSER - one last issue Tom, RFC5046 is very clear about the use of Send Message Type for iSCSI control-= type PDU. For specific iSCSI PDUs, Section 7 mandates the use of Send, Send= SE, or SendSEInv. Section 7 of RFC5046 clearly specifies the use of specifi= c Send Message Type for each iSCSI control-type PDUs (Login, Logout, Text r= equest, Text response...). Section 7.3.4 mandates the use of plain-old Send= also as needed. For example, see text below from 7.3.4. For unsolicited data, if the F bit is set to 0 in a SCSI Data-out PDU, the iSER layer at the initiator MUST use a Send Message to send the SCSI Data-out PDU. Regarding your comment in the second paragraph below, RFC5040 expectation i= s all the RDMAP messages specified in RFC5040 are supported (if that is not= the case then we cannot count on any of the other RDMAP messages to be sup= ported). So, an implementation that does not support SendSE at the sender i= s not compliant with RFC5040. So, your second point does not apply. I hope that addresses your comments. Hemal From: Tom Talpey [mailto:ttalpey@microsoft.com] Sent: Thursday, January 19, 2012 5:51 AM To: Hemal Shah; david.black@emc.com; Michael@hu= aweisymantec.com; storm@ietf.org Subject: RE: iSER - one last issue If using SendSE is so important here, when MUST a plain-old Send be used? T= here is no mention of any distinction in RFC5046 section 1.4.2 nor in this = draft's section 2.4.2. Even with a MUST on the SendSE, the behavior is indeterminate, since it wou= ld still be contingent on support for SendSE at the *sender*. This critical= exception voids any normative requirement, even over iWARP. The strongest = statement that can be made is SHOULD. Tom. From: storm-bounces@ietf.org [mailto:storm-b= ounces@ietf.org] On Behalf Of Hemal= Shah Sent: Wednesday, January 18, 2012 6:56 PM To: david.black@emc.com; Michael@huaweisymantec= .com; storm@ietf.org Subject: Re: [storm] iSER - one last issue David and Mike, I believe that the requirements should not be weakened and we should stick = to the requirement in RFC5046. RFC5046 required the use of specific Send Message Type for valid reasons. In the case of requiring SendSE message use, the intent was to give a hint = to the remote side to generate an event upon processing of SendSE message. = For example, the SendSE is specifically carrying SCSI information that need= s attention on the remote side e.g. SCSI command, Logout... In the case of requiring SendInvSE Message, the intent was to invalidate th= e STag used in the data transfer as well as inform the remote side to gener= ate an event. For example, mandating the use of SendInvSE for the SCSI resp= onse PDU for a SCSI Read limits the exposure of STag used in SCSI Read data= transfer and avoids the explicit invalidation of the STag at the initiator= . Plus, it allows the target to generate an event on the initiator upon the= completion of SCSI Read command. By relaxing RFC5046 requirements in the latest iSER draft, we will not only= impact all iWARP based iSER implementations that rely on the use of specif= ic Send Message Type for SCSI data transfers but also change the intended b= ehavior of initiators and targets. For iWARP, I strongly suggest that the iSER draft does not relax the requir= ements specified in RFC5046. I hope that helps. Hemal From: storm-bounces@ietf.org [mailto:storm-b= ounces@ietf.org] On Behalf Of david= .black@emc.com Sent: Wednesday, January 18, 2012 3:10 PM To: Michael@huaweisymantec.com; storm@ie= tf.org Subject: Re: [storm] iSER - one last issue Mike, Thank you for the explanation, but I don't believe you've correctly stated the intent of RFC 5046. Here are some examples from RFC 5046's text: 1.5. SCSI Read Overview The iSER Message is transferred to the target using a SendSE Message. The iSER layer at the target uses a SendInvSE Message to transfer the SCSI Response PDU back to the iSER layer at the initiator. Similar language occurs in 1.6 SCSI Write Overview. 5.2.1. Normal Connection Termination at the Initiator The iSER layer at the initiator MUST use a SendSE Message to send the Logout Request PDU to the target. Similar language occurs in 5.2.2 for target side normal connection terminat= ion, and more importantly in 7.3.1 SCSI Command: The iSER layer at the initiator MUST send the SCSI command in a SendSE Message to the target. None of the quoted text permits use of Send instead of SendSE or SendInv instead of SendInvSE. What you propose to do effectively changes "MUST" to "should" (lower case, weaker than "SHOULD"), and that sure looks like a change to RFC 5046. Are there good implementation-based reasons to weaken these requirements no= w? Does anyone else have a viewpoint on this topic? Thanks, --David From: Michael Ko [mailto:Michael@huaweisymantec.com] Sent: Wednesday, January 18, 2012 2:22 PM To: Black, David; storm@ietf.org Subject: Re: iSER - one last issue David, Here is my rationale for using the lower case "should". The intent of RFC5046 is that the Send Message type must be used instead of= RDMA Reads or Writes. The Solicited Event feature of the Send Message is = provided as a convenience. The receiver must still do the right thing in h= andling the Send Message regardless of whether the SE feature is used. In = other words, the sender is responsible for using the right format for the m= essage (Send vs RDMA) but the receiver must not rely on the sender to deter= mine how to handle the received message. The same rationale goes for the I= nvalidate feature. Mike ----- Original Message ----- From: david.black@emc.com To: Michael@huaweisymantec.com ; storm@i= etf.org Sent: Monday, January 16, 2012 2:06 PM Subject: iSER - one last issue Mike, Thanks for getting the -08 version of the iSER draft posted. I think that draft addresses all of the open issues, but I have a question for the WG about how to express the SendSE and SendInvSE requirements from RFC 5046. The -08 version of the iSER draft expresses requirements for the SendSE and SendInvSE messages (this is primarily for iSER over RDDP/iWARP) as: The SendSE Message should be used if supported by the RCaP layer (e.g., iWARP). My reading of RFC 5046 is that its requirements are tighter - to accurately reflect RFC 5046, I would replace "should" with "MUST" in the above text, at least for iWARP. In the alternative, if the "should"s remain, an explanatory item needs to be added to the Appendix A list of changes from RFC 5046. What do others think the right course of action is here, use "MUST" or explain weakening of requirement to "should" ? Thanks, --David From: storm-bounces@ietf.org [mailto:storm-b= ounces@ietf.org] On Behalf Of Micha= el Ko Sent: Sunday, January 15, 2012 8:43 AM To: storm@ietf.org Subject: Re: [storm] I-D Action: draft-ietf-storm-iser-08.txt This version contains all the updates as discussed in the latest exchanges = with Hemal Shah and David Black. Mike ----- Original Message ----- From: internet-drafts@ietf.org To: i-d-announce@ietf.org Cc: storm@ietf.org Sent: Sunday, January 15, 2012 5:40 AM Subject: [storm] I-D Action: draft-ietf-storm-iser-08.txt A New Internet-Draft is available from the on-line Internet-Drafts director= ies. This draft is a work item of the STORage Maintenance Working Group of = the IETF. Title : iSCSI Extensions for RDMA Specification Author(s) : Michael Ko Alexander Nezhinsky Filename : draft-ietf-storm-iser-08.txt Pages : 95 Date : 2012-01-15 iSCSI Extensions for RDMA provides the RDMA data transfer capability to iSCSI by layering iSCSI on top of an RDMA-Capable Protocol. An RDMA-Capable Protocol provides RDMA Read and Write services, which enable data to be transferred directly into SCSI I/O Buffers without intermediate data copies. This document describes the extensions to the iSCSI protocol to support RDMA services as provided by an RDMA- Capable Protocol. This document obsoletes RFC 5046. A URL for this Internet-Draft is: http://www.ietf.org/internet-drafts/draft-ietf-storm-iser-08.txt Internet-Drafts are also available by anonymous FTP at: ftp://ftp.ietf.org/internet-drafts/ This Internet-Draft can be retrieved at: ftp://ftp.ietf.org/internet-drafts/draft-ietf-storm-iser-08.txt _______________________________________________ storm mailing list storm@ietf.org https://www.ietf.org/mailman/listinfo/storm --_000_2D98DD3F898B6B4DA287BF3BA07DAE930234C6IRVEXCHMB11corpad_ Content-Type: text/html; charset=us-ascii Content-Transfer-Encoding: quoted-printable

Tom= ,

 

You= are welcome! You are right about 7.3.4 as the only section mandating Send = message.

 

I t= hink your concern about specifying the requirement for the other RCaP imple= mentation is valid. I suggest we should word the requirements as below:

 

“If the RCaP layer = is as specified in [RFC5040] and [RFC5041], the SendSE message MUST be used= . For any other RCaP layer, the Send message SHOULD be used. ”

 <= /p>

Hem= al

 

From: Tom Talp= ey [mailto:ttalpey@microsoft.com]
Sent: Thursday, January 19, 2012 8:09 AM
To: Hemal Shah; david.black@emc.com; Michael@huaweisymantec.com; sto= rm@ietf.org
Subject: RE: iSER - one last issue

 

Thanks for pointing out t= hat sentence in RFC5046 7.3.4. I believe it is the only such stipulation.

 <= /p>

I’m ok with stating= the requirement as RFC5046-over-RFC5040 compliance. But this new draft ope= ns the door to alternative behaviors, therefore it needs to be stated more clearly. What’s the requirement when SendSE is not suppo= rted? And, how does the receiver know what to expect?

 <= /p>

In other words, if itR= 17;s proposed that the statement be:

    &= nbsp;           “Wh= en the RCaP layer is as specified in [RFC5040] and [RFC5041], the SendSE me= ssage MUST be used.”

 <= /p>

… then what MUST be= used when the RCaP is not? That statement needs to be made, too.

 <= /p>

 <= /p>

From: Hemal Sh= ah [mailto:hemal@broadcom.com]
Sent: Thursday, January 19, 2012 10:38 AM
To: Tom Talpey; david.black@emc.com; Michael@huaweisymantec.com; sto= rm@ietf.org
Subject: RE: iSER - one last issue

 

Tom= ,

 

RFC= 5046 is very clear about the use of Send Message Type for iSCSI control-typ= e PDU. For specific iSCSI PDUs, Section 7 mandates the use of Send, SendSE,= or SendSEInv. Section 7 of RFC5046 clearly specifies the use of specific Send Message Type for each iSCSI con= trol-type PDUs (Login, Logout, Text request, Text response…). Section= 7.3.4 mandates the use of plain-old Send also as needed. For example, see = text below from 7.3.4.

 

For unsolicited data, if the= F bit is set to 0 in a SCSI Data-out

PDU, the iSER layer at the i= nitiator MUST use a Send Message to send

the SCSI Data-out PDU.

 

Reg= arding your comment in the second paragraph below, RFC5040 expectation is a= ll the RDMAP messages specified in RFC5040 are supported (if that is not th= e case then we cannot count on any of the other RDMAP messages to be supported). So, an implementation that does= not support SendSE at the sender is not compliant with RFC5040. So, your s= econd point does not apply.

 

I h= ope that addresses your comments.

 

Hem= al

 

From: Tom Talp= ey [mailto:ttalpey@microsoft= .com]
Sent: Thursday, January 19, 2012 5:51 AM
To: Hemal Shah; david.black@e= mc.com; Michael@huaweisymantec.com; storm@ietf.org
Subject: RE: iSER - one last issue

 

If using SendSE is so imp= ortant here, when MUST a plain-old Send be used? There is no mention of any= distinction in RFC5046 section 1.4.2 nor in this draft’s section 2.4.2.

 <= /p>

Even with a MUST on the S= endSE, the behavior is indeterminate, since it would still be contingent on= support for SendSE at the *sender*. This critical exception voids any normative requirement, even over iWARP. The strongest = statement that can be made is SHOULD.

 <= /p>

Tom.

 <= /p>

From: storm-bounces@ietf.org [mailto:storm-bounces@ietf.org] On Behalf Of Hemal Shah
Sent: Wednesday, January 18, 2012 6:56 PM
To: david.black@emc.com; = Michael@huaweisymantec.com; storm@iet= f.org
Subject: Re: [storm] iSER - one last issue

 

Dav= id and Mike,

 

I b= elieve that the requirements should not be weakened and we should stick to = the requirement in RFC5046.

 

RFC= 5046 required the use of specific Send Message Type for valid reasons.=

 

In = the case of requiring SendSE message use, the intent was to give a hint to = the remote side to generate an event upon processing of SendSE message. For= example, the SendSE is specifically carrying SCSI information that needs attention on the remote side e.g. SCS= I command, Logout…

 

In = the case of requiring SendInvSE Message, the intent was to invalidate the S= Tag used in the data transfer as well as inform the remote side to generate= an event. For example, mandating the use of SendInvSE for the SCSI response PDU for a SCSI Read limits the expo= sure of STag used in SCSI Read data transfer and avoids the explicit invali= dation of the STag at the initiator. Plus, it allows the target to generate= an event on the initiator upon the completion of SCSI Read command.

 

By = relaxing RFC5046 requirements in the latest iSER draft, we will not only im= pact all iWARP based iSER implementations that rely on the use of specific = Send Message Type for SCSI data transfers but also change the intended behavior of initiators and targets.

 

For= iWARP, I strongly suggest that the iSER draft does not relax the requireme= nts specified in RFC5046.

 

I h= ope that helps.

 

Hem= al    

 

From: storm-bounces@ietf.org [mailto:storm-bounces@ietf.org] On Behalf Of david.black@emc.com
Sent: Wednesday, January 18, 2012 3:10 PM
To: Michael@huaweisyma= ntec.com; storm@ietf.org
Subject: Re: [storm] iSER - one last issue

 

Mike,

 

Thank you for the explanation, but I don’= ;t believe you’ve correctly stated

the intent of RFC 5046.  Here are some ex= amples from RFC 5046’s text:

 

1.5.  SCSI Read Overview

 

   The iSER Messag= e is transferred to the target using a SendSE Message.

 

   The iSER layer = at the target uses a SendInvSE Message to transfer the

   SCSI Response PDU back to the iSER layer at t= he initiator.

 

Similar language occurs in 1.6 SCSI Write Overview.

 

5.2.1.  Normal Connecti= on Termination at the Initiator

 

   The iSER layer = at the initiator MUST

   use a SendSE Me= ssage to send the Logout Request PDU to the target.

 

Similar language occurs in 5= .2.2 for target side normal connection termination,

and more importantly in 7.3.= 1 SCSI Command:

 

   The iSER layer = at the initiator MUST send the SCSI command in a

   SendSE Message = to the target.

 

None of the quoted text perm= its use of Send instead of SendSE or SendInv

instead of SendInvSE.

 

What you propose to do effec= tively changes “MUST” to “should” (lower case,=

weaker than “SHOULD= 221;), and that sure looks like a change to RFC 5046.

 

Are there good implementatio= n-based reasons to weaken these requirements now?

 

Does anyone else have a viewpoint on this topi= c?

 

Thanks,
--David

 

From: Michael = Ko [mailto:Michael@huaw= eisymantec.com]
Sent: Wednesday, January 18, 2012 2:22 PM
To: Black, David; storm@ietf.org
Subject: Re: iSER - one last issue

 

David,<= /o:p>

 

Here is my rational= e for using the lower case "should".

 

The intent of RFC50= 46 is that the Send Message type must be used instead of RDMA Reads or Writ= es.  The Solicited Event feature of the Send Message is provided = as a convenience.  The receiver must still do the right thing in handling the Send Message regardless of whether the SE = feature is used.  In other words, the sender is responsible for using = the right format for the message (Send vs RDMA) but the receiver must = not rely on the sender to determine how to handle the received message.  The same rationale goes for the Invalid= ate feature. 

 

Mike

 

Mike,

 

Thanks for getting the -08 version of the iSER= draft posted.  I think that

draft addresses all of the open issues, but I = have a question for the WG

about how to express the SendSE and SendInvSE = requirements from RFC 5046.

 

The -08 version of the iSER draft expresses re= quirements for the SendSE

and SendInvSE messages (this is primarily for = iSER over RDDP/iWARP) as:

 

The SendSE Message = should be used if

supported by the RC= aP layer (e.g., iWARP).

 

My reading of RFC 5046 is that its requirement= s are tighter - to accurately

reflect RFC 5046, I would replace “shoul= d” with “MUST” in the above text,

at least for iWARP.

 

In the alternative, if the “should”= ;s remain, an explanatory item

needs to be added to the Appendix A list of ch= anges from RFC 5046.

 

What do others think the right course of actio= n is here, use “MUST” or

explain weakening of requirement to “sho= uld” ?

 

Thanks,
--David

 

From: storm-bounces@ietf.org [mailto:storm-bounces@ietf.org] On Behalf Of Michael Ko
Sent: Sunday, January 15, 2012 8:43 AM
To: storm@ietf.org
Subject: Re: [storm] I-D Action: draft-ietf-storm-iser-08.txt

 

This version contai= ns all the updates as discussed in the latest exchanges with Hemal Shah and= David Black.

 

Mike

 


A New Internet-Draft is available from the on-line Internet-Drafts director= ies. This draft is a work item of the STORage Maintenance Working Group of = the IETF.

Title           : iSCSI E= xtensions for RDMA Specification
Author(s)       : Michael Ko
            &nb= sp;            = Alexander Nezhinsky
Filename        : draft-ietf-storm-iser-= 08.txt
Pages           : 95
Date            : 20= 12-01-15

   iSCSI Extensions for RDMA provides the RDMA data transfer capa= bility
   to iSCSI by layering iSCSI on top of an RDMA-Capable Protocol.=   An
   RDMA-Capable Protocol provides RDMA Read and Write services, w= hich
   enable data to be transferred directly into SCSI I/O Buffers w= ithout
   intermediate data copies.  This document describes the ex= tensions to
   the iSCSI protocol to support RDMA services as provided by an = RDMA-
   Capable Protocol.

   This document obsoletes RFC 5046.


A URL for this Internet-Draft is:
http://www.ietf.org/internet-drafts/draft-ietf-storm-iser-08.txt

Internet-Drafts are also available by anonymous FTP at:
ftp://ftp.ietf.org/internet= -drafts/

This Internet-Draft can be retrieved at:
ftp://ftp.ietf.org/internet-drafts/draft-ietf-storm-iser-08.txt

_______________________________________________
storm mailing list
storm@ietf.org
https://www.ietf.or= g/mailman/listinfo/storm

--_000_2D98DD3F898B6B4DA287BF3BA07DAE930234C6IRVEXCHMB11corpad_-- From Michael@huaweisymantec.com Thu Jan 19 09:38:42 2012 Return-Path: X-Original-To: storm@ietfa.amsl.com Delivered-To: storm@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 92B3221F8619 for ; Thu, 19 Jan 2012 09:38:42 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.592 X-Spam-Level: X-Spam-Status: No, score=-1.592 tagged_above=-999 required=5 tests=[AWL=-1.006, BAYES_00=-2.599, FAKE_REPLY_C=2.012, HTML_MESSAGE=0.001] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id N6GKXn+dELam for ; Thu, 19 Jan 2012 09:38:41 -0800 (PST) Received: from mta1.huaweisymantec.com (mta1.huaweisymantec.com [218.17.155.14]) by ietfa.amsl.com (Postfix) with ESMTP id 5925321F8642 for ; Thu, 19 Jan 2012 09:38:40 -0800 (PST) MIME-version: 1.0 Content-type: multipart/alternative; boundary="Boundary_(ID_4Bwmj1YprrkYsFbgXCQgLg)" Received: from hstml02-in.huaweisymantec.com ([172.26.3.41]) by hstga01-in.huaweisymantec.com (Sun Java(tm) System Messaging Server 6.3-8.03 (built Apr 24 2009; 32bit)) with ESMTP id <0LY2008JH3OBQC10@hstga01-in.huaweisymantec.com> for storm@ietf.org; Fri, 20 Jan 2012 01:38:36 +0800 (CST) Received: from m90003900a ([10.212.245.80]) by hstml02-in.huaweisymantec.com (Sun Java(tm) System Messaging Server 6.3-8.03 (built Apr 24 2009; 32bit)) with ESMTPA id <0LY2008JT3O77D00@hstml02-in.huaweisymantec.com> for storm@ietf.org; Fri, 20 Jan 2012 01:38:35 +0800 (CST) Message-id: From: Michael Ko To: Hemal Shah , Tom Talpey , david.black@emc.com, storm@ietf.org Date: Thu, 19 Jan 2012 09:38:30 -0800 X-Priority: 3 X-MSMail-priority: Normal X-Mailer: Microsoft Outlook Express 6.00.2900.5931 X-MIMEOLE: Produced By Microsoft MimeOLE V6.00.2900.6109 Subject: Re: [storm] iSER - one last issue X-BeenThere: storm@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Storage Maintenance WG List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 19 Jan 2012 17:38:42 -0000 This is a multi-part message in MIME format. --Boundary_(ID_4Bwmj1YprrkYsFbgXCQgLg) Content-type: text/plain; charset=iso-8859-1 Content-transfer-encoding: 7BIT For all RCaP layers, the Send message (or any variant) MUST be used, as opposed to using RDMA Read or Write to accomplish the task. So a SHOULD for RCaP other than iWARP is incorrect. RFC 5040 can mandate that all RDMAP messages be supported, but it is still up to implementation to decide what messages to use in any situation. This iSER update is meant to reflect running code and to generalize the support of iSER over any RCaP, not just iWARP. So the question is are there any running code that uses SendSE and SendInvSE for iSER. Conversely, if the running code can get by using Send messages to accomplish the task, what is the point of mandating a MUST at this point for SendSE and SendInvSE? Note that the receiver cannot completely rely on the sender for the Invalidate feature anyway. The iSER layer at the initiator is required to explicitly invalidate the STag for bidirectional commands and abnormal completion of a command. Even when automatic invalidation is used, the iSER layer is required to sanity checking the automatic invalidation. In other words, the iSER spec puts the burden on the receiver for doing the right thing in handling the Send messages, and not relying on the sender for the mere convenience. Mike ----- Original Message ----- From: Hemal Shah To: Tom Talpey ; david.black@emc.com ; Michael@huaweisymantec.com ; storm@ietf.org Sent: Thursday, January 19, 2012 8:44 AM Subject: RE: iSER - one last issue Tom, You are welcome! You are right about 7.3.4 as the only section mandating Send message. I think your concern about specifying the requirement for the other RCaP implementation is valid. I suggest we should word the requirements as below: "If the RCaP layer is as specified in [RFC5040] and [RFC5041], the SendSE message MUST be used. For any other RCaP layer, the Send message SHOULD be used. " Hemal From: Tom Talpey [mailto:ttalpey@microsoft.com] Sent: Thursday, January 19, 2012 8:09 AM To: Hemal Shah; david.black@emc.com; Michael@huaweisymantec.com; storm@ietf.org Subject: RE: iSER - one last issue Thanks for pointing out that sentence in RFC5046 7.3.4. I believe it is the only such stipulation. I'm ok with stating the requirement as RFC5046-over-RFC5040 compliance. But this new draft opens the door to alternative behaviors, therefore it needs to be stated more clearly. What's the requirement when SendSE is not supported? And, how does the receiver know what to expect? In other words, if it's proposed that the statement be: "When the RCaP layer is as specified in [RFC5040] and [RFC5041], the SendSE message MUST be used." . then what MUST be used when the RCaP is not? That statement needs to be made, too. From: Hemal Shah [mailto:hemal@broadcom.com] Sent: Thursday, January 19, 2012 10:38 AM To: Tom Talpey; david.black@emc.com; Michael@huaweisymantec.com; storm@ietf.org Subject: RE: iSER - one last issue Tom, RFC5046 is very clear about the use of Send Message Type for iSCSI control-type PDU. For specific iSCSI PDUs, Section 7 mandates the use of Send, SendSE, or SendSEInv. Section 7 of RFC5046 clearly specifies the use of specific Send Message Type for each iSCSI control-type PDUs (Login, Logout, Text request, Text response.). Section 7.3.4 mandates the use of plain-old Send also as needed. For example, see text below from 7.3.4. For unsolicited data, if the F bit is set to 0 in a SCSI Data-out PDU, the iSER layer at the initiator MUST use a Send Message to send the SCSI Data-out PDU. Regarding your comment in the second paragraph below, RFC5040 expectation is all the RDMAP messages specified in RFC5040 are supported (if that is not the case then we cannot count on any of the other RDMAP messages to be supported). So, an implementation that does not support SendSE at the sender is not compliant with RFC5040. So, your second point does not apply. I hope that addresses your comments. Hemal From: Tom Talpey [mailto:ttalpey@microsoft.com] Sent: Thursday, January 19, 2012 5:51 AM To: Hemal Shah; david.black@emc.com; Michael@huaweisymantec.com; storm@ietf.org Subject: RE: iSER - one last issue If using SendSE is so important here, when MUST a plain-old Send be used? There is no mention of any distinction in RFC5046 section 1.4.2 nor in this draft's section 2.4.2. Even with a MUST on the SendSE, the behavior is indeterminate, since it would still be contingent on support for SendSE at the *sender*. This critical exception voids any normative requirement, even over iWARP. The strongest statement that can be made is SHOULD. Tom. From: storm-bounces@ietf.org [mailto:storm-bounces@ietf.org] On Behalf Of Hemal Shah Sent: Wednesday, January 18, 2012 6:56 PM To: david.black@emc.com; Michael@huaweisymantec.com; storm@ietf.org Subject: Re: [storm] iSER - one last issue David and Mike, I believe that the requirements should not be weakened and we should stick to the requirement in RFC5046. RFC5046 required the use of specific Send Message Type for valid reasons. In the case of requiring SendSE message use, the intent was to give a hint to the remote side to generate an event upon processing of SendSE message. For example, the SendSE is specifically carrying SCSI information that needs attention on the remote side e.g. SCSI command, Logout. In the case of requiring SendInvSE Message, the intent was to invalidate the STag used in the data transfer as well as inform the remote side to generate an event. For example, mandating the use of SendInvSE for the SCSI response PDU for a SCSI Read limits the exposure of STag used in SCSI Read data transfer and avoids the explicit invalidation of the STag at the initiator. Plus, it allows the target to generate an event on the initiator upon the completion of SCSI Read command. By relaxing RFC5046 requirements in the latest iSER draft, we will not only impact all iWARP based iSER implementations that rely on the use of specific Send Message Type for SCSI data transfers but also change the intended behavior of initiators and targets. For iWARP, I strongly suggest that the iSER draft does not relax the requirements specified in RFC5046. I hope that helps. Hemal From: storm-bounces@ietf.org [mailto:storm-bounces@ietf.org] On Behalf Of david.black@emc.com Sent: Wednesday, January 18, 2012 3:10 PM To: Michael@huaweisymantec.com; storm@ietf.org Subject: Re: [storm] iSER - one last issue Mike, Thank you for the explanation, but I don't believe you've correctly stated the intent of RFC 5046. Here are some examples from RFC 5046's text: 1.5. SCSI Read Overview The iSER Message is transferred to the target using a SendSE Message. The iSER layer at the target uses a SendInvSE Message to transfer the SCSI Response PDU back to the iSER layer at the initiator. Similar language occurs in 1.6 SCSI Write Overview. 5.2.1. Normal Connection Termination at the Initiator The iSER layer at the initiator MUST use a SendSE Message to send the Logout Request PDU to the target. Similar language occurs in 5.2.2 for target side normal connection termination, and more importantly in 7.3.1 SCSI Command: The iSER layer at the initiator MUST send the SCSI command in a SendSE Message to the target. None of the quoted text permits use of Send instead of SendSE or SendInv instead of SendInvSE. What you propose to do effectively changes "MUST" to "should" (lower case, weaker than "SHOULD"), and that sure looks like a change to RFC 5046. Are there good implementation-based reasons to weaken these requirements now? Does anyone else have a viewpoint on this topic? Thanks, --David From: Michael Ko [mailto:Michael@huaweisymantec.com] Sent: Wednesday, January 18, 2012 2:22 PM To: Black, David; storm@ietf.org Subject: Re: iSER - one last issue David, Here is my rationale for using the lower case "should". The intent of RFC5046 is that the Send Message type must be used instead of RDMA Reads or Writes. The Solicited Event feature of the Send Message is provided as a convenience. The receiver must still do the right thing in handling the Send Message regardless of whether the SE feature is used. In other words, the sender is responsible for using the right format for the message (Send vs RDMA) but the receiver must not rely on the sender to determine how to handle the received message. The same rationale goes for the Invalidate feature. Mike ----- Original Message ----- From: david.black@emc.com To: Michael@huaweisymantec.com ; storm@ietf.org Sent: Monday, January 16, 2012 2:06 PM Subject: iSER - one last issue Mike, Thanks for getting the -08 version of the iSER draft posted. I think that draft addresses all of the open issues, but I have a question for the WG about how to express the SendSE and SendInvSE requirements from RFC 5046. The -08 version of the iSER draft expresses requirements for the SendSE and SendInvSE messages (this is primarily for iSER over RDDP/iWARP) as: The SendSE Message should be used if supported by the RCaP layer (e.g., iWARP). My reading of RFC 5046 is that its requirements are tighter - to accurately reflect RFC 5046, I would replace "should" with "MUST" in the above text, at least for iWARP. In the alternative, if the "should"s remain, an explanatory item needs to be added to the Appendix A list of changes from RFC 5046. What do others think the right course of action is here, use "MUST" or explain weakening of requirement to "should" ? Thanks, --David --Boundary_(ID_4Bwmj1YprrkYsFbgXCQgLg) Content-type: text/html; charset=iso-8859-1 Content-transfer-encoding: quoted-printable
For all RCaP layers, the Send message (or any = variant) MUST be=20 used, as opposed to using RDMA Read or Write to accomplish the = task.  So a=20 SHOULD for RCaP other than iWARP is incorrect.  RFC 5040 can = mandate=20 that all RDMAP messages be supported, but it is still up to = implementation to=20 decide what messages to use in any situation.  This iSER update is = meant to=20 reflect running code and to generalize the support of iSER over any = RCaP, not=20 just iWARP.  So the question is are there any running code that = uses SendSE=20 and SendInvSE for iSER.  Conversely, if the running code can get by = using=20 Send messages to accomplish the task, what is the point of mandating a = MUST at=20 this point for SendSE and SendInvSE?  Note that the receiver cannot = completely rely on the sender for the Invalidate feature = anyway.  The=20 iSER layer at the initiator is required to explicitly invalidate the = STag for=20 bidirectional commands and abnormal completion of a = command.  Even=20 when automatic invalidation is used, the iSER layer is required to = sanity=20 checking the automatic invalidation.  In other words, the iSER = spec puts the burden on the receiver for doing the right thing in = handling=20 the Send messages, and not relying on the sender for the mere=20 convenience. 
 
Mike
----- Original Message -----=20
From: Hemal = Shah
To: Tom Talpey ; david.black@emc.com ; Michael@huaweisymantec.com= ; storm@ietf.org =
Sent: Thursday, January 19, 2012 8:44 AM
Subject: RE: iSER - one last issue

Tom,

 

You are=20 welcome! You are right about 7.3.4 as the only section mandating Send=20 message.

 

I = think your=20 concern about specifying the requirement for the other RCaP = implementation is=20 valid. I suggest we should word the requirements as=20 below:

 

=93If=20 the RCaP layer is as specified in [RFC5040] and [RFC5041], the SendSE = message=20 MUST be used. For any other RCaP layer, the Send message SHOULD be used. = =94

 

Hemal

 

From: Tom = Talpey=20 [mailto:ttalpey@microsoft.com]
Sent: Thursday, January 19, = 2012 8:09=20 AM
To: Hemal Shah; david.black@emc.com;=20 Michael@huaweisymantec.com; storm@ietf.org
Subject: RE: iSER - = one=20 last issue

 

Thanks=20 for pointing out that sentence in RFC5046 7.3.4. I believe it is the = only such=20 stipulation.

 

I=92m=20 ok with stating the requirement as RFC5046-over-RFC5040 compliance. But = this new=20 draft opens the door to alternative behaviors, therefore it needs to be = stated=20 more clearly. What=92s the requirement when SendSE is not supported? = And, how does=20 the receiver know what to expect?

 

In=20 other words, if it=92s proposed that the statement = be:

           &= nbsp;   =20 =93When the RCaP layer is as specified in [RFC5040] and [RFC5041], the = SendSE=20 message MUST be used.=94

 

=85=20 then what MUST be used when the RCaP is not? That statement needs to be = made,=20 too.

 

 

From: Hemal = Shah=20 [mailto:hemal@broadcom.com]
Sent: Thursday, January 19, 2012 = 10:38=20 AM
To: Tom Talpey; david.black@emc.com; = Michael@huaweisymantec.com;=20 storm@ietf.org
Subject: RE: iSER - one last=20 issue

 

Tom,

 

RFC5046 is=20 very clear about the use of Send Message Type for iSCSI control-type = PDU. For=20 specific iSCSI PDUs, Section 7 mandates the use of Send, SendSE, or = SendSEInv.=20 Section 7 of RFC5046 clearly specifies the use of specific Send Message = Type for=20 each iSCSI control-type PDUs (Login, Logout, Text request, Text = response=85).=20 Section 7.3.4 mandates the use of plain-old Send also as needed. For = example,=20 see text below from 7.3.4.

 

For=20 unsolicited data, if the F bit is set to 0 in a SCSI=20 Data-out

PDU, the iSER = layer at the=20 initiator MUST use a Send Message to send

the=20 SCSI Data-out PDU.

 

Regarding=20 your comment in the second paragraph below, RFC5040 expectation is all = the RDMAP=20 messages specified in RFC5040 are supported (if that is not the case = then we=20 cannot count on any of the other RDMAP messages to be supported). So, an = implementation that does not support SendSE at the sender is not = compliant with=20 RFC5040. So, your second point does not apply.

 

I = hope that=20 addresses your comments.

 

Hemal

 

From: Tom = Talpey [mailto:ttalpey@microsoft.= com]=20
Sent: Thursday, January 19, 2012 5:51 AM
To: Hemal = Shah; david.black@emc.com; Michael@huaweisymantec.com= ; storm@ietf.org
Subject: RE: = iSER -=20 one last issue

 

If=20 using SendSE is so important here, when MUST a plain-old Send be used? = There is=20 no mention of any distinction in RFC5046 section 1.4.2 nor in this = draft=92s=20 section 2.4.2.

 

Even=20 with a MUST on the SendSE, the behavior is indeterminate, since it would = still=20 be contingent on support for SendSE at the *sender*. This = critical=20 exception voids any normative requirement, even over iWARP. The = strongest=20 statement that can be made is SHOULD.

 

Tom.

 

From: storm-bounces@ietf.org [mailto:storm-bounces@iet= f.org]=20 On Behalf Of Hemal Shah
Sent: Wednesday, January 18, = 2012 6:56=20 PM
To: david.black@emc.com;=20 Michael@huaweisymantec.com= ; storm@ietf.org
Subject: Re: = [storm]=20 iSER - one last issue

 

David and=20 Mike,

 

I = believe=20 that the requirements should not be weakened and we should stick to the=20 requirement in RFC5046.

 

RFC5046=20 required the use of specific Send Message Type for valid=20 reasons.

 

In the case=20 of requiring SendSE message use, the intent was to give a hint to the = remote=20 side to generate an event upon processing of SendSE message. For = example, the=20 SendSE is specifically carrying SCSI information that needs attention on = the=20 remote side e.g. SCSI command, Logout=85

 

In the case=20 of requiring SendInvSE Message, the intent was to invalidate the STag = used in=20 the data transfer as well as inform the remote side to generate an = event. For=20 example, mandating the use of SendInvSE for the SCSI response PDU for a = SCSI=20 Read limits the exposure of STag used in SCSI Read data transfer and = avoids the=20 explicit invalidation of the STag at the initiator. Plus, it allows the = target=20 to generate an event on the initiator upon the completion of SCSI Read=20 command.

 

By relaxing=20 RFC5046 requirements in the latest iSER draft, we will not only impact = all iWARP=20 based iSER implementations that rely on the use of specific Send Message = Type=20 for SCSI data transfers but also change the intended behavior of = initiators and=20 targets.

 

For iWARP, I=20 strongly suggest that the iSER draft does not relax the requirements = specified=20 in RFC5046.

 

I = hope that=20 helps.

 

Hemal=20    

 

From: storm-bounces@ietf.org [mailto:storm-bounces@iet= f.org]=20 On Behalf Of david.black@emc.com
Sent:=20 Wednesday, January 18, 2012 3:10 PM
To: Michael@huaweisymantec.com= ; storm@ietf.org
Subject: Re: = [storm]=20 iSER - one last issue

 

Mike,

 

Thank you for=20 the explanation, but I don=92t believe you=92ve correctly=20 stated

the = intent of=20 RFC 5046.  Here are some examples from RFC 5046=92s=20 text:

 

1.5.  SCSI = Read=20 Overview

 

   The = iSER=20 Message is transferred to the target using a SendSE=20 Message.

 

   The = iSER layer=20 at the target uses a SendInvSE Message to transfer = the

   SCSI = Response=20 PDU back to the iSER layer at the initiator.

 

Similar language = occurs in=20 1.6 SCSI Write Overview.

 

5.2.1.  = Normal=20 Connection Termination at the Initiator

 

   The = iSER layer=20 at the initiator MUST

   use a = SendSE=20 Message to send the Logout Request PDU to the = target.

 

Similar language = occurs in=20 5.2.2 for target side normal connection = termination,

and=20 more importantly in 7.3.1 SCSI Command:

 

   The = iSER layer=20 at the initiator MUST send the SCSI command in a

   = SendSE Message=20 to the target.

 

None of the quoted = text=20 permits use of Send instead of SendSE or SendInv

instead of=20 SendInvSE.

 

What you propose = to do=20 effectively changes =93MUST=94 to =93should=94 (lower = case,

weaker than = =93SHOULD=94), and=20 that sure looks like a change to RFC 5046.

 

Are=20 there good implementation-based reasons to weaken these requirements=20 now?

 

Does = anyone=20 else have a viewpoint on this topic?

 

Thanks,
--David

 

From: Michael = Ko [mailto:Michael@huawe= isymantec.com]=20
Sent: Wednesday, January 18, 2012 2:22 PM
To: = Black, David;=20 storm@ietf.org
Subject: = Re: iSER -=20 one last issue

 

David,

 

Here is my = rationale for using=20 the lower case "should".

 

The intent of = RFC5046 is that=20 the Send Message type must be used instead of RDMA Reads or=20 Writes.  The Solicited Event feature of the Send Message is = provided=20 as a convenience.  The receiver must still do the right thing in = handling=20 the Send Message regardless of whether the SE feature is used.  In = other=20 words, the sender is responsible for using the right format for the = message=20 (Send vs RDMA) but the receiver must not rely on the sender to determine = how to=20 handle the received message.  The same rationale goes for the = Invalidate=20 feature. 

 

Mike

----- = Original=20 Message -----

From: david.black@emc.com=20

To: Michael@huaweisymantec.com= ; storm@ietf.org =

Sent: Monday, = January 16,=20 2012 2:06 PM

Subject: iSER - one = last=20 issue

 

Mike,

 

Thanks for=20 getting the -08 version of the iSER draft posted.  I think=20 that

draft=20 addresses all of the open issues, but I have a question for the=20 WG

about how to=20 express the SendSE and SendInvSE requirements from RFC=20 5046.

 

The = -08=20 version of the iSER draft expresses requirements for the=20 SendSE

and = SendInvSE=20 messages (this is primarily for iSER over RDDP/iWARP) = as:

 

The = SendSE=20 Message should be used if

supported by=20 the RCaP layer (e.g., iWARP).

 

My = reading of=20 RFC 5046 is that its requirements are tighter - to=20 accurately

reflect RFC=20 5046, I would replace =93should=94 with =93MUST=94 in the above=20 text,

at = least for=20 iWARP.

 

In = the=20 alternative, if the =93should=94s remain, an explanatory = item

needs to be=20 added to the Appendix A list of changes from RFC = 5046.

 

What = do others=20 think the right course of action is here, use =93MUST=94 = or

explain=20 weakening of requirement to =93should=94 ?

 

Thanks,
--David

--Boundary_(ID_4Bwmj1YprrkYsFbgXCQgLg)-- From ttalpey@microsoft.com Thu Jan 19 12:32:54 2012 Return-Path: X-Original-To: storm@ietfa.amsl.com Delivered-To: storm@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 4C8D721F8503 for ; Thu, 19 Jan 2012 12:32:54 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -105.098 X-Spam-Level: X-Spam-Status: No, score=-105.098 tagged_above=-999 required=5 tests=[AWL=1.500, BAYES_00=-2.599, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_MED=-4, USER_IN_WHITELIST=-100] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id k1Ht7CdUpX05 for ; Thu, 19 Jan 2012 12:32:48 -0800 (PST) Received: from TX2EHSOBE002.bigfish.com (tx2ehsobe001.messaging.microsoft.com [65.55.88.11]) by ietfa.amsl.com (Postfix) with ESMTP id 04C9221F853B for ; Thu, 19 Jan 2012 12:32:47 -0800 (PST) Received: from mail153-tx2-R.bigfish.com (10.9.14.245) by TX2EHSOBE002.bigfish.com (10.9.40.22) with Microsoft SMTP Server id 14.1.225.23; Thu, 19 Jan 2012 20:32:45 +0000 Received: from mail153-tx2 (localhost [127.0.0.1]) by mail153-tx2-R.bigfish.com (Postfix) with ESMTP id EB77C4800E1; Thu, 19 Jan 2012 20:32:45 +0000 (UTC) X-SpamScore: -50 X-BigFish: VS-50(zz9371I1503Mc85fh148cM542M1418Mc1dM4015Lzz1202hzz1033IL8275bh8275dhz2fhc1bhc31hc1ahc1bhc31hc1ah2a8h668h839h61h) X-Spam-TCS-SCL: 0:0 X-Forefront-Antispam-Report: CIP:131.107.125.8; KIP:(null); UIP:(null); IPV:NLI; H:TK5EX14HUBC101.redmond.corp.microsoft.com; RD:none; EFVD:NLI X-FB-SS: 0,0, Received-SPF: pass (mail153-tx2: domain of microsoft.com designates 131.107.125.8 as permitted sender) client-ip=131.107.125.8; envelope-from=ttalpey@microsoft.com; helo=TK5EX14HUBC101.redmond.corp.microsoft.com ; icrosoft.com ; Received: from mail153-tx2 (localhost.localdomain [127.0.0.1]) by mail153-tx2 (MessageSwitch) id 132700516485244_6465; Thu, 19 Jan 2012 20:32:44 +0000 (UTC) Received: from TX2EHSMHS031.bigfish.com (unknown [10.9.14.245]) by mail153-tx2.bigfish.com (Postfix) with ESMTP id F32DE340042; Thu, 19 Jan 2012 20:32:43 +0000 (UTC) Received: from TK5EX14HUBC101.redmond.corp.microsoft.com (131.107.125.8) by TX2EHSMHS031.bigfish.com (10.9.99.131) with Microsoft SMTP Server (TLS) id 14.1.225.23; Thu, 19 Jan 2012 20:32:41 +0000 Received: from TK5EX14MBXC118.redmond.corp.microsoft.com ([169.254.9.75]) by TK5EX14HUBC101.redmond.corp.microsoft.com ([157.54.7.153]) with mapi id 14.02.0247.005; Thu, 19 Jan 2012 12:32:32 -0800 From: Tom Talpey To: Michael Ko , Hemal Shah , "david.black@emc.com" , "storm@ietf.org" Thread-Topic: iSER - one last issue Thread-Index: AQHM1tEoMWLUJLTWDk+GYyBugVM345YUHEsw Date: Thu, 19 Jan 2012 20:32:31 +0000 Message-ID: References: In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [157.54.51.33] Content-Type: multipart/alternative; boundary="_000_F83812DF4B59B9499C1BC978336D917461CDEBCFTK5EX14MBXC118r_" MIME-Version: 1.0 X-OriginatorOrg: microsoft.com Subject: Re: [storm] iSER - one last issue X-BeenThere: storm@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Storage Maintenance WG List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 19 Jan 2012 20:32:54 -0000 --_000_F83812DF4B59B9499C1BC978336D917461CDEBCFTK5EX14MBXC118r_ Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable More precisely, are there any iSER *initiators* that *depend* on receiving = solicited events over the iWARP RDMA transport? I believe this to be a very= short list, if any. But I'd be ok with being proven wrong. From: Michael Ko [mailto:Michael@huaweisymantec.com] Sent: Thursday, January 19, 2012 12:39 PM To: Hemal Shah; Tom Talpey; david.black@emc.com; storm@ietf.org Subject: Re: iSER - one last issue For all RCaP layers, the Send message (or any variant) MUST be used, as opp= osed to using RDMA Read or Write to accomplish the task. So a SHOULD for R= CaP other than iWARP is incorrect. RFC 5040 can mandate that all RDMAP mes= sages be supported, but it is still up to implementation to decide what mes= sages to use in any situation. This iSER update is meant to reflect runnin= g code and to generalize the support of iSER over any RCaP, not just iWARP.= So the question is are there any running code that uses SendSE and SendIn= vSE for iSER. Conversely, if the running code can get by using Send messag= es to accomplish the task, what is the point of mandating a MUST at this po= int for SendSE and SendInvSE? Note that the receiver cannot completely rel= y on the sender for the Invalidate feature anyway. The iSER layer at the i= nitiator is required to explicitly invalidate the STag for bidirectional co= mmands and abnormal completion of a command. Even when automatic invalidat= ion is used, the iSER layer is required to sanity checking the automatic in= validation. In other words, the iSER spec puts the burden on the receiver = for doing the right thing in handling the Send messages, and not relying on= the sender for the mere convenience. Mike ----- Original Message ----- From: Hemal Shah To: Tom Talpey ; david.black@emc.com ; Michael@huaweisymantec.com ; storm@ietf.org Sent: Thursday, January 19, 2012 8:44 AM Subject: RE: iSER - one last issue Tom, You are welcome! You are right about 7.3.4 as the only section mandating Se= nd message. I think your concern about specifying the requirement for the other RCaP im= plementation is valid. I suggest we should word the requirements as below: "If the RCaP layer is as specified in [RFC5040] and [RFC5041], the SendSE m= essage MUST be used. For any other RCaP layer, the Send message SHOULD be u= sed. " Hemal From: Tom Talpey [mailto:ttalpey@microsoft.com] Sent: Thursday, January 19, 2012 8:09 AM To: Hemal Shah; david.black@emc.com; Michael@hu= aweisymantec.com; storm@ietf.org Subject: RE: iSER - one last issue Thanks for pointing out that sentence in RFC5046 7.3.4. I believe it is the= only such stipulation. I'm ok with stating the requirement as RFC5046-over-RFC5040 compliance. But= this new draft opens the door to alternative behaviors, therefore it needs= to be stated more clearly. What's the requirement when SendSE is not suppo= rted? And, how does the receiver know what to expect? In other words, if it's proposed that the statement be: "When the RCaP layer is as specified in [RFC5040] and [RFC5= 041], the SendSE message MUST be used." ... then what MUST be used when the RCaP is not? That statement needs to be= made, too. From: Hemal Shah [mailto:hemal@broadcom.com] Sent: Thursday, January 19, 2012 10:38 AM To: Tom Talpey; david.black@emc.com; Michael@hu= aweisymantec.com; storm@ietf.org Subject: RE: iSER - one last issue Tom, RFC5046 is very clear about the use of Send Message Type for iSCSI control-= type PDU. For specific iSCSI PDUs, Section 7 mandates the use of Send, Send= SE, or SendSEInv. Section 7 of RFC5046 clearly specifies the use of specifi= c Send Message Type for each iSCSI control-type PDUs (Login, Logout, Text r= equest, Text response...). Section 7.3.4 mandates the use of plain-old Send= also as needed. For example, see text below from 7.3.4. For unsolicited data, if the F bit is set to 0 in a SCSI Data-out PDU, the iSER layer at the initiator MUST use a Send Message to send the SCSI Data-out PDU. Regarding your comment in the second paragraph below, RFC5040 expectation i= s all the RDMAP messages specified in RFC5040 are supported (if that is not= the case then we cannot count on any of the other RDMAP messages to be sup= ported). So, an implementation that does not support SendSE at the sender i= s not compliant with RFC5040. So, your second point does not apply. I hope that addresses your comments. Hemal From: Tom Talpey [mailto:ttalpey@microsoft.com] Sent: Thursday, January 19, 2012 5:51 AM To: Hemal Shah; david.black@emc.com; Michael@hu= aweisymantec.com; storm@ietf.org Subject: RE: iSER - one last issue If using SendSE is so important here, when MUST a plain-old Send be used? T= here is no mention of any distinction in RFC5046 section 1.4.2 nor in this = draft's section 2.4.2. Even with a MUST on the SendSE, the behavior is indeterminate, since it wou= ld still be contingent on support for SendSE at the *sender*. This critical= exception voids any normative requirement, even over iWARP. The strongest = statement that can be made is SHOULD. Tom. From: storm-bounces@ietf.org [mailto:storm-b= ounces@ietf.org] On Behalf Of Hemal= Shah Sent: Wednesday, January 18, 2012 6:56 PM To: david.black@emc.com; Michael@huaweisymantec= .com; storm@ietf.org Subject: Re: [storm] iSER - one last issue David and Mike, I believe that the requirements should not be weakened and we should stick = to the requirement in RFC5046. RFC5046 required the use of specific Send Message Type for valid reasons. In the case of requiring SendSE message use, the intent was to give a hint = to the remote side to generate an event upon processing of SendSE message. = For example, the SendSE is specifically carrying SCSI information that need= s attention on the remote side e.g. SCSI command, Logout... In the case of requiring SendInvSE Message, the intent was to invalidate th= e STag used in the data transfer as well as inform the remote side to gener= ate an event. For example, mandating the use of SendInvSE for the SCSI resp= onse PDU for a SCSI Read limits the exposure of STag used in SCSI Read data= transfer and avoids the explicit invalidation of the STag at the initiator= . Plus, it allows the target to generate an event on the initiator upon the= completion of SCSI Read command. By relaxing RFC5046 requirements in the latest iSER draft, we will not only= impact all iWARP based iSER implementations that rely on the use of specif= ic Send Message Type for SCSI data transfers but also change the intended b= ehavior of initiators and targets. For iWARP, I strongly suggest that the iSER draft does not relax the requir= ements specified in RFC5046. I hope that helps. Hemal From: storm-bounces@ietf.org [mailto:storm-b= ounces@ietf.org] On Behalf Of david= .black@emc.com Sent: Wednesday, January 18, 2012 3:10 PM To: Michael@huaweisymantec.com; storm@ie= tf.org Subject: Re: [storm] iSER - one last issue Mike, Thank you for the explanation, but I don't believe you've correctly stated the intent of RFC 5046. Here are some examples from RFC 5046's text: 1.5. SCSI Read Overview The iSER Message is transferred to the target using a SendSE Message. The iSER layer at the target uses a SendInvSE Message to transfer the SCSI Response PDU back to the iSER layer at the initiator. Similar language occurs in 1.6 SCSI Write Overview. 5.2.1. Normal Connection Termination at the Initiator The iSER layer at the initiator MUST use a SendSE Message to send the Logout Request PDU to the target. Similar language occurs in 5.2.2 for target side normal connection terminat= ion, and more importantly in 7.3.1 SCSI Command: The iSER layer at the initiator MUST send the SCSI command in a SendSE Message to the target. None of the quoted text permits use of Send instead of SendSE or SendInv instead of SendInvSE. What you propose to do effectively changes "MUST" to "should" (lower case, weaker than "SHOULD"), and that sure looks like a change to RFC 5046. Are there good implementation-based reasons to weaken these requirements no= w? Does anyone else have a viewpoint on this topic? Thanks, --David From: Michael Ko [mailto:Michael@huaweisymantec.com] Sent: Wednesday, January 18, 2012 2:22 PM To: Black, David; storm@ietf.org Subject: Re: iSER - one last issue David, Here is my rationale for using the lower case "should". The intent of RFC5046 is that the Send Message type must be used instead of= RDMA Reads or Writes. The Solicited Event feature of the Send Message is = provided as a convenience. The receiver must still do the right thing in h= andling the Send Message regardless of whether the SE feature is used. In = other words, the sender is responsible for using the right format for the m= essage (Send vs RDMA) but the receiver must not rely on the sender to deter= mine how to handle the received message. The same rationale goes for the I= nvalidate feature. Mike ----- Original Message ----- From: david.black@emc.com To: Michael@huaweisymantec.com ; storm@i= etf.org Sent: Monday, January 16, 2012 2:06 PM Subject: iSER - one last issue Mike, Thanks for getting the -08 version of the iSER draft posted. I think that draft addresses all of the open issues, but I have a question for the WG about how to express the SendSE and SendInvSE requirements from RFC 5046. The -08 version of the iSER draft expresses requirements for the SendSE and SendInvSE messages (this is primarily for iSER over RDDP/iWARP) as: The SendSE Message should be used if supported by the RCaP layer (e.g., iWARP). My reading of RFC 5046 is that its requirements are tighter - to accurately reflect RFC 5046, I would replace "should" with "MUST" in the above text, at least for iWARP. In the alternative, if the "should"s remain, an explanatory item needs to be added to the Appendix A list of changes from RFC 5046. What do others think the right course of action is here, use "MUST" or explain weakening of requirement to "should" ? Thanks, --David --_000_F83812DF4B59B9499C1BC978336D917461CDEBCFTK5EX14MBXC118r_ Content-Type: text/html; charset="us-ascii" Content-Transfer-Encoding: quoted-printable

More precisely, are there= any iSER *initiators* that *depend* on receiving solicited e= vents over the iWARP RDMA transport? I believe this to be a very short list, if any. But I’d be ok with being proven wrong.

 <= /p>

 <= /p>

 <= /p>

From: Michael = Ko [mailto:Michael@huaweisymantec.com]
Sent: Thursday, January 19, 2012 12:39 PM
To: Hemal Shah; Tom Talpey; david.black@emc.com; storm@ietf.org
Subject: Re: iSER - one last issue

 

For all RCaP layers= , the Send message (or any variant) MUST be used, as opposed to using RDMA = Read or Write to accomplish the task.  So a SHOULD for RCaP other= than iWARP is incorrect.  RFC 5040 can mandate that all RDMAP messages be supported, but it is still up to implementation= to decide what messages to use in any situation.  This iSER update is= meant to reflect running code and to generalize the support of iSER over a= ny RCaP, not just iWARP.  So the question is are there any running code that uses SendSE and SendInvSE for iSER.&nbs= p; Conversely, if the running code can get by using Send messages to accomp= lish the task, what is the point of mandating a MUST at this point for Send= SE and SendInvSE?  Note that the receiver cannot completely rely on the sender for the Invalidate feature anyway.&nb= sp; The iSER layer at the initiator is required to explicitly invalida= te the STag for bidirectional commands and abnormal completion of a command= .  Even when automatic invalidation is used, the iSER layer is required to sanity checking the automatic invalidation.&= nbsp; In other words, the iSER spec puts the burden on the receiv= er for doing the right thing in handling the Send messages, and not re= lying on the sender for the mere convenience. 

 

Mike

 

Tom= ,

 

You= are welcome! You are right about 7.3.4 as the only section mandating Send = message.

 

I t= hink your concern about specifying the requirement for the other RCaP imple= mentation is valid. I suggest we should word the requirements as below:

 

“If the RCaP layer = is as specified in [RFC5040] and [RFC5041], the SendSE message MUST be used= . For any other RCaP layer, the Send message SHOULD be used. ”

 <= /p>

Hem= al

 

From: Tom Talp= ey [mailto:ttalpey@microsoft= .com]
Sent: Thursday, January 19, 2012 8:09 AM
To: Hemal Shah; david.black@e= mc.com; Michael@huaweisymantec.com; storm@ietf.org
Subject: RE: iSER - one last issue

 

Thanks for pointing out t= hat sentence in RFC5046 7.3.4. I believe it is the only such stipulation.

 <= /p>

I’m ok with stating= the requirement as RFC5046-over-RFC5040 compliance. But this new draft ope= ns the door to alternative behaviors, therefore it needs to be stated more clearly. What’s the requirement when SendSE is not suppo= rted? And, how does the receiver know what to expect?

 <= /p>

In other words, if itR= 17;s proposed that the statement be:

    &= nbsp;           “Wh= en the RCaP layer is as specified in [RFC5040] and [RFC5041], the SendSE me= ssage MUST be used.”

 <= /p>

… then what MUST be= used when the RCaP is not? That statement needs to be made, too.

 <= /p>

 <= /p>

 

Tom= ,

 

RFC= 5046 is very clear about the use of Send Message Type for iSCSI control-typ= e PDU. For specific iSCSI PDUs, Section 7 mandates the use of Send, SendSE,= or SendSEInv. Section 7 of RFC5046 clearly specifies the use of specific Send Message Type for each iSCSI con= trol-type PDUs (Login, Logout, Text request, Text response…). Section= 7.3.4 mandates the use of plain-old Send also as needed. For example, see = text below from 7.3.4.

 

For unsolicited data, if the F bit is set to 0 in a SCSI D= ata-out

PDU, the iSER layer at the initiator MUST use a Send Messa= ge to send

the SCSI Data-out PDU.

 

Reg= arding your comment in the second paragraph below, RFC5040 expectation is a= ll the RDMAP messages specified in RFC5040 are supported (if that is not th= e case then we cannot count on any of the other RDMAP messages to be supported). So, an implementation that does= not support SendSE at the sender is not compliant with RFC5040. So, your s= econd point does not apply.

 

I h= ope that addresses your comments.

 

Hem= al

 

From: Tom Talp= ey [mailto:ttalpey@microsoft= .com]
Sent: Thursday, January 19, 2012 5:51 AM
To: Hemal Shah; david.black@e= mc.com; Michael@huaweisymantec.com; storm@ietf.org
Subject: RE: iSER - one last issue

 

If using SendSE is so imp= ortant here, when MUST a plain-old Send be used? There is no mention of any= distinction in RFC5046 section 1.4.2 nor in this draft’s section 2.4.2.

 <= /p>

Even with a MUST on the S= endSE, the behavior is indeterminate, since it would still be contingent on= support for SendSE at the *sender*. This critical exception voids any normative requirement, even over iWARP. The strongest = statement that can be made is SHOULD.

 <= /p>

Tom.

 <= /p>

From: storm-bounces@ietf.org [mailto:storm-bounces@ietf.org] On Behalf Of Hemal Shah
Sent: Wednesday, January 18, 2012 6:56 PM
To: david.black@emc.com; = Michael@huaweisymantec.com; storm@iet= f.org
Subject: Re: [storm] iSER - one last issue

 

Dav= id and Mike,

 

I b= elieve that the requirements should not be weakened and we should stick to = the requirement in RFC5046.

 

RFC= 5046 required the use of specific Send Message Type for valid reasons.=

 

In = the case of requiring SendSE message use, the intent was to give a hint to = the remote side to generate an event upon processing of SendSE message. For= example, the SendSE is specifically carrying SCSI information that needs attention on the remote side e.g. SCS= I command, Logout…

 

In = the case of requiring SendInvSE Message, the intent was to invalidate the S= Tag used in the data transfer as well as inform the remote side to generate= an event. For example, mandating the use of SendInvSE for the SCSI response PDU for a SCSI Read limits the expo= sure of STag used in SCSI Read data transfer and avoids the explicit invali= dation of the STag at the initiator. Plus, it allows the target to generate= an event on the initiator upon the completion of SCSI Read command.

 

By = relaxing RFC5046 requirements in the latest iSER draft, we will not only im= pact all iWARP based iSER implementations that rely on the use of specific = Send Message Type for SCSI data transfers but also change the intended behavior of initiators and targets.

 

For= iWARP, I strongly suggest that the iSER draft does not relax the requireme= nts specified in RFC5046.

 

I h= ope that helps.

 

Hem= al    

 

From: storm-bounces@ietf.org [mailto:storm-bounces@ietf.org] On Behalf Of david.black@emc.com
Sent: Wednesday, January 18, 2012 3:10 PM
To: Michael@huaweisyma= ntec.com; storm@ietf.org
Subject: Re: [storm] iSER - one last issue

 

Mike,

 

Thank you for the explanation, but I don’= ;t believe you’ve correctly stated

the intent of RFC 5046.  Here are some ex= amples from RFC 5046’s text:

 

1.5.  SCSI Read Overview

 

   The iSER Message is transferred to the target= using a SendSE Message.

 

   The iSER layer at the target uses a SendInvSE= Message to transfer the

   SCSI Response PDU back to the iSER layer at t= he initiator.

 

Similar language occurs in 1.6 SCSI Write Overview.

 

5.2.1.  Normal Connection Termination at the Initiato= r

 

   The iSER layer at the initiator MUST

   use a SendSE Message to send the Logout Reque= st PDU to the target.

 

Similar language occurs in 5.2.2 for target side normal co= nnection termination,

and more importantly in 7.3.1 SCSI Command:

 

   The iSER layer at the initiator MUST send the= SCSI command in a

   SendSE Message to the target.

 

None of the quoted text permits use of Send instead of Sen= dSE or SendInv

instead of SendInvSE.

 

What you propose to do effectively changes “MUSTR= 21; to “should” (lower case,

weaker than “SHOULD”), and that sure looks lik= e a change to RFC 5046.

 

Are there good implementation-based reasons to weaken thes= e requirements now?

 

Does anyone else have a viewpoint on this topi= c?

 

Thanks,
--David

 

From: Michael = Ko [mailto:Michael@huaw= eisymantec.com]
Sent: Wednesday, January 18, 2012 2:22 PM
To: Black, David; storm@ietf.org
Subject: Re: iSER - one last issue

 

David,<= /o:p>

 

Here is my rational= e for using the lower case "should".

 

The intent of RFC50= 46 is that the Send Message type must be used instead of RDMA Reads or Writ= es.  The Solicited Event feature of the Send Message is provided = as a convenience.  The receiver must still do the right thing in handling the Send Message regardless of whether the SE = feature is used.  In other words, the sender is responsible for using = the right format for the message (Send vs RDMA) but the receiver must = not rely on the sender to determine how to handle the received message.  The same rationale goes for the Invalid= ate feature. 

 

Mike

 

Mike,

 

Thanks for getting the -08 version of the iSER= draft posted.  I think that

draft addresses all of the open issues, but I = have a question for the WG

about how to express the SendSE and SendInvSE = requirements from RFC 5046.

 

The -08 version of the iSER draft expresses re= quirements for the SendSE

and SendInvSE messages (this is primarily for = iSER over RDDP/iWARP) as:

 

The SendSE Message = should be used if

supported by the RC= aP layer (e.g., iWARP).

 

My reading of RFC 5046 is that its requirement= s are tighter - to accurately

reflect RFC 5046, I would replace “shoul= d” with “MUST” in the above text,

at least for iWARP.

 

In the alternative, if the “should”= ;s remain, an explanatory item

needs to be added to the Appendix A list of ch= anges from RFC 5046.

 

What do others think the right course of actio= n is here, use “MUST” or

explain weakening of requirement to “sho= uld” ?

 

Thanks,
--David

--_000_F83812DF4B59B9499C1BC978336D917461CDEBCFTK5EX14MBXC118r_-- From Michael@huaweisymantec.com Thu Jan 19 09:31:01 2012 Return-Path: X-Original-To: storm@ietfa.amsl.com Delivered-To: storm@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 7E2AD21F8642 for ; Thu, 19 Jan 2012 09:31:01 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.598 X-Spam-Level: X-Spam-Status: No, score=-2.598 tagged_above=-999 required=5 tests=[BAYES_00=-2.599, HTML_MESSAGE=0.001] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id sEoMvzJ-TNSP for ; Thu, 19 Jan 2012 09:30:59 -0800 (PST) Received: from mta1.huaweisymantec.com (mta1.huaweisymantec.com [218.17.155.14]) by ietfa.amsl.com (Postfix) with ESMTP id DCEC421F8574 for ; Thu, 19 Jan 2012 09:30:58 -0800 (PST) MIME-version: 1.0 Content-type: multipart/alternative; boundary="Boundary_(ID_ewEe8f5th1Tt8gBZ7EwHPw)" Received: from hstml02-in.huaweisymantec.com ([172.26.3.41]) by hstga01-in.huaweisymantec.com (Sun Java(tm) System Messaging Server 6.3-8.03 (built Apr 24 2009; 32bit)) with ESMTP id <0LY2008FL3BIQC10@hstga01-in.huaweisymantec.com> for storm@ietf.org; Fri, 20 Jan 2012 01:30:54 +0800 (CST) Received: from m90003900a ([10.212.245.80]) by hstml02-in.huaweisymantec.com (Sun Java(tm) System Messaging Server 6.3-8.03 (built Apr 24 2009; 32bit)) with ESMTPA id <0LY2008JF3BD2D00@hstml02-in.huaweisymantec.com> for storm@ietf.org; Fri, 20 Jan 2012 01:30:54 +0800 (CST) Message-id: <76514301743A45D5A0D6A86657A38C23@china.huawei.com> From: Michael Ko To: Hemal Shah , Tom Talpey , david.black@emc.com, storm@ietf.org References: <20120115134011.27535.67315.idtracker@ietfa.amsl.com> <7C4DFCE962635144B8FAE8CA11D0BF1E05A7CF06E5@MX14A.corp.emc.com> <6315937B35624B2E9C599A392DDB8EA1@china.huawei.com> <7C4DFCE962635144B8FAE8CA11D0BF1E05A7CF0CF7@MX14A.corp.emc.com> <2D98DD3F898B6B4DA287BF3BA07DAE93022E4D@IRVEXCHMB11.corp.ad.broadcom.com> <2D98DD3F898B6B4DA287BF3BA07DAE93023279@IRVEXCHMB11.corp.ad.broadcom.com> <2D98DD3F898B6B4DA287BF3BA07DAE930234C6@IRVEXCHMB11.corp.ad.broadcom.com> Date: Thu, 19 Jan 2012 09:30:48 -0800 X-Priority: 3 X-MSMail-priority: Normal X-Mailer: Microsoft Outlook Express 6.00.2900.5931 X-MIMEOLE: Produced By Microsoft MimeOLE V6.00.2900.6109 X-Mailman-Approved-At: Thu, 19 Jan 2012 16:09:31 -0800 Subject: Re: [storm] iSER - one last issue X-BeenThere: storm@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Storage Maintenance WG List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 19 Jan 2012 17:31:01 -0000 This is a multi-part message in MIME format. --Boundary_(ID_ewEe8f5th1Tt8gBZ7EwHPw) Content-type: text/plain; charset=iso-8859-1 Content-transfer-encoding: 7BIT For all RCaP layers, the Send message (or any variant) MUST be used, as opposed to using RDMA Read or Write to accomplish the task. So a SHOULD for RCaP other than iWARP is incorrect. RFC 5040 can mandate that all RDMAP messages be supported, but it is still up to implementation to decide what messages to use in any situation. This iSER update is meant to reflect running code and to generalize the support of iSER over any RCaP, not just iWARP. So the question is are there any running code that uses SendSE and SendInvSE for iSER. Conversely, if the running code can get by using Send messages to accomplish the task, what is the point of mandating a MUST at this point for SendSE and SendInvSE? Note that the receiver cannot completely rely on the sender for the Invalidate feature anyway. The iSER layer at the initiator is required to explicitly invalidate the STag for bidirectional commands and abnormal completion of a command. Even when automatic invalidation is used, the iSER layer is required to sanity checking the automatic invalidation. In other words, the iSER spec puts the burden on the receiver for doing the right thing in handling the Send messages, and not relying on the sender for the mere convenience. Mike ----- Original Message ----- From: Hemal Shah To: Tom Talpey ; david.black@emc.com ; Michael@huaweisymantec.com ; storm@ietf.org Sent: Thursday, January 19, 2012 8:44 AM Subject: RE: iSER - one last issue Tom, You are welcome! You are right about 7.3.4 as the only section mandating Send message. I think your concern about specifying the requirement for the other RCaP implementation is valid. I suggest we should word the requirements as below: "If the RCaP layer is as specified in [RFC5040] and [RFC5041], the SendSE message MUST be used. For any other RCaP layer, the Send message SHOULD be used. " Hemal From: Tom Talpey [mailto:ttalpey@microsoft.com] Sent: Thursday, January 19, 2012 8:09 AM To: Hemal Shah; david.black@emc.com; Michael@huaweisymantec.com; storm@ietf.org Subject: RE: iSER - one last issue Thanks for pointing out that sentence in RFC5046 7.3.4. I believe it is the only such stipulation. I'm ok with stating the requirement as RFC5046-over-RFC5040 compliance. But this new draft opens the door to alternative behaviors, therefore it needs to be stated more clearly. What's the requirement when SendSE is not supported? And, how does the receiver know what to expect? In other words, if it's proposed that the statement be: "When the RCaP layer is as specified in [RFC5040] and [RFC5041], the SendSE message MUST be used." . then what MUST be used when the RCaP is not? That statement needs to be made, too. From: Hemal Shah [mailto:hemal@broadcom.com] Sent: Thursday, January 19, 2012 10:38 AM To: Tom Talpey; david.black@emc.com; Michael@huaweisymantec.com; storm@ietf.org Subject: RE: iSER - one last issue Tom, RFC5046 is very clear about the use of Send Message Type for iSCSI control-type PDU. For specific iSCSI PDUs, Section 7 mandates the use of Send, SendSE, or SendSEInv. Section 7 of RFC5046 clearly specifies the use of specific Send Message Type for each iSCSI control-type PDUs (Login, Logout, Text request, Text response.). Section 7.3.4 mandates the use of plain-old Send also as needed. For example, see text below from 7.3.4. For unsolicited data, if the F bit is set to 0 in a SCSI Data-out PDU, the iSER layer at the initiator MUST use a Send Message to send the SCSI Data-out PDU. Regarding your comment in the second paragraph below, RFC5040 expectation is all the RDMAP messages specified in RFC5040 are supported (if that is not the case then we cannot count on any of the other RDMAP messages to be supported). So, an implementation that does not support SendSE at the sender is not compliant with RFC5040. So, your second point does not apply. I hope that addresses your comments. Hemal From: Tom Talpey [mailto:ttalpey@microsoft.com] Sent: Thursday, January 19, 2012 5:51 AM To: Hemal Shah; david.black@emc.com; Michael@huaweisymantec.com; storm@ietf.org Subject: RE: iSER - one last issue If using SendSE is so important here, when MUST a plain-old Send be used? There is no mention of any distinction in RFC5046 section 1.4.2 nor in this draft's section 2.4.2. Even with a MUST on the SendSE, the behavior is indeterminate, since it would still be contingent on support for SendSE at the *sender*. This critical exception voids any normative requirement, even over iWARP. The strongest statement that can be made is SHOULD. Tom. From: storm-bounces@ietf.org [mailto:storm-bounces@ietf.org] On Behalf Of Hemal Shah Sent: Wednesday, January 18, 2012 6:56 PM To: david.black@emc.com; Michael@huaweisymantec.com; storm@ietf.org Subject: Re: [storm] iSER - one last issue David and Mike, I believe that the requirements should not be weakened and we should stick to the requirement in RFC5046. RFC5046 required the use of specific Send Message Type for valid reasons. In the case of requiring SendSE message use, the intent was to give a hint to the remote side to generate an event upon processing of SendSE message. For example, the SendSE is specifically carrying SCSI information that needs attention on the remote side e.g. SCSI command, Logout. In the case of requiring SendInvSE Message, the intent was to invalidate the STag used in the data transfer as well as inform the remote side to generate an event. For example, mandating the use of SendInvSE for the SCSI response PDU for a SCSI Read limits the exposure of STag used in SCSI Read data transfer and avoids the explicit invalidation of the STag at the initiator. Plus, it allows the target to generate an event on the initiator upon the completion of SCSI Read command. By relaxing RFC5046 requirements in the latest iSER draft, we will not only impact all iWARP based iSER implementations that rely on the use of specific Send Message Type for SCSI data transfers but also change the intended behavior of initiators and targets. For iWARP, I strongly suggest that the iSER draft does not relax the requirements specified in RFC5046. I hope that helps. Hemal From: storm-bounces@ietf.org [mailto:storm-bounces@ietf.org] On Behalf Of david.black@emc.com Sent: Wednesday, January 18, 2012 3:10 PM To: Michael@huaweisymantec.com; storm@ietf.org Subject: Re: [storm] iSER - one last issue Mike, Thank you for the explanation, but I don't believe you've correctly stated the intent of RFC 5046. Here are some examples from RFC 5046's text: 1.5. SCSI Read Overview The iSER Message is transferred to the target using a SendSE Message. The iSER layer at the target uses a SendInvSE Message to transfer the SCSI Response PDU back to the iSER layer at the initiator. Similar language occurs in 1.6 SCSI Write Overview. 5.2.1. Normal Connection Termination at the Initiator The iSER layer at the initiator MUST use a SendSE Message to send the Logout Request PDU to the target. Similar language occurs in 5.2.2 for target side normal connection termination, and more importantly in 7.3.1 SCSI Command: The iSER layer at the initiator MUST send the SCSI command in a SendSE Message to the target. None of the quoted text permits use of Send instead of SendSE or SendInv instead of SendInvSE. What you propose to do effectively changes "MUST" to "should" (lower case, weaker than "SHOULD"), and that sure looks like a change to RFC 5046. Are there good implementation-based reasons to weaken these requirements now? Does anyone else have a viewpoint on this topic? Thanks, --David From: Michael Ko [mailto:Michael@huaweisymantec.com] Sent: Wednesday, January 18, 2012 2:22 PM To: Black, David; storm@ietf.org Subject: Re: iSER - one last issue David, Here is my rationale for using the lower case "should". The intent of RFC5046 is that the Send Message type must be used instead of RDMA Reads or Writes. The Solicited Event feature of the Send Message is provided as a convenience. The receiver must still do the right thing in handling the Send Message regardless of whether the SE feature is used. In other words, the sender is responsible for using the right format for the message (Send vs RDMA) but the receiver must not rely on the sender to determine how to handle the received message. The same rationale goes for the Invalidate feature. Mike ----- Original Message ----- From: david.black@emc.com To: Michael@huaweisymantec.com ; storm@ietf.org Sent: Monday, January 16, 2012 2:06 PM Subject: iSER - one last issue Mike, Thanks for getting the -08 version of the iSER draft posted. I think that draft addresses all of the open issues, but I have a question for the WG about how to express the SendSE and SendInvSE requirements from RFC 5046. The -08 version of the iSER draft expresses requirements for the SendSE and SendInvSE messages (this is primarily for iSER over RDDP/iWARP) as: The SendSE Message should be used if supported by the RCaP layer (e.g., iWARP). My reading of RFC 5046 is that its requirements are tighter - to accurately reflect RFC 5046, I would replace "should" with "MUST" in the above text, at least for iWARP. In the alternative, if the "should"s remain, an explanatory item needs to be added to the Appendix A list of changes from RFC 5046. What do others think the right course of action is here, use "MUST" or explain weakening of requirement to "should" ? Thanks, --David From: storm-bounces@ietf.org [mailto:storm-bounces@ietf.org] On Behalf Of Michael Ko Sent: Sunday, January 15, 2012 8:43 AM To: storm@ietf.org Subject: Re: [storm] I-D Action: draft-ietf-storm-iser-08.txt This version contains all the updates as discussed in the latest exchanges with Hemal Shah and David Black. Mike ----- Original Message ----- From: internet-drafts@ietf.org To: i-d-announce@ietf.org Cc: storm@ietf.org Sent: Sunday, January 15, 2012 5:40 AM Subject: [storm] I-D Action: draft-ietf-storm-iser-08.txt A New Internet-Draft is available from the on-line Internet-Drafts directories. This draft is a work item of the STORage Maintenance Working Group of the IETF. Title : iSCSI Extensions for RDMA Specification Author(s) : Michael Ko Alexander Nezhinsky Filename : draft-ietf-storm-iser-08.txt Pages : 95 Date : 2012-01-15 iSCSI Extensions for RDMA provides the RDMA data transfer capability to iSCSI by layering iSCSI on top of an RDMA-Capable Protocol. An RDMA-Capable Protocol provides RDMA Read and Write services, which enable data to be transferred directly into SCSI I/O Buffers without intermediate data copies. This document describes the extensions to the iSCSI protocol to support RDMA services as provided by an RDMA- Capable Protocol. This document obsoletes RFC 5046. A URL for this Internet-Draft is: http://www.ietf.org/internet-drafts/draft-ietf-storm-iser-08.txt Internet-Drafts are also available by anonymous FTP at: ftp://ftp.ietf.org/internet-drafts/ This Internet-Draft can be retrieved at: ftp://ftp.ietf.org/internet-drafts/draft-ietf-storm-iser-08.txt _______________________________________________ storm mailing list storm@ietf.org https://www.ietf.org/mailman/listinfo/storm --Boundary_(ID_ewEe8f5th1Tt8gBZ7EwHPw) Content-type: text/html; charset=iso-8859-1 Content-transfer-encoding: quoted-printable
For all RCaP layers, the Send message (or any = variant) MUST be=20 used, as opposed to using RDMA Read or Write to accomplish the = task.  So a=20 SHOULD for RCaP other than iWARP is incorrect.  RFC 5040 can = mandate=20 that all RDMAP messages be supported, but it is still up to = implementation to=20 decide what messages to use in any situation.  This iSER update is = meant to=20 reflect running code and to generalize the support of iSER over any = RCaP, not=20 just iWARP.  So the question is are there any running code that = uses SendSE=20 and SendInvSE for iSER.  Conversely, if the running code can get by = using=20 Send messages to accomplish the task, what is the point of mandating a = MUST at=20 this point for SendSE and SendInvSE?  Note that the receiver cannot = completely rely on the sender for the Invalidate feature = anyway.  The=20 iSER layer at the initiator is required to explicitly invalidate the = STag for=20 bidirectional commands and abnormal completion of a = command.  Even=20 when automatic invalidation is used, the iSER layer is required to = sanity=20 checking the automatic invalidation.  In other words, the iSER = spec puts the burden on the receiver for doing the right thing in = handling=20 the Send messages, and not relying on the sender for the mere=20 convenience. 
 
Mike
----- Original Message -----=20
To: Tom Talpey ; david.black@emc.com ; Michael@huaweisymantec.com= ; storm@ietf.org =
Sent: Thursday, January 19, 2012 8:44 AM
Subject: RE: iSER - one last issue

Tom,

 

You are=20 welcome! You are right about 7.3.4 as the only section mandating Send=20 message.

 

I = think your=20 concern about specifying the requirement for the other RCaP = implementation is=20 valid. I suggest we should word the requirements as=20 below:

 

=93If=20 the RCaP layer is as specified in [RFC5040] and [RFC5041], the SendSE = message=20 MUST be used. For any other RCaP layer, the Send message SHOULD be used. = =94

 

Hemal

 

From: Tom = Talpey=20 [mailto:ttalpey@microsoft.com]
Sent: Thursday, January 19, = 2012 8:09=20 AM
To: Hemal Shah; david.black@emc.com;=20 Michael@huaweisymantec.com; storm@ietf.org
Subject: RE: iSER - = one=20 last issue

 

Thanks=20 for pointing out that sentence in RFC5046 7.3.4. I believe it is the = only such=20 stipulation.

 

I=92m=20 ok with stating the requirement as RFC5046-over-RFC5040 compliance. But = this new=20 draft opens the door to alternative behaviors, therefore it needs to be = stated=20 more clearly. What=92s the requirement when SendSE is not supported? = And, how does=20 the receiver know what to expect?

 

In=20 other words, if it=92s proposed that the statement = be:

           &= nbsp;   =20 =93When the RCaP layer is as specified in [RFC5040] and [RFC5041], the = SendSE=20 message MUST be used.=94

 

=85=20 then what MUST be used when the RCaP is not? That statement needs to be = made,=20 too.

 

 

From: Hemal = Shah=20 [mailto:hemal@broadcom.com]
Sent: Thursday, January 19, 2012 = 10:38=20 AM
To: Tom Talpey; david.black@emc.com; = Michael@huaweisymantec.com;=20 storm@ietf.org
Subject: RE: iSER - one last=20 issue

 

Tom,

 

RFC5046 is=20 very clear about the use of Send Message Type for iSCSI control-type = PDU. For=20 specific iSCSI PDUs, Section 7 mandates the use of Send, SendSE, or = SendSEInv.=20 Section 7 of RFC5046 clearly specifies the use of specific Send Message = Type for=20 each iSCSI control-type PDUs (Login, Logout, Text request, Text = response=85).=20 Section 7.3.4 mandates the use of plain-old Send also as needed. For = example,=20 see text below from 7.3.4.

 

For=20 unsolicited data, if the F bit is set to 0 in a SCSI=20 Data-out

PDU, the iSER = layer at the=20 initiator MUST use a Send Message to send

the=20 SCSI Data-out PDU.

 

Regarding=20 your comment in the second paragraph below, RFC5040 expectation is all = the RDMAP=20 messages specified in RFC5040 are supported (if that is not the case = then we=20 cannot count on any of the other RDMAP messages to be supported). So, an = implementation that does not support SendSE at the sender is not = compliant with=20 RFC5040. So, your second point does not apply.

 

I = hope that=20 addresses your comments.

 

Hemal

 

From: Tom = Talpey [mailto:ttalpey@microsoft.= com]=20
Sent: Thursday, January 19, 2012 5:51 AM
To: Hemal = Shah; david.black@emc.com; Michael@huaweisymantec.com= ; storm@ietf.org
Subject: RE: = iSER -=20 one last issue

 

If=20 using SendSE is so important here, when MUST a plain-old Send be used? = There is=20 no mention of any distinction in RFC5046 section 1.4.2 nor in this = draft=92s=20 section 2.4.2.

 

Even=20 with a MUST on the SendSE, the behavior is indeterminate, since it would = still=20 be contingent on support for SendSE at the *sender*. This = critical=20 exception voids any normative requirement, even over iWARP. The = strongest=20 statement that can be made is SHOULD.

 

Tom.

 

From: storm-bounces@ietf.org [mailto:storm-bounces@iet= f.org]=20 On Behalf Of Hemal Shah
Sent: Wednesday, January 18, = 2012 6:56=20 PM
To: david.black@emc.com;=20 Michael@huaweisymantec.com= ; storm@ietf.org
Subject: Re: = [storm]=20 iSER - one last issue

 

David and=20 Mike,

 

I = believe=20 that the requirements should not be weakened and we should stick to the=20 requirement in RFC5046.

 

RFC5046=20 required the use of specific Send Message Type for valid=20 reasons.

 

In the case=20 of requiring SendSE message use, the intent was to give a hint to the = remote=20 side to generate an event upon processing of SendSE message. For = example, the=20 SendSE is specifically carrying SCSI information that needs attention on = the=20 remote side e.g. SCSI command, Logout=85

 

In the case=20 of requiring SendInvSE Message, the intent was to invalidate the STag = used in=20 the data transfer as well as inform the remote side to generate an = event. For=20 example, mandating the use of SendInvSE for the SCSI response PDU for a = SCSI=20 Read limits the exposure of STag used in SCSI Read data transfer and = avoids the=20 explicit invalidation of the STag at the initiator. Plus, it allows the = target=20 to generate an event on the initiator upon the completion of SCSI Read=20 command.

 

By relaxing=20 RFC5046 requirements in the latest iSER draft, we will not only impact = all iWARP=20 based iSER implementations that rely on the use of specific Send Message = Type=20 for SCSI data transfers but also change the intended behavior of = initiators and=20 targets.

 

For iWARP, I=20 strongly suggest that the iSER draft does not relax the requirements = specified=20 in RFC5046.

 

I = hope that=20 helps.

 

Hemal=20    

 

From: storm-bounces@ietf.org [mailto:storm-bounces@iet= f.org]=20 On Behalf Of david.black@emc.com
Sent:=20 Wednesday, January 18, 2012 3:10 PM
To: Michael@huaweisymantec.com= ; storm@ietf.org
Subject: Re: = [storm]=20 iSER - one last issue

 

Mike,

 

Thank you for=20 the explanation, but I don=92t believe you=92ve correctly=20 stated

the = intent of=20 RFC 5046.  Here are some examples from RFC 5046=92s=20 text:

 

1.5.  SCSI = Read=20 Overview

 

   The = iSER=20 Message is transferred to the target using a SendSE=20 Message.

 

   The = iSER layer=20 at the target uses a SendInvSE Message to transfer = the

   SCSI = Response=20 PDU back to the iSER layer at the initiator.

 

Similar language = occurs in=20 1.6 SCSI Write Overview.

 

5.2.1.  = Normal=20 Connection Termination at the Initiator

 

   The = iSER layer=20 at the initiator MUST

   use a = SendSE=20 Message to send the Logout Request PDU to the = target.

 

Similar language = occurs in=20 5.2.2 for target side normal connection = termination,

and=20 more importantly in 7.3.1 SCSI Command:

 

   The = iSER layer=20 at the initiator MUST send the SCSI command in a

   = SendSE Message=20 to the target.

 

None of the quoted = text=20 permits use of Send instead of SendSE or SendInv

instead of=20 SendInvSE.

 

What you propose = to do=20 effectively changes =93MUST=94 to =93should=94 (lower = case,

weaker than = =93SHOULD=94), and=20 that sure looks like a change to RFC 5046.

 

Are=20 there good implementation-based reasons to weaken these requirements=20 now?

 

Does = anyone=20 else have a viewpoint on this topic?

 

Thanks,
--David

 

From: Michael = Ko [mailto:Michael@huawe= isymantec.com]=20
Sent: Wednesday, January 18, 2012 2:22 PM
To: = Black, David;=20 storm@ietf.org
Subject: = Re: iSER -=20 one last issue

 

David,

 

Here is my = rationale for using=20 the lower case "should".

 

The intent of = RFC5046 is that=20 the Send Message type must be used instead of RDMA Reads or=20 Writes.  The Solicited Event feature of the Send Message is = provided=20 as a convenience.  The receiver must still do the right thing in = handling=20 the Send Message regardless of whether the SE feature is used.  In = other=20 words, the sender is responsible for using the right format for the = message=20 (Send vs RDMA) but the receiver must not rely on the sender to determine = how to=20 handle the received message.  The same rationale goes for the = Invalidate=20 feature. 

 

Mike

----- = Original=20 Message -----

From: david.black@emc.com=20

To: Michael@huaweisymantec.com= ; storm@ietf.org =

Sent: Monday, = January 16,=20 2012 2:06 PM

Subject: iSER - one = last=20 issue

 

Mike,

 

Thanks for=20 getting the -08 version of the iSER draft posted.  I think=20 that

draft=20 addresses all of the open issues, but I have a question for the=20 WG

about how to=20 express the SendSE and SendInvSE requirements from RFC=20 5046.

 

The = -08=20 version of the iSER draft expresses requirements for the=20 SendSE

and = SendInvSE=20 messages (this is primarily for iSER over RDDP/iWARP) = as:

 

The = SendSE=20 Message should be used if

supported by=20 the RCaP layer (e.g., iWARP).

 

My = reading of=20 RFC 5046 is that its requirements are tighter - to=20 accurately

reflect RFC=20 5046, I would replace =93should=94 with =93MUST=94 in the above=20 text,

at = least for=20 iWARP.

 

In = the=20 alternative, if the =93should=94s remain, an explanatory = item

needs to be=20 added to the Appendix A list of changes from RFC = 5046.

 

What = do others=20 think the right course of action is here, use =93MUST=94 = or

explain=20 weakening of requirement to =93should=94 ?

 

Thanks,
--David

 

From: storm-bounces@ietf.org [mailto:storm-bounces@iet= f.org]=20 On Behalf Of Michael Ko
Sent: Sunday, January 15, 2012 = 8:43=20 AM
To: storm@ietf.org
Subject: Re: = [storm]=20 I-D Action: = draft-ietf-storm-iser-08.txt

 

This version = contains all the=20 updates as discussed in the latest exchanges with Hemal Shah and David=20 Black.

 

Mike

----- = Original=20 Message -----

From: internet-drafts@ietf.org=20

To: i-d-announce@ietf.org=20

Cc: storm@ietf.org =

Sent: Sunday, = January 15,=20 2012 5:40 AM

Subject: [storm] = I-D Action:=20 draft-ietf-storm-iser-08.txt

 


A New Internet-Draft is available from the = on-line=20 Internet-Drafts directories. This draft is a work item of the STORage=20 Maintenance Working Group of the=20 IETF.

Title         &= nbsp; :=20 iSCSI Extensions for RDMA=20 Specification
Author(s)       : Michael = Ko
           &= nbsp;           &n= bsp; =20 Alexander = Nezhinsky
Filename        :=20 draft-ietf-storm-iser-08.txt
Pages      =     =20 : = 95
Date          &nb= sp; :=20 2012-01-15

   iSCSI Extensions for RDMA provides the = RDMA data=20 transfer capability
   to iSCSI by layering iSCSI on top of = an=20 RDMA-Capable Protocol.  An
   RDMA-Capable Protocol = provides=20 RDMA Read and Write services, which
   enable data to be=20 transferred directly into SCSI I/O Buffers without
   = intermediate=20 data copies.  This document describes the extensions = to
   the=20 iSCSI protocol to support RDMA services as provided by an = RDMA-
  =20 Capable Protocol.

   This document obsoletes RFC=20 5046.


A URL for this Internet-Draft is:
http://www.ietf.org/internet-drafts/draft-ietf-storm-iser-08.txt
=
Internet-Drafts=20 are also available by anonymous FTP at:
ftp://ftp.ietf.org/internet-= drafts/

This=20 Internet-Draft can be retrieved at:
= ftp://ftp.ietf.org/internet-drafts/draft-ietf-storm-iser-08.txt
_______________________________________________
storm=20 mailing list
storm@ietf.org
https://www.ietf.org= /mailman/listinfo/storm

--Boundary_(ID_ewEe8f5th1Tt8gBZ7EwHPw)-- From david.black@emc.com Thu Jan 19 16:03:55 2012 Return-Path: X-Original-To: storm@ietfa.amsl.com Delivered-To: storm@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 4501521F8625 for ; Thu, 19 Jan 2012 16:03:55 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -108.859 X-Spam-Level: X-Spam-Status: No, score=-108.859 tagged_above=-999 required=5 tests=[AWL=1.739, BAYES_00=-2.599, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_HI=-8, USER_IN_WHITELIST=-100] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id k2TiWXIzJZo2 for ; Thu, 19 Jan 2012 16:03:48 -0800 (PST) Received: from mexforward.lss.emc.com (mexforward.lss.emc.com [128.222.32.20]) by ietfa.amsl.com (Postfix) with ESMTP id 2C1D521F8621 for ; Thu, 19 Jan 2012 16:03:47 -0800 (PST) Received: from hop04-l1d11-si02.isus.emc.com (HOP04-L1D11-SI02.isus.emc.com [10.254.111.55]) by mexforward.lss.emc.com (Switch-3.4.3/Switch-3.4.3) with ESMTP id q0K03kv3030934 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO) for ; Thu, 19 Jan 2012 19:03:47 -0500 Received: from mailhub.lss.emc.com (mailhub.lss.emc.com [10.254.222.129]) by hop04-l1d11-si02.isus.emc.com (RSA Interceptor) for ; Thu, 19 Jan 2012 19:03:31 -0500 Received: from mxhub36.corp.emc.com (mxhub36.corp.emc.com [10.254.93.84]) by mailhub.lss.emc.com (Switch-3.4.3/Switch-3.4.3) with ESMTP id q0K03UTZ024497 for ; Thu, 19 Jan 2012 19:03:30 -0500 Received: from mx14a.corp.emc.com ([169.254.1.99]) by mxhub36.corp.emc.com ([::1]) with mapi; Thu, 19 Jan 2012 19:03:30 -0500 From: To: Date: Thu, 19 Jan 2012 19:03:29 -0500 Thread-Topic: iSER - one last issue Thread-Index: AQHM1tEoMWLUJLTWDk+GYyBugVM345YUHEswgABAqpA= Message-ID: <7C4DFCE962635144B8FAE8CA11D0BF1E05A7CF1027@MX14A.corp.emc.com> References: In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: acceptlanguage: en-US Content-Type: multipart/alternative; boundary="_000_7C4DFCE962635144B8FAE8CA11D0BF1E05A7CF1027MX14Acorpemcc_" MIME-Version: 1.0 X-EMM-MHVC: 1 X-Mailman-Approved-At: Thu, 19 Jan 2012 16:09:31 -0800 Subject: Re: [storm] iSER - one last issue X-BeenThere: storm@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Storage Maintenance WG List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 20 Jan 2012 00:03:55 -0000 --_000_7C4DFCE962635144B8FAE8CA11D0BF1E05A7CF1027MX14Acorpemcc_ Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable [WG co chair hat on] I think the implementation aspect of this discussion is useful. I also want to remind everyone that the following text from the storm WG ch= arter (http://datatracker.ietf.org/wg/storm/charter/) is highly relevant to the message usage topic under discussion: Stability is critical to the usage of these protocols, so backwards compatibility with existing implementations will be a requirement imposed on for all protocol changes and additions. Note that this is a requirement for implementation compatibility - if it is the case that all implementations of a protocol have done something different than what the RFC specifies, it is appropriate for a new RFC to document what the "running code" actually does and deprecate the unused original behavi= or. I think it's ok for iSER's message usage requirements to vary between iWARP and InfiniBand - such a difference could make it more difficult to run iSER over an RCaP gateway between iWARP and InfiniBand, but I don't think there'= s much interest in doing that. OTOH, all changes to RFC5046 need justificati= on under the "all implementations of a protocol have done something different than what the RFC specifies" condition quoted above. Thanks, --David (as WG co-chair) From: Tom Talpey [mailto:ttalpey@microsoft.com] Sent: Thursday, January 19, 2012 3:33 PM To: Michael Ko; Hemal Shah; Black, David; storm@ietf.org Subject: RE: iSER - one last issue More precisely, are there any iSER *initiators* that *depend* on receiving = solicited events over the iWARP RDMA transport? I believe this to be a very= short list, if any. But I'd be ok with being proven wrong. From: Michael Ko [mailto:Michael@huaweisymantec.com] Sent: Thursday, January 19, 2012 12:39 PM To: Hemal Shah; Tom Talpey; david.black@emc.com; storm@ietf.org Subject: Re: iSER - one last issue For all RCaP layers, the Send message (or any variant) MUST be used, as opp= osed to using RDMA Read or Write to accomplish the task. So a SHOULD for R= CaP other than iWARP is incorrect. RFC 5040 can mandate that all RDMAP mes= sages be supported, but it is still up to implementation to decide what mes= sages to use in any situation. This iSER update is meant to reflect runnin= g code and to generalize the support of iSER over any RCaP, not just iWARP.= So the question is are there any running code that uses SendSE and SendIn= vSE for iSER. Conversely, if the running code can get by using Send messag= es to accomplish the task, what is the point of mandating a MUST at this po= int for SendSE and SendInvSE? Note that the receiver cannot completely rel= y on the sender for the Invalidate feature anyway. The iSER layer at the i= nitiator is required to explicitly invalidate the STag for bidirectional co= mmands and abnormal completion of a command. Even when automatic invalidat= ion is used, the iSER layer is required to sanity checking the automatic in= validation. In other words, the iSER spec puts the burden on the receiver = for doing the right thing in handling the Send messages, and not relying on= the sender for the mere convenience. Mike ----- Original Message ----- From: Hemal Shah To: Tom Talpey ; david.black@emc.com ; Michael@huaweisymantec.com ; storm@ietf.org Sent: Thursday, January 19, 2012 8:44 AM Subject: RE: iSER - one last issue Tom, You are welcome! You are right about 7.3.4 as the only section mandating Se= nd message. I think your concern about specifying the requirement for the other RCaP im= plementation is valid. I suggest we should word the requirements as below: "If the RCaP layer is as specified in [RFC5040] and [RFC5041], the SendSE m= essage MUST be used. For any other RCaP layer, the Send message SHOULD be u= sed. " Hemal From: Tom Talpey [mailto:ttalpey@microsoft.com] Sent: Thursday, January 19, 2012 8:09 AM To: Hemal Shah; david.black@emc.com; Michael@hu= aweisymantec.com; storm@ietf.org Subject: RE: iSER - one last issue Thanks for pointing out that sentence in RFC5046 7.3.4. I believe it is the= only such stipulation. I'm ok with stating the requirement as RFC5046-over-RFC5040 compliance. But= this new draft opens the door to alternative behaviors, therefore it needs= to be stated more clearly. What's the requirement when SendSE is not suppo= rted? And, how does the receiver know what to expect? In other words, if it's proposed that the statement be: "When the RCaP layer is as specified in [RFC5040] and [RFC5= 041], the SendSE message MUST be used." ... then what MUST be used when the RCaP is not? That statement needs to be= made, too. From: Hemal Shah [mailto:hemal@broadcom.com] Sent: Thursday, January 19, 2012 10:38 AM To: Tom Talpey; david.black@emc.com; Michael@hu= aweisymantec.com; storm@ietf.org Subject: RE: iSER - one last issue Tom, RFC5046 is very clear about the use of Send Message Type for iSCSI control-= type PDU. For specific iSCSI PDUs, Section 7 mandates the use of Send, Send= SE, or SendSEInv. Section 7 of RFC5046 clearly specifies the use of specifi= c Send Message Type for each iSCSI control-type PDUs (Login, Logout, Text r= equest, Text response...). Section 7.3.4 mandates the use of plain-old Send= also as needed. For example, see text below from 7.3.4. For unsolicited data, if the F bit is set to 0 in a SCSI Data-out PDU, the iSER layer at the initiator MUST use a Send Message to send the SCSI Data-out PDU. Regarding your comment in the second paragraph below, RFC5040 expectation i= s all the RDMAP messages specified in RFC5040 are supported (if that is not= the case then we cannot count on any of the other RDMAP messages to be sup= ported). So, an implementation that does not support SendSE at the sender i= s not compliant with RFC5040. So, your second point does not apply. I hope that addresses your comments. Hemal From: Tom Talpey [mailto:ttalpey@microsoft.com] Sent: Thursday, January 19, 2012 5:51 AM To: Hemal Shah; david.black@emc.com; Michael@hu= aweisymantec.com; storm@ietf.org Subject: RE: iSER - one last issue If using SendSE is so important here, when MUST a plain-old Send be used? T= here is no mention of any distinction in RFC5046 section 1.4.2 nor in this = draft's section 2.4.2. Even with a MUST on the SendSE, the behavior is indeterminate, since it wou= ld still be contingent on support for SendSE at the *sender*. This critical= exception voids any normative requirement, even over iWARP. The strongest = statement that can be made is SHOULD. Tom. From: storm-bounces@ietf.org [mailto:storm-b= ounces@ietf.org] On Behalf Of Hemal= Shah Sent: Wednesday, January 18, 2012 6:56 PM To: david.black@emc.com; Michael@huaweisymantec= .com; storm@ietf.org Subject: Re: [storm] iSER - one last issue David and Mike, I believe that the requirements should not be weakened and we should stick = to the requirement in RFC5046. RFC5046 required the use of specific Send Message Type for valid reasons. In the case of requiring SendSE message use, the intent was to give a hint = to the remote side to generate an event upon processing of SendSE message. = For example, the SendSE is specifically carrying SCSI information that need= s attention on the remote side e.g. SCSI command, Logout... In the case of requiring SendInvSE Message, the intent was to invalidate th= e STag used in the data transfer as well as inform the remote side to gener= ate an event. For example, mandating the use of SendInvSE for the SCSI resp= onse PDU for a SCSI Read limits the exposure of STag used in SCSI Read data= transfer and avoids the explicit invalidation of the STag at the initiator= . Plus, it allows the target to generate an event on the initiator upon the= completion of SCSI Read command. By relaxing RFC5046 requirements in the latest iSER draft, we will not only= impact all iWARP based iSER implementations that rely on the use of specif= ic Send Message Type for SCSI data transfers but also change the intended b= ehavior of initiators and targets. For iWARP, I strongly suggest that the iSER draft does not relax the requir= ements specified in RFC5046. I hope that helps. Hemal From: storm-bounces@ietf.org [mailto:storm-b= ounces@ietf.org] On Behalf Of david= .black@emc.com Sent: Wednesday, January 18, 2012 3:10 PM To: Michael@huaweisymantec.com; storm@ie= tf.org Subject: Re: [storm] iSER - one last issue Mike, Thank you for the explanation, but I don't believe you've correctly stated the intent of RFC 5046. Here are some examples from RFC 5046's text: 1.5. SCSI Read Overview The iSER Message is transferred to the target using a SendSE Message. The iSER layer at the target uses a SendInvSE Message to transfer the SCSI Response PDU back to the iSER layer at the initiator. Similar language occurs in 1.6 SCSI Write Overview. 5.2.1. Normal Connection Termination at the Initiator The iSER layer at the initiator MUST use a SendSE Message to send the Logout Request PDU to the target. Similar language occurs in 5.2.2 for target side normal connection terminat= ion, and more importantly in 7.3.1 SCSI Command: The iSER layer at the initiator MUST send the SCSI command in a SendSE Message to the target. None of the quoted text permits use of Send instead of SendSE or SendInv instead of SendInvSE. What you propose to do effectively changes "MUST" to "should" (lower case, weaker than "SHOULD"), and that sure looks like a change to RFC 5046. Are there good implementation-based reasons to weaken these requirements no= w? Does anyone else have a viewpoint on this topic? Thanks, --David From: Michael Ko [mailto:Michael@huaweisymantec.com] Sent: Wednesday, January 18, 2012 2:22 PM To: Black, David; storm@ietf.org Subject: Re: iSER - one last issue David, Here is my rationale for using the lower case "should". The intent of RFC5046 is that the Send Message type must be used instead of= RDMA Reads or Writes. The Solicited Event feature of the Send Message is = provided as a convenience. The receiver must still do the right thing in h= andling the Send Message regardless of whether the SE feature is used. In = other words, the sender is responsible for using the right format for the m= essage (Send vs RDMA) but the receiver must not rely on the sender to deter= mine how to handle the received message. The same rationale goes for the I= nvalidate feature. Mike ----- Original Message ----- From: david.black@emc.com To: Michael@huaweisymantec.com ; storm@i= etf.org Sent: Monday, January 16, 2012 2:06 PM Subject: iSER - one last issue Mike, Thanks for getting the -08 version of the iSER draft posted. I think that draft addresses all of the open issues, but I have a question for the WG about how to express the SendSE and SendInvSE requirements from RFC 5046. The -08 version of the iSER draft expresses requirements for the SendSE and SendInvSE messages (this is primarily for iSER over RDDP/iWARP) as: The SendSE Message should be used if supported by the RCaP layer (e.g., iWARP). My reading of RFC 5046 is that its requirements are tighter - to accurately reflect RFC 5046, I would replace "should" with "MUST" in the above text, at least for iWARP. In the alternative, if the "should"s remain, an explanatory item needs to be added to the Appendix A list of changes from RFC 5046. What do others think the right course of action is here, use "MUST" or explain weakening of requirement to "should" ? Thanks, --David --_000_7C4DFCE962635144B8FAE8CA11D0BF1E05A7CF1027MX14Acorpemcc_ Content-Type: text/html; charset="us-ascii" Content-Transfer-Encoding: quoted-printable

= [WG = co chair hat on]

 

I think the implementation aspect of this discussi= on is useful.

 

I also want to remind everyone that the following tex= t from the storm WG charter

(http://datatracker.ietf.= org/wg/storm/charter/) is highly relevant to the

<= p class=3DMsoNormal>message usage topic under discussion:

=

 

  Stabil= ity is critical to the usage of these protocols, so backwards

  compatibility with existing implementations= will be a requirement

  imposed= on for all protocol changes and additions. Note that this is a<= /span>

  requirement for implementation compatibil= ity - if it is the case that

  a= ll implementations of a protocol have done something different than

  what the RFC specifies, it is appropr= iate for a new RFC to document what

&= nbsp; the "running code" actually does and deprecate the unused o= riginal behavior.

 

I think it’s ok for iSER’s message us= age requirements to vary between iWARP

over an RCaP gateway between iWARP= and InfiniBand, but I don’t think there’s

much interest in doing that.  OTOH, all changes to R= FC5046 need justification

under the &= #8220;all implementations of a protocol have done something different<= /o:p>

than what the RFC specifies” conditi= on quoted above.

 

Thanks,
--David (as WG co-chair)

 

From: Tom Talpey [mailto:ttalpey@microsoft.c= om]
Sent: Thursday, January 19, 2012 3:33 PM
To: Micha= el Ko; Hemal Shah; Black, David; storm@ietf.org
Subject: RE: iSER= - one last issue

 

More precisely, are there any= iSER *initiators* that *depend* on receiving solicited event= s over the iWARP RDMA transport? I believe this to be a very short list, if= any. But I’d be ok with being proven wrong.

 

<= span style=3D'font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F= 497D'> 

 

From: Michael Ko [mailto:Mi= chael@huaweisymantec.com]
Sent: Thursday, January 19, 2012 12:39= PM
To: Hemal Shah; Tom Talpey; david.black@emc.com; storm@ietf.o= rg
Subject: Re: iSER - one last issue

=

 

<= span style=3D'font-size:10.0pt'>For all RCaP layers, the Send message (or a= ny variant) MUST be used, as opposed to using RDMA Read or Write to accompl= ish the task.  So a SHOULD for RCaP other than iWARP is incorrect= .  RFC 5040 can mandate that all RDMAP messages be supported, but it i= s still up to implementation to decide what messages to use in any situatio= n.  This iSER update is meant to reflect running code and to generaliz= e the support of iSER over any RCaP, not just iWARP.  So the question = is are there any running code that uses SendSE and SendInvSE for iSER. = ; Conversely, if the running code can get by using Send messages to accompl= ish the task, what is the point of mandating a MUST at this point for SendS= E and SendInvSE?  Note that the receiver cannot completely rely on the= sender for the Invalidate feature anyway.  The iSER layer at the= initiator is required to explicitly invalidate the STag for bidirectional = commands and abnormal completion of a command.  Even when automat= ic invalidation is used, the iSER layer is required to sanity checking the = automatic invalidation.  In other words, the iSER spec puts = the burden on the receiver for doing the right thing in handling the Send m= essages, and not relying on the sender for the mere convenience. =

 

=

Mike

----- Original Message -----

Sent: Thursday, January 19, 2012 8:44 = AM

Subject: RE: iSER - one= last issue

 

Tom,

 

=

You a= re welcome! You are right about 7.3.4 as the only section mandating Send me= ssage.

 

I think your concern = about specifying the requirement for the other RCaP implementation is valid= . I suggest we should word the requirements as below:=

<= o:p> 

“If the RC= aP layer is as specified in [RFC5040] and [RFC5041], the SendSE message MUS= T be used. For any other RCaP layer, the Send message SHOULD be used. ̶= 1;

 

= Hemal

 

<= p class=3DMsoNormal>From: Tom Talpey [mailto:ttalpey@microsoft.com]
Sent: Thursday, Jan= uary 19, 2012 8:09 AM
To: Hemal Shah; david.black@emc.com; Michael@huaweisymantec.com; s= torm@ietf.org
Subject: RE: iSER - one last issue

 

Thanks for pointing out that sentence in RFC5046 7.3.4. I b= elieve it is the only such stipulation.

 

I= 217;m ok with stating the requirement as RFC5046-over-RFC5040 compliance. B= ut this new draft opens the door to alternative behaviors, therefore it nee= ds to be stated more clearly. What’s the requirement when SendSE is n= ot supported? And, how does the receiver know what to expect?

 

In other words, if it’s proposed that the statement = be:

   &nbs= p;            “= ;When the RCaP layer is as specified in [RFC5040] and [RFC5041], the SendSE= message MUST be used.”

 

… then wh= at MUST be used when the RCaP is not? That statement needs to be made, too.=

 <= /p>

 

<= p class=3DMsoNormal>From: Hemal Shah [mailto:hemal@broadcom.com]
Sent: Thursday, January 1= 9, 2012 10:38 AM
To: Tom Talpey; david.black@emc.com; Michael@huaweisymantec.com; storm@= ietf.org
Subject: RE: iSER - one last issue=

 

Tom,=

 

RFC5046 is very clear about the use of Send M= essage Type for iSCSI control-type PDU. For specific iSCSI PDUs, Section 7 = mandates the use of Send, SendSE, or SendSEInv. Section 7 of RFC5046 clearl= y specifies the use of specific Send Message Type for each iSCSI control-ty= pe PDUs (Login, Logout, Text request, Text response…). Section 7.3.4 = mandates the use of plain-old Send also as needed. For example, see text be= low from 7.3.4.

 

Fo= r unsolicited data, if the F bit is set to 0 in a SCSI Data-out<= /span>

PDU, the iSER layer at the initiator MUST use a Send Message= to send

the SCSI Data-out PDU.=

 

Regarding your comment in the second paragr= aph below, RFC5040 expectation is all the RDMAP messages specified in RFC50= 40 are supported (if that is not the case then we cannot count on any of th= e other RDMAP messages to be supported). So, an implementation that does no= t support SendSE at the sender is not compliant with RFC5040. So, your seco= nd point does not apply.

<= span style=3D'font-size:10.0pt;color:purple'> 

I h= ope that addresses your comments.

 

Hemal

 

From: Tom Talpey [mailto:ttalpey@microsoft.com]
Sent: Thursday,= January 19, 2012 5:51 AM
To: Hemal Shah; david.black@emc.com; Michael@huaweisymantec.com; storm@ietf.org
Subject: RE: iSER - one last issue

 

If using SendSE is so important here, when MUST a plain= -old Send be used? There is no mention of any distinction in RFC5046 sectio= n 1.4.2 nor in this draft’s section 2.4.2.

 

Even with a MUST on the SendSE, the behavior is indeterminate, since it= would still be contingent on support for SendSE at the *sender*. Th= is critical exception voids any normative requirement, even over iWARP. The= strongest statement that can be made is SHOULD.

 

Tom.

 =

From: storm-bounces@ietf.org [mailto:storm-bounces@ietf.org] On Behalf Of Hemal Shah
Sent: Wednesday, January 18, 2012 6:56 PM
To: david.black@emc.com; Michael@huaweisymantec.com; storm@ietf.org
Subject: Re: [st= orm] iSER - one last issue

 

David and Mike,

 =

I believe that the requirements should not be weakened and we sho= uld stick to the requirement in RFC5046.

 

RFC5046 required the use of specific Send Message Type for va= lid reasons.

 

In the case o= f requiring SendSE message use, the intent was to give a hint to the remote= side to generate an event upon processing of SendSE message. For example, = the SendSE is specifically carrying SCSI information that needs attention o= n the remote side e.g. SCSI command, Logout…

 

In the case of requiring SendInvSE Message, the inte= nt was to invalidate the STag used in the data transfer as well as inform t= he remote side to generate an event. For example, mandating the use of Send= InvSE for the SCSI response PDU for a SCSI Read limits the exposure of STag= used in SCSI Read data transfer and avoids the explicit invalidation of th= e STag at the initiator. Plus, it allows the target to generate an event on= the initiator upon the completion of SCSI Read command.<= /b>

 

By relaxing RFC5046 requirements in the latest= iSER draft, we will not only impact all iWARP based iSER implementations t= hat rely on the use of specific Send Message Type for SCSI data transfers b= ut also change the intended behavior of initiators and targets.<= /span>

 

For iWARP, I strongly suggest that the = iSER draft does not relax the requirements specified in RFC5046.=

 

I hope that helps.

 

Hemal    <= /p>

 

From: storm-bounces@ietf.org [mailto:storm-bounces@ietf.org] = On Behalf Of david.black@emc.= com
Sent: Wednesday, January 18, 2012 3:10 PM
To: <= a href=3D"mailto:Michael@huaweisymantec.com">Michael@huaweisymantec.com= ; storm@ietf.org
Subject: R= e: [storm] iSER - one last issue

 

Mike,<= /p>

 

Thank you = for the explanation, but I don’t believe you’ve correctly state= d

the intent of RFC 5046.  Here = are some examples from RFC 5046’s text:

 

1.5.  SCSI Read Overview

 

   The iS= ER Message is transferred to the target using a SendSE Message.<= /span>

 

   Th= e iSER layer at the target uses a SendInvSE Message to transfer the

   SCSI Response PDU back to the iSER layer at= the initiator.

 

= Similar language occurs in 1.6 SCSI Write Overview.

5.2.1.  Normal Connection Terminat= ion at the Initiator

 

   The iSER layer at the initiator MUST

   use a SendSE Message to send the Logout Request PDU to = the target.

 

Sim= ilar language occurs in 5.2.2 for target side normal connection termination= ,

and more importantly in 7.3.1 SCSI Command:

 

   The iS= ER layer at the initiator MUST send the SCSI command in a=

   SendSE Message to the target.

None of the quoted text permits use of = Send instead of SendSE or SendInv

instead of Send= InvSE.

 

What you = propose to do effectively changes “MUST” to “should”= ; (lower case,

weaker than “SHOULD”),= and that sure looks like a change to RFC 5046.

<= o:p> 

Are there good implementation-based reasons= to weaken these requirements now?

 

Does anyone else have a vie= wpoint on this topic?

 

Thanks,
--David

 

From: Michael Ko [mailto:Michael@huaweisymantec.com]
Sent: Wednesd= ay, January 18, 2012 2:22 PM
To: Black, David; storm@ietf.org
Subject: Re: iSER - one last i= ssue

 

David,

 

Here is my rati= onale for using the lower case "should".

 

The intent of RFC5046 is that the S= end Message type must be used instead of RDMA Reads or Writes.  T= he Solicited Event feature of the Send Message is provided as a convenience= .  The receiver must still do the right thing in handling the Send Mes= sage regardless of whether the SE feature is used.  In other words, th= e sender is responsible for using the right format for the message (Se= nd vs RDMA) but the receiver must not rely on the sender to determine how t= o handle the received message.  The same rationale goes for the Invali= date feature. 

&= nbsp;

Mike

----- Original M= essage -----

Sent: Monday, January 16, 2012 2:06 PM

Subject: iSER - one last iss= ue

 

Mike,

=  

Thanks for getting the -08 ver= sion of the iSER draft posted.  I think that

draft addresses all of the open issues, but I have a question = for the WG

about how to express the S= endSE and SendInvSE requirements from RFC 5046.

 

The -08 version of = the iSER draft expresses requirements for the SendSE

<= p class=3DMsoNormal>and SendInvSE messages (this is primarily for iSER over RDD= P/iWARP) as:

 =

The SendSE Message should b= e used if

= supported by the RCaP layer (e.g., iWARP).

 

My reading of RFC 5046 is that its requirements are tighter - to accurat= ely

reflect RFC 5046, I would replace= “should” with “MUST” in the above text,=

at least for iWARP.

 

In the alternative,= if the “should”s remain, an explanatory item=

needs to be added to the Appendix A list of changes fr= om RFC 5046.

 =

What do others think the right course of action is her= e, use “MUST” or

explain = weakening of requirement to “should” ?

 

Thanks,
= --David

= --_000_7C4DFCE962635144B8FAE8CA11D0BF1E05A7CF1027MX14Acorpemcc_-- From david.black@emc.com Mon Jan 23 17:05:13 2012 Return-Path: X-Original-To: storm@ietfa.amsl.com Delivered-To: storm@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 96B9B21F85C7 for ; Mon, 23 Jan 2012 17:05:13 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -109.111 X-Spam-Level: X-Spam-Status: No, score=-109.111 tagged_above=-999 required=5 tests=[AWL=1.487, BAYES_00=-2.599, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_HI=-8, USER_IN_WHITELIST=-100] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id soLPe4zMwbPw for ; Mon, 23 Jan 2012 17:05:06 -0800 (PST) Received: from mexforward.lss.emc.com (mexforward.lss.emc.com [128.222.32.20]) by ietfa.amsl.com (Postfix) with ESMTP id 0A03C21F85C4 for ; Mon, 23 Jan 2012 17:05:05 -0800 (PST) Received: from hop04-l1d11-si01.isus.emc.com (HOP04-L1D11-SI01.isus.emc.com [10.254.111.54]) by mexforward.lss.emc.com (Switch-3.4.3/Switch-3.4.3) with ESMTP id q0O1549R009232 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO) for ; Mon, 23 Jan 2012 20:05:05 -0500 Received: from mailhub.lss.emc.com (mailhub.lss.emc.com [10.254.221.145]) by hop04-l1d11-si01.isus.emc.com (RSA Interceptor) for ; Mon, 23 Jan 2012 20:04:56 -0500 Received: from mxhub32.corp.emc.com (mxhub32.corp.emc.com [128.222.70.172]) by mailhub.lss.emc.com (Switch-3.4.3/Switch-3.4.3) with ESMTP id q0O14uP2003696 for ; Mon, 23 Jan 2012 20:04:56 -0500 Received: from mx14a.corp.emc.com ([169.254.1.99]) by mxhub32.corp.emc.com ([128.222.70.172]) with mapi; Mon, 23 Jan 2012 20:04:56 -0500 From: To: Date: Mon, 23 Jan 2012 20:04:55 -0500 Thread-Topic: iSER - one last issue: update Thread-Index: AQHM1tEoMWLUJLTWDk+GYyBugVM345YUHEswgAaV+WA= Message-ID: <7C4DFCE962635144B8FAE8CA11D0BF1E05A7D9137A@MX14A.corp.emc.com> References: In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: acceptlanguage: en-US Content-Type: multipart/alternative; boundary="_000_7C4DFCE962635144B8FAE8CA11D0BF1E05A7D9137AMX14Acorpemcc_" MIME-Version: 1.0 X-EMM-MHVC: 1 X-Mailman-Approved-At: Mon, 23 Jan 2012 17:08:17 -0800 Subject: [storm] iSER - one last issue: update X-BeenThere: storm@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Storage Maintenance WG List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 24 Jan 2012 01:05:13 -0000 --_000_7C4DFCE962635144B8FAE8CA11D0BF1E05A7D9137AMX14Acorpemcc_ Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable [WG chair hat off] This is a message intended for discussion and comments. Looking for a way forward, this observation from Mike seems like a useful p= lace to start: > This iSER update is meant to reflect running code and to generalize the s= upport of iSER over any RCaP, not just iWARP. RFC 5046 (iSER) was written on the assumption that the RCaP layer supports = Solicited Event functionality. I also read RFC 5040 as requiring RDMAP implementatio= ns (part of iWARP) to support Solicited Event functionality. This first step seems easy: [1] There are RCaPs that don't support Solicited Event functionality; those= RCaPs have no choice - they MUST use Send (not SendSE) and SendInv (not SendInvSE). I would prefer to write iSER requirements in terms of whether the RCaP supp= orts Solicited Event functionality as opposed to calling out iWARP explicitly as= having different requirements. OTOH, I'm prepared to listen to arguments to the c= ontrary. Beyond that, I think there are two primary choices: [2a] Leave things alone - if the RCaP supports Solicited Events, then the "= MUST" requirements for SendSE and SendInvSE apply as they did in RFC 5046. [2b] In the alternative, if there are iSER implementations for iWARP that a= ren't using SendSE and SendInvSE in accordance with RFC 5046's requirements, then we ne= ed to change those requirements to match the implementations. I would hope we can agree on [1], and I believe that'll cover the InfiniBan= d case, right? For choosing between [2a] and [2b] it would help a lot to konw what iSER ov= er iWARP implementations are actually doing (SendSE/SendInvSE vs. Send/SendInv). *** Who is familiar with the iSER over iWARP implementation(s) ??? *** In general, what do folks think ought to be done here - [2a], [2b] or somet= hing else? [WG chair hat on] We do need to resolve this issue and reflect that resolution in a -09 iSER = draft before RFC publication can be requested. Thanks, --David From: Tom Talpey [mailto:ttalpey@microsoft.com] Sent: Thursday, January 19, 2012 3:33 PM To: Michael Ko; Hemal Shah; Black, David; storm@ietf.org Subject: RE: iSER - one last issue More precisely, are there any iSER *initiators* that *depend* on receiving = solicited events over the iWARP RDMA transport? I believe this to be a very= short list, if any. But I'd be ok with being proven wrong. From: Michael Ko [mailto:Michael@huaweisymantec.com] Sent: Thursday, January 19, 2012 12:39 PM To: Hemal Shah; Tom Talpey; david.black@emc.com; storm@ietf.org Subject: Re: iSER - one last issue For all RCaP layers, the Send message (or any variant) MUST be used, as opp= osed to using RDMA Read or Write to accomplish the task. So a SHOULD for R= CaP other than iWARP is incorrect. RFC 5040 can mandate that all RDMAP mes= sages be supported, but it is still up to implementation to decide what mes= sages to use in any situation. This iSER update is meant to reflect runnin= g code and to generalize the support of iSER over any RCaP, not just iWARP.= So the question is are there any running code that uses SendSE and SendIn= vSE for iSER. Conversely, if the running code can get by using Send messag= es to accomplish the task, what is the point of mandating a MUST at this po= int for SendSE and SendInvSE? Note that the receiver cannot completely rel= y on the sender for the Invalidate feature anyway. The iSER layer at the i= nitiator is required to explicitly invalidate the STag for bidirectional co= mmands and abnormal completion of a command. Even when automatic invalidat= ion is used, the iSER layer is required to sanity checking the automatic in= validation. In other words, the iSER spec puts the burden on the receiver = for doing the right thing in handling the Send messages, and not relying on= the sender for the mere convenience. Mike ----- Original Message ----- From: Hemal Shah To: Tom Talpey ; david.black@emc.com ; Michael@huaweisymantec.com ; storm@ietf.org Sent: Thursday, January 19, 2012 8:44 AM Subject: RE: iSER - one last issue Tom, You are welcome! You are right about 7.3.4 as the only section mandating Se= nd message. I think your concern about specifying the requirement for the other RCaP im= plementation is valid. I suggest we should word the requirements as below: "If the RCaP layer is as specified in [RFC5040] and [RFC5041], the SendSE m= essage MUST be used. For any other RCaP layer, the Send message SHOULD be u= sed. " Hemal From: Tom Talpey [mailto:ttalpey@microsoft.com] Sent: Thursday, January 19, 2012 8:09 AM To: Hemal Shah; david.black@emc.com; Michael@hu= aweisymantec.com; storm@ietf.org Subject: RE: iSER - one last issue Thanks for pointing out that sentence in RFC5046 7.3.4. I believe it is the= only such stipulation. I'm ok with stating the requirement as RFC5046-over-RFC5040 compliance. But= this new draft opens the door to alternative behaviors, therefore it needs= to be stated more clearly. What's the requirement when SendSE is not suppo= rted? And, how does the receiver know what to expect? In other words, if it's proposed that the statement be: "When the RCaP layer is as specified in [RFC5040] and [RFC5= 041], the SendSE message MUST be used." ... then what MUST be used when the RCaP is not? That statement needs to be= made, too. From: Hemal Shah [mailto:hemal@broadcom.com] Sent: Thursday, January 19, 2012 10:38 AM To: Tom Talpey; david.black@emc.com; Michael@hu= aweisymantec.com; storm@ietf.org Subject: RE: iSER - one last issue Tom, RFC5046 is very clear about the use of Send Message Type for iSCSI control-= type PDU. For specific iSCSI PDUs, Section 7 mandates the use of Send, Send= SE, or SendSEInv. Section 7 of RFC5046 clearly specifies the use of specifi= c Send Message Type for each iSCSI control-type PDUs (Login, Logout, Text r= equest, Text response...). Section 7.3.4 mandates the use of plain-old Send= also as needed. For example, see text below from 7.3.4. For unsolicited data, if the F bit is set to 0 in a SCSI Data-out PDU, the iSER layer at the initiator MUST use a Send Message to send the SCSI Data-out PDU. Regarding your comment in the second paragraph below, RFC5040 expectation i= s all the RDMAP messages specified in RFC5040 are supported (if that is not= the case then we cannot count on any of the other RDMAP messages to be sup= ported). So, an implementation that does not support SendSE at the sender i= s not compliant with RFC5040. So, your second point does not apply. I hope that addresses your comments. Hemal From: Tom Talpey [mailto:ttalpey@microsoft.com] Sent: Thursday, January 19, 2012 5:51 AM To: Hemal Shah; david.black@emc.com; Michael@hu= aweisymantec.com; storm@ietf.org Subject: RE: iSER - one last issue If using SendSE is so important here, when MUST a plain-old Send be used? T= here is no mention of any distinction in RFC5046 section 1.4.2 nor in this = draft's section 2.4.2. Even with a MUST on the SendSE, the behavior is indeterminate, since it wou= ld still be contingent on support for SendSE at the *sender*. This critical= exception voids any normative requirement, even over iWARP. The strongest = statement that can be made is SHOULD. Tom. From: storm-bounces@ietf.org [mailto:storm-b= ounces@ietf.org] On Behalf Of Hemal= Shah Sent: Wednesday, January 18, 2012 6:56 PM To: david.black@emc.com; Michael@huaweisymantec= .com; storm@ietf.org Subject: Re: [storm] iSER - one last issue David and Mike, I believe that the requirements should not be weakened and we should stick = to the requirement in RFC5046. RFC5046 required the use of specific Send Message Type for valid reasons. In the case of requiring SendSE message use, the intent was to give a hint = to the remote side to generate an event upon processing of SendSE message. = For example, the SendSE is specifically carrying SCSI information that need= s attention on the remote side e.g. SCSI command, Logout... In the case of requiring SendInvSE Message, the intent was to invalidate th= e STag used in the data transfer as well as inform the remote side to gener= ate an event. For example, mandating the use of SendInvSE for the SCSI resp= onse PDU for a SCSI Read limits the exposure of STag used in SCSI Read data= transfer and avoids the explicit invalidation of the STag at the initiator= . Plus, it allows the target to generate an event on the initiator upon the= completion of SCSI Read command. By relaxing RFC5046 requirements in the latest iSER draft, we will not only= impact all iWARP based iSER implementations that rely on the use of specif= ic Send Message Type for SCSI data transfers but also change the intended b= ehavior of initiators and targets. For iWARP, I strongly suggest that the iSER draft does not relax the requir= ements specified in RFC5046. I hope that helps. Hemal From: storm-bounces@ietf.org [mailto:storm-b= ounces@ietf.org] On Behalf Of david= .black@emc.com Sent: Wednesday, January 18, 2012 3:10 PM To: Michael@huaweisymantec.com; storm@ie= tf.org Subject: Re: [storm] iSER - one last issue Mike, Thank you for the explanation, but I don't believe you've correctly stated the intent of RFC 5046. Here are some examples from RFC 5046's text: 1.5. SCSI Read Overview The iSER Message is transferred to the target using a SendSE Message. The iSER layer at the target uses a SendInvSE Message to transfer the SCSI Response PDU back to the iSER layer at the initiator. Similar language occurs in 1.6 SCSI Write Overview. 5.2.1. Normal Connection Termination at the Initiator The iSER layer at the initiator MUST use a SendSE Message to send the Logout Request PDU to the target. Similar language occurs in 5.2.2 for target side normal connection terminat= ion, and more importantly in 7.3.1 SCSI Command: The iSER layer at the initiator MUST send the SCSI command in a SendSE Message to the target. None of the quoted text permits use of Send instead of SendSE or SendInv instead of SendInvSE. What you propose to do effectively changes "MUST" to "should" (lower case, weaker than "SHOULD"), and that sure looks like a change to RFC 5046. Are there good implementation-based reasons to weaken these requirements no= w? Does anyone else have a viewpoint on this topic? Thanks, --David From: Michael Ko [mailto:Michael@huaweisymantec.com] Sent: Wednesday, January 18, 2012 2:22 PM To: Black, David; storm@ietf.org Subject: Re: iSER - one last issue David, Here is my rationale for using the lower case "should". The intent of RFC5046 is that the Send Message type must be used instead of= RDMA Reads or Writes. The Solicited Event feature of the Send Message is = provided as a convenience. The receiver must still do the right thing in h= andling the Send Message regardless of whether the SE feature is used. In = other words, the sender is responsible for using the right format for the m= essage (Send vs RDMA) but the receiver must not rely on the sender to deter= mine how to handle the received message. The same rationale goes for the I= nvalidate feature. Mike ----- Original Message ----- From: david.black@emc.com To: Michael@huaweisymantec.com ; storm@i= etf.org Sent: Monday, January 16, 2012 2:06 PM Subject: iSER - one last issue Mike, Thanks for getting the -08 version of the iSER draft posted. I think that draft addresses all of the open issues, but I have a question for the WG about how to express the SendSE and SendInvSE requirements from RFC 5046. The -08 version of the iSER draft expresses requirements for the SendSE and SendInvSE messages (this is primarily for iSER over RDDP/iWARP) as: The SendSE Message should be used if supported by the RCaP layer (e.g., iWARP). My reading of RFC 5046 is that its requirements are tighter - to accurately reflect RFC 5046, I would replace "should" with "MUST" in the above text, at least for iWARP. In the alternative, if the "should"s remain, an explanatory item needs to be added to the Appendix A list of changes from RFC 5046. What do others think the right course of action is here, use "MUST" or explain weakening of requirement to "should" ? Thanks, --David --_000_7C4DFCE962635144B8FAE8CA11D0BF1E05A7D9137AMX14Acorpemcc_ Content-Type: text/html; charset="us-ascii" Content-Transfer-Encoding: quoted-printable

= [WG = chair hat off]

 

This is a message intended for discussion and commen= ts.

 

Looking for a way forward, this observation from Mike seems lik= e a useful place to start:

 = ;

>= ; This iSER update is meant to reflect running code and to generalize the s= upport of iSER over any RCaP, not just iWARP. 

 

RFC 504= 6 (iSER) was written on the assumption that the RCaP layer supports Solicit= ed

Event functionality.  I also = read RFC 5040 as requiring RDMAP implementations (part of=

iWARP) to support Solicited Event functionality.<= /o:p>

 

This first step seems easy:

&= nbsp;

[1] There are RCaPs that don’t= support Solicited Event functionality; those RCaPs have<= /p>

no choice - they MUST use Send (not SendSE) and SendInv= (not SendInvSE). 

  =             &nb= sp;            =             &nb= sp;            =             &nb= sp;            =             &nb= sp;  

I would prefer to wr= ite iSER requirements in terms of whether the RCaP supports

Solicited Event functionality as opposed to calling = out iWARP explicitly as having

differ= ent requirements.  OTOH, I’m prepared to listen to arguments to = the contrary.

 

Beyond that, I think there are two primary choices:

 

[2a] Leave things alone - if the RCaP supports Solicited Events, t= hen the “MUST”

requiremen= ts for SendSE and SendInvSE apply as they did in RFC 5046.

 

[2b] In = the alternative, if there are iSER implementations for iWARP that aren̵= 7;t using

SendSE and SendInvSE in acc= ordance with RFC 5046’s requirements, then we need to change

those requirements to match the implementatio= ns.

 

I would hope we can agree on [1], and I believe that’ll c= over the InfiniBand case, right?

=  

For choosing between [2a] and [2b]= it would help a lot to konw what iSER over iWARP

implementations are actually doing (SendSE/SendInvSE vs. Send/= SendInv).

 

*** Who is familiar with the iSER over iWARP implementati= on(s) ??? ***

 

In general, what do folks think ought to be done here= - [2a], [2b] or something else?

=  

[WG chair hat on]

 

We do ne= ed to resolve this issue and reflect that resolution in a -09 iSER draft be= fore

RFC publication can be requested= .

 

Thanks,
--David

 

<= span style=3D'font-size:10.0pt;font-family:"Tahoma","sans-serif"'>From: = Tom Talpey [mailto:ttalpey@microsoft.com]
Sent: Thursday, Januar= y 19, 2012 3:33 PM
To: Michael Ko; Hemal Shah; Black, David; stor= m@ietf.org
Subject: RE: iSER - one last issue

 

More precisely, are there any iSER *initiators* that *de= pend* on receiving solicited events over the iWARP RDMA transport? I be= lieve this to be a very short list, if any. But I’d be ok with being = proven wrong.

 <= /o:p>

 

 

From: Michael Ko [mailto:Michael@huaweisymantec.com]
Sent= : Thursday, January 19, 2012 12:39 PM
To: Hemal Shah; Tom Tal= pey; david.black@emc.com; storm@ietf.org
Subject: Re: iSER - one = last issue

 = ;

For a= ll RCaP layers, the Send message (or any variant) MUST be used, as opposed = to using RDMA Read or Write to accomplish the task.  So a SHOULD for&n= bsp;RCaP other than iWARP is incorrect.  RFC 5040 can mandate that all= RDMAP messages be supported, but it is still up to implementation to decid= e what messages to use in any situation.  This iSER update is meant to= reflect running code and to generalize the support of iSER over any RCaP, = not just iWARP.  So the question is are there any running code that us= es SendSE and SendInvSE for iSER.  Conversely, if the running code can= get by using Send messages to accomplish the task, what is the point of ma= ndating a MUST at this point for SendSE and SendInvSE?  Note that the = receiver cannot completely rely on the sender for the Invalidate feature an= yway.  The iSER layer at the initiator is required to explicitly = invalidate the STag for bidirectional commands and abnormal completion of a= command.  Even when automatic invalidation is used, the iSER lay= er is required to sanity checking the automatic invalidation.  In= other words, the iSER spec puts the burden on the receiver for doing = the right thing in handling the Send messages, and not relying on the = sender for the mere convenience. 

 

Mike

= Sent: Thursday, January 19, 2012 8:44 AM

=

Subject: RE: iSER - one last issue

 

Tom,

 

You are welcome! You are right about 7.3= .4 as the only section mandating Send message.

&nb= sp;

I think your concern about specifying the requirement fo= r the other RCaP implementation is valid. I suggest we should word the requ= irements as below:

 

“If the RCaP layer is as specified in [RFC504= 0] and [RFC5041], the SendSE message MUST be used. For any other RCaP layer= , the Send message SHOULD be used. ”

 

Hemal

&nbs= p;

From: Tom Talpey [mailto:ttalpey@microsoft.c= om]
Sent: Thursday, January 19, 2012 8:09 AM
To: H= emal Shah; david.black@emc.com; = Michael@huaweisymantec.com; storm@ietf.org
Subject: = RE: iSER - one last issue

 

Thanks for pointing o= ut that sentence in RFC5046 7.3.4. I believe it is the only such stipulatio= n.

 

I’m ok with stating the requirement = as RFC5046-over-RFC5040 compliance. But this new draft opens the door to al= ternative behaviors, therefore it needs to be stated more clearly. WhatR= 17;s the requirement when SendSE is not supported? And, how does the receiv= er know what to expect?

 

In other words, if it= ’s proposed that the statement be:

          =       “When the RCaP layer is as specified i= n [RFC5040] and [RFC5041], the SendSE message MUST be used.”

 

… then what MUST be used when the RCaP is not? = That statement needs to be made, too.

 

&n= bsp;

From: Hemal Shah [mailto:hemal@broadcom.com] =
Sent: Thursday, January 19, 2012 10:38 AM
To: Tom Talp= ey; david.black@emc.com; Michael@huaweisymantec.com; storm@ietf.org
Subject: RE: iSE= R - one last issue

 

Tom,

 

RFC5046 i= s very clear about the use of Send Message Type for iSCSI control-type PDU.= For specific iSCSI PDUs, Section 7 mandates the use of Send, SendSE, or Se= ndSEInv. Section 7 of RFC5046 clearly specifies the use of specific Send Me= ssage Type for each iSCSI control-type PDUs (Login, Logout, Text request, T= ext response…). Section 7.3.4 mandates the use of plain-old Send also= as needed. For example, see text below from 7.3.4.

 

For unsolicited data, if the F bit is set= to 0 in a SCSI Data-out

PDU, the iSER layer at t= he initiator MUST use a Send Message to send

the= SCSI Data-out PDU.

 

Rega= rding your comment in the second paragraph below, RFC5040 expectation is al= l the RDMAP messages specified in RFC5040 are supported (if that is not the= case then we cannot count on any of the other RDMAP messages to be support= ed). So, an implementation that does not support SendSE at the sender is no= t compliant with RFC5040. So, your second point does not apply.<= /span>

 

I hope that addresses your comments.

 

<= span style=3D'font-size:10.0pt;color:purple'>Hemal

 

From:<= span style=3D'font-size:10.0pt;font-family:"Tahoma","sans-serif"'> Tom Talp= ey [mailto:ttalpey@micros= oft.com]
Sent: Thursday, January 19, 2012 5:51 AM
To:<= /b> Hemal Shah; david.black@emc.com<= /a>; Michael@huaweisymantec.c= om; storm@ietf.org
Subject:= RE: iSER - one last issue

 

If using SendSE = is so important here, when MUST a plain-old Send be used? There is no menti= on of any distinction in RFC5046 section 1.4.2 nor in this draft’s se= ction 2.4.2.

 

Even with a MUST on the SendSE, = the behavior is indeterminate, since it would still be contingent on suppor= t for SendSE at the *sender*. This critical exception voids any norm= ative requirement, even over iWARP. The strongest statement that can be mad= e is SHOULD.

 

Tom.

 

From: storm-bounces@ietf.= org [mailto:storm-bo= unces@ietf.org] On Behalf Of Hemal Shah
Sent: Wednesda= y, January 18, 2012 6:56 PM
To: david.black@emc.com; Michael@huaweisymantec.com; storm@i= etf.org
Subject: Re: [storm] iSER - one last issue=

 

David and Mik= e,

 

I believe that the requir= ements should not be weakened and we should stick to the requirement in RFC= 5046.

 

RFC5046 required the u= se of specific Send Message Type for valid reasons.

 

In the case of requiring SendSE message use, the in= tent was to give a hint to the remote side to generate an event upon proces= sing of SendSE message. For example, the SendSE is specifically carrying SC= SI information that needs attention on the remote side e.g. SCSI command, L= ogout…

 

In the case o= f requiring SendInvSE Message, the intent was to invalidate the STag used i= n the data transfer as well as inform the remote side to generate an event.= For example, mandating the use of SendInvSE for the SCSI response PDU for = a SCSI Read limits the exposure of STag used in SCSI Read data transfer and= avoids the explicit invalidation of the STag at the initiator. Plus, it al= lows the target to generate an event on the initiator upon the completion o= f SCSI Read command.

 

By rela= xing RFC5046 requirements in the latest iSER draft, we will not only impact= all iWARP based iSER implementations that rely on the use of specific Send= Message Type for SCSI data transfers but also change the intended behavior= of initiators and targets.

<= b> 

= For iWARP, I strongly suggest that the iSER draft does not relax the requir= ements specified in RFC5046.

=  

I hope that helps.

 

Hemal &n= bsp;  

 

<= div style=3D'border:none;border-top:solid #B5C4DF 1.0pt;padding:3.0pt 0in 0= in 0in'>

From: = storm-bounces@ietf.org [mailto:storm-bounces@ietf.org] On Behalf Of david.black@emc.com
Sent: Wednesday, J= anuary 18, 2012 3:10 PM
To: Michael@huaweisymantec.com; = storm@ietf.org
Subject: Re: [storm] iSER - one last issue

 

Mike,

 

Thank you for the explanation, but I don̵= 7;t believe you’ve correctly stated

the intent of RFC 5046.  Here are some examples from RFC 5046R= 17;s text:

 

1.5.  SCSI Read Overview

 <= /o:p>

   The iSER Message is transferred to the ta= rget using a SendSE Message.

&nb= sp;

   The iSER layer at the target uses a S= endInvSE Message to transfer the

=    SCS= I Response PDU back to the iSER layer at the initiator.

 

Similar language occurs in 1.6 SCSI= Write Overview.

 

5.2.1.  Normal Connection Termination at the Initiator

 

   The iSER layer at= the initiator MUST

   use a SendSE M= essage to send the Logout Request PDU to the target.

<= p class=3DMsoNormal> 

Similar language occurs in 5.2.2 for t= arget side normal connection termination,

and mor= e importantly in 7.3.1 SCSI Command:

 <= /o:p>

   The iSER layer at the initiator MUST send= the SCSI command in a

   SendSE Messag= e to the target.

 

None of the quoted text permits use of Send instead of SendSE or SendInv

instead of SendInvSE.

 

What you propose to do effectively changes &= #8220;MUST” to “should” (lower case,

weaker than “SHOULD”), and that sure looks like a change = to RFC 5046.

 

Are= there good implementation-based reasons to weaken these requirements now?<= o:p>

 

Does anyone else have a viewpoint on this topic?

 

<= span style=3D'font-size:10.0pt;font-family:"Courier New";color:black'>Thank= s,
--David

 

From: Michael Ko [mailto:Michael@huaweisy= mantec.com]
Sent: Wednesday, January 18, 2012 2:22 PM
= To: Black, David; storm@ietf.org<= br>Subject: Re: iSER - one last issue

 

David,

 

Here is my rationale for using the lower case &quo= t;should".

 =

The intent of RFC5046 is that the Send Message type must be used inste= ad of RDMA Reads or Writes.  The Solicited Event feature of the S= end Message is provided as a convenience.  The receiver must still do = the right thing in handling the Send Message regardless of whether the SE f= eature is used.  In other words, the sender is responsible for using t= he right format for the message (Send vs RDMA) but the receiver must n= ot rely on the sender to determine how to handle the received message. = ; The same rationale goes for the Invalidate feature. 

 

Mike

----- Original Message -----

=

Se= nt: Monday, January 16, 2012 2:06 PM

Subject: iSER - one last issue

 

Mik= e,

 

Thanks for getting the -08 version of the iSER draft posted.&nbs= p; I think that

draft addresses all o= f the open issues, but I have a question for the WG

 

The -08 version of the iSER draft expresses requireme= nts for the SendSE

and SendInvSE mess= ages (this is primarily for iSER over RDDP/iWARP) as:

=

 

The SendSE Message should be used if

supported by the RCaP layer (e.g.,= iWARP).

<= o:p> 

My reading of RFC 5046 is that = its requirements are tighter - to accurately

reflect RFC 5046, I would replace “should” with “= ;MUST” in the above text,

<= span style=3D'font-size:10.0pt;font-family:"Courier New";color:black'>at le= ast for iWARP.

 

In the alternative, if the “should”s rem= ain, an explanatory item

needs to be = added to the Appendix A list of changes from RFC 5046.

 

What do othe= rs think the right course of action is here, use “MUST” or=

explain weakening of requirement to ̶= 0;should” ?

 

Thanks,
--David

= --_000_7C4DFCE962635144B8FAE8CA11D0BF1E05A7D9137AMX14Acorpemcc_-- From ttalpey@microsoft.com Tue Jan 24 05:48:50 2012 Return-Path: X-Original-To: storm@ietfa.amsl.com Delivered-To: storm@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 11E2021F854A for ; Tue, 24 Jan 2012 05:48:50 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -104.098 X-Spam-Level: X-Spam-Status: No, score=-104.098 tagged_above=-999 required=5 tests=[AWL=-0.500, BAYES_00=-2.599, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_LOW=-1, USER_IN_WHITELIST=-100] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id OPwguBezDELA for ; Tue, 24 Jan 2012 05:48:42 -0800 (PST) Received: from VA3EHSOBE010.bigfish.com (va3ehsobe010.messaging.microsoft.com [216.32.180.30]) by ietfa.amsl.com (Postfix) with ESMTP id CFA2721F84F4 for ; Tue, 24 Jan 2012 05:48:41 -0800 (PST) Received: from mail177-va3-R.bigfish.com (10.7.14.248) by VA3EHSOBE010.bigfish.com (10.7.40.12) with Microsoft SMTP Server id 14.1.225.22; Tue, 24 Jan 2012 13:48:41 +0000 Received: from mail177-va3 (localhost [127.0.0.1]) by mail177-va3-R.bigfish.com (Postfix) with ESMTP id 165422C0219 for ; Tue, 24 Jan 2012 13:48:41 +0000 (UTC) X-SpamScore: -56 X-BigFish: VS-56(zz9371I1503Mc85fh148cM542M1432N1418Mc1dM4015Lzz1202hzz1033IL8275bh8275dhz2fhc1bhc31hc1ahc1bhc31hc1ah2a8h668h839h) X-Forefront-Antispam-Report: CIP:131.107.125.8; KIP:(null); UIP:(null); IPV:NLI; H:TK5EX14MLTC101.redmond.corp.microsoft.com; RD:none; EFVD:NLI Received-SPF: pass (mail177-va3: domain of microsoft.com designates 131.107.125.8 as permitted sender) client-ip=131.107.125.8; envelope-from=ttalpey@microsoft.com; helo=TK5EX14MLTC101.redmond.corp.microsoft.com ; icrosoft.com ; Received: from mail177-va3 (localhost.localdomain [127.0.0.1]) by mail177-va3 (MessageSwitch) id 1327412916594650_7606; Tue, 24 Jan 2012 13:48:36 +0000 (UTC) Received: from VA3EHSMHS036.bigfish.com (unknown [10.7.14.248]) by mail177-va3.bigfish.com (Postfix) with ESMTP id 8B10C4C0084; Tue, 24 Jan 2012 13:48:36 +0000 (UTC) Received: from TK5EX14MLTC101.redmond.corp.microsoft.com (131.107.125.8) by VA3EHSMHS036.bigfish.com (10.7.99.46) with Microsoft SMTP Server (TLS) id 14.1.225.23; Tue, 24 Jan 2012 13:48:34 +0000 Received: from TK5EX14MBXC111.redmond.corp.microsoft.com ([169.254.2.216]) by TK5EX14MLTC101.redmond.corp.microsoft.com ([157.54.79.178]) with mapi id 14.02.0247.005; Tue, 24 Jan 2012 05:48:31 -0800 From: Tom Talpey To: "david.black@emc.com" , "storm@ietf.org" Thread-Topic: iSER - one last issue: update Thread-Index: AQHM2jStU9vvKyghkE2rGcTbpoeGCJYbg95A Date: Tue, 24 Jan 2012 13:48:30 +0000 Message-ID: References: <7C4DFCE962635144B8FAE8CA11D0BF1E05A7D9137A@MX14A.corp.emc.com> In-Reply-To: <7C4DFCE962635144B8FAE8CA11D0BF1E05A7D9137A@MX14A.corp.emc.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [157.54.51.34] Content-Type: multipart/alternative; boundary="_000_F83812DF4B59B9499C1BC978336D917461CF96F4TK5EX14MBXC111r_" MIME-Version: 1.0 X-OriginatorOrg: microsoft.com X-Mailman-Approved-At: Tue, 24 Jan 2012 06:32:22 -0800 Subject: Re: [storm] iSER - one last issue: update X-BeenThere: storm@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Storage Maintenance WG List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 24 Jan 2012 13:48:50 -0000 --_000_F83812DF4B59B9499C1BC978336D917461CF96F4TK5EX14MBXC111r_ Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable [WG co-chair hat off] > [2a] Leave things alone - if the RCaP supports Solicited Events, then the= "MUST" > requirements for SendSE and SendInvSE apply as they did in RFC 5046. The core issue here is the statement "if the RCaP supports Solicited Events= ". Both the iWARP and Infiniband protocols support them. But Infiniband doe= s not require all adapters support them - the architecture requires it for = HCAs but leaves it optional for TCAs (Infiniband architecture v1.2.1 table = 319 on page 1045). Because this latter property cannot be detected by the r= emote upper layer, it does not appear to be possible to make an interoperab= le decision for all protocols and all implementations. That said, in the absence of any existing iSER-over-iWARP SendSE implementa= tions, I agree the entire discussion is effectively moot. Tom. From: storm-bounces@ietf.org [mailto:storm-bounces@ietf.org] On Behalf Of d= avid.black@emc.com Sent: Monday, January 23, 2012 8:05 PM To: storm@ietf.org Subject: [storm] iSER - one last issue: update [WG chair hat off] This is a message intended for discussion and comments. Looking for a way forward, this observation from Mike seems like a useful p= lace to start: > This iSER update is meant to reflect running code and to generalize the s= upport of iSER over any RCaP, not just iWARP. RFC 5046 (iSER) was written on the assumption that the RCaP layer supports = Solicited Event functionality. I also read RFC 5040 as requiring RDMAP implementatio= ns (part of iWARP) to support Solicited Event functionality. This first step seems easy: [1] There are RCaPs that don't support Solicited Event functionality; those= RCaPs have no choice - they MUST use Send (not SendSE) and SendInv (not SendInvSE). I would prefer to write iSER requirements in terms of whether the RCaP supp= orts Solicited Event functionality as opposed to calling out iWARP explicitly as= having different requirements. OTOH, I'm prepared to listen to arguments to the c= ontrary. Beyond that, I think there are two primary choices: [2a] Leave things alone - if the RCaP supports Solicited Events, then the "= MUST" requirements for SendSE and SendInvSE apply as they did in RFC 5046. [2b] In the alternative, if there are iSER implementations for iWARP that a= ren't using SendSE and SendInvSE in accordance with RFC 5046's requirements, then we ne= ed to change those requirements to match the implementations. I would hope we can agree on [1], and I believe that'll cover the InfiniBan= d case, right? For choosing between [2a] and [2b] it would help a lot to konw what iSER ov= er iWARP implementations are actually doing (SendSE/SendInvSE vs. Send/SendInv). *** Who is familiar with the iSER over iWARP implementation(s) ??? *** In general, what do folks think ought to be done here - [2a], [2b] or somet= hing else? [WG chair hat on] We do need to resolve this issue and reflect that resolution in a -09 iSER = draft before RFC publication can be requested. Thanks, --David From: Tom Talpey [mailto:ttalpey@microsoft.com] Sent: Thursday, January 19, 2012 3:33 PM To: Michael Ko; Hemal Shah; Black, David; storm@ietf.org Subject: RE: iSER - one last issue More precisely, are there any iSER *initiators* that *depend* on receiving = solicited events over the iWARP RDMA transport? I believe this to be a very= short list, if any. But I'd be ok with being proven wrong. From: Michael Ko [mailto:Michael@huaweisymantec.com] Sent: Thursday, January 19, 2012 12:39 PM To: Hemal Shah; Tom Talpey; david.black@emc.com= ; storm@ietf.org Subject: Re: iSER - one last issue For all RCaP layers, the Send message (or any variant) MUST be used, as opp= osed to using RDMA Read or Write to accomplish the task. So a SHOULD for R= CaP other than iWARP is incorrect. RFC 5040 can mandate that all RDMAP mes= sages be supported, but it is still up to implementation to decide what mes= sages to use in any situation. This iSER update is meant to reflect runnin= g code and to generalize the support of iSER over any RCaP, not just iWARP.= So the question is are there any running code that uses SendSE and SendIn= vSE for iSER. Conversely, if the running code can get by using Send messag= es to accomplish the task, what is the point of mandating a MUST at this po= int for SendSE and SendInvSE? Note that the receiver cannot completely rel= y on the sender for the Invalidate feature anyway. The iSER layer at the i= nitiator is required to explicitly invalidate the STag for bidirectional co= mmands and abnormal completion of a command. Even when automatic invalidat= ion is used, the iSER layer is required to sanity checking the automatic in= validation. In other words, the iSER spec puts the burden on the receiver = for doing the right thing in handling the Send messages, and not relying on= the sender for the mere convenience. Mike ----- Original Message ----- From: Hemal Shah To: Tom Talpey ; david.black@emc.com ; Michael@huaweisymantec.com ; storm@ietf.org Sent: Thursday, January 19, 2012 8:44 AM Subject: RE: iSER - one last issue Tom, You are welcome! You are right about 7.3.4 as the only section mandating Se= nd message. I think your concern about specifying the requirement for the other RCaP im= plementation is valid. I suggest we should word the requirements as below: "If the RCaP layer is as specified in [RFC5040] and [RFC5041], the SendSE m= essage MUST be used. For any other RCaP layer, the Send message SHOULD be u= sed. " Hemal From: Tom Talpey [mailto:ttalpey@microsoft.com] Sent: Thursday, January 19, 2012 8:09 AM To: Hemal Shah; david.black@emc.com; Michael@hu= aweisymantec.com; storm@ietf.org Subject: RE: iSER - one last issue Thanks for pointing out that sentence in RFC5046 7.3.4. I believe it is the= only such stipulation. I'm ok with stating the requirement as RFC5046-over-RFC5040 compliance. But= this new draft opens the door to alternative behaviors, therefore it needs= to be stated more clearly. What's the requirement when SendSE is not suppo= rted? And, how does the receiver know what to expect? In other words, if it's proposed that the statement be: "When the RCaP layer is as specified in [RFC5040] and [RFC5= 041], the SendSE message MUST be used." ... then what MUST be used when the RCaP is not? That statement needs to be= made, too. From: Hemal Shah [mailto:hemal@broadcom.com] Sent: Thursday, January 19, 2012 10:38 AM To: Tom Talpey; david.black@emc.com; Michael@hu= aweisymantec.com; storm@ietf.org Subject: RE: iSER - one last issue Tom, RFC5046 is very clear about the use of Send Message Type for iSCSI control-= type PDU. For specific iSCSI PDUs, Section 7 mandates the use of Send, Send= SE, or SendSEInv. Section 7 of RFC5046 clearly specifies the use of specifi= c Send Message Type for each iSCSI control-type PDUs (Login, Logout, Text r= equest, Text response...). Section 7.3.4 mandates the use of plain-old Send= also as needed. For example, see text below from 7.3.4. For unsolicited data, if the F bit is set to 0 in a SCSI Data-out PDU, the iSER layer at the initiator MUST use a Send Message to send the SCSI Data-out PDU. Regarding your comment in the second paragraph below, RFC5040 expectation i= s all the RDMAP messages specified in RFC5040 are supported (if that is not= the case then we cannot count on any of the other RDMAP messages to be sup= ported). So, an implementation that does not support SendSE at the sender i= s not compliant with RFC5040. So, your second point does not apply. I hope that addresses your comments. Hemal From: Tom Talpey [mailto:ttalpey@microsoft.com] Sent: Thursday, January 19, 2012 5:51 AM To: Hemal Shah; david.black@emc.com; Michael@hu= aweisymantec.com; storm@ietf.org Subject: RE: iSER - one last issue If using SendSE is so important here, when MUST a plain-old Send be used? T= here is no mention of any distinction in RFC5046 section 1.4.2 nor in this = draft's section 2.4.2. Even with a MUST on the SendSE, the behavior is indeterminate, since it wou= ld still be contingent on support for SendSE at the *sender*. This critical= exception voids any normative requirement, even over iWARP. The strongest = statement that can be made is SHOULD. Tom. From: storm-bounces@ietf.org [mailto:storm-b= ounces@ietf.org] On Behalf Of Hemal= Shah Sent: Wednesday, January 18, 2012 6:56 PM To: david.black@emc.com; Michael@huaweisymantec= .com; storm@ietf.org Subject: Re: [storm] iSER - one last issue David and Mike, I believe that the requirements should not be weakened and we should stick = to the requirement in RFC5046. RFC5046 required the use of specific Send Message Type for valid reasons. In the case of requiring SendSE message use, the intent was to give a hint = to the remote side to generate an event upon processing of SendSE message. = For example, the SendSE is specifically carrying SCSI information that need= s attention on the remote side e.g. SCSI command, Logout... In the case of requiring SendInvSE Message, the intent was to invalidate th= e STag used in the data transfer as well as inform the remote side to gener= ate an event. For example, mandating the use of SendInvSE for the SCSI resp= onse PDU for a SCSI Read limits the exposure of STag used in SCSI Read data= transfer and avoids the explicit invalidation of the STag at the initiator= . Plus, it allows the target to generate an event on the initiator upon the= completion of SCSI Read command. By relaxing RFC5046 requirements in the latest iSER draft, we will not only= impact all iWARP based iSER implementations that rely on the use of specif= ic Send Message Type for SCSI data transfers but also change the intended b= ehavior of initiators and targets. For iWARP, I strongly suggest that the iSER draft does not relax the requir= ements specified in RFC5046. I hope that helps. Hemal From: storm-bounces@ietf.org [mailto:storm-b= ounces@ietf.org] On Behalf Of david= .black@emc.com Sent: Wednesday, January 18, 2012 3:10 PM To: Michael@huaweisymantec.com; storm@ie= tf.org Subject: Re: [storm] iSER - one last issue Mike, Thank you for the explanation, but I don't believe you've correctly stated the intent of RFC 5046. Here are some examples from RFC 5046's text: 1.5. SCSI Read Overview The iSER Message is transferred to the target using a SendSE Message. The iSER layer at the target uses a SendInvSE Message to transfer the SCSI Response PDU back to the iSER layer at the initiator. Similar language occurs in 1.6 SCSI Write Overview. 5.2.1. Normal Connection Termination at the Initiator The iSER layer at the initiator MUST use a SendSE Message to send the Logout Request PDU to the target. Similar language occurs in 5.2.2 for target side normal connection terminat= ion, and more importantly in 7.3.1 SCSI Command: The iSER layer at the initiator MUST send the SCSI command in a SendSE Message to the target. None of the quoted text permits use of Send instead of SendSE or SendInv instead of SendInvSE. What you propose to do effectively changes "MUST" to "should" (lower case, weaker than "SHOULD"), and that sure looks like a change to RFC 5046. Are there good implementation-based reasons to weaken these requirements no= w? Does anyone else have a viewpoint on this topic? Thanks, --David From: Michael Ko [mailto:Michael@huaweisymantec.com] Sent: Wednesday, January 18, 2012 2:22 PM To: Black, David; storm@ietf.org Subject: Re: iSER - one last issue David, Here is my rationale for using the lower case "should". The intent of RFC5046 is that the Send Message type must be used instead of= RDMA Reads or Writes. The Solicited Event feature of the Send Message is = provided as a convenience. The receiver must still do the right thing in h= andling the Send Message regardless of whether the SE feature is used. In = other words, the sender is responsible for using the right format for the m= essage (Send vs RDMA) but the receiver must not rely on the sender to deter= mine how to handle the received message. The same rationale goes for the I= nvalidate feature. Mike ----- Original Message ----- From: david.black@emc.com To: Michael@huaweisymantec.com ; storm@i= etf.org Sent: Monday, January 16, 2012 2:06 PM Subject: iSER - one last issue Mike, Thanks for getting the -08 version of the iSER draft posted. I think that draft addresses all of the open issues, but I have a question for the WG about how to express the SendSE and SendInvSE requirements from RFC 5046. The -08 version of the iSER draft expresses requirements for the SendSE and SendInvSE messages (this is primarily for iSER over RDDP/iWARP) as: The SendSE Message should be used if supported by the RCaP layer (e.g., iWARP). My reading of RFC 5046 is that its requirements are tighter - to accurately reflect RFC 5046, I would replace "should" with "MUST" in the above text, at least for iWARP. In the alternative, if the "should"s remain, an explanatory item needs to be added to the Appendix A list of changes from RFC 5046. What do others think the right course of action is here, use "MUST" or explain weakening of requirement to "should" ? Thanks, --David --_000_F83812DF4B59B9499C1BC978336D917461CF96F4TK5EX14MBXC111r_ Content-Type: text/html; charset="us-ascii" Content-Transfer-Encoding: quoted-printable

[WG co-chair hat off]

 <= /p>

> [2a] Leave things alone - if the RCaP sup= ports Solicited Events, then the “MUST”

> requirements for SendSE and SendInvSE app= ly as they did in RFC 5046.

 <= /p>

The core issue here is th= e statement “if the RCaP supports Solicited Events”. Both the i= WARP and Infiniband protocols support them. But Infiniband does not require all adapters support them – the architecture requires it for= HCAs but leaves it optional for TCAs (Infiniband architecture v1.2.1 table= 319 on page 1045). Because this latter property cannot be detected by the = remote upper layer, it does not appear to be possible to make an interoperable decision for all protocols and all= implementations.

 <= /p>

That said, in the absence= of any existing iSER-over-iWARP SendSE implementations, I agree the entire= discussion is effectively moot.

 <= /p>

Tom.

 <= /p>

From: storm-bo= unces@ietf.org [mailto:storm-bounces@ietf.org] On Behalf Of david.black@emc.com
Sent: Monday, January 23, 2012 8:05 PM
To: storm@ietf.org
Subject: [storm] iSER - one last issue: update

 

[WG chair hat off]

 

This is a message intended for discussion and = comments.

 

Looking for a way forward, this observation fr= om Mike seems like a useful place to start:

 

> This iSER upda= te is meant to reflect running code and to generalize the support of iSER o= ver any RCaP, not just iWARP. 

 

RFC 5046 (iSER) was written on the assumption = that the RCaP layer supports Solicited

Event functionality.  I also read RFC 504= 0 as requiring RDMAP implementations (part of

iWARP) to support Solicited Event functionalit= y.

 

This first step seems easy:<= /p>

 

[1] There are RCaPs that don’t support S= olicited Event functionality; those RCaPs have

no choice - they MUST use Send (not SendSE) an= d SendInv (not SendInvSE). 

       &nbs= p;            &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;         

I would prefer to write iSER requirements in t= erms of whether the RCaP supports

Solicited Event functionality as opposed to ca= lling out iWARP explicitly as having

different requirements.  OTOH, I’m = prepared to listen to arguments to the contrary.

 

Beyond that, I think there are two primary cho= ices:

 

[2a] Leave things alone - if the RCaP supports= Solicited Events, then the “MUST”

requirements for SendSE and SendInvSE apply as= they did in RFC 5046.

 

[2b] In the alternative, if there are iSER imp= lementations for iWARP that aren’t using

SendSE and SendInvSE in accordance with RFC 50= 46’s requirements, then we need to change

those requirements to match the implementation= s.

 

I would hope we can agree on [1], and I believ= e that’ll cover the InfiniBand case, right?

 

For choosing between [2a] and [2b] it would he= lp a lot to konw what iSER over iWARP

implementations are actually doing (SendSE/Sen= dInvSE vs. Send/SendInv).

 

*** Who is familiar with the iSER over iWARP i= mplementation(s) ??? ***

 

In general, what do folks think ought to be do= ne here - [2a], [2b] or something else?

 

[WG chair hat on]

 

We do need to resolve this issue and reflect t= hat resolution in a -09 iSER draft before

RFC publication can be requested.

 

Thanks,
--David

 

From: Tom Talp= ey [mailto:ttalpey@microsoft= .com]
Sent: Thursday, January 19, 2012 3:33 PM
To: Michael Ko; Hemal Shah; Black, David; storm@ietf.org
Subject: RE: iSER - one last issue

 

More precisely, are there= any iSER *initiators* that *depend* on receiving solicited e= vents over the iWARP RDMA transport? I believe this to be a very short list, if any. But I’d be ok with being proven wrong.

 <= /p>

 <= /p>

 <= /p>

From: Michael = Ko [mailto:Michael@huaw= eisymantec.com]
Sent: Thursday, January 19, 2012 12:39 PM
To: Hemal Shah; Tom Talpey; d= avid.black@emc.com; storm@ietf.org
Subject: Re: iSER - one last issue

 

For all RCaP layers= , the Send message (or any variant) MUST be used, as opposed to using RDMA = Read or Write to accomplish the task.  So a SHOULD for RCaP other= than iWARP is incorrect.  RFC 5040 can mandate that all RDMAP messages be supported, but it is still up to implementation= to decide what messages to use in any situation.  This iSER update is= meant to reflect running code and to generalize the support of iSER over a= ny RCaP, not just iWARP.  So the question is are there any running code that uses SendSE and SendInvSE for iSER.&nbs= p; Conversely, if the running code can get by using Send messages to accomp= lish the task, what is the point of mandating a MUST at this point for Send= SE and SendInvSE?  Note that the receiver cannot completely rely on the sender for the Invalidate feature anyway.&nb= sp; The iSER layer at the initiator is required to explicitly invalida= te the STag for bidirectional commands and abnormal completion of a command= .  Even when automatic invalidation is used, the iSER layer is required to sanity checking the automatic invalidation.&= nbsp; In other words, the iSER spec puts the burden on the receiv= er for doing the right thing in handling the Send messages, and not re= lying on the sender for the mere convenience. 

 

Mike

 

Tom= ,

 

You= are welcome! You are right about 7.3.4 as the only section mandating Send = message.

 

I t= hink your concern about specifying the requirement for the other RCaP imple= mentation is valid. I suggest we should word the requirements as below:

 

“If the RCaP layer = is as specified in [RFC5040] and [RFC5041], the SendSE message MUST be used= . For any other RCaP layer, the Send message SHOULD be used. ”

 <= /p>

Hem= al

 

From: Tom Talp= ey [mailto:ttalpey@microsoft= .com]
Sent: Thursday, January 19, 2012 8:09 AM
To: Hemal Shah; david.black@e= mc.com; Michael@huaweisymantec.com; storm@ietf.org
Subject: RE: iSER - one last issue

 

Thanks for pointing out t= hat sentence in RFC5046 7.3.4. I believe it is the only such stipulation.

 <= /p>

I’m ok with stating= the requirement as RFC5046-over-RFC5040 compliance. But this new draft ope= ns the door to alternative behaviors, therefore it needs to be stated more clearly. What’s the requirement when SendSE is not suppo= rted? And, how does the receiver know what to expect?

 <= /p>

In other words, if itR= 17;s proposed that the statement be:

    &= nbsp;           “Wh= en the RCaP layer is as specified in [RFC5040] and [RFC5041], the SendSE me= ssage MUST be used.”

 <= /p>

… then what MUST be= used when the RCaP is not? That statement needs to be made, too.

 <= /p>

 <= /p>

 

Tom= ,

 

RFC= 5046 is very clear about the use of Send Message Type for iSCSI control-typ= e PDU. For specific iSCSI PDUs, Section 7 mandates the use of Send, SendSE,= or SendSEInv. Section 7 of RFC5046 clearly specifies the use of specific Send Message Type for each iSCSI con= trol-type PDUs (Login, Logout, Text request, Text response…). Section= 7.3.4 mandates the use of plain-old Send also as needed. For example, see = text below from 7.3.4.

 

For unsolicited data, if the F bit is set to 0 in a SCSI D= ata-out

PDU, the iSER layer at the initiator MUST use a Send Messa= ge to send

the SCSI Data-out PDU.

 

Reg= arding your comment in the second paragraph below, RFC5040 expectation is a= ll the RDMAP messages specified in RFC5040 are supported (if that is not th= e case then we cannot count on any of the other RDMAP messages to be supported). So, an implementation that does= not support SendSE at the sender is not compliant with RFC5040. So, your s= econd point does not apply.

 

I h= ope that addresses your comments.

 

Hem= al

 

From: Tom Talp= ey [mailto:ttalpey@microsoft= .com]
Sent: Thursday, January 19, 2012 5:51 AM
To: Hemal Shah; david.black@e= mc.com; Michael@huaweisymantec.com; storm@ietf.org
Subject: RE: iSER - one last issue

 

If using SendSE is so imp= ortant here, when MUST a plain-old Send be used? There is no mention of any= distinction in RFC5046 section 1.4.2 nor in this draft’s section 2.4.2.

 <= /p>

Even with a MUST on the S= endSE, the behavior is indeterminate, since it would still be contingent on= support for SendSE at the *sender*. This critical exception voids any normative requirement, even over iWARP. The strongest = statement that can be made is SHOULD.

 <= /p>

Tom.

 <= /p>

From: storm-bounces@ietf.org [mailto:storm-bounces@ietf.org] On Behalf Of Hemal Shah
Sent: Wednesday, January 18, 2012 6:56 PM
To: david.black@emc.com; = Michael@huaweisymantec.com; storm@iet= f.org
Subject: Re: [storm] iSER - one last issue

 

Dav= id and Mike,

 

I b= elieve that the requirements should not be weakened and we should stick to = the requirement in RFC5046.

 

RFC= 5046 required the use of specific Send Message Type for valid reasons.=

 

In = the case of requiring SendSE message use, the intent was to give a hint to = the remote side to generate an event upon processing of SendSE message. For= example, the SendSE is specifically carrying SCSI information that needs attention on the remote side e.g. SCS= I command, Logout…

 

In = the case of requiring SendInvSE Message, the intent was to invalidate the S= Tag used in the data transfer as well as inform the remote side to generate= an event. For example, mandating the use of SendInvSE for the SCSI response PDU for a SCSI Read limits the expo= sure of STag used in SCSI Read data transfer and avoids the explicit invali= dation of the STag at the initiator. Plus, it allows the target to generate= an event on the initiator upon the completion of SCSI Read command.

 

By = relaxing RFC5046 requirements in the latest iSER draft, we will not only im= pact all iWARP based iSER implementations that rely on the use of specific = Send Message Type for SCSI data transfers but also change the intended behavior of initiators and targets.

 

For= iWARP, I strongly suggest that the iSER draft does not relax the requireme= nts specified in RFC5046.

 

I h= ope that helps.

 

Hem= al    

 

From: storm-bounces@ietf.org [mailto:storm-bounces@ietf.org] On Behalf Of david.black@emc.com
Sent: Wednesday, January 18, 2012 3:10 PM
To: Michael@huaweisyma= ntec.com; storm@ietf.org
Subject: Re: [storm] iSER - one last issue

 

Mike,

 

Thank you for the explanation, but I don’= ;t believe you’ve correctly stated

the intent of RFC 5046.  Here are some ex= amples from RFC 5046’s text:

 

1.5.  SCSI Read Overview

 

   The iSER Message is transferred to the target= using a SendSE Message.

 

   The iSER layer at the target uses a SendInvSE= Message to transfer the

   SCSI Response PDU back to the iSER layer at t= he initiator.

 

Similar language occurs in 1.6 SCSI Write Overview.

 

5.2.1.  Normal Connection Termination at the Initiato= r

 

   The iSER layer at the initiator MUST

   use a SendSE Message to send the Logout Reque= st PDU to the target.

 

Similar language occurs in 5.2.2 for target side normal co= nnection termination,

and more importantly in 7.3.1 SCSI Command:

 

   The iSER layer at the initiator MUST send the= SCSI command in a

   SendSE Message to the target.

 

None of the quoted text permits use of Send instead of Sen= dSE or SendInv

instead of SendInvSE.

 

What you propose to do effectively changes “MUSTR= 21; to “should” (lower case,

weaker than “SHOULD”), and that sure looks lik= e a change to RFC 5046.

 

Are there good implementation-based reasons to weaken thes= e requirements now?

 

Does anyone else have a viewpoint on this topi= c?

 

Thanks,
--David

 

From: Michael = Ko [mailto:Michael@huaw= eisymantec.com]
Sent: Wednesday, January 18, 2012 2:22 PM
To: Black, David; storm@ietf.org
Subject: Re: iSER - one last issue

 

David,<= /o:p>

 

Here is my rational= e for using the lower case "should".

 

The intent of RFC50= 46 is that the Send Message type must be used instead of RDMA Reads or Writ= es.  The Solicited Event feature of the Send Message is provided = as a convenience.  The receiver must still do the right thing in handling the Send Message regardless of whether the SE = feature is used.  In other words, the sender is responsible for using = the right format for the message (Send vs RDMA) but the receiver must = not rely on the sender to determine how to handle the received message.  The same rationale goes for the Invalid= ate feature. 

 

Mike

 

Mike,

 

Thanks for getting the -08 version of the iSER= draft posted.  I think that

draft addresses all of the open issues, but I = have a question for the WG

about how to express the SendSE and SendInvSE = requirements from RFC 5046.

 

The -08 version of the iSER draft expresses re= quirements for the SendSE

and SendInvSE messages (this is primarily for = iSER over RDDP/iWARP) as:

 

The SendSE Message = should be used if

supported by the RC= aP layer (e.g., iWARP).

 

My reading of RFC 5046 is that its requirement= s are tighter - to accurately

reflect RFC 5046, I would replace “shoul= d” with “MUST” in the above text,

at least for iWARP.

 

In the alternative, if the “should”= ;s remain, an explanatory item

needs to be added to the Appendix A list of ch= anges from RFC 5046.

 

What do others think the right course of actio= n is here, use “MUST” or

explain weakening of requirement to “sho= uld” ?

 

Thanks,
--David

--_000_F83812DF4B59B9499C1BC978336D917461CF96F4TK5EX14MBXC111r_-- From cbm@chadalapaka.com Tue Jan 24 16:25:48 2012 Return-Path: X-Original-To: storm@ietfa.amsl.com Delivered-To: storm@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 3417711E809D for ; Tue, 24 Jan 2012 16:25:48 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.598 X-Spam-Level: X-Spam-Status: No, score=-2.598 tagged_above=-999 required=5 tests=[BAYES_00=-2.599, HTML_MESSAGE=0.001] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id FKDNh1oEyBvn for ; Tue, 24 Jan 2012 16:25:40 -0800 (PST) Received: from snt0-omc4-s6.snt0.hotmail.com (snt0-omc4-s6.snt0.hotmail.com [65.55.90.209]) by ietfa.amsl.com (Postfix) with ESMTP id 8FD2511E8096 for ; Tue, 24 Jan 2012 16:25:37 -0800 (PST) Received: from SNT106-DS16 ([65.55.90.199]) by snt0-omc4-s6.snt0.hotmail.com with Microsoft SMTPSVC(6.0.3790.4675); Tue, 24 Jan 2012 16:25:37 -0800 X-Originating-IP: [131.107.0.85] X-Originating-Email: [cbm@chadalapaka.com] Message-ID: From: Mallikarjun Chadalapaka To: "Tom Talpey" , , References: <7C4DFCE962635144B8FAE8CA11D0BF1E05A7D9137A@MX14A.corp.emc.com> In-Reply-To: Date: Tue, 24 Jan 2012 16:25:21 -0800 MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="----=_NextPart_000_0015_01CCDAB4.CD31E770" X-Mailer: Microsoft Outlook 14.0 Thread-Index: AQCOMSfeSH/fN63SI/bKvLYFC94DHAImI546ApHpQMMBoXLTSphmqTBQ Content-Language: en-us X-OriginalArrivalTime: 25 Jan 2012 00:25:37.0090 (UTC) FILETIME=[DC12D220:01CCDAF7] Subject: Re: [storm] iSER - one last issue: update X-BeenThere: storm@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Storage Maintenance WG List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 25 Jan 2012 00:25:48 -0000 ------=_NextPart_000_0015_01CCDAB4.CD31E770 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit I think we should go with [1] and [2a] that David outlined. As Hemal said, there were good reasons why RFC 5046 stipulated SendInvSE and SendSE, where it did. I would rather not weaken it, at least for the iWARP RCaP ecosystem. Mallikarjun From: storm-bounces@ietf.org [mailto:storm-bounces@ietf.org] On Behalf Of Tom Talpey Sent: Tuesday, January 24, 2012 5:49 AM To: david.black@emc.com; storm@ietf.org Subject: Re: [storm] iSER - one last issue: update [WG co-chair hat off] > [2a] Leave things alone - if the RCaP supports Solicited Events, then the "MUST" > requirements for SendSE and SendInvSE apply as they did in RFC 5046. The core issue here is the statement "if the RCaP supports Solicited Events". Both the iWARP and Infiniband protocols support them. But Infiniband does not require all adapters support them - the architecture requires it for HCAs but leaves it optional for TCAs (Infiniband architecture v1.2.1 table 319 on page 1045). Because this latter property cannot be detected by the remote upper layer, it does not appear to be possible to make an interoperable decision for all protocols and all implementations. That said, in the absence of any existing iSER-over-iWARP SendSE implementations, I agree the entire discussion is effectively moot. Tom. From: storm-bounces@ietf.org [mailto:storm-bounces@ietf.org] On Behalf Of david.black@emc.com Sent: Monday, January 23, 2012 8:05 PM To: storm@ietf.org Subject: [storm] iSER - one last issue: update [WG chair hat off] This is a message intended for discussion and comments. Looking for a way forward, this observation from Mike seems like a useful place to start: > This iSER update is meant to reflect running code and to generalize the support of iSER over any RCaP, not just iWARP. RFC 5046 (iSER) was written on the assumption that the RCaP layer supports Solicited Event functionality. I also read RFC 5040 as requiring RDMAP implementations (part of iWARP) to support Solicited Event functionality. This first step seems easy: [1] There are RCaPs that don't support Solicited Event functionality; those RCaPs have no choice - they MUST use Send (not SendSE) and SendInv (not SendInvSE). I would prefer to write iSER requirements in terms of whether the RCaP supports Solicited Event functionality as opposed to calling out iWARP explicitly as having different requirements. OTOH, I'm prepared to listen to arguments to the contrary. Beyond that, I think there are two primary choices: [2a] Leave things alone - if the RCaP supports Solicited Events, then the "MUST" requirements for SendSE and SendInvSE apply as they did in RFC 5046. [2b] In the alternative, if there are iSER implementations for iWARP that aren't using SendSE and SendInvSE in accordance with RFC 5046's requirements, then we need to change those requirements to match the implementations. I would hope we can agree on [1], and I believe that'll cover the InfiniBand case, right? For choosing between [2a] and [2b] it would help a lot to konw what iSER over iWARP implementations are actually doing (SendSE/SendInvSE vs. Send/SendInv). *** Who is familiar with the iSER over iWARP implementation(s) ??? *** In general, what do folks think ought to be done here - [2a], [2b] or something else? [WG chair hat on] We do need to resolve this issue and reflect that resolution in a -09 iSER draft before RFC publication can be requested. Thanks, --David From: Tom Talpey [mailto:ttalpey@microsoft.com] Sent: Thursday, January 19, 2012 3:33 PM To: Michael Ko; Hemal Shah; Black, David; storm@ietf.org Subject: RE: iSER - one last issue More precisely, are there any iSER *initiators* that *depend* on receiving solicited events over the iWARP RDMA transport? I believe this to be a very short list, if any. But I'd be ok with being proven wrong. From: Michael Ko [mailto:Michael@huaweisymantec.com] Sent: Thursday, January 19, 2012 12:39 PM To: Hemal Shah; Tom Talpey; david.black@emc.com; storm@ietf.org Subject: Re: iSER - one last issue For all RCaP layers, the Send message (or any variant) MUST be used, as opposed to using RDMA Read or Write to accomplish the task. So a SHOULD for RCaP other than iWARP is incorrect. RFC 5040 can mandate that all RDMAP messages be supported, but it is still up to implementation to decide what messages to use in any situation. This iSER update is meant to reflect running code and to generalize the support of iSER over any RCaP, not just iWARP. So the question is are there any running code that uses SendSE and SendInvSE for iSER. Conversely, if the running code can get by using Send messages to accomplish the task, what is the point of mandating a MUST at this point for SendSE and SendInvSE? Note that the receiver cannot completely rely on the sender for the Invalidate feature anyway. The iSER layer at the initiator is required to explicitly invalidate the STag for bidirectional commands and abnormal completion of a command. Even when automatic invalidation is used, the iSER layer is required to sanity checking the automatic invalidation. In other words, the iSER spec puts the burden on the receiver for doing the right thing in handling the Send messages, and not relying on the sender for the mere convenience. Mike ----- Original Message ----- From: Hemal Shah To: Tom Talpey ; david.black@emc.com ; Michael@huaweisymantec.com ; storm@ietf.org Sent: Thursday, January 19, 2012 8:44 AM Subject: RE: iSER - one last issue Tom, You are welcome! You are right about 7.3.4 as the only section mandating Send message. I think your concern about specifying the requirement for the other RCaP implementation is valid. I suggest we should word the requirements as below: "If the RCaP layer is as specified in [RFC5040] and [RFC5041], the SendSE message MUST be used. For any other RCaP layer, the Send message SHOULD be used. " Hemal From: Tom Talpey [mailto:ttalpey@microsoft.com] Sent: Thursday, January 19, 2012 8:09 AM To: Hemal Shah; david.black@emc.com; Michael@huaweisymantec.com; storm@ietf.org Subject: RE: iSER - one last issue Thanks for pointing out that sentence in RFC5046 7.3.4. I believe it is the only such stipulation. I'm ok with stating the requirement as RFC5046-over-RFC5040 compliance. But this new draft opens the door to alternative behaviors, therefore it needs to be stated more clearly. What's the requirement when SendSE is not supported? And, how does the receiver know what to expect? In other words, if it's proposed that the statement be: "When the RCaP layer is as specified in [RFC5040] and [RFC5041], the SendSE message MUST be used." . then what MUST be used when the RCaP is not? That statement needs to be made, too. From: Hemal Shah [mailto:hemal@broadcom.com] Sent: Thursday, January 19, 2012 10:38 AM To: Tom Talpey; david.black@emc.com; Michael@huaweisymantec.com; storm@ietf.org Subject: RE: iSER - one last issue Tom, RFC5046 is very clear about the use of Send Message Type for iSCSI control-type PDU. For specific iSCSI PDUs, Section 7 mandates the use of Send, SendSE, or SendSEInv. Section 7 of RFC5046 clearly specifies the use of specific Send Message Type for each iSCSI control-type PDUs (Login, Logout, Text request, Text response.). Section 7.3.4 mandates the use of plain-old Send also as needed. For example, see text below from 7.3.4. For unsolicited data, if the F bit is set to 0 in a SCSI Data-out PDU, the iSER layer at the initiator MUST use a Send Message to send the SCSI Data-out PDU. Regarding your comment in the second paragraph below, RFC5040 expectation is all the RDMAP messages specified in RFC5040 are supported (if that is not the case then we cannot count on any of the other RDMAP messages to be supported). So, an implementation that does not support SendSE at the sender is not compliant with RFC5040. So, your second point does not apply. I hope that addresses your comments. Hemal From: Tom Talpey [mailto:ttalpey@microsoft.com] Sent: Thursday, January 19, 2012 5:51 AM To: Hemal Shah; david.black@emc.com; Michael@huaweisymantec.com; storm@ietf.org Subject: RE: iSER - one last issue If using SendSE is so important here, when MUST a plain-old Send be used? There is no mention of any distinction in RFC5046 section 1.4.2 nor in this draft's section 2.4.2. Even with a MUST on the SendSE, the behavior is indeterminate, since it would still be contingent on support for SendSE at the *sender*. This critical exception voids any normative requirement, even over iWARP. The strongest statement that can be made is SHOULD. Tom. From: storm-bounces@ietf.org [mailto:storm-bounces@ietf.org] On Behalf Of Hemal Shah Sent: Wednesday, January 18, 2012 6:56 PM To: david.black@emc.com; Michael@huaweisymantec.com; storm@ietf.org Subject: Re: [storm] iSER - one last issue David and Mike, I believe that the requirements should not be weakened and we should stick to the requirement in RFC5046. RFC5046 required the use of specific Send Message Type for valid reasons. In the case of requiring SendSE message use, the intent was to give a hint to the remote side to generate an event upon processing of SendSE message. For example, the SendSE is specifically carrying SCSI information that needs attention on the remote side e.g. SCSI command, Logout. In the case of requiring SendInvSE Message, the intent was to invalidate the STag used in the data transfer as well as inform the remote side to generate an event. For example, mandating the use of SendInvSE for the SCSI response PDU for a SCSI Read limits the exposure of STag used in SCSI Read data transfer and avoids the explicit invalidation of the STag at the initiator. Plus, it allows the target to generate an event on the initiator upon the completion of SCSI Read command. By relaxing RFC5046 requirements in the latest iSER draft, we will not only impact all iWARP based iSER implementations that rely on the use of specific Send Message Type for SCSI data transfers but also change the intended behavior of initiators and targets. For iWARP, I strongly suggest that the iSER draft does not relax the requirements specified in RFC5046. I hope that helps. Hemal From: storm-bounces@ietf.org [mailto:storm-bounces@ietf.org] On Behalf Of david.black@emc.com Sent: Wednesday, January 18, 2012 3:10 PM To: Michael@huaweisymantec.com; storm@ietf.org Subject: Re: [storm] iSER - one last issue Mike, Thank you for the explanation, but I don't believe you've correctly stated the intent of RFC 5046. Here are some examples from RFC 5046's text: 1.5. SCSI Read Overview The iSER Message is transferred to the target using a SendSE Message. The iSER layer at the target uses a SendInvSE Message to transfer the SCSI Response PDU back to the iSER layer at the initiator. Similar language occurs in 1.6 SCSI Write Overview. 5.2.1. Normal Connection Termination at the Initiator The iSER layer at the initiator MUST use a SendSE Message to send the Logout Request PDU to the target. Similar language occurs in 5.2.2 for target side normal connection termination, and more importantly in 7.3.1 SCSI Command: The iSER layer at the initiator MUST send the SCSI command in a SendSE Message to the target. None of the quoted text permits use of Send instead of SendSE or SendInv instead of SendInvSE. What you propose to do effectively changes "MUST" to "should" (lower case, weaker than "SHOULD"), and that sure looks like a change to RFC 5046. Are there good implementation-based reasons to weaken these requirements now? Does anyone else have a viewpoint on this topic? Thanks, --David From: Michael Ko [mailto:Michael@huaweisymantec.com] Sent: Wednesday, January 18, 2012 2:22 PM To: Black, David; storm@ietf.org Subject: Re: iSER - one last issue David, Here is my rationale for using the lower case "should". The intent of RFC5046 is that the Send Message type must be used instead of RDMA Reads or Writes. The Solicited Event feature of the Send Message is provided as a convenience. The receiver must still do the right thing in handling the Send Message regardless of whether the SE feature is used. In other words, the sender is responsible for using the right format for the message (Send vs RDMA) but the receiver must not rely on the sender to determine how to handle the received message. The same rationale goes for the Invalidate feature. Mike ----- Original Message ----- From: david.black@emc.com To: Michael@huaweisymantec.com ; storm@ietf.org Sent: Monday, January 16, 2012 2:06 PM Subject: iSER - one last issue Mike, Thanks for getting the -08 version of the iSER draft posted. I think that draft addresses all of the open issues, but I have a question for the WG about how to express the SendSE and SendInvSE requirements from RFC 5046. The -08 version of the iSER draft expresses requirements for the SendSE and SendInvSE messages (this is primarily for iSER over RDDP/iWARP) as: The SendSE Message should be used if supported by the RCaP layer (e.g., iWARP). My reading of RFC 5046 is that its requirements are tighter - to accurately reflect RFC 5046, I would replace "should" with "MUST" in the above text, at least for iWARP. In the alternative, if the "should"s remain, an explanatory item needs to be added to the Appendix A list of changes from RFC 5046. What do others think the right course of action is here, use "MUST" or explain weakening of requirement to "should" ? Thanks, --David ------=_NextPart_000_0015_01CCDAB4.CD31E770 Content-Type: text/html; charset="us-ascii" Content-Transfer-Encoding: quoted-printable

I think we should go with [1] and [2a] that David outlined.  As = Hemal said, there were good reasons why RFC 5046 stipulated SendInvSE = and SendSE, where it did.  I would rather not weaken it, at least = for the iWARP RCaP ecosystem.

 

Mallikarjun

 

 

From:= = storm-bounces@ietf.org [mailto:storm-bounces@ietf.org] On Behalf Of = Tom Talpey
Sent: Tuesday, January 24, 2012 5:49 = AM
To: david.black@emc.com; storm@ietf.org
Subject: = Re: [storm] iSER - one last issue: = update

 

[WG co-chair hat off]

 

> = [2a] Leave things alone - if the RCaP supports Solicited Events, then = the “MUST”

> = requirements for SendSE and SendInvSE apply as they did in RFC = 5046.

 

The core issue here is the statement “if the RCaP supports = Solicited Events”. Both the iWARP and Infiniband protocols support = them. But Infiniband does not require all adapters support them – = the architecture requires it for HCAs but leaves it optional for TCAs = (Infiniband architecture v1.2.1 table 319 on page 1045). Because this = latter property cannot be detected by the remote upper layer, it does = not appear to be possible to make an interoperable decision for all = protocols and all implementations.

 

That said, in the absence of any existing iSER-over-iWARP SendSE = implementations, I agree the entire discussion is effectively = moot.

 

Tom.

 

From:= = storm-bounces@ietf.org [mailto:storm-bounces@iet= f.org] On Behalf Of david.black@emc.com
Sent: Monday, January 23, 2012 8:05 PM
To: storm@ietf.org
Subject: = [storm] iSER - one last issue: = update

 

[WG = chair hat off]

 

This is = a message intended for discussion and comments.

 

Looking = for a way forward, this observation from Mike seems like a useful place = to start:

 

> This iSER update is meant to reflect = running code and to generalize the support of iSER over any RCaP, not = just iWARP. 

 

RFC 5046 (iSER) was written on the assumption that the = RCaP layer supports Solicited

Event functionality.  I also read RFC 5040 as = requiring RDMAP implementations (part of

iWARP) to support Solicited Event = functionality.

 

This = first step seems easy:

 

[1] = There are RCaPs that don’t support Solicited Event functionality; = those RCaPs have

no = choice - they MUST use Send (not SendSE) and SendInv (not = SendInvSE). 

         &= nbsp;           &n= bsp;           &nb= sp;           &nbs= p;            = ;            =             &= nbsp;          =

I would = prefer to write iSER requirements in terms of whether the RCaP = supports

Solicited Event functionality as opposed to calling = out iWARP explicitly as having

different requirements.  OTOH, I’m prepared = to listen to arguments to the contrary.

 

Beyond = that, I think there are two primary choices:

 

[2a] = Leave things alone - if the RCaP supports Solicited Events, then the = “MUST”

requirements for SendSE and SendInvSE apply as they = did in RFC 5046.

 

[2b] In = the alternative, if there are iSER implementations for iWARP that = aren’t using

SendSE = and SendInvSE in accordance with RFC 5046’s requirements, then we = need to change

those = requirements to match the implementations.

 

I would = hope we can agree on [1], and I believe that’ll cover the = InfiniBand case, right?

 

For = choosing between [2a] and [2b] it would help a lot to konw what iSER = over iWARP

implementations are actually doing (SendSE/SendInvSE = vs. Send/SendInv).

 

*** Who = is familiar with the iSER over iWARP implementation(s) ??? = ***

 

In = general, what do folks think ought to be done here - [2a], [2b] or = something else?

 

[WG = chair hat on]

 

We do = need to resolve this issue and reflect that resolution in a -09 iSER = draft before

RFC = publication can be requested.

 

Thanks,
--David

 

From:= = Tom Talpey [mailto:ttalpey@microsoft.= com]
Sent: Thursday, January 19, 2012 3:33 = PM
To: Michael Ko; Hemal Shah; Black, David; storm@ietf.org
Subject: RE: = iSER - one last issue

 

More precisely, are there any iSER *initiators* that = *depend* on receiving solicited events over the iWARP RDMA = transport? I believe this to be a very short list, if any. But I’d = be ok with being proven wrong.

 

 

 

From:= = Michael Ko [mailto:Michael@huawe= isymantec.com]
Sent: Thursday, January 19, 2012 12:39 = PM
To: Hemal Shah; Tom Talpey; david.black@emc.com; storm@ietf.org
Subject: Re: = iSER - one last issue

 

For all RCaP layers, the Send message (or any = variant) MUST be used, as opposed to using RDMA Read or Write to = accomplish the task.  So a SHOULD for RCaP other than iWARP is = incorrect.  RFC 5040 can mandate that all RDMAP messages be = supported, but it is still up to implementation to decide what messages = to use in any situation.  This iSER update is meant to reflect = running code and to generalize the support of iSER over any RCaP, not = just iWARP.  So the question is are there any running code that = uses SendSE and SendInvSE for iSER.  Conversely, if the running = code can get by using Send messages to accomplish the task, what is the = point of mandating a MUST at this point for SendSE and SendInvSE?  = Note that the receiver cannot completely rely on the sender for the = Invalidate feature anyway.  The iSER layer at the initiator is = required to explicitly invalidate the STag for bidirectional commands = and abnormal completion of a command.  Even when automatic = invalidation is used, the iSER layer is required to sanity checking the = automatic invalidation.  In other words, the iSER = spec puts the burden on the receiver for doing the right thing in = handling the Send messages, and not relying on the sender for the = mere convenience. 

 

Mike

----- = Original Message -----

From:<= /b> Hemal = Shah

Sent:<= /b> = Thursday, January 19, 2012 8:44 AM

Subject: = RE: iSER - one last issue

 

Tom,

 

<= p class=3DMsoNormal>You = are welcome! You are right about 7.3.4 as the only section mandating = Send message.

 

<= p class=3DMsoNormal>I = think your concern about specifying the requirement for the other RCaP = implementation is valid. I suggest we should word the requirements as = below:

 

<= p class=3DMsoNormal>“If the RCaP layer is as specified in [RFC5040] and [RFC5041], = the SendSE message MUST be used. For any other RCaP layer, the Send = message SHOULD be used. ”

 

Hemal

 

<= div>

From:= = Tom Talpey [mailto:ttalpey@microsoft.= com]
Sent: Thursday, January 19, 2012 8:09 = AM
To: Hemal Shah; david.black@emc.com; Michael@huaweisymantec.com= ; storm@ietf.org
Subject: RE: = iSER - one last issue

 

Thanks for pointing out that sentence in RFC5046 7.3.4. I believe it = is the only such stipulation.

 

I’m ok with stating the requirement as RFC5046-over-RFC5040 = compliance. But this new draft opens the door to alternative behaviors, = therefore it needs to be stated more clearly. What’s the = requirement when SendSE is not supported? And, how does the receiver = know what to expect?

 

In other words, if it’s proposed that the statement = be:

           &nbs= p;    “When the RCaP layer is as specified in = [RFC5040] and [RFC5041], the SendSE message MUST be = used.”

 

… then what MUST be used when the RCaP is not? That statement = needs to be made, too.

 

 

 

Tom,

 

<= p class=3DMsoNormal>RFC5046 is very clear about the = use of Send Message Type for iSCSI control-type PDU. For specific iSCSI = PDUs, Section 7 mandates the use of Send, SendSE, or SendSEInv. Section = 7 of RFC5046 clearly specifies the use of specific Send Message Type for = each iSCSI control-type PDUs (Login, Logout, Text request, Text = response…). Section 7.3.4 mandates the use of plain-old Send also = as needed. For example, see text below from = 7.3.4.

 

<= p class=3DMsoNormal>For unsolicited data, if the F bit is set to 0 in a SCSI = Data-out

PDU, the iSER layer = at the initiator MUST use a Send Message to send

the SCSI Data-out PDU.

 

Regarding your comment in the = second paragraph below, RFC5040 expectation is all the RDMAP messages = specified in RFC5040 are supported (if that is not the case then we = cannot count on any of the other RDMAP messages to be supported). So, an = implementation that does not support SendSE at the sender is not = compliant with RFC5040. So, your second point does not = apply.

 

<= p class=3DMsoNormal>I = hope that addresses your comments.

 

<= p class=3DMsoNormal>Hemal

 

<= div>

From:= = Tom Talpey [mailto:ttalpey@microsoft.= com]
Sent: Thursday, January 19, 2012 5:51 = AM
To: Hemal Shah; david.black@emc.com; Michael@huaweisymantec.com= ; storm@ietf.org
Subject: RE: = iSER - one last issue

 

If using SendSE is so important here, when MUST a plain-old Send be = used? There is no mention of any distinction in RFC5046 section 1.4.2 = nor in this draft’s section 2.4.2.

 

Even with a MUST on the SendSE, the behavior is indeterminate, since = it would still be contingent on support for SendSE at the = *sender*. This critical exception voids any normative = requirement, even over iWARP. The strongest statement that can be made = is SHOULD.

 

Tom.

 

From:= = storm-bounces@ietf.org [mailto:storm-bounces@iet= f.org] On Behalf Of Hemal Shah
Sent: Wednesday, = January 18, 2012 6:56 PM
To: david.black@emc.com; Michael@huaweisymantec.com= ; storm@ietf.org
Subject: Re: = [storm] iSER - one last issue

 

David and = Mike,

 

<= p class=3DMsoNormal>I = believe that the requirements should not be weakened and we should stick = to the requirement in RFC5046.

 

<= p class=3DMsoNormal>RFC5046 required the use of = specific Send Message Type for valid = reasons.

 

<= p class=3DMsoNormal>In = the case of requiring SendSE message use, the intent was to give a hint = to the remote side to generate an event upon processing of SendSE = message. For example, the SendSE is specifically carrying SCSI = information that needs attention on the remote side e.g. SCSI command, = Logout…

 

<= p class=3DMsoNormal>In = the case of requiring SendInvSE Message, the intent was to invalidate = the STag used in the data transfer as well as inform the remote side to = generate an event. For example, mandating the use of SendInvSE for the = SCSI response PDU for a SCSI Read limits the exposure of STag used in = SCSI Read data transfer and avoids the explicit invalidation of the STag = at the initiator. Plus, it allows the target to generate an event on the = initiator upon the completion of SCSI Read = command.

 

<= p class=3DMsoNormal>By = relaxing RFC5046 requirements in the latest iSER draft, we will not only = impact all iWARP based iSER implementations that rely on the use of = specific Send Message Type for SCSI data transfers but also change the = intended behavior of initiators and targets.

 

<= p class=3DMsoNormal>For = iWARP, I strongly suggest that the iSER draft does not relax the = requirements specified in RFC5046.

 

<= p class=3DMsoNormal>I = hope that helps.

 

<= p class=3DMsoNormal>Hemal =    

 

<= div>

From:= = storm-bounces@ietf.org [mailto:storm-bounces@iet= f.org] On Behalf Of david.black@emc.com
Sent: Wednesday, January 18, 2012 3:10 PM
To: Michael@huaweisymantec.com= ; storm@ietf.org
Subject: Re: = [storm] iSER - one last issue

 

Mike,

 

Thank = you for the explanation, but I don’t believe you’ve = correctly stated

the = intent of RFC 5046.  Here are some examples from RFC 5046’s = text:

 

1.5.  SCSI = Read Overview

 

   The = iSER Message is transferred to the target using a SendSE = Message.

 

   The = iSER layer at the target uses a SendInvSE Message to transfer = the

   SCSI = Response PDU back to the iSER layer at the = initiator.

 

Similar language = occurs in 1.6 SCSI Write Overview.

 

5.2.1.  Normal = Connection Termination at the Initiator

 

   The = iSER layer at the initiator MUST

   use a SendSE Message to send the Logout Request PDU = to the target.

 

Similar language = occurs in 5.2.2 for target side normal connection = termination,

and more = importantly in 7.3.1 SCSI Command:

 

   The = iSER layer at the initiator MUST send the SCSI command in = a

   SendSE = Message to the target.

 

None of the quoted = text permits use of Send instead of SendSE or = SendInv

instead of = SendInvSE.

 

What you propose to = do effectively changes “MUST” to “should” (lower = case,

weaker than = “SHOULD”), and that sure looks like a change to RFC = 5046.

 

Are there good = implementation-based reasons to weaken these requirements = now?

 

Does anyone else have a viewpoint on this = topic?

 

Thanks,
--David

 

From:= = Michael Ko [mailto:Michael@huawe= isymantec.com]
Sent: Wednesday, January 18, 2012 2:22 = PM
To: Black, David; storm@ietf.org
Subject: Re: = iSER - one last issue

 

David,

 

Here is my rationale = for using the lower case = "should".

 

The intent of RFC5046 = is that the Send Message type must be used instead of RDMA Reads or = Writes.  The Solicited Event feature of the Send Message is = provided as a convenience.  The receiver must still do the right = thing in handling the Send Message regardless of whether the SE feature = is used.  In other words, the sender is responsible for using the = right format for the message (Send vs RDMA) but the receiver must = not rely on the sender to determine how to handle the received = message.  The same rationale goes for the Invalidate feature.  =

 

Mike

----- = Original Message -----

From:<= /b> david.black@emc.com =

Sent:<= /b> = Monday, January 16, 2012 2:06 PM

Subject: = iSER - one last issue

 

Mike,

 

Thanks = for getting the -08 version of the iSER draft posted.  I think = that

draft = addresses all of the open issues, but I have a question for the = WG

about = how to express the SendSE and SendInvSE requirements from RFC = 5046.

 

The -08 = version of the iSER draft expresses requirements for the = SendSE

and = SendInvSE messages (this is primarily for iSER over RDDP/iWARP) = as:

 

The = SendSE Message should be used if

supported by the RCaP layer (e.g., = iWARP).

 

My = reading of RFC 5046 is that its requirements are tighter - to = accurately

reflect = RFC 5046, I would replace “should” with “MUST” = in the above text,

at = least for iWARP.

 

In the = alternative, if the “should”s remain, an explanatory = item

needs = to be added to the Appendix A list of changes from RFC = 5046.

 

What do = others think the right course of action is here, use “MUST” = or

explain = weakening of requirement to “should” = ?

 

Thanks,
--David

------=_NextPart_000_0015_01CCDAB4.CD31E770-- From david.black@emc.com Tue Jan 24 16:42:19 2012 Return-Path: X-Original-To: storm@ietfa.amsl.com Delivered-To: storm@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id F36C721F84D5 for ; Tue, 24 Jan 2012 16:42:18 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -109.13 X-Spam-Level: X-Spam-Status: No, score=-109.13 tagged_above=-999 required=5 tests=[AWL=1.468, BAYES_00=-2.599, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_HI=-8, USER_IN_WHITELIST=-100] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id Knd35t7xsUzF for ; Tue, 24 Jan 2012 16:42:09 -0800 (PST) Received: from mexforward.lss.emc.com (mexforward.lss.emc.com [128.222.32.20]) by ietfa.amsl.com (Postfix) with ESMTP id 684BD21F8574 for ; Tue, 24 Jan 2012 16:42:09 -0800 (PST) Received: from hop04-l1d11-si03.isus.emc.com (HOP04-L1D11-SI03.isus.emc.com [10.254.111.23]) by mexforward.lss.emc.com (Switch-3.4.3/Switch-3.4.3) with ESMTP id q0P0g6tL003961 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Tue, 24 Jan 2012 19:42:06 -0500 Received: from mailhub.lss.emc.com (mailhub.lss.emc.com [10.254.221.145]) by hop04-l1d11-si03.isus.emc.com (RSA Interceptor); Tue, 24 Jan 2012 19:41:58 -0500 Received: from mxhub07.corp.emc.com (mxhub07.corp.emc.com [128.222.70.204]) by mailhub.lss.emc.com (Switch-3.4.3/Switch-3.4.3) with ESMTP id q0P0fwK9001146; Tue, 24 Jan 2012 19:41:58 -0500 Received: from mx14a.corp.emc.com ([169.254.1.99]) by mxhub07.corp.emc.com ([128.222.70.204]) with mapi; Tue, 24 Jan 2012 19:41:58 -0500 From: To: , Date: Tue, 24 Jan 2012 19:41:56 -0500 Thread-Topic: [storm] iSER - one last issue: update Thread-Index: AQCOMSfeSH/fN63SI/bKvLYFC94DHAImI546ApHpQMMBoXLTSphmqTBQgAAZ7pA= Message-ID: <7C4DFCE962635144B8FAE8CA11D0BF1E05A7D91649@MX14A.corp.emc.com> References: <7C4DFCE962635144B8FAE8CA11D0BF1E05A7D9137A@MX14A.corp.emc.com> In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: acceptlanguage: en-US Content-Type: multipart/alternative; boundary="_000_7C4DFCE962635144B8FAE8CA11D0BF1E05A7D91649MX14Acorpemcc_" MIME-Version: 1.0 X-EMM-MHVC: 1 Subject: Re: [storm] iSER - one last issue: update X-BeenThere: storm@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Storage Maintenance WG List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 25 Jan 2012 00:42:19 -0000 --_000_7C4DFCE962635144B8FAE8CA11D0BF1E05A7D91649MX14Acorpemcc_ Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable [WG chair hat off] Proposed slight refinement: If the SendSE and SendInvSE requirements are ap= plied to RCaPs that "require all implementations to support Solicited Event= s", that set of RCaPs includes iWARP, but not InfiniBand. That refinement creates a need to figure out the complementary requirement = that applies to InfiniBand, which Tom more or less asked about earlier in t= his thread: > If using SendSE is so important here, when MUST a plain-old Send be used? There are at least a couple of approaches to this one: [3a] SendSE/SendInvSE MUST NOT be used with other RCaPs. [3b] Explain how a sender figures out whether to use SendSE/SendInvSE or no= t (aka "negotiation"). [3a] would be the simplest. The "running code" matters here - does the IB = implementation of iSER use SendSE/SendInvSE? If not, [3a] should be fine. Thanks, --David From: Mallikarjun Chadalapaka [mailto:cbm@chadalapaka.com] Sent: Tuesday, January 24, 2012 7:25 PM To: Tom Talpey; Black, David; storm@ietf.org Subject: RE: [storm] iSER - one last issue: update I think we should go with [1] and [2a] that David outlined. As Hemal said,= there were good reasons why RFC 5046 stipulated SendInvSE and SendSE, wher= e it did. I would rather not weaken it, at least for the iWARP RCaP ecosys= tem. Mallikarjun From: storm-bounces@ietf.org [mailto:storm-bounces@ietf.org] On Behalf Of T= om Talpey Sent: Tuesday, January 24, 2012 5:49 AM To: david.black@emc.com; storm@ietf.org Subject: Re: [storm] iSER - one last issue: update [WG co-chair hat off] > [2a] Leave things alone - if the RCaP supports Solicited Events, then the= "MUST" > requirements for SendSE and SendInvSE apply as they did in RFC 5046. The core issue here is the statement "if the RCaP supports Solicited Events= ". Both the iWARP and Infiniband protocols support them. But Infiniband doe= s not require all adapters support them - the architecture requires it for = HCAs but leaves it optional for TCAs (Infiniband architecture v1.2.1 table = 319 on page 1045). Because this latter property cannot be detected by the r= emote upper layer, it does not appear to be possible to make an interoperab= le decision for all protocols and all implementations. That said, in the absence of any existing iSER-over-iWARP SendSE implementa= tions, I agree the entire discussion is effectively moot. Tom. From: storm-bounces@ietf.org [mailto:storm-b= ounces@ietf.org] On Behalf Of david= .black@emc.com Sent: Monday, January 23, 2012 8:05 PM To: storm@ietf.org Subject: [storm] iSER - one last issue: update [WG chair hat off] This is a message intended for discussion and comments. Looking for a way forward, this observation from Mike seems like a useful p= lace to start: > This iSER update is meant to reflect running code and to generalize the s= upport of iSER over any RCaP, not just iWARP. RFC 5046 (iSER) was written on the assumption that the RCaP layer supports = Solicited Event functionality. I also read RFC 5040 as requiring RDMAP implementatio= ns (part of iWARP) to support Solicited Event functionality. This first step seems easy: [1] There are RCaPs that don't support Solicited Event functionality; those= RCaPs have no choice - they MUST use Send (not SendSE) and SendInv (not SendInvSE). I would prefer to write iSER requirements in terms of whether the RCaP supp= orts Solicited Event functionality as opposed to calling out iWARP explicitly as= having different requirements. OTOH, I'm prepared to listen to arguments to the c= ontrary. Beyond that, I think there are two primary choices: [2a] Leave things alone - if the RCaP supports Solicited Events, then the "= MUST" requirements for SendSE and SendInvSE apply as they did in RFC 5046. [2b] In the alternative, if there are iSER implementations for iWARP that a= ren't using SendSE and SendInvSE in accordance with RFC 5046's requirements, then we ne= ed to change those requirements to match the implementations. I would hope we can agree on [1], and I believe that'll cover the InfiniBan= d case, right? For choosing between [2a] and [2b] it would help a lot to konw what iSER ov= er iWARP implementations are actually doing (SendSE/SendInvSE vs. Send/SendInv). *** Who is familiar with the iSER over iWARP implementation(s) ??? *** In general, what do folks think ought to be done here - [2a], [2b] or somet= hing else? [WG chair hat on] We do need to resolve this issue and reflect that resolution in a -09 iSER = draft before RFC publication can be requested. Thanks, --David From: Tom Talpey [mailto:ttalpey@microsoft.com] Sent: Thursday, January 19, 2012 3:33 PM To: Michael Ko; Hemal Shah; Black, David; storm@ietf.org Subject: RE: iSER - one last issue More precisely, are there any iSER *initiators* that *depend* on receiving = solicited events over the iWARP RDMA transport? I believe this to be a very= short list, if any. But I'd be ok with being proven wrong. From: Michael Ko [mailto:Michael@huaweisymantec.com] Sent: Thursday, January 19, 2012 12:39 PM To: Hemal Shah; Tom Talpey; david.black@emc.com= ; storm@ietf.org Subject: Re: iSER - one last issue For all RCaP layers, the Send message (or any variant) MUST be used, as opp= osed to using RDMA Read or Write to accomplish the task. So a SHOULD for R= CaP other than iWARP is incorrect. RFC 5040 can mandate that all RDMAP mes= sages be supported, but it is still up to implementation to decide what mes= sages to use in any situation. This iSER update is meant to reflect runnin= g code and to generalize the support of iSER over any RCaP, not just iWARP.= So the question is are there any running code that uses SendSE and SendIn= vSE for iSER. Conversely, if the running code can get by using Send messag= es to accomplish the task, what is the point of mandating a MUST at this po= int for SendSE and SendInvSE? Note that the receiver cannot completely rel= y on the sender for the Invalidate feature anyway. The iSER layer at the i= nitiator is required to explicitly invalidate the STag for bidirectional co= mmands and abnormal completion of a command. Even when automatic invalidat= ion is used, the iSER layer is required to sanity checking the automatic in= validation. In other words, the iSER spec puts the burden on the receiver = for doing the right thing in handling the Send messages, and not relying on= the sender for the mere convenience. Mike ----- Original Message ----- From: Hemal Shah To: Tom Talpey ; david.black@emc.com ; Michael@huaweisymantec.com ; storm@ietf.org Sent: Thursday, January 19, 2012 8:44 AM Subject: RE: iSER - one last issue Tom, You are welcome! You are right about 7.3.4 as the only section mandating Se= nd message. I think your concern about specifying the requirement for the other RCaP im= plementation is valid. I suggest we should word the requirements as below: "If the RCaP layer is as specified in [RFC5040] and [RFC5041], the SendSE m= essage MUST be used. For any other RCaP layer, the Send message SHOULD be u= sed. " Hemal From: Tom Talpey [mailto:ttalpey@microsoft.com] Sent: Thursday, January 19, 2012 8:09 AM To: Hemal Shah; david.black@emc.com; Michael@hu= aweisymantec.com; storm@ietf.org Subject: RE: iSER - one last issue Thanks for pointing out that sentence in RFC5046 7.3.4. I believe it is the= only such stipulation. I'm ok with stating the requirement as RFC5046-over-RFC5040 compliance. But= this new draft opens the door to alternative behaviors, therefore it needs= to be stated more clearly. What's the requirement when SendSE is not suppo= rted? And, how does the receiver know what to expect? In other words, if it's proposed that the statement be: "When the RCaP layer is as specified in [RFC5040] and [RFC5= 041], the SendSE message MUST be used." ... then what MUST be used when the RCaP is not? That statement needs to be= made, too. From: Hemal Shah [mailto:hemal@broadcom.com] Sent: Thursday, January 19, 2012 10:38 AM To: Tom Talpey; david.black@emc.com; Michael@hu= aweisymantec.com; storm@ietf.org Subject: RE: iSER - one last issue Tom, RFC5046 is very clear about the use of Send Message Type for iSCSI control-= type PDU. For specific iSCSI PDUs, Section 7 mandates the use of Send, Send= SE, or SendSEInv. Section 7 of RFC5046 clearly specifies the use of specifi= c Send Message Type for each iSCSI control-type PDUs (Login, Logout, Text r= equest, Text response...). Section 7.3.4 mandates the use of plain-old Send= also as needed. For example, see text below from 7.3.4. For unsolicited data, if the F bit is set to 0 in a SCSI Data-out PDU, the iSER layer at the initiator MUST use a Send Message to send the SCSI Data-out PDU. Regarding your comment in the second paragraph below, RFC5040 expectation i= s all the RDMAP messages specified in RFC5040 are supported (if that is not= the case then we cannot count on any of the other RDMAP messages to be sup= ported). So, an implementation that does not support SendSE at the sender i= s not compliant with RFC5040. So, your second point does not apply. I hope that addresses your comments. Hemal From: Tom Talpey [mailto:ttalpey@microsoft.com] Sent: Thursday, January 19, 2012 5:51 AM To: Hemal Shah; david.black@emc.com; Michael@hu= aweisymantec.com; storm@ietf.org Subject: RE: iSER - one last issue If using SendSE is so important here, when MUST a plain-old Send be used? T= here is no mention of any distinction in RFC5046 section 1.4.2 nor in this = draft's section 2.4.2. Even with a MUST on the SendSE, the behavior is indeterminate, since it wou= ld still be contingent on support for SendSE at the *sender*. This critical= exception voids any normative requirement, even over iWARP. The strongest = statement that can be made is SHOULD. Tom. From: storm-bounces@ietf.org [mailto:storm-b= ounces@ietf.org] On Behalf Of Hemal= Shah Sent: Wednesday, January 18, 2012 6:56 PM To: david.black@emc.com; Michael@huaweisymantec= .com; storm@ietf.org Subject: Re: [storm] iSER - one last issue David and Mike, I believe that the requirements should not be weakened and we should stick = to the requirement in RFC5046. RFC5046 required the use of specific Send Message Type for valid reasons. In the case of requiring SendSE message use, the intent was to give a hint = to the remote side to generate an event upon processing of SendSE message. = For example, the SendSE is specifically carrying SCSI information that need= s attention on the remote side e.g. SCSI command, Logout... In the case of requiring SendInvSE Message, the intent was to invalidate th= e STag used in the data transfer as well as inform the remote side to gener= ate an event. For example, mandating the use of SendInvSE for the SCSI resp= onse PDU for a SCSI Read limits the exposure of STag used in SCSI Read data= transfer and avoids the explicit invalidation of the STag at the initiator= . Plus, it allows the target to generate an event on the initiator upon the= completion of SCSI Read command. By relaxing RFC5046 requirements in the latest iSER draft, we will not only= impact all iWARP based iSER implementations that rely on the use of specif= ic Send Message Type for SCSI data transfers but also change the intended b= ehavior of initiators and targets. For iWARP, I strongly suggest that the iSER draft does not relax the requir= ements specified in RFC5046. I hope that helps. Hemal From: storm-bounces@ietf.org [mailto:storm-b= ounces@ietf.org] On Behalf Of david= .black@emc.com Sent: Wednesday, January 18, 2012 3:10 PM To: Michael@huaweisymantec.com; storm@ie= tf.org Subject: Re: [storm] iSER - one last issue Mike, Thank you for the explanation, but I don't believe you've correctly stated the intent of RFC 5046. Here are some examples from RFC 5046's text: 1.5. SCSI Read Overview The iSER Message is transferred to the target using a SendSE Message. The iSER layer at the target uses a SendInvSE Message to transfer the SCSI Response PDU back to the iSER layer at the initiator. Similar language occurs in 1.6 SCSI Write Overview. 5.2.1. Normal Connection Termination at the Initiator The iSER layer at the initiator MUST use a SendSE Message to send the Logout Request PDU to the target. Similar language occurs in 5.2.2 for target side normal connection terminat= ion, and more importantly in 7.3.1 SCSI Command: The iSER layer at the initiator MUST send the SCSI command in a SendSE Message to the target. None of the quoted text permits use of Send instead of SendSE or SendInv instead of SendInvSE. What you propose to do effectively changes "MUST" to "should" (lower case, weaker than "SHOULD"), and that sure looks like a change to RFC 5046. Are there good implementation-based reasons to weaken these requirements no= w? Does anyone else have a viewpoint on this topic? Thanks, --David From: Michael Ko [mailto:Michael@huaweisymantec.com] Sent: Wednesday, January 18, 2012 2:22 PM To: Black, David; storm@ietf.org Subject: Re: iSER - one last issue David, Here is my rationale for using the lower case "should". The intent of RFC5046 is that the Send Message type must be used instead of= RDMA Reads or Writes. The Solicited Event feature of the Send Message is = provided as a convenience. The receiver must still do the right thing in h= andling the Send Message regardless of whether the SE feature is used. In = other words, the sender is responsible for using the right format for the m= essage (Send vs RDMA) but the receiver must not rely on the sender to deter= mine how to handle the received message. The same rationale goes for the I= nvalidate feature. Mike ----- Original Message ----- From: david.black@emc.com To: Michael@huaweisymantec.com ; storm@i= etf.org Sent: Monday, January 16, 2012 2:06 PM Subject: iSER - one last issue Mike, Thanks for getting the -08 version of the iSER draft posted. I think that draft addresses all of the open issues, but I have a question for the WG about how to express the SendSE and SendInvSE requirements from RFC 5046. The -08 version of the iSER draft expresses requirements for the SendSE and SendInvSE messages (this is primarily for iSER over RDDP/iWARP) as: The SendSE Message should be used if supported by the RCaP layer (e.g., iWARP). My reading of RFC 5046 is that its requirements are tighter - to accurately reflect RFC 5046, I would replace "should" with "MUST" in the above text, at least for iWARP. In the alternative, if the "should"s remain, an explanatory item needs to be added to the Appendix A list of changes from RFC 5046. What do others think the right course of action is here, use "MUST" or explain weakening of requirement to "should" ? Thanks, --David --_000_7C4DFCE962635144B8FAE8CA11D0BF1E05A7D91649MX14Acorpemcc_ Content-Type: text/html; charset="us-ascii" Content-Transfer-Encoding: quoted-printable

= [WG = chair hat off]

 

Proposed slight refinement: If the SendSE and SendIn= vSE requirements are applied to RCaPs that “require all implementatio= ns to support Solicited Events”, that set of RCaPs includes iWARP, bu= t not InfiniBand.

 

That refinement creates a need to figure out the = complementary requirement that applies to InfiniBand, which Tom more or les= s asked about earlier in this thread:

 

> If using S= endSE is so important here, when MUST a plain-old Send be used?<= /span>

 

There are at least a couple of approaches to this one:<= /span>

 

[3a= ] SendSE/SendInvSE MUST NOT be used with other RCaPs.

=

 

[3b] Explain = how a sender figures out whether to use SendSE/SendInvSE or not (aka “= ;negotiation”).

 

[3a] would be the simplest.  The “= running code” matters here - does the IB implementation of iSER use S= endSE/SendInvSE?  If not, [3a] should be fine.

Thanks,--David

 

From: Mallikarjun Chadal= apaka [mailto:cbm@chadalapaka.com]
Sent: Tuesday, January 24, 20= 12 7:25 PM
To: Tom Talpey; Black, David; storm@ietf.org
Sub= ject: RE: [storm] iSER - one last issue: update

 

I think we should go with [1] and [2a] that David outlined.  As H= emal said, there were good reasons why RFC 5046 stipulated SendInvSE and Se= ndSE, where it did.  I would rather not weaken it, at least for the iW= ARP RCaP ecosystem.

 

Mallikarjun<= /span>

 

 

From: storm-bounces@ietf.org [mailto:storm-bounces@ietf.org] On Be= half Of Tom Talpey
Sent: Tuesday, January 24, 2012 5:49 AMTo: david.black@emc.com; storm@ietf.org
Subject: Re: [sto= rm] iSER - one last issue: update

 

[WG co-chair = hat off]

 =

> [2a] Leave things alone - if the RCaP supp= orts Solicited Events, then the “MUST”

> requirements for SendSE and SendInvSE apply as they did = in RFC 5046.

 

The core issue here is the state= ment “if the RCaP supports Solicited Events”. Both the iWARP an= d Infiniband protocols support them. But Infiniband does not require all ad= apters support them – the architecture requires it for HCAs but leave= s it optional for TCAs (Infiniband architecture v1.2.1 table 319 on page 10= 45). Because this latter property cannot be detected by the remote upper la= yer, it does not appear to be possible to make an interoperable decision fo= r all protocols and all implementations.

 

That= said, in the absence of any existing iSER-over-iWARP SendSE implementation= s, I agree the entire discussion is effectively moot.

=

 

Tom.

 <= /o:p>

From: storm-bounces@ietf.org [mailto:storm-bounces@ietf.org] On Behalf= Of david.black@emc.com
<= b>Sent: Monday, January 23, 2012 8:05 PM
To: storm@ietf.org
Subject: [storm] iSER - one = last issue: update

 

[WG chair hat off]

 

This is a me= ssage intended for discussion and comments.

 

Looking for a way for= ward, this observation from Mike seems like a useful place to start:

 

> This iSER update is meant to re= flect running code and to generalize the support of iSER over any RCaP, not= just iWARP. 

 

RFC 5046 (iSER) was written on the assum= ption that the RCaP layer supports Solicited

Event functionality.  I also read RFC 5040 as requiring RDMAP= implementations (part of

iWARP) to s= upport Solicited Event functionality.

 

This first step seems easy:

 

[1] There are RCaPs that don’t support Solicited Event funct= ionality; those RCaPs have

no choice = - they MUST use Send (not SendSE) and SendInv (not SendInvSE).  <= /o:p>

       =             &nb= sp;            =             &nb= sp;            =             &nb= sp;            =           

I would prefer to write iSER requirements in terms= of whether the RCaP supports

Solicit= ed Event functionality as opposed to calling out iWARP explicitly as having=

different requirements.  OTOH, = I’m prepared to listen to arguments to the contrary.

 

Beyond t= hat, I think there are two primary choices:

 

[2a] Leave things alo= ne - if the RCaP supports Solicited Events, then the “MUST”

requirements for SendSE and SendInvSE ap= ply as they did in RFC 5046.

&nb= sp;

[2b] In the alternative, if there are = iSER implementations for iWARP that aren’t using

SendSE and SendInvSE in accordance with RFC 5046’s = requirements, then we need to change

= those requirements to match the implementations.

 

I would hope we ca= n agree on [1], and I believe that’ll cover the InfiniBand case, righ= t?

 

For choosing between [2a] and [2b] it would help a lot to konw w= hat iSER over iWARP

implementations= are actually doing (SendSE/SendInvSE vs. Send/SendInv).<= /p>

 

*** Who is= familiar with the iSER over iWARP implementation(s) ??? ***

 

In gen= eral, what do folks think ought to be done here - [2a], [2b] or something e= lse?

 

[WG chair hat on]

&= nbsp;

We do need to resolve this issue and= reflect that resolution in a -09 iSER draft before

 

Thanks,
--David=

 

From: Tom Talpey [mailto:ttalpey@microsoft.com]
= Sent: Thursday, January 19, 2012 3:33 PM
To: Michael Ko; Hema= l Shah; Black, David; storm@ietf.org<= br>Subject: RE: iSER - one last issue

 

Mo= re precisely, are there any iSER *initiators* that *depend* o= n receiving solicited events over the iWARP RDMA transport? I believe this = to be a very short list, if any. But I’d be ok with being proven wron= g.

 

 

 

Fro= m: Michael Ko [ma= ilto:Michael@huaweisymantec.com]
Sent: Thursday, January 19,= 2012 12:39 PM
To: Hemal Shah; Tom Talpey; david.black@emc.com; storm@ietf.org
Subject: Re: iSER - one last issue=

 

For all RCaP layers, the S= end message (or any variant) MUST be used, as opposed to using RDMA Read or= Write to accomplish the task.  So a SHOULD for RCaP other than i= WARP is incorrect.  RFC 5040 can mandate that all RDMAP messages be su= pported, but it is still up to implementation to decide what messages to us= e in any situation.  This iSER update is meant to reflect running code= and to generalize the support of iSER over any RCaP, not just iWARP. = So the question is are there any running code that uses SendSE and SendInv= SE for iSER.  Conversely, if the running code can get by using Send me= ssages to accomplish the task, what is the point of mandating a MUST at thi= s point for SendSE and SendInvSE?  Note that the receiver cannot compl= etely rely on the sender for the Invalidate feature anyway.  The = iSER layer at the initiator is required to explicitly invalidate the STag f= or bidirectional commands and abnormal completion of a command.  = Even when automatic invalidation is used, the iSER layer is required to san= ity checking the automatic invalidation.  In other words, the iSE= R spec puts the burden on the receiver for doing the right thing in ha= ndling the Send messages, and not relying on the sender for the mere c= onvenience. 

&nbs= p;

Mike

Tom,

 

You are welcome! You are right about 7.3.4 as the only sectio= n mandating Send message.

=  <= /p>

I = think your concern about specifying the requirement for the other RCaP impl= ementation is valid. I suggest we should word the requirements as below:

 

“If the RCaP layer is as specified in [RFC5040] and [RFC5041], the= SendSE message MUST be used. For any other RCaP layer, the Send message SH= OULD be used. ”

 

Hemal

<= span style=3D'font-size:10.0pt;color:purple'> 

From: Tom Talpey [mailto:ttalpey@microsoft.com]
Sent:<= /b> Thursday, January 19, 2012 8:09 AM
To: Hemal Shah; david.black@emc.com; Michael@huaweisymantec.com; storm@ietf.org
Subject: RE: iSER - one last i= ssue

 

Thanks for pointing out that sentence in R= FC5046 7.3.4. I believe it is the only such stipulation.<= /p>

 

I’m ok with stating the requirement as RFC5046-over-RFC50= 40 compliance. But this new draft opens the door to alternative behaviors, = therefore it needs to be stated more clearly. What’s the requirement = when SendSE is not supported? And, how does the receiver know what to expec= t?

 

In other words, if it’s proposed tha= t the statement be:

&nbs= p;            &= nbsp;  “When the RCaP layer is as specified in [RFC5040] and [RF= C5041], the SendSE message MUST be used.”

 

… then what MUST be used when the RCaP is not? That statement need= s to be made, too.

&n= bsp;

 

From: Hemal Shah [mailto:hemal@broadcom.com]
Sent: Th= ursday, January 19, 2012 10:38 AM
To: Tom Talpey; david.black@emc.com; Michael@huaweisymantec.com; storm@ietf.org
Subject: RE: iSER - one last issue<= o:p>

 

=

Tom,<= o:p>

 

RFC5046 is very clear about = the use of Send Message Type for iSCSI control-type PDU. For specific iSCSI= PDUs, Section 7 mandates the use of Send, SendSE, or SendSEInv. Section 7 = of RFC5046 clearly specifies the use of specific Send Message Type for each= iSCSI control-type PDUs (Login, Logout, Text request, Text response…= ). Section 7.3.4 mandates the use of plain-old Send also as needed. For exa= mple, see text below from 7.3.4.

 

For unsolicited data, if the F bit is set to 0 in a SCSI Dat= a-out

PDU, the iSER layer at the initiator MUST u= se a Send Message to send

the SCSI Data-out PDU.<= o:p>

 

= Regarding your comment in = the second paragraph below, RFC5040 expectation is all the RDMAP messages s= pecified in RFC5040 are supported (if that is not the case then we cannot c= ount on any of the other RDMAP messages to be supported). So, an implementa= tion that does not support SendSE at the sender is not compliant with RFC50= 40. So, your second point does not apply.

 

I hope that addresses your comments.

 

Hemal

=  

From: Tom Talpey [mailto:ttalpey@microsoft.com]
Se= nt: Thursday, January 19, 2012 5:51 AM
To: Hemal Shah; david.black@emc.com; Michael@huaweisymantec.com; storm@ietf.org
Subject: RE: iSER - one la= st issue

 <= /o:p>

If using SendSE is so important here, = when MUST a plain-old Send be used? There is no mention of any distinction = in RFC5046 section 1.4.2 nor in this draft’s section 2.4.2.

 

Even with a MUST on the SendSE, the behavior is indete= rminate, since it would still be contingent on support for SendSE at the *<= b>sender*. This critical exception voids any normative requirement, eve= n over iWARP. The strongest statement that can be made is SHOULD.

 

Tom.

=  

From:<= span style=3D'font-size:10.0pt;font-family:"Tahoma","sans-serif"'> storm-bounces@ietf.org [mailto:storm-bounces@ietf.org] <= b>On Behalf Of Hemal Shah
Sent: Wednesday, January 18, 2012 6= :56 PM
To: david.black@emc= .com; Michael@huaweisyman= tec.com; storm@ietf.org
Sub= ject: Re: [storm] iSER - one last issue

 

David and Mike,

=  

I believe that the requirements should not be we= akened and we should stick to the requirement in RFC5046.=

 

RFC5046 required the use of specific Send Mes= sage Type for valid reasons.

=  

In the case of requiring SendSE message use, the intent was to give a hint= to the remote side to generate an event upon processing of SendSE message.= For example, the SendSE is specifically carrying SCSI information that nee= ds attention on the remote side e.g. SCSI command, Logout…=

 

In the case of requiring SendInvSE Mes= sage, the intent was to invalidate the STag used in the data transfer as we= ll as inform the remote side to generate an event. For example, mandating t= he use of SendInvSE for the SCSI response PDU for a SCSI Read limits the ex= posure of STag used in SCSI Read data transfer and avoids the explicit inva= lidation of the STag at the initiator. Plus, it allows the target to genera= te an event on the initiator upon the completion of SCSI Read command.=

 

By relaxing RFC5046 requirements= in the latest iSER draft, we will not only impact all iWARP based iSER imp= lementations that rely on the use of specific Send Message Type for SCSI da= ta transfers but also change the intended behavior of initiators and target= s.

 

For iWARP, I strongly sug= gest that the iSER draft does not relax the requirements specified in RFC50= 46.

 

I hope that helps.<= /o:p>

 

Hemal    

 

F= rom: storm-bounces@ietf.org [mailto:storm-bounces@= ietf.org] On Behalf Of da= vid.black@emc.com
Sent: Wednesday, January 18, 2012 3:10 PMTo: Michael@huaweisy= mantec.com; storm@ietf.org
= Subject: Re: [storm] iSER - one last issue

<= /div>

 

Mike,<= /o:p>

 

Thank you for the explanation, but I don’t believe you’ve co= rrectly stated

the intent of RFC 5046= .  Here are some examples from RFC 5046’s text:

 

1.5.  SCSI Read= Overview

 

 =   The iSER Message is transferred to the target using a SendSE Message= .

 

&n= bsp;  The iSER layer at the target uses a SendInvSE Message to transfe= r the

   SCSI Response PDU back to the = iSER layer at the initiator.

 

Similar language occurs in 1.6 SCSI Write Overview.=

 

5.2.1.  Normal Connec= tion Termination at the Initiator

 

   The iSER layer at the initiator MUST

   use a SendSE Message to send the Logout Req= uest PDU to the target.

 <= /p>

Similar language occurs in 5.2.2 for target side normal connection = termination,

and more importantly in 7.3.1 SCSI C= ommand:

 

 &n= bsp; The iSER layer at the initiator MUST send the SCSI command in a

   SendSE Message to the target.

 

None of the quoted text per= mits use of Send instead of SendSE or SendInv

ins= tead of SendInvSE.

 

weaker than “SHO= ULD”), and that sure looks like a change to RFC 5046.

 

Are there good implementation-b= ased reasons to weaken these requirements now?

 

Does anyone els= e have a viewpoint on this topic?

 

Thanks,
--David<= /span>

 

From: Michael Ko [mailto:Michael@huaweisymantec.com]
Sent:= Wednesday, January 18, 2012 2:22 PM
To: Black, David; storm@ietf.org
Subject: Re: iSER = - one last issue

 

David,

 

Her= e is my rationale for using the lower case "should".<= /o:p>

 

The intent of RFC5046 i= s that the Send Message type must be used instead of RDMA Reads or Writes.&= nbsp; The Solicited Event feature of the Send Message is provided as a= convenience.  The receiver must still do the right thing in handling = the Send Message regardless of whether the SE feature is used.  In oth= er words, the sender is responsible for using the right format for the= message (Send vs RDMA) but the receiver must not rely on the sender to det= ermine how to handle the received message.  The same rationale goes fo= r the Invalidate feature. 

 

Mike

--= --- Original Message -----

From: david.black@emc.com

Sent: Monday, January 16, 20= 12 2:06 PM

Subject: iSER -= one last issue

=  

Mike,

 

Thanks for gett= ing the -08 version of the iSER draft posted.  I think that=

draft addresses all of the open issues, but I h= ave a question for the WG

about how t= o express the SendSE and SendInvSE requirements from RFC 5046.

 

= The = -08 version of the iSER draft expresses requirements for the SendSE

and SendInvSE messages (this is primarily fo= r iSER over RDDP/iWARP) as:

&nbs= p;

The SendSE M= essage should be used if

 

My reading of RFC 5046 is that its requirements are tight= er - to accurately

reflect RFC 5046, = I would replace “should” with “MUST” in the above t= ext,

at least for iWARP.

 

= In t= he alternative, if the “should”s remain, an explanatory item

needs to be added to the Appendix A lis= t of changes from RFC 5046.

&nbs= p;

What do others think the right course o= f action is here, use “MUST” or

explain weakening of requirement to “should” ?

 

Thanks,
--David

= --_000_7C4DFCE962635144B8FAE8CA11D0BF1E05A7D91649MX14Acorpemcc_-- From ttalpey@microsoft.com Tue Jan 24 19:50:27 2012 Return-Path: X-Original-To: storm@ietfa.amsl.com Delivered-To: storm@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 8382011E80A5 for ; Tue, 24 Jan 2012 19:50:27 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -105.473 X-Spam-Level: X-Spam-Status: No, score=-105.473 tagged_above=-999 required=5 tests=[AWL=1.125, BAYES_00=-2.599, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_MED=-4, USER_IN_WHITELIST=-100] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id c4BELHsRtuT2 for ; Tue, 24 Jan 2012 19:50:18 -0800 (PST) Received: from TX2EHSOBE007.bigfish.com (tx2ehsobe004.messaging.microsoft.com [65.55.88.14]) by ietfa.amsl.com (Postfix) with ESMTP id C964F21F85AE for ; Tue, 24 Jan 2012 19:50:17 -0800 (PST) Received: from mail29-tx2-R.bigfish.com (10.9.14.244) by TX2EHSOBE007.bigfish.com (10.9.40.27) with Microsoft SMTP Server id 14.1.225.23; Wed, 25 Jan 2012 03:50:17 +0000 Received: from mail29-tx2 (localhost [127.0.0.1]) by mail29-tx2-R.bigfish.com (Postfix) with ESMTP id 51BF7220171; Wed, 25 Jan 2012 03:50:17 +0000 (UTC) X-SpamScore: -56 X-BigFish: VS-56(zz9371I1503Mc85fh148cM542M1432N1418Mc1dM4015Lzz1202hzz1033IL8275bh8275dhz2fhc1bhc31hc1ahc1bhc31hc1ah2a8h668h839h) X-Forefront-Antispam-Report: CIP:131.107.125.8; KIP:(null); UIP:(null); IPV:NLI; H:TK5EX14HUBC105.redmond.corp.microsoft.com; RD:none; EFVD:NLI Received-SPF: pass (mail29-tx2: domain of microsoft.com designates 131.107.125.8 as permitted sender) client-ip=131.107.125.8; envelope-from=ttalpey@microsoft.com; helo=TK5EX14HUBC105.redmond.corp.microsoft.com ; icrosoft.com ; Received: from mail29-tx2 (localhost.localdomain [127.0.0.1]) by mail29-tx2 (MessageSwitch) id 1327463414292960_15934; Wed, 25 Jan 2012 03:50:14 +0000 (UTC) Received: from TX2EHSMHS018.bigfish.com (unknown [10.9.14.240]) by mail29-tx2.bigfish.com (Postfix) with ESMTP id 40829420046; Wed, 25 Jan 2012 03:50:14 +0000 (UTC) Received: from TK5EX14HUBC105.redmond.corp.microsoft.com (131.107.125.8) by TX2EHSMHS018.bigfish.com (10.9.99.118) with Microsoft SMTP Server (TLS) id 14.1.225.23; Wed, 25 Jan 2012 03:50:13 +0000 Received: from TK5EX14MBXC111.redmond.corp.microsoft.com ([169.254.2.216]) by TK5EX14HUBC105.redmond.corp.microsoft.com ([157.54.80.48]) with mapi id 14.02.0247.005; Tue, 24 Jan 2012 19:50:12 -0800 From: Tom Talpey To: "david.black@emc.com" , "cbm@chadalapaka.com" , "storm@ietf.org" Thread-Topic: [storm] iSER - one last issue: update Thread-Index: AQHM2jStU9vvKyghkE2rGcTbpoeGCJYbg95AgAE9eYCAAASjAP//oqIw Date: Wed, 25 Jan 2012 03:50:11 +0000 Message-ID: References: <7C4DFCE962635144B8FAE8CA11D0BF1E05A7D9137A@MX14A.corp.emc.com> <7C4DFCE962635144B8FAE8CA11D0BF1E05A7D91649@MX14A.corp.emc.com> In-Reply-To: <7C4DFCE962635144B8FAE8CA11D0BF1E05A7D91649@MX14A.corp.emc.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [157.54.51.78] Content-Type: multipart/alternative; boundary="_000_F83812DF4B59B9499C1BC978336D917461CFBEF1TK5EX14MBXC111r_" MIME-Version: 1.0 X-OriginatorOrg: microsoft.com Subject: Re: [storm] iSER - one last issue: update X-BeenThere: storm@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Storage Maintenance WG List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 25 Jan 2012 03:50:27 -0000 --_000_F83812DF4B59B9499C1BC978336D917461CFBEF1TK5EX14MBXC111r_ Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable To take [3a], making SendSE mandatory for iWARP and mandatory-to-not for al= l others, would be very inconsistent and somewhat contradictory. For exampl= e, Infiniband currently runs at speeds approximately triple that of Etherne= t, and up. Surely a performance-oriented tweak such as SendSE matters more = for IB than iWARP, if it matters at all. And, such a decision makes it more= difficult to design a common upper layer implementation. So, just to throw another log on this fire, [3c], I suggest we ask the Infi= niband community whether they're ok with SendSE/SendInvSE being a MUST for = solicited replies exactly as specified in RFC5046. It means that for Infini= band, all HCAs, but only those TCAs which support solicited events, can be = used with the protocol. And it means the sending side of the existing draft= implementations may have to change to send them where required. From: storm-bounces@ietf.org [mailto:storm-bounces@ietf.org] On Behalf Of d= avid.black@emc.com Sent: Tuesday, January 24, 2012 7:42 PM To: cbm@chadalapaka.com; storm@ietf.org Subject: Re: [storm] iSER - one last issue: update [WG chair hat off] Proposed slight refinement: If the SendSE and SendInvSE requirements are ap= plied to RCaPs that "require all implementations to support Solicited Event= s", that set of RCaPs includes iWARP, but not InfiniBand. That refinement creates a need to figure out the complementary requirement = that applies to InfiniBand, which Tom more or less asked about earlier in t= his thread: > If using SendSE is so important here, when MUST a plain-old Send be used? There are at least a couple of approaches to this one: [3a] SendSE/SendInvSE MUST NOT be used with other RCaPs. [3b] Explain how a sender figures out whether to use SendSE/SendInvSE or no= t (aka "negotiation"). [3a] would be the simplest. The "running code" matters here - does the IB = implementation of iSER use SendSE/SendInvSE? If not, [3a] should be fine. Thanks, --David From: Mallikarjun Chadalapaka [mailto:cbm@chadalapaka.com] Sent: Tuesday, January 24, 2012 7:25 PM To: Tom Talpey; Black, David; storm@ietf.org Subject: RE: [storm] iSER - one last issue: update I think we should go with [1] and [2a] that David outlined. As Hemal said,= there were good reasons why RFC 5046 stipulated SendInvSE and SendSE, wher= e it did. I would rather not weaken it, at least for the iWARP RCaP ecosys= tem. Mallikarjun From: storm-bounces@ietf.org [mailto:storm-b= ounces@ietf.org] On Behalf Of Tom T= alpey Sent: Tuesday, January 24, 2012 5:49 AM To: david.black@emc.com; storm@ietf.org Subject: Re: [storm] iSER - one last issue: update [WG co-chair hat off] > [2a] Leave things alone - if the RCaP supports Solicited Events, then the= "MUST" > requirements for SendSE and SendInvSE apply as they did in RFC 5046. The core issue here is the statement "if the RCaP supports Solicited Events= ". Both the iWARP and Infiniband protocols support them. But Infiniband doe= s not require all adapters support them - the architecture requires it for = HCAs but leaves it optional for TCAs (Infiniband architecture v1.2.1 table = 319 on page 1045). Because this latter property cannot be detected by the r= emote upper layer, it does not appear to be possible to make an interoperab= le decision for all protocols and all implementations. That said, in the absence of any existing iSER-over-iWARP SendSE implementa= tions, I agree the entire discussion is effectively moot. Tom. From: storm-bounces@ietf.org [mailto:storm-b= ounces@ietf.org] On Behalf Of david= .black@emc.com Sent: Monday, January 23, 2012 8:05 PM To: storm@ietf.org Subject: [storm] iSER - one last issue: update [WG chair hat off] This is a message intended for discussion and comments. Looking for a way forward, this observation from Mike seems like a useful p= lace to start: > This iSER update is meant to reflect running code and to generalize the s= upport of iSER over any RCaP, not just iWARP. RFC 5046 (iSER) was written on the assumption that the RCaP layer supports = Solicited Event functionality. I also read RFC 5040 as requiring RDMAP implementatio= ns (part of iWARP) to support Solicited Event functionality. This first step seems easy: [1] There are RCaPs that don't support Solicited Event functionality; those= RCaPs have no choice - they MUST use Send (not SendSE) and SendInv (not SendInvSE). I would prefer to write iSER requirements in terms of whether the RCaP supp= orts Solicited Event functionality as opposed to calling out iWARP explicitly as= having different requirements. OTOH, I'm prepared to listen to arguments to the c= ontrary. Beyond that, I think there are two primary choices: [2a] Leave things alone - if the RCaP supports Solicited Events, then the "= MUST" requirements for SendSE and SendInvSE apply as they did in RFC 5046. [2b] In the alternative, if there are iSER implementations for iWARP that a= ren't using SendSE and SendInvSE in accordance with RFC 5046's requirements, then we ne= ed to change those requirements to match the implementations. I would hope we can agree on [1], and I believe that'll cover the InfiniBan= d case, right? For choosing between [2a] and [2b] it would help a lot to konw what iSER ov= er iWARP implementations are actually doing (SendSE/SendInvSE vs. Send/SendInv). *** Who is familiar with the iSER over iWARP implementation(s) ??? *** In general, what do folks think ought to be done here - [2a], [2b] or somet= hing else? [WG chair hat on] We do need to resolve this issue and reflect that resolution in a -09 iSER = draft before RFC publication can be requested. Thanks, --David From: Tom Talpey [mailto:ttalpey@microsoft.com] Sent: Thursday, January 19, 2012 3:33 PM To: Michael Ko; Hemal Shah; Black, David; storm@ietf.org Subject: RE: iSER - one last issue More precisely, are there any iSER *initiators* that *depend* on receiving = solicited events over the iWARP RDMA transport? I believe this to be a very= short list, if any. But I'd be ok with being proven wrong. From: Michael Ko [mailto:Michael@huaweisymantec.com] Sent: Thursday, January 19, 2012 12:39 PM To: Hemal Shah; Tom Talpey; david.black@emc.com= ; storm@ietf.org Subject: Re: iSER - one last issue For all RCaP layers, the Send message (or any variant) MUST be used, as opp= osed to using RDMA Read or Write to accomplish the task. So a SHOULD for R= CaP other than iWARP is incorrect. RFC 5040 can mandate that all RDMAP mes= sages be supported, but it is still up to implementation to decide what mes= sages to use in any situation. This iSER update is meant to reflect runnin= g code and to generalize the support of iSER over any RCaP, not just iWARP.= So the question is are there any running code that uses SendSE and SendIn= vSE for iSER. Conversely, if the running code can get by using Send messag= es to accomplish the task, what is the point of mandating a MUST at this po= int for SendSE and SendInvSE? Note that the receiver cannot completely rel= y on the sender for the Invalidate feature anyway. The iSER layer at the i= nitiator is required to explicitly invalidate the STag for bidirectional co= mmands and abnormal completion of a command. Even when automatic invalidat= ion is used, the iSER layer is required to sanity checking the automatic in= validation. In other words, the iSER spec puts the burden on the receiver = for doing the right thing in handling the Send messages, and not relying on= the sender for the mere convenience. Mike ----- Original Message ----- From: Hemal Shah To: Tom Talpey ; david.black@emc.com ; Michael@huaweisymantec.com ; storm@ietf.org Sent: Thursday, January 19, 2012 8:44 AM Subject: RE: iSER - one last issue Tom, You are welcome! You are right about 7.3.4 as the only section mandating Se= nd message. I think your concern about specifying the requirement for the other RCaP im= plementation is valid. I suggest we should word the requirements as below: "If the RCaP layer is as specified in [RFC5040] and [RFC5041], the SendSE m= essage MUST be used. For any other RCaP layer, the Send message SHOULD be u= sed. " Hemal From: Tom Talpey [mailto:ttalpey@microsoft.com] Sent: Thursday, January 19, 2012 8:09 AM To: Hemal Shah; david.black@emc.com; Michael@hu= aweisymantec.com; storm@ietf.org Subject: RE: iSER - one last issue Thanks for pointing out that sentence in RFC5046 7.3.4. I believe it is the= only such stipulation. I'm ok with stating the requirement as RFC5046-over-RFC5040 compliance. But= this new draft opens the door to alternative behaviors, therefore it needs= to be stated more clearly. What's the requirement when SendSE is not suppo= rted? And, how does the receiver know what to expect? In other words, if it's proposed that the statement be: "When the RCaP layer is as specified in [RFC5040] and [RFC5= 041], the SendSE message MUST be used." ... then what MUST be used when the RCaP is not? That statement needs to be= made, too. From: Hemal Shah [mailto:hemal@broadcom.com] Sent: Thursday, January 19, 2012 10:38 AM To: Tom Talpey; david.black@emc.com; Michael@hu= aweisymantec.com; storm@ietf.org Subject: RE: iSER - one last issue Tom, RFC5046 is very clear about the use of Send Message Type for iSCSI control-= type PDU. For specific iSCSI PDUs, Section 7 mandates the use of Send, Send= SE, or SendSEInv. Section 7 of RFC5046 clearly specifies the use of specifi= c Send Message Type for each iSCSI control-type PDUs (Login, Logout, Text r= equest, Text response...). Section 7.3.4 mandates the use of plain-old Send= also as needed. For example, see text below from 7.3.4. For unsolicited data, if the F bit is set to 0 in a SCSI Data-out PDU, the iSER layer at the initiator MUST use a Send Message to send the SCSI Data-out PDU. Regarding your comment in the second paragraph below, RFC5040 expectation i= s all the RDMAP messages specified in RFC5040 are supported (if that is not= the case then we cannot count on any of the other RDMAP messages to be sup= ported). So, an implementation that does not support SendSE at the sender i= s not compliant with RFC5040. So, your second point does not apply. I hope that addresses your comments. Hemal From: Tom Talpey [mailto:ttalpey@microsoft.com] Sent: Thursday, January 19, 2012 5:51 AM To: Hemal Shah; david.black@emc.com; Michael@hu= aweisymantec.com; storm@ietf.org Subject: RE: iSER - one last issue If using SendSE is so important here, when MUST a plain-old Send be used? T= here is no mention of any distinction in RFC5046 section 1.4.2 nor in this = draft's section 2.4.2. Even with a MUST on the SendSE, the behavior is indeterminate, since it wou= ld still be contingent on support for SendSE at the *sender*. This critical= exception voids any normative requirement, even over iWARP. The strongest = statement that can be made is SHOULD. Tom. From: storm-bounces@ietf.org [mailto:storm-b= ounces@ietf.org] On Behalf Of Hemal= Shah Sent: Wednesday, January 18, 2012 6:56 PM To: david.black@emc.com; Michael@huaweisymantec= .com; storm@ietf.org Subject: Re: [storm] iSER - one last issue David and Mike, I believe that the requirements should not be weakened and we should stick = to the requirement in RFC5046. RFC5046 required the use of specific Send Message Type for valid reasons. In the case of requiring SendSE message use, the intent was to give a hint = to the remote side to generate an event upon processing of SendSE message. = For example, the SendSE is specifically carrying SCSI information that need= s attention on the remote side e.g. SCSI command, Logout... In the case of requiring SendInvSE Message, the intent was to invalidate th= e STag used in the data transfer as well as inform the remote side to gener= ate an event. For example, mandating the use of SendInvSE for the SCSI resp= onse PDU for a SCSI Read limits the exposure of STag used in SCSI Read data= transfer and avoids the explicit invalidation of the STag at the initiator= . Plus, it allows the target to generate an event on the initiator upon the= completion of SCSI Read command. By relaxing RFC5046 requirements in the latest iSER draft, we will not only= impact all iWARP based iSER implementations that rely on the use of specif= ic Send Message Type for SCSI data transfers but also change the intended b= ehavior of initiators and targets. For iWARP, I strongly suggest that the iSER draft does not relax the requir= ements specified in RFC5046. I hope that helps. Hemal From: storm-bounces@ietf.org [mailto:storm-b= ounces@ietf.org] On Behalf Of david= .black@emc.com Sent: Wednesday, January 18, 2012 3:10 PM To: Michael@huaweisymantec.com; storm@ie= tf.org Subject: Re: [storm] iSER - one last issue Mike, Thank you for the explanation, but I don't believe you've correctly stated the intent of RFC 5046. Here are some examples from RFC 5046's text: 1.5. SCSI Read Overview The iSER Message is transferred to the target using a SendSE Message. The iSER layer at the target uses a SendInvSE Message to transfer the SCSI Response PDU back to the iSER layer at the initiator. Similar language occurs in 1.6 SCSI Write Overview. 5.2.1. Normal Connection Termination at the Initiator The iSER layer at the initiator MUST use a SendSE Message to send the Logout Request PDU to the target. Similar language occurs in 5.2.2 for target side normal connection terminat= ion, and more importantly in 7.3.1 SCSI Command: The iSER layer at the initiator MUST send the SCSI command in a SendSE Message to the target. None of the quoted text permits use of Send instead of SendSE or SendInv instead of SendInvSE. What you propose to do effectively changes "MUST" to "should" (lower case, weaker than "SHOULD"), and that sure looks like a change to RFC 5046. Are there good implementation-based reasons to weaken these requirements no= w? Does anyone else have a viewpoint on this topic? Thanks, --David From: Michael Ko [mailto:Michael@huaweisymantec.com] Sent: Wednesday, January 18, 2012 2:22 PM To: Black, David; storm@ietf.org Subject: Re: iSER - one last issue David, Here is my rationale for using the lower case "should". The intent of RFC5046 is that the Send Message type must be used instead of= RDMA Reads or Writes. The Solicited Event feature of the Send Message is = provided as a convenience. The receiver must still do the right thing in h= andling the Send Message regardless of whether the SE feature is used. In = other words, the sender is responsible for using the right format for the m= essage (Send vs RDMA) but the receiver must not rely on the sender to deter= mine how to handle the received message. The same rationale goes for the I= nvalidate feature. Mike ----- Original Message ----- From: david.black@emc.com To: Michael@huaweisymantec.com ; storm@i= etf.org Sent: Monday, January 16, 2012 2:06 PM Subject: iSER - one last issue Mike, Thanks for getting the -08 version of the iSER draft posted. I think that draft addresses all of the open issues, but I have a question for the WG about how to express the SendSE and SendInvSE requirements from RFC 5046. The -08 version of the iSER draft expresses requirements for the SendSE and SendInvSE messages (this is primarily for iSER over RDDP/iWARP) as: The SendSE Message should be used if supported by the RCaP layer (e.g., iWARP). My reading of RFC 5046 is that its requirements are tighter - to accurately reflect RFC 5046, I would replace "should" with "MUST" in the above text, at least for iWARP. In the alternative, if the "should"s remain, an explanatory item needs to be added to the Appendix A list of changes from RFC 5046. What do others think the right course of action is here, use "MUST" or explain weakening of requirement to "should" ? Thanks, --David --_000_F83812DF4B59B9499C1BC978336D917461CFBEF1TK5EX14MBXC111r_ Content-Type: text/html; charset="us-ascii" Content-Transfer-Encoding: quoted-printable

To take [3a], making Send= SE mandatory for iWARP and mandatory-to-not for all others, would be very i= nconsistent and somewhat contradictory. For example, Infiniband currently runs at speeds approximately triple that of Ethernet, and up. Su= rely a performance-oriented tweak such as SendSE matters more for IB than i= WARP, if it matters at all. And, such a decision makes it more difficult to= design a common upper layer implementation.

 <= /p>

So, just to throw another= log on this fire, [3c], I suggest we ask the Infiniband community whether = they’re ok with SendSE/SendInvSE being a MUST for solicited replies exactly as specified in RFC5046. It means that for Infiniband, all= HCAs, but only those TCAs which support solicited events, can be used with= the protocol. And it means the sending side of the existing draft implemen= tations may have to change to send them where required.

 <= /p>

 <= /p>

From: storm-bo= unces@ietf.org [mailto:storm-bounces@ietf.org] On Behalf Of david.black@emc.com
Sent: Tuesday, January 24, 2012 7:42 PM
To: cbm@chadalapaka.com; storm@ietf.org
Subject: Re: [storm] iSER - one last issue: update=

 

[WG chair hat off]

 

Proposed slight refinement: If the SendSE and = SendInvSE requirements are applied to RCaPs that “require all impleme= ntations to support Solicited Events”, that set of RCaPs includes iWARP, but not InfiniBand.

 

That refinement creates a need to figure out t= he complementary requirement that applies to InfiniBand, which Tom more or = less asked about earlier in this thread:

 

> If using SendSE is so important here, whe= n MUST a plain-old Send be used?

 <= /p>

There are at least a couple of approaches to t= his one:

 

[3a] SendSE/SendInvSE MUST NOT be used with ot= her RCaPs.

 

[3b] Explain how a sender figures out whether = to use SendSE/SendInvSE or not (aka “negotiation”).<= /span>

 

[3a] would be the simplest.  The “r= unning code” matters here - does the IB implementation of iSER use Se= ndSE/SendInvSE?  If not, [3a] should be fine.

 

Thanks,
--David

 

From: Mallikar= jun Chadalapaka [mailto:cbm@chadalapaka.com= ]
Sent: Tuesday, January 24, 2012 7:25 PM
To: Tom Talpey; Black, David; stor= m@ietf.org
Subject: RE: [storm] iSER - one last issue: update=

 

I think we should go with= [1] and [2a] that David outlined.  As Hemal said, there were good rea= sons why RFC 5046 stipulated SendInvSE and SendSE, where it did.  I would rather not weaken it, at least for the iWARP RCaP ecosystem.<= /o:p>

 <= /p>

Mallikarjun

 <= /p>

 <= /p>

From: storm-bounces@ietf.org [mailto:storm-bounces@ietf.org] On Behalf Of Tom Talpey
Sent: Tuesday, January 24, 2012 5:49 AM
To: david.black@emc.com; = storm@ietf.org
Subject: Re: [storm] iSER - one last issue: update=

 

[WG co-chair hat off]

 <= /p>

> [2a] Leave things alone - if the RCaP sup= ports Solicited Events, then the “MUST”

> requirements for SendSE and SendInvSE app= ly as they did in RFC 5046.

 <= /p>

The core issue here is th= e statement “if the RCaP supports Solicited Events”. Both the i= WARP and Infiniband protocols support them. But Infiniband does not require all adapters support them – the architecture requires it for= HCAs but leaves it optional for TCAs (Infiniband architecture v1.2.1 table= 319 on page 1045). Because this latter property cannot be detected by the = remote upper layer, it does not appear to be possible to make an interoperable decision for all protocols and all= implementations.

 <= /p>

That said, in the absence= of any existing iSER-over-iWARP SendSE implementations, I agree the entire= discussion is effectively moot.

 <= /p>

Tom.

 <= /p>

From: storm-bounces@ietf.org [mailto:storm-bounces@ietf.org] On Behalf Of david.black@emc.com
Sent: Monday, January 23, 2012 8:05 PM
To: storm@ietf.org
Subject: [storm] iSER - one last issue: update

 

[WG chair hat off]

 

This is a message intended for discussion and = comments.

 

Looking for a way forward, this observation fr= om Mike seems like a useful place to start:

 

> This iSER upda= te is meant to reflect running code and to generalize the support of iSER o= ver any RCaP, not just iWARP. 

 

RFC 5046 (iSER) was written on the assumption = that the RCaP layer supports Solicited

Event functionality.  I also read RFC 504= 0 as requiring RDMAP implementations (part of

iWARP) to support Solicited Event functionalit= y.

 

This first step seems easy:<= /p>

 

[1] There are RCaPs that don’t support S= olicited Event functionality; those RCaPs have

no choice - they MUST use Send (not SendSE) an= d SendInv (not SendInvSE). 

       &nbs= p;            &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;         

I would prefer to write iSER requirements in t= erms of whether the RCaP supports

Solicited Event functionality as opposed to ca= lling out iWARP explicitly as having

different requirements.  OTOH, I’m = prepared to listen to arguments to the contrary.

 

Beyond that, I think there are two primary cho= ices:

 

[2a] Leave things alone - if the RCaP supports= Solicited Events, then the “MUST”

requirements for SendSE and SendInvSE apply as= they did in RFC 5046.

 

[2b] In the alternative, if there are iSER imp= lementations for iWARP that aren’t using

SendSE and SendInvSE in accordance with RFC 50= 46’s requirements, then we need to change

those requirements to match the implementation= s.

 

I would hope we can agree on [1], and I believ= e that’ll cover the InfiniBand case, right?

 

For choosing between [2a] and [2b] it would he= lp a lot to konw what iSER over iWARP

implementations are actually doing (SendSE/Sen= dInvSE vs. Send/SendInv).

 

*** Who is familiar with the iSER over iWARP i= mplementation(s) ??? ***

 

In general, what do folks think ought to be do= ne here - [2a], [2b] or something else?

 

[WG chair hat on]

 

We do need to resolve this issue and reflect t= hat resolution in a -09 iSER draft before

RFC publication can be requested.

 

Thanks,
--David

 

From: Tom Talp= ey [mailto:ttalpey@microsoft= .com]
Sent: Thursday, January 19, 2012 3:33 PM
To: Michael Ko; Hemal Shah; Black, David; storm@ietf.org
Subject: RE: iSER - one last issue

 

More precisely, are there= any iSER *initiators* that *depend* on receiving solicited e= vents over the iWARP RDMA transport? I believe this to be a very short list, if any. But I’d be ok with being proven wrong.

 <= /p>

 <= /p>

 <= /p>

From: Michael = Ko [mailto:Michael@huaw= eisymantec.com]
Sent: Thursday, January 19, 2012 12:39 PM
To: Hemal Shah; Tom Talpey; d= avid.black@emc.com; storm@ietf.org
Subject: Re: iSER - one last issue

 

For all RCaP layers= , the Send message (or any variant) MUST be used, as opposed to using RDMA = Read or Write to accomplish the task.  So a SHOULD for RCaP other= than iWARP is incorrect.  RFC 5040 can mandate that all RDMAP messages be supported, but it is still up to implementation= to decide what messages to use in any situation.  This iSER update is= meant to reflect running code and to generalize the support of iSER over a= ny RCaP, not just iWARP.  So the question is are there any running code that uses SendSE and SendInvSE for iSER.&nbs= p; Conversely, if the running code can get by using Send messages to accomp= lish the task, what is the point of mandating a MUST at this point for Send= SE and SendInvSE?  Note that the receiver cannot completely rely on the sender for the Invalidate feature anyway.&nb= sp; The iSER layer at the initiator is required to explicitly invalida= te the STag for bidirectional commands and abnormal completion of a command= .  Even when automatic invalidation is used, the iSER layer is required to sanity checking the automatic invalidation.&= nbsp; In other words, the iSER spec puts the burden on the receiv= er for doing the right thing in handling the Send messages, and not re= lying on the sender for the mere convenience. 

 

Mike

 

Tom= ,

 

You= are welcome! You are right about 7.3.4 as the only section mandating Send = message.

 

I t= hink your concern about specifying the requirement for the other RCaP imple= mentation is valid. I suggest we should word the requirements as below:

 

“If the RCaP layer = is as specified in [RFC5040] and [RFC5041], the SendSE message MUST be used= . For any other RCaP layer, the Send message SHOULD be used. ”

 <= /p>

Hem= al

 

From: Tom Talp= ey [mailto:ttalpey@microsoft= .com]
Sent: Thursday, January 19, 2012 8:09 AM
To: Hemal Shah; david.black@e= mc.com; Michael@huaweisymantec.com; storm@ietf.org
Subject: RE: iSER - one last issue

 

Thanks for pointing out t= hat sentence in RFC5046 7.3.4. I believe it is the only such stipulation.

 <= /p>

I’m ok with stating= the requirement as RFC5046-over-RFC5040 compliance. But this new draft ope= ns the door to alternative behaviors, therefore it needs to be stated more clearly. What’s the requirement when SendSE is not suppo= rted? And, how does the receiver know what to expect?

 <= /p>

In other words, if itR= 17;s proposed that the statement be:

    &= nbsp;           “Wh= en the RCaP layer is as specified in [RFC5040] and [RFC5041], the SendSE me= ssage MUST be used.”

 <= /p>

… then what MUST be= used when the RCaP is not? That statement needs to be made, too.

 <= /p>

 <= /p>

 

Tom= ,

 

RFC= 5046 is very clear about the use of Send Message Type for iSCSI control-typ= e PDU. For specific iSCSI PDUs, Section 7 mandates the use of Send, SendSE,= or SendSEInv. Section 7 of RFC5046 clearly specifies the use of specific Send Message Type for each iSCSI con= trol-type PDUs (Login, Logout, Text request, Text response…). Section= 7.3.4 mandates the use of plain-old Send also as needed. For example, see = text below from 7.3.4.

 

For unsolicited data, if the F bit is set to 0 in a SCSI D= ata-out

PDU, the iSER layer at the initiator MUST use a Send Messa= ge to send

the SCSI Data-out PDU.

 

Reg= arding your comment in the second paragraph below, RFC5040 expectation is a= ll the RDMAP messages specified in RFC5040 are supported (if that is not th= e case then we cannot count on any of the other RDMAP messages to be supported). So, an implementation that does= not support SendSE at the sender is not compliant with RFC5040. So, your s= econd point does not apply.

 

I h= ope that addresses your comments.

 

Hem= al

 

From: Tom Talp= ey [mailto:ttalpey@microsoft= .com]
Sent: Thursday, January 19, 2012 5:51 AM
To: Hemal Shah; david.black@e= mc.com; Michael@huaweisymantec.com; storm@ietf.org
Subject: RE: iSER - one last issue

 

If using SendSE is so imp= ortant here, when MUST a plain-old Send be used? There is no mention of any= distinction in RFC5046 section 1.4.2 nor in this draft’s section 2.4.2.

 <= /p>

Even with a MUST on the S= endSE, the behavior is indeterminate, since it would still be contingent on= support for SendSE at the *sender*. This critical exception voids any normative requirement, even over iWARP. The strongest = statement that can be made is SHOULD.

 <= /p>

Tom.

 <= /p>

From: storm-bounces@ietf.org [mailto:storm-bounces@ietf.org] On Behalf Of Hemal Shah
Sent: Wednesday, January 18, 2012 6:56 PM
To: david.black@emc.com; = Michael@huaweisymantec.com; storm@iet= f.org
Subject: Re: [storm] iSER - one last issue

 

Dav= id and Mike,

 

I b= elieve that the requirements should not be weakened and we should stick to = the requirement in RFC5046.

 

RFC= 5046 required the use of specific Send Message Type for valid reasons.=

 

In = the case of requiring SendSE message use, the intent was to give a hint to = the remote side to generate an event upon processing of SendSE message. For= example, the SendSE is specifically carrying SCSI information that needs attention on the remote side e.g. SCS= I command, Logout…

 

In = the case of requiring SendInvSE Message, the intent was to invalidate the S= Tag used in the data transfer as well as inform the remote side to generate= an event. For example, mandating the use of SendInvSE for the SCSI response PDU for a SCSI Read limits the expo= sure of STag used in SCSI Read data transfer and avoids the explicit invali= dation of the STag at the initiator. Plus, it allows the target to generate= an event on the initiator upon the completion of SCSI Read command.

 

By = relaxing RFC5046 requirements in the latest iSER draft, we will not only im= pact all iWARP based iSER implementations that rely on the use of specific = Send Message Type for SCSI data transfers but also change the intended behavior of initiators and targets.

 

For= iWARP, I strongly suggest that the iSER draft does not relax the requireme= nts specified in RFC5046.

 

I h= ope that helps.

 

Hem= al    

 

From: storm-bounces@ietf.org [mailto:storm-bounces@ietf.org] On Behalf Of david.black@emc.com
Sent: Wednesday, January 18, 2012 3:10 PM
To: Michael@huaweisyma= ntec.com; storm@ietf.org
Subject: Re: [storm] iSER - one last issue

 

Mike,

 

Thank you for the explanation, but I don’= ;t believe you’ve correctly stated

the intent of RFC 5046.  Here are some ex= amples from RFC 5046’s text:

 

1.5.  SCSI Read Overview

 

   The iSER Message is transferred to the target= using a SendSE Message.

 

   The iSER layer at the target uses a SendInvSE= Message to transfer the

   SCSI Response PDU back to the iSER layer at t= he initiator.

 

Similar language occurs in 1.6 SCSI Write Overview.

 

5.2.1.  Normal Connection Termination at the Initiato= r

 

   The iSER layer at the initiator MUST

   use a SendSE Message to send the Logout Reque= st PDU to the target.

 

Similar language occurs in 5.2.2 for target side normal co= nnection termination,

and more importantly in 7.3.1 SCSI Command:

 

   The iSER layer at the initiator MUST send the= SCSI command in a

   SendSE Message to the target.

 

None of the quoted text permits use of Send instead of Sen= dSE or SendInv

instead of SendInvSE.

 

What you propose to do effectively changes “MUSTR= 21; to “should” (lower case,

weaker than “SHOULD”), and that sure looks lik= e a change to RFC 5046.

 

Are there good implementation-based reasons to weaken thes= e requirements now?

 

Does anyone else have a viewpoint on this topi= c?

 

Thanks,
--David

 

From: Michael = Ko [mailto:Michael@huaw= eisymantec.com]
Sent: Wednesday, January 18, 2012 2:22 PM
To: Black, David; storm@ietf.org
Subject: Re: iSER - one last issue

 

David,<= /o:p>

 

Here is my rational= e for using the lower case "should".

 

The intent of RFC50= 46 is that the Send Message type must be used instead of RDMA Reads or Writ= es.  The Solicited Event feature of the Send Message is provided = as a convenience.  The receiver must still do the right thing in handling the Send Message regardless of whether the SE = feature is used.  In other words, the sender is responsible for using = the right format for the message (Send vs RDMA) but the receiver must = not rely on the sender to determine how to handle the received message.  The same rationale goes for the Invalid= ate feature. 

 

Mike

 

Mike,

 

Thanks for getting the -08 version of the iSER= draft posted.  I think that

draft addresses all of the open issues, but I = have a question for the WG

about how to express the SendSE and SendInvSE = requirements from RFC 5046.

 

The -08 version of the iSER draft expresses re= quirements for the SendSE

and SendInvSE messages (this is primarily for = iSER over RDDP/iWARP) as:

 

The SendSE Message = should be used if

supported by the RC= aP layer (e.g., iWARP).

 

My reading of RFC 5046 is that its requirement= s are tighter - to accurately

reflect RFC 5046, I would replace “shoul= d” with “MUST” in the above text,

at least for iWARP.

 

In the alternative, if the “should”= ;s remain, an explanatory item

needs to be added to the Appendix A list of ch= anges from RFC 5046.

 

What do others think the right course of actio= n is here, use “MUST” or

explain weakening of requirement to “sho= uld” ?

 

Thanks,
--David

--_000_F83812DF4B59B9499C1BC978336D917461CFBEF1TK5EX14MBXC111r_-- From hemal@broadcom.com Wed Jan 25 15:48:05 2012 Return-Path: X-Original-To: storm@ietfa.amsl.com Delivered-To: storm@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 88E0511E8089 for ; Wed, 25 Jan 2012 15:48:05 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -6.598 X-Spam-Level: X-Spam-Status: No, score=-6.598 tagged_above=-999 required=5 tests=[AWL=-0.000, BAYES_00=-2.599, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_MED=-4] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id woeMMbBGuGXE for ; Wed, 25 Jan 2012 15:47:56 -0800 (PST) Received: from MMS3.broadcom.com (mms3.broadcom.com [216.31.210.19]) by ietfa.amsl.com (Postfix) with ESMTP id 8EE9811E8071 for ; Wed, 25 Jan 2012 15:47:55 -0800 (PST) Received: from [10.9.208.27] by MMS3.broadcom.com with ESMTP (Broadcom SMTP Relay (Email Firewall v6.3.2)); Wed, 25 Jan 2012 15:55:47 -0800 X-Server-Uuid: B55A25B1-5D7D-41F8-BC53-C57E7AD3C201 Received: from IRVEXCHMB11.corp.ad.broadcom.com ( [fe80::9cdd:3a57:8694:f610]) by irvexchmb05.corp.ad.broadcom.com ( [::1]) with mapi id 14.01.0289.001; Wed, 25 Jan 2012 15:47:31 -0800 From: "Hemal Shah" To: "Tom Talpey" , "david.black@emc.com" , "cbm@chadalapaka.com" , "storm@ietf.org" Thread-Topic: [storm] iSER - one last issue: update Thread-Index: AQHM2jStRlyU+UWmekShwv3mXE7R65YcD2gAgACx74CAAASjAIAANJiAgADHgeA= Date: Wed, 25 Jan 2012 23:47:31 +0000 Message-ID: <2D98DD3F898B6B4DA287BF3BA07DAE93029056@IRVEXCHMB11.corp.ad.broadcom.com> References: <7C4DFCE962635144B8FAE8CA11D0BF1E05A7D9137A@MX14A.corp.emc.com> <7C4DFCE962635144B8FAE8CA11D0BF1E05A7D91649@MX14A.corp.emc.com> In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [10.12.148.247] MIME-Version: 1.0 X-WSS-ID: 633E490950417342957-01-01 Content-Type: multipart/alternative; boundary=_000_2D98DD3F898B6B4DA287BF3BA07DAE93029056IRVEXCHMB11corpad_ Subject: Re: [storm] iSER - one last issue: update X-BeenThere: storm@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Storage Maintenance WG List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 25 Jan 2012 23:48:05 -0000 --_000_2D98DD3F898B6B4DA287BF3BA07DAE93029056IRVEXCHMB11corpad_ Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable I think we should investigate [3c]. From the consistency standpoint, [3c] m= akes iSER consistent across different RCaPs. Can someone from InfiniBand side comment? Hemal From: storm-bounces@ietf.org [mailto:storm-bounces@ietf.org] On Behalf Of T= om Talpey Sent: Tuesday, January 24, 2012 7:50 PM To: david.black@emc.com; cbm@chadalapaka.com; storm@ietf.org Subject: Re: [storm] iSER - one last issue: update To take [3a], making SendSE mandatory for iWARP and mandatory-to-not for al= l others, would be very inconsistent and somewhat contradictory. For exampl= e, Infiniband currently runs at speeds approximately triple that of Etherne= t, and up. Surely a performance-oriented tweak such as SendSE matters more = for IB than iWARP, if it matters at all. And, such a decision makes it more= difficult to design a common upper layer implementation. So, just to throw another log on this fire, [3c], I suggest we ask the Infi= niband community whether they're ok with SendSE/SendInvSE being a MUST for = solicited replies exactly as specified in RFC5046. It means that for Infini= band, all HCAs, but only those TCAs which support solicited events, can be = used with the protocol. And it means the sending side of the existing draft= implementations may have to change to send them where required. From: storm-bounces@ietf.org [mailto:storm-bounces@ietf.org] On Behalf Of d= avid.black@emc.com Sent: Tuesday, January 24, 2012 7:42 PM To: cbm@chadalapaka.com; storm@ietf.org Subject: Re: [storm] iSER - one last issue: update [WG chair hat off] Proposed slight refinement: If the SendSE and SendInvSE requirements are ap= plied to RCaPs that "require all implementations to support Solicited Event= s", that set of RCaPs includes iWARP, but not InfiniBand. That refinement creates a need to figure out the complementary requirement = that applies to InfiniBand, which Tom more or less asked about earlier in t= his thread: > If using SendSE is so important here, when MUST a plain-old Send be used? There are at least a couple of approaches to this one: [3a] SendSE/SendInvSE MUST NOT be used with other RCaPs. [3b] Explain how a sender figures out whether to use SendSE/SendInvSE or no= t (aka "negotiation"). [3a] would be the simplest. The "running code" matters here - does the IB = implementation of iSER use SendSE/SendInvSE? If not, [3a] should be fine. Thanks, --David From: Mallikarjun Chadalapaka [mailto:cbm@chadalapaka.com] Sent: Tuesday, January 24, 2012 7:25 PM To: Tom Talpey; Black, David; storm@ietf.org Subject: RE: [storm] iSER - one last issue: update I think we should go with [1] and [2a] that David outlined. As Hemal said,= there were good reasons why RFC 5046 stipulated SendInvSE and SendSE, wher= e it did. I would rather not weaken it, at least for the iWARP RCaP ecosys= tem. Mallikarjun From: storm-bounces@ietf.org [mailto:storm-b= ounces@ietf.org] On Behalf Of Tom T= alpey Sent: Tuesday, January 24, 2012 5:49 AM To: david.black@emc.com; storm@ietf.org Subject: Re: [storm] iSER - one last issue: update [WG co-chair hat off] > [2a] Leave things alone - if the RCaP supports Solicited Events, then the= "MUST" > requirements for SendSE and SendInvSE apply as they did in RFC 5046. The core issue here is the statement "if the RCaP supports Solicited Events= ". Both the iWARP and Infiniband protocols support them. But Infiniband doe= s not require all adapters support them - the architecture requires it for = HCAs but leaves it optional for TCAs (Infiniband architecture v1.2.1 table = 319 on page 1045). Because this latter property cannot be detected by the r= emote upper layer, it does not appear to be possible to make an interoperab= le decision for all protocols and all implementations. That said, in the absence of any existing iSER-over-iWARP SendSE implementa= tions, I agree the entire discussion is effectively moot. Tom. From: storm-bounces@ietf.org [mailto:storm-b= ounces@ietf.org] On Behalf Of david= .black@emc.com Sent: Monday, January 23, 2012 8:05 PM To: storm@ietf.org Subject: [storm] iSER - one last issue: update [WG chair hat off] This is a message intended for discussion and comments. Looking for a way forward, this observation from Mike seems like a useful p= lace to start: > This iSER update is meant to reflect running code and to generalize the s= upport of iSER over any RCaP, not just iWARP. RFC 5046 (iSER) was written on the assumption that the RCaP layer supports = Solicited Event functionality. I also read RFC 5040 as requiring RDMAP implementatio= ns (part of iWARP) to support Solicited Event functionality. This first step seems easy: [1] There are RCaPs that don't support Solicited Event functionality; those= RCaPs have no choice - they MUST use Send (not SendSE) and SendInv (not SendInvSE). I would prefer to write iSER requirements in terms of whether the RCaP supp= orts Solicited Event functionality as opposed to calling out iWARP explicitly as= having different requirements. OTOH, I'm prepared to listen to arguments to the c= ontrary. Beyond that, I think there are two primary choices: [2a] Leave things alone - if the RCaP supports Solicited Events, then the "= MUST" requirements for SendSE and SendInvSE apply as they did in RFC 5046. [2b] In the alternative, if there are iSER implementations for iWARP that a= ren't using SendSE and SendInvSE in accordance with RFC 5046's requirements, then we ne= ed to change those requirements to match the implementations. I would hope we can agree on [1], and I believe that'll cover the InfiniBan= d case, right? For choosing between [2a] and [2b] it would help a lot to konw what iSER ov= er iWARP implementations are actually doing (SendSE/SendInvSE vs. Send/SendInv). *** Who is familiar with the iSER over iWARP implementation(s) ??? *** In general, what do folks think ought to be done here - [2a], [2b] or somet= hing else? [WG chair hat on] We do need to resolve this issue and reflect that resolution in a -09 iSER = draft before RFC publication can be requested. Thanks, --David From: Tom Talpey [mailto:ttalpey@microsoft.com] Sent: Thursday, January 19, 2012 3:33 PM To: Michael Ko; Hemal Shah; Black, David; storm@ietf.org Subject: RE: iSER - one last issue More precisely, are there any iSER *initiators* that *depend* on receiving = solicited events over the iWARP RDMA transport? I believe this to be a very= short list, if any. But I'd be ok with being proven wrong. From: Michael Ko [mailto:Michael@huaweisymantec.com] Sent: Thursday, January 19, 2012 12:39 PM To: Hemal Shah; Tom Talpey; david.black@emc.com= ; storm@ietf.org Subject: Re: iSER - one last issue For all RCaP layers, the Send message (or any variant) MUST be used, as opp= osed to using RDMA Read or Write to accomplish the task. So a SHOULD for R= CaP other than iWARP is incorrect. RFC 5040 can mandate that all RDMAP mes= sages be supported, but it is still up to implementation to decide what mes= sages to use in any situation. This iSER update is meant to reflect runnin= g code and to generalize the support of iSER over any RCaP, not just iWARP.= So the question is are there any running code that uses SendSE and SendIn= vSE for iSER. Conversely, if the running code can get by using Send messag= es to accomplish the task, what is the point of mandating a MUST at this po= int for SendSE and SendInvSE? Note that the receiver cannot completely rel= y on the sender for the Invalidate feature anyway. The iSER layer at the i= nitiator is required to explicitly invalidate the STag for bidirectional co= mmands and abnormal completion of a command. Even when automatic invalidat= ion is used, the iSER layer is required to sanity checking the automatic in= validation. In other words, the iSER spec puts the burden on the receiver = for doing the right thing in handling the Send messages, and not relying on= the sender for the mere convenience. Mike ----- Original Message ----- From: Hemal Shah To: Tom Talpey ; david.black@emc.com ; Michael@huaweisymantec.com ; storm@ietf.org Sent: Thursday, January 19, 2012 8:44 AM Subject: RE: iSER - one last issue Tom, You are welcome! You are right about 7.3.4 as the only section mandating Se= nd message. I think your concern about specifying the requirement for the other RCaP im= plementation is valid. I suggest we should word the requirements as below: "If the RCaP layer is as specified in [RFC5040] and [RFC5041], the SendSE m= essage MUST be used. For any other RCaP layer, the Send message SHOULD be u= sed. " Hemal From: Tom Talpey [mailto:ttalpey@microsoft.com] Sent: Thursday, January 19, 2012 8:09 AM To: Hemal Shah; david.black@emc.com; Michael@hu= aweisymantec.com; storm@ietf.org Subject: RE: iSER - one last issue Thanks for pointing out that sentence in RFC5046 7.3.4. I believe it is the= only such stipulation. I'm ok with stating the requirement as RFC5046-over-RFC5040 compliance. But= this new draft opens the door to alternative behaviors, therefore it needs= to be stated more clearly. What's the requirement when SendSE is not suppo= rted? And, how does the receiver know what to expect? In other words, if it's proposed that the statement be: "When the RCaP layer is as specified in [RFC5040] and [RFC5= 041], the SendSE message MUST be used." ... then what MUST be used when the RCaP is not? That statement needs to be= made, too. From: Hemal Shah [mailto:hemal@broadcom.com] Sent: Thursday, January 19, 2012 10:38 AM To: Tom Talpey; david.black@emc.com; Michael@hu= aweisymantec.com; storm@ietf.org Subject: RE: iSER - one last issue Tom, RFC5046 is very clear about the use of Send Message Type for iSCSI control-= type PDU. For specific iSCSI PDUs, Section 7 mandates the use of Send, Send= SE, or SendSEInv. Section 7 of RFC5046 clearly specifies the use of specifi= c Send Message Type for each iSCSI control-type PDUs (Login, Logout, Text r= equest, Text response...). Section 7.3.4 mandates the use of plain-old Send= also as needed. For example, see text below from 7.3.4. For unsolicited data, if the F bit is set to 0 in a SCSI Data-out PDU, the iSER layer at the initiator MUST use a Send Message to send the SCSI Data-out PDU. Regarding your comment in the second paragraph below, RFC5040 expectation i= s all the RDMAP messages specified in RFC5040 are supported (if that is not= the case then we cannot count on any of the other RDMAP messages to be sup= ported). So, an implementation that does not support SendSE at the sender i= s not compliant with RFC5040. So, your second point does not apply. I hope that addresses your comments. Hemal From: Tom Talpey [mailto:ttalpey@microsoft.com] Sent: Thursday, January 19, 2012 5:51 AM To: Hemal Shah; david.black@emc.com; Michael@hu= aweisymantec.com; storm@ietf.org Subject: RE: iSER - one last issue If using SendSE is so important here, when MUST a plain-old Send be used? T= here is no mention of any distinction in RFC5046 section 1.4.2 nor in this = draft's section 2.4.2. Even with a MUST on the SendSE, the behavior is indeterminate, since it wou= ld still be contingent on support for SendSE at the *sender*. This critical= exception voids any normative requirement, even over iWARP. The strongest = statement that can be made is SHOULD. Tom. From: storm-bounces@ietf.org [mailto:storm-b= ounces@ietf.org] On Behalf Of Hemal= Shah Sent: Wednesday, January 18, 2012 6:56 PM To: david.black@emc.com; Michael@huaweisymantec= .com; storm@ietf.org Subject: Re: [storm] iSER - one last issue David and Mike, I believe that the requirements should not be weakened and we should stick = to the requirement in RFC5046. RFC5046 required the use of specific Send Message Type for valid reasons. In the case of requiring SendSE message use, the intent was to give a hint = to the remote side to generate an event upon processing of SendSE message. = For example, the SendSE is specifically carrying SCSI information that need= s attention on the remote side e.g. SCSI command, Logout... In the case of requiring SendInvSE Message, the intent was to invalidate th= e STag used in the data transfer as well as inform the remote side to gener= ate an event. For example, mandating the use of SendInvSE for the SCSI resp= onse PDU for a SCSI Read limits the exposure of STag used in SCSI Read data= transfer and avoids the explicit invalidation of the STag at the initiator= . Plus, it allows the target to generate an event on the initiator upon the= completion of SCSI Read command. By relaxing RFC5046 requirements in the latest iSER draft, we will not only= impact all iWARP based iSER implementations that rely on the use of specif= ic Send Message Type for SCSI data transfers but also change the intended b= ehavior of initiators and targets. For iWARP, I strongly suggest that the iSER draft does not relax the requir= ements specified in RFC5046. I hope that helps. Hemal From: storm-bounces@ietf.org [mailto:storm-b= ounces@ietf.org] On Behalf Of david= .black@emc.com Sent: Wednesday, January 18, 2012 3:10 PM To: Michael@huaweisymantec.com; storm@ie= tf.org Subject: Re: [storm] iSER - one last issue Mike, Thank you for the explanation, but I don't believe you've correctly stated the intent of RFC 5046. Here are some examples from RFC 5046's text: 1.5. SCSI Read Overview The iSER Message is transferred to the target using a SendSE Message. The iSER layer at the target uses a SendInvSE Message to transfer the SCSI Response PDU back to the iSER layer at the initiator. Similar language occurs in 1.6 SCSI Write Overview. 5.2.1. Normal Connection Termination at the Initiator The iSER layer at the initiator MUST use a SendSE Message to send the Logout Request PDU to the target. Similar language occurs in 5.2.2 for target side normal connection terminat= ion, and more importantly in 7.3.1 SCSI Command: The iSER layer at the initiator MUST send the SCSI command in a SendSE Message to the target. None of the quoted text permits use of Send instead of SendSE or SendInv instead of SendInvSE. What you propose to do effectively changes "MUST" to "should" (lower case, weaker than "SHOULD"), and that sure looks like a change to RFC 5046. Are there good implementation-based reasons to weaken these requirements no= w? Does anyone else have a viewpoint on this topic? Thanks, --David From: Michael Ko [mailto:Michael@huaweisymantec.com] Sent: Wednesday, January 18, 2012 2:22 PM To: Black, David; storm@ietf.org Subject: Re: iSER - one last issue David, Here is my rationale for using the lower case "should". The intent of RFC5046 is that the Send Message type must be used instead of= RDMA Reads or Writes. The Solicited Event feature of the Send Message is = provided as a convenience. The receiver must still do the right thing in h= andling the Send Message regardless of whether the SE feature is used. In = other words, the sender is responsible for using the right format for the m= essage (Send vs RDMA) but the receiver must not rely on the sender to deter= mine how to handle the received message. The same rationale goes for the I= nvalidate feature. Mike ----- Original Message ----- From: david.black@emc.com To: Michael@huaweisymantec.com ; storm@i= etf.org Sent: Monday, January 16, 2012 2:06 PM Subject: iSER - one last issue Mike, Thanks for getting the -08 version of the iSER draft posted. I think that draft addresses all of the open issues, but I have a question for the WG about how to express the SendSE and SendInvSE requirements from RFC 5046. The -08 version of the iSER draft expresses requirements for the SendSE and SendInvSE messages (this is primarily for iSER over RDDP/iWARP) as: The SendSE Message should be used if supported by the RCaP layer (e.g., iWARP). My reading of RFC 5046 is that its requirements are tighter - to accurately reflect RFC 5046, I would replace "should" with "MUST" in the above text, at least for iWARP. In the alternative, if the "should"s remain, an explanatory item needs to be added to the Appendix A list of changes from RFC 5046. What do others think the right course of action is here, use "MUST" or explain weakening of requirement to "should" ? Thanks, --David --_000_2D98DD3F898B6B4DA287BF3BA07DAE93029056IRVEXCHMB11corpad_ Content-Type: text/html; charset=us-ascii Content-Transfer-Encoding: quoted-printable

I t= hink we should investigate [3c]. From the consistency standpoint, [3c] make= s iSER consistent across different RCaPs.

 

Can= someone from InfiniBand side comment?

 

Hem= al

 

From: storm-bo= unces@ietf.org [mailto:storm-bounces@ietf.org] On Behalf Of Tom Talpey
Sent: Tuesday, January 24, 2012 7:50 PM
To: david.black@emc.com; cbm@chadalapaka.com; storm@ietf.org
Subject: Re: [storm] iSER - one last issue: update=

 

To take [3a], making Send= SE mandatory for iWARP and mandatory-to-not for all others, would be very i= nconsistent and somewhat contradictory. For example, Infiniband currently runs at speeds approximately triple that of Ethernet, and up. Su= rely a performance-oriented tweak such as SendSE matters more for IB than i= WARP, if it matters at all. And, such a decision makes it more difficult to= design a common upper layer implementation.

 <= /p>

So, just to throw another= log on this fire, [3c], I suggest we ask the Infiniband community whether = they’re ok with SendSE/SendInvSE being a MUST for solicited replies exactly as specified in RFC5046. It means that for Infiniband, all= HCAs, but only those TCAs which support solicited events, can be used with= the protocol. And it means the sending side of the existing draft implemen= tations may have to change to send them where required.

 <= /p>

 <= /p>

From: storm-bo= unces@ietf.org [mailto:storm-bounces@ietf.org] On Behalf Of david.black@emc.com
Sent: Tuesday, January 24, 2012 7:42 PM
To: cbm@chadalapaka.com; storm@ietf.org
Subject: Re: [storm] iSER - one last issue: update=

 

[WG chair hat off]

 

Proposed slight refinement: If the SendSE and = SendInvSE requirements are applied to RCaPs that “require all impleme= ntations to support Solicited Events”, that set of RCaPs includes iWARP, but not InfiniBand.

 

That refinement creates a need to figure out t= he complementary requirement that applies to InfiniBand, which Tom more or = less asked about earlier in this thread:

 

> If using SendSE is so important here, whe= n MUST a plain-old Send be used?

 <= /p>

There are at least a couple of approaches to t= his one:

 

[3a] SendSE/SendInvSE MUST NOT be used with ot= her RCaPs.

 

[3b] Explain how a sender figures out whether = to use SendSE/SendInvSE or not (aka “negotiation”).<= /span>

 

[3a] would be the simplest.  The “r= unning code” matters here - does the IB implementation of iSER use Se= ndSE/SendInvSE?  If not, [3a] should be fine.

 

Thanks,
--David

 

From: Mallikar= jun Chadalapaka [mailto:cbm@chadalapaka.com= ]
Sent: Tuesday, January 24, 2012 7:25 PM
To: Tom Talpey; Black, David; stor= m@ietf.org
Subject: RE: [storm] iSER - one last issue: update=

 

I think we should go with= [1] and [2a] that David outlined.  As Hemal said, there were good rea= sons why RFC 5046 stipulated SendInvSE and SendSE, where it did.  I would rather not weaken it, at least for the iWARP RCaP ecosystem.<= /o:p>

 <= /p>

Mallikarjun

 <= /p>

 <= /p>

From: storm-bounces@ietf.org [mailto:storm-bounces@ietf.org] On Behalf Of Tom Talpey
Sent: Tuesday, January 24, 2012 5:49 AM
To: david.black@emc.com; = storm@ietf.org
Subject: Re: [storm] iSER - one last issue: update=

 

[WG co-chair hat off]

 <= /p>

> [2a] Leave things alone - if the RCaP sup= ports Solicited Events, then the “MUST”

> requirements for SendSE and SendInvSE app= ly as they did in RFC 5046.

 <= /p>

The core issue here is th= e statement “if the RCaP supports Solicited Events”. Both the i= WARP and Infiniband protocols support them. But Infiniband does not require all adapters support them – the architecture requires it for= HCAs but leaves it optional for TCAs (Infiniband architecture v1.2.1 table= 319 on page 1045). Because this latter property cannot be detected by the = remote upper layer, it does not appear to be possible to make an interoperable decision for all protocols and all= implementations.

 <= /p>

That said, in the absence= of any existing iSER-over-iWARP SendSE implementations, I agree the entire= discussion is effectively moot.

 <= /p>

Tom.

 <= /p>

From: storm-bounces@ietf.org [mailto:storm-bounces@ietf.org] On Behalf Of david.black@emc.com
Sent: Monday, January 23, 2012 8:05 PM
To: storm@ietf.org
Subject: [storm] iSER - one last issue: update

 

[WG chair hat off]

 

This is a message intended for discussion and = comments.

 

Looking for a way forward, this observation fr= om Mike seems like a useful place to start:

 

> This iSER upda= te is meant to reflect running code and to generalize the support of iSER o= ver any RCaP, not just iWARP. 

 

RFC 5046 (iSER) was written on the assumption = that the RCaP layer supports Solicited

Event functionality.  I also read RFC 504= 0 as requiring RDMAP implementations (part of

iWARP) to support Solicited Event functionalit= y.

 

This first step seems easy:<= /p>

 

[1] There are RCaPs that don’t support S= olicited Event functionality; those RCaPs have

no choice - they MUST use Send (not SendSE) an= d SendInv (not SendInvSE). 

       &nbs= p;            &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;         

I would prefer to write iSER requirements in t= erms of whether the RCaP supports

Solicited Event functionality as opposed to ca= lling out iWARP explicitly as having

different requirements.  OTOH, I’m = prepared to listen to arguments to the contrary.

 

Beyond that, I think there are two primary cho= ices:

 

[2a] Leave things alone - if the RCaP supports= Solicited Events, then the “MUST”

requirements for SendSE and SendInvSE apply as= they did in RFC 5046.

 

[2b] In the alternative, if there are iSER imp= lementations for iWARP that aren’t using

SendSE and SendInvSE in accordance with RFC 50= 46’s requirements, then we need to change

those requirements to match the implementation= s.

 

I would hope we can agree on [1], and I believ= e that’ll cover the InfiniBand case, right?

 

For choosing between [2a] and [2b] it would he= lp a lot to konw what iSER over iWARP

implementations are actually doing (SendSE/Sen= dInvSE vs. Send/SendInv).

 

*** Who is familiar with the iSER over iWARP i= mplementation(s) ??? ***

 

In general, what do folks think ought to be do= ne here - [2a], [2b] or something else?

 

[WG chair hat on]

 

We do need to resolve this issue and reflect t= hat resolution in a -09 iSER draft before

RFC publication can be requested.

 

Thanks,
--David

 

From: Tom Talp= ey [mailto:ttalpey@microsoft= .com]
Sent: Thursday, January 19, 2012 3:33 PM
To: Michael Ko; Hemal Shah; Black, David; storm@ietf.org
Subject: RE: iSER - one last issue

 

More precisely, are there= any iSER *initiators* that *depend* on receiving solicited e= vents over the iWARP RDMA transport? I believe this to be a very short list, if any. But I’d be ok with being proven wrong.

 <= /p>

 <= /p>

 <= /p>

From: Michael = Ko [mailto:Michael@huaw= eisymantec.com]
Sent: Thursday, January 19, 2012 12:39 PM
To: Hemal Shah; Tom Talpey; d= avid.black@emc.com; storm@ietf.org
Subject: Re: iSER - one last issue

 

For all RCaP layers= , the Send message (or any variant) MUST be used, as opposed to using RDMA = Read or Write to accomplish the task.  So a SHOULD for RCaP other= than iWARP is incorrect.  RFC 5040 can mandate that all RDMAP messages be supported, but it is still up to implementation= to decide what messages to use in any situation.  This iSER update is= meant to reflect running code and to generalize the support of iSER over a= ny RCaP, not just iWARP.  So the question is are there any running code that uses SendSE and SendInvSE for iSER.&nbs= p; Conversely, if the running code can get by using Send messages to accomp= lish the task, what is the point of mandating a MUST at this point for Send= SE and SendInvSE?  Note that the receiver cannot completely rely on the sender for the Invalidate feature anyway.&nb= sp; The iSER layer at the initiator is required to explicitly invalida= te the STag for bidirectional commands and abnormal completion of a command= .  Even when automatic invalidation is used, the iSER layer is required to sanity checking the automatic invalidation.&= nbsp; In other words, the iSER spec puts the burden on the receiv= er for doing the right thing in handling the Send messages, and not re= lying on the sender for the mere convenience. 

 

Mike

 

Tom= ,

 

You= are welcome! You are right about 7.3.4 as the only section mandating Send = message.

 

I t= hink your concern about specifying the requirement for the other RCaP imple= mentation is valid. I suggest we should word the requirements as below:

 

“If the RCaP layer = is as specified in [RFC5040] and [RFC5041], the SendSE message MUST be used= . For any other RCaP layer, the Send message SHOULD be used. ”

 <= /p>

Hem= al

 

From: Tom Talp= ey [mailto:ttalpey@microsoft= .com]
Sent: Thursday, January 19, 2012 8:09 AM
To: Hemal Shah; david.black@e= mc.com; Michael@huaweisymantec.com; storm@ietf.org
Subject: RE: iSER - one last issue

 

Thanks for pointing out t= hat sentence in RFC5046 7.3.4. I believe it is the only such stipulation.

 <= /p>

I’m ok with stating= the requirement as RFC5046-over-RFC5040 compliance. But this new draft ope= ns the door to alternative behaviors, therefore it needs to be stated more clearly. What’s the requirement when SendSE is not suppo= rted? And, how does the receiver know what to expect?

 <= /p>

In other words, if itR= 17;s proposed that the statement be:

    &= nbsp;           “Wh= en the RCaP layer is as specified in [RFC5040] and [RFC5041], the SendSE me= ssage MUST be used.”

 <= /p>

… then what MUST be= used when the RCaP is not? That statement needs to be made, too.

 <= /p>

 <= /p>

 

Tom= ,

 

RFC= 5046 is very clear about the use of Send Message Type for iSCSI control-typ= e PDU. For specific iSCSI PDUs, Section 7 mandates the use of Send, SendSE,= or SendSEInv. Section 7 of RFC5046 clearly specifies the use of specific Send Message Type for each iSCSI con= trol-type PDUs (Login, Logout, Text request, Text response…). Section= 7.3.4 mandates the use of plain-old Send also as needed. For example, see = text below from 7.3.4.

 

For unsolicited data, if the F bit is set to 0 in a SCSI D= ata-out

PDU, the iSER layer at the initiator MUST use a Send Messa= ge to send

the SCSI Data-out PDU.

 

Reg= arding your comment in the second paragraph below, RFC5040 expectation is a= ll the RDMAP messages specified in RFC5040 are supported (if that is not th= e case then we cannot count on any of the other RDMAP messages to be supported). So, an implementation that does= not support SendSE at the sender is not compliant with RFC5040. So, your s= econd point does not apply.

 

I h= ope that addresses your comments.

 

Hem= al

 

From: Tom Talp= ey [mailto:ttalpey@microsoft= .com]
Sent: Thursday, January 19, 2012 5:51 AM
To: Hemal Shah; david.black@e= mc.com; Michael@huaweisymantec.com; storm@ietf.org
Subject: RE: iSER - one last issue

 

If using SendSE is so imp= ortant here, when MUST a plain-old Send be used? There is no mention of any= distinction in RFC5046 section 1.4.2 nor in this draft’s section 2.4.2.

 <= /p>

Even with a MUST on the S= endSE, the behavior is indeterminate, since it would still be contingent on= support for SendSE at the *sender*. This critical exception voids any normative requirement, even over iWARP. The strongest = statement that can be made is SHOULD.

 <= /p>

Tom.

 <= /p>

From: storm-bounces@ietf.org [mailto:storm-bounces@ietf.org] On Behalf Of Hemal Shah
Sent: Wednesday, January 18, 2012 6:56 PM
To: david.black@emc.com; = Michael@huaweisymantec.com; storm@iet= f.org
Subject: Re: [storm] iSER - one last issue

 

Dav= id and Mike,

 

I b= elieve that the requirements should not be weakened and we should stick to = the requirement in RFC5046.

 

RFC= 5046 required the use of specific Send Message Type for valid reasons.=

 

In = the case of requiring SendSE message use, the intent was to give a hint to = the remote side to generate an event upon processing of SendSE message. For= example, the SendSE is specifically carrying SCSI information that needs attention on the remote side e.g. SCS= I command, Logout…

 

In = the case of requiring SendInvSE Message, the intent was to invalidate the S= Tag used in the data transfer as well as inform the remote side to generate= an event. For example, mandating the use of SendInvSE for the SCSI response PDU for a SCSI Read limits the expo= sure of STag used in SCSI Read data transfer and avoids the explicit invali= dation of the STag at the initiator. Plus, it allows the target to generate= an event on the initiator upon the completion of SCSI Read command.

 

By = relaxing RFC5046 requirements in the latest iSER draft, we will not only im= pact all iWARP based iSER implementations that rely on the use of specific = Send Message Type for SCSI data transfers but also change the intended behavior of initiators and targets.

 

For= iWARP, I strongly suggest that the iSER draft does not relax the requireme= nts specified in RFC5046.

 

I h= ope that helps.

 

Hem= al    

 

From: storm-bounces@ietf.org [mailto:storm-bounces@ietf.org] On Behalf Of david.black@emc.com
Sent: Wednesday, January 18, 2012 3:10 PM
To: Michael@huaweisyma= ntec.com; storm@ietf.org
Subject: Re: [storm] iSER - one last issue

 

Mike,

 

Thank you for the explanation, but I don’= ;t believe you’ve correctly stated

the intent of RFC 5046.  Here are some ex= amples from RFC 5046’s text:

 

1.5.  SCSI Read Overview

 

   The iSER Message is transferred to the target= using a SendSE Message.

 

   The iSER layer at the target uses a SendInvSE= Message to transfer the

   SCSI Response PDU back to the iSER layer at t= he initiator.

 

Similar language occurs in 1.6 SCSI Write Overview.

 

5.2.1.  Normal Connection Termination at the Initiato= r

 

   The iSER layer at the initiator MUST

   use a SendSE Message to send the Logout Reque= st PDU to the target.

 

Similar language occurs in 5.2.2 for target side normal co= nnection termination,

and more importantly in 7.3.1 SCSI Command:

 

   The iSER layer at the initiator MUST send the= SCSI command in a

   SendSE Message to the target.

 

None of the quoted text permits use of Send instead of Sen= dSE or SendInv

instead of SendInvSE.

 

What you propose to do effectively changes “MUSTR= 21; to “should” (lower case,

weaker than “SHOULD”), and that sure looks lik= e a change to RFC 5046.

 

Are there good implementation-based reasons to weaken thes= e requirements now?

 

Does anyone else have a viewpoint on this topi= c?

 

Thanks,
--David

 

From: Michael = Ko [mailto:Michael@huaw= eisymantec.com]
Sent: Wednesday, January 18, 2012 2:22 PM
To: Black, David; storm@ietf.org
Subject: Re: iSER - one last issue

 

David,<= /o:p>

 

Here is my rational= e for using the lower case "should".

 

The intent of RFC50= 46 is that the Send Message type must be used instead of RDMA Reads or Writ= es.  The Solicited Event feature of the Send Message is provided = as a convenience.  The receiver must still do the right thing in handling the Send Message regardless of whether the SE = feature is used.  In other words, the sender is responsible for using = the right format for the message (Send vs RDMA) but the receiver must = not rely on the sender to determine how to handle the received message.  The same rationale goes for the Invalid= ate feature. 

 

Mike

 

Mike,

 

Thanks for getting the -08 version of the iSER= draft posted.  I think that

draft addresses all of the open issues, but I = have a question for the WG

about how to express the SendSE and SendInvSE = requirements from RFC 5046.

 

The -08 version of the iSER draft expresses re= quirements for the SendSE

and SendInvSE messages (this is primarily for = iSER over RDDP/iWARP) as:

 

The SendSE Message = should be used if

supported by the RC= aP layer (e.g., iWARP).

 

My reading of RFC 5046 is that its requirement= s are tighter - to accurately

reflect RFC 5046, I would replace “shoul= d” with “MUST” in the above text,

at least for iWARP.

 

In the alternative, if the “should”= ;s remain, an explanatory item

needs to be added to the Appendix A list of ch= anges from RFC 5046.

 

What do others think the right course of actio= n is here, use “MUST” or

explain weakening of requirement to “sho= uld” ?

 

Thanks,
--David

--_000_2D98DD3F898B6B4DA287BF3BA07DAE93029056IRVEXCHMB11corpad_-- From hemal@broadcom.com Wed Jan 25 15:49:29 2012 Return-Path: X-Original-To: storm@ietfa.amsl.com Delivered-To: storm@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 8F2C411E8071 for ; Wed, 25 Jan 2012 15:49:29 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -6.598 X-Spam-Level: X-Spam-Status: No, score=-6.598 tagged_above=-999 required=5 tests=[AWL=-0.000, BAYES_00=-2.599, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_MED=-4] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id juq5djSq91uc for ; Wed, 25 Jan 2012 15:49:21 -0800 (PST) Received: from mms1.broadcom.com (mms1.broadcom.com [216.31.210.17]) by ietfa.amsl.com (Postfix) with ESMTP id BBE3111E8089 for ; Wed, 25 Jan 2012 15:49:20 -0800 (PST) Received: from [10.9.208.27] by mms1.broadcom.com with ESMTP (Broadcom SMTP Relay (Email Firewall v6.3.2)); Wed, 25 Jan 2012 15:52:01 -0800 X-Server-Uuid: 02CED230-5797-4B57-9875-D5D2FEE4708A Received: from IRVEXCHMB11.corp.ad.broadcom.com ( [fe80::9cdd:3a57:8694:f610]) by irvexchmb05.corp.ad.broadcom.com ( [::1]) with mapi id 14.01.0289.001; Wed, 25 Jan 2012 15:42:37 -0800 From: "Hemal Shah" To: "Mallikarjun Chadalapaka" , "Tom Talpey" , "david.black@emc.com" , "storm@ietf.org" Thread-Topic: [storm] iSER - one last issue: update Thread-Index: AQHM2jStRlyU+UWmekShwv3mXE7R65YcD2gAgACx74CAAP54IA== Date: Wed, 25 Jan 2012 23:42:36 +0000 Message-ID: <2D98DD3F898B6B4DA287BF3BA07DAE93029043@IRVEXCHMB11.corp.ad.broadcom.com> References: <7C4DFCE962635144B8FAE8CA11D0BF1E05A7D9137A@MX14A.corp.emc.com> In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [10.12.148.247] MIME-Version: 1.0 X-WSS-ID: 633E4A2A3DS19286279-01-01 Content-Type: multipart/alternative; boundary=_000_2D98DD3F898B6B4DA287BF3BA07DAE93029043IRVEXCHMB11corpad_ Subject: Re: [storm] iSER - one last issue: update X-BeenThere: storm@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Storage Maintenance WG List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 25 Jan 2012 23:49:29 -0000 --_000_2D98DD3F898B6B4DA287BF3BA07DAE93029043IRVEXCHMB11corpad_ Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable I'm also think we should go with [1] and [2a]. Hemal From: storm-bounces@ietf.org [mailto:storm-bounces@ietf.org] On Behalf Of M= allikarjun Chadalapaka Sent: Tuesday, January 24, 2012 4:25 PM To: Tom Talpey; david.black@emc.com; storm@ietf.org Subject: Re: [storm] iSER - one last issue: update I think we should go with [1] and [2a] that David outlined. As Hemal said,= there were good reasons why RFC 5046 stipulated SendInvSE and SendSE, wher= e it did. I would rather not weaken it, at least for the iWARP RCaP ecosys= tem. Mallikarjun From: storm-bounces@ietf.org [mailto:storm-bounces@ietf.org] On Behalf Of T= om Talpey Sent: Tuesday, January 24, 2012 5:49 AM To: david.black@emc.com; storm@ietf.org Subject: Re: [storm] iSER - one last issue: update [WG co-chair hat off] > [2a] Leave things alone - if the RCaP supports Solicited Events, then the= "MUST" > requirements for SendSE and SendInvSE apply as they did in RFC 5046. The core issue here is the statement "if the RCaP supports Solicited Events= ". Both the iWARP and Infiniband protocols support them. But Infiniband doe= s not require all adapters support them - the architecture requires it for = HCAs but leaves it optional for TCAs (Infiniband architecture v1.2.1 table = 319 on page 1045). Because this latter property cannot be detected by the r= emote upper layer, it does not appear to be possible to make an interoperab= le decision for all protocols and all implementations. That said, in the absence of any existing iSER-over-iWARP SendSE implementa= tions, I agree the entire discussion is effectively moot. Tom. From: storm-bounces@ietf.org [mailto:storm-b= ounces@ietf.org] On Behalf Of david= .black@emc.com Sent: Monday, January 23, 2012 8:05 PM To: storm@ietf.org Subject: [storm] iSER - one last issue: update [WG chair hat off] This is a message intended for discussion and comments. Looking for a way forward, this observation from Mike seems like a useful p= lace to start: > This iSER update is meant to reflect running code and to generalize the s= upport of iSER over any RCaP, not just iWARP. RFC 5046 (iSER) was written on the assumption that the RCaP layer supports = Solicited Event functionality. I also read RFC 5040 as requiring RDMAP implementatio= ns (part of iWARP) to support Solicited Event functionality. This first step seems easy: [1] There are RCaPs that don't support Solicited Event functionality; those= RCaPs have no choice - they MUST use Send (not SendSE) and SendInv (not SendInvSE). I would prefer to write iSER requirements in terms of whether the RCaP supp= orts Solicited Event functionality as opposed to calling out iWARP explicitly as= having different requirements. OTOH, I'm prepared to listen to arguments to the c= ontrary. Beyond that, I think there are two primary choices: [2a] Leave things alone - if the RCaP supports Solicited Events, then the "= MUST" requirements for SendSE and SendInvSE apply as they did in RFC 5046. [2b] In the alternative, if there are iSER implementations for iWARP that a= ren't using SendSE and SendInvSE in accordance with RFC 5046's requirements, then we ne= ed to change those requirements to match the implementations. I would hope we can agree on [1], and I believe that'll cover the InfiniBan= d case, right? For choosing between [2a] and [2b] it would help a lot to konw what iSER ov= er iWARP implementations are actually doing (SendSE/SendInvSE vs. Send/SendInv). *** Who is familiar with the iSER over iWARP implementation(s) ??? *** In general, what do folks think ought to be done here - [2a], [2b] or somet= hing else? [WG chair hat on] We do need to resolve this issue and reflect that resolution in a -09 iSER = draft before RFC publication can be requested. Thanks, --David From: Tom Talpey [mailto:ttalpey@microsoft.com] Sent: Thursday, January 19, 2012 3:33 PM To: Michael Ko; Hemal Shah; Black, David; storm@ietf.org Subject: RE: iSER - one last issue More precisely, are there any iSER *initiators* that *depend* on receiving = solicited events over the iWARP RDMA transport? I believe this to be a very= short list, if any. But I'd be ok with being proven wrong. From: Michael Ko [mailto:Michael@huaweisymantec.com] Sent: Thursday, January 19, 2012 12:39 PM To: Hemal Shah; Tom Talpey; david.black@emc.com= ; storm@ietf.org Subject: Re: iSER - one last issue For all RCaP layers, the Send message (or any variant) MUST be used, as opp= osed to using RDMA Read or Write to accomplish the task. So a SHOULD for R= CaP other than iWARP is incorrect. RFC 5040 can mandate that all RDMAP mes= sages be supported, but it is still up to implementation to decide what mes= sages to use in any situation. This iSER update is meant to reflect runnin= g code and to generalize the support of iSER over any RCaP, not just iWARP.= So the question is are there any running code that uses SendSE and SendIn= vSE for iSER. Conversely, if the running code can get by using Send messag= es to accomplish the task, what is the point of mandating a MUST at this po= int for SendSE and SendInvSE? Note that the receiver cannot completely rel= y on the sender for the Invalidate feature anyway. The iSER layer at the i= nitiator is required to explicitly invalidate the STag for bidirectional co= mmands and abnormal completion of a command. Even when automatic invalidat= ion is used, the iSER layer is required to sanity checking the automatic in= validation. In other words, the iSER spec puts the burden on the receiver = for doing the right thing in handling the Send messages, and not relying on= the sender for the mere convenience. Mike ----- Original Message ----- From: Hemal Shah To: Tom Talpey ; david.black@emc.com ; Michael@huaweisymantec.com ; storm@ietf.org Sent: Thursday, January 19, 2012 8:44 AM Subject: RE: iSER - one last issue Tom, You are welcome! You are right about 7.3.4 as the only section mandating Se= nd message. I think your concern about specifying the requirement for the other RCaP im= plementation is valid. I suggest we should word the requirements as below: "If the RCaP layer is as specified in [RFC5040] and [RFC5041], the SendSE m= essage MUST be used. For any other RCaP layer, the Send message SHOULD be u= sed. " Hemal From: Tom Talpey [mailto:ttalpey@microsoft.com] Sent: Thursday, January 19, 2012 8:09 AM To: Hemal Shah; david.black@emc.com; Michael@hu= aweisymantec.com; storm@ietf.org Subject: RE: iSER - one last issue Thanks for pointing out that sentence in RFC5046 7.3.4. I believe it is the= only such stipulation. I'm ok with stating the requirement as RFC5046-over-RFC5040 compliance. But= this new draft opens the door to alternative behaviors, therefore it needs= to be stated more clearly. What's the requirement when SendSE is not suppo= rted? And, how does the receiver know what to expect? In other words, if it's proposed that the statement be: "When the RCaP layer is as specified in [RFC5040] and [RFC5= 041], the SendSE message MUST be used." ... then what MUST be used when the RCaP is not? That statement needs to be= made, too. From: Hemal Shah [mailto:hemal@broadcom.com] Sent: Thursday, January 19, 2012 10:38 AM To: Tom Talpey; david.black@emc.com; Michael@hu= aweisymantec.com; storm@ietf.org Subject: RE: iSER - one last issue Tom, RFC5046 is very clear about the use of Send Message Type for iSCSI control-= type PDU. For specific iSCSI PDUs, Section 7 mandates the use of Send, Send= SE, or SendSEInv. Section 7 of RFC5046 clearly specifies the use of specifi= c Send Message Type for each iSCSI control-type PDUs (Login, Logout, Text r= equest, Text response...). Section 7.3.4 mandates the use of plain-old Send= also as needed. For example, see text below from 7.3.4. For unsolicited data, if the F bit is set to 0 in a SCSI Data-out PDU, the iSER layer at the initiator MUST use a Send Message to send the SCSI Data-out PDU. Regarding your comment in the second paragraph below, RFC5040 expectation i= s all the RDMAP messages specified in RFC5040 are supported (if that is not= the case then we cannot count on any of the other RDMAP messages to be sup= ported). So, an implementation that does not support SendSE at the sender i= s not compliant with RFC5040. So, your second point does not apply. I hope that addresses your comments. Hemal From: Tom Talpey [mailto:ttalpey@microsoft.com] Sent: Thursday, January 19, 2012 5:51 AM To: Hemal Shah; david.black@emc.com; Michael@hu= aweisymantec.com; storm@ietf.org Subject: RE: iSER - one last issue If using SendSE is so important here, when MUST a plain-old Send be used? T= here is no mention of any distinction in RFC5046 section 1.4.2 nor in this = draft's section 2.4.2. Even with a MUST on the SendSE, the behavior is indeterminate, since it wou= ld still be contingent on support for SendSE at the *sender*. This critical= exception voids any normative requirement, even over iWARP. The strongest = statement that can be made is SHOULD. Tom. From: storm-bounces@ietf.org [mailto:storm-b= ounces@ietf.org] On Behalf Of Hemal= Shah Sent: Wednesday, January 18, 2012 6:56 PM To: david.black@emc.com; Michael@huaweisymantec= .com; storm@ietf.org Subject: Re: [storm] iSER - one last issue David and Mike, I believe that the requirements should not be weakened and we should stick = to the requirement in RFC5046. RFC5046 required the use of specific Send Message Type for valid reasons. In the case of requiring SendSE message use, the intent was to give a hint = to the remote side to generate an event upon processing of SendSE message. = For example, the SendSE is specifically carrying SCSI information that need= s attention on the remote side e.g. SCSI command, Logout... In the case of requiring SendInvSE Message, the intent was to invalidate th= e STag used in the data transfer as well as inform the remote side to gener= ate an event. For example, mandating the use of SendInvSE for the SCSI resp= onse PDU for a SCSI Read limits the exposure of STag used in SCSI Read data= transfer and avoids the explicit invalidation of the STag at the initiator= . Plus, it allows the target to generate an event on the initiator upon the= completion of SCSI Read command. By relaxing RFC5046 requirements in the latest iSER draft, we will not only= impact all iWARP based iSER implementations that rely on the use of specif= ic Send Message Type for SCSI data transfers but also change the intended b= ehavior of initiators and targets. For iWARP, I strongly suggest that the iSER draft does not relax the requir= ements specified in RFC5046. I hope that helps. Hemal From: storm-bounces@ietf.org [mailto:storm-b= ounces@ietf.org] On Behalf Of david= .black@emc.com Sent: Wednesday, January 18, 2012 3:10 PM To: Michael@huaweisymantec.com; storm@ie= tf.org Subject: Re: [storm] iSER - one last issue Mike, Thank you for the explanation, but I don't believe you've correctly stated the intent of RFC 5046. Here are some examples from RFC 5046's text: 1.5. SCSI Read Overview The iSER Message is transferred to the target using a SendSE Message. The iSER layer at the target uses a SendInvSE Message to transfer the SCSI Response PDU back to the iSER layer at the initiator. Similar language occurs in 1.6 SCSI Write Overview. 5.2.1. Normal Connection Termination at the Initiator The iSER layer at the initiator MUST use a SendSE Message to send the Logout Request PDU to the target. Similar language occurs in 5.2.2 for target side normal connection terminat= ion, and more importantly in 7.3.1 SCSI Command: The iSER layer at the initiator MUST send the SCSI command in a SendSE Message to the target. None of the quoted text permits use of Send instead of SendSE or SendInv instead of SendInvSE. What you propose to do effectively changes "MUST" to "should" (lower case, weaker than "SHOULD"), and that sure looks like a change to RFC 5046. Are there good implementation-based reasons to weaken these requirements no= w? Does anyone else have a viewpoint on this topic? Thanks, --David From: Michael Ko [mailto:Michael@huaweisymantec.com] Sent: Wednesday, January 18, 2012 2:22 PM To: Black, David; storm@ietf.org Subject: Re: iSER - one last issue David, Here is my rationale for using the lower case "should". The intent of RFC5046 is that the Send Message type must be used instead of= RDMA Reads or Writes. The Solicited Event feature of the Send Message is = provided as a convenience. The receiver must still do the right thing in h= andling the Send Message regardless of whether the SE feature is used. In = other words, the sender is responsible for using the right format for the m= essage (Send vs RDMA) but the receiver must not rely on the sender to deter= mine how to handle the received message. The same rationale goes for the I= nvalidate feature. Mike ----- Original Message ----- From: david.black@emc.com To: Michael@huaweisymantec.com ; storm@i= etf.org Sent: Monday, January 16, 2012 2:06 PM Subject: iSER - one last issue Mike, Thanks for getting the -08 version of the iSER draft posted. I think that draft addresses all of the open issues, but I have a question for the WG about how to express the SendSE and SendInvSE requirements from RFC 5046. The -08 version of the iSER draft expresses requirements for the SendSE and SendInvSE messages (this is primarily for iSER over RDDP/iWARP) as: The SendSE Message should be used if supported by the RCaP layer (e.g., iWARP). My reading of RFC 5046 is that its requirements are tighter - to accurately reflect RFC 5046, I would replace "should" with "MUST" in the above text, at least for iWARP. In the alternative, if the "should"s remain, an explanatory item needs to be added to the Appendix A list of changes from RFC 5046. What do others think the right course of action is here, use "MUST" or explain weakening of requirement to "should" ? Thanks, --David --_000_2D98DD3F898B6B4DA287BF3BA07DAE93029043IRVEXCHMB11corpad_ Content-Type: text/html; charset=us-ascii Content-Transfer-Encoding: quoted-printable

I&#= 8217;m also think we should go with [1] and [2a].

 

Hem= al

 

From: storm-bo= unces@ietf.org [mailto:storm-bounces@ietf.org] On Behalf Of Mallikarjun Chadalapaka
Sent: Tuesday, January 24, 2012 4:25 PM
To: Tom Talpey; david.black@emc.com; storm@ietf.org
Subject: Re: [storm] iSER - one last issue: update=

 

I think we should go with= [1] and [2a] that David outlined.  As Hemal said, there were good rea= sons why RFC 5046 stipulated SendInvSE and SendSE, where it did.  I would rather not weaken it, at least for the iWARP RCaP ecosystem.<= /o:p>

 <= /p>

Mallikarjun

 <= /p>

 <= /p>

From: storm-bo= unces@ietf.org [mailto:storm-bounces@ietf.org] On Behalf Of Tom Talpey
Sent: Tuesday, January 24, 2012 5:49 AM
To: david.black@emc.com; storm@ietf.org
Subject: Re: [storm] iSER - one last issue: update=

 

[WG co-chair hat off]

 <= /p>

> [2a] Leave things alone - if the RCaP sup= ports Solicited Events, then the “MUST”

> requirements for SendSE and SendInvSE app= ly as they did in RFC 5046.

 <= /p>

The core issue here is th= e statement “if the RCaP supports Solicited Events”. Both the i= WARP and Infiniband protocols support them. But Infiniband does not require all adapters support them – the architecture requires it for= HCAs but leaves it optional for TCAs (Infiniband architecture v1.2.1 table= 319 on page 1045). Because this latter property cannot be detected by the = remote upper layer, it does not appear to be possible to make an interoperable decision for all protocols and all= implementations.

 <= /p>

That said, in the absence= of any existing iSER-over-iWARP SendSE implementations, I agree the entire= discussion is effectively moot.

 <= /p>

Tom.

 <= /p>

From: storm-bounces@ietf.org [mailto:storm-bounces@ietf.org] On Behalf Of david.black@emc.com
Sent: Monday, January 23, 2012 8:05 PM
To: storm@ietf.org
Subject: [storm] iSER - one last issue: update

 

[WG chair hat off]

 

This is a message intended for discussion and = comments.

 

Looking for a way forward, this observation fr= om Mike seems like a useful place to start:

 

> This iSER upda= te is meant to reflect running code and to generalize the support of iSER o= ver any RCaP, not just iWARP. 

 

RFC 5046 (iSER) was written on the assumption = that the RCaP layer supports Solicited

Event functionality.  I also read RFC 504= 0 as requiring RDMAP implementations (part of

iWARP) to support Solicited Event functionalit= y.

 

This first step seems easy:<= /p>

 

[1] There are RCaPs that don’t support S= olicited Event functionality; those RCaPs have

no choice - they MUST use Send (not SendSE) an= d SendInv (not SendInvSE). 

       &nbs= p;            &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;         

I would prefer to write iSER requirements in t= erms of whether the RCaP supports

Solicited Event functionality as opposed to ca= lling out iWARP explicitly as having

different requirements.  OTOH, I’m = prepared to listen to arguments to the contrary.

 

Beyond that, I think there are two primary cho= ices:

 

[2a] Leave things alone - if the RCaP supports= Solicited Events, then the “MUST”

requirements for SendSE and SendInvSE apply as= they did in RFC 5046.

 

[2b] In the alternative, if there are iSER imp= lementations for iWARP that aren’t using

SendSE and SendInvSE in accordance with RFC 50= 46’s requirements, then we need to change

those requirements to match the implementation= s.

 

I would hope we can agree on [1], and I believ= e that’ll cover the InfiniBand case, right?

 

For choosing between [2a] and [2b] it would he= lp a lot to konw what iSER over iWARP

implementations are actually doing (SendSE/Sen= dInvSE vs. Send/SendInv).

 

*** Who is familiar with the iSER over iWARP i= mplementation(s) ??? ***

 

In general, what do folks think ought to be do= ne here - [2a], [2b] or something else?

 

[WG chair hat on]

 

We do need to resolve this issue and reflect t= hat resolution in a -09 iSER draft before

RFC publication can be requested.

 

Thanks,
--David

 

From: Tom Talp= ey [mailto:ttalpey@microsoft= .com]
Sent: Thursday, January 19, 2012 3:33 PM
To: Michael Ko; Hemal Shah; Black, David; storm@ietf.org
Subject: RE: iSER - one last issue

 

More precisely, are there= any iSER *initiators* that *depend* on receiving solicited e= vents over the iWARP RDMA transport? I believe this to be a very short list, if any. But I’d be ok with being proven wrong.

 <= /p>

 <= /p>

 <= /p>

From: Michael = Ko [mailto:Michael@huaw= eisymantec.com]
Sent: Thursday, January 19, 2012 12:39 PM
To: Hemal Shah; Tom Talpey; d= avid.black@emc.com; storm@ietf.org
Subject: Re: iSER - one last issue

 

For all RCaP layers= , the Send message (or any variant) MUST be used, as opposed to using RDMA = Read or Write to accomplish the task.  So a SHOULD for RCaP other= than iWARP is incorrect.  RFC 5040 can mandate that all RDMAP messages be supported, but it is still up to implementation= to decide what messages to use in any situation.  This iSER update is= meant to reflect running code and to generalize the support of iSER over a= ny RCaP, not just iWARP.  So the question is are there any running code that uses SendSE and SendInvSE for iSER.&nbs= p; Conversely, if the running code can get by using Send messages to accomp= lish the task, what is the point of mandating a MUST at this point for Send= SE and SendInvSE?  Note that the receiver cannot completely rely on the sender for the Invalidate feature anyway.&nb= sp; The iSER layer at the initiator is required to explicitly invalida= te the STag for bidirectional commands and abnormal completion of a command= .  Even when automatic invalidation is used, the iSER layer is required to sanity checking the automatic invalidation.&= nbsp; In other words, the iSER spec puts the burden on the receiv= er for doing the right thing in handling the Send messages, and not re= lying on the sender for the mere convenience. 

 

Mike

 

Tom= ,

 

You= are welcome! You are right about 7.3.4 as the only section mandating Send = message.

 

I t= hink your concern about specifying the requirement for the other RCaP imple= mentation is valid. I suggest we should word the requirements as below:

 

“If the RCaP layer = is as specified in [RFC5040] and [RFC5041], the SendSE message MUST be used= . For any other RCaP layer, the Send message SHOULD be used. ”

 <= /p>

Hem= al

 

From: Tom Talp= ey [mailto:ttalpey@microsoft= .com]
Sent: Thursday, January 19, 2012 8:09 AM
To: Hemal Shah; david.black@e= mc.com; Michael@huaweisymantec.com; storm@ietf.org
Subject: RE: iSER - one last issue

 

Thanks for pointing out t= hat sentence in RFC5046 7.3.4. I believe it is the only such stipulation.

 <= /p>

I’m ok with stating= the requirement as RFC5046-over-RFC5040 compliance. But this new draft ope= ns the door to alternative behaviors, therefore it needs to be stated more clearly. What’s the requirement when SendSE is not suppo= rted? And, how does the receiver know what to expect?

 <= /p>

In other words, if itR= 17;s proposed that the statement be:

    &= nbsp;           “Wh= en the RCaP layer is as specified in [RFC5040] and [RFC5041], the SendSE me= ssage MUST be used.”

 <= /p>

… then what MUST be= used when the RCaP is not? That statement needs to be made, too.

 <= /p>

 <= /p>

 

Tom= ,

 

RFC= 5046 is very clear about the use of Send Message Type for iSCSI control-typ= e PDU. For specific iSCSI PDUs, Section 7 mandates the use of Send, SendSE,= or SendSEInv. Section 7 of RFC5046 clearly specifies the use of specific Send Message Type for each iSCSI con= trol-type PDUs (Login, Logout, Text request, Text response…). Section= 7.3.4 mandates the use of plain-old Send also as needed. For example, see = text below from 7.3.4.

 

For unsolicited data, if the F bit is set to 0 in a SCSI D= ata-out

PDU, the iSER layer at the initiator MUST use a Send Messa= ge to send

the SCSI Data-out PDU.

 

Reg= arding your comment in the second paragraph below, RFC5040 expectation is a= ll the RDMAP messages specified in RFC5040 are supported (if that is not th= e case then we cannot count on any of the other RDMAP messages to be supported). So, an implementation that does= not support SendSE at the sender is not compliant with RFC5040. So, your s= econd point does not apply.

 

I h= ope that addresses your comments.

 

Hem= al

 

From: Tom Talp= ey [mailto:ttalpey@microsoft= .com]
Sent: Thursday, January 19, 2012 5:51 AM
To: Hemal Shah; david.black@e= mc.com; Michael@huaweisymantec.com; storm@ietf.org
Subject: RE: iSER - one last issue

 

If using SendSE is so imp= ortant here, when MUST a plain-old Send be used? There is no mention of any= distinction in RFC5046 section 1.4.2 nor in this draft’s section 2.4.2.

 <= /p>

Even with a MUST on the S= endSE, the behavior is indeterminate, since it would still be contingent on= support for SendSE at the *sender*. This critical exception voids any normative requirement, even over iWARP. The strongest = statement that can be made is SHOULD.

 <= /p>

Tom.

 <= /p>

From: storm-bounces@ietf.org [mailto:storm-bounces@ietf.org] On Behalf Of Hemal Shah
Sent: Wednesday, January 18, 2012 6:56 PM
To: david.black@emc.com; = Michael@huaweisymantec.com; storm@iet= f.org
Subject: Re: [storm] iSER - one last issue

 

Dav= id and Mike,

 

I b= elieve that the requirements should not be weakened and we should stick to = the requirement in RFC5046.

 

RFC= 5046 required the use of specific Send Message Type for valid reasons.=

 

In = the case of requiring SendSE message use, the intent was to give a hint to = the remote side to generate an event upon processing of SendSE message. For= example, the SendSE is specifically carrying SCSI information that needs attention on the remote side e.g. SCS= I command, Logout…

 

In = the case of requiring SendInvSE Message, the intent was to invalidate the S= Tag used in the data transfer as well as inform the remote side to generate= an event. For example, mandating the use of SendInvSE for the SCSI response PDU for a SCSI Read limits the expo= sure of STag used in SCSI Read data transfer and avoids the explicit invali= dation of the STag at the initiator. Plus, it allows the target to generate= an event on the initiator upon the completion of SCSI Read command.

 

By = relaxing RFC5046 requirements in the latest iSER draft, we will not only im= pact all iWARP based iSER implementations that rely on the use of specific = Send Message Type for SCSI data transfers but also change the intended behavior of initiators and targets.

 

For= iWARP, I strongly suggest that the iSER draft does not relax the requireme= nts specified in RFC5046.

 

I h= ope that helps.

 

Hem= al    

 

From: storm-bounces@ietf.org [mailto:storm-bounces@ietf.org] On Behalf Of david.black@emc.com
Sent: Wednesday, January 18, 2012 3:10 PM
To: Michael@huaweisyma= ntec.com; storm@ietf.org
Subject: Re: [storm] iSER - one last issue

 

Mike,

 

Thank you for the explanation, but I don’= ;t believe you’ve correctly stated

the intent of RFC 5046.  Here are some ex= amples from RFC 5046’s text:

 

1.5.  SCSI Read Overview

 

   The iSER Message is transferred to the target= using a SendSE Message.

 

   The iSER layer at the target uses a SendInvSE= Message to transfer the

   SCSI Response PDU back to the iSER layer at t= he initiator.

 

Similar language occurs in 1.6 SCSI Write Overview.

 

5.2.1.  Normal Connection Termination at the Initiato= r

 

   The iSER layer at the initiator MUST

   use a SendSE Message to send the Logout Reque= st PDU to the target.

 

Similar language occurs in 5.2.2 for target side normal co= nnection termination,

and more importantly in 7.3.1 SCSI Command:

 

   The iSER layer at the initiator MUST send the= SCSI command in a

   SendSE Message to the target.

 

None of the quoted text permits use of Send instead of Sen= dSE or SendInv

instead of SendInvSE.

 

What you propose to do effectively changes “MUSTR= 21; to “should” (lower case,

weaker than “SHOULD”), and that sure looks lik= e a change to RFC 5046.

 

Are there good implementation-based reasons to weaken thes= e requirements now?

 

Does anyone else have a viewpoint on this topi= c?

 

Thanks,
--David

 

From: Michael = Ko [mailto:Michael@huaw= eisymantec.com]
Sent: Wednesday, January 18, 2012 2:22 PM
To: Black, David; storm@ietf.org
Subject: Re: iSER - one last issue

 

David,<= /o:p>

 

Here is my rational= e for using the lower case "should".

 

The intent of RFC50= 46 is that the Send Message type must be used instead of RDMA Reads or Writ= es.  The Solicited Event feature of the Send Message is provided = as a convenience.  The receiver must still do the right thing in handling the Send Message regardless of whether the SE = feature is used.  In other words, the sender is responsible for using = the right format for the message (Send vs RDMA) but the receiver must = not rely on the sender to determine how to handle the received message.  The same rationale goes for the Invalid= ate feature. 

 

Mike

 

Mike,

 

Thanks for getting the -08 version of the iSER= draft posted.  I think that

draft addresses all of the open issues, but I = have a question for the WG

about how to express the SendSE and SendInvSE = requirements from RFC 5046.

 

The -08 version of the iSER draft expresses re= quirements for the SendSE

and SendInvSE messages (this is primarily for = iSER over RDDP/iWARP) as:

 

The SendSE Message = should be used if

supported by the RC= aP layer (e.g., iWARP).

 

My reading of RFC 5046 is that its requirement= s are tighter - to accurately

reflect RFC 5046, I would replace “shoul= d” with “MUST” in the above text,

at least for iWARP.

 

In the alternative, if the “should”= ;s remain, an explanatory item

needs to be added to the Appendix A list of ch= anges from RFC 5046.

 

What do others think the right course of actio= n is here, use “MUST” or

explain weakening of requirement to “sho= uld” ?

 

Thanks,
--David

--_000_2D98DD3F898B6B4DA287BF3BA07DAE93029043IRVEXCHMB11corpad_--