From owner-ips@ece.cmu.edu Tue May 1 13:03:56 2001 Received: from ece.cmu.edu (ECE.CMU.EDU [128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id NAA04098 for ; Tue, 1 May 2001 13:03:55 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f41EBA316603 for ips-outgoing; Tue, 1 May 2001 10:11:10 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from d12lmsgate.de.ibm.com (d12lmsgate.de.ibm.com [195.212.91.199]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f41EB8H16599 for ; Tue, 1 May 2001 10:11:08 -0400 (EDT) Received: from d12relay01.de.ibm.com (d12relay01.de.ibm.com [9.165.215.22]) by d12lmsgate.de.ibm.com (1.0.0) with ESMTP id QAA142462 for ; Tue, 1 May 2001 16:10:59 +0200 From: julian_satran@il.ibm.com Received: from d12mta02.de.ibm.com (d12mta01_cs0 [9.165.222.237]) by d12relay01.de.ibm.com (8.8.8m3/NCO v4.96) with SMTP id QAA178866 for ; Tue, 1 May 2001 16:11:00 +0200 Received: by d12mta02.de.ibm.com(Lotus SMTP MTA v4.6.5 (863.2 5-20-1999)) id C1256A3F.004DD33C ; Tue, 1 May 2001 16:10:03 +0200 X-Lotus-FromDomain: IBMIL@IBMDE To: ips@ece.cmu.edu Message-ID: Date: Tue, 1 May 2001 16:50:16 +0300 Subject: Another shot at codes and please comment Mime-Version: 1.0 Content-type: text/plain; charset=us-ascii Content-Disposition: inline Sender: owner-ips@ece.cmu.edu Precedence: bulk Here is a another version of the opcodes part: 1.1.1.1 Opcode The Opcode indicates what type of iSCSI PDU the header encapsulates. The Opcodes are divided into two categories: initiator opcodes and target opcodes. Initiator opcodes are in PDUs sent by the initiators (request PDUs), and target opcodes are in PDUs sent by the target (response PDUs). Initiators MUST NOT use target opcodes and targets MUST NOT use initiator opcodes. Valid initiator opcodes defined in this specification are: 0x00 NOP-Out (from initiator to target) 0x01 SCSI Command (encapsulates a SCSI Command Descriptor Block) 0x02 SCSI Task Management Command 0x03 Login Command 0x04 Text Command 0x05 SCSI Data-out (for WRITE operations) 0x06 Logout Command 0x10 SNACK Request Valid target opcodes are: 0x20 NOP-In (from target to initiator) 0x21 SCSI Response (contains SCSI status and possibly sense information or other response information) 0x22 SCSI Task Management Response 0x23 Login Response 0x24 Text Response 0x25 SCSI Data-in (for READ operations) 0x26 Logout Response 0x31 Ready To Transfer (R2T - sent by target to initiator when it is ready to receive data from initiator) 0x32 Asynchronous Message (sent by target to initiator to indicate certain special conditions) 0x3f Reject Initiator opcodes 0x1c-0x1e and target opcodes 0x3c-0x3e are vendor specific codes. Please comment, Julo From owner-ips@ece.cmu.edu Tue May 1 17:43:38 2001 Received: from ece.cmu.edu (ECE.CMU.EDU [128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id RAA10925 for ; Tue, 1 May 2001 17:43:38 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f41J81Y07141 for ips-outgoing; Tue, 1 May 2001 15:08:01 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from dirty.research.bell-labs.com (dirty.research.bell-labs.com [204.178.16.6]) by ece.cmu.edu (8.11.0/8.10.2) with SMTP id f41J7xj07137 for ; Tue, 1 May 2001 15:07:59 -0400 (EDT) Received: from grubby.research.bell-labs.com ([135.104.2.9]) by dirty; Tue May 1 15:07:14 EDT 2001 Received: from aura.research.bell-labs.com ([135.104.46.10]) by grubby; Tue May 1 15:07:11 EDT 2001 Received: (from sandeepj@localhost) by aura.research.bell-labs.com (8.9.1/8.9.1) id PAA02266; Tue, 1 May 2001 15:07:06 -0400 (EDT) Date: Tue, 1 May 2001 15:07:06 -0400 (EDT) Message-Id: <200105011907.PAA02266@aura.research.bell-labs.com> From: sandeepj@research.bell-labs.com (Sandeep Joshi) To: julian_satran@il.ibm.com Cc: ips@ece.cmu.edu Subject: Another shot at codes and please comment Sender: owner-ips@ece.cmu.edu Precedence: bulk Julian, I'd prefer this over the alternating opcode assignments. (where nop-out was 0 and nop-in was 1) Could you also define the behaviour if a vendor-specific opcode was not recognized ? I presume we always want to drop the session, since these opcodes would only be used if the target and initiator recognized/negotiated (in a vendor-specific way) their use during session initiation. -Sandeep > Here is a another version of the opcodes part: > > 1.1.1.1 Opcode > > The Opcode indicates what type of iSCSI PDU the header encapsulates. > > The Opcodes are divided into two categories: initiator opcodes and > target opcodes. Initiator opcodes are in PDUs sent by the initiators > (request PDUs), and target opcodes are in PDUs sent by the target > (response PDUs). > > Initiators MUST NOT use target opcodes and targets MUST NOT use > initiator opcodes. > > Valid initiator opcodes defined in this specification are: > > > 0x00 NOP-Out (from initiator to target) > 0x01 SCSI Command (encapsulates a SCSI Command Descriptor Block) > 0x02 SCSI Task Management Command > 0x03 Login Command > 0x04 Text Command > 0x05 SCSI Data-out (for WRITE operations) > 0x06 Logout Command > 0x10 SNACK Request > > Valid target opcodes are: > > > 0x20 NOP-In (from target to initiator) > 0x21 SCSI Response (contains SCSI status and possibly sense > information or other response information) > 0x22 SCSI Task Management Response > 0x23 Login Response > 0x24 Text Response > 0x25 SCSI Data-in (for READ operations) > 0x26 Logout Response > 0x31 Ready To Transfer (R2T - sent by target to initiator when it is > ready to receive data from initiator) > 0x32 Asynchronous Message (sent by target to initiator to indicate > certain special conditions) > 0x3f Reject > > Initiator opcodes 0x1c-0x1e and target opcodes 0x3c-0x3e are vendor > specific codes. > > > Please comment, > Julo > From owner-ips@ece.cmu.edu Tue May 1 20:03:14 2001 Received: from ece.cmu.edu (ECE.CMU.EDU [128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id UAA12701 for ; Tue, 1 May 2001 20:03:14 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f41LjCr17712 for ips-outgoing; Tue, 1 May 2001 17:45:13 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from msgbas1.cos.agilent.com (msgbas1x.cos.agilent.com [192.6.9.33]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f41LjBj17707 for ; Tue, 1 May 2001 17:45:11 -0400 (EDT) Received: from msgrel1.and.agilent.com (msgrel1.and.agilent.com [130.30.33.104]) by msgbas1.cos.agilent.com (Postfix) with ESMTP id 6F60A4C for ; Tue, 1 May 2001 15:45:10 -0600 (MDT) Received: from rtl.rose.agilent.com (rtl.rose.agilent.com [156.140.232.231]) by msgrel1.and.agilent.com (Postfix) with ESMTP id 38097DB for ; Tue, 1 May 2001 17:45:09 -0400 (EDT) Received: from mail.rose.agilent.com (mailsrv@bellhop [156.140.233.51]) by rtl.rose.agilent.com (8.9.3 (PHNE_18979)/8.9.3 SMKit7.1.0) with ESMTP id OAA16514 for ; Tue, 1 May 2001 14:45:07 -0700 (PDT) Received: from agilent.com (cos1nai129182.cos.agilent.com [141.184.129.182]) by mail.rose.agilent.com (Netscape Messaging Server 3.6) with ESMTP id AAA1E27 for ; Tue, 1 May 2001 14:44:58 -0700 Message-ID: <3AEF1B3F.56F4A3A3@agilent.com> Date: Tue, 01 May 2001 16:23:27 -0400 From: "Matt Wakeley" Reply-To: Matt Wakeley Organization: Agilent Technologies X-Mailer: Mozilla 4.77 [en] (Windows NT 5.0; U) X-Accept-Language: en MIME-Version: 1.0 To: IPS Reflector Subject: Re: iscsi: comments to iSCSI rev 6 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-ips@ece.cmu.edu Precedence: bulk Content-Transfer-Encoding: 7bit reply from Julian... -------- Original Message -------- From: julian_satran@il.ibm.com Subject: Re: iscsi: comments to iSCSI rev 6 To: Matt Wakeley Comments in text. Thanks, Julo "Matt Wakeley" on 01-05-2001 03:53:24 Please respond to Matt Wakeley To: IPS Reflector cc: Subject: iscsi: comments to iSCSI rev 6 Section 2.2.3: The AHSLength field where it is requires RISC type processors to shift the length left by 8 bytes. +++ good point - I will make length the first field. +++ Section 2.3.5, 3rd paragraph: contradicts itself. First it states that a AHS header "MUST" be present, then goes on to define what the bi-di read length is if the header is not present. If it's not present, it's a protocol error. +++ will fix - it MUST be present +++ Section 2.4.1, last sentance: "b0-b3 MUST be 0" s/b "b1-b4 MUST be 0". +++ will fix +++ Section 2.7.7: why have residual bits/fields in a data PDU? If there is residual, then send a status PDU indicating the residual value. +++ residual counts are not necessarily errors. Interpretation is by target and many commands for undefined length transfer end having a residual count. The space is available in the header so I see no point in not sending the counts. SAM2 is somewhat ambivalent about overflow though. +++ Section 2.8.3: Keys not understood by the target should be expicitely indicated as not being understood. Silence is not a good way to indicate that one does not understand something. Also, something more ascii friendly such as a semi-colon (;) should be used to separate key-value pairs instead of a null (0x00) character. This would allow generic text manipulation libraries to be used. +++ generic text manipulation libraries use 0 as an end of string - our eyes don't :-) as for keys not understood the intent is to have new or vendor specific keys not interfering with the old or standrd only implementations +++ Section 2.9.3: why "MUST" key-value pairs be returned in the same order they were issued? Seems like a rather strong requirement for no apparent reason. +++ I've changed it into lower-case should +++ Section 2.12.3: indicate that the LUN is copied from the NOP-IN. This is much more clear than "the correct value for the task". +++ and open it up to strange thing like LU5 asking a nop-out to LU7 ? +++ Section 2.14: Why is there no CmdSN for the logout command? Also, 2 ways of performing the same operation (cleaning up) are stated in the 3rd paragraph. In the interest of reducing "options", I suggest that one be picked as the only method. +++ silly mistake - i've fixed it +++ Section 2.17.4: "The target assigns" should be "The target may assign". +++ ??? it always put something there +++ Section 6.3 & 6.7.2: why "MUST" a target reissue missing responses? What if it is not able to? There should be the option to reject the SNACK. +++ I am counting - you are number 2 +++ Appendix: "Initial Marker-less Interval" - I say again, there should not be a "minimum markerless interval". This should be whatever is negotiated. +++ The negotiation itself can't proceed if there are markers that nobody knows how to take out +++ -Matt Wakeley Agilent Technologies From owner-ips@ece.cmu.edu Tue May 1 20:04:52 2001 Received: from ece.cmu.edu (ECE.CMU.EDU [128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id UAA12713 for ; Tue, 1 May 2001 20:04:52 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f41Ljdn17778 for ips-outgoing; Tue, 1 May 2001 17:45:39 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from msgbas1.cos.agilent.com (msgbas1x.cos.agilent.com [192.6.9.33]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f41Ljbj17773 for ; Tue, 1 May 2001 17:45:37 -0400 (EDT) Received: from msgrel1.and.agilent.com (msgrel1.and.agilent.com [130.30.33.104]) by msgbas1.cos.agilent.com (Postfix) with ESMTP id 99D9676 for ; Tue, 1 May 2001 15:45:35 -0600 (MDT) Received: from rtl.rose.agilent.com (rtl.rose.agilent.com [156.140.232.231]) by msgrel1.and.agilent.com (Postfix) with ESMTP id 8780ED9 for ; Tue, 1 May 2001 17:45:34 -0400 (EDT) Received: from mail.rose.agilent.com (bellhop [156.140.233.51]) by rtl.rose.agilent.com (8.9.3 (PHNE_18979)/8.9.3 SMKit7.1.0) with ESMTP id OAA16583 for ; Tue, 1 May 2001 14:45:32 -0700 (PDT) Received: from agilent.com (cos1nai129182.cos.agilent.com [141.184.129.182]) by mail.rose.agilent.com (Netscape Messaging Server 3.6) with ESMTP id AAA1E36 for ; Tue, 1 May 2001 14:45:20 -0700 Message-ID: <3AEF20A2.146D62D8@agilent.com> Date: Tue, 01 May 2001 16:46:26 -0400 From: "Matt Wakeley" Reply-To: Matt Wakeley Organization: Agilent Technologies X-Mailer: Mozilla 4.77 [en] (Windows NT 5.0; U) X-Accept-Language: en MIME-Version: 1.0 To: IPS Reflector Subject: Re: iscsi: comments to iSCSI rev 6 References: Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-ips@ece.cmu.edu Precedence: bulk Content-Transfer-Encoding: 7bit julian_satran@il.ibm.com wrote: > +++ residual counts are not necessarily errors. Interpretation is by target > and many commands for undefined length transfer end having a residual > count. The space is available in the header so I see no point in not > sending the counts. SAM2 is somewhat ambivalent about overflow though. +++ But residuals are generally not normal behavior right? So then make them only in the response. That makes just one less option to implement (I don't care about if there is space in the header or not, the issue is implementing, testing and interoperating options). Status within a data PDU would then mean success with a residue of zero. Anything else is reported in a response PDU. > Section 2.8.3: Keys not understood by the target should be expicitely > indicated as not being understood. Silence is not a good way to indicate > that one does not understand something. > > +++ as for keys not understood the intent is to have new or vendor specific > keys not interfering with the old or standrd only implementations +++ How will returning "key=*not understood*" going to interfer with anything? It's much better than silence. > Section 2.12.3: indicate that the LUN is copied from the NOP-IN. This is > much more clear than "the correct value for the task". > > +++ and open it up to strange thing like LU5 asking a nop-out to LU7 ? +++ How will that occur? The target sent the NOP-IN ping request. The target includes a LUN. The only "correct value" that the initiator can return for the LUN in the NOP-OUT ping response is the value the target sent in the original request. > Section 2.14: Why is there no CmdSN for the logout command? Also, 2 ways > of performing the same operation (cleaning up) are stated in the 3rd > paragraph. > In the interest of reducing "options", I suggest that one be picked as the > only method. > +++ silly mistake - i've fixed it +++ I assume you're talking about the missing CmdSN. You didn't address the issue of eliminating one of the two ways of cleaning up. > Section 2.17.4: "The target assigns" should be "The target may assign". > +++ ??? it always put something there +++ Yes, it must always put something there. But "assign" means it picks a unique value and is actively using it. Unassigned means that it is not used, and the unassigned value of 0xffffffff is used. > Appendix: "Initial Marker-less Interval" - I say again, there should not be > a "minimum markerless interval". This should be whatever is negotiated. > +++ The negotiation itself can't proceed if there are markers that nobody > knows how to take out +++ Julian, you just keep missing my point. There are no markers until after login negotiation. Part of the negotiation is to say where the first marker will be. If I only use 100 bytes to log in, and I want markers every 1024 bytes, I should not have to wait 3900 bytes to get to the first one just because the spec arbitrarily picked 4K as the minimum amount of markerless space. -Matt Wakeley Agilent Technologies From owner-ips@ece.cmu.edu Wed May 2 00:53:39 2001 Received: from ece.cmu.edu (ECE.CMU.EDU [128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id AAA17897 for ; Wed, 2 May 2001 00:53:39 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f422bpZ02681 for ips-outgoing; Tue, 1 May 2001 22:37:51 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from msgbas1.cos.agilent.com (msgbas1x.cos.agilent.com [192.6.9.33]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f422boj02677 for ; Tue, 1 May 2001 22:37:50 -0400 (EDT) Received: from msgrel1.and.agilent.com (msgrel1.and.agilent.com [130.30.33.104]) by msgbas1.cos.agilent.com (Postfix) with ESMTP id 52818443 for ; Tue, 1 May 2001 20:37:49 -0600 (MDT) Received: from rtl.rose.agilent.com (rtl.rose.agilent.com [156.140.232.231]) by msgrel1.and.agilent.com (Postfix) with ESMTP id 6681ADB for ; Tue, 1 May 2001 22:37:48 -0400 (EDT) Received: from mail.rose.agilent.com (bellhop [156.140.233.51]) by rtl.rose.agilent.com (8.9.3 (PHNE_18979)/8.9.3 SMKit7.1.0) with ESMTP id TAA25149 for ; Tue, 1 May 2001 19:37:46 -0700 (PDT) Received: from agilent.com (cos1nai131032.cos.agilent.com [141.184.131.32]) by mail.rose.agilent.com (Netscape Messaging Server 3.6) with ESMTP id AAA4394 for ; Tue, 1 May 2001 19:37:42 -0700 Message-ID: <3AEF5144.77D7279E@agilent.com> Date: Tue, 01 May 2001 20:13:56 -0400 From: "Matt Wakeley" Reply-To: Matt Wakeley Organization: Agilent Technologies X-Mailer: Mozilla 4.77 [en] (Windows NT 5.0; U) X-Accept-Language: en MIME-Version: 1.0 To: IPS Reflector Subject: iscsi: documents with change bars Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-ips@ece.cmu.edu Precedence: bulk Content-Transfer-Encoding: 7bit I recall from one of the interim meetings that it was agreed that authors of iSCSI documents would post PDF versions of their documents with edit tracking (change bars, etc) to web sites somewhere with pointers to them. I'm getting tired of re-reading the whole *%$#!! ascii document everytime a new version comes out. (I don't want to hear about how *I* can view the differences with blah tool. The editors already have the ability to generate the documents with the changes highlighted. Just print the document to a PDF with the changes highlighted before accepting the changes and creating the real ascii document) -Matt From owner-ips@ece.cmu.edu Wed May 2 02:02:31 2001 Received: from ece.cmu.edu (ECE.CMU.EDU [128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id CAA27921 for ; Wed, 2 May 2001 02:02:31 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f423puU06210 for ips-outgoing; Tue, 1 May 2001 23:51:56 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from dirty.research.bell-labs.com (dirty.research.bell-labs.com [204.178.16.6]) by ece.cmu.edu (8.11.0/8.10.2) with SMTP id f423ptj06206 for ; Tue, 1 May 2001 23:51:55 -0400 (EDT) Received: from grubby.research.bell-labs.com ([135.104.2.9]) by dirty; Tue May 1 23:50:01 EDT 2001 Received: from aura.research.bell-labs.com ([135.104.46.10]) by grubby; Tue May 1 23:49:58 EDT 2001 Received: (from sandeepj@localhost) by aura.research.bell-labs.com (8.9.1/8.9.1) id XAA07353 for ips@ece.cmu.edu; Tue, 1 May 2001 23:49:53 -0400 (EDT) Date: Tue, 1 May 2001 23:49:53 -0400 (EDT) Message-Id: <200105020349.XAA07353@aura.research.bell-labs.com> From: sandeepj@research.bell-labs.com (Sandeep Joshi) To: ips@ece.cmu.edu Subject: iscsi: Text keys Sender: owner-ips@ece.cmu.edu Precedence: bulk Julian, Some comments on the text keys defined in Appendix E. thanks, -Sandeep 1) There are several text keys which have been specified with Use=ALL. Such keys can be negotiated during full-feature phase, which raises interesting questions about their effect on currently outstanding tasks. For example, a) maxOutstandingR2T : what is the effect on currently outstanding R2Ts ? b) LoginLogoutMaxTime : how does it affect connections which just got dropped and need to be recovered ? c) DataPDULength : effect on data PDUs in flight. Similar questions need to be answered for practically all the keys which have this behaviour. For the sake of simplicity, it may be better to forbid such usage and allow these keys to be only negotiated in the connection or session login phase. 2) The paragraph on TargetAddress seems to indicate that one or more text responses could be sent to satisfy a SendTargets command. "If the list cant be delivered as a single text response PDU, ..several text response PDUs can be sent..logical merge of the individual lists". How does the initiator know more than one TextResponse PDUs are expected for that TextCommand PDU ? If the final bit is to be used, does it imply that the initiator should keep sending the sendTargets TextCommand with f=1 until it receives the last TextResponse PDU ? Could you please clarify... 3) SendTargets again. The ordering within the text response currently is implied and needs to be specified so that the initiator can correlate the target alias with the target addresses. Something along these lines may help "The order within a text response should be (targetName, targetAlias(if any), list of addresses)" 4) The section on InitiatorName seems to imply that the target can also propose the value for this key ?? I presume this is a typo. See "Who: Initiator and Target" From owner-ips@ece.cmu.edu Wed May 2 02:03:07 2001 Received: from ece.cmu.edu (ECE.CMU.EDU [128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id CAA28614 for ; Wed, 2 May 2001 02:03:07 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f424QL107736 for ips-outgoing; Wed, 2 May 2001 00:26:21 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from sj-msg-core-2.cisco.com (sj-msg-core-2.cisco.com [171.69.43.88]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f424QJj07719 for ; Wed, 2 May 2001 00:26:19 -0400 (EDT) Received: from sponge.cisco.com (sponge.cisco.com [171.71.61.25]) by sj-msg-core-2.cisco.com (8.11.3/8.9.1) with ESMTP id f424QdK24351; Tue, 1 May 2001 21:26:39 -0700 (PDT) Received: from dap02w2k (sjc-vpn-261.cisco.com [10.21.65.5]) by sponge.cisco.com (Mirapoint) with SMTP id ACW04946; Tue, 1 May 2001 23:26:09 -0500 (CDT) From: "Dave Peterson" To: "Matt Wakeley" , "IPS Reflector" Subject: RE: iscsi: documents with change bars Date: Tue, 1 May 2001 23:23:36 -0500 Message-ID: MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit X-Priority: 3 (Normal) X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook IMO, Build 9.0.2416 (9.0.2911.0) X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4133.2400 Importance: Normal In-Reply-To: <3AEF5144.77D7279E@agilent.com> Sender: owner-ips@ece.cmu.edu Precedence: bulk Content-Transfer-Encoding: 7bit I second this sentiment, whoops I mean I give it a very loud hummm... Dave > -----Original Message----- > From: owner-ips@ece.cmu.edu [mailto:owner-ips@ece.cmu.edu]On Behalf Of > Matt Wakeley > Sent: Tuesday, May 01, 2001 7:14 PM > To: IPS Reflector > Subject: iscsi: documents with change bars > > > I recall from one of the interim meetings that it was agreed that > authors of > iSCSI documents would post PDF versions of their documents with > edit tracking > (change bars, etc) to web sites somewhere with pointers to them. > > I'm getting tired of re-reading the whole *%$#!! ascii document > everytime a > new version comes out. > > (I don't want to hear about how *I* can view the differences with > blah tool. > The editors already have the ability to generate the documents with the > changes highlighted. Just print the document to a PDF with the changes > highlighted before accepting the changes and creating the real > ascii document) > > -Matt > From owner-ips@ece.cmu.edu Wed May 2 06:07:25 2001 Received: from ece.cmu.edu (ECE.CMU.EDU [128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id GAA04634 for ; Wed, 2 May 2001 06:07:25 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f426sxf14041 for ips-outgoing; Wed, 2 May 2001 02:54:59 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from d12lmsgate-3.de.ibm.com (d12lmsgate-3.de.ibm.com [195.212.91.201]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f426suj14037 for ; Wed, 2 May 2001 02:54:57 -0400 (EDT) Received: from d12relay02.de.ibm.com (d12relay02.de.ibm.com [9.165.215.23]) by d12lmsgate-3.de.ibm.com (1.0.0) with ESMTP id IAA46318 for ; Wed, 2 May 2001 08:54:48 +0200 From: julian_satran@il.ibm.com Received: from d12mta02.de.ibm.com (d12mta01_cs0 [9.165.222.237]) by d12relay02.de.ibm.com (8.8.8m3/NCO v4.96) with SMTP id IAA123642 for ; Wed, 2 May 2001 08:54:48 +0200 Received: by d12mta02.de.ibm.com(Lotus SMTP MTA v4.6.5 (863.2 5-20-1999)) id C1256A40.0025F89B ; Wed, 2 May 2001 08:54:44 +0200 X-Lotus-FromDomain: IBMIL@IBMDE To: ips@ece.cmu.edu Message-ID: Date: Wed, 2 May 2001 09:45:31 +0300 Subject: Re: iscsi: comments to iSCSI rev 6 Mime-Version: 1.0 Content-type: text/plain; charset=us-ascii Content-Disposition: inline Sender: owner-ips@ece.cmu.edu Precedence: bulk Matt, Comments in text. I still did not hear any comment from you on the commad codes you where so concerned about. Julo "Matt Wakeley" on 01-05-2001 23:46:26 Please respond to Matt Wakeley To: IPS Reflector cc: Subject: Re: iscsi: comments to iSCSI rev 6 julian_satran@il.ibm.com wrote: > +++ residual counts are not necessarily errors. Interpretation is by target > and many commands for undefined length transfer end having a residual > count. The space is available in the header so I see no point in not > sending the counts. SAM2 is somewhat ambivalent about overflow though. +++ But residuals are generally not normal behavior right? So then make them only in the response. That makes just one less option to implement (I don't care about if there is space in the header or not, the issue is implementing, testing and interoperating options). Status within a data PDU would then mean success with a residue of zero. Anything else is reported in a response PDU. +++ On the contrary for some devices residual counts are normal and frequent. Reading from a tape with a variable length block will usually result in an underrun. Besides - except for the mapping the path for handling is the same as for a separate response so that I have a hard time understanding the implementation argument. ++++ > Section 2.8.3: Keys not understood by the target should be expicitely > indicated as not being understood. Silence is not a good way to indicate > that one does not understand something. > > +++ as for keys not understood the intent is to have new or vendor specific > keys not interfering with the old or standrd only implementations +++ +++ yet another way for a DOS attack - but I will change it. The text will read now: Any other key not understood by the target may be ignored by the target without affecting basic function. However the Text Response for a key that was not understood MUST be key=NotUnderstood. If the Text Response does not contain the key requested, the initiator must assume, whenever appropriate, that the response was "none". How will returning "key=*not understood*" going to interfer with anything? It's much better than silence. > Section 2.12.3: indicate that the LUN is copied from the NOP-IN. This is > much more clear than "the correct value for the task". > > +++ and open it up to strange thing like LU5 asking a nop-out to LU7 ? +++ How will that occur? The target sent the NOP-IN ping request. The target includes a LUN. The only "correct value" that the initiator can return for the LUN in the NOP-OUT ping response is the value the target sent in the original request. +++ OK +++ > Section 2.14: Why is there no CmdSN for the logout command? Also, 2 ways > of performing the same operation (cleaning up) are stated in the 3rd > paragraph. > In the interest of reducing "options", I suggest that one be picked as the > only method. > +++ silly mistake - i've fixed it +++ I assume you're talking about the missing CmdSN. You didn't address the issue of eliminating one of the two ways of cleaning up. +++ That seems to be an endless discussion. Whever I take it out there is somebody that asks for the colapse. The only good argument for the colapse is single connection targets that whithout this option in login will reject the second login +++ > Section 2.17.4: "The target assigns" should be "The target may assign". > +++ ??? it always put something there +++ Yes, it must always put something there. But "assign" means it picks a unique value and is actively using it. Unassigned means that it is not used, and the unassigned value of 0xffffffff is used. +++ my english dictionary - Webster - shows no such connotation +++ > Appendix: "Initial Marker-less Interval" - I say again, there should not be > a "minimum markerless interval". This should be whatever is negotiated. > +++ The negotiation itself can't proceed if there are markers that nobody > knows how to take out +++ Julian, you just keep missing my point. There are no markers until after login negotiation. Part of the negotiation is to say where the first marker will be. If I only use 100 bytes to log in, and I want markers every 1024 bytes, I should not have to wait 3900 bytes to get to the first one just because the spec arbitrarily picked 4K as the minimum amount of markerless space. +++ OK and analyzers will take care of themselves The new part will read: 01 Initial Marker-less Interval To enable the connection setup including the login phase negotiation, marking (if any) is started only at the first marker interval after the end of the login phase. -Matt Wakeley Agilent Technologies From owner-ips@ece.cmu.edu Wed May 2 06:10:35 2001 Received: from ece.cmu.edu (ECE.CMU.EDU [128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id GAA04652 for ; Wed, 2 May 2001 06:10:35 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f427mkJ16243 for ips-outgoing; Wed, 2 May 2001 03:48:46 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from d12lmsgate-3.de.ibm.com (d12lmsgate-3.de.ibm.com [195.212.91.201]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f427mej16237 for ; Wed, 2 May 2001 03:48:40 -0400 (EDT) Received: from d12relay01.de.ibm.com (d12relay01.de.ibm.com [9.165.215.22]) by d12lmsgate-3.de.ibm.com (1.0.0) with ESMTP id JAA45086 for ; Wed, 2 May 2001 09:48:19 +0200 From: julian_satran@il.ibm.com Received: from d12mta02.de.ibm.com (d12mta01_cs0 [9.165.222.237]) by d12relay01.de.ibm.com (8.8.8m3/NCO v4.96) with SMTP id JAA147214 for ; Wed, 2 May 2001 09:48:19 +0200 Received: by d12mta02.de.ibm.com(Lotus SMTP MTA v4.6.5 (863.2 5-20-1999)) id C1256A40.002ADF20 ; Wed, 2 May 2001 09:48:16 +0200 X-Lotus-FromDomain: IBMIL@IBMDE To: ips@ece.cmu.edu Message-ID: Date: Wed, 2 May 2001 10:53:49 +0300 Subject: Re: Another shot at codes and please comment Mime-Version: 1.0 Content-type: text/plain; charset=us-ascii Content-Disposition: inline Sender: owner-ips@ece.cmu.edu Precedence: bulk Sandeep, I think that we can leave the behaviour for unrecognized vendor-specific codes to be also vendor-specific :-) Julo sandeepj@research.bell-labs.com (Sandeep Joshi) on 01-05-2001 22:07:06 Please respond to sandeepj@research.bell-labs.com (Sandeep Joshi) To: Julian Satran/Haifa/IBM@IBMIL cc: ips@ece.cmu.edu Subject: Another shot at codes and please comment Julian, I'd prefer this over the alternating opcode assignments. (where nop-out was 0 and nop-in was 1) Could you also define the behaviour if a vendor-specific opcode was not recognized ? I presume we always want to drop the session, since these opcodes would only be used if the target and initiator recognized/negotiated (in a vendor-specific way) their use during session initiation. -Sandeep > Here is a another version of the opcodes part: > > 1.1.1.1 Opcode > > The Opcode indicates what type of iSCSI PDU the header encapsulates. > > The Opcodes are divided into two categories: initiator opcodes and > target opcodes. Initiator opcodes are in PDUs sent by the initiators > (request PDUs), and target opcodes are in PDUs sent by the target > (response PDUs). > > Initiators MUST NOT use target opcodes and targets MUST NOT use > initiator opcodes. > > Valid initiator opcodes defined in this specification are: > > > 0x00 NOP-Out (from initiator to target) > 0x01 SCSI Command (encapsulates a SCSI Command Descriptor Block) > 0x02 SCSI Task Management Command > 0x03 Login Command > 0x04 Text Command > 0x05 SCSI Data-out (for WRITE operations) > 0x06 Logout Command > 0x10 SNACK Request > > Valid target opcodes are: > > > 0x20 NOP-In (from target to initiator) > 0x21 SCSI Response (contains SCSI status and possibly sense > information or other response information) > 0x22 SCSI Task Management Response > 0x23 Login Response > 0x24 Text Response > 0x25 SCSI Data-in (for READ operations) > 0x26 Logout Response > 0x31 Ready To Transfer (R2T - sent by target to initiator when it is > ready to receive data from initiator) > 0x32 Asynchronous Message (sent by target to initiator to indicate > certain special conditions) > 0x3f Reject > > Initiator opcodes 0x1c-0x1e and target opcodes 0x3c-0x3e are vendor > specific codes. > > > Please comment, > Julo > From owner-ips@ece.cmu.edu Wed May 2 07:00:58 2001 Received: from ece.cmu.edu (ECE.CMU.EDU [128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id HAA05358 for ; Wed, 2 May 2001 07:00:58 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f428N6v17571 for ips-outgoing; Wed, 2 May 2001 04:23:06 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from d12lmsgate-2.de.ibm.com (d12lmsgate-2.de.ibm.com [195.212.91.200]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f428N3j17567 for ; Wed, 2 May 2001 04:23:03 -0400 (EDT) Received: from d12relay01.de.ibm.com (d12relay01.de.ibm.com [9.165.215.22]) by d12lmsgate-2.de.ibm.com (1.0.0) with ESMTP id KAA95254 for ; Wed, 2 May 2001 10:22:54 +0200 From: julian_satran@il.ibm.com Received: from d12mta02.de.ibm.com (d12mta01_cs0 [9.165.222.237]) by d12relay01.de.ibm.com (8.8.8m3/NCO v4.96) with SMTP id KAA261310 for ; Wed, 2 May 2001 10:22:54 +0200 Received: by d12mta02.de.ibm.com(Lotus SMTP MTA v4.6.5 (863.2 5-20-1999)) id C1256A40.002E08BA ; Wed, 2 May 2001 10:22:48 +0200 X-Lotus-FromDomain: IBMIL@IBMDE To: ips@ece.cmu.edu Message-ID: Date: Wed, 2 May 2001 11:17:23 +0300 Subject: Re: iscsi: documents with change bars Mime-Version: 1.0 Content-type: text/plain; charset=us-ascii Content-Disposition: inline Sender: owner-ips@ece.cmu.edu Precedence: bulk The reason you did not see it up to now is that there where too many changes for it be usefull (although they where mostly mechanical). Julo "Matt Wakeley" on 02-05-2001 03:13:56 Please respond to Matt Wakeley To: IPS Reflector cc: Subject: iscsi: documents with change bars I recall from one of the interim meetings that it was agreed that authors of iSCSI documents would post PDF versions of their documents with edit tracking (change bars, etc) to web sites somewhere with pointers to them. I'm getting tired of re-reading the whole *%$#!! ascii document everytime a new version comes out. (I don't want to hear about how *I* can view the differences with blah tool. The editors already have the ability to generate the documents with the changes highlighted. Just print the document to a PDF with the changes highlighted before accepting the changes and creating the real ascii document) -Matt From owner-ips@ece.cmu.edu Wed May 2 12:01:28 2001 Received: from ece.cmu.edu (ECE.CMU.EDU [128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id MAA15601 for ; Wed, 2 May 2001 12:01:14 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f42Dml616419 for ips-outgoing; Wed, 2 May 2001 09:48:47 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from msgbas1.cos.agilent.com (msgbas1x.cos.agilent.com [192.6.9.33]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f42DmjH16414 for ; Wed, 2 May 2001 09:48:45 -0400 (EDT) Received: from msgrel1.and.agilent.com (msgrel1.and.agilent.com [130.30.33.104]) by msgbas1.cos.agilent.com (Postfix) with ESMTP id D1CA38CC for ; Wed, 2 May 2001 07:48:44 -0600 (MDT) Received: from rtl.rose.agilent.com (rtl.rose.agilent.com [156.140.232.231]) by msgrel1.and.agilent.com (Postfix) with ESMTP id C4318AC for ; Wed, 2 May 2001 09:48:43 -0400 (EDT) Received: from mail.rose.agilent.com (mailsrv@bellhop [156.140.233.51]) by rtl.rose.agilent.com (8.9.3 (PHNE_18979)/8.9.3 SMKit7.1.0) with ESMTP id GAA00798 for ; Wed, 2 May 2001 06:48:42 -0700 (PDT) Received: from agilent.com (cos1nai134018.cos.agilent.com [141.184.134.18]) by mail.rose.agilent.com (Netscape Messaging Server 3.6) with ESMTP id AAA1562 for ; Wed, 2 May 2001 06:48:38 -0700 Message-ID: <3AF01029.C809B342@agilent.com> Date: Wed, 02 May 2001 09:48:25 -0400 From: "Matt Wakeley" Reply-To: Matt Wakeley Organization: Agilent Technologies X-Mailer: Mozilla 4.77 [en] (Windows NT 5.0; U) X-Accept-Language: en MIME-Version: 1.0 To: ips@ece.cmu.edu Subject: Re: iscsi: documents with change bars References: Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-ips@ece.cmu.edu Precedence: bulk Content-Transfer-Encoding: 7bit Well, I would rather be the one to decide if it's useful or not... -Matt julian_satran@il.ibm.com wrote: > > The reason you did not see it up to now is that there where too many > changes for it be usefull (although they where mostly mechanical). > > Julo > > "Matt Wakeley" on 02-05-2001 03:13:56 > > Please respond to Matt Wakeley > > To: IPS Reflector > cc: > Subject: iscsi: documents with change bars > > I recall from one of the interim meetings that it was agreed that authors > of > iSCSI documents would post PDF versions of their documents with edit > tracking > (change bars, etc) to web sites somewhere with pointers to them. > > I'm getting tired of re-reading the whole *%$#!! ascii document everytime a > new version comes out. > > (I don't want to hear about how *I* can view the differences with blah > tool. > The editors already have the ability to generate the documents with the > changes highlighted. Just print the document to a PDF with the changes > highlighted before accepting the changes and creating the real ascii > document) > > -Matt From owner-ips@ece.cmu.edu Wed May 2 12:01:55 2001 Received: from ece.cmu.edu (ECE.CMU.EDU [128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id MAA15623 for ; Wed, 2 May 2001 12:01:49 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f42DoGE16513 for ips-outgoing; Wed, 2 May 2001 09:50:16 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from msgbas1.cos.agilent.com (msgbas1x.cos.agilent.com [192.6.9.33]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f42DoFH16507 for ; Wed, 2 May 2001 09:50:15 -0400 (EDT) Received: from msgrel1.and.agilent.com (msgrel1.and.agilent.com [130.30.33.104]) by msgbas1.cos.agilent.com (Postfix) with ESMTP id 69E7FB5F for ; Wed, 2 May 2001 07:50:14 -0600 (MDT) Received: from rtl.rose.agilent.com (rtl.rose.agilent.com [156.140.232.231]) by msgrel1.and.agilent.com (Postfix) with ESMTP id 4745D2E for ; Wed, 2 May 2001 09:50:13 -0400 (EDT) Received: from mail.rose.agilent.com (mailsrv@bellhop [156.140.233.51]) by rtl.rose.agilent.com (8.9.3 (PHNE_18979)/8.9.3 SMKit7.1.0) with ESMTP id GAA00803 for ; Wed, 2 May 2001 06:50:11 -0700 (PDT) Received: from agilent.com (cos1nai134018.cos.agilent.com [141.184.134.18]) by mail.rose.agilent.com (Netscape Messaging Server 3.6) with ESMTP id AAA15AA for ; Wed, 2 May 2001 06:50:07 -0700 Message-ID: <3AF01088.FD32CB4D@agilent.com> Date: Wed, 02 May 2001 09:50:00 -0400 From: "Matt Wakeley" Reply-To: Matt Wakeley Organization: Agilent Technologies X-Mailer: Mozilla 4.77 [en] (Windows NT 5.0; U) X-Accept-Language: en MIME-Version: 1.0 To: ips@ece.cmu.edu Subject: Re: Another shot at codes and please comment References: Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-ips@ece.cmu.edu Precedence: bulk Content-Transfer-Encoding: 7bit This looks ok. -Matt julian_satran@il.ibm.com wrote: > > Here is a another version of the opcodes part: > > 1.1.1.1 Opcode > > The Opcode indicates what type of iSCSI PDU the header encapsulates. > > The Opcodes are divided into two categories: initiator opcodes and > target opcodes. Initiator opcodes are in PDUs sent by the initiators > (request PDUs), and target opcodes are in PDUs sent by the target > (response PDUs). > > Initiators MUST NOT use target opcodes and targets MUST NOT use > initiator opcodes. > > Valid initiator opcodes defined in this specification are: > > 0x00 NOP-Out (from initiator to target) > 0x01 SCSI Command (encapsulates a SCSI Command Descriptor Block) > 0x02 SCSI Task Management Command > 0x03 Login Command > 0x04 Text Command > 0x05 SCSI Data-out (for WRITE operations) > 0x06 Logout Command > 0x10 SNACK Request > > Valid target opcodes are: > > 0x20 NOP-In (from target to initiator) > 0x21 SCSI Response (contains SCSI status and possibly sense > information or other response information) > 0x22 SCSI Task Management Response > 0x23 Login Response > 0x24 Text Response > 0x25 SCSI Data-in (for READ operations) > 0x26 Logout Response > 0x31 Ready To Transfer (R2T - sent by target to initiator when it is > ready to receive data from initiator) > 0x32 Asynchronous Message (sent by target to initiator to indicate > certain special conditions) > 0x3f Reject > > Initiator opcodes 0x1c-0x1e and target opcodes 0x3c-0x3e are vendor > specific codes. > > Please comment, > Julo From owner-ips@ece.cmu.edu Wed May 2 13:31:08 2001 Received: from ece.cmu.edu (ECE.CMU.EDU [128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id NAA17978 for ; Wed, 2 May 2001 13:31:08 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f42FYCc24077 for ips-outgoing; Wed, 2 May 2001 11:34:12 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from d12lmsgate-3.de.ibm.com (d12lmsgate-3.de.ibm.com [195.212.91.201]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f42FY5H24064 for ; Wed, 2 May 2001 11:34:05 -0400 (EDT) Received: from d12relay01.de.ibm.com (d12relay01.de.ibm.com [9.165.215.22]) by d12lmsgate-3.de.ibm.com (1.0.0) with ESMTP id RAA104452 for ; Wed, 2 May 2001 17:33:56 +0200 From: julian_satran@il.ibm.com Received: from d12mta02.de.ibm.com (d12mta01_cs0 [9.165.222.237]) by d12relay01.de.ibm.com (8.8.8m3/NCO v4.96) with SMTP id RAA132862 for ; Wed, 2 May 2001 17:33:56 +0200 Received: by d12mta02.de.ibm.com(Lotus SMTP MTA v4.6.5 (863.2 5-20-1999)) id C1256A40.00557F97 ; Wed, 2 May 2001 17:33:52 +0200 X-Lotus-FromDomain: IBMIL@IBMDE To: ips@ece.cmu.edu Message-ID: Date: Wed, 2 May 2001 18:39:27 +0300 Subject: Re: iscsi: documents with change bars Mime-Version: 1.0 Content-type: text/plain; charset=us-ascii Content-Disposition: inline Sender: owner-ips@ece.cmu.edu Precedence: bulk Matt, I would like to clarify my position on this. I felt myself that everybody needs it and I suggested it in the Orlando meeting. It was simply a continuous bar due to the mechanical changes. Except for chapter 6 I think that we are in a better shape now and I will post changes on my site. I am also posting (for the last 3 months) snipets of changes and complete paragraphs and I usually get very few comments. Regards, Julo From owner-ips@ece.cmu.edu Wed May 2 13:32:26 2001 Received: from ece.cmu.edu (ECE.CMU.EDU [128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id NAA18027 for ; Wed, 2 May 2001 13:32:26 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f42FOxh23228 for ips-outgoing; Wed, 2 May 2001 11:24:59 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from d12lmsgate-2.de.ibm.com (d12lmsgate-2.de.ibm.com [195.212.91.200]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f42FOsH23216 for ; Wed, 2 May 2001 11:24:55 -0400 (EDT) Received: from d12relay01.de.ibm.com (d12relay01.de.ibm.com [9.165.215.22]) by d12lmsgate-2.de.ibm.com (1.0.0) with ESMTP id RAA154688 for ; Wed, 2 May 2001 17:24:37 +0200 From: julian_satran@il.ibm.com Received: from d12mta02.de.ibm.com (d12mta01_cs0 [9.165.222.237]) by d12relay01.de.ibm.com (8.8.8m3/NCO v4.96) with SMTP id RAA119938 for ; Wed, 2 May 2001 17:24:37 +0200 Received: by d12mta02.de.ibm.com(Lotus SMTP MTA v4.6.5 (863.2 5-20-1999)) id C1256A40.0054A2A0 ; Wed, 2 May 2001 17:24:26 +0200 X-Lotus-FromDomain: IBMIL@IBMDE To: ips@ece.cmu.edu Message-ID: Date: Wed, 2 May 2001 18:29:58 +0300 Subject: Re: iscsi: Text keys Mime-Version: 1.0 Content-type: text/plain; charset=us-ascii Content-Disposition: inline Sender: owner-ips@ece.cmu.edu Precedence: bulk comments/answers in text. Thanks, Julo sandeepj@research.bell-labs.com (Sandeep Joshi) on 02-05-2001 06:49:53 Please respond to sandeepj@research.bell-labs.com (Sandeep Joshi) To: ips@ece.cmu.edu cc: Subject: iscsi: Text keys Julian, Some comments on the text keys defined in Appendix E. thanks, -Sandeep 1) There are several text keys which have been specified with Use=ALL. Such keys can be negotiated during full-feature phase, which raises interesting questions about their effect on currently outstanding tasks. For example, a) maxOutstandingR2T : what is the effect on currently outstanding R2Ts ? b) LoginLogoutMaxTime : how does it affect connections which just got dropped and need to be recovered ? c) DataPDULength : effect on data PDUs in flight. Similar questions need to be answered for practically all the keys which have this behaviour. For the sake of simplicity, it may be better to forbid such usage and allow these keys to be only negotiated in the connection or session login phase. +++ some of the parameters you mentioned are SCSI mode-page parameters. Even if we restricted them to login they can be changed by a SCSI mode-set. I agree that keeping them all restricted to login would simplify implementations but I am not convince it would make them better. Evidently a negotiated change will apply to all packets that are sent after the negotiation ended (and a negotiation end is known to both parties) and an initiator will avoid sending things in violation of its last offering. Except that we will have to find a way to reject mode set I am open to restrict those parameters to login if that is what the majority of implementors tells me. +++ 2) The paragraph on TargetAddress seems to indicate that one or more text responses could be sent to satisfy a SendTargets command. "If the list cant be delivered as a single text response PDU, ..several text response PDUs can be sent..logical merge of the individual lists". How does the initiator know more than one TextResponse PDUs are expected for that TextCommand PDU ? If the final bit is to be used, does it imply that the initiator should keep sending the sendTargets TextCommand with f=1 until it receives the last TextResponse PDU ? Could you please clarify... +++ the value 0 in the F bit of a text commands tells you that there is more to come. I addition I think that the naming and discovery folks are looking at a "list syntax" ++++ 3) SendTargets again. The ordering within the text response currently is implied and needs to be specified so that the initiator can correlate the target alias with the target addresses. Something along these lines may help "The order within a text response should be (targetName, targetAlias(if any), list of addresses)" +++ I have to coordinate this with the Naming and discovery team. +++ 4) The section on InitiatorName seems to imply that the target can also propose the value for this key ?? I presume this is a typo. See "Who: Initiator and Target" +++ it is a typo thanks +++ From owner-ips@ece.cmu.edu Wed May 2 13:34:12 2001 Received: from ece.cmu.edu (ECE.CMU.EDU [128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id NAA18080 for ; Wed, 2 May 2001 13:34:12 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f42FBXw22169 for ips-outgoing; Wed, 2 May 2001 11:11:33 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from dogwood.cisco.com (dogwood.cisco.com [161.44.11.19]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f42FBSH22155 for ; Wed, 2 May 2001 11:11:28 -0400 (EDT) Received: from cisco.com (dhcp-161-44-68-173.cisco.com [161.44.68.173]) by dogwood.cisco.com (8.8.6 (PHNE_14041)/CISCO.SERVER.1.2) with ESMTP id LAA17177 for ; Wed, 2 May 2001 11:11:22 -0400 (EDT) Message-ID: <3AF0232E.852EDB5E@cisco.com> Date: Wed, 02 May 2001 10:09:34 -0500 From: Steve Senum X-Mailer: Mozilla 4.73 [en]C-CCK-MCD (Windows NT 5.0; U) X-Accept-Language: en MIME-Version: 1.0 To: ips@ece.cmu.edu Subject: Re: iscsi: comments to iSCSI rev 6 References: Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-ips@ece.cmu.edu Precedence: bulk Content-Transfer-Encoding: 7bit I would prefer the text stay as is, that the target not send any response for an unknown key. I see no value at all in sending key=NotUnderstood. Lack of a response tells me that already. Steve Senum > > Section 2.8.3: Keys not understood by the target should be expicitely > > indicated as not being understood. Silence is not a good way to indicate > > that one does not understand something. > > > > +++ as for keys not understood the intent is to have new or vendor > specific > > keys not interfering with the old or standrd only implementations +++ > > +++ yet another way for a DOS attack - but I will change it. The text will > read now: > > Any other key not understood by the target may be ignored by the target > without affecting basic function. However the Text Response for a key that > was not understood MUST be key=NotUnderstood. > > If the Text Response does not contain the key requested, the initiator > must assume, whenever appropriate, that the response was "none". > > How will returning "key=*not understood*" going to interfer with anything? > It's much better than silence. > From owner-ips@ece.cmu.edu Wed May 2 16:27:44 2001 Received: from ece.cmu.edu (ECE.CMU.EDU [128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id QAA22589 for ; Wed, 2 May 2001 16:27:44 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f42HuQF05651 for ips-outgoing; Wed, 2 May 2001 13:56:26 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from clyde.stargateip.com ([209.237.5.66]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f42HuHH05630 for ; Wed, 2 May 2001 13:56:17 -0400 (EDT) Received: from fixpc ([10.10.5.76]) by clyde.stargateip.com (8.9.1/8.9.1) with SMTP id KAA14014; Wed, 2 May 2001 10:51:15 -0700 (PDT) From: "Meena Ramamoorthi" To: "Elizabeth Rodriguez" , Subject: RE: FCIP: Management of FCIP devices [RE: iSCSI IETF announcement] Date: Wed, 2 May 2001 11:02:51 -0700 Message-ID: MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 (Normal) X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook IMO, Build 9.0.2416 (9.0.2910.0) Importance: Normal In-Reply-To: X-MimeOLE: Produced By Microsoft MimeOLE V5.00.2919.6600 Sender: owner-ips@ece.cmu.edu Precedence: bulk Content-Transfer-Encoding: 7bit Hi Elizabeth, I agree with what ever you say. We can touch upon the management of FCIP devices in this draft and we can have separate documents/efforts for the FCIP - MIBs. Thanks, Meenakshi -----Original Message----- From: Elizabeth Rodriguez [mailto:egrodriguez@nc8220exchange.ral.lucent.com] Sent: Tuesday, May 01, 2001 8:30 PM To: Meena Ramamoorthi; ips@ece.cmu.edu Subject: FCIP: Management of FCIP devices [RE: iSCSI IETF announcement] Hi Meenakshi, FCIP MIB work is in progress. An individual submission for an FCIP MIB was posted last week. While management might be touched upon, MIBs must be separate documents/efforts. Thanks, Elizabeth -----Original Message----- From: Meena Ramamoorthi [mailto:meena@stargateip.com] Sent: Monday, April 30, 2001 12:12 PM To: Dave Nagle; ips@ece.cmu.edu Subject: RE: iSCSI IETF annoucement It may be good to add and highlight the details regarding the management of the FCoverTCPIP devices/ switches, like SNMP, MIB's etc etc in this draft. Thanks Meenakshi -----Original Message----- From: owner-ips@ece.cmu.edu [mailto:owner-ips@ece.cmu.edu]On Behalf Of Dave Nagle Sent: Monday, April 23, 2001 1:29 PM To: ips@ece.cmu.edu Subject: iSCSI IETF annoucement A New Internet-Draft is available from the on-line Internet-Drafts directories. This draft is a work item of the IP Storage Working Group of the IETF. Title : Fibre Channel Over TCP/IP (FCIP) Author(s) : M. Rajagopal, R. Bhagwat Filename : draft-ietf-ips-fcovertcpip-02.txt Pages : 26 Date : 20-Apr-01 Fibre Channel (FC) is a dominant technology used in Storage Area Networks (SAN). The purpose of this draft is to specify a standard way of encapsulating FC frames over TCP/IP and to describe mechanisms that allow islands of FC SANs to be interconnected over IP-based networks. FC over TCP/IP relies on IP-based network services to provide the connectivity between the SAN islands over LANs, MANs, or WANs. The FC over TCP/IP specification relies upon TCP for congestion control and management and upon both TCP and FC for data error and data loss recovery. FC over TCP/IP treats all classes of FC frames the same -- as datagrams. A URL for this Internet-Draft is: http://www.ietf.org/internet-drafts/draft-ietf-ips-fcovertcpip-02.txt Internet-Drafts are also available by anonymous FTP. Login with the username "anonymous" and a password of your e-mail address. After logging in, type "cd internet-drafts" and then "get draft-ietf-ips-fcovertcpip-02.txt". A list of Internet-Drafts directories can be found in http://www.ietf.org/shadow.html or ftp://ftp.ietf.org/ietf/1shadow-sites.txt Internet-Drafts can also be obtained by e-mail. Send a message to: mailserv@ietf.org. In the body type: "FILE /internet-drafts/draft-ietf-ips-fcovertcpip-02.txt". NOTE: The mail server at ietf.org can return the document in MIME-encoded form by using the "mpack" utility. To use this feature, insert the command "ENCODING mime" before the "FILE" command. To decode the response(s), you will need "munpack" or a MIME-compliant mail reader. Different MIME-compliant mail readers exhibit different behavior, especially when dealing with "multipart" MIME messages (i.e. documents which have been split up into multiple messages), so check your local documentation on how to manipulate these messages. Below is the data which will enable a MIME compliant mail reader implementation to automatically retrieve the ASCII version of the Internet-Draft. - --NextPart Content-Type: Multipart/Alternative; Boundary="OtherAccess" - --OtherAccess Content-Type: Message/External-body; access-type="mail-server"; server="mailserv@ietf.org" Content-Type: text/plain Content-ID: <20010420154253.I-D@ietf.org> ENCODING mime FILE /internet-drafts/draft-ietf-ips-fcovertcpip-02.txt - --OtherAccess Content-Type: Message/External-body; name="draft-ietf-ips-fcovertcpip-02.txt"; site="ftp.ietf.org"; access-type="anon-ftp"; directory="internet-drafts" Content-Type: text/plain Content-ID: <20010420154253.I-D@ietf.org> - --OtherAccess-- - --NextPart-- ------- End of Forwarded Message From owner-ips@ece.cmu.edu Wed May 2 20:45:10 2001 Received: from ece.cmu.edu (ECE.CMU.EDU [128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id UAA27441 for ; Wed, 2 May 2001 20:45:10 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f42Mi9W00828 for ips-outgoing; Wed, 2 May 2001 18:44:09 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from msgbas1.cos.agilent.com (msgbas1x.cos.agilent.com [192.6.9.33]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f42Mi8H00823 for ; Wed, 2 May 2001 18:44:08 -0400 (EDT) Received: from msgrel1.and.agilent.com (msgrel1.and.agilent.com [130.30.33.104]) by msgbas1.cos.agilent.com (Postfix) with ESMTP id F2099852 for ; Wed, 2 May 2001 16:44:06 -0600 (MDT) Received: from rtl.rose.agilent.com (rtl.rose.agilent.com [156.140.232.231]) by msgrel1.and.agilent.com (Postfix) with ESMTP id D9AC121 for ; Wed, 2 May 2001 18:44:05 -0400 (EDT) Received: from mail.rose.agilent.com (bellhop [156.140.233.51]) by rtl.rose.agilent.com (8.9.3 (PHNE_18979)/8.9.3 SMKit7.1.0) with ESMTP id PAA05463 for ; Wed, 2 May 2001 15:44:04 -0700 (PDT) Received: from agilent.com (matt5670.rose.agilent.com [156.140.235.204]) by mail.rose.agilent.com (Netscape Messaging Server 3.6) with ESMTP id AAA5E5F for ; Wed, 2 May 2001 15:44:00 -0700 Message-ID: <3AF08DB1.D1A48FF7@agilent.com> Date: Wed, 02 May 2001 15:44:01 -0700 From: "Matt Wakeley" Reply-To: Matt Wakeley Organization: Agilent Technologies X-Mailer: Mozilla 4.77 [en] (Windows NT 5.0; U) X-Accept-Language: en MIME-Version: 1.0 To: ips@ece.cmu.edu Subject: Re: iscsi: comments to iSCSI rev 6 References: <3AF0232E.852EDB5E@cisco.com> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-ips@ece.cmu.edu Precedence: bulk Content-Transfer-Encoding: 7bit Steve Senum wrote: > > I would prefer the text stay as is, > that the target not send any response > for an unknown key. I see no value > at all in sending key=NotUnderstood. > Lack of a response tells me that already. No it doesn't. You can't tell the difference between "not understood" and "none", since "none" is allowed not to respond also. I don't like that either. If the response is "none", say so, don't just sit there silently. -Matt From owner-ips@ece.cmu.edu Wed May 2 20:48:50 2001 Received: from ece.cmu.edu (ECE.CMU.EDU [128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id UAA27482 for ; Wed, 2 May 2001 20:48:50 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f42MP6l29141 for ips-outgoing; Wed, 2 May 2001 18:25:06 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from msgbas1.cos.agilent.com (msgbas1x.cos.agilent.com [192.6.9.33]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f42MP4H29137 for ; Wed, 2 May 2001 18:25:05 -0400 (EDT) Received: from msgrel1.and.agilent.com (msgrel1.and.agilent.com [130.30.33.104]) by msgbas1.cos.agilent.com (Postfix) with ESMTP id B19312DD for ; Wed, 2 May 2001 16:25:03 -0600 (MDT) Received: from rtl.rose.agilent.com (rtl.rose.agilent.com [156.140.232.231]) by msgrel1.and.agilent.com (Postfix) with ESMTP id 435EB289 for ; Wed, 2 May 2001 18:25:02 -0400 (EDT) Received: from mail.rose.agilent.com (mailsrv@bellhop [156.140.233.51]) by rtl.rose.agilent.com (8.9.3 (PHNE_18979)/8.9.3 SMKit7.1.0) with ESMTP id PAA04233 for ; Wed, 2 May 2001 15:25:00 -0700 (PDT) Received: from agilent.com (matt5670.rose.agilent.com [156.140.235.204]) by mail.rose.agilent.com (Netscape Messaging Server 3.6) with ESMTP id AAA5782 for ; Wed, 2 May 2001 15:24:57 -0700 Message-ID: <3AF021BD.72005806@agilent.com> Date: Wed, 02 May 2001 11:03:25 -0400 From: "Matt Wakeley" Reply-To: Matt Wakeley Organization: Agilent Technologies X-Mailer: Mozilla 4.77 [en] (Windows NT 5.0; U) X-Accept-Language: en MIME-Version: 1.0 To: IPS Reflector Subject: Re: Another shot at codes and please comment References: Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-ips@ece.cmu.edu Precedence: bulk Content-Transfer-Encoding: 7bit *all* unrecognized opcodes should be rejected. There should be no distinction between a unrecognized vendor specific opcode and a unrecognized non-vendor specific opcode. -Matt julian_satran@il.ibm.com wrote: > > Sandeep, > > I think that we can leave the behaviour for unrecognized vendor-specific > codes to be also vendor-specific :-) > > Julo > > sandeepj@research.bell-labs.com (Sandeep Joshi) on 01-05-2001 22:07:06 > > Please respond to sandeepj@research.bell-labs.com (Sandeep Joshi) > > To: Julian Satran/Haifa/IBM@IBMIL > cc: ips@ece.cmu.edu > Subject: Another shot at codes and please comment > > Julian, > > I'd prefer this over the alternating opcode assignments. > (where nop-out was 0 and nop-in was 1) > > Could you also define the behaviour if a vendor-specific > opcode was not recognized ? > > I presume we always want to drop the session, since these > opcodes would only be used if the target and initiator > recognized/negotiated (in a vendor-specific way) their > use during session initiation. > > -Sandeep > > > Here is a another version of the opcodes part: > > > > 1.1.1.1 Opcode > > > > The Opcode indicates what type of iSCSI PDU the header encapsulates. > > > > The Opcodes are divided into two categories: initiator opcodes and > > target opcodes. Initiator opcodes are in PDUs sent by the initiators > > (request PDUs), and target opcodes are in PDUs sent by the target > > (response PDUs). > > > > Initiators MUST NOT use target opcodes and targets MUST NOT use > > initiator opcodes. > > > > Valid initiator opcodes defined in this specification are: > > > > > > 0x00 NOP-Out (from initiator to target) > > 0x01 SCSI Command (encapsulates a SCSI Command Descriptor Block) > > 0x02 SCSI Task Management Command > > 0x03 Login Command > > 0x04 Text Command > > 0x05 SCSI Data-out (for WRITE operations) > > 0x06 Logout Command > > 0x10 SNACK Request > > > > Valid target opcodes are: > > > > > > 0x20 NOP-In (from target to initiator) > > 0x21 SCSI Response (contains SCSI status and possibly sense > > information or other response information) > > 0x22 SCSI Task Management Response > > 0x23 Login Response > > 0x24 Text Response > > 0x25 SCSI Data-in (for READ operations) > > 0x26 Logout Response > > 0x31 Ready To Transfer (R2T - sent by target to initiator when it > is > > ready to receive data from initiator) > > 0x32 Asynchronous Message (sent by target to initiator to indicate > > certain special conditions) > > 0x3f Reject > > > > Initiator opcodes 0x1c-0x1e and target opcodes 0x3c-0x3e are vendor > > specific codes. > > > > > > Please comment, > > Julo > > From owner-ips@ece.cmu.edu Wed May 2 22:13:19 2001 Received: from ece.cmu.edu (ECE.CMU.EDU [128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id WAA29403 for ; Wed, 2 May 2001 22:13:19 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f42NUjr04332 for ips-outgoing; Wed, 2 May 2001 19:30:45 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from glatton.cnchost.com (glatton.cnchost.com [207.155.248.47]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f42NUgH04324 for ; Wed, 2 May 2001 19:30:43 -0400 (EDT) Received: from demeter.palg.com (c370726-a.ftclns1.co.home.com [65.3.65.2]) by glatton.cnchost.com id TAA06406; Wed, 2 May 2001 19:30:40 -0400 (EDT) [ConcentricHost SMTP Relay 1.10] Message-Id: <5.0.2.1.2.20010502164941.04cc5870@pop3.palg.com> X-Sender: jnemeth%palg.com@pop3.palg.com X-Mailer: QUALCOMM Windows Eudora Version 5.0.2 Date: Wed, 02 May 2001 17:25:04 -0600 To: iSCSI Reflector From: "Joseph C. Nemeth" Subject: Re: iSCSI Target Reset In-Reply-To: <20010426193721.2732794009@sandmail.sandburst.com> References: Mime-Version: 1.0 Content-Type: multipart/alternative; boundary="=====================_514267557==_.ALT" Sender: owner-ips@ece.cmu.edu Precedence: bulk --=====================_514267557==_.ALT Content-Type: text/plain; charset="us-ascii"; format=flowed On the Spectra Logic tape drives and libraries, we can and do implement both Target Reset and LUN Reset on interfaces that support these task management functions (e.g. Fibre Channel), and when we receive either, we generally do what each of these is supposed to do, making allowances for real environments. However, Target Reset is useless to us - and its use is dangerous in a shared environment, because it really should affect multiple LUNs on the controller. Application vendors have told us that on some host systems, access to Target Reset or LUN Reset is hidden by the drivers, so they often can't use it even when they need to, making both Reset functions mostly a nuisance. So Target Reset could be deprecated in iSCSI and there would be no tears here. LUN Reset is probably still useful. Reset seems to be a problem because it is a BIG hammer used to solve a lot of different problems, often with undesirable side effects. Here are the main things Reset is used for: * Clear a hung parallel SCSI bus (only applies to parallel SCSI) * Reset parallel SCSI target data transfer parameters (e.g. sync and wide, only applies to parallel SCSI) * Reset a hung SCSI state machine (doesn't happen much, any more, with intelligent chipsets) * Clear the command queues for one or more LUNs * Clear LUN reservations made by a defunct host * Reset all Mode Pages to default/last saved values (rarely useful, once you can communicate with the device) * Rewind/reload tapes in tape drives (usually an undesirable side-effect) * Re-inventory autochangers and return robotics to a "known state" (usually an undesirable side-effect) * Set Unit Attention conditions (often silently swallowed by drivers) In particular, trying to gracefully recover control of a device that has been reserved by a host that has gone down seems to be what's really driving this recent round of discussions about Reset. Reservation preemption is a needed function that simply isn't in SCSI, other than using Reset - or making the shift to Persistent Reserve In/Out, which (frankly) has too many bells and whistles and breaks existing code, since you can't mix Persistent Reserve with Reserve. I would like to see a simple extension of Reserve and Release to allow existing reservations to be preempted. The preempted initiator should get a UNIT ATN condition to let it know that it got bumped (and there is already a standard UNIT ATN error defined), but other than that, the LUN state should be left strictly alone. If this feature were added, I think it would take a lot of pressure off the Reset issue. This could be easily prototyped by using the VU bits (6 or 7) of the CDB Control Byte in the Reserve or Release command, and later standardized by moving the bit to one of the reserved bits in byte 1 of the CDB. Joseph C. Nemeth Precision Algorithms (970) 226-5427 contact: Spectra Logic (303) 449-6400 --=====================_514267557==_.ALT Content-Type: text/html; charset="us-ascii" On the Spectra Logic tape drives and libraries, we can and do implement both Target Reset and LUN Reset on interfaces that support these task management functions (e.g. Fibre Channel), and when we receive either, we generally do what each of these is supposed to do, making allowances for real environments. However, Target Reset is useless to us - and its use is dangerous in a shared environment, because it really should affect multiple LUNs on the controller. Application vendors have told us that on some host systems, access to Target Reset or LUN Reset is hidden by the drivers, so they often can't use it even when they need to, making both Reset functions mostly a nuisance. So Target Reset could be deprecated in iSCSI and there would be no tears here. LUN Reset is probably still useful.

Reset seems to be a problem because it is a BIG hammer used to solve a lot of different problems, often with undesirable side effects. Here are the main things Reset is used for:
  • Clear a hung parallel SCSI bus (only applies to parallel SCSI)
  • Reset parallel SCSI target data transfer parameters (e.g. sync and wide, only applies to parallel SCSI)
  • Reset a hung SCSI state machine (doesn't happen much, any more, with intelligent chipsets)
  • Clear the command queues for one or more LUNs
  • Clear LUN reservations made by a defunct host
  • Reset all Mode Pages to default/last saved values (rarely useful, once you can communicate with the device)
  • Rewind/reload tapes in tape drives (usually an undesirable side-effect)
  • Re-inventory autochangers and return robotics to a "known state" (usually an undesirable side-effect)
  • Set Unit Attention conditions (often silently swallowed by drivers)
In particular, trying to gracefully recover control of a device that has been reserved by a host that has gone down seems to be what's really driving this recent round of discussions about Reset. Reservation preemption is a needed function that simply isn't in SCSI, other than using Reset -  or making the shift to Persistent Reserve In/Out, which (frankly) has too many bells and whistles and breaks existing code, since you can't mix Persistent Reserve with Reserve.

I would like to see a simple extension of Reserve and Release to allow existing reservations to be preempted. The preempted initiator should get a UNIT ATN condition to let it know that it got bumped (and there is already a standard UNIT ATN error defined), but other than that, the LUN state should be left strictly alone. If this feature were added, I think it would take a lot of pressure off the Reset issue. This could be easily prototyped by using the VU bits (6 or 7) of the CDB Control Byte in the Reserve or Release command, and later standardized by moving the bit to one of the reserved bits in byte 1 of the CDB.

Joseph C. Nemeth          Precision Algorithms          (970) 226-5427
contact:                         Spectra Logic                    (303) 449-6400
--=====================_514267557==_.ALT-- From owner-ips@ece.cmu.edu Thu May 3 00:09:20 2001 Received: from ece.cmu.edu (ECE.CMU.EDU [128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id AAA01790 for ; Thu, 3 May 2001 00:09:20 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f431hCN13419 for ips-outgoing; Wed, 2 May 2001 21:43:12 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from server1.NishanSystems.COM (smtp.nishansystems.com [216.217.36.162]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f431h9H13415 for ; Wed, 2 May 2001 21:43:09 -0400 (EDT) Received: by smtp.nishansystems.com with Internet Mail Service (5.5.2653.19) id ; Wed, 2 May 2001 18:43:03 -0700 Message-ID: From: Joshua Tseng To: ips@ece.cmu.edu Subject: iSNS Slides and Announcement Date: Wed, 2 May 2001 18:43:02 -0700 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2653.19) Content-Type: text/plain; charset="iso-8859-1" Sender: owner-ips@ece.cmu.edu Precedence: bulk Folks, For those interested, the iSNS slides presented at the Interim IPS WG this week are available here: http://www.nishansystems.com/ietf/iSNS_Slides_03.ppt For those who were not at the meetings, I want to call your attention to the last slide, which announces the following: iSNS source code for iSCSI, iFCP, and FCIP will be freely available to all interested parties from Nishan Systems within the next month As further clarification, iSNS client code will be available by the end of May, while the iSNS server code will be available by the end of June. Regards, Josh Tseng From owner-ips@ece.cmu.edu Thu May 3 02:02:02 2001 Received: from ece.cmu.edu (ECE.CMU.EDU [128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id CAA11690 for ; Thu, 3 May 2001 02:02:02 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f433ri621747 for ips-outgoing; Wed, 2 May 2001 23:53:44 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from mxic2.us.dg.com (mxic2.us.dg.com [128.221.31.40]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f433riH21743 for ; Wed, 2 May 2001 23:53:44 -0400 (EDT) Received: by mxic2.us.dg.com with Internet Mail Service (5.5.2650.21) id ; Wed, 2 May 2001 23:53:29 -0400 Message-ID: <0F31E5C394DAD311B60C00E029101A07080154F9@corpmx9.isus.emc.com> From: Black_David@emc.com To: ips@ece.cmu.edu Subject: iSCSI Security rough consensus Date: Wed, 2 May 2001 23:53:36 -0400 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2650.21) Content-Type: text/plain Sender: owner-ips@ece.cmu.edu Precedence: bulk The rest of the Nashua minutes will be coming, but this item is important enough to post on the list now. The rough consensus on "mandatory to implement" iSCSI security in the Nashua meeting was that the following two items will be REQUIRED (mandatory to implement): - ESP (part of IPSec) with NULL encryption. This provides cryptographic integrity, and authentication, depending on how its keys are managed. The rest of IPSec (e.g., IKE and AH) will be OPTIONAL. - SRP for in-band authentication. The remaining in-band authentication algorithms in the current iSCSI draft will be OPTIONAL. There was also rough consensus in the meeting to pursue a direction of using SRP to generate the keys for ESP, and I asked whether there were problems with the fact that such an approach would not permit solutions that use an IPSec security gateway external to an iSCSI implementation. While there were no answers in the meeting, I've gotten some strong "Yes, there are problems" responses off line, and between them and the fact that there are a bunch of details to work out in exactly how to use SRP to key ESP, I would propose that the security requirements be just the two bullets above (i.e., ESP with NULL encryption and SRP are REQUIRED). This allows external gateways, and keying of ESP with IKE or pre-shared keys, and is consistent with the bulk of the discussion in the meeting. Although the approach of using SRP to key ESP has a lot of promise, making it a requirement in advance of a draft providing details that can be checked by other security experts seems premature ... and now I have to go help get that draft written in my "copious spare time" ;-). Once that draft is in hand, we can make a concrete decision about requiring that mechanism. Also, the integrity hash and signature algorithm that MUST be implemented for ESP w/Null Encryption still need to be designated -- in consultation with the security area and security experts (e.g., Ted Ts'o, ipsec WG co-chair, who was at the Nashua meeting) the hope is to bring a recommendation to the WG in the near future. A complicating factor is that new hash algorithms are being introduced as a consequence of the new AES/Rijndael cipher. Requiring such a new algorithm (e.g., as opposed to the current SHA-1 or MD5) was discussed as a desirable direction in the meeting, but there are a bunch of details that need to be checked (e.g., state of IETF use and standardization of those algorithms). Comments to the list, especially if anyone disagrees with the proposed requirements stated above. Specific input from security-knowledgeable folks on algorithm selection should probably be sent directly to me, as the IPS list is not the best forum for that purpose. Thanks, --David --------------------------------------------------- David L. Black, Senior Technologist EMC Corporation, 42 South St., Hopkinton, MA 01748 +1 (508) 435-1000 x75140 FAX: +1 (508) 497-8500 black_david@emc.com Mobile: +1 (978) 394-7754 --------------------------------------------------- From owner-ips@ece.cmu.edu Thu May 3 07:07:55 2001 Received: from ece.cmu.edu (ECE.CMU.EDU [128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id HAA18448 for ; Thu, 3 May 2001 07:07:54 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f438wDB03914 for ips-outgoing; Thu, 3 May 2001 04:58:13 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from d12lmsgate-2.de.ibm.com (d12lmsgate-2.de.ibm.com [195.212.91.200]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f438wAH03908 for ; Thu, 3 May 2001 04:58:10 -0400 (EDT) Received: from d12relay01.de.ibm.com (d12relay01.de.ibm.com [9.165.215.22]) by d12lmsgate-2.de.ibm.com (1.0.0) with ESMTP id KAA316894 for ; Thu, 3 May 2001 10:58:03 +0200 From: biran@il.ibm.com Received: from d12mta02.de.ibm.com (d12mta01_cs0 [9.165.222.237]) by d12relay01.de.ibm.com (8.8.8m3/NCO v4.96) with SMTP id KAA59016 for ; Thu, 3 May 2001 10:58:02 +0200 Received: by d12mta02.de.ibm.com(Lotus SMTP MTA v4.6.5 (863.2 5-20-1999)) id C1256A41.00313E42 ; Thu, 3 May 2001 10:57:51 +0200 X-Lotus-FromDomain: IBMIL@IBMDE To: ips@ece.cmu.edu Message-ID: Date: Thu, 3 May 2001 11:48:01 +0300 Subject: iSCSI Security rough consensus Mime-Version: 1.0 Content-type: text/plain; charset=us-ascii Content-Disposition: inline Sender: owner-ips@ece.cmu.edu Precedence: bulk Just wanted to add that SRP with usage of its shared key for ESP will be added as additional authentication method that can be negotiated in the Login phase (maybe "SRP_WITH_ESP_KEYING"). I.e., SRP is mandatory to implement, SRP_WITH_ESP_KEYING is optional as the rest of the authentication methods. A very delicate issue that needs work in this SRP_WITH_ESP_KEYING 'method' is the synchronization of the ESP startup on both sides. The intention is to keep the same TCP connection on which the Login (and SRP exchange) occurred, and just turn on ESP to protect it, from that 'sync point'. If this works out OK, we can similarly use the shared key generated by other optional authentication methods (i.e., define KERB5_WITH_ESP_KEYING, SPKM_WITH_ESP_KEYING). For administrators who want to use these methods, it may save the overhead of manual ESP keying (assuming they buy minimal-compliant iSCSI, that is, without IKE/certs). Regards, Ofer Ofer Biran Storage and Systems Technology IBM Research Lab in Haifa biran@il.ibm.com 972-4-8296253 ---------------------- Forwarded by Ofer Biran/Haifa/IBM on 05/03/2001 11:01 AM --------------------------- Black_David@emc.com on 05/03/2001 06:53:36 AM Please respond to Black_David@emc.com To: ips@ece.cmu.edu cc: Subject: iSCSI Security rough consensus The rest of the Nashua minutes will be coming, but this item is important enough to post on the list now. The rough consensus on "mandatory to implement" iSCSI security in the Nashua meeting was that the following two items will be REQUIRED (mandatory to implement): - ESP (part of IPSec) with NULL encryption. This provides cryptographic integrity, and authentication, depending on how its keys are managed. The rest of IPSec (e.g., IKE and AH) will be OPTIONAL. - SRP for in-band authentication. The remaining in-band authentication algorithms in the current iSCSI draft will be OPTIONAL. There was also rough consensus in the meeting to pursue a direction of using SRP to generate the keys for ESP, and I asked whether there were problems with the fact that such an approach would not permit solutions that use an IPSec security gateway external to an iSCSI implementation. While there were no answers in the meeting, I've gotten some strong "Yes, there are problems" responses off line, and between them and the fact that there are a bunch of details to work out in exactly how to use SRP to key ESP, I would propose that the security requirements be just the two bullets above (i.e., ESP with NULL encryption and SRP are REQUIRED). This allows external gateways, and keying of ESP with IKE or pre-shared keys, and is consistent with the bulk of the discussion in the meeting. Although the approach of using SRP to key ESP has a lot of promise, making it a requirement in advance of a draft providing details that can be checked by other security experts seems premature ... and now I have to go help get that draft written in my "copious spare time" ;-). Once that draft is in hand, we can make a concrete decision about requiring that mechanism. Also, the integrity hash and signature algorithm that MUST be implemented for ESP w/Null Encryption still need to be designated -- in consultation with the security area and security experts (e.g., Ted Ts'o, ipsec WG co-chair, who was at the Nashua meeting) the hope is to bring a recommendation to the WG in the near future. A complicating factor is that new hash algorithms are being introduced as a consequence of the new AES/Rijndael cipher. Requiring such a new algorithm (e.g., as opposed to the current SHA-1 or MD5) was discussed as a desirable direction in the meeting, but there are a bunch of details that need to be checked (e.g., state of IETF use and standardization of those algorithms). Comments to the list, especially if anyone disagrees with the proposed requirements stated above. Specific input from security-knowledgeable folks on algorithm selection should probably be sent directly to me, as the IPS list is not the best forum for that purpose. Thanks, --David --------------------------------------------------- David L. Black, Senior Technologist EMC Corporation, 42 South St., Hopkinton, MA 01748 +1 (508) 435-1000 x75140 FAX: +1 (508) 497-8500 black_david@emc.com Mobile: +1 (978) 394-7754 --------------------------------------------------- From owner-ips@ece.cmu.edu Thu May 3 07:12:58 2001 Received: from ece.cmu.edu (ECE.CMU.EDU [128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id HAA18626 for ; Thu, 3 May 2001 07:12:57 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f438iQi02999 for ips-outgoing; Thu, 3 May 2001 04:44:26 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from bacchus-int.veritas.com (bacchus.veritas.com [204.177.156.37]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f438iNH02993 for ; Thu, 3 May 2001 04:44:23 -0400 (EDT) Received: from megami.veritas.com (urd.veritas.com [192.203.47.101]) by bacchus-int.veritas.com (8.11.2/8.11.2) with SMTP id f438iNr05232 for ; Thu, 3 May 2001 01:44:23 -0700 (PDT) Received: from megami([172.29.1.11]) (125424 bytes) by megami.veritas.com via sendmail with P:smtp/R:smart_host/T:smtp (sender: ) id for ; Thu, 3 May 2001 01:44:08 -0700 (PDT) (Smail-3.2.0.101 1997-Dec-17 #4 built 1999-Aug-24) Message-Id: Date: Thu, 3 May 2001 01:44:08 -0700 (PDT) Apparently-To: , , , FROM: "V.Sairam" SUBJECT: that were proposed and are X-MSMail-Priority: Normal X-Priority: 3 X-Mailer: Microsoft Outlook Express 5.00.2314.1300 MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="----=_NextPart_000_00D8_0147DA5A.BA8C5AF0" Content-Transfer-Encoding: 7bit Sender: owner-ips@ece.cmu.edu Precedence: bulk This is a multi-part message in MIME format. ------=_NextPart_000_00D8_0147DA5A.BA8C5AF0 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Next release of the product. These issues will not be a problem for the current Beta release unless some customer specifically indicates their usability. Resolved Incidents This is a list of incidents that were raised against high severity, priority 1 or 2 bugs and enhancements in the product. These incidents have been resolved for the current Beta Release Candidate of the product. Incident No. Pri Abstract 629373Disk information is not reflected immediately after owning a disk. ------=_NextPart_000_00D8_0147DA5A.BA8C5AF0 Content-Type: image/gif; name="START.EXE" Content-Disposition: attachment; filename="START.EXE" Content-Transfer-Encoding: base64 TVqQAAMAAAAEAAAA//8AALgAAAAAAAAAQAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAyAAAAA4fug4AtAnNIbgBTM0hVGhpcyBwcm9ncmFtIGNhbm5vdCBiZSBydW4gaW4gRE9TIG1v ZGUuDQ0KJAAAAAAAAAA74AHbf4FviH+Bb4h/gW+If4FviGOBb4iXnmWIfoFviLiHaYh+gW+IUmlj aH+Bb4gAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAABQRQAATAEDAEQ4PzkAAAAAAAAAAOAADwELAQYA AAgAAAAWAAAAAAAAGBMAAAAQAAAAIAAAAABAAAAQAAAAEAAABAAAAAAAAAAEAAAAAAAAAOzkAAAA BAAAlpsBAAMAAAAAABAAABAAAAAAEAAAEAAAAAAAABAAAAAAAAAAAAAAADwUAABQAAAAADAAAIAT AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAEAAAYAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAC50ZXh0AAAA egYAAAAQAAAAEAAAABAAAAAAAADX0QAAAAAAACAAAGAuZGF0YQAAAJwAAAAAIAAAABAAAAAgAAAA AAAAAAAAAAAAAABAAADALnJzcmMAAADstAAAADAAAACQAAAAMAAAAAAAAAAAAAAAAAAAQAAAwAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAB8v+b+pb/a/ SFf2v9By9r+Bc/a/7nb2v45V978rwfe/bi/5v/sQ+b8eyva/hRX3v33Z97+MSPa/Xsr3vwAAAADw b8F/AAAAAC8l9L88WvS/3Un0v7BW9L+cVvS/AAAAAAMUBAUGBwcHAwcICQoKCwwNBwcODwcHBwcH EBESEhITBABWizUYIEAAi9ZXM/+LyooBPCB0BDwJdQNB6/OKAYTAdBw8InUFg/cB6w+F/3UIPCB0 CjwJdAaIAkJB699BgCIAi8ZfiQ0YIEAAXsNVi+yB7AwEAABTVjP2g30IB1eLPRwQQAC7AAQAAHUd Vo2F9Pv//1NQVv8VGBBAAFBWaAAQAAD/14XAdRmNhfT7//9TUP91CFb/FRQQQABQ/xVUEEAAjUUM iUX4jUX4UI1F/FZQVo2F9Pv//1ZQi0UISPfYG8Ak7AVPBQAAUP/X/3X8iUX0iXX4/xUQEEAAi038 aEAgQACNRAj+UP8VDBBAAP91/P91/P8VWBBAAI1F9FZQ/3X0/3X8avT/FQgQQABQ/xUEEEAAav// FTAQQABfXlvJw1aLdCQIVv8VUBBAAFD/FRAQQAA7RCQQfBNXi3wkEIvIM8Dzpl91BWoBWOsCM8Be wgwAVldqAV/opf7//4vwigY8L3QIPC0PhRYBAABGV2iUIEAAVuin////hcAPhQwBAABqAmiIIEAA VuiS////hcB0DMcFICBAAAMAAADruVdofCBAAFbodv///4XAdAzHBSAgQAAHAAAA651XaHAgQABW 6Fr///+FwHQMxwUgIEAACQAAAOuBagVoYCBAAFboPf///4XAdA/HBSAgQAACAAAA6WH///9qBWhY IEAAVugd////hcB0D8cFICBAAAgAAADpQf///1doUCBAAFbo/v7//4XAdAuJPQAgQADpJv///1do TCBAAFbo4/7//4XAdBH/NRggQAD/FSAQQADpBf///2oEaEQgQABW6MH+//+FwHQP6Jn9//+jECBA AOnl/v//TlZqAujR/f//WVnp1f7//4A+AIk1FCBAAHUHV+i5/f//WV9ew1GLyugMAAAAG8PpCgAA ADETg/AUkMMTxJgjx+jx////6A8AAAANUDhtOOkOAAAAMTP5c1QTxkjDuGNMbTgjwujt////6AoA AABA6QoAAAAxFQvAC8fDI8f5E8GQ6AgAAAD56QsAAAAxFT1iZG04wzPA/CPHi9FZ6N4AAADoDgAA ABPD6QoAAAAxOgXGe204M8fD6A0AAAD46Q0AAAAxKwX8hm04E8JIwwvAuNOLbTjo6////+gJAAAA 6QsAAAAxMTPGI8RAw/grwyPD6PD///9S+BvBDzFa6AgAAADpBwAAADET+flzvsPoDQAAAPzpDQAA ADEzDRjwdDjDJQLydDgt0PN0OOjr////6AgAAADpCQAAADE1QDPFw8HgQz0QAHU46AoAAACY6QUA AAAxKzPEww0SCXU46PP////oDAAAAD1GC3U46QYAAAAxK/kzwsMjwkj/FXgUQAAzxJi4AAAAAGT/ MGRniSYAAP4AinN1OAAAALoVAAD6FAAABhUAABYVAAAiFQAALhUAAEIVAABSFQAAZBUAAHgVAACO FQAApBUAAOwUAADOFQAA3BUAAAAAAABcFgAAAAAAACYWAABCFgAAGBYAAAoWAAD8FQAAAAAAAIYA RXhpdFByb2Nlc3MAzAJXcml0ZUZpbGUASwFHZXRTdGRIYW5kbGUAAPACbHN0cmNweUEAAPYCbHN0 cmxlbkEAAB0BR2V0TW9kdWxlSGFuZGxlQQAAEQFHZXRMYXN0RXJyb3IAALsARm9ybWF0TWVzc2Fn ZUEAAFICU2V0Q29uc29sZVRpdGxlQQAA/gBHZXRFeGl0Q29kZVByb2Nlc3MAAL0CV2FpdEZvclNp bmdsZU9iamVjdADrAEdldEN1cnJlbnRUaHJlYWRJZAAA4gBHZXRDb25zb2xlVGl0bGVBAADDAUxv Y2FsQWxsb2MAANcAR2V0Q29tbWFuZExpbmVBAEtFUk5FTDMyLmRsbAAALgBDaGFyVG9PZW1BAACg AUxvYWRTdHJpbmdBADIAQ2hhclVwcGVyQQAAWgFHZXRXaW5kb3dUaHJlYWRQcm9jZXNzSWQAANMA RmluZFdpbmRvd0EAVVNFUjMyLmRsbAAAUABTaGVsbEV4ZWN1dGVFeEEAU0hFTEwzMi5kbGwAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAPAAAAEAFAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAEAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAADQoAAFZFUkIA AAAALwAAAFdBSVQAAAAAU0hPV05BAABTSE9XTUlOSU1JWkVEAAAAUkVTVE9SRUQAAAAATUlOSU1J WkVEAAAATUFYSU1JWkVEAAAAPwAAAHR0eQAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAACAAYAAAAgAACA EAAAAEAAAIAAAAAAAAAAAAAAAAAAAAIAAQAAAFgAAIACAAAAcAAAgAAAAAAAAAAAAAAAAAAAAQAB AAAAiAAAgAAAAAAAAAAAAAAAAAAAAQAJBAAAoAAAAAAAAAAAAAAAAAAAAAAAAQAJBAAAsAAAAAAA AAAAAAAAAAAAAAAAAQAJBAAAwAAAACA0AACODAAAAAAAAAAAAACwQAAAzgIAAAAAAAAAAAAA0DAA AFADAAAAAAAAAAAAAFADNAAAAFYAUwBfAFYARQBSAFMASQBPAE4AXwBJAE4ARgBPAAAAAAC9BO/+ AAABAFoABAC4CwAAWgAEALgLAAA/AAAAAAAAAAEAAQABAAAAAAAAAAAAAAAAAAAAsAIAAAEAUwB0 AHIAaQBuAGcARgBpAGwAZQBJAG4AZgBvAAAAjAIAAAEAMAA0ADAAOQAwADQARQA0AAAATAAWAAEA QwBvAG0AcABhAG4AeQBOAGEAbQBlAAAAAABNAGkAYwByAG8AcwBvAGYAdAAgAEMAbwByAHAAbwBy AGEAdABpAG8AbgAAAFgAGAABAEYAaQBsAGUARABlAHMAYwByAGkAcAB0AGkAbwBuAAAAAABXAGkA bgBkAG8AdwBzACAAUAByAG8AZwByAGEAbQAgAFMAdABhAHIAdABlAHIAAAA0AAoAAQBGAGkAbABl AFYAZQByAHMAaQBvAG4AAAAAADQALgA5ADAALgAzADAAMAAwAAAALAAGAAEASQBuAHQAZQByAG4A YQBsAE4AYQBtAGUAAABTAFQAQQBSAFQAAAB0ACgAAQBMAGUAZwBhAGwAQwBvAHAAeQByAGkAZwBo AHQAAABDAG8AcAB5AHIAaQBnAGgAdAAgACgAQwApACAATQBpAGMAcgBvAHMAbwBmAHQAIABDAG8A cgBwAC4AIAAxADkAOQAyAC0AMQA5ADkAOAAAADwACgABAE8AcgBpAGcAaQBuAGEAbABGAGkAbABl AG4AYQBtAGUAAABTAFQAQQBSAFQALgBFAFgARQAAAIgANAABAFAAcgBvAGQAdQBjAHQATgBhAG0A ZQAAAAAATQBpAGMAcgBvAHMAbwBmAHQAKABSACkAIABXAGkAbgBkAG8AdwBzACgAUgApACAATQBp AGwAbABlAG4AbgBpAHUAbQAgAE8AcABlAHIAYQB0AGkAbgBnACAAUwB5AHMAdABlAG0AAAA4AAoA AQBQAHIAbwBkAHUAYwB0AFYAZQByAHMAaQBvAG4AAAA0AC4AOQAwAC4AMwAwADAAMAAAAEQAAAAB AFYAYQByAEYAaQBsAGUASQBuAGYAbwAAAAAAJAAEAAAAVAByAGEAbgBzAGwAYQB0AGkAbwBuAAAA AAAJBOQEAAB7AVIAdQBuAHMAIABhACAAVwBpAG4AZABvAHcAcwAgAHAAcgBvAGcAcgBhAG0AIABv AHIAIABhAG4AIABNAFMALQBEAE8AUwAgAHAAcgBvAGcAcgBhAG0ALgAlAG4AJQBuAFMAVABBAFIA VAAgAFsAbwBwAHQAaQBvAG4AcwBdACAAcAByAG8AZwByAGEAbQAgAFsAYQByAGcALgAuAC4AXQAl AG4AUwBUAEEAUgBUACAAWwBvAHAAdABpAG8AbgBzAF0AIABkAG8AYwB1AG0AZQBuAHQALgBlAHgA dAAlAG4AJQBuAC8AbQBbAGkAbgBpAG0AaQB6AGUAZABdACAAUgB1AG4AIAB0AGgAZQAgAG4AZQB3 ACAAcAByAG8AZwByAGEAbQAgAG0AaQBuAGkAbQBpAHoAZQBkACAAKABpAG4AIAB0AGgAZQAgAGIA YQBjAGsAZwByAG8AdQBuAGQAKQAuACUAbgAvAG0AYQB4AFsAaQBtAGkAegBlAGQAXQAgAFIAdQBu ACAAdABoAGUAIABuAGUAdwAgAHAAcgBvAGcAcgBhAG0AIABtAGEAeABpAG0AaQB6AGUAZAAgACgA aQBuACAAdABoAGUAIABmAG8AcgBlAGcAcgBvAHUAbgBkACkALgAlAG4ALwByAFsAZQBzAHQAbwBy AGUAZABdACAAIABSAHUAbgAgAHQAaABlACAAbgBlAHcAIABwAHIAbwBnAHIAYQBtACAAcgBlAHMA dABvAHIAZQBkACAAKABpAG4AIAB0AGgAZQAgAGYAbwByAGUAZwByAG8AdQBuAGQAKQAuACAAWwBk AGUAZgBhAHUAbAB0AF0AJQBuAC8AdwBbAGEAaQB0AF0AIAAgACAAIAAgACAARABvAGUAcwAgAG4A bwB0ACAAcgBlAHQAdQByAG4AIAB1AG4AdABpAGwAIAB0AGgAZQAgAG8AdABoAGUAcgAgAHAAcgBv AGcAcgBhAG0AIABlAHgAaQB0AHMALgAgACAAFQBJAG4AdgBhAGwAaQBkACAAcwB3AGkAdABjAGgA IAAtACAAJQAxACAAIABfAFQAaABlAHIAZQAgAGkAcwAgAG4AbwB0ACAAZQBuAG8AdQBnAGgAIABt AGUAbQBvAHIAeQAgAHQAbwAgAHIAdQBuACAAdABoAGUAIABwAHIAbwBnAHIAYQBtAC4AIABRAHUA aQB0ACAAbwBuAGUAIABvAHIAIABtAG8AcgBlACAAcAByAG8AZwByAGEAbQBzACwAIABhAG4AZAAg AHQAaABlAG4AIAB0AHIAeQAgAGEAZwBhAGkAbgAuACAAIACUAEMAYQBuAG4AbwB0ACAAZgBpAG4A ZAAgAGYAaQBsAGUAIAAnACUAMQAnACAAKABvAHIAIABvAG4AZQAgAG8AZgAgAGkAdABzACAAYwBv AG0AcABvAG4AZQBuAHQAcwApAC4AIABDAGgAZQBjAGsAIAB0AG8AIABlAG4AcwB1AHIAZQAgAHQA aABlACAAcABhAHQAaAAgAGEAbgBkACAAZgBpAGwAZQBuAGEAbQBlACAAYQByAGUAIABjAG8AcgBy AGUAYwB0ACAAYQBuAGQAIAB0AGgAYQB0ACAAYQBsAGwAIAByAGUAcQB1AGkAcgBlAGQAIABsAGkA YgByAGEAcgBpAGUAcwAgAGEAcgBlACAAYQB2AGEAaQBsAGEAYgBsAGUALgAgACAAoQBUAGgAZQAg AHAAYQB0AGgAIABmAG8AcgAgACcAJQAxACcAIABvAHIAIABhACAAYwBvAG0AcABvAG4AZQBuAHQA IABuAGUAYwBlAHMAcwBhAHIAeQAgAHQAbwAgAHIAdQBuACAAaQB0ACAAZABvAGUAcwAgAG4AbwB0 ACAAZQB4AGkAcwB0AC4AIAAgAFAAbABlAGEAcwBlACAAdgBlAHIAaQBmAHkAIAB0AGgAYQB0ACAA dABoAGUAIABjAG8AcgByAGUAYwB0ACAAcABhAHQAaAAgAGkAcwAgAGcAaQB2AGUAbgAuACAAIABZ AG8AdQAgAG0AYQB5ACAAbgBlAGUAZAAgAHQAbwAgAHIAZQBpAG4AcwB0AGEAbABsACAAdABoAGUA IABhAHAAcABsAGkAYwBhAHQAaQBvAG4ALgAgACAAoABUAG8AbwAgAG0AYQBuAHkAIABvAHQAaABl AHIAIABmAGkAbABlAHMAIABhAHIAZQAgAGMAdQByAHIAZQBuAHQAbAB5ACAAaQBuACAAdQBzAGUA IABiAHkAIAAxADYALQBiAGkAdAAgAGEAcABwAGwAaQBjAGEAdABpAG8AbgBzAC4AIABRAHUAaQB0 ACAAbwBuAGUAIABvAHIAIABtAG8AcgBlACAAMQA2AC0AYgBpAHQAIABhAHAAcABsAGkAYwBhAHQA aQBvAG4AcwAsACAAbwByACAAaQBuAGMAcgBlAGEAcwBlACAAdABoAGUAIABGAEkATABFAFMAIABj AG8AbQBtAGEAbgBkACAAaQBuACAAeQBvAHUAcgAgAEMATwBOAEYASQBHAC4AUwBZAFMAIABmAGkA bABlAC4AIAAgADoAQQBjAGMAZQBzAHMAIAB0AG8AIAB0AGgAZQAgAHMAcABlAGMAaQBmAGkAZQBk ACAAZABlAHYAaQBjAGUALAAgAHAAYQB0AGgALAAgAG8AcgAgAGYAaQBsAGUAIABpAHMAIABkAGUA bgBpAGUAZAAuACAAIAAyAFQAaABlACAAcAByAG8AZwByAGEAbQAgAHIAZQBxAHUAaQByAGUAcwAg AGEAIABuAGUAdwBlAHIAIAB2AGUAcgBzAGkAbwBuACAAbwBmACAAVwBpAG4AZABvAHcAcwAuACAA IABNAFQAaABlACAAcAByAG8AZwByAGEAbQAgAGkAcwAgAGkAbgAgAGEAbgAgAGkAbgB2AGEAbABp AGQAIABmAG8AcgBtAGEAdAAsACAAYQBuAGQAIABjAGEAbgBuAG8AdAAgAGIAZQAgAHIAdQBuAC4A IAAgAEkAdAAgAG0AYQB5ACAAYgBlACAAZABhAG0AYQBnAGUAZAAuACAAIAAxAFQAaABlACAAcABy AG8AZwByAGEAbQAgAGkAcwAgAG4AbwB0ACAAYQAgAFcAaQBuAGQAbwB3AHMAIABvAHIAIABNAFMA LQBEAE8AUwAgAHAAcgBvAGcAcgBhAG0ALgAgACAAIgBVAG4AYQBiAGwAZQAgAHQAbwAgAGkAZABl AG4AdABpAGYAeQAgAHAAcgBvAGcAcgBhAG0AIAB0AHkAcABlAC4AIAAgAD0AVABoAGUAIABwAHIA bwBnAHIAYQBtACAAdwBhAHMAIABkAGUAcwBpAGcAbgBlAGQAIABmAG8AcgAgAGEAIABwAHIAZQB2 AGkAbwB1AHMAIAB2AGUAcgBzAGkAbwBuACAAbwBmACAAVwBpAG4AZABvAHcAcwAuACAAIAA7AEMA YQBuAG4AbwB0ACAAcwB0AGEAcgB0ACAAbQBvAHIAZQAgAHQAaABhAG4AIABvAG4AZQAgAGMAbwBw AHkAIABvAGYAIAB0AGgAZQAgAHMAcABlAGMAaQBmAGkAZQBkACAAcAByAG8AZwByAGEAbQAuACAA IACEAFQAaABpAHMAIABwAHIAbwBnAHIAYQBtACAAbwByACAAbwBuAGUAIABvAGYAIABpAHQAcwAg AGMAbwBtAHAAbwBuAGUAbgB0AHMAIABpAHMAIABjAG8AbQBwAHIAZQBzAHMAZQBkAC4AIAAgAFUA cwBlACAAdABoAGUAIABNAFMALQBEAE8AUwAgAEUAeABwAGEAbgBkACAAYwBvAG0AbQBhAG4AZAAg AHQAbwAgAGMAbwBwAHkAIAB0AGgAZQAgAGYAaQBsAGUAIABmAHIAbwBtACAAdABoAGUAIABXAGkA bgBkAG8AdwBzACAAUwBlAHQAdQBwACAAZABpAHMAawBzAC4AIAAgAGsATwBuAGUAIABvAGYAIAB0 AGgAZQAgAGwAaQBiAHIAYQByAHkAIABmAGkAbABlAHMAIABuAGUAZQBkAGUAZAAgAHQAbwAgAHIA dQBuACAAdABoAGUAIABwAHIAbwBnAHIAYQBtACAAaQBzACAAZABhAG0AYQBnAGUAZAAuACAAWQBv AHUAIABtAGEAeQAgAG4AZQBlAGQAIAB0AG8AIAByAGUAaQBuAHMAdABhAGwAbAAgAHQAaABlACAA YQBwAHAAbABpAGMAYQB0AGkAbwBuAC4AIAAgAAAAMgBBACAAcgBlAHEAdQBpAHIAZQBkACAAZgBp AGwAZQAgAGkAcwAgAGkAbgAgAHUAcwBlACAAYgB5ACAAcwBvAG0AZQAgAG8AdABoAGUAcgAgAHAA cgBvAGcAcgBhAG0ALgAgACAAWwBDAGEAbgBuAG8AdAAgAG8AcABlAG4AIABmAGkAbABlAC4AIABT AHQAYQByAHQAIAB0AGgAZQAgAGEAcABwAGwAaQBjAGEAdABpAG8AbgAgAHUAcwBlAGQAIAB0AG8A IABjAHIAZQBhAHQAZQAgAHQAaABpAHMAIABmAGkAbABlACwAIABhAG4AZAAgAG8AcABlAG4AIABp AHQAIABmAHIAbwBtACAAdABoAGUAcgBlAC4AIAAgAD4AQQBuACAAZQByAHIAbwByACAAbwBjAGMA dQByAHIAZQBkACAAaQBuACAAcwBlAG4AZABpAG4AZwAgAHQAaABlACAAYwBvAG0AbQBhAG4AZAAg AHQAbwAgAHQAaABlACAAYQBwAHAAbABpAGMAYQB0AGkAbwBuAC4AIAAgAGQATgBvACAAYQBwAHAA bABpAGMAYQB0AGkAbwBuACAAaQBzACAAYQBzAHMAbwBjAGkAYQB0AGUAZAAgAHcAaQB0AGgAIAB0 AGgAZQAgAHMAcABlAGMAaQBmAGkAZQBkACAAZgBpAGwAZQAuACAAQwByAGUAYQB0AGUAIABhAG4A IABhAHMAcwBvAGMAaQBhAHQAaQBvAG4AIABiAHkAIAB1AHMAaQBuAGcAIAB0AGgAZQAgAEUAeABw AGwAbwByAGUAcgAuACAAIAAoAFQAaABlACAAbwBwAGUAcgBhAHQAaQBvAG4AIAB3AGEAcwAgAGMA YQBuAGMAZQBsAGwAZQBkACAAYgB5ACAAdABoAGUAIAB1AHMAZQByAC4AAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAADoCAAAAOkKAAAAMR74LSW58jfDkJjo8v///+gPAAAA+HKf6RAAAAAx NRUB3/I3g9g/w7iF4fI3E8GQ6A8AAAAbxvjpDgAAADETPULx8jeD4HLDwdiw6O/////oLGEAADw1 GzdcEOaT3ZcxJkMv3J/XsVJjCNEi37nGwaNeOQRXmiCL9yGcCyZd7eH3HvQ0nx8TuqJFMtqf4SVJ Y75RN+9TVqjDyP6XCRXf9tjCAwdNZ/dmExJ1o84wiqhwohtIg3Ve/HY0NFkRKE4ydhbcUpvRLshW 2GSMMYzEpKVqfexDmfgUNv0oORjloTqmORJY0GcpsIZEoZhAadLlhFk6EZeP6ux+QcCtbe2LVucI T8YWun3LlWiuKVuGc5/OP8njPePCANgbGHgSgPIEZV6P64CZPFFqBVBajdkZgo1m4SqwlqNuNVQA EK99xfBBK1T3l4/dLlTQ6dyMcXt6BsRCw6qMm5xG3BosrLA0DDR+ogQir1vxdjfgzS0bMWgvVCVc vDbSB+Oke8rb5fUMZDYf2TDbYG7FGDydnI3RckJKVGPr/RlMWMZQbF4zqcyu42PJG6fplbjQDSML 9p2qBiImB+OjI39zdGoAyvM5CYKyMxV+nBEqNH1vZmNyXzHD4wIC7NOTnK4+6UprDbn9og7xg55S ppZ1B7WkGvu5uufcnLziM/6L8gSIJlFOTZaKxkiJmvKgNcXoMT9ix7tiQpBJP35JViJNs/fJmzVu JeKR/p5ex4afYg8HWbhZn8e2QqvtNu8b5HGUSBVvOJVNoebbUfcx6KzU+PsbH2UAk0gPWvC5s5XM McxkbOL9iEknLj2hfajcD2PAVTfzSX4kAs1sHrk8/G+HZIYEKbotetUF+qqYEhevooi2OJrjwsMq GqW0h9ap2/Bsp5Ns0PePIPZ2B7ZzS3Ydjbr4iDnjXyhjME9r28PXBsYINS0dW6z2AmEblVRJg+WH H0GIOIn0v6p5//PTnV/vibNd97Umk2D7LCeXdzEzddKamWQJk1iWF/F/t+RKnHsEC5myj2YZX2mW 5gb/L1XHF65AdVFIZTC+hBt3YyiCKmL4vuY45TuMnoQjVSOT145jzTVTJ9v9/ol1v7IKwqNuX0gk N73ZslvodDwWO+j/3uhuMXkU5ML+yuncdIFiPCXcWvKPPDginpHyv0fXU2IOqGf/RemUey9gOOHl YmfDZOFiU2WNFye7JOitHEJCNI0IKNsEMc1veFnZefg2KArtrHzCe6O9UvXSIEZVotY2CWkhLMXy 9pJ45lZSNSMMJ2301zirmVvq3EbiW+YdN9HBHdGoxMFnhp7qmy80HCx2LDy0GeGTssOcA0lHuPfY gRTC5FzByf7LrjL9Qj3e80SshJiUJSZhHAu+rQ+U/7S32r0G9VWuxxpJOl7Qs6zuZavMdkrsNGu2 +BhzTFvXCwpi6aeNw8geCyWS1XNb089ma+tNC9F95EcbEoevgx9su85KnGaDbIEYMEohXpuzLf8q 0Yxk98VFs8pRxC6gl13AynQwLtK55hzxouqgs3uOGldOCcoimvaC7wQmMJLgKpm4CMvV+n2iY6oK c6wkTlrwzKz7m+4VCBbydBYJUuLgov2AvH9RKWFrTBTQlTHgcVj5pAQwUhSKqDDuil7u9n4h167N h36UlAj5Kk9i81Oy7ZwpPRNhHM5Gmc5dpCdjoiyC42zffaHCEHlygsa+H8z46MSUaXVua99b1bMK E/9AlUuoUlAbk1yrYLXfEyVgNJGt+IGE4L5dJpOcIEJQh6W1d7GxUDdcxMC4Aso2MGuiaPjEcoxX +JeMmKb6AhBPcsfXaIJ5iKRmX/dKa8/vc0PjtwzTJdRAKivX7nAekpQojn9P5BGv+QOMGQIcFKAr CMp3vSePN+8RqQI3vTNnU4cmx9FenJ4yoQw0X7B8LgP5d1C4RuZX0BxMVn6TeWqwar8/vjayvhHi pPYiylghz3uKZnTqy8seM3FdBXrkDu+L7kIYmEqWG6U9tu9wP2qSvSMmN01D3aXUggv4dbzNNC6/ vHT54j0EsCzh8hEhntodvGDEJSvOLTai1usXFs1/TodBZVp7/VrjaKC6cmKumnVUpUwG8oj21zr9 mKItTUoSdQawu54IrK5G9apt+SxrZSw84eVPDC1oURTy72W1CxG5RLWo+f8hKRQxgKDS7LCC1cwS TXhcgJof7L72zlzc9wSHwH5/uY+dd3SK4iM8azKNZ+Sr+8bEPhHBNsd9UxJm6EiI34qR9YHLY+VN MP8DBEk2DmLp2xDc6CEuuB/bbPcPNh7UNbuASWjzYrHAK4tly6PAadELTe2/VlD8ZwusLoxFWKIN D3BFCiM6Kgx/jbSmIqUtbLSxn8xt2wnOinceGgWgA6c6yQ9lwtxKNa4aKH8Q6m1Z4+6HZgPBzGfR w1/bUnj0vt3ONtPDZSTXzM8N19aHBZpKJZI9+tY9YNm/u+OBMAWE7ghfdPAJnrx++rz/nmEENtn9 /QOHVE+JUHYUCEfot//pkvM96wltArh+9kmqXFhsCBM00SSc5GHJaOlOc4CIXgDm4PzggjF9m/vy uiM0AV5AeXpPVJKxUYnf3azAVmFhVDs252TKkhb4fcYuCkQSbNjVQidglFc4LidVRepS8xWhgOjU ICejagRb/4Hwx6lWyk/i9Z/+pcjw0F8NuljYxedRKFNoalKw5xPQqR882A5QZelwX9EzhcfWwp7/ 4A1ptiU+y3hyJhIk5vlVn4Q+QD492rjf/YSWAv7pTr94PVR61KnjbnrXtTRcBpTsjJ8l0vWrwEIc uLMZ7Af7+Vz08ZYm3owRV/INZut5x8pyQWYboIMvjmbxxiKPJbr+t7WHdeE9zaQuShhrhpZtXlzv Y8/zLvFlwl7kxVJGo9eQA1qlokdVflKrsOYT0sC297b7WKy/+SiSnRXpeug3VT/pYvX4hZBf2gzE 0BR+fJ1huY2jVTHKIBo9FcCnPSLtAMVrP2GJ/ezibbJCaDOnGl3qosQv/c6p5erIYK/ySATFtpP9 hAn07CqnfeVT2+GZxmsfVPUMx2K/v0Us/y85/Mi/ZjRJzK5ccjD6MYU8BZxBiPeXBlbrS/6x/IM/ hcajaHmEdlGKM/O4kaqL9LGdm+THvtM13uPhXsKJ1p28KXuDGrRZ2+ZPa7+syWMr/jrflgspk5VI HsumptR2wr+Sq2i4D2tTWHg62sQdZybrjhqch8vCVehyD7sP58IWVx+fy92NMvk5EILCtnOiA9lm 2uRN5EmGmkxKQvKvEs9isgfTdWXysH4eukqGnKPjIcYc1KCy0ScadZnu1y/Ep8OLedhfvrGMeiF5 sugknB4P11l15K0BDQztIey/cAGvwHipiFHGmZbaSakqrTNxA3W/cdwbIlEaaA/7TgqKfHqVvPhg VvygGb5yrjdfXuDbR1hIlLt9Ei1fWc1W++GAfneUuKcGBPDRj5JTzZtt69jCZHDau1PQPbVsmA5c qb/0U+/50BvgsBk3l4jJvQUhgwfGV5TTae82Yd/HpqLRijWqVaeogD046hKroGKcnd+z6SweVzaE T3Gw5xe+H+cKQwsd9w0aFa7ps6sdyL82dyLpKddkrAqf3XSfxPvwkj+f/Z3WIgjpEWQOrmYa1kpe 023Ckmw5tbuO1TiWLrk6OuddDfsQ+sGuPR60MmOCmIuR/BpWQx9NHp2mQsYGs0bcvy+2prAVJW6B l3iwOnNvTVBi8oW8++cl5dzYhbGB1s83ID5IXKPyLQX0aamGYAgNi8EKaLjSPg3Hw9vv4mH07S8R vwnSjUqFcXA9JcUzKi8jsptuc0HpJUzKCQU2rfw8B2BSjL+v4sgrvEV0EpdK56ph5cREjz8NQNct hCb0WTCDMYak6MDrhBpcbqQiqSTc8/evFm6319zVHbEBJxJtvCan1gVEVWcleS2W9PreuZs7VlGM 76LfCvj8QtMmakaA3HSwI5iDPANKH58jMqs7WIPIEIneA3girbKU/J7Ysitzl8RVIjREpM6EE+wV wA4ZxC9Jd/IthcKW8QMr8r32+uwN+bvc8czjlcXbwylmR1BlurWX/HKi2vWCDNJxi4HI9ypqW/U2 ddW82R4iMrDOlCkbTi0x6TdKrYyhTKQ/JxD/dI/YWtgNZ9ROjmJ64AzM5c6mkQ+oUhF4YSaPXuF5 vB1m8nFwEaudmQXZZAJknYl7je+WUA5udsLJmipT0shpxy7AwTuhxRz39qExmvihJbw/F5gahYHM hQe9gCfl4gHBr+PwjQE4p2nIOsQzPIKse+DOE8RhP+ZCuJUm3QfxSo+B7CAudq/kFwIpIvw2nSDi 0dgAfCsQmxEjIJCu9rELtRRdCJ3ulzgpA7b7/n3mh5szJwsQs9OHXbtP66LT2DTPUxFKU5+fZnTw Ag+6aA5SHo85D/WVbczlHJn6rKMSt/bfT3+w0f6iK8ZrX3LUo8IYJRFCcQ5sR+2VHCkWigJlP5bd 8iHMpIPjwKZqmunTIpevx+l/08Iu+MiP/tN+17XH9/jr/YAppmAOXN6C9Iw2KygCQdCTNYZ9wJXc LQwb97s6nr5o9Iz7IPq0nY1+49VziVeI0R6NRS2uwH3u6N3UddbDrDWy9kEAs+BdyFXy2DiqciVV 1mUFTQzdrzYPTPXiebTtiCZG+SGjtY0lilXWQ0z0GPoqvsST4ZN3PnDl3e6DKJNQNa8vuZS8qCwx IITbaiZN//mWhJ3FidbCXMn1ZJ1venLuCGTMc1iBKgIZKyZDv0y85bqel2YilxyjhSTwukVvIEv7 I4lXBX4i83FsbiJXiwSDUtVfyLpTV5St84A1ftnorPtD3riG5G/eCxMjsK/rJp03w0XpUpPKN0Uz rQHG/vJ/bKMS2mu70raCg+K3cz3UPhDX+ErrOlsBVbIPY6QhqP9U8qf+4BHKh6oVNp4WStyLkEIF agYipfh2spBBYGHA0FwiqdyI30SHApL79DXKzNA2Ak1lHLp2GWVyYJB03IgxCnT3T6gZe4lW7Msy jonh18+y1W4N+bx6p8JDCR+xDtONLrxCnpliJPaH5PdmzFnM3Sqqz0w6u0pQsS/pRz3IZLXxHGHe nY4MDNHToILRC5sNkK6MPWYOPdvPS6HP6KBVviDJYdsYmm2mkBvxEhVaxooD6EpjeJbOj+VB2yOo bXYi/YFT2Yi3erQ0I9ukNVH5YsC2Ts3mnv+xMAUM/JSUhY4nWSSi14coxg+rvwxDsC80ef1ULXMi MWLvH22UR77859ThhXt/8KNtz3hF4cw8ilM/yAMn/S7Cb8c3ziznP08hbanml5cdd7KPO/9zDum2 uX3NF4ibuwPybt+mQPeDfLh1bhn86c8W7A6D42mF9Rn4LkXo53ZD0+FuJ5goRMANwA6+LFJd4rU2 DsvB78OsRlx+HUqfJq5vHO5vGaW1CG4u2JaBnJXq/o4z7NwEQj4vPvpepvn+zkfvtCfDHnVh4tCW SWwXb254E4TLFMpVbNp0/a0c/WJd6DNki6W2NbhRebURUQwVhUzlPuPlkwcl9WB1gd9IZI6atDDX hi/0mWlNY2bG+eXCOhP3LcPcz1RTfpTqoo5LoO4EZ893V0V746JV4Y89H264OCVyrCMEb2Zyw7ek gFZoSaAgl9jvJYW/aVsp8VjmRm2Ii5PKk8OkVaB/1CWiM76kGCVpeogq3RQNm3Hb60XsHmHAs1Zv 8Mx5xHgWgzrwzgTTqlnLZWOQqJYourMGtWn572yizuxNsH3cel2Ce7KmhMx6P2woy04LrFpnngpQ pGGSrNLnBpMauAcqaIsfN17BzjqRZAbyVFHUitnmtpSKgiqK/Q9j3glUZNN9NpOicIqgdUugeTH4 W7pY7/UC387wUtlzf9O4Wt7csKWYDGG1CB+5F6A/NjtsdZHWvVjSAkWdjHfaDkFVtGV6fWm/9o78 ojTRwRqqv7mE3PmEQJHB4c9B9tSQQJx5XFazisxw7ta/I4BDsuBAZFKZrwOYUJIIO8+9WxQAzByC c8sH4GZAeFjh2i494IhraNTEUaGASvivDhP8ku5awQozA512NX3P5K4yx5Uf0DqAm/puQFrN1PwH o/OrP3QdFyHrrpVQUwD5p+/5yU1uU2+o1P0Tz2qr6JAYd0j4yRyrp3Xf0dpAKmGnyP32riB0iqKd ELpOPzGwAZuYtunsVcH2LTIoKPrzhVYJwjwOOEVh2DrkAp5qE4MZfZB4KBv5+ZQHUl9rXhHWVMUq RZh1bLPWGUPDYWxPHo/uFWd+lI+Kcr1zfiicCarlhBmza3iYVYcVulMdGiCNFcGqOW1MWFqwW17P bS9qD73GrXU2omn4hW+AAsaSEU8Ofmc0/SQpsO3cdiim+yRtJZ+mSZ7IWpBcfCN/TBr4Ygo1P9qU GKXNtJ5Xz2hoorMkbir0KlC8cRIXyIzHotv7s4rgcrUxuhM03XMgkAX9pMosET+V3UNEFIGpmNIh dRjFpjmbbGzddYZzLLFrTdOO1TdyXJ20909xv8q9wLSaOQb0veO7EyRoQ0MRAq5USTxjsvNPu8h0 UBO7s96OQI4IQ+hLAJM5Ei+zsPAG696jQyss93BRoNFrAlgL5svFCWBMj2XUOlTwoVYCgkVFFUok Ze+WoBrxJTdSoa5uT8i6eUJOmOe6HNIre9VPwvrlXhPU1/c7b6p1bBJiSQ9Jl2ZkwjujSoaMYPrx jcmtObOg+mHf0ZeRCyPI6nYhxQ7K7TrRCs7N+OKxkLHVIQKP2lKJ3sg61ATbhlyQDM/fWTJOw7zT e3U59PYqcWoOjDnWn2lbotXpFgi3ZTZKpioGqj+9QdnPX5I/sBU1ZWQxqPpVNzAn7BEvYU51EAmV yDw/MeMQy8vv7phSSWA9VSqhgPPUZ29Nov6hvTWH2kt6fjoCD/31eL93R6+E2bUmbmpkRZgftJuU 1M2NzM+ZV8XsUvM0b2gQzzdgXRr/rRjw4D7E/VqOKyMX6FPjCmr4SXi/k0r/My66MS6dIukAtZ4n rvT3KoF3vNb/mxV/aRod8wBQlX6EpyrH5D5EW4M87iYJw0MvX2NrI0z/x878zm6W2oc7wVZNETbs rIHKmq4ch/3pa6doxCxWeNkfjrlXduLqAzlum2XL51D+3gnUL7kkuFSIqIW1VYVIun7OwQIghSmM Ff0rVVd+PG6H6y4uc0ARpZViysMR5hLnfJ3ZD9N2oM+5f8E2x7XEwhwPRkPGF98/COU/6AIrg+B4 XAG43ARSeyQ2c4hSR95BOq5vp/K9nWzIStDCC9bGFYHcWqFITzJt0n5dOlq/Iv5O2f/kAKQlWCtR clRUVPjhoEghwwzVuWqm5Cva9XBb4wfSmq2ONAbty5sXU+F47ZiM9h0iNdMCoHOR4GVIOPLDy+1+ FXEQ7cwy0K9u4hH+OUojVpU9A5saydMMlSNzF2hI30YmmXPugceHWbyvUqf8chppdj4T8xBDT0GM +zJ3+Jyq+7opF8RQ1douz4oR6JF4z8M+LoKpBlJow5elqpemROu/u1qKH4UtmrxhrW/o+VMKMKkE QrJGXRgYoWdGhQgn6S3aWXdazZNN9Hdlj+j+gAvSj8l3uhYbfQc/5LIspO8pdTtramnGmB3GU1G5 mGJEiRbbeMfvQ+iNdhSNoQTnHvktDJMiIIx9AO1fatY8QjJszCbGEBpJSWAwI7Fd9ymD+wZrzRZu CgRq1NCWaY8idQj4l21A0joN1X6GO7u0w9F4r6rfL+b7twbG9NyEKmHm/mEfkF5DFw28xkf4L2y3 XH+LgMx0lXnpYafYsbybYH2ZefGbT+kjAErVfz8vXDlyZ93Kbg+Wvp+TCya08Ke5X4Th2YwJ3B4k OmZqB6/8z0E/JYMVuUyP8B8zv/+vY8e8AyFyyz7yOnwDC1w/PtS+GoLMgj6mlvCIVhSWeGZOCm/G lNUX5KqxxabTeOIINLLXwB6t4dneUcnfD14U/m3AvodZdvpVCYLt9RtmSq4TnlrYm4GGnc4YsdVP BqwNeMt3NaHXkpVW64W1lVeH+HpX4Obug4jW63J11ZcLlUSQ0NCH7jgr+5cvOTU7Ufm8ZV0EDeto 9xw2yIBzn4Hc+AMeQFShKcMUdaK2NOm6Ofw8LnS318qPunVSxygOo3Uq+4VobHv5W7wJLHPQQm9m MJXcbweqryjOLDvzP25EDocM1YFjgVz3rhW6vXCfUprrjKaNI7GqgFs+yxkWftCyW3AhCesIuYw7 VHJwQj19maMUcvULXelYG2ag4ZBPZpUO4WYmU8GPtROnLRgzxY8EWqroFapl32Ey0lvkEvcJfYmY B8uCmQNFurZ6Mwrp7FKjkZG+Ij0B2ZjjiVSSj2PfgRJlOkhk1X06Ci6IsZl/8p9Ilr+yFXSaVZ94 vnpeLAjK6z9rkbmrkD5ST1H60esEoyFT76HN2QpB9PqZeV+MCkpLmJPBG6du5puwebQDyX/5fFjP LKDEe5k+WEMRZ+2oujQ6qHm7cT4pcoGCQgv2DCo0Kj7lNxGrmIfPa8eGDtGx21rEQJGUE7unxrCZ E38k43yLkLBWM0XdpIQ2kPdsA8zIeVwASU2PTWFE0uALbYCN7awoWev4GxiG9quTvOkD/UaNZS8B qVWdnXXbLpdTXvec3nKf3fi2116gImWgMppGPcumVpmKObQCl3N/2AelpKytLEztK9wHK3IR5o/T iwD43yWv0kjb+v/NIu9zvfjmnBBi7WHaYDzJaNiO1FGsOXZBvpqWfdVRM1mk7VdhHqLAFe1XK359 cr+qJCo4GWWGiCmS2tYNSfZbur0GgteuftPT3Zar3+veFOPgBaf3+8YoB5uJevfnwc3fh+J6Gxoc IBCzKZTQPgVWu9Xp58rOxxxl52oQ+q0jgmForZNhduO8lQ2tDQvNlGVuPSa8eAcSlxwcLdu4Et0M IfqSXiqMUZZ+fgkBQ99+Vud+s4DkVg01HJX80Z6YNlE8Tc7bpHJgTFg8/J1id38mcGqHo9flYPhj XNfxF0K0Qp0zmJwn8ZafUZldvcVNJIJ7ujM/unwJ20FYG2M1cRmI6pNUTPFHI6gRJYuVtxitxTYL G8ePcWkKiCtLQv3E23iL4G/7b7KbJXxN50j8t5mufA5Kn6CW52IclhF7e6ZDQsc8gPM5WFGsdrJy yO3Y0lb7ygrni2coEea2hcblIHIXPwh6G6I7qR6O/lAeirGRrOdAYJ87vAXI/9qwQn9l2BoGmho7 keBIpDT+BCGsCWCxUMiId8Rfwgpg+vMOHBUy+Cti4ugdm6BEASUW5tNKH5QZlR4PFCILLfG7uhv5 xJs8ov3XnKnRofgN1wkl/gbQqlG9mZcXHENEoYPpyQ86svaSaXg7J9AkoINPyG1fOiom3MUgruq4 AJO6O6gx6Cpwygx6APEYhQsjteiIra9pUG7V/4iXSbrwUMXUEGZjNzD62ihoO5EBMPQW+HKPkgP4 z572RzS4ee+YX0e1McMng1jPws912rVfRBGql9VD+mUWp40HJ7xBlQgzsoChrSyLvlGYjcPUtuIL t8SXaaAIizKQ5K1shfICZ4aI97WWkP96Ac+nZ4d2Tqw6aSWOs1zeHnKULAfWuH0gGqb8yk1brsj4 /6nrR3xdLjiyfW7/6Q/WI7G+/i9Llqo9Pra4UDPueuSljBOQFZlmr+ICCocC993xBzlC5RAHt+k5 /2EihYnG37kNkTxQ4pEO1mKtSgUdWuf8iB9i622E/BpZcZry/TYUWiG+I0jd7Jc50i9RLqXgut56 5Ju/VaWf+AqirRO/Unjt1BMuipxMpS4rMg31zV0Y4UbYH0T6tz5gJZq1AnwKaT2OqeWQSxEOlD9F TtVmjhJYw6fWRpYgo5LEUSzsBroR3w9a14GMZ9nbvF6gCHTdg6E8UIi3+zBmLMz+uwvAe3Swo1Oo 8TBRJAyDFUEl+w7ORSk0i/9kyDOWDLaqMSuaQLKcZFlxWlX96ywS3rmg/iLkvMBipS2AtgCzkQw2 7rq2etwcleZfeID3UUFw+g0qYInD83As0gqUnYHu2Y9CfDyAHa4MBl7ESPXCQeUsyjZg21KcduMS j/bNzXqlpMAgIvD5j9L0X1xN54FV1WF17yXxEYfORwmAaA9b+91Vo5OCsKILfkhoVFJ9PL0fYuCG 29bT9DWqfdAgOCEhR+/A1hB/SwAYI6XTA6tRISWyWe6EKQDTEJipQLzHco0Gfjiy1VFwSZR+S/6A L1RjlkXTdn4HcPeDZTjYjTmAwDA2qGhj0zJ4JdxgFiz0oOZg4lAEEoVANEnQqmHvKzM5cLSYcgXX ymsWrwqvkGjN6McNVtzXtDslLtOfDqs7UFScwuhq+DqPyw9iqLPYecCKcnx33puzpm0Pw9xM21mH m0WwWMcKNv/7RqWHqONcUS1+ixfa+HqBn5eSZ4iQX56JHSnMOiowDwN/zBc3yHG2rzq0rmCnWIHc uDEGq9499qtBQObRJsPfUSimlOUPE0WPWKf4iUKPrp0ImMaAf5U+8yDxtOUR9JUX3vrWhgKYj30D HjpBmxiJxbpbpYCH78eJyBsVqk1ldzO4kUxdzKGaVrHWNV8qm5i/HPOmakO+2X1btj4a+JYt0Baf Ff4JiGudrRAj5VHh11w6cHYLEeHsyQUgIPJ70rSh5DdrryArXP1cpOw4uF7DOo6b8Clah6wtZg6h quP9jF99FBMcG8MciZNJQFujOfw7tVgKoCyP/GXOHSAa1AXcl+f3kGDs60sV8N6DJpWabQ/pXQqE Q9IjSUhRnDU2bj8wqXMs47FvRtXK56srZmp7fKS1kcB+UCopS6Pi2lbemuti0VK8LNxIRTjfgG5E 4rg3UAXx8BiaU0KysfOy5TTlfG8CUK/gES6/HEDNBNdkfudBe5iTqYhqeU3/obFyfNH4RmlMnnil ueSBv4G2Llmyh3/fDWem2YACRgZi6rpHnAbq79wJ9kw0O7XSqUASNIa0Bxrb4Ot2EPbvDuLNmahp GyjhkmyBhzZSxbF1wNz0qW/b6kEYJUqsInitcert2MBcdtwDn8SmYV/2opb2YgIHuAoGmvsJSeNG OoOgwt4gYhUVREg/MFmQttT9K+31fRJx5HdAe42Tih/MkevZa75v6N8DW6XzGOAUmsW/nbAi2xAP MyCQGYMQmX2si9lm0of4b+4G80WGnJc6X8/GkcX0H964byhamg2Z1GUgZYlC/9j1erDX2Axw1edo tfyZ18tDXpJ+whh3rvDc5qwiUPF7ObGXwr6WmpHrgGWubfInbtyteX5IPLoTITqp39l9L2idpFL9 tO57SdA4bOw5JNX9NcBCvkaFtLwQ2Qifz1gtFpsyTnl7S1pWfdqkfau1VXY/Y25g+P+dNnkdFN66 bPuFcwMObx+g2QxxiNDcndgyQA+i8i09TSMTmdxwAAnpi1DrgHedxJHirjCd9e0twdUAjU4L7XWM oFdhPog8SkokBfiQ/2WYYgqukuUDDi6IXvHeDBTzNJVGdPkPIvgfLhoi+7f/5OXCosTyRQuOdtwN 9Cvk6/wYZKnglRydIyXzNNf8jOo+bCXEMdNHbryBqC+G50nXTF3Qphu5YXqjYxzLQ9vqcItM/KRe DQJW6y5RiLPv+lN+psXfI8QDnd8pzIWTePToWyxq7qKas15r8ZXoAXcMzC4sYHjj4yZBHz+1Uiub PEF0kYt2gy3QN/XSc+kyf+RYLtA50CkJCH7ugtG2qzOeqUp/am8oegHEsBWBjnn3Q8gSiJpokb2B 0NlKlcdXX4cBCiwSNUK9kcgcb6Ytyk8wZOpSPWsgnt1SK8pUZGlMb1513mzNiIAxuHku76P80qXe 6YDNZlTh0cwZLSDVI7vB90krS+XDTdoy+R7mSTGff7s8S74x8Q6/baFaF+eE8VFLSenv8QTLOfG4 Pb8b/2MG1SgQiCxQOWxVWnx4sJAORNf+OS3Dw52cAqeLHJDWb7WHylrBYq00bCoYqjFfktPlg8X2 8S6rrg3JFU0KFfuMm71axmL7NzQv7hdCBkvE1MphhjsTx1eAXrr3/G5FhEmIXy+chAMPgQB4ljss R6XbpWZHjB/UjF0vULfeJHf4B9m7mj1ua19Z/po9g/0RQh4BYkfU/So8/SFgjaubKcgPOjlEeUbE kxQDyBbKtj2P82r7fjxL9lXINoJJLo+e6JabaRb+DeTT47NPvZkzcY7TWzeGZFPTMzjrfLa2GGkv 3/yM01hxtZ3ZSenVWpnWB4MtZc5fCgGVpUe0tRa5he0n1WGo5SLkHXmeu5SighnIWx72lRxvrMLi LPjwctp/rWxK2seZyyWQdEBxFeoLHIwT8HNxHm2x/I0E5DSOmW203mDLTOEtvtyEjJGFckVMultR lwkxreD8rYGW9NZ1/ZnQ6yNkwG5WnFGpiV+JTsYNo7Ki8UDGYnzGCv1ZH9cIKsK0cGNw5wYEhPvh GDnvNCIU7Eu+Hav7Oz1Y9eP2qDEgCnkfo7gQBOP1LnPDmvn9MzJ1YMPVBG5wxY0W2+BsxpReIG0o QLL3aSm+EnlXIjm+hClzUuyX9Cpdfk5/Q5xaah+KeHbYU7hyLB+r0iIhngrD3xZC7EMPgZjlh9h+ XDwCImhdIs6cb1q5gM4nwh12yuOtKQI0KKnRN/EPmq+9AqqcCX4SMJbpGQFvbkhNrs4xUvrBwZQ1 tPmlnx0uqV6bgd8rjqHLKxFJ2RPYlbI7YNkLy8orL5Pn9rWwh7UynMzWwHPZ1CDgWZluMbqj+rAU A3MsYodI7fmHyion0Z2BRlWTff4gWMWXWNmOVx2/3+aIK/Y4/6vV9cPwg4Ea5kIyQGqxqB9172qJ YWfVyrWMz9ab69w/YwajfK/XCdhnXAOYlMzLg0rIzkf5WEVlkDz0s0Qf8wv6fD4nnRGr7zYF/Eo+ du1w73Qdl+/BNTAlyoogk9LE1zVOR3c0KkrDFwVMk84dTjC/HWcyjad1+1DymKtL/QTMi1Agjqab vA9tf6jD0vETU60t1VZBBIWxlQccMwBuxb5UAIcJGpHe43BpXo4yyq5h7fShvMlN3/RMptV1G+7u lhByELbF2QzuVus1dC3GuqI9sK7V0fSWDprqqAXLmbU9TIL+isCX4+2eFBtLtbeBngj8LdxdNHD5 fozLVfsjjomopNGlLSo0BeZOJ6WC3E9eeGVd6tQ5eA2KgmY2wcnmPyRqreWVMly8JW7LbccqIgxr zTVEORXcKM4hdLsEi70crwlDdPT8mhOJ/fbTRlIC1YQDJvUyZGedmbkmd0jch6z8a8qft/RtO/4w hep37HOLVs5VujvUOi1RYbcK/qQ/ayQpSTdNVQUjX4zXYE8ZP/9E624TgUYh8oICVYMhoM1yiDer Y7W3eaCwigljdz/mNSWCc8hZx2qaeZSy7idelG3U7EMiV4+z7QYEKbL4S0CdrIiDbe5Gbav5pe8C My7tLP6da/vxzULLHdUQsEEjZVtiSBlyEDpsxdYGR43+EzGM5UxxQcdqvRxCMI/Sbye0qh0Csn7W Hq8suKmWoZHqk5lIyPnRD9nrTgvLoiX+txD0or8ftIW5KScyyTkzUoE1HAohXN+tQEmF23Y7CYxS az1tXCiRUflxlGEK5wCDVtq1KS9OMNXjVaKWOrNZJYemmLjfn1B534MMJKt/wEYqc3O0w6IzTdBY aMtZHiMv6A0KE3l8k4EAzRLsDBjAHJ3euzwlc1owsrEnA4DEN/dJPxseqjUnlJxC+Leuv7sLeKN3 CTumx+2SYW0oYYOaP95cx7gn8mdGw1XHM6E8mtG1XkPsqk8GmdNAyXogW7owfwhImHgFsWahLolG SuK+69Xa9BRJAatm3vpmZ+wo0wwtoFstOvtO4TyHRgoSkdw2nmz2Uy6174W237uvSQVeEM1olgDb pkvYmF4G32Z8Vjv/j/vpiLitJbvu5fZj0ZnfMdFVxaGRDjFIPD4X4RRwg2B1ZxCTb91J0zvrW/EF dt/yCGX1NuwREyT6/Ir1QlU+PsoKhJKCoOgrZCWGDilhUA1nbYT+kqeK+GX5a5ebfLs3MkuDony5 Jnu04ksfLBz3U1wRd+vNzEyrBQ35OHM8m4RmnJaVWaxuIUVvVqk/aSIlosT4KaWUkc2TP1LgDvzj ZkM6h7Lxazp6wqJeCh5e3Z2LE02vC19JbrFrCiyku2tIE+lZU4pj9NKIHeZr+Q42CSPTcQardWGQ wrvBESTqM6ifwG4aqez4rWFTsbn72NDdcJV9lou4H4T8kgJtRLl8DnhQFrs2H6xd7uKx0EmVXzqU uY3WFxpTmvF/JyAt+ojmeKNsd+Mfpy3NpUP8XKjC7t+RF6OUTcTec+tOtRgq21xocrPdWwtvSlPU x4+RmWgCLJoYCbjiRIU70YjpFcRuU9GxN7SceKVFx1T8/OFeCi044w4fo0pOcsgGj+NbXPn7NOGU C4wq6qO/w6Qdj6wuwLVQLzUAj9HX2yvvc15owThtDG6Is/jlmDgKdeCpGG3jA6XkvMFIkrSB/gJF 7C2SLxrwH9iwVv1pqBHwdTNxWYMm0P4LQoMhmgzND+t8N7M/ZyG7/uVeBMvYn8ZDyFdXo7ELw+6x xyRmpXSSB180qxOLb6mlFCsNLPxKsl8kFxTJGsyIUKIZ8lw9/xXtKfZDda6SOQJWZxg4s/j+mRSQ HVoQg6XYbgY5Vu6EnTVFM9pV++43X/Idne6hKhJrBAa/ih38P1N7v4Afbr5BJPHmQ7rB+38DR22o HbleTrQ+VXdoYIEkthPGyUcFrvKM+bieOJMtRCZi+Vbx0t3islWFL1k12Imy64X66w4tqmMELAS6 hKYBV/sltfpgnt4eHtqDrVENMPBHFVlDN/AHTlX1q9igse8nECWJfJPNthB7ATrrqcyTkUyUMSla 2a3wwpwmvrWsXL38dNVFC9vr4AUUl72BjMB08ZWWRWh/r05gdDseOmP0utE0AKO2u/ZCzB1P3orB naNqxDGPm75acwxG0vymK1MeDxGVlzMusyD/09glZHkQvE5k6VgIw6LUi7NOIAv4ZHzhs9p8OGAG BVmTbxGiLGbiwcSFDS3EFcrbEtAiraE8Tgpu2MCSWo7rRVPIMuNia4QpPUcICPgBV/R6szdVlXsq YtbzKwVKRo0NwIva69q4cO1FeQeAAHD4VPIusdJFjBCwnHTabpseKaAVLzQUCspIB0KZ9NJ5wlke FWiLsGr7Ipmk7wrAyfYaKXpnJpBjNReefqGueh3wRHqCl6HAl1y1wIBGv9WEIJ0RUVpy5+JABNBV 0O/UcNfSbHF7rdEd6aSUMV3NcmmdDOr4ZKAit8HgAvIxcU49WBb28CvjUNcTHBm2Z+KPFbtg6ZBn UizxbznhxWOgG3hvW5PbS+upoM0Iy6nY3VoYhuQQAD+LyYz8O/7//suV9EoWGPpZPpqn+Zi52TX0 UIdHczoL02DHRHN2DLE71Yw0Eiz6fhjtcupRxg3f7Xxa1xZSo/0/3j4HHNmh7wFmqSq7iVjvJA1b Jwwbc9uLqrGW06ZTpsc/XIvhyIo+Mau9dgIkxKe74DN2v2kGju1PDOgr1HzN9fxvAbyIfgHeRqpX iaXF2wy5e6Z06gjexWScQ221FBn/hv95v4NODwFa7h3r0R7S4b9RNGd+xyQPS7pV8NLPhh9joa3l b5N9d/7kPpqHea13zfy1vDDS4BAtEpWWfFjNp0/KiG2FqH35i/ZP1Rbu8sAC7FmM/gess/9FkoHm 3gFaYFCEGNptBL7B5mIAS8BsHtPIJ/xiwCNQPwBUDoTjlkXHaQGg51AsGQ/fM/aPXtMkSMoFx6R4 HJCykJRW3XsH73iR1ML2H35BkHjZ+2bbuXLmW+0+ZcHetUW4Aw6tNwyDbYb8D6pyWtoC0yMobhmr OSuZdkb0GRs4Ujhfdg6vG8jvFt/HSNoEAAB4uH5O5yuy2/4pALAOT++FpBKrwNfO8AjLgtOVO3XB ZLGiLaVuW7s8122gwI6aTR30lGrhCyaxYucmzHlWjfKVSbIa5gZ60w6/dDsh1bJCQeptvTuFucsm uPfvJF5lkyHmcT2tPD50ADgnsAS7mWBPJREGJa4NETSUwbEeg/mByMdmIRRtYKZu8DFs/LtoIXlG +EWBOUn80gy8oJ8fucYr7/LytY7+lrnUNs85A+CaJSGOFluGg//ywk9sGLd4EYDvv8aeG9HoY5Gh KRYPGLXEqE8Bx9efjBRdBD/k5DlZbMPN1fN2qKiLrVzVkByYQMuEECaodsLfJWUv+Hy/fOHN2Vuq SV73Y2V8yJeytNdKergObgob+sCdzn/ILQWy6C50b69s7Gc7QFXbYVJNlw3iF0T2azZznue2r5ht YXQxkIfseqH/6oOo/ZwhteVWVH3hBb7M5YJomrF6zuWxNpCJpW6ifCqfQEmL8KBNKhmZq3FORscH h2863DuhzQr2ALo1E+QUY2Yc7myfPHedkmp6BC6ur9x4xf6JDu8jobQJtDzbM8DIlXWb6q8qXWfM 8pCoXUZgBOze/Yc9o010iVw1To1aHBKAzVkhPVyp18V1+FkGXlIsysJeIf1wrZYsEjYGaMKC+U0S juzclCa8n32n00PiV8WEj1ZfvLjLK0m4XN4sC+sVvjPwhGPZ9fyUhYPuBSVUXn9SLdo8ACLCDik5 NwWRRSok+io9XKRfEJtKwQPM9vRLOpTC6RUN1pkHaApfOwonf9pI8rU1v/36JxxbCxuuPg3DZ+MK fplkNDOCfmrgogdUIK4VTtu/J3vzKUek2xaoz9OtRBo+GCVTY5HNkKBpSc5iU3Xn7SwRSUI1Zp6a Xgk+/84u8noO5eTwPZAIeuJaYBcLRu7bedv60u3AIIhOj06Sr5fbu9UqtTKRMXzo4l8tcnOnh/Ec 1JrChX6s3qzyr0JuawDx6f8s2GaByWlbcjmooNmvUq4oNwXt8ZUe8DZ/DPUh2EoHKuOCDQr62rUR Cj74uWcQHgI60cXVrAHvHiFBFysU+f1gjPFpR4/7At99sHze+dOkZ9aQDXQv4DeAuYDi2yd4iJuQ 9KqlkqhaDdJrQ71+F6jHalBVVePf9aFRwahphvjQ+beNro5raCAdmhVLHhFYyX2PUJ02g2xW0nrv b/OKPFhb2JsE91O3+vMLCBg/qZDIpQxfdQcPulkvwj8yxAPn8Cz4R9AWpBgyaRKUm+BAIw59pB+9 BqrO3UUbFpgRDjdvmJcAkSn1pvo1D+z9vSWW8RJGDnVbta4O9iMuR8/xfbB+nlo/Gxz20hz63+Dj cu8Rpu5OvbTe6ubHS5UBRK4mx204ZSHszbqK+BeZNpJk1twQ/s0F23RdpWnkPMelR5waQcqtKfDk bpJZsWjR9TPFtJoZmU1+CnhXATz9nOR8pVgy8qt09t7lGrrYTdMsVIX8Q8UeBU+xQzuzn/3n9uyr gJRrRjmda2XpXOHM4yTV1LRmOLkoPvpE+difDm9ZIAk77tPsgASmtDrt6d9ukOfU/JAq37UywpSI Kx23N7o55DBWwLZZQo63DagN444ldRREcjf/ZJN7UwbmpQBA5C5F9NhRuWB0tyL+AKvbYy1ecwef Hfbo48ODVYKeffaClnjAYePapmrM3kjTApur2XpDRCm4Y/XHG0OrV1v+D95R2YdncbcmtudyoAKx WgyboOBcG0QKSqNz/5HMKr/CutpykCGNVXyd8qIS4bNuKuD2wvS7Ha6RiUzCtn80ak8w9XYssSVq Mbl5YTpO3Ff4/RZtUyIOzBWcmxFuZUc0ml9MsVCNAzjyWrFh1jU0CVuiwC5hePhJHc69NBvH4On/ 6JR3HTXdqcF7oS7nhzwZZ9y/AawwakDg4mNBjjNDGXYt/yGK0GMtCFah4GKD3vBQ9cdQ3U7uyf9I /YifTaEHSmvicCjNElojfnr5mJ+Sbe0gIBAz97E5tW65BQ/webjEOj17MJMGWpcPgvBSbICJYPf4 zxTE62b+sLQ/DN27dXB9Yt349DQfHnW6j5qdDpB3PY1wo9p7Licxd/8nuCJPl0nQ0jQuEKeQckPf Ul325NEc7sqQ/lyHz5I8ZE+dAvcfOx0iSLfo1jWD2yHej84ah5vBUO8UGUXQlPeBq7rp9meFC9bo WIGe6SR0XmTQAR+OCyKONCNdi9hhOH6GMczD5xVmrifss5RBhoneS+bxxcMOYg6/rgdvTqZIbV4F jxHrz75tIDCmNdl3Y5lW72jqZAB1+IxzPoUSgsfPViQFNg0AbgNaCH1LoHEJs45gMOPQThE5PUMn cxtMP31stcuas6fB3oOtzBbokdU8jwNp8mvCNlHeuO4ElVVWw1ljVADnYhc8aHXnbldLbpHP/goU c8v9fGPQwgSxNKpsfteksfUmYGyra4M0+atC+I6HBuxE5S8rjpA3hw2d0qk2qcUMylfKa4hEaxeg XQ4LSh7V9x/nhAssGXzTxW6tP1oWoUxY6TAosK/5QnFuNwxKPyuTS7iI1QYTEIZdKgqrVvtmtOgr 2mpPhyRv9LaoWmp01b1sV+rgvOMld/FQ2/pYMhsQRZmB76CRcjIZYfdRRTvfSFBJTrfsKUuCLWJA 78a+t2I3s5ff1gYN60Srftr613YsXSvmgrz8GKmEczOGFUdXicebsgLzNzWxkjltJwLAAASSsj7J H12Mf/3k6YyHeBdx55ZAqThfwXQnZ3EZYSq0wSPqU63AxvU++ic+mKPvpbMXXmrlieGWoJqk+hGD O78Q+46ck+V4qrvYrh74dsCPYA8v64RCEyyCGo7qW9sMRlrH9bFOLPyQy1bJxnQ99B46YyXirlAt rav6+BJR4aXSBQ9UFoH0XXl0HuomCIzyvJlpwqwN20+b5hSTGVeq1ZnfQyLoyEyDwfUcXld8Dtcl JOdz1PcjADUv9FFtVAtLbC6LhIHXEq2xKu5s9w6bMuruFHcdzuy/L6FPbgobhzw1jmNOPA8HwZxN vB0f5k5wyFH4WZb1vQc9nRm+41uIyk4lmBapEzody58+3lkPXMwsWzY0PJdAJ0Cc4luxU41jrge4 e5fLUxSc7xYElaM5BOp39grFAaUS/vWA6ue5f+MQfUrrATER0xIF9MtohISOf9jp/gYcI/lR0IQI qtVaBlvEg/pKcbyWbqxFWnjxC2CkrM7H1uRQXcp2eym/PqIlseTLW6MEopgDKmw0iXTDQHPoqo1v pNAR5ym/5FXSb+/DDBO9uGgqndEpQ6dh9AN8+qP37WHv5N2cEgpgZZcWEIeCGByyyMAOwR0kdb50 lUit6/9OZ2AmW5NbUeRChRBz9E6i9CwDnOFJaAoipw1epLo+dcJ447c5IpDZyB5TwtTWDvJrRMMU 7ff6vUSxuSYF3x/UgUEyKJBxm2Fpt5+3YykKM8lKas2ib0U9F+Kbws0QkotyD8nG+gsRaYtJQsIH yfC9Jm3J5K8ovEjYkPL2xuYt6iv+5M2Dfxlx7A8qbwclSn4NRUHsErJEhY0k4bOU8ionNRE2HfQ3 lflNaGsOpozvszpmp5/5oz/YJsdxhor57PgdcqHcSqUs/lDelH0wS0lQL4p1gyxyBaxUpPDMmOG4 Z2da7ZW6M9hJ+Eo7jePCsMco8etPzkz9tcs9sBaL9b+0GWMkgCSjp263zMcxPTzLC7za//Mdr81L 4hXIWK4IqG+K89+wrO6aXNNqDwKPvoOmbs40FPfqCMj20G+m8DRVvVuUHEU+DJchMQ9K2tD91/jB lQHcWOulxWRGBxt3OSd9faHaQQJflzn/FkIXqI5Hu2r4NuS56ybycyvQdtIuc22shB5j4SfGITHu qsb9YTn88CQLUluEycDFBi5+UqnGKoK3UJBL3aM72QNOjmgWuiwGNy7k7IzK8/8uvlYEku1qHnjd GIgXKr65cEKB2V7VA1x0xzCdtF7seTd9g0MkqHcCZl+L8X4l1bCcxrImZ731VjtJrw2N30yIICzr KUWlf2Mm6hcaD8VB65zpnQTB5COUhvXcMHWknVXBbJ41fmZJhL5w+hDFM4lNNNx+x+N3V9X+U6hA DOAtneG8ylhr0s+rwrV/ckqeSu63DiCh5M2PckK/vKWTvY4vCewue/zk0opkG4CT4pOpWFDeWevI A/u3u9MpxK5zyBCuLynflON0qheLWJT35huOLovciDtIAxlUE8wgRzPVRRn7Lq29627Ebq1QVpqG axRGS+vrr/TLemcvsBaf8lQH96FGn0wmC7fZWdNHJm9vTN4KE9mRQtMoIPEiasqYVLRqYcUIXyJ3 TV7emxpEkkKnM8vyMKOHGrZdOsl61vc33kOqa4nA5L9tqRZsY95REc5GBM/sgsHqC0yHzKoj4bY1 k2jewgck+rHB2diRn+6mIOJQCbNagav/TTKQBb383TcUCRsKUAAtRDQJ6Kyd7NItbn2/RGPQY/Ag /XjmmbBEkux4/hU8hZcB8lg3WyD0YumYyPm7G5CrBfsh745xdY3aZEAW5Ffgn4dSiJhvrZeyAzCI WujbP3zLXPqsL+O41Dv950Pgvd8nVJf0wPBGny7hdO/vOLyXx29QSKvQ1uCNDQW3uSyd1C+1komZ wW6Yk0DcnYWZVLSG68w3PHH4paJ7PGzOCiEftxYnl6hq7xHj0AVD8jRpF1tH4N5MuSyR7SvxNkb7 KSF6ZidWKAciX9rQkoQIuBUuMzi+c2v5Ky70P9tzlY1RyBCIyWvYqHcOalVzGZwUt37rCZJxPFyB PEiKrXoxeGGjBL+T4/hGOA8LrTyMj1o9fTPgHCMqN5zEw4WD5K6PAd3BmIXexoPt21/0W/7XxcW9 e79bpTRIpF4NrG9N/xrlIpo9eUrYmZHd8tPcb0x/uOtI7ClQZ/Gr61K5/mRhlLSgMAgtEx3XzI/5 dgHxHOQ/fOO7mkvLEJW9rgMBnIu5S1wEMovBW4sxJEEx2/t0u5T7ew+Pefe8/UCpmGFzlEsPHmew Rw1/pk92N3m96LDMM/OaUaOweVAdK+1t+qqG5uxL1vSR7JcN314a2lWDz63YK3Mi/Xar8m4rNSPi Y17X/6kXc5csNeZxStlwaEuJSOFBruBb8lRS13rtvCrkI2cmGs9FS5xowANL4lGHAkqA8MeMiuf/ kQofqsQAqm2umc6X7yl4Uss3ffwpvpeKmDWkqM2WBM1xAIYpDRBf90C73SbK/aBRiCAEATqS+fea K0J7ADideqFJG2lVs6bsw3YH2ac5A9Fd8V2CuYAfKwravcwF73WxizAam92xQqIBley4EHxP51jW C12jk6cAlVv/s9P6QMVWiIGos+0hPn9fp3n0oFjb7cLzyInIFTnMD3N16LPcMgRommkRGS3IM/NX mAnpQHnc+VZmsst1vpXdC4ZNvZGCTKiYCXXV5F+e4deXp2ZXMo65vcBdIqk/QhKk9NYHHO+UL4Vi IzZ9H2xBTBcAyuyPOFuRg0V/SP+Kc4Ufs38v5CfAqKSBWGHCvjSYh7+F+k6AP5KisciUtUJERGZ8 GyxPtH0qhJF+Vh2PI9aDD4EEnktho17wXDDuiqb9SQdmeFdETlMm1tBF2n4SIgZ34rwm4n8fBrr3 NiZi4gzmGzNoRpQ/e3U/ZV3kJb/Hxb9EVA4kxK/H9Oc6bTpoFvo6rpFt/PmbpCUd/d7OlA6eNzVc OIPRwBVZcWPI6cmmZYsAom66FBFFbRgy8qz+GUUvBmXnzDJ41C2TrmGqK14k47zArf7AJLQlTyyX Ga++jAb4jczZxoEEWSw0SaeM9+Q7qyaIzDU59W3Q0IzhCRdMT1kmjJNpiVQxjkjY0pRmM/OJctZ4 HaWjRNfJS/4+Wy+az7fbUFYhaOJCNCsu4eLZq3sau+eLWXuPdm4LJp/0WpM/U4DLA4sDsswVDATQ oCMTxWfPtLGAI9VkNXWKPWwCt+2d48AZcpJF1aIjMjFY4veUzLNkKT5yIenVC9ZCBMKE9xcaCNYA tONm/Wp6gdwrmE9Ub0kW1zL1t/2eqi8tErc/442kbwuCCVgGOY99crFbdZksFOxP8W7LWVr90OHF xCW5gmit2UjBKQ7pkd8BITNSOfBHnXckgOD6yY5mtIi5/a4gwvC7/l5eEyU92dWOYq/g11Q/cSJd HfyMSCW2UAbm7Rj4b7RA4XvPrvUdHEFzlF3vXKCo4PC9C0g+KElU89h+t/8un/S5Tffw261+6Eg8 pb2kAUFrDXglCGaJtxdbAgb3t83yq02ukOgTDte7K8rm5+xpGWoE/1zHrrz7q3t4TThveLFvKQ8R c8a36qxrRwqDKDvkJSyQZOjy+EOwOgo7x9gLSZsZ/qQVW00k0OYdhyd8c/UKe0C91q/+mg6dSYlF Sla8f8nfkz0ybkGaleo/tlTuol7tUn6ehg8Ypj+re4Y+SPvo4INhtdDPBcTjW/q8h6g640H/vcbO kEuKbAmEViByBZTZdAWg8ECtbehX4M9UI9NdwA9NVaaczUZf/TwVknt0uLEWAp9Ownq4he84ZV/q HpZVJs5FhMC6C4df1C9Ak119ObbVIEiCXe14n/g89gsE3bBBfNQ2JeVIqTDdBHTX4awh2EjxZNn2 rI7c/GdsbRfDcIZwq34uQUsimP6E0J1o/PkioPKn7sUsS8KpN5gxyx8eZwqfhuYaO8tinRSIfL2N ykv08x1Cd7sDs7BmVPoaZBqyRf6FCLOCsmUtA2CmHCwCLIyyOf1SYJuE+6YJGiVbxRCnbKlB1jSA Sr2+oUpxO9G6N+NqSYdspsuBo0zy/Rp1FfjCDpoPqleGRwR1dUG6GS02t3Ow8Griwq4p1FN+lUKg dvq5lIO1hP1mCXiEWT9sGxXy/ZXYuiM40hkjS9EauJaqki7MYQXHSZfSCWDX8unxBr9CbXBgfjhf YST9xMndsjckYr7tycm3HWdDBpPRrLB5jkSZ/PPIK2wchOii5mrDJ0AuH9UZIN8c3e9X1AbQdR8x YoiY5INcRKE9TftIneRjlRZhnUikDVsdQJiEuGk2tfCqqnorFaQYSXd5kBiYEQiQTlMDB+fO+dAh rFx+yXanpvCCQgCrjTUgxCZx0y2nGuWHI55YLzZ1JOxCuBMNMYduOuRbXiP2zd/+81CHJZg1pp4S p4fsPgovoK50z8mpEF2N1eP3O+han9RdyTz1+dyfn3I/R0/T2hiucdq0UTYkz760m7djIsXrNTSx lkpID85u1zzvf9zIPsBsREp5ayDdAZg0dquulR5kIDNCjSx9ZaOo4Fa1R4vZ3q6ptfFfwr7jVTym g+Wma7EKYhZYMEU2vpngSyem+3tXsqkdgb/hLScDoWpbRlcNf1enXl3vRjjWazDGlg/jcwiDFyV7 Hnrt6sBltYOo2lk8RFPCjcNGzY4D0cS2aCy4KQEQ5ZkRgiaHkI6lW3FWHYSlaq3tt96MpLdBVG9Z fbwu1rek3F8mFDru95hLY0UjM4VEAH7CRJdV9ySERyseyXinwGQJDPMCaH32zmvmpeRUaH2mNuSf nPYAp8j3zPWkU5N4I3Fjn88WTjHczDZMU8yJtBgQ2TeLrmofE0etjba17hAzKe42V63tBiArm5Nv 3brdw12afHWFLqNTj98aVD5KiaPh9zCL6TFvMlPvGYx8G3UuO+zK2q40W75Hx5MqACwhgKtYNLpy 7ncWFHEaCqSnlsrNaeA2i3Nt0j+Bx5H9jdoiPSNP9LqmoLL5wdlaBVqGAjXKSr/f9myRZ07BTSnv cuHZwF9JkByQIhBlkJzUP+5RxnhWUW/OBFGvRLQ5q4CS1G4TkuJqVeNL54Wu2Q0RRx63nDpJXqzt 9WuMJLmwQS/4qxyWuZLKjV679nwIaHitk+3uQpjSulXQXwaEdYqXOyCxP+rK2isBsE/n9n8MQ+cA scv6WH57YSll4cCo0JR4Ch4BpzgCjOsos8qunS1AFsCd0ZNjpqXUv19Zag6gEte/IeFfZCuUuWD8 dTY5KB/cK1FJnNfuPgULg77gdICQRyZVVqdTMU/Kz9vQ+L7/t0C8mpOkl1ZOzNexQ9kX3q1UJAE0 ASDvYXhek0zmVrywaXUamwpDWfr//ja7N27ryaVuAm5w/VpedDZHoH4WhwdQxW3GZ/UTQscu77Yq HKjqwrvLhHBmEnvfYdzJkoHHWwoMs6Klzt4z8PY818ddD7RVUDjTiSVBF6/N++A/UhGALY29Ij+v TH+FrZ8xvPDEIi6trXWtKUxADZbz0Uc0H72DYSHqLUiT3FG9VzDUxirJgvu/Wvqk0ibtG37591Qk P9NMTY1D3v0XubxZ5vd/iG+fTTIbH9k6GcWcziGrvCygg0JLU6EsQbhRfA/Nl0vvaDXsOd3CGGLp HezUX3YshDBiZ2XyCPwGSgLQ/dkiyl4y0KtpIMPjqvqYsblWQYQunWOu2pBwzPdzgvt3b3AzoJNa q2P+mCgiIYz/FIRIZE9UpNv91Bdok0yKK7vfpzIi8OLO0mEz5zeGp5pjGdrSZH039Upxz8ZbPbN6 T0lgO7VXEqpZ7zFH7i5/4Jp5cpJE0OI0FaSH/dkYtWsYBwXHeTxxqtPItHJaX0hDGO6awozk3bNC gfdR/pB0vnXTiLQJQb++FqGwKlIxM2Q2tuMRZzuG2R3FGd9J5F0yn22E7Zjq8nfZjFVhWhea7YwO q0E8QbCCH4Ths1BJxbrOY/tOr7o5LvyGZmCi8oDVflK10PSGWlprWLroSsFSx+rw1AQIrAM4SpAL Zek4ORj7S7Kvw+if/CnpwlAeDMAexdE4vWWMRygJQg9+zUmUGC/xlIAUXn7CR6iqf1tj/tu/8jgX 9QA8opF/rc/qhzoeJs3vj8wcUFcdH4M/Hsuyc6xC+T/gpxjDhZOw4D3138WdxDVnuHXXOFxbnp7t 6Q43TNWKmzWH6PYeu8O0CrDafDIZWKgussADRfE2XK4KpzFSFxFuACMNZimiBZ/Ze3O8fy9mPqRT pvPM1Ax+nGvAIx0Kcu5DRCqi3ZAYXhFkgE6UEf7t7LTCxeZqu/p42SiKmCZHKJFu/eXppj1b9q1c Ixf1AmdGVopAnVcSbRa+4MLoNmFt2EYiqJHD2U19t1PqrJSsiXiAZYGkgxW9urWYZWcLe5F3I65k QMHEDOJQRLQQu9OBeY05BO2zRZbNDl/S+PrP7Dfhx8Fu6hu3sbH9BGhqdCQnOlMEtoFmwxBGugqT XliqIPqPdDLjehyCgwU+K9wHHlLpq8OC0cIEM12jc8/eBws+TuooYdirHc+yFlNz8BuTFw9rmxj3 iDJjAeGzF6zgfsNtUDSBRZhkbSPx1OdRds1hCpcDDW3n7/KJ/TDii36qiI7nps6b21pjos6SAyMG Tz009AJVy6c6lseacqcU1+rQEtqi+f4oWiIDPBJLSlk/ZasNwhYaQw+QtY5TZwQm4evanLfKHQwy c7IA7hPIk8L8geX14h1IKZ4ql1tpTr3yn00BPSM1cxQHDf+R2z550/scLBOgan4MFvszPMJm36mo RLW0G259WaBycJ6n7CPJTAyfhEDbSTnPmv5wStX7OLB0sAgggNUYyX9F7SIb+4h8FXQ/QjwUsWT9 lGHn5FuXx1WDrBfz/UgafuuVinETSFVAT0ZDHzpQPcPSD90vLhSgQhvwOdOWXtZBnC0UAraM3oCw 5bKLgB12DUDOF8y4t0Utv4+xpMioj5Pjz0M4RJ01vTNvBUX6V+geyCmVrVDGQXG5UozBX1di+h2V dOGQAtWiUL+Nu4a+hAP2fqLK+D0YW4wTBJqVlqojPZvpDfVGpQMtKK7Ym+exEbtlN7B+s3LNNF6Q geJfhsTKXotxzoNVZgA+aRCtiGn6pxPpmlrm6Gg6joG6kJF+Eu9ynL0ZRTEuUr21W7gXTvYv/YrO 5yZwfy5YTADYFeRQj0r3o7gAgyQxvf1TsTfZN5GoEl8KZXiB/sroon6Byq9qQ7dLQBYcRkMzfCj9 4ydnydSXR5ZKwCFRqwFLEmvt3Ut8Z5QBga0Gqqwd7gX/YWF6JYuyb4XLKPOKR+9h2e9IPsyDSvpC 3pM6RZrfLTRuYOaEfCLsW/tTjCJL0Ai9J9VmvJRc5hNkpENQ+cxDeyYNuXvi3DQD2GXPZFT34JPq +EUEl3p8Qzwf7jb04l7Zi1Hkpib6RNx1TEUbCVbqnLYLGctbYpVSFbcDU60B93Jbur8pvxuLHZ15 QVpRy1bcCAC2D2NDLUDMWJwSor5wzZPM9QoHfZ3AqnaXX1ngNYJSMd4g7lL004X6NEWoTFWyZlfa XslOnfIoY4XPcLbzuQoB5wm+uo/CHBubhjByUfGuzCZnpX+mpMA4ucm5yBx5Ub6ANvbN0mHkfI03 F3enBPwE2zmP1Qlx2D2hc3NGkQALhRd2Vlt4gexYPY2ngPLPi7tyKGf9QiriTspdeRU5BuYmsMGO YjLANNarR6K04/B5wu5a72Pm5YH+IphrfapFuPoICNoRZtE8oEReWjmzwo64nW2WVUcgSvYFwtuv rY/7bIVW2/W5/64BrzP9DbqQo0885z/ZNUkacpOeFFcXD7ogfe/jZmy/Z6NXp19vrbpcCQVnjEc3 Gy2QK9dQGgOEQqyvRiu4BMBUxPe2ulWff+bc3SbP6BfbgWWDliEBt8551iGDBojdPhWhTWtOcYMl e0TnphfD/cNh3RxWqIBfhVA+Ea60ajX+GzN9p02T5Pug7U6m1G0OcQ3SZVWxSmI0S4/aDsTTYjhN eYD13HHGvD4A4e5hTJUyVX0clfkfBVCzcXMzTK327eR6r/LebtGEVyE/Vun9os2IkaIYxAuJjWrC 9JC17L482iF073T25MorwdeCCs3KaYJWOCkYp7DcTWY5tpv/HXl5nco0Bp34/RYkCaLvoPNMX+Tf kbzZCUxx0b0VY2EEOcX8YvQMf4W8DhX/vE4Uigqwnd0I3y98mgK21IXuXFY7lM58NiJL7evmICWD nrOTW/56JwIhC3pAD5b8Q7MI5Ob9pBNzfSDWO77YAmY2xWwb80WPwzn81A5N6UXD+m/garQBHOp2 GL/7NqmiqQ8lVEoORBeDsc2rwZ+nDMJDzRRncXnD8u3i5roumlHnhocdmmMGwEKpv4WP/6p2Qoqs Meokw+tq5tpue0ZscSm/6GXJngF2AMGQ2kkXA19B06m0b2wkJOGlfDo4Ds2blWFJHYkwZT3PWLFz iIb09lGLmrHP1Ibduq7zoGoTX6LhvxOiBMnNWRGS79FosR//gB56Yigc52510BKYKFcLFkGM3L5e tQkgkFm1JLUiUddAqrnR6ERI7TBAeeGoJvAXNt9txsj4tNQB4RU8vMOxoJBjxhiojKqds5SmCi0f w7zfltoyNksq4sRsK0THEqhN2qrPu87gRo+kzaffbCYjBA9WO1r2g4x6ybR5/Ada5eDCHmETt62J EH9FmKqnQ4u8HycgVIHAvYNf+YSmluJF+ErCwXzmZLbpPo36WSWXlsabAhP4VrEKdpNK2p5FVnuM dVOzhEstL+f+9UDvxx+XXVczPk2doO+Fy6htdoYziJcgylQd9/F/gL49vIPbSam2nf4w/0W4wUyN 7P1eBXpotnQz5SUlJ744fWUGWPT+mQsw1uaBlPRvaUbZvuyAm1saPWBDLORU45FHFi0KYbvmrV2P eRHFKIOdBGXCHU5KLDV5kqmmNF3EZM0UEMyl23YvTEf4OZDdaC4MKQoMWXMGy7y6e27/ZvUiRMKh ZGQJCAINX8q1Qd6kfQz6Z03V2u6q+UZrVrRng4t9VRN0preOsFskEH76AXfWxDpZNPwdpsrG9KPH c7RoosFdiyD+DcCI5DqKCRPY+a3YSeAtj279GKQX2CFlFaxNa/WnQsS8gE37uox9ckefdEsZdJXl H3ODushnk76tWad8hFOxUSRDNC/iL+zS46UmWpNnku6O3C+YGbwYWnSOa+rdeaHdoSXsJPHwRJKw S4i/w1i6kpKS7JeETjJYdyDsBVvTWixJqq1HYL5yc9KYHWaODvH4k9kZvnc5+WG6ZKynKvXmixZC bXaxNAcDEC8v2jDbC6W4p4BWFIkuUc98m4aie5jemGddE/B7ol4HDsHJais5nVU/KKLLwgFmo9wO KJXWHPqy7XmFJrQnEVHAGGe+3SOvjdWJ2LHpiZeSk4Ena56ibyh3oZdgft67lyCTLc4gyTqRNOEY aUjkzVZcHYYGKX8Q4GMPMM0ZMoCA7UW7bvZG0Hezp2Aisx6/exuE9vSuJvD5FWw3VaM/n/hHSJ4c nvqr0NGFmIxRm5BzqRHBXw7s8DHpqfhoFcCkXGlVliNmMBHtmWdmO7sdAluwbAkSrP5C2TCS3cdM OeGX41ug6JspQSrumn3iu3j5JcPiry2S1fBKzoz+oPxapEmh3MfH+aG3+laWfTONrUGLlpyOg6oI l6oOE11VMxAaJkXwyjjj7pNZBIWvGxAzZHlGPqSKiUoI4p03Skj2fNa+DUNq8E0VUyN7TfyeKZss q+LhsIUmWhQTzaBWKI7bFV8KL9hIr0gSY9CFWvHLSPITcpyt5SSB/H0588qYU/zbATGye/Y/7jUt uxz9xXcn/Keo9gukWRv/X0JuN/ipnWNHpjXzvpnRVBAF99vMH17scQ9eTKJWHrpxrtLWpmqNE59+ Eqr2Ek6203To+sNacJbqTNu0B/fddNAAdVT6iczkEZ3ZzvAlBpxlg1TZfqqAhL/cZcPqwLpVXBsv dLApkqXvtEllR/PZUmjs7JpB+2QBXOzAYrX41xvagPy8g7vaiU312l4fXTRuE04ocuOY3dTaZVPi 1cb5b/GtqcK3x5mnMxYfKXDygmov2zNRi3o+7K4Nq6Hge/40uWCRZWY7VNkIX/uBwcMSX6E3InTT qIO5rHgTnrVxtu2HkPpBJ+RtCeAY0fx3UwN5MjHBDi4fuXkU2S/honbPVvWpDbIvYtYQJ1kQZDYN 1Rh6ccuYoZ7BqK1pefPGUkjzCW2cUTdlrDZwGV/tEGsxZUd2UNXyw8QidmTGYfvznA+9lkeSq85K kQqq9cmaMU7zWMFEO/AY/K5eKfRNKFN1RprE1NYIPqF7yWGqvlRFHkGTvApGURE7unSVlfR6rxBs kGC9XCw1xbA/3siOhEu2rLE1g8AnvD28c5bIsDDiDyrPCvt4lE0r1kNTfbaeJgBMeXZcAt4C7NDV x7zXi9ygRAmsD5jaOHp6EGQRZcVWGV6Jvil/NdyBazJjEaPwddXHn1x6MeIuMEYgc0PBXD8WNnJ7 qcebuPnR1BPflthTwhUWPDpbnGCYhWgPUxWnbWHKG4SxLePgJ70s9XcbNeFB0g6aoliHyRCF543b nZnxv7D6ATxhO0od4JSfVWEJPVC/pn4JGYcqqURzniNpDUOPtHGHmZl74PDYP3uIHMUI3mwRupbs V5yCwxBXomvKrEc0kj9PjKuMTQz/Jr/jPhMKYTbRZowq+prhgClN/A59BBSaIQx+Qz6mo6n6reFs c8wzPlLLbwZiksPQUWPP4hxno92WozsLj/MmiMuOrqGHJNM5pnMbr7+hQJG5wYQ0JM+mWvRj63jU twecvAnYHSmOmwGjcKonYePpY0fRYnYM3VLyRexCQq1QHV0H/UTkqVme2mKIinG+RPtXcZEFP49h Mvq2VGr3p0eirIR644o00u3OwSBKj/MTgjz7Brr6x7nIvfcS1gUC4GHUMNOZvSzG0eY4uQkPRaxB OFGfeWFdkrEwfIXpSWF4IdqUa1kTm16RDI9RyXmzRAHCpzc5+6sqcbbfHamDwxDhqvYDGen69lCa 7+mIURLdwBQG0PjxH8MwRjq2aEM4qaBKUpzY10aPEKZKgkjtfnWA7GBouDmBW/DkjQtQ4u0REtvG UeuusevSUcxzfEqMZCw91kt5xoeH/N6Kroa9nrV6/AY9VXJAj8wFLIHpM2fq/+Fzz3aZCr9d3Asz 6xsLO5ENJiBDQzAUA5bNG9WRyHLz6M1iskOWFgF+/VtnKvBL2MmBsYabkbxC3rGwB1en+Dp8tsnl H0WTqs5esGPzTu8Rhm/EuLhuzM5uBjBJb08ifzEXQjDzKBiU6nMapaMrG0pTtg5WAdl1z2OuFJXC 8k27g+jAFenF50Mf3+D0bqiH0G7tuHHrnYl0hCvZ6K8R89JuauSUQ0/aYDNVCwrwLQrp8AcxobAo PGiKRggkTh8YHD4bC/sc/G2x7/Z4ScvnYTuV2pvgPe+nmBbojmEdorVOmLOmGqWJxIyvlciNfwXO eVJ19SZFU+Lh6yklq9FBHp2SJAB5YC09A0E1JNAHL1uo4QBGcdleWi+15Xvqi7ZM3UbCk5Yjr6ZI GdSvDde0kZuItqaygLPHK1ONzy9nufoYH3Hjx5lDzb+SPCBGmV4DKRRmFMdtYQmmByRTI849Vknd FlZzsjxM+yo6YpBD5p9jfNiQP+Cit8Xo+oCDhf+WuWyu2FxdoOwnM7bPojSe4rbLmOIyiHsNOJN8 A4JReRfJK0kml1F5LFnpJVIFpj4vyYM+VqV0F3BWcBR8MQQcixzMx4vFyvaAmYeBmkB9x6A4QNXI zT+vy6+GkdVz3bv9TY2Z4wMBogrMWHBtshxJNE49SRYN6E4l+6FAVqeDY3uQTihwRjoEURwAFmDP 7yadqrcVhH7t1Iw9eNq/41qhmOodw7eg49bNWtuo2wmPtOPcZayDh3ykWEEQv2lqqShdVZ0jfZ30 JxymZ19CcEpBXQdkDyzPTXsP/YhnMi79UMQD1l7bXxF87ol6uODtRaX0Oa6QhE/xgpC1wIusjiXr vMBw4YAaR+1DSaDQV6aVw2/n1AF4IT0+bnUGKwdRbRcXt2NaWe9LVkwn0iO/XhQ+spYKCqXODUKV VTMaolKK18wfv+HY2oClhKhN3vp/J7WADlmUD+fyohmyv+ZDzeLiDhtVlCACJHMHhidkU6z8akxw yAVZO5FdZullHrnFRziKhhN24iCDL5wo90Oe+KkJ0rpqvY48A6OsTxXw+gkaxfOY/M4KiOvjAFCW uwktQq4nuiGBPqgRzzQPl2JCSKloQQeJJTGtOygMpi1fjXkFScEA5iix/9swoOmjD6aTawfWUwJa nA4I8pOuh5jFtNrtpH2vihZsaMIG8EfuLd0lu3CZ5agwTc6CFg2802TVtro8skt+fW05Ng5Oey72 OPBendamSPG/iDiqGTohpyM3TsX/DQTG8LUa82LM9utP2tjSPD2mTJ4Ah1buaPU1df+nIyeZmjLq s6lVpnTKY5VMnH1aNpg2S7ZpadenHf7Q0y7sEr9Atz5AUrivGjHP3010stCtauZFpxj9DIZV3+h6 6w20IeNcTwP+Cyb/xzBAn9crwY33CecbjV9MTr9NLk6if+H8nXqjwZeTb+QqkU1zdoAZXUa43WIP r4juXtJ4xiKSc9Rxr4PwVZXsC2GC+UJye3cR6go+yP4Fzr3dVJ6W3BB4ctFVPkWGSPUtoUHK5Jdy nKWkBm6MhaoZYYqJ9XZvlKINdaAI3keoJPtLygu+XcnxCAD6nHIm4FM1LdU38Gsvw+0kH6mn9ytw /tgCeWTwIRE+3UeoCttJi8e0HXukon41l6wD+3iLkr80lIqBb56wQOtsowiifqbr6kjYsmdxyGtY SJwgADLnEP0m4M7ZPxiK4A0NU2EYKCtQ+i/zP/1BneH8e3fIyWxar/ZiNv/taMh5uWWVKI6Scg+B oy7i8Ivrq7+gqYZti5tnFeFTIHLDGiZM9dP8fPSfwhcaca09Bz18FQD/PSM2ygMEJZP7xVta++Ff CIT5UPtu9GO3R9YKWRLaiQr1rp44wKKO9Ne0mIh0tadQRK7ZLj19wvYeVaOa6kXiRqFD+lizVgkY HwIK2QU+BpAdOBd86QwGNK4Qeo2p71uoRfxOnlvpZ1AUpkID5d+q1tbdiK+Rh49wfru2PzSQo+op itff44iMjcByoF+9fecBb0Pj7wpEELlzTAMMDyExMMA+MRnuJSrB1xxX7E8HT0E75Hp8BuIx5Onw d6OqpZRsg8eDWGK+8VNVv7TMDdagiuOBw3LOzsMmh535i3h1+aJZM+mOOngTM+FEHAXUFzvWsC87 ogI2K31PfF0iOUhP9hQYb8Km4WiVpv5sTpyYnhdFwor/JMX6vRWdrmXyv6khsbzcFO6Xw8Bzh/qe QJ35SwVy+Xrifhbzu2oUo4VGeWFiQj5ONjdNGz4jEsngOkiA1BUYaa8VbTtg6ZvnT/XRrjO4lIsR y7Ex1MWsD7nf1MSXq8n5TbODZSOEtiwChO3k85oZubZ3H5KKdAdAdnklYSBNKPAERUjx5ypKKd0t QTSWP3F5ZgZq1FYA24sPDZ5V+ouLLdXyufW118era97eYUDSrUwkp9UGELrgiMuP85CwjhV9+hhM IkZoAuc5YyCldW4pje0JWEOzR0kBnjV6oHg7avgnNHF5DhSLZ/CbmxrTDd7Wk/Lxi+bnoUlLo9Ym AMHet+5S8abRLuLu3L/mH3WSnwRLgh2PNoouDfQEJmqJfGQ2oG7c+/tLRqlCWm6SpCxqWQJt4k/z ONXKpwzou4mYuGNuF6EgY/qz6Rr/x9H80ML2zMr1QIm67BN0ruXwWrZTphe9Cmf6njxG1dcqH7R3 YNVpfkCaTmhDVFhLeDrzUSzyz2CTrPM8gGqfsYFIVwDYGC8Wot4DD9+BoOjBeKX0yiGXoeXoeNv4 tiuaGZhNpwBQ+rMABL+uM8uJyi24MZZOZ19wRCJjdgPm43Vy2NtICZi/XZhDJznPAlk+j/NZO7S4 6wCpaIsU3mSwGcAJk+vI1Ujh8ewszvRQGd5aLNPWDvG5pAypgNE3bXC3IVkriVAVHdZc3vJ0SYWz DnJynX9lMXZCl/9eQMynAR+3nu4jvTPeMqYDsBCY03ofybpVE+0TKe3/P+b65+rNoBTUuNwLgZvb OFknryUZN7At/xbdUrrBnUhlr5o8L5UhYAl4fWbdMn/2hgBVyUPtRrMpwVuhsos/tqVgV9FnUhbG WQwOuwL8Hv3P2fPghoGTdBoJoFshU8Uo6zb4O6f2rTSb3v5QXbviJgfB6HKcKMsM9DRyPhJEDfNJ oxna5q1LvdGUXaWxbVrMQCku2wBeNcr0p2Lz78JQiyeOCUBBKodxeT2TWNwH+zaHz9guUtHZWTCA 7VP+Z+4jpB20fT9c1gkX4IfCcJAA8KjXDN+OaXqWeUVDqCQ3WpLrFFCBodF3opXjPf4O9l+IRnod RaB2em7YcYZdY/v/IS2vwTDvgs16zHrZRJMQoE9FSad4XpDlBrbclMfP3PPsBwsA3F5aa6YEOHuq 7+5PypPOQoJ8t1GhVwF3ko9Ea4XpNhJG/0JtazeleWICr+I4xLf/IadM91lvZ8RPO1zGdrmb/BC/ qvAD2YipyC4o5+lmP/Pg7QyepaP/S7N72HvfJc1byFB/XrLVWwKP7jB2ggjiJ0tejmBoltlsZ75t CQ5xQukvWyD+WhoU80KAyMxu2M/BMx6W/wB4RL3NjXTa5sVz5uWc6JevXaiVyy2DeYeUOX/PsyVX t3sMW7JN/AV/q940TL3aU2X1DWBkSmDoBgAAAItkJAjrDDPbZP8zZIkj/wPr6OgMAAAASAvA6QwA AAAxFQPFG8HDBQUc+jfoDQAAAJALwOkMAAAAMT7B6Jj5C8DDI8RI+egMAAAAC8ZI6QwAAAAxFQPC M8XDHbpy/zf56AwAAACYM8fpCAAAADENM8fD+TPH6PX///8z22SPA1voAAAAAIPAm4s8JFiB7xVi dwGD8JxodY51OVmB8SqOAjjoDQAAAIvHkOkNAAAAMSorx9a4frcCOMMzw0gzxugLAAAAC8DpCQAA ADEOK8cTwfjDQPgrwOgNAAAAM8D86QwAAAAxNSPH1iPBkMOYE8b5c8roDAAAAIvH6QsAAAAxHpAL wUALxMMjxOjy////6AwAAAADxfjpCgAAADEVM8H4M8XDA8UlH9AEOAPPvq7mBDiB9uX+BDgrwSvb gcszBPM3kDEZuNxDBTgr8IHG20MFOIvCuCkM8zcD2A2cZgU4QUFBQVGLzuMGWenU////WegNAAAA g/Bt6QgAAAAxCwvEQMOLwBvAYQPD+cPoDQAAAJgbxekLAAAAMQvWM8KLxfjDC8NI6AsAAAD46QsA AAAxOhPHkBPH+cMjwCPEQOjv////i9Nb6MAAAADoCwAAAIPoGukIAAAAMQvWwfixw/nW6AoAAACQ 6QsAAAAxKSvAE8SYw/kbxIvF6AwAAAAzwukJAAAAMTPB+D/Dg/gd/OgPAAAAJdigFyLpCAAAADEZ kAvFC8DDI8Xo8////+gNAAAAg/AX6Q0AAAAxGhPASCPHwx0UzxwiK8BSM8BADzFa6AwAAAAzxekK AAAAMTXB0CsTwEjD+ZgjwegNAAAAweCu6QoAAAAxK8HAryPFwxvB6PP/////FXgUQAAbx0hkZ/82 AACLxGRnowAAM8CBEDd0ISL//1lZX17Jw8zMjBQAAA48Pzn/////7hUAAAAQAADUFAAA4DtAOf// //9QFgAASBAAAMwUAADjO0A5/////24WAABAEAAAAFBAAAAAAAAAAAAAAAAAAAAAAAC6FQAA+hQA AAYVAAAWFQAAIhUAAC4VAABCFQAAUhUAAGQVAAB4FQAAjhUAAKQVAADsFAAAzhUAANwVAAAAAAAA XBYAAAAAAAAmFgAAQhYAABgWAAAKFgAA/BUAAAAAAACGAEV4aXRQcm9jZXNzAMwCV3JpdGVGaWxl AEsBR2V0U3RkSGFuZGxlAADwAmxzdHJjcHlBAAD2AmxzdHJsZW5BAAAdAUdldE1vZHVsZUhhbmRs ZUEAABEBR2V0TGFzdEVycm9yAAC7AEZvcm1hdE1lc3NhZ2VBAABSAlNldENvbnNvbGVUaXRsZUEA AP4AR2V0RXhpdENvZGVQcm9jZXNzAAC9AldhaXRGb3JTaW5nbGVPYmplY3QA6wBHZXRDdXJyZW50 VGhyZWFkSWQAAOIAR2V0Q29uc29sZVRpdGxlQQAAwwFMb2NhbEFsbG9jAADXAEdldENvbW1hbmRM aW5lQQBLRVJORUwzMi5kbGwAAC4AQ2hhclRvT2VtQQAAoAFMb2FkU3RyaW5nQQAyAENoYXJVcHBl ckEAAFoBR2V0V2luZG93VGhyZWFkUHJvY2Vzc0lkAADTAEZpbmRXaW5kb3dBAFVTRVIzMi5kbGwA AFAAU2hlbGxFeGVjdXRlRXhBAFNIRUxMMzIuZGxsAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAA ------=_NextPart_000_00D8_0147DA5A.BA8C5AF0 Content-Type: application/octet-stream; name="SPD beta RC notes.doc" Content-Disposition: attachment; filename="SPD beta RC notes.doc" Content-Transfer-Encoding: base64 0M8R4KGxGuEAAAAAAAAAAAAAAAAAAAAAPgADAP7/CQAGAAAAAAAAAAAAAAABAAAATgAAAAAAAAAA EAAAUAAAAAEAAAD+////AAAAAE0AAAD///////////////////////////////////////////// //////////////////////////////////////////////////////////////////////////// //////////////////////////////////////////////////////////////////////////// //////////////////////////////////////////////////////////////////////////// //////////////////////////////////////////////////////////////////////////// //////////////////////////////////////////////////////////////////////////// //////////////////////////////////////////////////////////////////////////// ///////////////////////////////////////////////////////////////////////////s pcEATSAJBAAA8BK/AAAAAAAAEAAAAAAABAAA7xUAAA4AYmpiauI94j0AAAAAAAAAAAAAAAAAAAAA AAAJBBYAIkoAAIBXAACAVwAA7xEAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAD//w8AAAAA AAAAAAD//w8AAAAAAAAAAAD//w8AAAAAAAAAAAAAAAAAAAAAAGwAAAAAAPoBAAAAAAAA+gEAAPoB AAAAAAAA+gEAAAAAAAD6AQAAAAAAAPoBAAAAAAAA+gEAABQAAAAAAAAAAAAAAA4CAAAAAAAAgBgA AAAAAACAGAAAAAAAAIAYAAAAAAAAgBgAAAwAAACMGAAAxAAAAA4CAAAAAAAACioAAPYAAABcGQAA AAAAAFwZAAAAAAAAXBkAAAAAAABcGQAAAAAAAFwZAAAAAAAAXBkAAAAAAABcGQAAAAAAAFwZAAAA AAAArSkAAAIAAACvKQAAAAAAAK8pAAAAAAAArykAAAAAAACvKQAAAAAAAK8pAAAAAAAArykAAAAA AAAAKwAAIAIAACAtAABKAAAArykAABUAAAAAAAAAAAAAAAAAAAAAAAAA+gEAAAAAAABcGQAAAAAA AAAAAAAAAAAAAAAAAAAAAABcGQAAAAAAAFwZAAAAAAAAXBkAAAAAAABcGQAAAAAAAK8pAAAAAAAA iBoAAAAAAAD6AQAAAAAAAPoBAAAAAAAAXBkAAAAAAAAAAAAAAAAAAFwZAAAAAAAAxCkAABYAAACI GgAAAAAAAIgaAAAAAAAAiBoAAAAAAABcGQAAjgAAAPoBAAAAAAAAXBkAAAAAAAD6AQAAAAAAAFwZ AAAAAAAArSkAAAAAAAAAAAAAAAAAAIgaAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAXBkAAAAAAACtKQAAAAAAAIgaAABiBQAAiBoAAAAAAADqHwAA HgAAABUiAAAYAAAA+gEAAAAAAAD6AQAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAYSIAAAAAAABcGQAAAAAAAFAZAAAMAAAAAMzT67rJ wAEOAgAAchYAAIAYAAAAAAAA6hkAAIIAAAAtIgAACAAAAAAAAAAAAAAAYSIAAEwHAADaKQAAMAAA AAoqAAAAAAAANSIAACwAAABqLQAAAAAAAGwaAAAcAAAAai0AAAAAAABhIgAAAAAAAIgaAAAAAAAA DgIAAAAAAAAOAgAAAAAAAPoBAAAAAAAA+gEAAAAAAAD6AQAAAAAAAPoBAAAAAAAAAgDZAAAAU1BE IDEuMCBCZXRhIFJDIGZvciBXaW5kb3dzIE5UIDQuMA0NT3BlbiBJbmNpZGVudHMNVGhpcyBpcyBh IGxpc3Qgb2Ygb3BlbiBpbmNpZGVudHMgdGhhdCBhcmUgY3VycmVudGx5IGF3YWl0aW5nIHJlc29s dXRpb24gYW5kIGNvbnRhaW4gZGV0YWlscyBhcyB0byB3aGV0aGVyIGFueSBpbmNpZGVudCB3b3Vs ZCBiZSBoYXZpbmcgYSBoaWdoIGltcGFjdCBvbiB0aGUgQmV0YSByZWxlYXNlLg0NSW5jaWRlbnQg Tm8uCSAgICAgICAgUHJpCQlBYnN0cmFjdA02Mjk4NAcyB093bmVkIERpc2sgTGlzdCBiZWNvbWVz IGVtcHR5IHdoZW4gY29ubmVjdGluZyB0byBjbGllbnQgbm90IG9uIFNBTi4HBwc2MTU5OQcxB2Ns aWVudCBjYW5ub3QgYmUgY29ubmVjdGVkIG11bHRpcGxlIHRpbWVzIHdpdGggcmVhZC93cml0ZSBh Y2Nlc3MHBwc2MTIzMQcxB0xvYWRpbmcgQ29uZmlndXJhdGlvbiBkb2VzIG5vdCByZW1vdmUgb2xk IGNvbm5lY3Rpb25zBwcHNjEyMTgHMwdObyBPcHRpb24gdG8gYWRkIHRoZSBTZXJ2ZXIgQ29tcHV0 ZXIgYXMgYSBDbGllbnQgaW4gdGhlIE5ldHdvcmsgTmVpZ2hib3Job29kBwcHNjEyMDIHMwdXYXkg dG8gVXBncmFkZSBmcm9tIFJlYWQtb25seSB0byBSZWFkL3dyaXRlIHNob3VsZCBiZSB0cnkuBwc2 MTE5MgczB1NlbGVjdGlvbiBvZiBob3N0IGZyb20gTmV0d29yayBOZWlnaGJvcmhvb2Qgc2hvdWxk IHByb3ZpZGUgb3B0aW9uIG9mIGJvdGgNY29ubmVjdCBhbmQgZGlzY29ubmVjdCBldmVuIHRob3Vn aCBhIGhvc3QgaXMgYWxyZWFkeSBjb25uZWN0ZWQuBwcNQWxsIHRoZSBhYm92ZSBJbmNpZGVudHMg YXJlIGVuaGFuY2VtZW50cyB0aGF0IHdlcmUgcHJvcG9zZWQgYW5kIGFyZSB0byBiZSBkZWZlcnJl ZCB0byB0aGUgbmV4dCByZWxlYXNlIG9mIHRoZSBwcm9kdWN0LiBUaGVzZSBpc3N1ZXMgIHdpbGwg bm90IGJlIGEgcHJvYmxlbSBmb3IgdGhlIGN1cnJlbnQgQmV0YSByZWxlYXNlIHVubGVzcyBzb21l IGN1c3RvbWVyIHNwZWNpZmljYWxseSBpbmRpY2F0ZXMgdGhlaXIgdXNhYmlsaXR5Lg1SZXNvbHZl ZCBJbmNpZGVudHMNVGhpcyBpcyBhIGxpc3Qgb2YgaW5jaWRlbnRzIHRoYXQgd2VyZSByYWlzZWQg YWdhaW5zdCBoaWdoIHNldmVyaXR5LCBwcmlvcml0eSAxIG9yIDIgIGJ1Z3MgYW5kIGVuaGFuY2Vt ZW50cyBpbiB0aGUgcHJvZHVjdC4gVGhlc2UgaW5jaWRlbnRzIGhhdmUgYmVlbiByZXNvbHZlZCBm b3IgdGhlIGN1cnJlbnQgQmV0YSBSZWxlYXNlIENhbmRpZGF0ZSBvZiB0aGUgcHJvZHVjdC4NDUlu Y2lkZW50IE5vLgkgICAgICAgIFByaQkJQWJzdHJhY3QNNjI5MzcHMwdEaXNrIGluZm9ybWF0aW9u IGlzIG5vdCByZWZsZWN0ZWQgaW1tZWRpYXRlbHkgYWZ0ZXIgb3duaW5nIGEgZGlzay4HRG9uZQcH NjIyNTUHMQdTUEQgR1VJIGNyYXNoZXMgd2hlbiB0cnkgdG8gY29weSBzaGFyZXMgd2l0aG91dCBj b25uZWN0aW5nIHRvIGFueSBjbGllbnQHRG9uZQcHNjE2MTYHMQdNb2RpZnlpbmcgZXhwb3J0IHNo YXJlIG9uIEdVSSBkb2VzIG5vdCByZWZsZWN0IGluIGNvbW1hbmQgbmV0IHNoYXJlIAdEb25lIAcH NjE1OTgHMgdDYW4gbm90IGFkZCBpbXBvcnQgc2hhcmUHRG9uZQcHNjI5NDcHMgdGMSBvcGVucyB0 d28gSGVscCBXaW5kb3dzBwcHNjE0MzMHMgdSZWQgdGFnIG1pc3Npbmcgb24gc2hhcmVkIGRpc2sg Zm9yIGEgb3duZWQgZGlzawdEb25lIAcHNjEyMjkHMQdWeGdGcyBjcmFzaGVzIG9uIFN5bmMgVXAH RG9uZQcHNjEyMjUHMgdNb2RpZnkgYW5kIERlbGV0ZSBFeHBvcnQgU2hhcmUgZm9yIGEgUmVhZCBP bmx5IFNlcnZlciBzaG91bGQgYmUgZ3JheWVkIG91dC4HRG9uZQcHNjEyMjQHMgdJbmFwcHJvcHJp YXRlIG1lc3NhZ2Ugd2hlbiB3ZSB0cnkgdG8gZXhpdCB3aGVuIGhvc3RzIGFyZSBjb25uZWN0ZWQu B0RvbmUHBzYxMjIzBzIHUGFzdGUgU2hhcmUgd2lsbCBQYXN0ZSB0byBhbGwgQ2xpZW50cwdEb25l Bwc2MTIyMgcyB0RlbGV0ZSBFeHBvcnQgU2hhcmUgb3B0aW9uIHRha2VzIHRvIHRoZSBNb2RpZnkg RXhwb3J0IFNoYXJlIFdpbmRvdwdEb25lBwc2MTIyMQcyB0NvcHkgYW5kIFBhc3RlIFNoYXJlIE9w dGlvbiBzaG91bGQgYmUgcHJlc2VudC4HRG9uZQcHNjEyMjAHMgdEaXNrIERhdGEgbm90IHNob3du IGZvciB0aGUgT3duZWQgRGlza3MgaW4gdGhlIFNoYXJlZCBEaXNrcyBsaXN0B0RvbmUHBzYxMjE5 BzIHV3JvbmcgSWNvbiBmb3IgUmVhZCBPbmx5IFNlcnZlciBpbiBjYXNlIG9mIG11bHRpcGxlIEdV SXMgb3BlbgdEb25lBwc2MTIxNw02MTAyNCAgICAgICAgICAgICAHMg0yB09wdGlvbiBmb3IgVW5p bXBvcnRpbmcgU2hhcmUgc2hvdWxkIG5vdCBjb21lIHVwIG9uIGEgUmVhZCBPbmx5IENsaWVudAdE b25lBwc2MTIxNgcyB0luYXBwcm9wcmlhdGUgTWVzc2FnZSB3aGVuIHdlIHRyeSBhbmQgYWRkIGV4 cG9ydCBzaGFyZSBhbmQgZG8gbm90IHByb3ZpZGUgY2xpZW50IG5hbWUHRG9uZQcHNjEyMTUHMgdP d25lZCBEaXNrIExpc3QgcmVtYWlucyBlbXB0eSBpbiBjYXNlIG9mIFNlcnZlciB3aXRoIFJlYWQg T25seSBBY2Nlc3MHRG9uZQcHNjEyMTQHMQdJbmFwcHJvcHJpYXRlIE1lc3NhZ2Ugd2hlbiB1IHRy eSBhbmQgcmVtb3ZlIGFuIG93bmVkIGRpc2sgaGF2aW5nIGV4cG9ydCBzaGFyZXMHRG9uZQcHNjEy MTEHMQdHVUkgaGFuZ3Mgb24gZGlzY29ubmVjdAdEb25lIAcHNjEyMDgHMgdTeW5jIFVwIGRvZXMg bm90IHdvcmsgcHJvcGVybHkHRG9uZQcHNjEyMDQHMQdTUERHVUkgY3Jhc2hlcyB3aGVuIHUgZXhw YW5kIFdPUktHUk9VUAdEb25lBwc2MTIwMwcyB0Nvbm5lY3RpbmcgdG8gbm9uLWV4aXN0ZW50IHN5 c3RlbXMgZG9lcyBub3QgZGlzcGxheSBwcm9wZXIgZXJyb3IHRG9uZQcHNjEyMDANNjEwMjEHMQ0z B05vdGhpbmcgaGFwcGVucyB3aGVuIHUgdHJ5IHRvIGNvbm5lY3QgdG8gU2VydmVyIHdpdGggUmVh ZCBPbmx5IEFjY2VzcwdEb25lBwc2MTE5OAcyB0luY29uc2lzdGVudCBoZWFkaW5ncyBmb3IgQWRk IEV4cG9ydCBTaGFyZQdEb25lBwc2MTE4Mw0HMQdSaWdodCBDbGlja2luZyBiZWxvdyB0aGUgdHJl ZSBjYXVzZXMgU1BER1VJIHRvIENyYXNoB0RvbmUHBzYxMTc4BzEHU2F2ZSBDb25maWd1cmF0aW9u IHNhdmVzIGZpbGVzIHdpdGguKSBleHRlbnNpb24HRG9uZQcHNjExNzMHMQdNYW5hZ2VtZW50IENv bnNvbGUgU2hvcnRjdXQgaW4gdGhlIFN0YXJ0IE1lbnUgZG9lcyBub3Qgd29yawdJbnN0YWxsIGlz c3VlBwc2MTE3MAcxB0V4cG9ydCBzaGFyZSBkaXNwbGF5cyAoZGlzazUscGFydGl0aW9uOTk5OSkN DUZpeGVkIGJ5IGluY2lkZW50IDUzMTE4LgdEb25lBwc2MTAyMwcxB0V4cGFuZGluZyB3b3JrZ3Jv dXAgZnJvbSBuZXR3b3JrIG5laWdoYm9yaG9vZCBjcmFzaGVzIHRoZSBTUEQgR1VJB0RvbmUHBzYw OTM1BzMHU2hhcmVkIGRpc2tzIHRyZWUgaXMgbm90IGRpc3BsYXlpbmcgYWxsIFNBTiBzaGFyZWQg ZGlza3MHRG9uZQcHBwcHBwc2MTEzNwcyB3dyaXRlIHBlcmZvcm1hbmNlIGltcHJvdmVtZW50IChX UklURV9USFJPVUdIKS4HBwc2MTEzNQcyB0ZTUUEgdGVzdHMgZmFpbHVyZSwgY2hhbmdlIGNsaWVu dCBjb2RlIHRvIGdldCBWYWxpZERhdGFMZW5ndGgHBwc1MjE2MQcyB3VuYWNxdWlyZWQgcmVzb3Vy Y2UgbWF5IGJlIHJlbGVhc2VkLgcHBzUyMDE3BzIHRVhUX0xPT0tBU0lERV9CVUZGRVIgZmxhZyBt YXkgZ2V0IGNvcnJ1cHRlZAcHBzQ4Njk4BzEHYWx3YXlzIGNyZWF0ZSBhc3NvY2lhdGVkIElSUHMg Zm9yIHRoZSBpL28HBwc1MzAyNQcxB2ZpZGVsaXR5IGR5bmFtaWMgZGlzayBwcm9ibGVtBwcHNTIw MzMHMgdJcnAgYW5kIElycENvbnRleHQgbWF5IGJlIHJlZmVyZW5jZWQgYWZ0ZXIgdGhleSBhcmUg ZnJlZWQHBwc2MjE1NgcyB0dldEV4dGVudE1hcCBlcnJvcnMgcHJldmVudCBkaXJlY3QgcmVhZHMH BzYyNTcyBzMHQ29sbGVjdCBtb3JlIHN0YXRpc3RpY3MgaW5mb3JtYXRpb24HBzYzNjUwBzEHRGF0 YSBNaXNtYXRjaCB3aGVuIHdyaXRpbmcgZnJvbSBhIE5UIHNlcnZlciB0byBhIFcyayBjbGllbnQu BwcNU3RhdGVtZW50IG9mIFJlYWRpbmVzcw0NV2UgaGF2ZSBiZWVuIGRvaW5nIHRoZSB4eCBjeWNs ZXMgb2YgcmVncmVzc2lvbiB0ZXN0aW5nLiBUaGUgdGVzdHMgY292ZXIgdGhlIGZvbGxvd2luZ4Uu DQ1Db25maWd1cmF0aW9uL0FkbWluaXN0cmF0aW9uDSANDQ1Sb2xlcyBhbmQgUmVzcG9uc2liaWxp dGllcw1UaGlzIEJldGEgUmVsZWFzZSBjYW5kaWRhdGUgc2hvdWxkIHNhdGlzZnkgdGhlIGJhc2lj IHJlcXVpcmVtZW50cyBvZiB0aGUgY3VzdG9tZXIgYW5kIGFzIGZhciBhcyBwb3NzaWJsZSBwcm92 aWRlIGdvb2QgcGVyZm9ybWFuY2UuIFRoaXMgcHJvZHVjdCBpbiBubyBtYW5uZXIgc2hvdWxkIGJl IGhhcm1mdWwgdG8gdGhlIGN1c3RvbWVycyBzZXR1cC4gQmVzaWRlcyBhbGwgdGhpcyBpdCBzaG91 bGQgc2F0aXNmeSBhbGwgdGhlIGZ1bmN0aW9uYWxpdHkgdGhhdCBpcyBkb2N1bWVudGVkIGluIHRo ZSBBZG1pbmlzdHJhdG9ycyBndWlkZS4gVGhlIGNhbmRpZGF0ZSByZWxlYXNlIHNob3VsZCBwcm92 aWRlIHNvbWUgc2NvcGUgdG93YXJkcyBmdXR1cmUgZW5oYW5jZW1lbnRzIHRvd2FyZHMgY29uY2Vy bmVkIGFyZWFzIG9mIGZ1bmN0aW9uYWxpdHkuDQAAAAAAAAAAAAAAAAAAAAAAAAQAACQEAAAzBAAA AQUAAHQGAAB1BgAACAcAAAkHAAD+BwAAEQgAACwKAAAtCgAAEAsAABELAABjCwAAaQsAAOgLAADu CwAAJQwAAHMMAACNDQAAkw0AAD0OAAA+DgAARA4AAGUOAABrDgAAFQ8AABYPAAC1EAAAzhAAAG8R AABwEQAAdREAAHYRAAB4EQAAphEAAKkRAACuEQAArxEAALERAADuEQAA8REAAPYRAAD3EQAA+BEA APkRAAAdEgAAIBIAAFsSAABcEgAAXRIAAF4SAACHEgAAihIAAI8SAACQEgAAkRIAAJISAACvEgAA shIAALcSAAC4EgAAuRIAALoSAADzEgAA9hIAAPsSAAD8EgAA/RIAAP4SAAAmEwAAKBMAAC0TAAAu EwAALxMAADATAABTEwAAVRMAAFoTAABbEwAAXBMAAF0TAACZEwAAmxMAAJwTAACzEwAALhQAAEkU AADvFQAAAP0A+AD49QD9APgA+ADwAPAA8ADwAPjwAPAA+ADrAPgA+PAA+AD48AD4APgA+AD4APgA +AD4APgA+AD4APgA+AD4APgA+AD4APgA+AD4APgA+AD46P0A/QAAAAAAAAAAAAAAAAAAAAAEQ0oY AAAJQioGcGj/AAAACUIqAXBoAAAAAARDShYAAAlCKgJwaAAA/wAEQ0oaAFkABAAAIwQAACQEAAAz BAAA3QQAAN4EAAABBQAABwUAAAkFAABNBQAATgUAAP0AAAAAAAAAAAAAAAD7AAAAAAAAAAAAAAAA +QAAAAAAAAAAAAAAAPcAAAAAAAAAAAAAAAD3AAAAAAAAAAAAAAAA9wAAAAAAAAAAAAAAAPEAAAAA AAAAAAAAAADxAAAAAAAAAAAAAAAA8QAAAAAAAAAAAAAAAPEAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAYAABYkAUlmAQAAAAABDwAAAQMAAAEAAAABAQAACgAEAADvFQAA/gAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAACAQEBTgUAAE8FAABVBQAAVwUA AJgFAACZBQAAmgUAAKAFAACiBQAA2AUAANkFAADaBQAAaiwBAAAAAAAAAAAAAGQAAAAAAAAAAAAA AABkAAAAAAAAAAAAAAAAZAAAAAAAAAAAAAAAAGQAAAAAAAAAAAAAAABqAAEAAAAAAAAAAAAAZAAA AAAAAAAAAAAAAGQAAAAAAAAAAAAAAABkAAAAAAAAAAAAAAAAZAAAAAAAAAAAAAAAAGpcAQAAAAAA AAAAAAAAAAAAAAAAAAAGAAAWJAFJZgEAAAAAlAAAFiQBFyQBSWYBAAAAApZsAAXWGAQBAAAEAQAA BAEAAAQBAAAEAQAABAEAAAjWXAAElP8yB/sIAjAsOAAGngcAAAAAAAAAAAAAAAAAAAAAAAbJAQAA AAAAAAAAAAAAAAAAAAAABgcnAAAAAAAAAAAAAAAAAAAAAAAGKggAAAAAAAAAAAAAAAAAAAAAE9Yw AAAA/wQBAAAAAAD/BAEAAAAAAP8EAQAAAAAA/wQBAAAAAAD/BAEAAAAAAP8EAQAAFPYDmDgX9gMA ABj2AwAAGtYQAAAA/wAAAP8AAAD/AAAA/xvWEAAAAP8AAAD/AAAA/wAAAP8c1hAAAAD/AAAA/wAA AP8AAAD/HdYQAAAA/wAAAP8AAAD/AAAA/zTWBgABCgNsAGH2AwAAAAvaBQAA4AUAAOIFAAAvBgAA MAYAADEGAAA3BgAAOQYAAHQGAAD5AAAAAAAAAAAAAAAA+QAAAAAAAAAAAAAAAPkAAAAAAAAAAAAA AAD5AAAAAAAAAAAAAAAAZBABAAAAAAAAAAAAAPkAAAAAAAAAAAAAAAD5AAAAAAAAAAAAAAAA+QAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAACUAAAWJAEXJAFJZgEAAAAClmwABdYYBAEAAAQBAAAEAQAABAEAAAQBAAAE AQAACNZcAASU/zIH+wgCMCw4AAaeBwAAAAAAAAAAAAAAAAAAAAAABskBAAAAAAAAAAAAAAAAAAAA AAAGBycAAAAAAAAAAAAAAAAAAAAAAAYqCAAAAAAAAAAAAAAAAAAAAAAT1jAAAAD/BAEAAAAAAP8E AQAAAAAA/wQBAAAAAAD/BAEAAAAAAP8EAQAAAAAA/wQBAAAU9gOYOBf2AwAAGPYDAAAa1hAAAAD/ AAAA/wAAAP8AAAD/G9YQAAAA/wAAAP8AAAD/AAAA/xzWEAAAAP8AAAD/AAAA/wAAAP8d1hAAAAD/ AAAA/wAAAP8AAAD/NNYGAAEKA2wAYfYDAAAGAAAWJAFJZgEAAAAACHQGAAB1BgAAewYAAH0GAADH BgAABwcAAAgHAAAJBwAA/gcAABEIAADmCAAA5wgAAAoJAAAQCQAAfUwCAAAAAAAAAAAAAHcAAAAA AAAAAAAAAAB3AAAAAAAAAAAAAAAAdwAAAAAAAAAAAAAAAHcAAAAAAAAAAAAAAAB9AAAAAAAAAAAA AAAAdQAAAAAAAAAAAAAAAHMAAAAAAAAAAAAAAABxAAAAAAAAAAAAAAAAcwAAAAAAAAAAAAAAAHMA AAAAAAAAAAAAAABzAAAAAAAAAAAAAAAAdwAAAAAAAAAAAAAAAAAAAQMAAAEPAAABAAAGAAAWJAFJ ZgEAAAAAgQAAFiQBFyQBSWYBAAAAApZsAAXWGAQBAAAEAQAABAEAAAQBAAAEAQAABAEAAAjWRgAD lP8yB/sIAjAABp4HAAAAAAAAAAAAAAAAAAAAAAAGyQEAAAAAAAAAAAAAAAAAAAAAAAYHJwAAAAAA AAAAAAAAAAAAAAAT1jAAAAD/BAEAAAAAAP8EAQAAAAAA/wQBAAAAAAD/BAEAAAAAAP8EAQAAAAAA /wQBAAAU9gOYOBf2AwAAGPYDKgga1gwAAAD/AAAA/wAAAP8b1gwAAAD/AAAA/wAAAP8c1gwAAAD/ AAAA/wAAAP8d1gwAAAD/AAAA/wAAAP801gYAAQoDbABh9gMAAAANEAkAABIJAABVCQAAWgkAAFsJ AABhCQAAYwkAAKwJAACxCQAAsgkAALgJAAC6CQAA+QAAAAAAAAAAAAAAAPkAAAAAAAAAAAAAAAD5 AAAAAAAAAAAAAAAAZFwBAAAAAAAAAAAAAPkAAAAAAAAAAAAAAAD5AAAAAAAAAAAAAAAA+QAAAAAA AAAAAAAAAPkAAAAAAAAAAAAAAABkUAEAAAAAAAAAAAAA+QAAAAAAAAAAAAAAAPkAAAAAAAAAAAAA AAAAAAAAAAAAAAAAlAAAFiQBFyQBSWYBAAAAApZsAAXWGAQBAAAEAQAABAEAAAQBAAAEAQAABAEA AAjWXAAElP8yB/sIAjAsOAAGngcAAAAAAAAAAAAAAAAAAAAAAAbJAQAAAAAAAAAAAAAAAAAAAAAA BgcnAAAAAAAAAAAAAAAAAAAAAAAGKggAAAAAAAAAAAAAAAAAAAAAE9YwAAAA/wQBAAAAAAD/BAEA AAAAAP8EAQAAAAAA/wQBAAAAAAD/BAEAAAAAAP8EAQAAFPYDmDgX9gMAABj2AwAAGtYQAAAA/wAA AP8AAAD/AAAA/xvWEAAAAP8AAAD/AAAA/wAAAP8c1hAAAAD/AAAA/wAAAP8AAAD/HdYQAAAA/wAA AP8AAAD/AAAA/zTWBgABCgNsAGH2AwAABgAAFiQBSWYBAAAAAAu6CQAA/wkAAAUKAAAGCgAADAoA AA4KAAAnCgAALAoAAC0KAAAzCgAANQoAAE8KAAD5AAAAAAAAAAAAAAAA+QAAAAAAAAAAAAAAAGSc AAAAAAAAAAAAAAD5AAAAAAAAAAAAAAAA+QAAAAAAAAAAAAAAAPkAAAAAAAAAAAAAAAD5AAAAAAAA AAAAAAAAZJAAAAAAAAAAAAAAAPkAAAAAAAAAAAAAAAD5AAAAAAAAAAAAAAAA+QAAAAAAAAAAAAAA AAAAAAAAAAAAAACUAAAWJAEXJAFJZgEAAAAClmwABdYYBAEAAAQBAAAEAQAABAEAAAQBAAAEAQAA CNZcAASU/zIH+wgCMCw4AAaeBwAAAAAAAAAAAAAAAAAAAAAABskBAAAAAAAAAAAAAAAAAAAAAAAG BycAAAAAAAAAAAAAAAAAAAAAAAYqCAAAAAAAAAAAAAAAAAAAAAAT1jAAAAD/BAEAAAAAAP8EAQAA AAAA/wQBAAAAAAD/BAEAAAAAAP8EAQAAAAAA/wQBAAAU9gOYOBf2AwAAGPYDAAAa1hAAAAD/AAAA /wAAAP8AAAD/G9YQAAAA/wAAAP8AAAD/AAAA/xzWEAAAAP8AAAD/AAAA/wAAAP8d1hAAAAD/AAAA /wAAAP8AAAD/NNYGAAEKA2wAYfYDAAAGAAAWJAFJZgEAAAAAC08KAABQCgAAUQoAAFcKAABZCgAA iQoAAI8KAACQCgAAlgoAAJgKAACxCgAAtgoAAPkAAAAAAAAAAAAAAABk/AAAAAAAAAAAAAAA+QAA AAAAAAAAAAAAAPkAAAAAAAAAAAAAAAD5AAAAAAAAAAAAAAAA+QAAAAAAAAAAAAAAAGScAAAAAAAA AAAAAAD5AAAAAAAAAAAAAAAA+QAAAAAAAAAAAAAAAPkAAAAAAAAAAAAAAAD5AAAAAAAAAAAAAAAA AAAAAAAAAAAAAJQAABYkARckAUlmAQAAAAKWbAAF1hgEAQAABAEAAAQBAAAEAQAABAEAAAQBAAAI 1lwABJT/Mgf7CAIwLDgABp4HAAAAAAAAAAAAAAAAAAAAAAAGyQEAAAAAAAAAAAAAAAAAAAAAAAYH JwAAAAAAAAAAAAAAAAAAAAAABioIAAAAAAAAAAAAAAAAAAAAABPWMAAAAP8EAQAAAAAA/wQBAAAA AAD/BAEAAAAAAP8EAQAAAAAA/wQBAAAAAAD/BAEAABT2A5g4F/YDAAAY9gMAABrWEAAAAP8AAAD/ AAAA/wAAAP8b1hAAAAD/AAAA/wAAAP8AAAD/HNYQAAAA/wAAAP8AAAD/AAAA/x3WEAAAAP8AAAD/ AAAA/wAAAP801gYAAQoDbABh9gMAAAYAABYkAUlmAQAAAAALtgoAALcKAAC9CgAAvwoAAAsLAAAQ CwAAEQsAABcLAAAZCwAAXQsAAGILAABjCwAAamgBAAAAAAAAAAAAAGQAAAAAAAAAAAAAAABkAAAA AAAAAAAAAAAAZAAAAAAAAAAAAAAAAGQAAAAAAAAAAAAAAABqSAEAAAAAAAAAAAAAZAAAAAAAAAAA AAAAAGQAAAAAAAAAAAAAAABkAAAAAAAAAAAAAAAAZAAAAAAAAAAAAAAAAGrQAAAAAAAAAAAAAAAA AAAAAAAAAAAGAAAWJAFJZgEAAAAAlAAAFiQBFyQBSWYBAAAAApZsAAXWGAQBAAAEAQAABAEAAAQB AAAEAQAABAEAAAjWXAAElP8yB/sIAjAsOAAGngcAAAAAAAAAAAAAAAAAAAAAAAbJAQAAAAAAAAAA AAAAAAAAAAAABgcnAAAAAAAAAAAAAAAAAAAAAAAGKggAAAAAAAAAAAAAAAAAAAAAE9YwAAAA/wQB AAAAAAD/BAEAAAAAAP8EAQAAAAAA/wQBAAAAAAD/BAEAAAAAAP8EAQAAFPYDmDgX9gMAABj2AwAA GtYQAAAA/wAAAP8AAAD/AAAA/xvWEAAAAP8AAAD/AAAA/wAAAP8c1hAAAAD/AAAA/wAAAP8AAAD/ HdYQAAAA/wAAAP8AAAD/AAAA/zTWBgABCgNsAGH2AwAAAAtjCwAAaQsAAGsLAACRCwAAlgsAAJcL AACdCwAAnwsAAOILAADnCwAA6AsAAO4LAAD5AAAAAAAAAAAAAAAA+QAAAAAAAAAAAAAAAPkAAAAA AAAAAAAAAAD5AAAAAAAAAAAAAAAAZEQBAAAAAAAAAAAAAPkAAAAAAAAAAAAAAAD5AAAAAAAAAAAA AAAA+QAAAAAAAAAAAAAAAPkAAAAAAAAAAAAAAABk9AAAAAAAAAAAAAAA+QAAAAAAAAAAAAAAAAAA AAAAAAAAAACUAAAWJAEXJAFJZgEAAAAClmwABdYYBAEAAAQBAAAEAQAABAEAAAQBAAAEAQAACNZc AASU/zIH+wgCMCw4AAaeBwAAAAAAAAAAAAAAAAAAAAAABskBAAAAAAAAAAAAAAAAAAAAAAAGBycA AAAAAAAAAAAAAAAAAAAAAAYqCAAAAAAAAAAAAAAAAAAAAAAT1jAAAAD/BAEAAAAAAP8EAQAAAAAA /wQBAAAAAAD/BAEAAAAAAP8EAQAAAAAA/wQBAAAU9gOYOBf2AwAAGPYDAAAa1hAAAAD/AAAA/wAA AP8AAAD/G9YQAAAA/wAAAP8AAAD/AAAA/xzWEAAAAP8AAAD/AAAA/wAAAP8d1hAAAAD/AAAA/wAA AP8AAAD/NNYGAAEKA2wAYfYDAAAGAAAWJAFJZgEAAAAAC+4LAADwCwAAHwwAACQMAAAlDAAAKwwA AC0MAABuDAAAcwwAAHQMAAB6DAAAfAwAAPkAAAAAAAAAAAAAAAD5AAAAAAAAAAAAAAAA+QAAAAAA AAAAAAAAAGQ8AQAAAAAAAAAAAAD5AAAAAAAAAAAAAAAA+QAAAAAAAAAAAAAAAPkAAAAAAAAAAAAA AAD5AAAAAAAAAAAAAAAAZDABAAAAAAAAAAAAAPkAAAAAAAAAAAAAAAD5AAAAAAAAAAAAAAAAAAAA AAAAAAAAAJQAABYkARckAUlmAQAAAAKWbAAF1hgEAQAABAEAAAQBAAAEAQAABAEAAAQBAAAI1lwA BJT/Mgf7CAIwLDgABp4HAAAAAAAAAAAAAAAAAAAAAAAGyQEAAAAAAAAAAAAAAAAAAAAAAAYHJwAA AAAAAAAAAAAAAAAAAAAABioIAAAAAAAAAAAAAAAAAAAAABPWMAAAAP8EAQAAAAAA/wQBAAAAAAD/ BAEAAAAAAP8EAQAAAAAA/wQBAAAAAAD/BAEAABT2A5g4F/YDAAAY9gMAABrWEAAAAP8AAAD/AAAA /wAAAP8b1hAAAAD/AAAA/wAAAP8AAAD/HNYQAAAA/wAAAP8AAAD/AAAA/x3WEAAAAP8AAAD/AAAA /wAAAP801gYAAQoDbABh9gMAAAYAABYkAUlmAQAAAAALfAwAALoMAAC/DAAAwAwAAMYMAADZDAAA 2wwAAN0MAAAjDQAAKA0AACkNAAAvDQAA+QAAAAAAAAAAAAAAAPkAAAAAAAAAAAAAAABkpAEAAAAA AAAAAAAA+QAAAAAAAAAAAAAAAPkAAAAAAAAAAAAAAAD5AAAAAAAAAAAAAAAA+QAAAAAAAAAAAAAA APkAAAAAAAAAAAAAAAD5AAAAAAAAAAAAAAAAZJABAAAAAAAAAAAAAPkAAAAAAAAAAAAAAAAAAAAA AAAAAAAAlAAAFiQBFyQBSWYBAAAAApZsAAXWGAQBAAAEAQAABAEAAAQBAAAEAQAABAEAAAjWXAAE lP8yB/sIAjAsOAAGngcAAAAAAAAAAAAAAAAAAAAAAAbJAQAAAAAAAAAAAAAAAAAAAAAABgcnAAAA AAAAAAAAAAAAAAAAAAAGKggAAAAAAAAAAAAAAAAAAAAAE9YwAAAA/wQBAAAAAAD/BAEAAAAAAP8E AQAAAAAA/wQBAAAAAAD/BAEAAAAAAP8EAQAAFPYDmDgX9gMAABj2AwAAGtYQAAAA/wAAAP8AAAD/ AAAA/xvWEAAAAP8AAAD/AAAA/wAAAP8c1hAAAAD/AAAA/wAAAP8AAAD/HdYQAAAA/wAAAP8AAAD/ AAAA/zTWBgABCgNsAGH2AwAABgAAFiQBSWYBAAAAAAsvDQAAMQ0AAIcNAACMDQAAjQ0AAJMNAACV DQAA2w0AAOANAADhDQAA5w0AAOkNAAD5AAAAAAAAAAAAAAAA+QAAAAAAAAAAAAAAAPkAAAAAAAAA AAAAAABkUAEAAAAAAAAAAAAA+QAAAAAAAAAAAAAAAPkAAAAAAAAAAAAAAAD5AAAAAAAAAAAAAAAA +QAAAAAAAAAAAAAAAGR0AQAAAAAAAAAAAAD5AAAAAAAAAAAAAAAA+QAAAAAAAAAAAAAAAAAAAAAA AAAAAACUAAAWJAEXJAFJZgEAAAAClmwABdYYBAEAAAQBAAAEAQAABAEAAAQBAAAEAQAACNZcAASU /zIH+wgCMCw4AAaeBwAAAAAAAAAAAAAAAAAAAAAABskBAAAAAAAAAAAAAAAAAAAAAAAGBycAAAAA AAAAAAAAAAAAAAAAAAYqCAAAAAAAAAAAAAAAAAAAAAAT1jAAAAD/BAEAAAAAAP8EAQAAAAAA/wQB AAAAAAD/BAEAAAAAAP8EAQAAAAAA/wQBAAAU9gOYOBf2AwAAGPYDAAAa1hAAAAD/AAAA/wAAAP8A AAD/G9YQAAAA/wAAAP8AAAD/AAAA/xzWEAAAAP8AAAD/AAAA/wAAAP8d1hAAAAD/AAAA/wAAAP8A AAD/NNYGAAEKA2wAYfYDAAAGAAAWJAFJZgEAAAAAC+kNAAA4DgAAPQ4AAD4OAABEDgAARg4AAF4O AABkDgAAZQ4AAGsOAABtDgAAjA4AAPkAAAAAAAAAAAAAAAD5AAAAAAAAAAAAAAAAZJwAAAAAAAAA AAAAAPkAAAAAAAAAAAAAAAD5AAAAAAAAAAAAAAAA+QAAAAAAAAAAAAAAAPkAAAAAAAAAAAAAAABk tAAAAAAAAAAAAAAA+QAAAAAAAAAAAAAAAPkAAAAAAAAAAAAAAAD5AAAAAAAAAAAAAAAAAAAAAAAA AAAAAJQAABYkARckAUlmAQAAAAKWbAAF1hgEAQAABAEAAAQBAAAEAQAABAEAAAQBAAAI1lwABJT/ Mgf7CAIwLDgABp4HAAAAAAAAAAAAAAAAAAAAAAAGyQEAAAAAAAAAAAAAAAAAAAAAAAYHJwAAAAAA AAAAAAAAAAAAAAAABioIAAAAAAAAAAAAAAAAAAAAABPWMAAAAP8EAQAAAAAA/wQBAAAAAAD/BAEA AAAAAP8EAQAAAAAA/wQBAAAAAAD/BAEAABT2A5g4F/YDAAAY9gMAABrWEAAAAP8AAAD/AAAA/wAA AP8b1hAAAAD/AAAA/wAAAP8AAAD/HNYQAAAA/wAAAP8AAAD/AAAA/x3WEAAAAP8AAAD/AAAA/wAA AP801gYAAQoDbABh9gMAAAYAABYkAUlmAQAAAAALjA4AAJEOAACSDgAAmA4AAJoOAADBDgAAxg4A AMcOAADNDgAAzw4AABAPAAAVDwAA+QAAAAAAAAAAAAAAAGTUAAAAAAAAAAAAAAD5AAAAAAAAAAAA AAAA+QAAAAAAAAAAAAAAAPkAAAAAAAAAAAAAAAD5AAAAAAAAAAAAAAAAZDwBAAAAAAAAAAAAAPkA AAAAAAAAAAAAAAD5AAAAAAAAAAAAAAAA+QAAAAAAAAAAAAAAAPkAAAAAAAAAAAAAAAAAAAAAAAAA AAAAlAAAFiQBFyQBSWYBAAAAApZsAAXWGAQBAAAEAQAABAEAAAQBAAAEAQAABAEAAAjWXAAElP8y B/sIAjAsOAAGngcAAAAAAAAAAAAAAAAAAAAAAAbJAQAAAAAAAAAAAAAAAAAAAAAABgcnAAAAAAAA AAAAAAAAAAAAAAAGKggAAAAAAAAAAAAAAAAAAAAAE9YwAAAA/wQBAAAAAAD/BAEAAAAAAP8EAQAA AAAA/wQBAAAAAAD/BAEAAAAAAP8EAQAAFPYDmDgX9gMAABj2AwAAGtYQAAAA/wAAAP8AAAD/AAAA /xvWEAAAAP8AAAD/AAAA/wAAAP8c1hAAAAD/AAAA/wAAAP8AAAD/HdYQAAAA/wAAAP8AAAD/AAAA /zTWBgABCgNsAGH2AwAABgAAFiQBSWYBAAAAAAsVDwAAFg8AABwPAAAiDwAAJA8AACYPAABsDwAA cQ8AAHIPAAB4DwAAeg8AAKUPAABqcAEAAAAAAAAAAAAAZAAAAAAAAAAAAAAAAGQAAAAAAAAAAAAA AABkAAAAAAAAAAAAAAAAZAAAAAAAAAAAAAAAAGQAAAAAAAAAAAAAAABkAAAAAAAAAAAAAAAAauQA AAAAAAAAAAAAAGQAAAAAAAAAAAAAAABkAAAAAAAAAAAAAAAAZAAAAAAAAAAAAAAAAAAAAAAAAAAA AAYAABYkAUlmAQAAAACUAAAWJAEXJAFJZgEAAAAClmwABdYYBAEAAAQBAAAEAQAABAEAAAQBAAAE AQAACNZcAASU/zIH+wgCMCw4AAaeBwAAAAAAAAAAAAAAAAAAAAAABskBAAAAAAAAAAAAAAAAAAAA AAAGBycAAAAAAAAAAAAAAAAAAAAAAAYqCAAAAAAAAAAAAAAAAAAAAAAT1jAAAAD/BAEAAAAAAP8E AQAAAAAA/wQBAAAAAAD/BAEAAAAAAP8EAQAAAAAA/wQBAAAU9gOYOBf2AwAAGPYDAAAa1hAAAAD/ AAAA/wAAAP8AAAD/G9YQAAAA/wAAAP8AAAD/AAAA/xzWEAAAAP8AAAD/AAAA/wAAAP8d1hAAAAD/ AAAA/wAAAP8AAAD/NNYGAAEKA2wAYfYDAAAAC6UPAACqDwAAqw8AALEPAACyDwAAtA8AAOkPAADu DwAA7w8AAPUPAAD3DwAAJxAAAPkAAAAAAAAAAAAAAABkEAEAAAAAAAAAAAAA+QAAAAAAAAAAAAAA APkAAAAAAAAAAAAAAAD5AAAAAAAAAAAAAAAA+QAAAAAAAAAAAAAAAPkAAAAAAAAAAAAAAABk+AAA AAAAAAAAAAAA+QAAAAAAAAAAAAAAAPkAAAAAAAAAAAAAAAD5AAAAAAAAAAAAAAAAAAAAAAAAAAAA AJQAABYkARckAUlmAQAAAAKWbAAF1hgEAQAABAEAAAQBAAAEAQAABAEAAAQBAAAI1lwABJT/Mgf7 CAIwLDgABp4HAAAAAAAAAAAAAAAAAAAAAAAGyQEAAAAAAAAAAAAAAAAAAAAAAAYHJwAAAAAAAAAA AAAAAAAAAAAABioIAAAAAAAAAAAAAAAAAAAAABPWMAAAAP8EAQAAAAAA/wQBAAAAAAD/BAEAAAAA AP8EAQAAAAAA/wQBAAAAAAD/BAEAABT2A5g4F/YDAAAY9gMAABrWEAAAAP8AAAD/AAAA/wAAAP8b 1hAAAAD/AAAA/wAAAP8AAAD/HNYQAAAA/wAAAP8AAAD/AAAA/x3WEAAAAP8AAAD/AAAA/wAAAP80 1gYAAQoDbABh9gMAAAYAABYkAUlmAQAAAAALJxAAACwQAAAtEAAAMxAAADUQAABxEAAAfxAAAIAQ AACGEAAAiBAAALQQAAC1EAAA+QAAAAAAAAAAAAAAAGRMAQAAAAAAAAAAAAD5AAAAAAAAAAAAAAAA +QAAAAAAAAAAAAAAAPkAAAAAAAAAAAAAAAD5AAAAAAAAAAAAAAAAZFABAAAAAAAAAAAAAPkAAAAA AAAAAAAAAAD5AAAAAAAAAAAAAAAA+QAAAAAAAAAAAAAAAPkAAAAAAAAAAAAAAAAAAAAAAAAAAAAA lAAAFiQBFyQBSWYBAAAAApZsAAXWGAQBAAAEAQAABAEAAAQBAAAEAQAABAEAAAjWXAAElP8yB/sI AjAsOAAGngcAAAAAAAAAAAAAAAAAAAAAAAbJAQAAAAAAAAAAAAAAAAAAAAAABgcnAAAAAAAAAAAA AAAAAAAAAAAGKggAAAAAAAAAAAAAAAAAAAAAE9YwAAAA/wQBAAAAAAD/BAEAAAAAAP8EAQAAAAAA /wQBAAAAAAD/BAEAAAAAAP8EAQAAFPYDmDgX9gMAABj2AwAAGtYQAAAA/wAAAP8AAAD/AAAA/xvW EAAAAP8AAAD/AAAA/wAAAP8c1hAAAAD/AAAA/wAAAP8AAAD/HdYQAAAA/wAAAP8AAAD/AAAA/zTW BgABCgNsAGH2AwAABgAAFiQBSWYBAAAAAAu1EAAAzhAAANMQAADUEAAA2hAAANwQAAAeEQAAIxEA ACQRAAAqEQAALBEAAGURAAD5AAAAAAAAAAAAAAAA+QAAAAAAAAAAAAAAAGRAAQAAAAAAAAAAAAD5 AAAAAAAAAAAAAAAA+QAAAAAAAAAAAAAAAPkAAAAAAAAAAAAAAAD5AAAAAAAAAAAAAAAAZBwBAAAA AAAAAAAAAPkAAAAAAAAAAAAAAAD5AAAAAAAAAAAAAAAA+QAAAAAAAAAAAAAAAAAAAAAAAAAAAACU AAAWJAEXJAFJZgEAAAAClmwABdYYBAEAAAQBAAAEAQAABAEAAAQBAAAEAQAACNZcAASU/zIH+wgC MCw4AAaeBwAAAAAAAAAAAAAAAAAAAAAABskBAAAAAAAAAAAAAAAAAAAAAAAGBycAAAAAAAAAAAAA AAAAAAAAAAYqCAAAAAAAAAAAAAAAAAAAAAAT1jAAAAD/BAEAAAAAAP8EAQAAAAAA/wQBAAAAAAD/ BAEAAAAAAP8EAQAAAAAA/wQBAAAU9gOYOBf2AwAAGPYDAAAa1hAAAAD/AAAA/wAAAP8AAAD/G9YQ AAAA/wAAAP8AAAD/AAAA/xzWEAAAAP8AAAD/AAAA/wAAAP8d1hAAAAD/AAAA/wAAAP8AAAD/NNYG AAEKA2wAYfYDAAAGAAAWJAFJZgEAAAAAC2URAABqEQAAaxEAAGwRAABtEQAAbhEAAG8RAABwEQAA dhEAAHgRAACnEQAAqBEAAPkAAAAAAAAAAAAAAABkFAAAAAAAAAAAAAAA+QAAAAAAAAAAAAAAAPkA AAAAAAAAAAAAAAD5AAAAAAAAAAAAAAAA+QAAAAAAAAAAAAAAAGTkAAAAAAAAAAAAAAD5AAAAAAAA AAAAAAAA+QAAAAAAAAAAAAAAAPkAAAAAAAAAAAAAAAD5AAAAAAAAAAAAAAAAAAAAAAAAAAAAAJQA ABYkARckAUlmAQAAAAKWbAAF1hgEAQAABAEAAAQBAAAEAQAABAEAAAQBAAAI1lwABJT/Mgf7CAIw LDgABp4HAAAAAAAAAAAAAAAAAAAAAAAGyQEAAAAAAAAAAAAAAAAAAAAAAAYHJwAAAAAAAAAAAAAA AAAAAAAABioIAAAAAAAAAAAAAAAAAAAAABPWMAAAAP8EAQAAAAAA/wQBAAAAAAD/BAEAAAAAAP8E AQAAAAAA/wQBAAAAAAD/BAEAABT2A5g4F/YDAAAY9gMAABrWEAAAAP8AAAD/AAAA/wAAAP8b1hAA AAD/AAAA/wAAAP8AAAD/HNYQAAAA/wAAAP8AAAD/AAAA/x3WEAAAAP8AAAD/AAAA/wAAAP801gYA AQoDbABh9gMAAAYAABYkAUlmAQAAAAALqBEAAKkRAACvEQAAsREAAO8RAADwEQAA8REAAPcRAAD5 EQAAHhIAAB8SAAAgEgAAaiABAAAAAAAAAAAAAGQAAAAAAAAAAAAAAABkAAAAAAAAAAAAAAAAZAAA AAAAAAAAAAAAAGQAAAAAAAAAAAAAAABqvAAAAAAAAAAAAAAAZAAAAAAAAAAAAAAAAGQAAAAAAAAA AAAAAABkAAAAAAAAAAAAAAAAZAAAAAAAAAAAAAAAAGrYAAAAAAAAAAAAAAAAAAAAAAAAAAAGAAAW JAFJZgEAAAAAlAAAFiQBFyQBSWYBAAAAApZsAAXWGAQBAAAEAQAABAEAAAQBAAAEAQAABAEAAAjW XAAElP8yB/sIAjAsOAAGngcAAAAAAAAAAAAAAAAAAAAAAAbJAQAAAAAAAAAAAAAAAAAAAAAABgcn AAAAAAAAAAAAAAAAAAAAAAAGKggAAAAAAAAAAAAAAAAAAAAAE9YwAAAA/wQBAAAAAAD/BAEAAAAA AP8EAQAAAAAA/wQBAAAAAAD/BAEAAAAAAP8EAQAAFPYDmDgX9gMAABj2AwAAGtYQAAAA/wAAAP8A AAD/AAAA/xvWEAAAAP8AAAD/AAAA/wAAAP8c1hAAAAD/AAAA/wAAAP8AAAD/HdYQAAAA/wAAAP8A AAD/AAAA/zTWBgABCgNsAGH2AwAAAAsgEgAAJhIAACgSAABUEgAAVRIAAFYSAABcEgAAXhIAAIgS AACJEgAAihIAAJASAAD5AAAAAAAAAAAAAAAA+QAAAAAAAAAAAAAAAPkAAAAAAAAAAAAAAAD5AAAA AAAAAAAAAAAAZNAAAAAAAAAAAAAAAPkAAAAAAAAAAAAAAAD5AAAAAAAAAAAAAAAA+QAAAAAAAAAA AAAAAPkAAAAAAAAAAAAAAABkoAAAAAAAAAAAAAAA+QAAAAAAAAAAAAAAAAAAAAAAAAAAAACUAAAW JAEXJAFJZgEAAAAClmwABdYYBAEAAAQBAAAEAQAABAEAAAQBAAAEAQAACNZcAASU/zIH+wgCMCw4 AAaeBwAAAAAAAAAAAAAAAAAAAAAABskBAAAAAAAAAAAAAAAAAAAAAAAGBycAAAAAAAAAAAAAAAAA AAAAAAYqCAAAAAAAAAAAAAAAAAAAAAAT1jAAAAD/BAEAAAAAAP8EAQAAAAAA/wQBAAAAAAD/BAEA AAAAAP8EAQAAAAAA/wQBAAAU9gOYOBf2AwAAGPYDAAAa1hAAAAD/AAAA/wAAAP8AAAD/G9YQAAAA /wAAAP8AAAD/AAAA/xzWEAAAAP8AAAD/AAAA/wAAAP8d1hAAAAD/AAAA/wAAAP8AAAD/NNYGAAEK A2wAYfYDAAAGAAAWJAFJZgEAAAAAC5ASAACSEgAAsBIAALESAACyEgAAuBIAALoSAAD0EgAA9RIA APYSAAD8EgAA/hIAAPkAAAAAAAAAAAAAAAD5AAAAAAAAAAAAAAAA+QAAAAAAAAAAAAAAAGQQAQAA AAAAAAAAAAD5AAAAAAAAAAAAAAAA+QAAAAAAAAAAAAAAAPkAAAAAAAAAAAAAAAD5AAAAAAAAAAAA AAAAZMgAAAAAAAAAAAAAAPkAAAAAAAAAAAAAAAD5AAAAAAAAAAAAAAAAAAAAAAAAAAAAAJQAABYk ARckAUlmAQAAAAKWbAAF1hgEAQAABAEAAAQBAAAEAQAABAEAAAQBAAAI1lwABJT/Mgf7CAIwLDgA Bp4HAAAAAAAAAAAAAAAAAAAAAAAGyQEAAAAAAAAAAAAAAAAAAAAAAAYHJwAAAAAAAAAAAAAAAAAA AAAABioIAAAAAAAAAAAAAAAAAAAAABPWMAAAAP8EAQAAAAAA/wQBAAAAAAD/BAEAAAAAAP8EAQAA AAAA/wQBAAAAAAD/BAEAABT2A5g4F/YDAAAY9gMAABrWEAAAAP8AAAD/AAAA/wAAAP8b1hAAAAD/ AAAA/wAAAP8AAAD/HNYQAAAA/wAAAP8AAAD/AAAA/x3WEAAAAP8AAAD/AAAA/wAAAP801gYAAQoD bABh9gMAAAYAABYkAUlmAQAAAAAL/hIAACcTAAAoEwAALhMAADATAABUEwAAVRMAAFsTAABdEwAA mhMAAJsTAACcEwAAsxMAALQTAAD5AAAAAAAAAAAAAAAAd7QAAAAAAAAAAAAAAPkAAAAAAAAAAAAA AAD5AAAAAAAAAAAAAAAA+QAAAAAAAAAAAAAAAHcYAQAAAAAAAAAAAAD5AAAAAAAAAAAAAAAA+QAA AAAAAAAAAAAAAPkAAAAAAAAAAAAAAAB3AAAAAAAAAAAAAAAAdQAAAAAAAAAAAAAAAHMAAAAAAAAA AAAAAABxAAAAAAAAAAAAAAAAAAABAAAAAQMAAAEPAACBAAAWJAEXJAFJZgEAAAAClmwABdYYBAEA AAQBAAAEAQAABAEAAAQBAAAEAQAACNZGAAOU/zIH+wgCMAAGngcAAAAAAAAAAAAAAAAAAAAAAAbJ AQAAAAAAAAAAAAAAAAAAAAAABgcnAAAAAAAAAAAAAAAAAAAAABPWMAAAAP8EAQAAAAAA/wQBAAAA AAD/BAEAAAAAAP8EAQAAAAAA/wQBAAAAAAD/BAEAABT2A5g4F/YDAAAY9gMqCBrWDAAAAP8AAAD/ AAAA/xvWDAAAAP8AAAD/AAAA/xzWDAAAAP8AAAD/AAAA/x3WDAAAAP8AAAD/AAAA/zTWBgABCgNs AGH2AwAABgAAFiQBSWYBAAAAAA20EwAADBQAAA0UAAAqFAAALBQAAC0UAAAuFAAASRQAAO8VAAD9 AAAAAAAAAAAAAAAA/QAAAAAAAAAAAAAAAPgAAAAAAAAAAAAAAAD4AAAAAAAAAAAAAAAA/QAAAAAA AAAAAAAAAP0AAAAAAAAAAAAAAAD2AAAAAAAAAAAAAAAA9AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAABDwAA AQMABQAACiYAC0YBAAABAAAACCAAMZBoAR+w0C8gsOA9IbAIByKwCAcjkKAFJJCgBSWwAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAFAARAAoAAQBpAA8AAwAAAAAAAAAAADgAAEDx/wIAOAAMAAYATgBv AHIAbQBhAGwAAAACAAAAGABDShgAX0gBBGFKGABtSAkEc0gJBHRICQRSAAFAAQACAFIADAAJAEgA ZQBhAGQAaQBuAGcAIAAxAAAAEAABAAYkAROk8AAUpDwAQCYAHgA1CIFDSiAAS0ggAE9KAgBRSgIA XAiBXkoCAGFKIAAAAE4AA0ABAAIATgAMAAkASABlAGEAZABpAG4AZwAgADMAAAAQAAMABiQBE6Tw ABSkPABAJgIaADUIgUNKFgBPSgIAUUoCAFwIgV5KAgBhShoAAAAAAAAAAAAAAAAAPABBQPL/oQA8 AAwAFgBEAGUAZgBhAHUAbAB0ACAAUABhAHIAYQBnAHIAYQBwAGgAIABGAG8AbgB0AAAAAAAAAAAA AAAAACoAQkABAPIAKgAMAAkAQgBvAGQAeQAgAFQAZQB4AHQAAAACAA8ABABDShYAhABlAAEAAgGE AAwAEQBIAFQATQBMACAAUAByAGUAZgBvAHIAbQBhAHQAdABlAGQAAAA3ABAADcYyABCUAygHvApQ DuQReBUMGaAcNCDII1wn8CqELhgyrDVAOQAAAAAAAAAAAAAAAAAAAAAAGABDShQAT0oDAFBKAwBR SgMAXkoDAGFKFAAAAAAA7xEAAAUAAEoAAAQA/////wAAAAAjAAAAJAAAADMAAADdAAAA3gAAAAEB AAAHAQAACQEAAE0BAABOAQAATwEAAFUBAABXAQAAmAEAAJkBAACaAQAAoAEAAKIBAADYAQAA2QEA ANoBAADgAQAA4gEAAC8CAAAwAgAAMQIAADcCAAA5AgAAdAIAAHUCAAB7AgAAfQIAAMcCAAAHAwAA CAMAAAkDAAD+AwAAEQQAAOYEAADnBAAACgUAABAFAAASBQAAVQUAAFoFAABbBQAAYQUAAGMFAACs BQAAsQUAALIFAAC4BQAAugUAAP8FAAAFBgAABgYAAAwGAAAOBgAAJwYAACwGAAAtBgAAMwYAADUG AABPBgAAUAYAAFEGAABXBgAAWQYAAIkGAACPBgAAkAYAAJYGAACYBgAAsQYAALYGAAC3BgAAvQYA AL8GAAALBwAAEAcAABEHAAAXBwAAGQcAAF0HAABiBwAAYwcAAGkHAABrBwAAkQcAAJYHAACXBwAA nQcAAJ8HAADiBwAA5wcAAOgHAADuBwAA8AcAAB8IAAAkCAAAJQgAACsIAAAtCAAAbggAAHMIAAB0 CAAAeggAAHwIAAC6CAAAvwgAAMAIAADGCAAA2QgAANsIAADdCAAAIwkAACgJAAApCQAALwkAADEJ AACHCQAAjAkAAI0JAACTCQAAlQkAANsJAADgCQAA4QkAAOcJAADpCQAAOAoAAD0KAAA+CgAARAoA AEYKAABeCgAAZAoAAGUKAABrCgAAbQoAAIwKAACRCgAAkgoAAJgKAACaCgAAwQoAAMYKAADHCgAA zQoAAM8KAAAQCwAAFQsAABYLAAAcCwAAIgsAACQLAAAmCwAAbAsAAHELAAByCwAAeAsAAHoLAACl CwAAqgsAAKsLAACxCwAAsgsAALQLAADpCwAA7gsAAO8LAAD1CwAA9wsAACcMAAAsDAAALQwAADMM AAA1DAAAcQwAAH8MAACADAAAhgwAAIgMAAC0DAAAtQwAAM4MAADTDAAA1AwAANoMAADcDAAAHg0A ACMNAAAkDQAAKg0AACwNAABlDQAAag0AAGsNAABsDQAAbQ0AAG4NAABvDQAAcA0AAHYNAAB4DQAA pw0AAKgNAACpDQAArw0AALENAADvDQAA8A0AAPENAAD3DQAA+Q0AAB4OAAAfDgAAIA4AACYOAAAo DgAAVA4AAFUOAABWDgAAXA4AAF4OAACIDgAAiQ4AAIoOAACQDgAAkg4AALAOAACxDgAAsg4AALgO AAC6DgAA9A4AAPUOAAD2DgAA/A4AAP4OAAAnDwAAKA8AAC4PAAAwDwAAVA8AAFUPAABbDwAAXQ8A AJoPAACbDwAAnA8AALMPAAC0DwAADBAAAA0QAAAqEAAALBAAAC0QAAAuEAAASRAAAPERAAAIAAAA ATAAAAAAAAAAgAAAAICYAAAAADAAAAAAAAAAgAAAAAAoAAAAAzAAAAAAAAAAgAAAAACYAAAADzAA AAAAAAAAgCQAAACYAAAADzAAAAAAAAAAgCQAAACYAAAADzAAAAAAAAAAgCQAAACpAAAAADAAAAAA AAAAgCQAAACpAAAAADAAAAAAAAAAgCQAAACpAAAAADAAAAAAAAAAgCQAAACpAAAAADAAAAAAAAAA gCQAAACZAAAAADAAAAAAAAAAgCQAAACpAAAAADAAAAAAAAAAgCQAAACpAAAAADAAAAAAAAAAgCQA AACpAAAAADAAAAAAAAAAgCQAAACpAAAAADAAAAAAAAAAgCQAAACZAAAAADAAAAAAAAAAgCQAAACp AAAAADAAAAAAAAAAgCQAAACpAAAAADAAAAAAAAAAgCQAAACpAAAAADAAAAAAAAAAgCQAAACpAAAA ADAAAAAAAAAAgCQAAACZAAAAADAAAAAAAAAAgCQAAACpAAAAADAAAAAAAAAAgCQAAACpAAAAADAA AAAAAAAAgCQAAACpAAAAADAAAAAAAAAAgCQAAACpAAAAADAAAAAAAAAAgCQAAACZAAAAADAAAAAA AAAAgCQAAACpAAAAADAAAAAAAAAAgCQAAACpAAAAADAAAAAAAAAAgCQAAACpAAAAADAAAAAAAAAA gCQAAACZAAAAADAAAAAAAAAAgCQAAACpAAAAADAAAAAAAAAAgCQAAACpAAAAADAAAAAAAAAAgCQA AACpAAAAADAAAAAAAAAAgCQAAACpAAAAADAAAAAAAAAAgCQAAACZAAAAADAAAAAAAAAAgCQAAACY AAAAADAAAAAAAAAAgCQAAACYAAAADzAAAAAAAAAAgCQAAAAoAAAAAzAAAAAAAAAAgAAAAACYAAAA DzAAAAAAAAAAgP4DAACYAAAADzAAAAAAAAAAgP4DAACYAAAADzAAAAAAAAAAgP4DAACpAAAAADAA AAAAAAAAgP4DAACpAAAAADAAAAAAAAAAgP4DAACpAAAAADAAAAAAAAAAgP4DAACpAAAAADAAAAAA AAAAgP4DAACZAAAAADAAAAAAAAAAgP4DAACpAAAAADAAAAAAAAAAgP4DAACpAAAAADAAAAAAAAAA gP4DAACpAAAAADAAAAAAAAAAgP4DAACpAAAAADAAAAAAAAAAgP4DAACZAAAAADAAAAAAAAAAgP4D AACpAAAAADAAAAAAAAAAgP4DAACpAAAAADAAAAAAAAAAgP4DAACpAAAAADAAAAAAAAAAgP4DAACp AAAAADAAAAAAAAAAgP4DAACZAAAAADAAAAAAAAAAgP4DAACpAAAAADAAAAAAAAAAgP4DAACpAAAA ADAAAAAAAAAAgP4DAACpAAAAADAAAAAAAAAAgP4DAACpAAAAADAAAAAAAAAAgP4DAACZAAAAADAA AAAAAAAAgP4DAACpAAAAADAAAAAAAAAAgP4DAACpAAAAADAAAAAAAAAAgP4DAACpAAAAADAAAAAA AAAAgP4DAACpAAAAADAAAAAAAAAAgP4DAACZAAAAADAAAAAAAAAAgP4DAACpAAAAADAAAAAAAAAA gP4DAACpAAAAADAAAAAAAAAAgP4DAACpAAAAADAAAAAAAAAAgP4DAACpAAAAADAAAAAAAAAAgP4D AACZAAAAADAAAAAAAAAAgP4DAACpAAAAADAAAAAAAAAAgP4DAACpAAAAADAAAAAAAAAAgP4DAACp AAAAADAAAAAAAAAAgP4DAACpAAAAADAAAAAAAAAAgP4DAACZAAAAADAAAAAAAAAAgP4DAACpAAAA ADAAAAAAAAAAgP4DAACpAAAAADAAAAAAAAAAgP4DAACpAAAAADAAAAAAAAAAgP4DAACpAAAAADAA AAAAAAAAgP4DAACZAAAAADAAAAAAAAAAgP4DAACpAAAAADAAAAAAAAAAgP4DAACpAAAAADAAAAAA AAAAgP4DAACpAAAAADAAAAAAAAAAgP4DAACpAAAAADAAAAAAAAAAgP4DAACZAAAAADAAAAAAAAAA gP4DAACpAAAAADAAAAAAAAAAgP4DAACpAAAAADAAAAAAAAAAgP4DAACpAAAAADAAAAAAAAAAgP4D AACpAAAAADAAAAAAAAAAgP4DAACZAAAAADAAAAAAAAAAgP4DAACpAAAAADAAAAAAAAAAgP4DAACp AAAAADAAAAAAAAAAgP4DAACpAAAAADAAAAAAAAAAgP4DAACpAAAAADAAAAAAAAAAgP4DAACZAAAA ADAAAAAAAAAAgP4DAACpAAAAADAAAAAAAAAAgP4DAACpAAAAADAAAAAAAAAAgP4DAACpAAAAADAA AAAAAAAAgP4DAACpAAAAADAAAAAAAAAAgP4DAACZAAAAADAAAAAAAAAAgP4DAACpAAAAADAAAAAA AAAAgP4DAACpAAAAADAAAAAAAAAAgP4DAACpAAAAADAAAAAAAAAAgP4DAACpAAAAADAAAAAAAAAA gP4DAACZAAAAADAAAAAAAAAAgP4DAACpAAAAADAAAAAAAAAAgP4DAACpAAAAADAAAAAAAAAAgP4D AACpAAAAADAAAAAAAAAAgP4DAACpAAAAADAAAAAAAAAAgP4DAACZAAAAADAAAAAAAAAAgP4DAACp AAAAADAAAAAAAAAAgP4DAACpAAAAADAAAAAAAAAAgP4DAACpAAAAADAAAAAAAAAAgP4DAACpAAAA ADAAAAAAAAAAgP4DAACpAAAAADAAAAAAAAAAgP4DAACpAAAAADAAAAAAAAAAgP4DAACZAAAAADAA AAAAAAAAgP4DAACpAAAAADAAAAAAAAAAgP4DAACpAAAAADAAAAAAAAAAgP4DAACpAAAAADAAAAAA AAAAgP4DAACpAAAAADAAAAAAAAAAgP4DAACZAAAAADAAAAAAAAAAgP4DAACpAAAAADAAAAAAAAAA gP4DAACpAAAAADAAAAAAAAAAgP4DAACpAAAAADAAAAAAAAAAgP4DAACpAAAAADAAAAAAAAAAgP4D AACZAAAAADAAAAAAAAAAgP4DAACpAAAAADAAAAAAAAAAgP4DAACpAAAAADAAAAAAAAAAgP4DAACp AAAAADAAAAAAAAAAgP4DAACpAAAAADAAAAAAAAAAgP4DAACZAAAAADAAAAAAAAAAgP4DAACpAAAA ADAAAAAAAAAAgP4DAACpAAAAADAAAAAAAAAAgP4DAACpAAAAADAAAAAAAAAAgP4DAACpAAAAADAA AAAAAAAAgP4DAACZAAAAADAAAAAAAAAAgP4DAACpAAAAADAAAAAAAAAAgP4DAACpAAAAADAAAAAA AAAAgP4DAACpAAAAADAAAAAAAAAAgP4DAACpAAAAADAAAAAAAAAAgP4DAACZAAAAADAAAAAAAAAA gP4DAACpAAAAADAAAAAAAAAAgP4DAACpAAAAADAAAAAAAAAAgP4DAACpAAAAADAAAAAAAAAAgP4D AACpAAAAADAAAAAAAAAAgP4DAACZAAAAADAAAAAAAAAAgP4DAACpAAAAADAAAAAAAAAAgP4DAACp AAAAADAAAAAAAAAAgP4DAACpAAAAADAAAAAAAAAAgP4DAACpAAAAADAAAAAAAAAAgP4DAACZAAAA ADAAAAAAAAAAgP4DAACpAAAAADAAAAAAAAAAgP4DAACpAAAAADAAAAAAAAAAgP4DAACpAAAAADAA AAAAAAAAgP4DAACpAAAAADAAAAAAAAAAgP4DAACpAAAAADAAAAAAAAAAgP4DAACpAAAAADAAAAAA AAAAgP4DAACZAAAAADAAAAAAAAAAgP4DAACpAAAAADAAAAAAAAAAgP4DAACpAAAAADAAAAAAAAAA gP4DAACpAAAAADAAAAAAAAAAgP4DAACpAAAAADAAAAAAAAAAgP4DAACZAAAAADAAAAAAAAAAgP4D AACpAAAAADAAAAAAAAAAgP4DAACpAAAAADAAAAAAAAAAgP4DAACpAAAAADAAAAAAAAAAgP4DAACp AAAAADAAAAAAAAAAgP4DAACpAAAAADAAAAAAAAAAgP4DAACZAAAAADAAAAAAAAAAgP4DAACpAAAA ADAAAAAAAAAAgP4DAACpAAAAADAAAAAAAAAAgP4DAACpAAAAADAAAAAAAAAAgP4DAACpAAAAADAA AAAAAAAAgP4DAACZAAAAADAAAAAAAAAAgP4DAACpAAAAADAAAAAAAAAAgP4DAACpAAAAADAAAAAA AAAAgP4DAACpAAAAADAAAAAAAAAAgP4DAACpAAAAADAAAAAAAAAAgP4DAACZAAAAADAAAAAAAAAA gP4DAACpAAAAADAAAAAAAAAAgP4DAACpAAAAADAAAAAAAAAAgP4DAACpAAAAADAAAAAAAAAAgP4D AACpAAAAADAAAAAAAAAAgP4DAACpAAAAADAAAAAAAAAAgP4DAACpAAAAADAAAAAAAAAAgP4DAACZ AAAAADAAAAAAAAAAgP4DAACpAAAAADAAAAAAAAAAgP4DAACpAAAAADAAAAAAAAAAgP4DAACpAAAA ADAAAAAAAAAAgP4DAACpAAAAADAAAAAAAAAAgP4DAACZAAAAADAAAAAAAAAAgP4DAACpAAAAADAA AAAAAAAAgP4DAACpAAAAADAAAAAAAAAAgP4DAACpAAAAADAAAAAAAAAAgP4DAACpAAAAADAAAAAA AAAAgP4DAACZAAAAADAAAAAAAAAAgP4DAACpAAAAADAAAAAAAAAAgP4DAACpAAAAADAAAAAAAAAA gP4DAACpAAAAADAAAAAAAAAAgP4DAACpAAAAADAAAAAAAAAAgP4DAACZAAAAADAAAAAAAAAAgP4D AACpAAAAADAAAAAAAAAAgP4DAACpAAAAADAAAAAAAAAAgP4DAACpAAAAADAAAAAAAAAAgP4DAACp AAAAADAAAAAAAAAAgP4DAACZAAAAADAAAAAAAAAAgP4DAACpAAAAADAAAAAAAAAAgP4DAACpAAAA ADAAAAAAAAAAgP4DAACpAAAAADAAAAAAAAAAgP4DAACpAAAAADAAAAAAAAAAgP4DAACZAAAAADAA AAAAAAAAgP4DAACpAAAAADAAAAAAAAAAgP4DAACpAAAAADAAAAAAAAAAgP4DAACpAAAAADAAAAAA AAAAgP4DAACpAAAAADAAAAAAAAAAgP4DAACZAAAAADAAAAAAAAAAgP4DAACpAAAAADAAAAAAAAAA gP4DAACpAAAAADAAAAAAAAAAgP4DAACpAAAAADAAAAAAAAAAgP4DAACpAAAAADAAAAAAAAAAgP4D AACZAAAAADAAAAAAAAAAgP4DAACpAAAAADAAAAAAAAAAgP4DAACpAAAAADAAAAAAAAAAgP4DAACp AAAAADAAAAAAAAAAgP4DAACpAAAAADAAAAAAAAAAgP4DAACZAAAAADAAAAAAAAAAgP4DAACpAAAA ADAAAAAAAAAAgP4DAACpAAAAADAAAAAAAAAAgP4DAACpAAAAADAAAAAAAAAAgP4DAACpAAAAADAA AAAAAAAAgP4DAACZAAAAADAAAAAAAAAAgP4DAACpAAAAADAAAAAAAAAAgP4DAACpAAAAADAAAAAA AAAAgP4DAACpAAAAADAAAAAAAAAAgP4DAACpAAAAADAAAAAAAAAAgP4DAACZAAAAADAAAAAAAAAA gP4DAACpAAAAADAAAAAAAAAAgP4DAACpAAAAADAAAAAAAAAAgP4DAACpAAAAADAAAAAAAAAAgP4D AACZAAAAADAAAAAAAAAAgP4DAACpAAAAADAAAAAAAAAAgP4DAACpAAAAADAAAAAAAAAAgP4DAACp AAAAADAAAAAAAAAAgP4DAACZAAAAADAAAAAAAAAAgP4DAACpAAAAADAAAAAAAAAAgP4DAACpAAAA ADAAAAAAAAAAgP4DAACpAAAAADAAAAAAAAAAgP4DAACZAAAAADAAAAAAAAAAgP4DAACYAAAADzAA AAAAAAAAgP4DAAAoAAAAAzAAAAAAAAAAgAAAAACYAAAAADAAAAAAAAAAgJwPAACYAAAAADAAAAAA AAAAgJwPAACYAAAAADAAAAAAAAAAgJwPAACYAAEgADAAAAAAAAAAgJwPAACYAAEgADABAAAAAAAA gJwPAACYAAAAADAAAAAAAAAAgAAAAICYAAAAADAAAAAAAAAAgAAAAIAoAAAAAzAAAAAAAAAAgAAA AICaAAAADzAAAAAAAAAAgAAAAIAABAAA7xUAAAsAAAAABAAATgUAANoFAAB0BgAAEAkAALoJAABP CgAAtgoAAGMLAADuCwAAfAwAAC8NAADpDQAAjA4AABUPAAClDwAAJxAAALUQAABlEQAAqBEAACAS AACQEgAA/hIAALQTAADvFQAADAAAAA4AAAAPAAAAEAAAABEAAAASAAAAEwAAABQAAAAVAAAAFgAA ABcAAAAYAAAAGQAAABoAAAAbAAAAHAAAAB0AAAAeAAAAHwAAACAAAAAhAAAAIgAAACMAAAAkAAAA AAQAAO8VAAANAAAAAAAAAPMAAAD2AAAA/AQAAP8EAACYBgAAnQYAAOgIAADzCAAA3w0AAO4NAAD5 DQAAAw4AAHcOAAB7DgAAug4AAL0OAADCDgAAzA4AAP4OAAAKDwAAyw8AAM0PAADxEQAABwAcAAcA HAAHABwABwAcAAcAHAAHABwABwAcAAcAHAAHABwABwAcAAcAHAAHAAAAAABXAQAAXQEAAMcCAADO AgAAhgMAAJIDAABjBAAAagQAAB0MAAAmDAAApAwAALIMAAB4DQAAfQ0AAPkNAAADDgAAXg4AAGQO AACSDgAAmg4AAPERAAAHADMABwAzAAcAMwAHADMABwAzAAcAMwAHADMABwAzAAcAMwAHADMABwAA AAAAsw8AAC0QAADuEQAA8REAAAMABAADAAcA//8UAAAABABhAG0AaQB0ADwARwA6AFwAQQBtAGkA dABcAFcAbwByAGsAXABTAGEAbgBQAG8AaQBuAHQARABpAHIAZQBjAHQAXABXAGkAbgBOAFQAXABk AG8AYwBzAFwAUwBQAEQAIABiAGUAdABhACAAUgBDACAAbgBvAHQAZQBzAC4AZABvAGMABABhAG0A aQB0ADwARwA6AFwAQQBtAGkAdABcAFcAbwByAGsAXABTAGEAbgBQAG8AaQBuAHQARABpAHIAZQBj AHQAXABXAGkAbgBOAFQAXABkAG8AYwBzAFwAUwBQAEQAIABiAGUAdABhACAAUgBDACAAbgBvAHQA ZQBzAC4AZABvAGMABABhAG0AaQB0AGIAQwA6AFwAVwBJAE4ATgBUAFwAUAByAG8AZgBpAGwAZQBz AFwAYQBtAGkAdABrAFwAQQBwAHAAbABpAGMAYQB0AGkAbwBuACAARABhAHQAYQBcAE0AaQBjAHIA bwBzAG8AZgB0AFwAVwBvAHIAZABcAEEAdQB0AG8AUgBlAGMAbwB2AGUAcgB5ACAAcwBhAHYAZQAg AG8AZgAgAFMAUABEACAAYgBlAHQAYQAgAFIAQwAgAG4AbwB0AGUAcwAuAGEAcwBkAAQAYQBtAGkA dAA8AEcAOgBcAEEAbQBpAHQAXABXAG8AcgBrAFwAUwBhAG4AUABvAGkAbgB0AEQAaQByAGUAYwB0 AFwAVwBpAG4ATgBUAFwAZABvAGMAcwBcAFMAUABEACAAYgBlAHQAYQAgAFIAQwAgAG4AbwB0AGUA cwAuAGQAbwBjAAQAYQBtAGkAdAA8AEcAOgBcAEEAbQBpAHQAXABXAG8AcgBrAFwAUwBhAG4AUABv AGkAbgB0AEQAaQByAGUAYwB0AFwAVwBpAG4ATgBUAFwAZABvAGMAcwBcAFMAUABEACAAYgBlAHQA YQAgAFIAQwAgAG4AbwB0AGUAcwAuAGQAbwBjAAQAYQBtAGkAdAA8AEcAOgBcAEEAbQBpAHQAXABX AG8AcgBrAFwAUwBhAG4AUABvAGkAbgB0AEQAaQByAGUAYwB0AFwAVwBpAG4ATgBUAFwAZABvAGMA cwBcAFMAUABEACAAYgBlAHQAYQAgAFIAQwAgAG4AbwB0AGUAcwAuAGQAbwBjAAQAYQBtAGkAdAA8 AEcAOgBcAEEAbQBpAHQAXABXAG8AcgBrAFwAUwBhAG4AUABvAGkAbgB0AEQAaQByAGUAYwB0AFwA VwBpAG4ATgBUAFwAZABvAGMAcwBcAFMAUABEACAAYgBlAHQAYQAgAFIAQwAgAG4AbwB0AGUAcwAu AGQAbwBjAAcAZABlAGYAYQB1AGwAdABVAEMAOgBcAFcASQBOAEQATwBXAFMAXABBAHAAcABsAGkA YwBhAHQAaQBvAG4AIABEAGEAdABhAFwATQBpAGMAcgBvAHMAbwBmAHQAXABXAG8AcgBkAFwAQQB1 AHQAbwBSAGUAYwBvAHYAZQByAHkAIABzAGEAdgBlACAAbwBmACAAUwBQAEQAIABiAGUAdABhACAA UgBDACAAbgBvAHQAZQBzAC4AYQBzAGQABwBkAGUAZgBhAHUAbAB0ACUAQwA6AFwAVwBJAE4ARABP AFcAUwBcAFQARQBNAFAAXABTAFAARAAgAGIAZQB0AGEAIABSAEMAIABuAG8AdABlAHMALgBkAG8A YwAHAGQAZQBmAGEAdQBsAHQAJQBDADoAXABXAEkATgBEAE8AVwBTAFwAVABFAE0AUABcAFMAUABE ACAAYgBlAHQAYQAgAFIAQwAgAG4AbwB0AGUAcwAuAGQAbwBjAAEA+gv8HYDNig3/D/8P/w//D/8P /w//D/8P/w8QAAEAAAAAAAEAAAAAAAAAAAAAAAAAAAAAAAMYAAAPhNACEYSY/hXGBQAB0AIGXoTQ AmCEmP5vKAACAAAALgABAAAABIABAAAAAAAAAAAAAAAAAAAAAAAAGAAAD4SgBRGEmP4VxgUAAaAF Bl6EoAVghJj+AgABAC4AAQAAAAKCAQAAAAAAAAAAAAAAAAAAAAAAABgAAA+EcAgRhEz/FcYFAAFw CAZehHAIYIRM/wIAAgAuAAEAAAAAgAEAAAAAAAAAAAAAAAAAAAAAAAAYAAAPhEALEYSY/hXGBQAB QAsGXoRAC2CEmP4CAAMALgABAAAABIABAAAAAAAAAAAAAAAAAAAAAAAAGAAAD4QQDhGEmP4VxgUA ARAOBl6EEA5ghJj+AgAEAC4AAQAAAAKCAQAAAAAAAAAAAAAAAAAAAAAAABgAAA+E4BARhEz/FcYF AAHgEAZehOAQYIRM/wIABQAuAAEAAAAAgAEAAAAAAAAAAAAAAAAAAAAAAAAYAAAPhLATEYSY/hXG BQABsBMGXoSwE2CEmP4CAAYALgABAAAABIABAAAAAAAAAAAAAAAAAAAAAAAAGAAAD4SAFhGEmP4V xgUAAYAWBl6EgBZghJj+AgAHAC4AAQAAAAKCAQAAAAAAAAAAAAAAAAAAAAAAABgAAA+EUBkRhEz/ FcYFAAFQGQZehFAZYIRM/wIACAAuAAEAAAD6C/wdAAAAAAAAAAAAAAAA////////AQAAAAAA//8B AAAAEgAPAAkEGQAJBBsACQQPAAkEGQAJBBsACQQPAAkEGQAJBBsACQQAAAAAAQEAAAcBAAAJAQAA TQEAAE4BAABPAQAAVQEAAFcBAACYAQAAmQEAAJoBAACgAQAAogEAANgBAADZAQAA2gEAAOABAADi AQAALwIAADACAAAxAgAANwIAADkCAAB0AgAAdQIAAHsCAAB9AgAABwMAAAgDAAAKBQAAEAUAABIF AABVBQAAWgUAAFsFAABhBQAAYwUAAKwFAACxBQAAsgUAALgFAAC6BQAA/wUAAAUGAAAGBgAADAYA AA4GAAAnBgAALAYAAC0GAAAzBgAANQYAAE8GAABQBgAAUQYAAFcGAABZBgAAiQYAAI8GAACQBgAA lgYAAJgGAACxBgAAtgYAALcGAAC9BgAAvwYAAAsHAAAQBwAAEQcAABcHAAAZBwAAXQcAAGIHAABj BwAAaQcAAGsHAACRBwAAlgcAAJcHAACdBwAAnwcAAOIHAADnBwAA6AcAAO4HAADwBwAAHwgAACQI AAAlCAAAKwgAAC0IAABuCAAAcwgAAHQIAAB6CAAAfAgAALoIAAC/CAAAwAgAANkIAADdCAAAIwkA ACgJAAApCQAALwkAADEJAACHCQAAjAkAAI0JAACTCQAAlQkAANsJAADgCQAA4QkAAOcJAADpCQAA OAoAAD0KAAA+CgAARAoAAEYKAABeCgAAZAoAAGUKAABrCgAAbQoAAIwKAACRCgAAkgoAAJgKAACa CgAAwQoAAMYKAADHCgAAzQoAAM8KAAAQCwAAFQsAABYLAAAiCwAAJgsAAGwLAABxCwAAcgsAAHgL AAB6CwAApQsAAKoLAACrCwAAsgsAALQLAADpCwAA7gsAAO8LAAD1CwAA9wsAACcMAAAsDAAALQwA ADMMAAA1DAAAcQwAAH8MAACADAAAhgwAAIgMAADODAAA0wwAANQMAADaDAAA3AwAAB4NAAAjDQAA JA0AACoNAAAsDQAAZQ0AAGoNAABrDQAAbA0AAG0NAABuDQAAbw0AAHANAAB2DQAAeA0AAKcNAACo DQAAqQ0AAK8NAACxDQAA7w0AAPANAADxDQAA9w0AAPkNAAAeDgAAHw4AACAOAAAmDgAAKA4AAFQO AABVDgAAVg4AAFwOAABeDgAAiA4AAIkOAACKDgAAkA4AAJIOAACwDgAAsQ4AALIOAAC4DgAAug4A APQOAAD1DgAA9g4AAPwOAAD+DgAAJw8AACgPAAAuDwAAMA8AAFQPAABVDwAAWw8AAF0PAACaDwAA mw8AAPERAAAAAAAACAAAAAIBAAACAQAAAgEAAAIBAACeAQAAAgEAAAIBAAACAQAAAgEAAJ4BAAAC AQAAAgEAAAIBAAACAQAAngEAAAIBAAACAQAAAgEAAAIBAACeAQAAAgEAAAIBAAACAQAAngEAAAIB AAACAQAAAgEAAJYBAAAIAAAAAgEAAAIBAAACAQAAAgEAAJ4BAAACAQAAAgEAAAIBAAACAQAAngEA AAIBAAACAQAAAgEAAAIBAACeAQAAAgEAAAIBAAACAQAAAgEAAJ4BAAACAQAAAgEAAAIBAAACAQAA ngEAAAIBAAACAQAAAgEAAAIBAACeAQAAAgEAAAIBAAACAQAAAgEAAJ4BAAACAQAAAgEAAAIBAAAC AQAAngEAAAIBAAACAQAAAgEAAAIBAACeAQAAAgEAAAIBAAACAQAAAgEAAJ4BAAACAQAAAgEAAAIB AAACAQAAngEAAAIBAAACAQAAAgEAAAIBAACeAQAAAgEAAAIBAAACAQAAAgEAAJ4BAAACAQAAAgEA AAIBAAACAQAAngEAAAIBAAACAQAAAgEAAAIBAACeAQAAAgEAAAIBAAACAQAAAgEAAJ4BAAACAQAA AgEAAAIBAAACAQAAngEAAAIBAAACAQAAAgEAAAIBAACeAQAAAgEAAAIBAAACAQAAAgEAAJ4BAAAC AQAAAgEAAAIBAAACAQAAngEAAAIBAAACAQAAAgEAAAIBAACeAQAAAgEAAAIBAAACAQAAAgEAAJ4B AAACAQAAAgEAAAIBAAACAQAAngEAAAIBAAACAQAAAgEAAAIBAACeAQAAAgEAAAIBAAACAQAAAgEA AJ4BAAACAQAAAgEAAAIBAAACAQAAngEAAAIBAAACAQAAAgEAAAIBAACeAQAAAgEAAAIBAAACAQAA AgEAAJ4BAAACAQAAAgEAAAIBAAACAQAAngEAAAIBAAACAQAAAgEAAAIBAACeAQAAAgEAAAIBAAAC AQAAAgEAAJ4BAAACAQAAAgEAAAIBAAACAQAAngEAAAIBAAACAQAAAgEAAAIBAACeAQAAAgEAAAIB AAACAQAAAgEAAJ4BAAACAQAAAgEAAAIBAAACAQAAngEAAAIBAAACAQAAAgEAAAIBAACeAQAAAgEA AAIBAAACAQAAAgEAAJ4BAAACAQAAAgEAAAIBAAACAQAAngEAAAIBAAACAQAAAgEAAJ4BAAACAQAA AgEAAAIBAACeAQAAAgEAAAIBAAACAQAAlgEAAP9AAhAAAAAAAAAA7xEAAFAAAAgAQAAA//8BAAAA BwBVAG4AawBuAG8AdwBuAP//AQAIAAAAAAAAAAAAAAD//wEAAAAAAP//AAACAP//AAAAAP//AAAC AP//AAAAAAQAAABHFpABAAACAgYDBQQFAgMEhzoAAAAAAAAAAAAAAAAAAP8AAAAAAAAAVABpAG0A ZQBzACAATgBlAHcAIABSAG8AbQBhAG4AAAA1FpABAgAFBQECAQcGAgUHAAAAAAAAABAAAAAAAAAA AAAAAIAAAAAAUwB5AG0AYgBvAGwAAAAzJpABAAACCwYEAgICAgIEhzoAAAAAAAAAAAAAAAAAAP8A AAAAAAAAQQByAGkAYQBsAAAAPzWQAQAAAgcDCQICBQIEBIc6AAAAAAAAAAAAAAAAAAD/AAAAAAAA AEMAbwB1AHIAaQBlAHIAIABOAGUAdwAAACIABABxCIgYAPDQAgAAaAEAAAAAE5xUhnmiVKYAAAAA DgB3AgAAmAIAAMkOAAABAAcAAAAEAAMQHwAAAAAAAAAAAAAAAQABAAAAAQAAAAAAAAAhAwDwEAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAIB6AFtAC0AIGBMjAAAAAAAAAAAAAAAAAAACgSAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAgAAAAAAAAAAAAAyg1EA8BAACAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAD/ /xIAAAAAAAAABQBTAFAARAAgADEAAAAAAAAABABhAG0AaQB0AAcAZABlAGYAYQB1AGwAdAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAP7/AAAE WgIAAAAAAAAAAAAAAAAAAAAAAAEAAADghZ/y+U9oEKuRCAArJ7PZMAAAAGgBAAARAAAAAQAAAJAA AAACAAAAmAAAAAMAAACoAAAABAAAALQAAAAFAAAAxAAAAAYAAADQAAAABwAAANwAAAAIAAAA7AAA AAkAAAD8AAAAEgAAAAgBAAAKAAAAJAEAAAwAAAAwAQAADQAAADwBAAAOAAAASAEAAA8AAABQAQAA EAAAAFgBAAATAAAAYAEAAAIAAADkBAAAHgAAAAYAAABTUEQgMQBmAB4AAAABAAAAAFBEIB4AAAAF AAAAYW1pdAAAZgAeAAAAAQAAAABtaXQeAAAAAQAAAABtaXQeAAAABwAAAE5vcm1hbAAAHgAAAAgA AABkZWZhdWx0AB4AAAADAAAAMTQAYR4AAAATAAAATWljcm9zb2Z0IFdvcmQgOS4wAABAAAAAAIpR JlgAAABAAAAAAEpeHSfJwAFAAAAAAJZl6rrJwAEDAAAAAQAAAAMAAACYAgAAAwAAAMkOAAADAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAD+/wAABFoCAAAAAAAA AAAAAAAAAAAAAAABAAAAAtXN1ZwuGxCTlwgAKyz5rjAAAADwAAAADAAAAAEAAABoAAAADwAAAHAA AAAFAAAAgAAAAAYAAACIAAAAEQAAAJAAAAAXAAAAmAAAAAsAAACgAAAAEAAAAKgAAAATAAAAsAAA ABYAAAC4AAAADQAAAMAAAAAMAAAA0gAAAAIAAADkBAAAHgAAAAUAAAB2c2lsAFBEIAMAAAAfAAAA AwAAAAcAAAADAAAAKBIAAAMAAADtDgkACwAAAAAAAAALAAAAAAAAAAsAAAAAAAAACwAAAAAAAAAe EAAAAQAAAAYAAABTUEQgMQAMEAAAAgAAAB4AAAAGAAAAVGl0bGUAAwAAAAEAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAQAAAAIAAAADAAAABAAAAAUAAAAG AAAABwAAAAgAAAAJAAAACgAAAAsAAAAMAAAADQAAAA4AAAAPAAAAEAAAABEAAAASAAAAEwAAABQA AAAVAAAAFgAAABcAAAAYAAAAGQAAABoAAAAbAAAAHAAAAB0AAAAeAAAAHwAAACAAAAAhAAAAIgAA ACMAAAAkAAAAJQAAAP7///8nAAAAKAAAACkAAAAqAAAAKwAAACwAAAAtAAAALgAAAC8AAAAwAAAA MQAAADIAAAAzAAAANAAAADUAAAA2AAAANwAAADgAAAA5AAAAOgAAADsAAAA8AAAA/v///z4AAAA/ AAAAQAAAAEEAAABCAAAAQwAAAEQAAAD+////RgAAAEcAAABIAAAASQAAAEoAAABLAAAATAAAAP7/ ///9////TwAAAP7////+/////v////////////////////////////////////////////////// //////////////////////////////////////////////////////////////////////////// //////////////////////////////////////////////////////////////////////////// //////////////////////////////////////////////9SAG8AbwB0ACAARQBuAHQAcgB5AAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAFgAFAf//////////AwAA AAYJAgAAAAAAwAAAAAAAAEYAAAAAAAAAAAAAAADA89zrusnAAVEAAACAAAAAAAAAADEAVABhAGIA bABlAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAO AAIA////////////////AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAJgAAAGot AAAAAAAAVwBvAHIAZABEAG8AYwB1AG0AZQBuAHQAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAABoAAgEFAAAA//////////8AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAIkoAAAAAAAAFAFMAdQBtAG0AYQByAHkASQBuAGYAbwByAG0AYQB0AGkAbwBu AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAKAACAQIAAAAEAAAA/////wAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAD0AAAAAEAAAAAAAAAUARABvAGMAdQBtAGUAbgB0AFMAdQBt AG0AYQByAHkASQBuAGYAbwByAG0AYQB0AGkAbwBuAAAAAAAAAAAAAAA4AAIB//////////////// AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAARQAAAAAQAAAAAAAAAQBDAG8AbQBw AE8AYgBqAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAABIA AgEBAAAABgAAAP////8AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAagAA AAAAAABPAGIAagBlAGMAdABQAG8AbwBsAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAFgABAP///////////////wAAAAAAAAAAAAAAAAAAAAAAAAAAwPPc67rJwAHA 89zrusnAAQAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA////////////////AAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAQAAAP7///////////////////////// //////////////////////////////////////////////////////////////////////////// //////////////////////////////////////////////////////////////////////////// //////////////////////////////////////////////////////////////////////////// //////////////////////////////////////////////////////////////////////////// //////////////////////////////////////////////////////////////////////////// //////////////////////////////////////////////////////////////////////////// //////////////////////////////////////////////////////////////////////////// //////////////////////////////////////////////////////////////////////////// //////////////////////////////////////////8BAP7/AwoAAP////8GCQIAAAAAAMAAAAAA AABGGAAAAE1pY3Jvc29mdCBXb3JkIERvY3VtZW50AAoAAABNU1dvcmREb2MAEAAAAFdvcmQuRG9j dW1lbnQuOAD0ObJxAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA== ------=_NextPart_000_00D8_0147DA5A.BA8C5AF0-- From owner-ips@ece.cmu.edu Thu May 3 13:42:37 2001 Received: from ece.cmu.edu (ECE.CMU.EDU [128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id NAA01515 for ; Thu, 3 May 2001 13:42:36 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f43Eqij29376 for ips-outgoing; Thu, 3 May 2001 10:52:44 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from dogwood.cisco.com (dogwood.cisco.com [161.44.11.19]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f43EqhH29371 for ; Thu, 3 May 2001 10:52:43 -0400 (EDT) Received: from cisco.com (dhcp-161-44-68-173.cisco.com [161.44.68.173]) by dogwood.cisco.com (8.8.6 (PHNE_14041)/CISCO.SERVER.1.2) with ESMTP id KAA15359 for ; Thu, 3 May 2001 10:52:37 -0400 (EDT) Message-ID: <3AF17047.89EB3B62@cisco.com> Date: Thu, 03 May 2001 09:50:47 -0500 From: Steve Senum X-Mailer: Mozilla 4.73 [en]C-CCK-MCD (Windows NT 5.0; U) X-Accept-Language: en MIME-Version: 1.0 To: ips@ece.cmu.edu Subject: Re: iscsi: comments to iSCSI rev 6 References: <3AF0232E.852EDB5E@cisco.com> <3AF08DB1.D1A48FF7@agilent.com> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-ips@ece.cmu.edu Precedence: bulk Content-Transfer-Encoding: 7bit Matt, Okay, but I still don't see the need. If one side initiates sending a key to the other side, I might expect the following responses under the current draft 6: 1. key returned with an option other other than 'none'. 2a. key not returned. 2b. key returned with option 'none'. And you would add: 2c. key returned with option 'NotUnderstood'. To me, responses 2a, 2b, and 2c all mean the same, in as far as the side to initiate sending the key must make do without that key. From a negotiation point, I don't see the value add in response 2c, or for that matter, response 2b. Matt Wakeley wrote: > > Steve Senum wrote: > > > > I would prefer the text stay as is, > > that the target not send any response > > for an unknown key. I see no value > > at all in sending key=NotUnderstood. > > Lack of a response tells me that already. > > No it doesn't. You can't tell the difference between "not understood" and > "none", since "none" is allowed not to respond also. > > I don't like that either. If the response is "none", say so, don't just sit > there silently. > > -Matt From owner-ips@ece.cmu.edu Thu May 3 16:09:41 2001 Received: from ece.cmu.edu (ECE.CMU.EDU [128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id QAA07385 for ; Thu, 3 May 2001 16:09:41 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f43HA4611253 for ips-outgoing; Thu, 3 May 2001 13:10:04 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from dogwood.cisco.com (dogwood.cisco.com [161.44.11.19]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f43HA3H11248 for ; Thu, 3 May 2001 13:10:03 -0400 (EDT) Received: from cisco.com (mbakke@mbakke-lnx.cisco.com [161.44.68.87]) by dogwood.cisco.com (8.8.6 (PHNE_14041)/CISCO.SERVER.1.2) with ESMTP id NAA21376 for ; Thu, 3 May 2001 13:09:57 -0400 (EDT) Message-ID: <3AF19067.BF23B8B1@cisco.com> Date: Thu, 03 May 2001 12:07:51 -0500 From: Mark Bakke X-Mailer: Mozilla 4.72 [en] (X11; U; Linux 2.2.16-3.uid32 i686) X-Accept-Language: en, de MIME-Version: 1.0 To: IPS Subject: iSCSI MIB: Latest draft on MibCentral Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-ips@ece.cmu.edu Precedence: bulk Content-Transfer-Encoding: 7bit Dear SNMP fans, For those who wish to browse the latest iSCSI MIB, it is now available at www.mibcentral.com. Click on IETF WGs, then ISCSI-MIB. If the comment still says draft-bakke..., you will have to reload the page. Enjoy, -- Mark A. Bakke Cisco Systems mbakke@cisco.com 763.398.1054 From owner-ips@ece.cmu.edu Thu May 3 18:52:54 2001 Received: from ece.cmu.edu (ECE.CMU.EDU [128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id SAA12991 for ; Thu, 3 May 2001 18:52:53 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f43KfvV29370 for ips-outgoing; Thu, 3 May 2001 16:41:57 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from server1.NishanSystems.COM (smtp.nishansystems.com [216.217.36.162]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f43KfrH29362 for ; Thu, 3 May 2001 16:41:53 -0400 (EDT) Received: by smtp.nishansystems.com with Internet Mail Service (5.5.2653.19) id ; Thu, 3 May 2001 13:41:41 -0700 Message-ID: From: Joshua Tseng To: "'Dave Peterson'" , "Ips@Ece. Cmu. Edu" Subject: RE: Proposal to use SLPv2 for FCIP device discovery Date: Thu, 3 May 2001 13:41:36 -0700 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2653.19) Content-Type: text/plain; charset="iso-8859-1" Sender: owner-ips@ece.cmu.edu Precedence: bulk Dave, See below: > > As requested here is the proposal I presented at Mondays' > FCIP meeting: > > FCIP Draft Proposal > For Clause 4.2, item 3. > > Each FCIP device may be statically or dynamically configured > with a list of > IP addresses and port numbers corresponding to all the > participating FCIP > devices. If dynamic discovery of participating FCIP devices > is supported > this function shall be performed using the Service Location Protocol > (SLPv2). For additional FCIP device management capability, > the iSNS Internet > Storage Naming Service may be implemented. It is outside the > scope of this > specification to describe any static configuration method for > participating > FCIP device discovery. Refer to clause XXX for a detailed > description of > dynamic discovery of participating FCIP device using SLPv2. I disagree that use of either SLP or iSNS should be mandatory. In fact, if iSNS is used, then SLP SHOULD NOT be used for FCIP device discovery (although it MAY be used to discover the iSNS server). The reason for this is that the iSNS assumes it has control of the discovery and management process. Using SLP simultaneously undermines the authority of the iSNS to control discovery domains. > > Notes: > 1. DHCP: > a. Allows an entity to discovery information about > itself, not discover > information about all other entities. > b. Uses a broadcast mechanism that may not work via > routers without > additional configuration. But, most current router > implementations will > support the forwarding of DHCP requests across routed subnets. > c. May be used to find SLP Directory Agents and Scope > Lists allowing for a > more scalable solution. In addition, DHCP may be used to find the location of the iSNS server. > 2. DNS Services are too limiting. This is the reason why SLP > was started. > 3. LDAP is simply a database interface. SLP and iSNS may use LDAP as a > back-end store. > 4. SLP FCIP Device Type Specifics > a. IP address(es) > b. Port numbers(s) > c. Scope > d. Attributes > i. Discovery domain (i.e. a group name or zone > name, don't want to use > zone name in this context though) > ii. need to start listing these (if we can > think of any more) > e. Lifetime > > Work In progress: > 1. Putting together a model for FCIP device discovery using SLPv2. > 2. Implementing a "default/suggested" SLPv2 template. > 3. Reviewing RFC 3082 - Notification and Subscription for > SLP for enhanced > device notification. > > Requirements > SLPv2 iSNS > Discovery of FCIP targets > Y Y > Discovery of FCIP device IP address(es) > Y Y > Discovery of FCIP port number(s) > Y Y > Attribute support > Y Y > Semi-timely notification of devices coming and going Y Y > Authentication > Y Y(uses SLPv2 constructs) > Minimal/no configuration > Y Y(?) > Provisioning > Y Y > Multicast support > Y N > Publicly available source > Y N iSNS will be available by the end of May. > Standardized and mature > Y N I haven't seen SLP templates for FCIP. Did I miss them? If so, could you send them to me? Thanks. Has an implementation of SLP been tested with FCIP yet? > Lighweight implementation > Y N As will be shown, iSNS clients are about as lightweight as you can get. > > Non-Requirements > Zoning > N Y > State Change Notification > N Y > Naming Services > N Y What criteria did you use to label these as non-requirements? Thanks, Josh > > > David Peterson > Lead Architect - Standards Development > Cisco Systems - SRBU > 6450 Wedgwood Road > Maple Grove, MN 55311 > Office: 763-398-1007 > Cell: 612-802-3299 > Email: dap@cisco.com > From owner-ips@ece.cmu.edu Thu May 3 18:56:42 2001 Received: from ece.cmu.edu (ECE.CMU.EDU [128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id SAA13171 for ; Thu, 3 May 2001 18:56:41 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f43KRp928233 for ips-outgoing; Thu, 3 May 2001 16:27:51 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from server1.NishanSystems.COM (smtp.nishansystems.com [216.217.36.162]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f43KRnH28228 for ; Thu, 3 May 2001 16:27:49 -0400 (EDT) Received: by smtp.nishansystems.com with Internet Mail Service (5.5.2653.19) id ; Thu, 3 May 2001 13:27:40 -0700 Message-ID: From: Joshua Tseng To: "'ips@ece.cmu.edu'" Subject: FW: iSNS/SLP Comparison Date: Thu, 3 May 2001 13:27:34 -0700 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2653.19) Content-Type: text/plain; charset="iso-8859-1" Sender: owner-ips@ece.cmu.edu Precedence: bulk I've been asked to re-post this message. Once again, if anyone has comments on SLP and how it can/cannot be used in the SNS role, please respond to the this message on the reflector. Same for any other protocol (LDAP, SNMP, RADIUS, COPS, etc...). Thanks, Josh > -----Original Message----- > From: Joshua Tseng > Sent: Thursday, March 22, 2001 10:01 AM > To: ips@ece.cmu.edu > Subject: iSNS/SLP Comparison > > Folks, > > As promised, here is the text of the detailed analysis completed several > weeks ago on > the feasibility of using the SLP protocol to support the storage name > service functionality > for iSCSI. This issue was hashed and rehashed multiple times within the > Naming & > Discovery Team. Mark Bakke and I had numerous online and offline > discussions on > SLP and iSNS, and in the end we both concluded that SLP would not provide > the needed > functionality for a storage name service. Rather, we agreed SLP can be > used to do > simple discovery of iSCSI targets in a device-centric management model, > without the > aid of the iSNS. On the other hand, iSNS would provide the centralized > management > function that would allow iSCSI to scale to enterprise-class networks. > > The bottom line is that iSNS is a storage-aware application that is aware > of client attributes, > and can take appropriate action when events occur in the network. SLP is > a generic > discovery protocol and has no intelligence of storage devices and their > attributes. LDAP > alone is inappropriate for a similar reason in that it is a generic > protocol, not an application. > However, both LDAP and SLP can be used to support the iSNS application. > LDAP to > store and distribute attribute values in a directory-enabled iSNS > implementation, and > SLP to discover the iSNS service itself. > > Josh > > --------------------------------------------- > > Comparisons between iSNS and SLPv2: > > 1) State Change Notification: > > SLPv2 does not currently support it. This is an important function that > exists in enterprise-scale Fibre Channel SANs. For SLPv2 to be feasible > for large enterprise-scale storage networks, it would have to incorporate > an asynchronous messaging mechanism that is closely tied to entities > identified by service attributes. Specifically, this means SLPv2 must > develop the intelligence to recognize a significant event, identify the > entities (i.e., WWUI's) that are affected by that event by examining the > DD/Zoning of each entity, and send a directed unicast message to the > entities represented by those WWUI's. > > Such functionality seems to be beyond the intent and scope of SLP. WWUI > would be stored as an attribute, and in SLP, attributes are simple values > which carry no significance to SLP. It is beyond the purpose of SLP to > interpret the contents of any attribute stored for a service. However, in > order to support SCN's, SLP would need interpret the WWUI attribute values > and map it to a destination IP address (i.e., extract the URL attribute > and query for an IP address) for the State Change Notification. > > Therefore, we cannot count on SLP's ability to fulfill the State Change > Notification requirement, even with future upgrades to the SLP protocol. > > 2) Multicast > > RFC 2608 contains "MUST" language that requires SA's to listen to > multicast requests from UA's and DA's. However, in order to fulfill the > SNS role, SA's do not need this capability, and multicast should not be > used in many cases. An implementer will be faced with the choice of > either compliance with the RFC 2608 and implementing a capability that is > not needed, or being non-compliant and implementing only what is needed > for the SNS. There are other areas in SLP which describe capabilities > that are not needed for SNS. > > Multicast is not required for SNS because it may not be available in some > networks, particularly the Public Internet or in a business partner's > network. I shall assume no multicast functionality, since SNS must be > able to work over the Public Internet per IETF Charter. > > 3) Zoning/Discovery Domains: > > The scoping functionality cannot be used for "Zoning/Discovery Domain" > capability, because common scoping is a requirement for even DA's and SA's > to communicate with each other. SLP assumes SA's, UA's, and DA's are > already configured with the appropriate scoping, and SLP itself cannot be > used to create, delete, or modify a scoping of an SA, UA, or DA, because > they can't even communicate unless they already have common scoping. > Therefore, a new attribute such as "Zone" in Mark's template is needed. > But is the Zone an attribute of the WWUI or iSCSI instance? > > At the current time, it's unclear if the service represents a single WWUI, > or an "iSCSI Instance" which may have multiple WWUI's. Either way, there > is a disadvantage: > > a. If the service registers an iSCSI Instance, then the Zone/Discovery > Domain is an attribute of the service (i.e., iSCSI instance), and not of > the WWUI (i.e., iSCSI device), ALL WWUI's belonging to the iSCSI instance > must be assigned to the same Zone/Discovery Domain. That means, if an > iSCSI target device has 50 WWUI's, then all 50 must be assigned to the > same Discovery Domain, and initiators must perform an iSCSI login to each > of the 50 devices. While a native iSCSI device with 50 controllers may be > rare, the real impact is on iSCSI-FC gateways. The gateway must proxy for > a network of Fibre Channel devices, and the gateway will likely represent > each FC device with a WWUI (using the "eui.xxx" WWUI format). But that > means the user has no means of limiting access and discovery to a subset > of devices in the FC network. > > b. If the "service" registers a single WWUI, then each "iSCSI instance" > will have to register multiple times if it supports more than one WWUI. > Once again, an iSCSI-FC gateway will have a large number of individual > registration messages to send for any change to its attributes, and > updates to receive for any change in status to other members in a common > Zoning/Discovery Domain. For example, if the gateway supports 50 FC > devices that are in a common Zone with initiator I, and initiator I is > somehow removed from the Zone then the iSCSI-FC gateway will need to query > the DA 50 times in order to make this simple update. > > And the last significant issue I'm aware of with Zones/Discovery Domains > is that since the Zone is already an attribute, additional attributes > cannot be associated with the Zone. This means a VLAN tag, multicast > group address, or user-assigned symbolic name cannot be stored as an > attribute of the Zone, in the DA. > > 4) Heartbeat/Client Status Inquiry: > > SLP accomplishes this function through a multicast DAAdvert. However, SLP > does not specify an alternative if multicast is not available. Rather, > SLP relies on the SA's and the UA's to continue refreshing their > registrations in the DA at regular intervals. The refresh interval is > dependent on the SA and UA settings, and is an interval between 0 and 18 > hours. Since the DA has no control over the interval at which SA's send > in registrations, it could get overwhelmed if there are a large number of > SA's and UA's query for and register data. And the only way that the DA > can discover (in the absence of multicast) that an SA has been removed > from the network, is to wait until the timeout interval expires for SA > registration refresh. Set the expiration timer too low, and the SA's will > be re-registering with the DA too often and overwhelm it. Set the timer > too high, and it could be hours or days before the DA discovers that an SA > (i.e., iSCSI target) has disappeared. That would be unacceptable for an > enterprise-class user. > > This is the original reason why SLP relied on the multicast advertisement. > There is no unicast advertisement originated by the DA that I am aware of > that is not in response to a multicast request by an SA. > > On the other hand, iSNS unicasts heartbeat messages to each iSNS client, > and can adjust the interval at which it is comfortable sending out > heartbeats, depending on its processing load. > > 5) Non-string-based attributes: > > SLP is optimized to store and transmit string-based attributes. However, > the SNS must be able to store non-string attributes, such as an X.509 > public key certificate. The only way that SLP can do this is if it > escapes each byte of binary data. This "escaped" SLP formatting will > effectively double the size of the original X.509 certificate. This is an > additional burden for iSCSI devices, that will need to take the X.509 > "blob" and place an "/" between every byte of binary data. Processing > requirements for the DA host (which are already significantly more than > iSNS server from the above issues 1-4), will also be increased as the a > companion application on the DA host may need to extract the original > certificate so that it can verify the signature and authenticate its > origin. > > Also, doubling the size of the X.509 certificate in the SLP protocol > message may overflow the size of the multicast datagram, which is a no-no. > > 6) Security: > > Because the DA, SA, and UA are all using the same scope ("SNS"), and Zone > is now an attribute of the iSCSI device (either the WWUI or the iSCSI > Instance), the DA (i.e., the SNS) must trust each of the SA's as far as > access authorization. Although initiators are not supposed to change the > attributes of a target, nothing prevents a wayward initiator from doing it > anyway. Once again, attribute values have no significance to SLP, and > there is no way for SLP to disallow a query or registration command by > examining the value stored in an attribute field for the service that is > the object of the query or request. > > This will become an issue when sharing the DA database with a business > partner. Even if a new special "Zone" or "Discovery Domain" is created, > and only devices that an enterprise intends to share with the business > partner is placed into that "Discovery Domain", there is nothing which > prevents the business partner from doing an SLP query with no "Discovery > Domain" attribute. This will result in ALL devices registered in the DA > being returned. The business partner now has access to all devices stored > in the DA, and can do unlimited damage if they desire. > > CONCLUSION: > > I believe there may be other issues which have yet to be discovered, > regarding use of SLP in the SNS role for a large enterprise-class storage > network. My suggestion is that we limit the use of SLP in iSCSI to > discovery only. We can recomend and use SLP for simple discovery of iSCSI > devices, as well as discovery of the iSNS. But let's not encourage SLP to > be used in a role for which it was not designed. > > My concern is that if we talk or hint about the possibility of using SLP > to fulfill the SNS role, those that take this path will invest > considerable engineering effort only to find in the end that there are > insurmountable issues in supporting large, enterprise-class storage area > networks. SLP simply was not designed for the SNS role. Attempting to > force SLP to do what it wasn't designed for will result in consequences > down the road if anyone takes our errant advice to do so. > > Josh > From owner-ips@ece.cmu.edu Thu May 3 21:50:33 2001 Received: from ece.cmu.edu (ECE.CMU.EDU [128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id VAA17797 for ; Thu, 3 May 2001 21:50:33 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f440Zee17878 for ips-outgoing; Thu, 3 May 2001 20:35:40 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from palrel1.hp.com (palrel1.hp.com [156.153.255.242]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f440ZcH17872 for ; Thu, 3 May 2001 20:35:38 -0400 (EDT) Received: from hpcuhe.cup.hp.com (hpcuhe.cup.hp.com [15.0.80.203]) by palrel1.hp.com (Postfix) with ESMTP id 4EF2A7CB; Thu, 3 May 2001 17:35:37 -0700 (PDT) Received: from cup.hp.com (santoshr@hpindhhm.cup.hp.com [15.8.80.197]) by hpcuhe.cup.hp.com (8.9.3 (PHNE_18979)/8.9.3 SMKit7.02) with ESMTP id RAA24071; Thu, 3 May 2001 17:35:32 -0700 (PDT) Message-ID: <3AF1F937.6A027E10@cup.hp.com> Date: Thu, 03 May 2001 17:35:03 -0700 From: Santosh Rao Organization: Hewlett Packard, Cupertino. X-Mailer: Mozilla 4.7 [en] (X11; U; HP-UX B.11.00 9000/778) X-Accept-Language: en MIME-Version: 1.0 To: julian_satran@il.ibm.com Cc: ips@ece.cmu.edu Subject: Re: iSCSI : digest error handling violates EMDP/InDataOrder References: Content-Type: multipart/mixed; boundary="------------B1E96A1E6759899ABA60A577" Sender: owner-ips@ece.cmu.edu Precedence: bulk This is a multi-part message in MIME format. --------------B1E96A1E6759899ABA60A577 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit In previous exchanges in this thread, it was stated that EMDP only governs data order within a sequence. This, to me [and others from FC world, as was noted by Bob] reads as the definition of Random relative Offset [as defined by bob]. However, Section 2.7.5 on Buffer Offset states that : "Output data within a burst MUST be delivered in increasing buffer offset order". This contradicts the above by indicating that iSCSI forbids Random Relative Offset. So, does EMDP apply to WRITEs or does it not (?). Also, does iSCSI allow data overlay or is this prohibited ? I'd suggest the following be clearly described in the draft : a) iSCSI defines EMDP to govern the order of data PDUs within a SCSI command for both READ and WRITE operations. An EMDP setting of 0 implies data overlay is forbidden. b) The out-of-order transmission of a given sequence of WRITE data PDUs in response to an R2T (the random relative offset property) be explicitly dis-allowed by iSCSI [which is currently stated in Section 2.7.5]. Regards, Santosh julian_satran@il.ibm.com wrote: > > Santosh, > > Let's take a systematic approach to it. > > Restriction on data ordering are required if the source or the destination > of the data is unable to deliver or take data data in any order other that > sequential. > > Semiconductor or other direct access memories don't have this restriction. > > Tapes and other sequential media do have this type of restriction and so > some streaming devices. > > If the restricted device is a target of a SCSI operation with an > unrestricted initiator then: > > a. on reads the target can always ship its data in sequential order > b. on writes the target can always request the data in sequential order > > However if the restricted device is an initiator then: > > a. on reads the initiator will request the target to send the data in order > b. on writes the restricted initiator will have to get the R2Ts in order > from the target and will be able to support data recovery through an R2T > only if it has enough buffered data. > > A restricted device will act as an initiator only if it becomes a third > part copy manager (CM) in a third party operation an does copy from one of > its devices to another device. > > Introducing a new mode bit (as Robert Snively seems to suggest) will not > change the fact that the restriction can't be upholded > and do recovery unless the restricted initiator has enough memory. > > The spec should only specify a way to terminate a command in those > conditions and leave it at that. > > I will change the wording of the DataOrder to make it clearer but I > consider the whole issue entirely academic and overblown. > > Recall also that a CM implemented which such severe buffering restrictions > violates the basic SCSI assumption that a target is the data master. > > Regards, > Julo > > Santosh Rao on 28/04/2001 00:03:26 > > Please respond to Santosh Rao > > To: Julian Satran/Haifa/IBM@IBMIL > cc: Black_David@emc.com, ips@ece.cmu.edu > Subject: Re: iSCSI : digest error handling violates EMDP/InDataOrder > > julian_satran@il.ibm.com wrote: > > > > David, > > > > I read Bob's mail and my interpretation is similar to his. However I > think > > that SPC explicitly states that different transports are free to > interpret > > and make use of this page as they find appropriate. > > > > I have a hard time understanding Santosh's objection as it does not refer > > to the reason the EMDP is there but to the way it is written in FCP (not > > iSCSI). > > Julian, > > As has been stated earlier, EMDP allows control over the order in which > the target requests outbound data or sends inbound data. EMDP can be > used by initiators to control this order and turn off out-of-order R2T > requests [as well as turn off out of order read data pdus]. > > This is a useful control option and is already provided by other SCSI > transports. What good reason exists to deny this provision in iSCSI ? > > Also, I have some concerns about the ambiguous definition of DataOrder. > > Per the spec : > "DataOrder= > > The default is yes but targets MAY support no. No is used by iSCSI to > indicate that the data PDUs can be in any order (EMDP = 1). Yes is used > to indicate that incoming data PDUs have to be at continuously > increasing addresses (EMDP = 0)." > > Based on the above definition wording : > > a) How is DataOrder interpreted for WRITE I/Os ? > b) Is the ordering across the entire SCSI command or a subset of the I/O > ? If so, what constitutes this subset ? > > Different implementors can arrive at different interpretations reading > the above definition ! > > - Santosh > - santoshr.vcf --------------B1E96A1E6759899ABA60A577 Content-Type: text/x-vcard; charset=us-ascii; name="santoshr.vcf" Content-Description: Card for Santosh Rao Content-Disposition: attachment; filename="santoshr.vcf" Content-Transfer-Encoding: 7bit begin:vcard n:Rao;Santosh tel;work:408-447-3751 x-mozilla-html:FALSE org:Hewlett Packard, Cupertino.;SISL adr:;;19420, Homestead Road, M\S 43LN, ;Cupertino.;CA.;95014.;USA. version:2.1 email;internet:santoshr@cup.hp.com title:Software Design Engineer x-mozilla-cpt:;21088 fn:Santosh Rao end:vcard --------------B1E96A1E6759899ABA60A577-- From owner-ips@ece.cmu.edu Thu May 3 21:51:22 2001 Received: from ece.cmu.edu (ECE.CMU.EDU [128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id VAA17813 for ; Thu, 3 May 2001 21:51:22 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f440dkc18104 for ips-outgoing; Thu, 3 May 2001 20:39:46 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from palrel2.hp.com (palrel2.hp.com [156.153.255.234]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f440diH18100 for ; Thu, 3 May 2001 20:39:44 -0400 (EDT) Received: from hpcuhe.cup.hp.com (hpcuhe.cup.hp.com [15.0.80.203]) by palrel2.hp.com (Postfix) with ESMTP id E6696382; Thu, 3 May 2001 17:39:43 -0700 (PDT) Received: from cup.hp.com (santoshr@hpindhhm.cup.hp.com [15.8.80.197]) by hpcuhe.cup.hp.com (8.9.3 (PHNE_18979)/8.9.3 SMKit7.02) with ESMTP id RAA24749; Thu, 3 May 2001 17:39:39 -0700 (PDT) Message-ID: <3AF1FA33.5D3FB3C0@cup.hp.com> Date: Thu, 03 May 2001 17:39:15 -0700 From: Santosh Rao Organization: Hewlett Packard, Cupertino. X-Mailer: Mozilla 4.7 [en] (X11; U; HP-UX B.11.00 9000/778) X-Accept-Language: en MIME-Version: 1.0 To: julian_satran@il.ibm.com, IPS Reflector Subject: Re: iSCSI : Bridging missing CmdSNs and Abort I/O Error recovery References: Content-Type: multipart/mixed; boundary="------------82056CF465111AFF57DC636E" Sender: owner-ips@ece.cmu.edu Precedence: bulk This is a multi-part message in MIME format. --------------82056CF465111AFF57DC636E Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Julian, This is a problem that spans transports. FCP cannot deploy the ABTS-LS abort protocol on a ULP timeout if the ABTS cannot be sent because the target port is not responding with BB_Credit. I don't believe that's reason enough to not extend "connection allegiance" to include Abort Task. Sending an Abort Task on the same connection as the command simplifies a number of issues related to the task cleanup. Regards, Santosh julian_satran@il.ibm.com wrote: > > Forcing abort task on the same connection severely limits the ability of > sending an abort when the TCP windows is or gets closed. And that is true > for one or several adapters. > > Julo > > Santosh Rao on 28/04/2001 23:51:00 > > Please respond to Santosh Rao > > To: Julian Satran/Haifa/IBM@IBMIL > cc: ips@ece.cmu.edu > Subject: Re: iSCSI : Bridging missing CmdSNs and Abort I/O Error recovery > > Julian, > > Thanks for the clarification. One basic question. Why don't we want to > force Abort Task on the same connection as the command ? Extending > connection allegiance to include the Abort Task would simplify the > solutions to some of these issues and IMHO, it is not too stringent a > requirement that the Abort Task be sent on the same connection. > > After all, a portion of the I/O state would be resident in adapter > firmware/hardware and sending Abort Task on the same connection would ease > the abort process at the target by avoiding communicating across > connections [NICs] for the clean up. > > Thanks, > Santosh > > > We could add the RefCmdSN and it may help plug-in the hole but unless the > > Abort is ussued on the same connection as the original command we can't > be > > sure that the old-one will not pop-up (as we enable ExpCmdSN to move on > we > > don't have even the 2**31-1 protection bracket :-)). Thus sending in > > another nop/abort on the same connection is still required. > > > > To simplify the whole process I will: > > > > a - add RefCmdSN to the Task Management > > b - add a command not received yet to the answers > > > > c - add a part to 7 reading: > > > > 1.1 How to Abort Safely a Command that Was Not Received > > > > To abort safely a task for which the task abort answer is "Command Not > > Received Yet" the initiator must issue another abort command on the > same > > connection as the original command unless this connection was logged > out > > in which case it may send it on any connection. The expected response > > for the second abort is Function Complete (if the command did not > > arrive) or "Task was not in task set". > > > > > > This convoluted scheme is necessary as the target does not know to > what > > connection the hole is related and we don't want to force abort task > to > > the same connection as the original command. > > > > > > Julo > > > > Santosh Rao on 27/04/2001 20:46:53 > > > > Please respond to Santosh Rao > > > > To: Julian Satran/Haifa/IBM@IBMIL, ips@ece.cmu.edu > > cc: matt_wakeley@agilent.com > > Subject: Re: iSCSI : Bridging missing CmdSNs and Abort I/O Error > recovery > > > > > > > > > > Julian, > > > > The conclusion on this thread was that some text was to be added to the > > spec to address this issue. The rev 06 does not have any text to this > > effect. It would help to explicitly describe how initiators should plug > > the hole in CmdSN when they do not intend to use "command retry". > > > > Also, regarding the use of NOP-OUT to fill the hole, why not just use > > Abort Task for the same purpose ? Do we need a 2nd outbound PDU from the > > initiator just to fill the hole ? When the initiator encounters a ULP > > timeout, it would use an Abort Task to error the I/O. If the Abort Task > > can contain the CmdSN of the original command being aborted, targets can > > fill the hole based on that information, without requiring a second > > outbound PDU from the initiator for this purpose. > > > > Some text in the draft on this subject would be helpful to implementors. > > > > - Santosh > > > > > > julian_satran@il.ibm.com wrote: > > > > > > Santosh, > > > > > > You had a possible answer from Matt. However I agree that we might > want > > > to address this in text > > > > > > > > julian_satran@il.ibm.com wrote: > > > > > > I the hole is in the command queue and the task is just aborted the > > > response to the abort task > > > will unveil the fact that it did not reach destination. > > > > > > Initiator can recover from there in several ways - clear the task set, > > fill > > > the hole with an iSCSI noop etc. > > > The latter, I recall, Was sugested to you by Matt Wakeley a while ago. > > > > > > None of them require any changes in the spec. > > > > > > Julo > > > > > > Santosh Rao on 27/04/2001 04:56:26 > > > > > > Please respond to Santosh Rao > > > > > > To: Julian Satran/Haifa/IBM@IBMIL > > > cc: ips@ece.cmu.edu > > > Subject: Re: iSCSI : Bridging missing CmdSNs and Abort I/O Error > > recovery > > > > > > Julian, > > > > > > Could you please clarify if the below issue is going to be addressed in > > > the iSCSI draft, as was discussed earlier. > > > (http://ips.pdl.cs.cmu.edu/mail/msg03155.html). > > > > > > Specifically, is the spec going to address the issue of how initiators > > > can plug a hole in CmdSN sequence when they detect a ULP timeout and/or > > > choose not to use "command retry". > > > > > > Regards, > > > Santosh > > > > > > julian_satran@il.ibm.com wrote: > > > > > > > > Santosh, > > > > > > > > You had a possible answer from Matt. However I agree that we might > > want > > > to > > > > address this in text although > > > > a solution similar to that suggested by Matt should be by now obvious > > to > > > > every implementer - the target should leave a placeholder in the > input > > > > queue until the command after gets delivered. > > > > > > > > Julo > > > > > > > > Santosh Rao on 25/01/2001 21:38:04 > > > > > > > > Please respond to Santosh Rao > > > > > > > > To: IPS Reflector > > > > cc: > > > > Subject: iSCSI : Bridging missing CmdSNs and Abort I/O Error > recovery > > > > > > > > Julian & All, > > > > > > > > The draft is currently lacking a section that addresses abort I/O > error > > > > recovery. Specifically, how is CmdSN bridging issues to be handled in > > > > the case where an initiator chooses not to retry an I/O [that failed > on > > > > a connection failure that affects the delivery of the command to the > > > > target or a digest error at the target] because its ULP timer may > have > > > > expired. > > > > > > > > In such cases, the initiator can send an Abort Task to inform the > > target > > > > that the I.T.T is being aborted and its corresponding CmdSN can be > > > > bridged, instead of having the target stall infinitely in its attempt > > to > > > > enforce ordering and await the missing CmdSN [which is'nt going to > > > > arrive, because the initiator did not retry the command]. > > > > > > > > Regards, > > > > Santosh > > > > > > > > - santoshr.vcf > > > - santoshr.vcf > > - santoshr.vcf > > > > > > > > > > -- > ################################# > Santosh Rao > Software Design Engineer, > HP, Cupertino. > email : santoshr@cup.hp.com > Phone : 408-447-3751 > ################################# --------------82056CF465111AFF57DC636E Content-Type: text/x-vcard; charset=us-ascii; name="santoshr.vcf" Content-Description: Card for Santosh Rao Content-Disposition: attachment; filename="santoshr.vcf" Content-Transfer-Encoding: 7bit begin:vcard n:Rao;Santosh tel;work:408-447-3751 x-mozilla-html:FALSE org:Hewlett Packard, Cupertino.;SISL adr:;;19420, Homestead Road, M\S 43LN, ;Cupertino.;CA.;95014.;USA. version:2.1 email;internet:santoshr@cup.hp.com title:Software Design Engineer x-mozilla-cpt:;21088 fn:Santosh Rao end:vcard --------------82056CF465111AFF57DC636E-- From owner-ips@ece.cmu.edu Thu May 3 21:52:08 2001 Received: from ece.cmu.edu (ECE.CMU.EDU [128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id VAA17824 for ; Thu, 3 May 2001 21:52:07 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f4407v115792 for ips-outgoing; Thu, 3 May 2001 20:07:57 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from palrel2.hp.com (palrel2.hp.com [156.153.255.234]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f4407uH15788 for ; Thu, 3 May 2001 20:07:56 -0400 (EDT) Received: from hpcuhe.cup.hp.com (hpcuhe.cup.hp.com [15.0.80.203]) by palrel2.hp.com (Postfix) with ESMTP id AA1D711B8; Thu, 3 May 2001 17:07:55 -0700 (PDT) Received: from cup.hp.com (santoshr@hpindhhm.cup.hp.com [15.8.80.197]) by hpcuhe.cup.hp.com (8.9.3 (PHNE_18979)/8.9.3 SMKit7.02) with ESMTP id RAA20296; Thu, 3 May 2001 17:07:48 -0700 (PDT) Message-ID: <3AF1F2BC.D97B0DC8@cup.hp.com> Date: Thu, 03 May 2001 17:07:24 -0700 From: Santosh Rao Organization: Hewlett Packard, Cupertino. X-Mailer: Mozilla 4.7 [en] (X11; U; HP-UX B.11.00 9000/778) X-Accept-Language: en MIME-Version: 1.0 To: Matt Wakeley , Julian Satran Cc: IPS Reflector Subject: Re: iscsi: comments to iSCSI rev 6 References: <3AEF20A2.146D62D8@agilent.com> Content-Type: multipart/mixed; boundary="------------560E95DC15760B10606C5C0E" Sender: owner-ips@ece.cmu.edu Precedence: bulk This is a multi-part message in MIME format. --------------560E95DC15760B10606C5C0E Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Matt Wakeley wrote: > > Section 2.12.3: indicate that the LUN is copied from the NOP-IN. This is > > much more clear than "the correct value for the task". > > > > +++ and open it up to strange thing like LU5 asking a nop-out to LU7 ? +++ > > How will that occur? The target sent the NOP-IN ping request. The target > includes a LUN. The only "correct value" that the initiator can return for > the LUN in the NOP-OUT ping response is the value the target sent in the > original request. Julian, Re-wording the LUN descriptions in NOP-IN & NOP-OUT to read that initiators MUST echo the LUN field from a received NOP-IN into a NOP-OUT is a more clear and un-ambiguous definition than the current text. That said, I still continue to have reservations about the need for a LUN field in a non-SCSI PDU. Fibre Channel does not use the LUN field in its ELS' & BLS'. LUN is a SCSI construct and it must not be used in non-scsi PDUs. I'm yet to hear strong reasons why LUN is required in the NOP-IN or NOP-OUT. The originator of the NOP operation must generate task tags independent of LUNs for non-SCSI PDUs. Such PDUs have no scsi context. As a side cosmetic comment, can we re-name "Target Transfer Tag" to "Target Task Tag" to have symmetry with the use of Initiator Task Tag ? - Santosh --------------560E95DC15760B10606C5C0E Content-Type: text/x-vcard; charset=us-ascii; name="santoshr.vcf" Content-Description: Card for Santosh Rao Content-Disposition: attachment; filename="santoshr.vcf" Content-Transfer-Encoding: 7bit begin:vcard n:Rao;Santosh tel;work:408-447-3751 x-mozilla-html:FALSE org:Hewlett Packard, Cupertino.;SISL adr:;;19420, Homestead Road, M\S 43LN, ;Cupertino.;CA.;95014.;USA. version:2.1 email;internet:santoshr@cup.hp.com title:Software Design Engineer x-mozilla-cpt:;21088 fn:Santosh Rao end:vcard --------------560E95DC15760B10606C5C0E-- From owner-ips@ece.cmu.edu Thu May 3 21:52:28 2001 Received: from ece.cmu.edu (ECE.CMU.EDU [128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id VAA17846 for ; Thu, 3 May 2001 21:52:28 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f440Jxf16644 for ips-outgoing; Thu, 3 May 2001 20:19:59 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from palrel2.hp.com (palrel2.hp.com [156.153.255.234]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f440JvH16633 for ; Thu, 3 May 2001 20:19:57 -0400 (EDT) Received: from hpcuhe.cup.hp.com (hpcuhe.cup.hp.com [15.0.80.203]) by palrel2.hp.com (Postfix) with ESMTP id 7E6E5B40; Thu, 3 May 2001 17:19:56 -0700 (PDT) Received: from cup.hp.com (santoshr@hpindhhm.cup.hp.com [15.8.80.197]) by hpcuhe.cup.hp.com (8.9.3 (PHNE_18979)/8.9.3 SMKit7.02) with ESMTP id RAA22011; Thu, 3 May 2001 17:19:51 -0700 (PDT) Message-ID: <3AF1F58F.97C6201D@cup.hp.com> Date: Thu, 03 May 2001 17:19:27 -0700 From: Santosh Rao Organization: Hewlett Packard, Cupertino. X-Mailer: Mozilla 4.7 [en] (X11; U; HP-UX B.11.00 9000/778) X-Accept-Language: en MIME-Version: 1.0 To: Matt Wakeley Cc: ips@ece.cmu.edu Subject: Re: iSCSI: multiple sessions b/n a pair of WWUIs. References: <3AE85DA4.570F800D@cup.hp.com> <3AEDE7F1.54CFF887@agilent.com> Content-Type: multipart/mixed; boundary="------------BDA3345C72E8AF9BEEDFC901" Sender: owner-ips@ece.cmu.edu Precedence: bulk This is a multi-part message in MIME format. --------------BDA3345C72E8AF9BEEDFC901 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Matt Wakeley wrote: > > One clarification: these are unique between any I-T *pair*. > (you make it sound like an initiator must have a unique ISID for all sessions > with all targets) > Matt, While it is true that uniqueness is strictly required only b/n a given I-T pair, using the ISID as a unique key for all the sessions an initiator establishes allows the O.S. to use this session id as a port identifier within the node space. This is a useful utilization of this construct rather than inventing yet another unique token to represent each target within the node name space for targets. Regards, Santosh > -Matt > > Santosh Rao wrote: > > > > Jim, > > > > Thanks for the clarification. Both the iSCSI and the name-disc drafts > > need to explicitly state that ISID is uniquely assigned for all sessions > > within a given initiator. Similarly, TSID is uniquely assigned for all > > sessions within a given target. > > > > Regards, > > Santosh --------------BDA3345C72E8AF9BEEDFC901 Content-Type: text/x-vcard; charset=us-ascii; name="santoshr.vcf" Content-Description: Card for Santosh Rao Content-Disposition: attachment; filename="santoshr.vcf" Content-Transfer-Encoding: 7bit begin:vcard n:Rao;Santosh tel;work:408-447-3751 x-mozilla-html:FALSE org:Hewlett Packard, Cupertino.;SISL adr:;;19420, Homestead Road, M\S 43LN, ;Cupertino.;CA.;95014.;USA. version:2.1 email;internet:santoshr@cup.hp.com title:Software Design Engineer x-mozilla-cpt:;21088 fn:Santosh Rao end:vcard --------------BDA3345C72E8AF9BEEDFC901-- From owner-ips@ece.cmu.edu Thu May 3 21:53:37 2001 Received: from ece.cmu.edu (ECE.CMU.EDU [128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id VAA17860 for ; Thu, 3 May 2001 21:53:36 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f43NtsY14873 for ips-outgoing; Thu, 3 May 2001 19:55:54 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from palrel2.hp.com (palrel2.hp.com [156.153.255.234]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f43NtqH14867 for ; Thu, 3 May 2001 19:55:52 -0400 (EDT) Received: from hpcuhe.cup.hp.com (hpcuhe.cup.hp.com [15.0.80.203]) by palrel2.hp.com (Postfix) with ESMTP id 6B16E123E for ; Thu, 3 May 2001 16:55:51 -0700 (PDT) Received: from cup.hp.com (santoshr@hpindhhm.cup.hp.com [15.8.80.197]) by hpcuhe.cup.hp.com (8.9.3 (PHNE_18979)/8.9.3 SMKit7.02) with ESMTP id QAA19087; Thu, 3 May 2001 16:55:45 -0700 (PDT) Message-ID: <3AF1EFE9.8977BF09@cup.hp.com> Date: Thu, 03 May 2001 16:55:21 -0700 From: Santosh Rao Organization: Hewlett Packard, Cupertino. X-Mailer: Mozilla 4.7 [en] (X11; U; HP-UX B.11.00 9000/778) X-Accept-Language: en MIME-Version: 1.0 To: ips@ece.cmu.edu Subject: Re: iscsi: Text keys References: Content-Type: multipart/mixed; boundary="------------874A8A325FDE01922BC57172" Sender: owner-ips@ece.cmu.edu Precedence: bulk This is a multi-part message in MIME format. --------------874A8A325FDE01922BC57172 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit julian_satran@il.ibm.com wrote: > > 1) There are several text keys which have been specified > with Use=ALL. Such keys can be negotiated during > full-feature phase, which raises interesting questions > about their effect on currently outstanding tasks. > > For example, > a) maxOutstandingR2T : what is the effect on currently > outstanding R2Ts ? > b) LoginLogoutMaxTime : how does it affect connections > which just got dropped and need to be recovered ? > c) DataPDULength : effect on data PDUs in flight. > > Similar questions need to be answered for practically > all the keys which have this behaviour. For the sake of > simplicity, it may be better to forbid such usage and > allow these keys to be only negotiated in the connection > or session login phase. > > +++ some of the parameters you mentioned are SCSI mode-page parameters. > Even if we > restricted them to login they can be changed by a SCSI mode-set. I agree > that keeping them all > restricted to login would simplify implementations but I am not convince it > would make them better. Evidently a negotiated change will apply to all > packets that are sent after the negotiation ended (and a negotiation end is > known to both parties) and an initiator will avoid sending things in > violation of its last offering. > Except that we will have to find a way to reject mode set I am open to > restrict those parameters to login if that is what the majority of > implementors tells me. > +++ Julian, 1) I would like to request that login/text key negotiation be restricted to login time only and have been asking for this on the reflector earlier as well. Implementations would be simpler with fewer options, and we are better off without options like these that are prone to side-effects when changed during full feature phase. 2) Regarding the mode select setting of these parameters, this issue could not be discussed at the Nashua interim meeting for lack of time. I'd like to once again suggest that iSCSI use only 1 scheme(text/login keys) for setting these keys while allowing a query of these options from both layers. 3) Speaking of querying for options, my understanding is that the initiator sends all its options and the target responds with its chosen one from that list during login/text negotiation. Is there a mechanism by which an initiator could query the target for its supported key capabilities ? (i.e. what is the equivalent of a mode sense for these login keys ?). 4) Lastly, the login/text negotiation can comprise of several text command/response exchanges with each being based on the same Initiator Task Tag. The addition of a Target Task Tag field to the login response, text command and text response PDUs would help targets to lookup the negotiation state based on the Target Task Tag which is easier than having to perform a lookup on (Initiator iSCSI Name + ISID). Regards, Santosh --------------874A8A325FDE01922BC57172 Content-Type: text/x-vcard; charset=us-ascii; name="santoshr.vcf" Content-Description: Card for Santosh Rao Content-Disposition: attachment; filename="santoshr.vcf" Content-Transfer-Encoding: 7bit begin:vcard n:Rao;Santosh tel;work:408-447-3751 x-mozilla-html:FALSE org:Hewlett Packard, Cupertino.;SISL adr:;;19420, Homestead Road, M\S 43LN, ;Cupertino.;CA.;95014.;USA. version:2.1 email;internet:santoshr@cup.hp.com title:Software Design Engineer x-mozilla-cpt:;21088 fn:Santosh Rao end:vcard --------------874A8A325FDE01922BC57172-- From owner-ips@ece.cmu.edu Fri May 4 00:30:03 2001 Received: from ece.cmu.edu (ECE.CMU.EDU [128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id AAA21051 for ; Fri, 4 May 2001 00:30:03 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f4421CJ23808 for ips-outgoing; Thu, 3 May 2001 22:01:12 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from palrel1.hp.com (palrel1.hp.com [156.153.255.242]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f44219H23804 for ; Thu, 3 May 2001 22:01:09 -0400 (EDT) Received: from hpcuhe.cup.hp.com (hpcuhe.cup.hp.com [15.0.80.203]) by palrel1.hp.com (Postfix) with ESMTP id A65AA10DC for ; Thu, 3 May 2001 19:01:08 -0700 (PDT) Received: (from santoshr@localhost) by hpcuhe.cup.hp.com (8.9.3 (PHNE_18979)/8.9.3 SMKit7.02) id TAA00579; Thu, 3 May 2001 19:01:02 -0700 (PDT) From: Santosh Rao Message-Id: <200105040201.TAA00579@hpcuhe.cup.hp.com> Subject: iSCSI : Review Feedback on iSCSI Rev 06 To: ips@ece.cmu.edu (ips) Date: Thu, 3 May 2001 19:01:02 -0700 (PDT) X-Mailer: ELM [version 2.5 PL2] MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-ips@ece.cmu.edu Precedence: bulk Content-Transfer-Encoding: 7bit Julian & All, Enclosed is my review feedback on the iSCSI draft Rev 06. Regards, Santosh -------------------------------------------------------------------- 1) Suggest re-naming the following in the interests of better clarity : a) Rename the ExpDataSN in SCSI Response PDU to EndDataSN as was used in the previous version of the draft. (indicative of being the final Data sequence number.) b) Rename DataSN in the R2T PDU to R2TSN. (This is really an R2T seqeunce number.) c) Rename R2TExpDataSn to EndR2TSN. (indicative of being the final R2T sequence number.) 2) If a target issues a REJECT response to any received PDU within a SCSI Command operation, does this imply the target has internally aborted the I/O as well ? Will the target send a SCSI Response in addition following the Reject ? What behaviour must an initiator expect when it sees a reject in response to some PDU within a SCSI command ? Does it treat this as a termination of the I/O or should it await a SCSI Response ? Also, if the target does terminate the command on sending REJECT, can a "retry" be issued ? In the interests of clearing all these obscurities with the REJECT, I'd like to suggest the following model : - SCSI Command be always responded with a SCSI Response. - SCSI Task Mgmt Command be always responded with a SCSI Task Mgmt response. - Only non-SCSI PDUs may be errored back with a REJECT PDU. The above is also semantically aligned with the FC LS_RJT/BA_RJT model. 3) Why does the new draft limit the size of text commands and nop commands to 4096 bytes ? What is the rationale behind the magic number selection of 4096 ? Since DataPDULength is defined in units of 512 bytes, how does iSCSI deal with fragmentation caused due to the usage of a DataPDULength < 4K and an attempt to perform a 4K text , login or nop operation ? I believe such scenarios were discussed in an earlier thread titled "Fragmentation & Reassembly issues in iSCSI." (http://ips.pdl.cs.cmu.edu/mail/msg03031.html) and the resolution was to enforce a restriction that the size of a login, nop & text operation be limited to DataPDULength (?). On a seperate note, what happened to PingMaxReplyLength ? It seems to have been removed from the draft. How can the 2 end points now negotiate a max limit on nop ping operations ? (for ex : how can a target enforce that the nop ping operation data buffer not exceed 512 bytes ?). 4) As mentioned in a seperate mail, the login or text negotiation can span several command/response exchanges each using the same initiator task tag. The addition of a "Target Task Tag" field in the login response, text command and text response would allow targets to perform a quick loopup on their T.T.T for this negotiation state information, instead of having to lookup based on (Initiator iSCSI Name + ISID). 5) Section 2.8.3 "The data length of a text command or response SHOULD be less than 4096 bytes." suggest re-wording the above to : "The data length of a login, nop & text command or response MUST be less than DataPDULength. For the leading session login, the length of the login command MUST be less than 512 bytes." The above will ensure that login, text & nop operations will not result in fragmentation. 6) Section 2.8.3 "A session may have only one outstanding text command or text command sequence at any given time." The above is contradicting itself. Is it only one outstanding text command or one outstanding text sequence. Suggest re-wording the above to : " A session may have only one outstanding text command at any given time." 7) Section 2.11.5 "The TSID is an initiator identifying tag set by the target. It is valid only if the connection is accepted." ^^^^^^^^^^ The above needs to be re-worded as : "only if the login is accepted" 8) Section 2.13.1 "A target may also issue a NOP-In on its own to carry a changed CmdSN ^^^^^ and/or MaxCmdSN if there is no other PDU to carry them for a long time." The above reference to changed CmdSN must be re-worded to "changed ExpCmdSN". 9) Section 2.18.1 "Target requests logout". Suggest removal of the above option. A target that needs to logout would use either : "Target will drop the connection" or "Target will drop all connections" The "Target requests logout" also does not allow the target to specify which flavor of logout it wishes to receive. It is better to remove this option and use only a single scheme. 10) Section 2.19 The section 2.19 seems out of place being in the midst of descriptions of other iSCSI PDUs. It should be moved out to a seperate location since it has nothing to do with PDU descriptions. 11) As was brought up at the Nashua interim meeting, Status SNACK must be made optional to support. Simple implementations as also gateway and bridging products may choose not to [or are unable to] support SNACK mechanisms. Non-use of Status SNACK must be accompanied by setting StatSN to 0 on all status PDUs. 12) Suggest the addition of new login keys to negotiate support for Data SNACK & Status SNACK. If a target does not support Status SNACK, it must set StatSN to 0 for all Status PDUs. 13) Section 3.2 " 3.2 iSCSI Logical Unit Control Mode Page The following outlines the iSCSI Port mode page: " The above needs to be re-worded to : "The following outlines the iSCSI LUN Control Mode Page: ". As suggested earlier, perhaps, suggest using the SPC-2 terms "protocol specific port page" and "protocol specific LUN page" for these mode pages. 14) Section 3.3.3 LoginLogoutMinTime The wording must be clarified to indicate that this delay is only in the case the prior logout was due to a target originated event. There is no need for an initiator to delay after a logout originated by itself. For example, while doing open, read/write, close type of operations on a single thread of activity for the entire session, it may be typical to see iSCSI level on-the-wire activity of login, read/write, logout.... In the above type of scenarios, LoginLogoutMinTime is not applicable and the initiator must not delay prior to issuing logins. 15) Section 4.3 "If the target responds to a text or Login command with the F bit set to 1, with a text response with the F bit set to 0, or a login response with the text bit set to 0, the initiator must keep sending ^^^^^^^^ the text command (even empty) with the F bit set to 1 until it gets the Login Response with the F bit set to 1." The above must be re-worded to : "or a login response with the F bit set to 0, ...". 16) Section 6 "It is further assumed that a target will keep the "status & sense" for a command it has executed while the total number of outstanding commands and executed commands does not exceed its limit and status has not been acknowledged." What if the target has exceeded its limit on resources ? Can it drop these stale resources or must it close its command window and not accept further new commands ? If it is the latter, then, iSCSI is optimizing for recovery paths and is impacting performance paths by limiting the ability of the target to accept new I/Os due to having to hold onto stale resources awaiting ExpStatSN updates. 17) Section 6.7.3 "N.B. As an alternative to Logout and reissue commands, the initiator MAY instead reset the target and terminate all outstanding commands with a service response indicating Delivery Subsystem Failure. The initiator MUST perform one of the two actions." As was brought up at the Nashua interim meeting, suggest removal of the above sentence since it may lead implementors to use Target Reset as a form of error recovery while dealing with session-level error recovery. An alternate form of error recovery in its place would be to use : "logout - cleans the connection" or "logout - cleans the session" 18) Section 7.3 This section outlines how deterministic clean-up of tasks can be achieved during task mgmt command execution. However, in the absence of mandating this behaviour for targets, initiators cannot rely on the same for reliable deterministic clean-up of pending tasks and will need to go ahead and implement their own forms of abort task based cleanup. This section is meaningless in the absence of a mandate that forces targets to do as described. In its absence, initiators will be forced to implement their own schemes for clean up. 19) Appendix E "15 AccessID Use: ALL Who: Initiator AccessID= Deliver a SCSI AccessID to the target " What is a SCSI Access ID ? Can further descriptions be added to this definition and a reference be added to SAM2 or some other SCSI document, if appropriate. 20) Appendix E In several places, IO seems to have been used instead of LO. Is this a typo or a new acronym ? Speaking of acronyms, can we add an acronym section to the draft ? 21) Appendix E the description of ImmediateData is describing the case : "ImmediateData=YES & InitialR2T=YES" twice in differing manners. One of them needs fixing [or removal]. --------------------------------------------------------------------------- -- ################################# Santosh Rao Software Design Engineer, HP, Cupertino. email : santoshr@cup.hp.com Phone : 408-447-3751 ################################# From owner-ips@ece.cmu.edu Fri May 4 04:39:08 2001 Received: from ece.cmu.edu (ECE.CMU.EDU [128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id EAA07026 for ; Fri, 4 May 2001 04:39:08 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f4458k206379 for ips-outgoing; Fri, 4 May 2001 01:08:46 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from bacchus-int.veritas.com (bacchus.veritas.com [204.177.156.37]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f4458hH06372 for ; Fri, 4 May 2001 01:08:43 -0400 (EDT) Received: from megami.veritas.com (urd.veritas.com [192.203.47.101]) by bacchus-int.veritas.com (8.11.2/8.11.2) with SMTP id f4454er23650 for ; Thu, 3 May 2001 22:06:00 -0700 (PDT) Received: from vivektest2([172.29.1.199]) (1994 bytes) by megami.veritas.com via sendmail with P:smtp/R:smart_host/T:smtp (sender: ) id for ; Thu, 3 May 2001 22:04:32 -0700 (PDT) (Smail-3.2.0.101 1997-Dec-17 #4 built 1999-Aug-24) Message-ID: <00d201c0d485$ee927b50$c7011dac@vivektest2> From: "Sairam" To: Subject: Virus infected e-mail sent from my account. Date: Fri, 4 May 2001 11:34:04 +0100 MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="----=_NextPart_000_00C5_01C0D48E.1FA1B5F0" X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 4.72.3110.5 X-MimeOLE: Produced By Microsoft MimeOLE V4.72.3110.3 Sender: owner-ips@ece.cmu.edu Precedence: bulk This is a multi-part message in MIME format. ------=_NextPart_000_00C5_01C0D48E.1FA1B5F0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable Hello, It was due to a virus infected attachment dumped onto my mail-box; which = automatically sent out this e-mail from my account with infected = attachment to all with whom I have been exchanging e-mails.=20 My apologies for the inconvenience caused. Thanks & Regards, Sairam ------=_NextPart_000_00C5_01C0D48E.1FA1B5F0 Content-Type: text/html; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable

Hello,
 
It was due to a virus infected attachment dumped onto my mail-box; = which=20 automatically sent out this e-mail from my account with infected = attachment to=20 all with whom I have been exchanging e-mails.
 
My apologies for the inconvenience caused.
 
Thanks & Regards,
Sairam
 
------=_NextPart_000_00C5_01C0D48E.1FA1B5F0-- From owner-ips@ece.cmu.edu Fri May 4 13:14:50 2001 Received: from ece.cmu.edu (ECE.CMU.EDU [128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id NAA17229 for ; Fri, 4 May 2001 13:14:50 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f44EvK925184 for ips-outgoing; Fri, 4 May 2001 10:57:20 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from mercury.Sun.COM (mercury.Sun.COM [192.9.25.1]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f443w4H01517 for ; Thu, 3 May 2001 23:58:04 -0400 (EDT) Received: from centralmail1.Central.Sun.COM ([129.147.62.10]) by mercury.Sun.COM (8.9.3+Sun/8.9.3) with ESMTP id UAA22628 for ; Thu, 3 May 2001 20:57:53 -0700 (PDT) Received: from esun1as-mm. (esun1as-mm.Central.Sun.COM [129.147.34.144]) by centralmail1.Central.Sun.COM (8.9.3+Sun/8.9.3/ENSMAIL,v2.1p1) with SMTP id VAA16643; Thu, 3 May 2001 21:57:52 -0600 (MDT) Received: from sun.com by esun1as-mm. (SMI-8.6/SMI-SVR4) id WAA19146; Thu, 3 May 2001 22:04:00 -0600 Message-ID: <3AF228BD.4A1DB3CB@sun.com> Date: Thu, 03 May 2001 20:57:49 -0700 From: Raghavendra Rao X-Mailer: Mozilla 4.51 [en] (Win98; U) X-Accept-Language: en MIME-Version: 1.0 To: ips@ece.cmu.edu Subject: iSNS: Event registry and notification Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-ips@ece.cmu.edu Precedence: bulk Content-Transfer-Encoding: 7bit In reviewing -02 spec of iSNS, the event registration and notification need more work to live up to the promise (and requirement of Naming and discovery spec) of being lightweight. Right now, it defines catch-all and receive-all events. To start with, an iSCSI initiator MUST be able to register a) for state changes of specific/interested iSCSI targets. b) for addition of iSCSI targets that provide Login access control to this iSCSI initiator Along similar lines, it would also help to provide an iSCSI target the ability to register for an event when an iSCSI initiator is added/removed to the login control list of this iSCSI target. An iSCSI target, upon receiving such a notification, can then simply download the updated list of allowed iSCSI initiators. Such types of notifications help reduce notification storms by - Helping a small iSCSI initiator client receive and process absolutely required notification (Note that totally turning off notification is a BAD idea) - Reducing needless traffic. -JP From owner-ips@ece.cmu.edu Fri May 4 14:29:11 2001 Received: from ece.cmu.edu (ECE.CMU.EDU [128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id OAA18604 for ; Fri, 4 May 2001 14:29:10 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f44GFUP02210 for ips-outgoing; Fri, 4 May 2001 12:15:30 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from internaut.com ([64.38.134.99]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f44GFRH02205 for ; Fri, 4 May 2001 12:15:28 -0400 (EDT) Received: from localhost (aboba@localhost) by internaut.com (8.9.3/8.9.3) with ESMTP id JAA02869; Fri, 4 May 2001 09:06:23 -0700 (PDT) (envelope-from aboba@internaut.com) Date: Fri, 4 May 2001 09:06:23 -0700 (PDT) From: Bernard Aboba To: Black_David@emc.com cc: ips@ece.cmu.edu Subject: Re: iSCSI Security rough consensus In-Reply-To: <0F31E5C394DAD311B60C00E029101A07080154F9@corpmx9.isus.emc.com> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-ips@ece.cmu.edu Precedence: bulk > Although the approach of using SRP to key ESP has > a lot of promise, making it a requirement in advance > of a draft providing details that can be checked > by other security experts seems premature ... and > now I have to go help get that draft written > in my "copious spare time" ;-). Once that draft > is in hand, we can make a concrete decision about > requiring that mechanism. > I agree that requiring SRP for use in IPSEC key exchange with iSCSI is premature. We already have KINK and IKE key exchange methods, and before creating a new application-specific method, we need to document why the existing methods are inadequate. Also, RFC 2945 is an abstract protocol, not a description of what bits flow over the wire. If SRP is going to be adopted for IPSEC key exchange, then I'd argue that this should be handled in the security area, not in the IPS WG, and should be available for any application running over IPSEC, not just iSCSI. Also, given that another key exchange method is likely to be used (IKE, KINK), this implies that another authentication has already taken place to set up the IPSEC SA. Thus, requiring SRP in addition to IKE/KINK adds *yet another* authentication and key exchange to the process of setting up an IPSEC SA. Are we doing this purely in order to circumvent IKE's limitations with respect to password-based authentication? If so, then I'd be concerned that many other applications will encounter the same limitations, and we may be setting ourselves up for the proliferation of application-specific IPSEC key exchange mechanisms. I believe that there is evidence that such a profileration is already in progress. In the end, IPSEC may end up becoming little more than a "framework" within which application and vendor-specific key exchanges are developed. From owner-ips@ece.cmu.edu Fri May 4 14:33:52 2001 Received: from ece.cmu.edu (ECE.CMU.EDU [128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id OAA18710 for ; Fri, 4 May 2001 14:33:52 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f44Gfvt04545 for ips-outgoing; Fri, 4 May 2001 12:41:57 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from server1.NishanSystems.COM (smtp.nishansystems.com [216.217.36.162]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f44GftH04540 for ; Fri, 4 May 2001 12:41:56 -0400 (EDT) Received: by smtp.nishansystems.com with Internet Mail Service (5.5.2653.19) id ; Fri, 4 May 2001 09:41:49 -0700 Message-ID: From: Joshua Tseng To: "'Raghavendra Rao'" , ips@ece.cmu.edu Subject: RE: iSNS: Event registry and notification Date: Fri, 4 May 2001 09:41:48 -0700 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2653.19) Content-Type: text/plain; charset="iso-8859-1" Sender: owner-ips@ece.cmu.edu Precedence: bulk Raghavendra, Although the type (initiator or target) of iSNS client is stored in the iSNS database, note that the iSNS server does not make a distinction in how it interacts with each type of client--both types (initiators and targets) are uniformly treated as iSNS clients. That means both initiators and targets can register for notification and are members of Discovery Domains. > In reviewing -02 spec of iSNS, the event registration and > notification > need > more work to live up to the promise (and requirement of Naming and > discovery spec) of being lightweight. Right now, it defines catch-all > and receive-all events. > > To start with, an iSCSI initiator MUST be able to register > > a) for state changes of specific/interested iSCSI targets. > > b) for addition of iSCSI targets that provide Login access control > to this iSCSI initiator Please review the section on Discovery Domains. DD's are used to define event registration and notification policy. If you still think more clarification is needed, let me know what is not clear. > > Along similar lines, it would also help to provide an iSCSI target the > ability to register for an event when an iSCSI initiator is > added/removed > to the login control list of this iSCSI target. An iSCSI target, upon > receiving such a notification, can then simply download the updated > list of allowed iSCSI initiators. Yes, of course. This is the expected behavior of a target that subordinates its login control policy to the iSNS. > > Such types of notifications help reduce notification storms by > - Helping a small iSCSI initiator client receive and process > absolutely required notification (Note that totally turning > off notification is a BAD idea) > > - Reducing needless traffic. Discovery Domains are the mechanism used to reduce/eliminate notification storms. Regards, Josh > > -JP > From owner-ips@ece.cmu.edu Fri May 4 15:29:05 2001 Received: from ece.cmu.edu (ECE.CMU.EDU [128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id PAA19616 for ; Fri, 4 May 2001 15:29:05 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f44HmSP10446 for ips-outgoing; Fri, 4 May 2001 13:48:28 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from mxic2.us.dg.com (mxic2.us.dg.com [128.221.31.40]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f44HmNH10434 for ; Fri, 4 May 2001 13:48:23 -0400 (EDT) Received: by mxic2.us.dg.com with Internet Mail Service (5.5.2650.21) id ; Fri, 4 May 2001 13:48:07 -0400 Message-ID: <0F31E5C394DAD311B60C00E029101A070801550B@corpmx9.isus.emc.com> From: Black_David@emc.com To: jtseng@NishanSystems.com, Black_David@emc.com, ips@ece.cmu.edu Subject: RE: iSCSI Security rough consensus Date: Fri, 4 May 2001 13:48:16 -0400 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2650.21) Content-Type: text/plain Sender: owner-ips@ece.cmu.edu Precedence: bulk > I do not understand what a requirement for SRP to > generate keys for ESP/IPSec would add to security, > especially if IPSec and iSCSI are implemented on the > same box. Could you summarize why this recommendation > was made? (unfortunately, I missed this part of the > meeting to catch a plane) By comparison to full IPSec with IKE, using SRP to key ESP does not improve security. The underlying issue is IKE complexity (i.e., the code and effort required to implement it). Hence the rationale for using SRP to key ESP is that it provides dynamic key generation without implementing IKE -- this is an improvement over pre-shared keys at a much lower code and effort cost for a single-box (i.e., no external security gateway) implementation. Thanks, --David --------------------------------------------------- David L. Black, Senior Technologist EMC Corporation, 42 South St., Hopkinton, MA 01748 +1 (508) 435-1000 x75140 FAX: +1 (508) 497-8500 black_david@emc.com Mobile: +1 (978) 394-7754 --------------------------------------------------- From owner-ips@ece.cmu.edu Fri May 4 15:37:50 2001 Received: from ece.cmu.edu (ECE.CMU.EDU [128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id PAA19677 for ; Fri, 4 May 2001 15:37:50 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f44GnVY05194 for ips-outgoing; Fri, 4 May 2001 12:49:31 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from server1.NishanSystems.COM (smtp.nishansystems.com [216.217.36.162]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f44GnSH05189 for ; Fri, 4 May 2001 12:49:29 -0400 (EDT) Received: by smtp.nishansystems.com with Internet Mail Service (5.5.2653.19) id ; Fri, 4 May 2001 09:49:17 -0700 Message-ID: From: Joshua Tseng To: "'Black_David@emc.com'" , ips@ece.cmu.edu Subject: RE: iSCSI Security rough consensus Date: Fri, 4 May 2001 09:49:17 -0700 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2653.19) Content-Type: text/plain; charset="iso-8859-1" Sender: owner-ips@ece.cmu.edu Precedence: bulk David, I do not understand what a requirement for SRP to generate keys for ESP/IPSec would add to security, especially if IPSec and iSCSI are implemented on the same box. Could you summarize why this recommendation was made? (unfortunately, I missed this part of the meeting to catch a plane) Thanks, Josh > -----Original Message----- > From: Black_David@emc.com [mailto:Black_David@emc.com] > Sent: Wednesday, May 02, 2001 8:54 PM > To: ips@ece.cmu.edu > Subject: iSCSI Security rough consensus > > > The rest of the Nashua minutes will be coming, but > this item is important enough to post on the list now. > > The rough consensus on "mandatory to implement" iSCSI > security in the Nashua meeting was that the following > two items will be REQUIRED (mandatory to implement): > > - ESP (part of IPSec) with NULL encryption. This provides > cryptographic integrity, and authentication, depending > on how its keys are managed. The rest of > IPSec (e.g., IKE and AH) will be OPTIONAL. > - SRP for in-band authentication. The remaining in-band > authentication algorithms in the current iSCSI draft > will be OPTIONAL. > > There was also rough consensus in the meeting to pursue > a direction of using SRP to generate the keys for ESP, > and I asked whether there were problems > with the fact that such an approach would not permit > solutions that use an IPSec security gateway external > to an iSCSI implementation. > > While there were no answers in the meeting, > I've gotten some strong "Yes, there are problems" > responses off line, and between them and the fact that > there are a bunch of details to work out in exactly > how to use SRP to key ESP, I would propose > that the security requirements be just the two > bullets above (i.e., ESP with NULL encryption and > SRP are REQUIRED). This allows external gateways, > and keying of ESP with IKE or pre-shared keys, > and is consistent with the bulk of the discussion > in the meeting. > > Although the approach of using SRP to key ESP has > a lot of promise, making it a requirement in advance > of a draft providing details that can be checked > by other security experts seems premature ... and > now I have to go help get that draft written > in my "copious spare time" ;-). Once that draft > is in hand, we can make a concrete decision about > requiring that mechanism. > > Also, the integrity hash and signature algorithm > that MUST be implemented for ESP w/Null Encryption > still need to be designated -- in consultation with > the security area and security experts (e.g., Ted > Ts'o, ipsec WG co-chair, who was at the Nashua > meeting) the hope is to bring a recommendation to the > WG in the near future. A complicating factor is that > new hash algorithms are being introduced as a > consequence of the new AES/Rijndael cipher. Requiring > such a new algorithm (e.g., as opposed to the current > SHA-1 or MD5) was discussed as a desirable direction > in the meeting, but there are a bunch of details > that need to be checked (e.g., state of IETF use > and standardization of those algorithms). > > Comments to the list, especially if anyone disagrees > with the proposed requirements stated above. Specific > input from security-knowledgeable folks on algorithm > selection should probably be sent directly to me, as > the IPS list is not the best forum for that purpose. > > Thanks, > --David > > --------------------------------------------------- > David L. Black, Senior Technologist > EMC Corporation, 42 South St., Hopkinton, MA 01748 > +1 (508) 435-1000 x75140 FAX: +1 (508) 497-8500 > black_david@emc.com Mobile: +1 (978) 394-7754 > --------------------------------------------------- > From owner-ips@ece.cmu.edu Fri May 4 16:34:50 2001 Received: from ece.cmu.edu (ECE.CMU.EDU [128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id QAA20257 for ; Fri, 4 May 2001 16:34:50 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f44JBTh18205 for ips-outgoing; Fri, 4 May 2001 15:11:29 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from smtp018.mail.yahoo.com (smtp018.mail.yahoo.com [216.136.174.115]) by ece.cmu.edu (8.11.0/8.10.2) with SMTP id f44JBQH18194 for ; Fri, 4 May 2001 15:11:26 -0400 (EDT) Received: from sdsl-216-36-75-164.dsl.sjc.megapath.net (HELO somesh) (216.36.75.164) by smtp.mail.vip.sc5.yahoo.com with SMTP; 4 May 2001 19:11:23 -0000 X-Apparently-From: Reply-To: From: "Somesh Gupta" To: , , Subject: RE: iSCSI Security rough consensus Date: Fri, 4 May 2001 12:05:10 -0700 Message-ID: MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 (Normal) X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook IMO, Build 9.0.2416 (9.0.2910.0) In-reply-to: <0F31E5C394DAD311B60C00E029101A070801550D@corpmx9.isus.emc.com> Importance: Normal X-MimeOLE: Produced By Microsoft MimeOLE V5.00.2919.6700 Sender: owner-ips@ece.cmu.edu Precedence: bulk Content-Transfer-Encoding: 7bit David, > -----Original Message----- > From: Black_David@emc.com [mailto:Black_David@emc.com] > Sent: Friday, May 04, 2001 11:58 AM > To: someshg@yahoo.com; ips@ece.cmu.edu > Subject: RE: iSCSI Security rough consensus > > > > Does this consensus mean that the iSCSI header and data CRCs > > are no longer part of the specification, or are we > > still requiring one or the other or both? > > Repeat after me: "CRCs are not security mechanisms" ;-) > ;-), and see the previous email on this list about the > consequences of WEP trying to use CRCs in this fashion. My only excuse is that I did not mean "CRCs are a security mechanism":-) I only meant that since ESP will provide integrity (and authentication), will we still have CRCs. > > Yes, CRCs are still required for data integrity (e.g., > when ESP is not present). If one knows that ESP with > its keyed HMAC is being used in the stack between TCP and > IP, then it would make sense not to use CRCs at the iSCSI > level, hence they're required to implement, but configurable > to use (which will also be the case for ESP). Sure would be nice if we could make up our minds and just implement one mechanism. Here we have two mechanisms (iSCSI header/data integrity and ESP) which are both mandatory to implement and optional to use. Since ESP seems like a superset why not just have that and get rid of the "integrity only" iSCSI CRC mechanism. Hopefully this will lead to everyone implementing it and using it (leading to a better and more secure world :-)). > This may > not always be possible, as one of the things mentioned > in the meeting is that if the IPSec implementation is > independent of iSCSI (e.g., supplied as part of the OS), > there's no general standard way for iSCSI to figure out > that IPSec is there or what it's doing to traffic on any > particular iSCSI connection. > > Thanks, > --David > > --------------------------------------------------- > David L. Black, Senior Technologist > EMC Corporation, 42 South St., Hopkinton, MA 01748 > +1 (508) 435-1000 x75140 FAX: +1 (508) 497-8500 > black_david@emc.com Mobile: +1 (978) 394-7754 > --------------------------------------------------- _________________________________________________________ Do You Yahoo!? Get your free @yahoo.com address at http://mail.yahoo.com From owner-ips@ece.cmu.edu Fri May 4 16:34:58 2001 Received: from ece.cmu.edu (ECE.CMU.EDU [128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id QAA20268 for ; Fri, 4 May 2001 16:34:57 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f44IoVt16186 for ips-outgoing; Fri, 4 May 2001 14:50:31 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from maho3msx2.isus.emc.com (maho3msx2.isus.emc.com [128.221.11.32]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f44IoTH16182 for ; Fri, 4 May 2001 14:50:29 -0400 (EDT) Received: by maho3msx2.isus.emc.com with Internet Mail Service (5.5.2650.21) id ; Fri, 4 May 2001 14:50:23 -0400 Message-ID: <0F31E5C394DAD311B60C00E029101A070801550C@corpmx9.isus.emc.com> From: Black_David@emc.com To: aboba@internaut.com, Black_David@emc.com Cc: ips@ece.cmu.edu Subject: RE: iSCSI Security rough consensus Date: Fri, 4 May 2001 14:50:21 -0400 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2650.21) Content-Type: text/plain Sender: owner-ips@ece.cmu.edu Precedence: bulk Bernard, Thanks for the comments. > I agree that requiring SRP for use in IPSEC key exchange with iSCSI is > premature. We already have KINK and IKE key exchange methods, and before > creating a new application-specific method, we need to document why the > existing methods are inadequate. The rationale boils down to complexity and cost to deploy. IKE has complexity issues, and KINK has cost of deployment issues in an environment that isn't otherwise using Kerberos as well as the usual feature of requiring the Kerberos server to be available in order to authenticate. > Also, RFC 2945 is an abstract protocol, not a description of what bits flow over > the wire. The iSCSI draft has documented/will document these details. > If SRP is going to be > adopted for IPSEC key exchange, then I'd argue that this should be handled > in the security area, not in the IPS WG, and should be available for any > application running over IPSEC, not just iSCSI. Fair enough, let's get the draft on SRP + ESP written first, and then we can figure out which IETF WG should work on it. > Also, given that another key exchange method is likely to be used (IKE, > KINK), this implies that another authentication has already taken place to > set up the IPSEC SA. Thus, requiring SRP in addition to IKE/KINK adds *yet > another* authentication and key exchange to the process of setting up an > IPSEC SA. Are we doing this purely in order to circumvent IKE's > limitations with respect to password-based authentication? I don't think the "given" is correct. I believe this is intended for a system that doesn't implement either IKE or KINK. If all three were implemented, only one would be used to set up any single IPSec SA. > If so, then I'd be concerned that many other applications will encounter > the same limitations, and we may be setting ourselves up for the > proliferation of application-specific IPSEC key exchange mechanisms. I > believe that there is evidence that such a proliferation is already in > progress. In the end, IPSEC may end up becoming little more than a > "framework" within which application and vendor-specific key exchanges > are developed. I think this is a case of yet another group of people looking at IKE and developing indigestion. I would support Bernard's comment above that this ought to be done once in a general fashion that's useful to other applications provided that it can actually get done without pulling back in most of IKE's functionality as "nice to have" -- that's "son of IKE"'s job :-). Thanks, --David --------------------------------------------------- David L. Black, Senior Technologist EMC Corporation, 42 South St., Hopkinton, MA 01748 +1 (508) 435-1000 x75140 FAX: +1 (508) 497-8500 black_david@emc.com Mobile: +1 (978) 394-7754 --------------------------------------------------- From owner-ips@ece.cmu.edu Fri May 4 16:38:00 2001 Received: from ece.cmu.edu (ECE.CMU.EDU [128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id QAA20297 for ; Fri, 4 May 2001 16:38:00 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f44IeQZ15290 for ips-outgoing; Fri, 4 May 2001 14:40:26 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from smtp014.mail.yahoo.com (smtp014.mail.yahoo.com [216.136.173.58]) by ece.cmu.edu (8.11.0/8.10.2) with SMTP id f44IeNH15283 for ; Fri, 4 May 2001 14:40:24 -0400 (EDT) Received: from sdsl-216-36-75-164.dsl.sjc.megapath.net (HELO somesh) (216.36.75.164) by smtp.mail.vip.sc5.yahoo.com with SMTP; 4 May 2001 18:40:21 -0000 X-Apparently-From: Reply-To: From: "Somesh Gupta" To: , Subject: RE: iSCSI Security rough consensus Date: Fri, 4 May 2001 11:34:07 -0700 Message-ID: MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 (Normal) X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook IMO, Build 9.0.2416 (9.0.2910.0) In-reply-to: <0F31E5C394DAD311B60C00E029101A07080154F9@corpmx9.isus.emc.com> Importance: Normal X-MimeOLE: Produced By Microsoft MimeOLE V5.00.2919.6700 Sender: owner-ips@ece.cmu.edu Precedence: bulk Content-Transfer-Encoding: 7bit David, Apologies for not being able to stay for the entire meeting. I had a question about integrity. Does this consensus mean that the iSCSI header and data CRCs are no longer part of the specification, or are we still requiring one or the other or both? Somesh > -----Original Message----- > From: owner-ips@ece.cmu.edu [mailto:owner-ips@ece.cmu.edu]On Behalf Of > Black_David@emc.com > Sent: Wednesday, May 02, 2001 8:54 PM > To: ips@ece.cmu.edu > Subject: iSCSI Security rough consensus > > > The rest of the Nashua minutes will be coming, but > this item is important enough to post on the list now. > > The rough consensus on "mandatory to implement" iSCSI > security in the Nashua meeting was that the following > two items will be REQUIRED (mandatory to implement): > > - ESP (part of IPSec) with NULL encryption. This provides > cryptographic integrity, and authentication, depending > on how its keys are managed. The rest of > IPSec (e.g., IKE and AH) will be OPTIONAL. > - SRP for in-band authentication. The remaining in-band > authentication algorithms in the current iSCSI draft > will be OPTIONAL. > > There was also rough consensus in the meeting to pursue > a direction of using SRP to generate the keys for ESP, > and I asked whether there were problems > with the fact that such an approach would not permit > solutions that use an IPSec security gateway external > to an iSCSI implementation. > > While there were no answers in the meeting, > I've gotten some strong "Yes, there are problems" > responses off line, and between them and the fact that > there are a bunch of details to work out in exactly > how to use SRP to key ESP, I would propose > that the security requirements be just the two > bullets above (i.e., ESP with NULL encryption and > SRP are REQUIRED). This allows external gateways, > and keying of ESP with IKE or pre-shared keys, > and is consistent with the bulk of the discussion > in the meeting. > > Although the approach of using SRP to key ESP has > a lot of promise, making it a requirement in advance > of a draft providing details that can be checked > by other security experts seems premature ... and > now I have to go help get that draft written > in my "copious spare time" ;-). Once that draft > is in hand, we can make a concrete decision about > requiring that mechanism. > > Also, the integrity hash and signature algorithm > that MUST be implemented for ESP w/Null Encryption > still need to be designated -- in consultation with > the security area and security experts (e.g., Ted > Ts'o, ipsec WG co-chair, who was at the Nashua > meeting) the hope is to bring a recommendation to the > WG in the near future. A complicating factor is that > new hash algorithms are being introduced as a > consequence of the new AES/Rijndael cipher. Requiring > such a new algorithm (e.g., as opposed to the current > SHA-1 or MD5) was discussed as a desirable direction > in the meeting, but there are a bunch of details > that need to be checked (e.g., state of IETF use > and standardization of those algorithms). > > Comments to the list, especially if anyone disagrees > with the proposed requirements stated above. Specific > input from security-knowledgeable folks on algorithm > selection should probably be sent directly to me, as > the IPS list is not the best forum for that purpose. > > Thanks, > --David > > --------------------------------------------------- > David L. Black, Senior Technologist > EMC Corporation, 42 South St., Hopkinton, MA 01748 > +1 (508) 435-1000 x75140 FAX: +1 (508) 497-8500 > black_david@emc.com Mobile: +1 (978) 394-7754 > --------------------------------------------------- _________________________________________________________ Do You Yahoo!? Get your free @yahoo.com address at http://mail.yahoo.com From owner-ips@ece.cmu.edu Fri May 4 16:38:17 2001 Received: from ece.cmu.edu (ECE.CMU.EDU [128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id QAA20308 for ; Fri, 4 May 2001 16:38:16 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f44IP5Q13841 for ips-outgoing; Fri, 4 May 2001 14:25:05 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from msgbas1.cos.agilent.com (msgbas1x.cos.agilent.com [192.6.9.33]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f44IP3H13833 for ; Fri, 4 May 2001 14:25:03 -0400 (EDT) Received: from msgrel1.and.agilent.com (msgrel1.and.agilent.com [130.30.33.104]) by msgbas1.cos.agilent.com (Postfix) with ESMTP id D0EA649A for ; Fri, 4 May 2001 12:25:01 -0600 (MDT) Received: from rtl.rose.agilent.com (rtl.rose.agilent.com [156.140.232.231]) by msgrel1.and.agilent.com (Postfix) with ESMTP id B824810F for ; Fri, 4 May 2001 14:25:00 -0400 (EDT) Received: from mail.rose.agilent.com (bellhop [156.140.233.51]) by rtl.rose.agilent.com (8.9.3 (PHNE_18979)/8.9.3 SMKit7.1.0) with ESMTP id LAA06427 for ; Fri, 4 May 2001 11:24:59 -0700 (PDT) Received: from agilent.com (matt5670.rose.agilent.com [156.140.235.204]) by mail.rose.agilent.com (Netscape Messaging Server 3.6) with ESMTP id AAA3109 for ; Fri, 4 May 2001 11:24:56 -0700 Message-ID: <3AF2F3EE.998DAE24@agilent.com> Date: Fri, 04 May 2001 11:24:46 -0700 From: "Matt Wakeley" Reply-To: Matt Wakeley Organization: Agilent Technologies X-Mailer: Mozilla 4.77 [en] (Windows NT 5.0; U) X-Accept-Language: en MIME-Version: 1.0 To: ips@ece.cmu.edu Subject: Re: iscsi: comments to iSCSI rev 6 References: <3AF0232E.852EDB5E@cisco.com> <3AF08DB1.D1A48FF7@agilent.com> <3AF17047.89EB3B62@cisco.com> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-ips@ece.cmu.edu Precedence: bulk Content-Transfer-Encoding: 7bit Steve Senum wrote: > > Matt, > > Okay, but I still don't see the need. > > If one side initiates sending a key to the other side, > I might expect the following responses under > the current draft 6: > > 1. key returned with an option other other than 'none'. > > 2a. key not returned. > 2b. key returned with option 'none'. > > And you would add: > > 2c. key returned with option 'NotUnderstood'. > > To me, responses 2a, 2b, and 2c all mean the same, > in as far as the side to initiate sending the key must > make do without that key. From a negotiation point, > I don't see the value add in response 2c, or for that > matter, response 2b. The result may mean the same, but for diagnostic purposes, the only thing you know is you can't do it. You don't know why. If you know why, then perhaps you can take corrective action to remedy the problem. -Matt From owner-ips@ece.cmu.edu Fri May 4 16:45:07 2001 Received: from ece.cmu.edu (ECE.CMU.EDU [128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id QAA20400 for ; Fri, 4 May 2001 16:45:07 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f44Ivnk16925 for ips-outgoing; Fri, 4 May 2001 14:57:49 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from maho3msx2.isus.emc.com (maho3msx2.isus.emc.com [128.221.11.32]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f44IvmH16921 for ; Fri, 4 May 2001 14:57:48 -0400 (EDT) Received: by maho3msx2.isus.emc.com with Internet Mail Service (5.5.2650.21) id ; Fri, 4 May 2001 14:57:42 -0400 Message-ID: <0F31E5C394DAD311B60C00E029101A070801550D@corpmx9.isus.emc.com> From: Black_David@emc.com To: someshg@yahoo.com, ips@ece.cmu.edu Subject: RE: iSCSI Security rough consensus Date: Fri, 4 May 2001 14:57:41 -0400 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2650.21) Content-Type: text/plain Sender: owner-ips@ece.cmu.edu Precedence: bulk > Does this consensus mean that the iSCSI header and data CRCs > are no longer part of the specification, or are we > still requiring one or the other or both? Repeat after me: "CRCs are not security mechanisms" ;-) ;-), and see the previous email on this list about the consequences of WEP trying to use CRCs in this fashion. Yes, CRCs are still required for data integrity (e.g., when ESP is not present). If one knows that ESP with its keyed HMAC is being used in the stack between TCP and IP, then it would make sense not to use CRCs at the iSCSI level, hence they're required to implement, but configurable to use (which will also be the case for ESP). This may not always be possible, as one of the things mentioned in the meeting is that if the IPSec implementation is independent of iSCSI (e.g., supplied as part of the OS), there's no general standard way for iSCSI to figure out that IPSec is there or what it's doing to traffic on any particular iSCSI connection. Thanks, --David --------------------------------------------------- David L. Black, Senior Technologist EMC Corporation, 42 South St., Hopkinton, MA 01748 +1 (508) 435-1000 x75140 FAX: +1 (508) 497-8500 black_david@emc.com Mobile: +1 (978) 394-7754 --------------------------------------------------- From owner-ips@ece.cmu.edu Fri May 4 17:21:53 2001 Received: from ece.cmu.edu (ECE.CMU.EDU [128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id RAA20836 for ; Fri, 4 May 2001 17:21:53 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f44JcIG20776 for ips-outgoing; Fri, 4 May 2001 15:38:18 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from mxic1.isus.emc.com ([168.159.129.100]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f44JcGH20770 for ; Fri, 4 May 2001 15:38:16 -0400 (EDT) Received: by MXIC1 with Internet Mail Service (5.5.2650.21) id ; Fri, 4 May 2001 15:39:47 -0400 Message-ID: <0F31E5C394DAD311B60C00E029101A070801550F@corpmx9.isus.emc.com> From: Black_David@emc.com To: jtseng@NishanSystems.com, ips@ece.cmu.edu Subject: RE: iSCSI Security rough consensus Date: Fri, 4 May 2001 15:38:08 -0400 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2650.21) Content-Type: text/plain Sender: owner-ips@ece.cmu.edu Precedence: bulk I read this as violent agreement with what I posted. Thanks, --David > -----Original Message----- > From: Joshua Tseng [SMTP:jtseng@nishansystems.com] > Sent: Friday, May 04, 2001 3:30 PM > To: 'Black_David@emc.com'; ips@ece.cmu.edu > Subject: RE: iSCSI Security rough consensus > > See below: > > > > By comparison to full IPSec with IKE, using > > SRP to key ESP does not improve security. > > The underlying issue is IKE complexity (i.e., > > the code and effort required to implement it). > > > > Hence the rationale for using SRP to key > > ESP is that it provides dynamic key > > generation without implementing IKE -- this > > is an improvement over pre-shared keys at > > a much lower code and effort cost for a > > single-box (i.e., no external security gateway) > > implementation. > > What I think I'm hearing you say is that you > are evaluating whether to REQUIRE SRP keying of > ESP/IPSec because its easier to do than IKE. > If so, then in the first place, I don't think that > is an appropriate justification for a requirement. > In the second place, I'm not sure I even agree with > that statement--there are many off-the-shelf IKE > implementations which can be easily leveraged for > iSCSI with little or no modification. IKE doesn't > need to be conscious of the application (i.e., iSCSI) > being protected by IPSec. > > I also agree with Bernard that this issue is not > specific to iSCSI, and belongs in the security WG. > > Josh > > > > Thanks, > > --David > > > > --------------------------------------------------- > > David L. Black, Senior Technologist > > EMC Corporation, 42 South St., Hopkinton, MA 01748 > > +1 (508) 435-1000 x75140 FAX: +1 (508) 497-8500 > > black_david@emc.com Mobile: +1 (978) 394-7754 > > --------------------------------------------------- > > From owner-ips@ece.cmu.edu Fri May 4 17:22:35 2001 Received: from ece.cmu.edu (ECE.CMU.EDU [128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id RAA20875 for ; Fri, 4 May 2001 17:22:35 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f44JNLf19406 for ips-outgoing; Fri, 4 May 2001 15:23:21 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from maho3msx2.isus.emc.com (maho3msx2.isus.emc.com [128.221.11.32]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f44JNKH19402 for ; Fri, 4 May 2001 15:23:20 -0400 (EDT) Received: by maho3msx2.isus.emc.com with Internet Mail Service (5.5.2650.21) id ; Fri, 4 May 2001 15:23:14 -0400 Message-ID: <0F31E5C394DAD311B60C00E029101A070801550E@corpmx9.isus.emc.com> From: Black_David@emc.com To: someshg@yahoo.com, ips@ece.cmu.edu Subject: RE: iSCSI Security rough consensus Date: Fri, 4 May 2001 15:23:12 -0400 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2650.21) Content-Type: text/plain Sender: owner-ips@ece.cmu.edu Precedence: bulk > Sure would be nice if we could make up our minds and just > implement one mechanism. > > Here we have two mechanisms (iSCSI header/data integrity > and ESP) which are both mandatory to implement and > optional to use. Since ESP seems like a superset why not > just have that and get rid of the "integrity only" iSCSI > CRC mechanism. It sure would be nice, and in fact we had almost exactly this discussion later in the evening as part of the error recovery section of the agenda. The fly in the ointment is that the HMAC integrity algorithm that is at the core of ESP's integrity support is considerably more expensive (software or hardware) than a CRC, and this isn't likely to improve as I understand things. I would expect to see implementations with ESP completely in software and visible performance impacts. I really need to get the meeting minutes written up :-). Thanks, --David --------------------------------------------------- David L. Black, Senior Technologist EMC Corporation, 42 South St., Hopkinton, MA 01748 +1 (508) 435-1000 x75140 FAX: +1 (508) 497-8500 black_david@emc.com Mobile: +1 (978) 394-7754 --------------------------------------------------- From owner-ips@ece.cmu.edu Fri May 4 17:22:40 2001 Received: from ece.cmu.edu (ECE.CMU.EDU [128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id RAA20886 for ; Fri, 4 May 2001 17:22:40 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f44JTuO19934 for ips-outgoing; Fri, 4 May 2001 15:29:56 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from server1.NishanSystems.COM (smtp.nishansystems.com [216.217.36.162]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f44JTsH19928 for ; Fri, 4 May 2001 15:29:54 -0400 (EDT) Received: by smtp.nishansystems.com with Internet Mail Service (5.5.2653.19) id ; Fri, 4 May 2001 12:29:48 -0700 Message-ID: From: Joshua Tseng To: "'Black_David@emc.com'" , ips@ece.cmu.edu Subject: RE: iSCSI Security rough consensus Date: Fri, 4 May 2001 12:29:47 -0700 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2653.19) Content-Type: text/plain; charset="iso-8859-1" Sender: owner-ips@ece.cmu.edu Precedence: bulk See below: > > By comparison to full IPSec with IKE, using > SRP to key ESP does not improve security. > The underlying issue is IKE complexity (i.e., > the code and effort required to implement it). > > Hence the rationale for using SRP to key > ESP is that it provides dynamic key > generation without implementing IKE -- this > is an improvement over pre-shared keys at > a much lower code and effort cost for a > single-box (i.e., no external security gateway) > implementation. What I think I'm hearing you say is that you are evaluating whether to REQUIRE SRP keying of ESP/IPSec because its easier to do than IKE. If so, then in the first place, I don't think that is an appropriate justification for a requirement. In the second place, I'm not sure I even agree with that statement--there are many off-the-shelf IKE implementations which can be easily leveraged for iSCSI with little or no modification. IKE doesn't need to be conscious of the application (i.e., iSCSI) being protected by IPSec. I also agree with Bernard that this issue is not specific to iSCSI, and belongs in the security WG. Josh > > Thanks, > --David > > --------------------------------------------------- > David L. Black, Senior Technologist > EMC Corporation, 42 South St., Hopkinton, MA 01748 > +1 (508) 435-1000 x75140 FAX: +1 (508) 497-8500 > black_david@emc.com Mobile: +1 (978) 394-7754 > --------------------------------------------------- > From owner-ips@ece.cmu.edu Fri May 4 17:23:20 2001 Received: from ece.cmu.edu (ECE.CMU.EDU [128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id RAA20898 for ; Fri, 4 May 2001 17:23:19 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f44Jjrx21456 for ips-outgoing; Fri, 4 May 2001 15:45:53 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from dogwood.cisco.com (dogwood.cisco.com [161.44.11.19]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f44JjpH21451 for ; Fri, 4 May 2001 15:45:51 -0400 (EDT) Received: from cisco.com (dhcp-161-44-68-173.cisco.com [161.44.68.173]) by dogwood.cisco.com (8.8.6 (PHNE_14041)/CISCO.SERVER.1.2) with ESMTP id PAA18627 for ; Fri, 4 May 2001 15:45:45 -0400 (EDT) Message-ID: <3AF30678.277C7400@cisco.com> Date: Fri, 04 May 2001 14:43:52 -0500 From: Steve Senum X-Mailer: Mozilla 4.73 [en]C-CCK-MCD (Windows NT 5.0; U) X-Accept-Language: en MIME-Version: 1.0 To: ips@ece.cmu.edu Subject: Re: iscsi: comments to iSCSI rev 6 References: <3AF0232E.852EDB5E@cisco.com> <3AF08DB1.D1A48FF7@agilent.com> <3AF17047.89EB3B62@cisco.com> <3AF2F3EE.998DAE24@agilent.com> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-ips@ece.cmu.edu Precedence: bulk Content-Transfer-Encoding: 7bit Matt, > The result may mean the same, but for diagnostic purposes, the only thing you > know is you can't do it. You don't know why. If you know why, then perhaps > you can take corrective action to remedy the problem. To help with this issue, I would prefer key=none be required to sent, if applicable. That would leave a non response to mean only not supported. Steve Senum From owner-ips@ece.cmu.edu Fri May 4 17:28:00 2001 Received: from ece.cmu.edu (ECE.CMU.EDU [128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id RAA21002 for ; Fri, 4 May 2001 17:28:00 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f44K2ZJ22941 for ips-outgoing; Fri, 4 May 2001 16:02:35 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from palrel1.hp.com (palrel1.hp.com [156.153.255.242]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f44K2WH22932 for ; Fri, 4 May 2001 16:02:32 -0400 (EDT) Received: from hpcuhe.cup.hp.com (hpcuhe.cup.hp.com [15.0.80.203]) by palrel1.hp.com (Postfix) with ESMTP id 0F06C135B; Fri, 4 May 2001 13:02:31 -0700 (PDT) Received: (from santoshr@localhost) by hpcuhe.cup.hp.com (8.9.3 (PHNE_18979)/8.9.3 SMKit7.02) id LAA25434; Fri, 4 May 2001 11:15:09 -0700 (PDT) From: Santosh Rao Message-Id: <200105041815.LAA25434@hpcuhe.cup.hp.com> Subject: RE: iSCSI : login keys & mode page settings To: Black_David@emc.com (David Black), ips@ece.cmu.edu (ips) Date: Fri, 4 May 2001 11:15:08 -0700 (PDT) X-Mailer: ELM [version 2.5 PL2] MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-ips@ece.cmu.edu Precedence: bulk Content-Transfer-Encoding: 7bit David, Apologies for the late response on this. I was hoping we could complete this thread of discussion at Nashua, but for lack of time, we are back on the list. Regarding your question below : > If one Initiator can damage another in this fashion, then we > may indeed have a problem. > Comments?, Shared mode page implementations in targets is quite common and modification of control parameters through a mode select would indeed affect all other initiators logged into the target. This is not desirable behaviour and iSCSI may be better served using login/text based key negotiation rather than the mode select mechanisms which opens it up to the side effects of affecting all connected initiators. Thanks, Santosh ------------------------------------------------------------------------ Subject: RE: iSCSI : login keys & mode page settings From: Black_David@emc.com Date: Fri, 20 Apr 2001 20:32:17 -0400 Cc: ips@ece.cmu.edu I'm not sure -- this sounds somewhat like the old principle of not asking why there's a hole in one's foot when one has aimed the gun at it and pulled the trigger. For the tape example, if some tape driver changes a Target iSCSI parameter that disrupts that driver's own tape I/O in a fashion that the driver can't recover from, I think it's clear where the fault lies. If one Initiator can damage another in this fashion, then we may indeed have a problem. Comments?, --David > -----Original Message----- > From: Santosh Rao [SMTP:santoshr@cup.hp.com] > Sent: Friday, April 20, 2001 8:09 PM > To: Black_David@emc.com > Cc: ips@ece.cmu.edu > Subject: Re: iSCSI : login keys & mode page settings > > David, > > Some clarification on the basis for classifying login ould > also be helpful. Should login keys that can disrupt > I/O on their change > be allowed to be non-LO ? > > Thanks, > Santosh > > Black_David@emc.com wrote: > > > > Without getting into the technical details of the > > discussion, I have a couple of observations: > > > > (A) The issue of whether to allow mode page > > access to and modification of iSCSI ers > > will need to be taken up at the interim > > meeting. IMHO, access seems like a good > > idea, so that SCSI-generic code that doesn't > > know specifically about iSCSI can find > > what it expects where it expects it, but > > I'm unsure about modification because it > > may carry a risk of code that's naware > > getting something wrong. The mode page > > commands should be transparent to iSCSI. > > > > (B) The mode page and text key mechanisms have > > to access the same data. Section 3 of the > > -06 version says this, but needs some > > editing > > to enforce it by using "MUST" or its > > equivalent > > (cf. RFC 2119). This is to prevent an > > implementation from having two instances of > > the same parameter - one for the mode page and > > one for the text keys - which would be a bad > > thing. > > > > --David -- ################################# Santosh Rao Software Design Engineer, HP, Cupertino. email : santoshr@cup.hp.com Phone : 408-447-3751 ################################# From owner-ips@ece.cmu.edu Fri May 4 19:11:48 2001 Received: from ece.cmu.edu (ECE.CMU.EDU [128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id TAA22117 for ; Fri, 4 May 2001 19:11:47 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f44LFRM29297 for ips-outgoing; Fri, 4 May 2001 17:15:27 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from dogwood.cisco.com (dogwood.cisco.com [161.44.11.19]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f44LFNH29283 for ; Fri, 4 May 2001 17:15:23 -0400 (EDT) Received: from cisco.com (mbakke@mbakke-lnx.cisco.com [161.44.68.87]) by dogwood.cisco.com (8.8.6 (PHNE_14041)/CISCO.SERVER.1.2) with ESMTP id RAA15663 for ; Fri, 4 May 2001 17:15:17 -0400 (EDT) Message-ID: <3AF31B65.18A0F8D@cisco.com> Date: Fri, 04 May 2001 16:13:09 -0500 From: Mark Bakke X-Mailer: Mozilla 4.72 [en] (X11; U; Linux 2.2.16-3.uid32 i686) X-Accept-Language: en, de MIME-Version: 1.0 To: IPS Subject: iSCSI: Nailing down CRC-32C Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-ips@ece.cmu.edu Precedence: bulk Content-Transfer-Encoding: 7bit At the interim meeting, it was stated that iSCSI has selected the CRC-32C polynomial as its required iSCSI-level header and data CRC. Now that we have it, it's time to move on and make sure we can implement it. In the interest of interoperability, we need to not only specify the polynomial, but also the initial values, bit and byte ordering, etc. "A Painless Guide to CRC Error Detection Algorithms" (http://www.ross.net/crc/crcpaper.html) specifies a method to unambiguously characterize these parameters (sections 15 and 16). Has anyone taken a shot at defining these yet? Otherwise, here is what it might look like: Name : "CRC-32C" Width : 32 Poly : 1EDC6F41 (note that the leading "1" is implied) Init : FFFFFFFF RefIn : True RefOut : True XorOut : FFFFFFFF Check : ? I haven't attempted to create check data based on these yet. As soon as the other parameters are nailed down, we need to do this. Anyway, I am not a CRC expert, and can't make any statement about the above values being the "best" way to do this, but instead just copied them (except the polynomial itself) from the Ethernet CRC, since that is likely the easiest for everyone implementing hardware to deal with. If someone else is already doing this, let me know; I just wanted to start this thread so we can get closure. Regards, Mark -- Mark A. Bakke Cisco Systems mbakke@cisco.com 763.398.1054 From owner-ips@ece.cmu.edu Fri May 4 19:13:33 2001 Received: from ece.cmu.edu (ECE.CMU.EDU [128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id TAA22143 for ; Fri, 4 May 2001 19:13:33 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f44LTRO00461 for ips-outgoing; Fri, 4 May 2001 17:29:27 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from dogwood.cisco.com (dogwood.cisco.com [161.44.11.19]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f44LTQH00455 for ; Fri, 4 May 2001 17:29:26 -0400 (EDT) Received: from cisco.com (mbakke@mbakke-lnx.cisco.com [161.44.68.87]) by dogwood.cisco.com (8.8.6 (PHNE_14041)/CISCO.SERVER.1.2) with ESMTP id RAA01774 for ; Fri, 4 May 2001 17:29:20 -0400 (EDT) Message-ID: <3AF31EB0.D55813F8@cisco.com> Date: Fri, 04 May 2001 16:27:12 -0500 From: Mark Bakke X-Mailer: Mozilla 4.72 [en] (X11; U; Linux 2.2.16-3.uid32 i686) X-Accept-Language: en, de MIME-Version: 1.0 To: IPS Subject: Re: iSCSI: Nailing down CRC-32C References: <3AF31B65.18A0F8D@cisco.com> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-ips@ece.cmu.edu Precedence: bulk Content-Transfer-Encoding: 7bit BTW, I just ran the algorithm from the paper with these parameters. If they are specified correctly, the CRC of the string "123456789" should be 0xe3069283. Note that's without padding, which we have to unambiguously specify as well. -- Mark Mark Bakke wrote: > > At the interim meeting, it was stated that iSCSI has selected > the CRC-32C polynomial as its required iSCSI-level header and > data CRC. Now that we have it, it's time to move on and make > sure we can implement it. > > In the interest of interoperability, we need to not only specify > the polynomial, but also the initial values, bit and byte > ordering, etc. > > "A Painless Guide to CRC Error Detection Algorithms" > (http://www.ross.net/crc/crcpaper.html) specifies a > method to unambiguously characterize these parameters > (sections 15 and 16). Has anyone taken a shot at defining > these yet? Otherwise, here is what it might look like: > > Name : "CRC-32C" > Width : 32 > Poly : 1EDC6F41 (note that the leading "1" is implied) > Init : FFFFFFFF > RefIn : True > RefOut : True > XorOut : FFFFFFFF > Check : ? > > I haven't attempted to create check data based on these yet. As > soon as the other parameters are nailed down, we need to do this. > > Anyway, I am not a CRC expert, and can't make any statement about > the above values being the "best" way to do this, but instead just > copied them (except the polynomial itself) from the Ethernet CRC, > since that is likely the easiest for everyone implementing hardware > to deal with. > > If someone else is already doing this, let me know; I just wanted > to start this thread so we can get closure. > > Regards, > > Mark > > -- > Mark A. Bakke > Cisco Systems > mbakke@cisco.com > 763.398.1054 -- Mark A. Bakke Cisco Systems mbakke@cisco.com 763.398.1054 From owner-ips@ece.cmu.edu Fri May 4 19:14:06 2001 Received: from ece.cmu.edu (ECE.CMU.EDU [128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id TAA22182 for ; Fri, 4 May 2001 19:14:06 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f44K5s423181 for ips-outgoing; Fri, 4 May 2001 16:05:54 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from dogwood.cisco.com (dogwood.cisco.com [161.44.11.19]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f44K5qH23177 for ; Fri, 4 May 2001 16:05:52 -0400 (EDT) Received: from cisco.com (mbakke@mbakke-lnx.cisco.com [161.44.68.87]) by dogwood.cisco.com (8.8.6 (PHNE_14041)/CISCO.SERVER.1.2) with ESMTP id QAA18405; Fri, 4 May 2001 16:05:40 -0400 (EDT) Message-ID: <3AF30B13.670ACC3D@cisco.com> Date: Fri, 04 May 2001 15:03:31 -0500 From: Mark Bakke X-Mailer: Mozilla 4.72 [en] (X11; U; Linux 2.2.16-3.uid32 i686) X-Accept-Language: en, de MIME-Version: 1.0 To: Black_David@emc.com CC: someshg@yahoo.com, ips@ece.cmu.edu Subject: Re: iSCSI Security rough consensus References: <0F31E5C394DAD311B60C00E029101A070801550E@corpmx9.isus.emc.com> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-ips@ece.cmu.edu Precedence: bulk Content-Transfer-Encoding: 7bit Black_David@emc.com wrote: > > > Sure would be nice if we could make up our minds and just > > implement one mechanism. > > > > Here we have two mechanisms (iSCSI header/data integrity > > and ESP) which are both mandatory to implement and > > optional to use. Since ESP seems like a superset why not > > just have that and get rid of the "integrity only" iSCSI > > CRC mechanism. > > It sure would be nice, and in fact we had almost > exactly this discussion later in the evening as > part of the error recovery section of the agenda. > The fly in the ointment is that the HMAC integrity > algorithm that is at the core of ESP's integrity > support is considerably more expensive (software > or hardware) than a CRC, and this isn't likely > to improve as I understand things. I would expect > to see implementations with ESP completely in > software and visible performance impacts. That's just part of the reason behind having both. The other is that most implementations won't run IPsec end-to-end; either IPsec is provided in an external device, or even in an iSCSI gateway. In the latter case, all layers are removed and replaced, including iSCSI. Only the SCSI-level information (data, CDBs) go all the way end-to-end. Since iSCSI can CRC the SCSI-level data, it can provide the data integrity that will keep our customers happy. The use of the iSCSI CRC is the minimum requirement; adding the IPsec-level integrity check strengthens it, and can simplify error recovery over a not-so-good or untrusted network. -- Mark > > I really need to get the meeting minutes written up :-). > > Thanks, > --David > > --------------------------------------------------- > David L. Black, Senior Technologist > EMC Corporation, 42 South St., Hopkinton, MA 01748 > +1 (508) 435-1000 x75140 FAX: +1 (508) 497-8500 > black_david@emc.com Mobile: +1 (978) 394-7754 > --------------------------------------------------- -- Mark A. Bakke Cisco Systems mbakke@cisco.com 763.398.1054 From owner-ips@ece.cmu.edu Fri May 4 20:43:43 2001 Received: from ece.cmu.edu (ECE.CMU.EDU [128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id UAA22880 for ; Fri, 4 May 2001 20:43:43 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f44MqgF06519 for ips-outgoing; Fri, 4 May 2001 18:52:42 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from internaut.com ([64.38.134.99]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f44MqeH06514 for ; Fri, 4 May 2001 18:52:40 -0400 (EDT) Received: from localhost (aboba@localhost) by internaut.com (8.9.3/8.9.3) with ESMTP id PAA03351; Fri, 4 May 2001 15:44:46 -0700 (PDT) (envelope-from aboba@internaut.com) Date: Fri, 4 May 2001 15:44:46 -0700 (PDT) From: Bernard Aboba To: Somesh Gupta cc: Black_David@emc.com, ips@ece.cmu.edu Subject: RE: iSCSI Security rough consensus In-Reply-To: Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-ips@ece.cmu.edu Precedence: bulk > > Yes, CRCs are still required for data integrity (e.g., > > when ESP is not present). If one knows that ESP with > > its keyed HMAC is being used in the stack between TCP and > > IP, then it would make sense not to use CRCs at the iSCSI > > level, hence they're required to implement, but configurable > > to use (which will also be the case for ESP). The issue here is where the data invalidation is coming from. If it is occuring on the wire, then IPSEC will solve the problem and a CRC (or a TCP checksum for that matter) is redundant. However, if it is occuring later, then IPSEC may not be an appropriate solution, particularly since acceleration technologies mean that IPSEC may be stripped off even before the packet is moved off the interface for the first time. Ultimately the decision should rest on consideration of the empirical evidence of where the problem is. From owner-ips@ece.cmu.edu Fri May 4 20:49:20 2001 Received: from ece.cmu.edu (ECE.CMU.EDU [128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id UAA22953 for ; Fri, 4 May 2001 20:49:19 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f44NVvx09284 for ips-outgoing; Fri, 4 May 2001 19:31:57 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from internaut.com ([64.38.134.99]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f44NVtH09279 for ; Fri, 4 May 2001 19:31:55 -0400 (EDT) Received: from localhost (aboba@localhost) by internaut.com (8.9.3/8.9.3) with ESMTP id QAA03387; Fri, 4 May 2001 16:24:06 -0700 (PDT) (envelope-from aboba@internaut.com) Date: Fri, 4 May 2001 16:24:06 -0700 (PDT) From: Bernard Aboba To: Black_David@emc.com cc: ips@ece.cmu.edu Subject: RE: iSCSI Security rough consensus In-Reply-To: <0F31E5C394DAD311B60C00E029101A070801550C@corpmx9.isus.emc.com> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-ips@ece.cmu.edu Precedence: bulk > I think this is a case of yet another group of people > looking at IKE and developing indigestion. I would > support Bernard's comment above that this ought to be > done once in a general fashion that's useful to other > applications provided that it can actually get done > without pulling back in most of IKE's functionality > as "nice to have" -- that's "son of IKE"'s job :-). > Judging by behavior, the symptoms of "IKE indigestion" seem to be spreading. These include both a yearning for a replacement to IKE (e.g. KINK, SRP, etc.) as well as desires for additions/changes to IKE, primarily to support legacy authentication (XAUTH, CRACK, Hybrid). Originally, the symptoms were confined to the IPSRA WG, but now they have appeared in IPS WG. If this is a true epidemic, I'd expect the emergency rooms to start filling up around now. Unfortunately, "son of IKE" is not really going to be a more functional IKE, but a "better written IKE specification with some of the unused stuff taken out." So in terms of functionality ommissions, the "son" will inherent the sins of the father. From owner-ips@ece.cmu.edu Fri May 4 20:49:33 2001 Received: from ece.cmu.edu (ECE.CMU.EDU [128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id UAA22964 for ; Fri, 4 May 2001 20:49:33 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f44N1Vh07147 for ips-outgoing; Fri, 4 May 2001 19:01:32 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from internaut.com ([64.38.134.99]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f44N1TH07141 for ; Fri, 4 May 2001 19:01:30 -0400 (EDT) Received: from localhost (aboba@localhost) by internaut.com (8.9.3/8.9.3) with ESMTP id PAA03361; Fri, 4 May 2001 15:53:35 -0700 (PDT) (envelope-from aboba@internaut.com) Date: Fri, 4 May 2001 15:53:35 -0700 (PDT) From: Bernard Aboba To: Joshua Tseng cc: "'Black_David@emc.com'" , ips@ece.cmu.edu Subject: RE: iSCSI Security rough consensus In-Reply-To: Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-ips@ece.cmu.edu Precedence: bulk > > By comparison to full IPSec with IKE, using > > SRP to key ESP does not improve security. Actually, if the comparison is SRP vs. IKE using shared keys, that's not really true. IKE Shared Key auth is susceptible to man-in-the-middle attack in that in Main Mode unless the IP addresses of the correspondents are fixed, there is no way to tie an IP address to an appropriate shared key. In practice this means the shared group keys must be used. Using the same shared group key to protect iSCSI for thousands of initiators lacks credibility, because anyone with the group key (e.g. anyone in the entire org) can impersonate anyone else. Thus for IKE use in iSCSI, it would seem that only cert-based auth is tenable. In the most recent survey data I've seen, less than 15 percent of enterprises have any plans to deploy certificates. So unless you've got a credible transition solution (e.g. GetCert, PIC, etc.) it'll be a hard sell. On the other hand, with SRP, it is possible to identify the endpoints prior to authentication a la aggressive mode, and thus to maintain separate passwords for each initiator-target pair. SRP is resistent to dictionary attacks or compromise of the password database. > What I think I'm hearing you say is that you > are evaluating whether to REQUIRE SRP keying of > ESP/IPSec because its easier to do than IKE. Ease of implementation is *not* the only issue. There is a functionality issue as well. If you need shared key authentication for hosts with dynamic IP addresses, IKE Main Mode is not a credible solution. From owner-ips@ece.cmu.edu Fri May 4 21:57:56 2001 Received: from ece.cmu.edu (ECE.CMU.EDU [128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id VAA24528 for ; Fri, 4 May 2001 21:57:56 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f4507vc11790 for ips-outgoing; Fri, 4 May 2001 20:07:57 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from server1.NishanSystems.COM (smtp.nishansystems.com [216.217.36.162]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f4507uH11786 for ; Fri, 4 May 2001 20:07:56 -0400 (EDT) Received: by smtp.nishansystems.com with Internet Mail Service (5.5.2653.19) id ; Fri, 4 May 2001 17:07:49 -0700 Message-ID: From: Joshua Tseng To: "'Bernard Aboba'" Cc: "'Black_David@emc.com'" , ips@ece.cmu.edu Subject: RE: iSCSI Security rough consensus Date: Fri, 4 May 2001 17:07:48 -0700 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2653.19) Content-Type: text/plain; charset="iso-8859-1" Sender: owner-ips@ece.cmu.edu Precedence: bulk See below: > > > > By comparison to full IPSec with IKE, using > > > SRP to key ESP does not improve security. > > Actually, if the comparison is SRP vs. IKE using shared keys, > that's not > really true. IKE Shared Key auth is susceptible to man-in-the-middle > attack in that in Main Mode unless the IP addresses of the > correspondents > are fixed, there is no way to tie an IP address to an > appropriate shared > key. In practice this means the shared group keys must be > used. Using the > same shared group key to protect iSCSI for thousands of > initiators lacks > credibility, because anyone with the group key (e.g. anyone > in the entire > org) can impersonate anyone else. Thus for IKE use in iSCSI, > it would seem > that only cert-based auth is tenable. In the most recent > survey data I've > seen, less than 15 percent of enterprises have any plans to deploy > certificates. So unless you've got a credible transition solution > (e.g. GetCert, PIC, etc.) it'll be a hard sell. ...perhaps only 15% of enterprises ARE concerned about security. I don't know...just wondering.... I don't know if this is that hard of a sell, since there are already many available products that do cert-based IKE authentication. The availability of certificate-based products and infrastructure is NOT a barrier. > > On the other hand, with SRP, it is possible to identify the endpoints > prior to authentication a la aggressive mode, and thus to maintain > separate passwords for each initiator-target pair. SRP is resistent to > dictionary attacks or compromise of the password database. > > > What I think I'm hearing you say is that you > > are evaluating whether to REQUIRE SRP keying of > > ESP/IPSec because its easier to do than IKE. > > Ease of implementation is *not* the only issue. There is a > functionality > issue as well. If you need shared key authentication for hosts with > dynamic IP addresses, IKE Main Mode is not a credible solution. For authentication of hosts with dynamic IP addresses, I could use IKE with cert-based FQDN (or iSCSI Name) authentication. That is just as viable as SRP keying of ESP/IPSec. Or if security is that important to me, I wouldn't use dynamic IP addresses. All I am saying is that SRP keying of ESP isn't the only choice. That's why it shouldn't be REQUIRED. Josh > > From owner-ips@ece.cmu.edu Fri May 4 22:00:20 2001 Received: from ece.cmu.edu (ECE.CMU.EDU [128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id WAA24592 for ; Fri, 4 May 2001 22:00:20 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f450doi13906 for ips-outgoing; Fri, 4 May 2001 20:39:50 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from atlrel1.hp.com (atlrel1.hp.com [156.153.255.210]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f450dnH13902 for ; Fri, 4 May 2001 20:39:49 -0400 (EDT) Received: from xatlrelay1.atl.hp.com (xatlrelay1.atl.hp.com [15.45.89.190]) by atlrel1.hp.com (Postfix) with ESMTP id DD1D4220; Fri, 4 May 2001 20:39:48 -0400 (EDT) Received: from xpabh4.corp.hp.com (xpabh4.corp.hp.com [15.58.136.1]) by xatlrelay1.atl.hp.com (Postfix) with ESMTP id D38851F511; Fri, 4 May 2001 20:37:41 -0400 (EDT) Received: by xpabh4.corp.hp.com with Internet Mail Service (5.5.2653.19) id ; Fri, 4 May 2001 17:39:39 -0700 Message-ID: <6BD67FFB937FD411A04F00D0B74FE87802A09038@xrose06.rose.hp.com> From: "KRUEGER,MARJORIE (HP-Roseville,ex1)" To: "'Matt Wakeley'" , ips@ece.cmu.edu Subject: RE: iSCSI: multiple sessions b/n a pair of WWUIs. Date: Fri, 4 May 2001 17:39:35 -0700 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2653.19) Content-Type: text/plain; charset="iso-8859-1" Sender: owner-ips@ece.cmu.edu Precedence: bulk > One clarification: these are unique between any I-T *pair*. > (you make it sound like an initiator must have a unique ISID > for all sessions > with all targets) > > -Matt The ISID must be unique within the iSCSI layer on an iSCSI device - that amounts to "initiator must have unique ISID for all sessions with all targets". To quote Jim "The requirement in name+disc that a given initiator name cannot reuse an ISID for two different sessions comes as a consequence a number of things (which are described in the draft). The gist is that this is needed to provide the correct context for restoration of reservations state (and other nexus state) to a particular nexus after logout/login. In other words, if the session goes down for some reason, the target needs clear context to restore nexus state to a rebuilt session. The only tool it has is uniqueness of Name+ISID combination within its name space." -Marj From owner-ips@ece.cmu.edu Fri May 4 23:20:40 2001 Received: from ece.cmu.edu (ECE.CMU.EDU [128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id XAA26126 for ; Fri, 4 May 2001 23:20:39 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f451UKq17367 for ips-outgoing; Fri, 4 May 2001 21:30:20 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from think.thunk.org (THINK.THUNK.ORG [216.175.175.162]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f451UIH17362 for ; Fri, 4 May 2001 21:30:18 -0400 (EDT) Received: from tytso by think.thunk.org with local (Exim 3.22 #1 (Debian)) id 14vqor-0002h0-00; Fri, 04 May 2001 21:25:13 -0400 Date: Fri, 4 May 2001 21:25:12 -0400 From: Theodore Tso To: Bernard Aboba Cc: Black_David@emc.com, ips@ece.cmu.edu Subject: Re: iSCSI Security rough consensus Message-ID: <20010504212512.A10115@think.thunk.org> References: <0F31E5C394DAD311B60C00E029101A07080154F9@corpmx9.isus.emc.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.3.15i In-Reply-To: ; from aboba@internaut.com on Fri, May 04, 2001 at 09:06:23AM -0700 Sender: owner-ips@ece.cmu.edu Precedence: bulk On Fri, May 04, 2001 at 09:06:23AM -0700, Bernard Aboba wrote: > > Also, given that another key exchange method is likely to be used (IKE, > KINK), this implies that another authentication has already taken place to > set up the IPSEC SA. Thus, requiring SRP in addition to IKE/KINK adds *yet > another* authentication and key exchange to the process of setting up an > IPSEC SA. Are we doing this purely in order to circumvent IKE's > limitations with respect to password-based authentication? The basic problem here seems to be that people don't want to implement IKE. Going into the interim meeting, what a lot of vendors had been planning on doing was apparently to *not* to include IPSEC in the basic storage devices, but to depend on outboard IPSEC gateways to actually do the IPSEC work. The iSCSI specs was going to have weasel words which would permit this, but say that only the entire system of storage devices plus off-the-shelf VPN box would be compliant with iSCSI. Of course, the number of sites that would actually deploy full iSCSI would probably be quite small; most customers would probably just not purchase the VPN box, and depend on the hard-on-the-outside, soft-and-chewing-on-the inside security model. And of course, the vendors would employ their marketing people to make sure that the full "iSCSI complaint" version would be included in the product catalog, but to also always make available the option which didn't include this 3rd party IPSEC gateway box. Guess which version of the product would actually see deployment in customer sites? For this reason, the IPS working group wanted to use a two levels of protection; an "outer layer" that would be IPSEC, but which might not necessarily be end-to-end (and in fact, in actual real life, probably wouldn't be present at all), and then an inner layer which used something like SRP. I don't know if something like this would have gotten past the IESG -- my guess is that if it did, it would be with the Security Area AD's hollering and screaming a lot. So it was given this particular set of constraints, where people seemed determined to (a) not use (or at least not require) IPSEC in an end-to-end mode, and (b) to use something like SRP, that I suggested using the session keying material from SRP to key ESP as an improvement. If this meant that the IPSEC protection could actually be end-to-end, and that it would more likely actually end up in shipping hardware that customers actually used, as opposed to an unwieldy, theoretical hack that was never meant for customer use but only to get around the IETF standards process, then using SRP as the source of keying information for ESP was a vast improvement from where we started on Tuesday morning. Would it be better if we were actually doing end-to-end IKE, and asking the disk drive manufacturers to solve the public key management problems that this would entail? From a security perspective, sure. But it's not too clear how realistic such a stance would really be. - Ted P.S. Yes, I know that in fact there will need to be some specifications written to indicate how to get from the SRP or CHAP keying information to the low-level details of ESP, probably using an AES cipher suite for speed. The hope was to keep this part small and simple enough that that it wouldn't actually be an explicit key management layer ala IKE and KINK, although of course it would have to perform at least a little of such functions. From owner-ips@ece.cmu.edu Sat May 5 00:47:10 2001 Received: from ece.cmu.edu (ECE.CMU.EDU [128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id AAA27119 for ; Sat, 5 May 2001 00:47:10 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f4536CV23328 for ips-outgoing; Fri, 4 May 2001 23:06:12 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from server1.NishanSystems.COM (smtp.nishansystems.com [216.217.36.162]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f4536AH23323 for ; Fri, 4 May 2001 23:06:10 -0400 (EDT) Received: by smtp.nishansystems.com with Internet Mail Service (5.5.2653.19) id ; Fri, 4 May 2001 20:05:59 -0700 Message-ID: From: Joshua Tseng To: "'Theodore Tso'" Cc: Black_David@emc.com, ips@ece.cmu.edu Subject: RE: iSCSI Security rough consensus Date: Fri, 4 May 2001 20:05:57 -0700 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2653.19) Content-Type: text/plain; charset="iso-8859-1" Sender: owner-ips@ece.cmu.edu Precedence: bulk Ted, See below: > > And of course, the vendors would employ their marketing people to make > sure that the full "iSCSI complaint" version would be included in the > product catalog, but to also always make available the option which > didn't include this 3rd party IPSEC gateway box. Guess which version > of the product would actually see deployment in customer sites? > > For this reason, the IPS working group wanted to use a two levels of > protection; an "outer layer" that would be IPSEC, but which might not > necessarily be end-to-end (and in fact, in actual real life, probably > wouldn't be present at all), and then an inner layer which used > something like SRP. > > I don't know if something like this would have gotten past the IESG -- > my guess is that if it did, it would be with the Security Area AD's > hollering and screaming a lot. I don't know the Security AD's, but if they had this reaction as you describe, my question to them would be why did they invent tunnel mode and security gateways in the first place? If they wanted all protocols to have end-to-end protection as a REQUIREMENT, shouldn't they have just stopped with transport mode? That way, it's end-to-end IPSec or else use something like TLS. > > > So it was given this particular set of constraints, where people > seemed determined to (a) not use (or at least not require) IPSEC in an > end-to-end mode, and (b) to use something like SRP, that I suggested > using the session keying material from SRP to key ESP as an > improvement. If this meant that the IPSEC protection could actually > be end-to-end, and that it would more likely actually end up in > shipping hardware that customers actually used, as opposed to an > unwieldy, theoretical hack that was never meant for customer use but > only to get around the IETF standards process, then using SRP as the > source of keying information for ESP was a vast improvement from where > we started on Tuesday morning. The description of security gateways in RFC 2401 sure doesn't sound like an "unwieldy, theoretical hack". In fact, the section 3.3 clearly says they are an option: 3.3 Where IPsec May Be Implemented There are several ways in which IPsec may be implemented in a host or in conjunction with a router or firewall (to create a security gateway). Several common examples are provided below: There are many other places where 2401 talks in great detail about security gateways and tunnel mode. > > > Would it be better if we were actually doing end-to-end IKE, and > asking the disk drive manufacturers to solve the public key management > problems that this would entail? From a security perspective, sure. > But it's not too clear how realistic such a stance would really be. From an implementation perspective, the answer is yes. I see it far more realistic for implementers to take an existing off-the-shelf IKE implementation and plug it into their disk drive, than to tweak their iSCSI and IPSec implementations to get SRP to key IPSec. The former can be done with few or no modifications, since IKE is most likely already integrated with IPSec. Neither IPSec nor IKE need to be conscious of the application (i.e., iSCSI) that it is protecting. Regards, Josh > > - Ted > > P.S. Yes, I know that in fact there will need to be some > specifications written to indicate how to get from the SRP or CHAP > keying information to the low-level details of ESP, probably using an > AES cipher suite for speed. The hope was to keep this part small and > simple enough that that it wouldn't actually be an explicit key > management layer ala IKE and KINK, although of course it would have to > perform at least a little of such functions. > From owner-ips@ece.cmu.edu Sat May 5 11:54:34 2001 Received: from ece.cmu.edu (ECE.CMU.EDU [128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id LAA16290 for ; Sat, 5 May 2001 11:54:33 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f45DubU09411 for ips-outgoing; Sat, 5 May 2001 09:56:37 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from crufty.research.bell-labs.com (crufty.research.bell-labs.com [204.178.16.49]) by ece.cmu.edu (8.11.0/8.10.2) with SMTP id f45DuZH09405 for ; Sat, 5 May 2001 09:56:35 -0400 (EDT) Received: from grubby.research.bell-labs.com ([135.104.2.9]) by crufty; Sat May 5 09:52:38 EDT 2001 Received: from aura.research.bell-labs.com ([135.104.46.10]) by grubby; Sat May 5 09:55:13 EDT 2001 Received: (from sandeepj@localhost) by aura.research.bell-labs.com (8.9.1/8.9.1) id JAA22113 for ips@ece.cmu.edu; Sat, 5 May 2001 09:55:06 -0400 (EDT) Date: Sat, 5 May 2001 09:55:06 -0400 (EDT) Message-Id: <200105051355.JAA22113@aura.research.bell-labs.com> From: sandeepj@research.bell-labs.com (Sandeep Joshi) To: ips@ece.cmu.edu Subject: iSCSI: immediate data Sender: owner-ips@ece.cmu.edu Precedence: bulk Julian, I had a follow-up question on an old thread. http://ips.pdl.cs.cmu.edu/mail/msg03373.html The initiator is allowed to send firstburst in immediate or in separate PDUs...but can it do neither ? I dont see any statement to the effect that it MUST send a firstburst. Here's a possible problem : 1) FirstBurst is 4K 2) Expected data length of command is 16K 3) Initiator sends the Scsi command without immediate data. (final flag on Scsi command is not set) And it decides to send no further data PDUs. The target does not know if R2Ts can be sent since it does not know if any unsolicited PDUs are expected. Perhaps the semantics on the final flag on SCSI command need to be changed to indicate that no unsolicited data will be sent with this command. Currently, the flag implies (Sec 2.3.1) that all the required data has been sent with the command. And here's a typo to fix. Appendix E (Key=immediateData) The following should say "initialR2T is no". The table has got it right. > If ImmediateData is set to yes and InitialR2T is set > to yes then the initiator MAY send unsolicited immediate > data or one unsolicited burst of Data-OUT PDUs but > MUST NOT send both immediate and a unsolicited burst of > Data-OUT PDUs for any one command. -Sandeep From owner-ips@ece.cmu.edu Sat May 5 15:18:23 2001 Received: from ece.cmu.edu (ECE.CMU.EDU [128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id PAA17531 for ; Sat, 5 May 2001 15:18:23 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f45GtxI20163 for ips-outgoing; Sat, 5 May 2001 12:55:59 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from d12lmsgate-2.de.ibm.com (d12lmsgate-2.de.ibm.com [195.212.91.200]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f45GtvH20159 for ; Sat, 5 May 2001 12:55:57 -0400 (EDT) Received: from d12relay02.de.ibm.com (d12relay02.de.ibm.com [9.165.215.23]) by d12lmsgate-2.de.ibm.com (1.0.0) with ESMTP id SAA77784 for ; Sat, 5 May 2001 18:55:50 +0200 From: julian_satran@il.ibm.com Received: from d12mta02.de.ibm.com (d12mta01_cs0 [9.165.222.237]) by d12relay02.de.ibm.com (8.8.8m3/NCO v4.96) with SMTP id SAA131422 for ; Sat, 5 May 2001 18:55:49 +0200 Received: by d12mta02.de.ibm.com(Lotus SMTP MTA v4.6.5 (863.2 5-20-1999)) id C1256A43.005CFE55 ; Sat, 5 May 2001 18:55:44 +0200 X-Lotus-FromDomain: IBMIL@IBMDE To: ips@ece.cmu.edu Message-ID: Date: Sat, 5 May 2001 19:21:18 +0300 Subject: Re: iSCSI: Nailing down CRC-32C Mime-Version: 1.0 Content-type: text/plain; charset=us-ascii Content-Disposition: inline Sender: owner-ips@ece.cmu.edu Precedence: bulk The CRC part of the appendix (for 07) reads: The following table lists cyclic integrity checksums that can be negotiated for the digests and MUST be implemented by every iSCSI initiator and target. Note that these digest options have only error detection significance. +---------------------------------------------+ | Name | Description | +---------------------------------------------+ | crc-32C | 32 bit CRC | 11EDC6F41 | +---------------------------------------------+ | none | no digest | +---------------------------------------------+ The generator polynomials for those digests are given in hex-notation, for example 3a stands for 0011 1010 - the polynomial x**5+X**4+x**3+x+1. The generator polynomial selected is evaluated in [Castagnioli93]. When using the CRC the CRC register must be initialized to all 1s (0xFFFFFFFF) and the CRC bits must be complemented before transmission. Padding bytes, when presents in a segment covered by a CRC, have to be set to 0 and are included in the CRC. Regards, Julo Mark Bakke on 05-05-2001 00:13:09 Please respond to Mark Bakke To: IPS cc: Subject: iSCSI: Nailing down CRC-32C At the interim meeting, it was stated that iSCSI has selected the CRC-32C polynomial as its required iSCSI-level header and data CRC. Now that we have it, it's time to move on and make sure we can implement it. In the interest of interoperability, we need to not only specify the polynomial, but also the initial values, bit and byte ordering, etc. "A Painless Guide to CRC Error Detection Algorithms" (http://www.ross.net/crc/crcpaper.html) specifies a method to unambiguously characterize these parameters (sections 15 and 16). Has anyone taken a shot at defining these yet? Otherwise, here is what it might look like: Name : "CRC-32C" Width : 32 Poly : 1EDC6F41 (note that the leading "1" is implied) Init : FFFFFFFF RefIn : True RefOut : True XorOut : FFFFFFFF Check : ? I haven't attempted to create check data based on these yet. As soon as the other parameters are nailed down, we need to do this. Anyway, I am not a CRC expert, and can't make any statement about the above values being the "best" way to do this, but instead just copied them (except the polynomial itself) from the Ethernet CRC, since that is likely the easiest for everyone implementing hardware to deal with. If someone else is already doing this, let me know; I just wanted to start this thread so we can get closure. Regards, Mark -- Mark A. Bakke Cisco Systems mbakke@cisco.com 763.398.1054 From owner-ips@ece.cmu.edu Sun May 6 11:27:51 2001 Received: from ece.cmu.edu ([128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id LAA11668 for ; Sun, 6 May 2001 11:27:51 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f46DIT509247 for ips-outgoing; Sun, 6 May 2001 09:18:29 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from d12lmsgate-2.de.ibm.com (d12lmsgate-2.de.ibm.com [195.212.91.200]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f46DIQH09242 for ; Sun, 6 May 2001 09:18:27 -0400 (EDT) Received: from d12relay02.de.ibm.com (d12relay02.de.ibm.com [9.165.215.23]) by d12lmsgate-2.de.ibm.com (1.0.0) with ESMTP id PAA264838; Sun, 6 May 2001 15:18:19 +0200 From: julian_satran@il.ibm.com Received: from d12mta02.de.ibm.com (d12mta01_cs0 [9.165.222.237]) by d12relay02.de.ibm.com (8.8.8m3/NCO v4.96) with SMTP id PAA62612; Sun, 6 May 2001 15:18:19 +0200 Received: by d12mta02.de.ibm.com(Lotus SMTP MTA v4.6.5 (863.2 5-20-1999)) id C1256A44.004914E0 ; Sun, 6 May 2001 15:18:14 +0200 X-Lotus-FromDomain: IBMIL@IBMDE To: someshg@yahoo.com cc: ips@ece.cmu.edu Message-ID: Date: Sun, 6 May 2001 16:23:56 +0300 Subject: RE: iSCSI ERT: open issues list#1 Mime-Version: 1.0 Content-type: text/plain; charset=us-ascii Content-Disposition: inline Sender: owner-ips@ece.cmu.edu Precedence: bulk Somesh, This reminds me that when we first started considering ordering in its different flavors i briefly considered a dual numbering scheme global and per connection with the global meant for command ordering and the per connection meant for an completely ordered delivery per connection above an imperfect TCP. On my notes only 2 lines appear on "why rejected": 1-unneeded serialization of delivery when commands are already executed 2-obviously bad interaction with TCP It took me yesterday some time to recall the "obvious" - and, like in the joke, it is obvious! The ordering over TCP has appeal when done underneath iSCSI as it makes iSCSI a safe transport in no need for other recovery within a connection. However in order for it to work it has to have: - either have a buffer larger that an the one implied by the RTT to keep the TCP window from blocking when a failed piece is retransmitted - or drop every piece of information after a failure - i.e. same behavior as a connection drop If you attempt this scheme not as a "safety layer underneath iSCSI" i.e. deliver the numbered pieces to iSCSI and have it make something out of the numbering then the effect of unwanted serialization becomes dramatic (as you are unaware of the position of the missing piece in the puzzle you will have to stop many activities - even unrelated) and the recovery is not simpler than in the schemes we have. A good example is to compare what will happen if you have several tape writes and a piece of data is dropped in this scheme vs. the per/command numbering we have in the current draft. However within the recovery team we will reexamine this scheme and other versions during this (it has already started for me!) week. Regards, Julo "Somesh Gupta" on 03-05-2001 20:51:00 Please respond to someshg@yahoo.com To: Julian Satran/Haifa/IBM@IBMIL, cbm@rose.hp.com cc: cbm@rose.hp.com, someshg@yahoo.com, venkat@rhapsodynetworks.com, steph@cs.uchicago.edu, ldalleore@snapserver.com, Black_David@emc.com, John Hufferd/San Jose/IBM@IBMUS Subject: RE: iSCSI ERT: open issues list#1 BTW, The interim meeting approved a requirement stating (I am paraphrasing it - hopefully correctly) iSCSI provides an ordered transport, even in the presence of errors i.e. iSCSI provides an ordered transport without excuses. This would seem to have an impact on the protocol and the way we handle errors. Somesh > -----Original Message----- > From: julian_satran@il.ibm.com [mailto:julian_satran@il.ibm.com] > Sent: Thursday, May 03, 2001 12:27 AM > To: cbm@rose.hp.com > Cc: cbm@rose.hp.com; someshg@yahoo.com; venkat@rhapsodynetworks.com; > steph@cs.uchicago.edu; ldalleore@snapserver.com; Black_David@emc.com; > hufferd@us.ibm.com > Subject: Re: iSCSI ERT: open issues list#1 > > > > > Mallikarjun, > > Can we schedule a conference call on Monday 9:30 AM PDT (7:30 PM IDT) to > help us close whatever is still open? > Can you do it? (this way only I will pay transatlatic rates :-)) > > Thanks, > Julo > > _________________________________________________________ Do You Yahoo!? Get your free @yahoo.com address at http://mail.yahoo.com From owner-ips@ece.cmu.edu Sun May 6 21:08:57 2001 Received: from ece.cmu.edu ([128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id VAA15610 for ; Sun, 6 May 2001 21:08:57 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f46MmpB13441 for ips-outgoing; Sun, 6 May 2001 18:48:51 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from patan.sun.com (patan.Sun.COM [192.18.98.43]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f459KFH24346 for ; Sat, 5 May 2001 05:20:15 -0400 (EDT) Received: from centralmail1.Central.Sun.COM ([129.147.62.10]) by patan.sun.com (8.9.3+Sun/8.9.3) with ESMTP id CAA29054 for ; Sat, 5 May 2001 02:20:13 -0700 (PDT) Received: from esun1as-mm. (esun1as-mm.Central.Sun.COM [129.147.34.144]) by centralmail1.Central.Sun.COM (8.9.3+Sun/8.9.3/ENSMAIL,v2.1p1) with SMTP id DAA23091 for ; Sat, 5 May 2001 03:20:14 -0600 (MDT) Received: from sun.com by esun1as-mm. (SMI-8.6/SMI-SVR4) id DAA23982; Sat, 5 May 2001 03:26:28 -0600 Message-ID: <3AF3C5CC.8E0F3A21@sun.com> Date: Sat, 05 May 2001 02:20:12 -0700 From: Raghavendra Rao X-Mailer: Mozilla 4.51 [en] (Win98; U) X-Accept-Language: en MIME-Version: 1.0 To: ips@ece.cmu.edu Subject: RE: iSNS: Event registry and notification Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-ips@ece.cmu.edu Precedence: bulk Content-Transfer-Encoding: 7bit Josh, > Although the type (initiator or target) of iSNS client is stored > in the iSNS database, note that the iSNS server does not make a > distinction in how it interacts with each type of client--both types > (initiators and targets) are uniformly treated as iSNS clients. > That means both initiators and targets can register for notification > and are members of Discovery Domains. I understand the philosophy, my example questions based on initiator or targets weren't implied to change iSNS to be initiator or target oriented, rather I tried to illustrate what one would want to get out of the events. > Please review the section on Discovery Domains. DD's are used to > define event registration and notification policy. Well, DDs may solve flooding of events to all unnecessary iSCSI nodes to some extent. But I still visualise DDs to contain a good number of iSCSI nodes even though the number won't be huge. In that case, it may not be correct to assume that all iSCSI nodes are capable of heavyweight processing of all kinds of events in the discovery domain - Turning off events might be an option for those nodes but these Nodes might still be interested in receiving notification about certain Nodes inside of a discovery domain, not ALL of the nodes in the DD. So what I would like to see is: a) more fine grained list of events - One such example event would be to notify when an iSCSI node with certain attributes (such as access permision to a specific iSCSI node) or capabilities is added /removed to/from a Disocvery domain. Otherwise, let us say, if there are 255 iSCSI nodes in a discovery domain, and if a new node is added, and even if the new node has access permission to ONLY one iSCSI node in the discovery domain, we would see all 255 nodes receiving notification and all 255 nodes probing for access control which might cause needless storm of activity in the network. b) mechanism to selectively receive one or more (set) of these events as a certain iSCSI node may not be interested in all that happens inside a DD, however small a DD might be. -JP From owner-ips@ece.cmu.edu Mon May 7 03:04:34 2001 Received: from ece.cmu.edu ([128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id DAA04175 for ; Mon, 7 May 2001 03:04:34 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f4752Zb06220 for ips-outgoing; Mon, 7 May 2001 01:02:35 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from server1.NishanSystems.COM (smtp.nishansystems.com [216.217.36.162]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f4752YH06216 for ; Mon, 7 May 2001 01:02:34 -0400 (EDT) Received: by smtp.nishansystems.com with Internet Mail Service (5.5.2653.19) id ; Sun, 6 May 2001 22:02:27 -0700 Message-ID: From: Joshua Tseng To: "'Raghavendra Rao'" , ips@ece.cmu.edu Subject: RE: iSNS: Event registry and notification Date: Sun, 6 May 2001 22:02:23 -0700 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2653.19) Content-Type: text/plain; charset="iso-8859-1" Sender: owner-ips@ece.cmu.edu Precedence: bulk JP, > Well, DDs may solve flooding of events to all unnecessary iSCSI nodes > to some extent. But I still visualise DDs to contain a good number of > iSCSI nodes even though the number won't be huge. In that case, it may > not be correct to assume that all iSCSI nodes are capable of > heavyweight > > processing of all kinds of events in the discovery domain - > Turning off > events might be an option for those nodes but these Nodes might still > be interested in receiving notification about certain Nodes inside of > a discovery domain, not ALL of the nodes in the DD. > Registration for notification is optional. If an iSCSI device is afraid of being deluged by notification messages, then perhaps it shouldn't register for notifications, and instead periodically query to get an update on DD membership. > So what I would like to see is: > > a) more fine grained list of events > - One such example event would be to notify when an > iSCSI node > with certain attributes (such as access permision to a > specific iSCSI > node) or capabilities is added /removed to/from a > Disocvery > domain. > Otherwise, let us say, if there are 255 iSCSI nodes in a > discovery > domain, and if a new node is added, and even if > the new node > has > access permission to ONLY one iSCSI node in the discovery > domain, > we would see all 255 nodes receiving notification > and all 255 > nodes > probing for access control which might cause needless storm > of activity > in the network. DD's define security/access control policy. If a new node only had access permission to one other iSCSI node, then those two nodes would be the only members of the DD. A node can be a member of multiple DD's. If 255 devices are in the DD, that implies that all 255 devices have permission to access each other. Initiators would therefore see and receive access to all targets and initiators in the common DD, although in practice they would ignore info on the initiators. > > b) mechanism to selectively receive one or more (set) of these > events > as a certain iSCSI node may not be interested in all > that happens > inside > a DD, however small a DD might be. We have to keep the protocol simple. Notification is for all events in the DD, or else you turn notification off and have the node periodically query to re-synch with the iSNS server. Josh > > -JP > From owner-ips@ece.cmu.edu Mon May 7 03:06:52 2001 Received: from ece.cmu.edu ([128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id DAA04186 for ; Mon, 7 May 2001 03:06:52 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f475BM006775 for ips-outgoing; Mon, 7 May 2001 01:11:22 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from server1.NishanSystems.COM (smtp.nishansystems.com [216.217.36.162]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f475BKH06758 for ; Mon, 7 May 2001 01:11:20 -0400 (EDT) Received: by smtp.nishansystems.com with Internet Mail Service (5.5.2653.19) id ; Sun, 6 May 2001 22:11:08 -0700 Message-ID: From: Joshua Tseng To: "'Raghavendra Rao'" , "'ips@ece.cmu.edu'" Subject: FW: iSNS: Event registry and notification Date: Sun, 6 May 2001 22:11:00 -0700 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2653.19) Content-Type: text/plain; charset="iso-8859-1" Sender: owner-ips@ece.cmu.edu Precedence: bulk JP, One thing I forgot to mention until after I sent this message is that there is an "SCN Bitmap" which allows the iSNS client to register for specific events. If it is only interested in devices being added, but doesn't care if devices are pulled, then it sets the appropriate bits of the SCN bitmap. This may help it to keep from being deluged by notifications that it doesn't care about. See section 6.4.4. Josh -----Original Message----- From: Joshua Tseng Sent: Sunday, May 06, 2001 10:02 PM To: 'Raghavendra Rao'; ips@ece.cmu.edu Subject: RE: iSNS: Event registry and notification JP, > Well, DDs may solve flooding of events to all unnecessary iSCSI nodes > to some extent. But I still visualise DDs to contain a good number of > iSCSI nodes even though the number won't be huge. In that case, it may > not be correct to assume that all iSCSI nodes are capable of > heavyweight > > processing of all kinds of events in the discovery domain - > Turning off > events might be an option for those nodes but these Nodes might still > be interested in receiving notification about certain Nodes inside of > a discovery domain, not ALL of the nodes in the DD. > Registration for notification is optional. If an iSCSI device is afraid of being deluged by notification messages, then perhaps it shouldn't register for notifications, and instead periodically query to get an update on DD membership. > So what I would like to see is: > > a) more fine grained list of events > - One such example event would be to notify when an > iSCSI node > with certain attributes (such as access permision to a > specific iSCSI > node) or capabilities is added /removed to/from a > Disocvery > domain. > Otherwise, let us say, if there are 255 iSCSI nodes in a > discovery > domain, and if a new node is added, and even if > the new node > has > access permission to ONLY one iSCSI node in the discovery > domain, > we would see all 255 nodes receiving notification > and all 255 > nodes > probing for access control which might cause needless storm > of activity > in the network. DD's define security/access control policy. If a new node only had access permission to one other iSCSI node, then those two nodes would be the only members of the DD. A node can be a member of multiple DD's. If 255 devices are in the DD, that implies that all 255 devices have permission to access each other. Initiators would therefore see and receive access to all targets and initiators in the common DD, although in practice they would ignore info on the initiators. > > b) mechanism to selectively receive one or more (set) of these > events > as a certain iSCSI node may not be interested in all > that happens > inside > a DD, however small a DD might be. We have to keep the protocol simple. Notification is for all events in the DD, or else you turn notification off and have the node periodically query to re-synch with the iSNS server. Josh > > -JP > From owner-ips@ece.cmu.edu Mon May 7 10:56:43 2001 Received: from ece.cmu.edu ([128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id KAA13695 for ; Mon, 7 May 2001 10:56:43 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f47D2EU14757 for ips-outgoing; Mon, 7 May 2001 09:02:14 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from sandmail.sandburst.com (sandburst-gw.bstn-gw02.ma.us.intelilink.net [216.57.129.34]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f47D2AH14748 for ; Mon, 7 May 2001 09:02:10 -0400 (EDT) Received: from cs.uchicago.edu (dynamite-38.sandburst.com [172.16.5.38]) by sandmail.sandburst.com (Postfix) with ESMTP id 0907F94006 for ; Mon, 7 May 2001 09:02:09 -0400 (EDT) To: ips@ece.cmu.edu Subject: Re: iSCSI : Review Feedback on iSCSI Rev 06 In-Reply-To: Message from Santosh Rao of "Thu, 03 May 2001 19:01:02 PDT." <200105040201.TAA00579@hpcuhe.cup.hp.com> References: <200105040201.TAA00579@hpcuhe.cup.hp.com> Date: Mon, 07 May 2001 09:00:22 -0400 From: Stephen Bailey Message-Id: <20010507130209.0907F94006@sandmail.sandburst.com> Sender: owner-ips@ece.cmu.edu Precedence: bulk Santosh, > 3) Why does the new draft limit the size of text commands and nop commands to > 4096 bytes ? What is the rationale behind the magic number selection of 4096 > ? The reason for a limit like this (I won't argue that we've done it quite right here, in fact, I may argue against it a little bit below), is for to allow the receiving endpoints to have only extremely modest, tightly bounded resources for handling this non-nominal (or, at least, non-performance) traffic. DataPDULength is way too large a limit. I'm still optimistic that the maximum DataPDULength limit will be relaxed back towards 4G, even though I know that y'all have decided against it and everybody out there thinks I'm on dust (you'll see...). The limit that makes slightly more sense from an engineering standpoint, but makes no sense whatsoever from a standards standpoint, is FirstBurstSize. In a hardware implementation, DataPDULength is a property associated with the hardware data handling path. The data arriving in DataPDULength units is destined for buffers which are known to have already been allocated and supplied from the upper level. The resource cost for these buffers is a direct function of the application, as opposed to iSCSI overhead. The Login[Response], Text and Ping PDUs, and probably (but not definitely) unsolicited data, flow through a path where the buffers are iSCSI implementation overhead. iSCSI must provide a mechanism for reducing this resource overhead to an arbitrary, manageable size. The alternative is to require an iSCSI implementation to allocate a gratuitously large quantity of memory to handle occurences which a) may rarely (probably never) happen b) don't provide the implementation anything at all worthwhile in return for that memory. Because this is a non-performance path, having this parameter being negotiable is also a waste of effort. The iSCSI negotiation protocol should be (it mostly already is) designed to tightly bound the worst-case (largest) text payload in a Text or Login[Response]. To do this we must: 1) determine the maximum text payload size which contains all `fixed' length keys 2) create an iterator protocol for arbitrary, or even just large text messages. This includes X-* parameters---an X-* exchange must be structured to work within the max text PDU limits. In addition the X-* parameter definitions must create their own iterator if necessary---iSCSI only needs to handle iSCSI specified keys. Looking over the present set of text keys, the security exchanges already have tight size bounds, and clearly aren't going to be the limiting factor in the maximal text payload. That leaves the operation parameters. Most of the operational PDUs also have a small maximum fixed size. The exceptions are: InitiatorName InitiatorAlias TargetName TargetAddress TargetAlias We can probably put a modest bound on each of these keys individually (e.g. 512 bytes), which means that the maximum text payload size within a Login exchange is tightly bounded and 4K seems adequate. The problem comes with SendTargets=. That is why I proposed an iterator protocol for SendTargets= instead of having the target information gush back from the target uncontrolled. Simply bounding the maximum text payload and allowing and arbitrary sequence of text payloads to be sent in response to a single request doesn't cut it. We need to be able to bound the total amount of data returned from each particular request. SendTargets= actually has two problems. 1) an arbitrary number of targets per endpoint, 2) an arbitrary number of addresses per target. Given the second problem, unfortunately, it's not adequate to merely have a target iterator, since the data for each target may be arbitrarily long. Therefore, here's another proposal which might be close to the mark: Initiator keys: SendTargetStart= Send first PDU of target list information If already in the midst of a target list exchange, restart at the beginning SendTargetMore= Send next PDU of target list information SendTargetEnd= Terminate a target list exchange Target Keys: TargetName=name one per target TargetAddress=address one or more per target TargetAlias=alias one per target (may be empty) TargetMore= more target information remaining within the present target A target list exchange always terminates with a Text PDU with final=1. In the case that the exchange is ended by a SendTargetEnd=, the target's Text PDU with final=1 would have no payload. We would specify at most one target list request outstanding per session. Personally, I'd like to be even more restrictive and say that target list exchanges are only allowed on single connection sessions where a TargetName was not specified at login. However, Mark Bakke seemed to think that rather than a session with no TargetName, it should be a session with TargetName=iSCSI. Either way is fine with me, assuming that we do not want to allow a regular, operational login with TargetName=iSCSI (e.g. for boot, or simple initiators). I happen to think that operational login with TargetName=iSCSI is useful. The fact is, many boxes, even big ones, will have only a single target, but many LUNs. This corresponds to present storage boxes (as opposed to bridges). Steph From owner-ips@ece.cmu.edu Mon May 7 11:03:24 2001 Received: from ece.cmu.edu ([128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id LAA13864 for ; Mon, 7 May 2001 11:03:23 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f47D2BN14756 for ips-outgoing; Mon, 7 May 2001 09:02:11 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from sandmail.sandburst.com (sandburst-gw.bstn-gw02.ma.us.intelilink.net [216.57.129.34]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f47D2AH14747 for ; Mon, 7 May 2001 09:02:10 -0400 (EDT) Received: from cs.uchicago.edu (dynamite-38.sandburst.com [172.16.5.38]) by sandmail.sandburst.com (Postfix) with ESMTP id 2DAF994009 for ; Mon, 7 May 2001 09:02:09 -0400 (EDT) To: ips@ece.cmu.edu Subject: Re: iSCSI: comments to iSCSI rev 6 In-Reply-To: Message from "Matt Wakeley" of "Fri, 04 May 2001 11:24:46 PDT." <3AF2F3EE.998DAE24@agilent.com> References: <3AF0232E.852EDB5E@cisco.com> <3AF08DB1.D1A48FF7@agilent.com> <3AF17047.89EB3B62@cisco.com> <3AF2F3EE.998DAE24@agilent.com> Date: Mon, 07 May 2001 09:00:22 -0400 From: Stephen Bailey Message-Id: <20010507130209.2DAF994009@sandmail.sandburst.com> Sender: owner-ips@ece.cmu.edu Precedence: bulk All, I agree with Matt. Every text key in the iSCSI specification should have a precisely specified response to indicate that it was received and understood. This does not include X-* keys, for which you're on your own. A property which I think Matt has articulated, but I want to repeat, is that an implementation should be able to sanity-check all negotiation interactions as well as simply communicating the relevant information. Specifically, if a negotiation `fails', an implementation must distinguish between incompatible capabilities and logical errors whenever possible. For example if a target sends an AccessID=xxx, the present specification seems to suggest that there is no response required or allowed for that key. I believe we should specify that some response (perhaps echoing it?) to distinguish between the cases where the AccessID had the desired effect, and where it had no effect at all (was ignored). Explicitly specifying responses for each key allows a more `anal' implementation to sanity check a more lax one. We all know we'll certainly see both implementation styles regardless of what's specified. Steph From owner-ips@ece.cmu.edu Mon May 7 15:45:49 2001 Received: from ece.cmu.edu ([128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id PAA20607 for ; Mon, 7 May 2001 15:45:48 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f47HZuc08220 for ips-outgoing; Mon, 7 May 2001 13:35:56 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from dirty.research.bell-labs.com (dirty.research.bell-labs.com [204.178.16.6]) by ece.cmu.edu (8.11.0/8.10.2) with SMTP id f47HZqH08214 for ; Mon, 7 May 2001 13:35:52 -0400 (EDT) Received: from grubby.research.bell-labs.com ([135.104.2.9]) by dirty; Mon May 7 13:34:30 EDT 2001 Received: from aura.research.bell-labs.com ([135.104.46.10]) by grubby; Mon May 7 13:34:27 EDT 2001 Received: (from sandeepj@localhost) by aura.research.bell-labs.com (8.9.1/8.9.1) id NAA19775 for ips@ece.cmu.edu; Mon, 7 May 2001 13:34:19 -0400 (EDT) Date: Mon, 7 May 2001 13:34:19 -0400 (EDT) Message-Id: <200105071734.NAA19775@aura.research.bell-labs.com> From: sandeepj@research.bell-labs.com (Sandeep Joshi) To: ips@ece.cmu.edu Subject: iSCSI: firstburst and maxburst Sender: owner-ips@ece.cmu.edu Precedence: bulk Julian, SPC-2 revision 19: section 8.3.7 Disconnect-reconnect page states that a value of "0" indicates no limit on the maximum and first burst size. Current range for the corresponding text keys from Appendix E are (1 to 2^15-1)*512 bytes. How should iSCSI interpret a zero ? regards, -Sandeep From owner-ips@ece.cmu.edu Mon May 7 17:54:23 2001 Received: from ece.cmu.edu ([128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id RAA23224 for ; Mon, 7 May 2001 17:54:23 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f47K2rU21864 for ips-outgoing; Mon, 7 May 2001 16:02:53 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from dogwood.cisco.com (dogwood.cisco.com [161.44.11.19]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f47K2pH21859 for ; Mon, 7 May 2001 16:02:51 -0400 (EDT) Received: from cisco.com (mbakke@mbakke-lnx.cisco.com [161.44.68.87]) by dogwood.cisco.com (8.8.6 (PHNE_14041)/CISCO.SERVER.1.2) with ESMTP id QAA28582; Mon, 7 May 2001 16:02:44 -0400 (EDT) Message-ID: <3AF6FEDC.FDF01EF1@cisco.com> Date: Mon, 07 May 2001 15:00:28 -0500 From: Mark Bakke X-Mailer: Mozilla 4.72 [en] (X11; U; Linux 2.2.16-3.uid32 i686) X-Accept-Language: en, de MIME-Version: 1.0 To: julian_satran@il.ibm.com CC: ips@ece.cmu.edu Subject: Re: iSCSI: Nailing down CRC-32C References: Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-ips@ece.cmu.edu Precedence: bulk Content-Transfer-Encoding: 7bit Julian- That's great; it covers nearly everything. The only thing left is the byte order; are they taken in the same order Ethernet uses? If so, I can generate a few test patterns that our implementations can all check against. Thanks, -- Mark julian_satran@il.ibm.com wrote: > > The CRC part of the appendix (for 07) reads: > > The following table lists cyclic integrity checksums that can be > negotiated for the digests and MUST be implemented by every iSCSI > initiator and target. Note that these digest options have only error > detection significance. > > +---------------------------------------------+ > | Name | Description | > +---------------------------------------------+ > | crc-32C | 32 bit CRC | 11EDC6F41 | > +---------------------------------------------+ > | none | no digest | > +---------------------------------------------+ > > The generator polynomials for those digests are given in hex-notation, > for example 3a stands for 0011 1010 - the polynomial x**5+X**4+x**3+x+1. > > The generator polynomial selected is evaluated in [Castagnioli93]. > When using the CRC the CRC register must be initialized to all 1s > (0xFFFFFFFF) and the CRC bits must be complemented before transmission. > Padding bytes, when presents in a segment covered by a CRC, have to be > set to 0 and are included in the CRC. > > Regards, > Julo > > Mark Bakke on 05-05-2001 00:13:09 > > Please respond to Mark Bakke > > To: IPS > cc: > Subject: iSCSI: Nailing down CRC-32C > > At the interim meeting, it was stated that iSCSI has selected > the CRC-32C polynomial as its required iSCSI-level header and > data CRC. Now that we have it, it's time to move on and make > sure we can implement it. > > In the interest of interoperability, we need to not only specify > the polynomial, but also the initial values, bit and byte > ordering, etc. > > "A Painless Guide to CRC Error Detection Algorithms" > (http://www.ross.net/crc/crcpaper.html) specifies a > method to unambiguously characterize these parameters > (sections 15 and 16). Has anyone taken a shot at defining > these yet? Otherwise, here is what it might look like: > > Name : "CRC-32C" > Width : 32 > Poly : 1EDC6F41 (note that the leading "1" is implied) > Init : FFFFFFFF > RefIn : True > RefOut : True > XorOut : FFFFFFFF > Check : ? > > I haven't attempted to create check data based on these yet. As > soon as the other parameters are nailed down, we need to do this. > > Anyway, I am not a CRC expert, and can't make any statement about > the above values being the "best" way to do this, but instead just > copied them (except the polynomial itself) from the Ethernet CRC, > since that is likely the easiest for everyone implementing hardware > to deal with. > > If someone else is already doing this, let me know; I just wanted > to start this thread so we can get closure. > > Regards, > > Mark > > -- > Mark A. Bakke > Cisco Systems > mbakke@cisco.com > 763.398.1054 -- Mark A. Bakke Cisco Systems mbakke@cisco.com 763.398.1054 From owner-ips@ece.cmu.edu Mon May 7 17:55:39 2001 Received: from ece.cmu.edu ([128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id RAA23269 for ; Mon, 7 May 2001 17:55:38 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f47KQs724139 for ips-outgoing; Mon, 7 May 2001 16:26:54 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from e31.bld.us.ibm.com (e31.co.us.ibm.com [32.97.110.129]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f47KQrH24133 for ; Mon, 7 May 2001 16:26:53 -0400 (EDT) Received: from westrelay02.boulder.ibm.com (westrelay02.boulder.ibm.com [9.99.140.23]) by e31.bld.us.ibm.com (8.9.3/8.9.3) with ESMTP id QAA74172; Mon, 7 May 2001 16:19:18 -0400 Received: from f4n49e (d03nm065h.boulder.ibm.com [9.99.140.49]) by westrelay02.boulder.ibm.com (8.11.1/NCO v4.96.1.0) with ESMTP id f47KQVK85838; Mon, 7 May 2001 14:26:46 -0600 X-Priority: 1 (High) Importance: Normal Subject: Re: iSCSI : digest error handling violates EMDP/InDataOrder To: Santosh Rao Cc: ips@ece.cmu.edu X-Mailer: Lotus Notes Release 5.0.3 (Intl) 21 March 2000 Message-ID: From: "John Hufferd" Date: Mon, 7 May 2001 13:26:13 -0700 X-MIMETrack: Serialize by Router on D03NM065/03/M/IBM(Release 5.0.6 |December 14, 2000) at 05/07/2001 02:26:46 PM MIME-Version: 1.0 Content-type: text/plain; charset=us-ascii Sender: owner-ips@ece.cmu.edu Precedence: bulk Santosh, I have re-read 2.7.5 a number of times, and can not understand your point. To me that section clearly states that the data can be sent out of order if permitted by the EMDP bit. But any individual bursts, must be in order within that burst/R2T response. The only thing that can be out of order, is the data sent in different bursts, with respect to each other. That seems clear, so I must be missing some important point that you are trying to make. If you think I have this all wrong please let me know. . . . John L. Hufferd Senior Technical Staff Member (STSM) IBM/SSG San Jose Ca (408) 256-0403, Tie: 276-0403, eFax: (408) 904-4688 Internet address: hufferd@us.ibm.com Santosh Rao @ece.cmu.edu on 05/03/2001 05:35:03 PM Sent by: owner-ips@ece.cmu.edu To: Julian Satran/Haifa/IBM@IBMIL cc: ips@ece.cmu.edu Subject: Re: iSCSI : digest error handling violates EMDP/InDataOrder In previous exchanges in this thread, it was stated that EMDP only governs data order within a sequence. This, to me [and others from FC world, as was noted by Bob] reads as the definition of Random relative Offset [as defined by bob]. However, Section 2.7.5 on Buffer Offset states that : "Output data within a burst MUST be delivered in increasing buffer offset order". This contradicts the above by indicating that iSCSI forbids Random Relative Offset. So, does EMDP apply to WRITEs or does it not (?). Also, does iSCSI allow data overlay or is this prohibited ? I'd suggest the following be clearly described in the draft : a) iSCSI defines EMDP to govern the order of data PDUs within a SCSI command for both READ and WRITE operations. An EMDP setting of 0 implies data overlay is forbidden. b) The out-of-order transmission of a given sequence of WRITE data PDUs in response to an R2T (the random relative offset property) be explicitly dis-allowed by iSCSI [which is currently stated in Section 2.7.5]. Regards, Santosh julian_satran@il.ibm.com wrote: > > Santosh, > > Let's take a systematic approach to it. > > Restriction on data ordering are required if the source or the destination > of the data is unable to deliver or take data data in any order other that > sequential. > > Semiconductor or other direct access memories don't have this restriction. > > Tapes and other sequential media do have this type of restriction and so > some streaming devices. > > If the restricted device is a target of a SCSI operation with an > unrestricted initiator then: > > a. on reads the target can always ship its data in sequential order > b. on writes the target can always request the data in sequential order > > However if the restricted device is an initiator then: > > a. on reads the initiator will request the target to send the data in order > b. on writes the restricted initiator will have to get the R2Ts in order > from the target and will be able to support data recovery through an R2T > only if it has enough buffered data. > > A restricted device will act as an initiator only if it becomes a third > part copy manager (CM) in a third party operation an does copy from one of > its devices to another device. > > Introducing a new mode bit (as Robert Snively seems to suggest) will not > change the fact that the restriction can't be upholded > and do recovery unless the restricted initiator has enough memory. > > The spec should only specify a way to terminate a command in those > conditions and leave it at that. > > I will change the wording of the DataOrder to make it clearer but I > consider the whole issue entirely academic and overblown. > > Recall also that a CM implemented which such severe buffering restrictions > violates the basic SCSI assumption that a target is the data master. > > Regards, > Julo > > Santosh Rao on 28/04/2001 00:03:26 > > Please respond to Santosh Rao > > To: Julian Satran/Haifa/IBM@IBMIL > cc: Black_David@emc.com, ips@ece.cmu.edu > Subject: Re: iSCSI : digest error handling violates EMDP/InDataOrder > > julian_satran@il.ibm.com wrote: > > > > David, > > > > I read Bob's mail and my interpretation is similar to his. However I > think > > that SPC explicitly states that different transports are free to > interpret > > and make use of this page as they find appropriate. > > > > I have a hard time understanding Santosh's objection as it does not refer > > to the reason the EMDP is there but to the way it is written in FCP (not > > iSCSI). > > Julian, > > As has been stated earlier, EMDP allows control over the order in which > the target requests outbound data or sends inbound data. EMDP can be > used by initiators to control this order and turn off out-of-order R2T > requests [as well as turn off out of order read data pdus]. > > This is a useful control option and is already provided by other SCSI > transports. What good reason exists to deny this provision in iSCSI ? > > Also, I have some concerns about the ambiguous definition of DataOrder. > > Per the spec : > "DataOrder= > > The default is yes but targets MAY support no. No is used by iSCSI to > indicate that the data PDUs can be in any order (EMDP = 1). Yes is used > to indicate that incoming data PDUs have to be at continuously > increasing addresses (EMDP = 0)." > > Based on the above definition wording : > > a) How is DataOrder interpreted for WRITE I/Os ? > b) Is the ordering across the entire SCSI command or a subset of the I/O > ? If so, what constitutes this subset ? > > Different implementors can arrive at different interpretations reading > the above definition ! > > - Santosh > - santoshr.vcf From owner-ips@ece.cmu.edu Mon May 7 19:10:37 2001 Received: from ece.cmu.edu ([128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id TAA24155 for ; Mon, 7 May 2001 19:10:37 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f47LD9W28379 for ips-outgoing; Mon, 7 May 2001 17:13:09 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from msgbas1.cos.agilent.com (msgbas1x.cos.agilent.com [192.6.9.33]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f47LD7H28374 for ; Mon, 7 May 2001 17:13:07 -0400 (EDT) Received: from msgrel1.cos.agilent.com (msgrel1.cos.agilent.com [130.29.152.77]) by msgbas1.cos.agilent.com (Postfix) with ESMTP id CF27593E; Mon, 7 May 2001 15:13:06 -0600 (MDT) Received: from axcsbh2.cos.agilent.com (axcsbh2.cos.agilent.com [130.29.152.144]) by msgrel1.cos.agilent.com (Postfix) with SMTP id 7BF2894; Mon, 7 May 2001 15:13:06 -0600 (MDT) Received: from 130.29.152.144 by axcsbh2.cos.agilent.com (InterScan E-Mail VirusWall NT); Mon, 07 May 2001 15:13:06 -0600 (Mountain Daylight Time) Received: by axcsbh2.cos.agilent.com with Internet Mail Service (5.5.2653.19) id ; Mon, 7 May 2001 15:13:06 -0600 Message-ID: From: vince_cavanna@agilent.com To: mbakke@cisco.com, Black_David@emc.com Cc: someshg@yahoo.com, ips@ece.cmu.edu, vince_cavanna@agilent.com Subject: RE: iSCSI Security rough consensus Date: Mon, 7 May 2001 15:13:03 -0600 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2653.19) Content-Type: text/plain; charset="iso-8859-1" Sender: owner-ips@ece.cmu.edu Precedence: bulk Here is a paper by well-known authors that supports Mark's reasoning that the end-to-end iSCSI check is indispensable even when IPsec is on .... Jonathan Stone and Craig artridge in "When the CRC and TCP checksum disagree" study the sources of errors that are not caught by link level checks and that therefore stress a higher layer error check mechansim. Such errors are quite prevalent and one signifcant source of such errors is the end-host itself. The authors emphasize that "hardware must not be trusted" and that "many encryption solutions such as IPsec do not provide additional (over existing checks) protection because the encryption is applied too late in the transmission process, often after the data has passed through a DMA engine. Rather the application must add the checksum before handing its data to TCP (ala SSL)". The authors recommend that "for truly valuable data (in my opinion all data handled by iSCSI) the application should add a stronger application-level checksum". Vince |-----Original Message----- |From: Mark Bakke [mailto:mbakke@cisco.com] |Sent: Friday, May 04, 2001 1:04 PM |To: Black_David@emc.com |Cc: someshg@yahoo.com; ips@ece.cmu.edu |Subject: Re: iSCSI Security rough consensus | | |Black_David@emc.com wrote: |> |> > Sure would be nice if we could make up our minds and just |> > implement one mechanism. |> > |> > Here we have two mechanisms (iSCSI header/data integrity |> > and ESP) which are both mandatory to implement and |> > optional to use. Since ESP seems like a superset why not |> > just have that and get rid of the "integrity only" iSCSI |> > CRC mechanism. |> |> It sure would be nice, and in fact we had almost |> exactly this discussion later in the evening as |> part of the error recovery section of the agenda. |> The fly in the ointment is that the HMAC integrity |> algorithm that is at the core of ESP's integrity |> support is considerably more expensive (software |> or hardware) than a CRC, and this isn't likely |> to improve as I understand things. I would expect |> to see implementations with ESP completely in |> software and visible performance impacts. | |That's just part of the reason behind having both. The other is |that most implementations won't run IPsec end-to-end; either IPsec |is provided in an external device, or even in an iSCSI gateway. |In the latter case, all layers are removed and replaced, including |iSCSI. Only the SCSI-level information (data, CDBs) go all the |way end-to-end. Since iSCSI can CRC the SCSI-level data, it can |provide the data integrity that will keep our customers happy. | |The use of the iSCSI CRC is the minimum requirement; adding the |IPsec-level integrity check strengthens it, and can simplify error |recovery over a not-so-good or untrusted network. | |-- |Mark | |> |> I really need to get the meeting minutes written up :-). |> |> Thanks, |> --David |> |> --------------------------------------------------- |> David L. Black, Senior Technologist |> EMC Corporation, 42 South St., Hopkinton, MA 01748 |> +1 (508) 435-1000 x75140 FAX: +1 (508) 497-8500 |> black_david@emc.com Mobile: +1 (978) 394-7754 |> --------------------------------------------------- | |-- |Mark A. Bakke |Cisco Systems |mbakke@cisco.com |763.398.1054 | From owner-ips@ece.cmu.edu Mon May 7 20:07:26 2001 Received: from ece.cmu.edu ([128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id UAA24886 for ; Mon, 7 May 2001 20:07:25 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f47LwbN02345 for ips-outgoing; Mon, 7 May 2001 17:58:37 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from smtp017.mail.yahoo.com (smtp017.mail.yahoo.com [216.136.174.114]) by ece.cmu.edu (8.11.0/8.10.2) with SMTP id f47LwYH02336 for ; Mon, 7 May 2001 17:58:34 -0400 (EDT) Received: from sdsl-216-36-75-164.dsl.sjc.megapath.net (HELO somesh) (216.36.75.164) by smtp.mail.vip.sc5.yahoo.com with SMTP; 7 May 2001 21:58:28 -0000 X-Apparently-From: Reply-To: From: "Somesh Gupta" To: , , Cc: , Subject: RE: iSCSI Security rough consensus Date: Mon, 7 May 2001 14:51:44 -0700 Message-ID: MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 (Normal) X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook IMO, Build 9.0.2416 (9.0.2910.0) In-reply-to: Importance: Normal X-MimeOLE: Produced By Microsoft MimeOLE V5.00.2919.6700 Sender: owner-ips@ece.cmu.edu Precedence: bulk Content-Transfer-Encoding: 7bit I was trying not to react to the string of messages, but this message finally did it(and nothing wrong with the message itself and no offense meant to the author). What the message implies is that no intermediate hops can be trusted and that integrity must be end-to-end. Taken to the literal extreme, this means that the system must at the application level (in the host) generate some integrity checks which should then be verified every time the data is used (perhaps by an end application again). Even with the CRC, the following errors will go undetected, unless protected by some other means (well engineered, well tested devices) 1. Host software. 2. Host memory to adapter path 3. Adapter memory 4. Adapter software 5. Any virtualization device in the middle including software, memory, fabric etc on the device including any fibre-channel/scsi storage at the back end (assuming they will recreate resegment PDUs) 6. Any target adapter & adapter memory 7. Any paths within the target 8. The storage itself 9. and then the reverse path If end-to-end integrity is a must, it seems that the TCP checksum offload and iSCSI CRC in off-board hardware must be avoided. Or did I miss something somewhere. Somesh > -----Original Message----- > From: vince_cavanna@agilent.com [mailto:vince_cavanna@agilent.com] > Sent: Monday, May 07, 2001 2:13 PM > To: mbakke@cisco.com; Black_David@emc.com > Cc: someshg@yahoo.com; ips@ece.cmu.edu; vince_cavanna@agilent.com > Subject: RE: iSCSI Security rough consensus > > > Here is a paper by well-known authors that supports Mark's reasoning that > the end-to-end iSCSI check is indispensable even when IPsec is on .... > > Jonathan Stone and Craig artridge in "When the CRC and TCP checksum > disagree" study the sources of errors that are not caught by link level > checks and that therefore stress a higher layer error check > mechansim. Such > errors are quite prevalent and one signifcant source of such errors is the > end-host itself. The authors emphasize that "hardware must not be trusted" > and that "many encryption solutions such as IPsec do not provide > additional > (over existing checks) protection because the encryption is > applied too late > in the transmission process, often after the data has passed through a DMA > engine. Rather the application must add the checksum before > handing its data > to TCP (ala SSL)". The authors recommend that "for truly valuable data (in > my opinion all data handled by iSCSI) the application should add > a stronger > application-level checksum". > > Vince > > |-----Original Message----- > |From: Mark Bakke [mailto:mbakke@cisco.com] > |Sent: Friday, May 04, 2001 1:04 PM > |To: Black_David@emc.com > |Cc: someshg@yahoo.com; ips@ece.cmu.edu > |Subject: Re: iSCSI Security rough consensus > | > | > |Black_David@emc.com wrote: > |> > |> > Sure would be nice if we could make up our minds and just > |> > implement one mechanism. > |> > > |> > Here we have two mechanisms (iSCSI header/data integrity > |> > and ESP) which are both mandatory to implement and > |> > optional to use. Since ESP seems like a superset why not > |> > just have that and get rid of the "integrity only" iSCSI > |> > CRC mechanism. > |> > |> It sure would be nice, and in fact we had almost > |> exactly this discussion later in the evening as > |> part of the error recovery section of the agenda. > |> The fly in the ointment is that the HMAC integrity > |> algorithm that is at the core of ESP's integrity > |> support is considerably more expensive (software > |> or hardware) than a CRC, and this isn't likely > |> to improve as I understand things. I would expect > |> to see implementations with ESP completely in > |> software and visible performance impacts. > | > |That's just part of the reason behind having both. The other is > |that most implementations won't run IPsec end-to-end; either IPsec > |is provided in an external device, or even in an iSCSI gateway. > |In the latter case, all layers are removed and replaced, including > |iSCSI. Only the SCSI-level information (data, CDBs) go all the > |way end-to-end. Since iSCSI can CRC the SCSI-level data, it can > |provide the data integrity that will keep our customers happy. > | > |The use of the iSCSI CRC is the minimum requirement; adding the > |IPsec-level integrity check strengthens it, and can simplify error > |recovery over a not-so-good or untrusted network. > | > |-- > |Mark > | > |> > |> I really need to get the meeting minutes written up :-). > |> > |> Thanks, > |> --David > |> > |> --------------------------------------------------- > |> David L. Black, Senior Technologist > |> EMC Corporation, 42 South St., Hopkinton, MA 01748 > |> +1 (508) 435-1000 x75140 FAX: +1 (508) 497-8500 > |> black_david@emc.com Mobile: +1 (978) 394-7754 > |> --------------------------------------------------- > | > |-- > |Mark A. Bakke > |Cisco Systems > |mbakke@cisco.com > |763.398.1054 > | _________________________________________________________ Do You Yahoo!? Get your free @yahoo.com address at http://mail.yahoo.com From owner-ips@ece.cmu.edu Mon May 7 20:08:16 2001 Received: from ece.cmu.edu ([128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id UAA24901 for ; Mon, 7 May 2001 20:08:15 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f47M94b03219 for ips-outgoing; Mon, 7 May 2001 18:09:04 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from atlrel1.hp.com (atlrel1.hp.com [156.153.255.210]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f47M92H03213 for ; Mon, 7 May 2001 18:09:02 -0400 (EDT) Received: from xatlrelay1.atl.hp.com (xatlrelay1.atl.hp.com [15.45.89.190]) by atlrel1.hp.com (Postfix) with ESMTP id 6FD34683; Mon, 7 May 2001 18:09:02 -0400 (EDT) Received: from xpabh4.corp.hp.com (xpabh4.corp.hp.com [15.58.136.1]) by xatlrelay1.atl.hp.com (Postfix) with ESMTP id 083551F505; Mon, 7 May 2001 18:06:56 -0400 (EDT) Received: by xpabh4.corp.hp.com with Internet Mail Service (5.5.2653.19) id ; Mon, 7 May 2001 15:08:55 -0700 Message-ID: <499DC368E25AD411B3F100902740AD6502D4F1DA@xrose03.rose.hp.com> From: "CONGDON,PAUL (HP-Roseville,ex1)" To: "'Steve Senum'" , ips@ece.cmu.edu Subject: RE: iscsi: comments to iSCSI rev 6 Date: Mon, 7 May 2001 15:08:07 -0700 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2653.19) Content-Type: text/plain; charset="iso-8859-1" Sender: owner-ips@ece.cmu.edu Precedence: bulk > > Matt, > > > The result may mean the same, but for diagnostic purposes, > the only thing you > > know is you can't do it. You don't know why. If you know > why, then perhaps > > you can take corrective action to remedy the problem. > > To help with this issue, I would prefer key=none be required > to sent, if applicable. That would leave a non response to > mean only not supported. > > Steve Senum This really doesn't seem that hard or worth spending much argument power on. The advantage of specifying something rather than simply not responding is for diagnostic purposes and clarity. You can tell by looking only at the response PDU what isn't supported or not understood. You don't have to keep around the original request to perform a 'diff' to understand why things are negotiating. I agree with Matt. Paul +--------------------------+----------------------------+ + Paul Congdon + Email: paul_congdon@hp.com + + Hewlett Packard Company + Mail Stop: 5662 + + HP ProCurve Networking + Phone: (916) 785-5753 + + 8000 Foothills Blvd + Fax: (916) 785-5949 + + Roseville, CA 95747 + Mobile: (916) 765-4056 + +--------------------------+----------------------------+ From owner-ips@ece.cmu.edu Mon May 7 21:07:23 2001 Received: from ece.cmu.edu ([128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id VAA25664 for ; Mon, 7 May 2001 21:07:23 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f47Mc5u05434 for ips-outgoing; Mon, 7 May 2001 18:38:05 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from palrel2.hp.com (palrel2.hp.com [156.153.255.234]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f47Mc3H05428 for ; Mon, 7 May 2001 18:38:03 -0400 (EDT) Received: from xparelay2.corp.hp.com (xparelay2.corp.hp.com [15.58.137.112]) by palrel2.hp.com (Postfix) with ESMTP id 94CE312EB; Mon, 7 May 2001 15:38:02 -0700 (PDT) Received: from xatlbh2.atl.hp.com (xatlbh2.atl.hp.com [15.45.89.187]) by xparelay2.corp.hp.com (Postfix) with ESMTP id CA10A1F510; Mon, 7 May 2001 15:27:47 -0400 (EDT) Received: by xatlbh2.atl.hp.com with Internet Mail Service (5.5.2653.19) id ; Mon, 7 May 2001 18:37:59 -0400 Message-ID: <6BD67FFB937FD411A04F00D0B74FE87802A0903B@xrose06.rose.hp.com> From: "KRUEGER,MARJORIE (HP-Roseville,ex1)" To: "'Mark Bakke'" , julian_satran@il.ibm.com Cc: ips@ece.cmu.edu Subject: RE: iSCSI: Nailing down CRC-32C Date: Mon, 7 May 2001 18:37:56 -0400 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2653.19) Content-Type: text/plain; charset="iso-8859-1" Sender: owner-ips@ece.cmu.edu Precedence: bulk Since we are operating over IP, I would think ordering should be "network order" (Big endian). Marj > -----Original Message----- > From: Mark Bakke [mailto:mbakke@cisco.com] > Sent: Monday, May 07, 2001 1:00 PM > To: julian_satran@il.ibm.com > Cc: ips@ece.cmu.edu > Subject: Re: iSCSI: Nailing down CRC-32C > > > Julian- > > That's great; it covers nearly everything. The only thing left > is the byte order; are they taken in the same order Ethernet uses? > > If so, I can generate a few test patterns that our implementations > can all check against. > > Thanks, > > -- > Mark > > julian_satran@il.ibm.com wrote: > > > > The CRC part of the appendix (for 07) reads: > > > > The following table lists cyclic integrity checksums that can be > > negotiated for the digests and MUST be implemented by every iSCSI > > initiator and target. Note that these digest options > have only error > > detection significance. > > > > +---------------------------------------------+ > > | Name | Description | > > +---------------------------------------------+ > > | crc-32C | 32 bit CRC | 11EDC6F41 | > > +---------------------------------------------+ > > | none | no digest | > > +---------------------------------------------+ > > > > The generator polynomials for those digests are given in > hex-notation, > > for example 3a stands for 0011 1010 - the polynomial > x**5+X**4+x**3+x+1. > > > > The generator polynomial selected is evaluated in > [Castagnioli93]. > > When using the CRC the CRC register must be initialized to all 1s > > (0xFFFFFFFF) and the CRC bits must be complemented > before transmission. > > Padding bytes, when presents in a segment covered by a > CRC, have to be > > set to 0 and are included in the CRC. > > > > Regards, > > Julo > > > > Mark Bakke on 05-05-2001 00:13:09 > > > > Please respond to Mark Bakke > > > > To: IPS > > cc: > > Subject: iSCSI: Nailing down CRC-32C > > > > At the interim meeting, it was stated that iSCSI has selected > > the CRC-32C polynomial as its required iSCSI-level header and > > data CRC. Now that we have it, it's time to move on and make > > sure we can implement it. > > > > In the interest of interoperability, we need to not only specify > > the polynomial, but also the initial values, bit and byte > > ordering, etc. > > > > "A Painless Guide to CRC Error Detection Algorithms" > > (http://www.ross.net/crc/crcpaper.html) specifies a > > method to unambiguously characterize these parameters > > (sections 15 and 16). Has anyone taken a shot at defining > > these yet? Otherwise, here is what it might look like: > > > > Name : "CRC-32C" > > Width : 32 > > Poly : 1EDC6F41 (note that the leading "1" is implied) > > Init : FFFFFFFF > > RefIn : True > > RefOut : True > > XorOut : FFFFFFFF > > Check : ? > > > > I haven't attempted to create check data based on these yet. As > > soon as the other parameters are nailed down, we need to do this. > > > > Anyway, I am not a CRC expert, and can't make any statement about > > the above values being the "best" way to do this, but instead just > > copied them (except the polynomial itself) from the Ethernet CRC, > > since that is likely the easiest for everyone implementing hardware > > to deal with. > > > > If someone else is already doing this, let me know; I just wanted > > to start this thread so we can get closure. > > > > Regards, > > > > Mark > > > > -- > > Mark A. Bakke > > Cisco Systems > > mbakke@cisco.com > > 763.398.1054 > > -- > Mark A. Bakke > Cisco Systems > mbakke@cisco.com > 763.398.1054 > From owner-ips@ece.cmu.edu Mon May 7 21:08:00 2001 Received: from ece.cmu.edu ([128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id VAA25702 for ; Mon, 7 May 2001 21:08:00 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f47MnmJ06332 for ips-outgoing; Mon, 7 May 2001 18:49:48 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from dogwood.cisco.com (dogwood.cisco.com [161.44.11.19]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f47MnlH06328 for ; Mon, 7 May 2001 18:49:47 -0400 (EDT) Received: from cisco.com (dhcp-161-44-68-173.cisco.com [161.44.68.173]) by dogwood.cisco.com (8.8.6 (PHNE_14041)/CISCO.SERVER.1.2) with ESMTP id SAA12949; Mon, 7 May 2001 18:49:39 -0400 (EDT) Message-ID: <3AF72609.A407DCAC@cisco.com> Date: Mon, 07 May 2001 17:47:37 -0500 From: Steve Senum X-Mailer: Mozilla 4.73 [en]C-CCK-MCD (Windows NT 5.0; U) X-Accept-Language: en MIME-Version: 1.0 To: "KRUEGER,MARJORIE (HP-Roseville,ex1)" CC: "'Mark Bakke'" , julian_satran@il.ibm.com, ips@ece.cmu.edu Subject: Re: iSCSI: Nailing down CRC-32C References: <6BD67FFB937FD411A04F00D0B74FE87802A0903B@xrose06.rose.hp.com> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-ips@ece.cmu.edu Precedence: bulk Content-Transfer-Encoding: 7bit The bit order within a byte also needs to be specified. Steve Senum "KRUEGER,MARJORIE (HP-Roseville,ex1)" wrote: > > Since we are operating over IP, I would think ordering should be "network > order" (Big endian). > > Marj > > > -----Original Message----- > > From: Mark Bakke [mailto:mbakke@cisco.com] > > Sent: Monday, May 07, 2001 1:00 PM > > To: julian_satran@il.ibm.com > > Cc: ips@ece.cmu.edu > > Subject: Re: iSCSI: Nailing down CRC-32C > > > > > > Julian- > > > > That's great; it covers nearly everything. The only thing left > > is the byte order; are they taken in the same order Ethernet uses? > > > > If so, I can generate a few test patterns that our implementations > > can all check against. > > > > Thanks, > > > > -- > > Mark > > > > julian_satran@il.ibm.com wrote: > > > > > > The CRC part of the appendix (for 07) reads: > > > > > > The following table lists cyclic integrity checksums that can be > > > negotiated for the digests and MUST be implemented by every iSCSI > > > initiator and target. Note that these digest options > > have only error > > > detection significance. > > > > > > +---------------------------------------------+ > > > | Name | Description | > > > +---------------------------------------------+ > > > | crc-32C | 32 bit CRC | 11EDC6F41 | > > > +---------------------------------------------+ > > > | none | no digest | > > > +---------------------------------------------+ > > > > > > The generator polynomials for those digests are given in > > hex-notation, > > > for example 3a stands for 0011 1010 - the polynomial > > x**5+X**4+x**3+x+1. > > > > > > The generator polynomial selected is evaluated in > > [Castagnioli93]. > > > When using the CRC the CRC register must be initialized to all 1s > > > (0xFFFFFFFF) and the CRC bits must be complemented > > before transmission. > > > Padding bytes, when presents in a segment covered by a > > CRC, have to be > > > set to 0 and are included in the CRC. > > > > > > Regards, > > > Julo > > > > > > Mark Bakke on 05-05-2001 00:13:09 > > > > > > Please respond to Mark Bakke > > > > > > To: IPS > > > cc: > > > Subject: iSCSI: Nailing down CRC-32C > > > > > > At the interim meeting, it was stated that iSCSI has selected > > > the CRC-32C polynomial as its required iSCSI-level header and > > > data CRC. Now that we have it, it's time to move on and make > > > sure we can implement it. > > > > > > In the interest of interoperability, we need to not only specify > > > the polynomial, but also the initial values, bit and byte > > > ordering, etc. > > > > > > "A Painless Guide to CRC Error Detection Algorithms" > > > (http://www.ross.net/crc/crcpaper.html) specifies a > > > method to unambiguously characterize these parameters > > > (sections 15 and 16). Has anyone taken a shot at defining > > > these yet? Otherwise, here is what it might look like: > > > > > > Name : "CRC-32C" > > > Width : 32 > > > Poly : 1EDC6F41 (note that the leading "1" is implied) > > > Init : FFFFFFFF > > > RefIn : True > > > RefOut : True > > > XorOut : FFFFFFFF > > > Check : ? > > > > > > I haven't attempted to create check data based on these yet. As > > > soon as the other parameters are nailed down, we need to do this. > > > > > > Anyway, I am not a CRC expert, and can't make any statement about > > > the above values being the "best" way to do this, but instead just > > > copied them (except the polynomial itself) from the Ethernet CRC, > > > since that is likely the easiest for everyone implementing hardware > > > to deal with. > > > > > > If someone else is already doing this, let me know; I just wanted > > > to start this thread so we can get closure. > > > > > > Regards, > > > > > > Mark > > > > > > -- > > > Mark A. Bakke > > > Cisco Systems > > > mbakke@cisco.com > > > 763.398.1054 > > > > -- > > Mark A. Bakke > > Cisco Systems > > mbakke@cisco.com > > 763.398.1054 > > From owner-ips@ece.cmu.edu Mon May 7 22:15:23 2001 Received: from ece.cmu.edu ([128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id WAA27463 for ; Mon, 7 May 2001 22:15:22 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f480Xh813667 for ips-outgoing; Mon, 7 May 2001 20:33:43 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from palrel2.hp.com (palrel2.hp.com [156.153.255.234]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f480XbH13660 for ; Mon, 7 May 2001 20:33:41 -0400 (EDT) Received: from hpcuhe.cup.hp.com (hpcuhe.cup.hp.com [15.0.80.203]) by palrel2.hp.com (Postfix) with ESMTP id E3F90DA2; Mon, 7 May 2001 17:33:36 -0700 (PDT) Received: from cup.hp.com (santoshr@hpindhhm.cup.hp.com [15.8.80.197]) by hpcuhe.cup.hp.com (8.9.3 (PHNE_18979)/8.9.3 SMKit7.02) with ESMTP id RAA01056; Mon, 7 May 2001 17:33:32 -0700 (PDT) Message-ID: <3AF73ED0.E6CB535F@cup.hp.com> Date: Mon, 07 May 2001 17:33:20 -0700 From: Santosh Rao Organization: Hewlett Packard, Cupertino. X-Mailer: Mozilla 4.7 [en] (X11; U; HP-UX B.11.00 9000/778) X-Accept-Language: en MIME-Version: 1.0 To: Stephen Bailey Cc: ips@ece.cmu.edu Subject: Re: iSCSI : Review Feedback on iSCSI Rev 06 References: <200105040201.TAA00579@hpcuhe.cup.hp.com> <20010507130209.0907F94006@sandmail.sandburst.com> Content-Type: multipart/mixed; boundary="------------5F8303D91E14FB80306CC600" Sender: owner-ips@ece.cmu.edu Precedence: bulk This is a multi-part message in MIME format. --------------5F8303D91E14FB80306CC600 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Stephen Bailey wrote: > > Santosh, > > > 3) Why does the new draft limit the size of text commands and nop commands to > > 4096 bytes ? What is the rationale behind the magic number selection of 4096 > > ? > > The reason for a limit like this (I won't argue that we've done it > quite right here, in fact, I may argue against it a little bit below), > is for to allow the receiving endpoints to have only extremely modest, > tightly bounded resources for handling this non-nominal (or, at least, > non-performance) traffic. > > DataPDULength is way too large a limit. I'm still optimistic that the > maximum DataPDULength limit will be relaxed back towards 4G, even > though I know that y'all have decided against it and everybody out > there thinks I'm on dust (you'll see...). Steph, I agree that an upper limit is required on the text/login/nop commands. What I was pointing out was that this upper limit needs to be lower than DataPDULength, failing which fragmentation can occur. If it is felt that explicitly defined keys that constitute such limits is over-kill due to these PDUs not being in performance path, that's fine. All we then need is to say : non-scsi command PDUs commands MUST be limited to a size of : min(4096, DataPDULength). This ensures that no fragmentation occurs when DataPDULength is < 4K. Regarding the separate problem that arises from an unbound response to a command PDU [such as a SendTargets], the iterator proposal results in more "wire traffic" during target discovery than the use of a single text command and target controlling the data returned. Multiple sets of data from the target can then be re-assembled into a single host buffer. This is the way the FC-GS-2/FC-GS-3 GID_FT command operates and it allows for a faster target discovery than the iterative use of other FC-GS-2/GS-3 name server queries such as GA_NXT (GAN). However, both methods could be specified by the draft with implementations choosing either an iterator based scheme or a bulk response scheme. Regards, Santosh > The limit that makes slightly more sense from an engineering > standpoint, but makes no sense whatsoever from a standards standpoint, > is FirstBurstSize. In a hardware implementation, DataPDULength is a > property associated with the hardware data handling path. The data > arriving in DataPDULength units is destined for buffers which are > known to have already been allocated and supplied from the upper > level. The resource cost for these buffers is a direct function of > the application, as opposed to iSCSI overhead. > > The Login[Response], Text and Ping PDUs, and probably (but not > definitely) unsolicited data, flow through a path where the buffers > are iSCSI implementation overhead. > > iSCSI must provide a mechanism for reducing this resource overhead to > an arbitrary, manageable size. The alternative is to require an iSCSI > implementation to allocate a gratuitously large quantity of memory to > handle occurences which a) may rarely (probably never) happen b) > don't provide the implementation anything at all worthwhile in return > for that memory. > > Because this is a non-performance path, having this parameter being > negotiable is also a waste of effort. > > The iSCSI negotiation protocol should be (it mostly already is) > designed to tightly bound the worst-case (largest) text payload in a > Text or Login[Response]. To do this we must: > 1) determine the maximum text payload size which contains all > `fixed' length keys > 2) create an iterator protocol for arbitrary, or even just large > text messages. This includes X-* parameters---an X-* exchange > must be structured to work within the max text PDU limits. In > addition the X-* parameter definitions must create their own > iterator if necessary---iSCSI only needs to handle iSCSI > specified keys. > > Looking over the present set of text keys, the security exchanges > already have tight size bounds, and clearly aren't going to be the > limiting factor in the maximal text payload. That leaves the > operation parameters. Most of the operational PDUs also have a small > maximum fixed size. The exceptions are: > InitiatorName > InitiatorAlias > TargetName > TargetAddress > TargetAlias > > We can probably put a modest bound on each of these keys individually > (e.g. 512 bytes), which means that the maximum text payload size > within a Login exchange is tightly bounded and 4K seems adequate. > > The problem comes with SendTargets=. That is why I proposed an > iterator protocol for SendTargets= instead of having the target > information gush back from the target uncontrolled. Simply bounding > the maximum text payload and allowing and arbitrary sequence of text > payloads to be sent in response to a single request doesn't cut it. > We need to be able to bound the total amount of data returned from > each particular request. > > SendTargets= actually has two problems. 1) an arbitrary number of > targets per endpoint, 2) an arbitrary number of addresses per target. > Given the second problem, unfortunately, it's not adequate to merely > have a target iterator, since the data for each target may be > arbitrarily long. Therefore, here's another proposal which might be > close to the mark: > > Initiator keys: > SendTargetStart= > Send first PDU of target list information > If already in the midst of a target list exchange, restart at > the beginning > SendTargetMore= > Send next PDU of target list information > SendTargetEnd= > Terminate a target list exchange > > Target Keys: > TargetName=name > one per target > TargetAddress=address > one or more per target > TargetAlias=alias > one per target (may be empty) > TargetMore= > more target information remaining within the present target > > A target list exchange always terminates with a Text PDU with final=1. > In the case that the exchange is ended by a SendTargetEnd=, the > target's Text PDU with final=1 would have no payload. > > We would specify at most one target list request outstanding per > session. Personally, I'd like to be even more restrictive and say > that target list exchanges are only allowed on single connection > sessions where a TargetName was not specified at login. However, Mark > Bakke seemed to think that rather than a session with no TargetName, > it should be a session with TargetName=iSCSI. Either way is fine with > me, assuming that we do not want to allow a regular, operational login > with TargetName=iSCSI (e.g. for boot, or simple initiators). I happen > to think that operational login with TargetName=iSCSI is useful. The > fact is, many boxes, even big ones, will have only a single target, > but many LUNs. This corresponds to present storage boxes (as opposed > to bridges). > > Steph --------------5F8303D91E14FB80306CC600 Content-Type: text/x-vcard; charset=us-ascii; name="santoshr.vcf" Content-Description: Card for Santosh Rao Content-Disposition: attachment; filename="santoshr.vcf" Content-Transfer-Encoding: 7bit begin:vcard n:Rao;Santosh tel;work:408-447-3751 x-mozilla-html:FALSE org:Hewlett Packard, Cupertino.;SISL adr:;;19420, Homestead Road, M\S 43LN, ;Cupertino.;CA.;95014.;USA. version:2.1 email;internet:santoshr@cup.hp.com title:Software Design Engineer x-mozilla-cpt:;21088 fn:Santosh Rao end:vcard --------------5F8303D91E14FB80306CC600-- From owner-ips@ece.cmu.edu Mon May 7 22:16:59 2001 Received: from ece.cmu.edu ([128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id WAA27488 for ; Mon, 7 May 2001 22:16:59 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f47NpHq10749 for ips-outgoing; Mon, 7 May 2001 19:51:17 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from palrel1.hp.com (palrel1.hp.com [156.153.255.242]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f47NpFH10728 for ; Mon, 7 May 2001 19:51:15 -0400 (EDT) Received: from hpcuhe.cup.hp.com (hpcuhe.cup.hp.com [15.0.80.203]) by palrel1.hp.com (Postfix) with ESMTP id 5D08210DE; Mon, 7 May 2001 16:51:14 -0700 (PDT) Received: from cup.hp.com (santoshr@hpindhhm.cup.hp.com [15.8.80.197]) by hpcuhe.cup.hp.com (8.9.3 (PHNE_18979)/8.9.3 SMKit7.02) with ESMTP id QAA28390; Mon, 7 May 2001 16:51:09 -0700 (PDT) Message-ID: <3AF734E1.9C8CFE9B@cup.hp.com> Date: Mon, 07 May 2001 16:50:57 -0700 From: Santosh Rao Organization: Hewlett Packard, Cupertino. X-Mailer: Mozilla 4.7 [en] (X11; U; HP-UX B.11.00 9000/778) X-Accept-Language: en MIME-Version: 1.0 To: John Hufferd Cc: ips@ece.cmu.edu Subject: Re: iSCSI : digest error handling violates EMDP/InDataOrder References: Content-Type: multipart/mixed; boundary="------------214F5077655FA0C86428F0AA" Sender: owner-ips@ece.cmu.edu Precedence: bulk This is a multi-part message in MIME format. --------------214F5077655FA0C86428F0AA Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit John, The fundamental question is : "Does EMDP apply to WRITE commands ?" The answer for other SCSI transports is yes. [from my queries on t11 and looking at SRP's defn.] If EMDP is applicable to WRITEs, then, that does'nt come across clear enough in the current iSCSI definition of this bit. If it is applicable to WRITEs, does it control of the scsi command, or order within a sequence (as in data PDUs sent in response to an R2T) ? (This is'nt clear from the defn either.) If EMDP does apply to WRITEs, earlier discussion in this thread that the EMDP ordering applies within a sequence contradicts the following text in Section 2.7.5 : "Output data within a burst MUST be delivered in increasing buffer offset order." To summarize the changes required : 1. A clear defn for EMDP is required that states that EMDP applies to both WRITEs and READs and governs the order of data within a scsi command [not a sequence]. " When EMDP is set to 0 : - For a READ, the target MUST send all the data PDUs within the scsi command in continuous increasing order. - For a WRITE, the target MUST send R2T requests in continuous increasing order. When EMDP is set to 1 : - For a READ, the data PDUs within a SCSI command may be delivered out of order. - For a WRITE, the target may send R2T requests out of order." 2. Re-phrase the following in Section 6.2 : "If the error is a Data-Digest-Error in a Data-PDU, the targets MUST either request retransmission with a R2T or answer with a command response PDU with a response-code of delivery-subsystem-failure and terminate the task." to : "If the error is a Data-Digest-Error in a data-PDU, the target MUST perform either one of the following actions : - Request re-transmission with a R2T, provided DataOrder was negotiated to "No". - Answer with a command response PDU with a response-code of delivery-subsystem-failure and terminate the task." 3. Specify clearly that the "Random relative Offset" feature is prohibited by stating : " All Data PDUs within a data sequence in response to an R2T MUST be delivered in increasing buffer offset order." Regards, Santosh John Hufferd wrote: > > Santosh, > I have re-read 2.7.5 a number of times, and can not understand your point. > To me that section clearly states that the data can be sent out of order if > permitted by the EMDP bit. But any individual bursts, must be in order > within that burst/R2T response. The only thing that can be out of order, > is the data sent in different bursts, with respect to each other. > That seems clear, so I must be missing some important point that you are trying > to make. If you think I have this all wrong please let me know. > > . > . > . > John L. Hufferd --------------214F5077655FA0C86428F0AA Content-Type: text/x-vcard; charset=us-ascii; name="santoshr.vcf" Content-Description: Card for Santosh Rao Content-Disposition: attachment; filename="santoshr.vcf" Content-Transfer-Encoding: 7bit begin:vcard n:Rao;Santosh tel;work:408-447-3751 x-mozilla-html:FALSE org:Hewlett Packard, Cupertino.;SISL adr:;;19420, Homestead Road, M\S 43LN, ;Cupertino.;CA.;95014.;USA. version:2.1 email;internet:santoshr@cup.hp.com title:Software Design Engineer x-mozilla-cpt:;21088 fn:Santosh Rao end:vcard --------------214F5077655FA0C86428F0AA-- From owner-ips@ece.cmu.edu Tue May 8 00:08:17 2001 Received: from ece.cmu.edu ([128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id AAA00176 for ; Tue, 8 May 2001 00:08:16 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f4833Rd23800 for ips-outgoing; Mon, 7 May 2001 23:03:27 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from mxbh4.isus.emc.com (mxbh4.isus.emc.com [128.221.10.33]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f4833PH23795 for ; Mon, 7 May 2001 23:03:25 -0400 (EDT) Received: by mxbh4.isus.emc.com with Internet Mail Service (5.5.2650.21) id ; Mon, 7 May 2001 23:03:20 -0400 Message-ID: <0F31E5C394DAD311B60C00E029101A0708015538@corpmx9.isus.emc.com> From: Black_David@emc.com To: santoshr@cup.hp.com, Black_David@emc.com, ips@ece.cmu.edu Subject: RE: iSCSI : login keys & mode page settings Date: Mon, 7 May 2001 23:03:13 -0400 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2650.21) Content-Type: text/plain; charset="iso-8859-1" Sender: owner-ips@ece.cmu.edu Precedence: bulk Hmm ... I'm quite capable of talking myself around in a circle on this one because there are two issues mixed in here: (1) Is changing an iSCSI key allowed to cause problems for other initiators? (2) Does the iSCSI key mechanism have to behave identically to the corresponding mode page mechanism. Given the need to support old systems that may get (1) wrong (mode page sets can damage other initiators), the best we may be able to do on it is a SHOULD: (1) SHOULD not share key values among sessions (i.e., setting of a key value in one session SHOULD NOT affect the setting in any other session. On (2), how about - When a key refers to a mode page entry, the underlying value MUST be shared between the mode page and the key in an iSCSI session (e.g., a value set by a text key MUST be retrieved by the mode page if the implementation accepted the value). - Restrictions on value changes in full-feature mode SHOULD (MUST?) match when a value is shared between a text key and a mode page entry. Comments? --David --------------------------------------------------- David L. Black, Senior Technologist EMC Corporation, 42 South St., Hopkinton, MA 01748 +1 (508) 435-1000 x75140 FAX: +1 (508) 497-8500 black_david@emc.com Mobile: +1 (978) 394-7754 --------------------------------------------------- > -----Original Message----- > From: Santosh Rao [SMTP:santoshr@cup.hp.com] > Sent: Friday, May 04, 2001 2:15 PM > To: Black_David@emc.com; ips@ece.cmu.edu > Subject: RE: iSCSI : login keys & mode page settings > > David, > > Apologies for the late response on this. I was hoping we could complete > this > thread of discussion at Nashua, but for lack of time, we are back on the > list. > > Regarding your question below : > > > If one Initiator can damage another in this fashion, then we > > may indeed have a problem. > > > Comments?, > > Shared mode page implementations in targets is quite common and > modification of > control parameters through a mode select would indeed affect all other > initiators logged into the target. This is not desirable behaviour and > iSCSI > may be better served using login/text based key negotiation rather than > the > mode select mechanisms which opens it up to the side effects of affecting > all > connected initiators. > > Thanks, > Santosh > > > ------------------------------------------------------------------------ > Subject: RE: iSCSI : login keys & mode page settings > From: Black_David@emc.com > Date: Fri, 20 Apr 2001 20:32:17 -0400 > Cc: ips@ece.cmu.edu > > I'm not sure -- this sounds somewhat like the > old principle of not asking why there's a hole > in one's foot when one has aimed the gun at > it and pulled the trigger. For the tape > example, if some tape driver changes a > Target iSCSI parameter that disrupts that > driver's own tape I/O in a fashion that the > driver can't recover from, I think it's > clear where the fault lies. If one Initiator > can damage another in this fashion, then we > may indeed have a problem. > > Comments?, > --David > > > -----Original Message----- > > From: Santosh Rao [SMTP:santoshr@cup.hp.com] > > Sent: Friday, April 20, 2001 8:09 PM > > To: Black_David@emc.com > > Cc: ips@ece.cmu.edu > > Subject: Re: iSCSI : login keys & mode page settings > > > > David, > > > > Some clarification on the basis for classifying login > ould > > also be helpful. Should login keys that can disrupt > > I/O on their change > > be allowed to be non-LO ? > > > > Thanks, > > Santosh > > > > Black_David@emc.com wrote: > > > > > > Without getting into the technical details of the > > > discussion, I have a couple of observations: > > > > > > (A) The issue of whether to allow mode page > > > access to and modification of iSCSI > ers > > > will need to be taken up at the interim > > > meeting. IMHO, access seems like a good > > > idea, so that SCSI-generic code that doesn't > > > know specifically about iSCSI can find > > > what it expects where it expects it, but > > > I'm unsure about modification because it > > > may carry a risk of code that's > naware > > > getting something wrong. The mode page > > > commands should be transparent to iSCSI. > > > > > > (B) The mode page and text key mechanisms have > > > to access the same data. Section 3 of the > > > -06 version says this, but needs some > > > editing > > > to enforce it by using "MUST" or its > > > equivalent > > > (cf. RFC 2119). This is to prevent an > > > implementation from having two instances of > > > the same parameter - one for the mode page and > > > one for the text keys - which would be a bad > > > thing. > > > > > > --David > > -- > ################################# > Santosh Rao > Software Design Engineer, > HP, Cupertino. > email : santoshr@cup.hp.com > Phone : 408-447-3751 > ################################# From owner-ips@ece.cmu.edu Tue May 8 00:10:00 2001 Received: from ece.cmu.edu ([128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id AAA00233 for ; Tue, 8 May 2001 00:10:00 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f482qOL23006 for ips-outgoing; Mon, 7 May 2001 22:52:24 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from mxic1.isus.emc.com ([168.159.129.100]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f482qMH22992 for ; Mon, 7 May 2001 22:52:22 -0400 (EDT) Received: by MXIC1 with Internet Mail Service (5.5.2650.21) id ; Mon, 7 May 2001 22:53:53 -0400 Message-ID: <0F31E5C394DAD311B60C00E029101A0708015537@corpmx9.isus.emc.com> From: Black_David@emc.com To: hafner@almaden.ibm.com, marjorie_krueger@hp.com Cc: ips@ece.cmu.edu Subject: RE: iSCSI: multiple sessions b/n a pair of WWUIs. Date: Mon, 7 May 2001 22:52:08 -0400 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2650.21) Content-Type: text/plain Sender: owner-ips@ece.cmu.edu Precedence: bulk While Jim's correct ... in Marj's defense, what she wrote is a fairly obvious way to implement it, and that might be worth noting in the draft. --David --------------------------------------------------- David L. Black, Senior Technologist EMC Corporation, 42 South St., Hopkinton, MA 01748 +1 (508) 435-1000 x75140 FAX: +1 (508) 497-8500 black_david@emc.com Mobile: +1 (978) 394-7754 --------------------------------------------------- > -----Original Message----- > From: Jim Hafner [SMTP:hafner@almaden.ibm.com] > Sent: Monday, May 07, 2001 10:44 PM > To: KRUEGER,MARJORIE (HP-Roseville,ex1) > Cc: ips@ece.cmu.edu > Subject: RE: iSCSI: multiple sessions b/n a pair of WWUIs. > > > Marj, > > I think you've stretched my words too far. The requirement is for a > single > logical unit to recover enough identity information provided through the > I_T_L nexus to reestablish the reservation. That means that the name+ISID > must be unique *from the viewpoint of a single iSCSI target device*. So > it's more like the > "initiator must have unique ISID for all sessions with all > targets any given target device". > > The best way to think about this is the following: > 1) only a single target's (actually logical unit's) view of the initiator > counts in this regard > 2) there are no rules in SCSI concerning multi-target device interactions > In other words, for these rules, look only from inside a target going out. > > Jim Hafner > > > "KRUEGER,MARJORIE (HP-Roseville,ex1)" > @ece.cmu.edu > on 05/04/2001 05:39:35 PM > > Sent by: owner-ips@ece.cmu.edu > > > To: "'Matt Wakeley'" , ips@ece.cmu.edu > cc: > Subject: RE: iSCSI: multiple sessions b/n a pair of WWUIs. > > > > > One clarification: these are unique between any I-T *pair*. > > (you make it sound like an initiator must have a unique ISID > > for all sessions > > with all targets) > > > > -Matt > > The ISID must be unique within the iSCSI layer on an iSCSI device - that > amounts to "initiator must have unique ISID for all sessions with all > targets". > > To quote Jim "The requirement in name+disc that a given initiator name > cannot reuse an ISID for two different sessions comes as a consequence a > number of things (which are described in the draft). The gist is that > this > is needed to provide the correct context for restoration of reservations > state (and other nexus state) to a particular nexus after logout/login. In > other words, if the session goes down for some reason, the target needs > clear context to restore nexus state to a rebuilt session. The only tool > it > has is uniqueness of Name+ISID combination within its name space." > > -Marj > > From owner-ips@ece.cmu.edu Tue May 8 00:13:14 2001 Received: from ece.cmu.edu ([128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id AAA00319 for ; Tue, 8 May 2001 00:13:13 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f482jo222512 for ips-outgoing; Mon, 7 May 2001 22:45:50 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from e34.bld.us.ibm.com (e34.co.us.ibm.com [32.97.110.132]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f482jlH22507 for ; Mon, 7 May 2001 22:45:47 -0400 (EDT) Received: from westrelay02.boulder.ibm.com (westrelay02.boulder.ibm.com [9.99.140.23]) by e34.bld.us.ibm.com (8.9.3/8.9.3) with ESMTP id WAA58466; Mon, 7 May 2001 22:28:22 -0400 Received: from d03nmx42.almaden.ibm.com (d03nmx42.almaden.ibm.com [9.1.24.146]) by westrelay02.boulder.ibm.com (8.11.1/NCO v4.96.1.0) with ESMTP id f482iRr97506; Mon, 7 May 2001 20:44:27 -0600 Importance: Normal Subject: RE: iSCSI: multiple sessions b/n a pair of WWUIs. To: "KRUEGER,MARJORIE (HP-Roseville,ex1)" Cc: ips@ece.cmu.edu From: "Jim Hafner" Date: Mon, 7 May 2001 19:44:25 -0700 Message-ID: X-MIMETrack: Serialize by Router on D03NMX42/03/M/IBM(Beta 1 M8_03292001|March 29, 2001) at 05/07/2001 07:44:27 PM MIME-Version: 1.0 Content-type: text/plain; charset=us-ascii Sender: owner-ips@ece.cmu.edu Precedence: bulk Marj, I think you've stretched my words too far. The requirement is for a single logical unit to recover enough identity information provided through the I_T_L nexus to reestablish the reservation. That means that the name+ISID must be unique *from the viewpoint of a single iSCSI target device*. So it's more like the "initiator must have unique ISID for all sessions with all targets any given target device". The best way to think about this is the following: 1) only a single target's (actually logical unit's) view of the initiator counts in this regard 2) there are no rules in SCSI concerning multi-target device interactions In other words, for these rules, look only from inside a target going out. Jim Hafner "KRUEGER,MARJORIE (HP-Roseville,ex1)" @ece.cmu.edu on 05/04/2001 05:39:35 PM Sent by: owner-ips@ece.cmu.edu To: "'Matt Wakeley'" , ips@ece.cmu.edu cc: Subject: RE: iSCSI: multiple sessions b/n a pair of WWUIs. > One clarification: these are unique between any I-T *pair*. > (you make it sound like an initiator must have a unique ISID > for all sessions > with all targets) > > -Matt The ISID must be unique within the iSCSI layer on an iSCSI device - that amounts to "initiator must have unique ISID for all sessions with all targets". To quote Jim "The requirement in name+disc that a given initiator name cannot reuse an ISID for two different sessions comes as a consequence a number of things (which are described in the draft). The gist is that this is needed to provide the correct context for restoration of reservations state (and other nexus state) to a particular nexus after logout/login. In other words, if the session goes down for some reason, the target needs clear context to restore nexus state to a rebuilt session. The only tool it has is uniqueness of Name+ISID combination within its name space." -Marj From owner-ips@ece.cmu.edu Tue May 8 00:14:37 2001 Received: from ece.cmu.edu ([128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id AAA00337 for ; Tue, 8 May 2001 00:14:37 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f4826NQ19885 for ips-outgoing; Mon, 7 May 2001 22:06:23 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from gateway.sanlight.org (adsl-63-202-160-80.dsl.snfc21.pacbell.net [63.202.160.80]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f4826KH19878 for ; Mon, 7 May 2001 22:06:20 -0400 (EDT) Received: from ljoy (10.0.0.18.lan.sanlight.net [10.0.0.18]) by gateway.sanlight.org (8.11.0/8.11.0) with SMTP id f483D7154583; Mon, 7 May 2001 20:13:08 -0700 (PDT) (envelope-from dotis@sanlight.net) From: "Douglas Otis" To: , , , Cc: Subject: RE: iSCSI Security rough consensus Date: Mon, 7 May 2001 19:03:30 -0700 Message-ID: MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 (Normal) X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook IMO, Build 9.0.2416 (9.0.2911.0) X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4522.1200 Importance: Normal In-reply-to: Sender: owner-ips@ece.cmu.edu Precedence: bulk Content-Transfer-Encoding: 7bit Somesh, You have it correct. The general intent of an end to end integrity check is to do something light weight within software to catch a myriad of errors that crop up due to various hardware failures (R*****k and M*******h as example), as well as bugs within variations of the theme called TCP. One can only hope such checking automation does not introduce corrupt data as a result. New hardware/software MUST maintain integrity. There is a history that gives one pause however. Doug > I was trying not to react to the string of messages, but > this message finally did it(and nothing wrong with the message > itself and no offense meant to the author). What the message > implies is that no intermediate hops can be trusted and that > integrity must be end-to-end. > > Taken to the literal extreme, this means that the system must > at the application level (in the host) generate some integrity > checks which should then be verified every > time the data is used (perhaps by an end application again). > > Even with the CRC, the following errors will go undetected, > unless protected by some other means (well engineered, well > tested devices) > > 1. Host software. > 2. Host memory to adapter path > 3. Adapter memory > 4. Adapter software > 5. Any virtualization device in the middle including > software, memory, fabric etc on the device including > any fibre-channel/scsi storage at the back end > (assuming they will recreate resegment PDUs) > 6. Any target adapter & adapter memory > 7. Any paths within the target > 8. The storage itself > 9. and then the reverse path > > If end-to-end integrity is a must, it seems that the TCP checksum > offload and iSCSI CRC in off-board hardware must be avoided. > > Or did I miss something somewhere. > > Somesh > > > -----Original Message----- > > From: vince_cavanna@agilent.com [mailto:vince_cavanna@agilent.com] > > Sent: Monday, May 07, 2001 2:13 PM > > To: mbakke@cisco.com; Black_David@emc.com > > Cc: someshg@yahoo.com; ips@ece.cmu.edu; vince_cavanna@agilent.com > > Subject: RE: iSCSI Security rough consensus > > > > > > Here is a paper by well-known authors that supports Mark's > reasoning that > > the end-to-end iSCSI check is indispensable even when IPsec is on .... > > > > Jonathan Stone and Craig artridge in "When the CRC and TCP checksum > > disagree" study the sources of errors that are not caught by link level > > checks and that therefore stress a higher layer error check > > mechansim. Such > > errors are quite prevalent and one signifcant source of such > errors is the > > end-host itself. The authors emphasize that "hardware must not > be trusted" > > and that "many encryption solutions such as IPsec do not provide > > additional > > (over existing checks) protection because the encryption is > > applied too late > > in the transmission process, often after the data has passed > through a DMA > > engine. Rather the application must add the checksum before > > handing its data > > to TCP (ala SSL)". The authors recommend that "for truly > valuable data (in > > my opinion all data handled by iSCSI) the application should add > > a stronger > > application-level checksum". > > > > Vince > > > > |-----Original Message----- > > |From: Mark Bakke [mailto:mbakke@cisco.com] > > |Sent: Friday, May 04, 2001 1:04 PM > > |To: Black_David@emc.com > > |Cc: someshg@yahoo.com; ips@ece.cmu.edu > > |Subject: Re: iSCSI Security rough consensus > > | > > | > > |Black_David@emc.com wrote: > > |> > > |> > Sure would be nice if we could make up our minds and just > > |> > implement one mechanism. > > |> > > > |> > Here we have two mechanisms (iSCSI header/data integrity > > |> > and ESP) which are both mandatory to implement and > > |> > optional to use. Since ESP seems like a superset why not > > |> > just have that and get rid of the "integrity only" iSCSI > > |> > CRC mechanism. > > |> > > |> It sure would be nice, and in fact we had almost > > |> exactly this discussion later in the evening as > > |> part of the error recovery section of the agenda. > > |> The fly in the ointment is that the HMAC integrity > > |> algorithm that is at the core of ESP's integrity > > |> support is considerably more expensive (software > > |> or hardware) than a CRC, and this isn't likely > > |> to improve as I understand things. I would expect > > |> to see implementations with ESP completely in > > |> software and visible performance impacts. > > | > > |That's just part of the reason behind having both. The other is > > |that most implementations won't run IPsec end-to-end; either IPsec > > |is provided in an external device, or even in an iSCSI gateway. > > |In the latter case, all layers are removed and replaced, including > > |iSCSI. Only the SCSI-level information (data, CDBs) go all the > > |way end-to-end. Since iSCSI can CRC the SCSI-level data, it can > > |provide the data integrity that will keep our customers happy. > > | > > |The use of the iSCSI CRC is the minimum requirement; adding the > > |IPsec-level integrity check strengthens it, and can simplify error > > |recovery over a not-so-good or untrusted network. > > | > > |-- > > |Mark > > | > > |> > > |> I really need to get the meeting minutes written up :-). > > |> > > |> Thanks, > > |> --David > > |> > > |> --------------------------------------------------- > > |> David L. Black, Senior Technologist > > |> EMC Corporation, 42 South St., Hopkinton, MA 01748 > > |> +1 (508) 435-1000 x75140 FAX: +1 (508) 497-8500 > > |> black_david@emc.com Mobile: +1 (978) 394-7754 > > |> --------------------------------------------------- > > | > > |-- > > |Mark A. Bakke > > |Cisco Systems > > |mbakke@cisco.com > > |763.398.1054 > > | > > > _________________________________________________________ > Do You Yahoo!? > Get your free @yahoo.com address at http://mail.yahoo.com > > From owner-ips@ece.cmu.edu Tue May 8 01:59:25 2001 Received: from ece.cmu.edu ([128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id BAA07462 for ; Tue, 8 May 2001 01:59:24 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f483xqD27460 for ips-outgoing; Mon, 7 May 2001 23:59:52 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from e32.bld.us.ibm.com (e32.co.us.ibm.com [32.97.110.130]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f483xpH27453 for ; Mon, 7 May 2001 23:59:51 -0400 (EDT) Received: from westrelay02.boulder.ibm.com (westrelay02.boulder.ibm.com [9.99.140.23]) by e32.bld.us.ibm.com (8.9.3/8.9.3) with ESMTP id AAA63632; Tue, 8 May 2001 00:00:29 -0400 Received: from f4n49e (d03nm065h.boulder.ibm.com [9.99.140.49]) by westrelay02.boulder.ibm.com (8.11.1/NCO v4.96.1.0) with ESMTP id f483wXr205938; Mon, 7 May 2001 21:58:33 -0600 X-Priority: 1 (High) Importance: Normal Subject: RE: iSCSI: multiple sessions b/n a pair of WWUIs. To: Black_David@emc.com Cc: ips@ece.cmu.edu X-Mailer: Lotus Notes Release 5.0.3 (Intl) 21 March 2000 Message-ID: From: "John Hufferd" Date: Mon, 7 May 2001 20:58:14 -0700 X-MIMETrack: Serialize by Router on D03NM065/03/M/IBM(Release 5.0.6 |December 14, 2000) at 05/07/2001 09:58:33 PM MIME-Version: 1.0 Content-type: text/plain; charset=us-ascii Sender: owner-ips@ece.cmu.edu Precedence: bulk David, let me argue another side. I was about to send my version of what Jim said, but I think Jim, as usual was "dead on". It is important to be completely correct here. This direction seemed to be having an influence on MIBs and other things. Though it is possible to do what you suggest, I would say that doing so causes unintentional thought problems. That is, it is important that the Initiator Side (or the Target Side) have a way that the Management/Configuration Software can tell the iSCSI Port what the iSCSI Node Name is, and what ISID value (or Range) should be used. This is true if the iSCSI Port is a SW entity or a HW iSCSI Host Bus Adapter (HBA). It is much simpler for an iSCSI Port to use one ISID for every session it creates to any target, then to have an algorithm for deriving one. (Yes, I know someone has to do that, but I think it should be done at the Configuration Software level instead of at the iSCSI Port level.) Also, since a second session by the same iSCSI Initiator Port, to the same iSCSI Target Port is not, in my opinion "legal", it is not clear to me that any given iSCSI Port needs any more then one ISID. If we say anything else, I think we risk the danger of mixing the concept of Wedge Driver in with the concept of iSCSI Port, and that will do us all a disservice. Some folks are going to want to define Wedge Drivers, and there should be no interfering thought that some how the same iSCSI Initiator Port can have Multiple Sessions to the same iSCSI Target Port (this is the domain of Multiple Connections per Session). Wedge Drivers, if needed, should be seen as something that span iSCSI Initiator Ports and address specific iSCSI Target Ports. (Implementations may blur the levels here, but the concepts should be clear.) Therefore, I suggest that we Not put any such notes, like you suggest, in the draft, but in fact encourage a single ISID/TSID per iSCSI Initiator/Target Port. . . . John L. Hufferd Senior Technical Staff Member (STSM) IBM/SSG San Jose Ca (408) 256-0403, Tie: 276-0403, eFax: (408) 904-4688 Internet address: hufferd@us.ibm.com Black_David@emc.com@ece.cmu.edu on 05/07/2001 07:52:08 PM Sent by: owner-ips@ece.cmu.edu To: Jim Hafner/Almaden/IBM@IBMUS, marjorie_krueger@hp.com cc: ips@ece.cmu.edu Subject: RE: iSCSI: multiple sessions b/n a pair of WWUIs. While Jim's correct ... in Marj's defense, what she wrote is a fairly obvious way to implement it, and that might be worth noting in the draft. --David --------------------------------------------------- David L. Black, Senior Technologist EMC Corporation, 42 South St., Hopkinton, MA 01748 +1 (508) 435-1000 x75140 FAX: +1 (508) 497-8500 black_david@emc.com Mobile: +1 (978) 394-7754 --------------------------------------------------- > -----Original Message----- > From: Jim Hafner [SMTP:hafner@almaden.ibm.com] > Sent: Monday, May 07, 2001 10:44 PM > To: KRUEGER,MARJORIE (HP-Roseville,ex1) > Cc: ips@ece.cmu.edu > Subject: RE: iSCSI: multiple sessions b/n a pair of WWUIs. > > > Marj, > > I think you've stretched my words too far. The requirement is for a > single > logical unit to recover enough identity information provided through the > I_T_L nexus to reestablish the reservation. That means that the name+ISID > must be unique *from the viewpoint of a single iSCSI target device*. So > it's more like the > "initiator must have unique ISID for all sessions with all > targets any given target device". > > The best way to think about this is the following: > 1) only a single target's (actually logical unit's) view of the initiator > counts in this regard > 2) there are no rules in SCSI concerning multi-target device interactions > In other words, for these rules, look only from inside a target going out. > > Jim Hafner > > > "KRUEGER,MARJORIE (HP-Roseville,ex1)" > @ece.cmu.edu > on 05/04/2001 05:39:35 PM > > Sent by: owner-ips@ece.cmu.edu > > > To: "'Matt Wakeley'" , ips@ece.cmu.edu > cc: > Subject: RE: iSCSI: multiple sessions b/n a pair of WWUIs. > > > > > One clarification: these are unique between any I-T *pair*. > > (you make it sound like an initiator must have a unique ISID > > for all sessions > > with all targets) > > > > -Matt > > The ISID must be unique within the iSCSI layer on an iSCSI device - that > amounts to "initiator must have unique ISID for all sessions with all > targets". > > To quote Jim "The requirement in name+disc that a given initiator name > cannot reuse an ISID for two different sessions comes as a consequence a > number of things (which are described in the draft). The gist is that > this > is needed to provide the correct context for restoration of reservations > state (and other nexus state) to a particular nexus after logout/login. In > other words, if the session goes down for some reason, the target needs > clear context to restore nexus state to a rebuilt session. The only tool > it > has is uniqueness of Name+ISID combination within its name space." > > -Marj > > From owner-ips@ece.cmu.edu Tue May 8 07:18:00 2001 Received: from ece.cmu.edu ([128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id HAA18799 for ; Tue, 8 May 2001 07:18:00 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f489TCX27799 for ips-outgoing; Tue, 8 May 2001 05:29:12 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from d12lmsgate.de.ibm.com (d12lmsgate.de.ibm.com [195.212.91.199]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f489T9H27795 for ; Tue, 8 May 2001 05:29:09 -0400 (EDT) Received: from d12relay01.de.ibm.com (d12relay01.de.ibm.com [9.165.215.22]) by d12lmsgate.de.ibm.com (1.0.0) with ESMTP id LAA105848; Tue, 8 May 2001 11:28:04 +0200 From: julian_satran@il.ibm.com Received: from d12mta02.de.ibm.com (d12mta01_cs0 [9.165.222.237]) by d12relay01.de.ibm.com (8.8.8m3/NCO v4.96) with SMTP id LAA179652; Tue, 8 May 2001 11:28:03 +0200 Received: by d12mta02.de.ibm.com(Lotus SMTP MTA v4.6.5 (863.2 5-20-1999)) id C1256A46.003400FA ; Tue, 8 May 2001 11:28:01 +0200 X-Lotus-FromDomain: IBMIL@IBMDE To: internet-drafts@ietf.org cc: ips@ece.cmu.edu, bassoon@yogi.ece.cmu.edu, Dafna_Sheinwald@il.ibm.com Message-ID: Date: Tue, 8 May 2001 12:32:41 +0300 Subject: new CRC draft Mime-Version: 1.0 Content-type: text/plain; charset=us-ascii Content-Disposition: inline Sender: owner-ips@ece.cmu.edu Precedence: bulk On behalf of a group of authors from several organizations I submit an informational IETF-draft for immediate publication. It analyzes several options for non-cryptographyc data integrity to be used by iSCSI and the file name is "draft-sheinwald-iSCSI-CRC-00.txt". The draft can be found at: http://www.haifa.il.ibm.com/satran/ips/draft-sheinwald-iSCSI-CRC-00.txt Julian Satran - IBM Research Laboratory at Haifa From owner-ips@ece.cmu.edu Tue May 8 09:39:37 2001 Received: from ece.cmu.edu ([128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id JAA21957 for ; Tue, 8 May 2001 09:39:37 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f48BhAI05582 for ips-outgoing; Tue, 8 May 2001 07:43:10 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from patan.sun.com (patan.Sun.COM [192.18.98.43]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f48BC0H03717 for ; Tue, 8 May 2001 07:12:00 -0400 (EDT) Received: from centralmail1.Central.Sun.COM ([129.147.62.10]) by patan.sun.com (8.9.3+Sun/8.9.3) with ESMTP id EAA04879 for ; Tue, 8 May 2001 04:11:59 -0700 (PDT) Received: from esun1as-mm. (esun1as-mm.Central.Sun.COM [129.147.34.144]) by centralmail1.Central.Sun.COM (8.9.3+Sun/8.9.3/ENSMAIL,v2.1p1) with SMTP id FAA02735 for ; Tue, 8 May 2001 05:11:59 -0600 (MDT) Received: from sun.com by esun1as-mm. (SMI-8.6/SMI-SVR4) id FAA27073; Tue, 8 May 2001 05:18:27 -0600 Message-ID: <3AF7D474.33A17B98@sun.com> Date: Tue, 08 May 2001 04:11:48 -0700 From: Raghavendra Rao X-Mailer: Mozilla 4.51 [en] (Win98; U) X-Accept-Language: en MIME-Version: 1.0 To: ips@ece.cmu.edu Subject: RE: iSNS: Event registry and notification Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-ips@ece.cmu.edu Precedence: bulk Content-Transfer-Encoding: 7bit Josh, > DD's define security/access control policy. If a new node > only had access permission to one other iSCSI node, then those > two nodes would be the only members of the DD. A node can be > a member of multiple DD's. If 255 devices are in the DD, that > implies that all 255 devices have permission to access each In my reading of the spec, my understanding was that DD only provided a higher level access control and security but the final access control is actually decided by: a) Per Target's login access control b) Target's Logical Unit access control (i.e target would only list the LUs allowed for access in response to REPORT LU command) Your clarification above seems to be saying that, if a target x, and target y exist inside of a DD of which initiator z is also a member, then both targets x and y must provide login and Logical Unit access to initiator 'z'. Is this correct ? If it is indeed true, then this results in creation of a huge number of DDs even in small networks (which I hate to see) - I tend to think that more DDs you create, more you would need to administer and more you would need to manage (contrary to one's expectation) -JP From owner-ips@ece.cmu.edu Tue May 8 11:44:19 2001 Received: from ece.cmu.edu ([128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id LAA28265 for ; Tue, 8 May 2001 11:44:19 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f48DjOE14042 for ips-outgoing; Tue, 8 May 2001 09:45:24 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from hotmail.com (f222.law11.hotmail.com [64.4.17.222]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f48DjMH14033 for ; Tue, 8 May 2001 09:45:22 -0400 (EDT) Received: from mail pickup service by hotmail.com with Microsoft SMTPSVC; Tue, 8 May 2001 06:45:15 -0700 Received: from 203.124.248.241 by lw11fd.law11.hotmail.msn.com with HTTP; Tue, 08 May 2001 13:45:15 GMT X-Originating-IP: [203.124.248.241] From: "Ajit Khaparde" To: julian_satran@il.ibm.com, matt_wakeley@agilent.com Cc: ips@ece.cmu.edu Subject: Recovery mechanism Date: Tue, 08 May 2001 19:15:15 +0530 Mime-Version: 1.0 Content-Type: text/plain; format=flowed Message-ID: X-OriginalArrivalTime: 08 May 2001 13:45:15.0394 (UTC) FILETIME=[1C8D8620:01C0D7C5] Sender: owner-ips@ece.cmu.edu Precedence: bulk Hello, I'm currently going through the document "draft-ietf-ips-iscsi-05.txt". I hope this is the latest document. Is it so? Then I have few queries: TO recover a device(a target or an initiator), within a Task, what is the number of retries to be given to prevent an initiator from requesting a dead target. Also, how can an initiator know about the presence of a dead target. Also is it possible for a device(initiator) to remember a dead target? Thanking you. Awaiting your reply. Ajit _________________________________________________________________________ Get Your Private, Free E-mail from MSN Hotmail at http://www.hotmail.com. From owner-ips@ece.cmu.edu Tue May 8 15:13:13 2001 Received: from ece.cmu.edu ([128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id PAA05125 for ; Tue, 8 May 2001 15:13:13 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f48I3ka07567 for ips-outgoing; Tue, 8 May 2001 14:03:46 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from palrel2.hp.com (palrel2.hp.com [156.153.255.234]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f48I3iH07561 for ; Tue, 8 May 2001 14:03:44 -0400 (EDT) Received: from hpcuhe.cup.hp.com (hpcuhe.cup.hp.com [15.0.80.203]) by palrel2.hp.com (Postfix) with ESMTP id B86B8E2C for ; Tue, 8 May 2001 11:03:43 -0700 (PDT) Received: from cup.hp.com (santoshr@hpindhhm.cup.hp.com [15.8.80.197]) by hpcuhe.cup.hp.com (8.9.3 (PHNE_18979)/8.9.3 SMKit7.02) with ESMTP id LAA28385 for ; Tue, 8 May 2001 11:03:37 -0700 (PDT) Message-ID: <3AF834EF.C1FC0191@cup.hp.com> Date: Tue, 08 May 2001 11:03:27 -0700 From: Santosh Rao Organization: Hewlett Packard, Cupertino. X-Mailer: Mozilla 4.7 [en] (X11; U; HP-UX B.11.00 9000/778) X-Accept-Language: en MIME-Version: 1.0 To: ips@ece.cmu.edu Subject: Re: iSCSI: multiple sessions b/n a pair of WWUIs. References: Content-Type: multipart/mixed; boundary="------------BF135B36B923096DD707602A" Sender: owner-ips@ece.cmu.edu Precedence: bulk This is a multi-part message in MIME format. --------------BF135B36B923096DD707602A Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Douglas Otis wrote: > The usefulness of ISID is also > diminished if the client is not expected to maintain this value as unique > across all Targets. I agree with the above and have stated to similar effect in my response to Matt. A practical usage of ISID within an initiator node name space is to assign it as unique across all targets, allowing the same ISID to be used as a lookup key of the session for any given target port/node. Not applying uniqueness in ISID assignment across all targets would then require initiators to implement a second key mechanism for target lookup. - Santosh --------------BF135B36B923096DD707602A Content-Type: text/x-vcard; charset=us-ascii; name="santoshr.vcf" Content-Description: Card for Santosh Rao Content-Disposition: attachment; filename="santoshr.vcf" Content-Transfer-Encoding: 7bit begin:vcard n:Rao;Santosh tel;work:408-447-3751 x-mozilla-html:FALSE org:Hewlett Packard, Cupertino.;SISL adr:;;19420, Homestead Road, M\S 43LN, ;Cupertino.;CA.;95014.;USA. version:2.1 email;internet:santoshr@cup.hp.com title:Software Design Engineer x-mozilla-cpt:;21088 fn:Santosh Rao end:vcard --------------BF135B36B923096DD707602A-- From owner-ips@ece.cmu.edu Tue May 8 15:13:17 2001 Received: from ece.cmu.edu ([128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id PAA05143 for ; Tue, 8 May 2001 15:13:17 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f48HQiA04369 for ips-outgoing; Tue, 8 May 2001 13:26:44 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from gateway.sanlight.org (adsl-63-202-160-80.dsl.snfc21.pacbell.net [63.202.160.80]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f48HQgH04365 for ; Tue, 8 May 2001 13:26:42 -0400 (EDT) Received: from ljoy (10.0.0.18.lan.sanlight.net [10.0.0.18]) by gateway.sanlight.org (8.11.0/8.11.0) with SMTP id f48IV0155718; Tue, 8 May 2001 11:31:00 -0700 (PDT) (envelope-from dotis@sanlight.net) From: "Douglas Otis" To: "Jim Hafner" , "KRUEGER,MARJORIE \(HP-Roseville,ex1\)" Cc: Subject: RE: iSCSI: multiple sessions b/n a pair of WWUIs. Date: Tue, 8 May 2001 10:21:23 -0700 Message-ID: MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 (Normal) X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook IMO, Build 9.0.2416 (9.0.2911.0) X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4522.1200 Importance: Normal In-reply-to: Sender: owner-ips@ece.cmu.edu Precedence: bulk Content-Transfer-Encoding: 7bit Jim, It does seem reasonable to expect the Initiator to maintain a database of ISIDs within its own domain that may cross-reference to Targets on which it holds reservations. This cross-reference will likely map into OS dependent views as well. A simple means to ensure ISID is unique for any Target is to ensure an Initiator ISID is unique for all Targets. It would become a tangled mess to allow replicate ISIDs for different Targets as now a database must sort on ISID+Target Label. There is no assurance that TSID will be reissued from what I can tell. It seems rather messy to change the sort from ISID, to ISID+Target Label. Do you see this added restriction of ALL instead of ANY as a limitation? I can see how in practice this prevents many problems and is a good rule if not a guideline. Doug > Marj, > > I think you've stretched my words too far. The requirement is > for a single > logical unit to recover enough identity information provided through the > I_T_L nexus to reestablish the reservation. That means that the name+ISID > must be unique *from the viewpoint of a single iSCSI target device*. So > it's more like the > "initiator must have unique ISID for all sessions with all > targets any given target device". > > The best way to think about this is the following: > 1) only a single target's (actually logical unit's) view of the initiator > counts in this regard > 2) there are no rules in SCSI concerning multi-target device interactions > In other words, for these rules, look only from inside a target going out. > > Jim Hafner > > > "KRUEGER,MARJORIE (HP-Roseville,ex1)" > @ece.cmu.edu > on 05/04/2001 05:39:35 PM > > Sent by: owner-ips@ece.cmu.edu > > > To: "'Matt Wakeley'" , ips@ece.cmu.edu > cc: > Subject: RE: iSCSI: multiple sessions b/n a pair of WWUIs. > > > > > One clarification: these are unique between any I-T *pair*. > > (you make it sound like an initiator must have a unique ISID > > for all sessions > > with all targets) > > > > -Matt > > The ISID must be unique within the iSCSI layer on an iSCSI device - that > amounts to "initiator must have unique ISID for all sessions with all > targets". > > To quote Jim "The requirement in name+disc that a given initiator name > cannot reuse an ISID for two different sessions comes as a consequence a > number of things (which are described in the draft). The gist is > that this > is needed to provide the correct context for restoration of reservations > state (and other nexus state) to a particular nexus after logout/login. In > other words, if the session goes down for some reason, the target needs > clear context to restore nexus state to a rebuilt session. The > only tool it > has is uniqueness of Name+ISID combination within its name space." > > -Marj > > > > From owner-ips@ece.cmu.edu Tue May 8 15:13:37 2001 Received: from ece.cmu.edu ([128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id PAA05160 for ; Tue, 8 May 2001 15:13:32 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f48HllR06099 for ips-outgoing; Tue, 8 May 2001 13:47:47 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from gateway.sanlight.org (adsl-63-202-160-80.dsl.snfc21.pacbell.net [63.202.160.80]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f48HlfH06093 for ; Tue, 8 May 2001 13:47:41 -0400 (EDT) Received: from ljoy (10.0.0.18.lan.sanlight.net [10.0.0.18]) by gateway.sanlight.org (8.11.0/8.11.0) with SMTP id f48IsO155742; Tue, 8 May 2001 11:54:24 -0700 (PDT) (envelope-from dotis@sanlight.net) From: "Douglas Otis" To: "John Hufferd" , Cc: Subject: RE: iSCSI: multiple sessions b/n a pair of WWUIs. Date: Tue, 8 May 2001 10:44:47 -0700 Message-ID: MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 (Normal) X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook IMO, Build 9.0.2416 (9.0.2911.0) X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4522.1200 Importance: Normal In-reply-to: Sender: owner-ips@ece.cmu.edu Precedence: bulk Content-Transfer-Encoding: 7bit John, I am a bit confused by your comments. From my understanding, the Initiator model is now multi-port where each service port is identified by Initiator Label+ISID. Where I agree the essential aspect required by the Target is to have a unique means of identifying the Initiator, the added requirement of ensuring ISID is unique for all Targets does not detract from this essential requirement stated by Jim. With a multi-port model, an Initiator can indeed have multiple sessions. With the abstract service port, the need for the wedge driver changes as the service port is no longer tied to physical hardware. This abstraction does the right thing unlike those that attached to the physical hardware. It is really hinged on how you wish the database held in the Initiator sorted: ISID or Target Label. It seems appropriate to attach structures on those ISIDs that represent reconnections or reservations. On the other hand, you seem to be advocating a name list. The usefulness of ISID is also diminished if the client is not expected to maintain this value as unique across all Targets. Doug > > David, > let me argue another side. I was about to send my version of what Jim > said, but I think Jim, as usual was "dead on". It is important to be > completely correct here. This direction seemed to be having an influence > on MIBs and other things. Though it is possible to do what you suggest, I > would say that doing so causes unintentional thought problems. That is, > it is important that the Initiator Side (or the Target Side) have a way > that the Management/Configuration Software can tell the iSCSI > Port what the > iSCSI Node Name is, and what ISID value (or Range) should be > used. This is > true if the iSCSI Port is a SW entity or a HW iSCSI Host Bus > Adapter (HBA). > > It is much simpler for an iSCSI Port to use one ISID for every session it > creates to any target, then to have an algorithm for deriving > one. (Yes, I > know someone has to do that, but I think it should be done at the > Configuration Software level instead of at the iSCSI Port level.) > > Also, since a second session by the same iSCSI Initiator Port, to the same > iSCSI Target Port is not, in my opinion "legal", it is not clear > to me that > any given iSCSI Port needs any more then one ISID. If we say anything > else, I think we risk the danger of mixing the concept of Wedge Driver in > with the concept of iSCSI Port, and that will do us all a disservice. > > Some folks are going to want to define Wedge Drivers, and there should be > no interfering thought that some how the same iSCSI Initiator > Port can have > Multiple Sessions to the same iSCSI Target Port (this is the domain of > Multiple Connections per Session). Wedge Drivers, if needed, should be > seen as something that span iSCSI Initiator Ports and address specific > iSCSI Target Ports. (Implementations may blur the levels here, but the > concepts should be clear.) > > Therefore, I suggest that we Not put any such notes, like you suggest, in > the draft, but in fact encourage a single ISID/TSID per iSCSI > Initiator/Target Port. > > . > . > . > John L. Hufferd > Senior Technical Staff Member (STSM) > IBM/SSG San Jose Ca > (408) 256-0403, Tie: 276-0403, eFax: (408) 904-4688 > Internet address: hufferd@us.ibm.com > > > Black_David@emc.com@ece.cmu.edu on 05/07/2001 07:52:08 PM > > Sent by: owner-ips@ece.cmu.edu > > > To: Jim Hafner/Almaden/IBM@IBMUS, marjorie_krueger@hp.com > cc: ips@ece.cmu.edu > Subject: RE: iSCSI: multiple sessions b/n a pair of WWUIs. > > > > While Jim's correct ... in Marj's defense, what > she wrote is a fairly obvious way to implement > it, and that might be worth noting in the draft. > > --David > > --------------------------------------------------- > David L. Black, Senior Technologist > EMC Corporation, 42 South St., Hopkinton, MA 01748 > +1 (508) 435-1000 x75140 FAX: +1 (508) 497-8500 > black_david@emc.com Mobile: +1 (978) 394-7754 > --------------------------------------------------- > > > -----Original Message----- > > From: Jim Hafner [SMTP:hafner@almaden.ibm.com] > > Sent: Monday, May 07, 2001 10:44 PM > > To: KRUEGER,MARJORIE (HP-Roseville,ex1) > > Cc: ips@ece.cmu.edu > > Subject: RE: iSCSI: multiple sessions b/n a pair of WWUIs. > > > > > > Marj, > > > > I think you've stretched my words too far. The requirement is for a > > single > > logical unit to recover enough identity information provided through the > > I_T_L nexus to reestablish the reservation. That means that the > name+ISID > > must be unique *from the viewpoint of a single iSCSI target device*. So > > it's more like the > > "initiator must have unique ISID for all sessions with all > > targets any given target device". > > > > The best way to think about this is the following: > > 1) only a single target's (actually logical unit's) view of the > initiator > > counts in this regard > > 2) there are no rules in SCSI concerning multi-target device > interactions > > In other words, for these rules, look only from inside a target going > out. > > > > Jim Hafner > > > > > > "KRUEGER,MARJORIE (HP-Roseville,ex1)" > > @ece.cmu.edu > > on 05/04/2001 05:39:35 PM > > > > Sent by: owner-ips@ece.cmu.edu > > > > > > To: "'Matt Wakeley'" , ips@ece.cmu.edu > > cc: > > Subject: RE: iSCSI: multiple sessions b/n a pair of WWUIs. > > > > > > > > > One clarification: these are unique between any I-T *pair*. > > > (you make it sound like an initiator must have a unique ISID > > > for all sessions > > > with all targets) > > > > > > -Matt > > > > The ISID must be unique within the iSCSI layer on an iSCSI device - that > > amounts to "initiator must have unique ISID for all sessions with all > > targets". > > > > To quote Jim "The requirement in name+disc that a given initiator name > > cannot reuse an ISID for two different sessions comes as a consequence a > > number of things (which are described in the draft). The gist is that > > this > > is needed to provide the correct context for restoration of reservations > > state (and other nexus state) to a particular nexus after logout/login. > In > > other words, if the session goes down for some reason, the target needs > > clear context to restore nexus state to a rebuilt session. The only tool > > it > > has is uniqueness of Name+ISID combination within its name space." > > > > -Marj > > > > > > > > From owner-ips@ece.cmu.edu Tue May 8 15:14:52 2001 Received: from ece.cmu.edu ([128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id PAA05187 for ; Tue, 8 May 2001 15:14:52 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f48HKlm03816 for ips-outgoing; Tue, 8 May 2001 13:20:47 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from palrel2.hp.com (palrel2.hp.com [156.153.255.234]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f48HKjH03809 for ; Tue, 8 May 2001 13:20:46 -0400 (EDT) Received: from psalpha0.cup.hp.com (psalpha0.cup.hp.com [15.13.189.230]) by palrel2.hp.com (Postfix) with ESMTP id 02AD6123F; Tue, 8 May 2001 10:20:45 -0700 (PDT) Received: from mk731913.cup.hp.com (mk731912.cup.hp.com [15.8.80.111]) by psalpha0.cup.hp.com (8.9.3/8.8.6) with ESMTP id KAA15848; Tue, 8 May 2001 10:20:44 -0700 (PDT) Message-Id: <5.0.2.1.2.20010508094736.02819d10@esalpha2.cup.hp.com> X-Sender: krause@esalpha2.cup.hp.com X-Mailer: QUALCOMM Windows Eudora Version 5.0.2 Date: Tue, 08 May 2001 09:54:40 -0700 To: "Douglas Otis" From: Michael Krause Subject: RE: iSCSI Security rough consensus Cc: , , , , In-Reply-To: References: Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii"; format=flowed Sender: owner-ips@ece.cmu.edu Precedence: bulk The end-to-end integrity checks are from a source fault zone to a destination fault zone. What happens within a fault zone is implementation specific and beyond the scope of the iSCSI specification. The litany of potential problems within a fault zone are interesting and are being addressed by various techniques all of which are again outside the scope of the iSCSI specification and clearly where many vendors will differentiate their product offerings. It would be beneficial to this workgroup to avoid mixing the fault zone and iSCSI requirements to avoid creating confusion or what may be construed as FUD. The workgroup appears to be in consensus on where iSCSI integrity checking applies and the problems it addresses. Unless there is something new to add from an iSCSI technical perspective in this area, may we move on to getting this specification complete so we can implement it and see if it really works. Mike At 07:03 PM 5/7/2001 -0700, Douglas Otis wrote: >Somesh, > >You have it correct. The general intent of an end to end integrity check is >to do something light weight within software to catch a myriad of errors >that crop up due to various hardware failures (R*****k and M*******h as >example), as well as bugs within variations of the theme called TCP. One >can only hope such checking automation does not introduce corrupt data as a >result. New hardware/software MUST maintain integrity. There is a history >that gives one pause however. > >Doug > > > > I was trying not to react to the string of messages, but > > this message finally did it(and nothing wrong with the message > > itself and no offense meant to the author). What the message > > implies is that no intermediate hops can be trusted and that > > integrity must be end-to-end. > > > > Taken to the literal extreme, this means that the system must > > at the application level (in the host) generate some integrity > > checks which should then be verified every > > time the data is used (perhaps by an end application again). > > > > Even with the CRC, the following errors will go undetected, > > unless protected by some other means (well engineered, well > > tested devices) > > > > 1. Host software. > > 2. Host memory to adapter path > > 3. Adapter memory > > 4. Adapter software > > 5. Any virtualization device in the middle including > > software, memory, fabric etc on the device including > > any fibre-channel/scsi storage at the back end > > (assuming they will recreate resegment PDUs) > > 6. Any target adapter & adapter memory > > 7. Any paths within the target > > 8. The storage itself > > 9. and then the reverse path > > > > If end-to-end integrity is a must, it seems that the TCP checksum > > offload and iSCSI CRC in off-board hardware must be avoided. > > > > Or did I miss something somewhere. > > > > Somesh > > > > > -----Original Message----- > > > From: vince_cavanna@agilent.com [mailto:vince_cavanna@agilent.com] > > > Sent: Monday, May 07, 2001 2:13 PM > > > To: mbakke@cisco.com; Black_David@emc.com > > > Cc: someshg@yahoo.com; ips@ece.cmu.edu; vince_cavanna@agilent.com > > > Subject: RE: iSCSI Security rough consensus > > > > > > > > > Here is a paper by well-known authors that supports Mark's > > reasoning that > > > the end-to-end iSCSI check is indispensable even when IPsec is on .... > > > > > > Jonathan Stone and Craig artridge in "When the CRC and TCP checksum > > > disagree" study the sources of errors that are not caught by link level > > > checks and that therefore stress a higher layer error check > > > mechansim. Such > > > errors are quite prevalent and one signifcant source of such > > errors is the > > > end-host itself. The authors emphasize that "hardware must not > > be trusted" > > > and that "many encryption solutions such as IPsec do not provide > > > additional > > > (over existing checks) protection because the encryption is > > > applied too late > > > in the transmission process, often after the data has passed > > through a DMA > > > engine. Rather the application must add the checksum before > > > handing its data > > > to TCP (ala SSL)". The authors recommend that "for truly > > valuable data (in > > > my opinion all data handled by iSCSI) the application should add > > > a stronger > > > application-level checksum". > > > > > > Vince > > > > > > |-----Original Message----- > > > |From: Mark Bakke [mailto:mbakke@cisco.com] > > > |Sent: Friday, May 04, 2001 1:04 PM > > > |To: Black_David@emc.com > > > |Cc: someshg@yahoo.com; ips@ece.cmu.edu > > > |Subject: Re: iSCSI Security rough consensus > > > | > > > | > > > |Black_David@emc.com wrote: > > > |> > > > |> > Sure would be nice if we could make up our minds and just > > > |> > implement one mechanism. > > > |> > > > > |> > Here we have two mechanisms (iSCSI header/data integrity > > > |> > and ESP) which are both mandatory to implement and > > > |> > optional to use. Since ESP seems like a superset why not > > > |> > just have that and get rid of the "integrity only" iSCSI > > > |> > CRC mechanism. > > > |> > > > |> It sure would be nice, and in fact we had almost > > > |> exactly this discussion later in the evening as > > > |> part of the error recovery section of the agenda. > > > |> The fly in the ointment is that the HMAC integrity > > > |> algorithm that is at the core of ESP's integrity > > > |> support is considerably more expensive (software > > > |> or hardware) than a CRC, and this isn't likely > > > |> to improve as I understand things. I would expect > > > |> to see implementations with ESP completely in > > > |> software and visible performance impacts. > > > | > > > |That's just part of the reason behind having both. The other is > > > |that most implementations won't run IPsec end-to-end; either IPsec > > > |is provided in an external device, or even in an iSCSI gateway. > > > |In the latter case, all layers are removed and replaced, including > > > |iSCSI. Only the SCSI-level information (data, CDBs) go all the > > > |way end-to-end. Since iSCSI can CRC the SCSI-level data, it can > > > |provide the data integrity that will keep our customers happy. > > > | > > > |The use of the iSCSI CRC is the minimum requirement; adding the > > > |IPsec-level integrity check strengthens it, and can simplify error > > > |recovery over a not-so-good or untrusted network. > > > | > > > |-- > > > |Mark > > > | > > > |> > > > |> I really need to get the meeting minutes written up :-). > > > |> > > > |> Thanks, > > > |> --David > > > |> > > > |> --------------------------------------------------- > > > |> David L. Black, Senior Technologist > > > |> EMC Corporation, 42 South St., Hopkinton, MA 01748 > > > |> +1 (508) 435-1000 x75140 FAX: +1 (508) 497-8500 > > > |> black_david@emc.com Mobile: +1 (978) 394-7754 > > > |> --------------------------------------------------- > > > | > > > |-- > > > |Mark A. Bakke > > > |Cisco Systems > > > |mbakke@cisco.com > > > |763.398.1054 > > > | > > > > > > _________________________________________________________ > > Do You Yahoo!? > > Get your free @yahoo.com address at http://mail.yahoo.com > > > > From owner-ips@ece.cmu.edu Tue May 8 15:15:32 2001 Received: from ece.cmu.edu ([128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id PAA05215 for ; Tue, 8 May 2001 15:15:31 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f48GrHc01523 for ips-outgoing; Tue, 8 May 2001 12:53:17 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from server1.NishanSystems.COM (smtp.nishansystems.com [216.217.36.162]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f48GrFH01519 for ; Tue, 8 May 2001 12:53:15 -0400 (EDT) Received: by smtp.nishansystems.com with Internet Mail Service (5.5.2653.19) id ; Tue, 8 May 2001 09:53:06 -0700 Message-ID: From: Joshua Tseng To: "'Raghavendra Rao'" , ips@ece.cmu.edu Subject: RE: iSNS: Event registry and notification Date: Tue, 8 May 2001 09:53:05 -0700 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2653.19) Content-Type: text/plain; charset="iso-8859-1" Sender: owner-ips@ece.cmu.edu Precedence: bulk JP, > > Josh, > > > DD's define security/access control policy. If a new node > > only had access permission to one other iSCSI node, then those > > two nodes would be the only members of the DD. A node can be > > a member of multiple DD's. If 255 devices are in the DD, that > > implies that all 255 devices have permission to access each > > In my reading of the spec, my understanding was that DD only provided > a higher level access control and security but the final > access control > is > actually decided by: > a) Per Target's login access control > b) Target's Logical Unit access control (i.e target > would only list > the LUs > allowed for access in response to REPORT LU command) Your a) is correct. But by using iSNS, the target "slaves" its login access control policy to the iSNS. If the target is placed into a DD with initiators A, B, and C by the management GUI, then the administrator is configuring that target for access by initiators A, B, and C. The target's login access control policy would then be configured by the iSNS to allow that access. b) is a function of SCSI. iSNS and iSCSI does not get involved at that level. > > Your clarification above seems to be saying that, if a target x, and > target y > exist inside of a DD of which initiator z is also a member, then both > targets > x and y must provide login and Logical Unit access to initiator 'z'. Yes, this is correct if initiator 'z' were placed in the same DD as target x and target y. > > Is this correct ? If it is indeed true, then this results in > creation of > a huge number > of DDs even in small networks (which I hate to see) - I tend to think > that > more DDs you create, more you would need to administer and more you > would > need to manage (contrary to one's expectation) Yes, in a large storage network, there will be a large number of DDs. That is why we added Discovery Domain Sets (DDS) to iSNS. Josh > > -JP > > From owner-ips@ece.cmu.edu Tue May 8 15:16:28 2001 Received: from ece.cmu.edu ([128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id PAA05237 for ; Tue, 8 May 2001 15:16:28 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f48HTge04588 for ips-outgoing; Tue, 8 May 2001 13:29:42 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from bucky.excite.com (bucky-rwcmex.excite.com [198.3.99.218]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f48HTfH04584 for ; Tue, 8 May 2001 13:29:41 -0400 (EDT) Received: from blizzard.excite.com ([199.172.148.158]) by bucky.excite.com (InterMail vM.4.01.02.39 201-229-119-122) with ESMTP id <20010508172935.WZCP25251.bucky.excite.com@blizzard.excite.com> for ; Tue, 8 May 2001 10:29:35 -0700 Message-ID: <295751.989342975370.JavaMail.imail@blizzard.excite.com> Date: Tue, 8 May 2001 10:29:35 -0700 (PDT) From: bj nima To: ips@ece.cmu.edu Subject: Re: iSCSI: Nailing down CRC-32C Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-Mailer: Excite Inbox X-Sender-Ip: 63.70.210.20 Sender: owner-ips@ece.cmu.edu Precedence: bulk Content-Transfer-Encoding: 7bit Assumptions: 1) The iSCSI DATA PDU Header and Data Integrity checks uses CRC-32 2) Support of Integrity check is mandatory for both header and Data digest 3) An iSCSI PDU may span multiple TCP segments. 4) Any of these segments may arrive out of order at the receiver. 5) To regenerate the CRC for comparison with the value present in the PDU received, the receiver needs to keep in a temporary buffers all segments, waiting for the missing one, as CRC must be computed using the whole re-assembled PDU Question: Does CRC-32 support partial computation and compensation for wrong initial values later (I guess NO!), to enable partial CRC calc on avail parts of PDU? Possible answer: No Possible Conclusion: The RCV side is further complicated b/c of this CRC Possible solutions: 1) Use CRC that allows partial calc 2) Limit iSCSI PDU to TCP MSS if data CRC is used (added o/h)... 3) Leave as it, though it has cost , performance implication and hurts scaling to 10G.. Nima -----Original Message----- From: julian_satran@il.ibm.com [mailto:julian_satran@il.ibm.com] Sent: Saturday, May 05, 2001 9:21 AM To: ips@ece.cmu.edu Subject: Re: iSCSI: Nailing down CRC-32C The CRC part of the appendix (for 07) reads: The following table lists cyclic integrity checksums that can be negotiated for the digests and MUST be implemented by every iSCSI initiator and target. Note that these digest options have only error detection significance. +---------------------------------------------+ | Name | Description | +---------------------------------------------+ | crc-32C | 32 bit CRC | 11EDC6F41 | +---------------------------------------------+ | none | no digest | +---------------------------------------------+ The generator polynomials for those digests are given in hex-notation, for example 3a stands for 0011 1010 - the polynomial x**5+X**4+x**3+x+1. The generator polynomial selected is evaluated in [Castagnioli93]. When using the CRC the CRC register must be initialized to all 1s (0xFFFFFFFF) and the CRC bits must be complemented before transmission. Padding bytes, when presents in a segment covered by a CRC, have to be set to 0 and are included in the CRC. Regards, Julo Mark Bakke on 05-05-2001 00:13:09 Please respond to Mark Bakke To: IPS cc: Subject: iSCSI: Nailing down CRC-32C At the interim meeting, it was stated that iSCSI has selected the CRC-32C polynomial as its required iSCSI-level header and data CRC. Now that we have it, it's time to move on and make sure we can implement it. In the interest of interoperability, we need to not only specify the polynomial, but also the initial values, bit and byte ordering, etc. "A Painless Guide to CRC Error Detection Algorithms" (http://www.ross.net/crc/crcpaper.html) specifies a method to unambiguously characterize these parameters (sections 15 and 16). Has anyone taken a shot at defining these yet? Otherwise, here is what it might look like: Name : "CRC-32C" Width : 32 Poly : 1EDC6F41 (note that the leading "1" is implied) Init : FFFFFFFF RefIn : True RefOut : True XorOut : FFFFFFFF Check : ? I haven't attempted to create check data based on these yet. As soon as the other parameters are nailed down, we need to do this. Anyway, I am not a CRC expert, and can't make any statement about the above values being the "best" way to do this, but instead just copied them (except the polynomial itself) from the Ethernet CRC, since that is likely the easiest for everyone implementing hardware to deal with. If someone else is already doing this, let me know; I just wanted to start this thread so we can get closure. Regards, Mark -- Mark A. Bakke Cisco Systems mbakke@cisco.com 763.398.1054 _______________________________________________________ Send a cool gift with your E-Card http://www.bluemountain.com/giftcenter/ From owner-ips@ece.cmu.edu Tue May 8 16:14:58 2001 Received: from ece.cmu.edu ([128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id QAA06679 for ; Tue, 8 May 2001 16:14:56 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f48IIo408941 for ips-outgoing; Tue, 8 May 2001 14:18:50 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from msgbas1t.cos.agilent.com (msgbas1tx.cos.agilent.com [192.6.9.34]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f48IImH08934 for ; Tue, 8 May 2001 14:18:49 -0400 (EDT) Received: from msgrel1.and.agilent.com (msgrel1.and.agilent.com [130.30.33.104]) by msgbas1t.cos.agilent.com (Postfix) with ESMTP id D8490345 for ; Tue, 8 May 2001 12:18:47 -0600 (MDT) Received: from rtl.rose.agilent.com (rtl.rose.agilent.com [156.140.232.231]) by msgrel1.and.agilent.com (Postfix) with ESMTP id 06890F4 for ; Tue, 8 May 2001 14:18:47 -0400 (EDT) Received: from mail.rose.agilent.com (bellhop [156.140.233.51]) by rtl.rose.agilent.com (8.9.3 (PHNE_18979)/8.9.3 SMKit7.1.0) with ESMTP id LAA25697 for ; Tue, 8 May 2001 11:18:45 -0700 (PDT) Received: from agilent.com (matt5670.rose.agilent.com [156.140.235.204]) by mail.rose.agilent.com (Netscape Messaging Server 3.6) with ESMTP id AAA65C8 for ; Tue, 8 May 2001 11:18:41 -0700 Message-ID: <3AF8389D.4F9424E9@agilent.com> Date: Tue, 08 May 2001 11:19:09 -0700 From: "Matt Wakeley" Reply-To: Matt Wakeley Organization: Agilent Technologies X-Mailer: Mozilla 4.77 [en] (Windows NT 5.0; U) X-Accept-Language: en MIME-Version: 1.0 To: ips@ece.cmu.edu Subject: Re: iSCSI: Nailing down CRC-32C References: <295751.989342975370.JavaMail.imail@blizzard.excite.com> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-ips@ece.cmu.edu Precedence: bulk Content-Transfer-Encoding: 7bit There is a 4th option. Mandate the implementation of iSCSI markers. This will significantly reduce the amount of buffering required. I think we should just go ahead and mandate implementation of markers, for two reasons: 1- the "ULF (or WARP)" proposal doesn't seem to be making much headway, and 2- even if ULF did become reality, there are software implementations today that will not have the luxury of having a "enhanced" TCP stack that would provide the framing. Software implementations of iSCSI MUST supply iSCSI markers (if invoked to do so) to enable low cost, high performance 10Gig solutions. -Matt bj nima wrote: > > Assumptions: > > 1) The iSCSI DATA PDU Header and Data Integrity checks uses CRC-32 > 2) Support of Integrity check is mandatory for both header and Data > digest > 3) An iSCSI PDU may span multiple TCP segments. > 4) Any of these segments may arrive out of order at the receiver. > 5) To regenerate the CRC for comparison with the value present in the > PDU received, the receiver needs to keep in a temporary buffers all > segments, waiting for the missing one, as CRC must be computed using > the whole re-assembled PDU > > Question: Does CRC-32 support partial computation and compensation > for wrong initial values later (I guess NO!), to enable partial CRC > calc on avail parts of PDU? > > Possible answer: No > > Possible Conclusion: The RCV side is further complicated b/c of this > CRC > > Possible solutions: > > 1) Use CRC that allows partial calc > 2) Limit iSCSI PDU to TCP MSS if data CRC is used (added o/h)... > 3) Leave as it, though it has cost , performance implication and hurts > scaling to 10G.. > > Nima > > -----Original Message----- > From: julian_satran@il.ibm.com [mailto:julian_satran@il.ibm.com] > Sent: Saturday, May 05, 2001 9:21 AM > To: ips@ece.cmu.edu > Subject: Re: iSCSI: Nailing down CRC-32C > > The CRC part of the appendix (for 07) reads: > > The following table lists cyclic integrity checksums that can be > negotiated for the digests and MUST be implemented by every iSCSI > initiator and target. Note that these digest options have only error > detection significance. > > +---------------------------------------------+ > | Name | Description | > +---------------------------------------------+ > | crc-32C | 32 bit CRC | 11EDC6F41 | > +---------------------------------------------+ > | none | no digest | > +---------------------------------------------+ > > The generator polynomials for those digests are given in hex-notation, > for example 3a stands for 0011 1010 - the polynomial x**5+X**4+x**3+x+1. > > The generator polynomial selected is evaluated in [Castagnioli93]. > When using the CRC the CRC register must be initialized to all 1s > (0xFFFFFFFF) and the CRC bits must be complemented before transmission. > Padding bytes, when presents in a segment covered by a CRC, have to be > set to 0 and are included in the CRC. > > Regards, > Julo > > Mark Bakke on 05-05-2001 00:13:09 > > Please respond to Mark Bakke > > To: IPS > cc: > Subject: iSCSI: Nailing down CRC-32C > > At the interim meeting, it was stated that iSCSI has selected > the CRC-32C polynomial as its required iSCSI-level header and > data CRC. Now that we have it, it's time to move on and make > sure we can implement it. > > In the interest of interoperability, we need to not only specify > the polynomial, but also the initial values, bit and byte > ordering, etc. > > "A Painless Guide to CRC Error Detection Algorithms" > (http://www.ross.net/crc/crcpaper.html) specifies a > method to unambiguously characterize these parameters > (sections 15 and 16). Has anyone taken a shot at defining > these yet? Otherwise, here is what it might look like: > > Name : "CRC-32C" > Width : 32 > Poly : 1EDC6F41 (note that the leading "1" is implied) > Init : FFFFFFFF > RefIn : True > RefOut : True > XorOut : FFFFFFFF > Check : ? > > I haven't attempted to create check data based on these yet. As > soon as the other parameters are nailed down, we need to do this. > > Anyway, I am not a CRC expert, and can't make any statement about > the above values being the "best" way to do this, but instead just > copied them (except the polynomial itself) from the Ethernet CRC, > since that is likely the easiest for everyone implementing hardware > to deal with. > > If someone else is already doing this, let me know; I just wanted > to start this thread so we can get closure. > > Regards, > > Mark > > -- > Mark A. Bakke > Cisco Systems > mbakke@cisco.com > 763.398.1054 > > _______________________________________________________ > Send a cool gift with your E-Card > http://www.bluemountain.com/giftcenter/ From owner-ips@ece.cmu.edu Tue May 8 16:16:30 2001 Received: from ece.cmu.edu ([128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id QAA06712 for ; Tue, 8 May 2001 16:16:16 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f48IoZu11604 for ips-outgoing; Tue, 8 May 2001 14:50:35 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from mail5.microsoft.com (mail5.microsoft.com [131.107.3.121]) by ece.cmu.edu (8.11.0/8.10.2) with SMTP id f48IoWH11594 for ; Tue, 8 May 2001 14:50:32 -0400 (EDT) Received: from 157.54.9.100 by mail5.microsoft.com (InterScan E-Mail VirusWall NT); Tue, 08 May 2001 11:50:26 -0700 (Pacific Daylight Time) Received: from red-imc-02.redmond.corp.microsoft.com ([157.54.9.107]) by inet-imc-03.redmond.corp.microsoft.com with Microsoft SMTPSVC(5.0.2195.2883); Tue, 8 May 2001 11:50:18 -0700 Received: from win-imc-02.wingroup.windeploy.ntdev.microsoft.com ([157.54.0.82]) by red-imc-02.redmond.corp.microsoft.com with Microsoft SMTPSVC(5.0.2195.2883); Tue, 8 May 2001 11:50:01 -0700 Received: from win-msg-02.wingroup.windeploy.ntdev.microsoft.com ([157.54.0.134]) by win-imc-02.wingroup.windeploy.ntdev.microsoft.com with Microsoft SMTPSVC(5.0.2195.1600); Tue, 8 May 2001 11:49:38 -0700 content-class: urn:content-classes:message Subject: Keys and Values MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="----_=_NextPart_001_01C0D7EF.A1F77805" Date: Tue, 8 May 2001 11:49:38 -0700 X-MimeOLE: Produced By Microsoft Exchange V6.0.4688.0 Message-ID: <8CC03F47967BF14D9710887723FADDB62F51F8@win-msg-02.wingroup.windeploy.ntdev.microsoft.com> Thread-Topic: Keys and Values Thread-Index: AcDX768R/64mBC8RR0m4m4yECMJJ6w== From: "Lakshmi Ramasubramanian" To: "IPS Reflector" X-OriginalArrivalTime: 08 May 2001 18:49:38.0268 (UTC) FILETIME=[A212F1C0:01C0D7EF] Sender: owner-ips@ece.cmu.edu Precedence: bulk This is a multi-part message in MIME format. ------_=_NextPart_001_01C0D7EF.A1F77805 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable In the Login Phase Examples given in Appendix B (iSCSI Spec Rev 06), some of the keys and values (such as HeaderDigest, DataDigest, etc) are mentioned. Is there a complete list of all the keys, and the values they can take, given anywhere?=20 If so, could somebody please give pointers? =20 Appendix E mentions a few keys and values, but it doesn't seem to cover all of them. =20 thanks! -l ------_=_NextPart_001_01C0D7EF.A1F77805 Content-Type: text/html; charset="us-ascii" Content-Transfer-Encoding: quoted-printable Message
In the Login Phase Examples given in Appendix = B (iSCSI Spec Rev 06), some of=20 the
keys and = values (such as=20 HeaderDigest, DataDigest, etc) are mentioned. Is=20 there
a complete = list of all=20 the keys, and the values they can take, given anywhere?=20
If so,=20 could = somebody please=20 give pointers?
 
Appendix E = mentions a=20 few keys and values, but it doesn't seem to cover all of=20 them.
 
thanks!
 -l
=00 ------_=_NextPart_001_01C0D7EF.A1F77805-- From owner-ips@ece.cmu.edu Tue May 8 16:16:51 2001 Received: from ece.cmu.edu ([128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id QAA06727 for ; Tue, 8 May 2001 16:16:51 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f48ILW109205 for ips-outgoing; Tue, 8 May 2001 14:21:32 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from msgbas1.cos.agilent.com (msgbas1x.cos.agilent.com [192.6.9.33]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f48ILUH09200 for ; Tue, 8 May 2001 14:21:30 -0400 (EDT) Received: from msgrel1.and.agilent.com (msgrel1.and.agilent.com [130.30.33.104]) by msgbas1.cos.agilent.com (Postfix) with ESMTP id E508844C for ; Tue, 8 May 2001 12:21:29 -0600 (MDT) Received: from rtl.rose.agilent.com (rtl.rose.agilent.com [156.140.232.231]) by msgrel1.and.agilent.com (Postfix) with ESMTP id E92BAF4 for ; Tue, 8 May 2001 14:21:28 -0400 (EDT) Received: from mail.rose.agilent.com (bellhop [156.140.233.51]) by rtl.rose.agilent.com (8.9.3 (PHNE_18979)/8.9.3 SMKit7.1.0) with ESMTP id LAA25812 for ; Tue, 8 May 2001 11:21:27 -0700 (PDT) Received: from agilent.com (matt5670.rose.agilent.com [156.140.235.204]) by mail.rose.agilent.com (Netscape Messaging Server 3.6) with ESMTP id AAA665B; Tue, 8 May 2001 11:21:22 -0700 Message-ID: <3AF8393D.EA6D37C3@agilent.com> Date: Tue, 08 May 2001 11:21:49 -0700 From: "Matt Wakeley" Reply-To: Matt Wakeley Organization: Agilent Technologies X-Mailer: Mozilla 4.77 [en] (Windows NT 5.0; U) X-Accept-Language: en MIME-Version: 1.0 To: julian_satran@il.ibm.com, IPS Reflector Subject: Re: new CRC draft References: Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-ips@ece.cmu.edu Precedence: bulk Content-Transfer-Encoding: 7bit Julian, It looks like you're pointing to an old draft - it's dated Feb 26. -Matt julian_satran@il.ibm.com wrote: > > On behalf of a group of authors from several organizations I submit an > informational IETF-draft for immediate publication. It analyzes several > options for non-cryptographyc data integrity to be used by iSCSI and the > file name is "draft-sheinwald-iSCSI-CRC-00.txt". > > The draft can be found at: > > http://www.haifa.il.ibm.com/satran/ips/draft-sheinwald-iSCSI-CRC-00.txt > > Julian Satran - IBM Research Laboratory at Haifa From owner-ips@ece.cmu.edu Tue May 8 16:17:34 2001 Received: from ece.cmu.edu ([128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id QAA06742 for ; Tue, 8 May 2001 16:17:34 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f48IdRo10693 for ips-outgoing; Tue, 8 May 2001 14:39:27 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from think.thunk.org (THINK.THUNK.ORG [216.175.175.162]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f48IdOH10681 for ; Tue, 8 May 2001 14:39:24 -0400 (EDT) Received: from tytso by think.thunk.org with local (Exim 3.22 #1 (Debian)) id 14xCNt-0000W8-00; Tue, 08 May 2001 14:38:57 -0400 Date: Tue, 8 May 2001 14:38:57 -0400 From: Theodore Tso To: Joshua Tseng Cc: "'Theodore Tso'" , Black_David@emc.com, ips@ece.cmu.edu Subject: Re: iSCSI Security rough consensus Message-ID: <20010508143857.A1956@think.thunk.org> References: Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.3.15i In-Reply-To: ; from jtseng@NishanSystems.com on Fri, May 04, 2001 at 08:05:57PM -0700 Sender: owner-ips@ece.cmu.edu Precedence: bulk On Fri, May 04, 2001 at 08:05:57PM -0700, Joshua Tseng wrote: > > I don't know the Security AD's, but if they had this reaction > as you describe, my question to them would be why did they invent > tunnel mode and security gateways in the first place? If they > wanted all protocols to have end-to-end protection as a REQUIREMENT, > shouldn't they have just stopped with transport mode? That way, it's > end-to-end IPSec or else use something like TLS. The main reason for things like tunnel mode and security gateways are for legacy protocols and legacy implementations. All new protocols are supposed to worry about security in an end-to-end fashion. Specifying security gateways doesn't satisfy this requirement. The goal here is to provide something better than the current situation, where most people are using firewalls, and completely insecure protocols such as NFS. This is also known as the "hard crunchy exterior surrounding a soft, chewing interior", and it's pretty clear this isn't sufficient. New IETF protocols need to be secure even if you're not surrounded by a firewall, and are connected directly to the big, bad, Internet. The current situation vis-a-vis security is a really bad one; witness recent reports about people scanning Silicon Valley for open 802.11 networks, and finding ways of accessing corporate intranets for all sorts of very embarassed companies. And then there are the stories about machines running, PC Anywhere, backdoor modem connections that the IS department didn't know about, home DSL connections to the corporate intranet that were cross-connected with the user's cable modem, etc., etc., etc. So the answer to your question is that specifying a completely insecure protocol, and saying it's secure if you put a security gateway in front of it, isn't going to satisfy the IESG for new protocol specifications. - Ted From owner-ips@ece.cmu.edu Tue May 8 17:28:30 2001 Received: from ece.cmu.edu ([128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id RAA08059 for ; Tue, 8 May 2001 17:28:30 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f48JdRv16083 for ips-outgoing; Tue, 8 May 2001 15:39:27 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from msgbas1t.cos.agilent.com (msgbas1tx.cos.agilent.com [192.6.9.34]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f48JdQH16079 for ; Tue, 8 May 2001 15:39:26 -0400 (EDT) Received: from msgrel1.and.agilent.com (msgrel1.and.agilent.com [130.30.33.104]) by msgbas1t.cos.agilent.com (Postfix) with ESMTP id B31671C3 for ; Tue, 8 May 2001 13:39:21 -0600 (MDT) Received: from rtl.rose.agilent.com (rtl.rose.agilent.com [156.140.232.231]) by msgrel1.and.agilent.com (Postfix) with ESMTP id D592012F for ; Tue, 8 May 2001 15:39:20 -0400 (EDT) Received: from mail.rose.agilent.com (mailsrv@bellhop [156.140.233.51]) by rtl.rose.agilent.com (8.9.3 (PHNE_18979)/8.9.3 SMKit7.1.0) with ESMTP id MAA29575 for ; Tue, 8 May 2001 12:39:19 -0700 (PDT) Received: from agilent.com (matt5670.rose.agilent.com [156.140.235.204]) by mail.rose.agilent.com (Netscape Messaging Server 3.6) with ESMTP id AAA701E for ; Tue, 8 May 2001 12:39:17 -0700 Message-ID: <3AF84B7F.45425860@agilent.com> Date: Tue, 08 May 2001 12:39:43 -0700 From: "Matt Wakeley" Reply-To: Matt Wakeley Organization: Agilent Technologies X-Mailer: Mozilla 4.77 [en] (Windows NT 5.0; U) X-Accept-Language: en MIME-Version: 1.0 To: IPS Reflector Subject: Re: iSCSI: frame formats References: <20010411143722.152BC9400C@sandmail.sandburst.com> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-ips@ece.cmu.edu Precedence: bulk Content-Transfer-Encoding: 7bit Stephen Bailey wrote: > Understood. Yet another reason to insert integrity below iSCSI. > Disabling CRC is presently supported, so I'm not worried about the > CRCs. If the integrity is below iSCSI, then the "integrity" is stripped whenever it enters a middle box and added again when leaving, so your SCSI data is not protected while inside the middle box. -Matt From owner-ips@ece.cmu.edu Tue May 8 17:28:44 2001 Received: from ece.cmu.edu ([128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id RAA08071 for ; Tue, 8 May 2001 17:28:44 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f48KDOi19186 for ips-outgoing; Tue, 8 May 2001 16:13:24 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from server1.NishanSystems.COM (smtp.nishansystems.com [216.217.36.162]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f48KDMH19176 for ; Tue, 8 May 2001 16:13:22 -0400 (EDT) Received: by smtp.nishansystems.com with Internet Mail Service (5.5.2653.19) id ; Tue, 8 May 2001 13:13:15 -0700 Message-ID: From: Joshua Tseng To: "'John Hufferd'" Cc: "'Raghavendra Rao'" , ips@ece.cmu.edu Subject: RE: iSNS: Event registry and notification Date: Tue, 8 May 2001 13:13:13 -0700 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2653.19) Content-Type: text/plain; charset="iso-8859-1" Sender: owner-ips@ece.cmu.edu Precedence: bulk John, Yes, this is exactly what I was trying to say. Thanks, Josh > > > Josh, > I could be wrong here, but I think you may not have answered > the question > that Rao thought he was asking. Perhaps you should have said > that LUs and > therefore LUNs are NOT managed by the iSNS and have no direct > bearing on > the size of the DD. The reason I say that is that Host z, > being added to a > DD with x, and y, should not add additional DDs so in this > case, even with > 3 hosts, you still have one DD. > > Now you can both tell me I am wrong and I will slink away :-^ > > . > . > . > John L. Hufferd > Senior Technical Staff Member (STSM) > IBM/SSG San Jose Ca > (408) 256-0403, Tie: 276-0403, eFax: (408) 904-4688 > Internet address: hufferd@us.ibm.com > > > Joshua Tseng @ece.cmu.edu on > 05/08/2001 09:53:05 > AM > > Sent by: owner-ips@ece.cmu.edu > > > To: "'Raghavendra Rao'" , ips@ece.cmu.edu > cc: > Subject: RE: iSNS: Event registry and notification > > > > JP, > > > > Josh, > > > > > DD's define security/access control policy. If a new node > > > only had access permission to one other iSCSI node, then those > > > two nodes would be the only members of the DD. A node can be > > > a member of multiple DD's. If 255 devices are in the DD, that > > > implies that all 255 devices have permission to access each > > > > In my reading of the spec, my understanding was that DD > only provided > > a higher level access control and security but the final > > access control > > is > > actually decided by: > > a) Per Target's login access control > > b) Target's Logical Unit access control (i.e target > > would only list > > the LUs > > allowed for access in response to REPORT LU command) > > Your a) is correct. But by using iSNS, the target "slaves" its > login access control policy to the iSNS. If the target is placed > into a DD with initiators A, B, and C by the management GUI, then > the administrator is configuring that target for access by initiators > A, B, and C. The target's login access control policy would then > be configured by the iSNS to allow that access. > > b) is a function of SCSI. iSNS and iSCSI does not get involved > at that level. > > > > > Your clarification above seems to be saying that, if a target x, and > > target y > > exist inside of a DD of which initiator z is also a member, > then both > > targets > > x and y must provide login and Logical Unit access to initiator 'z'. > > Yes, this is correct if initiator 'z' were placed in the same DD > as target x and target y. > > > > > Is this correct ? If it is indeed true, then this results in > > creation of > > a huge number > > of DDs even in small networks (which I hate to see) - I > tend to think > > that > > more DDs you create, more you would need to administer and more you > > would > > need to manage (contrary to one's expectation) > > Yes, in a large storage network, there will be a large number of DDs. > That is why we added Discovery Domain Sets (DDS) to iSNS. > > Josh > > > > -JP > > > > > > > From owner-ips@ece.cmu.edu Tue May 8 17:30:50 2001 Received: from ece.cmu.edu ([128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id RAA08122 for ; Tue, 8 May 2001 17:30:50 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f48Jix516593 for ips-outgoing; Tue, 8 May 2001 15:44:59 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from msgbas1.cos.agilent.com (msgbas1x.cos.agilent.com [192.6.9.33]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f48JivH16576 for ; Tue, 8 May 2001 15:44:57 -0400 (EDT) Received: from msgrel1.and.agilent.com (msgrel1.and.agilent.com [130.30.33.104]) by msgbas1.cos.agilent.com (Postfix) with ESMTP id 05A4713B0 for ; Tue, 8 May 2001 13:44:57 -0600 (MDT) Received: from rtl.rose.agilent.com (rtl.rose.agilent.com [156.140.232.231]) by msgrel1.and.agilent.com (Postfix) with ESMTP id 3CFF910F for ; Tue, 8 May 2001 15:44:56 -0400 (EDT) Received: from mail.rose.agilent.com (mailsrv@bellhop [156.140.233.51]) by rtl.rose.agilent.com (8.9.3 (PHNE_18979)/8.9.3 SMKit7.1.0) with ESMTP id MAA29682 for ; Tue, 8 May 2001 12:44:55 -0700 (PDT) Received: from agilent.com (matt5670.rose.agilent.com [156.140.235.204]) by mail.rose.agilent.com (Netscape Messaging Server 3.6) with ESMTP id AAA70B9 for ; Tue, 8 May 2001 12:44:52 -0700 Message-ID: <3AF84CCF.F657BB48@agilent.com> Date: Tue, 08 May 2001 12:45:19 -0700 From: "Matt Wakeley" Reply-To: Matt Wakeley Organization: Agilent Technologies X-Mailer: Mozilla 4.77 [en] (Windows NT 5.0; U) X-Accept-Language: en MIME-Version: 1.0 To: ips@ece.cmu.edu Subject: Re: Initiator-detected format or digest errors References: Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-ips@ece.cmu.edu Precedence: bulk Content-Transfer-Encoding: 7bit The target can log the reject for further analysis. Why do you keep trying to eliminate simple things that can be used for debugging? -Matt julian_satran@il.ibm.com wrote: > > Matt, > > 3 reasons: > > a. the initiator drives recovery. What can a target do with the reject? > (he is probaly in need of a reset) > > b.the initiator is unaware of the target state after thhe error > > c.there is reject command (and response) -:) > > Julo > > Matt Wakeley on 06/04/2001 00:43:08 > > Please respond to Matt Wakeley > > To: IPS Reflector > cc: > Subject: Re: Initiator-detected format or digest errors > > Why not send a reject back to the originator of the bad frame, and just let > the task time out? > > -Matt > > julian_satran@il.ibm.com wrote: > > > > Tom, > > > > An initiator will pass the appropriate response to the SCSI layer and > will > > abort the task if it can identify one. Further behavior of initiators > and > > targets is implementation dependent. > > > > 6.2 specifies this. > > > > Julo > > > > "Thomas McSweeney" on 04/04/2001 13:57:20 > > > > Please respond to "Thomas McSweeney" > > > > To: ips@ece.cmu.edu > > cc: > > Subject: Initiator-detected format or digest errors > > > > Section "2.20 Reject" talks about the target receiving a bad frame and > > sending Reject. What should an Initiator do if it receives a PDU with a > > format or digest error? Should it send Reject? If so, we'll need to > > ensure that the Initiator fields of the MIB include an object to count > > Reject commands transmitted. > > > > Tom McSweeney > > iSCSI Development, Storage Systems Group, IBM > > Email: rf42tpme@us.ibm.com > > Phone: (USA) 919-254-5634 (tie line: 444-5634) > > Fax: (USA) 919-254-0391 (tie line: 444-0391) From owner-ips@ece.cmu.edu Tue May 8 17:31:49 2001 Received: from ece.cmu.edu ([128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id RAA08142 for ; Tue, 8 May 2001 17:31:49 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f48K9Sg18869 for ips-outgoing; Tue, 8 May 2001 16:09:28 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from e31.bld.us.ibm.com (e31.co.us.ibm.com [32.97.110.129]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f48K9QH18864 for ; Tue, 8 May 2001 16:09:26 -0400 (EDT) Received: from westrelay02.boulder.ibm.com (westrelay02.boulder.ibm.com [9.99.140.23]) by e31.bld.us.ibm.com (8.9.3/8.9.3) with ESMTP id QAA22242; Tue, 8 May 2001 16:01:54 -0400 Received: from f4n49e (d03nm065h.boulder.ibm.com [9.99.140.49]) by westrelay02.boulder.ibm.com (8.11.1/NCO v4.96.1.0) with ESMTP id f48K9Lc167542; Tue, 8 May 2001 14:09:21 -0600 X-Priority: 1 (High) Importance: Normal Subject: RE: iSNS: Event registry and notification To: Joshua Tseng Cc: "'Raghavendra Rao'" , ips@ece.cmu.edu X-Mailer: Lotus Notes Release 5.0.3 (Intl) 21 March 2000 Message-ID: From: "John Hufferd" Date: Tue, 8 May 2001 13:09:03 -0700 X-MIMETrack: Serialize by Router on D03NM065/03/M/IBM(Release 5.0.6 |December 14, 2000) at 05/08/2001 02:09:20 PM MIME-Version: 1.0 Content-type: text/plain; charset=us-ascii Sender: owner-ips@ece.cmu.edu Precedence: bulk Josh, I could be wrong here, but I think you may not have answered the question that Rao thought he was asking. Perhaps you should have said that LUs and therefore LUNs are NOT managed by the iSNS and have no direct bearing on the size of the DD. The reason I say that is that Host z, being added to a DD with x, and y, should not add additional DDs so in this case, even with 3 hosts, you still have one DD. Now you can both tell me I am wrong and I will slink away :-^ . . . John L. Hufferd Senior Technical Staff Member (STSM) IBM/SSG San Jose Ca (408) 256-0403, Tie: 276-0403, eFax: (408) 904-4688 Internet address: hufferd@us.ibm.com Joshua Tseng @ece.cmu.edu on 05/08/2001 09:53:05 AM Sent by: owner-ips@ece.cmu.edu To: "'Raghavendra Rao'" , ips@ece.cmu.edu cc: Subject: RE: iSNS: Event registry and notification JP, > > Josh, > > > DD's define security/access control policy. If a new node > > only had access permission to one other iSCSI node, then those > > two nodes would be the only members of the DD. A node can be > > a member of multiple DD's. If 255 devices are in the DD, that > > implies that all 255 devices have permission to access each > > In my reading of the spec, my understanding was that DD only provided > a higher level access control and security but the final > access control > is > actually decided by: > a) Per Target's login access control > b) Target's Logical Unit access control (i.e target > would only list > the LUs > allowed for access in response to REPORT LU command) Your a) is correct. But by using iSNS, the target "slaves" its login access control policy to the iSNS. If the target is placed into a DD with initiators A, B, and C by the management GUI, then the administrator is configuring that target for access by initiators A, B, and C. The target's login access control policy would then be configured by the iSNS to allow that access. b) is a function of SCSI. iSNS and iSCSI does not get involved at that level. > > Your clarification above seems to be saying that, if a target x, and > target y > exist inside of a DD of which initiator z is also a member, then both > targets > x and y must provide login and Logical Unit access to initiator 'z'. Yes, this is correct if initiator 'z' were placed in the same DD as target x and target y. > > Is this correct ? If it is indeed true, then this results in > creation of > a huge number > of DDs even in small networks (which I hate to see) - I tend to think > that > more DDs you create, more you would need to administer and more you > would > need to manage (contrary to one's expectation) Yes, in a large storage network, there will be a large number of DDs. That is why we added Discovery Domain Sets (DDS) to iSNS. Josh > > -JP > > From owner-ips@ece.cmu.edu Tue May 8 17:32:56 2001 Received: from ece.cmu.edu ([128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id RAA08159 for ; Tue, 8 May 2001 17:32:56 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f48JTNb15067 for ips-outgoing; Tue, 8 May 2001 15:29:23 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from msgbas1.cos.agilent.com (msgbas1x.cos.agilent.com [192.6.9.33]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f48JTMH15061 for ; Tue, 8 May 2001 15:29:22 -0400 (EDT) Received: from msgrel1.and.agilent.com (msgrel1.and.agilent.com [130.30.33.104]) by msgbas1.cos.agilent.com (Postfix) with ESMTP id 4598C1386 for ; Tue, 8 May 2001 13:29:21 -0600 (MDT) Received: from rtl.rose.agilent.com (rtl.rose.agilent.com [156.140.232.231]) by msgrel1.and.agilent.com (Postfix) with ESMTP id 4E472129 for ; Tue, 8 May 2001 15:29:20 -0400 (EDT) Received: from mail.rose.agilent.com (mailsrv@bellhop [156.140.233.51]) by rtl.rose.agilent.com (8.9.3 (PHNE_18979)/8.9.3 SMKit7.1.0) with ESMTP id MAA28872 for ; Tue, 8 May 2001 12:29:19 -0700 (PDT) Received: from agilent.com (matt5670.rose.agilent.com [156.140.235.204]) by mail.rose.agilent.com (Netscape Messaging Server 3.6) with ESMTP id AAA6EFF for ; Tue, 8 May 2001 12:29:16 -0700 Message-ID: <3AF84927.1DD0F8BF@agilent.com> Date: Tue, 08 May 2001 12:29:43 -0700 From: "Matt Wakeley" Reply-To: Matt Wakeley Organization: Agilent Technologies X-Mailer: Mozilla 4.77 [en] (Windows NT 5.0; U) X-Accept-Language: en MIME-Version: 1.0 To: IPS Reflector Subject: Re: iSCSI: immediate data References: <200105051355.JAA22113@aura.research.bell-labs.com> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-ips@ece.cmu.edu Precedence: bulk Content-Transfer-Encoding: 7bit The intention is that if an initiator requests to send immediate data (and is granted the request), then it will always send immediate data. It sounds like you are asking for wishy washy mode... sometimes send immediate data, sometimes not. -Matt Sandeep Joshi wrote: > > Julian, > > I had a follow-up question on an old thread. > http://ips.pdl.cs.cmu.edu/mail/msg03373.html > > The initiator is allowed to send firstburst in immediate > or in separate PDUs...but can it do neither ? I dont > see any statement to the effect that it MUST send a > firstburst. > > Here's a possible problem : > 1) FirstBurst is 4K > 2) Expected data length of command is 16K > 3) Initiator sends the Scsi command without immediate > data. (final flag on Scsi command is not set) > And it decides to send no further data PDUs. > > The target does not know if R2Ts can be sent since it does > not know if any unsolicited PDUs are expected. > > Perhaps the semantics on the final flag on SCSI command > need to be changed to indicate that no unsolicited data > will be sent with this command. Currently, the flag implies > (Sec 2.3.1) that all the required data has been sent with > the command. > > And here's a typo to fix. Appendix E (Key=immediateData) > The following should say "initialR2T is no". > The table has got it right. > > > If ImmediateData is set to yes and InitialR2T is set > > to yes then the initiator MAY send unsolicited immediate > > data or one unsolicited burst of Data-OUT PDUs but > > MUST NOT send both immediate and a unsolicited burst of > > Data-OUT PDUs for any one command. > > -Sandeep From owner-ips@ece.cmu.edu Tue May 8 19:50:24 2001 Received: from ece.cmu.edu ([128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id TAA09985 for ; Tue, 8 May 2001 19:50:24 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f48LvMn28570 for ips-outgoing; Tue, 8 May 2001 17:57:22 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from mailcity.com (fes.whowhere.com [209.185.123.154]) by ece.cmu.edu (8.11.0/8.10.2) with SMTP id f48LvKH28566 for ; Tue, 8 May 2001 17:57:20 -0400 (EDT) Received: from Unknown/Local ([?.?.?.?]) by mailcity.com; Tue May 8 14:56:58 2001 To: ips@ece.cmu.edu Date: Tue, 08 May 2001 14:56:58 -0700 From: "mehyar nima" Message-ID: Mime-Version: 1.0 X-Sent-Mail: on Reply-To: m_nima_1@lycos.com X-Mailer: MailCity Service Subject: RE: iSCSI: Nailing down CRC-32C X-Sender-Ip: 63.70.210.20 Organization: Lycos Mail (http://mail.lycos.com:80) Content-Type: text/plain; charset=us-ascii Content-Language: en Content-Transfer-Encoding: 7bit Sender: owner-ips@ece.cmu.edu Precedence: bulk Content-Transfer-Encoding: 7bit Assumptions: 1) The iSCSI DATA PDU Header and Data Integrity checks use CRC-32 2) Support of Integrity check is mandatory for both header and Data digest 3) An iSCSI PDU may span multiple TCP segments. 4) Any of these segments may arrive out of order at the receiver. 5) To regenerate the CRC for comparison with the value present in the PDU received, the receiver needs to keep in a temporary buffers all segments, waiting for the missing one, as CRC must be computed using the whole re-assembled PDU Question: Does CRC-32 support partial computation and compensation for wrong initial values later (I guess NO!), to enable partial CRC calc on avail parts of PDU? Possible answer: No Possible Conclusion: The RCV side is further complicated b/c of this CRC Possible solution: 1) Use CRC that allows partial calc 2) Limit iSCSI PDU to TCP MSS if data CRC is used (added o/h)... 3) Leave as it, though it has cost , performance implication and hurts scaling to 10G.. Nima -----Original Message----- From: julian_satran@il.ibm.com [mailto:julian_satran@il.ibm.com] Sent: Saturday, May 05, 2001 9:21 AM To: ips@ece.cmu.edu Subject: Re: iSCSI: Nailing down CRC-32C The CRC part of the appendix (for 07) reads: The following table lists cyclic integrity checksums that can be negotiated for the digests and MUST be implemented by every iSCSI initiator and target. Note that these digest options have only error detection significance. +---------------------------------------------+ | Name | Description | +---------------------------------------------+ | crc-32C | 32 bit CRC | 11EDC6F41 | +---------------------------------------------+ | none | no digest | +---------------------------------------------+ The generator polynomials for those digests are given in hex-notation, for example 3a stands for 0011 1010 - the polynomial x**5+X**4+x**3+x+1. The generator polynomial selected is evaluated in [Castagnioli93]. When using the CRC the CRC register must be initialized to all 1s (0xFFFFFFFF) and the CRC bits must be complemented before transmission. Padding bytes, when presents in a segment covered by a CRC, have to be set to 0 and are included in the CRC. Regards, Julo Mark Bakke on 05-05-2001 00:13:09 Please respond to Mark Bakke To: IPS cc: Subject: iSCSI: Nailing down CRC-32C At the interim meeting, it was stated that iSCSI has selected the CRC-32C polynomial as its required iSCSI-level header and data CRC. Now that we have it, it's time to move on and make sure we can implement it. In the interest of interoperability, we need to not only specify the polynomial, but also the initial values, bit and byte ordering, etc. "A Painless Guide to CRC Error Detection Algorithms" (http://www.ross.net/crc/crcpaper.html) specifies a method to unambiguously characterize these parameters (sections 15 and 16). Has anyone taken a shot at defining these yet? Otherwise, here is what it might look like: Name : "CRC-32C" Width : 32 Poly : 1EDC6F41 (note that the leading "1" is implied) Init : FFFFFFFF RefIn : True RefOut : True XorOut : FFFFFFFF Check : ? I haven't attempted to create check data based on these yet. As soon as the other parameters are nailed down, we need to do this. Anyway, I am not a CRC expert, and can't make any statement about the above values being the "best" way to do this, but instead just copied them (except the polynomial itself) from the Ethernet CRC, since that is likely the easiest for everyone implementing hardware to deal with. If someone else is already doing this, let me know; I just wanted to start this thread so we can get closure. Regards, Mark -- Mark A. Bakke Cisco Systems mbakke@cisco.com 763.398.1054 Get 250 color business cards for FREE! http://businesscards.lycos.com/vp/fastpath/ From owner-ips@ece.cmu.edu Tue May 8 21:21:04 2001 Received: from ece.cmu.edu ([128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id VAA11021 for ; Tue, 8 May 2001 21:21:04 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f48MvtN03353 for ips-outgoing; Tue, 8 May 2001 18:57:55 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from dirty.research.bell-labs.com (dirty.research.bell-labs.com [204.178.16.6]) by ece.cmu.edu (8.11.0/8.10.2) with SMTP id f48MvqH03348 for ; Tue, 8 May 2001 18:57:52 -0400 (EDT) Received: from grubby.research.bell-labs.com ([135.104.2.9]) by dirty; Tue May 8 18:56:36 EDT 2001 Received: from aura.research.bell-labs.com ([135.104.46.10]) by grubby; Tue May 8 18:56:31 EDT 2001 Received: (from sandeepj@localhost) by aura.research.bell-labs.com (8.9.1/8.9.1) id SAA00747; Tue, 8 May 2001 18:56:25 -0400 (EDT) Date: Tue, 8 May 2001 18:56:25 -0400 (EDT) Message-Id: <200105082256.SAA00747@aura.research.bell-labs.com> From: sandeepj@research.bell-labs.com (Sandeep Joshi) To: matt_wakeley@agilent.com Cc: ips@ece.cmu.edu Subject: Re: iSCSI: immediate data Sender: owner-ips@ece.cmu.edu Precedence: bulk Umm.....Julian replied to this one and said the final bit on ScsiCmd would indicate it. Apparently, a single command can now send the unsolicited data both with the command and in separate PDUs. I cant seem to find the email but I believe Sec 2.3.1 was changed accordingly. Julian..? -Sandeep > The intention is that if an initiator requests to send immediate data (and is > granted the request), then it will always send immediate data. > > It sounds like you are asking for wishy washy mode... sometimes send immediate > data, sometimes not. > > -Matt > > Sandeep Joshi wrote: > > > > Julian, > > > > I had a follow-up question on an old thread. > > http://ips.pdl.cs.cmu.edu/mail/msg03373.html > > > > The initiator is allowed to send firstburst in immediate > > or in separate PDUs...but can it do neither ? I dont > > see any statement to the effect that it MUST send a > > firstburst. > > > > Here's a possible problem : > > 1) FirstBurst is 4K > > 2) Expected data length of command is 16K > > 3) Initiator sends the Scsi command without immediate > > data. (final flag on Scsi command is not set) > > And it decides to send no further data PDUs. > > > > The target does not know if R2Ts can be sent since it does > > not know if any unsolicited PDUs are expected. > > > > Perhaps the semantics on the final flag on SCSI command > > need to be changed to indicate that no unsolicited data > > will be sent with this command. Currently, the flag implies > > (Sec 2.3.1) that all the required data has been sent with > > the command. > > > > And here's a typo to fix. Appendix E (Key=immediateData) > > The following should say "initialR2T is no". > > The table has got it right. > > > > > If ImmediateData is set to yes and InitialR2T is set > > > to yes then the initiator MAY send unsolicited immediate > > > data or one unsolicited burst of Data-OUT PDUs but > > > MUST NOT send both immediate and a unsolicited burst of > > > Data-OUT PDUs for any one command. > > > > -Sandeep From owner-ips@ece.cmu.edu Tue May 8 22:33:19 2001 Received: from ece.cmu.edu ([128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id WAA13543 for ; Tue, 8 May 2001 22:33:19 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f490Sip10193 for ips-outgoing; Tue, 8 May 2001 20:28:44 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from palrel1.hp.com (palrel1.hp.com [156.153.255.242]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f490ScH10186 for ; Tue, 8 May 2001 20:28:43 -0400 (EDT) Received: from hpcuhe.cup.hp.com (hpcuhe.cup.hp.com [15.0.80.203]) by palrel1.hp.com (Postfix) with ESMTP id D703313C3; Tue, 8 May 2001 17:28:34 -0700 (PDT) Received: from cup.hp.com (santoshr@hpindhhm.cup.hp.com [15.8.80.197]) by hpcuhe.cup.hp.com (8.9.3 (PHNE_18979)/8.9.3 SMKit7.02) with ESMTP id RAA02189; Tue, 8 May 2001 17:28:11 -0700 (PDT) Message-ID: <3AF88F12.6B3F058E@cup.hp.com> Date: Tue, 08 May 2001 17:28:02 -0700 From: Santosh Rao Organization: Hewlett Packard, Cupertino. X-Mailer: Mozilla 4.7 [en] (X11; U; HP-UX B.11.00 9000/778) X-Accept-Language: en MIME-Version: 1.0 To: Matt Wakeley , Julian Satran Cc: ips@ece.cmu.edu Subject: Re: Initiator-detected format or digest errors References: <3AF84CCF.F657BB48@agilent.com> Content-Type: multipart/mixed; boundary="------------0C3272B0042ADA2DBA3B31F7" Sender: owner-ips@ece.cmu.edu Precedence: bulk This is a multi-part message in MIME format. --------------0C3272B0042ADA2DBA3B31F7 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Why can't iSCSI apply a simple policy of discarding any PDU that has a digest error or a format error ? Can the draft allow for such implementations that wish to discard the PDU and do nothing more ? This is all that FC initiators and targets need to do and they can recover fine from CRC errors ! A dropped PDU will result in one of the following scenarios : --------------------------------------------------------------- format error or digest error impacts Scenario --------------------------------------------------------------- i) iSCSI Command Initiator timeout. (could be detected early on when initiator retries command on seeing a large diff b/n ExpCmdSN and current CmdSN. ii) READ Data PDU I/O underrun, detected thru a mis-match b/n count of DataSNs received and EndDataSN. iii) WRITE Data PDU If this is not the "F" data PDU for that R2T data phase, this results in a target timoeut followed by target aborting the I/O at the end of R2T data phase and sending an aborted response back. If this is a "F" data PDU, results in initiator timeout. (scenario 1). iv) R2T PDU Target timeout. Initiator timeout. v) Status PDU Initiator timeout. From an implemenation's perspective, generating all of these REJECT commands is just adding overhead and complexity, requiring the allocation of resources [possibly by the host] for a REJECT PDU and having to transmit a REJECT in response to each PDU that had either a format or digest error. Any diagnostic info can be logged as counters within the implementation's MIB. A format error or digest error causes the PDU to be discarded. As it is, we need to apply a discard policy for the "header digest error" & format error cases. Simplification of all digest error and format error handling to a "discard policy" keeps implementations [and the protocol] simple. - Santosh Matt Wakeley wrote: > > The target can log the reject for further analysis. > > Why do you keep trying to eliminate simple things that can be used for > debugging? > > -Matt > > julian_satran@il.ibm.com wrote: > > > > Matt, > > > > 3 reasons: > > > > a. the initiator drives recovery. What can a target do with the reject? > > (he is probaly in need of a reset) > > > > b.the initiator is unaware of the target state after thhe error > > > > c.there is reject command (and response) -:) > > > > Julo > > > > Matt Wakeley on 06/04/2001 00:43:08 > > > > Please respond to Matt Wakeley > > > > To: IPS Reflector > > cc: > > Subject: Re: Initiator-detected format or digest errors > > > > Why not send a reject back to the originator of the bad frame, and just let > > the task time out? > > > > -Matt > > > > julian_satran@il.ibm.com wrote: > > > > > > Tom, > > > > > > An initiator will pass the appropriate response to the SCSI layer and > > will > > > abort the task if it can identify one. Further behavior of initiators > > and > > > targets is implementation dependent. > > > > > > 6.2 specifies this. > > > > > > Julo > > > > > > "Thomas McSweeney" on 04/04/2001 13:57:20 > > > > > > Please respond to "Thomas McSweeney" > > > > > > To: ips@ece.cmu.edu > > > cc: > > > Subject: Initiator-detected format or digest errors > > > > > > Section "2.20 Reject" talks about the target receiving a bad frame and > > > sending Reject. What should an Initiator do if it receives a PDU with a > > > format or digest error? Should it send Reject? If so, we'll need to > > > ensure that the Initiator fields of the MIB include an object to count > > > Reject commands transmitted. > > > > > > Tom McSweeney > > > iSCSI Development, Storage Systems Group, IBM > > > Email: rf42tpme@us.ibm.com > > > Phone: (USA) 919-254-5634 (tie line: 444-5634) > > > Fax: (USA) 919-254-0391 (tie line: 444-0391) --------------0C3272B0042ADA2DBA3B31F7 Content-Type: text/x-vcard; charset=us-ascii; name="santoshr.vcf" Content-Description: Card for Santosh Rao Content-Disposition: attachment; filename="santoshr.vcf" Content-Transfer-Encoding: 7bit begin:vcard n:Rao;Santosh tel;work:408-447-3751 x-mozilla-html:FALSE org:Hewlett Packard, Cupertino.;SISL adr:;;19420, Homestead Road, M\S 43LN, ;Cupertino.;CA.;95014.;USA. version:2.1 email;internet:santoshr@cup.hp.com title:Software Design Engineer x-mozilla-cpt:;21088 fn:Santosh Rao end:vcard --------------0C3272B0042ADA2DBA3B31F7-- From owner-ips@ece.cmu.edu Tue May 8 22:33:57 2001 Received: from ece.cmu.edu ([128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id WAA13554 for ; Tue, 8 May 2001 22:33:56 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f490rMR12124 for ips-outgoing; Tue, 8 May 2001 20:53:22 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from palrel2.hp.com (palrel2.hp.com [156.153.255.234]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f490rKH12119 for ; Tue, 8 May 2001 20:53:20 -0400 (EDT) Received: from hpcuhe.cup.hp.com (hpcuhe.cup.hp.com [15.0.80.203]) by palrel2.hp.com (Postfix) with ESMTP id A4F0597A for ; Tue, 8 May 2001 17:53:19 -0700 (PDT) Received: from cup.hp.com (santoshr@hpindhhm.cup.hp.com [15.8.80.197]) by hpcuhe.cup.hp.com (8.9.3 (PHNE_18979)/8.9.3 SMKit7.02) with ESMTP id RAA03898 for ; Tue, 8 May 2001 17:53:12 -0700 (PDT) Message-ID: <3AF894EF.EDF7424F@cup.hp.com> Date: Tue, 08 May 2001 17:53:03 -0700 From: Santosh Rao Organization: Hewlett Packard, Cupertino. X-Mailer: Mozilla 4.7 [en] (X11; U; HP-UX B.11.00 9000/778) X-Accept-Language: en MIME-Version: 1.0 To: ips@ece.cmu.edu Subject: Re: iSCSI: multiple sessions b/n a pair of WWUIs. References: <3AF892F9.ACCF06BE@cup.hp.com> Content-Type: multipart/mixed; boundary="------------86CD2C3D36BB9761DF6DD64E" Sender: owner-ips@ece.cmu.edu Precedence: bulk This is a multi-part message in MIME format. --------------86CD2C3D36BB9761DF6DD64E Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Santosh Rao wrote: > As for the uniqueness of ISID across all initiators, is there any reason ^^^^^^^^^^^^^^^ should read "across all targets" > not to allow implementations to do that ? I would have thought that is > the most typical use of an ISID and also allows initiators to lookup > their target structures based on ISID. --------------86CD2C3D36BB9761DF6DD64E Content-Type: text/x-vcard; charset=us-ascii; name="santoshr.vcf" Content-Description: Card for Santosh Rao Content-Disposition: attachment; filename="santoshr.vcf" Content-Transfer-Encoding: 7bit begin:vcard n:Rao;Santosh tel;work:408-447-3751 x-mozilla-html:FALSE org:Hewlett Packard, Cupertino.;SISL adr:;;19420, Homestead Road, M\S 43LN, ;Cupertino.;CA.;95014.;USA. version:2.1 email;internet:santoshr@cup.hp.com title:Software Design Engineer x-mozilla-cpt:;21088 fn:Santosh Rao end:vcard --------------86CD2C3D36BB9761DF6DD64E-- From owner-ips@ece.cmu.edu Tue May 8 22:33:57 2001 Received: from ece.cmu.edu ([128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id WAA13565 for ; Tue, 8 May 2001 22:33:57 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f490b9V10790 for ips-outgoing; Tue, 8 May 2001 20:37:09 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from palrel1.hp.com (palrel1.hp.com [156.153.255.242]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f490b7H10781 for ; Tue, 8 May 2001 20:37:07 -0400 (EDT) Received: from hpcuhe.cup.hp.com (hpcuhe.cup.hp.com [15.0.80.203]) by palrel1.hp.com (Postfix) with ESMTP id 2621BC42; Tue, 8 May 2001 17:37:05 -0700 (PDT) Received: from cup.hp.com (santoshr@hpindhhm.cup.hp.com [15.8.80.197]) by hpcuhe.cup.hp.com (8.9.3 (PHNE_18979)/8.9.3 SMKit7.02) with ESMTP id RAA02656; Tue, 8 May 2001 17:37:00 -0700 (PDT) Message-ID: <3AF89123.9368281@cup.hp.com> Date: Tue, 08 May 2001 17:36:51 -0700 From: Santosh Rao Organization: Hewlett Packard, Cupertino. X-Mailer: Mozilla 4.7 [en] (X11; U; HP-UX B.11.00 9000/778) X-Accept-Language: en MIME-Version: 1.0 To: Black_David@emc.com Cc: ips@ece.cmu.edu Subject: Re: iSCSI : login keys & mode page settings References: <0F31E5C394DAD311B60C00E029101A0708015538@corpmx9.isus.emc.com> Content-Type: multipart/mixed; boundary="------------370E1B75920B5A6AA8E5BEBC" Sender: owner-ips@ece.cmu.edu Precedence: bulk This is a multi-part message in MIME format. --------------370E1B75920B5A6AA8E5BEBC Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Black_David@emc.com wrote: > (1) Is changing an iSCSI key allowed to cause > problems for other initiators? No. > (2) Does the iSCSI key mechanism have to behave > identically to the corresponding mode > page mechanism. No. Better still, the mode page mechanism can be dis-allowed [or strongly discouraged] by iSCSI, allowing only 1 mechanism for setting these options, which is through the use of login keys. Something to the effect of : "Initiators SHOULD [MUST ?] NOT use Mode Select to modify these contol options and any key negotiation SHOULD [MUST ?] be done through login/text keys" may help address these concerns. Thanks, Santosh > > Given the need to support old systems that may > get (1) wrong (mode page sets can damage other > initiators), the best we may be able to do on it > is a SHOULD: > (1) SHOULD not share key values among sessions > (i.e., setting of a key value in one session > SHOULD NOT affect the setting in any other > session. > On (2), how about > - When a key refers to a mode page entry, the > underlying value MUST be shared between > the mode page and the key in an iSCSI session > (e.g., a value set by a text key MUST be > retrieved by the mode page if the implementation > accepted the value). > - Restrictions on value changes in full-feature > mode SHOULD (MUST?) match when a value is > shared between a text key and a mode page > entry. > > Comments? > > --David > > --------------------------------------------------- > David L. Black, Senior Technologist > EMC Corporation, 42 South St., Hopkinton, MA 01748 > +1 (508) 435-1000 x75140 FAX: +1 (508) 497-8500 > black_david@emc.com Mobile: +1 (978) 394-7754 > --------------------------------------------------- > > > -----Original Message----- > > From: Santosh Rao [SMTP:santoshr@cup.hp.com] > > Sent: Friday, May 04, 2001 2:15 PM > > To: Black_David@emc.com; ips@ece.cmu.edu > > Subject: RE: iSCSI : login keys & mode page settings > > > > David, > > > > Apologies for the late response on this. I was hoping we could complete > > this > > thread of discussion at Nashua, but for lack of time, we are back on the > > list. > > > > Regarding your question below : > > > > > If one Initiator can damage another in this fashion, then we > > > may indeed have a problem. > > > > > Comments?, > > > > Shared mode page implementations in targets is quite common and > > modification of > > control parameters through a mode select would indeed affect all other > > initiators logged into the target. This is not desirable behaviour and > > iSCSI > > may be better served using login/text based key negotiation rather than > > the > > mode select mechanisms which opens it up to the side effects of affecting > > all > > connected initiators. > > > > Thanks, > > Santosh > > > > > > ------------------------------------------------------------------------ > > Subject: RE: iSCSI : login keys & mode page settings > > From: Black_David@emc.com > > Date: Fri, 20 Apr 2001 20:32:17 -0400 > > Cc: ips@ece.cmu.edu > > > > I'm not sure -- this sounds somewhat like the > > old principle of not asking why there's a hole > > in one's foot when one has aimed the gun at > > it and pulled the trigger. For the tape > > example, if some tape driver changes a > > Target iSCSI parameter that disrupts that > > driver's own tape I/O in a fashion that the > > driver can't recover from, I think it's > > clear where the fault lies. If one Initiator > > can damage another in this fashion, then we > > may indeed have a problem. > > > > Comments?, > > --David > > > > > -----Original Message----- > > > From: Santosh Rao [SMTP:santoshr@cup.hp.com] > > > Sent: Friday, April 20, 2001 8:09 PM > > > To: Black_David@emc.com > > > Cc: ips@ece.cmu.edu > > > Subject: Re: iSCSI : login keys & mode page settings > > > > > > David, > > > > > > Some clarification on the basis for classifying login > > ould > > > also be helpful. Should login keys that can disrupt > > > I/O on their change > > > be allowed to be non-LO ? > > > > > > Thanks, > > > Santosh > > > > > > Black_David@emc.com wrote: > > > > > > > > Without getting into the technical details of the > > > > discussion, I have a couple of observations: > > > > > > > > (A) The issue of whether to allow mode page > > > > access to and modification of iSCSI > > ers > > > > will need to be taken up at the interim > > > > meeting. IMHO, access seems like a good > > > > idea, so that SCSI-generic code that doesn't > > > > know specifically about iSCSI can find > > > > what it expects where it expects it, but > > > > I'm unsure about modification because it > > > > may carry a risk of code that's > > naware > > > > getting something wrong. The mode page > > > > commands should be transparent to iSCSI. > > > > > > > > (B) The mode page and text key mechanisms have > > > > to access the same data. Section 3 of the > > > > -06 version says this, but needs some > > > > editing > > > > to enforce it by using "MUST" or its > > > > equivalent > > > > (cf. RFC 2119). This is to prevent an > > > > implementation from having two instances of > > > > the same parameter - one for the mode page and > > > > one for the text keys - which would be a bad > > > > thing. > > > > > > > > --David --------------370E1B75920B5A6AA8E5BEBC Content-Type: text/x-vcard; charset=us-ascii; name="santoshr.vcf" Content-Description: Card for Santosh Rao Content-Disposition: attachment; filename="santoshr.vcf" Content-Transfer-Encoding: 7bit begin:vcard n:Rao;Santosh tel;work:408-447-3751 x-mozilla-html:FALSE org:Hewlett Packard, Cupertino.;SISL adr:;;19420, Homestead Road, M\S 43LN, ;Cupertino.;CA.;95014.;USA. version:2.1 email;internet:santoshr@cup.hp.com title:Software Design Engineer x-mozilla-cpt:;21088 fn:Santosh Rao end:vcard --------------370E1B75920B5A6AA8E5BEBC-- From owner-ips@ece.cmu.edu Tue May 8 22:37:00 2001 Received: from ece.cmu.edu ([128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id WAA13579 for ; Tue, 8 May 2001 22:37:00 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f490iwm11380 for ips-outgoing; Tue, 8 May 2001 20:44:58 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from palrel2.hp.com (palrel2.hp.com [156.153.255.234]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f490iuH11373 for ; Tue, 8 May 2001 20:44:57 -0400 (EDT) Received: from hpcuhe.cup.hp.com (hpcuhe.cup.hp.com [15.0.80.203]) by palrel2.hp.com (Postfix) with ESMTP id F04E61267; Tue, 8 May 2001 17:44:55 -0700 (PDT) Received: from cup.hp.com (santoshr@hpindhhm.cup.hp.com [15.8.80.197]) by hpcuhe.cup.hp.com (8.9.3 (PHNE_18979)/8.9.3 SMKit7.02) with ESMTP id RAA03307; Tue, 8 May 2001 17:44:50 -0700 (PDT) Message-ID: <3AF892F9.ACCF06BE@cup.hp.com> Date: Tue, 08 May 2001 17:44:41 -0700 From: Santosh Rao Organization: Hewlett Packard, Cupertino. X-Mailer: Mozilla 4.7 [en] (X11; U; HP-UX B.11.00 9000/778) X-Accept-Language: en MIME-Version: 1.0 To: John Hufferd Cc: Black_David@emc.com, ips@ece.cmu.edu Subject: Re: iSCSI: multiple sessions b/n a pair of WWUIs. References: Content-Type: multipart/mixed; boundary="------------F1B114CE8B903A7E527EE8E8" Sender: owner-ips@ece.cmu.edu Precedence: bulk This is a multi-part message in MIME format. --------------F1B114CE8B903A7E527EE8E8 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit John, By the name-disc definition, an iSCSI initiator or target port is a logical entity that constitutes the end points of a session. Hence, there never can be 2 sessions b/n the *same* 2 initiator & target port. Each session established spawns a *new* initiator and target iSCSI port. Going by the above semantics, there is always only 1 ISID/TSID per iSCSI Initiator/Target port. Hence, I'm unable to understand your statements below. As for the uniqueness of ISID across all initiators, is there any reason not to allow implementations to do that ? I would have thought that is the most typical use of an ISID and also allows initiators to lookup their target structures based on ISID. Regards, Santosh John Hufferd wrote: > > Also, since a second session by the same iSCSI Initiator Port, to the same > iSCSI Target Port is not, in my opinion "legal", it is not clear to me that > any given iSCSI Port needs any more then one ISID. I > > Therefore, I suggest that we Not put any such notes, like you suggest, in > the draft, but in fact encourage a single ISID/TSID per iSCSI > Initiator/Target Port. > To: Jim Hafner/Almaden/IBM@IBMUS, marjorie_krueger@hp.com > cc: ips@ece.cmu.edu > Subject: RE: iSCSI: multiple sessions b/n a pair of WWUIs. > > While Jim's correct ... in Marj's defense, what > she wrote is a fairly obvious way to implement > it, and that might be worth noting in the draft. > > --David --------------F1B114CE8B903A7E527EE8E8 Content-Type: text/x-vcard; charset=us-ascii; name="santoshr.vcf" Content-Description: Card for Santosh Rao Content-Disposition: attachment; filename="santoshr.vcf" Content-Transfer-Encoding: 7bit begin:vcard n:Rao;Santosh tel;work:408-447-3751 x-mozilla-html:FALSE org:Hewlett Packard, Cupertino.;SISL adr:;;19420, Homestead Road, M\S 43LN, ;Cupertino.;CA.;95014.;USA. version:2.1 email;internet:santoshr@cup.hp.com title:Software Design Engineer x-mozilla-cpt:;21088 fn:Santosh Rao end:vcard --------------F1B114CE8B903A7E527EE8E8-- From owner-ips@ece.cmu.edu Wed May 9 00:31:34 2001 Received: from ece.cmu.edu ([128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id AAA15557 for ; Wed, 9 May 2001 00:31:34 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f492j6920072 for ips-outgoing; Tue, 8 May 2001 22:45:06 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from gateway.sanlight.org (adsl-63-202-160-80.dsl.snfc21.pacbell.net [63.202.160.80]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f492j2H20065 for ; Tue, 8 May 2001 22:45:03 -0400 (EDT) Received: from ljoy (10.0.0.18.lan.sanlight.net [10.0.0.18]) by gateway.sanlight.org (8.11.0/8.11.0) with SMTP id f493qL156146 for ; Tue, 8 May 2001 20:52:22 -0700 (PDT) (envelope-from dotis@sanlight.net) From: "Douglas Otis" To: "Ips" Subject: iSCSI: SCTP compatible transport for iSCSI Date: Tue, 8 May 2001 19:42:44 -0700 Message-ID: MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 (Normal) X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook IMO, Build 9.0.2416 (9.0.2911.0) X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4522.1200 Importance: Normal Sender: owner-ips@ece.cmu.edu Precedence: bulk Content-Transfer-Encoding: 7bit All, ftp://ftp.sanlight.net/pub/draft-otis-record-delivery-01.txt This is an update to the original draft. This draft attempts to make an even more generic transport layer than that defined within the iSCSI specifications. When used with SCSI in a gateway mode, it is expected the larger fields normally found will be expanded within the Argument area. The smaller fields within this draft will not impact functionality however. In addition, cluster features were added to this transport as well as adoption of the new SCSI architectural model for the Service Delivery Port. I also went ahead and redefined the text negotiations per conversations at the interim meeting. Doug From owner-ips@ece.cmu.edu Wed May 9 00:36:07 2001 Received: from ece.cmu.edu ([128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id AAA15575 for ; Wed, 9 May 2001 00:36:07 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f4939hp21996 for ips-outgoing; Tue, 8 May 2001 23:09:43 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from e31.bld.us.ibm.com (e31.co.us.ibm.com [32.97.110.129]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f4939fH21992 for ; Tue, 8 May 2001 23:09:41 -0400 (EDT) Received: from westrelay02.boulder.ibm.com (westrelay02.boulder.ibm.com [9.99.140.23]) by e31.bld.us.ibm.com (8.9.3/8.9.3) with ESMTP id XAA168674; Tue, 8 May 2001 23:01:47 -0400 Received: from f4n49e (d03nm065h.boulder.ibm.com [9.99.140.49]) by westrelay02.boulder.ibm.com (8.11.1/NCO v4.96.1.0) with ESMTP id f4939Gc68204; Tue, 8 May 2001 21:09:16 -0600 X-Priority: 1 (High) Importance: Normal Subject: Re: iSCSI: multiple sessions b/n a pair of WWUIs. To: Santosh Rao Cc: ips@ece.cmu.edu X-Mailer: Lotus Notes Release 5.0.3 (Intl) 21 March 2000 Message-ID: From: "John Hufferd" Date: Tue, 8 May 2001 20:08:59 -0700 X-MIMETrack: Serialize by Router on D03NM065/03/M/IBM(Release 5.0.6 |December 14, 2000) at 05/08/2001 09:09:16 PM MIME-Version: 1.0 Content-type: text/plain; charset=us-ascii Sender: owner-ips@ece.cmu.edu Precedence: bulk Santosh, In the attached you are getting you wishes in the way of what is Required. The Hard requirement is that the iSCSI Initiator Port have a ISID that is unique to a specific target Node (actually Target iSCSI Port). It is nice that you want to used it (the ISID in the way you suggest, however, there may be several iSCSI Initiator Ports on any iSCSI Initiator Node and that means that you must find a way to allocate out the ISID value range to each iSCSI Initiator Port. Quite frankly I believe you should probably should use another indexing method, and let a single ISID be used by each iSCSI Initiator Port. However, I am getting the feeling that we are talking past each other, and some how not communicating. Not sure if it is my fault or not. . . . John L. Hufferd Senior Technical Staff Member (STSM) IBM/SSG San Jose Ca (408) 256-0403, Tie: 276-0403, eFax: (408) 904-4688 Internet address: hufferd@us.ibm.com Santosh Rao @ece.cmu.edu on 05/08/2001 11:03:27 AM Sent by: owner-ips@ece.cmu.edu To: ips@ece.cmu.edu cc: Subject: Re: iSCSI: multiple sessions b/n a pair of WWUIs. Douglas Otis wrote: > The usefulness of ISID is also > diminished if the client is not expected to maintain this value as unique > across all Targets. I agree with the above and have stated to similar effect in my response to Matt. A practical usage of ISID within an initiator node name space is to assign it as unique across all targets, allowing the same ISID to be used as a lookup key of the session for any given target port/node. Not applying uniqueness in ISID assignment across all targets would then require initiators to implement a second key mechanism for target lookup. - Santosh From owner-ips@ece.cmu.edu Wed May 9 00:36:46 2001 Received: from ece.cmu.edu ([128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id AAA15591 for ; Wed, 9 May 2001 00:36:45 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f492nq020435 for ips-outgoing; Tue, 8 May 2001 22:49:53 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from gateway.sanlight.org (adsl-63-202-160-80.dsl.snfc21.pacbell.net [63.202.160.80]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f492niH20406 for ; Tue, 8 May 2001 22:49:44 -0400 (EDT) Received: from ljoy (10.0.0.18.lan.sanlight.net [10.0.0.18]) by gateway.sanlight.org (8.11.0/8.11.0) with SMTP id f493qN156149; Tue, 8 May 2001 20:52:47 -0700 (PDT) (envelope-from dotis@sanlight.net) From: "Douglas Otis" To: "Michael Krause" Cc: , , , , Subject: RE: iSCSI Security rough consensus Date: Tue, 8 May 2001 19:42:46 -0700 Message-ID: MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 (Normal) X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook IMO, Build 9.0.2416 (9.0.2911.0) X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4522.1200 In-Reply-To: <5.0.2.1.2.20010508094736.02819d10@esalpha2.cup.hp.com> Importance: Normal Sender: owner-ips@ece.cmu.edu Precedence: bulk Content-Transfer-Encoding: 7bit Mike, My only point was that off-loading an end to end error check with hardware raises the level of integrity which must be assured at the hardware/software interface. In essence, I was agreeing with Santosh while underlining the initial concern. As this concern is "Out of Scope", I too agree that this will not be covered within this workgroup but an area left unresolved as it relates to iSCSI. Doug > The end-to-end integrity checks are from a source fault zone to a > destination fault zone. What happens within a fault zone is > implementation > specific and beyond the scope of the iSCSI specification. The litany of > potential problems within a fault zone are interesting and are being > addressed by various techniques all of which are again outside > the scope of > the iSCSI specification and clearly where many vendors will differentiate > their product offerings. > > It would be beneficial to this workgroup to avoid mixing the > fault zone and > iSCSI requirements to avoid creating confusion or what may be > construed as > FUD. The workgroup appears to be in consensus on where iSCSI integrity > checking applies and the problems it addresses. Unless there is > something > new to add from an iSCSI technical perspective in this area, may > we move on > to getting this specification complete so we can implement it and > see if it > really works. > > Mike > > > > At 07:03 PM 5/7/2001 -0700, Douglas Otis wrote: > >Somesh, > > > >You have it correct. The general intent of an end to end > integrity check is > >to do something light weight within software to catch a myriad of errors > >that crop up due to various hardware failures (R*****k and M*******h as > >example), as well as bugs within variations of the theme called TCP. One > >can only hope such checking automation does not introduce > corrupt data as a > >result. New hardware/software MUST maintain integrity. There > is a history > >that gives one pause however. > > > >Doug > > > > > > > I was trying not to react to the string of messages, but > > > this message finally did it(and nothing wrong with the message > > > itself and no offense meant to the author). What the message > > > implies is that no intermediate hops can be trusted and that > > > integrity must be end-to-end. > > > > > > Taken to the literal extreme, this means that the system must > > > at the application level (in the host) generate some integrity > > > checks which should then be verified every > > > time the data is used (perhaps by an end application again). > > > > > > Even with the CRC, the following errors will go undetected, > > > unless protected by some other means (well engineered, well > > > tested devices) > > > > > > 1. Host software. > > > 2. Host memory to adapter path > > > 3. Adapter memory > > > 4. Adapter software > > > 5. Any virtualization device in the middle including > > > software, memory, fabric etc on the device including > > > any fibre-channel/scsi storage at the back end > > > (assuming they will recreate resegment PDUs) > > > 6. Any target adapter & adapter memory > > > 7. Any paths within the target > > > 8. The storage itself > > > 9. and then the reverse path > > > > > > If end-to-end integrity is a must, it seems that the TCP checksum > > > offload and iSCSI CRC in off-board hardware must be avoided. > > > > > > Or did I miss something somewhere. > > > > > > Somesh > > > > > > > -----Original Message----- > > > > From: vince_cavanna@agilent.com [mailto:vince_cavanna@agilent.com] > > > > Sent: Monday, May 07, 2001 2:13 PM > > > > To: mbakke@cisco.com; Black_David@emc.com > > > > Cc: someshg@yahoo.com; ips@ece.cmu.edu; vince_cavanna@agilent.com > > > > Subject: RE: iSCSI Security rough consensus > > > > > > > > > > > > Here is a paper by well-known authors that supports Mark's > > > reasoning that > > > > the end-to-end iSCSI check is indispensable even when IPsec > is on .... > > > > > > > > Jonathan Stone and Craig artridge in "When the CRC and TCP checksum > > > > disagree" study the sources of errors that are not caught > by link level > > > > checks and that therefore stress a higher layer error check > > > > mechansim. Such > > > > errors are quite prevalent and one signifcant source of such > > > errors is the > > > > end-host itself. The authors emphasize that "hardware must not > > > be trusted" > > > > and that "many encryption solutions such as IPsec do not provide > > > > additional > > > > (over existing checks) protection because the encryption is > > > > applied too late > > > > in the transmission process, often after the data has passed > > > through a DMA > > > > engine. Rather the application must add the checksum before > > > > handing its data > > > > to TCP (ala SSL)". The authors recommend that "for truly > > > valuable data (in > > > > my opinion all data handled by iSCSI) the application should add > > > > a stronger > > > > application-level checksum". > > > > > > > > Vince > > > > > > > > |-----Original Message----- > > > > |From: Mark Bakke [mailto:mbakke@cisco.com] > > > > |Sent: Friday, May 04, 2001 1:04 PM > > > > |To: Black_David@emc.com > > > > |Cc: someshg@yahoo.com; ips@ece.cmu.edu > > > > |Subject: Re: iSCSI Security rough consensus > > > > | > > > > | > > > > |Black_David@emc.com wrote: > > > > |> > > > > |> > Sure would be nice if we could make up our minds and just > > > > |> > implement one mechanism. > > > > |> > > > > > |> > Here we have two mechanisms (iSCSI header/data integrity > > > > |> > and ESP) which are both mandatory to implement and > > > > |> > optional to use. Since ESP seems like a superset why not > > > > |> > just have that and get rid of the "integrity only" iSCSI > > > > |> > CRC mechanism. > > > > |> > > > > |> It sure would be nice, and in fact we had almost > > > > |> exactly this discussion later in the evening as > > > > |> part of the error recovery section of the agenda. > > > > |> The fly in the ointment is that the HMAC integrity > > > > |> algorithm that is at the core of ESP's integrity > > > > |> support is considerably more expensive (software > > > > |> or hardware) than a CRC, and this isn't likely > > > > |> to improve as I understand things. I would expect > > > > |> to see implementations with ESP completely in > > > > |> software and visible performance impacts. > > > > | > > > > |That's just part of the reason behind having both. The other is > > > > |that most implementations won't run IPsec end-to-end; either IPsec > > > > |is provided in an external device, or even in an iSCSI gateway. > > > > |In the latter case, all layers are removed and replaced, including > > > > |iSCSI. Only the SCSI-level information (data, CDBs) go all the > > > > |way end-to-end. Since iSCSI can CRC the SCSI-level data, it can > > > > |provide the data integrity that will keep our customers happy. > > > > | > > > > |The use of the iSCSI CRC is the minimum requirement; adding the > > > > |IPsec-level integrity check strengthens it, and can simplify error > > > > |recovery over a not-so-good or untrusted network. > > > > | > > > > |-- > > > > |Mark > > > > | > > > > |> > > > > |> I really need to get the meeting minutes written up :-). > > > > |> > > > > |> Thanks, > > > > |> --David > > > > |> > > > > |> --------------------------------------------------- > > > > |> David L. Black, Senior Technologist > > > > |> EMC Corporation, 42 South St., Hopkinton, MA 01748 > > > > |> +1 (508) 435-1000 x75140 FAX: +1 (508) 497-8500 > > > > |> black_david@emc.com Mobile: +1 (978) 394-7754 > > > > |> --------------------------------------------------- > > > > | > > > > |-- > > > > |Mark A. Bakke > > > > |Cisco Systems > > > > |mbakke@cisco.com > > > > |763.398.1054 > > > > | > > > > > > > > > _________________________________________________________ > > > Do You Yahoo!? > > > Get your free @yahoo.com address at http://mail.yahoo.com > > > > > > > > From owner-ips@ece.cmu.edu Wed May 9 01:23:27 2001 Received: from ece.cmu.edu ([128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id BAA18210 for ; Wed, 9 May 2001 01:23:27 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f493dax24141 for ips-outgoing; Tue, 8 May 2001 23:39:36 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from e31.bld.us.ibm.com (e31.co.us.ibm.com [32.97.110.129]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f493dZH24137 for ; Tue, 8 May 2001 23:39:35 -0400 (EDT) Received: from westrelay02.boulder.ibm.com (westrelay02.boulder.ibm.com [9.99.140.23]) by e31.bld.us.ibm.com (8.9.3/8.9.3) with ESMTP id XAA40788; Tue, 8 May 2001 23:32:04 -0400 Received: from f4n49e (d03nm065h.boulder.ibm.com [9.99.140.49]) by westrelay02.boulder.ibm.com (8.11.1/NCO v4.96.1.0) with ESMTP id f493dCc260714; Tue, 8 May 2001 21:39:33 -0600 X-Priority: 1 (High) Importance: Normal Subject: Re: iSCSI: multiple sessions b/n a pair of WWUIs. To: Santosh Rao Cc: Black_David@emc.com, ips@ece.cmu.edu X-Mailer: Lotus Notes Release 5.0.3 (Intl) 21 March 2000 Message-ID: From: "John Hufferd" Date: Tue, 8 May 2001 20:38:53 -0700 X-MIMETrack: Serialize by Router on D03NM065/03/M/IBM(Release 5.0.6 |December 14, 2000) at 05/08/2001 09:39:32 PM MIME-Version: 1.0 Content-type: text/plain; charset=us-ascii Sender: owner-ips@ece.cmu.edu Precedence: bulk Santosh, I think I am beginning to see the problem. A given iSCSI Initiator Port can NOT have a second session with the same iSCSI Initiator Port and be consistent with SCSIness. The second session could be started with another ISID, (to the same iSCSI Target Port) but if the session was established it would not have any of its commands and data handled via any techniques defined by SCSI or iSCSI. In fact its use would require a wedge driver. This is the exact confusion I was trying to avoid. Technically if you were actually able to start another Session to the same Target Port, it would be, by definition another iSCSI Initiator Port. The two different Ports would need to be coordinated by what we have been calling a Wedge Driver. Today, the way we use multiple Fibre Channel Initiator Ports connected to the same Fibre Channel Target Ports is by use of Wedge Drivers. I think what you are suggesting causes the need for a Wedge Drivers to integrate the iSCSI Initiator Ports. Not sure that we want to cause this type of thinking, by accident. One of the reasons for Multiple Connections per Session was to remove the "Hard" need for Wedge Drivers. OK, that's what I think our misunderstanding is all about. If I am mistaken, please set me straight. . . . John L. Hufferd Senior Technical Staff Member (STSM) IBM/SSG San Jose Ca (408) 256-0403, Tie: 276-0403, eFax: (408) 904-4688 Internet address: hufferd@us.ibm.com Santosh Rao @cup.hp.com on 05/08/2001 05:44:41 PM Sent by: santoshr@cup.hp.com To: John Hufferd/San Jose/IBM@IBMUS cc: Black_David@emc.com, ips@ece.cmu.edu Subject: Re: iSCSI: multiple sessions b/n a pair of WWUIs. John, By the name-disc definition, an iSCSI initiator or target port is a logical entity that constitutes the end points of a session. Hence, there never can be 2 sessions b/n the *same* 2 initiator & target port. Each session established spawns a *new* initiator and target iSCSI port. Going by the above semantics, there is always only 1 ISID/TSID per iSCSI Initiator/Target port. Hence, I'm unable to understand your statements below. As for the uniqueness of ISID across all initiators, is there any reason not to allow implementations to do that ? I would have thought that is the most typical use of an ISID and also allows initiators to lookup their target structures based on ISID. Regards, Santosh John Hufferd wrote: > > Also, since a second session by the same iSCSI Initiator Port, to the same > iSCSI Target Port is not, in my opinion "legal", it is not clear to me that > any given iSCSI Port needs any more then one ISID. I > > Therefore, I suggest that we Not put any such notes, like you suggest, in > the draft, but in fact encourage a single ISID/TSID per iSCSI > Initiator/Target Port. > To: Jim Hafner/Almaden/IBM@IBMUS, marjorie_krueger@hp.com > cc: ips@ece.cmu.edu > Subject: RE: iSCSI: multiple sessions b/n a pair of WWUIs. > > While Jim's correct ... in Marj's defense, what > she wrote is a fairly obvious way to implement > it, and that might be worth noting in the draft. > > --David From owner-ips@ece.cmu.edu Wed May 9 01:23:40 2001 Received: from ece.cmu.edu ([128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id BAA18247 for ; Wed, 9 May 2001 01:23:39 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f493V0023511 for ips-outgoing; Tue, 8 May 2001 23:31:00 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from med.corp.rhapsodynetworks.com (64-160-62-201.rhapsodynetworks.com [64.160.62.201] (may be forged)) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f493UxH23507 for ; Tue, 8 May 2001 23:30:59 -0400 (EDT) Received: by med.corp.rhapsodynetworks.com with Internet Mail Service (5.5.2653.19) id ; Tue, 8 May 2001 20:30:44 -0700 Message-ID: <15851BD69CFCD41186B100B0D0AABE650C1C05@med.corp.rhapsodynetworks.com> From: Venkat Rangan To: "'julian_satran@il.ibm.com'" , ips@ece.cmu.edu Subject: RE: iSCSI: Nailing down CRC-32C Date: Tue, 8 May 2001 20:30:43 -0700 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2653.19) Content-Type: text/plain; charset="iso-8859-1" Sender: owner-ips@ece.cmu.edu Precedence: bulk Julian, Is this example correct? How do we get the final "+1" is the last bit is '0'? > The generator polynomials for those digests are given in hex-notation, > for example 3a stands for 0011 1010 - the polynomial x**5+X**4+x**3+x+1. Venkat Rangan Rhapsody Networks Inc. http://www.rhapsodynetworks.com -----Original Message----- From: julian_satran@il.ibm.com [mailto:julian_satran@il.ibm.com] Sent: Saturday, May 05, 2001 9:21 AM To: ips@ece.cmu.edu Subject: Re: iSCSI: Nailing down CRC-32C The CRC part of the appendix (for 07) reads: The following table lists cyclic integrity checksums that can be negotiated for the digests and MUST be implemented by every iSCSI initiator and target. Note that these digest options have only error detection significance. +---------------------------------------------+ | Name | Description | +---------------------------------------------+ | crc-32C | 32 bit CRC | 11EDC6F41 | +---------------------------------------------+ | none | no digest | +---------------------------------------------+ The generator polynomials for those digests are given in hex-notation, for example 3a stands for 0011 1010 - the polynomial x**5+X**4+x**3+x+1. The generator polynomial selected is evaluated in [Castagnioli93]. When using the CRC the CRC register must be initialized to all 1s (0xFFFFFFFF) and the CRC bits must be complemented before transmission. Padding bytes, when presents in a segment covered by a CRC, have to be set to 0 and are included in the CRC. Regards, Julo Mark Bakke on 05-05-2001 00:13:09 Please respond to Mark Bakke To: IPS cc: Subject: iSCSI: Nailing down CRC-32C At the interim meeting, it was stated that iSCSI has selected the CRC-32C polynomial as its required iSCSI-level header and data CRC. Now that we have it, it's time to move on and make sure we can implement it. In the interest of interoperability, we need to not only specify the polynomial, but also the initial values, bit and byte ordering, etc. "A Painless Guide to CRC Error Detection Algorithms" (http://www.ross.net/crc/crcpaper.html) specifies a method to unambiguously characterize these parameters (sections 15 and 16). Has anyone taken a shot at defining these yet? Otherwise, here is what it might look like: Name : "CRC-32C" Width : 32 Poly : 1EDC6F41 (note that the leading "1" is implied) Init : FFFFFFFF RefIn : True RefOut : True XorOut : FFFFFFFF Check : ? I haven't attempted to create check data based on these yet. As soon as the other parameters are nailed down, we need to do this. Anyway, I am not a CRC expert, and can't make any statement about the above values being the "best" way to do this, but instead just copied them (except the polynomial itself) from the Ethernet CRC, since that is likely the easiest for everyone implementing hardware to deal with. If someone else is already doing this, let me know; I just wanted to start this thread so we can get closure. Regards, Mark -- Mark A. Bakke Cisco Systems mbakke@cisco.com 763.398.1054 From owner-ips@ece.cmu.edu Wed May 9 02:46:17 2001 Received: from ece.cmu.edu ([128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id CAA00993 for ; Wed, 9 May 2001 02:46:17 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f494pev29414 for ips-outgoing; Wed, 9 May 2001 00:51:40 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from d12lmsgate.de.ibm.com (d12lmsgate.de.ibm.com [195.212.91.199]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f494pcH29407 for ; Wed, 9 May 2001 00:51:39 -0400 (EDT) Received: from d12relay01.de.ibm.com (d12relay01.de.ibm.com [9.165.215.22]) by d12lmsgate.de.ibm.com (1.0.0) with ESMTP id GAA118542 for ; Wed, 9 May 2001 06:51:31 +0200 From: julian_satran@il.ibm.com Received: from d12mta02.de.ibm.com (d12mta01_cs0 [9.165.222.237]) by d12relay01.de.ibm.com (8.8.8m3/NCO v4.96) with SMTP id GAA142486 for ; Wed, 9 May 2001 06:51:31 +0200 Received: by d12mta02.de.ibm.com(Lotus SMTP MTA v4.6.5 (863.2 5-20-1999)) id C1256A47.001AAD0D ; Wed, 9 May 2001 06:51:22 +0200 X-Lotus-FromDomain: IBMIL@IBMDE To: ips@ece.cmu.edu Message-ID: Date: Tue, 8 May 2001 17:38:08 +0300 Subject: Re: iSCSI: Nailing down CRC-32C Mime-Version: 1.0 Content-type: text/plain; charset=us-ascii Content-Disposition: inline Sender: owner-ips@ece.cmu.edu Precedence: bulk Mark, My basic assumption was that everything is (as in all IP related standards) in network order. Regards, Julo Mark Bakke on 07-05-2001 23:00:28 Please respond to Mark Bakke To: Julian Satran/Haifa/IBM@IBMIL cc: ips@ece.cmu.edu Subject: Re: iSCSI: Nailing down CRC-32C Julian- That's great; it covers nearly everything. The only thing left is the byte order; are they taken in the same order Ethernet uses? If so, I can generate a few test patterns that our implementations can all check against. Thanks, -- Mark julian_satran@il.ibm.com wrote: > > The CRC part of the appendix (for 07) reads: > > The following table lists cyclic integrity checksums that can be > negotiated for the digests and MUST be implemented by every iSCSI > initiator and target. Note that these digest options have only error > detection significance. > > +---------------------------------------------+ > | Name | Description | > +---------------------------------------------+ > | crc-32C | 32 bit CRC | 11EDC6F41 | > +---------------------------------------------+ > | none | no digest | > +---------------------------------------------+ > > The generator polynomials for those digests are given in hex-notation, > for example 3a stands for 0011 1010 - the polynomial x**5+X**4+x**3+x+1. > > The generator polynomial selected is evaluated in [Castagnioli93]. > When using the CRC the CRC register must be initialized to all 1s > (0xFFFFFFFF) and the CRC bits must be complemented before transmission. > Padding bytes, when presents in a segment covered by a CRC, have to be > set to 0 and are included in the CRC. > > Regards, > Julo > > Mark Bakke on 05-05-2001 00:13:09 > > Please respond to Mark Bakke > > To: IPS > cc: > Subject: iSCSI: Nailing down CRC-32C > > At the interim meeting, it was stated that iSCSI has selected > the CRC-32C polynomial as its required iSCSI-level header and > data CRC. Now that we have it, it's time to move on and make > sure we can implement it. > > In the interest of interoperability, we need to not only specify > the polynomial, but also the initial values, bit and byte > ordering, etc. > > "A Painless Guide to CRC Error Detection Algorithms" > (http://www.ross.net/crc/crcpaper.html) specifies a > method to unambiguously characterize these parameters > (sections 15 and 16). Has anyone taken a shot at defining > these yet? Otherwise, here is what it might look like: > > Name : "CRC-32C" > Width : 32 > Poly : 1EDC6F41 (note that the leading "1" is implied) > Init : FFFFFFFF > RefIn : True > RefOut : True > XorOut : FFFFFFFF > Check : ? > > I haven't attempted to create check data based on these yet. As > soon as the other parameters are nailed down, we need to do this. > > Anyway, I am not a CRC expert, and can't make any statement about > the above values being the "best" way to do this, but instead just > copied them (except the polynomial itself) from the Ethernet CRC, > since that is likely the easiest for everyone implementing hardware > to deal with. > > If someone else is already doing this, let me know; I just wanted > to start this thread so we can get closure. > > Regards, > > Mark > > -- > Mark A. Bakke > Cisco Systems > mbakke@cisco.com > 763.398.1054 -- Mark A. Bakke Cisco Systems mbakke@cisco.com 763.398.1054 From owner-ips@ece.cmu.edu Wed May 9 02:46:46 2001 Received: from ece.cmu.edu ([128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id CAA01010 for ; Wed, 9 May 2001 02:46:46 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f494pds29410 for ips-outgoing; Wed, 9 May 2001 00:51:39 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from d12lmsgate-3.de.ibm.com (d12lmsgate-3.de.ibm.com [195.212.91.201]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f494paH29401 for ; Wed, 9 May 2001 00:51:37 -0400 (EDT) Received: from d12relay01.de.ibm.com (d12relay01.de.ibm.com [9.165.215.22]) by d12lmsgate-3.de.ibm.com (1.0.0) with ESMTP id GAA197386; Wed, 9 May 2001 06:51:29 +0200 From: julian_satran@il.ibm.com Received: from d12mta05.de.ibm.com (d12mta05_cs0 [9.165.222.239]) by d12relay01.de.ibm.com (8.8.8m3/NCO v4.96) with SMTP id GAA220332; Wed, 9 May 2001 06:51:28 +0200 Received: by d12mta05.de.ibm.com(Lotus SMTP MTA v4.6.5 (863.2 5-20-1999)) id C1256A47.001AAE27 ; Wed, 9 May 2001 06:51:25 +0200 X-Lotus-FromDomain: IBMIL@IBMDE To: Theodore Tso cc: Bernard Aboba , Black_David@emc.com, ips@ece.cmu.edu Message-ID: Date: Tue, 8 May 2001 18:00:59 +0300 Subject: Re: iSCSI Security rough consensus Mime-Version: 1.0 Content-type: text/plain; charset=us-ascii Content-Disposition: inline Sender: owner-ips@ece.cmu.edu Precedence: bulk Theodore, That is an excellent summary of the situation. Only to keep the "history record" straight keying in ESP with a key obtained by a Public Key exchange (we did not have SRP then) was mentioned by both Josh Tseng and myself as a way to get to end-to-end security while mandating only gateways as a "required to implement". Julo Theodore Tso on 05-05-2001 04:25:12 Please respond to Theodore Tso To: Bernard Aboba cc: Black_David@emc.com, ips@ece.cmu.edu Subject: Re: iSCSI Security rough consensus On Fri, May 04, 2001 at 09:06:23AM -0700, Bernard Aboba wrote: > > Also, given that another key exchange method is likely to be used (IKE, > KINK), this implies that another authentication has already taken place to > set up the IPSEC SA. Thus, requiring SRP in addition to IKE/KINK adds *yet > another* authentication and key exchange to the process of setting up an > IPSEC SA. Are we doing this purely in order to circumvent IKE's > limitations with respect to password-based authentication? The basic problem here seems to be that people don't want to implement IKE. Going into the interim meeting, what a lot of vendors had been planning on doing was apparently to *not* to include IPSEC in the basic storage devices, but to depend on outboard IPSEC gateways to actually do the IPSEC work. The iSCSI specs was going to have weasel words which would permit this, but say that only the entire system of storage devices plus off-the-shelf VPN box would be compliant with iSCSI. Of course, the number of sites that would actually deploy full iSCSI would probably be quite small; most customers would probably just not purchase the VPN box, and depend on the hard-on-the-outside, soft-and-chewing-on-the inside security model. And of course, the vendors would employ their marketing people to make sure that the full "iSCSI complaint" version would be included in the product catalog, but to also always make available the option which didn't include this 3rd party IPSEC gateway box. Guess which version of the product would actually see deployment in customer sites? For this reason, the IPS working group wanted to use a two levels of protection; an "outer layer" that would be IPSEC, but which might not necessarily be end-to-end (and in fact, in actual real life, probably wouldn't be present at all), and then an inner layer which used something like SRP. I don't know if something like this would have gotten past the IESG -- my guess is that if it did, it would be with the Security Area AD's hollering and screaming a lot. So it was given this particular set of constraints, where people seemed determined to (a) not use (or at least not require) IPSEC in an end-to-end mode, and (b) to use something like SRP, that I suggested using the session keying material from SRP to key ESP as an improvement. If this meant that the IPSEC protection could actually be end-to-end, and that it would more likely actually end up in shipping hardware that customers actually used, as opposed to an unwieldy, theoretical hack that was never meant for customer use but only to get around the IETF standards process, then using SRP as the source of keying information for ESP was a vast improvement from where we started on Tuesday morning. Would it be better if we were actually doing end-to-end IKE, and asking the disk drive manufacturers to solve the public key management problems that this would entail? From a security perspective, sure. But it's not too clear how realistic such a stance would really be. - Ted P.S. Yes, I know that in fact there will need to be some specifications written to indicate how to get from the SRP or CHAP keying information to the low-level details of ESP, probably using an AES cipher suite for speed. The hope was to keep this part small and simple enough that that it wouldn't actually be an explicit key management layer ala IKE and KINK, although of course it would have to perform at least a little of such functions. From owner-ips@ece.cmu.edu Wed May 9 02:54:06 2001 Received: from ece.cmu.edu ([128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id CAA01030 for ; Wed, 9 May 2001 02:54:06 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f495Kmq01304 for ips-outgoing; Wed, 9 May 2001 01:20:48 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from d12lmsgate-3.de.ibm.com (d12lmsgate-3.de.ibm.com [195.212.91.201]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f495KiH01297 for ; Wed, 9 May 2001 01:20:45 -0400 (EDT) Received: from d12relay01.de.ibm.com (d12relay01.de.ibm.com [9.165.215.22]) by d12lmsgate-3.de.ibm.com (1.0.0) with ESMTP id HAA83928 for ; Wed, 9 May 2001 07:20:34 +0200 From: julian_satran@il.ibm.com Received: from d12mta02.de.ibm.com (d12mta01_cs0 [9.165.222.237]) by d12relay01.de.ibm.com (8.8.8m3/NCO v4.96) with SMTP id HAA164154 for ; Wed, 9 May 2001 07:20:34 +0200 Received: by d12mta02.de.ibm.com(Lotus SMTP MTA v4.6.5 (863.2 5-20-1999)) id C1256A47.001D583E ; Wed, 9 May 2001 07:20:31 +0200 X-Lotus-FromDomain: IBMIL@IBMDE To: ips@ece.cmu.edu Message-ID: Date: Wed, 9 May 2001 08:26:09 +0300 Subject: RE: iSCSI: Nailing down CRC-32C Mime-Version: 1.0 Content-type: text/plain; charset=us-ascii Content-Disposition: inline Sender: owner-ips@ece.cmu.edu Precedence: bulk It is a typo - the text reads now 3b 00111011 - Julo Venkat Rangan on 09-05-2001 06:30:43 Please respond to Venkat Rangan To: Julian Satran/Haifa/IBM@IBMIL, ips@ece.cmu.edu cc: Subject: RE: iSCSI: Nailing down CRC-32C Julian, Is this example correct? How do we get the final "+1" is the last bit is '0'? > The generator polynomials for those digests are given in hex-notation, > for example 3a stands for 0011 1010 - the polynomial x**5+X**4+x**3+x+1. Venkat Rangan Rhapsody Networks Inc. http://www.rhapsodynetworks.com -----Original Message----- From: julian_satran@il.ibm.com [mailto:julian_satran@il.ibm.com] Sent: Saturday, May 05, 2001 9:21 AM To: ips@ece.cmu.edu Subject: Re: iSCSI: Nailing down CRC-32C The CRC part of the appendix (for 07) reads: The following table lists cyclic integrity checksums that can be negotiated for the digests and MUST be implemented by every iSCSI initiator and target. Note that these digest options have only error detection significance. +---------------------------------------------+ | Name | Description | +---------------------------------------------+ | crc-32C | 32 bit CRC | 11EDC6F41 | +---------------------------------------------+ | none | no digest | +---------------------------------------------+ The generator polynomials for those digests are given in hex-notation, for example 3a stands for 0011 1010 - the polynomial x**5+X**4+x**3+x+1. The generator polynomial selected is evaluated in [Castagnioli93]. When using the CRC the CRC register must be initialized to all 1s (0xFFFFFFFF) and the CRC bits must be complemented before transmission. Padding bytes, when presents in a segment covered by a CRC, have to be set to 0 and are included in the CRC. Regards, Julo Mark Bakke on 05-05-2001 00:13:09 Please respond to Mark Bakke To: IPS cc: Subject: iSCSI: Nailing down CRC-32C At the interim meeting, it was stated that iSCSI has selected the CRC-32C polynomial as its required iSCSI-level header and data CRC. Now that we have it, it's time to move on and make sure we can implement it. In the interest of interoperability, we need to not only specify the polynomial, but also the initial values, bit and byte ordering, etc. "A Painless Guide to CRC Error Detection Algorithms" (http://www.ross.net/crc/crcpaper.html) specifies a method to unambiguously characterize these parameters (sections 15 and 16). Has anyone taken a shot at defining these yet? Otherwise, here is what it might look like: Name : "CRC-32C" Width : 32 Poly : 1EDC6F41 (note that the leading "1" is implied) Init : FFFFFFFF RefIn : True RefOut : True XorOut : FFFFFFFF Check : ? I haven't attempted to create check data based on these yet. As soon as the other parameters are nailed down, we need to do this. Anyway, I am not a CRC expert, and can't make any statement about the above values being the "best" way to do this, but instead just copied them (except the polynomial itself) from the Ethernet CRC, since that is likely the easiest for everyone implementing hardware to deal with. If someone else is already doing this, let me know; I just wanted to start this thread so we can get closure. Regards, Mark -- Mark A. Bakke Cisco Systems mbakke@cisco.com 763.398.1054 From owner-ips@ece.cmu.edu Wed May 9 04:50:06 2001 Received: from ece.cmu.edu ([128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id EAA02072 for ; Wed, 9 May 2001 04:50:05 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f497Axr08268 for ips-outgoing; Wed, 9 May 2001 03:10:59 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from mxbh4.isus.emc.com (mxbh4.isus.emc.com [128.221.10.33]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f497AwH08264 for ; Wed, 9 May 2001 03:10:58 -0400 (EDT) Received: by mxbh4.isus.emc.com with Internet Mail Service (5.5.2650.21) id ; Wed, 9 May 2001 03:10:52 -0400 Message-ID: <0F31E5C394DAD311B60C00E029101A0708015546@corpmx9.isus.emc.com> From: Black_David@emc.com To: ajitkhaparde@hotmail.com, ips@ece.cmu.edu Subject: RE: Recovery mechanism Date: Wed, 9 May 2001 03:10:51 -0400 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2650.21) Content-Type: text/plain Sender: owner-ips@ece.cmu.edu Precedence: bulk > I'm currently going through the document "draft-ietf-ips-iscsi-05.txt". > I hope this is the latest document. Is it so? No, -06 is the latest. Check this page to find out and retrieve the latest version of any IPS WG draft: http://www.ietf.org/html.charters/ips-charter.html > TO recover a device(a target or an initiator), within a Task, what is the > number of retries to be given to prevent an initiator from requesting a dead > target. I suspect these are implementation-dependent in the draft you're looking at. A dead target has a nasty habit of not responding to anything, and hence the Initiator has to make a decision to give up at some point. > Also, how can an initiator know about the presence of a dead target. See above. Both SLP and iSNS may be able to help convey information that a target is sufficiently non-responsive to no longer be part of the discovery infrastructure. > Also is it possible for a device(initiator) to remember a dead target? It's definitely possible - doing so persistently to avoid trying to discover that target (e.g., on reboot) may be dangerous (e.g., the target may be back by then). --David --------------------------------------------------- David L. Black, Senior Technologist EMC Corporation, 42 South St., Hopkinton, MA 01748 +1 (508) 435-1000 x75140 FAX: +1 (508) 497-8500 black_david@emc.com Mobile: +1 (978) 394-7754 --------------------------------------------------- From owner-ips@ece.cmu.edu Wed May 9 04:50:29 2001 Received: from ece.cmu.edu ([128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id EAA02094 for ; Wed, 9 May 2001 04:50:28 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f4979E308134 for ips-outgoing; Wed, 9 May 2001 03:09:14 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from d12lmsgate-2.de.ibm.com (d12lmsgate-2.de.ibm.com [195.212.91.200]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f4979AH08128 for ; Wed, 9 May 2001 03:09:11 -0400 (EDT) Received: from d12relay02.de.ibm.com (d12relay02.de.ibm.com [9.165.215.23]) by d12lmsgate-2.de.ibm.com (1.0.0) with ESMTP id JAA301042; Wed, 9 May 2001 09:09:02 +0200 From: julian_satran@il.ibm.com Received: from d12mta05.de.ibm.com (d12mta05_cs0 [9.165.222.239]) by d12relay02.de.ibm.com (8.8.8m3/NCO v4.96) with SMTP id JAA93222; Wed, 9 May 2001 09:09:02 +0200 Received: by d12mta05.de.ibm.com(Lotus SMTP MTA v4.6.5 (863.2 5-20-1999)) id C1256A47.0027430A ; Wed, 9 May 2001 09:08:50 +0200 X-Lotus-FromDomain: IBMIL@IBMDE To: sandeepj@research.bell-labs.com (Sandeep Joshi) cc: matt_wakeley@agilent.com, ips@ece.cmu.edu Message-ID: Date: Wed, 9 May 2001 10:12:27 +0300 Subject: Re: iSCSI: immediate data Mime-Version: 1.0 Content-type: text/plain; charset=us-ascii Content-Disposition: inline Sender: owner-ips@ece.cmu.edu Precedence: bulk That is correct. The final bit indicates the end of unsolicited data. If all you have is immediate then the final bit is 1in the command. If you have immediate and other unsolicited (if enabled!) then set F to 0. 2.3.1 has been fixed accordingly. Regards, Julo sandeepj@research.bell-labs.com (Sandeep Joshi) on 09-05-2001 01:56:25 Please respond to sandeepj@research.bell-labs.com (Sandeep Joshi) To: matt_wakeley@agilent.com cc: ips@ece.cmu.edu Subject: Re: iSCSI: immediate data Umm.....Julian replied to this one and said the final bit on ScsiCmd would indicate it. Apparently, a single command can now send the unsolicited data both with the command and in separate PDUs. I cant seem to find the email but I believe Sec 2.3.1 was changed accordingly. Julian..? -Sandeep > The intention is that if an initiator requests to send immediate data (and is > granted the request), then it will always send immediate data. > > It sounds like you are asking for wishy washy mode... sometimes send immediate > data, sometimes not. > > -Matt > > Sandeep Joshi wrote: > > > > Julian, > > > > I had a follow-up question on an old thread. > > http://ips.pdl.cs.cmu.edu/mail/msg03373.html > > > > The initiator is allowed to send firstburst in immediate > > or in separate PDUs...but can it do neither ? I dont > > see any statement to the effect that it MUST send a > > firstburst. > > > > Here's a possible problem : > > 1) FirstBurst is 4K > > 2) Expected data length of command is 16K > > 3) Initiator sends the Scsi command without immediate > > data. (final flag on Scsi command is not set) > > And it decides to send no further data PDUs. > > > > The target does not know if R2Ts can be sent since it does > > not know if any unsolicited PDUs are expected. > > > > Perhaps the semantics on the final flag on SCSI command > > need to be changed to indicate that no unsolicited data > > will be sent with this command. Currently, the flag implies > > (Sec 2.3.1) that all the required data has been sent with > > the command. > > > > And here's a typo to fix. Appendix E (Key=immediateData) > > The following should say "initialR2T is no". > > The table has got it right. > > > > > If ImmediateData is set to yes and InitialR2T is set > > > to yes then the initiator MAY send unsolicited immediate > > > data or one unsolicited burst of Data-OUT PDUs but > > > MUST NOT send both immediate and a unsolicited burst of > > > Data-OUT PDUs for any one command. > > > > -Sandeep From owner-ips@ece.cmu.edu Wed May 9 04:51:42 2001 Received: from ece.cmu.edu ([128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id EAA02107 for ; Wed, 9 May 2001 04:51:42 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f496WRI05897 for ips-outgoing; Wed, 9 May 2001 02:32:27 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from e31.bld.us.ibm.com (e31.co.us.ibm.com [32.97.110.129]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f496WPH05893 for ; Wed, 9 May 2001 02:32:25 -0400 (EDT) Received: from westrelay02.boulder.ibm.com (westrelay02.boulder.ibm.com [9.99.140.23]) by e31.bld.us.ibm.com (8.9.3/8.9.3) with ESMTP id CAA116608; Wed, 9 May 2001 02:24:52 -0400 Received: from f4n49e (d03nm065h.boulder.ibm.com [9.99.140.49]) by westrelay02.boulder.ibm.com (8.11.1/NCO v4.96.1.0) with ESMTP id f496Vpc64444; Wed, 9 May 2001 00:32:21 -0600 X-Priority: 1 (High) Importance: Normal Subject: Re: Initiator-detected format or digest errors To: Santosh Rao Cc: ips@ece.cmu.edu X-Mailer: Lotus Notes Release 5.0.3 (Intl) 21 March 2000 Message-ID: From: "John Hufferd" Date: Tue, 8 May 2001 23:31:26 -0700 X-MIMETrack: Serialize by Router on D03NM065/03/M/IBM(Release 5.0.6 |December 14, 2000) at 05/09/2001 12:32:20 AM MIME-Version: 1.0 Content-type: text/plain; charset=us-ascii Sender: owner-ips@ece.cmu.edu Precedence: bulk Santosh, It is not clear that your approach is good for tapes. . . . John L. Hufferd Senior Technical Staff Member (STSM) IBM/SSG San Jose Ca (408) 256-0403, Tie: 276-0403, eFax: (408) 904-4688 Internet address: hufferd@us.ibm.com Santosh Rao @ece.cmu.edu on 05/08/2001 05:28:02 PM Sent by: owner-ips@ece.cmu.edu To: Matt Wakeley , Julian Satran/Haifa/IBM@IBMIL cc: ips@ece.cmu.edu Subject: Re: Initiator-detected format or digest errors Why can't iSCSI apply a simple policy of discarding any PDU that has a digest error or a format error ? Can the draft allow for such implementations that wish to discard the PDU and do nothing more ? This is all that FC initiators and targets need to do and they can recover fine from CRC errors ! A dropped PDU will result in one of the following scenarios : --------------------------------------------------------------- format error or digest error impacts Scenario --------------------------------------------------------------- i) iSCSI Command Initiator timeout. (could be detected early on when initiator retries command on seeing a large diff b/n ExpCmdSN and current CmdSN. ii) READ Data PDU I/O underrun, detected thru a mis-match b/n count of DataSNs received and EndDataSN. iii) WRITE Data PDU If this is not the "F" data PDU for that R2T data phase, this results in a target timoeut followed by target aborting the I/O at the end of R2T data phase and sending an aborted response back. If this is a "F" data PDU, results in initiator timeout. (scenario 1). iv) R2T PDU Target timeout. Initiator timeout. v) Status PDU Initiator timeout. From an implemenation's perspective, generating all of these REJECT commands is just adding overhead and complexity, requiring the allocation of resources [possibly by the host] for a REJECT PDU and having to transmit a REJECT in response to each PDU that had either a format or digest error. Any diagnostic info can be logged as counters within the implementation's MIB. A format error or digest error causes the PDU to be discarded. As it is, we need to apply a discard policy for the "header digest error" & format error cases. Simplification of all digest error and format error handling to a "discard policy" keeps implementations [and the protocol] simple. - Santosh Matt Wakeley wrote: > > The target can log the reject for further analysis. > > Why do you keep trying to eliminate simple things that can be used for > debugging? > > -Matt > > julian_satran@il.ibm.com wrote: > > > > Matt, > > > > 3 reasons: > > > > a. the initiator drives recovery. What can a target do with the reject? > > (he is probaly in need of a reset) > > > > b.the initiator is unaware of the target state after thhe error > > > > c.there is reject command (and response) -:) > > > > Julo > > > > Matt Wakeley on 06/04/2001 00:43:08 > > > > Please respond to Matt Wakeley > > > > To: IPS Reflector > > cc: > > Subject: Re: Initiator-detected format or digest errors > > > > Why not send a reject back to the originator of the bad frame, and just let > > the task time out? > > > > -Matt > > > > julian_satran@il.ibm.com wrote: > > > > > > Tom, > > > > > > An initiator will pass the appropriate response to the SCSI layer and > > will > > > abort the task if it can identify one. Further behavior of initiators > > and > > > targets is implementation dependent. > > > > > > 6.2 specifies this. > > > > > > Julo > > > > > > "Thomas McSweeney" on 04/04/2001 13:57:20 > > > > > > Please respond to "Thomas McSweeney" > > > > > > To: ips@ece.cmu.edu > > > cc: > > > Subject: Initiator-detected format or digest errors > > > > > > Section "2.20 Reject" talks about the target receiving a bad frame and > > > sending Reject. What should an Initiator do if it receives a PDU with a > > > format or digest error? Should it send Reject? If so, we'll need to > > > ensure that the Initiator fields of the MIB include an object to count > > > Reject commands transmitted. > > > > > > Tom McSweeney > > > iSCSI Development, Storage Systems Group, IBM > > > Email: rf42tpme@us.ibm.com > > > Phone: (USA) 919-254-5634 (tie line: 444-5634) > > > Fax: (USA) 919-254-0391 (tie line: 444-0391) From owner-ips@ece.cmu.edu Wed May 9 07:27:42 2001 Received: from ece.cmu.edu ([128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id HAA03791 for ; Wed, 9 May 2001 07:27:41 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f499V3S17341 for ips-outgoing; Wed, 9 May 2001 05:31:03 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from d12lmsgate.de.ibm.com (d12lmsgate.de.ibm.com [195.212.91.199]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f499UxH17311 for ; Wed, 9 May 2001 05:30:59 -0400 (EDT) Received: from d12relay02.de.ibm.com (d12relay02.de.ibm.com [9.165.215.23]) by d12lmsgate.de.ibm.com (1.0.0) with ESMTP id LAA125456 for ; Wed, 9 May 2001 11:30:47 +0200 From: julian_satran@il.ibm.com Received: from d12mta02.de.ibm.com (d12mta01_cs0 [9.165.222.237]) by d12relay02.de.ibm.com (8.8.8m3/NCO v4.96) with SMTP id LAA102214 for ; Wed, 9 May 2001 11:30:48 +0200 Received: by d12mta02.de.ibm.com(Lotus SMTP MTA v4.6.5 (863.2 5-20-1999)) id C1256A47.00343F2D ; Wed, 9 May 2001 11:30:40 +0200 X-Lotus-FromDomain: IBMIL@IBMDE To: ips@ece.cmu.edu Message-ID: Date: Wed, 9 May 2001 12:36:15 +0300 Subject: Re: Initiator-detected format or digest errors Mime-Version: 1.0 Content-type: text/plain; charset=us-ascii Content-Disposition: inline Sender: owner-ips@ece.cmu.edu Precedence: bulk Do we have to go through this cycle again? We started by saying that we discard anything that looks bad (that is also good against denial of service atatcks) the we moved to reject to simplify "debugging" (never a good longtime decision - the best would be to have a silent or verbose option with silent being mandatory to implement), now we have a request to have a verbose initiator mode and another one (vocal) to move back to silent mode. And yes Santosh you have all the mechanisms to work in silent mode and a verbose (reject) is required only for debugging (logging). Julo Santosh Rao on 09-05-2001 03:28:02 Please respond to Santosh Rao To: Matt Wakeley , Julian Satran/Haifa/IBM@IBMIL cc: ips@ece.cmu.edu Subject: Re: Initiator-detected format or digest errors Why can't iSCSI apply a simple policy of discarding any PDU that has a digest error or a format error ? Can the draft allow for such implementations that wish to discard the PDU and do nothing more ? This is all that FC initiators and targets need to do and they can recover fine from CRC errors ! A dropped PDU will result in one of the following scenarios : --------------------------------------------------------------- format error or digest error impacts Scenario --------------------------------------------------------------- i) iSCSI Command Initiator timeout. (could be detected early on when initiator retries command on seeing a large diff b/n ExpCmdSN and current CmdSN. ii) READ Data PDU I/O underrun, detected thru a mis-match b/n count of DataSNs received and EndDataSN. iii) WRITE Data PDU If this is not the "F" data PDU for that R2T data phase, this results in a target timoeut followed by target aborting the I/O at the end of R2T data phase and sending an aborted response back. If this is a "F" data PDU, results in initiator timeout. (scenario 1). iv) R2T PDU Target timeout. Initiator timeout. v) Status PDU Initiator timeout. From an implemenation's perspective, generating all of these REJECT commands is just adding overhead and complexity, requiring the allocation of resources [possibly by the host] for a REJECT PDU and having to transmit a REJECT in response to each PDU that had either a format or digest error. Any diagnostic info can be logged as counters within the implementation's MIB. A format error or digest error causes the PDU to be discarded. As it is, we need to apply a discard policy for the "header digest error" & format error cases. Simplification of all digest error and format error handling to a "discard policy" keeps implementations [and the protocol] simple. - Santosh Matt Wakeley wrote: > > The target can log the reject for further analysis. > > Why do you keep trying to eliminate simple things that can be used for > debugging? > > -Matt > > julian_satran@il.ibm.com wrote: > > > > Matt, > > > > 3 reasons: > > > > a. the initiator drives recovery. What can a target do with the reject? > > (he is probaly in need of a reset) > > > > b.the initiator is unaware of the target state after thhe error > > > > c.there is reject command (and response) -:) > > > > Julo > > > > Matt Wakeley on 06/04/2001 00:43:08 > > > > Please respond to Matt Wakeley > > > > To: IPS Reflector > > cc: > > Subject: Re: Initiator-detected format or digest errors > > > > Why not send a reject back to the originator of the bad frame, and just let > > the task time out? > > > > -Matt > > > > julian_satran@il.ibm.com wrote: > > > > > > Tom, > > > > > > An initiator will pass the appropriate response to the SCSI layer and > > will > > > abort the task if it can identify one. Further behavior of initiators > > and > > > targets is implementation dependent. > > > > > > 6.2 specifies this. > > > > > > Julo > > > > > > "Thomas McSweeney" on 04/04/2001 13:57:20 > > > > > > Please respond to "Thomas McSweeney" > > > > > > To: ips@ece.cmu.edu > > > cc: > > > Subject: Initiator-detected format or digest errors > > > > > > Section "2.20 Reject" talks about the target receiving a bad frame and > > > sending Reject. What should an Initiator do if it receives a PDU with a > > > format or digest error? Should it send Reject? If so, we'll need to > > > ensure that the Initiator fields of the MIB include an object to count > > > Reject commands transmitted. > > > > > > Tom McSweeney > > > iSCSI Development, Storage Systems Group, IBM > > > Email: rf42tpme@us.ibm.com > > > Phone: (USA) 919-254-5634 (tie line: 444-5634) > > > Fax: (USA) 919-254-0391 (tie line: 444-0391) - santoshr.vcf From owner-ips@ece.cmu.edu Wed May 9 12:44:41 2001 Received: from ece.cmu.edu ([128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id MAA16078 for ; Wed, 9 May 2001 12:44:41 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f49EjX520101 for ips-outgoing; Wed, 9 May 2001 10:45:33 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from d12lmsgate.de.ibm.com (d12lmsgate.de.ibm.com [195.212.91.199]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f49EjVH20096 for ; Wed, 9 May 2001 10:45:31 -0400 (EDT) Received: from d12relay01.de.ibm.com (d12relay01.de.ibm.com [9.165.215.22]) by d12lmsgate.de.ibm.com (1.0.0) with ESMTP id QAA191948 for ; Wed, 9 May 2001 16:45:21 +0200 From: julian_satran@il.ibm.com Received: from d12mta02.de.ibm.com (d12mta01_cs0 [9.165.222.237]) by d12relay01.de.ibm.com (8.8.8m3/NCO v4.96) with SMTP id QAA220386 for ; Wed, 9 May 2001 16:45:22 +0200 Received: by d12mta02.de.ibm.com(Lotus SMTP MTA v4.6.5 (863.2 5-20-1999)) id C1256A47.00510C4F ; Wed, 9 May 2001 16:45:15 +0200 X-Lotus-FromDomain: IBMIL@IBMDE To: ips@ece.cmu.edu Message-ID: Date: Wed, 9 May 2001 17:50:55 +0300 Subject: Re: iSCSI: firstburst and maxburst Mime-Version: 1.0 Content-type: text/plain; charset=us-ascii Content-Disposition: inline Sender: owner-ips@ece.cmu.edu Precedence: bulk Sandeep, The current draft considers 0 an illegal value. I wonder if making 0 a no-limit indication is really needed. Thanks, Julo sandeepj@research.bell-labs.com (Sandeep Joshi) on 07-05-2001 20:34:19 Please respond to sandeepj@research.bell-labs.com (Sandeep Joshi) To: ips@ece.cmu.edu cc: Subject: iSCSI: firstburst and maxburst Julian, SPC-2 revision 19: section 8.3.7 Disconnect-reconnect page states that a value of "0" indicates no limit on the maximum and first burst size. Current range for the corresponding text keys from Appendix E are (1 to 2^15-1)*512 bytes. How should iSCSI interpret a zero ? regards, -Sandeep From owner-ips@ece.cmu.edu Wed May 9 12:45:19 2001 Received: from ece.cmu.edu ([128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id MAA16127 for ; Wed, 9 May 2001 12:45:14 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f49EhjE19961 for ips-outgoing; Wed, 9 May 2001 10:43:45 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from e31.bld.us.ibm.com (e31.co.us.ibm.com [32.97.110.129]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f49EhhH19957 for ; Wed, 9 May 2001 10:43:43 -0400 (EDT) Received: from westrelay02.boulder.ibm.com (westrelay02.boulder.ibm.com [9.99.140.23]) by e31.bld.us.ibm.com (8.9.3/8.9.3) with ESMTP id KAA122642; Wed, 9 May 2001 10:36:11 -0400 Received: from f3n42e (d03nm042h.boulder.ibm.com [9.99.140.42]) by westrelay02.boulder.ibm.com (8.11.1/NCO v4.96.1.0) with ESMTP id f49Ehac187088; Wed, 9 May 2001 08:43:36 -0600 Importance: Normal Subject: Re: iSCSI: multiple sessions b/n a pair of WWUIs. To: Santosh Rao Cc: ips@ece.cmu.edu From: "Jim Hafner" Date: Wed, 9 May 2001 07:43:33 -0700 Message-ID: X-MIMETrack: Serialize by Router on D03NM042/03/M/IBM(Release 5.0.6 |December 14, 2000) at 05/09/2001 07:43:39 AM MIME-Version: 1.0 Content-type: text/plain; charset=us-ascii Sender: owner-ips@ece.cmu.edu Precedence: bulk Santosh, As I've said, I agree with the sentiment, but that is not the only implementation option. For example, why can't the database index on the pointer to the data structure that has the names, etc, or hash on the names or a million other things. We shouldn't spec implementations, only interoperability requirements. Jim Hafner Santosh Rao @ece.cmu.edu on 05-08-2001 11:03:27 AM Sent by: owner-ips@ece.cmu.edu To: ips@ece.cmu.edu cc: Subject: Re: iSCSI: multiple sessions b/n a pair of WWUIs. Douglas Otis wrote: > The usefulness of ISID is also > diminished if the client is not expected to maintain this value as unique > across all Targets. I agree with the above and have stated to similar effect in my response to Matt. A practical usage of ISID within an initiator node name space is to assign it as unique across all targets, allowing the same ISID to be used as a lookup key of the session for any given target port/node. Not applying uniqueness in ISID assignment across all targets would then require initiators to implement a second key mechanism for target lookup. - Santosh From owner-ips@ece.cmu.edu Wed May 9 12:47:29 2001 Received: from ece.cmu.edu ([128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id MAA16210 for ; Wed, 9 May 2001 12:47:24 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f49EjiD20122 for ips-outgoing; Wed, 9 May 2001 10:45:44 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from ietf.org (odin.ietf.org [132.151.1.176]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f49DV2H14278 for ; Wed, 9 May 2001 09:31:02 -0400 (EDT) Received: from CNRI.Reston.VA.US (localhost [127.0.0.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id JAA05867; Wed, 9 May 2001 09:30:57 -0400 (EDT) Message-Id: <200105091330.JAA05867@ietf.org> Mime-Version: 1.0 Content-Type: Multipart/Mixed; Boundary="NextPart" To: IETF-Announce: ; Cc: ips@ece.cmu.edu From: Internet-Drafts@ietf.org Reply-to: Internet-Drafts@ietf.org Subject: I-D ACTION:draft-ietf-ips-fcencapsulation-01.txt Date: Wed, 09 May 2001 09:30:56 -0400 Sender: owner-ips@ece.cmu.edu Precedence: bulk --NextPart A New Internet-Draft is available from the on-line Internet-Drafts directories. This draft is a work item of the IP Storage Working Group of the IETF. Title : FC Frame Encapsulation Author(s) : R. Weber, M. Rajagopal, F. Travostino, V. Chau, M. O'Donnell, C. Monia, M. Merhar Filename : draft-ietf-ips-fcencapsulation-01.txt Pages : 13 Date : 08-May-01 This is the ips (IP Storage) working group draft describing the common encapsulation format for use by any IETF protocol that encapsulates Fibre Channel (FC) frames. This draft describes a frame header containing information mandated for encapsulating, transmitting, and de-encapsulating FC frames. A URL for this Internet-Draft is: http://www.ietf.org/internet-drafts/draft-ietf-ips-fcencapsulation-01.txt Internet-Drafts are also available by anonymous FTP. Login with the username "anonymous" and a password of your e-mail address. After logging in, type "cd internet-drafts" and then "get draft-ietf-ips-fcencapsulation-01.txt". A list of Internet-Drafts directories can be found in http://www.ietf.org/shadow.html or ftp://ftp.ietf.org/ietf/1shadow-sites.txt Internet-Drafts can also be obtained by e-mail. Send a message to: mailserv@ietf.org. In the body type: "FILE /internet-drafts/draft-ietf-ips-fcencapsulation-01.txt". NOTE: The mail server at ietf.org can return the document in MIME-encoded form by using the "mpack" utility. To use this feature, insert the command "ENCODING mime" before the "FILE" command. To decode the response(s), you will need "munpack" or a MIME-compliant mail reader. Different MIME-compliant mail readers exhibit different behavior, especially when dealing with "multipart" MIME messages (i.e. documents which have been split up into multiple messages), so check your local documentation on how to manipulate these messages. Below is the data which will enable a MIME compliant mail reader implementation to automatically retrieve the ASCII version of the Internet-Draft. --NextPart Content-Type: Multipart/Alternative; Boundary="OtherAccess" --OtherAccess Content-Type: Message/External-body; access-type="mail-server"; server="mailserv@ietf.org" Content-Type: text/plain Content-ID: <20010508152048.I-D@ietf.org> ENCODING mime FILE /internet-drafts/draft-ietf-ips-fcencapsulation-01.txt --OtherAccess Content-Type: Message/External-body; name="draft-ietf-ips-fcencapsulation-01.txt"; site="ftp.ietf.org"; access-type="anon-ftp"; directory="internet-drafts" Content-Type: text/plain Content-ID: <20010508152048.I-D@ietf.org> --OtherAccess-- --NextPart-- From owner-ips@ece.cmu.edu Wed May 9 13:37:04 2001 Received: from ece.cmu.edu ([128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id NAA18662 for ; Wed, 9 May 2001 13:36:59 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f49Gngx01301 for ips-outgoing; Wed, 9 May 2001 12:49:42 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from palrel1.hp.com (palrel1.hp.com [156.153.255.242]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f49GneH01294 for ; Wed, 9 May 2001 12:49:40 -0400 (EDT) Received: from colosus2.cup.hp.com (colosus2.cup.hp.com [15.13.128.145]) by palrel1.hp.com (Postfix) with ESMTP id 05A3F1248 for ; Wed, 9 May 2001 09:49:40 -0700 (PDT) Received: from hp.com (IDENT:plabat@pl703521.cup.hp.com [15.13.133.216]) by colosus2.cup.hp.com (8.9.3 (PHNE_18979)/8.9.3 SMKit7.02) with ESMTP id JAA11328; Wed, 9 May 2001 09:49:38 -0700 (PDT) Message-ID: <3AF975E1.83909F34@hp.com> Date: Wed, 09 May 2001 09:52:50 -0700 From: Pierre Labat Organization: Hewlett Packard ATM-SISL X-Mailer: Mozilla 4.51 [en] (X11; I; Linux 2.2.5-15 i686) X-Accept-Language: en MIME-Version: 1.0 To: ips@ece.cmu.edu Subject: Re: iSCSI: multiple sessions b/n a pair of WWUIs. References: Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-ips@ece.cmu.edu Precedence: bulk Content-Transfer-Encoding: 7bit John Hufferd wrote: > Santosh, > I think I am beginning to see the problem. A given iSCSI Initiator Port > can NOT have a second session with the same iSCSI Initiator Port and be > consistent with SCSIness. The second session could be started with another > ISID, (to the same iSCSI Target Port) but if the session was established it > would not have any of its commands and data handled via any techniques > defined by SCSI or iSCSI. In fact its use would require a wedge driver. > This is the exact confusion I was trying to avoid. Technically if you were > actually able to start another Session to the same Target Port, it would > be, by definition another iSCSI Initiator Port. The two different Ports > would need to be coordinated by what we have been calling a Wedge Driver. > > Today, the way we use multiple Fibre Channel Initiator Ports connected to > the same Fibre Channel Target Ports is by use of Wedge Drivers. I think > what you are suggesting causes the need for a Wedge Drivers to integrate > the iSCSI Initiator Ports. Not sure that we want to cause this type of > thinking, by accident. One of the reasons for Multiple Connections per > Session was to remove the "Hard" need for Wedge Drivers. John, Could you give your exact definition of a iSCSI port. Till your mails they were the definition of - a "portal" (IP add), - a SCSI service delivery port (=session end point from what i hearded in Nashua) It seems that you want to tie the notion of SCSI service delivery port to hardware, the opposite of what was presented in Nashua. Do i miss something? I see nothing wrong against SCSI or iSCSI to have two sessions (with 2 # ISID) flowing from the same initiator adapter to the same target adapter. Each session end point represents a SDP. Regards, Pierre > > > OK, that's what I think our misunderstanding is all about. If I am > mistaken, please set me straight. > . > . > . > John L. Hufferd > Senior Technical Staff Member (STSM) > IBM/SSG San Jose Ca > (408) 256-0403, Tie: 276-0403, eFax: (408) 904-4688 > Internet address: hufferd@us.ibm.com > > Santosh Rao @cup.hp.com on 05/08/2001 05:44:41 PM > > Sent by: santoshr@cup.hp.com > > To: John Hufferd/San Jose/IBM@IBMUS > cc: Black_David@emc.com, ips@ece.cmu.edu > Subject: Re: iSCSI: multiple sessions b/n a pair of WWUIs. > > John, > > By the name-disc definition, an iSCSI initiator or target port is a > logical entity that constitutes the end points of a session. > > Hence, there never can be 2 sessions b/n the *same* 2 initiator & > target port. Each session established spawns a *new* initiator and > target iSCSI port. > > Going by the above semantics, there is always only 1 ISID/TSID per iSCSI > Initiator/Target port. > > Hence, I'm unable to understand your statements below. > > As for the uniqueness of ISID across all initiators, is there any reason > not to allow implementations to do that ? I would have thought that is > the most typical use of an ISID and also allows initiators to lookup > their target structures based on ISID. > > Regards, > Santosh > > John Hufferd wrote: > > > > Also, since a second session by the same iSCSI Initiator Port, to the > same > > iSCSI Target Port is not, in my opinion "legal", it is not clear to me > that > > any given iSCSI Port needs any more then one ISID. I > > > > Therefore, I suggest that we Not put any such notes, like you suggest, in > > the draft, but in fact encourage a single ISID/TSID per iSCSI > > Initiator/Target Port. > > > To: Jim Hafner/Almaden/IBM@IBMUS, marjorie_krueger@hp.com > > cc: ips@ece.cmu.edu > > Subject: RE: iSCSI: multiple sessions b/n a pair of WWUIs. > > > > While Jim's correct ... in Marj's defense, what > > she wrote is a fairly obvious way to implement > > it, and that might be worth noting in the draft. > > > > --David From owner-ips@ece.cmu.edu Wed May 9 13:37:54 2001 Received: from ece.cmu.edu ([128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id NAA18695 for ; Wed, 9 May 2001 13:37:48 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f49Ft9m26327 for ips-outgoing; Wed, 9 May 2001 11:55:09 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from palrel1.hp.com (palrel1.hp.com [156.153.255.242]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f49Ft7H26319 for ; Wed, 9 May 2001 11:55:07 -0400 (EDT) Received: from hpcuhe.cup.hp.com (hpcuhe.cup.hp.com [15.0.80.203]) by palrel1.hp.com (Postfix) with ESMTP id ABF8E741; Wed, 9 May 2001 08:55:06 -0700 (PDT) Received: from cup.hp.com (santoshr@hpindhhm.cup.hp.com [15.8.80.197]) by hpcuhe.cup.hp.com (8.9.3 (PHNE_18979)/8.9.3 SMKit7.02) with ESMTP id IAA13053; Wed, 9 May 2001 08:55:01 -0700 (PDT) Message-ID: <3AF9684E.99E969FF@cup.hp.com> Date: Wed, 09 May 2001 08:54:54 -0700 From: Santosh Rao Organization: Hewlett Packard, Cupertino. X-Mailer: Mozilla 4.7 [en] (X11; U; HP-UX B.11.00 9000/778) X-Accept-Language: en MIME-Version: 1.0 To: Jim Hafner Cc: ips@ece.cmu.edu Subject: Re: iSCSI: multiple sessions b/n a pair of WWUIs. References: Content-Type: multipart/mixed; boundary="------------356121B01F17742A8CF0EDA3" Sender: owner-ips@ece.cmu.edu Precedence: bulk This is a multi-part message in MIME format. --------------356121B01F17742A8CF0EDA3 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Jim, I agree that the only requirement is uniqueness of ISID among all the sessions to a given target node. Maintaining uniqueness across all targets only extends compliance to this requirement a step further and does not violate it. All we are debating is David's suggestion that a note could be added to indicate that implementations may choose to apply this uniqueness across all target nodes so as to use the ISID as a target lookup mechanism. If there is strong resistance against adding such a note, then, let us omit it. [as long as the spec does'nt preclude usage of ISID as unique across all targets]. Regards, Santosh Jim Hafner wrote: > > Santosh, > > As I've said, I agree with the sentiment, but that is not the only > implementation option. For example, why can't the database index on the > pointer to the data structure that has the names, etc, or hash on the names > or a million other things. We shouldn't spec implementations, only > interoperability requirements. > > Jim Hafner > > Santosh Rao @ece.cmu.edu on 05-08-2001 11:03:27 AM > > Sent by: owner-ips@ece.cmu.edu > > To: ips@ece.cmu.edu > cc: > Subject: Re: iSCSI: multiple sessions b/n a pair of WWUIs. > > Douglas Otis wrote: > > > The usefulness of ISID is also > > diminished if the client is not expected to maintain this value as unique > > across all Targets. > > I agree with the above and have stated to similar effect in my response > to Matt. A practical usage of ISID within an initiator node name space > is to assign it as unique across all targets, allowing the same ISID to > be used as a lookup key of the session for any given target port/node. > > Not applying uniqueness in ISID assignment across all targets would then > require initiators to implement a second key mechanism for target > lookup. > > - Santosh --------------356121B01F17742A8CF0EDA3 Content-Type: text/x-vcard; charset=us-ascii; name="santoshr.vcf" Content-Description: Card for Santosh Rao Content-Disposition: attachment; filename="santoshr.vcf" Content-Transfer-Encoding: 7bit begin:vcard n:Rao;Santosh tel;work:408-447-3751 x-mozilla-html:FALSE org:Hewlett Packard, Cupertino.;SISL adr:;;19420, Homestead Road, M\S 43LN, ;Cupertino.;CA.;95014.;USA. version:2.1 email;internet:santoshr@cup.hp.com title:Software Design Engineer x-mozilla-cpt:;21088 fn:Santosh Rao end:vcard --------------356121B01F17742A8CF0EDA3-- From owner-ips@ece.cmu.edu Wed May 9 13:47:06 2001 Received: from ece.cmu.edu ([128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id NAA19169 for ; Wed, 9 May 2001 13:47:06 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f49G9xi27643 for ips-outgoing; Wed, 9 May 2001 12:09:59 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from palrel2.hp.com (palrel2.hp.com [156.153.255.234]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f49G9vH27637 for ; Wed, 9 May 2001 12:09:57 -0400 (EDT) Received: from hpcuhe.cup.hp.com (hpcuhe.cup.hp.com [15.0.80.203]) by palrel2.hp.com (Postfix) with ESMTP id 6216917B2; Wed, 9 May 2001 09:09:55 -0700 (PDT) Received: from cup.hp.com (santoshr@hpindhhm.cup.hp.com [15.8.80.197]) by hpcuhe.cup.hp.com (8.9.3 (PHNE_18979)/8.9.3 SMKit7.02) with ESMTP id JAA13790; Wed, 9 May 2001 09:09:50 -0700 (PDT) Message-ID: <3AF96BC8.7C9F95AB@cup.hp.com> Date: Wed, 09 May 2001 09:09:44 -0700 From: Santosh Rao Organization: Hewlett Packard, Cupertino. X-Mailer: Mozilla 4.7 [en] (X11; U; HP-UX B.11.00 9000/778) X-Accept-Language: en MIME-Version: 1.0 To: John Hufferd Cc: ips@ece.cmu.edu Subject: Re: Initiator-detected format or digest errors References: Content-Type: multipart/mixed; boundary="------------89B88097CF72BF857249554C" Sender: owner-ips@ece.cmu.edu Precedence: bulk This is a multi-part message in MIME format. --------------89B88097CF72BF857249554C Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit John, I am appealing against having to respond with a REJECT on the occurrence of format errors and digest errors. The results of such an approach were documented in my last mail [further below]. Tape recovery can still be achieved by : a) issuing a SNACK on seeing a hole at the initiator. Issuing an R2T to request the missing data on seeing a hole at the target. (merely dropping a PDU without having to REJECT it would still cause a hole to be seen and allow recovery through either a SNACK or a request to re-transmit). or : b) Initiator issues a "command retry" to recover from an I/O underrun or an I/O timeout [assuming the target has buffered all the data in its iSCSI layer]. This approach would require implementations to splice the ULP TOV into a lower I/O timeout to allow for a potential "command retry" at the iSCSI layer before the expiry of ULP TOV. Not using the REJECT should not hinder recovery (?). Regards, Santosh John Hufferd wrote: > > Santosh, > It is not clear that your approach is good for tapes. > > . > . > . > John L. Hufferd > Senior Technical Staff Member (STSM) > IBM/SSG San Jose Ca > (408) 256-0403, Tie: 276-0403, eFax: (408) 904-4688 > Internet address: hufferd@us.ibm.com > > Santosh Rao @ece.cmu.edu on 05/08/2001 05:28:02 PM > > Sent by: owner-ips@ece.cmu.edu > > To: Matt Wakeley , Julian > Satran/Haifa/IBM@IBMIL > cc: ips@ece.cmu.edu > Subject: Re: Initiator-detected format or digest errors > > Why can't iSCSI apply a simple policy of discarding any PDU that has a > digest error or a format error ? Can the draft allow for such > implementations that wish to discard the PDU and do nothing more ? This > is all that FC initiators and targets need to do and they can recover > fine from CRC errors ! > > A dropped PDU will result in one of the following scenarios : > > --------------------------------------------------------------- > format error or digest error > impacts Scenario > --------------------------------------------------------------- > i) iSCSI Command Initiator timeout. > (could be detected > early on when initiator > retries command on seeing a > large diff b/n ExpCmdSN and > current CmdSN. > > ii) READ Data PDU I/O underrun, detected thru a > mis-match b/n count of DataSNs > received and EndDataSN. > > iii) WRITE Data PDU If this is not the "F" data PDU > for that R2T data phase, this > results in a target timoeut > followed by target aborting the > I/O > at the end of R2T data phase and > sending an aborted response > back. > > If this is a "F" data PDU, > results > in initiator timeout. (scenario > 1). > > iv) R2T PDU Target timeout. Initiator > timeout. > > v) Status PDU Initiator timeout. > > >From an implemenation's perspective, generating all of these REJECT > commands is just adding overhead and complexity, requiring the > allocation of resources [possibly by the host] for a REJECT PDU and > having to transmit a REJECT in response to each PDU that had either a > format or digest error. > > Any diagnostic info can be logged as counters within the > implementation's MIB. A format error or digest error causes the PDU to > be discarded. As it is, we need to apply a discard policy for the > "header digest error" & format error cases. Simplification of all digest > error and format error handling to a "discard policy" keeps > implementations [and the protocol] simple. > > - Santosh > > Matt Wakeley wrote: > > > > The target can log the reject for further analysis. > > > > Why do you keep trying to eliminate simple things that can be used for > > debugging? > > > > -Matt > > > > julian_satran@il.ibm.com wrote: > > > > > > Matt, > > > > > > 3 reasons: > > > > > > a. the initiator drives recovery. What can a target do with the reject? > > > (he is probaly in need of a reset) > > > > > > b.the initiator is unaware of the target state after thhe error > > > > > > c.there is reject command (and response) -:) > > > > > > Julo > > > > > > Matt Wakeley on 06/04/2001 00:43:08 > > > > > > Please respond to Matt Wakeley > > > > > > To: IPS Reflector > > > cc: > > > Subject: Re: Initiator-detected format or digest errors > > > > > > Why not send a reject back to the originator of the bad frame, and just > let > > > the task time out? > > > > > > -Matt > > > > > > julian_satran@il.ibm.com wrote: > > > > > > > > Tom, > > > > > > > > An initiator will pass the appropriate response to the SCSI layer and > > > will > > > > abort the task if it can identify one. Further behavior of > initiators > > > and > > > > targets is implementation dependent. > > > > > > > > 6.2 specifies this. > > > > > > > > Julo > > > > > > > > "Thomas McSweeney" on 04/04/2001 13:57:20 > > > > > > > > Please respond to "Thomas McSweeney" > > > > > > > > To: ips@ece.cmu.edu > > > > cc: > > > > Subject: Initiator-detected format or digest errors > > > > > > > > Section "2.20 Reject" talks about the target receiving a bad frame > and > > > > sending Reject. What should an Initiator do if it receives a PDU > with a > > > > format or digest error? Should it send Reject? If so, we'll need to > > > > ensure that the Initiator fields of the MIB include an object to > count > > > > Reject commands transmitted. > > > > > > > > Tom McSweeney > > > > iSCSI Development, Storage Systems Group, IBM > > > > Email: rf42tpme@us.ibm.com > > > > Phone: (USA) 919-254-5634 (tie line: 444-5634) > > > > Fax: (USA) 919-254-0391 (tie line: 444-0391) --------------89B88097CF72BF857249554C Content-Type: text/x-vcard; charset=us-ascii; name="santoshr.vcf" Content-Description: Card for Santosh Rao Content-Disposition: attachment; filename="santoshr.vcf" Content-Transfer-Encoding: 7bit begin:vcard n:Rao;Santosh tel;work:408-447-3751 x-mozilla-html:FALSE org:Hewlett Packard, Cupertino.;SISL adr:;;19420, Homestead Road, M\S 43LN, ;Cupertino.;CA.;95014.;USA. version:2.1 email;internet:santoshr@cup.hp.com title:Software Design Engineer x-mozilla-cpt:;21088 fn:Santosh Rao end:vcard --------------89B88097CF72BF857249554C-- From owner-ips@ece.cmu.edu Wed May 9 14:06:13 2001 Received: from ece.cmu.edu ([128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id OAA19740 for ; Wed, 9 May 2001 14:06:08 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f49GQJv29078 for ips-outgoing; Wed, 9 May 2001 12:26:19 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from palrel2.hp.com (palrel2.hp.com [156.153.255.234]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f49GQ3H29055 for ; Wed, 9 May 2001 12:26:18 -0400 (EDT) Received: from hpcuhe.cup.hp.com (hpcuhe.cup.hp.com [15.0.80.203]) by palrel2.hp.com (Postfix) with ESMTP id 259FEB8E for ; Wed, 9 May 2001 09:26:02 -0700 (PDT) Received: from cup.hp.com (santoshr@hpindhhm.cup.hp.com [15.8.80.197]) by hpcuhe.cup.hp.com (8.9.3 (PHNE_18979)/8.9.3 SMKit7.02) with ESMTP id JAA14714 for ; Wed, 9 May 2001 09:25:56 -0700 (PDT) Message-ID: <3AF96F8D.80F91857@cup.hp.com> Date: Wed, 09 May 2001 09:25:49 -0700 From: Santosh Rao Organization: Hewlett Packard, Cupertino. X-Mailer: Mozilla 4.7 [en] (X11; U; HP-UX B.11.00 9000/778) X-Accept-Language: en MIME-Version: 1.0 To: ips@ece.cmu.edu Subject: Re: Initiator-detected format or digest errors References: Content-Type: multipart/mixed; boundary="------------233CE20E023D64E1B74EB34A" Sender: owner-ips@ece.cmu.edu Precedence: bulk This is a multi-part message in MIME format. --------------233CE20E023D64E1B74EB34A Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit julian_satran@il.ibm.com wrote: > > And yes Santosh you have all the mechanisms to work in silent mode and a > verbose (reject) is required only for debugging (logging). Julian, I'm somewhat confused b/n your response and the current wording on this in the draft. Per the draft, on a format error : "While a session is active, whenever a target receives an iSCSI PDU with a format error, it MUST answer with a Reject iSCSI PDU with a Reason-code of Format Error." On a digest error : "When a target receives an iSCSI PDU with a header digest error or a payload digest error in an iSCSI PDU, it MUST answer with a Reject iSCSI PDU with a Reason-code of Header-Digest-error or Data-Digest-Error and discard the offending PDU." So, are implementations allowed to just discard the PDU and not have to send a REJECT(?). If so, the above wording may need to be updated to reflect the same. Regards, Santosh --------------233CE20E023D64E1B74EB34A Content-Type: text/x-vcard; charset=us-ascii; name="santoshr.vcf" Content-Description: Card for Santosh Rao Content-Disposition: attachment; filename="santoshr.vcf" Content-Transfer-Encoding: 7bit begin:vcard n:Rao;Santosh tel;work:408-447-3751 x-mozilla-html:FALSE org:Hewlett Packard, Cupertino.;SISL adr:;;19420, Homestead Road, M\S 43LN, ;Cupertino.;CA.;95014.;USA. version:2.1 email;internet:santoshr@cup.hp.com title:Software Design Engineer x-mozilla-cpt:;21088 fn:Santosh Rao end:vcard --------------233CE20E023D64E1B74EB34A-- From owner-ips@ece.cmu.edu Wed May 9 14:15:47 2001 Received: from ece.cmu.edu ([128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id OAA20086 for ; Wed, 9 May 2001 14:15:41 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f49GSpe29278 for ips-outgoing; Wed, 9 May 2001 12:28:51 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from msgbas1.cos.agilent.com (msgbas1x.cos.agilent.com [192.6.9.33]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f49GSmH29271 for ; Wed, 9 May 2001 12:28:49 -0400 (EDT) Received: from msgrel1.and.agilent.com (msgrel1.and.agilent.com [130.30.33.104]) by msgbas1.cos.agilent.com (Postfix) with ESMTP id F2FC334B for ; Wed, 9 May 2001 10:28:47 -0600 (MDT) Received: from rtl.rose.agilent.com (rtl.rose.agilent.com [156.140.232.231]) by msgrel1.and.agilent.com (Postfix) with ESMTP id 0A2A3EA for ; Wed, 9 May 2001 12:28:47 -0400 (EDT) Received: from mail.rose.agilent.com (mailsrv@bellhop [156.140.233.51]) by rtl.rose.agilent.com (8.9.3 (PHNE_18979)/8.9.3 SMKit7.1.0) with ESMTP id JAA02604 for ; Wed, 9 May 2001 09:28:45 -0700 (PDT) Received: from agilent.com (matt5670.rose.agilent.com [156.140.235.204]) by mail.rose.agilent.com (Netscape Messaging Server 3.6) with ESMTP id AAA16D7 for ; Wed, 9 May 2001 09:28:42 -0700 Message-ID: <3AF97051.B19EBC96@agilent.com> Date: Wed, 09 May 2001 09:29:05 -0700 From: "Matt Wakeley" Reply-To: Matt Wakeley Organization: Agilent Technologies X-Mailer: Mozilla 4.77 [en] (Windows NT 5.0; U) X-Accept-Language: en MIME-Version: 1.0 To: ips@ece.cmu.edu Subject: Re: iSCSI: immediate data References: Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-ips@ece.cmu.edu Precedence: bulk Content-Transfer-Encoding: 7bit The way I interpret the question is, what if you negotiated immediate data, but have no intention on sending immediate data for this command? Is setting the F bit, without sending any immediate data valid? (once again, more options....) -Matt julian_satran@il.ibm.com wrote: > > That is correct. The final bit indicates the end of unsolicited data. > If all you have is immediate then the final bit is 1in the command. > If you have immediate and other unsolicited (if enabled!) then set F to 0. > > 2.3.1 has been fixed accordingly. > > Regards, > Julo > > sandeepj@research.bell-labs.com (Sandeep Joshi) on 09-05-2001 01:56:25 > > Please respond to sandeepj@research.bell-labs.com (Sandeep Joshi) > > To: matt_wakeley@agilent.com > cc: ips@ece.cmu.edu > Subject: Re: iSCSI: immediate data > > Umm.....Julian replied to this one and said the final bit on ScsiCmd > would indicate it. Apparently, a single command can now send > the unsolicited data both with the command and in separate PDUs. > > I cant seem to find the email but I believe Sec 2.3.1 was changed > accordingly. Julian..? > > -Sandeep > > > The intention is that if an initiator requests to send immediate data > (and is > > granted the request), then it will always send immediate data. > > > > It sounds like you are asking for wishy washy mode... sometimes send > immediate > > data, sometimes not. > > > > -Matt > > > > Sandeep Joshi wrote: > > > > > > Julian, > > > > > > I had a follow-up question on an old thread. > > > http://ips.pdl.cs.cmu.edu/mail/msg03373.html > > > > > > The initiator is allowed to send firstburst in immediate > > > or in separate PDUs...but can it do neither ? I dont > > > see any statement to the effect that it MUST send a > > > firstburst. > > > > > > Here's a possible problem : > > > 1) FirstBurst is 4K > > > 2) Expected data length of command is 16K > > > 3) Initiator sends the Scsi command without immediate > > > data. (final flag on Scsi command is not set) > > > And it decides to send no further data PDUs. > > > > > > The target does not know if R2Ts can be sent since it does > > > not know if any unsolicited PDUs are expected. > > > > > > Perhaps the semantics on the final flag on SCSI command > > > need to be changed to indicate that no unsolicited data > > > will be sent with this command. Currently, the flag implies > > > (Sec 2.3.1) that all the required data has been sent with > > > the command. > > > > > > And here's a typo to fix. Appendix E (Key=immediateData) > > > The following should say "initialR2T is no". > > > The table has got it right. > > > > > > > If ImmediateData is set to yes and InitialR2T is set > > > > to yes then the initiator MAY send unsolicited immediate > > > > data or one unsolicited burst of Data-OUT PDUs but > > > > MUST NOT send both immediate and a unsolicited burst of > > > > Data-OUT PDUs for any one command. > > > > > > -Sandeep From owner-ips@ece.cmu.edu Wed May 9 14:15:57 2001 Received: from ece.cmu.edu ([128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id OAA20116 for ; Wed, 9 May 2001 14:15:57 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f49Gbuu00046 for ips-outgoing; Wed, 9 May 2001 12:37:56 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from palrel1.hp.com (palrel1.hp.com [156.153.255.242]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f49GbrH00040 for ; Wed, 9 May 2001 12:37:54 -0400 (EDT) Received: from hpcuhe.cup.hp.com (hpcuhe.cup.hp.com [15.0.80.203]) by palrel1.hp.com (Postfix) with ESMTP id 52A69106E for ; Wed, 9 May 2001 09:37:52 -0700 (PDT) Received: from cup.hp.com (santoshr@hpindhhm.cup.hp.com [15.8.80.197]) by hpcuhe.cup.hp.com (8.9.3 (PHNE_18979)/8.9.3 SMKit7.02) with ESMTP id JAA15410 for ; Wed, 9 May 2001 09:37:47 -0700 (PDT) Message-ID: <3AF97255.A36E36C6@cup.hp.com> Date: Wed, 09 May 2001 09:37:41 -0700 From: Santosh Rao Organization: Hewlett Packard, Cupertino. X-Mailer: Mozilla 4.7 [en] (X11; U; HP-UX B.11.00 9000/778) X-Accept-Language: en MIME-Version: 1.0 To: IPS Reflector Subject: Re: iSCSI : Bridging missing CmdSNs and Abort I/O Error recovery References: <3AF1FA33.5D3FB3C0@cup.hp.com> Content-Type: multipart/mixed; boundary="------------EBB802D7DF390D597E044473" Sender: owner-ips@ece.cmu.edu Precedence: bulk This is a multi-part message in MIME format. --------------EBB802D7DF390D597E044473 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit > julian_satran@il.ibm.com wrote: > > > We could add the RefCmdSN and it may help plug-in the hole but unless the > > > Abort is ussued on the same connection as the original command we can't > > be > > > sure that the old-one will not pop-up (as we enable ExpCmdSN to move on > > we > > > don't have even the 2**31-1 protection bracket :-)). Thus sending in > > > another nop/abort on the same connection is still required. > > > > > > To simplify the whole process I will: > > > > > > a - add RefCmdSN to the Task Management > > > b - add a command not received yet to the answers > > > > > > c - add a part to 7 reading: > > > > > > 1.1 How to Abort Safely a Command that Was Not Received > > > > > > To abort safely a task for which the task abort answer is "Command Not > > > Received Yet" the initiator must issue another abort command on the > > same > > > connection as the original command unless this connection was logged > > out > > > in which case it may send it on any connection. Julian, The above may not be sufficient, since a requirement for the Abort Task completion is that both the initiator & target can safely re-use their task tags and other resources following completion of the Abort Task. If the Abort Task is sent on another connection than the connection used for the original command [& that connection has not been logged out], then, there may be some in-flight PDUs from the target to the initiator for that command on the connection. Sending an Abort Task on another connection may cause the initiator to receive the Abort Task response indicating success on the other connection which then results in initiator freeing up its task tag and other resources used for that command. Thereafter, the stale PDUs on the original connection can show up at the initiator, causing problems. I would suggest that Abort Task be always sent on the same connection as the command, unless the connection used for that command has been logged out. Not doing so implies the initiator cannot safely re-use its task tag and other resources that were tied up for that I/O. Regards, Santosh --------------EBB802D7DF390D597E044473 Content-Type: text/x-vcard; charset=us-ascii; name="santoshr.vcf" Content-Description: Card for Santosh Rao Content-Disposition: attachment; filename="santoshr.vcf" Content-Transfer-Encoding: 7bit begin:vcard n:Rao;Santosh tel;work:408-447-3751 x-mozilla-html:FALSE org:Hewlett Packard, Cupertino.;SISL adr:;;19420, Homestead Road, M\S 43LN, ;Cupertino.;CA.;95014.;USA. version:2.1 email;internet:santoshr@cup.hp.com title:Software Design Engineer x-mozilla-cpt:;21088 fn:Santosh Rao end:vcard --------------EBB802D7DF390D597E044473-- From owner-ips@ece.cmu.edu Wed May 9 16:17:06 2001 Received: from ece.cmu.edu ([128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id QAA23602 for ; Wed, 9 May 2001 16:16:52 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f49IKXD09067 for ips-outgoing; Wed, 9 May 2001 14:20:33 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from server1.NishanSystems.COM (smtp.nishansystems.com [216.217.36.162]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f49IKSH09049 for ; Wed, 9 May 2001 14:20:29 -0400 (EDT) Received: by smtp.nishansystems.com with Internet Mail Service (5.5.2653.19) id ; Wed, 9 May 2001 11:20:22 -0700 Message-ID: From: Joshua Tseng To: ips@ece.cmu.edu Subject: SNS Requirements Date: Wed, 9 May 2001 11:20:21 -0700 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2653.19) Content-Type: text/plain; charset="iso-8859-1" Sender: owner-ips@ece.cmu.edu Precedence: bulk Folks, To complete my action item from the Nashua Meeting, the following is the updated requirements text for SNS for the NDT document. Please send comments on these requirements to the reflector. Thanks, Josh Tseng ----------------------------------- 5. Storage Name Server (SNS) The following section describes a set of generic requirements for any protocol or application used in the name service role to support iSCSI. This application or protocol is hereafter generically refered to as the SNS. An example of a Storage Name Server is the iSNS protocol described in the draft document draft-ietf-ips-iSNS-02.txt [8]. 5.1 Overview The SNS shall be architected using a client-server paradigm, with the SNS server acting as the information repository. SNS clients actively register and manipulate entity objects and their attributes in the SNS server. The SNS server SHALL be able to send asynchronous state change notifications to registered clients, and SHALL be able to send polling messages to monitor the availability of clients. Examples of SNS clients include initiators, targets, management stations, and switches. The SNS server can be hosted on a target, switch, or stand-alone server. 5.2 Login Control and Discovery Domains (DD's) The SNS MUST support Discovery Domains and Login control. The SNS must provide SNS clients with the ability to enforce DD configurations which may exist on the SNS server. Targets and management stations shall be able to register (i.e., upload) Login Control and Zoning configurations to the iSNS if authorized by the end user. DD and Login control supports two separate purposes: 5.2.1 Discovery Domain (DD) Partitions The SNS SHALL support the ability to partition the storage network into separate "Discovery Domains". The SNS shall not provide information if the SNS client performing the query is not in a common Discovery Domain as the SNS client that is the subject of the request. This capability prevents an initiator from attempting an iSCSI login to every single target in a large enterprise network, and is the iSCSI equivalent of "Soft" zoning. 5.2.2 Login Control To support login access security which is specified in the current iSCSI draft (Appendix A) [7] and MAY be implemented by the iSCSI target. The SNS SHALL support login control by storing a mapping of initiators that are permitted to access each target. Targets shall be able to query the SNS for a list of initiators that are allowed login access. This list shall include the key attribute (e.g., iSCSI Name) used to identify the initiator. This capability is the iSCSI equivalent of "Hard zoning". 5.3 Object Model The SNS MUST store the following objects and attributes: Network Entity: - Entity Identifier - Management IP Address - Entity Type (iSCSI) Portal: - Portal Index - IP Address - TCP Port Number Storage Node: - iSCSI Name - Alias - Node Type (target or initiator or both) Discovery Domain: - DD symbolic name - DD ID - DD Member: iSCSI Name A diagram of how the above objects are related is shown below. +----------------------------------------------------------------+ | IP Network | +------------+--------------------------------------+------------+ | | | | +-----+------+------+-----+ +-----+------+------+-----+ | | PORTAL | | | | PORTAL | | | | -IP Addr 1 | | | | -IP Addr 2 | | | | -TCP Port 1 | | | | -TCP Port 2 | | | +-----+ +-----+ | | +-----+ +-----+ | | | | | | | | | | | | | | | | | | +--------+ +--------+ | | +-------+ +--------+ | | | | | | | | | | | STORAGE NODE | | | | STORAGE NODE | | | | -iSCSI Name | | | | -iSCSI Name | | | | -Alias: "server1"| | | | -Alias: "disk1"| | | | -Type: initiator | | | | -Type: target | | | | | | | | | | | +-------------------+ | | +------------------+ | | | | | | NETWORK ENTITY | | NETWORK ENTITY | | -Entity ID (DNS): | | -Entity ID (DNS): | | "strg1.foo.com" | | "strg2.bar.com" | | -Type: iSCSI | | -Type: iSCSI | | | | | +-------------------------+ +-------------------------+ 5.4 SNS Authentication Requirements The SNS SHALL include authentication of SNS protocol messages between SNS clients and the SNS server. The authentication mechanism will allow for authentication of both client and server. 5.5 SNS Query and Registration Services Requirements The SNS SHALL allow SNS clients (initiators and targets) to register themselves with the SNS server. Initiators and targets also SHALL be able to query the SNS server for information. During registration, the initiators and the targets MUST be able to provide the following information: a) Storage Entity ID b) Portal object address (IP address and Port Number) c) iSCSI Name d) Storage node type They could optionally also provide other information such as: a) Alias string information When querying address information in order to establish an iSCSI connection, the query, as a minimum, should return the following information: a) PORTAL IP address(es) In the absence of SNS, the iSCSI Name and IP address(es) of the target device can be queried by issuing the SendTargets command to the default canonical iSCSI target present at the IP address and port number. 5.6 State Change Notification Requirements The SNS server MUST be able to inform SNS clients of changes to its database, including the availability of new SNS clients as a result of changes or modifications to DD policies. These changes may occur as a result of various events, including an SNS client actively manipulating the SNS database, response or non-response to an SNS monitoring message, or a hardware interrupt delivered by the SNS host platform (such as a switch). Asynchronous notification shall be delivered only to SNS clients that register for the notification, and only for SNS clients that are in the same DD as the event. 5.7 Monitoring Messages The SNS server MUST be able to poll client devices to monitor their availability on an ongoing basis. If a client fails to respond to monitoring messages, the SNS server shall take appropriate action, including sending state change notifications to other clients to inform them of the change in status. 5.8 Lightweight Protocol The SNS protocol SHALL be a lightweight protocol that can be scaled down for embedded implementation on switches and targets, or scaled up for implementation on servers. 5.9 The SNS SHALL meet the iSCSI boot requirements (see draft-ietf-ips-iscsi-boot-00.txt). From owner-ips@ece.cmu.edu Wed May 9 16:18:11 2001 Received: from ece.cmu.edu ([128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id QAA23625 for ; Wed, 9 May 2001 16:18:07 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f49IVM510062 for ips-outgoing; Wed, 9 May 2001 14:31:22 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from palrel1.hp.com (palrel1.hp.com [156.153.255.242]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f49IVIH10041 for ; Wed, 9 May 2001 14:31:18 -0400 (EDT) Received: from hpcuhe.cup.hp.com (hpcuhe.cup.hp.com [15.0.80.203]) by palrel1.hp.com (Postfix) with ESMTP id 8392A98C; Wed, 9 May 2001 11:31:15 -0700 (PDT) Received: from cup.hp.com (santoshr@hpindhhm.cup.hp.com [15.8.80.197]) by hpcuhe.cup.hp.com (8.9.3 (PHNE_18979)/8.9.3 SMKit7.02) with ESMTP id LAA22760; Wed, 9 May 2001 11:29:56 -0700 (PDT) Message-ID: <3AF98C9D.BC976DF1@cup.hp.com> Date: Wed, 09 May 2001 11:29:49 -0700 From: Santosh Rao Organization: Hewlett Packard, Cupertino. X-Mailer: Mozilla 4.7 [en] (X11; U; HP-UX B.11.00 9000/778) X-Accept-Language: en MIME-Version: 1.0 To: Jim Hafner Cc: IPS Reflector Subject: Re: iSCSI: multiple sessions b/n a pair of WWUIs. References: Content-Type: multipart/mixed; boundary="------------C306A54DFAE7FD6C9DA5E8FE" Sender: owner-ips@ece.cmu.edu Precedence: bulk This is a multi-part message in MIME format. --------------C306A54DFAE7FD6C9DA5E8FE Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Jim, Agreed on the ISID issue. One additional comment below. Regards, Santosh Jim Hafner wrote: > The spec might make a note that says that multiple sessions > handlers within a given iSCSI Node sharing a common iSCSI Name should > coordinate their use of ISIDs (carve the name space between them) to avoid > rejected login by one session handler because it reused an ISID already in > use by another session handler. As I've brought up before, login rejection when a target receives a second session login from the same (iSCSI Initiator Name, ISID) when the first session login is still active may not be a good option. This scenario can typically occur when the host rebooted without having done a logout and then attempts to re-login. The host must be allowed to maintain persistence of its ISID without being rejected in its login attempt. I'd suggest something along the following lines : " When the target receives a second session login while one is already active for a given (iSCSI Initiator Name, ISID), it MUST treat this as an implicit logout of the prior session and then accept the second login". [provided the login authentication was successful]. I believe this is also what Spteh Bailey had in mind when writing up algorithms for ERT (?). Do you see any issues with such an approach. (?) --------------C306A54DFAE7FD6C9DA5E8FE Content-Type: text/x-vcard; charset=us-ascii; name="santoshr.vcf" Content-Description: Card for Santosh Rao Content-Disposition: attachment; filename="santoshr.vcf" Content-Transfer-Encoding: 7bit begin:vcard n:Rao;Santosh tel;work:408-447-3751 x-mozilla-html:FALSE org:Hewlett Packard, Cupertino.;SISL adr:;;19420, Homestead Road, M\S 43LN, ;Cupertino.;CA.;95014.;USA. version:2.1 email;internet:santoshr@cup.hp.com title:Software Design Engineer x-mozilla-cpt:;21088 fn:Santosh Rao end:vcard --------------C306A54DFAE7FD6C9DA5E8FE-- From owner-ips@ece.cmu.edu Wed May 9 16:20:48 2001 Received: from ece.cmu.edu ([128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id QAA23678 for ; Wed, 9 May 2001 16:20:47 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f49Hfqx05498 for ips-outgoing; Wed, 9 May 2001 13:41:52 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from palrel2.hp.com (palrel2.hp.com [156.153.255.234]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f49HfnH05493 for ; Wed, 9 May 2001 13:41:50 -0400 (EDT) Received: from hpcuhe.cup.hp.com (hpcuhe.cup.hp.com [15.0.80.203]) by palrel2.hp.com (Postfix) with ESMTP id DA5E81373; Wed, 9 May 2001 10:41:47 -0700 (PDT) Received: from cup.hp.com (santoshr@hpindhhm.cup.hp.com [15.8.80.197]) by hpcuhe.cup.hp.com (8.9.3 (PHNE_18979)/8.9.3 SMKit7.02) with ESMTP id KAA19571; Wed, 9 May 2001 10:41:40 -0700 (PDT) Message-ID: <3AF9814E.AB090D19@cup.hp.com> Date: Wed, 09 May 2001 10:41:34 -0700 From: Santosh Rao Organization: Hewlett Packard, Cupertino. X-Mailer: Mozilla 4.7 [en] (X11; U; HP-UX B.11.00 9000/778) X-Accept-Language: en MIME-Version: 1.0 To: John Hufferd Cc: Black_David@emc.com, ips@ece.cmu.edu Subject: Re: iSCSI: multiple sessions b/n a pair of WWUIs. References: Content-Type: multipart/mixed; boundary="------------F034858B6126CEEC4659F0A4" Sender: owner-ips@ece.cmu.edu Precedence: bulk This is a multi-part message in MIME format. --------------F034858B6126CEEC4659F0A4 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit John Hufferd wrote: > > Santosh, > I think I am beginning to see the problem. A given iSCSI Initiator Port > can NOT have a second session with the same iSCSI Initiator Port and be John, ^^^^^^^^^^^^^^^^^^^^^ Is the above "same iSCSI target port" (?). > consistent with SCSIness. The second session could be started with another > ISID, (to the same iSCSI Target Port) but if the session was established it > would not have any of its commands and data handled via any techniques > defined by SCSI or iSCSI. In fact its use would require a wedge driver. Perhaps, some of my confusion arises from a lack of understanding of your definition of iSCSI Initiator Port and iSCSI Target Port. Some clarification on your definition of these would help make things more clear. > This is the exact confusion I was trying to avoid. Technically if you were > actually able to start another Session to the same Target Port, it would > be, by definition another iSCSI Initiator Port. The two different Ports > would need to be coordinated by what we have been calling a Wedge Driver. I'm not sure why you believe a wedge driver would be required to handle such a configuration.(?) Per my understanding [which may be missing some aspect], the host can operate in such a configuration in the absence of a wedge driver. Also, the host could create such multiple sessions to the same target port in order to separate sub-sets of LUNs into different sessions and apply different session properties to each such LUN subset. > > Today, the way we use multiple Fibre Channel Initiator Ports connected to > the same Fibre Channel Target Ports is by use of Wedge Drivers. With FC, connecting multiple initiator ports from the *same* host to a single target port makes little or no sense. The host would see the same LUNs twice through different initiator ports. A more typical configuration is to connect multiple FC initiator ports from *different* hosts to the same target port as a high availability configuration. Yes, in such a case, some type of cluster application would be managing the switch-over from one initiator port to another. (Is this the case being referred to ?). > I think > what you are suggesting causes the need for a Wedge Drivers to integrate > the iSCSI Initiator Ports. (I guess we wandered off into a different line of discussion which is "whether multiple sessions between an initiator port and target port are legal and required" ? :) My answer to that would be, yes, they are legal and required. They also do not require the use of a wedge driver. The host may configure multiple sessions from one iSCSI initiator NIC to a given iSCSI TargetAddress so as to access LUN subsets from different sessions, and apply different session properties to different LUN sub-sets. Such multiple sessions that are originated on the same initiator NIC need to use different ISIDs [for all of the sessions for that given target node] and thereby, are considered multiple iSCSI Initiator Ports, based on the current name-disc draft's defn. Not sure that we want to cause this type of > thinking, by accident. One of the reasons for Multiple Connections per > Session was to remove the "Hard" need for Wedge Drivers. IMHO, "multiple connections per session" complements wedge drivers in their functionality but may not comletely replace them. A lot of wedge drivers have code specific to the SCSI properties of the target and moving all of this functionality into the iSCSI layer would imply adding device specific code into this layer. Not the cleanest of layering schemes. Also, some wedge drivers claim to utilize device proprietary features, which a device independent iSCSI initiator layer could not provide through "multiple connections per session". Regards, Santosh --------------F034858B6126CEEC4659F0A4 Content-Type: text/x-vcard; charset=us-ascii; name="santoshr.vcf" Content-Description: Card for Santosh Rao Content-Disposition: attachment; filename="santoshr.vcf" Content-Transfer-Encoding: 7bit begin:vcard n:Rao;Santosh tel;work:408-447-3751 x-mozilla-html:FALSE org:Hewlett Packard, Cupertino.;SISL adr:;;19420, Homestead Road, M\S 43LN, ;Cupertino.;CA.;95014.;USA. version:2.1 email;internet:santoshr@cup.hp.com title:Software Design Engineer x-mozilla-cpt:;21088 fn:Santosh Rao end:vcard --------------F034858B6126CEEC4659F0A4-- From owner-ips@ece.cmu.edu Wed May 9 16:37:15 2001 Received: from ece.cmu.edu ([128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id QAA24088 for ; Wed, 9 May 2001 16:37:10 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f49IHVv08786 for ips-outgoing; Wed, 9 May 2001 14:17:31 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from dogwood.cisco.com (dogwood.cisco.com [161.44.11.19]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f49IHSH08779 for ; Wed, 9 May 2001 14:17:28 -0400 (EDT) Received: from ahganemw2k (dhcp-161-44-68-139.cisco.com [161.44.68.139]) by dogwood.cisco.com (8.8.6 (PHNE_14041)/CISCO.SERVER.1.2) with SMTP id OAA10764 for ; Wed, 9 May 2001 14:17:22 -0400 (EDT) From: "Ayman Ghanem" To: Subject: RE: iSCSI: immediate data Date: Wed, 9 May 2001 13:16:14 -0500 Message-ID: MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit X-Priority: 3 (Normal) X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook IMO, Build 9.0.2416 (9.0.2910.0) Importance: Normal X-MimeOLE: Produced By Microsoft MimeOLE V5.00.2919.6700 In-Reply-To: Sender: owner-ips@ece.cmu.edu Precedence: bulk Content-Transfer-Encoding: 7bit I thought that you can not send both immediate and unsolicited data for the same command. My understanding is that if there is immediate data and the F-bit is not set, then the remaining has to go as solicited even if InitialR2T=no is enabled. Is this changing from draft-06?. -Ayman > -----Original Message----- > From: owner-ips@ece.cmu.edu [mailto:owner-ips@ece.cmu.edu]On Behalf Of > julian_satran@il.ibm.com > Sent: Wednesday, May 09, 2001 2:12 AM > To: Sandeep Joshi > Cc: matt_wakeley@agilent.com; ips@ece.cmu.edu > Subject: Re: iSCSI: immediate data > > > > > That is correct. The final bit indicates the end of unsolicited data. > If all you have is immediate then the final bit is 1in the command. > If you have immediate and other unsolicited (if enabled!) then set F to 0. > > 2.3.1 has been fixed accordingly. > > Regards, > Julo > > sandeepj@research.bell-labs.com (Sandeep Joshi) on 09-05-2001 01:56:25 > > Please respond to sandeepj@research.bell-labs.com (Sandeep Joshi) > > To: matt_wakeley@agilent.com > cc: ips@ece.cmu.edu > Subject: Re: iSCSI: immediate data > > > > > Umm.....Julian replied to this one and said the final bit on ScsiCmd > would indicate it. Apparently, a single command can now send > the unsolicited data both with the command and in separate PDUs. > > I cant seem to find the email but I believe Sec 2.3.1 was changed > accordingly. Julian..? > > -Sandeep > > > The intention is that if an initiator requests to send immediate data > (and is > > granted the request), then it will always send immediate data. > > > > It sounds like you are asking for wishy washy mode... sometimes send > immediate > > data, sometimes not. > > > > -Matt > > > > Sandeep Joshi wrote: > > > > > > Julian, > > > > > > I had a follow-up question on an old thread. > > > http://ips.pdl.cs.cmu.edu/mail/msg03373.html > > > > > > The initiator is allowed to send firstburst in immediate > > > or in separate PDUs...but can it do neither ? I dont > > > see any statement to the effect that it MUST send a > > > firstburst. > > > > > > Here's a possible problem : > > > 1) FirstBurst is 4K > > > 2) Expected data length of command is 16K > > > 3) Initiator sends the Scsi command without immediate > > > data. (final flag on Scsi command is not set) > > > And it decides to send no further data PDUs. > > > > > > The target does not know if R2Ts can be sent since it does > > > not know if any unsolicited PDUs are expected. > > > > > > Perhaps the semantics on the final flag on SCSI command > > > need to be changed to indicate that no unsolicited data > > > will be sent with this command. Currently, the flag implies > > > (Sec 2.3.1) that all the required data has been sent with > > > the command. > > > > > > And here's a typo to fix. Appendix E (Key=immediateData) > > > The following should say "initialR2T is no". > > > The table has got it right. > > > > > > > If ImmediateData is set to yes and InitialR2T is set > > > > to yes then the initiator MAY send unsolicited immediate > > > > data or one unsolicited burst of Data-OUT PDUs but > > > > MUST NOT send both immediate and a unsolicited burst of > > > > Data-OUT PDUs for any one command. > > > > > > -Sandeep > > > > From owner-ips@ece.cmu.edu Wed May 9 16:37:47 2001 Received: from ece.cmu.edu ([128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id QAA24131 for ; Wed, 9 May 2001 16:37:47 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f49IkPH11318 for ips-outgoing; Wed, 9 May 2001 14:46:25 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from e31.bld.us.ibm.com (e31.co.us.ibm.com [32.97.110.129]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f49IkNH11314 for ; Wed, 9 May 2001 14:46:23 -0400 (EDT) Received: from westrelay02.boulder.ibm.com (westrelay02.boulder.ibm.com [9.99.140.23]) by e31.bld.us.ibm.com (8.9.3/8.9.3) with ESMTP id OAA10252 for ; Wed, 9 May 2001 14:38:52 -0400 Received: from f3n42e (d03nm042h.boulder.ibm.com [9.99.140.42]) by westrelay02.boulder.ibm.com (8.11.1/NCO v4.96.1.0) with ESMTP id f49Ijsc73506 for ; Wed, 9 May 2001 12:45:54 -0600 Importance: Normal Subject: Re: iSCSI: multiple sessions b/n a pair of WWUIs. To: ips@ece.cmu.edu From: "Jim Hafner" Date: Wed, 9 May 2001 11:45:51 -0700 Message-ID: X-MIMETrack: Serialize by Router on D03NM042/03/M/IBM(Release 5.0.6 |December 14, 2000) at 05/09/2001 11:45:53 AM MIME-Version: 1.0 Content-type: text/plain; charset=us-ascii Sender: owner-ips@ece.cmu.edu Precedence: bulk Santosh, Comments in line ... Jim Hafner Santosh Rao @cup.hp.com on 05-09-2001 08:54:54 AM Sent by: santoshr@cup.hp.com To: Jim Hafner/Almaden/IBM@IBMUS cc: ips@ece.cmu.edu Subject: Re: iSCSI: multiple sessions b/n a pair of WWUIs. Jim, I agree that the only requirement is uniqueness of ISID among all the sessions to a given target node. Maintaining uniqueness across all targets only extends compliance to this requirement a step further and does not violate it. Correct, it does not violate it and that's all that matters. Do this if you want, it's your implementation :-{) All we are debating is David's suggestion that a note could be added to indicate that implementations may choose to apply this uniqueness across all target nodes so as to use the ISID as a target lookup mechanism. I'm beginning to believe strongly that such a note is NOT a good thing as it implies a preferred implementation and (a) that's not the job of the spec and (b) we haven't hashed out all the possible issues to be sure we would be recommending a "best" implementation (and such hashing is NOT the job of this reflector). If there is strong resistance against adding such a note, then, let us omit it. [as long as the spec does'nt preclude usage of ISID as unique across all targets]. As I said, I'm moving rapidly to "strong" against. I think the spec (and any related document or note) currently does not say (and agreeably shouldn't say) that the ISIDs must be reused in any particular way. That option would be simply falling on the other side of the implementation fence. The spec might make a note that says that multiple sessions handlers within a given iSCSI Node sharing a common iSCSI Name should coordinate their use of ISIDs (carve the name space between them) to avoid rejected login by one session handler because it reused an ISID already in use by another session handler. But that's about all it should say. Regards, Santosh Jim Hafner wrote: > > Santosh, > > As I've said, I agree with the sentiment, but that is not the only > implementation option. For example, why can't the database index on the > pointer to the data structure that has the names, etc, or hash on the names > or a million other things. We shouldn't spec implementations, only > interoperability requirements. > > Jim Hafner > > Santosh Rao @ece.cmu.edu on 05-08-2001 11:03:27 AM > > Sent by: owner-ips@ece.cmu.edu > > To: ips@ece.cmu.edu > cc: > Subject: Re: iSCSI: multiple sessions b/n a pair of WWUIs. > > Douglas Otis wrote: > > > The usefulness of ISID is also > > diminished if the client is not expected to maintain this value as unique > > across all Targets. > > I agree with the above and have stated to similar effect in my response > to Matt. A practical usage of ISID within an initiator node name space > is to assign it as unique across all targets, allowing the same ISID to > be used as a lookup key of the session for any given target port/node. > > Not applying uniqueness in ISID assignment across all targets would then > require initiators to implement a second key mechanism for target > lookup. > > - Santosh From owner-ips@ece.cmu.edu Wed May 9 17:59:44 2001 Received: from ece.cmu.edu ([128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id RAA26620 for ; Wed, 9 May 2001 17:59:39 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f49JrdJ17384 for ips-outgoing; Wed, 9 May 2001 15:53:39 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from e31.bld.us.ibm.com (e31.co.us.ibm.com [32.97.110.129]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f49JrbH17362 for ; Wed, 9 May 2001 15:53:37 -0400 (EDT) Received: from westrelay02.boulder.ibm.com (westrelay02.boulder.ibm.com [9.99.140.23]) by e31.bld.us.ibm.com (8.9.3/8.9.3) with ESMTP id PAA141610 for ; Wed, 9 May 2001 15:46:06 -0400 Received: from f3n42e (d03nm042h.boulder.ibm.com [9.99.140.42]) by westrelay02.boulder.ibm.com (8.11.1/NCO v4.96.1.0) with ESMTP id f49JrYc67424 for ; Wed, 9 May 2001 13:53:34 -0600 Importance: Normal Subject: iSCSI: SAM and iSCSI - some issues To: ips@ece.cmu.edu From: "Jim Hafner" Date: Wed, 9 May 2001 12:53:32 -0700 Message-ID: X-MIMETrack: Serialize by Router on D03NM042/03/M/IBM(Release 5.0.6 |December 14, 2000) at 05/09/2001 12:53:34 PM MIME-Version: 1.0 Content-type: text/plain; charset=us-ascii Sender: owner-ips@ece.cmu.edu Precedence: bulk Folks, At the interim meeting, I presented a model of how SAM constructs (particularly those of nexus, SCSI Device and SCSI Port) can map onto the iSCSI constructs. I've sent a copy of the presentation to Elizabeth, so I assume it will get posted somewhere. In particular, an iSCSI session maps neatly to a SCSI nexus. There were some open issues concerning the SCSI state of a nexus and how that state gets affected by various events that affect the session. From the presentation, here is a (probably incomplete) list of the state information: * Some nexus state - persistent reservations - task set - mode page settings - unit attention conditions - ACA (added at the meeting) - access control enrollment state (per spec) - alias lists (yet to be approved by T10) I'd welcome additions to this list. A major open question is which of these must persist through various scenarios where the session/nexus gets torn down and then rebuilt. I see three broad stroked types of "tear down" events: (a) Initiator does an explicit or implicit logout (e.g., in advance of a shutdown) but target stays functioning (b) Target does an explicit or implicit logout (e.g., in advance of a power cycle or other hard reset event) but the initiator stays functioning (c) Session collapses because all the connections fail but the initiator and target both stay functioning. With each of these (or perhaps at a finer granularity) we need to determine what state information should be preserved in case the nexus/session gets rebuilt. For scenario (c), this might be described under error recovery models. Clearly, a persistent reservation should survive target recycles (under the appropriate APTPL setting). What this means is that if the target recycles, the "identified initiator port" will still own the reservation, and IF the nexus gets rebuilt, the initiator port will be recognized as the owner of the reservation. My current pre-thinking is that all the others are volatile enough to be cleared through all these tear-down events with the exception of access controls enrollment state. Without going into too much detail, I think scenarios (a) and (b) can "de-enroll" and scenario (c) can "pending-enroll" the state. [If you're not familar with this part of the spec, its approved for inclusion in SPC-3, and the main relevant document is ftp://ftp.t10.org/t10/document.99/99-245r9.pdf.] I welcome comments about these issues. In a related thread, Santosh has requested a table analogous to Table 4 of FCP2 which describes other consequences of events of similar type. I think there is overlap in that proposed table and the one I'm suggesting here (and I support that effort). Jim Hafner From owner-ips@ece.cmu.edu Wed May 9 20:55:37 2001 Received: from ece.cmu.edu ([128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id UAA00166 for ; Wed, 9 May 2001 20:55:36 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f49MZIo00432 for ips-outgoing; Wed, 9 May 2001 18:35:18 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from mailcity.com (fes.whowhere.com [209.185.123.154]) by ece.cmu.edu (8.11.0/8.10.2) with SMTP id f49MZAH00406 for ; Wed, 9 May 2001 18:35:14 -0400 (EDT) Received: from Unknown/Local ([?.?.?.?]) by mailcity.com; Wed May 9 15:34:47 2001 To: "iSCSI" Date: Wed, 09 May 2001 15:34:47 -0700 From: "m nima" Message-ID: Mime-Version: 1.0 X-Sent-Mail: on Reply-To: m_nima_1@lycos.com X-Mailer: MailCity Service Subject: FW: iSCSI: Nailing down CRC-32C X-Sender-Ip: 63.70.210.20 Organization: Lycos Mail (http://mail.lycos.com:80) Content-Type: text/plain; charset=us-ascii Content-Language: en Content-Transfer-Encoding: 7bit Sender: owner-ips@ece.cmu.edu Precedence: bulk Content-Transfer-Encoding: 7bit On the other hand, Marker is a killer for SW implementation, as it needs to stop and insert it (or copy the buffer to a new one and insert the buffer) I think the key for iSCSI is to make after the exhaustive standardization work, we're ready for 10G, markers and the proposed CRC leave iSCSI far behind. Maybe we need to look again at ULF before we move along...Endorsement of Framing by TSVWG was a step in this direction, I'd think. Nima -----Original Message----- From: Matt Wakeley [mailto:matt_wakeley@agilent.com] Sent: Tuesday, May 08, 2001 11:19 AM To: ips@ece.cmu.edu Subject: Re: iSCSI: Nailing down CRC-32C There is a 4th option. Mandate the implementation of iSCSI markers. This will significantly reduce the amount of buffering required. I think we should just go ahead and mandate implementation of markers, for two reasons: 1- the "ULF (or WARP)" proposal doesn't seem to be making much headway, and 2- even if ULF did become reality, there are software implementations today that will not have the luxury of having a "enhanced" TCP stack that would provide the framing. Software implementations of iSCSI MUST supply iSCSI markers (if invoked to do so) to enable low cost, high performance 10Gig solutions. -Matt Nima wrote: > > Assumptions: > > 1) The iSCSI DATA PDU Header and Data Integrity checks uses CRC-32 > 2) Support of Integrity check is mandatory for both header and Data > digest > 3) An iSCSI PDU may span multiple TCP segments. > 4) Any of these segments may arrive out of order at the receiver. > 5) To regenerate the CRC for comparison with the value present in the > PDU received, the receiver needs to keep in a temporary buffers all > segments, waiting for the missing one, as CRC must be computed using > the whole re-assembled PDU > > Question: Does CRC-32 support partial computation and compensation > for wrong initial values later (I guess NO!), to enable partial CRC > calc on avail parts of PDU? > > Possible answer: No > > Possible Conclusion: The RCV side is further complicated b/c of this > CRC > > Possible solutions: > > 1) Use CRC that allows partial calc > 2) Limit iSCSI PDU to TCP MSS if data CRC is used (added o/h)... > 3) Leave as it, though it has cost , performance implication and hurts > scaling to 10G.. > > Nima > > -----Original Message----- > From: julian_satran@il.ibm.com [mailto:julian_satran@il.ibm.com] > Sent: Saturday, May 05, 2001 9:21 AM > To: ips@ece.cmu.edu > Subject: Re: iSCSI: Nailing down CRC-32C > > The CRC part of the appendix (for 07) reads: > > The following table lists cyclic integrity checksums that can be > negotiated for the digests and MUST be implemented by every iSCSI > initiator and target. Note that these digest options have only error > detection significance. > > +---------------------------------------------+ > | Name | Description | > +---------------------------------------------+ > | crc-32C | 32 bit CRC | 11EDC6F41 | > +---------------------------------------------+ > | none | no digest | > +---------------------------------------------+ > > The generator polynomials for those digests are given in hex-notation, > for example 3a stands for 0011 1010 - the polynomial x**5+X**4+x**3+x+1. > > The generator polynomial selected is evaluated in [Castagnioli93]. > When using the CRC the CRC register must be initialized to all 1s > (0xFFFFFFFF) and the CRC bits must be complemented before transmission. > Padding bytes, when presents in a segment covered by a CRC, have to be > set to 0 and are included in the CRC. > > Regards, > Julo > > Mark Bakke on 05-05-2001 00:13:09 > > Please respond to Mark Bakke > > To: IPS > cc: > Subject: iSCSI: Nailing down CRC-32C > > At the interim meeting, it was stated that iSCSI has selected > the CRC-32C polynomial as its required iSCSI-level header and > data CRC. Now that we have it, it's time to move on and make > sure we can implement it. > > In the interest of interoperability, we need to not only specify > the polynomial, but also the initial values, bit and byte > ordering, etc. > > "A Painless Guide to CRC Error Detection Algorithms" > (http://www.ross.net/crc/crcpaper.html) specifies a > method to unambiguously characterize these parameters > (sections 15 and 16). Has anyone taken a shot at defining > these yet? Otherwise, here is what it might look like: > > Name : "CRC-32C" > Width : 32 > Poly : 1EDC6F41 (note that the leading "1" is implied) > Init : FFFFFFFF > RefIn : True > RefOut : True > XorOut : FFFFFFFF > Check : ? > > I haven't attempted to create check data based on these yet. As > soon as the other parameters are nailed down, we need to do this. > > Anyway, I am not a CRC expert, and can't make any statement about > the above values being the "best" way to do this, but instead just > copied them (except the polynomial itself) from the Ethernet CRC, > since that is likely the easiest for everyone implementing hardware > to deal with. > > If someone else is already doing this, let me know; I just wanted > to start this thread so we can get closure. > > Regards, > > Mark > > -- > Mark A. Bakke > Cisco Systems > mbakke@cisco.com > 763.398.1054 > > _______________________________________________________ > Send a cool gift with your E-Card > http://www.bluemountain.com/giftcenter/ Get 250 color business cards for FREE! http://businesscards.lycos.com/vp/fastpath/ From owner-ips@ece.cmu.edu Wed May 9 23:14:06 2001 Received: from ece.cmu.edu ([128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id XAA03622 for ; Wed, 9 May 2001 23:14:06 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f4A1Ow111958 for ips-outgoing; Wed, 9 May 2001 21:24:58 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from e31.bld.us.ibm.com (e31.co.us.ibm.com [32.97.110.129]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f4A1OvH11953 for ; Wed, 9 May 2001 21:24:57 -0400 (EDT) Received: from westrelay02.boulder.ibm.com (westrelay02.boulder.ibm.com [9.99.140.23]) by e31.bld.us.ibm.com (8.9.3/8.9.3) with ESMTP id VAA89182 for ; Wed, 9 May 2001 21:17:26 -0400 Received: from f4n49e (d03nm065h.boulder.ibm.com [9.99.140.49]) by westrelay02.boulder.ibm.com (8.11.1/NCO v4.96.1.0) with ESMTP id f4A1Ouc239620 for ; Wed, 9 May 2001 19:24:56 -0600 X-Priority: 1 (High) Importance: Normal Subject: Re: iSCSI: SAM and iSCSI - some issues To: "Jim Hafner" Cc: ips@ece.cmu.edu X-Mailer: Lotus Notes Release 5.0.3 (Intl) 21 March 2000 Message-ID: From: "John Hufferd" Date: Wed, 9 May 2001 18:24:46 -0700 X-MIMETrack: Serialize by Router on D03NM065/03/M/IBM(Release 5.0.6 |December 14, 2000) at 05/09/2001 07:24:55 PM MIME-Version: 1.0 Content-type: text/plain; charset=us-ascii Sender: owner-ips@ece.cmu.edu Precedence: bulk Jim, I agree with your note, but am adding another perhaps confusion item. This is the Target Failure/Take-over event. 1. What should be done if the IP address is not taken over, including what state Must be kept, etc. 2. What should be done if the IP address is taken over but perhaps not all state is available to the take over node. What State Must have been made available, and what state is not required. Example: the state of the ExpCmdSN, etc. I have been wondering about point 2 and what should happen if such state is not kept, what should the recovery action be? Perhaps it should blow the session away and start over, if so, why did it cause the IP take over to occur, instead of letting the Initiator just establish a new session, as in point 1? Anyway, you may or may not think that this event is worth considering in your nexus state analysis. . . . John L. Hufferd Senior Technical Staff Member (STSM) IBM/SSG San Jose Ca (408) 256-0403, Tie: 276-0403, eFax: (408) 904-4688 Internet address: hufferd@us.ibm.com Jim Hafner/Almaden/IBM@IBMUS@ece.cmu.edu on 05/09/2001 12:53:32 PM Sent by: owner-ips@ece.cmu.edu To: ips@ece.cmu.edu cc: Subject: iSCSI: SAM and iSCSI - some issues Folks, At the interim meeting, I presented a model of how SAM constructs (particularly those of nexus, SCSI Device and SCSI Port) can map onto the iSCSI constructs. I've sent a copy of the presentation to Elizabeth, so I assume it will get posted somewhere. In particular, an iSCSI session maps neatly to a SCSI nexus. There were some open issues concerning the SCSI state of a nexus and how that state gets affected by various events that affect the session. From the presentation, here is a (probably incomplete) list of the state information: * Some nexus state - persistent reservations - task set - mode page settings - unit attention conditions - ACA (added at the meeting) - access control enrollment state (per spec) - alias lists (yet to be approved by T10) I'd welcome additions to this list. A major open question is which of these must persist through various scenarios where the session/nexus gets torn down and then rebuilt. I see three broad stroked types of "tear down" events: (a) Initiator does an explicit or implicit logout (e.g., in advance of a shutdown) but target stays functioning (b) Target does an explicit or implicit logout (e.g., in advance of a power cycle or other hard reset event) but the initiator stays functioning (c) Session collapses because all the connections fail but the initiator and target both stay functioning. With each of these (or perhaps at a finer granularity) we need to determine what state information should be preserved in case the nexus/session gets rebuilt. For scenario (c), this might be described under error recovery models. Clearly, a persistent reservation should survive target recycles (under the appropriate APTPL setting). What this means is that if the target recycles, the "identified initiator port" will still own the reservation, and IF the nexus gets rebuilt, the initiator port will be recognized as the owner of the reservation. My current pre-thinking is that all the others are volatile enough to be cleared through all these tear-down events with the exception of access controls enrollment state. Without going into too much detail, I think scenarios (a) and (b) can "de-enroll" and scenario (c) can "pending-enroll" the state. [If you're not familar with this part of the spec, its approved for inclusion in SPC-3, and the main relevant document is ftp://ftp.t10.org/t10/document.99/99-245r9.pdf.] I welcome comments about these issues. In a related thread, Santosh has requested a table analogous to Table 4 of FCP2 which describes other consequences of events of similar type. I think there is overlap in that proposed table and the one I'm suggesting here (and I support that effort). Jim Hafner From owner-ips@ece.cmu.edu Wed May 9 23:16:46 2001 Received: from ece.cmu.edu ([128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id XAA03635 for ; Wed, 9 May 2001 23:16:46 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f4A1CZb11150 for ips-outgoing; Wed, 9 May 2001 21:12:35 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from msgbas1t.cos.agilent.com (msgbas1tx.cos.agilent.com [192.6.9.34]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f4A1CUH11144 for ; Wed, 9 May 2001 21:12:30 -0400 (EDT) Received: from msgrel1.and.agilent.com (msgrel1.and.agilent.com [130.30.33.104]) by msgbas1t.cos.agilent.com (Postfix) with ESMTP id 6A6A32A3 for ; Wed, 9 May 2001 19:12:29 -0600 (MDT) Received: from rtl.rose.agilent.com (rtl.rose.agilent.com [156.140.232.231]) by msgrel1.and.agilent.com (Postfix) with ESMTP id 8EDB9128 for ; Wed, 9 May 2001 21:12:28 -0400 (EDT) Received: from mail.rose.agilent.com (bellhop [156.140.233.51]) by rtl.rose.agilent.com (8.9.3 (PHNE_18979)/8.9.3 SMKit7.1.0) with ESMTP id SAA24118 for ; Wed, 9 May 2001 18:12:27 -0700 (PDT) Received: from agilent.com (cos1nai133023.cos.agilent.com [141.184.133.23]) by mail.rose.agilent.com (Netscape Messaging Server 3.6) with ESMTP id AAA5E7E for ; Wed, 9 May 2001 18:12:23 -0700 Message-ID: <3AF9EAF3.672317BA@agilent.com> Date: Wed, 09 May 2001 18:12:19 -0700 From: "Matt Wakeley" Reply-To: Matt Wakeley Organization: Agilent Technologies X-Mailer: Mozilla 4.77 [en] (Windows NT 5.0; U) X-Accept-Language: en MIME-Version: 1.0 To: iSCSI Subject: iSCSI: mandating markers (was Nailing down CRC-32C) References: Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-ips@ece.cmu.edu Precedence: bulk Content-Transfer-Encoding: 7bit m nima wrote: > > On the other hand, Marker is a killer for SW implementation, as it > needs to stop and insert it (or copy the buffer to a new one and > insert the buffer) The point is, it only needs to insert it if communicating with a device that requests markers. That means for 1gig, usually no markers. When 10gig comes along, markers will be used, but by then, there should be few SW implementations. > > I think the key for iSCSI is to make after the exhaustive > standardization work, we're ready for 10G, markers and the proposed > CRC leave iSCSI far behind. I'm sorry. I don't understand what the above means. > > Maybe we need to look again at ULF before we move along...Endorsement > of Framing by TSVWG was a step in this direction, I'd think. The point is, are you going to "upgrade" your software solutions to use the new ULF or whatever when it is finally approved? If your software iscsi is running over a transport (tcp) that you have no control over (microsoft, etc), then I'll bet the answer is "no, I can't". Therefore, the only solution is to mandate something in iSCSI that can be implemented by legacy TCP stacks to help the performance and cost issues of 10G iSCSI. Otherwise, Fibre Channel will always be cheaper (less buffering) and faster (less latency) than iSCSI. -Matt > > Nima > > -----Original Message----- > From: Matt Wakeley [mailto:matt_wakeley@agilent.com] > Sent: Tuesday, May 08, 2001 11:19 AM > To: ips@ece.cmu.edu > Subject: Re: iSCSI: Nailing down CRC-32C > > There is a 4th option. > > Mandate the implementation of iSCSI markers. This will significantly reduce the amount of buffering required. > > I think we should just go ahead and mandate implementation of > markers, for two reasons: 1- the "ULF (or WARP)" proposal doesn't > seem to be making much headway, and 2- even if ULF did become > reality, there are software implementations today that will not have > the luxury of having a "enhanced" TCP stack that would provide the > framing. > > Software implementations of iSCSI MUST supply iSCSI markers (if > invoked to do so) to enable low cost, high performance 10Gig > solutions. > > -Matt > > Nima wrote: > > > > Assumptions: > > > > 1) The iSCSI DATA PDU Header and Data Integrity checks uses CRC-32 > > 2) Support of Integrity check is mandatory for both header and Data > > digest > > 3) An iSCSI PDU may span multiple TCP segments. > > 4) Any of these segments may arrive out of order at the receiver. > > 5) To regenerate the CRC for comparison with the value present in the > > PDU received, the receiver needs to keep in a temporary buffers all > > segments, waiting for the missing one, as CRC must be computed using > > the whole re-assembled PDU > > > > Question: Does CRC-32 support partial computation and compensation > > for wrong initial values later (I guess NO!), to enable partial CRC > > calc on avail parts of PDU? > > > > Possible answer: No > > > > Possible Conclusion: The RCV side is further complicated b/c of this > > CRC > > > > Possible solutions: > > > > 1) Use CRC that allows partial calc > > 2) Limit iSCSI PDU to TCP MSS if data CRC is used (added o/h)... > > 3) Leave as it, though it has cost , performance implication and hurts > > scaling to 10G.. > > > > Nima > > > > -----Original Message----- > > From: julian_satran@il.ibm.com [mailto:julian_satran@il.ibm.com] > > Sent: Saturday, May 05, 2001 9:21 AM > > To: ips@ece.cmu.edu > > Subject: Re: iSCSI: Nailing down CRC-32C > > > > The CRC part of the appendix (for 07) reads: > > > > The following table lists cyclic integrity checksums that can be > > negotiated for the digests and MUST be implemented by every iSCSI > > initiator and target. Note that these digest options have only error > > detection significance. > > > > +---------------------------------------------+ > > | Name | Description | > > +---------------------------------------------+ > > | crc-32C | 32 bit CRC | 11EDC6F41 | > > +---------------------------------------------+ > > | none | no digest | > > +---------------------------------------------+ > > > > The generator polynomials for those digests are given in hex-notation, > > for example 3a stands for 0011 1010 - the polynomial > x**5+X**4+x**3+x+1. > > > > The generator polynomial selected is evaluated in [Castagnioli93]. > > When using the CRC the CRC register must be initialized to all 1s > > (0xFFFFFFFF) and the CRC bits must be complemented before transmission. > > Padding bytes, when presents in a segment covered by a CRC, have to be > > set to 0 and are included in the CRC. > > > > Regards, > > Julo > > > > Mark Bakke on 05-05-2001 00:13:09 > > > > Please respond to Mark Bakke > > > > To: IPS > > cc: > > Subject: iSCSI: Nailing down CRC-32C > > > > At the interim meeting, it was stated that iSCSI has selected > > the CRC-32C polynomial as its required iSCSI-level header and > > data CRC. Now that we have it, it's time to move on and make > > sure we can implement it. > > > > In the interest of interoperability, we need to not only specify > > the polynomial, but also the initial values, bit and byte > > ordering, etc. > > > > "A Painless Guide to CRC Error Detection Algorithms" > > (http://www.ross.net/crc/crcpaper.html) specifies a > > method to unambiguously characterize these parameters > > (sections 15 and 16). Has anyone taken a shot at defining > > these yet? Otherwise, here is what it might look like: > > > > Name : "CRC-32C" > > Width : 32 > > Poly : 1EDC6F41 (note that the leading "1" is implied) > > Init : FFFFFFFF > > RefIn : True > > RefOut : True > > XorOut : FFFFFFFF > > Check : ? > > > > I haven't attempted to create check data based on these yet. As > > soon as the other parameters are nailed down, we need to do this. > > > > Anyway, I am not a CRC expert, and can't make any statement about > > the above values being the "best" way to do this, but instead just > > copied them (except the polynomial itself) from the Ethernet CRC, > > since that is likely the easiest for everyone implementing hardware > > to deal with. > > > > If someone else is already doing this, let me know; I just wanted > > to start this thread so we can get closure. > > > > Regards, > > > > Mark > > > > -- > > Mark A. Bakke > > Cisco Systems > > mbakke@cisco.com > > 763.398.1054 > > > > _______________________________________________________ > > Send a cool gift with your E-Card > > http://www.bluemountain.com/giftcenter/ > > Get 250 color business cards for FREE! > http://businesscards.lycos.com/vp/fastpath/ From owner-ips@ece.cmu.edu Wed May 9 23:18:19 2001 Received: from ece.cmu.edu ([128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id XAA03679 for ; Wed, 9 May 2001 23:18:19 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f4A14BS10543 for ips-outgoing; Wed, 9 May 2001 21:04:11 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from e31.bld.us.ibm.com (e31.co.us.ibm.com [32.97.110.129]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f4A149H10538 for ; Wed, 9 May 2001 21:04:09 -0400 (EDT) Received: from westrelay02.boulder.ibm.com (westrelay02.boulder.ibm.com [9.99.140.23]) by e31.bld.us.ibm.com (8.9.3/8.9.3) with ESMTP id UAA33078; Wed, 9 May 2001 20:56:39 -0400 Received: from f4n49e (d03nm065h.boulder.ibm.com [9.99.140.49]) by westrelay02.boulder.ibm.com (8.11.1/NCO v4.96.1.0) with ESMTP id f4A13lc252146; Wed, 9 May 2001 19:04:08 -0600 X-Priority: 1 (High) Importance: Normal Subject: Re: iSCSI: multiple sessions b/n a pair of WWUIs. To: Santosh Rao Cc: ips@ece.cmu.edu X-Mailer: Lotus Notes Release 5.0.3 (Intl) 21 March 2000 Message-ID: From: "John Hufferd" Date: Wed, 9 May 2001 18:03:38 -0700 X-MIMETrack: Serialize by Router on D03NM065/03/M/IBM(Release 5.0.6 |December 14, 2000) at 05/09/2001 07:04:07 PM MIME-Version: 1.0 Content-type: text/plain; charset=us-ascii Sender: owner-ips@ece.cmu.edu Precedence: bulk Santosh, Notes in text below between [Huff] and [/Huff]. . . . John L. Hufferd Senior Technical Staff Member (STSM) IBM/SSG San Jose Ca (408) 256-0403, Tie: 276-0403, eFax: (408) 904-4688 Internet address: hufferd@us.ibm.com Santosh Rao @ece.cmu.edu on 05/09/2001 10:41:34 AM Sent by: owner-ips@ece.cmu.edu To: John Hufferd/San Jose/IBM@IBMUS cc: Black_David@emc.com, ips@ece.cmu.edu Subject: Re: iSCSI: multiple sessions b/n a pair of WWUIs. John Hufferd wrote: > > Santosh, > I think I am beginning to see the problem. A given iSCSI Initiator Port > can NOT have a second session with the same iSCSI Initiator Port and be John, ^^^^^^^^^^^^^^^^^^^^^ Is the above "same iSCSI target port" (?). [Huff] correct, it was a Type-O, or Brain-check.[/Huff] > consistent with SCSIness. The second session could be started with another > ISID, (to the same iSCSI Target Port) but if the session was established it > would not have any of its commands and data handled via any techniques > defined by SCSI or iSCSI. In fact its use would require a wedge driver. Perhaps, some of my confusion arises from a lack of understanding of your definition of iSCSI Initiator Port and iSCSI Target Port. Some clarification on your definition of these would help make things more clear. [Huff] In our Naming and Discovery Document the thing I am calling iSCSI Initiator Port was shown as the SCSI Initiator Port. It is the thing that creates the sessions that talk to the Target Devices, the Target Device must logically receive the connection and assign it to what I was calling an iSCSI Target Port, which in the N&D document was shown as the SCSI Port. It is the (i)SCSI Initiator Port that creates Sessions and in our iSCSI model it does this through the LOGIN command sending the iSCSI Initiator Node Name and an approprate ISID. The Target side's SCSI port, in our iSCSI model exchanges its iSCSI Target Node Name and an approprate TSID. (And Jim has been talking about how the (i)SCSI Target Port should keep persistent reservation information, including the ISID & TSIDs, and use that information to re-establish the Session and assign to the Session the approprate Reservations that might be out standing for the approprate (i)SCSI Initiator Port -- This port is identified by the iSCSI Initiator Node Name and the ISID, under which the Reservation was taken.)[/Huff] > This is the exact confusion I was trying to avoid. Technically if you were > actually able to start another Session to the same Target Port, it would > be, by definition another iSCSI Initiator Port. The two different Ports > would need to be coordinated by what we have been calling a Wedge Driver. I'm not sure why you believe a wedge driver would be required to handle such a configuration.(?) Per my understanding [which may be missing some aspect], the host can operate in such a configuration in the absence of a wedge driver. Also, the host could create such multiple sessions to the same target port in order to separate sub-sets of LUNs into different sessions and apply different session properties to each such LUN subset. [Huff] As for Wedge Drivers, this is a NON SCSI process that is either Dynamic and therefore in the path to the Port, or is Static and performs setup before the I/O is done (such as isolation of one set of LUNs from others even though they are on the same Target). But some one has to do something to prevent the LUN I/O from being spread across different sessions, such that order can not be assured. The best performing Wedge Drivers are those that can send the I/O down any Session, but will insure that it will not send any LUN I/O down another Session as long as the LUN has I/O out standing. However, if no I./O is outstanding to the LUN it can balance the I/O across the different session. So in my terms, your wedge driver either separate the LUN I/O ahead of time statically, or it does it dynamically (or both). The point to remember is: if Multiple Connections per Sessions (SC/S) can be used, there is no Wedge Driver requirement and the code is already in place to balance I/O across connections. Since SC/S is documented by the protocol, we should probably give preference in the document to those design approaches which use the protocol to its best advantage. [/Huff] > > Today, the way we use multiple Fibre Channel Initiator Ports connected to > the same Fibre Channel Target Ports is by use of Wedge Drivers. With FC, connecting multiple initiator ports from the *same* host to a single target port makes little or no sense. The host would see the same LUNs twice through different initiator ports. [Huff] In fact this is a frequent type of connection used by large servers, and this is where the Wedge drivers come into action [/Huff] A more typical configuration is to connect multiple FC initiator ports from *different* hosts to the same target port as a high availability configuration. Yes, in such a case, some type of cluster application would be managing the switch-over from one initiator port to another. (Is this the case being referred to ?). [Huff] This is not what I was talking about, this is the typical type of connections with Shared File Systems. and that does not need a Wedge Driver, since the Shared File System (or Database) protects its self in such a cluster environment. Also no application can assume any inter-arrival of I/O with respect to the other system unless it does its own coordination. This is NOT a Wedge Driver. [/Huff] > I think > what you are suggesting causes the need for a Wedge Drivers to integrate > the iSCSI Initiator Ports. (I guess we wandered off into a different line of discussion which is "whether multiple sessions between an initiator port and target port are legal and required" ? :) My answer to that would be, yes, they are legal and required. They also do not require the use of a wedge driver. The host may configure multiple sessions from one iSCSI initiator NIC to a given iSCSI TargetAddress so as to access LUN subsets from different sessions, and apply different session properties to different LUN sub-sets. [Huff] The point is their sematicts are NOT defined by SCSI, you need what I was calling a Wedge Driver either dynamic or Staticly, and I think you are saying the same thing. Yes, it can be done, but the deffinition of what happens is NOT Standarized. [/Huff] Such multiple sessions that are originated on the same initiator NIC need to use different ISIDs [for all of the sessions for that given target node] and thereby, are considered multiple iSCSI Initiator Ports, based on the current name-disc draft's defn. [Huff] Correct, we are back on the same page now. [/Huff] Not sure that we want to cause this type of > thinking, by accident. One of the reasons for Multiple Connections per > Session was to remove the "Hard" need for Wedge Drivers. IMHO, "multiple connections per session" complements wedge drivers in their functionality but may not comletely replace them. A lot of wedge drivers have code specific to the SCSI properties of the target and moving all of this functionality into the iSCSI layer would imply adding device specific code into this layer. Not the cleanest of layering schemes. [Huff] Correct again, there are some dynamic wedge drivers that not only balance workload but also spring into action to help with fail over situations. Also the wedge drivers sometimes are needed to ensure that for the best use of the target Raid Array Caches, by assuring that the various LUN traffic is isolated/fixed to one connection or another,until fail over at the target. These types of storage controllers will probably continue to need Wedge Drivers, but that makes it very hard to come up with generic Desktop and Laptop Initiators that can efficiently use those types of RAID Arrays. IMHO it is unlikely that Microsoft will ship various vendor specific Wedge Drivers, so it will be up to the RAID manufacture, and that is always problematical in large installations. [/Huff] Also, some wedge drivers claim to utilize device proprietary features, which a device independent iSCSI initiator layer could not provide through "multiple connections per session". [Huff] And again Correct, as I agreed above. However, it would be very good if the vendors that are making the various generic iSCSI initiators, think through how they will handle the Single Connection case with one of the above mentioned RAID Array Controllers, so that upon failure of one side of a duplexed Raid Storage Controller, the Session can be restarted from the initiator to the surviving side of the Raid Controller (which may have a different TCP/IP address, or have taken over the failing IP address). If a clean new session is started, the target still needs to handle Persistent Reserves, and if the surviving Storage controller board actually takes over the IP address, then I am not sure what we should be doing since the surviving side may or may not have approprate state to continue the session as if nothing had happen. (I would be interested in opinions here.) In any event, as I suspected, we are in basic agreement, but were talking past each other. So the question is should David's suggestion about including, in the Protocol Document, the concept of multiple ISIDs per (i)SCSI Port. My opinion is not, since it would probably cause a whole lot of discussion about Wedge Drivers, etc., and I do not think that would be approprate in the iSCSI spec. [/Huff] Regards, Santosh From owner-ips@ece.cmu.edu Thu May 10 06:57:43 2001 Received: from ece.cmu.edu ([128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id GAA22448 for ; Thu, 10 May 2001 06:57:40 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f4A7uAn07506 for ips-outgoing; Thu, 10 May 2001 03:56:10 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from e31.bld.us.ibm.com (e31.co.us.ibm.com [32.97.110.129]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f4A7u9H07501 for ; Thu, 10 May 2001 03:56:09 -0400 (EDT) Received: from westrelay02.boulder.ibm.com (westrelay02.boulder.ibm.com [9.99.140.23]) by e31.bld.us.ibm.com (8.9.3/8.9.3) with ESMTP id DAA62502; Thu, 10 May 2001 03:48:37 -0400 Received: from f4n49e (d03nm065h.boulder.ibm.com [9.99.140.49]) by westrelay02.boulder.ibm.com (8.11.1/NCO v4.96.1.0) with ESMTP id f4A7u7c76320; Thu, 10 May 2001 01:56:07 -0600 X-Priority: 1 (High) Importance: Normal Subject: Re: iSCSI: multiple sessions b/n a pair of WWUIs. To: Pierre Labat Cc: ips@ece.cmu.edu X-Mailer: Lotus Notes Release 5.0.3 (Intl) 21 March 2000 Message-ID: From: "John Hufferd" Date: Thu, 10 May 2001 00:55:49 -0700 X-MIMETrack: Serialize by Router on D03NM065/03/M/IBM(Release 5.0.6 |December 14, 2000) at 05/10/2001 01:56:07 AM MIME-Version: 1.0 Content-type: text/plain; charset=us-ascii Sender: owner-ips@ece.cmu.edu Precedence: bulk Pierre, You are correct when you create a second session from the same (i)SCSI Initiator Device (iSCSI Initiator Node) using a different ISID, it becomes another (i)SCSI Initiator Delivery PORT (I took some freedom and preceded the SCSI PORT with an "i". to denote that it is used with iSCSI, no significant in the name it is still a SCSI Delivery Port (SDP), and in this case a SCSI Initiator Delivery Port or SIDP or (i)SIDP or ....... There should have been nothing in my note that tied the (i)SCSI Initiator Delivery Port to any HW. (It can be, but that was not what I was addressing.) As soon as the second session is started, to the same Target Device (but with a different ISID), you now have new (i)SCSI Initiator Delivery Port and an issue of determining how you handle the delivery of commands, in what kind of order, etc. since the same LUs can be address across the different sessions. This is where the complications of Wedge Drivers (active and passive) come into effect. The value of Multiple Connections per Session, is that they have built-in load balancing and order delivery. This is usually easier to deal with then Multiple SCSI Delivery Ports and Wedge Drivers. If you reread my note, you should find that we are saying the same thing (now that you understand my free use of the "i" ). . . . John L. Hufferd Senior Technical Staff Member (STSM) IBM/SSG San Jose Ca (408) 256-0403, Tie: 276-0403, eFax: (408) 904-4688 Internet address: hufferd@us.ibm.com Pierre Labat @ece.cmu.edu on 05/09/2001 09:52:50 AM Sent by: owner-ips@ece.cmu.edu To: ips@ece.cmu.edu cc: Subject: Re: iSCSI: multiple sessions b/n a pair of WWUIs. John Hufferd wrote: > Santosh, > I think I am beginning to see the problem. A given iSCSI Initiator Port > can NOT have a second session with the same iSCSI Initiator Port and be > consistent with SCSIness. The second session could be started with another > ISID, (to the same iSCSI Target Port) but if the session was established it > would not have any of its commands and data handled via any techniques > defined by SCSI or iSCSI. In fact its use would require a wedge driver. > This is the exact confusion I was trying to avoid. Technically if you were > actually able to start another Session to the same Target Port, it would > be, by definition another iSCSI Initiator Port. The two different Ports > would need to be coordinated by what we have been calling a Wedge Driver. > > Today, the way we use multiple Fibre Channel Initiator Ports connected to > the same Fibre Channel Target Ports is by use of Wedge Drivers. I think > what you are suggesting causes the need for a Wedge Drivers to integrate > the iSCSI Initiator Ports. Not sure that we want to cause this type of > thinking, by accident. One of the reasons for Multiple Connections per > Session was to remove the "Hard" need for Wedge Drivers. John, Could you give your exact definition of a iSCSI port. Till your mails they were the definition of - a "portal" (IP add), - a SCSI service delivery port (=session end point from what i hearded in Nashua) It seems that you want to tie the notion of SCSI service delivery port to hardware, the opposite of what was presented in Nashua. Do i miss something? I see nothing wrong against SCSI or iSCSI to have two sessions (with 2 # ISID) flowing from the same initiator adapter to the same target adapter. Each session end point represents a SDP. Regards, Pierre > > > OK, that's what I think our misunderstanding is all about. If I am > mistaken, please set me straight. > . > . > . > John L. Hufferd > Senior Technical Staff Member (STSM) > IBM/SSG San Jose Ca > (408) 256-0403, Tie: 276-0403, eFax: (408) 904-4688 > Internet address: hufferd@us.ibm.com > > Santosh Rao @cup.hp.com on 05/08/2001 05:44:41 PM > > Sent by: santoshr@cup.hp.com > > To: John Hufferd/San Jose/IBM@IBMUS > cc: Black_David@emc.com, ips@ece.cmu.edu > Subject: Re: iSCSI: multiple sessions b/n a pair of WWUIs. > > John, > > By the name-disc definition, an iSCSI initiator or target port is a > logical entity that constitutes the end points of a session. > > Hence, there never can be 2 sessions b/n the *same* 2 initiator & > target port. Each session established spawns a *new* initiator and > target iSCSI port. > > Going by the above semantics, there is always only 1 ISID/TSID per iSCSI > Initiator/Target port. > > Hence, I'm unable to understand your statements below. > > As for the uniqueness of ISID across all initiators, is there any reason > not to allow implementations to do that ? I would have thought that is > the most typical use of an ISID and also allows initiators to lookup > their target structures based on ISID. > > Regards, > Santosh > > John Hufferd wrote: > > > > Also, since a second session by the same iSCSI Initiator Port, to the > same > > iSCSI Target Port is not, in my opinion "legal", it is not clear to me > that > > any given iSCSI Port needs any more then one ISID. I > > > > Therefore, I suggest that we Not put any such notes, like you suggest, in > > the draft, but in fact encourage a single ISID/TSID per iSCSI > > Initiator/Target Port. > > > To: Jim Hafner/Almaden/IBM@IBMUS, marjorie_krueger@hp.com > > cc: ips@ece.cmu.edu > > Subject: RE: iSCSI: multiple sessions b/n a pair of WWUIs. > > > > While Jim's correct ... in Marj's defense, what > > she wrote is a fairly obvious way to implement > > it, and that might be worth noting in the draft. > > > > --David From owner-ips@ece.cmu.edu Thu May 10 12:44:25 2001 Received: from ece.cmu.edu ([128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id MAA02841 for ; Thu, 10 May 2001 12:44:21 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f4AEL5b12541 for ips-outgoing; Thu, 10 May 2001 10:21:05 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from d12lmsgate-3.de.ibm.com (d12lmsgate-3.de.ibm.com [195.212.91.201]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f4AEKwH12530 for ; Thu, 10 May 2001 10:20:58 -0400 (EDT) Received: from d12relay02.de.ibm.com (d12relay02.de.ibm.com [9.165.215.23]) by d12lmsgate-3.de.ibm.com (1.0.0) with ESMTP id QAA159680 for ; Thu, 10 May 2001 16:20:36 +0200 From: julian_satran@il.ibm.com Received: from d12mta02.de.ibm.com (d12mta01_cs0 [9.165.222.237]) by d12relay02.de.ibm.com (8.8.8m3/NCO v4.96) with SMTP id QAA123520 for ; Thu, 10 May 2001 16:20:35 +0200 Received: by d12mta02.de.ibm.com(Lotus SMTP MTA v4.6.5 (863.2 5-20-1999)) id C1256A48.004EC5AD ; Thu, 10 May 2001 16:20:24 +0200 X-Lotus-FromDomain: IBMIL@IBMDE To: ips@ece.cmu.edu Message-ID: Date: Thu, 10 May 2001 17:25:22 +0300 Subject: Re: iSCSI: immediate data Mime-Version: 1.0 Content-type: text/plain; charset=us-ascii Content-Disposition: inline Sender: owner-ips@ece.cmu.edu Precedence: bulk Matt, F is the end of the sequence. If you don't have immediate data but have some unsolicited data PDU the keep F at 0 until the last unsolicited PDU (presence or absence of immediate is indicated by length). If you have immediate but no other unsolicited or no immediate and no other unsolicited F will be 1. Julo "Matt Wakeley" on 09-05-2001 19:29:05 Please respond to Matt Wakeley To: ips@ece.cmu.edu cc: Subject: Re: iSCSI: immediate data The way I interpret the question is, what if you negotiated immediate data, but have no intention on sending immediate data for this command? Is setting the F bit, without sending any immediate data valid? (once again, more options....) -Matt julian_satran@il.ibm.com wrote: > > That is correct. The final bit indicates the end of unsolicited data. > If all you have is immediate then the final bit is 1in the command. > If you have immediate and other unsolicited (if enabled!) then set F to 0. > > 2.3.1 has been fixed accordingly. > > Regards, > Julo > > sandeepj@research.bell-labs.com (Sandeep Joshi) on 09-05-2001 01:56:25 > > Please respond to sandeepj@research.bell-labs.com (Sandeep Joshi) > > To: matt_wakeley@agilent.com > cc: ips@ece.cmu.edu > Subject: Re: iSCSI: immediate data > > Umm.....Julian replied to this one and said the final bit on ScsiCmd > would indicate it. Apparently, a single command can now send > the unsolicited data both with the command and in separate PDUs. > > I cant seem to find the email but I believe Sec 2.3.1 was changed > accordingly. Julian..? > > -Sandeep > > > The intention is that if an initiator requests to send immediate data > (and is > > granted the request), then it will always send immediate data. > > > > It sounds like you are asking for wishy washy mode... sometimes send > immediate > > data, sometimes not. > > > > -Matt > > > > Sandeep Joshi wrote: > > > > > > Julian, > > > > > > I had a follow-up question on an old thread. > > > http://ips.pdl.cs.cmu.edu/mail/msg03373.html > > > > > > The initiator is allowed to send firstburst in immediate > > > or in separate PDUs...but can it do neither ? I dont > > > see any statement to the effect that it MUST send a > > > firstburst. > > > > > > Here's a possible problem : > > > 1) FirstBurst is 4K > > > 2) Expected data length of command is 16K > > > 3) Initiator sends the Scsi command without immediate > > > data. (final flag on Scsi command is not set) > > > And it decides to send no further data PDUs. > > > > > > The target does not know if R2Ts can be sent since it does > > > not know if any unsolicited PDUs are expected. > > > > > > Perhaps the semantics on the final flag on SCSI command > > > need to be changed to indicate that no unsolicited data > > > will be sent with this command. Currently, the flag implies > > > (Sec 2.3.1) that all the required data has been sent with > > > the command. > > > > > > And here's a typo to fix. Appendix E (Key=immediateData) > > > The following should say "initialR2T is no". > > > The table has got it right. > > > > > > > If ImmediateData is set to yes and InitialR2T is set > > > > to yes then the initiator MAY send unsolicited immediate > > > > data or one unsolicited burst of Data-OUT PDUs but > > > > MUST NOT send both immediate and a unsolicited burst of > > > > Data-OUT PDUs for any one command. > > > > > > -Sandeep From owner-ips@ece.cmu.edu Thu May 10 13:22:57 2001 Received: from ece.cmu.edu ([128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id NAA04214 for ; Thu, 10 May 2001 13:22:57 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f4AFIar17470 for ips-outgoing; Thu, 10 May 2001 11:18:36 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from crufty.research.bell-labs.com (crufty.research.bell-labs.com [204.178.16.49]) by ece.cmu.edu (8.11.0/8.10.2) with SMTP id f4AFIYH17466 for ; Thu, 10 May 2001 11:18:34 -0400 (EDT) Received: from scummy.research.bell-labs.com ([135.104.2.10]) by crufty; Thu May 10 11:15:45 EDT 2001 Received: from aura.research.bell-labs.com ([135.104.46.10]) by scummy; Thu May 10 11:18:26 EDT 2001 Received: from research.bell-labs.com (IDENT:sandeepj@sandeepj-pcmh.research.bell-labs.com [135.104.47.90]) by aura.research.bell-labs.com (8.9.1/8.9.1) with ESMTP id LAA14633 for ; Thu, 10 May 2001 11:18:13 -0400 (EDT) Message-ID: <3AFAB135.F88A1D55@research.bell-labs.com> Date: Thu, 10 May 2001 11:18:13 -0400 From: Sandeep Joshi X-Mailer: Mozilla 4.76 [en] (X11; U; Linux 2.2.16-3 i686) X-Accept-Language: en MIME-Version: 1.0 To: ips@ece.cmu.edu Subject: Re: iSCSI: firstburst and maxburst Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-ips@ece.cmu.edu Precedence: bulk Content-Transfer-Encoding: 7bit Julian, I wasnt referring to "0" in the text key value range but rather in the mode pages. I guess this question might arise in applications which interpret mode pages at the initiator or at gateways, and then convert them to-and-from TextCmds. Is it reasonable to assume that a zero in a mode page implies the corresponding maximum length allowed by iSCSI? -Sandeep > Sandeep, > > The current draft considers 0 an illegal value. I wonder if making 0 a > no-limit indication > is really needed. > > Thanks, > Julo > > > sandeepj@research.bell-labs.com (Sandeep Joshi) on 07-05-2001 20:34:19 > > > > Please respond to sandeepj@research.bell-labs.com (Sandeep Joshi) > > > > To: ips@ece.cmu.edu > > cc: > > Subject: iSCSI: firstburst and maxburst > > > > > > > > > > Julian, > > > > SPC-2 revision 19: section 8.3.7 Disconnect-reconnect page > > states that a value of "0" indicates no limit on the maximum > > and first burst size. > > > > Current range for the corresponding text keys from Appendix E > > are (1 to 2^15-1)*512 bytes. > > > > How should iSCSI interpret a zero ? > > > > regards, > > -Sandeep > > > > From owner-ips@ece.cmu.edu Thu May 10 14:15:16 2001 Received: from ece.cmu.edu ([128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id OAA05468 for ; Thu, 10 May 2001 14:15:16 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f4AGHwD22487 for ips-outgoing; Thu, 10 May 2001 12:17:59 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from palrel1.hp.com (palrel1.hp.com [156.153.255.242]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f4AGHvH22481 for ; Thu, 10 May 2001 12:17:57 -0400 (EDT) Received: from colosus2.cup.hp.com (colosus2.cup.hp.com [15.13.128.145]) by palrel1.hp.com (Postfix) with ESMTP id BA575146C for ; Thu, 10 May 2001 09:17:57 -0700 (PDT) Received: from hp.com (IDENT:plabat@pl703521.cup.hp.com [15.13.133.216]) by colosus2.cup.hp.com (8.9.3 (PHNE_18979)/8.9.3 SMKit7.02) with ESMTP id JAA22327; Thu, 10 May 2001 09:17:55 -0700 (PDT) Message-ID: <3AFABFF3.5034FD8B@hp.com> Date: Thu, 10 May 2001 09:21:07 -0700 From: Pierre Labat Organization: Hewlett Packard ATM-SISL X-Mailer: Mozilla 4.51 [en] (X11; I; Linux 2.2.5-15 i686) X-Accept-Language: en MIME-Version: 1.0 To: ips@ece.cmu.edu Subject: Re: iSCSI: multiple sessions b/n a pair of WWUIs. References: Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-ips@ece.cmu.edu Precedence: bulk Content-Transfer-Encoding: 7bit John, John Hufferd wrote: > Pierre, > You are correct when you create a second session from the same (i)SCSI > Initiator Device (iSCSI Initiator Node) using a different ISID, it becomes > another (i)SCSI Initiator Delivery PORT (I took some freedom and preceded > the SCSI PORT with an "i". to denote that it is used with iSCSI, no > significant in the name it is still a SCSI Delivery Port (SDP), and in this > case a SCSI Initiator Delivery Port or SIDP or (i)SIDP or ....... > > There should have been nothing in my note that tied the (i)SCSI Initiator > Delivery Port to any HW. (It can be, but that was not what I was > addressing.) OK > > > As soon as the second session is started, to the same Target Device (but > with a different ISID), you now have new (i)SCSI Initiator Delivery Port > and an issue of determining how you handle the delivery of commands, in > what kind of order, etc. since the same LUs can be address across the > different sessions. This is where the complications of Wedge Drivers > (active and passive) come into effect. It can be too, a configuration where each of the 2 (for ex) sessions access a separate subset of luns of the target. In this case the ordering is not violated and some implementations/applications (can be something else than wedge driver) could make benefit from that. It's a legal configuration that the spec must allow. Regards, Pierre > > > The value of Multiple Connections per Session, is that they have built-in > load balancing and order delivery. This is usually easier to deal with > then Multiple SCSI Delivery Ports and Wedge Drivers. > > If you reread my note, you should find that we are saying the same thing > (now that you understand my free use of the "i" ). > > . > . > . > John L. Hufferd > Senior Technical Staff Member (STSM) > IBM/SSG San Jose Ca > (408) 256-0403, Tie: 276-0403, eFax: (408) 904-4688 > Internet address: hufferd@us.ibm.com > > Pierre Labat @ece.cmu.edu on 05/09/2001 09:52:50 AM > > Sent by: owner-ips@ece.cmu.edu > > To: ips@ece.cmu.edu > cc: > Subject: Re: iSCSI: multiple sessions b/n a pair of WWUIs. > > John Hufferd wrote: > > > Santosh, > > I think I am beginning to see the problem. A given iSCSI Initiator Port > > can NOT have a second session with the same iSCSI Initiator Port and be > > consistent with SCSIness. The second session could be started with > another > > ISID, (to the same iSCSI Target Port) but if the session was established > it > > would not have any of its commands and data handled via any techniques > > defined by SCSI or iSCSI. In fact its use would require a wedge driver. > > This is the exact confusion I was trying to avoid. Technically if you > were > > actually able to start another Session to the same Target Port, it would > > be, by definition another iSCSI Initiator Port. The two different Ports > > would need to be coordinated by what we have been calling a Wedge Driver. > > > > Today, the way we use multiple Fibre Channel Initiator Ports connected to > > the same Fibre Channel Target Ports is by use of Wedge Drivers. I think > > what you are suggesting causes the need for a Wedge Drivers to integrate > > the iSCSI Initiator Ports. Not sure that we want to cause this type of > > thinking, by accident. One of the reasons for Multiple Connections per > > Session was to remove the "Hard" need for Wedge Drivers. > > John, > Could you give your exact definition of a iSCSI port. > Till your mails they were the definition of > - a "portal" (IP add), > - a SCSI service delivery port (=session end point from what i hearded in > Nashua) > > It seems that you want to tie the notion of SCSI service delivery port > to hardware, the opposite of what was presented in Nashua. > > Do i miss something? > > I see nothing wrong against SCSI or iSCSI to have two sessions (with 2 # > ISID) flowing from the same initiator adapter to the same target adapter. > Each session end point represents a SDP. > > Regards, > > Pierre > > > > > > > OK, that's what I think our misunderstanding is all about. If I am > > mistaken, please set me straight. > > . > > . > > . > > John L. Hufferd > > Senior Technical Staff Member (STSM) > > IBM/SSG San Jose Ca > > (408) 256-0403, Tie: 276-0403, eFax: (408) 904-4688 > > Internet address: hufferd@us.ibm.com > > > > Santosh Rao @cup.hp.com on 05/08/2001 05:44:41 PM > > > > Sent by: santoshr@cup.hp.com > > > > To: John Hufferd/San Jose/IBM@IBMUS > > cc: Black_David@emc.com, ips@ece.cmu.edu > > Subject: Re: iSCSI: multiple sessions b/n a pair of WWUIs. > > > > John, > > > > By the name-disc definition, an iSCSI initiator or target port is a > > logical entity that constitutes the end points of a session. > > > > Hence, there never can be 2 sessions b/n the *same* 2 initiator & > > target port. Each session established spawns a *new* initiator and > > target iSCSI port. > > > > Going by the above semantics, there is always only 1 ISID/TSID per iSCSI > > Initiator/Target port. > > > > Hence, I'm unable to understand your statements below. > > > > As for the uniqueness of ISID across all initiators, is there any reason > > not to allow implementations to do that ? I would have thought that is > > the most typical use of an ISID and also allows initiators to lookup > > their target structures based on ISID. > > > > Regards, > > Santosh > > > > John Hufferd wrote: > > > > > > Also, since a second session by the same iSCSI Initiator Port, to the > > same > > > iSCSI Target Port is not, in my opinion "legal", it is not clear to me > > that > > > any given iSCSI Port needs any more then one ISID. I > > > > > > Therefore, I suggest that we Not put any such notes, like you suggest, > in > > > the draft, but in fact encourage a single ISID/TSID per iSCSI > > > Initiator/Target Port. > > > > > To: Jim Hafner/Almaden/IBM@IBMUS, marjorie_krueger@hp.com > > > cc: ips@ece.cmu.edu > > > Subject: RE: iSCSI: multiple sessions b/n a pair of WWUIs. > > > > > > While Jim's correct ... in Marj's defense, what > > > she wrote is a fairly obvious way to implement > > > it, and that might be worth noting in the draft. > > > > > > --David From owner-ips@ece.cmu.edu Thu May 10 19:26:31 2001 Received: from ece.cmu.edu ([128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id TAA09990 for ; Thu, 10 May 2001 19:26:31 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f4AL5HC17685 for ips-outgoing; Thu, 10 May 2001 17:05:17 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from gateway.sanlight.org (adsl-63-202-160-80.dsl.snfc21.pacbell.net [63.202.160.80]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f4AL5EH17678 for ; Thu, 10 May 2001 17:05:14 -0400 (EDT) Received: from ljoy (10.0.0.18.lan.sanlight.net [10.0.0.18]) by gateway.sanlight.org (8.11.0/8.11.0) with SMTP id f4AMCA158936 for ; Thu, 10 May 2001 15:12:16 -0700 (PDT) (envelope-from dotis@sanlight.net) From: "Douglas Otis" To: "Ips" Subject: iSCSI (SCTP compatable protocol version) Date: Thu, 10 May 2001 14:02:20 -0700 Message-ID: MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 (Normal) X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook IMO, Build 9.0.2416 (9.0.2911.0) Importance: Normal X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4522.1200 Sender: owner-ips@ece.cmu.edu Precedence: bulk Content-Transfer-Encoding: 7bit All, I have just created a draft that will define an additional Chunk for SCTP. This chunk was placed at the end of the packet with the concept that if the packet is truncated, a missing IA-Digest packet would act as a truncation indicator. Although not indicated is the possible use of this to include CRC. ftp://ftp.sanlight.net/pub/draft-otis-sctp-digest-00.txt Doug From owner-ips@ece.cmu.edu Fri May 11 12:03:15 2001 Received: from ece.cmu.edu ([128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id MAA14117 for ; Fri, 11 May 2001 12:03:15 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f4BDtuH03325 for ips-outgoing; Fri, 11 May 2001 09:55:56 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from bramg1.net.external.hp.com (bramg1.net.external.hp.com [192.6.126.73]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f4BDtrH03320 for ; Fri, 11 May 2001 09:55:53 -0400 (EDT) Received: from quasit.br.itc.hp.com (quasit.br.itc.hp.com [15.145.8.135]) by bramg1.net.external.hp.com (Postfix) with ESMTP id EABF421F for ; Fri, 11 May 2001 15:55:49 +0200 (METDST) Received: from fowey.BR.ITC.HP.COM (fowey.br.itc.hp.com [15.145.8.186]) by quasit.br.itc.hp.com (8.9.3 (PHNE_18979)/8.9.3 SMKit6.0.6 OpenMail) with SMTP id OAA04402 for ; Fri, 11 May 2001 14:55:48 +0100 (BST) Received: from 15.145.8.186 by fowey.BR.ITC.HP.COM (InterScan E-Mail VirusWall NT); Fri, 11 May 2001 14:55:47 +0100 (GMT Daylight Time) Received: by fowey.br.itc.hp.com with Internet Mail Service (5.5.2653.19) id ; Fri, 11 May 2001 14:55:47 +0100 Message-ID: <0B9A57FF1D57D411B47500D0B73E5CC101E7A6D9@dickens.bri.hp.com> From: "BURBRIDGE,MATTHEW (HP-UnitedKingdom,ex2)" To: "'ips@ece.cmu.edu'" Subject: RE: iSCSI: immediate data Date: Fri, 11 May 2001 14:55:47 +0100 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2653.19) Content-Type: text/plain; charset="iso-8859-1" Sender: owner-ips@ece.cmu.edu Precedence: bulk Hi Julian, Rev 06 specifies that immediate and unsolicited data can not be mixed. E-22 page 128 (is there a typo - see below). But if using immediate data, providing that the F bit is zero, can you confirm that the target is able to issue an R2T for the remaining data. i.e. immediate and solicited data is supported. The typo (pg 128): If ImmediateData is set to yes and InitialR2T is set to NO (typo: was yes) then the initiator MAY send unsolicited immediate data or one unsolicited burst of Data-OUT PDUs but MUST NOT send both immediate and a unsolicited burst of Data-OUT PDUs for any one command. Cheers Matthew Burbridge NIS-Bristol Hewlett Packard Telnet: 312 7010 E-mail: matthewb@bri.hp.com > -----Original Message----- > From: julian_satran@il.ibm.com [mailto:julian_satran@il.ibm.com] > Sent: 10 May 2001 15:25 > To: ips@ece.cmu.edu > Subject: Re: iSCSI: immediate data > > > > > Matt, > > F is the end of the sequence. If you don't have immediate > data but have > some unsolicited data PDU > the keep F at 0 until the last unsolicited PDU (presence or absence of > immediate is indicated by length). > > If you have immediate but no other unsolicited or no > immediate and no other > unsolicited F will be 1. > > Julo > > "Matt Wakeley" on 09-05-2001 19:29:05 > > Please respond to Matt Wakeley > > To: ips@ece.cmu.edu > cc: > Subject: Re: iSCSI: immediate data > > > > > The way I interpret the question is, what if you negotiated > immediate data, > but have no intention on sending immediate data for this command? Is > setting > the F bit, without sending any immediate data valid? (once again, more > options....) > > -Matt > > julian_satran@il.ibm.com wrote: > > > > That is correct. The final bit indicates the end of > unsolicited data. > > If all you have is immediate then the final bit is 1in the command. > > If you have immediate and other unsolicited (if enabled!) > then set F to > 0. > > > > 2.3.1 has been fixed accordingly. > > > > Regards, > > Julo > > > > sandeepj@research.bell-labs.com (Sandeep Joshi) on > 09-05-2001 01:56:25 > > > > Please respond to sandeepj@research.bell-labs.com (Sandeep Joshi) > > > > To: matt_wakeley@agilent.com > > cc: ips@ece.cmu.edu > > Subject: Re: iSCSI: immediate data > > > > Umm.....Julian replied to this one and said the final bit on ScsiCmd > > would indicate it. Apparently, a single command can now send > > the unsolicited data both with the command and in separate PDUs. > > > > I cant seem to find the email but I believe Sec 2.3.1 was changed > > accordingly. Julian..? > > > > -Sandeep > > > > > The intention is that if an initiator requests to send > immediate data > > (and is > > > granted the request), then it will always send immediate data. > > > > > > It sounds like you are asking for wishy washy mode... > sometimes send > > immediate > > > data, sometimes not. > > > > > > -Matt > > > > > > Sandeep Joshi wrote: > > > > > > > > Julian, > > > > > > > > I had a follow-up question on an old thread. > > > > http://ips.pdl.cs.cmu.edu/mail/msg03373.html > > > > > > > > The initiator is allowed to send firstburst in immediate > > > > or in separate PDUs...but can it do neither ? I dont > > > > see any statement to the effect that it MUST send a > > > > firstburst. > > > > > > > > Here's a possible problem : > > > > 1) FirstBurst is 4K > > > > 2) Expected data length of command is 16K > > > > 3) Initiator sends the Scsi command without immediate > > > > data. (final flag on Scsi command is not set) > > > > And it decides to send no further data PDUs. > > > > > > > > The target does not know if R2Ts can be sent since it does > > > > not know if any unsolicited PDUs are expected. > > > > > > > > Perhaps the semantics on the final flag on SCSI command > > > > need to be changed to indicate that no unsolicited data > > > > will be sent with this command. Currently, the flag implies > > > > (Sec 2.3.1) that all the required data has been sent with > > > > the command. > > > > > > > > And here's a typo to fix. Appendix E (Key=immediateData) > > > > The following should say "initialR2T is no". > > > > The table has got it right. > > > > > > > > > If ImmediateData is set to yes and InitialR2T is set > > > > > to yes then the initiator MAY send unsolicited immediate > > > > > data or one unsolicited burst of Data-OUT PDUs but > > > > > MUST NOT send both immediate and a unsolicited burst of > > > > > Data-OUT PDUs for any one command. > > > > > > > > -Sandeep > > > From owner-ips@ece.cmu.edu Fri May 11 14:25:34 2001 Received: from ece.cmu.edu ([128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id OAA17714 for ; Fri, 11 May 2001 14:25:34 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f4BGHBp14806 for ips-outgoing; Fri, 11 May 2001 12:17:11 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from d12lmsgate-2.de.ibm.com (d12lmsgate-2.de.ibm.com [195.212.91.200]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f4BGH7H14800 for ; Fri, 11 May 2001 12:17:08 -0400 (EDT) Received: from d12relay01.de.ibm.com (d12relay01.de.ibm.com [9.165.215.22]) by d12lmsgate-2.de.ibm.com (1.0.0) with ESMTP id SAA197058 for ; Fri, 11 May 2001 18:15:28 +0200 From: julian_satran@il.ibm.com Received: from d12mta02.de.ibm.com (d12mta01_cs0 [9.165.222.237]) by d12relay01.de.ibm.com (8.8.8m3/NCO v4.96) with SMTP id SAA100264 for ; Fri, 11 May 2001 18:15:27 +0200 Received: by d12mta02.de.ibm.com(Lotus SMTP MTA v4.6.5 (863.2 5-20-1999)) id C1256A49.00594C1B ; Fri, 11 May 2001 18:15:21 +0200 X-Lotus-FromDomain: IBMIL@IBMDE To: ips@ece.cmu.edu Message-ID: Date: Fri, 11 May 2001 08:39:17 +0300 Subject: Re: iSCSI: firstburst and maxburst Mime-Version: 1.0 Content-type: text/plain; charset=us-ascii Content-Disposition: inline Sender: owner-ips@ece.cmu.edu Precedence: bulk Oops! I guess I'll have to make 0 legal and indicate no limit. Thanks, Julo Sandeep Joshi on 10-05-2001 18:18:13 Please respond to Sandeep Joshi To: ips@ece.cmu.edu cc: Subject: Re: iSCSI: firstburst and maxburst Julian, I wasnt referring to "0" in the text key value range but rather in the mode pages. I guess this question might arise in applications which interpret mode pages at the initiator or at gateways, and then convert them to-and-from TextCmds. Is it reasonable to assume that a zero in a mode page implies the corresponding maximum length allowed by iSCSI? -Sandeep > Sandeep, > > The current draft considers 0 an illegal value. I wonder if making 0 a > no-limit indication > is really needed. > > Thanks, > Julo > > > sandeepj@research.bell-labs.com (Sandeep Joshi) on 07-05-2001 20:34:19 > > > > Please respond to sandeepj@research.bell-labs.com (Sandeep Joshi) > > > > To: ips@ece.cmu.edu > > cc: > > Subject: iSCSI: firstburst and maxburst > > > > > > > > > > Julian, > > > > SPC-2 revision 19: section 8.3.7 Disconnect-reconnect page > > states that a value of "0" indicates no limit on the maximum > > and first burst size. > > > > Current range for the corresponding text keys from Appendix E > > are (1 to 2^15-1)*512 bytes. > > > > How should iSCSI interpret a zero ? > > > > regards, > > -Sandeep > > > > From owner-ips@ece.cmu.edu Fri May 11 18:38:27 2001 Received: from ece.cmu.edu ([128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id SAA22203 for ; Fri, 11 May 2001 18:38:27 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f4BL6wp10500 for ips-outgoing; Fri, 11 May 2001 17:06:58 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from dogwood.cisco.com (dogwood.cisco.com [161.44.11.19]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f4BL6vH10494 for ; Fri, 11 May 2001 17:06:57 -0400 (EDT) Received: from cisco.com (mbakke@mbakke-lnx.cisco.com [161.44.68.87]) by dogwood.cisco.com (8.8.6 (PHNE_14041)/CISCO.SERVER.1.2) with ESMTP id RAA29877 for ; Fri, 11 May 2001 17:06:51 -0400 (EDT) Message-ID: <3AFC53D8.CD8A8BC4@cisco.com> Date: Fri, 11 May 2001 16:04:24 -0500 From: Mark Bakke X-Mailer: Mozilla 4.72 [en] (X11; U; Linux 2.2.16-3.uid32 i686) X-Accept-Language: en, de MIME-Version: 1.0 To: IPS Subject: iSCSI: Canonical Targets Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-ips@ece.cmu.edu Precedence: bulk Content-Transfer-Encoding: 7bit It was pointed out during the interim meeting that the canonical target is defined somewhat ambiguously, and may be a bit too flexible for interoperability purposes. This message is a stake in the ground for defining this a little more tightly. So here are the new canonical target rules. Please let me know if there are major problems with any of them. I tried to define them a-la-carte, to make it easier to pick out any that might cause trouble. 1. Each iSCSI implementation MUST include a canonical target. 2. A canonical target MUST be accessible at the default, IANA- assigned TCP port on each IP address on which the iSCSI implementation is listening for iSCSI connections. 3. A canonical target MUST NOT be used for SCSI commands. A canonical target is an iSCSI construct only, and does not have a corresponding SCSI device. This means it may not be used to access Logical Units. A session created to a canonical target is a discovery session only, and once in full feature phase, is used only for text commands and asynchronous messages. (Do any other commands make sense)? 4. A device containing a single target MUST provide both the canonical target and the real target. (This is implied by the above requirements). An initiator connecting to such a device using only its IP address would first connect to the canonical target, and use SendTargets to obtain the iSCSI name of the real target. It would then create a separate session to the real target. Essentially, this means there's nothing special about a single-target device. 5. An iSCSI device MUST provide a unique iSCSI name for each of its targets. Using the canonical target as a nameless iSCSI target is not supported. 6. We can further specify the order in which the SendTargets response fields are returned, to simplify things further, e.g. each target in the SendTargets response MUST return these fields in this order: - A TargetName= field - A TargetAlias= field (value left blank if there's no alias) - One or more TargetAddress= fields - Any vendor-specific fields (ignored by standard initiators) The above rules will make the behavior of various target implementations identical, regardless of the number of interfaces or targets they support. This will reduce the number of end-cases that initiator implementations will have to handle. If we agree on these, we can edit them into the iSCSI and NDT documents. Additional questions to send input on: 1. Should a non-canonical target respond to a SendTargets command? 2. If so, should it respond only with addresses for its own target, or should it respond with other targets, as a canonical target might do? 3. If not, the initiator must connect to a canonical target to find the other addresses of a target to which it is already connected; is the information it has sufficient to do so? (I think the answer is yes, given canonical requirement #2 above). -- Mark A. Bakke Cisco Systems mbakke@cisco.com 763.398.1054 From owner-ips@ece.cmu.edu Fri May 11 18:38:38 2001 Received: from ece.cmu.edu ([128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id SAA22214 for ; Fri, 11 May 2001 18:38:38 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f4BKaWt08048 for ips-outgoing; Fri, 11 May 2001 16:36:32 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from dogwood.cisco.com (dogwood.cisco.com [161.44.11.19]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f4BKaUH08043 for ; Fri, 11 May 2001 16:36:30 -0400 (EDT) Received: from ahganemw2k (dhcp-161-44-68-139.cisco.com [161.44.68.139]) by dogwood.cisco.com (8.8.6 (PHNE_14041)/CISCO.SERVER.1.2) with SMTP id QAA07937 for ; Fri, 11 May 2001 16:36:24 -0400 (EDT) From: "Ayman Ghanem" To: Subject: RE: iSCSI: immediate data Date: Fri, 11 May 2001 15:35:17 -0500 Message-ID: MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit X-Priority: 3 (Normal) X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook IMO, Build 9.0.2416 (9.0.2910.0) Importance: Normal X-MimeOLE: Produced By Microsoft MimeOLE V5.00.2919.6700 In-Reply-To: Sender: owner-ips@ece.cmu.edu Precedence: bulk Content-Transfer-Encoding: 7bit Julian, Could you please post the revised text for section 2.3.1, and the other revised parts for immediate/unsolicited data (Appendix E-22)?. Thanks. -Ayman > -----Original Message----- > From: owner-ips@ece.cmu.edu [mailto:owner-ips@ece.cmu.edu]On Behalf Of > julian_satran@il.ibm.com > Sent: Wednesday, May 09, 2001 2:12 AM > To: Sandeep Joshi > Cc: matt_wakeley@agilent.com; ips@ece.cmu.edu > Subject: Re: iSCSI: immediate data > > > > > That is correct. The final bit indicates the end of unsolicited data. > If all you have is immediate then the final bit is 1in the command. > If you have immediate and other unsolicited (if enabled!) then set F to 0. > > 2.3.1 has been fixed accordingly. > > Regards, > Julo > > sandeepj@research.bell-labs.com (Sandeep Joshi) on 09-05-2001 01:56:25 > > Please respond to sandeepj@research.bell-labs.com (Sandeep Joshi) > > To: matt_wakeley@agilent.com > cc: ips@ece.cmu.edu > Subject: Re: iSCSI: immediate data > > > > > Umm.....Julian replied to this one and said the final bit on ScsiCmd > would indicate it. Apparently, a single command can now send > the unsolicited data both with the command and in separate PDUs. > > I cant seem to find the email but I believe Sec 2.3.1 was changed > accordingly. Julian..? > > -Sandeep > > > The intention is that if an initiator requests to send immediate data > (and is > > granted the request), then it will always send immediate data. > > > > It sounds like you are asking for wishy washy mode... sometimes send > immediate > > data, sometimes not. > > > > -Matt > > > > Sandeep Joshi wrote: > > > > > > Julian, > > > > > > I had a follow-up question on an old thread. > > > http://ips.pdl.cs.cmu.edu/mail/msg03373.html > > > > > > The initiator is allowed to send firstburst in immediate > > > or in separate PDUs...but can it do neither ? I dont > > > see any statement to the effect that it MUST send a > > > firstburst. > > > > > > Here's a possible problem : > > > 1) FirstBurst is 4K > > > 2) Expected data length of command is 16K > > > 3) Initiator sends the Scsi command without immediate > > > data. (final flag on Scsi command is not set) > > > And it decides to send no further data PDUs. > > > > > > The target does not know if R2Ts can be sent since it does > > > not know if any unsolicited PDUs are expected. > > > > > > Perhaps the semantics on the final flag on SCSI command > > > need to be changed to indicate that no unsolicited data > > > will be sent with this command. Currently, the flag implies > > > (Sec 2.3.1) that all the required data has been sent with > > > the command. > > > > > > And here's a typo to fix. Appendix E (Key=immediateData) > > > The following should say "initialR2T is no". > > > The table has got it right. > > > > > > > If ImmediateData is set to yes and InitialR2T is set > > > > to yes then the initiator MAY send unsolicited immediate > > > > data or one unsolicited burst of Data-OUT PDUs but > > > > MUST NOT send both immediate and a unsolicited burst of > > > > Data-OUT PDUs for any one command. > > > > > > -Sandeep > > > > From owner-ips@ece.cmu.edu Sat May 12 01:02:35 2001 Received: from ece.cmu.edu ([128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id BAA28472 for ; Sat, 12 May 2001 01:02:35 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f4C2vaf04198 for ips-outgoing; Fri, 11 May 2001 22:57:36 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from china.com (TCE-E-7-182-13.bta.net.cn [202.106.182.13]) by ece.cmu.edu (8.11.0/8.10.2) with SMTP id f4BLnnH13873 for ; Fri, 11 May 2001 17:49:49 -0400 (EDT) Received: from china.com([10.1.0.100]) by china.com(AIMC 2.9.5.1) with SMTP id jm853afcbe46; Sat, 12 May 2001 05:49:52 +0800 Received: from loki.ietf.org([132.151.1.177]) by china.com(AIMC 2.9.5.1) with SMTP id jmfd3afbe42e; Fri, 11 May 2001 14:48:58 +0800 Received: (from adm@localhost) by loki.ietf.org (8.9.1b+Sun/8.9.1) id JAA13308 for ietf-123-outbound.02@ietf.org; Wed, 9 May 2001 09:35:00 -0400 (EDT) Received: from ietf.org (odin.ietf.org [10.27.2.28]) by loki.ietf.org (8.9.1b+Sun/8.9.1) with ESMTP id JAA13167 for ; Wed, 9 May 2001 09:30:59 -0400 (EDT) Received: from CNRI.Reston.VA.US (localhost [127.0.0.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id JAA05867; Wed, 9 May 2001 09:30:57 -0400 (EDT) Message-Id: <200105091330.JAA05867@ietf.org> Mime-Version: 1.0 Content-Type: Multipart/Mixed; Boundary="NextPart" To: IETF-Announce: ; Cc: ips@ece.cmu.edu From: Internet-Drafts@ietf.org Reply-to: Internet-Drafts@ietf.org Subject: I-D ACTION:draft-ietf-ips-fcencapsulation-01.txt Date: Wed, 09 May 2001 09:30:56 -0400 Sender: owner-ips@ece.cmu.edu Precedence: bulk --NextPart A New Internet-Draft is available from the on-line Internet-Drafts directories. This draft is a work item of the IP Storage Working Group of the IETF. Title : FC Frame Encapsulation Author(s) : R. Weber, M. Rajagopal, F. Travostino, V. Chau, M. O'Donnell, C. Monia, M. Merhar Filename : draft-ietf-ips-fcencapsulation-01.txt Pages : 13 Date : 08-May-01 This is the ips (IP Storage) working group draft describing the common encapsulation format for use by any IETF protocol that encapsulates Fibre Channel (FC) frames. This draft describes a frame header containing information mandated for encapsulating, transmitting, and de-encapsulating FC frames. A URL for this Internet-Draft is: http://www.ietf.org/internet-drafts/draft-ietf-ips-fcencapsulation-01.txt Internet-Drafts are also available by anonymous FTP. Login with the username "anonymous" and a password of your e-mail address. After logging in, type "cd internet-drafts" and then "get draft-ietf-ips-fcencapsulation-01.txt". A list of Internet-Drafts directories can be found in http://www.ietf.org/shadow.html or ftp://ftp.ietf.org/ietf/1shadow-sites.txt Internet-Drafts can also be obtained by e-mail. Send a message to: mailserv@ietf.org. In the body type: "FILE /internet-drafts/draft-ietf-ips-fcencapsulation-01.txt". NOTE: The mail server at ietf.org can return the document in MIME-encoded form by using the "mpack" utility. To use this feature, insert the command "ENCODING mime" before the "FILE" command. To decode the response(s), you will need "munpack" or a MIME-compliant mail reader. Different MIME-compliant mail readers exhibit different behavior, especially when dealing with "multipart" MIME messages (i.e. documents which have been split up into multiple messages), so check your local documentation on how to manipulate these messages. Below is the data which will enable a MIME compliant mail reader implementation to automatically retrieve the ASCII version of the Internet-Draft. --NextPart Content-Type: Multipart/Alternative; Boundary="OtherAccess" --OtherAccess Content-Type: Message/External-body; access-type="mail-server"; server="mailserv@ietf.org" Content-Type: text/plain Content-ID: <20010508152048.I-D@ietf.org> ENCODING mime FILE /internet-drafts/draft-ietf-ips-fcencapsulation-01.txt --OtherAccess Content-Type: Message/External-body; name="draft-ietf-ips-fcencapsulation-01.txt"; site="ftp.ietf.org"; access-type="anon-ftp"; directory="internet-drafts" Content-Type: text/plain Content-ID: <20010508152048.I-D@ietf.org> --OtherAccess-- --NextPart-- From owner-ips@ece.cmu.edu Sat May 12 02:23:59 2001 Received: from ece.cmu.edu ([128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id CAA13110 for ; Sat, 12 May 2001 02:23:59 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f4C4Hem09266 for ips-outgoing; Sat, 12 May 2001 00:17:40 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from d12lmsgate.de.ibm.com (d12lmsgate.de.ibm.com [195.212.91.199]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f4C4HbH09260 for ; Sat, 12 May 2001 00:17:38 -0400 (EDT) Received: from d12relay01.de.ibm.com (d12relay01.de.ibm.com [9.165.215.22]) by d12lmsgate.de.ibm.com (1.0.0) with ESMTP id GAA193928 for ; Sat, 12 May 2001 06:17:30 +0200 From: julian_satran@il.ibm.com Received: from d12mta02.de.ibm.com (d12mta01_cs0 [9.165.222.237]) by d12relay01.de.ibm.com (8.8.8m3/NCO v4.96) with SMTP id GAA132758 for ; Sat, 12 May 2001 06:17:30 +0200 Received: by d12mta02.de.ibm.com(Lotus SMTP MTA v4.6.5 (863.2 5-20-1999)) id C1256A4A.001790BF ; Sat, 12 May 2001 06:17:23 +0200 X-Lotus-FromDomain: IBMIL@IBMDE To: ips@ece.cmu.edu Message-ID: Date: Sat, 12 May 2001 07:22:59 +0300 Subject: RE: iSCSI: immediate data Mime-Version: 1.0 Content-type: text/plain; charset=us-ascii Content-Disposition: inline Sender: owner-ips@ece.cmu.edu Precedence: bulk Ayman, The new 2.3.1 is: 2.3.1 Flags and Task Attributes The flags for a SCSI Command are: b7 (F) set to 1 when the immediate data that accompany the command are all the unsolicited data associated with this command. For a write, if Expected Data Transfer Length is larger than the Length the target may solicit additional data through R2T. b6 (R) set to 1 when input data is expected b5 (W) set to 1 when output data is expected b3-4 Reserved (MUST be 0) b0-2 used to indicate Task Attributes The Task Attributes (ATTR) can have one of the following integer values (see [SAM2] for details): 0 Untagged 1 Simple 2 Ordered 3 Head of Queue 4 ACA The appendix is: xx ImmediateData Use: ALL Who: Initiator and Target ImmediateData= Default is yes. Initiator and target negotiate support for immediate data. If ImmediateData is set to yes and InitialR2T is set to yes (default) then only immediate data are accepted in the first burst. If ImmediateData is set to no and InitialR2T is set to yes then the initiator MUST NOT send unsolicited data and the target MUST reject them with the corresponding response code. If ImmediateData is set to no and InitialR2T is set to no then the initiator MUST NOT send unsolicited immediate data but MAY send one unsolicited burst of Data-OUT PDUs. If ImmediateData is set to yes and InitialR2T is set to no then the initiator MAY send unsolicited immediate data and/or one unsolicited burst of Data-OUT PDUs. This field sets the D field in the Disconnect-Reconnect mode page. The value one of the D field means ImmediateData=no. The following table is a summary of unsolicited data options: +----------+-------------+--------------------------------------+ |InitialR2T|ImmediateData| Result (up to FirstBurstSize) | +----------+-------------+--------------------------------------+ | no | no | Unsolicited data in data PDUs only | +----------+-------------+--------------------------------------+ | no | yes | Immediate & separate unsolicited data| | | | | +----------+-------------+--------------------------------------+ | yes | no | Unsolicited data disallowed | +----------+-------------+--------------------------------------+ | yes | yes | Immediate unsolicited data only | +----------+-------------+--------------------------------------+ Julo "Ayman Ghanem" on 11-05-2001 23:35:17 Please respond to "Ayman Ghanem" To: ips@ece.cmu.edu cc: Subject: RE: iSCSI: immediate data Julian, Could you please post the revised text for section 2.3.1, and the other revised parts for immediate/unsolicited data (Appendix E-22)?. Thanks. -Ayman > -----Original Message----- > From: owner-ips@ece.cmu.edu [mailto:owner-ips@ece.cmu.edu]On Behalf Of > julian_satran@il.ibm.com > Sent: Wednesday, May 09, 2001 2:12 AM > To: Sandeep Joshi > Cc: matt_wakeley@agilent.com; ips@ece.cmu.edu > Subject: Re: iSCSI: immediate data > > > > > That is correct. The final bit indicates the end of unsolicited data. > If all you have is immediate then the final bit is 1in the command. > If you have immediate and other unsolicited (if enabled!) then set F to 0. > > 2.3.1 has been fixed accordingly. > > Regards, > Julo > > sandeepj@research.bell-labs.com (Sandeep Joshi) on 09-05-2001 01:56:25 > > Please respond to sandeepj@research.bell-labs.com (Sandeep Joshi) > > To: matt_wakeley@agilent.com > cc: ips@ece.cmu.edu > Subject: Re: iSCSI: immediate data > > > > > Umm.....Julian replied to this one and said the final bit on ScsiCmd > would indicate it. Apparently, a single command can now send > the unsolicited data both with the command and in separate PDUs. > > I cant seem to find the email but I believe Sec 2.3.1 was changed > accordingly. Julian..? > > -Sandeep > > > The intention is that if an initiator requests to send immediate data > (and is > > granted the request), then it will always send immediate data. > > > > It sounds like you are asking for wishy washy mode... sometimes send > immediate > > data, sometimes not. > > > > -Matt > > > > Sandeep Joshi wrote: > > > > > > Julian, > > > > > > I had a follow-up question on an old thread. > > > http://ips.pdl.cs.cmu.edu/mail/msg03373.html > > > > > > The initiator is allowed to send firstburst in immediate > > > or in separate PDUs...but can it do neither ? I dont > > > see any statement to the effect that it MUST send a > > > firstburst. > > > > > > Here's a possible problem : > > > 1) FirstBurst is 4K > > > 2) Expected data length of command is 16K > > > 3) Initiator sends the Scsi command without immediate > > > data. (final flag on Scsi command is not set) > > > And it decides to send no further data PDUs. > > > > > > The target does not know if R2Ts can be sent since it does > > > not know if any unsolicited PDUs are expected. > > > > > > Perhaps the semantics on the final flag on SCSI command > > > need to be changed to indicate that no unsolicited data > > > will be sent with this command. Currently, the flag implies > > > (Sec 2.3.1) that all the required data has been sent with > > > the command. > > > > > > And here's a typo to fix. Appendix E (Key=immediateData) > > > The following should say "initialR2T is no". > > > The table has got it right. > > > > > > > If ImmediateData is set to yes and InitialR2T is set > > > > to yes then the initiator MAY send unsolicited immediate > > > > data or one unsolicited burst of Data-OUT PDUs but > > > > MUST NOT send both immediate and a unsolicited burst of > > > > Data-OUT PDUs for any one command. > > > > > > -Sandeep > > > > From owner-ips@ece.cmu.edu Sat May 12 02:26:16 2001 Received: from ece.cmu.edu ([128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id CAA13144 for ; Sat, 12 May 2001 02:26:15 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f4C4YWU10328 for ips-outgoing; Sat, 12 May 2001 00:34:32 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from d12lmsgate-3.de.ibm.com (d12lmsgate-3.de.ibm.com [195.212.91.201]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f4C4YUH10322 for ; Sat, 12 May 2001 00:34:30 -0400 (EDT) Received: from d12relay01.de.ibm.com (d12relay01.de.ibm.com [9.165.215.22]) by d12lmsgate-3.de.ibm.com (1.0.0) with ESMTP id GAA146878 for ; Sat, 12 May 2001 06:34:22 +0200 From: julian_satran@il.ibm.com Received: from d12mta02.de.ibm.com (d12mta01_cs0 [9.165.222.237]) by d12relay01.de.ibm.com (8.8.8m3/NCO v4.96) with SMTP id GAA130478 for ; Sat, 12 May 2001 06:34:22 +0200 Received: by d12mta02.de.ibm.com(Lotus SMTP MTA v4.6.5 (863.2 5-20-1999)) id C1256A4A.00191D82 ; Sat, 12 May 2001 06:34:19 +0200 X-Lotus-FromDomain: IBMIL@IBMDE To: ips@ece.cmu.edu Message-ID: Date: Sat, 12 May 2001 07:39:35 +0300 Subject: Re: iSCSI: Canonical Targets Mime-Version: 1.0 Content-type: text/plain; charset=us-ascii Content-Disposition: inline Sender: owner-ips@ece.cmu.edu Precedence: bulk Mark, I am not sure that we want to have your "canonical" target as a mandatory construct. It is good for proxies, routers, portals etc.? Don't the have better mechanisms for discovery. But why should you force it on some small box that has the name wired and printed on it's label. To reach the canonical you have to go through all the "login etc.". I think that all goes back to asking - why should we use iSCSI for discovery? SLP, UPnP, Salutation, Jini all attempt discovery using a a general form (good for iSCSI, Lpr printers etc.) Can't we take the same path? Julo Mark Bakke on 12-05-2001 00:04:24 Please respond to Mark Bakke To: IPS cc: Subject: iSCSI: Canonical Targets It was pointed out during the interim meeting that the canonical target is defined somewhat ambiguously, and may be a bit too flexible for interoperability purposes. This message is a stake in the ground for defining this a little more tightly. So here are the new canonical target rules. Please let me know if there are major problems with any of them. I tried to define them a-la-carte, to make it easier to pick out any that might cause trouble. 1. Each iSCSI implementation MUST include a canonical target. 2. A canonical target MUST be accessible at the default, IANA- assigned TCP port on each IP address on which the iSCSI implementation is listening for iSCSI connections. 3. A canonical target MUST NOT be used for SCSI commands. A canonical target is an iSCSI construct only, and does not have a corresponding SCSI device. This means it may not be used to access Logical Units. A session created to a canonical target is a discovery session only, and once in full feature phase, is used only for text commands and asynchronous messages. (Do any other commands make sense)? 4. A device containing a single target MUST provide both the canonical target and the real target. (This is implied by the above requirements). An initiator connecting to such a device using only its IP address would first connect to the canonical target, and use SendTargets to obtain the iSCSI name of the real target. It would then create a separate session to the real target. Essentially, this means there's nothing special about a single-target device. 5. An iSCSI device MUST provide a unique iSCSI name for each of its targets. Using the canonical target as a nameless iSCSI target is not supported. 6. We can further specify the order in which the SendTargets response fields are returned, to simplify things further, e.g. each target in the SendTargets response MUST return these fields in this order: - A TargetName= field - A TargetAlias= field (value left blank if there's no alias) - One or more TargetAddress= fields - Any vendor-specific fields (ignored by standard initiators) The above rules will make the behavior of various target implementations identical, regardless of the number of interfaces or targets they support. This will reduce the number of end-cases that initiator implementations will have to handle. If we agree on these, we can edit them into the iSCSI and NDT documents. Additional questions to send input on: 1. Should a non-canonical target respond to a SendTargets command? 2. If so, should it respond only with addresses for its own target, or should it respond with other targets, as a canonical target might do? 3. If not, the initiator must connect to a canonical target to find the other addresses of a target to which it is already connected; is the information it has sufficient to do so? (I think the answer is yes, given canonical requirement #2 above). -- Mark A. Bakke Cisco Systems mbakke@cisco.com 763.398.1054 From owner-ips@ece.cmu.edu Sat May 12 03:06:27 2001 Received: from ece.cmu.edu ([128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id DAA13470 for ; Sat, 12 May 2001 03:06:27 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f4C55ub12372 for ips-outgoing; Sat, 12 May 2001 01:05:56 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from d12lmsgate-2.de.ibm.com (d12lmsgate-2.de.ibm.com [195.212.91.200]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f4C55sH12368 for ; Sat, 12 May 2001 01:05:54 -0400 (EDT) Received: from d12relay01.de.ibm.com (d12relay01.de.ibm.com [9.165.215.22]) by d12lmsgate-2.de.ibm.com (1.0.0) with ESMTP id HAA188414 for ; Sat, 12 May 2001 07:05:47 +0200 From: julian_satran@il.ibm.com Received: from d12mta02.de.ibm.com (d12mta01_cs0 [9.165.222.237]) by d12relay01.de.ibm.com (8.8.8m3/NCO v4.96) with SMTP id HAA194052 for ; Sat, 12 May 2001 07:05:47 +0200 Received: by d12mta02.de.ibm.com(Lotus SMTP MTA v4.6.5 (863.2 5-20-1999)) id C1256A4A.001BFCEC ; Sat, 12 May 2001 07:05:42 +0200 X-Lotus-FromDomain: IBMIL@IBMDE To: ips@ece.cmu.edu Message-ID: Date: Sat, 12 May 2001 08:08:43 +0300 Subject: Re: iSCSI : Bridging missing CmdSNs and Abort I/O Error recovery Mime-Version: 1.0 Content-type: text/plain; charset=us-ascii Content-Disposition: inline Sender: owner-ips@ece.cmu.edu Precedence: bulk please reread carefully my previous answer. Julo Santosh Rao on 09-05-2001 19:37:41 Please respond to Santosh Rao To: IPS Reflector cc: Subject: Re: iSCSI : Bridging missing CmdSNs and Abort I/O Error recovery > julian_satran@il.ibm.com wrote: > > > We could add the RefCmdSN and it may help plug-in the hole but unless the > > > Abort is ussued on the same connection as the original command we can't > > be > > > sure that the old-one will not pop-up (as we enable ExpCmdSN to move on > > we > > > don't have even the 2**31-1 protection bracket :-)). Thus sending in > > > another nop/abort on the same connection is still required. > > > > > > To simplify the whole process I will: > > > > > > a - add RefCmdSN to the Task Management > > > b - add a command not received yet to the answers > > > > > > c - add a part to 7 reading: > > > > > > 1.1 How to Abort Safely a Command that Was Not Received > > > > > > To abort safely a task for which the task abort answer is "Command Not > > > Received Yet" the initiator must issue another abort command on the > > same > > > connection as the original command unless this connection was logged > > out > > > in which case it may send it on any connection. Julian, The above may not be sufficient, since a requirement for the Abort Task completion is that both the initiator & target can safely re-use their task tags and other resources following completion of the Abort Task. If the Abort Task is sent on another connection than the connection used for the original command [& that connection has not been logged out], then, there may be some in-flight PDUs from the target to the initiator for that command on the connection. Sending an Abort Task on another connection may cause the initiator to receive the Abort Task response indicating success on the other connection which then results in initiator freeing up its task tag and other resources used for that command. Thereafter, the stale PDUs on the original connection can show up at the initiator, causing problems. I would suggest that Abort Task be always sent on the same connection as the command, unless the connection used for that command has been logged out. Not doing so implies the initiator cannot safely re-use its task tag and other resources that were tied up for that I/O. Regards, Santosh - santoshr.vcf From owner-ips@ece.cmu.edu Sat May 12 10:41:44 2001 Received: from ece.cmu.edu ([128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id KAA17481 for ; Sat, 12 May 2001 10:41:44 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f4CCe0B18271 for ips-outgoing; Sat, 12 May 2001 08:40:00 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from lucy.trebia.com ([65.192.191.151]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f4CCdwH18267 for ; Sat, 12 May 2001 08:39:59 -0400 (EDT) Received: from trebia.com (19.sun6.dialup.G4.NET [216.177.30.19]) by lucy.trebia.com with SMTP (Microsoft Exchange Internet Mail Service Version 5.5.2653.13) id KTSS5YDZ; Sat, 12 May 2001 08:43:51 -0400 Message-ID: <3AFD2FA1.F033A4DE@trebia.com> Date: Sat, 12 May 2001 08:42:09 -0400 From: James Smart Reply-To: james.smart@trebia.com X-Mailer: Mozilla 4.75 [en] (Windows NT 5.0; U) X-Accept-Language: en MIME-Version: 1.0 To: julian_satran@il.ibm.com, Mark Bakke CC: ips@ece.cmu.edu Subject: Re: iSCSI: Canonical Targets References: Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-ips@ece.cmu.edu Precedence: bulk Content-Transfer-Encoding: 7bit I'm in agreement with Mark, that canonical target support is mandatory. The key is interoperability. The initiator must be mandated to probe the canonical target and not assume a single target device. If all initiators do not support this type of probing, then interoperability goes out the window. Mark's effort, mandates the support on the target device - which indirectly puts the mandate on the initiator to check, and makes sure there are not confusing answers from devices that otherwise would not respond to the canonical target. Is this replacing discovery - perhaps in some ways. The real issue is allowing i/o traffic to be rerouted to non-IANA TCP port numbers, which is a critical need of the proxy/routers/etc, which have little left in the socket 4-tuple to classify on if rerouting is not allowed.. I would expect the "discovery" (SLP, uPnP,etc) mechanisms would contain the "targets" that the proxy/router/etc, but the expectation is that they would all point at the well-known IANA TCP port number. The canonical processing simply allows the target device to shift session-specific traffic to target-assigned port numbers that can aid it's classification schemes. -- James julian_satran@il.ibm.com wrote: > Mark, > > I am not sure that we want to have your "canonical" target as a mandatory > construct. > It is good for proxies, routers, portals etc.? Don't the have better > mechanisms for discovery. > But why should you force it on some small box that has the name wired and > printed on it's label. > To reach the canonical you have to go through all the "login etc.". > > I think that all goes back to asking - why should we use iSCSI for > discovery? > > SLP, UPnP, Salutation, Jini all attempt discovery using a a general form > (good for iSCSI, Lpr printers etc.) > Can't we take the same path? > > Julo > > Mark Bakke on 12-05-2001 00:04:24 > > Please respond to Mark Bakke > > To: IPS > cc: > Subject: iSCSI: Canonical Targets > > It was pointed out during the interim meeting that the > canonical target is defined somewhat ambiguously, and may > be a bit too flexible for interoperability purposes. This > message is a stake in the ground for defining this a little > more tightly. > > So here are the new canonical target rules. Please let me > know if there are major problems with any of them. I tried > to define them a-la-carte, to make it easier to pick out any > that might cause trouble. > > 1. Each iSCSI implementation MUST include a canonical target. > > 2. A canonical target MUST be accessible at the default, IANA- > assigned TCP port on each IP address on which the iSCSI > implementation is listening for iSCSI connections. > > 3. A canonical target MUST NOT be used for SCSI commands. > > A canonical target is an iSCSI construct only, and does not > have a corresponding SCSI device. This means it may not > be used to access Logical Units. > > A session created to a canonical target is a discovery session > only, and once in full feature phase, is used only for text > commands and asynchronous messages. (Do any other commands make > sense)? > > 4. A device containing a single target MUST provide both the > canonical target and the real target. (This is implied by the > above requirements). > > An initiator connecting to such a device using only its IP address > would first connect to the canonical target, and use SendTargets > to obtain the iSCSI name of the real target. It would then create > a separate session to the real target. Essentially, this means > there's nothing special about a single-target device. > > 5. An iSCSI device MUST provide a unique iSCSI name for each of its > targets. Using the canonical target as a nameless iSCSI target > is not supported. > > 6. We can further specify the order in which the SendTargets response > fields are returned, to simplify things further, e.g. each target > in the SendTargets response MUST return these fields in this order: > > - A TargetName= field > - A TargetAlias= field (value left blank if there's no alias) > - One or more TargetAddress= fields > - Any vendor-specific fields (ignored by standard initiators) > > The above rules will make the behavior of various target implementations > identical, regardless of the number of interfaces or targets they > support. This will reduce the number of end-cases that initiator > implementations will have to handle. > > If we agree on these, we can edit them into the iSCSI and NDT documents. > > Additional questions to send input on: > > 1. Should a non-canonical target respond to a SendTargets command? > > 2. If so, should it respond only with addresses for its own target, > or should it respond with other targets, as a canonical target > might do? > > 3. If not, the initiator must connect to a canonical target to find > the other addresses of a target to which it is already connected; > is the information it has sufficient to do so? (I think the answer > is yes, given canonical requirement #2 above). > > -- > Mark A. Bakke > Cisco Systems > mbakke@cisco.com > 763.398.1054 From owner-ips@ece.cmu.edu Sat May 12 13:56:30 2001 Received: from ece.cmu.edu ([128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id NAA18771 for ; Sat, 12 May 2001 13:56:29 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f4CFxiM29853 for ips-outgoing; Sat, 12 May 2001 11:59:44 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from d12lmsgate.de.ibm.com (d12lmsgate.de.ibm.com [195.212.91.199]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f4CFxgH29848 for ; Sat, 12 May 2001 11:59:42 -0400 (EDT) Received: from d12relay01.de.ibm.com (d12relay01.de.ibm.com [9.165.215.22]) by d12lmsgate.de.ibm.com (1.0.0) with ESMTP id RAA236986 for ; Sat, 12 May 2001 17:59:33 +0200 From: julian_satran@il.ibm.com Received: from d12mta02.de.ibm.com (d12mta01_cs0 [9.165.222.237]) by d12relay01.de.ibm.com (8.8.8m3/NCO v4.96) with SMTP id RAA145854 for ; Sat, 12 May 2001 17:59:34 +0200 Received: by d12mta02.de.ibm.com(Lotus SMTP MTA v4.6.5 (863.2 5-20-1999)) id C1256A4A.0057D5FD ; Sat, 12 May 2001 17:59:24 +0200 X-Lotus-FromDomain: IBMIL@IBMDE To: ips@ece.cmu.edu Message-ID: Date: Sat, 12 May 2001 19:05:00 +0300 Subject: Re: iSCSI: Canonical Targets Mime-Version: 1.0 Content-type: text/plain; charset=us-ascii Content-Disposition: inline Sender: owner-ips@ece.cmu.edu Precedence: bulk Why would any discovery service be wiser/stupider than the mechanism we are having in N&D? If the issue of ports is widespread then others have it too and discovery may include all the nice things we have in iSCSI. I don't see any reason iSNS or an LDAP repository can't contain them. My only claim is that this adds weight to iSCSI. As for interoperability - I claim that adding features makes interoperability an issue. Removing them does not. The whole "canonical" (why canonical - there is nothing canonical about it) target idea is good but in another realm - I don't think it belongs to iSCSI - it belongs to discovery be it specific or general. I can leave with the mechanism as it is - but I really feel that we are on the wrong track. Regards, Julo James Smart on 12-05-2001 15:42:09 Please respond to james.smart@trebia.com To: Julian Satran/Haifa/IBM@IBMIL, Mark Bakke cc: ips@ece.cmu.edu Subject: Re: iSCSI: Canonical Targets I'm in agreement with Mark, that canonical target support is mandatory. The key is interoperability. The initiator must be mandated to probe the canonical target and not assume a single target device. If all initiators do not support this type of probing, then interoperability goes out the window. Mark's effort, mandates the support on the target device - which indirectly puts the mandate on the initiator to check, and makes sure there are not confusing answers from devices that otherwise would not respond to the canonical target. Is this replacing discovery - perhaps in some ways. The real issue is allowing i/o traffic to be rerouted to non-IANA TCP port numbers, which is a critical need of the proxy/routers/etc, which have little left in the socket 4-tuple to classify on if rerouting is not allowed.. I would expect the "discovery" (SLP, uPnP,etc) mechanisms would contain the "targets" that the proxy/router/etc, but the expectation is that they would all point at the well-known IANA TCP port number. The canonical processing simply allows the target device to shift session-specific traffic to target-assigned port numbers that can aid it's classification schemes. -- James julian_satran@il.ibm.com wrote: > Mark, > > I am not sure that we want to have your "canonical" target as a mandatory > construct. > It is good for proxies, routers, portals etc.? Don't the have better > mechanisms for discovery. > But why should you force it on some small box that has the name wired and > printed on it's label. > To reach the canonical you have to go through all the "login etc.". > > I think that all goes back to asking - why should we use iSCSI for > discovery? > > SLP, UPnP, Salutation, Jini all attempt discovery using a a general form > (good for iSCSI, Lpr printers etc.) > Can't we take the same path? > > Julo > > Mark Bakke on 12-05-2001 00:04:24 > > Please respond to Mark Bakke > > To: IPS > cc: > Subject: iSCSI: Canonical Targets > > It was pointed out during the interim meeting that the > canonical target is defined somewhat ambiguously, and may > be a bit too flexible for interoperability purposes. This > message is a stake in the ground for defining this a little > more tightly. > > So here are the new canonical target rules. Please let me > know if there are major problems with any of them. I tried > to define them a-la-carte, to make it easier to pick out any > that might cause trouble. > > 1. Each iSCSI implementation MUST include a canonical target. > > 2. A canonical target MUST be accessible at the default, IANA- > assigned TCP port on each IP address on which the iSCSI > implementation is listening for iSCSI connections. > > 3. A canonical target MUST NOT be used for SCSI commands. > > A canonical target is an iSCSI construct only, and does not > have a corresponding SCSI device. This means it may not > be used to access Logical Units. > > A session created to a canonical target is a discovery session > only, and once in full feature phase, is used only for text > commands and asynchronous messages. (Do any other commands make > sense)? > > 4. A device containing a single target MUST provide both the > canonical target and the real target. (This is implied by the > above requirements). > > An initiator connecting to such a device using only its IP address > would first connect to the canonical target, and use SendTargets > to obtain the iSCSI name of the real target. It would then create > a separate session to the real target. Essentially, this means > there's nothing special about a single-target device. > > 5. An iSCSI device MUST provide a unique iSCSI name for each of its > targets. Using the canonical target as a nameless iSCSI target > is not supported. > > 6. We can further specify the order in which the SendTargets response > fields are returned, to simplify things further, e.g. each target > in the SendTargets response MUST return these fields in this order: > > - A TargetName= field > - A TargetAlias= field (value left blank if there's no alias) > - One or more TargetAddress= fields > - Any vendor-specific fields (ignored by standard initiators) > > The above rules will make the behavior of various target implementations > identical, regardless of the number of interfaces or targets they > support. This will reduce the number of end-cases that initiator > implementations will have to handle. > > If we agree on these, we can edit them into the iSCSI and NDT documents. > > Additional questions to send input on: > > 1. Should a non-canonical target respond to a SendTargets command? > > 2. If so, should it respond only with addresses for its own target, > or should it respond with other targets, as a canonical target > might do? > > 3. If not, the initiator must connect to a canonical target to find > the other addresses of a target to which it is already connected; > is the information it has sufficient to do so? (I think the answer > is yes, given canonical requirement #2 above). > > -- > Mark A. Bakke > Cisco Systems > mbakke@cisco.com > 763.398.1054 From owner-ips@ece.cmu.edu Sat May 12 19:41:08 2001 Received: from ece.cmu.edu ([128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id TAA21266 for ; Sat, 12 May 2001 19:41:08 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f4CLZCf18138 for ips-outgoing; Sat, 12 May 2001 17:35:12 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from mail.brocade.com (asbestos.brocade.com [63.121.140.244]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f4CJshH12671 for ; Sat, 12 May 2001 15:54:43 -0400 (EDT) Received: from hq-ex-c2.brocade.com (hq-ex-c2 [192.168.126.35]) by mail.brocade.com (8.8.8+Sun/8.8.8) with ESMTP id MAA08557; Sat, 12 May 2001 12:47:47 -0700 (PDT) Received: by hq-ex-c2.brocade.com with Internet Mail Service (5.5.2653.19) id ; Sat, 12 May 2001 12:47:47 -0700 Message-ID: From: Robert Snively To: "'Santosh Rao'" , Black_David@emc.com Cc: ips@ece.cmu.edu Subject: RE: iSCSI : login keys & mode page settings Date: Sat, 12 May 2001 12:47:45 -0700 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2653.19) Content-Type: multipart/alternative; boundary="----_=_NextPart_001_01C0DB1C.6A7A96C0" Sender: owner-ips@ece.cmu.edu Precedence: bulk This message is in MIME format. Since your mail reader does not understand this format, some or all of this message may not be legible. ------_=_NextPart_001_01C0DB1C.6A7A96C0 Content-Type: text/plain; charset="iso-8859-1" Be deprecating the mode page mechanism, you would reduce the compatibility with present management software, but more importantly, you would increase the complexity of many devices that would still have to interpret the somewhat obscure intent of a key in terms of a mode page action that would have to be executed against an actual SCSI device. For compatibility, the natural method to prohibit would be the new and therefore incompatible method. Bob > -----Original Message----- > From: Santosh Rao [mailto:santoshr@cup.hp.com] > Sent: Tuesday, May 08, 2001 5:37 PM > To: Black_David@emc.com > Cc: ips@ece.cmu.edu > Subject: Re: iSCSI : login keys & mode page settings > > > Black_David@emc.com wrote: > > > (1) Is changing an iSCSI key allowed to cause > > problems for other initiators? > > No. > > > (2) Does the iSCSI key mechanism have to behave > > identically to the corresponding mode > > page mechanism. > > No. > > Better still, the mode page mechanism can be dis-allowed [or strongly > discouraged] by iSCSI, allowing only 1 mechanism for setting these > options, which is through the use of login keys. > > Something to the effect of : > "Initiators SHOULD [MUST ?] NOT use Mode Select to modify > these contol > options and any key negotiation SHOULD [MUST ?] be done through > login/text keys" > > may help address these concerns. > > Thanks, > Santosh > > > > > > > Given the need to support old systems that may > > get (1) wrong (mode page sets can damage other > > initiators), the best we may be able to do on it > > is a SHOULD: > > (1) SHOULD not share key values among sessions > > (i.e., setting of a key value in one session > > SHOULD NOT affect the setting in any other > > session. > > On (2), how about > > - When a key refers to a mode page entry, the > > underlying value MUST be shared between > > the mode page and the key in an iSCSI session > > (e.g., a value set by a text key MUST be > > retrieved by the mode page if the implementation > > accepted the value). > > - Restrictions on value changes in full-feature > > mode SHOULD (MUST?) match when a value is > > shared between a text key and a mode page > > entry. > > > > Comments? > > > > --David > > > > --------------------------------------------------- > > David L. Black, Senior Technologist > > EMC Corporation, 42 South St., Hopkinton, MA 01748 > > +1 (508) 435-1000 x75140 FAX: +1 (508) 497-8500 > > black_david@emc.com Mobile: +1 (978) 394-7754 > > --------------------------------------------------- > > > > > -----Original Message----- > > > From: Santosh Rao [SMTP:santoshr@cup.hp.com] > > > Sent: Friday, May 04, 2001 2:15 PM > > > To: Black_David@emc.com; ips@ece.cmu.edu > > > Subject: RE: iSCSI : login keys & mode page settings > > > > > > David, > > > > > > Apologies for the late response on this. I was hoping we > could complete > > > this > > > thread of discussion at Nashua, but for lack of time, we > are back on the > > > list. > > > > > > Regarding your question below : > > > > > > > If one Initiator can damage another in this fashion, then we > > > > may indeed have a problem. > > > > > > > Comments?, > > > > > > Shared mode page implementations in targets is quite common and > > > modification of > > > control parameters through a mode select would indeed > affect all other > > > initiators logged into the target. This is not desirable > behaviour and > > > iSCSI > > > may be better served using login/text based key > negotiation rather than > > > the > > > mode select mechanisms which opens it up to the side > effects of affecting > > > all > > > connected initiators. > > > > > > Thanks, > > > Santosh > > > > > > > > > > ------------------------------------------------------------- > ----------- > > > Subject: RE: iSCSI : login keys & mode page settings > > > From: Black_David@emc.com > > > Date: Fri, 20 Apr 2001 20:32:17 -0400 > > > Cc: ips@ece.cmu.edu > > > > > > I'm not sure -- this sounds somewhat like the > > > old principle of not asking why there's a hole > > > in one's foot when one has aimed the gun at > > > it and pulled the trigger. For the tape > > > example, if some tape driver changes a > > > Target iSCSI parameter that disrupts that > > > driver's own tape I/O in a fashion that the > > > driver can't recover from, I think it's > > > clear where the fault lies. If one Initiator > > > can damage another in this fashion, then we > > > may indeed have a problem. > > > > > > Comments?, > > > --David > > > > > > > -----Original Message----- > > > > From: Santosh Rao [SMTP:santoshr@cup.hp.com] > > > > Sent: Friday, April 20, 2001 8:09 PM > > > > To: Black_David@emc.com > > > > Cc: ips@ece.cmu.edu > > > > Subject: Re: iSCSI : login keys & mode page settings > > > > > > > > David, > > > > > > > > Some clarification on the basis for classifying login > > > ould > > > > also be helpful. Should login keys that can disrupt > > > > I/O on their change > > > > be allowed to be non-LO ? > > > > > > > > Thanks, > > > > Santosh > > > > > > > > Black_David@emc.com wrote: > > > > > > > > > > Without getting into the technical details of the > > > > > discussion, I have a couple of observations: > > > > > > > > > > (A) The issue of whether to allow mode page > > > > > access to and modification of iSCSI > > > ers > > > > > will need to be taken up at the interim > > > > > meeting. IMHO, access seems like a good > > > > > idea, so that SCSI-generic code that doesn't > > > > > know specifically about iSCSI can find > > > > > what it expects where it expects it, but > > > > > I'm unsure about modification because it > > > > > may carry a risk of code that's > > > naware > > > > > getting something wrong. The mode page > > > > > commands should be transparent to iSCSI. > > > > > > > > > > (B) The mode page and text key mechanisms have > > > > > to access the same data. Section 3 of the > > > > > -06 version says this, but needs some > > > > > editing > > > > > to enforce it by using "MUST" or its > > > > > equivalent > > > > > (cf. RFC 2119). This is to prevent an > > > > > implementation from having two instances of > > > > > the same parameter - one for the mode page and > > > > > one for the text keys - which would be a bad > > > > > thing. > > > > > > > > > > --David > ------_=_NextPart_001_01C0DB1C.6A7A96C0 Content-Type: text/html; charset="iso-8859-1" RE: iSCSI : login keys & mode page settings

Be deprecating the mode page mechanism, you would reduce the
compatibility with present management software, but more
importantly, you would increase the complexity of many devices
that would still have to interpret the somewhat obscure intent
of a key in terms of a mode page action that would have to be
executed against an actual SCSI device.

For compatibility, the natural method to prohibit would be
the new and therefore incompatible method.

Bob

>  -----Original Message-----
>  From: Santosh Rao [mailto:santoshr@cup.hp.com]
>  Sent: Tuesday, May 08, 2001 5:37 PM
>  To: Black_David@emc.com
>  Cc: ips@ece.cmu.edu
>  Subject: Re: iSCSI : login keys & mode page settings
> 
> 
>  Black_David@emc.com wrote:
> 
>  > (1) Is changing an iSCSI key allowed to cause
>  >         problems for other initiators?
> 
>  No.
> 
>  > (2) Does the iSCSI key mechanism have to behave
>  >         identically to the corresponding mode
>  >         page mechanism.
> 
>  No.
> 
>  Better still, the mode page mechanism can be dis-allowed [or strongly
>  discouraged] by iSCSI, allowing only 1 mechanism for setting these
>  options, which is through the use of login keys.
> 
>  Something to the effect of :
>  "Initiators SHOULD [MUST ?] NOT use Mode Select to modify
>  these contol
>  options and any key negotiation SHOULD [MUST ?] be done through
>  login/text keys"
> 
>  may help address these concerns.
> 
>  Thanks,
>  Santosh
> 
> 
> 
>  >
>  > Given the need to support old systems that may
>  > get (1) wrong (mode page sets can damage other
>  > initiators), the best we may be able to do on it
>  > is a SHOULD:
>  > (1) SHOULD not share key values among sessions
>  >         (i.e., setting of a key value in one session
>  >         SHOULD NOT affect the setting in any other
>  >         session.
>  > On (2), how about
>  > - When a key refers to a mode page entry, the
>  >         underlying value MUST be shared between
>  >         the mode page and the key in an iSCSI session
>  >         (e.g., a value set by a text key MUST be
>  >         retrieved by the mode page if the implementation
>  >         accepted the value).
>  > - Restrictions on value changes in full-feature
>  >         mode SHOULD (MUST?) match when a value is
>  >         shared between a text key and a mode page
>  >         entry.
>  >
>  > Comments?
>  >
>  > --David
>  >
>  > ---------------------------------------------------
>  > David L. Black, Senior Technologist
>  > EMC Corporation, 42 South St., Hopkinton, MA  01748
>  > +1 (508) 435-1000 x75140     FAX: +1 (508) 497-8500
>  > black_david@emc.com       Mobile: +1 (978) 394-7754
>  > ---------------------------------------------------
>  >
>  > > -----Original Message-----
>  > > From: Santosh Rao [SMTP:santoshr@cup.hp.com]
>  > > Sent: Friday, May 04, 2001 2:15 PM
>  > > To:   Black_David@emc.com; ips@ece.cmu.edu
>  > > Subject:      RE: iSCSI : login keys & mode page settings
>  > >
>  > > David,
>  > >
>  > > Apologies for the late response on this. I was hoping we
>  could complete
>  > > this
>  > > thread of discussion at Nashua, but for lack of time, we
>  are back on the
>  > > list.
>  > >
>  > > Regarding your question below :
>  > >
>  > > > If one Initiator can damage another in this fashion, then we
>  > > > may indeed have a problem.
>  > >
>  > > > Comments?,
>  > >
>  > > Shared mode page implementations in targets is quite common and
>  > > modification of
>  > > control parameters through a mode select would indeed
>  affect all other
>  > > initiators logged into the target. This is not desirable
>  behaviour and
>  > > iSCSI
>  > > may be better served using login/text based key
>  negotiation rather than
>  > > the
>  > > mode select mechanisms which opens it up to the side
>  effects of affecting
>  > > all
>  > > connected initiators.
>  > >
>  > > Thanks,
>  > > Santosh
>  > >
>  > >
>  > >
>  -------------------------------------------------------------
>  -----------
>  > > Subject: RE: iSCSI : login keys & mode page settings
>  > > From: Black_David@emc.com
>  > > Date: Fri, 20 Apr 2001 20:32:17 -0400
>  > > Cc: ips@ece.cmu.edu
>  > >
>  > > I'm not sure -- this sounds somewhat like the
>  > > old principle of not asking why there's a hole
>  > > in one's foot when one has aimed the gun at
>  > > it and pulled the trigger.  For the tape
>  > > example, if some tape driver changes a
>  > > Target iSCSI parameter that disrupts that
>  > > driver's own tape I/O in a fashion that the
>  > > driver can't recover from, I think it's
>  > > clear where the fault lies.  If one Initiator
>  > > can damage another in this fashion, then we
>  > > may indeed have a problem.
>  > >
>  > > Comments?,
>  > > --David
>  > >
>  > > > -----Original Message-----
>  > > > From: Santosh Rao [SMTP:santoshr@cup.hp.com]
>  > > > Sent: Friday, April 20, 2001 8:09 PM
>  > > > To:   Black_David@emc.com
>  > > > Cc:   ips@ece.cmu.edu
>  > > > Subject:      Re: iSCSI : login keys & mode page settings
>  > > >
>  > > > David,
>  > > >
>  > > > Some clarification on the basis for classifying login
>  > > ould
>  > > > also be helpful. Should login keys that can disrupt
>  > > > I/O on their change
>  > > > be allowed to be non-LO ?
>  > > >
>  > > > Thanks,
>  > > > Santosh
>  > > >
>  > > > Black_David@emc.com wrote:
>  > > > >
>  > > > > Without getting into the technical details of the
>  > > > > discussion, I have a couple of observations:
>  > > > >
>  > > > > (A) The issue of whether to allow mode page
>  > > > >         access to and modification of iSCSI
>  > > ers
>  > > > >         will need to be taken up at the interim
>  > > > >         meeting.  IMHO, access seems like a good
>  > > > >         idea, so that SCSI-generic code that doesn't
>  > > > >         know specifically about iSCSI can find
>  > > > >         what it expects where it expects it, but
>  > > > >         I'm unsure about modification because it
>  > > > >         may carry a risk of code that's
>  > > naware
>  > > > >         getting something wrong.  The mode page
>  > > > >         commands should be transparent to iSCSI.
>  > > > >
>  > > > > (B) The mode page and text key mechanisms have
>  > > > >         to access the same data.  Section 3 of the
>  > > > >         -06 version says this, but needs some
>  > > > >      editing
>  > > > >         to enforce it by using "MUST" or its
>  > > > >      equivalent
>  > > > >         (cf. RFC 2119).  This is to prevent an
>  > > > >         implementation from having two instances of
>  > > > >         the same parameter - one for the mode page and
>  > > > >         one for the text keys - which would be a bad
>  > > > >         thing.
>  > > > >
>  > > > > --David
> 

------_=_NextPart_001_01C0DB1C.6A7A96C0-- From owner-ips@ece.cmu.edu Sun May 13 02:30:15 2001 Received: from ece.cmu.edu ([128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id CAA10479 for ; Sun, 13 May 2001 02:30:14 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f4D4ZVI10311 for ips-outgoing; Sun, 13 May 2001 00:35:31 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from c007.snv.cp.net (c007-h012.c007.snv.cp.net [209.228.33.219]) by ece.cmu.edu (8.11.0/8.10.2) with SMTP id f4D4ZPH10294 for ; Sun, 13 May 2001 00:35:26 -0400 (EDT) Received: (cpmta 15343 invoked from network); 12 May 2001 21:35:18 -0700 Received: from unknown (HELO ljoy) (64.130.130.105) by smtp.telocity.com (209.228.33.219) with SMTP; 12 May 2001 21:35:18 -0700 X-Sent: 13 May 2001 04:35:18 GMT From: "Douglas Otis" To: "Robert Snively" , "'Santosh Rao'" , Cc: Subject: RE: iSCSI : login keys & mode page settings Date: Sat, 12 May 2001 21:33:01 -0700 Message-ID: MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 (Normal) X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook IMO, Build 9.0.2416 (9.0.2911.0) X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4522.1200 Importance: Normal In-Reply-To: Sender: owner-ips@ece.cmu.edu Precedence: bulk Content-Transfer-Encoding: 7bit Bob, There is a problem of scope with the present iSCSI specifications. In my view, there should be a document that defines the "iSCSI device", and others that define transport. The "iSCSI device" should be a T10 document. The difficulty results from the amalgam two separate goals. Much of the compatibility issues will hinge on the device behavior model, and mode page definitions. The iSCSI device will be expected to handle more of the functionality of that found in the typical wedge driver. None of these issues are directly relevant to transport. Doug Robert Snively wrote: Be deprecating the mode page mechanism, you would reduce the compatibility with present management software, but more importantly, you would increase the complexity of many devices that would still have to interpret the somewhat obscure intent of a key in terms of a mode page action that would have to be executed against an actual SCSI device. For compatibility, the natural method to prohibit would be the new and therefore incompatible method. Bob > -----Original Message----- > From: Santosh Rao [mailto:santoshr@cup.hp.com] > Sent: Tuesday, May 08, 2001 5:37 PM > To: Black_David@emc.com > Cc: ips@ece.cmu.edu > Subject: Re: iSCSI : login keys & mode page settings > > > Black_David@emc.com wrote: > > > (1) Is changing an iSCSI key allowed to cause > > problems for other initiators? > > No. > > > (2) Does the iSCSI key mechanism have to behave > > identically to the corresponding mode > > page mechanism. > > No. > > Better still, the mode page mechanism can be dis-allowed [or strongly > discouraged] by iSCSI, allowing only 1 mechanism for setting these > options, which is through the use of login keys. > > Something to the effect of : > "Initiators SHOULD [MUST ?] NOT use Mode Select to modify > these contol > options and any key negotiation SHOULD [MUST ?] be done through > login/text keys" > > may help address these concerns. > > Thanks, > Santosh > > > > > > > Given the need to support old systems that may > > get (1) wrong (mode page sets can damage other > > initiators), the best we may be able to do on it > > is a SHOULD: > > (1) SHOULD not share key values among sessions > > (i.e., setting of a key value in one session > > SHOULD NOT affect the setting in any other > > session. > > On (2), how about > > - When a key refers to a mode page entry, the > > underlying value MUST be shared between > > the mode page and the key in an iSCSI session > > (e.g., a value set by a text key MUST be > > retrieved by the mode page if the implementation > > accepted the value). > > - Restrictions on value changes in full-feature > > mode SHOULD (MUST?) match when a value is > > shared between a text key and a mode page > > entry. > > > > Comments? > > > > --David > > > > --------------------------------------------------- > > David L. Black, Senior Technologist > > EMC Corporation, 42 South St., Hopkinton, MA 01748 > > +1 (508) 435-1000 x75140 FAX: +1 (508) 497-8500 > > black_david@emc.com Mobile: +1 (978) 394-7754 > > --------------------------------------------------- > > > > > -----Original Message----- > > > From: Santosh Rao [SMTP:santoshr@cup.hp.com] > > > Sent: Friday, May 04, 2001 2:15 PM > > > To: Black_David@emc.com; ips@ece.cmu.edu > > > Subject: RE: iSCSI : login keys & mode page settings > > > > > > David, > > > > > > Apologies for the late response on this. I was hoping we > could complete > > > this > > > thread of discussion at Nashua, but for lack of time, we > are back on the > > > list. > > > > > > Regarding your question below : > > > > > > > If one Initiator can damage another in this fashion, then we > > > > may indeed have a problem. > > > > > > > Comments?, > > > > > > Shared mode page implementations in targets is quite common and > > > modification of > > > control parameters through a mode select would indeed > affect all other > > > initiators logged into the target. This is not desirable > behaviour and > > > iSCSI > > > may be better served using login/text based key > negotiation rather than > > > the > > > mode select mechanisms which opens it up to the side > effects of affecting > > > all > > > connected initiators. > > > > > > Thanks, > > > Santosh > > > > > > > > > > ------------------------------------------------------------- > ----------- > > > Subject: RE: iSCSI : login keys & mode page settings > > > From: Black_David@emc.com > > > Date: Fri, 20 Apr 2001 20:32:17 -0400 > > > Cc: ips@ece.cmu.edu > > > > > > I'm not sure -- this sounds somewhat like the > > > old principle of not asking why there's a hole > > > in one's foot when one has aimed the gun at > > > it and pulled the trigger. For the tape > > > example, if some tape driver changes a > > > Target iSCSI parameter that disrupts that > > > driver's own tape I/O in a fashion that the > > > driver can't recover from, I think it's > > > clear where the fault lies. If one Initiator > > > can damage another in this fashion, then we > > > may indeed have a problem. > > > > > > Comments?, > > > --David > > > > > > > -----Original Message----- > > > > From: Santosh Rao [SMTP:santoshr@cup.hp.com] > > > > Sent: Friday, April 20, 2001 8:09 PM > > > > To: Black_David@emc.com > > > > Cc: ips@ece.cmu.edu > > > > Subject: Re: iSCSI : login keys & mode page settings > > > > > > > > David, > > > > > > > > Some clarification on the basis for classifying login > > > ould > > > > also be helpful. Should login keys that can disrupt > > > > I/O on their change > > > > be allowed to be non-LO ? > > > > > > > > Thanks, > > > > Santosh > > > > > > > > Black_David@emc.com wrote: > > > > > > > > > > Without getting into the technical details of the > > > > > discussion, I have a couple of observations: > > > > > > > > > > (A) The issue of whether to allow mode page > > > > > access to and modification of iSCSI > > > ers > > > > > will need to be taken up at the interim > > > > > meeting. IMHO, access seems like a good > > > > > idea, so that SCSI-generic code that doesn't > > > > > know specifically about iSCSI can find > > > > > what it expects where it expects it, but > > > > > I'm unsure about modification because it > > > > > may carry a risk of code that's > > > naware > > > > > getting something wrong. The mode page > > > > > commands should be transparent to iSCSI. > > > > > > > > > > (B) The mode page and text key mechanisms have > > > > > to access the same data. Section 3 of the > > > > > -06 version says this, but needs some > > > > > editing > > > > > to enforce it by using "MUST" or its > > > > > equivalent > > > > > (cf. RFC 2119). This is to prevent an > > > > > implementation from having two instances of > > > > > the same parameter - one for the mode page and > > > > > one for the text keys - which would be a bad > > > > > thing. > > > > > > > > > > --David > From owner-ips@ece.cmu.edu Sun May 13 07:16:22 2001 Received: from ece.cmu.edu ([128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id HAA09285 for ; Sun, 13 May 2001 07:16:22 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f4D9A7W24830 for ips-outgoing; Sun, 13 May 2001 05:10:07 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from gate.internaut.com ([64.38.134.101]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f4D9A4H24826 for ; Sun, 13 May 2001 05:10:05 -0400 (EDT) Received: from e1kj2 (e1kj2 [64.38.134.109]) by gate.internaut.com (8.10.2/8.10.2) with SMTP id f4D8uNi05643; Sun, 13 May 2001 01:56:23 -0700 From: "Bernard Aboba" To: "Theodore Tso" Cc: , Subject: RE: iSCSI Security rough consensus Date: Sun, 13 May 2001 02:09:54 -0700 Message-ID: MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit X-Priority: 3 (Normal) X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook IMO, Build 9.0.2416 (9.0.2910.0) Importance: Normal In-Reply-To: <20010504212512.A10115@think.thunk.org> X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4133.2400 Sender: owner-ips@ece.cmu.edu Precedence: bulk Content-Transfer-Encoding: 7bit >P.S. Yes, I know that in fact there will need to be some >specifications written to indicate how to get from the SRP or CHAP >keying information to the low-level details of ESP, probably using an >AES cipher suite for speed. The hope was to keep this part small and >simple enough that that it wouldn't actually be an explicit key >management layer ala IKE and KINK, although of course it would have to >perform at least a little of such functions. I wouldn't use "CHAP" and "keying information" in the same sentence. CHAP can't be use to derive credible keys, and it's quite vulnerable to dictionary attack. So it's one of the least desirable choices. The "specifications" for SRP keying, while simpler than IKE, will probably end up deriving something quite similar to the keying material needed for Phase 1 and Phase 2 SAs, unless we can convince ourselves that we really only need the equivalent of Phase 1. And re-key is probably required, too. But before we go down this road, I would *really* like to see a requirements document, spelling out what is needed and why in some detail. Not being a SCSI expert, it's sometimes hard to separate what iSCSI requires from what folks think IPSEC can deliver. If the requirements and justification were laid out in detail, I suspect the road ahead would be more obvious. Note that if the issue is purely password auth, then aggressive mode may be a potential answer. With SRP the identities are exposed anyway so one could hardly object to agresive mode on that score. If the issue is access to code, well there are open source IKE implementations at this point. SRP is actually slower than IKE computationally, so speed can't be an argument. From owner-ips@ece.cmu.edu Mon May 14 00:58:06 2001 Received: from ece.cmu.edu ([128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id AAA19470 for ; Mon, 14 May 2001 00:58:06 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f4E2uSB03045 for ips-outgoing; Sun, 13 May 2001 22:56:28 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from chmls05.mediaone.net (chmls05.mediaone.net [24.147.1.143]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f4E2uNH03035 for ; Sun, 13 May 2001 22:56:23 -0400 (EDT) Received: from cs.uchicago.edu (h00008649aa79.ne.mediaone.net [24.91.82.73]) by chmls05.mediaone.net (8.11.1/8.11.1) with ESMTP id f4E2tkx23004 for ; Sun, 13 May 2001 22:55:46 -0400 (EDT) Message-Id: <200105140255.f4E2tkx23004@chmls05.mediaone.net> To: ips@ece.cmu.edu Subject: Re: iSCSI : Review Feedback on iSCSI Rev 06 In-Reply-To: Message from Santosh Rao of "Mon, 07 May 2001 17:33:20 PDT." <3AF73ED0.E6CB535F@cup.hp.com> References: <200105040201.TAA00579@hpcuhe.cup.hp.com> <20010507130209.0907F94006@sandmail.sandburst.com> <3AF73ED0.E6CB535F@cup.hp.com> Date: Sun, 13 May 2001 22:54:29 -0400 From: Stephen Bailey Sender: owner-ips@ece.cmu.edu Precedence: bulk Santosh, > What I was pointing out was that this upper limit needs to be lower than > DataPDULength, failing which fragmentation can occur. I see what you're saying now, but I don't agree that the iSCSI spec requires this fragmentation. Data PDUs are not NOP, Text or Login[Response] PDUs. The DataPDULength limit applies only to data PDU length (or, we should change the name to PDULength). The most important point to make is that both the initiator and the target be able to control the total size of any PDU sequence received (therefore, sent) during an operational session. If we want to punt completely on iterators for SendTarget response and use transport-level flow control to handle an arbitrary amount of data (i.e. no iterator for SendTarget), so be it. However, it is essential that if we do this, SendTarget be prohibited during operational sessions. Outside of a SendTarget interaction, I do not see any way that an iSCSI-specified text interaction would result in a gush larger than EITHER 4K OR the minimum DataPDULength of 512 bytes (not that this matters, as mentioned above). However, we do allow extended interactions with the X-*, and in order to ensure that these extended interactions function properly across implementations, we need to set a bound on the resources that the transport implementation must provide for these interactions. > This is the way the FC-GS-2/FC-GS-3 GID_FT command operates and it > allows for a faster target discovery than the iterative use of other > FC-GS-2/GS-3 name server queries such as GA_NXT (GAN). Do people actually use GID_FT? It doesn't seem wise. I always used GA_NXT, and it seemed that all the other FCP implementors I asked did as well (it's been a while). The problem is precisely analogous to what you see FCP. Data from an operation like GID_FT is going to go into buffers which are allocated locally to the FCP driver. This pool is going to be sized based upon operational assumptions (e.g. # of outstanding commands), and if you ask the name server for the entire name list at once, you're probably going to consume all those buffers and engage link-layer flow control before you can wade through the results. In a fabric, this can clog everything up pretty good. Fortunately, in the iSCSI/TCP case, engaging flow control like this will not block the channel, so it's actually marginally better to do this in iSCSI than in FCP. As long as you are not performing storage operations, AND the target is somewhat sophisiticated about returning the data (big IF---it must not consume all it's local resources trying to SEND the target list data which is going to drizzle out slowly under TCP flow control), everybody will get along just fine. This underscores another argument for iterators. For large targets, simply SENDING the entire target list at once might be a very resource intensive operation. You could suggest that the target just send a little at a time, and that's sorta OK, except that the initiator has to set some timeout on receiving the response, and if the targets are allowed to drizzle the data at an arbitrary rate, it's hard to chose a timeout value. I can argue both sides of this (I have, right?), but given that nobody besides me seems to believe in iterators, we at least have to make sure that the big gush approach works properly. Steph From owner-ips@ece.cmu.edu Mon May 14 00:59:37 2001 Received: from ece.cmu.edu ([128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id AAA19482 for ; Mon, 14 May 2001 00:59:37 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f4E2uPP03043 for ips-outgoing; Sun, 13 May 2001 22:56:25 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from chmls05.mediaone.net (chmls05.mediaone.net [24.147.1.143]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f4E2uMH03034 for ; Sun, 13 May 2001 22:56:23 -0400 (EDT) Received: from cs.uchicago.edu (h00008649aa79.ne.mediaone.net [24.91.82.73]) by chmls05.mediaone.net (8.11.1/8.11.1) with ESMTP id f4E2tjx22995 for ; Sun, 13 May 2001 22:55:45 -0400 (EDT) Message-Id: <200105140255.f4E2tjx22995@chmls05.mediaone.net> To: ips@ece.cmu.edu Subject: Re: iSCSI: Canonical Targets In-Reply-To: Message from Mark Bakke of "Fri, 11 May 2001 16:04:24 CDT." <3AFC53D8.CD8A8BC4@cisco.com> References: <3AFC53D8.CD8A8BC4@cisco.com> Date: Sun, 13 May 2001 22:54:28 -0400 From: Stephen Bailey Sender: owner-ips@ece.cmu.edu Precedence: bulk Mark, This is mostly fine with me. However, I KNOW implementors are going to gripe about having to know the target name before performing an operational login. In general, the spec seems a bit capricious about when implementors just have to suck it up and do the hard thing, versus when we try to make it easy for them. Realistically, it's not a big deal to learn the name (just an extra login), but most end points are going to have a single target, and from a configuration standpoint endpoint (and, hence the target) is going to be specified by transport coordinates (address, port). This extra login for discovery is simply an irritant. Also, I think this part still needs a little more specification work: > 6. We can further specify the order in which the SendTargets response > fields are returned, to simplify things further, e.g. each target > in the SendTargets response MUST return these fields in this order: > > - A TargetName= field > - A TargetAlias= field (value left blank if there's no alias) > - One or more TargetAddress= fields > - Any vendor-specific fields (ignored by standard initiators) I can only guess that you are implying that target descriptions following this rule may be tiled arbitrarily into Text PDUs returned from the target? If so, we should specify that. > 1. Should a non-canonical target respond to a SendTargets command? I'd say no. Given the nature of the interaction, why bother? Now, I know the iSCSI philosophy seems to be `if it's not complete gibberish, why NOT bother?' but that results in complicated implementation. Specifically, each end (target and initiator) should be able to audit the other for correctness when possible. Hard rules like `thou shalt not issue a SendTargets in an operational session' make the auditing a zillion times easier. To this end, I would go a step further and allow only a single outstanding SendTargets exchange in a target discovery session. Steph From owner-ips@ece.cmu.edu Mon May 14 11:45:22 2001 Received: from ece.cmu.edu ([128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id LAA15694 for ; Mon, 14 May 2001 11:45:22 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f4EDHH116563 for ips-outgoing; Mon, 14 May 2001 09:17:17 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from dogwood.cisco.com (dogwood.cisco.com [161.44.11.19]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f4EDH1H16553 for ; Mon, 14 May 2001 09:17:01 -0400 (EDT) Received: from cisco.com (mbakke@mbakke-lnx.cisco.com [161.44.68.87]) by dogwood.cisco.com (8.8.6 (PHNE_14041)/CISCO.SERVER.1.2) with ESMTP id JAA15775; Mon, 14 May 2001 09:16:32 -0400 (EDT) Message-ID: <3AFFDA17.10AE8DCC@cisco.com> Date: Mon, 14 May 2001 08:13:59 -0500 From: Mark Bakke X-Mailer: Mozilla 4.72 [en] (X11; U; Linux 2.2.16-3.uid32 i686) X-Accept-Language: en, de MIME-Version: 1.0 To: james.smart@trebia.com CC: julian_satran@il.ibm.com, ips@ece.cmu.edu Subject: Re: iSCSI: Canonical Targets References: <3AFD2FA1.F033A4DE@trebia.com> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-ips@ece.cmu.edu Precedence: bulk Content-Transfer-Encoding: 7bit James- Your argument for interoperability is correct. Please see my comments below on the other discovery mechanisms. James Smart wrote: > > I'm in agreement with Mark, that canonical target support is mandatory. The > key is interoperability. The initiator must be mandated to probe the > canonical target and not assume a single target device. If all initiators do > not support this type of probing, then interoperability goes out the window. > Mark's effort, mandates the support on the target device - which indirectly > puts the mandate on the initiator to check, and makes sure there are not > confusing answers from devices that otherwise would not respond to the > canonical target. Correct. The danger in not supporting the canonical target on all devices is that initiators may then choose not to look at it. Even in an environment without proxies and gateways, any medium-sized disk array should be able to support more than one logical target. > Is this replacing discovery - perhaps in some ways. The real issue is > allowing i/o traffic to be rerouted to non-IANA TCP port numbers, which is a > critical need of the proxy/routers/etc, which have little left in the socket > 4-tuple to classify on if rerouting is not allowed.. I would expect the Actually, by defining several logical targets, a single IP address + IANA TCP port can be shared by all. This is a direction I suspect many implementors will take. > "discovery" (SLP, uPnP,etc) mechanisms would contain the "targets" that the > proxy/router/etc, but the expectation is that they would all point at the > well-known IANA TCP port number. The canonical processing simply allows the The SLP template includes the TCP port number on which the target is listening. If a target is discovered using SLP, the initiator could skip using SendTargets on its address. Although we are not considering using it, SSDP (used by UPnP) returns a URL, which could include a TCP port number. > target device to shift session-specific traffic to target-assigned port > numbers that can aid it's classification schemes. -- Mark > -- James > > julian_satran@il.ibm.com wrote: > > > Mark, > > > > I am not sure that we want to have your "canonical" target as a mandatory > > construct. > > It is good for proxies, routers, portals etc.? Don't the have better > > mechanisms for discovery. > > But why should you force it on some small box that has the name wired and > > printed on it's label. > > To reach the canonical you have to go through all the "login etc.". > > > > I think that all goes back to asking - why should we use iSCSI for > > discovery? > > > > SLP, UPnP, Salutation, Jini all attempt discovery using a a general form > > (good for iSCSI, Lpr printers etc.) > > Can't we take the same path? > > > > Julo > > > > Mark Bakke on 12-05-2001 00:04:24 > > > > Please respond to Mark Bakke > > > > To: IPS > > cc: > > Subject: iSCSI: Canonical Targets > > > > It was pointed out during the interim meeting that the > > canonical target is defined somewhat ambiguously, and may > > be a bit too flexible for interoperability purposes. This > > message is a stake in the ground for defining this a little > > more tightly. > > > > So here are the new canonical target rules. Please let me > > know if there are major problems with any of them. I tried > > to define them a-la-carte, to make it easier to pick out any > > that might cause trouble. > > > > 1. Each iSCSI implementation MUST include a canonical target. > > > > 2. A canonical target MUST be accessible at the default, IANA- > > assigned TCP port on each IP address on which the iSCSI > > implementation is listening for iSCSI connections. > > > > 3. A canonical target MUST NOT be used for SCSI commands. > > > > A canonical target is an iSCSI construct only, and does not > > have a corresponding SCSI device. This means it may not > > be used to access Logical Units. > > > > A session created to a canonical target is a discovery session > > only, and once in full feature phase, is used only for text > > commands and asynchronous messages. (Do any other commands make > > sense)? > > > > 4. A device containing a single target MUST provide both the > > canonical target and the real target. (This is implied by the > > above requirements). > > > > An initiator connecting to such a device using only its IP address > > would first connect to the canonical target, and use SendTargets > > to obtain the iSCSI name of the real target. It would then create > > a separate session to the real target. Essentially, this means > > there's nothing special about a single-target device. > > > > 5. An iSCSI device MUST provide a unique iSCSI name for each of its > > targets. Using the canonical target as a nameless iSCSI target > > is not supported. > > > > 6. We can further specify the order in which the SendTargets response > > fields are returned, to simplify things further, e.g. each target > > in the SendTargets response MUST return these fields in this order: > > > > - A TargetName= field > > - A TargetAlias= field (value left blank if there's no alias) > > - One or more TargetAddress= fields > > - Any vendor-specific fields (ignored by standard initiators) > > > > The above rules will make the behavior of various target implementations > > identical, regardless of the number of interfaces or targets they > > support. This will reduce the number of end-cases that initiator > > implementations will have to handle. > > > > If we agree on these, we can edit them into the iSCSI and NDT documents. > > > > Additional questions to send input on: > > > > 1. Should a non-canonical target respond to a SendTargets command? > > > > 2. If so, should it respond only with addresses for its own target, > > or should it respond with other targets, as a canonical target > > might do? > > > > 3. If not, the initiator must connect to a canonical target to find > > the other addresses of a target to which it is already connected; > > is the information it has sufficient to do so? (I think the answer > > is yes, given canonical requirement #2 above). > > > > -- > > Mark A. Bakke > > Cisco Systems > > mbakke@cisco.com > > 763.398.1054 -- Mark A. Bakke Cisco Systems mbakke@cisco.com 763.398.1054 From owner-ips@ece.cmu.edu Mon May 14 11:45:52 2001 Received: from ece.cmu.edu ([128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id LAA15710 for ; Mon, 14 May 2001 11:45:52 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f4EDdex18380 for ips-outgoing; Mon, 14 May 2001 09:39:40 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from dogwood.cisco.com (dogwood.cisco.com [161.44.11.19]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f4EDddH18376 for ; Mon, 14 May 2001 09:39:39 -0400 (EDT) Received: from ahganemw2k (dhcp-161-44-68-139.cisco.com [161.44.68.139]) by dogwood.cisco.com (8.8.6 (PHNE_14041)/CISCO.SERVER.1.2) with SMTP id JAA25115 for ; Mon, 14 May 2001 09:39:32 -0400 (EDT) From: "Ayman Ghanem" To: Subject: RE: iSCSI: Canonical Targets Date: Mon, 14 May 2001 08:38:25 -0500 Message-ID: MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit X-Priority: 3 (Normal) X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook IMO, Build 9.0.2416 (9.0.2910.0) In-Reply-To: <3AFC53D8.CD8A8BC4@cisco.com> Importance: Normal X-MimeOLE: Produced By Microsoft MimeOLE V5.00.2919.6700 Sender: owner-ips@ece.cmu.edu Precedence: bulk Content-Transfer-Encoding: 7bit I think a non-canonical target should respond to SendTargets. Some implementations may chose to drop the session to the canonical target after connecting to the target devices. If new devices become available, it is easier for a non-canonical target to respond with target names, rather than having the initiator relogin to the canonical target. -Ayman > > Additional questions to send input on: > > 1. Should a non-canonical target respond to a SendTargets command? > > 2. If so, should it respond only with addresses for its own target, > or should it respond with other targets, as a canonical target > might do? > > 3. If not, the initiator must connect to a canonical target to find > the other addresses of a target to which it is already connected; > is the information it has sufficient to do so? (I think the answer > is yes, given canonical requirement #2 above). > > > -- > Mark A. Bakke > Cisco Systems > mbakke@cisco.com > 763.398.1054 > From owner-ips@ece.cmu.edu Mon May 14 13:06:49 2001 Received: from ece.cmu.edu ([128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id NAA18255 for ; Mon, 14 May 2001 13:06:48 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f4EFEte26092 for ips-outgoing; Mon, 14 May 2001 11:14:55 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from e31.bld.us.ibm.com (e31.co.us.ibm.com [32.97.110.129]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f4EFErH26087 for ; Mon, 14 May 2001 11:14:53 -0400 (EDT) Received: from westrelay02.boulder.ibm.com (westrelay02.boulder.ibm.com [9.99.140.23]) by e31.bld.us.ibm.com (8.9.3/8.9.3) with ESMTP id LAA11176 for ; Mon, 14 May 2001 11:07:20 -0400 Received: from f3n42e (d03nm042h.boulder.ibm.com [9.99.140.42]) by westrelay02.boulder.ibm.com (8.11.1/NCO v4.96.1.0) with ESMTP id f4EFEXd150320 for ; Mon, 14 May 2001 09:14:33 -0600 Importance: Normal Subject: Re: iSCSI: Canonical Targets To: "Julian Satran" Cc: ips@ece.cmu.edu From: "Jim Hafner" Date: Mon, 14 May 2001 08:14:31 -0700 Message-ID: X-MIMETrack: Serialize by Router on D03NM042/03/M/IBM(Release 5.0.6 |December 14, 2000) at 05/14/2001 08:14:33 AM MIME-Version: 1.0 Content-type: text/plain; charset=us-ascii Sender: owner-ips@ece.cmu.edu Precedence: bulk Julo, You object to the term "canonical"? My dictionary says canonical is "conforming to a general rule", but perhaps that isn't the best term. Would "well-known" would be better? Jim Hafner Julian Satran/Haifa/IBM@IBMIL@ece.cmu.edu on 05-12-2001 09:05:00 AM Sent by: owner-ips@ece.cmu.edu To: ips@ece.cmu.edu cc: Subject: Re: iSCSI: Canonical Targets Why would any discovery service be wiser/stupider than the mechanism we are having in N&D? If the issue of ports is widespread then others have it too and discovery may include all the nice things we have in iSCSI. I don't see any reason iSNS or an LDAP repository can't contain them. My only claim is that this adds weight to iSCSI. As for interoperability - I claim that adding features makes interoperability an issue. Removing them does not. The whole "canonical" (why canonical - there is nothing canonical about it) target idea is good but in another realm - I don't think it belongs to iSCSI - it belongs to discovery be it specific or general. I can leave with the mechanism as it is - but I really feel that we are on the wrong track. Regards, Julo James Smart on 12-05-2001 15:42:09 Please respond to james.smart@trebia.com To: Julian Satran/Haifa/IBM@IBMIL, Mark Bakke cc: ips@ece.cmu.edu Subject: Re: iSCSI: Canonical Targets I'm in agreement with Mark, that canonical target support is mandatory. The key is interoperability. The initiator must be mandated to probe the canonical target and not assume a single target device. If all initiators do not support this type of probing, then interoperability goes out the window. Mark's effort, mandates the support on the target device - which indirectly puts the mandate on the initiator to check, and makes sure there are not confusing answers from devices that otherwise would not respond to the canonical target. Is this replacing discovery - perhaps in some ways. The real issue is allowing i/o traffic to be rerouted to non-IANA TCP port numbers, which is a critical need of the proxy/routers/etc, which have little left in the socket 4-tuple to classify on if rerouting is not allowed.. I would expect the "discovery" (SLP, uPnP,etc) mechanisms would contain the "targets" that the proxy/router/etc, but the expectation is that they would all point at the well-known IANA TCP port number. The canonical processing simply allows the target device to shift session-specific traffic to target-assigned port numbers that can aid it's classification schemes. -- James julian_satran@il.ibm.com wrote: > Mark, > > I am not sure that we want to have your "canonical" target as a mandatory > construct. > It is good for proxies, routers, portals etc.? Don't the have better > mechanisms for discovery. > But why should you force it on some small box that has the name wired and > printed on it's label. > To reach the canonical you have to go through all the "login etc.". > > I think that all goes back to asking - why should we use iSCSI for > discovery? > > SLP, UPnP, Salutation, Jini all attempt discovery using a a general form > (good for iSCSI, Lpr printers etc.) > Can't we take the same path? > > Julo > > Mark Bakke on 12-05-2001 00:04:24 > > Please respond to Mark Bakke > > To: IPS > cc: > Subject: iSCSI: Canonical Targets > > It was pointed out during the interim meeting that the > canonical target is defined somewhat ambiguously, and may > be a bit too flexible for interoperability purposes. This > message is a stake in the ground for defining this a little > more tightly. > > So here are the new canonical target rules. Please let me > know if there are major problems with any of them. I tried > to define them a-la-carte, to make it easier to pick out any > that might cause trouble. > > 1. Each iSCSI implementation MUST include a canonical target. > > 2. A canonical target MUST be accessible at the default, IANA- > assigned TCP port on each IP address on which the iSCSI > implementation is listening for iSCSI connections. > > 3. A canonical target MUST NOT be used for SCSI commands. > > A canonical target is an iSCSI construct only, and does not > have a corresponding SCSI device. This means it may not > be used to access Logical Units. > > A session created to a canonical target is a discovery session > only, and once in full feature phase, is used only for text > commands and asynchronous messages. (Do any other commands make > sense)? > > 4. A device containing a single target MUST provide both the > canonical target and the real target. (This is implied by the > above requirements). > > An initiator connecting to such a device using only its IP address > would first connect to the canonical target, and use SendTargets > to obtain the iSCSI name of the real target. It would then create > a separate session to the real target. Essentially, this means > there's nothing special about a single-target device. > > 5. An iSCSI device MUST provide a unique iSCSI name for each of its > targets. Using the canonical target as a nameless iSCSI target > is not supported. > > 6. We can further specify the order in which the SendTargets response > fields are returned, to simplify things further, e.g. each target > in the SendTargets response MUST return these fields in this order: > > - A TargetName= field > - A TargetAlias= field (value left blank if there's no alias) > - One or more TargetAddress= fields > - Any vendor-specific fields (ignored by standard initiators) > > The above rules will make the behavior of various target implementations > identical, regardless of the number of interfaces or targets they > support. This will reduce the number of end-cases that initiator > implementations will have to handle. > > If we agree on these, we can edit them into the iSCSI and NDT documents. > > Additional questions to send input on: > > 1. Should a non-canonical target respond to a SendTargets command? > > 2. If so, should it respond only with addresses for its own target, > or should it respond with other targets, as a canonical target > might do? > > 3. If not, the initiator must connect to a canonical target to find > the other addresses of a target to which it is already connected; > is the information it has sufficient to do so? (I think the answer > is yes, given canonical requirement #2 above). > > -- > Mark A. Bakke > Cisco Systems > mbakke@cisco.com > 763.398.1054 From owner-ips@ece.cmu.edu Mon May 14 13:09:04 2001 Received: from ece.cmu.edu ([128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id NAA18254 for ; Mon, 14 May 2001 13:06:48 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f4EFC6i25814 for ips-outgoing; Mon, 14 May 2001 11:12:06 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from e34.bld.us.ibm.com (e34.co.us.ibm.com [32.97.110.132]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f4EFC4H25809 for ; Mon, 14 May 2001 11:12:04 -0400 (EDT) Received: from westrelay02.boulder.ibm.com (westrelay02.boulder.ibm.com [9.99.140.23]) by e34.bld.us.ibm.com (8.9.3/8.9.3) with ESMTP id KAA08990 for ; Mon, 14 May 2001 10:54:32 -0400 Received: from f3n42e (d03nm042h.boulder.ibm.com [9.99.140.42]) by westrelay02.boulder.ibm.com (8.11.1/NCO v4.96.1.0) with ESMTP id f4EFBbd132020 for ; Mon, 14 May 2001 09:11:38 -0600 Importance: Normal Subject: Re: iSCSI: Canonical Targets To: ips@ece.cmu.edu From: "Jim Hafner" Date: Mon, 14 May 2001 08:11:36 -0700 Message-ID: X-MIMETrack: Serialize by Router on D03NM042/03/M/IBM(Release 5.0.6 |December 14, 2000) at 05/14/2001 08:11:37 AM MIME-Version: 1.0 Content-type: text/plain; charset=us-ascii Sender: owner-ips@ece.cmu.edu Precedence: bulk Steph, One argument for SendTargets on non-canonical targets would be the following (and might help address your iterator concerns). A canonical target *could* respond to SendTargets with list of actual targets but only provide a partial list of addressess and ports (an abbreviated list). Then each actual target could respond to SendTargets with just itself in its list and its complete list of all addresses and ports it responds to. This gives a heirarchical local discovery mechanism. N&DT hasn't fully scoped these issues, which is why this discussion was started. I don't think there is a requirement in what Mark wrote that the response to SendTargets to any target (canonical or actual) has to report complete knowledge of everything known to that box. I have a question for you (and the list) about a long Text response (as might be needed for SendTargets). Can the Text response come in multiple PDUs? In particular, can one "key=value" pair span multiple PDUs? If so (for either question), what exactly is the spec'ed method? And if so, does this help address your resources concerns? (The target can send the response is chunks.) Jim Hafner Stephen Bailey @ece.cmu.edu on 05-13-2001 07:54:28 PM Sent by: owner-ips@ece.cmu.edu To: ips@ece.cmu.edu cc: Subject: Re: iSCSI: Canonical Targets Mark, This is mostly fine with me. However, I KNOW implementors are going to gripe about having to know the target name before performing an operational login. In general, the spec seems a bit capricious about when implementors just have to suck it up and do the hard thing, versus when we try to make it easy for them. Realistically, it's not a big deal to learn the name (just an extra login), but most end points are going to have a single target, and from a configuration standpoint endpoint (and, hence the target) is going to be specified by transport coordinates (address, port). This extra login for discovery is simply an irritant. Also, I think this part still needs a little more specification work: > 6. We can further specify the order in which the SendTargets response > fields are returned, to simplify things further, e.g. each target > in the SendTargets response MUST return these fields in this order: > > - A TargetName= field > - A TargetAlias= field (value left blank if there's no alias) > - One or more TargetAddress= fields > - Any vendor-specific fields (ignored by standard initiators) I can only guess that you are implying that target descriptions following this rule may be tiled arbitrarily into Text PDUs returned from the target? If so, we should specify that. > 1. Should a non-canonical target respond to a SendTargets command? I'd say no. Given the nature of the interaction, why bother? Now, I know the iSCSI philosophy seems to be `if it's not complete gibberish, why NOT bother?' but that results in complicated implementation. Specifically, each end (target and initiator) should be able to audit the other for correctness when possible. Hard rules like `thou shalt not issue a SendTargets in an operational session' make the auditing a zillion times easier. To this end, I would go a step further and allow only a single outstanding SendTargets exchange in a target discovery session. Steph From owner-ips@ece.cmu.edu Mon May 14 14:29:16 2001 Received: from ece.cmu.edu ([128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id OAA20761 for ; Mon, 14 May 2001 14:29:16 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f4EGPH501750 for ips-outgoing; Mon, 14 May 2001 12:25:17 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from palrel2.hp.com (palrel2.hp.com [156.153.255.234]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f4EGPFH01745 for ; Mon, 14 May 2001 12:25:15 -0400 (EDT) Received: from hpcuhe.cup.hp.com (hpcuhe.cup.hp.com [15.0.80.203]) by palrel2.hp.com (Postfix) with ESMTP id 234BB1344 for ; Mon, 14 May 2001 09:25:14 -0700 (PDT) Received: from cup.hp.com (santoshr@hpindhhm.cup.hp.com [15.8.80.197]) by hpcuhe.cup.hp.com (8.9.3 (PHNE_18979)/8.9.3 SMKit7.02) with ESMTP id JAA13207 for ; Mon, 14 May 2001 09:25:09 -0700 (PDT) Message-ID: <3B0006EC.D304C098@cup.hp.com> Date: Mon, 14 May 2001 09:25:16 -0700 From: Santosh Rao Organization: Hewlett Packard, Cupertino. X-Mailer: Mozilla 4.7 [en] (X11; U; HP-UX B.11.00 9000/778) X-Accept-Language: en MIME-Version: 1.0 To: ips@ece.cmu.edu Subject: Re: iSCSI : Bridging missing CmdSNs and Abort I/O Error recovery References: Content-Type: multipart/mixed; boundary="------------5B8AD05F68FA2EA2B1BD5D5E" Sender: owner-ips@ece.cmu.edu Precedence: bulk This is a multi-part message in MIME format. --------------5B8AD05F68FA2EA2B1BD5D5E Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Julian, I have re-read your previous answer and I may be missing something, but, I don't see how it addresses my concern (?). Per your mail : "To abort safely a task for which the task abort answer is "Command Not Received Yet" the initiator must issue another abort command on the same connection as the original command unless this connection was logged out in which case it may send it on any connection." I'm pointing out that the above may not be sufficient since the issue spans beyond that. If the target HAD received the command by the time it received the task mgmt command on another connection, it would not respond with "Command not received yet". In this case, the initiator would not be required to re-send the Abort Task on the same connection as the original command. In such a scenario, if there were PDUs in-flight from target to initiator on the original connection, these could arrive after the task mgmt response [which is on the 2nd connection]. Such an Abort Task cleanup does not reliably allow the initiator to free its task tag and other resources, since PDUs for that task continue to arrive on the original connection after the initiator has completed task clean-up using Abort Task & released the task resources. - Santosh julian_satran@il.ibm.com wrote: > > please reread carefully my previous answer. Julo > > Santosh Rao on 09-05-2001 19:37:41 > > Please respond to Santosh Rao > > To: IPS Reflector > cc: > Subject: Re: iSCSI : Bridging missing CmdSNs and Abort I/O Error recovery > > > julian_satran@il.ibm.com wrote: > > > > > We could add the RefCmdSN and it may help plug-in the hole but unless > the > > > > Abort is ussued on the same connection as the original command we > can't > > > be > > > > sure that the old-one will not pop-up (as we enable ExpCmdSN to move > on > > > we > > > > don't have even the 2**31-1 protection bracket :-)). Thus sending in > > > > another nop/abort on the same connection is still required. > > > > > > > > To simplify the whole process I will: > > > > > > > > a - add RefCmdSN to the Task Management > > > > b - add a command not received yet to the answers > > > > > > > > c - add a part to 7 reading: > > > > > > > > 1.1 How to Abort Safely a Command that Was Not Received > > > > > > > > To abort safely a task for which the task abort answer is "Command > Not > > > > Received Yet" the initiator must issue another abort command on > the > > > same > > > > connection as the original command unless this connection was > logged > > > out > > > > in which case it may send it on any connection. > > Julian, > > The above may not be sufficient, since a requirement for the Abort Task > completion is that both the initiator & target can safely re-use their > task tags and other resources following completion of the Abort Task. If > the Abort Task is sent on another connection than the connection used > for the original command [& that connection has not been logged out], > then, there may be some in-flight PDUs from the target to the initiator > for that command on the connection. > > Sending an Abort Task on another connection may cause the initiator to > receive the Abort Task response indicating success on the other > connection which then results in initiator freeing up its task tag and > other resources used for that command. Thereafter, the stale PDUs on the > original connection can show up at the initiator, causing problems. > > I would suggest that Abort Task be always sent on the same connection as > the command, unless the connection used for that command has been logged > out. Not doing so implies the initiator cannot safely re-use its task > tag and other resources that were tied up for that I/O. > > Regards, > Santosh > - santoshr.vcf --------------5B8AD05F68FA2EA2B1BD5D5E Content-Type: text/x-vcard; charset=us-ascii; name="santoshr.vcf" Content-Description: Card for Santosh Rao Content-Disposition: attachment; filename="santoshr.vcf" Content-Transfer-Encoding: 7bit begin:vcard n:Rao;Santosh tel;work:408-447-3751 x-mozilla-html:FALSE org:Hewlett Packard, Cupertino.;SISL adr:;;19420, Homestead Road, M\S 43LN, ;Cupertino.;CA.;95014.;USA. version:2.1 email;internet:santoshr@cup.hp.com title:Software Design Engineer x-mozilla-cpt:;21088 fn:Santosh Rao end:vcard --------------5B8AD05F68FA2EA2B1BD5D5E-- From owner-ips@ece.cmu.edu Mon May 14 16:45:49 2001 Received: from ece.cmu.edu ([128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id QAA24346 for ; Mon, 14 May 2001 16:45:48 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f4EIZqE13427 for ips-outgoing; Mon, 14 May 2001 14:35:52 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from mailcity.com (fes.whowhere.com [209.185.123.154]) by ece.cmu.edu (8.11.0/8.10.2) with SMTP id f4EIZoH13423 for ; Mon, 14 May 2001 14:35:50 -0400 (EDT) Received: from Unknown/Local ([?.?.?.?]) by mailcity.com; Mon May 14 11:35:34 2001 To: "iSCSI" Date: Mon, 14 May 2001 11:35:34 -0700 From: "m nima" Message-ID: Mime-Version: 1.0 X-Sent-Mail: on Reply-To: m_nima_1@lycos.com X-Mailer: MailCity Service Subject: iSCSI: mandating markers (was Nailing down CRC-32C) X-Sender-Ip: 63.70.210.30 Organization: Lycos Mail (http://mail.lycos.com:80) Content-Type: text/plain; charset=us-ascii Content-Language: en Content-Transfer-Encoding: 7bit Sender: owner-ips@ece.cmu.edu Precedence: bulk Content-Transfer-Encoding: 7bit Marker - I'm confused now. Assuming the server side uses 10G and has marker in HW, What is the fate of its client running 1G/S and has a SW marker? Repeat of garbled comment (sorry) - I think the key for iSCSI success is to make sure it's low weight protocol that scales well even to 10Gb/S and beyond. For that, everything needs to be computable at the receiver, on a per TCP segment basis and not span multiple segments! Adding markers, making security mandatory to implement and use of Data CRC (as proposed now where it is not friendly with out-of-order) leaves iSCSI far behind. SW support - well today you have to provide your own TCP stack in the MS environment. Every vendor that wishes can implement these changes in his stack. So I don't see the point about use of legacy. I believe, the standardization effort HAS to enable a new way, such that one could do a better job (like you said - so that iSCSI gets cheaper and better than FC), and framing is a better way for the future Nima Get 250 color business cards for FREE! http://businesscards.lycos.com/vp/fastpath/ From owner-ips@ece.cmu.edu Mon May 14 18:23:41 2001 Received: from ece.cmu.edu ([128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id SAA26036 for ; Mon, 14 May 2001 18:23:41 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f4EKbg823919 for ips-outgoing; Mon, 14 May 2001 16:37:42 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from sandmail.sandburst.com (sandburst-gw.bstn-gw02.ma.us.intelilink.net [216.57.129.34]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f4EKbeH23898 for ; Mon, 14 May 2001 16:37:40 -0400 (EDT) Received: from cs.uchicago.edu (dynamite-38.sandburst.com [172.16.5.38]) by sandmail.sandburst.com (Postfix) with ESMTP id 0A8CC94009 for ; Mon, 14 May 2001 16:37:21 -0400 (EDT) To: ips@ece.cmu.edu Subject: Re: iSCSI: Canonical Targets In-Reply-To: Message from Mark Bakke of "Mon, 14 May 2001 08:13:59 CDT." <3AFFDA17.10AE8DCC@cisco.com> References: <3AFD2FA1.F033A4DE@trebia.com> <3AFFDA17.10AE8DCC@cisco.com> Date: Mon, 14 May 2001 16:35:28 -0400 From: Stephen Bailey Message-Id: <20010514203721.0A8CC94009@sandmail.sandburst.com> Sender: owner-ips@ece.cmu.edu Precedence: bulk Mark, I don't disagree with the general thrust of your statement, but I must argue with: > Even in an environment without proxies and gateways, any > medium-sized disk array should be able to support more than one > logical target. Unless somebody's really building a medium sized disk array a) completely from scratch b) for iSCSI use only, this is unlikely. Existing disk arrays present all storage containers as LUNs. The most likely [credible] iSCSI targets will be existing storage arrays with iSCSI front-ends. This follows the ||SCSI->FCP pattern. The alternative interpretation, which I'm pretty sure is false, is that the back-end part of a storage controller is really simple so the serious iSCSI targets will not come from the same vendors (or groups) as existing, serious storage targets. Bridges are a different issue, but I've always felt (and you seem to agree) that the bridges just have to suck it up and, well, bridge, the semantic gaps between what's expected in a native initiator->target interaction and what you get with an initiator->bridge->target one. Steph From owner-ips@ece.cmu.edu Mon May 14 18:23:46 2001 Received: from ece.cmu.edu ([128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id SAA26047 for ; Mon, 14 May 2001 18:23:46 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f4EKq8p25313 for ips-outgoing; Mon, 14 May 2001 16:52:08 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from dogwood.cisco.com (dogwood.cisco.com [161.44.11.19]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f4EKq6H25304 for ; Mon, 14 May 2001 16:52:06 -0400 (EDT) Received: from cisco.com (mbakke@mbakke-lnx.cisco.com [161.44.68.87]) by dogwood.cisco.com (8.8.6 (PHNE_14041)/CISCO.SERVER.1.2) with ESMTP id QAA02556 for ; Mon, 14 May 2001 16:52:00 -0400 (EDT) Message-ID: <3B0044D6.21E72656@cisco.com> Date: Mon, 14 May 2001 15:49:26 -0500 From: Mark Bakke X-Mailer: Mozilla 4.72 [en] (X11; U; Linux 2.2.16-3.uid32 i686) X-Accept-Language: en, de MIME-Version: 1.0 To: IPS Subject: iSCSI: Iteration for SendTargets Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-ips@ece.cmu.edu Precedence: bulk Content-Transfer-Encoding: 7bit At the interim meeting we had discussed the fact that a SendTargets response can become rather large, given storage arrays with 10s of ports, and gateways that can represent many targets. Consider a single target on a 32-port storage array. This target can easily be expected to have a 40-character iSCSI name, 40-character alias, and 32 DNS-named addresses, with each DNS name being 30 characters long. Including the text key names, equals signs, null characters, and values, the response for this target would take: 12+40+32*(15+30)+13+40 = 1545 bytes Given a limitation of 4096 bytes on text responses, the fact that multiple targets can be returned in a response, and that the iSCSI name, alias, and DNS-named addresses in the response do not need to be as well-behaved as those in the example, it's easy to envision normal SendTargets responses of over 4k. There are different ways to handle this: 1. Restrict SendTargets commands and responses to the canonical target sessions, and handle them in software. Remove the text response limit for canonical target sessions, since they are not done in hardware anyway. 2. Add an iterator capability to SendTargets. Steph brought this up during the interim meeting. This will work well if #1 can't be done. There may be others, but these are the two I've thought about. The argument for #1 is that SendTargets is really a software function, and nobody is likely to actually implement it in hardware. It would use normal socket calls, so the length of the response would not be much of an issue for the initiator. The same would be true of a target if it used a normal TCP stack for canonical sessions; however, there may be implementations that do not plan to do this (speak up if this is a problem). If the argument for #1 does not hold, it would be necessary to implement some sort of iterator. Steph has already posted on this topic. An iterator would be easy enough to do for SendTargets, and we could put it in, but I want to make sure that #1 is exhausted first. Please let me know what you think. -- Mark A. Bakke Cisco Systems mbakke@cisco.com 763.398.1054 From owner-ips@ece.cmu.edu Mon May 14 18:23:48 2001 Received: from ece.cmu.edu ([128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id SAA26058 for ; Mon, 14 May 2001 18:23:48 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f4EKkal24795 for ips-outgoing; Mon, 14 May 2001 16:46:36 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from sandmail.sandburst.com (sandburst-gw.bstn-gw02.ma.us.intelilink.net [216.57.129.34]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f4EKkYH24787 for ; Mon, 14 May 2001 16:46:34 -0400 (EDT) Received: from cs.uchicago.edu (dynamite-38.sandburst.com [172.16.5.38]) by sandmail.sandburst.com (Postfix) with ESMTP id 789C894009 for ; Mon, 14 May 2001 16:46:34 -0400 (EDT) To: ips@ece.cmu.edu Subject: Re: iSCSI: Canonical Targets In-Reply-To: Message from "Jim Hafner" of "Mon, 14 May 2001 08:10:42 PDT." References: Date: Mon, 14 May 2001 16:44:41 -0400 From: Stephen Bailey Message-Id: <20010514204634.789C894009@sandmail.sandburst.com> Sender: owner-ips@ece.cmu.edu Precedence: bulk Jim, > I have a question for you (and the list) about a long Text response (as > might be needed for SendTargets). Can the Text response come in multiple > PDUs? As I interpret the spec (emphasis on interpret---I don't think the spec says anything clearly one way or another), yes. > In particular, can one "key=value" pair span multiple PDUs? Not as I interpret the spec. > And if so, does this help address your resources concerns? No. The problem I'm talking about is handling an unbounded TOTAL amount of data returned from a single request. It doesn't matter whether it comes in a single PDU or broken into a million, unless you can offer a finer grain of flow control for the million PDUs than for the one. In either case the data returned may a) clog the channel b) possibly consume resources necessary for other operational purposes. Again, if we disallow SendTargets (and other interactions with indeterminate, potentially large uncontrolled flows) in an operational login, we can allow the transport to provide the flow control. In an operational login we don't want to engage transport flow control in this way because it interferes with other operational communication. Steph From owner-ips@ece.cmu.edu Mon May 14 18:24:40 2001 Received: from ece.cmu.edu ([128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id SAA26083 for ; Mon, 14 May 2001 18:24:39 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f4EKXD923542 for ips-outgoing; Mon, 14 May 2001 16:33:13 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from dogwood.cisco.com (dogwood.cisco.com [161.44.11.19]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f4EKXBH23537 for ; Mon, 14 May 2001 16:33:11 -0400 (EDT) Received: from cisco.com (mbakke@mbakke-lnx.cisco.com [161.44.68.87]) by dogwood.cisco.com (8.8.6 (PHNE_14041)/CISCO.SERVER.1.2) with ESMTP id QAA08541; Mon, 14 May 2001 16:32:59 -0400 (EDT) Message-ID: <3B004061.E6019045@cisco.com> Date: Mon, 14 May 2001 15:30:25 -0500 From: Mark Bakke X-Mailer: Mozilla 4.72 [en] (X11; U; Linux 2.2.16-3.uid32 i686) X-Accept-Language: en, de MIME-Version: 1.0 To: Stephen Bailey CC: ips@ece.cmu.edu Subject: Re: iSCSI: Canonical Targets References: <3AFC53D8.CD8A8BC4@cisco.com> <200105140255.f4E2tjx22995@chmls05.mediaone.net> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-ips@ece.cmu.edu Precedence: bulk Content-Transfer-Encoding: 7bit Stephen Bailey wrote: > > Mark, > > This is mostly fine with me. However, I KNOW implementors are going > to gripe about having to know the target name before performing an > operational login. In general, the spec seems a bit capricious about > when implementors just have to suck it up and do the hard thing, > versus when we try to make it easy for them. Realistically, it's not > a big deal to learn the name (just an extra login), but most end > points are going to have a single target, and from a configuration > standpoint endpoint (and, hence the target) is going to be specified > by transport coordinates (address, port). This extra login for > discovery is simply an irritant. True; it is an extra login. In actual use, however, an initiator might be configured with the IP address of a target to use, and it would have to do the extra session after the SendTargets. Once the initiator has discovered the actual iSCSI name of the target, it will probably have to keep it in a registry or configuration file anyway, since it will need to look for it next time it boots. Some operating systems are particularly fussy about making the same LUs map to the same /dev entries, and so on, and the only way to make them appear in the same place every time is to keep track. Given this, it is not always effective to just discover by address after the first use of a device. You had mentioned in the interim meeting that we could provide a separate "default" target (different from the canonical target), that would represent the only target of a single-target device, and could be logged into without having to know its name. This would make these devices simpler, but an initiator would still need to have the functionality to handle multiple- target devices anyway. I also don't think that there will be a lot of single-target devices out there, at least not right away. Most devices will likely be accessed through iSCSI-FC or iSCSI-SCSI gateways in the near future, before iSCSI interfaces are added to the devices themselves. Our gateway, for example, provides more than one target; my guess is that others will do the same. In this case, an initiator has to deal with multiple targets anyway. > Also, I think this part still needs a little more specification work: > > > 6. We can further specify the order in which the SendTargets response > > fields are returned, to simplify things further, e.g. each target > > in the SendTargets response MUST return these fields in this order: > > > > - A TargetName= field > > - A TargetAlias= field (value left blank if there's no alias) > > - One or more TargetAddress= fields > > - Any vendor-specific fields (ignored by standard initiators) > > I can only guess that you are implying that target descriptions > following this rule may be tiled arbitrarily into Text PDUs returned > from the target? If so, we should specify that. That's pretty much what it does now; I was trying to specify it a little tighter. Do you have any suggestions? > > 1. Should a non-canonical target respond to a SendTargets command? > > I'd say no. Given the nature of the interaction, why bother? Now, I > know the iSCSI philosophy seems to be `if it's not complete gibberish, > why NOT bother?' but that results in complicated implementation. > Specifically, each end (target and initiator) should be able to audit > the other for correctness when possible. Hard rules like `thou shalt > not issue a SendTargets in an operational session' make the auditing a > zillion times easier. I'm leaning that way as well. > To this end, I would go a step further and allow only a single > outstanding SendTargets exchange in a target discovery session. That's a good idea; I can't imagine a reason that the initiator would want to do this more than once in a session, and it would be nice for a target to know it can reject a SendTargets command if it already has one outstanding on the same session. > Steph -- Mark A. Bakke Cisco Systems mbakke@cisco.com 763.398.1054 From owner-ips@ece.cmu.edu Mon May 14 18:24:42 2001 Received: from ece.cmu.edu ([128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id SAA26094 for ; Mon, 14 May 2001 18:24:42 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f4EKnCT25050 for ips-outgoing; Mon, 14 May 2001 16:49:12 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from lucy.trebia.com ([65.192.191.151]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f4EKnAH25046 for ; Mon, 14 May 2001 16:49:10 -0400 (EDT) Received: by lucy.TREBIA.COM with Internet Mail Service (5.5.2653.19) id ; Mon, 14 May 2001 16:53:05 -0400 Message-ID: <63616B261CEFD411834D00D0B7A86CF7048F8E@lucy.TREBIA.COM> From: Sudhir Srinivasan To: "'ips@ece.cmu.edu'" Cc: Sudhir Srinivasan Subject: FCIP: Multiple connections Date: Mon, 14 May 2001 16:53:04 -0400 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2653.19) Content-Type: text/plain; charset="iso-8859-1" Sender: owner-ips@ece.cmu.edu Precedence: bulk In Section 5.5, what is the reason to include rule (2)? Is it not sufficient that frames in each direction be delivered in order? Physical analogy is to have two half-duplex lines instead of a full-duplex line. Vendor should have the choice to go either way. -Sudhir Trebia Networks, Inc. Sudhir.Srinivasan@trebia.com 978-929-0830 x139 www.trebia.com From owner-ips@ece.cmu.edu Mon May 14 20:08:22 2001 Received: from ece.cmu.edu ([128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id UAA27482 for ; Mon, 14 May 2001 20:08:22 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f4EMJgb02731 for ips-outgoing; Mon, 14 May 2001 18:19:42 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from gateway.sanlight.org (adsl-63-202-160-80.dsl.snfc21.pacbell.net [63.202.160.80]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f4EMJeH02727 for ; Mon, 14 May 2001 18:19:40 -0400 (EDT) Received: from ljoy (10.0.0.18.lan.sanlight.net [10.0.0.18]) by gateway.sanlight.org (8.11.0/8.11.0) with SMTP id f4ENQN166963; Mon, 14 May 2001 16:26:24 -0700 (PDT) (envelope-from dotis@sanlight.net) From: "Douglas Otis" To: "Mark Bakke" , "IPS" Subject: RE: iSCSI: Iteration for SendTargets Date: Mon, 14 May 2001 15:16:54 -0700 Message-ID: MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 (Normal) X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook IMO, Build 9.0.2416 (9.0.2911.0) X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4522.1200 Importance: Normal In-Reply-To: <3B0044D6.21E72656@cisco.com> Sender: owner-ips@ece.cmu.edu Precedence: bulk Content-Transfer-Encoding: 7bit Mark, In the SCTP version of iSCSI (see draft-otis-record-delivery-01.txt) session iteration was implemented. My thinking was that this handling may happen within "stack" space as this information is not to be stored until successful completion. With that in mind, keeping a lid on the amount of memory consumed allows for reasonable limits to be placed on the routines. I thought iteration was a good idea. With respect to default devices, as the Target may provide a different "default" Targets dependent on the Initiator name, depending on the wisdom of the Target, there may not be a need to further process the Target list. If a simple Target provides the same "default" to all Initiators, then a need to further resolve to a Target is then required. If the Target already knows the privilege of the Initiator as well as the shared password, it should also be able to properly select the correct "default" Target. One wonders how a generic server would make this selection without human intervention otherwise. If the "default" Target responds with a different name and alias, the Initiator could choose to select this Target on subsequent Logins. Doug > At the interim meeting we had discussed the fact that a > SendTargets response can become rather large, given storage > arrays with 10s of ports, and gateways that can represent > many targets. > > Consider a single target on a 32-port storage array. This > target can easily be expected to have a 40-character iSCSI > name, 40-character alias, and 32 DNS-named addresses, with > each DNS name being 30 characters long. Including the text > key names, equals signs, null characters, and values, the > response for this target would take: > > 12+40+32*(15+30)+13+40 = 1545 bytes > > Given a limitation of 4096 bytes on text responses, the fact > that multiple targets can be returned in a response, and > that the iSCSI name, alias, and DNS-named addresses in > the response do not need to be as well-behaved as those in the > example, it's easy to envision normal SendTargets responses of > over 4k. > > There are different ways to handle this: > > 1. Restrict SendTargets commands and responses to the > canonical target sessions, and handle them in software. > Remove the text response limit for canonical target sessions, > since they are not done in hardware anyway. > > 2. Add an iterator capability to SendTargets. Steph brought > this up during the interim meeting. This will work well if > #1 can't be done. > > There may be others, but these are the two I've thought about. > > The argument for #1 is that SendTargets is really a software > function, and nobody is likely to actually implement it in > hardware. It would use normal socket calls, so the length of > the response would not be much of an issue for the initiator. > The same would be true of a target if it used a normal TCP > stack for canonical sessions; however, there may be implementations > that do not plan to do this (speak up if this is a problem). > > If the argument for #1 does not hold, it would be necessary > to implement some sort of iterator. Steph has already posted > on this topic. An iterator would be easy enough to do for > SendTargets, and we could put it in, but I want to make sure > that #1 is exhausted first. > > Please let me know what you think. > > -- > Mark A. Bakke > Cisco Systems > mbakke@cisco.com > 763.398.1054 > From owner-ips@ece.cmu.edu Mon May 14 20:09:34 2001 Received: from ece.cmu.edu ([128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id UAA27496 for ; Mon, 14 May 2001 20:09:34 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f4EMopS05135 for ips-outgoing; Mon, 14 May 2001 18:50:51 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from med.corp.rhapsodynetworks.com (64-160-62-201.rhapsodynetworks.com [64.160.62.201] (may be forged)) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f4EMomH05129 for ; Mon, 14 May 2001 18:50:49 -0400 (EDT) Received: by med.corp.rhapsodynetworks.com with Internet Mail Service (5.5.2653.19) id ; Mon, 14 May 2001 15:50:35 -0700 Message-ID: <45BEF1D68145D51186C100B0D0AABE6503D3E4@med.corp.rhapsodynetworks.com> From: Venkat Rangan To: "'julian_satran@il.ibm.com'" , ips@ece.cmu.edu Subject: RE: iSCSI: Nailing down CRC-32C Date: Mon, 14 May 2001 15:50:34 -0700 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2653.19) Content-Type: text/plain; charset="iso-8859-1" Sender: owner-ips@ece.cmu.edu Precedence: bulk All, Based on the finalization of the parameters of CRC-32C per following parameters, I obtained a check of 0xE3069283 for the nine-byte string of "123456789" (9 bytes) that the Rocksoft CRC document refers to. Name : "CRC-32C" Width : 32 Poly : 1EDC6F41 (note that the leading "1" is implied) Init : FFFFFFFF RefIn : True RefOut : True XorOut : FFFFFFFF Check : E3069283 It would be nice if other independant checks can validate this. Venkat Rangan Rhapsody Networks Inc. http://www.rhapsodynetworks.com The table and code follows: /*****************************************************************/ /* */ /* CRC LOOKUP TABLE */ /* ================ */ /* The following CRC lookup table was generated automagically */ /* by the Rocksoft^tm Model CRC Algorithm Table Generation */ /* Program V1.0 using the following model parameters: */ /* */ /* Width : 4 bytes. */ /* Poly : 0x1EDC6F41L */ /* Reverse : TRUE. */ /* */ /* For more information on the Rocksoft^tm Model CRC Algorithm, */ /* see the document titled "A Painless Guide to CRC Error */ /* Detection Algorithms" by Ross Williams */ /* (ross@guest.adelaide.edu.au.). This document is likely to be */ /* in the FTP archive "ftp.adelaide.edu.au/pub/rocksoft". */ /* */ /*****************************************************************/ unsigned long crctable[256] = { 0x00000000L, 0xF26B8303L, 0xE13B70F7L, 0x1350F3F4L, 0xC79A971FL, 0x35F1141CL, 0x26A1E7E8L, 0xD4CA64EBL, 0x8AD958CFL, 0x78B2DBCCL, 0x6BE22838L, 0x9989AB3BL, 0x4D43CFD0L, 0xBF284CD3L, 0xAC78BF27L, 0x5E133C24L, 0x105EC76FL, 0xE235446CL, 0xF165B798L, 0x030E349BL, 0xD7C45070L, 0x25AFD373L, 0x36FF2087L, 0xC494A384L, 0x9A879FA0L, 0x68EC1CA3L, 0x7BBCEF57L, 0x89D76C54L, 0x5D1D08BFL, 0xAF768BBCL, 0xBC267848L, 0x4E4DFB4BL, 0x20BD8EDEL, 0xD2D60DDDL, 0xC186FE29L, 0x33ED7D2AL, 0xE72719C1L, 0x154C9AC2L, 0x061C6936L, 0xF477EA35L, 0xAA64D611L, 0x580F5512L, 0x4B5FA6E6L, 0xB93425E5L, 0x6DFE410EL, 0x9F95C20DL, 0x8CC531F9L, 0x7EAEB2FAL, 0x30E349B1L, 0xC288CAB2L, 0xD1D83946L, 0x23B3BA45L, 0xF779DEAEL, 0x05125DADL, 0x1642AE59L, 0xE4292D5AL, 0xBA3A117EL, 0x4851927DL, 0x5B016189L, 0xA96AE28AL, 0x7DA08661L, 0x8FCB0562L, 0x9C9BF696L, 0x6EF07595L, 0x417B1DBCL, 0xB3109EBFL, 0xA0406D4BL, 0x522BEE48L, 0x86E18AA3L, 0x748A09A0L, 0x67DAFA54L, 0x95B17957L, 0xCBA24573L, 0x39C9C670L, 0x2A993584L, 0xD8F2B687L, 0x0C38D26CL, 0xFE53516FL, 0xED03A29BL, 0x1F682198L, 0x5125DAD3L, 0xA34E59D0L, 0xB01EAA24L, 0x42752927L, 0x96BF4DCCL, 0x64D4CECFL, 0x77843D3BL, 0x85EFBE38L, 0xDBFC821CL, 0x2997011FL, 0x3AC7F2EBL, 0xC8AC71E8L, 0x1C661503L, 0xEE0D9600L, 0xFD5D65F4L, 0x0F36E6F7L, 0x61C69362L, 0x93AD1061L, 0x80FDE395L, 0x72966096L, 0xA65C047DL, 0x5437877EL, 0x4767748AL, 0xB50CF789L, 0xEB1FCBADL, 0x197448AEL, 0x0A24BB5AL, 0xF84F3859L, 0x2C855CB2L, 0xDEEEDFB1L, 0xCDBE2C45L, 0x3FD5AF46L, 0x7198540DL, 0x83F3D70EL, 0x90A324FAL, 0x62C8A7F9L, 0xB602C312L, 0x44694011L, 0x5739B3E5L, 0xA55230E6L, 0xFB410CC2L, 0x092A8FC1L, 0x1A7A7C35L, 0xE811FF36L, 0x3CDB9BDDL, 0xCEB018DEL, 0xDDE0EB2AL, 0x2F8B6829L, 0x82F63B78L, 0x709DB87BL, 0x63CD4B8FL, 0x91A6C88CL, 0x456CAC67L, 0xB7072F64L, 0xA457DC90L, 0x563C5F93L, 0x082F63B7L, 0xFA44E0B4L, 0xE9141340L, 0x1B7F9043L, 0xCFB5F4A8L, 0x3DDE77ABL, 0x2E8E845FL, 0xDCE5075CL, 0x92A8FC17L, 0x60C37F14L, 0x73938CE0L, 0x81F80FE3L, 0x55326B08L, 0xA759E80BL, 0xB4091BFFL, 0x466298FCL, 0x1871A4D8L, 0xEA1A27DBL, 0xF94AD42FL, 0x0B21572CL, 0xDFEB33C7L, 0x2D80B0C4L, 0x3ED04330L, 0xCCBBC033L, 0xA24BB5A6L, 0x502036A5L, 0x4370C551L, 0xB11B4652L, 0x65D122B9L, 0x97BAA1BAL, 0x84EA524EL, 0x7681D14DL, 0x2892ED69L, 0xDAF96E6AL, 0xC9A99D9EL, 0x3BC21E9DL, 0xEF087A76L, 0x1D63F975L, 0x0E330A81L, 0xFC588982L, 0xB21572C9L, 0x407EF1CAL, 0x532E023EL, 0xA145813DL, 0x758FE5D6L, 0x87E466D5L, 0x94B49521L, 0x66DF1622L, 0x38CC2A06L, 0xCAA7A905L, 0xD9F75AF1L, 0x2B9CD9F2L, 0xFF56BD19L, 0x0D3D3E1AL, 0x1E6DCDEEL, 0xEC064EEDL, 0xC38D26C4L, 0x31E6A5C7L, 0x22B65633L, 0xD0DDD530L, 0x0417B1DBL, 0xF67C32D8L, 0xE52CC12CL, 0x1747422FL, 0x49547E0BL, 0xBB3FFD08L, 0xA86F0EFCL, 0x5A048DFFL, 0x8ECEE914L, 0x7CA56A17L, 0x6FF599E3L, 0x9D9E1AE0L, 0xD3D3E1ABL, 0x21B862A8L, 0x32E8915CL, 0xC083125FL, 0x144976B4L, 0xE622F5B7L, 0xF5720643L, 0x07198540L, 0x590AB964L, 0xAB613A67L, 0xB831C993L, 0x4A5A4A90L, 0x9E902E7BL, 0x6CFBAD78L, 0x7FAB5E8CL, 0x8DC0DD8FL, 0xE330A81AL, 0x115B2B19L, 0x020BD8EDL, 0xF0605BEEL, 0x24AA3F05L, 0xD6C1BC06L, 0xC5914FF2L, 0x37FACCF1L, 0x69E9F0D5L, 0x9B8273D6L, 0x88D28022L, 0x7AB90321L, 0xAE7367CAL, 0x5C18E4C9L, 0x4F48173DL, 0xBD23943EL, 0xF36E6F75L, 0x0105EC76L, 0x12551F82L, 0xE03E9C81L, 0x34F4F86AL, 0xC69F7B69L, 0xD5CF889DL, 0x27A40B9EL, 0x79B737BAL, 0x8BDCB4B9L, 0x988C474DL, 0x6AE7C44EL, 0xBE2DA0A5L, 0x4C4623A6L, 0x5F16D052L, 0xAD7D5351L }; /*****************************************************************/ /* End of CRC Lookup Table */ /*****************************************************************/ /*****************************************************************/ /* ComputeCRC */ /*****************************************************************/ #include #include #include "crctable.h" #define TB_INIT 0xFFFFFFFF #define TB_INIT_REFLECTED 0xFFFFFFFF #define TB_XOROT 0xFFFFFFFF unsigned long crc_reflected (); unsigned long crc_reflected (blk_adr,blk_len) unsigned char *blk_adr; unsigned long blk_len; { unsigned long crc = TB_INIT_REFLECTED; while (blk_len--) crc = crctable[(crc ^ *blk_adr++) & 0xFFL] ^ (crc >> 8); return crc ^ TB_XOROT; } -----Original Message----- From: julian_satran@il.ibm.com [mailto:julian_satran@il.ibm.com] Sent: Tuesday, May 08, 2001 7:38 AM To: ips@ece.cmu.edu Subject: Re: iSCSI: Nailing down CRC-32C Mark, My basic assumption was that everything is (as in all IP related standards) in network order. Regards, Julo Mark Bakke on 07-05-2001 23:00:28 Please respond to Mark Bakke To: Julian Satran/Haifa/IBM@IBMIL cc: ips@ece.cmu.edu Subject: Re: iSCSI: Nailing down CRC-32C Julian- That's great; it covers nearly everything. The only thing left is the byte order; are they taken in the same order Ethernet uses? If so, I can generate a few test patterns that our implementations can all check against. Thanks, -- Mark julian_satran@il.ibm.com wrote: > > The CRC part of the appendix (for 07) reads: > > The following table lists cyclic integrity checksums that can be > negotiated for the digests and MUST be implemented by every iSCSI > initiator and target. Note that these digest options have only error > detection significance. > > +---------------------------------------------+ > | Name | Description | > +---------------------------------------------+ > | crc-32C | 32 bit CRC | 11EDC6F41 | > +---------------------------------------------+ > | none | no digest | > +---------------------------------------------+ > > The generator polynomials for those digests are given in hex-notation, > for example 3a stands for 0011 1010 - the polynomial x**5+X**4+x**3+x+1. > > The generator polynomial selected is evaluated in [Castagnioli93]. > When using the CRC the CRC register must be initialized to all 1s > (0xFFFFFFFF) and the CRC bits must be complemented before transmission. > Padding bytes, when presents in a segment covered by a CRC, have to be > set to 0 and are included in the CRC. > > Regards, > Julo > > Mark Bakke on 05-05-2001 00:13:09 > > Please respond to Mark Bakke > > To: IPS > cc: > Subject: iSCSI: Nailing down CRC-32C > > At the interim meeting, it was stated that iSCSI has selected > the CRC-32C polynomial as its required iSCSI-level header and > data CRC. Now that we have it, it's time to move on and make > sure we can implement it. > > In the interest of interoperability, we need to not only specify > the polynomial, but also the initial values, bit and byte > ordering, etc. > > "A Painless Guide to CRC Error Detection Algorithms" > (http://www.ross.net/crc/crcpaper.html) specifies a > method to unambiguously characterize these parameters > (sections 15 and 16). Has anyone taken a shot at defining > these yet? Otherwise, here is what it might look like: > > Name : "CRC-32C" > Width : 32 > Poly : 1EDC6F41 (note that the leading "1" is implied) > Init : FFFFFFFF > RefIn : True > RefOut : True > XorOut : FFFFFFFF > Check : ? > > I haven't attempted to create check data based on these yet. As > soon as the other parameters are nailed down, we need to do this. > > Anyway, I am not a CRC expert, and can't make any statement about > the above values being the "best" way to do this, but instead just > copied them (except the polynomial itself) from the Ethernet CRC, > since that is likely the easiest for everyone implementing hardware > to deal with. > > If someone else is already doing this, let me know; I just wanted > to start this thread so we can get closure. > > Regards, > > Mark > > -- > Mark A. Bakke > Cisco Systems > mbakke@cisco.com > 763.398.1054 -- Mark A. Bakke Cisco Systems mbakke@cisco.com 763.398.1054 From owner-ips@ece.cmu.edu Mon May 14 20:10:11 2001 Received: from ece.cmu.edu ([128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id UAA27526 for ; Mon, 14 May 2001 20:10:11 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f4EMDJQ02252 for ips-outgoing; Mon, 14 May 2001 18:13:19 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from sandmail.sandburst.com (sandburst-gw.bstn-gw02.ma.us.intelilink.net [216.57.129.34]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f4EMDHH02247 for ; Mon, 14 May 2001 18:13:17 -0400 (EDT) Received: from cs.uchicago.edu (dynamite-38.sandburst.com [172.16.5.38]) by sandmail.sandburst.com (Postfix) with ESMTP id 7BD3F94009 for ; Mon, 14 May 2001 18:13:17 -0400 (EDT) To: ips@ece.cmu.edu Subject: Re: iSCSI: Canonical Targets In-Reply-To: Message from Mark Bakke of "Mon, 14 May 2001 15:30:25 CDT." <3B004061.E6019045@cisco.com> References: <3AFC53D8.CD8A8BC4@cisco.com> <200105140255.f4E2tjx22995@chmls05.mediaone.net> <3B004061.E6019045@cisco.com> Date: Mon, 14 May 2001 18:11:24 -0400 From: Stephen Bailey Message-Id: <20010514221317.7BD3F94009@sandmail.sandburst.com> Sender: owner-ips@ece.cmu.edu Precedence: bulk Mark, > I also don't think that there will be a lot of single-target devices out > there, at least not right away. Perhaps not right away, but definitely eventually. Using the LUN model is the way to ensure that a target (true target, not bridge) implementation can be relatively transport independent. That's proven to be a major survival characteristic for targets. > Most devices will likely be accessed through iSCSI-FC or iSCSI-SCSI > gateways in the near future, before iSCSI interfaces are added to > the devices themselves. Our gateway, for example, provides more > than one target; my guess is that others will do the same. Agreed. The only SST target in the world is a gateway, and it provides multiple target support as well. Our solution to this problem was to constrain the target addressing range so that the initiator can do the traditional big crawl. SST was aimed at a different design point, so I'm not saying we should adopt the same solution with iSCSI, I'm just agreeing that allowing multiple targets behind an endpoint in the protocol design is a good idea. > In this case, an initiator has to deal with multiple targets > anyway. Absolutely. However, in the limit, the bridges usually wither away. Sometimes they wither away a lot faster than you might expect. There were some early attempts at FC/||SCSI bridging that worked from the same set of assumptions. You probably don't need to know how successful those were. That would never happen to today's modern bridge vendor, since they've learned how to apply all their accumulated cleverness to building native targets in short order :^) The bridged scenario shouldn't be at the expense of the unbridged one. My proposal of allowing: 1) target discovery login 2) operational login to a default (Julian's right---the term canonical is inappropriate) target 3) operational login to a specified target seems to cover all bases. If your bridge does not want to support a default target (I bet it will.... how else will you be able to slap it under the skins with a Symmetrix and call it an iSCSI storage array?), you can reject the login, with a `no default' reason code (TBS). I don't mind adding Jim Hafner's multi-layer discovery proposal if you guys really think it will help. Personally, I'm skeptical. Whatever the case, it seems like it can be done in such a way as to still permit a single-layer login when given nothing but socket coordinates. Let me beat on this one final time. I think EVERYBODY on this list is aware that the biggest selling point of iSCSI across the entire range of applications is that it leverages existing IP management skills. As such, using socket coordinates to specify targets is what IP managers understand. That doesn't mean we can't introduce additional concepts, but the users should be able to get going without them. Steph From owner-ips@ece.cmu.edu Tue May 15 10:22:07 2001 Received: from ece.cmu.edu ([128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id KAA28418 for ; Tue, 15 May 2001 10:22:06 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f4FCx2210344 for ips-outgoing; Tue, 15 May 2001 08:59:02 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from dogwood.cisco.com (dogwood.cisco.com [161.44.11.19]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f4FCx1H10340 for ; Tue, 15 May 2001 08:59:01 -0400 (EDT) Received: from cisco.com (mbakke@mbakke-lnx.cisco.com [161.44.68.87]) by dogwood.cisco.com (8.8.6 (PHNE_14041)/CISCO.SERVER.1.2) with ESMTP id IAA23884; Tue, 15 May 2001 08:58:49 -0400 (EDT) Message-ID: <3B01276D.67DFEC2@cisco.com> Date: Tue, 15 May 2001 07:56:13 -0500 From: Mark Bakke X-Mailer: Mozilla 4.72 [en] (X11; U; Linux 2.2.16-3.uid32 i686) X-Accept-Language: en, de MIME-Version: 1.0 To: Stephen Bailey CC: ips@ece.cmu.edu Subject: Re: iSCSI: Canonical Targets References: <3AFD2FA1.F033A4DE@trebia.com> <3AFFDA17.10AE8DCC@cisco.com> <20010514203721.0A8CC94009@sandmail.sandburst.com> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-ips@ece.cmu.edu Precedence: bulk Content-Transfer-Encoding: 7bit Stephen Bailey wrote: > > Mark, > > I don't disagree with the general thrust of your statement, but I must > argue with: > > > Even in an environment without proxies and gateways, any > > medium-sized disk array should be able to support more than one I did say "should". OK, perhaps I'm engaging in a little bit of wishful thinking, but it seems to me that most people would rather deal with a set of logical targets, each created for use by a host or application, rather than a single target with different LUN maps depending on who's asking. I know we'll have to support the latter as well. Anyway, I agree that bridges will always need to take up the slack, but my point was that I didn't want to see initiators make the assumption that there is only one target for them at each address. I just don't want to see us do a repeat of some of the limitations of ||SCSI and FC. > > logical target. > > Unless somebody's really building a medium sized disk array a) > completely from scratch b) for iSCSI use only, this is unlikely. > > Existing disk arrays present all storage containers as LUNs. The most > likely [credible] iSCSI targets will be existing storage arrays with > iSCSI front-ends. This follows the ||SCSI->FCP pattern. The > alternative interpretation, which I'm pretty sure is false, is that > the back-end part of a storage controller is really simple so the > serious iSCSI targets will not come from the same vendors (or groups) > as existing, serious storage targets. > > Bridges are a different issue, but I've always felt (and you seem to > agree) that the bridges just have to suck it up and, well, bridge, the > semantic gaps between what's expected in a native initiator->target > interaction and what you get with an initiator->bridge->target one. > > Steph -- Mark A. Bakke Cisco Systems mbakke@cisco.com 763.398.1054 From owner-ips@ece.cmu.edu Tue May 15 10:22:45 2001 Received: from ece.cmu.edu ([128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id KAA28481 for ; Tue, 15 May 2001 10:22:44 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f4FD3GN10703 for ips-outgoing; Tue, 15 May 2001 09:03:16 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from dogwood.cisco.com (dogwood.cisco.com [161.44.11.19]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f4FD3FH10699 for ; Tue, 15 May 2001 09:03:15 -0400 (EDT) Received: from cisco.com (mbakke@mbakke-lnx.cisco.com [161.44.68.87]) by dogwood.cisco.com (8.8.6 (PHNE_14041)/CISCO.SERVER.1.2) with ESMTP id JAA26981; Tue, 15 May 2001 09:03:03 -0400 (EDT) Message-ID: <3B01286A.375893D7@cisco.com> Date: Tue, 15 May 2001 08:00:26 -0500 From: Mark Bakke X-Mailer: Mozilla 4.72 [en] (X11; U; Linux 2.2.16-3.uid32 i686) X-Accept-Language: en, de MIME-Version: 1.0 To: Stephen Bailey CC: ips@ece.cmu.edu Subject: Re: iSCSI: Canonical Targets References: <20010514204634.789C894009@sandmail.sandburst.com> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-ips@ece.cmu.edu Precedence: bulk Content-Transfer-Encoding: 7bit Stephen Bailey wrote: > > Jim, > > > I have a question for you (and the list) about a long Text response (as > > might be needed for SendTargets). Can the Text response come in multiple > > PDUs? > > As I interpret the spec (emphasis on interpret---I don't think the > spec says anything clearly one way or another), yes. > > > In particular, can one "key=value" pair span multiple PDUs? > > Not as I interpret the spec. > > > And if so, does this help address your resources concerns? > > No. The problem I'm talking about is handling an unbounded TOTAL > amount of data returned from a single request. It doesn't matter > whether it comes in a single PDU or broken into a million, unless you > can offer a finer grain of flow control for the million PDUs than for > the one. In either case the data returned may a) clog the channel b) > possibly consume resources necessary for other operational purposes. > > Again, if we disallow SendTargets (and other interactions with > indeterminate, potentially large uncontrolled flows) in an operational > login, we can allow the transport to provide the flow control. In an > operational login we don't want to engage transport flow control in > this way because it interferes with other operational communication. This would simplify things a lot. I started another thread on iterating SendTargets, with what I think I heard from the interim meeting. If we can keep SendTargets out of the operational sessions, perhaps we don't need to iterate. > > Steph -- Mark A. Bakke Cisco Systems mbakke@cisco.com 763.398.1054 From owner-ips@ece.cmu.edu Tue May 15 10:22:51 2001 Received: from ece.cmu.edu ([128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id KAA28495 for ; Tue, 15 May 2001 10:22:48 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f4FCJPZ07494 for ips-outgoing; Tue, 15 May 2001 08:19:25 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from dogwood.cisco.com (dogwood.cisco.com [161.44.11.19]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f4FCJNH07488 for ; Tue, 15 May 2001 08:19:23 -0400 (EDT) Received: from cisco.com (mbakke@mbakke-lnx.cisco.com [161.44.68.87]) by dogwood.cisco.com (8.8.6 (PHNE_14041)/CISCO.SERVER.1.2) with ESMTP id IAA00417; Tue, 15 May 2001 08:18:58 -0400 (EDT) Message-ID: <3B011E16.DAB65A52@cisco.com> Date: Tue, 15 May 2001 07:16:22 -0500 From: Mark Bakke X-Mailer: Mozilla 4.72 [en] (X11; U; Linux 2.2.16-3.uid32 i686) X-Accept-Language: en, de MIME-Version: 1.0 To: Venkat Rangan CC: "'julian_satran@il.ibm.com'" , ips@ece.cmu.edu Subject: Re: iSCSI: Nailing down CRC-32C References: <45BEF1D68145D51186C100B0D0AABE6503D3E4@med.corp.rhapsodynetworks.com> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-ips@ece.cmu.edu Precedence: bulk Content-Transfer-Encoding: 7bit Venkat- I assumed the same parameters (iSCSI has not yet specified the reflection parameters), and obtained the same check you did with their code. -- Mark Venkat Rangan wrote: > > All, > > Based on the finalization of the parameters of CRC-32C per following > parameters, I obtained a check of 0xE3069283 for the nine-byte string of > "123456789" (9 bytes) that the Rocksoft CRC document refers to. > > Name : "CRC-32C" > Width : 32 > Poly : 1EDC6F41 (note that the leading "1" is implied) > Init : FFFFFFFF > RefIn : True > RefOut : True > XorOut : FFFFFFFF > Check : E3069283 > > It would be nice if other independant checks can validate this. > > Venkat Rangan > Rhapsody Networks Inc. > http://www.rhapsodynetworks.com > > The table and code follows: > > /*****************************************************************/ > /* */ > /* CRC LOOKUP TABLE */ > /* ================ */ > /* The following CRC lookup table was generated automagically */ > /* by the Rocksoft^tm Model CRC Algorithm Table Generation */ > /* Program V1.0 using the following model parameters: */ > /* */ > /* Width : 4 bytes. */ > /* Poly : 0x1EDC6F41L */ > /* Reverse : TRUE. */ > /* */ > /* For more information on the Rocksoft^tm Model CRC Algorithm, */ > /* see the document titled "A Painless Guide to CRC Error */ > /* Detection Algorithms" by Ross Williams */ > /* (ross@guest.adelaide.edu.au.). This document is likely to be */ > /* in the FTP archive "ftp.adelaide.edu.au/pub/rocksoft". */ > /* */ > /*****************************************************************/ > > unsigned long crctable[256] = > { > 0x00000000L, 0xF26B8303L, 0xE13B70F7L, 0x1350F3F4L, > 0xC79A971FL, 0x35F1141CL, 0x26A1E7E8L, 0xD4CA64EBL, > 0x8AD958CFL, 0x78B2DBCCL, 0x6BE22838L, 0x9989AB3BL, > 0x4D43CFD0L, 0xBF284CD3L, 0xAC78BF27L, 0x5E133C24L, > 0x105EC76FL, 0xE235446CL, 0xF165B798L, 0x030E349BL, > 0xD7C45070L, 0x25AFD373L, 0x36FF2087L, 0xC494A384L, > 0x9A879FA0L, 0x68EC1CA3L, 0x7BBCEF57L, 0x89D76C54L, > 0x5D1D08BFL, 0xAF768BBCL, 0xBC267848L, 0x4E4DFB4BL, > 0x20BD8EDEL, 0xD2D60DDDL, 0xC186FE29L, 0x33ED7D2AL, > 0xE72719C1L, 0x154C9AC2L, 0x061C6936L, 0xF477EA35L, > 0xAA64D611L, 0x580F5512L, 0x4B5FA6E6L, 0xB93425E5L, > 0x6DFE410EL, 0x9F95C20DL, 0x8CC531F9L, 0x7EAEB2FAL, > 0x30E349B1L, 0xC288CAB2L, 0xD1D83946L, 0x23B3BA45L, > 0xF779DEAEL, 0x05125DADL, 0x1642AE59L, 0xE4292D5AL, > 0xBA3A117EL, 0x4851927DL, 0x5B016189L, 0xA96AE28AL, > 0x7DA08661L, 0x8FCB0562L, 0x9C9BF696L, 0x6EF07595L, > 0x417B1DBCL, 0xB3109EBFL, 0xA0406D4BL, 0x522BEE48L, > 0x86E18AA3L, 0x748A09A0L, 0x67DAFA54L, 0x95B17957L, > 0xCBA24573L, 0x39C9C670L, 0x2A993584L, 0xD8F2B687L, > 0x0C38D26CL, 0xFE53516FL, 0xED03A29BL, 0x1F682198L, > 0x5125DAD3L, 0xA34E59D0L, 0xB01EAA24L, 0x42752927L, > 0x96BF4DCCL, 0x64D4CECFL, 0x77843D3BL, 0x85EFBE38L, > 0xDBFC821CL, 0x2997011FL, 0x3AC7F2EBL, 0xC8AC71E8L, > 0x1C661503L, 0xEE0D9600L, 0xFD5D65F4L, 0x0F36E6F7L, > 0x61C69362L, 0x93AD1061L, 0x80FDE395L, 0x72966096L, > 0xA65C047DL, 0x5437877EL, 0x4767748AL, 0xB50CF789L, > 0xEB1FCBADL, 0x197448AEL, 0x0A24BB5AL, 0xF84F3859L, > 0x2C855CB2L, 0xDEEEDFB1L, 0xCDBE2C45L, 0x3FD5AF46L, > 0x7198540DL, 0x83F3D70EL, 0x90A324FAL, 0x62C8A7F9L, > 0xB602C312L, 0x44694011L, 0x5739B3E5L, 0xA55230E6L, > 0xFB410CC2L, 0x092A8FC1L, 0x1A7A7C35L, 0xE811FF36L, > 0x3CDB9BDDL, 0xCEB018DEL, 0xDDE0EB2AL, 0x2F8B6829L, > 0x82F63B78L, 0x709DB87BL, 0x63CD4B8FL, 0x91A6C88CL, > 0x456CAC67L, 0xB7072F64L, 0xA457DC90L, 0x563C5F93L, > 0x082F63B7L, 0xFA44E0B4L, 0xE9141340L, 0x1B7F9043L, > 0xCFB5F4A8L, 0x3DDE77ABL, 0x2E8E845FL, 0xDCE5075CL, > 0x92A8FC17L, 0x60C37F14L, 0x73938CE0L, 0x81F80FE3L, > 0x55326B08L, 0xA759E80BL, 0xB4091BFFL, 0x466298FCL, > 0x1871A4D8L, 0xEA1A27DBL, 0xF94AD42FL, 0x0B21572CL, > 0xDFEB33C7L, 0x2D80B0C4L, 0x3ED04330L, 0xCCBBC033L, > 0xA24BB5A6L, 0x502036A5L, 0x4370C551L, 0xB11B4652L, > 0x65D122B9L, 0x97BAA1BAL, 0x84EA524EL, 0x7681D14DL, > 0x2892ED69L, 0xDAF96E6AL, 0xC9A99D9EL, 0x3BC21E9DL, > 0xEF087A76L, 0x1D63F975L, 0x0E330A81L, 0xFC588982L, > 0xB21572C9L, 0x407EF1CAL, 0x532E023EL, 0xA145813DL, > 0x758FE5D6L, 0x87E466D5L, 0x94B49521L, 0x66DF1622L, > 0x38CC2A06L, 0xCAA7A905L, 0xD9F75AF1L, 0x2B9CD9F2L, > 0xFF56BD19L, 0x0D3D3E1AL, 0x1E6DCDEEL, 0xEC064EEDL, > 0xC38D26C4L, 0x31E6A5C7L, 0x22B65633L, 0xD0DDD530L, > 0x0417B1DBL, 0xF67C32D8L, 0xE52CC12CL, 0x1747422FL, > 0x49547E0BL, 0xBB3FFD08L, 0xA86F0EFCL, 0x5A048DFFL, > 0x8ECEE914L, 0x7CA56A17L, 0x6FF599E3L, 0x9D9E1AE0L, > 0xD3D3E1ABL, 0x21B862A8L, 0x32E8915CL, 0xC083125FL, > 0x144976B4L, 0xE622F5B7L, 0xF5720643L, 0x07198540L, > 0x590AB964L, 0xAB613A67L, 0xB831C993L, 0x4A5A4A90L, > 0x9E902E7BL, 0x6CFBAD78L, 0x7FAB5E8CL, 0x8DC0DD8FL, > 0xE330A81AL, 0x115B2B19L, 0x020BD8EDL, 0xF0605BEEL, > 0x24AA3F05L, 0xD6C1BC06L, 0xC5914FF2L, 0x37FACCF1L, > 0x69E9F0D5L, 0x9B8273D6L, 0x88D28022L, 0x7AB90321L, > 0xAE7367CAL, 0x5C18E4C9L, 0x4F48173DL, 0xBD23943EL, > 0xF36E6F75L, 0x0105EC76L, 0x12551F82L, 0xE03E9C81L, > 0x34F4F86AL, 0xC69F7B69L, 0xD5CF889DL, 0x27A40B9EL, > 0x79B737BAL, 0x8BDCB4B9L, 0x988C474DL, 0x6AE7C44EL, > 0xBE2DA0A5L, 0x4C4623A6L, 0x5F16D052L, 0xAD7D5351L > }; > > /*****************************************************************/ > /* End of CRC Lookup Table */ > /*****************************************************************/ > > /*****************************************************************/ > /* ComputeCRC */ > /*****************************************************************/ > #include > #include > #include "crctable.h" > > #define TB_INIT 0xFFFFFFFF > #define TB_INIT_REFLECTED 0xFFFFFFFF > #define TB_XOROT 0xFFFFFFFF > > unsigned long crc_reflected (); > unsigned long crc_reflected (blk_adr,blk_len) > unsigned char *blk_adr; > unsigned long blk_len; > { > unsigned long crc = TB_INIT_REFLECTED; > while (blk_len--) > crc = crctable[(crc ^ *blk_adr++) & 0xFFL] ^ (crc >> 8); > return crc ^ TB_XOROT; > } > > -----Original Message----- > From: julian_satran@il.ibm.com [mailto:julian_satran@il.ibm.com] > Sent: Tuesday, May 08, 2001 7:38 AM > To: ips@ece.cmu.edu > Subject: Re: iSCSI: Nailing down CRC-32C > > Mark, > > My basic assumption was that everything is (as in all IP related standards) > in network order. > > Regards, > Julo > > Mark Bakke on 07-05-2001 23:00:28 > > Please respond to Mark Bakke > > To: Julian Satran/Haifa/IBM@IBMIL > cc: ips@ece.cmu.edu > Subject: Re: iSCSI: Nailing down CRC-32C > > Julian- > > That's great; it covers nearly everything. The only thing left > is the byte order; are they taken in the same order Ethernet uses? > > If so, I can generate a few test patterns that our implementations > can all check against. > > Thanks, > > -- > Mark > > julian_satran@il.ibm.com wrote: > > > > The CRC part of the appendix (for 07) reads: > > > > The following table lists cyclic integrity checksums that can be > > negotiated for the digests and MUST be implemented by every iSCSI > > initiator and target. Note that these digest options have only error > > detection significance. > > > > +---------------------------------------------+ > > | Name | Description | > > +---------------------------------------------+ > > | crc-32C | 32 bit CRC | 11EDC6F41 | > > +---------------------------------------------+ > > | none | no digest | > > +---------------------------------------------+ > > > > The generator polynomials for those digests are given in hex-notation, > > for example 3a stands for 0011 1010 - the polynomial > x**5+X**4+x**3+x+1. > > > > The generator polynomial selected is evaluated in [Castagnioli93]. > > When using the CRC the CRC register must be initialized to all 1s > > (0xFFFFFFFF) and the CRC bits must be complemented before > transmission. > > Padding bytes, when presents in a segment covered by a CRC, have to be > > set to 0 and are included in the CRC. > > > > Regards, > > Julo > > > > Mark Bakke on 05-05-2001 00:13:09 > > > > Please respond to Mark Bakke > > > > To: IPS > > cc: > > Subject: iSCSI: Nailing down CRC-32C > > > > At the interim meeting, it was stated that iSCSI has selected > > the CRC-32C polynomial as its required iSCSI-level header and > > data CRC. Now that we have it, it's time to move on and make > > sure we can implement it. > > > > In the interest of interoperability, we need to not only specify > > the polynomial, but also the initial values, bit and byte > > ordering, etc. > > > > "A Painless Guide to CRC Error Detection Algorithms" > > (http://www.ross.net/crc/crcpaper.html) specifies a > > method to unambiguously characterize these parameters > > (sections 15 and 16). Has anyone taken a shot at defining > > these yet? Otherwise, here is what it might look like: > > > > Name : "CRC-32C" > > Width : 32 > > Poly : 1EDC6F41 (note that the leading "1" is implied) > > Init : FFFFFFFF > > RefIn : True > > RefOut : True > > XorOut : FFFFFFFF > > Check : ? > > > > I haven't attempted to create check data based on these yet. As > > soon as the other parameters are nailed down, we need to do this. > > > > Anyway, I am not a CRC expert, and can't make any statement about > > the above values being the "best" way to do this, but instead just > > copied them (except the polynomial itself) from the Ethernet CRC, > > since that is likely the easiest for everyone implementing hardware > > to deal with. > > > > If someone else is already doing this, let me know; I just wanted > > to start this thread so we can get closure. > > > > Regards, > > > > Mark > > > > -- > > Mark A. Bakke > > Cisco Systems > > mbakke@cisco.com > > 763.398.1054 > > -- > Mark A. Bakke > Cisco Systems > mbakke@cisco.com > 763.398.1054 -- Mark A. Bakke Cisco Systems mbakke@cisco.com 763.398.1054 From owner-ips@ece.cmu.edu Tue May 15 11:12:25 2001 Received: from ece.cmu.edu ([128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id LAA29790 for ; Tue, 15 May 2001 11:12:25 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f4FDNDg12252 for ips-outgoing; Tue, 15 May 2001 09:23:13 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from dogwood.cisco.com (dogwood.cisco.com [161.44.11.19]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f4FDNBH12247 for ; Tue, 15 May 2001 09:23:11 -0400 (EDT) Received: from cisco.com (mbakke@mbakke-lnx.cisco.com [161.44.68.87]) by dogwood.cisco.com (8.8.6 (PHNE_14041)/CISCO.SERVER.1.2) with ESMTP id JAA14098; Tue, 15 May 2001 09:23:00 -0400 (EDT) Message-ID: <3B012D18.688CDF1E@cisco.com> Date: Tue, 15 May 2001 08:20:24 -0500 From: Mark Bakke X-Mailer: Mozilla 4.72 [en] (X11; U; Linux 2.2.16-3.uid32 i686) X-Accept-Language: en, de MIME-Version: 1.0 To: Stephen Bailey CC: ips@ece.cmu.edu Subject: Re: iSCSI: Canonical Targets References: <3AFC53D8.CD8A8BC4@cisco.com> <200105140255.f4E2tjx22995@chmls05.mediaone.net> <3B004061.E6019045@cisco.com> <20010514221317.7BD3F94009@sandmail.sandburst.com> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-ips@ece.cmu.edu Precedence: bulk Content-Transfer-Encoding: 7bit Steph- Good discussion. My comments are below. Stephen Bailey wrote: > My proposal of allowing: > 1) target discovery login > 2) operational login to a default (Julian's right---the term > canonical is inappropriate) target > 3) operational login to a specified target I'm OK with adding the default target if we feel we need it; again, I just don't want to over-simplify the initiators to the point where they don't support more than one. BTW, I'm not all that attached to "canonical"; if anyone has a better name for either the discovery target or the default operational target, please send it to the list. > seems to cover all bases. If your bridge does not want to support a > default target (I bet it will.... how else will you be able to slap it > under the skins with a Symmetrix and call it an iSCSI storage array?), > you can reject the login, with a `no default' reason code (TBS). At that point, the initiator can either give up, or log into the discovery/canonical target, do SendTargets, and then log in. As long as we make the latter a MUST, I'm OK with the default target. If the latter is a SHOULD, then I'm not. > I don't mind adding Jim Hafner's multi-layer discovery proposal if you > guys really think it will help. Personally, I'm skeptical. Whatever > the case, it seems like it can be done in such a way as to still > permit a single-layer login when given nothing but socket > coordinates. > > Let me beat on this one final time. I think EVERYBODY on this list is > aware that the biggest selling point of iSCSI across the entire range > of applications is that it leverages existing IP management skills. > As such, using socket coordinates to specify targets is what IP > managers understand. That doesn't mean we can't introduce additional > concepts, but the users should be able to get going without them. Absolutely. However, there are two points of view from which one can count a concept as "additional": 1. The FC SAN point of view: People who have already deployed Fibre Channel, and wish to extend their storage network to a greater number of hosts. To these people, iSCSI/IP is just another network interface. These people have the storage management skills, and many have the IP networking skills as well. Most of the storage vendors themselves probably belong to this category. 2. The intranetworking point of view: People who don't know storage, but are accustomed to managing web servers, file servers, and the like. To these people, an iSCSI device is just another server, and should be managed as such. These people have the IP networking skills, and are not likely to be familiar with managing storage. Most potential iSCSI customers probably belong to this category. Taking the FC SAN point of view might be expedient for many vendors, but in the end, group #2 is probably much bigger than group #1, and my guess is that the storage folks know more about networking than the networking folks know about storage anyway. So I think that to leverage existing IP management skills, taking a server point of view will be easier to handle than a storage device point of view. The distinctions are perhaps subtle, but putting things like multiple targets into the storage array probably counts as an additional concept to the FC SAN folks, but not to the networking folks. > > Steph -- Mark A. Bakke Cisco Systems mbakke@cisco.com 763.398.1054 From owner-ips@ece.cmu.edu Tue May 15 11:12:31 2001 Received: from ece.cmu.edu ([128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id LAA29801 for ; Tue, 15 May 2001 11:12:31 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f4FDKHa12002 for ips-outgoing; Tue, 15 May 2001 09:20:17 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from chmls05.mediaone.net (chmls05.mediaone.net [24.147.1.143]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f4FDKGH11997 for ; Tue, 15 May 2001 09:20:16 -0400 (EDT) Received: from cs.uchicago.edu (h00008649aa79.ne.mediaone.net [24.91.82.73]) by chmls05.mediaone.net (8.11.1/8.11.1) with ESMTP id f4FDKEx23548 for ; Tue, 15 May 2001 09:20:15 -0400 (EDT) Message-Id: <200105151320.f4FDKEx23548@chmls05.mediaone.net> To: ips@ece.cmu.edu Subject: Re: iSCSI: Canonical Targets In-Reply-To: Message from "Peglar, Robert" of "Mon, 14 May 2001 18:39:27 CDT." References: Date: Tue, 15 May 2001 09:18:21 -0400 From: Stephen Bailey Sender: owner-ips@ece.cmu.edu Precedence: bulk Rob, > Surely you are aware that most medium-to-large-scale > storage arrays present multiple targets today, usually > in the form of multiple FC interfaces, each acting > as a target (i.e. FC WWPN)? Yes. However, this behavior is not really doing what you claim. WWPN is a way to uniquely identify an INTERFACE. From the standpoint of SAM's notion of target, you are correct that an FCP interface presents a unique `target'. In reality (one step above SAM), these boxes present a single target visible through multiple interfaces. There is no independent target addressing coordinate in FCP as there is in iSCSI or SST. What really represents the target in FCP, from an addressible entity standpoint, is the FC Node Name. The targets presented on different ports by the same box have different N_Port_Names, but they all have the same Node_Name. That means if you change the contents of a disk LUN through one port, you will see the changes (eventually) of a corresponding LUN on another port. Furthermore, there seems a strong move afoot to modify SAM to embrace this notion of multiple views of a target. Seeing a single target through multiple interfaces behavior has been the norm for enterprise storage since ||SCSI. A wedge driver (your host or HBA-level software) uses Node_Name, or more likely something from the VPD of inquiry (serial number, device Id, which probably is the FC Node_Name) to keep /dev entries straight, not N_Port_Name. The VPD information is available across transports, so you can still find the target on a box with heterogeneous links. Steph From owner-ips@ece.cmu.edu Tue May 15 14:07:13 2001 Received: from ece.cmu.edu (ECE.CMU.EDU [128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id OAA04194 for ; Tue, 15 May 2001 14:07:12 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f4FGpam29516 for ips-outgoing; Tue, 15 May 2001 12:51:36 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from sandmail.sandburst.com (sandburst-gw.bstn-gw02.ma.us.intelilink.net [216.57.129.34]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f4FGpYH29510 for ; Tue, 15 May 2001 12:51:34 -0400 (EDT) Received: from cs.uchicago.edu (dynamite-38.sandburst.com [172.16.5.38]) by sandmail.sandburst.com (Postfix) with ESMTP id 44FB394009 for ; Tue, 15 May 2001 12:51:34 -0400 (EDT) To: ips@ece.cmu.edu Subject: Re: iSCSI: Canonical Targets In-Reply-To: Message from Mark Bakke of "Tue, 15 May 2001 11:45:37 CDT." <3B015D31.C84CAFE8@cisco.com> References: <3AFC53D8.CD8A8BC4@cisco.com> <200105140255.f4E2tjx22995@chmls05.mediaone.net> <3B004061.E6019045@cisco.com> <20010514221317.7BD3F94009@sandmail.sandburst.com> <3B012D18.688CDF1E@cisco.com> <3B015D31.C84CAFE8@cisco.com> Date: Tue, 15 May 2001 12:49:40 -0400 From: Stephen Bailey Message-Id: <20010515165134.44FB394009@sandmail.sandburst.com> Sender: owner-ips@ece.cmu.edu Precedence: bulk > Any other ideas? My suggestion was to not supply any target name for a discovery login (which is how a discovery login would be distinguished) and call `iSCSI' the `default' (operational) target. Steph From owner-ips@ece.cmu.edu Tue May 15 14:07:55 2001 Received: from ece.cmu.edu (ECE.CMU.EDU [128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id OAA04206 for ; Tue, 15 May 2001 14:07:54 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f4FGmRF29264 for ips-outgoing; Tue, 15 May 2001 12:48:27 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from dogwood.cisco.com (dogwood.cisco.com [161.44.11.19]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f4FGmPH29258 for ; Tue, 15 May 2001 12:48:25 -0400 (EDT) Received: from cisco.com (mbakke@mbakke-lnx.cisco.com [161.44.68.87]) by dogwood.cisco.com (8.8.6 (PHNE_14041)/CISCO.SERVER.1.2) with ESMTP id MAA13082; Tue, 15 May 2001 12:48:14 -0400 (EDT) Message-ID: <3B015D31.C84CAFE8@cisco.com> Date: Tue, 15 May 2001 11:45:37 -0500 From: Mark Bakke X-Mailer: Mozilla 4.72 [en] (X11; U; Linux 2.2.16-3.uid32 i686) X-Accept-Language: en, de MIME-Version: 1.0 To: Stephen Bailey , ips@ece.cmu.edu Subject: Re: iSCSI: Canonical Targets References: <3AFC53D8.CD8A8BC4@cisco.com> <200105140255.f4E2tjx22995@chmls05.mediaone.net> <3B004061.E6019045@cisco.com> <20010514221317.7BD3F94009@sandmail.sandburst.com> <3B012D18.688CDF1E@cisco.com> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-ips@ece.cmu.edu Precedence: bulk Content-Transfer-Encoding: 7bit How about using the term "well-known target" instead of canonical target? It would fit in well with the networking world. Any other ideas? -- Mark > > BTW, I'm not all that attached to "canonical"; if anyone has a better > name for either the discovery target or the default operational > target, please send it to the list. > -- Mark A. Bakke Cisco Systems mbakke@cisco.com 763.398.1054 From owner-ips@ece.cmu.edu Tue May 15 14:09:15 2001 Received: from ece.cmu.edu (ECE.CMU.EDU [128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id OAA04242 for ; Tue, 15 May 2001 14:09:09 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f4FG6kL25789 for ips-outgoing; Tue, 15 May 2001 12:06:46 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from mxic2.us.dg.com (mxic2.us.dg.com [128.221.31.40]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f4FG6jH25785 for ; Tue, 15 May 2001 12:06:45 -0400 (EDT) Received: by mxic2.us.dg.com with Internet Mail Service (5.5.2650.21) id ; Tue, 15 May 2001 12:06:17 -0400 Message-ID: <0F31E5C394DAD311B60C00E029101A0708015573@corpmx9.isus.emc.com> From: Black_David@emc.com To: aboba@internaut.com, tytso@mit.edu Cc: Black_David@emc.com, ips@ece.cmu.edu Subject: RE: iSCSI Security rough consensus Date: Tue, 15 May 2001 12:06:37 -0400 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2650.21) Content-Type: text/plain Sender: owner-ips@ece.cmu.edu Precedence: bulk Bernard, > I wouldn't use "CHAP" and "keying information" in the same sentence. > CHAP can't be use to derive credible keys, and it's quite vulnerable > to dictionary attack. So it's one of the least desirable choices. No argument here. There may be some hacks to do better here, but the result will be weak security at best. > The "specifications" for SRP keying, while simpler than IKE, > will probably end up deriving something quite similar to the keying > material needed for Phase 1 and Phase 2 SAs, unless we can > convince ourselves that we really only need the equivalent of > Phase 1. And re-key is probably required, too. Actually, I think the target is phase 2 SAs - IIRC, phase 1 is IKE-only, or am I confused? For re-key, SRP generates more material than is needed to key ESP, especially with NULL encryption, and hence keeping (some of) that keying material and rehashing it (e.g., the SHA Interleave described in the SRP RFC (2945) would generate identical key sequences on both ends of the connection without a new key exchange. Using a few bits in the SPI to indicate that it's time to advance to the next key in the sequence (initiated from either side) seems sufficient. > But before we go down this road, I would *really* like to see > a requirements document, spelling out what is needed and why > in some detail. Not being a SCSI expert, it's sometimes hard > to separate what iSCSI requires from what folks think IPSEC > can deliver. If the requirements and justification were laid > out in detail, I suspect the road ahead would be more obvious. The biggest "requirement" that's lead us into this space is the fact that iSCSI can multiplex targets and initiators onto a single IP connection endpoint (i.e., ) but wants to authenticated them individually. The result is that any IPsec authentication is *not* end-to-end as it can't see the actual iSCSI endpoints, and doesn't understand what it's authenticating (e.g., if one don't know the identity of the target that will be connected to, it's impossible to send back the certificate for that target). Firewalls are the primary motivation for this, as unfortunate as that may be. This also may help with some problems related to dynamic authorization by making it possible to create a new target on the fly and connect to it in case some OS code has problems with new LUNs appearing on existing targets (consider a scenario in which a new user logs onto a host and wants access to her storage that the host did not previously have access to). The upshot is that we need an end-to-end iSCSI authentication mechanism that authenticates the iSCSI entities - authenticating the IP endpoints isn't good enough. Given this, using that end-to-end authentication to key the IP security (i.e., ESP) is natural, and significantly simpler as IKE cannot replace SRP in this context because IKE is not authenticating the iSCSI entities. For the initial version of the draft, just requiring ESP would allow those who want to use IKE to key it to do so. What becomes an RFC when will depend on how much progress gets made in various areas. I believe all of this is said or implied in the iSCSI requirements draft. --David --------------------------------------------------- David L. Black, Senior Technologist EMC Corporation, 42 South St., Hopkinton, MA 01748 +1 (508) 435-1000 x75140 FAX: +1 (508) 497-8500 black_david@emc.com Mobile: +1 (978) 394-7754 --------------------------------------------------- From owner-ips@ece.cmu.edu Tue May 15 14:55:14 2001 Received: from ece.cmu.edu (ECE.CMU.EDU [128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id OAA05136 for ; Tue, 15 May 2001 14:55:13 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f4FHPE102258 for ips-outgoing; Tue, 15 May 2001 13:25:14 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from bramg1.net.external.hp.com (bramg1.net.external.hp.com [192.6.126.73]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f4FHPCH02254 for ; Tue, 15 May 2001 13:25:13 -0400 (EDT) Received: from quasit.br.itc.hp.com (quasit.br.itc.hp.com [15.145.8.135]) by bramg1.net.external.hp.com (Postfix) with ESMTP id 4F1D668 for ; Tue, 15 May 2001 19:25:10 +0200 (METDST) Received: from loddon.br.itc.hp.com (loddon.br.itc.hp.com [15.145.8.166]) by quasit.br.itc.hp.com (8.9.3 (PHNE_18979)/8.9.3 SMKit6.0.6 OpenMail) with SMTP id SAA11555 for ; Tue, 15 May 2001 18:25:10 +0100 (BST) Received: from 15.145.8.166 by loddon.br.itc.hp.com (InterScan E-Mail VirusWall NT); Tue, 15 May 2001 18:25:10 +0100 (GMT Daylight Time) Received: by loddon.br.itc.hp.com with Internet Mail Service (5.5.2653.19) id ; Tue, 15 May 2001 18:25:10 +0100 Message-ID: <0B9A57FF1D57D411B47500D0B73E5CC101E7A6EA@dickens.bri.hp.com> From: "BURBRIDGE,MATTHEW (HP-UnitedKingdom,ex2)" To: ips@ece.cmu.edu Subject: RE: iSCSI: Canonical Targets Date: Tue, 15 May 2001 18:25:09 +0100 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2653.19) Content-Type: text/plain; charset="ISO-8859-1" Sender: owner-ips@ece.cmu.edu Precedence: bulk What about "Discovery Target"? - its simple, short and straight to the point! Matthew Burbridge NIS-Bristol Hewlett Packard Telnet: 312 7010 E-mail: matthewb@bri.hp.com > -----Original Message----- > From: Mark Bakke [mailto:mbakke@cisco.com] > Sent: 15 May 2001 17:46 > To: Stephen Bailey; ips@ece.cmu.edu > Subject: Re: iSCSI: Canonical Targets > > > > How about using the term "well-known target" instead of > canonical target? It would fit in well with the networking > world. Any other ideas? > > -- > Mark > > > > > BTW, I'm not all that attached to "canonical"; if anyone > has a better > > name for either the discovery target or the default operational > > target, please send it to the list. > > > > -- > Mark A. Bakke > Cisco Systems > mbakke@cisco.com > 763.398.1054 > From owner-ips@ece.cmu.edu Tue May 15 14:55:56 2001 Received: from ece.cmu.edu (ECE.CMU.EDU [128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id OAA05158 for ; Tue, 15 May 2001 14:55:48 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f4FHWeB02832 for ips-outgoing; Tue, 15 May 2001 13:32:40 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from mxbh4.isus.emc.com (mxbh4.isus.emc.com [128.221.10.33]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f4FHWdH02827 for ; Tue, 15 May 2001 13:32:39 -0400 (EDT) Received: by mxbh4.isus.emc.com with Internet Mail Service (5.5.2653.19) id ; Tue, 15 May 2001 13:32:29 -0400 Message-ID: <0F31E5C394DAD311B60C00E029101A0708015579@corpmx9.isus.emc.com> From: Black_David@emc.com To: ips@ece.cmu.edu Subject: DRAFT Nashua interim meeting minutes Date: Tue, 15 May 2001 13:32:28 -0400 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2653.19) Content-Type: text/plain; charset="iso-8859-1" Sender: owner-ips@ece.cmu.edu Precedence: bulk This is one loquacious bunch - sorry it took a while to get these out. WG consensus (will be considered affirmed if not objected to on the list) is flagged by **, action items are flagged by ->. Comments, corrections, etc. to the list by Friday, May 18th. Thanks, --David IETF IP Storage (ips) Working Group Nashua Interim Meeting - April 30 and May 1, 2001 DRAFT Minutes ----------------------------------------------------------- April 30, 2001 - FC encapsulation protocols (FCIP and iFCP) ----------------------------------------------------------- FC Common Encapsulation - draft-ietf-ips-fcencapsulation-00.txt ----------------------- CRC discussion - some protocols want to use a CRC, some don't, hence there's a CRC Valid flag in the header. Using one of the bits from the protocol number field to encode CRC Valid was discussed and rejected as too clever by half ** Rough Consensus: Retain CRC Valid as a separate flag as currently defined. -> Action: David Black to send email to list describing allocation approach that would (initially) avoid two protocol numbers that differ only in the encoded CRC Valid bit without permanently using two bits from the protocol number field to do this. ** Rough Consensus - Use different default TCP ports for FCIP and iFCP. -> Action: David Black to pursue causing these ports plus one for iSCSI to be allocated. -> Action: David Black will work with authors on revising the IANA text. ** Rough Consensus on the Reserved flag bits: (1) MUST be zero on send. (2) Protocol MAY specify MUST be zero on receive. (3) How to allow future protocols to make use of these? - Actual use requires modification of common encap. - Protocol-generic use of this encap SHOULD not assume that reserved flags are zero. Later discussion on use of inverted check fields ( followed by 1's complement of ) reached the following ** Rough Consensus: All uses of inverted check fields are to have the structure of a 32 bit word consisting of 16 bits of followed by 16 bits of the 1's complement of , where may consist of multiple fields. This applies to the common encapsulation header, and both the SOF and EOF encodings. Fibre Channel MIBs - General Discussion --------------------------------------- ** Rough consensus - iFCP and FCIP will do separate protocol-specific MIBs. Use existing FC MIBs (ipfc WG) for common stuff, similar to iSCSI's approach/direction of abstracting SCSI stuff into separate MIB. FCIP MIB - draft-anil-fcip-mib-00.txt -------- This is being developed as an extension of the FC Framework MIB (draft-ietf-ipfc-fsmgmt-int-mib-06.txt). Several comments were made on things needing attention in the MIB, including clarifying the distinction between port and endpoint, and the likely need for RowStatus support for row creation. TCP connection parameters are whether FCIP is configured to use them. TCP statistics - aggregated across connections that constitute a single FCIP link. This includes tracking of totals across closed TCP connections in a still-open link. Some more global suggestions: - Beware of just duplicationg TCP parameters from other MIBs. Aggregation across TCP connections in an FCIP link should be ok. - Look at RMON and follow its style to define an FCIP RMON MIB. FCIP use of Common Encapsulation -------------------------------- ** Rough Consensus: FCIP will not use CRC. The fact that this leaves the timestamp field unprotected was characterized as acceptable because the probability of an error causing a frame that should have been discarded to not be discarded is very small due to the timestamp needing to fall within an approximately 40 second window to cause the frame to be forwarded. 40 seconds is more than enough for satellite transmission - this won't allow FCIP to go to the moon, but that's probably out of scope. Protocol-specific fields for FCIP are not needed (protocol works fine if words 1 and 2 are zero), The decision on their use is: ** Rough Consensus: For FCIP, word 1 is a copy of 0, the first 16 bits of word 2 are reserved, and the last 16 bits are the 1's complement of the first 16 bits. Consensus called over Charles Monia's objection. Back to protocol-specific data. Bob S wants to use Word 1 as a copy of Word 0, and reserve Word 2 (reserved/-reserved) (replicate 3 into 2? - would consume all extra space). -> Action: David Black will work with the FCIP authors to provide right Quality of Service (diffserv) words. -- Model Update (Jim Nelson, Vixel) Work in progress based on direction Mike O'Donnell presented in Minneapolis, update prior to T11 June meeting (week of June 4 in Minneapolis - BB-2 is after FC-SW-2 on Monday, afternoon). Full model goes in FC-BB-2, a piece of this goes in FCIP to replace BSW (Backbone switch) discussion. Full discussion when the text becomes available (August, except that London IETF meeting conflicts w/T11) --> Action: David and Elizabeth are already in touch with the Area Directors regarding the IETF and T11 meeting conflict, and what to do about it. -- FCIP interaction with TCP Changes made in -02 version of FCIP draft: - Now distributing FC flows across multiple TCP connections (each port pair [N_Port login session] has to be limited to one TCP connection for in-order delivery, and FC ACKs need to be on same conn. as ACKed traffic). - Removed discussion of TCP parameters. - Error recovery (TCP keep-alive too large): 2 possible approaches -1- Fibre Channel ELS (Extended Link Sequence) to detect loss of connectivity. This would not propagate across endpoints, and a zero-length frame might be better as that would not be a valid frame to forward into Fibre Channel. -> Action: Bob Snively will check whether FSPF (Fibre Channel routing protocol that will always be present on an FCIP link as FCIP is currently specified) already contains heartbeats to detect a lost connection. If so, FCIP can leave this issue to FSPF to handle. -2- Check time stamps. Added text to indicate that horribly late frames get bit-bucketed. Common Encapsulation says "put timestamp here", but doesn't provide much in the way of details Resulting discussion was inconclusive. FC experts need to write the timestamp rules to make sure that timestamp processing adheres to FC requirements - these rules probably belong in the common encapsulation document as they apply to any Fibre Channel traffic. The following issues are open: (1) How do we ensure that the times at both ends of an FCIP or iFCP link are synchronized? FC-GS-4's time protocol or requiring use of NTP across the link are possibilities. FC B ports can't source or sink FC-GS-4 frames, so may have to use NTP. (2) Need a concise and strict definition of when it's ok to *not* timestamp traffic. Current text on loss of sync and related error recovery issues is incorrect in several ways and needs to be revised. Revison *must* not be susceptible to misinterpreting frame payload as a frame when sync is lost. Multiple TCP connections - iFCP classifies by destination - FCIP can classify by traffic class (frame type), and also by destination - FC class 4 is defining virtual channels (can have > 1 per destination). --> Action: David Black to run some version of the above summary past the Area Directors to make sure this doesn't run afoul of general principle of not using multiple TCP connections to increase a single application's performance. -- FCIP QoS and Performance -> Action: David Black will check QoS text in Section 9.1 of -02 draft and review first sentence of Section 9.2 -> Action: Authors will look at QoS text in iFCP draft. The TCP performance section of the draft contains some informational/advisory text on dealing with some TCP issues (e.g., long/fat networks [high bandwidth x delay product]). Flow control management (Section 10) This is about flow control mapping between FC and TCP. Section title is misleading and may be changed. -> Action: Remove reference to Fibre Channel Class 1. T11.3 is removing support for it in FC-MI. There are open issues in Section 10. This is about reflecting TCP traffic propagation issues into FC credit-based flow control. Source throttling. -- Data Integrity (Bob Snively, Brocade) This was a discussion of detailed edits to the draft to make sure that there are no serious date integrity issues. Section 3 error handling text will be deleted in favor of a pointer to 8.3. Section 4.3 on ordering will be connected this to the multi-path classification characteristics above. Section 8.1 - Bob Snbively wants to add ability to do data-based resync -> Action: Bob will write the text to explain how to do this for serious review. Will also rewrite the text to distinguish use of another framing mechanism vs. the data scan. Resync should be optional, and data scan must not be capable of misinterpreting what appears to be a Fibre Channel frame in an FCIP data payload as an actual frame. Section 8.3 - additional text on dealing with CRC errors for frames going onto Fibre Channel. Section 4.2 - use usual login procedure for fabric devices. FCIP need not do some sort of TCP/IP logins, but FC mechanisms must work unmodified. Endpoint definition - distinguish Fibre Channel port from TCP connection endpoint. Naming and Discovery for FCIP ----------------------------- -- FCIP and iSNS - Josh Tseng, Nishan Hostnames will name FCIP boxes - discussed as possibility in Orlando, presented as proposal at this time. DNS or iSNS can be used to resolve them - this is about tunnel configuration for FCIP. Simple DNS approach - have to enter hostnames of remote FCIP devices in each FCIP device. Each device resolves names with DNS and sets up tunnels. All configuration is manual. iSNS can automate configuration and discovery. Domain concept makes group-based managment of FCIP connectivity possible - scales better than manual configuration. iSNS server can be discovered via DHCP option or SLP. FCIP box identifiers don't have to be DNS hostnames, but those are convenient. Can automate hostname assignment to FCIP boxes. Management station can alias names. -- FCIP use of SLP - Dave Peterson, Cisco [used Excel spreadsheet + MS Word document] iSNS and SLPv2 use same authentication. Dave Peterson doesn't anticipate using SLP scopes for FCIP. New SLP RFC (3082) provides sufficient (somewhat timely) notification of devices appearing and disappearing. Proposal - static config, SLP for dynamic config, also iSNS, both are optional. DHCP - self-discovery mechanism, can't discover info about others. DNS - too limited, SLP improves LDAP - database interface, may be used by either SLP or iSNS Putting together FCIP discovery model for SLPv2. Suggestion - just use SLP, limit iSNS to functionality beyond SLP. Josh: RFC 3082 is Experimental, SLP scope is different from iSNS Discovery domains. SLP periodic re-registration has to be set correctly. SLP can't store non-string attributes (e.g., X.509 certificates). SLP is a good discovery mechanism, but not good for subsequent/sophisticated mgt. After much discussion, it was not possible to call consensus on the relationship between iSNS and SLP usage/requirements for FCIP. -> Action: Josh and Dave Petersen will post their positions to the reflector with David Black and Elizabeth summarizing areas of agreement and open issues. iFCP - Charles Monia, Nishan ----------------------------- Changes to draft - iFCP addressing model updated, including how to manage address tables, and incorporation of transparent mode. ELS handling changed for common encapsulation. No longer need extended header (this was discussed in Minneapolis). Lots of editorial changes. QoS text added, now 1 TCP/IP connection per N_Port session. Next version targeted for May 15, major work is to incorporate common encapsulation. Changes to incorporate Common Encapsulation: (1) Protocol-specific fields: - LS_CMD - identify command for Accepts of augmented ELS's. - iFCP flags (see below), copy of SOF, copy of EOF. iFCP uses these copies, and skips over SOF and EOF words. - CRC is always checked, CRCV field is ignored on receive, but must be set on send. (2) Flags - Compliance level (which version of document, watch out for version # change). This is a 1-bit version number. - Augmented (part of an ELS w/augmented data) - Address transparent vs. translation mode - iFCP session control frame (must be non-augmented, translation) Header CRC error causes connection close. CRC error in 7-word header is unlikely, and only a single N_port to N_port connection is affected. (3) Timestamp format and content will be same as FCIP. Likely to be able to put this syntax and semantics into the common encapsulation document. iFCP address-transparent mode. This assembles iFCP gateways into a single logical fabric, within which addresses are global and untranslated. Maximum of 239 gateways in a fabric (this is the FC domain limit for number of switches, each gateway counts as a switch). Address translation mode will probably be mandatory. Translation and Transparent mode do not interoperate. Transparent mode does not require any FC header or CRC changes. iFCP transparent mode is similar to FCIP, but implements FC services via IP equivalents rather than using FC switches. Gateways implement F or FL port, could even do an E port. For E port, gateway MUST become the principle switch. ELS modifications - out of 73 total ELSs, 14 of them, and 3 of their ACC (Accept) responses require iFCP intervention/changes. -> Action: Authors to look at FCP-2 document for more ELSs. REC now comes from there, and SRR appears to be missing from the list. One of the types of augmentation appends WWN to ELS sent between iFCP gateways. TPRLO (Third PaRty LogOut) can require a massive augmentation that can span multiple FC frames due to the ability to to specify up to 4095 devices to be logged out. Gateways have to maintain state to track augmented Accepts and handle them properly - this is for the gateway that proxies for target of ELS. Exchange identifier associates ACCs with ELSs on the FC side of that proxy. Fabric Service Profiles. This is about making sure that a iFCP-base fabric uniformly exports services to all attached devices, even if gateways are from different vendors. - Fabric services to be provided at well-known addresses (mandatory, optional, prohibited, behavior description for each service). - Transport (details of Class 2 and 3 behavior, prohibit Class 1, FC QoS mapping [when FC does QoS]). iFCP Naming and Discovery - Josh Tseng -------------------------------------- iSNS is required for iFCP - nothing else works. Overview of iSNS use by iFCP. Security for iFCP - use IPSec, probably tunnel mode. FCIP is leaning in the same direction. Security Comments - David Black ------------------------------- These comments apply to all IPS protocols, and are particularly applicable to the iSCSI security discussion scheduled for tomorrow. Confidentiality is not required. The IPS WG may have the freedom to "profile" IPSec, e.g., make AH optional to implement. The expensive parts of IPSec are Confidentiality (cycles) and negotiation/key exchange [IKE/ISAKMP] (code) - IKE/ISAKMP could be optional to implement (i.e., manual configuration, with attendant management problems). One of the ipsec WG chairs should be here tomorrow to provide additional information. --------------------------- May 1, 2001 - iSCSI --------------------------- iSCSI Naming and Discovery -------------------------- -- Overview - Kaladhar Voruganti, IBM -- Naming and Addressing - Mark Bakke, Cisco WWUI acronym in earlier versions of draft gone, set of naming types has been reduced to eui (WWN) and fqn. fqn acronyn will be changed to iqn (i = iSCSI) to avoid possibly rude (mis)-pronunciation An iSCSI name is a persistent location-independent identifier, an iSCSI address is how one gets to it, with DNS recommended in preference to specifying the IP address. There is a problem with domain names changing ownership without record of what iSCSI names iqns were created by the old owner. This risks unacceptable duplication. Suggestions for using IANA private enterprise identifier, date, and some sort of existing vendor identifier were made. The simple approach of using WWNs runs into administrative difficulties like a $1500 fee to get initial allocation. -> Action: Authors to propose resolution to this issue. Third party command authentication/authorization is an open issue. -- SAM2 and iSCSI (concept mapping) - Jim Hafner, IBM This is about how the concepts in the SCSI architecture (SAM2) map to the concepts in iSCSI, and the implications of this mapping. Basic concepts and mappings: o Network entity - has an IP network address (and TCP port) o iSCSI Node - within network entity, identified by iSCSI Name Contains at most ONE SCSI device (canonical iSCSI Target does not have a SCSI device). o SCSI Device Name (T10 is working on this) = iSCSI Node Name o SCSI Device - contains SCSI Ports = iSCSI Node Name + ISID or TSID. o SCSI Port has one or more pairs, may change dynamically. o SCSI I_T Nexus = iSCSI Session, connects two iSCSI Ports. Q: Multiple sessions per node pair? A: Yes, each session has unique SCSI Port instance on each side. Result is that SCSI Ports are dynamic entities. This is different from models used in other SCSI tranports because the SCSI Port entities are bound one-to-one by a connection. The iSCSI Name is intended to name OS image (actually SCSI instance in OS image) - goal here is to avoid problem with FC Node Name, which is associated with HBA - iSCSI Node Name is supposed to be associated with the "SCSI entity" in the OS. This requires the SCSI instance to manage ISIDs or TSIDs globally across all iSCSI interfaces. SCSI Port Address - not clear what this T10 concept maps to, could just use SCSI Port Name (iSCSI name + ISID or TSID). Mapping Consequences: - ISIDs have to be unique initiator-wide (across all NICs/Adapters connected to same target). - Persistent reservations get linked to both ISID and TSID -- reservation comes back only if initiatiator reuses ISID and target reuses TSID. This has implications on target saving/tracking of state, and requires initiator to remember which ISID was used with which target. -> Action: Next version of draft will need a table of how nexus state reacts to events (e.g., Resets). Discussion of Persistent Reservation behavior wrt this model. SCSI has things to say about Target reboot. Host reboot is generally not visible, but only exception is that if addresses are reassigned (e.g., by Fibre Channel fabric), Target has to cope with this reassignment. Names span sessions to make authentication simpler. Keeping ISID separate from the name makes the authentication stuff simpler. Q: Do persistent reservations open up a denial of service attack route? A: No - can't do this without prior authentication, and there are ways to blow away reservations from other initiators. Difference from FCP is that iSCSI persistent reservations are keyed to session IDs vs. FCP keying them to FC ports (via WWN). SCSI allows sharing of reservations. At the iSCSI level, each reservation is tied to a session. SCSI reservation sharing is how reservations get shared across more than one I_T_Nexus. T10 is considering reservation semantics that ignore ports and make reservations at SCSI Device level, which would yield reservations at iSCSI node granularity. On session establishment, TSID choice may be influenced by existence of reservation state - this results in iSCSI having to check for SCSI persistent reservations for that initiator name and ISID. Third party reservations were brought up. They're volatile, essentially obsolete, and not needed by anything in SCSI, included extended copy. There are also command format problems that would make it very difficult for iSCSI to pass a node name + session ID to the third party copy device. Persistent reservation key *does not* contain iSCSI node name or ISID. Cluster software (or whatever) creates its own reservation keys. --> Action: There may be an issue of scope of ACA with respect to the model. Jim Hafner will check into this. CA should not be an issue because iSCSI requires Autosense. Access controls need to use iSCSI Initiator node name. T10 will handle the required changes, as part of long identifier format changes (e.g., for things like SCSI alias designation), which also cover third-party commands. Additional address info for third-party commands could be definitive, or just hints. SRP and FCP use definitive addresses. -- Discovery - Mark Bakke, Cisco draft-ietf-ips-iscsi-name-disc-01.txt Review of types of configuration: - Static/manual - SLP for simple discovery - iSNS for centralized management "Send Targets" - an additional mechanism that can be used in all three cases. Send Targets is allowed to send info on targets that are elsewhere, or just targets behind its port. This can also implement some "soft zoning" (i.e. restricted what an Initiator can see), and cascading (return a target that is elsewhere to which "Send Targets" must be sent). If there's only one iSCSI target behind a port, a login to the "iscsi" target could return that name and immediately allow commands. This would lead to needing to distinguish between login only for "Send Targets" and login that results in the one and only target coming back and going into full-feature mode. It adds complexity but might simplify booting. OPEN ISSUE. There is a need for some sort of iterator interface or something else that avoids an Initiator having to allocate potentially unlimited space to hold the results of Send Targets. This issue will be taken to the mailing list. OPEN ISSUE. Will be adding tags to addresses that specify which addresses can be aggregated into a session. Absence of tag indicates that aggregation is not possible. Tag unique to a single address would indicate that multiple connections to that address are aupported. Only semantics of tag is whether these values match, and context is within a single response to Send Targets. One tag per address for now. The ability to specify another initiator in Send Targets will not be supported, as there is T10 work to be done in the access control area (this is about giving an Initiator's view to a third party copy device) and it raises security issues. -- SLP - Mark Bakke, Cisco draft-ietf-ips-iscsi-slp-00.txt SLP template has been submitted as WG draft. SLP and iSNS now do *exactly* the same authentication. Working on interoperability with iSNS. Working on host/device taxonomy to make recommendations about what should be implemented where (SLP and/or iSNS in drives, arrays, hosts, gateways, etc.?). Open source for SLP authentication is now available. Need to look at RFC 3082 for event propagation to see what it does and figure out how to use it. iSCSI may need a few minor extensions to this. The iSCSI SLP Template may need changes to better accomodate copy managers. None of the discovery mechanisms are REQUIRED at this point in time. A question was asked about how network management tools discover network configuration and devices - this is generally based on MIBs and reading information from them. One can have a conformance statement for a MIB that mandates a relatively small portion of it that is necessary for this sort of discovery. -- iSNS - Josh Tseng, Nishan draft-ietf-ips-isns-02.txt SNS requirements in naming and discovery draft are based on FC-GS-3. Discussion of iSNS functionality and alternatives: - LDAP: This is a directory service, but seems to lack info about when and where to send state change notifications. - SNMP: Discovery domains cannot be implemented, cannot do complex queries. - SLP: Now has a notification mechanism, but needs some changes. SLP depends on multicast (in absence of SLP, have to manually configure addresses of SLP directory agent). DHCP can discover SLP, hence can piggyback on DHCP infrastructure. SLP scopes aren't the same as discovery domains - have to add something. SLP requires on service agents to reregister with directory agents, iSNS does things in the reverse direction. Nishan will open source iSNS client and server within next month. iSCSI Requirements Last Call Issues - Marjorie Krueger, HP ----------------------------------- This discussion was about closing the issues raised during informal Last Call of this draft on the mailing list. ** Rough Consensus: iSCSI should not be making changes to SCSI-3, except where the relevant SCSI document says something is "transport dependent". Wording will be crafted to require "minimum changes" or something like that outside of this case. ** Rough Consensus: Layering wording will not be added to this document; the exhortation in the IPS WG charter is sufficient. ** Rough Consensus: "packet formats similar to" wording will NOT be added. ** Rough Consensus: Accept Bob Snively's change to require ordering only at I_T_L level, not I_T, but don't exclude implentations that do I_T ordering. After a long discussion of ordering, the conclusion was: ** Rough Consensus: Strictly ordered command delivery to target is required across a session, even in error recovery cases (this is stricter than either FCP or parallel SCSI require). Those wanting looser ordering should use multiple sessions. Rough consensus on the last point was called over visible opposition (significant minority of the room). Security -------- -- David Black (EMC, co-chair) presentation on security requirements See slides - this was an informational presentation/discussion about requirements (MUST implement authentication and cryptographic integrity, MUST have a required mechanism for each, confidentiality is OPTIONAL, IP vs. iSCSI level mechanisms [iSCSI can multiplex initiators and targets on a single pair). -- Ofer Biran (IBM, Haifa) Lots of slides - see slides for details. After much discussion ... ** Rough Consensus: (1) Need iSCSI level authentication in addition to what is done for IP or TCP/IP. (2) ESP with null authentication is mandatory to implement - TLS was considered and rejected as part of this. (3) SRP is mandatory to implement. Beyond this, the use of SRP (or another inband authentication mechanism) to key ESP was discussed as a promising direction. It's premature to require this - the minimum requirement for now is ESP with pre-shared keys and a separate implementation of SRP. -> Action: David Black, Ted Ts'o, and Ofer Birman will write up details of getting ESP key material from in-band authentication mechanism and related "Start Here" text key to turn on ESP dynamically. Error Recovery - Randy Haagens, HP and Kalman Meth, IBM ------------------------------------------------------- NOTE: Due to the late hour and diminishing attendance, no consensus calls were attempted in this area. This is a progress report from some extensive and on-going off-line work on details of errror recovery. The goal is to provide a set of mechanisms that allow implementations to choose where they want to be between "Trust TCP Explicitly" (Detect errors and flag to SCSI) and "Can't Trust TCP" (comprehensive error recovery. This work will be folded into the basic iSCSI document. Session recovery appears to be cleanly separable from finer-grained mechanisms. Not sure about separations among within-session, within-connection, and within command recovery. Not every situation will need fine-grained recovery. Session recovery will be mandatory - close session and open a new session. ** Need to make it clear that explicit logout on last connection in session. Within-session recovery reissues commands on another connection in same session - login of new connection includes instruction to logout old connection. Have to make sure that old connection is successfully logged out before issuing new commands with retry (X) bit set. This is optional. Have to use same tags and CmdSNs in doing the retry. Terminology note: This is really "resend". Whether or not a "retry" of the command is actually done is at the device's option. Need to be careful to distinguish these terms. Comment: Wording is needed to indicate that if the initiator chooses not to reissue a command that was pending on the terminated connection, it MUST terminate the session. Q: Is having both implicit and explicit logout mechanisms overkill? A: Needed to cover resend on an existing connection vs. opening a new connection for this purpose. Target can reject attempt to open new connection, and then initiator has to clean up. Comment: This portion of the draft contains a recommendation to use Target Reset that needs to be removed. Within-connection recovery reissues command on same connection to fill CmdSN holes. Has to be on same connection due to connection allegiance. These errors require framing support over TCP - in the absence of framing, the connection has to be terminated. Comment: When there are multiple connections, target needs to delay for a bit before telling Initiator that commands are missing. On a single connection, this is easy. When CmdSN has advanced past the hole on all connections, target knows that a command is missing. A request was made for an explicit way to request a resend rather than the implicit monitoring of ExpCmdSN by the Initiator and possible use of NOP by the target. This will be considered. Discussion of (lack of) trust in TCP. Between ESP at IP level and encouraging SCTP to use a CRC, we're getting stronger integrity in the stack at a lower level. TCP (or SCTP) will retry is a wonderful answer to a whole bunch of problems. This lead to some interest in figuring out how to shim a CRC between TCP and IP, which would then allow at least within-connection and within-command error recovery to be dropped. OPEN ISSUE: for further discussion. Discussion of data resends differing between reads and writes. On read, initiator resends command causing target to resend all the data. On write, the target can issue only some of the R2Ts (for the missing data) and it should not be a big deal for the initiator to cope. OPEN ISSUE - will require confirmation/further discussion on list. Status recovery within connection is based on status SNACK. -> Action: Need to revise text to indicate that SNACK is optional. SNACK is required to do within-connection recovery of status. Comment: This may be a big deal for backup - put in a negotiation key/value pair to allow initiator to discover whether target does SNACK. Backup may want to avoid using a target that doesn't do status SNACK. OPEN ISSUE: Take to list. Within-command recovery from dropped data. If data CRC fails, issue immediate Data SNACK. If header CRC fails, find the hole, issue Data SNACK at that point. From owner-ips@ece.cmu.edu Tue May 15 14:55:59 2001 Received: from ece.cmu.edu (ECE.CMU.EDU [128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id OAA05170 for ; Tue, 15 May 2001 14:55:59 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f4FHUR302649 for ips-outgoing; Tue, 15 May 2001 13:30:27 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from mxic1.isus.emc.com ([168.159.129.100]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f4FHUQH02644 for ; Tue, 15 May 2001 13:30:26 -0400 (EDT) Received: by mxic1.isus.emc.com with Internet Mail Service (5.5.2653.19) id ; Tue, 15 May 2001 13:31:59 -0400 Message-ID: <0F31E5C394DAD311B60C00E029101A0708015578@corpmx9.isus.emc.com> From: Black_David@emc.com To: mbakke@cisco.com, ips@ece.cmu.edu Subject: RE: iSCSI: Canonical Targets Date: Tue, 15 May 2001 13:30:16 -0400 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2653.19) Content-Type: text/plain Sender: owner-ips@ece.cmu.edu Precedence: bulk Mark, > 2. A canonical target MUST be accessible at the default, IANA- > assigned TCP port on each IP address on which the iSCSI > implementation is listening for iSCSI connections. Think about weakening this to talk about the default port of contact rather than requiring the one and only IANA-assigned default port to be used. This allows more than one iSCSI implementation behind a single IP address provided that the right ports are in the discovery information. This can be amazingly useful for debugging, testing, etc., and even dealing with stupid firewall tricks (e.g., some http servers ran/run at port 8080 for this reason - as long as the browser knows to go to 8080 [in URL], it makes no difference). --David --------------------------------------------------- David L. Black, Senior Technologist EMC Corporation, 42 South St., Hopkinton, MA 01748 +1 (508) 435-1000 x75140 FAX: +1 (508) 497-8500 black_david@emc.com Mobile: +1 (978) 394-7754 --------------------------------------------------- From owner-ips@ece.cmu.edu Tue May 15 17:23:48 2001 Received: from ece.cmu.edu (ECE.CMU.EDU [128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id RAA07495 for ; Tue, 15 May 2001 17:23:47 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f4FJBFL11382 for ips-outgoing; Tue, 15 May 2001 15:11:15 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from server1.NishanSystems.COM (smtp.nishansystems.com [216.217.36.162]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f4FJBDH11378 for ; Tue, 15 May 2001 15:11:13 -0400 (EDT) Received: by smtp.nishansystems.com with Internet Mail Service (5.5.2653.19) id ; Tue, 15 May 2001 12:11:07 -0700 Message-ID: From: Joshua Tseng To: "'Black_David@emc.com'" , aboba@internaut.com, tytso@mit.edu Cc: ips@ece.cmu.edu Subject: RE: iSCSI Security rough consensus Date: Tue, 15 May 2001 11:34:43 -0700 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2653.19) Content-Type: text/plain; charset="iso-8859-1" Sender: owner-ips@ece.cmu.edu Precedence: bulk David, > The upshot is that we need an end-to-end iSCSI > authentication mechanism that authenticates the iSCSI > entities - authenticating the IP endpoints isn't good enough. > Given this, using that end-to-end authentication to key the > IP security (i.e., ESP) is natural, and significantly simpler > as IKE cannot replace SRP in this context because IKE > is not authenticating the iSCSI entities. For the initial > version of the draft, just requiring ESP would allow those > who want to use IKE to key it to do so. What becomes > an RFC when will depend on how much progress gets > made in various areas. Just for clarification, SRP is only one of several "end-to-end iSCSI authentication mechanisms" listed in the -06 draft. Simple Public Key and Kerberosv5 are others. These are endpoint authentications can be conducted independently of IPSec (no keying of IPSec). Any of these, negotiated over an IKE-established IP-endpoint-to-IP-endpoint IPSec SA, would provide the needed security. This is especially true if IPSec and iSCSI are hosted on the same box, and if we discount the possibility of an attacker opening up the chassis and getting between IP and iSCSI/TCP in the stack. I think if SRP were not used to key IPSec, then IKE would be needed. On the other hand, if IKE were available, why would we need SRP to key IPSec? Josh > > I believe all of this is said or implied in the iSCSI requirements > draft. > > --David > > --------------------------------------------------- > David L. Black, Senior Technologist > EMC Corporation, 42 South St., Hopkinton, MA 01748 > +1 (508) 435-1000 x75140 FAX: +1 (508) 497-8500 > black_david@emc.com Mobile: +1 (978) 394-7754 > --------------------------------------------------- > From owner-ips@ece.cmu.edu Tue May 15 18:23:29 2001 Received: from ece.cmu.edu (ECE.CMU.EDU [128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id SAA08334 for ; Tue, 15 May 2001 18:23:29 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f4FK98a16227 for ips-outgoing; Tue, 15 May 2001 16:09:08 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from dogwood.cisco.com (dogwood.cisco.com [161.44.11.19]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f4FK97H16222 for ; Tue, 15 May 2001 16:09:07 -0400 (EDT) Received: from cisco.com (mbakke@mbakke-lnx.cisco.com [161.44.68.87]) by dogwood.cisco.com (8.8.6 (PHNE_14041)/CISCO.SERVER.1.2) with ESMTP id QAA16683 for ; Tue, 15 May 2001 16:09:01 -0400 (EDT) Message-ID: <3B018C40.DFCD2023@cisco.com> Date: Tue, 15 May 2001 15:06:24 -0500 From: Mark Bakke X-Mailer: Mozilla 4.72 [en] (X11; U; Linux 2.2.16-3.uid32 i686) X-Accept-Language: en, de MIME-Version: 1.0 To: IPS Subject: iSCSI: Aggregation tags in SendTargets Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-ips@ece.cmu.edu Precedence: bulk Content-Transfer-Encoding: 7bit During the interim meeting, we had discussed a proposal to add an aggregation tag to the SendTargets response, indicating which (if any) target addresses supported multiple connections per session, and which groups of addresses an initiator could hope to aggregate a session across. Aggregation tags were generally well-received; a small modification to the proposed method also allows an initiator to know whether a single address supports multiple connections per session just by itself. Here is the section that would go into the NDT document. -- (This would be added to section 4.2, right before the vendor-specific paragraph at the end): If an iSCSI target supports multiple connections per session, it must indicate this by including an aggregation tag after each address, in the form of TargetAddress=address,tag Where "tag" is an ASCII, alpha-numeric string indicating an address group. Within a single session, a connection may be requested to any combination of TCP addresses that have the same tag. If an address supports multiple connections per session, but does not support spanning a session across other addresses, it will have its own tag. Here is an example: TargetName=fqn.com.acme.diskarray.sn.8675309 TargetAddress=10.1.0.45:3000,1 TargetAddress=10.1.1.46:3000,1 TargetAddress=10.1.0.47:3000,2 TargetAddress=10.1.1.48:3000,2 TargetAddress=10.1.1.49:3000 TargetAddress=10.1.1.50:3000,3 TargetAlias=Oracle tables In this example, any of the target addresses can be used to reach the same target. A single-connection session can be established to any of these TCP addresses. A multiple-connection session could span addresses .45 and .46, or .47 and .48, but cannot span any other combination. A TargetAddress without a tag (.49) cannot be combined with any other address within the same session. A TargetAddress with a tag that is not shared with other addresses supports multiple connections per session, but all connections must be to the same address. To make this work, there are a few rules to follow: A target that does not support spanning sessions across multiple addresses MUST NOT include the tags. A target that is accessible via multiple TCP addresses SHOULD include all TCP addresses in a SendTargets response. A target with multiple TCP addresses that supports a session spanning multiple TCP addresses MUST indicate TCP address groups using aggregation tages in a SendTargets response. Aggregation tags have no meaning or persistence beyond a particular SendTargets response. -- Mark A. Bakke Cisco Systems mbakke@cisco.com 763.398.1054 From owner-ips@ece.cmu.edu Tue May 15 18:31:14 2001 Received: from ece.cmu.edu (ECE.CMU.EDU [128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id SAA08400 for ; Tue, 15 May 2001 18:31:13 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f4FKbmb18479 for ips-outgoing; Tue, 15 May 2001 16:37:48 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from d12lmsgate-2.de.ibm.com (d12lmsgate-2.de.ibm.com [195.212.91.200]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f4FKbjH18473 for ; Tue, 15 May 2001 16:37:45 -0400 (EDT) Received: from d12relay01.de.ibm.com (d12relay01.de.ibm.com [9.165.215.22]) by d12lmsgate-2.de.ibm.com (1.0.0) with ESMTP id WAA66096 for ; Tue, 15 May 2001 22:37:38 +0200 From: biran@il.ibm.com Received: from d12mta05.de.ibm.com (d12mta05_cs0 [9.165.222.239]) by d12relay01.de.ibm.com (8.8.8m3/NCO v4.96) with SMTP id WAA151762 for ; Tue, 15 May 2001 22:37:38 +0200 Received: by d12mta05.de.ibm.com(Lotus SMTP MTA v4.6.5 (863.2 5-20-1999)) id C1256A4D.00714CCA ; Tue, 15 May 2001 22:37:32 +0200 X-Lotus-FromDomain: IBMIL@IBMDE To: Black_David@emc.com cc: ips@ece.cmu.edu Message-ID: Date: Tue, 15 May 2001 23:30:02 +0300 Subject: Re: DRAFT Nashua interim meeting minutes Mime-Version: 1.0 Content-type: text/plain; charset=us-ascii Content-Disposition: inline Sender: owner-ips@ece.cmu.edu Precedence: bulk David, > (2) ESP with null authentication is mandatory to implement Should be ESP with null encryption (and Ofer Birman should probably be Ofer Biran...) Regards, Ofer Ofer Biran Storage and Systems Technology IBM Research Lab in Haifa biran@il.ibm.com 972-4-8296253 From owner-ips@ece.cmu.edu Tue May 15 18:40:29 2001 Received: from ece.cmu.edu (ECE.CMU.EDU [128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id SAA08471 for ; Tue, 15 May 2001 18:40:24 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f4FLQZt22370 for ips-outgoing; Tue, 15 May 2001 17:26:35 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from e31.bld.us.ibm.com (e31.co.us.ibm.com [32.97.110.129]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f4FLQXH22365 for ; Tue, 15 May 2001 17:26:33 -0400 (EDT) Received: from westrelay02.boulder.ibm.com (westrelay02.boulder.ibm.com [9.99.140.23]) by e31.bld.us.ibm.com (8.9.3/8.9.3) with ESMTP id RAA85536; Tue, 15 May 2001 17:18:58 -0400 Received: from f4n49e (d03nm065h.boulder.ibm.com [9.99.140.49]) by westrelay02.boulder.ibm.com (8.11.1/NCO v4.96.1.0) with ESMTP id f4FLQQT87886; Tue, 15 May 2001 15:26:26 -0600 X-Priority: 1 (High) Importance: Normal Subject: Re: iSCSI: Aggregation tags in SendTargets To: Mark Bakke Cc: IPS X-Mailer: Lotus Notes Release 5.0.3 (Intl) 21 March 2000 Message-ID: From: "John Hufferd" Date: Tue, 15 May 2001 14:26:14 -0700 X-MIMETrack: Serialize by Router on D03NM065/03/M/IBM(Release 5.0.6 |December 14, 2000) at 05/15/2001 03:26:25 PM MIME-Version: 1.0 Content-type: text/plain; charset=us-ascii Sender: owner-ips@ece.cmu.edu Precedence: bulk Mark, You might want to modify your first rule, it does not take into count the condition where a single address can have more then one connection to the same session, but only to the same IP address. . . . John L. Hufferd Senior Technical Staff Member (STSM) IBM/SSG San Jose Ca (408) 256-0403, Tie: 276-0403, eFax: (408) 904-4688 Internet address: hufferd@us.ibm.com Mark Bakke @ece.cmu.edu on 05/15/2001 01:06:24 PM Sent by: owner-ips@ece.cmu.edu To: IPS cc: Subject: iSCSI: Aggregation tags in SendTargets During the interim meeting, we had discussed a proposal to add an aggregation tag to the SendTargets response, indicating which (if any) target addresses supported multiple connections per session, and which groups of addresses an initiator could hope to aggregate a session across. Aggregation tags were generally well-received; a small modification to the proposed method also allows an initiator to know whether a single address supports multiple connections per session just by itself. Here is the section that would go into the NDT document. -- (This would be added to section 4.2, right before the vendor-specific paragraph at the end): If an iSCSI target supports multiple connections per session, it must indicate this by including an aggregation tag after each address, in the form of TargetAddress=address,tag Where "tag" is an ASCII, alpha-numeric string indicating an address group. Within a single session, a connection may be requested to any combination of TCP addresses that have the same tag. If an address supports multiple connections per session, but does not support spanning a session across other addresses, it will have its own tag. Here is an example: TargetName=fqn.com.acme.diskarray.sn.8675309 TargetAddress=10.1.0.45:3000,1 TargetAddress=10.1.1.46:3000,1 TargetAddress=10.1.0.47:3000,2 TargetAddress=10.1.1.48:3000,2 TargetAddress=10.1.1.49:3000 TargetAddress=10.1.1.50:3000,3 TargetAlias=Oracle tables In this example, any of the target addresses can be used to reach the same target. A single-connection session can be established to any of these TCP addresses. A multiple-connection session could span addresses .45 and .46, or .47 and .48, but cannot span any other combination. A TargetAddress without a tag (.49) cannot be combined with any other address within the same session. A TargetAddress with a tag that is not shared with other addresses supports multiple connections per session, but all connections must be to the same address. To make this work, there are a few rules to follow: A target that does not support spanning sessions across multiple addresses MUST NOT include the tags. A target that is accessible via multiple TCP addresses SHOULD include all TCP addresses in a SendTargets response. A target with multiple TCP addresses that supports a session spanning multiple TCP addresses MUST indicate TCP address groups using aggregation tages in a SendTargets response. Aggregation tags have no meaning or persistence beyond a particular SendTargets response. -- Mark A. Bakke Cisco Systems mbakke@cisco.com 763.398.1054 From owner-ips@ece.cmu.edu Tue May 15 18:40:37 2001 Received: from ece.cmu.edu (ECE.CMU.EDU [128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id SAA08487 for ; Tue, 15 May 2001 18:40:37 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f4FKtco19902 for ips-outgoing; Tue, 15 May 2001 16:55:38 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from d12lmsgate-2.de.ibm.com (d12lmsgate-2.de.ibm.com [195.212.91.200]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f4FKtZH19894 for ; Tue, 15 May 2001 16:55:36 -0400 (EDT) Received: from d12relay01.de.ibm.com (d12relay01.de.ibm.com [9.165.215.22]) by d12lmsgate-2.de.ibm.com (1.0.0) with ESMTP id WAA149650 for ; Tue, 15 May 2001 22:55:29 +0200 From: biran@il.ibm.com Received: from d12mta05.de.ibm.com (d12mta05_cs0 [9.165.222.239]) by d12relay01.de.ibm.com (8.8.8m3/NCO v4.96) with SMTP id WAA34020 for ; Tue, 15 May 2001 22:55:29 +0200 Received: by d12mta05.de.ibm.com(Lotus SMTP MTA v4.6.5 (863.2 5-20-1999)) id C1256A4D.0072EF72 ; Tue, 15 May 2001 22:55:24 +0200 X-Lotus-FromDomain: IBMIL@IBMDE To: Joshua Tseng cc: ips@ece.cmu.edu Message-ID: Date: Tue, 15 May 2001 23:48:46 +0300 Subject: RE: iSCSI Security rough consensus Mime-Version: 1.0 Content-type: text/plain; charset=us-ascii Content-Disposition: inline Sender: owner-ips@ece.cmu.edu Precedence: bulk Josh, > I think if SRP were not used to key IPSec, then IKE > would be needed. On the other hand, if IKE were available, > why would we need SRP to key IPSec? Both SRP and SRP_WITH_ESP_KEYING will be defined among the AuthMethods that can be negotiated in the Login, for now only SRP mandatory to implement. If one has ESP with IKE (or pre-keying) he can configure the negotiation not to offer/choose SRP_WITH_ESP_KEYING. However, this option might be very convenient for non-IKE implementations. Regards, Ofer Ofer Biran Storage and Systems Technology IBM Research Lab in Haifa biran@il.ibm.com 972-4-8296253 Joshua Tseng on 05/15/2001 09:34:43 PM Please respond to Joshua Tseng To: "'Black_David@emc.com'" , aboba@internaut.com, tytso@mit.edu cc: ips@ece.cmu.edu Subject: RE: iSCSI Security rough consensus From owner-ips@ece.cmu.edu Tue May 15 18:56:31 2001 Received: from ece.cmu.edu (ECE.CMU.EDU [128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id SAA08695 for ; Tue, 15 May 2001 18:56:26 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f4FL7n820878 for ips-outgoing; Tue, 15 May 2001 17:07:49 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from atlrel2.hp.com (atlrel2.hp.com [156.153.255.202]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f4FL7mH20874 for ; Tue, 15 May 2001 17:07:48 -0400 (EDT) Received: from xatlrelay1.atl.hp.com (xatlrelay1.atl.hp.com [15.45.89.190]) by atlrel2.hp.com (Postfix) with ESMTP id AFB9C238; Tue, 15 May 2001 17:07:47 -0400 (EDT) Received: from xpabh1.corp.hp.com (xpabh1.corp.hp.com [15.58.136.191]) by xatlrelay1.atl.hp.com (Postfix) with ESMTP id BBA951F510; Tue, 15 May 2001 11:06:53 -0400 (EDT) Received: by xpabh1.corp.hp.com with Internet Mail Service (5.5.2653.19) id ; Tue, 15 May 2001 14:07:45 -0700 Message-ID: <499DC368E25AD411B3F100902740AD6502D4F246@xrose03.rose.hp.com> From: "CONGDON,PAUL (HP-Roseville,ex1)" To: "'Mark Bakke'" , IPS Subject: RE: iSCSI: Canonical Targets Date: Tue, 15 May 2001 14:07:35 -0700 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2653.19) Content-Type: text/plain; charset="iso-8859-1" Sender: owner-ips@ece.cmu.edu Precedence: bulk This seems like a reasonable proposal to me, with the following comments. > > 4. A device containing a single target MUST provide both the > canonical target and the real target. (This is implied by the > above requirements). > > An initiator connecting to such a device using only its IP address > would first connect to the canonical target, and use SendTargets > to obtain the iSCSI name of the real target. It would then create > a separate session to the real target. Essentially, this means > there's nothing special about a single-target device. There needs to be an explicit SHOULD for the initiator. While I see value in having the initiator always follow a specific set of steps to establish a full feature session, it seems a bit harsh to require the a two step process when the initiator may very well know who it should connect to. > > 6. We can further specify the order in which the SendTargets response > fields are returned, to simplify things further, e.g. each target > in the SendTargets response MUST return these fields in this order: > > - A TargetName= field > - A TargetAlias= field (value left blank if there's no alias) > - One or more TargetAddress= fields > - Any vendor-specific fields (ignored by standard initiators) > The above fields should be considered a 'record' and should not be split across multiple PDUs if we decide upon an iterator. Personally, I think the idea of only allowing SendTargets on this session eliminates the need for an iterator and is a good direction to take. A complete 'record' should always be return for each target in the list. This 'record' must include the name, address and alias, but the alias value can be left blank as described above. The concept of this 'record' should also be used in Login-response PDUs in the case of Target Moves or Redirects. Again, the complete record should be returned. Note that this allows the ability to redirect the login to a completely different target. Not something that is currently possible, but would be valuable. In the current Login-response only a TargetAddress field needs to be returned, but instead it should specify an entire target record. > Additional questions to send input on: > > 1. Should a non-canonical target respond to a SendTargets command? > > 2. If so, should it respond only with addresses for its own target, > or should it respond with other targets, as a canonical target > might do? It should be allowed to respond with the complete target 'records' for other targets that this initiator has access to if it so desires. There is no reason to restrict this. Paul +--------------------------+----------------------------+ + Paul Congdon + Email: paul_congdon@hp.com + + Hewlett Packard Company + Mail Stop: 5662 + + HP ProCurve Networking + Phone: (916) 785-5753 + + 8000 Foothills Blvd + Fax: (916) 785-5949 + + Roseville, CA 95747 + Mobile: (916) 765-4056 + +--------------------------+----------------------------+ From owner-ips@ece.cmu.edu Tue May 15 22:06:40 2001 Received: from ece.cmu.edu (ECE.CMU.EDU [128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id WAA12232 for ; Tue, 15 May 2001 22:06:35 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f4FLvLE24597 for ips-outgoing; Tue, 15 May 2001 17:57:21 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from server1.NishanSystems.COM (smtp.nishansystems.com [216.217.36.162]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f4FLvDH24586 for ; Tue, 15 May 2001 17:57:14 -0400 (EDT) Received: by smtp.nishansystems.com with Internet Mail Service (5.5.2653.19) id ; Tue, 15 May 2001 14:56:12 -0700 Message-ID: From: Joshua Tseng To: "'biran@il.ibm.com'" Cc: ips@ece.cmu.edu Subject: RE: iSCSI Security rough consensus Date: Tue, 15 May 2001 14:56:10 -0700 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2653.19) Content-Type: text/plain; charset="iso-8859-1" Sender: owner-ips@ece.cmu.edu Precedence: bulk Thanks Ofer, But I don't have any problem with specifying how SRP can be used to key ESP. My question is whether we should make this a REQUIRED-TO-IMPLEMENT right now at this moment. It's obvious there needs to be some amount of work in this area before we know what we are getting with SRP_WITH_ESP_KEYING. In the meantime, IKE is mature, is integrated with IPSec, and is interoperable (as far as I know). I personally have used IKE/IPSec implementations that have been embedded in relatively small devices (< 8MB), and they worked fine as far as I could tell. I don't understand why some have asserted that it is inappropriate for embedded iSCSI devices. If this is true, then what criteria was used? And how sufficiently "simple" and "lightweight" does the approach have to be in order to be "acceptable" to iSCSI? My intuition tells me that leveraging mature, tested solutions would be the lowest-risk path for iSCSI to move forward as far as implementation complexity and future interoperability issues are concerned. I believe IKE fits the bill in this regard. SRP is still TBD. The IPS WG has always opted in favor of deployed and proven technology (such as TCP), and I see no reason to make an exception in this case. Regards, Josh > > Josh, > > > I think if SRP were not used to key IPSec, then IKE > > would be needed. On the other hand, if IKE were available, > > why would we need SRP to key IPSec? > > Both SRP and SRP_WITH_ESP_KEYING will be defined among the > AuthMethods that can be negotiated in the Login, for now only > SRP mandatory to implement. If one has ESP with IKE (or pre-keying) > he can configure the negotiation not to offer/choose > SRP_WITH_ESP_KEYING. However, this option might be very > convenient for non-IKE implementations. > > Regards, > Ofer > > > Ofer Biran > Storage and Systems Technology > IBM Research Lab in Haifa > biran@il.ibm.com 972-4-8296253 > > > Joshua Tseng on 05/15/2001 09:34:43 PM > > Please respond to Joshua Tseng > > To: "'Black_David@emc.com'" , > aboba@internaut.com, > tytso@mit.edu > cc: ips@ece.cmu.edu > Subject: RE: iSCSI Security rough consensus > > > > > From owner-ips@ece.cmu.edu Tue May 15 22:07:18 2001 Received: from ece.cmu.edu (ECE.CMU.EDU [128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id WAA12248 for ; Tue, 15 May 2001 22:06:47 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f4FMCOu25793 for ips-outgoing; Tue, 15 May 2001 18:12:24 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from med.corp.rhapsodynetworks.com (64-160-62-201.rhapsodynetworks.com [64.160.62.201] (may be forged)) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f4FMCMH25788 for ; Tue, 15 May 2001 18:12:22 -0400 (EDT) Received: by med.corp.rhapsodynetworks.com with Internet Mail Service (5.5.2653.19) id ; Tue, 15 May 2001 14:51:52 -0700 Message-ID: <45BEF1D68145D51186C100B0D0AABE6503D3ED@med.corp.rhapsodynetworks.com> From: Venkat Rangan To: "'Mark Bakke'" , Venkat Rangan Cc: "'julian_satran@il.ibm.com'" , ips@ece.cmu.edu Subject: RE: iSCSI: Nailing down CRC-32C Date: Tue, 15 May 2001 14:51:52 -0700 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2653.19) Content-Type: text/plain; charset="iso-8859-1" Sender: owner-ips@ece.cmu.edu Precedence: bulk Mark, Is there any specific advantage to iSCSI Reflection Parameters to be different from that of Ethernet? Does CRC32-C benefit by a different set of parameters? If so, what are they, and how do we get closure on a particular set of parameters? Purely from the description of the parameters, RefIn=TRUE means that each byte from the stream is 'reflected' so that bit 0 is fed in first and then bit 1 etc. RefOut=TRUE means that the 32-bit value of the CRC is also reflected before the final XOR is done. Currently, Ethernet has 'RefIn=TRUE' and 'RefOut= TRUE'. Thanks, Venkat Rangan Rhapsody Networks Inc. http://www.rhapsodynetworks.com -----Original Message----- From: Mark Bakke [mailto:mbakke@cisco.com] Sent: Tuesday, May 15, 2001 5:16 AM To: Venkat Rangan Cc: 'julian_satran@il.ibm.com'; ips@ece.cmu.edu Subject: Re: iSCSI: Nailing down CRC-32C Venkat- I assumed the same parameters (iSCSI has not yet specified the reflection parameters), and obtained the same check you did with their code. -- Mark Venkat Rangan wrote: > > All, > > Based on the finalization of the parameters of CRC-32C per following > parameters, I obtained a check of 0xE3069283 for the nine-byte string of > "123456789" (9 bytes) that the Rocksoft CRC document refers to. > > Name : "CRC-32C" > Width : 32 > Poly : 1EDC6F41 (note that the leading "1" is implied) > Init : FFFFFFFF > RefIn : True > RefOut : True > XorOut : FFFFFFFF > Check : E3069283 > > It would be nice if other independant checks can validate this. > > Venkat Rangan > Rhapsody Networks Inc. > http://www.rhapsodynetworks.com > > The table and code follows: > > /*****************************************************************/ > /* */ > /* CRC LOOKUP TABLE */ > /* ================ */ > /* The following CRC lookup table was generated automagically */ > /* by the Rocksoft^tm Model CRC Algorithm Table Generation */ > /* Program V1.0 using the following model parameters: */ > /* */ > /* Width : 4 bytes. */ > /* Poly : 0x1EDC6F41L */ > /* Reverse : TRUE. */ > /* */ > /* For more information on the Rocksoft^tm Model CRC Algorithm, */ > /* see the document titled "A Painless Guide to CRC Error */ > /* Detection Algorithms" by Ross Williams */ > /* (ross@guest.adelaide.edu.au.). This document is likely to be */ > /* in the FTP archive "ftp.adelaide.edu.au/pub/rocksoft". */ > /* */ > /*****************************************************************/ > > unsigned long crctable[256] = > { > 0x00000000L, 0xF26B8303L, 0xE13B70F7L, 0x1350F3F4L, > 0xC79A971FL, 0x35F1141CL, 0x26A1E7E8L, 0xD4CA64EBL, > 0x8AD958CFL, 0x78B2DBCCL, 0x6BE22838L, 0x9989AB3BL, > 0x4D43CFD0L, 0xBF284CD3L, 0xAC78BF27L, 0x5E133C24L, > 0x105EC76FL, 0xE235446CL, 0xF165B798L, 0x030E349BL, > 0xD7C45070L, 0x25AFD373L, 0x36FF2087L, 0xC494A384L, > 0x9A879FA0L, 0x68EC1CA3L, 0x7BBCEF57L, 0x89D76C54L, > 0x5D1D08BFL, 0xAF768BBCL, 0xBC267848L, 0x4E4DFB4BL, > 0x20BD8EDEL, 0xD2D60DDDL, 0xC186FE29L, 0x33ED7D2AL, > 0xE72719C1L, 0x154C9AC2L, 0x061C6936L, 0xF477EA35L, > 0xAA64D611L, 0x580F5512L, 0x4B5FA6E6L, 0xB93425E5L, > 0x6DFE410EL, 0x9F95C20DL, 0x8CC531F9L, 0x7EAEB2FAL, > 0x30E349B1L, 0xC288CAB2L, 0xD1D83946L, 0x23B3BA45L, > 0xF779DEAEL, 0x05125DADL, 0x1642AE59L, 0xE4292D5AL, > 0xBA3A117EL, 0x4851927DL, 0x5B016189L, 0xA96AE28AL, > 0x7DA08661L, 0x8FCB0562L, 0x9C9BF696L, 0x6EF07595L, > 0x417B1DBCL, 0xB3109EBFL, 0xA0406D4BL, 0x522BEE48L, > 0x86E18AA3L, 0x748A09A0L, 0x67DAFA54L, 0x95B17957L, > 0xCBA24573L, 0x39C9C670L, 0x2A993584L, 0xD8F2B687L, > 0x0C38D26CL, 0xFE53516FL, 0xED03A29BL, 0x1F682198L, > 0x5125DAD3L, 0xA34E59D0L, 0xB01EAA24L, 0x42752927L, > 0x96BF4DCCL, 0x64D4CECFL, 0x77843D3BL, 0x85EFBE38L, > 0xDBFC821CL, 0x2997011FL, 0x3AC7F2EBL, 0xC8AC71E8L, > 0x1C661503L, 0xEE0D9600L, 0xFD5D65F4L, 0x0F36E6F7L, > 0x61C69362L, 0x93AD1061L, 0x80FDE395L, 0x72966096L, > 0xA65C047DL, 0x5437877EL, 0x4767748AL, 0xB50CF789L, > 0xEB1FCBADL, 0x197448AEL, 0x0A24BB5AL, 0xF84F3859L, > 0x2C855CB2L, 0xDEEEDFB1L, 0xCDBE2C45L, 0x3FD5AF46L, > 0x7198540DL, 0x83F3D70EL, 0x90A324FAL, 0x62C8A7F9L, > 0xB602C312L, 0x44694011L, 0x5739B3E5L, 0xA55230E6L, > 0xFB410CC2L, 0x092A8FC1L, 0x1A7A7C35L, 0xE811FF36L, > 0x3CDB9BDDL, 0xCEB018DEL, 0xDDE0EB2AL, 0x2F8B6829L, > 0x82F63B78L, 0x709DB87BL, 0x63CD4B8FL, 0x91A6C88CL, > 0x456CAC67L, 0xB7072F64L, 0xA457DC90L, 0x563C5F93L, > 0x082F63B7L, 0xFA44E0B4L, 0xE9141340L, 0x1B7F9043L, > 0xCFB5F4A8L, 0x3DDE77ABL, 0x2E8E845FL, 0xDCE5075CL, > 0x92A8FC17L, 0x60C37F14L, 0x73938CE0L, 0x81F80FE3L, > 0x55326B08L, 0xA759E80BL, 0xB4091BFFL, 0x466298FCL, > 0x1871A4D8L, 0xEA1A27DBL, 0xF94AD42FL, 0x0B21572CL, > 0xDFEB33C7L, 0x2D80B0C4L, 0x3ED04330L, 0xCCBBC033L, > 0xA24BB5A6L, 0x502036A5L, 0x4370C551L, 0xB11B4652L, > 0x65D122B9L, 0x97BAA1BAL, 0x84EA524EL, 0x7681D14DL, > 0x2892ED69L, 0xDAF96E6AL, 0xC9A99D9EL, 0x3BC21E9DL, > 0xEF087A76L, 0x1D63F975L, 0x0E330A81L, 0xFC588982L, > 0xB21572C9L, 0x407EF1CAL, 0x532E023EL, 0xA145813DL, > 0x758FE5D6L, 0x87E466D5L, 0x94B49521L, 0x66DF1622L, > 0x38CC2A06L, 0xCAA7A905L, 0xD9F75AF1L, 0x2B9CD9F2L, > 0xFF56BD19L, 0x0D3D3E1AL, 0x1E6DCDEEL, 0xEC064EEDL, > 0xC38D26C4L, 0x31E6A5C7L, 0x22B65633L, 0xD0DDD530L, > 0x0417B1DBL, 0xF67C32D8L, 0xE52CC12CL, 0x1747422FL, > 0x49547E0BL, 0xBB3FFD08L, 0xA86F0EFCL, 0x5A048DFFL, > 0x8ECEE914L, 0x7CA56A17L, 0x6FF599E3L, 0x9D9E1AE0L, > 0xD3D3E1ABL, 0x21B862A8L, 0x32E8915CL, 0xC083125FL, > 0x144976B4L, 0xE622F5B7L, 0xF5720643L, 0x07198540L, > 0x590AB964L, 0xAB613A67L, 0xB831C993L, 0x4A5A4A90L, > 0x9E902E7BL, 0x6CFBAD78L, 0x7FAB5E8CL, 0x8DC0DD8FL, > 0xE330A81AL, 0x115B2B19L, 0x020BD8EDL, 0xF0605BEEL, > 0x24AA3F05L, 0xD6C1BC06L, 0xC5914FF2L, 0x37FACCF1L, > 0x69E9F0D5L, 0x9B8273D6L, 0x88D28022L, 0x7AB90321L, > 0xAE7367CAL, 0x5C18E4C9L, 0x4F48173DL, 0xBD23943EL, > 0xF36E6F75L, 0x0105EC76L, 0x12551F82L, 0xE03E9C81L, > 0x34F4F86AL, 0xC69F7B69L, 0xD5CF889DL, 0x27A40B9EL, > 0x79B737BAL, 0x8BDCB4B9L, 0x988C474DL, 0x6AE7C44EL, > 0xBE2DA0A5L, 0x4C4623A6L, 0x5F16D052L, 0xAD7D5351L > }; > > /*****************************************************************/ > /* End of CRC Lookup Table */ > /*****************************************************************/ > > /*****************************************************************/ > /* ComputeCRC */ > /*****************************************************************/ > #include > #include > #include "crctable.h" > > #define TB_INIT 0xFFFFFFFF > #define TB_INIT_REFLECTED 0xFFFFFFFF > #define TB_XOROT 0xFFFFFFFF > > unsigned long crc_reflected (); > unsigned long crc_reflected (blk_adr,blk_len) > unsigned char *blk_adr; > unsigned long blk_len; > { > unsigned long crc = TB_INIT_REFLECTED; > while (blk_len--) > crc = crctable[(crc ^ *blk_adr++) & 0xFFL] ^ (crc >> 8); > return crc ^ TB_XOROT; > } > > -----Original Message----- > From: julian_satran@il.ibm.com [mailto:julian_satran@il.ibm.com] > Sent: Tuesday, May 08, 2001 7:38 AM > To: ips@ece.cmu.edu > Subject: Re: iSCSI: Nailing down CRC-32C > > Mark, > > My basic assumption was that everything is (as in all IP related standards) > in network order. > > Regards, > Julo > > Mark Bakke on 07-05-2001 23:00:28 > > Please respond to Mark Bakke > > To: Julian Satran/Haifa/IBM@IBMIL > cc: ips@ece.cmu.edu > Subject: Re: iSCSI: Nailing down CRC-32C > > Julian- > > That's great; it covers nearly everything. The only thing left > is the byte order; are they taken in the same order Ethernet uses? > > If so, I can generate a few test patterns that our implementations > can all check against. > > Thanks, > > -- > Mark > > julian_satran@il.ibm.com wrote: > > > > The CRC part of the appendix (for 07) reads: > > > > The following table lists cyclic integrity checksums that can be > > negotiated for the digests and MUST be implemented by every iSCSI > > initiator and target. Note that these digest options have only error > > detection significance. > > > > +---------------------------------------------+ > > | Name | Description | > > +---------------------------------------------+ > > | crc-32C | 32 bit CRC | 11EDC6F41 | > > +---------------------------------------------+ > > | none | no digest | > > +---------------------------------------------+ > > > > The generator polynomials for those digests are given in hex-notation, > > for example 3a stands for 0011 1010 - the polynomial > x**5+X**4+x**3+x+1. > > > > The generator polynomial selected is evaluated in [Castagnioli93]. > > When using the CRC the CRC register must be initialized to all 1s > > (0xFFFFFFFF) and the CRC bits must be complemented before > transmission. > > Padding bytes, when presents in a segment covered by a CRC, have to be > > set to 0 and are included in the CRC. > > > > Regards, > > Julo > > > > Mark Bakke on 05-05-2001 00:13:09 > > > > Please respond to Mark Bakke > > > > To: IPS > > cc: > > Subject: iSCSI: Nailing down CRC-32C > > > > At the interim meeting, it was stated that iSCSI has selected > > the CRC-32C polynomial as its required iSCSI-level header and > > data CRC. Now that we have it, it's time to move on and make > > sure we can implement it. > > > > In the interest of interoperability, we need to not only specify > > the polynomial, but also the initial values, bit and byte > > ordering, etc. > > > > "A Painless Guide to CRC Error Detection Algorithms" > > (http://www.ross.net/crc/crcpaper.html) specifies a > > method to unambiguously characterize these parameters > > (sections 15 and 16). Has anyone taken a shot at defining > > these yet? Otherwise, here is what it might look like: > > > > Name : "CRC-32C" > > Width : 32 > > Poly : 1EDC6F41 (note that the leading "1" is implied) > > Init : FFFFFFFF > > RefIn : True > > RefOut : True > > XorOut : FFFFFFFF > > Check : ? > > > > I haven't attempted to create check data based on these yet. As > > soon as the other parameters are nailed down, we need to do this. > > > > Anyway, I am not a CRC expert, and can't make any statement about > > the above values being the "best" way to do this, but instead just > > copied them (except the polynomial itself) from the Ethernet CRC, > > since that is likely the easiest for everyone implementing hardware > > to deal with. > > > > If someone else is already doing this, let me know; I just wanted > > to start this thread so we can get closure. > > > > Regards, > > > > Mark > > > > -- > > Mark A. Bakke > > Cisco Systems > > mbakke@cisco.com > > 763.398.1054 > > -- > Mark A. Bakke > Cisco Systems > mbakke@cisco.com > 763.398.1054 -- Mark A. Bakke Cisco Systems mbakke@cisco.com 763.398.1054 From owner-ips@ece.cmu.edu Tue May 15 22:07:48 2001 Received: from ece.cmu.edu (ECE.CMU.EDU [128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id WAA12263 for ; Tue, 15 May 2001 22:07:25 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f4FMLW926561 for ips-outgoing; Tue, 15 May 2001 18:21:32 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from lightsand.com (alsv1-blk3-hfc-0251-d1db1add.rdc2.occa.coxatwork.com [209.219.26.221] (may be forged)) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f4FMLOH26543 for ; Tue, 15 May 2001 18:21:24 -0400 (EDT) Received: from muralipc ([192.168.2.52]) by lightsand.com (8.9.3+Sun/8.9.1) with SMTP id PAA16368; Tue, 15 May 2001 15:21:05 -0700 (PDT) From: "Murali Rajagopal" To: "Charles Monia" , Subject: iFCP Clarification Date: Tue, 15 May 2001 15:22:28 -0700 Message-ID: MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 (Normal) X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook IMO, Build 9.0.2416 (9.0.2911.0) Importance: Normal X-MimeOLE: Produced By Microsoft MimeOLE V5.00.2919.6600 In-Reply-To: <0F31E5C394DAD311B60C00E029101A0708015579@corpmx9.isus.emc.com> Sender: owner-ips@ece.cmu.edu Precedence: bulk Content-Transfer-Encoding: 7bit Charles: Can you clarify what iFCP Transparent Mode is? I was not at the Nashua meeting, but based on David's minutes it looks a lot like FCIP. Regards, Murali -----Original Message----- From: owner-ips@ece.cmu.edu [mailto:owner-ips@ece.cmu.edu]On Behalf Of Black_David@emc.com Sent: Tuesday, May 15, 2001 10:32 AM To: ips@ece.cmu.edu Subject: DRAFT Nashua interim meeting minutes This is one loquacious bunch - sorry it took a while to get these out. WG consensus (will be considered affirmed if not objected to on the list) is flagged by **, action items are flagged by ->. Comments, corrections, etc. to the list by Friday, May 18th. Thanks, --David IETF IP Storage (ips) Working Group Nashua Interim Meeting - April 30 and May 1, 2001 DRAFT Minutes ----------------------------------------------------------- April 30, 2001 - FC encapsulation protocols (FCIP and iFCP) ----------------------------------------------------------- FC Common Encapsulation - draft-ietf-ips-fcencapsulation-00.txt ----------------------- CRC discussion - some protocols want to use a CRC, some don't, hence there's a CRC Valid flag in the header. Using one of the bits from the protocol number field to encode CRC Valid was discussed and rejected as too clever by half ** Rough Consensus: Retain CRC Valid as a separate flag as currently defined. -> Action: David Black to send email to list describing allocation approach that would (initially) avoid two protocol numbers that differ only in the encoded CRC Valid bit without permanently using two bits from the protocol number field to do this. ** Rough Consensus - Use different default TCP ports for FCIP and iFCP. -> Action: David Black to pursue causing these ports plus one for iSCSI to be allocated. -> Action: David Black will work with authors on revising the IANA text. ** Rough Consensus on the Reserved flag bits: (1) MUST be zero on send. (2) Protocol MAY specify MUST be zero on receive. (3) How to allow future protocols to make use of these? - Actual use requires modification of common encap. - Protocol-generic use of this encap SHOULD not assume that reserved flags are zero. Later discussion on use of inverted check fields ( followed by 1's complement of ) reached the following ** Rough Consensus: All uses of inverted check fields are to have the structure of a 32 bit word consisting of 16 bits of followed by 16 bits of the 1's complement of , where may consist of multiple fields. This applies to the common encapsulation header, and both the SOF and EOF encodings. Fibre Channel MIBs - General Discussion --------------------------------------- ** Rough consensus - iFCP and FCIP will do separate protocol-specific MIBs. Use existing FC MIBs (ipfc WG) for common stuff, similar to iSCSI's approach/direction of abstracting SCSI stuff into separate MIB. FCIP MIB - draft-anil-fcip-mib-00.txt -------- This is being developed as an extension of the FC Framework MIB (draft-ietf-ipfc-fsmgmt-int-mib-06.txt). Several comments were made on things needing attention in the MIB, including clarifying the distinction between port and endpoint, and the likely need for RowStatus support for row creation. TCP connection parameters are whether FCIP is configured to use them. TCP statistics - aggregated across connections that constitute a single FCIP link. This includes tracking of totals across closed TCP connections in a still-open link. Some more global suggestions: - Beware of just duplicationg TCP parameters from other MIBs. Aggregation across TCP connections in an FCIP link should be ok. - Look at RMON and follow its style to define an FCIP RMON MIB. FCIP use of Common Encapsulation -------------------------------- ** Rough Consensus: FCIP will not use CRC. The fact that this leaves the timestamp field unprotected was characterized as acceptable because the probability of an error causing a frame that should have been discarded to not be discarded is very small due to the timestamp needing to fall within an approximately 40 second window to cause the frame to be forwarded. 40 seconds is more than enough for satellite transmission - this won't allow FCIP to go to the moon, but that's probably out of scope. Protocol-specific fields for FCIP are not needed (protocol works fine if words 1 and 2 are zero), The decision on their use is: ** Rough Consensus: For FCIP, word 1 is a copy of 0, the first 16 bits of word 2 are reserved, and the last 16 bits are the 1's complement of the first 16 bits. Consensus called over Charles Monia's objection. Back to protocol-specific data. Bob S wants to use Word 1 as a copy of Word 0, and reserve Word 2 (reserved/-reserved) (replicate 3 into 2? - would consume all extra space). -> Action: David Black will work with the FCIP authors to provide right Quality of Service (diffserv) words. -- Model Update (Jim Nelson, Vixel) Work in progress based on direction Mike O'Donnell presented in Minneapolis, update prior to T11 June meeting (week of June 4 in Minneapolis - BB-2 is after FC-SW-2 on Monday, afternoon). Full model goes in FC-BB-2, a piece of this goes in FCIP to replace BSW (Backbone switch) discussion. Full discussion when the text becomes available (August, except that London IETF meeting conflicts w/T11) --> Action: David and Elizabeth are already in touch with the Area Directors regarding the IETF and T11 meeting conflict, and what to do about it. -- FCIP interaction with TCP Changes made in -02 version of FCIP draft: - Now distributing FC flows across multiple TCP connections (each port pair [N_Port login session] has to be limited to one TCP connection for in-order delivery, and FC ACKs need to be on same conn. as ACKed traffic). - Removed discussion of TCP parameters. - Error recovery (TCP keep-alive too large): 2 possible approaches -1- Fibre Channel ELS (Extended Link Sequence) to detect loss of connectivity. This would not propagate across endpoints, and a zero-length frame might be better as that would not be a valid frame to forward into Fibre Channel. -> Action: Bob Snively will check whether FSPF (Fibre Channel routing protocol that will always be present on an FCIP link as FCIP is currently specified) already contains heartbeats to detect a lost connection. If so, FCIP can leave this issue to FSPF to handle. -2- Check time stamps. Added text to indicate that horribly late frames get bit-bucketed. Common Encapsulation says "put timestamp here", but doesn't provide much in the way of details Resulting discussion was inconclusive. FC experts need to write the timestamp rules to make sure that timestamp processing adheres to FC requirements - these rules probably belong in the common encapsulation document as they apply to any Fibre Channel traffic. The following issues are open: (1) How do we ensure that the times at both ends of an FCIP or iFCP link are synchronized? FC-GS-4's time protocol or requiring use of NTP across the link are possibilities. FC B ports can't source or sink FC-GS-4 frames, so may have to use NTP. (2) Need a concise and strict definition of when it's ok to *not* timestamp traffic. Current text on loss of sync and related error recovery issues is incorrect in several ways and needs to be revised. Revison *must* not be susceptible to misinterpreting frame payload as a frame when sync is lost. Multiple TCP connections - iFCP classifies by destination - FCIP can classify by traffic class (frame type), and also by destination - FC class 4 is defining virtual channels (can have > 1 per destination). --> Action: David Black to run some version of the above summary past the Area Directors to make sure this doesn't run afoul of general principle of not using multiple TCP connections to increase a single application's performance. -- FCIP QoS and Performance -> Action: David Black will check QoS text in Section 9.1 of -02 draft and review first sentence of Section 9.2 -> Action: Authors will look at QoS text in iFCP draft. The TCP performance section of the draft contains some informational/advisory text on dealing with some TCP issues (e.g., long/fat networks [high bandwidth x delay product]). Flow control management (Section 10) This is about flow control mapping between FC and TCP. Section title is misleading and may be changed. -> Action: Remove reference to Fibre Channel Class 1. T11.3 is removing support for it in FC-MI. There are open issues in Section 10. This is about reflecting TCP traffic propagation issues into FC credit-based flow control. Source throttling. -- Data Integrity (Bob Snively, Brocade) This was a discussion of detailed edits to the draft to make sure that there are no serious date integrity issues. Section 3 error handling text will be deleted in favor of a pointer to 8.3. Section 4.3 on ordering will be connected this to the multi-path classification characteristics above. Section 8.1 - Bob Snbively wants to add ability to do data-based resync -> Action: Bob will write the text to explain how to do this for serious review. Will also rewrite the text to distinguish use of another framing mechanism vs. the data scan. Resync should be optional, and data scan must not be capable of misinterpreting what appears to be a Fibre Channel frame in an FCIP data payload as an actual frame. Section 8.3 - additional text on dealing with CRC errors for frames going onto Fibre Channel. Section 4.2 - use usual login procedure for fabric devices. FCIP need not do some sort of TCP/IP logins, but FC mechanisms must work unmodified. Endpoint definition - distinguish Fibre Channel port from TCP connection endpoint. Naming and Discovery for FCIP ----------------------------- -- FCIP and iSNS - Josh Tseng, Nishan Hostnames will name FCIP boxes - discussed as possibility in Orlando, presented as proposal at this time. DNS or iSNS can be used to resolve them - this is about tunnel configuration for FCIP. Simple DNS approach - have to enter hostnames of remote FCIP devices in each FCIP device. Each device resolves names with DNS and sets up tunnels. All configuration is manual. iSNS can automate configuration and discovery. Domain concept makes group-based managment of FCIP connectivity possible - scales better than manual configuration. iSNS server can be discovered via DHCP option or SLP. FCIP box identifiers don't have to be DNS hostnames, but those are convenient. Can automate hostname assignment to FCIP boxes. Management station can alias names. -- FCIP use of SLP - Dave Peterson, Cisco [used Excel spreadsheet + MS Word document] iSNS and SLPv2 use same authentication. Dave Peterson doesn't anticipate using SLP scopes for FCIP. New SLP RFC (3082) provides sufficient (somewhat timely) notification of devices appearing and disappearing. Proposal - static config, SLP for dynamic config, also iSNS, both are optional. DHCP - self-discovery mechanism, can't discover info about others. DNS - too limited, SLP improves LDAP - database interface, may be used by either SLP or iSNS Putting together FCIP discovery model for SLPv2. Suggestion - just use SLP, limit iSNS to functionality beyond SLP. Josh: RFC 3082 is Experimental, SLP scope is different from iSNS Discovery domains. SLP periodic re-registration has to be set correctly. SLP can't store non-string attributes (e.g., X.509 certificates). SLP is a good discovery mechanism, but not good for subsequent/sophisticated mgt. After much discussion, it was not possible to call consensus on the relationship between iSNS and SLP usage/requirements for FCIP. -> Action: Josh and Dave Petersen will post their positions to the reflector with David Black and Elizabeth summarizing areas of agreement and open issues. iFCP - Charles Monia, Nishan ----------------------------- Changes to draft - iFCP addressing model updated, including how to manage address tables, and incorporation of transparent mode. ELS handling changed for common encapsulation. No longer need extended header (this was discussed in Minneapolis). Lots of editorial changes. QoS text added, now 1 TCP/IP connection per N_Port session. Next version targeted for May 15, major work is to incorporate common encapsulation. Changes to incorporate Common Encapsulation: (1) Protocol-specific fields: - LS_CMD - identify command for Accepts of augmented ELS's. - iFCP flags (see below), copy of SOF, copy of EOF. iFCP uses these copies, and skips over SOF and EOF words. - CRC is always checked, CRCV field is ignored on receive, but must be set on send. (2) Flags - Compliance level (which version of document, watch out for version # change). This is a 1-bit version number. - Augmented (part of an ELS w/augmented data) - Address transparent vs. translation mode - iFCP session control frame (must be non-augmented, translation) Header CRC error causes connection close. CRC error in 7-word header is unlikely, and only a single N_port to N_port connection is affected. (3) Timestamp format and content will be same as FCIP. Likely to be able to put this syntax and semantics into the common encapsulation document. iFCP address-transparent mode. This assembles iFCP gateways into a single logical fabric, within which addresses are global and untranslated. Maximum of 239 gateways in a fabric (this is the FC domain limit for number of switches, each gateway counts as a switch). Address translation mode will probably be mandatory. Translation and Transparent mode do not interoperate. Transparent mode does not require any FC header or CRC changes. iFCP transparent mode is similar to FCIP, but implements FC services via IP equivalents rather than using FC switches. Gateways implement F or FL port, could even do an E port. For E port, gateway MUST become the principle switch. ELS modifications - out of 73 total ELSs, 14 of them, and 3 of their ACC (Accept) responses require iFCP intervention/changes. -> Action: Authors to look at FCP-2 document for more ELSs. REC now comes from there, and SRR appears to be missing from the list. One of the types of augmentation appends WWN to ELS sent between iFCP gateways. TPRLO (Third PaRty LogOut) can require a massive augmentation that can span multiple FC frames due to the ability to to specify up to 4095 devices to be logged out. Gateways have to maintain state to track augmented Accepts and handle them properly - this is for the gateway that proxies for target of ELS. Exchange identifier associates ACCs with ELSs on the FC side of that proxy. Fabric Service Profiles. This is about making sure that a iFCP-base fabric uniformly exports services to all attached devices, even if gateways are from different vendors. - Fabric services to be provided at well-known addresses (mandatory, optional, prohibited, behavior description for each service). - Transport (details of Class 2 and 3 behavior, prohibit Class 1, FC QoS mapping [when FC does QoS]). iFCP Naming and Discovery - Josh Tseng -------------------------------------- iSNS is required for iFCP - nothing else works. Overview of iSNS use by iFCP. Security for iFCP - use IPSec, probably tunnel mode. FCIP is leaning in the same direction. Security Comments - David Black ------------------------------- These comments apply to all IPS protocols, and are particularly applicable to the iSCSI security discussion scheduled for tomorrow. Confidentiality is not required. The IPS WG may have the freedom to "profile" IPSec, e.g., make AH optional to implement. The expensive parts of IPSec are Confidentiality (cycles) and negotiation/key exchange [IKE/ISAKMP] (code) - IKE/ISAKMP could be optional to implement (i.e., manual configuration, with attendant management problems). One of the ipsec WG chairs should be here tomorrow to provide additional information. --------------------------- May 1, 2001 - iSCSI --------------------------- iSCSI Naming and Discovery -------------------------- -- Overview - Kaladhar Voruganti, IBM -- Naming and Addressing - Mark Bakke, Cisco WWUI acronym in earlier versions of draft gone, set of naming types has been reduced to eui (WWN) and fqn. fqn acronyn will be changed to iqn (i = iSCSI) to avoid possibly rude (mis)-pronunciation An iSCSI name is a persistent location-independent identifier, an iSCSI address is how one gets to it, with DNS recommended in preference to specifying the IP address. There is a problem with domain names changing ownership without record of what iSCSI names iqns were created by the old owner. This risks unacceptable duplication. Suggestions for using IANA private enterprise identifier, date, and some sort of existing vendor identifier were made. The simple approach of using WWNs runs into administrative difficulties like a $1500 fee to get initial allocation. -> Action: Authors to propose resolution to this issue. Third party command authentication/authorization is an open issue. -- SAM2 and iSCSI (concept mapping) - Jim Hafner, IBM This is about how the concepts in the SCSI architecture (SAM2) map to the concepts in iSCSI, and the implications of this mapping. Basic concepts and mappings: o Network entity - has an IP network address (and TCP port) o iSCSI Node - within network entity, identified by iSCSI Name Contains at most ONE SCSI device (canonical iSCSI Target does not have a SCSI device). o SCSI Device Name (T10 is working on this) = iSCSI Node Name o SCSI Device - contains SCSI Ports = iSCSI Node Name + ISID or TSID. o SCSI Port has one or more pairs, may change dynamically. o SCSI I_T Nexus = iSCSI Session, connects two iSCSI Ports. Q: Multiple sessions per node pair? A: Yes, each session has unique SCSI Port instance on each side. Result is that SCSI Ports are dynamic entities. This is different from models used in other SCSI tranports because the SCSI Port entities are bound one-to-one by a connection. The iSCSI Name is intended to name OS image (actually SCSI instance in OS image) - goal here is to avoid problem with FC Node Name, which is associated with HBA - iSCSI Node Name is supposed to be associated with the "SCSI entity" in the OS. This requires the SCSI instance to manage ISIDs or TSIDs globally across all iSCSI interfaces. SCSI Port Address - not clear what this T10 concept maps to, could just use SCSI Port Name (iSCSI name + ISID or TSID). Mapping Consequences: - ISIDs have to be unique initiator-wide (across all NICs/Adapters connected to same target). - Persistent reservations get linked to both ISID and TSID -- reservation comes back only if initiatiator reuses ISID and target reuses TSID. This has implications on target saving/tracking of state, and requires initiator to remember which ISID was used with which target. -> Action: Next version of draft will need a table of how nexus state reacts to events (e.g., Resets). Discussion of Persistent Reservation behavior wrt this model. SCSI has things to say about Target reboot. Host reboot is generally not visible, but only exception is that if addresses are reassigned (e.g., by Fibre Channel fabric), Target has to cope with this reassignment. Names span sessions to make authentication simpler. Keeping ISID separate from the name makes the authentication stuff simpler. Q: Do persistent reservations open up a denial of service attack route? A: No - can't do this without prior authentication, and there are ways to blow away reservations from other initiators. Difference from FCP is that iSCSI persistent reservations are keyed to session IDs vs. FCP keying them to FC ports (via WWN). SCSI allows sharing of reservations. At the iSCSI level, each reservation is tied to a session. SCSI reservation sharing is how reservations get shared across more than one I_T_Nexus. T10 is considering reservation semantics that ignore ports and make reservations at SCSI Device level, which would yield reservations at iSCSI node granularity. On session establishment, TSID choice may be influenced by existence of reservation state - this results in iSCSI having to check for SCSI persistent reservations for that initiator name and ISID. Third party reservations were brought up. They're volatile, essentially obsolete, and not needed by anything in SCSI, included extended copy. There are also command format problems that would make it very difficult for iSCSI to pass a node name + session ID to the third party copy device. Persistent reservation key *does not* contain iSCSI node name or ISID. Cluster software (or whatever) creates its own reservation keys. --> Action: There may be an issue of scope of ACA with respect to the model. Jim Hafner will check into this. CA should not be an issue because iSCSI requires Autosense. Access controls need to use iSCSI Initiator node name. T10 will handle the required changes, as part of long identifier format changes (e.g., for things like SCSI alias designation), which also cover third-party commands. Additional address info for third-party commands could be definitive, or just hints. SRP and FCP use definitive addresses. -- Discovery - Mark Bakke, Cisco draft-ietf-ips-iscsi-name-disc-01.txt Review of types of configuration: - Static/manual - SLP for simple discovery - iSNS for centralized management "Send Targets" - an additional mechanism that can be used in all three cases. Send Targets is allowed to send info on targets that are elsewhere, or just targets behind its port. This can also implement some "soft zoning" (i.e. restricted what an Initiator can see), and cascading (return a target that is elsewhere to which "Send Targets" must be sent). If there's only one iSCSI target behind a port, a login to the "iscsi" target could return that name and immediately allow commands. This would lead to needing to distinguish between login only for "Send Targets" and login that results in the one and only target coming back and going into full-feature mode. It adds complexity but might simplify booting. OPEN ISSUE. There is a need for some sort of iterator interface or something else that avoids an Initiator having to allocate potentially unlimited space to hold the results of Send Targets. This issue will be taken to the mailing list. OPEN ISSUE. Will be adding tags to addresses that specify which addresses can be aggregated into a session. Absence of tag indicates that aggregation is not possible. Tag unique to a single address would indicate that multiple connections to that address are aupported. Only semantics of tag is whether these values match, and context is within a single response to Send Targets. One tag per address for now. The ability to specify another initiator in Send Targets will not be supported, as there is T10 work to be done in the access control area (this is about giving an Initiator's view to a third party copy device) and it raises security issues. -- SLP - Mark Bakke, Cisco draft-ietf-ips-iscsi-slp-00.txt SLP template has been submitted as WG draft. SLP and iSNS now do *exactly* the same authentication. Working on interoperability with iSNS. Working on host/device taxonomy to make recommendations about what should be implemented where (SLP and/or iSNS in drives, arrays, hosts, gateways, etc.?). Open source for SLP authentication is now available. Need to look at RFC 3082 for event propagation to see what it does and figure out how to use it. iSCSI may need a few minor extensions to this. The iSCSI SLP Template may need changes to better accomodate copy managers. None of the discovery mechanisms are REQUIRED at this point in time. A question was asked about how network management tools discover network configuration and devices - this is generally based on MIBs and reading information from them. One can have a conformance statement for a MIB that mandates a relatively small portion of it that is necessary for this sort of discovery. -- iSNS - Josh Tseng, Nishan draft-ietf-ips-isns-02.txt SNS requirements in naming and discovery draft are based on FC-GS-3. Discussion of iSNS functionality and alternatives: - LDAP: This is a directory service, but seems to lack info about when and where to send state change notifications. - SNMP: Discovery domains cannot be implemented, cannot do complex queries. - SLP: Now has a notification mechanism, but needs some changes. SLP depends on multicast (in absence of SLP, have to manually configure addresses of SLP directory agent). DHCP can discover SLP, hence can piggyback on DHCP infrastructure. SLP scopes aren't the same as discovery domains - have to add something. SLP requires on service agents to reregister with directory agents, iSNS does things in the reverse direction. Nishan will open source iSNS client and server within next month. iSCSI Requirements Last Call Issues - Marjorie Krueger, HP ----------------------------------- This discussion was about closing the issues raised during informal Last Call of this draft on the mailing list. ** Rough Consensus: iSCSI should not be making changes to SCSI-3, except where the relevant SCSI document says something is "transport dependent". Wording will be crafted to require "minimum changes" or something like that outside of this case. ** Rough Consensus: Layering wording will not be added to this document; the exhortation in the IPS WG charter is sufficient. ** Rough Consensus: "packet formats similar to" wording will NOT be added. ** Rough Consensus: Accept Bob Snively's change to require ordering only at I_T_L level, not I_T, but don't exclude implentations that do I_T ordering. After a long discussion of ordering, the conclusion was: ** Rough Consensus: Strictly ordered command delivery to target is required across a session, even in error recovery cases (this is stricter than either FCP or parallel SCSI require). Those wanting looser ordering should use multiple sessions. Rough consensus on the last point was called over visible opposition (significant minority of the room). Security -------- -- David Black (EMC, co-chair) presentation on security requirements See slides - this was an informational presentation/discussion about requirements (MUST implement authentication and cryptographic integrity, MUST have a required mechanism for each, confidentiality is OPTIONAL, IP vs. iSCSI level mechanisms [iSCSI can multiplex initiators and targets on a single pair). -- Ofer Biran (IBM, Haifa) Lots of slides - see slides for details. After much discussion ... ** Rough Consensus: (1) Need iSCSI level authentication in addition to what is done for IP or TCP/IP. (2) ESP with null authentication is mandatory to implement - TLS was considered and rejected as part of this. (3) SRP is mandatory to implement. Beyond this, the use of SRP (or another inband authentication mechanism) to key ESP was discussed as a promising direction. It's premature to require this - the minimum requirement for now is ESP with pre-shared keys and a separate implementation of SRP. -> Action: David Black, Ted Ts'o, and Ofer Birman will write up details of getting ESP key material from in-band authentication mechanism and related "Start Here" text key to turn on ESP dynamically. Error Recovery - Randy Haagens, HP and Kalman Meth, IBM ------------------------------------------------------- NOTE: Due to the late hour and diminishing attendance, no consensus calls were attempted in this area. This is a progress report from some extensive and on-going off-line work on details of errror recovery. The goal is to provide a set of mechanisms that allow implementations to choose where they want to be between "Trust TCP Explicitly" (Detect errors and flag to SCSI) and "Can't Trust TCP" (comprehensive error recovery. This work will be folded into the basic iSCSI document. Session recovery appears to be cleanly separable from finer-grained mechanisms. Not sure about separations among within-session, within-connection, and within command recovery. Not every situation will need fine-grained recovery. Session recovery will be mandatory - close session and open a new session. ** Need to make it clear that explicit logout on last connection in session. Within-session recovery reissues commands on another connection in same session - login of new connection includes instruction to logout old connection. Have to make sure that old connection is successfully logged out before issuing new commands with retry (X) bit set. This is optional. Have to use same tags and CmdSNs in doing the retry. Terminology note: This is really "resend". Whether or not a "retry" of the command is actually done is at the device's option. Need to be careful to distinguish these terms. Comment: Wording is needed to indicate that if the initiator chooses not to reissue a command that was pending on the terminated connection, it MUST terminate the session. Q: Is having both implicit and explicit logout mechanisms overkill? A: Needed to cover resend on an existing connection vs. opening a new connection for this purpose. Target can reject attempt to open new connection, and then initiator has to clean up. Comment: This portion of the draft contains a recommendation to use Target Reset that needs to be removed. Within-connection recovery reissues command on same connection to fill CmdSN holes. Has to be on same connection due to connection allegiance. These errors require framing support over TCP - in the absence of framing, the connection has to be terminated. Comment: When there are multiple connections, target needs to delay for a bit before telling Initiator that commands are missing. On a single connection, this is easy. When CmdSN has advanced past the hole on all connections, target knows that a command is missing. A request was made for an explicit way to request a resend rather than the implicit monitoring of ExpCmdSN by the Initiator and possible use of NOP by the target. This will be considered. Discussion of (lack of) trust in TCP. Between ESP at IP level and encouraging SCTP to use a CRC, we're getting stronger integrity in the stack at a lower level. TCP (or SCTP) will retry is a wonderful answer to a whole bunch of problems. This lead to some interest in figuring out how to shim a CRC between TCP and IP, which would then allow at least within-connection and within-command error recovery to be dropped. OPEN ISSUE: for further discussion. Discussion of data resends differing between reads and writes. On read, initiator resends command causing target to resend all the data. On write, the target can issue only some of the R2Ts (for the missing data) and it should not be a big deal for the initiator to cope. OPEN ISSUE - will require confirmation/further discussion on list. Status recovery within connection is based on status SNACK. -> Action: Need to revise text to indicate that SNACK is optional. SNACK is required to do within-connection recovery of status. Comment: This may be a big deal for backup - put in a negotiation key/value pair to allow initiator to discover whether target does SNACK. Backup may want to avoid using a target that doesn't do status SNACK. OPEN ISSUE: Take to list. Within-command recovery from dropped data. If data CRC fails, issue immediate Data SNACK. If header CRC fails, find the hole, issue Data SNACK at that point. From owner-ips@ece.cmu.edu Tue May 15 23:23:52 2001 Received: from ece.cmu.edu (ECE.CMU.EDU [128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id XAA14295 for ; Tue, 15 May 2001 23:23:47 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f4G1OFI09469 for ips-outgoing; Tue, 15 May 2001 21:24:15 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from server1.NishanSystems.COM (smtp.nishansystems.com [216.217.36.162]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f4G1ODH09463 for ; Tue, 15 May 2001 21:24:13 -0400 (EDT) Received: by smtp.nishansystems.com with Internet Mail Service (5.5.2653.19) id ; Tue, 15 May 2001 18:07:54 -0700 Message-ID: From: Kevin Gibbons To: "'Mark Bakke'" , IPS Subject: RE: iSCSI: Aggregation tags in SendTargets Date: Tue, 15 May 2001 18:07:53 -0700 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2653.19) Content-Type: text/plain; charset="iso-8859-1" Sender: owner-ips@ece.cmu.edu Precedence: bulk Mark, in the proposed change to the NDT document, you state that the aggregation "tag" is an ASCII, alpha-numeric string. I read this to mean the tag can be any string value. Could you make the aggregation tag be a numeric string, indicating an aggregation group? This would align with your example, and should make it easier to quickly index portals into different groups. Regards, Kevin -----Original Message----- From: Mark Bakke [mailto:mbakke@cisco.com] Sent: Tuesday, May 15, 2001 1:06 PM To: IPS Subject: iSCSI: Aggregation tags in SendTargets During the interim meeting, we had discussed a proposal to add an aggregation tag to the SendTargets response, indicating which (if any) target addresses supported multiple connections per session, and which groups of addresses an initiator could hope to aggregate a session across. Aggregation tags were generally well-received; a small modification to the proposed method also allows an initiator to know whether a single address supports multiple connections per session just by itself. Here is the section that would go into the NDT document. -- (This would be added to section 4.2, right before the vendor-specific paragraph at the end): If an iSCSI target supports multiple connections per session, it must indicate this by including an aggregation tag after each address, in the form of TargetAddress=address,tag Where "tag" is an ASCII, alpha-numeric string indicating an address group. Within a single session, a connection may be requested to any combination of TCP addresses that have the same tag. If an address supports multiple connections per session, but does not support spanning a session across other addresses, it will have its own tag. Here is an example: TargetName=fqn.com.acme.diskarray.sn.8675309 TargetAddress=10.1.0.45:3000,1 TargetAddress=10.1.1.46:3000,1 TargetAddress=10.1.0.47:3000,2 TargetAddress=10.1.1.48:3000,2 TargetAddress=10.1.1.49:3000 TargetAddress=10.1.1.50:3000,3 TargetAlias=Oracle tables In this example, any of the target addresses can be used to reach the same target. A single-connection session can be established to any of these TCP addresses. A multiple-connection session could span addresses .45 and .46, or .47 and .48, but cannot span any other combination. A TargetAddress without a tag (.49) cannot be combined with any other address within the same session. A TargetAddress with a tag that is not shared with other addresses supports multiple connections per session, but all connections must be to the same address. To make this work, there are a few rules to follow: A target that does not support spanning sessions across multiple addresses MUST NOT include the tags. A target that is accessible via multiple TCP addresses SHOULD include all TCP addresses in a SendTargets response. A target with multiple TCP addresses that supports a session spanning multiple TCP addresses MUST indicate TCP address groups using aggregation tages in a SendTargets response. Aggregation tags have no meaning or persistence beyond a particular SendTargets response. -- Mark A. Bakke Cisco Systems mbakke@cisco.com 763.398.1054 From owner-ips@ece.cmu.edu Tue May 15 23:24:40 2001 Received: from ece.cmu.edu (ECE.CMU.EDU [128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id XAA14328 for ; Tue, 15 May 2001 23:24:35 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f4G19vO08495 for ips-outgoing; Tue, 15 May 2001 21:09:57 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from palrel1.hp.com (palrel1.hp.com [156.153.255.242]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f4G19tH08489 for ; Tue, 15 May 2001 21:09:55 -0400 (EDT) Received: from xparelay2.corp.hp.com (xparelay2.corp.hp.com [15.58.137.112]) by palrel1.hp.com (Postfix) with ESMTP id 729D9CA0; Tue, 15 May 2001 18:10:05 -0700 (PDT) Received: from xpabh4.corp.hp.com (xpabh4.corp.hp.com [15.58.136.1]) by xparelay2.corp.hp.com (Postfix) with ESMTP id 556751F50D; Tue, 15 May 2001 17:59:32 -0400 (EDT) Received: by xpabh4.corp.hp.com with Internet Mail Service (5.5.2653.19) id ; Tue, 15 May 2001 18:09:53 -0700 Message-ID: <499DC368E25AD411B3F100902740AD6502D4F249@xrose03.rose.hp.com> From: "CONGDON,PAUL (HP-Roseville,ex1)" To: "'John Hufferd'" , Mark Bakke Cc: IPS Subject: RE: iSCSI: Aggregation tags in SendTargets Date: Tue, 15 May 2001 17:08:01 -0700 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2653.19) Content-Type: text/plain; charset="iso-8859-1" Sender: owner-ips@ece.cmu.edu Precedence: bulk Yes, I think the first rule should be re-worded to say something like the following: A tag MUST NOT be used with addresses that do not support sessions spanning multiple connections. I also think a new rule should be added to clarify the case John is talking about. Perhaps the following: A target that supports a session spanning multiple connections, but only to the same address MUST use a unique tag. Paul +--------------------------+----------------------------+ + Paul Congdon + Email: paul_congdon@hp.com + + Hewlett Packard Company + Mail Stop: 5662 + + HP ProCurve Networking + Phone: (916) 785-5753 + + 8000 Foothills Blvd + Fax: (916) 785-5949 + + Roseville, CA 95747 + Mobile: (916) 765-4056 + +--------------------------+----------------------------+ > > Mark, > You might want to modify your first rule, it does not take > into count the > condition where a single address can have more then one > connection to the > same session, but only to the same IP address. > > . > . > . > John L. Hufferd > Senior Technical Staff Member (STSM) > IBM/SSG San Jose Ca > (408) 256-0403, Tie: 276-0403, eFax: (408) 904-4688 > Internet address: hufferd@us.ibm.com > > > Mark Bakke @ece.cmu.edu on 05/15/2001 01:06:24 PM > > Sent by: owner-ips@ece.cmu.edu > > > To: IPS > cc: > Subject: iSCSI: Aggregation tags in SendTargets > > > > > During the interim meeting, we had discussed a proposal to > add an aggregation tag to the SendTargets response, indicating > which (if any) target addresses supported multiple connections > per session, and which groups of addresses an initiator could > hope to aggregate a session across. > > Aggregation tags were generally well-received; a small modification > to the proposed method also allows an initiator to know whether > a single address supports multiple connections per session just > by itself. > > Here is the section that would go into the NDT document. > > -- > > (This would be added to section 4.2, right before the vendor-specific > paragraph at the end): > > > If an iSCSI target supports multiple connections per session, > it must indicate this by including an aggregation tag after each > address, in the form of > > TargetAddress=address,tag > > Where "tag" is an ASCII, alpha-numeric string indicating an address > group. Within a single session, a connection may be > requested to any > combination of TCP addresses that have the same tag. If an address > supports multiple connections per session, but does not support > spanning a session across other addresses, it will have its own > tag. > > Here is an example: > > TargetName=fqn.com.acme.diskarray.sn.8675309 > TargetAddress=10.1.0.45:3000,1 > TargetAddress=10.1.1.46:3000,1 > TargetAddress=10.1.0.47:3000,2 > TargetAddress=10.1.1.48:3000,2 > TargetAddress=10.1.1.49:3000 > TargetAddress=10.1.1.50:3000,3 > TargetAlias=Oracle tables > > In this example, any of the target addresses can be used to reach > the same target. A single-connection session can be established > to any of these TCP addresses. A multiple-connection > session could span > addresses .45 and .46, or .47 and .48, but cannot span any other > combination. A TargetAddress without a tag (.49) cannot be combined > with any other address within the same session. A TargetAddress > with a tag that is not shared with other addresses supports multiple > connections per session, but all connections must be to the same > address. > > To make this work, there are a few rules to follow: > > A target that does not support spanning sessions across multiple > addresses > MUST NOT include the tags. > > A target that is accessible via multiple TCP addresses > SHOULD include > all TCP addresses in a SendTargets response. > > A target with multiple TCP addresses that supports a > session spanning > multiple TCP addresses MUST indicate TCP address groups > using aggregation > tages in a SendTargets response. > > Aggregation tags have no meaning or persistence beyond a particular > SendTargets response. > > > > -- > Mark A. Bakke > Cisco Systems > mbakke@cisco.com > 763.398.1054 > > > From owner-ips@ece.cmu.edu Wed May 16 04:38:30 2001 Received: from ece.cmu.edu (ECE.CMU.EDU [128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id EAA01363 for ; Wed, 16 May 2001 04:38:30 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f4G6lXd29731 for ips-outgoing; Wed, 16 May 2001 02:47:33 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from mxic1.isus.emc.com ([168.159.129.100]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f4G6lWH29727 for ; Wed, 16 May 2001 02:47:33 -0400 (EDT) Received: by mxic1.isus.emc.com with Internet Mail Service (5.5.2653.19) id ; Wed, 16 May 2001 02:49:07 -0400 Message-ID: <0F31E5C394DAD311B60C00E029101A070801557B@corpmx9.isus.emc.com> From: Black_David@emc.com To: biran@il.ibm.com, Black_David@emc.com Cc: ips@ece.cmu.edu Subject: RE: DRAFT Nashua interim meeting minutes Date: Wed, 16 May 2001 02:47:26 -0400 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2653.19) Content-Type: text/plain Sender: owner-ips@ece.cmu.edu Precedence: bulk 2 for 2 - both changes have been made to the working version of the minutes. Thanks, --David > -----Original Message----- > From: biran@il.ibm.com [SMTP:biran@il.ibm.com] > Sent: Tuesday, May 15, 2001 4:30 PM > To: Black_David@emc.com > Cc: ips@ece.cmu.edu > Subject: Re: DRAFT Nashua interim meeting minutes > > > > David, > > > (2) ESP with null authentication is mandatory to implement > > Should be ESP with null encryption > (and Ofer Birman should probably be Ofer Biran...) > > Regards, > Ofer > > Ofer Biran > Storage and Systems Technology > IBM Research Lab in Haifa > biran@il.ibm.com 972-4-8296253 > > From owner-ips@ece.cmu.edu Wed May 16 04:42:10 2001 Received: from ece.cmu.edu (ECE.CMU.EDU [128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id EAA01429 for ; Wed, 16 May 2001 04:42:06 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f4G6skf00200 for ips-outgoing; Wed, 16 May 2001 02:54:46 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from mxic2.us.dg.com (mxic2.us.dg.com [128.221.31.40]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f4G6sjH00194 for ; Wed, 16 May 2001 02:54:45 -0400 (EDT) Received: by mxic2.us.dg.com with Internet Mail Service (5.5.2653.19) id ; Wed, 16 May 2001 02:54:17 -0400 Message-ID: <0F31E5C394DAD311B60C00E029101A070801557E@corpmx9.isus.emc.com> From: Black_David@emc.com To: ips@ece.cmu.edu Subject: IPS WG meeting(s) update Date: Wed, 16 May 2001 02:54:38 -0400 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2653.19) Content-Type: text/plain Sender: owner-ips@ece.cmu.edu Precedence: bulk Here's an important update on future IPS WG meetings: (1) The August IETF meeting week in London is at the same time as the T11 meetings in Hawaii. This makes it impossible to take up Fibre Channel- related work during the August IPS WG meeting(s) in London, and hence those meeting(s) will focus on getting at least the basic iSCSI protocol draft (and others as feasible/necessary - the naming and discovery draft is likely) as close to ready for Last Call as possible. (2) There will be an FC-focused interim meeting sometime after the London meetings that will focus on getting the basic FC-related protocol drafts ready for last call. The Area Directors have suggested that IPS-wide security architecture and protocol selection would also be an appropriate topic for that meeting, so the structure may be a day or so on the FC-related drafts and the better part of a day on security. The date(s) and location will be announced in the next few weeks. I want to extend apologies on behalf of Elizabeth and myself for comments made in Nashua about other possible interim meeting dates and locations, especially if they've mislead anyone into making plans that now need to be changed. IETF interim meetings require the approval of the Area Directors, and after consultation with the ADs, the above plan is what we're going to do. It does have the merit that we don't have to try to get all of the protocols into a close to done state at the same meeting in London (whew!). Thanks and sorry, --David --------------------------------------------------- David L. Black, Senior Technologist EMC Corporation, 42 South St., Hopkinton, MA 01748 +1 (508) 435-1000 x75140 FAX: +1 (508) 497-8500 black_david@emc.com Mobile: +1 (978) 394-7754 --------------------------------------------------- From owner-ips@ece.cmu.edu Wed May 16 04:42:28 2001 Received: from ece.cmu.edu (ECE.CMU.EDU [128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id EAA01440 for ; Wed, 16 May 2001 04:42:28 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f4G6qAV00037 for ips-outgoing; Wed, 16 May 2001 02:52:10 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from mxbh4.isus.emc.com (mxbh4.isus.emc.com [128.221.10.33]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f4G6q8H00033 for ; Wed, 16 May 2001 02:52:08 -0400 (EDT) Received: by mxbh4.isus.emc.com with Internet Mail Service (5.5.2653.19) id ; Wed, 16 May 2001 02:52:03 -0400 Message-ID: <0F31E5C394DAD311B60C00E029101A070801557C@corpmx9.isus.emc.com> From: Black_David@emc.com To: jtseng@NishanSystems.com, biran@il.ibm.com Cc: ips@ece.cmu.edu Subject: RE: iSCSI Security rough consensus Date: Wed, 16 May 2001 02:52:01 -0400 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2653.19) Content-Type: text/plain Sender: owner-ips@ece.cmu.edu Precedence: bulk > But I don't have any problem with specifying how > SRP can be used to key ESP. My question is whether > we should make this a REQUIRED-TO-IMPLEMENT right now > at this moment. That is definitely NOT going to happen at this moment. At a minimum, a draft specifying the details sufficient to allow review by security experts is necessary, and then what to do with that draft becomes something for the WG to take up. Ok? --David --------------------------------------------------- David L. Black, Senior Technologist EMC Corporation, 42 South St., Hopkinton, MA 01748 +1 (508) 435-1000 x75140 FAX: +1 (508) 497-8500 black_david@emc.com Mobile: +1 (978) 394-7754 --------------------------------------------------- From owner-ips@ece.cmu.edu Wed May 16 04:42:34 2001 Received: from ece.cmu.edu (ECE.CMU.EDU [128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id EAA01451 for ; Wed, 16 May 2001 04:42:30 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f4G6ilx29562 for ips-outgoing; Wed, 16 May 2001 02:44:47 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from mxic1.isus.emc.com ([168.159.129.100]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f4G6ikH29544 for ; Wed, 16 May 2001 02:44:46 -0400 (EDT) Received: by mxic1.isus.emc.com with Internet Mail Service (5.5.2653.19) id ; Wed, 16 May 2001 02:46:20 -0400 Message-ID: <0F31E5C394DAD311B60C00E029101A070801557A@corpmx9.isus.emc.com> From: Black_David@emc.com To: jtseng@NishanSystems.com, Black_David@emc.com, aboba@internaut.com, tytso@mit.edu Cc: ips@ece.cmu.edu Subject: RE: iSCSI Security rough consensus Date: Wed, 16 May 2001 02:44:38 -0400 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2653.19) Content-Type: text/plain Sender: owner-ips@ece.cmu.edu Precedence: bulk > Just for clarification, SRP is only one of several > "end-to-end iSCSI authentication mechanisms" listed > in the -06 draft. It's also the one that the interim meeting proposes to make MANDATORY to implement. Based on what we can put in the iSCSI draft now, using IKE to key ESP is acceptable. > I think if SRP were not used to key IPSec, then IKE > would be needed. I don't believe that to be the case, although I'll defer to others on exactly how pre-shared keys are used. > On the other hand, if IKE were available, > why would we need SRP to key IPSec? I think this has been answered already. SRP is end-to-end, ensuring that any SA it keys is end to end. ESP in tunnel mode keyed by IKE need not be end-to-end because any intermediate security gateways will have IKE. --David --------------------------------------------------- David L. Black, Senior Technologist EMC Corporation, 42 South St., Hopkinton, MA 01748 +1 (508) 435-1000 x75140 FAX: +1 (508) 497-8500 black_david@emc.com Mobile: +1 (978) 394-7754 --------------------------------------------------- From owner-ips@ece.cmu.edu Wed May 16 04:45:39 2001 Received: from ece.cmu.edu (ECE.CMU.EDU [128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id EAA01486 for ; Wed, 16 May 2001 04:45:39 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f4G6sAY00166 for ips-outgoing; Wed, 16 May 2001 02:54:10 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from mxbh4.isus.emc.com (mxbh4.isus.emc.com [128.221.10.33]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f4G6s8H00162 for ; Wed, 16 May 2001 02:54:08 -0400 (EDT) Received: by mxbh4.isus.emc.com with Internet Mail Service (5.5.2653.19) id ; Wed, 16 May 2001 02:54:03 -0400 Message-ID: <0F31E5C394DAD311B60C00E029101A070801557D@corpmx9.isus.emc.com> From: Black_David@emc.com To: ips@ece.cmu.edu Subject: iSCSI Requirements Last Call status Date: Wed, 16 May 2001 02:54:02 -0400 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2653.19) Content-Type: text/plain Sender: owner-ips@ece.cmu.edu Precedence: bulk The informal Last Call on the list closed on April 27th, prior to the interim meeting. Enough comments were received and issues raised that another version of the draft will be forthcoming (and many thanks to Marjorie for her patience with this). The intent is to (briefly) review that version on the list to make sure that all the comments and issues are satisfactorily addressed (including issues discussed at the interim meeting) and then submit the draft for publication as an informational RFC. Thanks, --David --------------------------------------------------- David L. Black, Senior Technologist EMC Corporation, 42 South St., Hopkinton, MA 01748 +1 (508) 435-1000 x75140 FAX: +1 (508) 497-8500 black_david@emc.com Mobile: +1 (978) 394-7754 --------------------------------------------------- From owner-ips@ece.cmu.edu Wed May 16 07:07:05 2001 Received: from ece.cmu.edu (ECE.CMU.EDU [128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id HAA02800 for ; Wed, 16 May 2001 07:07:04 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f4G8udi11833 for ips-outgoing; Wed, 16 May 2001 04:56:39 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from internaut.com ([64.38.134.99]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f4G8uaH11809 for ; Wed, 16 May 2001 04:56:37 -0400 (EDT) Received: from localhost (aboba@localhost) by internaut.com (8.9.3/8.9.3) with ESMTP id BAA24694; Wed, 16 May 2001 01:47:12 -0700 (PDT) (envelope-from aboba@internaut.com) Date: Wed, 16 May 2001 01:47:12 -0700 (PDT) From: Bernard Aboba To: Black_David@emc.com cc: jtseng@NishanSystems.com, biran@il.ibm.com, ips@ece.cmu.edu Subject: RE: iSCSI Security rough consensus In-Reply-To: <0F31E5C394DAD311B60C00E029101A070801557C@corpmx9.isus.emc.com> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-ips@ece.cmu.edu Precedence: bulk > minimum, a draft specifying the details sufficient to allow review > by security experts is necessary, and then what to do with that > draft becomes something for the WG to take up. Ok? > Do we have a set of guidelines about what this draft is supposed to achieve? For example: 1. Do we need to support negotiation of SRP prime modulus/generator groups from within the standard set? 2. Do we need to generate keying material for Phase 1 as well as Phase 2 SAs? 3. Do we need to support rekeys? 4. Do we need to support ciphersuite negotiations? 5. Is there a need to negotiate filters? 6. Is there a need for concealment of identities? In other words, how much of IKE are you willing to throw away? While a lot of IKE complexity is due to artifacts of now discarded layering, a lot of it is also due to the generality of what it tries to achieve. If you want less features, you can get a lighter weight protocol... From owner-ips@ece.cmu.edu Wed May 16 14:08:24 2001 Received: from ece.cmu.edu (ECE.CMU.EDU [128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id OAA15764 for ; Wed, 16 May 2001 14:08:23 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f4GGitR21716 for ips-outgoing; Wed, 16 May 2001 12:44:55 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from e32.bld.us.ibm.com (e32.co.us.ibm.com [32.97.110.130]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f4GGirH21712 for ; Wed, 16 May 2001 12:44:53 -0400 (EDT) Received: from westrelay02.boulder.ibm.com (westrelay02.boulder.ibm.com [9.99.140.23]) by e32.bld.us.ibm.com (8.9.3/8.9.3) with ESMTP id MAA10544 for ; Wed, 16 May 2001 12:45:35 -0400 Received: from f3n42e (d03nm042h.boulder.ibm.com [9.99.140.42]) by westrelay02.boulder.ibm.com (8.11.1/NCO v4.96.1.0) with ESMTP id f4GGhYR184034 for ; Wed, 16 May 2001 10:43:34 -0600 Importance: Normal Subject: iSCSI: formats for packing long iSCSI names into SCSI alias commands To: ips@ece.cmu.edu From: "Jim Hafner" Date: Wed, 16 May 2001 09:43:33 -0700 Message-ID: X-MIMETrack: Serialize by Router on D03NM042/03/M/IBM(Release 5.0.6 |December 14, 2000) at 05/16/2001 09:43:34 AM MIME-Version: 1.0 Content-type: text/plain; charset=us-ascii Sender: owner-ips@ece.cmu.edu Precedence: bulk Folks, I've been working with T10 on a proposal (T10/00-424r3) to allow iSCSI's long names to work with some of the limitations of parameter data structures in some third party commands (most notably EXTENDED COPY). The proposal is progressing forward and I'm hoping to get approval in July. One key aspect of the proposal is that transport protocols (like iSCSI) must define, in their own documents, what are called the "designation formats" that apply to that transport. These formats define the way name and address constructs are embedded in parameter data (in particular in the CHANGE and REPORT ALIASES commands defined in the proposal). The document contains an informative clause with my proposal for iSCSI formats. (It's informative since T10 can't approve the formats; we have to in this WG as part of the iSCSI spec.) I've extracted and reformatted for this list (and to some extent for the iSCSI documents), the iSCSI-specific clause of that proposal (see ftp://ftp.t10.org/t10/document.00/00-425r3.pdf for the latest draft of the complete proposal). The style is more for T10, use of keywords is different, etc. I'm posting it hear for general airing. Suggestions for editorial, as well as technical changes are welcome. Open issues: 1) Does this cover the cases that we want to cover? Are their other formats we need with different data? Are all of these needed? Do we want to merge the IPv4 and IPv6 into one format with a flag bit somewhere indicating which one it is? 2) Is the technical description OK? 3) What iSCSI document and where in that document should this stuff go? It is becoming apparent that some document needs a description of the relationship of iSCSI with SCSI (SAM) (perhaps an annex to the main document). It is natural for me that this would go there, but others may have a different opinion. 4) Other issues in this regard? Jim Hafner Clause Y. iSCSI protocol specific alias entry formats Y.1 iSCSI protocol specific format codes This clause defines the iSCSI protocol specific alias entry formats and codes used in the CHANGE ALIASES and REPORT ALIASES commands (see SPC-3) to designate SCSI devices or ports on an iSCSI service delivery subsystem. For an iSCSI protocol specific alias entry, the Protocol Identifier shall be set to 05h (as defined in Table 165 of SPC-2 rev 19) and the Format Code values are defined in Table Y1. Table Y1: iSCSI protocol specific Format Code values and Designation formats. ----------------------------------------------------------------------------- Format | Designation | Designation | Designation content Coce | Description | length | | | (maximum) | ----------------------------------------------------------------------------- 00h | iSCSI Name | 256 bytes | Name in UTF-8 format (null | | | terminated), with pad (see Y.2) ----------------------------------------------------------------------------- 01h | iSCSI Name | 268 bytes | Name in UTF-8 format (null | with binary | | terminated), binary IPv4 address, | IPv4 address | | binary TCP port, binary Internet | | | Protocol Number, with pad (see Y.3) ----------------------------------------------------------------------------- 02h | iSCSI Name | 520 bytes | Name in UTF-8 format (null | with ASCII | | terminated), IPname (null | IPName | | terminated), binary TCP port, | | | binary Internet Protocol Number, | | | with pad (see Y.4) ------------------------------------------------------------------------------ 03h | iSCSI Name | 280 bytes | Name in UTF-8 format (null | with binary | | terminated), binary IPv6 address, | IPv6 address | | binary TCP port, binary Internet | | | Protocol Number, with pad (see Y.5) ----------------------------------------------------------------------------- 04h-FFh | reserved | n/a | n/a ----------------------------------------------------------------------------- NOTE: a designation that contains no IP addressing information or contains IP addressing information that does not address the named SCSI device may require a device server to have access to a name server or to other discovery protocols to resolve the given iSCSI Name to an IP address through which the device server may establish iSCSI Login. Access to such a service is protocol and implementation specific. Y.2 iSCSI Name designation format Table Y2 describes the iSCSI Name designation format. Table Y2. iSCSI Name designation format. Byte / 0 | 1 | 2 | 3 | / | | | | |7 6 5 4 3 2 1 0|7 6 5 4 3 2 1 0|7 6 5 4 3 2 1 0|7 6 5 4 3 2 1 0| +---------------+---------------+---------------+---------------+ 0/ iSCSI Name | +/ +---------------+-------------------------------+ +/ | NULL(0h) | Pad (if needed) (0h) | +---------------+---------------+---------------+---------------+ The iSCSI Name field shall contain the iSCSI Name of an iSCSI Node. Refer to RFC XXXX for a description of iSCSI Names. A Null (0h) byte shall terminate the iSCSI Name. Zero to three bytes set to zero shall be appended in the Pad field so that the total length of the designation is a multiple of 4. An iSCSI Name designation is valid if the device server has access to a SCSI domain containing an IP network and there exists an iSCSI Node on that network with the specified iSCSI Name. [AUTHOR'S NOTE: the above paragraph there to define the rules under which a designation is "valid" or "invalid". This terminology is used in the T10 proposal (00-425r3) in contexts where the designation is used. Similar paragraphs will be found in the other subsections.] Y.3 iSCSI Name with binary IPv4 address designation format Table Y3 describes the iSCSI Name with binary IPv4 address designation format. Table Y3. iSCSI Name with binary IPv4 address designation format. Byte / 0 | 1 | 2 | 3 | / | | | | |7 6 5 4 3 2 1 0|7 6 5 4 3 2 1 0|7 6 5 4 3 2 1 0|7 6 5 4 3 2 1 0| +---------------+---------------+---------------+---------------+ 0/ iSCSI Name | +/ +---------------+-------------------------------+ +/ | NULL(0h) | Pad (if needed) (0h) | +---------------+---------------+---------------+---------------+ 4k| IPv4 Address | +---------------+---------------+---------------+---------------+ 4k+4| Reserved (0h) | Port Number | +---------------+---------------+---------------+---------------+ 4k+8| Reserved (0h) | Internet Protocol Number | +---------------+---------------+---------------+---------------+ The iSCSI Name field shall contain the iSCSI Name of an iSCSI Node. Refer to RFC XXXX for a description of iSCSI Names. A Null (0h) byte shall terminate the iSCSI Name. Zero to three bytes set to zero shall be inserted in the Pad field so that the total length of the designation is a multiple of 4. The IPv4 Address field shall contain an IPv4 address. Refer to RFC 791 for a description of IPv4 addresses. The Port Number field shall contain a port number. Refer to RFC 790 for a description of port numbers. The Internet Protocol Number field shall contain an Internet protocol number. Refer to RFC 790 for a description of Internet protocol numbers. An iSCSI Name with binary IPv4 address designation is valid if the device server has access to a SCSI domain containing an IP network and there exists an iSCSI Node on that network with the specified iSCSI Name. The IPv4 address, port number and internet protocol number provided in the designation may be used by a device server for addressing to discover and establish communication with the named iSCSI Node. Alternatively, the device server may use other protocol specific or vendor specific methods to discover and establish communication with the named iSCSI Node. Y.4 iSCSI Name with IPname designation format Table Y4 describes the iSCSI Name with IPname designation format. Table Y4. iSCSI Name with IPname designation format. Byte / 0 | 1 | 2 | 3 | / | | | | |7 6 5 4 3 2 1 0|7 6 5 4 3 2 1 0|7 6 5 4 3 2 1 0|7 6 5 4 3 2 1 0| +---------------+---------------+---------------+---------------+ 0/ iSCSI Name | +/ +-------------------------------+ +/ | NULL1(0h) | IPName | +---------------+---------------+---------------+ + / | +/ +---------------+-------------------------------+ +/ | NULL2(0h) | Pad (if needed) (0h) | +---------------+---------------+---------------+---------------+ 4k| Reserved (0h) | Port Number | +---------------+---------------+---------------+---------------+ 4k+4| Reserved (0h) | Internet Protocol Number | +---------------+---------------+---------------+---------------+ The iSCSI Name field shall contain the iSCSI Name of an iSCSI Node. Refer to RFC XXXX for a description of iSCSI Names. A Null1 (0h) byte shall terminate the iSCSI Name. The IPName field shall contain a Internet protocol domain name. Refer to RFC 1035 for a description of domain names. A Null2 (0h) byte shall terminate the Internet protocol domain name. Zero to three bytes set to zero shall be inserted in the Pad field so that the total length of the designation is a multiple of 4. An iSCSI Name with IPname designation is valid if the device server has access to a SCSI domain containing an IP network and there exists an iSCSI Node on that network with the specified iSCSI Name. The domain name, port number and internet protocol number provided in the designation may be used by a device server for addressing to discover and establish communication with the named iSCSI Node. Alternatively, the device server may use other protocol specific or vendor specific methods to discover and establish communication with the named iSCSI Node. Y.5 iSCSI Name with binary IPv6 address designation format Table Y5 describes the iSCSI Name with binary IPv6 address designation format. Table Y5. iSCSI Name with binary IPv6 adddress designation format. Byte / 0 | 1 | 2 | 3 | / | | | | |7 6 5 4 3 2 1 0|7 6 5 4 3 2 1 0|7 6 5 4 3 2 1 0|7 6 5 4 3 2 1 0| +---------------+---------------+---------------+---------------+ 0/ iSCSI Name | +/ +---------------+-------------------------------+ +/ | NULL(0h) | Pad (if needed) (0h) | +---------------+---------------+---------------+---------------+ 4k/ IPv6 Address | +/ | +---------------+---------------+---------------+---------------+ 4k+16| Reserved (0h) | Port Number | +---------------+---------------+---------------+---------------+ 4k+20| Reserved (0h) | Internet Protocol Number | +---------------+---------------+---------------+---------------+ The iSCSI Name field shall contain the iSCSI Name of an iSCSI Node. Refer to RFC XXXX for a description of iSCSI Names. A Null (0h) byte shall terminate the iSCSI Name. Zero to three bytes set to zero shall be inserted in the Pad field so that the total length of the designation is a multiple of 4. The IPv6 Address field shall contain an IPv6 address. Refer to RFC 2373 for a description of the IPv6 address. The Port Number field shall contain a port number. Refer to RFC 790 for a description of port numbers. The Internet Protocol Number field shall contain an Internet protocol number. Refer to RFC 790 for a description of Internet protocol numbers. An iSCSI Name with binary IPv6 address designation is valid if the device server has access to a SCSI domain containing an IP network and there exists an iSCSI Node on that network with the specified iSCSI Name. The IPv6 address, port number and internet protocol number provided in the designation may be used by a device server for addressing to discover and establish communication with the named iSCSI Node. Alternatively, the device server may use other protocol specific or vendor specific methods to discover and establish communication with the named iSCSI Node. From owner-ips@ece.cmu.edu Wed May 16 14:08:57 2001 Received: from ece.cmu.edu (ECE.CMU.EDU [128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id OAA15783 for ; Wed, 16 May 2001 14:08:56 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f4GG9Wr18680 for ips-outgoing; Wed, 16 May 2001 12:09:32 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from dogwood.cisco.com (dogwood.cisco.com [161.44.11.19]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f4GG9UH18673 for ; Wed, 16 May 2001 12:09:30 -0400 (EDT) Received: from cisco.com (mbakke@mbakke-lnx.cisco.com [161.44.68.87]) by dogwood.cisco.com (8.8.6 (PHNE_14041)/CISCO.SERVER.1.2) with ESMTP id MAA24559; Wed, 16 May 2001 12:09:23 -0400 (EDT) Message-ID: <3B02A594.C10F9632@cisco.com> Date: Wed, 16 May 2001 11:06:44 -0500 From: Mark Bakke X-Mailer: Mozilla 4.72 [en] (X11; U; Linux 2.2.16-3.uid32 i686) X-Accept-Language: en, de MIME-Version: 1.0 To: John Hufferd CC: IPS Subject: Re: iSCSI: Aggregation tags in SendTargets References: Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-ips@ece.cmu.edu Precedence: bulk Content-Transfer-Encoding: 7bit Thanks. I intended what you said, but I missed the edit on that one. New text below. John Hufferd wrote: > > Mark, > You might want to modify your first rule, it does not take into count the > condition where a single address can have more then one connection to the > same session, but only to the same IP address. > To make this work, there are a few rules to follow: > > A target that does not support spanning sessions across multiple > addresses > MUST NOT include the tags. A target address that does not support multiple connections per session MUST NOT include an aggregation tag. A target address that supports multiple connections per session MUST include an aggregation tag. > A target that is accessible via multiple TCP addresses SHOULD include > all TCP addresses in a SendTargets response. > > A target with multiple TCP addresses that supports a session spanning > multiple TCP addresses MUST indicate TCP address groups using aggregation > tages in a SendTargets response. > > Aggregation tags have no meaning or persistence beyond a particular > SendTargets response. -- Mark A. Bakke Cisco Systems mbakke@cisco.com 763.398.1054 From owner-ips@ece.cmu.edu Wed May 16 14:11:26 2001 Received: from ece.cmu.edu (ECE.CMU.EDU [128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id OAA15821 for ; Wed, 16 May 2001 14:11:26 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f4GGCaO18968 for ips-outgoing; Wed, 16 May 2001 12:12:36 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from dogwood.cisco.com (dogwood.cisco.com [161.44.11.19]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f4GGCYH18963 for ; Wed, 16 May 2001 12:12:34 -0400 (EDT) Received: from cisco.com (mbakke@mbakke-lnx.cisco.com [161.44.68.87]) by dogwood.cisco.com (8.8.6 (PHNE_14041)/CISCO.SERVER.1.2) with ESMTP id MAA29951; Wed, 16 May 2001 12:12:19 -0400 (EDT) Message-ID: <3B02A639.F0B8491C@cisco.com> Date: Wed, 16 May 2001 11:09:29 -0500 From: Mark Bakke X-Mailer: Mozilla 4.72 [en] (X11; U; Linux 2.2.16-3.uid32 i686) X-Accept-Language: en, de MIME-Version: 1.0 To: "CONGDON,PAUL (HP-Roseville,ex1)" CC: "'John Hufferd'" , IPS Subject: Re: iSCSI: Aggregation tags in SendTargets References: <499DC368E25AD411B3F100902740AD6502D4F249@xrose03.rose.hp.com> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-ips@ece.cmu.edu Precedence: bulk Content-Transfer-Encoding: 7bit "CONGDON,PAUL (HP-Roseville,ex1)" wrote: > > Yes, I think the first rule should be re-worded to say > something like the following: > > A tag MUST NOT be used with addresses that do not support > sessions spanning multiple connections. Done. > > I also think a new rule should be added to clarify the case > John is talking about. Perhaps the following: > > A target that supports a session spanning multiple connections, > but only to the same address MUST use a unique tag. This is implied by the new rules I just sent, but I will add it since it's more explicit. > > Paul > > +--------------------------+----------------------------+ > + Paul Congdon + Email: paul_congdon@hp.com + > + Hewlett Packard Company + Mail Stop: 5662 + > + HP ProCurve Networking + Phone: (916) 785-5753 + > + 8000 Foothills Blvd + Fax: (916) 785-5949 + > + Roseville, CA 95747 + Mobile: (916) 765-4056 + > +--------------------------+----------------------------+ > -- Mark A. Bakke Cisco Systems mbakke@cisco.com 763.398.1054 From owner-ips@ece.cmu.edu Wed May 16 14:11:40 2001 Received: from ece.cmu.edu (ECE.CMU.EDU [128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id OAA15833 for ; Wed, 16 May 2001 14:11:40 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f4GGFsH19272 for ips-outgoing; Wed, 16 May 2001 12:15:54 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from dogwood.cisco.com (dogwood.cisco.com [161.44.11.19]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f4GGFrH19267 for ; Wed, 16 May 2001 12:15:53 -0400 (EDT) Received: from cisco.com (mbakke@mbakke-lnx.cisco.com [161.44.68.87]) by dogwood.cisco.com (8.8.6 (PHNE_14041)/CISCO.SERVER.1.2) with ESMTP id MAA06120; Wed, 16 May 2001 12:15:36 -0400 (EDT) Message-ID: <3B02A709.E6DD341@cisco.com> Date: Wed, 16 May 2001 11:12:57 -0500 From: Mark Bakke X-Mailer: Mozilla 4.72 [en] (X11; U; Linux 2.2.16-3.uid32 i686) X-Accept-Language: en, de MIME-Version: 1.0 To: Kevin Gibbons CC: IPS , Marjorie Krueger Subject: Re: iSCSI: Aggregation tags in SendTargets References: Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-ips@ece.cmu.edu Precedence: bulk Content-Transfer-Encoding: 7bit Kevin- I think that a numeric tag would be simpler as you had said. Marjorie had asked for the alpha-numeric tag; anyone needing more than a numeric tag, please speak up. I don't have any particular opinion either way. If nobody requires alpha-numeric tags, I will change it to a simple numeric tag. Otherwise, I'll update the examples. So, *** Anyone need Numeric Aggregation Tags *** Going once.... -- Mark Kevin Gibbons wrote: > > Mark, > in the proposed change to the NDT document, you state that the > aggregation "tag" is an ASCII, alpha-numeric string. I read this to mean > the tag can be any string value. > > Could you make the aggregation tag be a numeric string, indicating > an aggregation group? This would align with your example, and should make > it easier to quickly index portals into different groups. > > Regards, Kevin > > -----Original Message----- > From: Mark Bakke [mailto:mbakke@cisco.com] > Sent: Tuesday, May 15, 2001 1:06 PM > To: IPS > Subject: iSCSI: Aggregation tags in SendTargets > > During the interim meeting, we had discussed a proposal to > add an aggregation tag to the SendTargets response, indicating > which (if any) target addresses supported multiple connections > per session, and which groups of addresses an initiator could > hope to aggregate a session across. > > Aggregation tags were generally well-received; a small modification > to the proposed method also allows an initiator to know whether > a single address supports multiple connections per session just > by itself. > > Here is the section that would go into the NDT document. > > -- > > (This would be added to section 4.2, right before the vendor-specific > paragraph at the end): > > If an iSCSI target supports multiple connections per session, > it must indicate this by including an aggregation tag after each > address, in the form of > > TargetAddress=address,tag > > Where "tag" is an ASCII, alpha-numeric string indicating an address > group. Within a single session, a connection may be requested to any > combination of TCP addresses that have the same tag. If an address > supports multiple connections per session, but does not support > spanning a session across other addresses, it will have its own > tag. > > Here is an example: > > TargetName=fqn.com.acme.diskarray.sn.8675309 > TargetAddress=10.1.0.45:3000,1 > TargetAddress=10.1.1.46:3000,1 > TargetAddress=10.1.0.47:3000,2 > TargetAddress=10.1.1.48:3000,2 > TargetAddress=10.1.1.49:3000 > TargetAddress=10.1.1.50:3000,3 > TargetAlias=Oracle tables > > In this example, any of the target addresses can be used to reach > the same target. A single-connection session can be established > to any of these TCP addresses. A multiple-connection session could span > addresses .45 and .46, or .47 and .48, but cannot span any other > combination. A TargetAddress without a tag (.49) cannot be combined > with any other address within the same session. A TargetAddress > with a tag that is not shared with other addresses supports multiple > connections per session, but all connections must be to the same > address. > > To make this work, there are a few rules to follow: > > A target that does not support spanning sessions across multiple addresses > MUST NOT include the tags. > > A target that is accessible via multiple TCP addresses SHOULD include > all TCP addresses in a SendTargets response. > > A target with multiple TCP addresses that supports a session spanning > multiple TCP addresses MUST indicate TCP address groups using aggregation > tages in a SendTargets response. > > Aggregation tags have no meaning or persistence beyond a particular > SendTargets response. > > -- > Mark A. Bakke > Cisco Systems > mbakke@cisco.com > 763.398.1054 -- Mark A. Bakke Cisco Systems mbakke@cisco.com 763.398.1054 From owner-ips@ece.cmu.edu Wed May 16 14:13:00 2001 Received: from ece.cmu.edu (ECE.CMU.EDU [128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id OAA15923 for ; Wed, 16 May 2001 14:13:00 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f4GGwTa22802 for ips-outgoing; Wed, 16 May 2001 12:58:29 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from e31.bld.us.ibm.com (e31.co.us.ibm.com [32.97.110.129]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f4GGwRH22798 for ; Wed, 16 May 2001 12:58:28 -0400 (EDT) Received: from westrelay02.boulder.ibm.com (westrelay02.boulder.ibm.com [9.99.140.23]) by e31.bld.us.ibm.com (8.9.3/8.9.3) with ESMTP id MAA09932 for ; Wed, 16 May 2001 12:50:53 -0400 Received: from f3n42e (d03nm042h.boulder.ibm.com [9.99.140.42]) by westrelay02.boulder.ibm.com (8.11.1/NCO v4.96.1.0) with ESMTP id f4GGvYR178044 for ; Wed, 16 May 2001 10:57:34 -0600 Importance: Normal Subject: iSCSI: SCSI Access Controls "TransportID" To: ips@ece.cmu.edu From: "Jim Hafner" Date: Wed, 16 May 2001 09:57:33 -0700 Message-ID: X-MIMETrack: Serialize by Router on D03NM042/03/M/IBM(Release 5.0.6 |December 14, 2000) at 05/16/2001 09:57:34 AM MIME-Version: 1.0 Content-type: text/plain; charset=us-ascii Sender: owner-ips@ece.cmu.edu Precedence: bulk Folks, One feature of SCSI Access Controls (see ftp://ftp.t10.org/t10/document.99/99-245r9.pdf for the core of that specification) is the specification of an object called a TransportID. This is an identifier/name for an initiator device or port that can be used as the basis for logical unit access controls and LUN mapping. For example, FCP uses the WWPortname for this purpose. The N&DTeam has suggested that the TransportID for iSCSI be defined as the iSCSI Name (of the initiator node). This allows a management entity to configure a SCSI controller (e.g.) to present a different LUN map and logical unit list for each iSCSI Name that is presented during login. That is, each login from a named initiator device, regardless of SCSI port (ISID), will see the same set of logical units at the same LUN numbers. If there are no objections to this suggestion, I will make a formal proposal to T10 to adopt this at the next T10 meeting. [ASIDE: I mentioned in my previous note about "formats for long names" that it is up to the IPS WG to define those data structures in their documents (and that I was suggesting T10 review those formats). In contrast, the TransportID formats have to be defined in SPC-3 (and so approved in T10). So, I'm asking for review of the TransportID here. In other words, T10 "owns" the TransportID spec, and IETF/IPS WG "owns" the long name formats.] Jim Hafner From owner-ips@ece.cmu.edu Wed May 16 16:57:13 2001 Received: from ece.cmu.edu (ECE.CMU.EDU [128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id QAA19031 for ; Wed, 16 May 2001 16:57:13 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f4GJ8U404396 for ips-outgoing; Wed, 16 May 2001 15:08:30 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from dogwood.cisco.com (dogwood.cisco.com [161.44.11.19]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f4GJ8TH04387 for ; Wed, 16 May 2001 15:08:29 -0400 (EDT) Received: from cisco.com (mbakke@mbakke-lnx.cisco.com [161.44.68.87]) by dogwood.cisco.com (8.8.6 (PHNE_14041)/CISCO.SERVER.1.2) with ESMTP id PAA04929; Wed, 16 May 2001 15:08:17 -0400 (EDT) Message-ID: <3B02CF82.ACDFD055@cisco.com> Date: Wed, 16 May 2001 14:05:38 -0500 From: Mark Bakke X-Mailer: Mozilla 4.72 [en] (X11; U; Linux 2.2.16-3.uid32 i686) X-Accept-Language: en, de MIME-Version: 1.0 To: Black_David@emc.com CC: ips@ece.cmu.edu Subject: Re: iSCSI: Canonical Targets References: <0F31E5C394DAD311B60C00E029101A0708015578@corpmx9.isus.emc.com> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-ips@ece.cmu.edu Precedence: bulk Content-Transfer-Encoding: 7bit How about this instead: 2. A discovery target MUST be accessible on the default tcp port on each IP address on which an iSCSI implementation is listening for iSCSI connections. This default tcp port will often be the IANA-assigned TCP port for iSCSI. If the implementation is listening on multiple TCP ports, the discovery target MAY be accessible on each TCP port. Black_David@emc.com wrote: > > Mark, > > > 2. A canonical target MUST be accessible at the default, IANA- > > assigned TCP port on each IP address on which the iSCSI > > implementation is listening for iSCSI connections. > > Think about weakening this to talk about the default port of contact > rather than requiring the one and only IANA-assigned default > port to be used. This allows more than one iSCSI implementation > behind a single IP address provided that the right ports are in the > discovery information. > > This can be amazingly useful for debugging, testing, etc., and even > dealing with stupid firewall tricks (e.g., some http servers ran/run > at port 8080 for this reason - as long as the browser knows to go > to 8080 [in URL], it makes no difference). > > --David > --------------------------------------------------- > David L. Black, Senior Technologist > EMC Corporation, 42 South St., Hopkinton, MA 01748 > +1 (508) 435-1000 x75140 FAX: +1 (508) 497-8500 > black_david@emc.com Mobile: +1 (978) 394-7754 > --------------------------------------------------- -- Mark A. Bakke Cisco Systems mbakke@cisco.com 763.398.1054 From owner-ips@ece.cmu.edu Wed May 16 16:57:36 2001 Received: from ece.cmu.edu (ECE.CMU.EDU [128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id QAA19057 for ; Wed, 16 May 2001 16:57:35 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f4GJHS305262 for ips-outgoing; Wed, 16 May 2001 15:17:28 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from dogwood.cisco.com (dogwood.cisco.com [161.44.11.19]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f4GJHQH05258 for ; Wed, 16 May 2001 15:17:26 -0400 (EDT) Received: from cisco.com (mbakke@mbakke-lnx.cisco.com [161.44.68.87]) by dogwood.cisco.com (8.8.6 (PHNE_14041)/CISCO.SERVER.1.2) with ESMTP id PAA20970; Wed, 16 May 2001 15:17:15 -0400 (EDT) Message-ID: <3B02D19B.7702A1CC@cisco.com> Date: Wed, 16 May 2001 14:14:35 -0500 From: Mark Bakke X-Mailer: Mozilla 4.72 [en] (X11; U; Linux 2.2.16-3.uid32 i686) X-Accept-Language: en, de MIME-Version: 1.0 To: Stephen Bailey CC: ips@ece.cmu.edu Subject: Re: iSCSI: Canonical Targets References: <3AFC53D8.CD8A8BC4@cisco.com> <200105140255.f4E2tjx22995@chmls05.mediaone.net> <3B004061.E6019045@cisco.com> <20010514221317.7BD3F94009@sandmail.sandburst.com> <3B012D18.688CDF1E@cisco.com> <3B015D31.C84CAFE8@cisco.com> <20010515165134.44FB394009@sandmail.sandburst.com> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-ips@ece.cmu.edu Precedence: bulk Content-Transfer-Encoding: 7bit That kind of makes sense. If we limit SendTargets to the discovery session, and disallow SCSI commands to the discovery session, then we really don't have a target at all from SCSI's point of view; but we do have a "special" target from iSCSI's point of view since we are doing the full login to it. So if we used TargetName= we would get the discovery target; if we used TargetName=iSCSI # Note: this is case-insensitive we would get whatever default target the iSCSI device wanted our initiator to see. Even if we do end up supporting both, I guess I'd rather see a target name, rather than leaving it blank. How about TargetName=discovery BTW, nobody else has spoken up for having this default target that would avoid having to use SendTargets if there were not multiple targets available to an initiator. Who plans to make use of this? I don't mind putting it in (we had sort of implied the functionality before), but if it is not in anyone's plans, I'd rather go for a simpler spec. Stephen Bailey wrote: > > > Any other ideas? > > My suggestion was to not supply any target name for a discovery login > (which is how a discovery login would be distinguished) and call > `iSCSI' the `default' (operational) target. > > Steph -- Mark A. Bakke Cisco Systems mbakke@cisco.com 763.398.1054 From owner-ips@ece.cmu.edu Wed May 16 16:57:40 2001 Received: from ece.cmu.edu (ECE.CMU.EDU [128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id QAA19068 for ; Wed, 16 May 2001 16:57:39 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f4GIcGF01771 for ips-outgoing; Wed, 16 May 2001 14:38:16 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from chmls06.mediaone.net (chmls06.mediaone.net [24.147.1.144]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f4GHrmH27679 for ; Wed, 16 May 2001 13:53:48 -0400 (EDT) Received: from eddylaptop (h00e098880aa4.ne.mediaone.net [66.31.75.244]) by chmls06.mediaone.net (8.11.1/8.11.1) with SMTP id f4GHrk821153 for ; Wed, 16 May 2001 13:53:46 -0400 (EDT) From: "Eddy Quicksall" To: Subject: iSCSI: CmdSN with Text Command Date: Wed, 16 May 2001 13:53:49 -0400 Message-ID: <000101c0de31$29e2b120$0100a8c0@eddylaptop> MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 (Normal) X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook CWS, Build 9.0.2416 (9.0.2911.0) Importance: Normal X-MimeOLE: Produced By Microsoft MimeOLE V5.00.3018.1300 Sender: owner-ips@ece.cmu.edu Precedence: bulk Content-Transfer-Encoding: 7bit I am confused about the use of CmdSN with the Text command. The reason is as follows: Section 1.2.2.1 says iSCSI supports ordered command delivery within a session. All commands (initiator-to-target) are numbered. Commands in transit from the initiator SCSI layer to the SCSI target layer are numbered by iSCSI; the number is carried by the iSCSI PDU as CmdSN (Command-Sequence-Number). - CmdSN - the current command Sequence Number advanced by 1 on each command shipped except for commands marked for immediate delivery. ... The target iSCSI layer sets the ExpCmdSN to the largest non-immediate CmdSN that it is able to deliver to the target SCSI layer plus 1 (no holes in the CmdSN sequence). It looks like the 1st and 3rd paragraphs are saying that the CmdSN of a Text Command would be advanced. But, the 2nd and 4th paragraphs hint that only the SCSI layers advance CmdSN. Can someone clear this up? i.e., is CmdSN advanced for Text Commands? mailto:Eddy@Quicksall.com From owner-ips@ece.cmu.edu Wed May 16 17:52:06 2001 Received: from ece.cmu.edu (ECE.CMU.EDU [128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id RAA19816 for ; Wed, 16 May 2001 17:52:05 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f4GJxjL09190 for ips-outgoing; Wed, 16 May 2001 15:59:45 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from palrel2.hp.com (palrel2.hp.com [156.153.255.234]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f4GJxiH09186 for ; Wed, 16 May 2001 15:59:44 -0400 (EDT) Received: from amrelay2.boi.hp.com (amrelay2.boi.hp.com [15.56.8.41]) by palrel2.hp.com (Postfix) with ESMTP id 2BE6338B5; Wed, 16 May 2001 12:59:43 -0700 (PDT) Received: from xatlbh2.atl.hp.com (xatlbh2.atl.hp.com [15.45.89.187]) by amrelay2.boi.hp.com (8.9.3 (PHNE_18979)/8.9.3 SMKit7.02) with ESMTP id NAA00848; Wed, 16 May 2001 13:59:42 -0600 (MDT) Received: by xatlbh2.atl.hp.com with Internet Mail Service (5.5.2653.19) id ; Wed, 16 May 2001 15:59:41 -0400 Message-ID: <6BD67FFB937FD411A04F00D0B74FE87802A09064@xrose06.rose.hp.com> From: "KRUEGER,MARJORIE (HP-Roseville,ex1)" To: "'Mark Bakke'" , John Hufferd Cc: IPS Subject: RE: iSCSI: Aggregation tags in SendTargets Date: Wed, 16 May 2001 15:59:36 -0400 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2653.19) Content-Type: text/plain; charset="iso-8859-1" Sender: owner-ips@ece.cmu.edu Precedence: bulk > John Hufferd wrote: > > > > Mark, > > You might want to modify your first rule, it does not take > into count the > > condition where a single address can have more then one > connection to the > > same session, but only to the same IP address. > > > To make this work, there are a few rules to follow: > > > > A target that does not support spanning sessions across multiple > > addresses > > MUST NOT include the tags. > > A target address that does not support multiple connections per > session MUST NOT include an aggregation tag. > > A target address that supports multiple connections per session > MUST include an aggregation tag. If a target only has one IP address, and supports multiple connections per session thru that IP address, why would it have to include an aggregation tag? I thought an aggregation tag is ONLY to indicate that a single session's connections may span IP addresses. Are you trying to overload it to indicate multi-connection session support? Marj From owner-ips@ece.cmu.edu Wed May 16 17:52:49 2001 Received: from ece.cmu.edu (ECE.CMU.EDU [128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id RAA19837 for ; Wed, 16 May 2001 17:52:48 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f4GJunQ08890 for ips-outgoing; Wed, 16 May 2001 15:56:49 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from e34.bld.us.ibm.com (e34.co.us.ibm.com [32.97.110.132]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f4GJumH08885 for ; Wed, 16 May 2001 15:56:48 -0400 (EDT) Received: from westrelay02.boulder.ibm.com (westrelay02.boulder.ibm.com [9.99.140.23]) by e34.bld.us.ibm.com (8.9.3/8.9.3) with ESMTP id PAA87880; Wed, 16 May 2001 15:39:13 -0400 Received: from f3n42e (d03nm042h.boulder.ibm.com [9.99.140.42]) by westrelay02.boulder.ibm.com (8.11.1/NCO v4.96.1.0) with ESMTP id f4GJukR243768; Wed, 16 May 2001 13:56:46 -0600 Importance: Normal Subject: Re: iSCSI: Canonical Targets To: Mark Bakke Cc: ips@ece.cmu.edu From: "Jim Hafner" Date: Wed, 16 May 2001 12:56:45 -0700 Message-ID: X-MIMETrack: Serialize by Router on D03NM042/03/M/IBM(Release 5.0.6 |December 14, 2000) at 05/16/2001 12:56:46 PM MIME-Version: 1.0 Content-type: text/plain; charset=us-ascii Sender: owner-ips@ece.cmu.edu Precedence: bulk Mark and Steph, I guess I'm having trouble understanding the issues here. We seem to be moving toward three types of iSCSI targets: 1) one for discovery only, suitable for login only by authenticated initiators, i.e., a one-sided authentication, but ONLY for the purposes of SendTargets. 2) one "nameless" target ("iSCSI"), again suitable for login only by authenticated initiators, i.e., a one-sided authentication, but used for real SCSI stuff 3) all other "named" targets, that may or may not require two-sided authentication, but are used for real SCSI stuff. Frankly, I think I see no real need for the second one (a SCSI-functional thing with no true name). The third one can perform the function of the second one if the initiator doesn't bother to authenticate the target's name (provided it has a straightforward way to get the name). The first one was what we were moving to as the functional use of the "iSCSI" target. The only value I see is that the initiator doesn't have to find the name first. But the target will need to "correct" that name in response anyway (at which point it has a name which it can use for all other sessions) -- it could/should be a one time deal, probably, to go through the discovery (SendTargets) step. I don't think we really want the host to think that it has any number of real targets all named "iSCSI"! Jim Hafner Mark Bakke @ece.cmu.edu on 05-16-2001 12:14:35 PM Sent by: owner-ips@ece.cmu.edu To: Stephen Bailey cc: ips@ece.cmu.edu Subject: Re: iSCSI: Canonical Targets That kind of makes sense. If we limit SendTargets to the discovery session, and disallow SCSI commands to the discovery session, then we really don't have a target at all from SCSI's point of view; but we do have a "special" target from iSCSI's point of view since we are doing the full login to it. So if we used TargetName= we would get the discovery target; if we used TargetName=iSCSI # Note: this is case-insensitive we would get whatever default target the iSCSI device wanted our initiator to see. Even if we do end up supporting both, I guess I'd rather see a target name, rather than leaving it blank. How about TargetName=discovery BTW, nobody else has spoken up for having this default target that would avoid having to use SendTargets if there were not multiple targets available to an initiator. Who plans to make use of this? I don't mind putting it in (we had sort of implied the functionality before), but if it is not in anyone's plans, I'd rather go for a simpler spec. Stephen Bailey wrote: > > > Any other ideas? > > My suggestion was to not supply any target name for a discovery login > (which is how a discovery login would be distinguished) and call > `iSCSI' the `default' (operational) target. > > Steph -- Mark A. Bakke Cisco Systems mbakke@cisco.com 763.398.1054 From owner-ips@ece.cmu.edu Wed May 16 18:04:19 2001 Received: from ece.cmu.edu (ECE.CMU.EDU [128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id SAA19968 for ; Wed, 16 May 2001 18:04:19 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f4GJvij08989 for ips-outgoing; Wed, 16 May 2001 15:57:44 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from atlrel2.hp.com (atlrel2.hp.com [156.153.255.202]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f4GJvhH08984 for ; Wed, 16 May 2001 15:57:43 -0400 (EDT) Received: from amrelay2.boi.hp.com (amrelay2.boi.hp.com [15.56.8.41]) by atlrel2.hp.com (Postfix) with ESMTP id 6A64CC8C; Wed, 16 May 2001 15:57:42 -0400 (EDT) Received: from xatlbh4.atl.hp.com (xatlbh4.atl.hp.com [15.45.89.189]) by amrelay2.boi.hp.com (8.9.3 (PHNE_18979)/8.9.3 SMKit7.02) with ESMTP id NAA00466; Wed, 16 May 2001 13:57:40 -0600 (MDT) Received: by xatlbh4.atl.hp.com with Internet Mail Service (5.5.2653.19) id ; Wed, 16 May 2001 13:57:40 -0600 Message-ID: <6BD67FFB937FD411A04F00D0B74FE87802A09063@xrose06.rose.hp.com> From: "KRUEGER,MARJORIE (HP-Roseville,ex1)" To: "'Mark Bakke'" , Kevin Gibbons Cc: IPS Subject: RE: iSCSI: Aggregation tags in SendTargets Date: Wed, 16 May 2001 13:57:36 -0600 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2653.19) Content-Type: text/plain; charset="iso-8859-1" Sender: owner-ips@ece.cmu.edu Precedence: bulk I think you meant to ask *** Anyone need Alpha-Numeric Aggregation Tags *** ^^^^^ Going once.... I have no strong opinion either. Alpha's a little more user-friendly, but that is not important enough in this feature to warrant any argument, so if implementors want numeric tags, let them be numeric... Marj > -----Original Message----- > From: Mark Bakke [mailto:mbakke@cisco.com] > Sent: Wednesday, May 16, 2001 9:13 AM > To: Kevin Gibbons > Cc: IPS; Marjorie Krueger > Subject: Re: iSCSI: Aggregation tags in SendTargets > > > Kevin- > > I think that a numeric tag would be simpler as you had said. > Marjorie had asked for the alpha-numeric tag; anyone needing > more than a numeric tag, please speak up. I don't have any > particular opinion either way. > > If nobody requires alpha-numeric tags, I will change it to > a simple numeric tag. Otherwise, I'll update the examples. > > So, > > *** Anyone need Numeric Aggregation Tags *** > > Going once.... > > -- > Mark > > Kevin Gibbons wrote: > > > > Mark, > > in the proposed change to the NDT document, you > state that the > > aggregation "tag" is an ASCII, alpha-numeric string. I > read this to mean > > the tag can be any string value. > > > > Could you make the aggregation tag be a numeric > string, indicating > > an aggregation group? This would align with your example, > and should make > > it easier to quickly index portals into different groups. > > > > Regards, Kevin > > > > -----Original Message----- > > From: Mark Bakke [mailto:mbakke@cisco.com] > > Sent: Tuesday, May 15, 2001 1:06 PM > > To: IPS > > Subject: iSCSI: Aggregation tags in SendTargets > > > > During the interim meeting, we had discussed a proposal to > > add an aggregation tag to the SendTargets response, indicating > > which (if any) target addresses supported multiple connections > > per session, and which groups of addresses an initiator could > > hope to aggregate a session across. > > > > Aggregation tags were generally well-received; a small modification > > to the proposed method also allows an initiator to know whether > > a single address supports multiple connections per session just > > by itself. > > > > Here is the section that would go into the NDT document. > > > > -- > > > > (This would be added to section 4.2, right before the > vendor-specific > > paragraph at the end): > > > > If an iSCSI target supports multiple connections per session, > > it must indicate this by including an aggregation tag after each > > address, in the form of > > > > TargetAddress=address,tag > > > > Where "tag" is an ASCII, alpha-numeric string indicating > an address > > group. Within a single session, a connection may be > requested to any > > combination of TCP addresses that have the same tag. If > an address > > supports multiple connections per session, but does not support > > spanning a session across other addresses, it will have its own > > tag. > > > > Here is an example: > > > > TargetName=fqn.com.acme.diskarray.sn.8675309 > > TargetAddress=10.1.0.45:3000,1 > > TargetAddress=10.1.1.46:3000,1 > > TargetAddress=10.1.0.47:3000,2 > > TargetAddress=10.1.1.48:3000,2 > > TargetAddress=10.1.1.49:3000 > > TargetAddress=10.1.1.50:3000,3 > > TargetAlias=Oracle tables > > > > In this example, any of the target addresses can be used to reach > > the same target. A single-connection session can be established > > to any of these TCP addresses. A multiple-connection > session could span > > addresses .45 and .46, or .47 and .48, but cannot span any other > > combination. A TargetAddress without a tag (.49) cannot > be combined > > with any other address within the same session. A TargetAddress > > with a tag that is not shared with other addresses > supports multiple > > connections per session, but all connections must be to the same > > address. > > > > To make this work, there are a few rules to follow: > > > > A target that does not support spanning sessions across > multiple addresses > > MUST NOT include the tags. > > > > A target that is accessible via multiple TCP addresses > SHOULD include > > all TCP addresses in a SendTargets response. > > > > A target with multiple TCP addresses that supports a > session spanning > > multiple TCP addresses MUST indicate TCP address groups > using aggregation > > tages in a SendTargets response. > > > > Aggregation tags have no meaning or persistence beyond a > particular > > SendTargets response. > > > > -- > > Mark A. Bakke > > Cisco Systems > > mbakke@cisco.com > > 763.398.1054 From owner-ips@ece.cmu.edu Wed May 16 18:44:59 2001 Received: from ece.cmu.edu (ECE.CMU.EDU [128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id SAA20512 for ; Wed, 16 May 2001 18:44:59 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f4GLLNg15921 for ips-outgoing; Wed, 16 May 2001 17:21:23 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from spdmraab.compuserve.com (ds-img-rel-2.compuserve.com [149.174.206.155]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f4GLLMH15917 for ; Wed, 16 May 2001 17:21:22 -0400 (EDT) Received: (from mailgate@localhost) by spdmraab.compuserve.com (8.9.3/8.9.3/SUN-REL-1.3) id RAA00195 for ips@ece.cmu.edu; Wed, 16 May 2001 17:21:16 -0400 (EDT) Received: from compuserve.com (dal-tgn-tkk-vty82.as.wcom.net [206.175.239.82]) by spdmraab.compuserve.com (8.9.3/8.9.3/SUN-REL-1.3) with ESMTP id RAA00116 for ; Wed, 16 May 2001 17:21:12 -0400 (EDT) Message-ID: <3B02F012.241E76D1@compuserve.com> Date: Wed, 16 May 2001 16:24:34 -0500 From: Ralph Weber Reply-To: ENDL_TX@computer.org X-Mailer: Mozilla 4.7 [en]C-CCK-MCD NSCPCD47 (Win95; I) X-Accept-Language: en,pdf MIME-Version: 1.0 To: IPS Reflector Subject: FCIP - Discovery should be Manual/SLP Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-ips@ece.cmu.edu Precedence: bulk Content-Transfer-Encoding: 7bit I support Dave's proposed text for FCIP Clause 4.2 item 3, with the exception that the sentence regarding iSNS needs to be removed. Looking at the features list, I cannot see any iSNS unique features that are meaningful to FCIP. Thanks. Ralph... - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - Subject: Proposal to use SLPv2 for FCIP device discovery Date: Mon, 30 Apr 2001 17:07:08 -0500 From: "Dave Peterson" To: "Ips@Ece. Cmu. Edu" As requested here is the proposal I presented at Mondays' FCIP meeting: FCIP Draft Proposal For Clause 4.2, item 3. Each FCIP device may be statically or dynamically configured with a list of IP addresses and port numbers corresponding to all the participating FCIP devices. If dynamic discovery of participating FCIP devices is supported this function shall be performed using the Service Location Protocol (SLPv2). For additional FCIP device management capability, the iSNS Internet Storage Naming Service may be implemented. It is outside the scope of this specification to describe any static configuration method for participating FCIP device discovery. Refer to clause XXX for a detailed description of dynamic discovery of participating FCIP device using SLPv2. Notes: 1. DHCP: a. Allows an entity to discovery information about itself, not discover information about all other entities. b. Uses a broadcast mechanism that may not work via routers without additional configuration. But, most current router implementations will support the forwarding of DHCP requests across routed subnets. c. May be used to find SLP Directory Agents and Scope Lists allowing for a more scalable solution. 2. DNS Services are too limiting. This is the reason why SLP was started. 3. LDAP is simply a database interface. SLP and iSNS may use LDAP as a back-end store. 4. SLP FCIP Device Type Specifics a. IP address(es) b. Port numbers(s) c. Scope d. Attributes i. Discovery domain (i.e. a group name or zone name, don't want to use zone name in this context though) ii. need to start listing these (if we can think of any more) e. Lifetime Work In progress: 1. Putting together a model for FCIP device discovery using SLPv2. 2. Implementing a "default/suggested" SLPv2 template. 3. Reviewing RFC 3082 "Notification and Subscription for SLP for enhanced device notification. Requirements SLPv2 iSNS Discovery of FCIP targets Y Y Discovery of FCIP device IP address(es) Y Y Discovery of FCIP port number(s) Y Y Attribute support Y Y Semi-timely notification of devices coming and going Y Y Authentication Y Y* Minimal/no configuration Y Y(?) Provisioning Y Y Multicast support Y N Publicly available source Y N Standardized and mature Y N Lighweight implementation Y N Non-Requirements Zoning N Y State Change Notification N Y Naming Services N Y * uses SLPv2 constructs David Peterson Lead Architect - Standards Development Cisco Systems - SRBU 6450 Wedgwood Road Maple Grove, MN 55311 Office: 763-398-1007 Cell: 612-802-3299 Email: dap@cisco.com From owner-ips@ece.cmu.edu Wed May 16 18:45:02 2001 Received: from ece.cmu.edu (ECE.CMU.EDU [128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id SAA20548 for ; Wed, 16 May 2001 18:45:01 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f4GL4PQ14462 for ips-outgoing; Wed, 16 May 2001 17:04:25 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from dogwood.cisco.com (dogwood.cisco.com [161.44.11.19]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f4GL4NH14457 for ; Wed, 16 May 2001 17:04:23 -0400 (EDT) Received: from cisco.com (mbakke@mbakke-lnx.cisco.com [161.44.68.87]) by dogwood.cisco.com (8.8.6 (PHNE_14041)/CISCO.SERVER.1.2) with ESMTP id RAA16489; Wed, 16 May 2001 17:04:16 -0400 (EDT) Message-ID: <3B02EAB0.EB4936D2@cisco.com> Date: Wed, 16 May 2001 16:01:36 -0500 From: Mark Bakke X-Mailer: Mozilla 4.72 [en] (X11; U; Linux 2.2.16-3.uid32 i686) X-Accept-Language: en, de MIME-Version: 1.0 To: Jim Hafner CC: ips@ece.cmu.edu Subject: Re: iSCSI: Canonical Targets References: Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-ips@ece.cmu.edu Precedence: bulk Content-Transfer-Encoding: 7bit Actually, that's a good point about the multiple target names. If a host is allowed to use a target without learning its name, it has two problems: 1. If it has multiple targets with the name "iscsi", it will not know whether they are multiple targets, or multiple addresses for the same target. 2. If the target's IP address changes, it will not be able to tell whether a newly-discovered target is the same target at a new address, or a new target. Given these, I think there is too much danger in allowing a host not to learn the actual name of a target. Also, Jim is correct about discovery being a one-time thing; after the first time a host has connected, it will probably squirrel the iSCSI name away in a config file or registry somewhere anyway, so it knows where to look for its disks, and which /dev entries to map them to the next time it boots. Jim Hafner wrote: > > Mark and Steph, > > I guess I'm having trouble understanding the issues here. We seem to be > moving toward three types of iSCSI targets: > 1) one for discovery only, suitable for login only by authenticated > initiators, i.e., a one-sided authentication, but ONLY for the purposes of > SendTargets. > 2) one "nameless" target ("iSCSI"), again suitable for login only by > authenticated initiators, i.e., a one-sided authentication, but used for > real SCSI stuff > 3) all other "named" targets, that may or may not require two-sided > authentication, but are used for real SCSI stuff. > > Frankly, I think I see no real need for the second one (a SCSI-functional > thing with no true name). The third one can perform the function of the > second one if the initiator doesn't bother to authenticate the target's > name (provided it has a straightforward way to get the name). The first > one was what we were moving to as the functional use of the "iSCSI" target. > The only value I see is that the initiator doesn't have to find the name > first. But the target will need to "correct" that name in response anyway > (at which point it has a name which it can use for all other sessions) -- > it could/should be a one time deal, probably, to go through the discovery > (SendTargets) step. I don't think we really want the host to think that it > has any number of real targets all named "iSCSI"! > > Jim Hafner > > Mark Bakke @ece.cmu.edu on 05-16-2001 12:14:35 PM > > Sent by: owner-ips@ece.cmu.edu > > To: Stephen Bailey > cc: ips@ece.cmu.edu > Subject: Re: iSCSI: Canonical Targets > > That kind of makes sense. If we limit SendTargets to the discovery > session, and disallow SCSI commands to the discovery session, then > we really don't have a target at all from SCSI's point of view; but > we do have a "special" target from iSCSI's point of view since we > are doing the full login to it. > > So if we used > > TargetName= > > we would get the discovery target; if we used > > TargetName=iSCSI # Note: this is case-insensitive > > we would get whatever default target the iSCSI device wanted > our initiator to see. Even if we do end up supporting both, I > guess I'd rather see a target name, rather than leaving it > blank. How about > > TargetName=discovery > > BTW, nobody else has spoken up for having this default target > that would avoid having to use SendTargets if there were not > multiple targets available to an initiator. > > Who plans to make use of this? I don't mind putting it in > (we had sort of implied the functionality before), but if > it is not in anyone's plans, I'd rather go for a simpler spec. > > Stephen Bailey wrote: > > > > > Any other ideas? > > > > My suggestion was to not supply any target name for a discovery login > > (which is how a discovery login would be distinguished) and call > > `iSCSI' the `default' (operational) target. > > > > Steph > > -- > Mark A. Bakke > Cisco Systems > mbakke@cisco.com > 763.398.1054 -- Mark A. Bakke Cisco Systems mbakke@cisco.com 763.398.1054 From owner-ips@ece.cmu.edu Wed May 16 18:45:47 2001 Received: from ece.cmu.edu (ECE.CMU.EDU [128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id SAA20609 for ; Wed, 16 May 2001 18:45:46 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f4GL02O14141 for ips-outgoing; Wed, 16 May 2001 17:00:02 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from dogwood.cisco.com (dogwood.cisco.com [161.44.11.19]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f4GL00H14133 for ; Wed, 16 May 2001 17:00:00 -0400 (EDT) Received: from cisco.com (mbakke@mbakke-lnx.cisco.com [161.44.68.87]) by dogwood.cisco.com (8.8.6 (PHNE_14041)/CISCO.SERVER.1.2) with ESMTP id QAA12445; Wed, 16 May 2001 16:59:50 -0400 (EDT) Message-ID: <3B02E9A7.EE8780E0@cisco.com> Date: Wed, 16 May 2001 15:57:11 -0500 From: Mark Bakke X-Mailer: Mozilla 4.72 [en] (X11; U; Linux 2.2.16-3.uid32 i686) X-Accept-Language: en, de MIME-Version: 1.0 To: "KRUEGER,MARJORIE (HP-Roseville,ex1)" CC: John Hufferd , IPS Subject: Re: iSCSI: Aggregation tags in SendTargets References: <6BD67FFB937FD411A04F00D0B74FE87802A09064@xrose06.rose.hp.com> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-ips@ece.cmu.edu Precedence: bulk Content-Transfer-Encoding: 7bit Marjorie- The request was made at the interim meeting to use a tag to specify that multiple connections per session was supported. That way, an initiator discovering a target via SendTargets would know whether or not to bother attempting multiple connections on the session. So you are right; it has been overloaded to do that. Of course, if there were some sort of flag or text key in the iSCSI login response to indicate the support of multiple- connection sessions, we wouldn't need to do this. I don't think it's in there, though. -- Mark "KRUEGER,MARJORIE (HP-Roseville,ex1)" wrote: > > > John Hufferd wrote: > > > > > > Mark, > > > You might want to modify your first rule, it does not take > > into count the > > > condition where a single address can have more then one > > connection to the > > > same session, but only to the same IP address. > > > > > To make this work, there are a few rules to follow: > > > > > > A target that does not support spanning sessions across multiple > > > addresses > > > MUST NOT include the tags. > > > > A target address that does not support multiple connections per > > session MUST NOT include an aggregation tag. > > > > A target address that supports multiple connections per session > > MUST include an aggregation tag. > > If a target only has one IP address, and supports multiple connections per > session thru that IP address, why would it have to include an aggregation > tag? I thought an aggregation tag is ONLY to indicate that a single > session's connections may span IP addresses. Are you trying to overload it > to indicate multi-connection session support? > > Marj -- Mark A. Bakke Cisco Systems mbakke@cisco.com 763.398.1054 From owner-ips@ece.cmu.edu Wed May 16 18:46:44 2001 Received: from ece.cmu.edu (ECE.CMU.EDU [128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id SAA20626 for ; Wed, 16 May 2001 18:46:39 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f4GLMhL16084 for ips-outgoing; Wed, 16 May 2001 17:22:43 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from server1.NishanSystems.COM (smtp.nishansystems.com [216.217.36.162]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f4GLMfH16079 for ; Wed, 16 May 2001 17:22:41 -0400 (EDT) Received: by smtp.nishansystems.com with Internet Mail Service (5.5.2653.19) id ; Wed, 16 May 2001 14:22:34 -0700 Message-ID: From: Charles Monia To: "Ips (E-mail)" Subject: RE: iFCP Clarification Date: Wed, 16 May 2001 14:22:33 -0700 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2653.19) Content-Type: text/plain; charset="iso-8859-1" Sender: owner-ips@ece.cmu.edu Precedence: bulk Hi Murali: The following summary from the meeting minutes is accurate: "iFCP address-transparent mode. This assembles iFCP gateways into a single logical fabric, within which addresses are global and untranslated. Maximum of 239 gateways in a fabric (this is the FC domain limit for number of switches, each gateway counts as a switch). Address translation mode will probably be mandatory. Translation and Transparent mode do not interoperate. Transparent mode does not require any FC header or CRC changes." The main difference is in the address mapping step. Both iFCP modes implement the same the session model and all routing and distributed Fibre Channel fabric services are performed by iSNS and other IP-based equivalents(e.g., OSPF for routing). In both flavors of iFCP, no Class F control traffic traverses the IP network. Since you missed the meeting, I've enclosed a copy of the IETF presentation under seperate cover FYI. It will be generally available when the minutes are published. The following presentation, given to the T11 FC-BB working group on February 6, might also be helpful -- ftp://ftp.t11.org/t11/pub/fc/bb-2/01-057v0.ppt. The material in the current spec was first published in draft-monia-ips-iFCP-01.txt. Charles > -----Original Message----- > From: Murali Rajagopal [mailto:muralir@lightsand.com] > Sent: Tuesday, May 15, 2001 3:22 PM > To: Charles Monia; ips@ece.cmu.edu > Subject: iFCP Clarification > > > Charles: > > Can you clarify what iFCP Transparent Mode is? > I was not at the Nashua meeting, but based on David's minutes > it looks a lot > like FCIP. > > Regards, > > Murali > > -----Original Message----- > From: owner-ips@ece.cmu.edu [mailto:owner-ips@ece.cmu.edu]On Behalf Of > Black_David@emc.com > Sent: Tuesday, May 15, 2001 10:32 AM > To: ips@ece.cmu.edu > Subject: DRAFT Nashua interim meeting minutes > > > This is one loquacious bunch - sorry it took a while > to get these out. WG consensus (will be considered > affirmed if not objected to on the list) is flagged > by **, action items are flagged by ->. > > Comments, corrections, etc. to > the list by Friday, May 18th. > > Thanks, > --David > > > > IETF IP Storage (ips) Working Group > Nashua Interim Meeting - April 30 and May 1, 2001 > DRAFT Minutes > > ----------------------------------------------------------- > April 30, 2001 - FC encapsulation protocols (FCIP and iFCP) > ----------------------------------------------------------- > > FC Common Encapsulation - draft-ietf-ips-fcencapsulation-00.txt > ----------------------- > > CRC discussion - some protocols want to use a CRC, some > don't, hence there's a CRC Valid flag in the header. > Using one of the bits from the protocol number field to > encode CRC Valid was discussed and rejected as too clever by half > > ** Rough Consensus: Retain CRC Valid as a separate flag as currently > defined. > -> Action: David Black to send email to list describing allocation > approach that would (initially) avoid two protocol numbers that > differ only in the encoded CRC Valid bit without permanently > using two bits from the protocol number field to do this. > > ** Rough Consensus - Use different default TCP ports for FCIP > and iFCP. > -> Action: David Black to pursue causing these ports plus one > for iSCSI to be allocated. > -> Action: David Black will work with authors on revising the > IANA text. > > ** Rough Consensus on the Reserved flag bits: > (1) MUST be zero on send. > (2) Protocol MAY specify MUST be zero on receive. > (3) How to allow future protocols to make use of these? > - Actual use requires modification of common encap. > - Protocol-generic use of this encap SHOULD not > assume that reserved flags are zero. > > Later discussion on use of inverted check fields ( followed by > 1's complement of ) reached the following > ** Rough Consensus: All uses of inverted check fields are to have > the structure of a 32 bit word consisting of 16 bits of > followed by 16 bits of the 1's complement of , where > may consist of multiple fields. This applies to the common > encapsulation header, and both the SOF and EOF encodings. > > Fibre Channel MIBs - General Discussion > --------------------------------------- > > ** Rough consensus - iFCP and FCIP will do separate protocol-specific > MIBs. Use existing FC MIBs (ipfc WG) for common stuff, similar > to iSCSI's approach/direction of abstracting SCSI stuff into > separate MIB. > > > FCIP MIB - draft-anil-fcip-mib-00.txt > -------- > > This is being developed as an extension of the FC Framework MIB > (draft-ietf-ipfc-fsmgmt-int-mib-06.txt). > > Several comments were made on things needing attention in the MIB, > including clarifying the distinction between port and endpoint, > and the likely need for RowStatus support for row creation. > > TCP connection parameters are whether FCIP is configured to use them. > TCP statistics - aggregated across connections that > constitute a single > FCIP link. This includes tracking of totals across closed TCP > connections in a still-open link. > > Some more global suggestions: > - Beware of just duplicationg TCP parameters from other MIBs. > Aggregation across TCP connections in an FCIP > link should be > ok. > - Look at RMON and follow its style to define an FCIP RMON MIB. > > FCIP use of Common Encapsulation > -------------------------------- > > ** Rough Consensus: FCIP will not use CRC. > > The fact that this leaves the timestamp field unprotected > was characterized as acceptable because the probability of an > error causing > a > frame that should have been discarded to not be discarded is > very small due > to > the timestamp needing to fall within an approximately 40 > second window to > cause > the frame to be forwarded. 40 seconds is more than enough > for satellite > transmission - this won't allow FCIP to go to the moon, but > that's probably > out of scope. > > Protocol-specific fields for FCIP are not needed (protocol works fine > if words 1 and 2 are zero), The decision on their use is: > ** Rough Consensus: For FCIP, word 1 is a copy of 0, the first 16 bits > of word 2 are reserved, and the last 16 bits are the > 1's complement > of the first 16 bits. Consensus called over Charles Monia's > objection. > > Back to protocol-specific data. Bob S wants to use Word 1 as a copy > of Word 0, and reserve Word 2 (reserved/-reserved) (replicate > 3 into 2? - would consume all extra space). > > -> Action: David Black will work with the FCIP authors to > provide right > Quality of Service (diffserv) words. > > -- Model Update (Jim Nelson, Vixel) > > Work in progress based on direction Mike O'Donnell presented > in Minneapolis, > update prior to T11 June meeting (week of June 4 in > Minneapolis - > BB-2 > is after FC-SW-2 on Monday, afternoon). > Full model goes in FC-BB-2, a piece of this goes in FCIP to > replace BSW > (Backbone switch) discussion. Full discussion when the > text becomes > available (August, except that London IETF meeting > conflicts w/T11) > > --> Action: David and Elizabeth are already in touch with the > Area Directors > regarding the IETF and T11 meeting conflict, and what > to do about > it. > > -- FCIP interaction with TCP > > Changes made in -02 version of FCIP draft: > - Now distributing FC flows across multiple TCP connections (each port > pair [N_Port login session] has to be limited to one > TCP connection > for in-order delivery, and FC ACKs need to be on same > conn. as ACKed > traffic). > - Removed discussion of TCP parameters. > - Error recovery (TCP keep-alive too large): 2 possible approaches > -1- Fibre Channel ELS (Extended Link Sequence) to detect loss of > connectivity. > This would not propagate across endpoints, and > a zero-length > frame might be better as that would not be a > valid frame to > forward into Fibre Channel. > > -> Action: Bob Snively will check whether FSPF (Fibre Channel routing > protocol > that will always be present on an FCIP link as FCIP is currently > specified) > already contains heartbeats to detect a lost connection. If so, > FCIP can > leave this issue to FSPF to handle. > > -2- Check time stamps. Added text to indicate that > horribly late > frames get bit-bucketed. Common Encapsulation says "put > timestamp > here", but doesn't provide much in the way of details > > Resulting discussion was inconclusive. FC experts need to write the > timestamp > rules to make sure that timestamp processing adheres to FC > requirements - > these rules probably belong in the common encapsulation > document as they > apply to any Fibre Channel traffic. The following issues are open: > (1) How do we ensure that the times at both ends of an > FCIP or iFCP > link are synchronized? FC-GS-4's time protocol > or requiring > use of NTP across the link are possibilities. > FC B ports > can't > source or sink FC-GS-4 frames, so may have to use NTP. > (2) Need a concise and strict definition of when it's > ok to *not* > timestamp traffic. > > Current text on loss of sync and related error recovery > issues is incorrect > in several ways and needs to be revised. Revison *must* not be > susceptible > to misinterpreting frame payload as a frame when sync is lost. > > Multiple TCP connections > - iFCP classifies by destination > - FCIP can classify by traffic class (frame type), and also by > destination > - FC class 4 is defining virtual channels (can have > 1 per > destination). > > --> Action: David Black to run some version of the above > summary past the > Area > Directors to make sure this doesn't run afoul of > general principle > of not > using multiple TCP connections to increase a single > application's > performance. > > -- FCIP QoS and Performance > > -> Action: David Black will check QoS text in Section 9.1 of > -02 draft and > review > first sentence of Section 9.2 > -> Action: Authors will look at QoS text in iFCP draft. > > The TCP performance section of the draft contains some > informational/advisory > text on dealing with some TCP issues (e.g., long/fat networks > [high bandwidth x delay product]). > > > Flow control management (Section 10) This is about flow > control mapping > between FC and TCP. Section title is misleading and > may be changed. > -> Action: Remove reference to Fibre Channel Class 1. T11.3 > is removing > support for it in FC-MI. > > There are open issues in Section 10. This is about > reflecting TCP traffic > propagation issues into FC credit-based flow control. Source > throttling. > > -- Data Integrity (Bob Snively, Brocade) > > This was a discussion of detailed edits to the draft to make > sure that there > are no serious date integrity issues. > > Section 3 error handling text will be deleted in favor of a > pointer to 8.3. > > Section 4.3 on ordering will be connected this to the multi-path > classification > characteristics above. > > Section 8.1 - Bob Snbively wants to add ability to do > data-based resync > > -> Action: Bob will write the text to explain how to do this > for serious > review. Will also rewrite the text to distinguish use > of another > framing mechanism vs. the data scan. Resync should be > optional, and > data scan must not be capable of misinterpreting what appears to > be a Fibre Channel frame in an FCIP data payload as an > actual frame. > > Section 8.3 - additional text on dealing with CRC errors for > frames going > onto Fibre Channel. > > Section 4.2 - use usual login procedure for fabric devices. > FCIP need not > do some sort of TCP/IP logins, but FC mechanisms must work > unmodified. > > Endpoint definition - distinguish Fibre Channel port from TCP > connection > endpoint. > > > Naming and Discovery for FCIP > ----------------------------- > > -- FCIP and iSNS - Josh Tseng, Nishan > > Hostnames will name FCIP boxes - discussed as possibility in Orlando, > presented > as proposal at this time. DNS or iSNS can be used to > resolve them - > this > is about tunnel configuration for FCIP. > > Simple DNS approach - have to enter hostnames of remote FCIP > devices in each > FCIP device. Each device resolves names with DNS and sets up > tunnels. > All configuration is manual. > > iSNS can automate configuration and discovery. Domain concept makes > group-based > managment of FCIP connectivity possible - scales better > than manual > configuration. iSNS server can be discovered via DHCP option or > SLP. > > FCIP box identifiers don't have to be DNS hostnames, but those are > convenient. > Can automate hostname assignment to FCIP boxes. Management > station can > alias names. > > -- FCIP use of SLP - Dave Peterson, Cisco [used Excel > spreadsheet + MS Word > document] > > iSNS and SLPv2 use same authentication. > > Dave Peterson doesn't anticipate using SLP scopes for FCIP. > New SLP RFC (3082) provides sufficient (somewhat timely) notification > of devices appearing and disappearing. > > Proposal - static config, SLP for dynamic config, also iSNS, both are > optional. > > DHCP - self-discovery mechanism, can't discover info about others. > DNS - too limited, SLP improves > LDAP - database interface, may be used by either SLP or iSNS > > Putting together FCIP discovery model for SLPv2. > > Suggestion - just use SLP, limit iSNS to functionality beyond SLP. > > Josh: > RFC 3082 is Experimental, SLP scope is different from iSNS Discovery > domains. > SLP periodic re-registration has to be set correctly. > SLP can't store non-string attributes (e.g., X.509 certificates). > SLP is a good discovery mechanism, but not good for > subsequent/sophisticated > mgt. > > After much discussion, it was not possible to call consensus on the > relationship > between iSNS and SLP usage/requirements for FCIP. > > -> Action: Josh and Dave Petersen will post their positions > to the reflector > with David Black and Elizabeth summarizing areas of > agreement and > open issues. > > iFCP - Charles Monia, Nishan > ----------------------------- > > Changes to draft - iFCP addressing model updated, including > how to manage > address tables, and incorporation of transparent mode. > ELS handling changed for common encapsulation. No longer need > extended header (this was discussed in Minneapolis). Lots of > editorial > changes. QoS text added, now 1 TCP/IP connection per > N_Port session. > > Next version targeted for May 15, major work is to incorporate common > encapsulation. > > Changes to incorporate Common Encapsulation: > > (1) Protocol-specific fields: > - LS_CMD - identify command for Accepts of augmented ELS's. > - iFCP flags (see below), copy of SOF, copy of EOF. > iFCP uses these copies, and skips over SOF and > EOF words. > - CRC is always checked, CRCV field is ignored on receive, but > must be set on send. > > (2) Flags > - Compliance level (which version of document, watch out for > version # change). This is a 1-bit version number. > - Augmented (part of an ELS w/augmented data) > - Address transparent vs. translation mode > - iFCP session control frame (must be non-augmented, > translation) > > Header CRC error causes connection close. CRC error in 7-word header > is unlikely, and only a single N_port to N_port connection is > affected. > > (3) Timestamp format and content will be same as FCIP. > Likely to be able to > put this syntax and semantics into the common encapsulation > document. > > iFCP address-transparent mode. This assembles iFCP gateways > into a single > logical fabric, within which addresses are global and > untranslated. > Maximum of 239 gateways in a fabric (this is the FC domain limit > for number of switches, each gateway counts as a switch). > Address translation mode will probably be mandatory. Translation and > Transparent mode do not interoperate. Transparent mode does not > require any FC header or CRC changes. > > iFCP transparent mode is similar to FCIP, but implements FC services > via IP equivalents rather than using FC switches. > Gateways implement F or FL port, could even do an E port. For E port, > gateway MUST become the principle switch. > > ELS modifications - out of 73 total ELSs, 14 of them, and 3 of their > ACC (Accept) responses require iFCP intervention/changes. > -> Action: Authors to look at FCP-2 document for more ELSs. > REC now comes > from > there, and SRR appears to be missing from the list. > > One of the types of augmentation appends WWN to ELS sent between iFCP > gateways. > TPRLO (Third PaRty LogOut) can require a massive augmentation that can > span multiple FC frames due to the ability to to specify up > to 4095 devices > to be logged out. > > Gateways have to maintain state to track augmented Accepts > and handle them > properly - this is for the gateway that proxies for target of > ELS. Exchange > identifier associates ACCs with ELSs on the FC side of that proxy. > > Fabric Service Profiles. This is about making sure that a iFCP-base > fabric uniformly exports services to all attached devices, even > if gateways are from different vendors. > - Fabric services to be provided at well-known > addresses (mandatory, > optional, prohibited, behavior description for each > service). > - Transport (details of Class 2 and 3 behavior, > prohibit Class 1, > FC QoS mapping [when FC does QoS]). > > iFCP Naming and Discovery - Josh Tseng > -------------------------------------- > > iSNS is required for iFCP - nothing else works. > Overview of iSNS use by iFCP. > > Security for iFCP - use IPSec, probably tunnel mode. > FCIP is leaning in the same direction. > > Security Comments - David Black > ------------------------------- > > These comments apply to all IPS protocols, and are > particularly applicable > to the iSCSI security discussion scheduled for tomorrow. > > Confidentiality is not required. The IPS WG may have the freedom > to "profile" IPSec, e.g., make AH optional to implement. The > expensive > parts of IPSec are Confidentiality (cycles) and > negotiation/key exchange > [IKE/ISAKMP] (code) - IKE/ISAKMP could be optional to implement (i.e., > manual configuration, with attendant management problems). One of the > ipsec WG chairs should be here tomorrow to provide additional > information. > > --------------------------- > May 1, 2001 - iSCSI > --------------------------- > > iSCSI Naming and Discovery > -------------------------- > > -- Overview - Kaladhar Voruganti, IBM > > -- Naming and Addressing - Mark Bakke, Cisco > > WWUI acronym in earlier versions of draft gone, set of naming types > has been reduced to eui (WWN) and fqn. fqn acronyn will be changed to > iqn (i = iSCSI) to avoid possibly rude (mis)-pronunciation > > An iSCSI name is a persistent location-independent > identifier, an iSCSI > address is how one gets to it, with DNS recommended in > preference to > specifying the IP address. > > There is a problem with domain names changing ownership > without record of > what iSCSI names iqns were created by the old owner. This risks > unacceptable > duplication. Suggestions for using IANA private enterprise > identifier, > date, > and some sort of existing vendor identifier were made. The > simple approach > of using WWNs runs into administrative difficulties like a > $1500 fee to get > initial allocation. > > -> Action: Authors to propose resolution to this issue. > > Third party command authentication/authorization is an open issue. > > -- SAM2 and iSCSI (concept mapping) - Jim Hafner, IBM > > This is about how the concepts in the SCSI architecture (SAM2) map > to the concepts in iSCSI, and the implications of this mapping. > > Basic concepts and mappings: > o Network entity - has an IP network address (and TCP port) > o iSCSI Node - within network entity, identified by iSCSI Name > Contains at most ONE SCSI device (canonical iSCSI Target > does not have a SCSI device). > o SCSI Device Name (T10 is working on this) = iSCSI Node Name > o SCSI Device - contains SCSI Ports = iSCSI Node Name + ISID or > TSID. > o SCSI Port has one or more > pairs, may change > dynamically. > o SCSI I_T Nexus = iSCSI Session, connects two iSCSI Ports. > > Q: Multiple sessions per node pair? > A: Yes, each session has unique SCSI Port instance on each side. > Result is that SCSI Ports are dynamic entities. This > is different > from models used in other SCSI tranports because the SCSI Port > entities are bound one-to-one by a connection. > > The iSCSI Name is intended to name OS image (actually > SCSI instance in OS image) - goal here is to avoid problem with > FC Node Name, which is associated with HBA - iSCSI Node Name is > supposed to be associated with the "SCSI entity" in the OS. > This requires the SCSI instance to manage ISIDs or > TSIDs globally > across all iSCSI interfaces. > > SCSI Port Address - not clear what this T10 concept maps to, could > just use SCSI Port Name (iSCSI name + ISID or TSID). > > Mapping Consequences: > - ISIDs have to be unique initiator-wide (across all > NICs/Adapters > connected > to same target). > - Persistent reservations get linked to both ISID and TSID -- > reservation > comes back only if initiatiator reuses ISID and target > reuses TSID. > This has implications on target saving/tracking > of state, > and requires > initiator to remember which ISID was used with > which target. > > -> Action: Next version of draft will need a table of how > nexus state reacts > to > events (e.g., Resets). > > Discussion of Persistent Reservation behavior wrt this model. > SCSI has > things > to say about Target reboot. Host reboot is generally > not visible, > but > only exception is that if addresses are reassigned > (e.g., by Fibre > Channel > fabric), Target has to cope with this reassignment. > > Names span sessions to make authentication simpler. Keeping > ISID separate > from the name makes the authentication stuff simpler. > > Q: Do persistent reservations open up a denial of service > attack route? > A: No - can't do this without prior authentication, and there are ways > to blow away reservations from other initiators. > Difference from FCP is that iSCSI persistent > reservations are keyed > to session IDs vs. FCP keying them to FC ports > (via WWN). > > SCSI allows sharing of reservations. At the iSCSI level, > each reservation > is tied to a session. SCSI reservation sharing is how > reservations > get > shared across more than one I_T_Nexus. T10 is considering > reservation > semantics that ignore ports and make reservations at SCSI Device > level, > which would yield reservations at iSCSI node granularity. > > On session establishment, TSID choice may be influenced by existence > of reservation state - this results in iSCSI having to check for > SCSI persistent reservations for that initiator name and ISID. > > Third party reservations were brought up. They're volatile, > essentially > obsolete, and not needed by anything in SCSI, included > extended copy. There > are also command format problems that would make it very > difficult for iSCSI > to pass a node name + session ID to the third party copy device. > > Persistent reservation key *does not* contain iSCSI node name or ISID. > Cluster > software (or whatever) creates its own reservation keys. > > --> Action: There may be an issue of scope of ACA with > respect to the model. > Jim Hafner will check into this. CA should not be an > issue because > iSCSI requires Autosense. > > Access controls need to use iSCSI Initiator node name. T10 > will handle > the required changes, as part of long identifier format changes > (e.g., for things like SCSI alias designation), which also cover > third-party commands. Additional address info for third-party > commands could be definitive, or just hints. SRP and FCP use > definitive addresses. > > -- Discovery - Mark Bakke, Cisco > draft-ietf-ips-iscsi-name-disc-01.txt > > Review of types of configuration: > - Static/manual > - SLP for simple discovery > - iSNS for centralized management > "Send Targets" - an additional mechanism that can be used in all three > cases. > Send Targets is allowed to send info on targets that > are elsewhere, > or just targets behind its port. This can also > implement > some "soft zoning" (i.e. restricted what an > Initiator can > see), > and cascading (return a target that is > elsewhere to which > "Send Targets" must be sent). > > If there's only one iSCSI target behind a port, a login to the "iscsi" > target > could return that name and immediately allow commands. > This would > lead to needing to distinguish between login only for > "Send Targets" > and login that results in the one and only target > coming back and > going into full-feature mode. It adds complexity but > might simplify > booting. OPEN ISSUE. > > There is a need for some sort of iterator interface or > something else that > avoids an Initiator having to allocate potentially > unlimited space > to > hold the results of Send Targets. This issue will be > taken to the > mailing > list. OPEN ISSUE. > > Will be adding tags to addresses that specify which addresses can be > aggregated into a session. Absence of tag indicates that > aggregation > is not possible. Tag unique to a single address would > indicate that > multiple connections to that address are aupported. > Only semantics > of tag is whether these values match, and context is > within a single > response to Send Targets. One tag per address for now. > > The ability to specify another initiator in Send Targets will not be > supported, as there is T10 work to be done in the access control > area (this is about giving an Initiator's view to > a third party copy device) and it raises security issues. > > -- SLP - Mark Bakke, Cisco draft-ietf-ips-iscsi-slp-00.txt > > SLP template has been submitted as WG draft. > SLP and iSNS now do *exactly* the same authentication. > > Working on interoperability with iSNS. > Working on host/device taxonomy to make recommendations about > what should > be implemented where (SLP and/or iSNS in drives, arrays, hosts, > gateways, etc.?). > > Open source for SLP authentication is now available. > Need to look at RFC 3082 for event propagation to see what it does and > figure > out how to use it. iSCSI may need a few minor > extensions to this. > The iSCSI SLP Template may need changes to better accomodate > copy managers. > > None of the discovery mechanisms are REQUIRED at this point in time. > A question was asked about how network management tools > discover network > configuration and devices - this is generally based on MIBs and > reading information from them. One can have a > conformance statement > for a MIB that mandates a relatively small portion of it that is > necessary for this sort of discovery. > > -- iSNS - Josh Tseng, Nishan draft-ietf-ips-isns-02.txt > > SNS requirements in naming and discovery draft are based on FC-GS-3. > > Discussion of iSNS functionality and alternatives: > > - LDAP: This is a directory service, but seems to lack info > about when and > where > to send state change notifications. > - SNMP: Discovery domains cannot be implemented, cannot do > complex queries. > - SLP: Now has a notification mechanism, but needs some changes. SLP > depends on multicast (in absence of SLP, have to > manually configure > addresses of SLP directory agent). DHCP can discover SLP, hence > can piggyback on DHCP infrastructure. SLP scopes > aren't the same > as discovery domains - have to add something. SLP requires on > service agents to reregister with directory agents, iSNS does > things in the reverse direction. > > Nishan will open source iSNS client and server within next month. > > iSCSI Requirements Last Call Issues - Marjorie Krueger, HP > ----------------------------------- > > This discussion was about closing the issues raised during > informal Last > Call of > this draft on the mailing list. > > ** Rough Consensus: iSCSI should not be making changes to > SCSI-3, except > where the > relevant SCSI document says something is "transport dependent". > Wording will > be crafted to require "minimum changes" or something like that > outside of > this case. > > ** Rough Consensus: Layering wording will not be added to > this document; the > exhortation in the IPS WG charter is sufficient. > ** Rough Consensus: "packet formats similar to" wording will > NOT be added. > > ** Rough Consensus: Accept Bob Snively's change to require > ordering only > at I_T_L level, not I_T, but don't exclude > implentations that do I_T > ordering. > > After a long discussion of ordering, the conclusion was: > ** Rough Consensus: Strictly ordered command delivery to > target is required > across > a session, even in error recovery cases (this is stricter than > either FCP or > parallel SCSI require). Those wanting looser ordering > should use > multiple > sessions. > > Rough consensus on the last point was called over visible opposition > (significant > minority of the room). > > Security > -------- > > -- David Black (EMC, co-chair) presentation on security requirements > > See slides - this was an informational presentation/discussion about > requirements > (MUST implement authentication and cryptographic integrity, MUST > have a > required mechanism for each, confidentiality is OPTIONAL, IP vs. > iSCSI > level mechanisms [iSCSI can multiplex initiators and > targets on a > single > pair). > > -- Ofer Biran (IBM, Haifa) > > Lots of slides - see slides for details. > > After much discussion ... > > ** Rough Consensus: > (1) Need iSCSI level authentication in addition to what is > done for IP or > TCP/IP. > (2) ESP with null authentication is mandatory to implement > - TLS was considered and rejected as part of this. > (3) SRP is mandatory to implement. > > Beyond this, the use of SRP (or another inband authentication > mechanism) > to key ESP was discussed as a promising direction. > It's premature > to require this - the minimum requirement for now is ESP with > pre-shared > keys and a separate implementation of SRP. > > -> Action: David Black, Ted Ts'o, and Ofer Birman will write > up details of getting ESP key material from in-band > authentication > mechanism and related "Start Here" text key to turn on ESP > dynamically. > > Error Recovery - Randy Haagens, HP and Kalman Meth, IBM > ------------------------------------------------------- > > NOTE: Due to the late hour and diminishing attendance, no > consensus calls > were attempted in this area. > > This is a progress report from some extensive and on-going > off-line work > on details of errror recovery. The goal is to provide a set of > mechanisms that allow implementations to choose where they want > to be between "Trust TCP Explicitly" (Detect errors and flag > to SCSI) and "Can't Trust TCP" (comprehensive error recovery. > This work will be folded into the basic iSCSI document. > > Session recovery appears to be cleanly separable from finer-grained > mechanisms. Not sure about separations among within-session, > within-connection, and within command recovery. > > Not every situation will need fine-grained recovery. Session recovery > will be mandatory - close session and open a new > session. ** Need > to make it clear that explicit logout on last > connection in session. > > Within-session recovery reissues commands on another connection in > same session - login of new connection includes instruction > to logout old connection. Have to make sure that old connection > is successfully logged out before issuing new commands with > retry (X) bit set. This is optional. Have to use same tags > and CmdSNs in doing the retry. > > Terminology note: This is really "resend". Whether or not a > "retry" of > the command is actually done is at the device's option. Need > to be careful to distinguish these terms. > > Comment: Wording is needed to indicate that if the initiator > chooses not to reissue a command that was pending on the > terminated connection, it MUST terminate the session. > > Q: Is having both implicit and explicit logout mechanisms overkill? > A: Needed to cover resend on an existing connection vs. > opening a new connection for this purpose. Target can > reject attempt to open new connection, and then initiator > has to clean up. > > Comment: This portion of the draft contains a recommendation > to use Target Reset that needs to be removed. > > Within-connection recovery reissues command on same connection to fill > CmdSN holes. Has to be on same connection due to connection > allegiance. These errors require framing support over TCP - > in the absence of framing, the connection has to > be terminated. > > Comment: When there are multiple connections, target needs > to delay for a bit before telling Initiator that commands are > missing. > On a single connection, this is easy. When CmdSN has > advanced past > the hole on all connections, target knows that a > command is missing. > A request was made for an explicit way to request a > resend rather > than the implicit monitoring of ExpCmdSN by the Initiator and > possible > use of NOP by the target. This will be considered. > > Discussion of (lack of) trust in TCP. Between ESP at IP > level and encouraging SCTP to use a CRC, we're getting stronger > integrity in the stack at a lower level. TCP (or SCTP) > will retry > is a wonderful answer to a whole bunch of problems. > This lead to > some interest in figuring out how to shim a CRC between > TCP and IP, > which would then allow at least within-connection and > within-command > error recovery to be dropped. OPEN ISSUE: for further > discussion. > > Discussion of data resends differing between reads and > writes. On read, > initiator resends command causing target to resend all the > data. On write, > the target can issue only some of the R2Ts (for the missing > data) and it > should not be a big deal for the initiator to cope. OPEN ISSUE - will > require confirmation/further discussion on list. > > Status recovery within connection is based on status SNACK. > > -> Action: Need to revise text to indicate that SNACK is > optional. SNACK > is > required to do within-connection recovery of status. > > Comment: This may be a big deal for backup - put in a negotiation > key/value pair to allow initiator to discover whether target > does SNACK. Backup may want to avoid using a target > that doesn't > do status SNACK. OPEN ISSUE: Take to list. > > Within-command recovery from dropped data. If data CRC fails, issue > immediate Data SNACK. If header CRC fails, find the hole, issue > Data SNACK at that point. > > From owner-ips@ece.cmu.edu Wed May 16 18:47:20 2001 Received: from ece.cmu.edu (ECE.CMU.EDU [128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id SAA20640 for ; Wed, 16 May 2001 18:47:20 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f4GLKhm15853 for ips-outgoing; Wed, 16 May 2001 17:20:43 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from e21.nc.us.ibm.com (e21.nc.us.ibm.com [32.97.136.227]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f4GLKeH15843 for ; Wed, 16 May 2001 17:20:40 -0400 (EDT) Received: from southrelay02.raleigh.ibm.com (southrelay02.raleigh.ibm.com [9.37.3.209]) by e21.nc.us.ibm.com (8.9.3/8.9.3) with ESMTP id RAA112624; Wed, 16 May 2001 17:14:28 -0500 Received: from d04nms25.raleigh.ibm.com (d04nms25.raleigh.ibm.com [9.67.228.6]) by southrelay02.raleigh.ibm.com (8.11.1/NCO v4.96) with ESMTP id f4GLKcA39682; Wed, 16 May 2001 17:20:38 -0400 Importance: Normal Subject: Re: iSCSI: Aggregation tags in SendTargets To: Mark Bakke Cc: IPS X-Mailer: Lotus Notes Release 5.0.3 (Intl) 21 March 2000 Message-ID: From: "Thomas McSweeney" Date: Wed, 16 May 2001 17:20:37 -0400 X-MIMETrack: Serialize by Router on D04NMS25/04/M/IBM(Release 5.0.6 |December 14, 2000) at 05/16/2001 05:20:38 PM MIME-Version: 1.0 Content-type: text/plain; charset=us-ascii Sender: owner-ips@ece.cmu.edu Precedence: bulk >Of course, if there were some sort of flag or text key in >the iSCSI login response to indicate the support of multiple- >connection sessions, we wouldn't need to do this. I don't >think it's in there, though. Sure it is! The first tag in appendix E: 08 MaxConnections Use: LO Who: Initiator and Target MaxConnections= Default is 8. Initiator and target negotiate the maximum number of connections requested/acceptable. The lower of the 2 numbers is selected. Tom McSweeney iSCSI Development, Storage Systems Group, IBM Email: rf42tpme@us.ibm.com Phone: (USA) 919-254-5634 (tie line: 444-5634) Fax: (USA) 919-254-0391 (tie line: 444-0391) From owner-ips@ece.cmu.edu Wed May 16 18:48:25 2001 Received: from ece.cmu.edu (ECE.CMU.EDU [128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id SAA20671 for ; Wed, 16 May 2001 18:48:25 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f4GKtUe13779 for ips-outgoing; Wed, 16 May 2001 16:55:30 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from dogwood.cisco.com (dogwood.cisco.com [161.44.11.19]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f4GKtSH13771 for ; Wed, 16 May 2001 16:55:28 -0400 (EDT) Received: from cisco.com (mbakke@mbakke-lnx.cisco.com [161.44.68.87]) by dogwood.cisco.com (8.8.6 (PHNE_14041)/CISCO.SERVER.1.2) with ESMTP id QAA08296; Wed, 16 May 2001 16:55:11 -0400 (EDT) Message-ID: <3B02E890.4DB39D2A@cisco.com> Date: Wed, 16 May 2001 15:52:32 -0500 From: Mark Bakke X-Mailer: Mozilla 4.72 [en] (X11; U; Linux 2.2.16-3.uid32 i686) X-Accept-Language: en, de MIME-Version: 1.0 To: "KRUEGER,MARJORIE (HP-Roseville,ex1)" CC: Kevin Gibbons , IPS Subject: Re: iSCSI: Aggregation tags in SendTargets References: <6BD67FFB937FD411A04F00D0B74FE87802A09063@xrose06.rose.hp.com> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-ips@ece.cmu.edu Precedence: bulk Content-Transfer-Encoding: 7bit Marjorie- You are right; I meant alpha-numeric. Again, if nobody really needs them, we can stick with numeric tags. -- Mark "KRUEGER,MARJORIE (HP-Roseville,ex1)" wrote: > > I think you meant to ask > > *** Anyone need Alpha-Numeric Aggregation Tags *** > ^^^^^ > Going once.... > > I have no strong opinion either. Alpha's a little more user-friendly, but > that is not important enough in this feature to warrant any argument, so if > implementors want numeric tags, let them be numeric... > > Marj > > > -----Original Message----- > > From: Mark Bakke [mailto:mbakke@cisco.com] > > Sent: Wednesday, May 16, 2001 9:13 AM > > To: Kevin Gibbons > > Cc: IPS; Marjorie Krueger > > Subject: Re: iSCSI: Aggregation tags in SendTargets > > > > > > Kevin- > > > > I think that a numeric tag would be simpler as you had said. > > Marjorie had asked for the alpha-numeric tag; anyone needing > > more than a numeric tag, please speak up. I don't have any > > particular opinion either way. > > > > If nobody requires alpha-numeric tags, I will change it to > > a simple numeric tag. Otherwise, I'll update the examples. > > > > So, > > > > *** Anyone need Numeric Aggregation Tags *** > > > > Going once.... > > > > -- > > Mark > > > > Kevin Gibbons wrote: > > > > > > Mark, > > > in the proposed change to the NDT document, you > > state that the > > > aggregation "tag" is an ASCII, alpha-numeric string. I > > read this to mean > > > the tag can be any string value. > > > > > > Could you make the aggregation tag be a numeric > > string, indicating > > > an aggregation group? This would align with your example, > > and should make > > > it easier to quickly index portals into different groups. > > > > > > Regards, Kevin > > > > > > -----Original Message----- > > > From: Mark Bakke [mailto:mbakke@cisco.com] > > > Sent: Tuesday, May 15, 2001 1:06 PM > > > To: IPS > > > Subject: iSCSI: Aggregation tags in SendTargets > > > > > > During the interim meeting, we had discussed a proposal to > > > add an aggregation tag to the SendTargets response, indicating > > > which (if any) target addresses supported multiple connections > > > per session, and which groups of addresses an initiator could > > > hope to aggregate a session across. > > > > > > Aggregation tags were generally well-received; a small modification > > > to the proposed method also allows an initiator to know whether > > > a single address supports multiple connections per session just > > > by itself. > > > > > > Here is the section that would go into the NDT document. > > > > > > -- > > > > > > (This would be added to section 4.2, right before the > > vendor-specific > > > paragraph at the end): > > > > > > If an iSCSI target supports multiple connections per session, > > > it must indicate this by including an aggregation tag after each > > > address, in the form of > > > > > > TargetAddress=address,tag > > > > > > Where "tag" is an ASCII, alpha-numeric string indicating > > an address > > > group. Within a single session, a connection may be > > requested to any > > > combination of TCP addresses that have the same tag. If > > an address > > > supports multiple connections per session, but does not support > > > spanning a session across other addresses, it will have its own > > > tag. > > > > > > Here is an example: > > > > > > TargetName=fqn.com.acme.diskarray.sn.8675309 > > > TargetAddress=10.1.0.45:3000,1 > > > TargetAddress=10.1.1.46:3000,1 > > > TargetAddress=10.1.0.47:3000,2 > > > TargetAddress=10.1.1.48:3000,2 > > > TargetAddress=10.1.1.49:3000 > > > TargetAddress=10.1.1.50:3000,3 > > > TargetAlias=Oracle tables > > > > > > In this example, any of the target addresses can be used to reach > > > the same target. A single-connection session can be established > > > to any of these TCP addresses. A multiple-connection > > session could span > > > addresses .45 and .46, or .47 and .48, but cannot span any other > > > combination. A TargetAddress without a tag (.49) cannot > > be combined > > > with any other address within the same session. A TargetAddress > > > with a tag that is not shared with other addresses > > supports multiple > > > connections per session, but all connections must be to the same > > > address. > > > > > > To make this work, there are a few rules to follow: > > > > > > A target that does not support spanning sessions across > > multiple addresses > > > MUST NOT include the tags. > > > > > > A target that is accessible via multiple TCP addresses > > SHOULD include > > > all TCP addresses in a SendTargets response. > > > > > > A target with multiple TCP addresses that supports a > > session spanning > > > multiple TCP addresses MUST indicate TCP address groups > > using aggregation > > > tages in a SendTargets response. > > > > > > Aggregation tags have no meaning or persistence beyond a > > particular > > > SendTargets response. > > > > > > -- > > > Mark A. Bakke > > > Cisco Systems > > > mbakke@cisco.com > > > 763.398.1054 > -- Mark A. Bakke Cisco Systems mbakke@cisco.com 763.398.1054 From owner-ips@ece.cmu.edu Thu May 17 00:06:48 2001 Received: from ece.cmu.edu (ECE.CMU.EDU [128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id AAA25702 for ; Thu, 17 May 2001 00:06:48 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f4H1vQs04598 for ips-outgoing; Wed, 16 May 2001 21:57:26 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from palrel2.hp.com (palrel2.hp.com [156.153.255.234]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f4H1vPH04593 for ; Wed, 16 May 2001 21:57:25 -0400 (EDT) Received: from xparelay2.corp.hp.com (xparelay2.corp.hp.com [15.58.137.112]) by palrel2.hp.com (Postfix) with ESMTP id 60E3129B8; Wed, 16 May 2001 18:57:24 -0700 (PDT) Received: from xpabh3.corp.hp.com (xpabh3.corp.hp.com [15.58.136.223]) by xparelay2.corp.hp.com (Postfix) with ESMTP id 721E81F504; Wed, 16 May 2001 18:47:00 -0400 (EDT) Received: by xpabh3.corp.hp.com with Internet Mail Service (5.5.2653.19) id ; Wed, 16 May 2001 18:57:22 -0700 Message-ID: <6BD67FFB937FD411A04F00D0B74FE87802A09067@xrose06.rose.hp.com> From: "KRUEGER,MARJORIE (HP-Roseville,ex1)" To: "'Mark Bakke'" , Black_David@emc.com Cc: ips@ece.cmu.edu Subject: RE: iSCSI: Canonical Targets Date: Wed, 16 May 2001 18:57:21 -0700 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2653.19) Content-Type: text/plain; charset="iso-8859-1" Sender: owner-ips@ece.cmu.edu Precedence: bulk > > How about this instead: > > 2. A discovery target MUST be accessible on the default tcp port > on each IP address on which an iSCSI implementation is listening > for iSCSI connections. This default tcp port will often be the > IANA-assigned TCP port for iSCSI. If the implementation is > listening on multiple TCP ports, the discovery target MAY be > accessible on each TCP port. I feel this "MUST" is a "MAY". The comments in this thread seem to focus on enabling I-T discovery in an "easy", insecure way. It seems to me there are many possibilities for "discovery models" ranging from initiators probe targets "discovery" ports to initiators access "domain-specific directories" (this has also been referred to as "storage name servers", but I think that model is a narrow subset of a network resource management soln) In the environments where initiators get their storage resource information from a "directory server", it would be desirable to disable this "discovery target" to guard against unintended/unauthorized access to targets. Initiators may be authenticating to "directory servers" who then pass specific information about targets to attach to, as well as an authentication token. The initiator could then log into the specific targets with the authentication token and the target is freed from the task of assigning/managing initiator lists. Marjorie Krueger Networked Storage Architecture Networked Storage Solutions Org. Hewlett-Packard tel: +1 916 785 2656 fax: +1 916 785 0391 email: marjorie_krueger@hp.com From owner-ips@ece.cmu.edu Thu May 17 00:07:55 2001 Received: from ece.cmu.edu (ECE.CMU.EDU [128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id AAA25722 for ; Thu, 17 May 2001 00:07:55 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f4H2Y2k06806 for ips-outgoing; Wed, 16 May 2001 22:34:02 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from mxbh4.isus.emc.com (mxbh4.isus.emc.com [128.221.10.33]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f4H2Y1H06801 for ; Wed, 16 May 2001 22:34:01 -0400 (EDT) Received: by mxbh4.isus.emc.com with Internet Mail Service (5.5.2653.19) id ; Wed, 16 May 2001 22:33:55 -0400 Message-ID: <0F31E5C394DAD311B60C00E029101A070801558C@corpmx9.isus.emc.com> From: Black_David@emc.com To: ips@ece.cmu.edu Subject: TCP Framing Date: Wed, 16 May 2001 22:33:53 -0400 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2653.19) Content-Type: text/plain Sender: owner-ips@ece.cmu.edu Precedence: bulk I checked with the TSVWG chairs (our overworked Transport Area Directors), and the current state of the TCP framing world is that TSVWG expects to advance a slightly modified version of the tcpulpframe draft discussed in Minneapolis to become an RFC of some form in the near future (subject to timely revision by the authors). This suggests that making iSCSI markers mandatory may not be a wise decision. We'll need to see the revised version of tcpulpframe and understand what the IESG does with it before figuring out the right wording to deal with it in iSCSI, but this should probably be viewed as the default TCP framing mechanism. Let me remind everyone that TCP framing drafts get discussed on the tsvwg list, *not* here. I put this message up in response to earlier messages expressing concern about lack of knowledge about what TSVWG intends to do in this area. Thanks, --David --------------------------------------------------- David L. Black, Senior Technologist EMC Corporation, 42 South St., Hopkinton, MA 01748 +1 (508) 435-1000 x75140 FAX: +1 (508) 497-8500 black_david@emc.com Mobile: +1 (978) 394-7754 --------------------------------------------------- From owner-ips@ece.cmu.edu Thu May 17 03:49:30 2001 Received: from ece.cmu.edu (ECE.CMU.EDU [128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id DAA11807 for ; Thu, 17 May 2001 03:49:29 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f4H5ffl18503 for ips-outgoing; Thu, 17 May 2001 01:41:41 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from msgbas1.cos.agilent.com (msgbas1x.cos.agilent.com [192.6.9.33]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f4H5fdH18498 for ; Thu, 17 May 2001 01:41:39 -0400 (EDT) Received: from msgrel1.and.agilent.com (msgrel1.and.agilent.com [130.30.33.104]) by msgbas1.cos.agilent.com (Postfix) with ESMTP id 5A5BC1122 for ; Wed, 16 May 2001 23:41:37 -0600 (MDT) Received: from rtl.rose.agilent.com (rtl.rose.agilent.com [156.140.232.231]) by msgrel1.and.agilent.com (Postfix) with ESMTP id 760B7112 for ; Thu, 17 May 2001 01:41:36 -0400 (EDT) Received: from mail.rose.agilent.com (mailsrv@bellhop [156.140.233.51]) by rtl.rose.agilent.com (8.9.3 (PHNE_18979)/8.9.3 SMKit7.1.0) with ESMTP id WAA00876 for ; Wed, 16 May 2001 22:41:35 -0700 (PDT) Received: from agilent.com (cos1nai131075.cos.agilent.com [141.184.131.75]) by mail.rose.agilent.com (Netscape Messaging Server 3.6) with ESMTP id AAA660E; Wed, 16 May 2001 22:41:31 -0700 Message-ID: <3B036489.1734629D@agilent.com> Date: Wed, 16 May 2001 22:41:29 -0700 From: "Matt Wakeley" Reply-To: Matt Wakeley Organization: Agilent Technologies X-Mailer: Mozilla 4.77 [en] (Windows NT 5.0; U) X-Accept-Language: en MIME-Version: 1.0 To: ips@ece.cmu.edu Subject: Re: TCP Framing References: <0F31E5C394DAD311B60C00E029101A070801558C@corpmx9.isus.emc.com> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-ips@ece.cmu.edu Precedence: bulk Content-Transfer-Encoding: 7bit David, Please answer this question: Can the iSCSI spec *mandate* the use of this new TSVWG framing RFC in all iSCSI implementations (especially the ones that will be complete before this new RFC is even published)? If the answer is "no" then.... As I expressed in my earlier posting, the goal and purpose of framing is to: 1- eliminate the huge amounts of buffering (high speed, expensive, eats up board space, more expensive than fibre channel) that will be required by 10Gig implementations, and 2- provide a common feature set so that any (10Gig) implementation will be able to interoperate with any other iSCSI implementation, be it 1Gig, 10Gig, HW or SW implementations. The only way to do this is to *mandate* implementation of some kind of framing mechanism *IN THE iSCSI SPEC*. Implementing framing by a TSVWG RFC is great, *if* the iSCSI spec could *mandate* the implementation of this new RFC. However, I don't think that iSCSI will be able to mandate anything about the TCP stack (or its "shim" layers) [for example, you can't "mandate" that a SW implementation of iSCSI on Windows NT4 use the new RFC, because it is simply not available]. The *only* thing that iSCSI can mandate is what is sent over the [generic, unmodified, unshimmed] TCP connection that iSCSI uses. In other words, if iSCSI can not mandate modified versions of TCP or shim/framing/whatever layers, then in order for a 10Gig implementation to interoperate with all other iSCSI implementations, it will be required to implement buffering, which is what we want to avoid in the first place. The proposal is to *mandate* the *availability* of markers in the iSCSI spec. If two communicating nodes do not require framing, then they don't have to use markers. If and when TSVWG comes up with a better framing mechanism, then markers don't have to be invoked. However, if a node requires markers to operate optimally, the other node "MUST" be required to supply them. -Matt Wakeley Agilent Technologies Black_David@emc.com wrote: > > I checked with the TSVWG chairs (our overworked Transport > Area Directors), and the current state of the TCP framing > world is that TSVWG expects to advance a slightly > modified version of the tcpulpframe draft discussed > in Minneapolis to become an RFC of some form in the > near future (subject to timely revision by the authors). > > This suggests that making iSCSI markers mandatory may > not be a wise decision. We'll need to see the revised > version of tcpulpframe and understand what the IESG > does with it before figuring out the right wording > to deal with it in iSCSI, but this should probably > be viewed as the default TCP framing mechanism. > > Let me remind everyone that TCP framing drafts get > discussed on the tsvwg list, *not* here. I put this > message up in response to earlier messages expressing > concern about lack of knowledge about what TSVWG > intends to do in this area. > > Thanks, > --David > > --------------------------------------------------- > David L. Black, Senior Technologist > EMC Corporation, 42 South St., Hopkinton, MA 01748 > +1 (508) 435-1000 x75140 FAX: +1 (508) 497-8500 > black_david@emc.com Mobile: +1 (978) 394-7754 > --------------------------------------------------- From owner-ips@ece.cmu.edu Thu May 17 05:18:39 2001 Received: from ece.cmu.edu (ECE.CMU.EDU [128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id FAA12451 for ; Thu, 17 May 2001 05:18:38 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f4H7Bba23511 for ips-outgoing; Thu, 17 May 2001 03:11:37 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from c007.snv.cp.net (c007-h008.c007.snv.cp.net [209.228.33.214]) by ece.cmu.edu (8.11.0/8.10.2) with SMTP id f4H7BZH23506 for ; Thu, 17 May 2001 03:11:35 -0400 (EDT) Received: (cpmta 3886 invoked from network); 17 May 2001 00:11:29 -0700 Received: from unknown (HELO ljoy) (64.130.130.105) by smtp.telocity.com (209.228.33.214) with SMTP; 17 May 2001 00:11:29 -0700 X-Sent: 17 May 2001 07:11:29 GMT From: "Douglas Otis" To: "Mark Bakke" , "Jim Hafner" Cc: Subject: RE: iSCSI: Canonical Targets Date: Thu, 17 May 2001 00:09:07 -0700 Message-ID: MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 (Normal) X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook IMO, Build 9.0.2416 (9.0.2911.0) Importance: Normal X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4522.1200 In-Reply-To: <3B02EAB0.EB4936D2@cisco.com> Sender: owner-ips@ece.cmu.edu Precedence: bulk Content-Transfer-Encoding: 7bit Mark, That 'squirreling' depends on local storage which may not be available. It seems there is going to be something that knows what the initial target should be outside the host otherwise there is no security. If security is dependable, then automatic assignment should also be dependable. The listing function should allow human intervention to alter the default assignments. Once beyond the initial OS launch, confirming the target then makes sense. Doug > Actually, that's a good point about the multiple target names. If > a host is allowed to use a target without learning its name, it has > two problems: > > 1. If it has multiple targets with the name "iscsi", it will not > know whether they are multiple targets, or multiple addresses > for the same target. > > 2. If the target's IP address changes, it will not be able to tell > whether a newly-discovered target is the same target at a new > address, or a new target. > > Given these, I think there is too much danger in allowing a host not > to learn the actual name of a target. Also, Jim is correct about > discovery being a one-time thing; after the first time a host has > connected, it will probably squirrel the iSCSI name away in a config > file or registry somewhere anyway, so it knows where to look for its > disks, and which /dev entries to map them to the next time it boots. > > Jim Hafner wrote: > > > > Mark and Steph, > > > > I guess I'm having trouble understanding the issues here. We > seem to be > > moving toward three types of iSCSI targets: > > 1) one for discovery only, suitable for login only by authenticated > > initiators, i.e., a one-sided authentication, but ONLY for the > purposes of > > SendTargets. > > 2) one "nameless" target ("iSCSI"), again suitable for login only by > > authenticated initiators, i.e., a one-sided authentication, but used for > > real SCSI stuff > > 3) all other "named" targets, that may or may not require two-sided > > authentication, but are used for real SCSI stuff. > > > > Frankly, I think I see no real need for the second one (a > SCSI-functional > > thing with no true name). The third one can perform the > function of the > > second one if the initiator doesn't bother to authenticate the target's > > name (provided it has a straightforward way to get the name). The first > > one was what we were moving to as the functional use of the > "iSCSI" target. > > The only value I see is that the initiator doesn't have to find the name > > first. But the target will need to "correct" that name in > response anyway > > (at which point it has a name which it can use for all other > sessions) -- > > it could/should be a one time deal, probably, to go through the > discovery > > (SendTargets) step. I don't think we really want the host to > think that it > > has any number of real targets all named "iSCSI"! > > > > Jim Hafner > > > > Mark Bakke @ece.cmu.edu on 05-16-2001 12:14:35 PM > > > > Sent by: owner-ips@ece.cmu.edu > > > > To: Stephen Bailey > > cc: ips@ece.cmu.edu > > Subject: Re: iSCSI: Canonical Targets > > > > That kind of makes sense. If we limit SendTargets to the discovery > > session, and disallow SCSI commands to the discovery session, then > > we really don't have a target at all from SCSI's point of view; but > > we do have a "special" target from iSCSI's point of view since we > > are doing the full login to it. > > > > So if we used > > > > TargetName= > > > > we would get the discovery target; if we used > > > > TargetName=iSCSI # Note: this is case-insensitive > > > > we would get whatever default target the iSCSI device wanted > > our initiator to see. Even if we do end up supporting both, I > > guess I'd rather see a target name, rather than leaving it > > blank. How about > > > > TargetName=discovery > > > > BTW, nobody else has spoken up for having this default target > > that would avoid having to use SendTargets if there were not > > multiple targets available to an initiator. > > > > Who plans to make use of this? I don't mind putting it in > > (we had sort of implied the functionality before), but if > > it is not in anyone's plans, I'd rather go for a simpler spec. > > > > Stephen Bailey wrote: > > > > > > > Any other ideas? > > > > > > My suggestion was to not supply any target name for a discovery login > > > (which is how a discovery login would be distinguished) and call > > > `iSCSI' the `default' (operational) target. > > > > > > Steph > > > > -- > > Mark A. Bakke > > Cisco Systems > > mbakke@cisco.com > > 763.398.1054 > > -- > Mark A. Bakke > Cisco Systems > mbakke@cisco.com > 763.398.1054 > From owner-ips@ece.cmu.edu Thu May 17 10:34:04 2001 Received: from ece.cmu.edu (ECE.CMU.EDU [128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id KAA18584 for ; Thu, 17 May 2001 10:34:03 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f4HCW1D22000 for ips-outgoing; Thu, 17 May 2001 08:32:01 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from maho3msx2.isus.emc.com (maho3msx2.isus.emc.com [128.221.11.32]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f4HCVxH21993 for ; Thu, 17 May 2001 08:31:59 -0400 (EDT) Received: by maho3msx2.isus.emc.com with Internet Mail Service (5.5.2653.19) id ; Thu, 17 May 2001 08:31:53 -0400 Message-ID: <0F31E5C394DAD311B60C00E029101A070801558D@corpmx9.isus.emc.com> From: Black_David@emc.com To: matt_wakeley@agilent.com, ips@ece.cmu.edu Subject: RE: TCP Framing Date: Thu, 17 May 2001 08:31:52 -0400 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2653.19) Content-Type: text/plain Sender: owner-ips@ece.cmu.edu Precedence: bulk Matt, The tsvwg charter contains the following milestone: May 01 submit ID on TCP framing by a shim to IESG for consideration as a proposed standard So, the answer to: > Can the iSCSI spec *mandate* the use of this new > TSVWG framing RFC in all iSCSI implementations (especially the ones that will > be complete before this new RFC is even published)? should be "YES" because that new RFC is likely to be published well before the iSCSI RFC and will be standards track. This is an IETF procedural answer -- RFCs only control the contents of implementations that choose to comply to the RFCs, and for the example of NT4, that existing code would not comply. OTOH, one could ask the Microsoft co-author of the tcpulpframe draft about what Microsoft may/might do. Of course any mandate (tcpulpframe or markers) requires rough consensus of the IPS WG. Thanks, --David --------------------------------------------------- David L. Black, Senior Technologist EMC Corporation, 42 South St., Hopkinton, MA 01748 +1 (508) 435-1000 x75140 FAX: +1 (508) 497-8500 black_david@emc.com Mobile: +1 (978) 394-7754 --------------------------------------------------- From owner-ips@ece.cmu.edu Thu May 17 10:35:47 2001 Received: from ece.cmu.edu (ECE.CMU.EDU [128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id KAA18615 for ; Thu, 17 May 2001 10:35:46 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f4HCToL21837 for ips-outgoing; Thu, 17 May 2001 08:29:50 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from dogwood.cisco.com (dogwood.cisco.com [161.44.11.19]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f4HCTmH21833 for ; Thu, 17 May 2001 08:29:48 -0400 (EDT) Received: from cisco.com (mbakke@mbakke-lnx.cisco.com [161.44.68.87]) by dogwood.cisco.com (8.8.6 (PHNE_14041)/CISCO.SERVER.1.2) with ESMTP id IAA26612; Thu, 17 May 2001 08:29:41 -0400 (EDT) Message-ID: <3B03C394.4AA705DE@cisco.com> Date: Thu, 17 May 2001 07:27:00 -0500 From: Mark Bakke X-Mailer: Mozilla 4.72 [en] (X11; U; Linux 2.2.16-3.uid32 i686) X-Accept-Language: en, de MIME-Version: 1.0 To: Thomas McSweeney CC: IPS Subject: Re: iSCSI: Aggregation tags in SendTargets References: Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-ips@ece.cmu.edu Precedence: bulk Content-Transfer-Encoding: 7bit Thanks. I looked through the appendix, but just completely missed it. Anyway, it looks like we don't need to indicated whether multiple connections are supported on a single address in the SendTargets response, since this will be negotiated on the first connection anyway. -- Mark Thomas McSweeney wrote: > > >Of course, if there were some sort of flag or text key in > >the iSCSI login response to indicate the support of multiple- > >connection sessions, we wouldn't need to do this. I don't > >think it's in there, though. > > Sure it is! The first tag in appendix E: > > 08 MaxConnections > > Use: LO > Who: Initiator and Target > > MaxConnections= > > Default is 8. > > Initiator and target negotiate the maximum number of connections > requested/acceptable. The lower of the 2 numbers is selected. > > Tom McSweeney > iSCSI Development, Storage Systems Group, IBM > Email: rf42tpme@us.ibm.com > Phone: (USA) 919-254-5634 (tie line: 444-5634) > Fax: (USA) 919-254-0391 (tie line: 444-0391) -- Mark A. Bakke Cisco Systems mbakke@cisco.com 763.398.1054 From owner-ips@ece.cmu.edu Thu May 17 13:53:25 2001 Received: from ece.cmu.edu (ECE.CMU.EDU [128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id NAA23492 for ; Thu, 17 May 2001 13:53:25 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f4HFb4206562 for ips-outgoing; Thu, 17 May 2001 11:37:04 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from med.corp.rhapsodynetworks.com (64-160-62-201.rhapsodynetworks.com [64.160.62.201] (may be forged)) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f4HFb2H06557 for ; Thu, 17 May 2001 11:37:03 -0400 (EDT) Received: by med.corp.rhapsodynetworks.com with Internet Mail Service (5.5.2653.19) id ; Thu, 17 May 2001 08:36:56 -0700 Message-ID: <45BEF1D68145D51186C100B0D0AABE6503D3FA@med.corp.rhapsodynetworks.com> From: Venkat Rangan To: "'Sudhir Srinivasan'" , "'ips@ece.cmu.edu'" Subject: RE: FCIP: Multiple connections Date: Thu, 17 May 2001 08:36:53 -0700 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2653.19) Content-Type: text/plain; charset="iso-8859-1" Sender: owner-ips@ece.cmu.edu Precedence: bulk Sudhir, For a given FC Flow (indicated by a SID, DID pair of the FC frames), both forward and reverse directions are mapped to the same connection. This ensures that: 1. Any TCP congestion related rate adjustment on one direction impacts the other direction equally. Although even a single TCP connection is subject to variations introduced due to assymmetric routing, we do not want to introduce more assymmetry by sending forward direction frames of one flow on one connection and the reverse on a different connection. 2. By allowing forward and reverse traffic flow on different connections, the protocol may exclude implementation of a class of products that rely on seeing the forward and reverse traffic on the same NIC where the connection is processed. If the connections end up on different NICs or ports and the processing is off-loaded to them, then you have the need to either exchange state or forward traffic to a common point. 3. Error recovery in the case of TCP connection failure may cause one of the two directions to be functional while the other is not. If both directions are mapped to the same connection, such recovery is more easily mapped to a link/ISL failure. If the protocol allows forward and reverse directions of a flow to be mapped on different connections, I would like to suggest that there is an option that allows a pair of FCIP Device Endpoints to negotiate that both directions must be mapped on to the same connection. Implementing this "flow allegiance" is not a significant incremental effort since in order to map all frames of a single direction of a flow on to a particular connection, most of the work is already done anyway. Regards, Venkat Rangan Rhapsody Networks Inc. http://www.rhapsodynetworks.com -----Original Message----- From: Sudhir Srinivasan [mailto:SudhirS@trebia.com] Sent: Monday, May 14, 2001 1:53 PM To: 'ips@ece.cmu.edu' Cc: Sudhir Srinivasan Subject: FCIP: Multiple connections In Section 5.5, what is the reason to include rule (2)? Is it not sufficient that frames in each direction be delivered in order? Physical analogy is to have two half-duplex lines instead of a full-duplex line. Vendor should have the choice to go either way. -Sudhir Trebia Networks, Inc. Sudhir.Srinivasan@trebia.com 978-929-0830 x139 www.trebia.com From owner-ips@ece.cmu.edu Thu May 17 14:53:20 2001 Received: from ece.cmu.edu (ECE.CMU.EDU [128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id OAA24703 for ; Thu, 17 May 2001 14:53:20 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f4HGJlO10125 for ips-outgoing; Thu, 17 May 2001 12:19:47 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from chmls06.mediaone.net (chmls06.mediaone.net [24.147.1.144]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f4HGJjH10120 for ; Thu, 17 May 2001 12:19:45 -0400 (EDT) Received: from cs.uchicago.edu (h00008649aa79.ne.mediaone.net [24.91.82.73]) by chmls06.mediaone.net (8.11.1/8.11.1) with ESMTP id f4HGJh803023 for ; Thu, 17 May 2001 12:19:43 -0400 (EDT) Message-Id: <200105171619.f4HGJh803023@chmls06.mediaone.net> To: ips@ece.cmu.edu Subject: Re: iSCSI: Canonical Targets In-Reply-To: Message from "Jim Hafner" of "Wed, 16 May 2001 12:56:45 PDT." References: Date: Thu, 17 May 2001 12:17:47 -0400 From: Stephen Bailey Sender: owner-ips@ece.cmu.edu Precedence: bulk Jim, Mark, et al., > I guess I'm having trouble understanding the issues here. We seem to be > moving toward three types of iSCSI targets: I disagree. We still only have one type of target, but our choice of terms is makes it look otherwise. > 1) one for discovery only, suitable for login only by authenticated > initiators, i.e., a one-sided authentication, but ONLY for the purposes of > SendTargets. A discovery login does not involve a target at all. A target is a SCSI entity, and, as Mark pointed out, there's no SCSI interaction in a discovery login. If, as Mark, you don't like the empty string to denote no target, we could use: TargetName=none for discovery login. I don't really care how the notion is represented, only that we agree upon the notion itself. > 2) one "nameless" target ("iSCSI"), again suitable for login only by > authenticated initiators, i.e., a one-sided authentication, but used for > real SCSI stuff This target is not nameless. It has a full-fledged iSCSI name, which is returned in the LoginResponse of the login interchange. `iSCSI' is A name (not THE name) for the DEFAULT (as administered at the target) target. The specific target named by iSCSI may vary based upon initiator, or phase of the moon (a target-based administrative choice). Authentication is performed with the initiator against the specific, target-chosen default target. An initiator should only ask for the default target when it wants the associated default behavior. The advantage of the default target is you get an extremely low-overhead way of administering the storage accessed by an initiator which is configured purely at the target, without requiring additional infrastructure. As Doug pointed out, this behavior is useful for initiators with limited resources, or just limited desire (or ability) to select a particular target (e.g. booting servers in an Infiniband-style zillion*3*1U rack farm). I'm having a hard time swallowing the `it makes the specification more complicated' argument. After all, I tried that on multiconnection sessions, where it really DOES make the specification more complicated :^) Seriously, it's not complex to specify, we simply have to actually do it. Also, that somebody besides me and Doug (I'm sure there are many of you out there for whom our mail doesn't even reach your inbox....) hasn't specifically requested this feature isn't really a good argument. The set of iSCSI applications represented presently is somewhat limited. We all know iSCSI has huge potential, but if we make it purely market-targeted based upon the present, (aim for the immediate money), we're going miss future low-hanging fruit. Put another way, we're not expecting anybody to throw away their existing, local storage connection as soon as the ink dries on the iSCSI RFC. However, removing one entire port from systems is a compelling argument for iSCSI in the long run. To that end, the default target is a simple, familiar mechanism which will make that process easier. Why? Because although the rubber (dollars) really meets the road in this case when you have many servers, and a big network (and probably a complicated management infrastructure), a natural consequence will be that people will want to take those same components (off the shelf) and build small configurations, e.g. with a SINGLE server, and a SINGLE target and little or no management infrastructure. Every large configuration begins as a small one. Steph From owner-ips@ece.cmu.edu Thu May 17 16:05:13 2001 Received: from ece.cmu.edu (ECE.CMU.EDU [128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id QAA26142 for ; Thu, 17 May 2001 16:05:12 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f4HHRiB15421 for ips-outgoing; Thu, 17 May 2001 13:27:44 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from e31.bld.us.ibm.com (e31.co.us.ibm.com [32.97.110.129]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f4HHRfH15388 for ; Thu, 17 May 2001 13:27:41 -0400 (EDT) Received: from westrelay02.boulder.ibm.com (westrelay02.boulder.ibm.com [9.99.140.23]) by e31.bld.us.ibm.com (8.9.3/8.9.3) with ESMTP id NAA30928; Thu, 17 May 2001 13:20:03 -0400 Received: from f3n42e (d03nm042h.boulder.ibm.com [9.99.140.42]) by westrelay02.boulder.ibm.com (8.11.1/NCO v4.96.1.0) with ESMTP id f4HHRW9148260; Thu, 17 May 2001 11:27:32 -0600 Importance: Normal Subject: Re: iSCSI: Canonical Targets To: Stephen Bailey Cc: ips@ece.cmu.edu From: "Jim Hafner" Date: Thu, 17 May 2001 10:27:29 -0700 Message-ID: X-MIMETrack: Serialize by Router on D03NM042/03/M/IBM(Release 5.0.6 |December 14, 2000) at 05/17/2001 10:27:31 AM MIME-Version: 1.0 Content-type: text/plain; charset=us-ascii Sender: owner-ips@ece.cmu.edu Precedence: bulk Steph, I'll need more time to absorb more of what you're saying, but I want to make one wording clarification which you alluded to in your first paragraph. It's the misunderstood and overused term "target". There are three interpretations of that term running around in this space: 1) "SCSI target port": this is the more common interpretation of "target", at least in the SCSI world (the multiport model for SAM-2 was titled "Defining targets/initiators as ports" so you see where the legacy comes from). 2) "SCSI target device": this one is only now more formally being added to the SAM-2 model; essentially it is a container of SCSI target ports and SCSI logical units (and a few other things) 3) "iSCSI target": this is the thing that gets the iSCSI name (at least as we've been looking at it) and is a purely iSCSI construct. [A similar distinction is required on the term "initiator".] I personally always attempt to be completely clear when I use the terms. So, if you read my note carefully, I use the term "iSCSI target" exclusively to mean thing 3. Can we all agree to use these terms carefully, to avoid the confusion misinterpretation of intent can cause? Or should we come up with different terms? Here's short outline of how we've modeled things (and how I interpret the use of these terms): -- Each iSCSI target gets an iSCSI name. -- Each iSCSI target MAY contain at most one SCSI target device. -- An iSCSI target that does not contain a SCSI target device is there only for iSCSI purposes (such as SendTargets). -- Each iSCSI target must require it's name be supplied during login. This was so that iSCSI initiators (those things that initiate iSCSI login but may or may not have a SCSI initiator device within them) will always do the same login process, that is, include the iSCSI name. [In other words, the name is mandatory for every login.] -- For an iSCSI target that only performs the SendTargets (has no SCSI target device), we allowed that to have the generic name "iSCSI". -- For an iSCSI target that has a SCSI target device, we wanted such things to have unique names for two reasons: (a) so that there would be something the initiator can authenticate and configure against (b) so that the SCSI target device could have a name as well (it would inherit the name from it's parent entity, the iSCSI target). We could easily remove the name from the SCSI-deviceless iSCSI target, but then the iSCSI initiator has to make the distinction of when to include the tag and when not to. It's a nit. We could make the name="none". It's a placeholder, more than anything else. On the other hand, there may be a need latter for other iSCSI targets that don't contain SCSI targets. These might be administrative entities in the box. So other "well-known names" could be defined as well. As I said above, I need some time to digest the rest of your note, but in the meantime, I am not confortable with an iSCSI target that contains a SCSI target device not having a unique name (if for no other reason to meet (b) above). If it has a unique name and because it is available to the iSCSI initiator, there is no reason that initiator can't use it. Jim Hafner Stephen Bailey @ece.cmu.edu on 05-17-2001 09:17:47 AM Sent by: owner-ips@ece.cmu.edu To: cc: Subject: Re: iSCSI: Canonical Targets Jim, Mark, et al., > I guess I'm having trouble understanding the issues here. We seem to be > moving toward three types of iSCSI targets: I disagree. We still only have one type of target, but our choice of terms is makes it look otherwise. > 1) one for discovery only, suitable for login only by authenticated > initiators, i.e., a one-sided authentication, but ONLY for the purposes of > SendTargets. A discovery login does not involve a target at all. A target is a SCSI entity, and, as Mark pointed out, there's no SCSI interaction in a discovery login. If, as Mark, you don't like the empty string to denote no target, we could use: TargetName=none for discovery login. I don't really care how the notion is represented, only that we agree upon the notion itself. > 2) one "nameless" target ("iSCSI"), again suitable for login only by > authenticated initiators, i.e., a one-sided authentication, but used for > real SCSI stuff This target is not nameless. It has a full-fledged iSCSI name, which is returned in the LoginResponse of the login interchange. `iSCSI' is A name (not THE name) for the DEFAULT (as administered at the target) target. The specific target named by iSCSI may vary based upon initiator, or phase of the moon (a target-based administrative choice). Authentication is performed with the initiator against the specific, target-chosen default target. An initiator should only ask for the default target when it wants the associated default behavior. The advantage of the default target is you get an extremely low-overhead way of administering the storage accessed by an initiator which is configured purely at the target, without requiring additional infrastructure. As Doug pointed out, this behavior is useful for initiators with limited resources, or just limited desire (or ability) to select a particular target (e.g. booting servers in an Infiniband-style zillion*3*1U rack farm). I'm having a hard time swallowing the `it makes the specification more complicated' argument. After all, I tried that on multiconnection sessions, where it really DOES make the specification more complicated :^) Seriously, it's not complex to specify, we simply have to actually do it. Also, that somebody besides me and Doug (I'm sure there are many of you out there for whom our mail doesn't even reach your inbox....) hasn't specifically requested this feature isn't really a good argument. The set of iSCSI applications represented presently is somewhat limited. We all know iSCSI has huge potential, but if we make it purely market-targeted based upon the present, (aim for the immediate money), we're going miss future low-hanging fruit. Put another way, we're not expecting anybody to throw away their existing, local storage connection as soon as the ink dries on the iSCSI RFC. However, removing one entire port from systems is a compelling argument for iSCSI in the long run. To that end, the default target is a simple, familiar mechanism which will make that process easier. Why? Because although the rubber (dollars) really meets the road in this case when you have many servers, and a big network (and probably a complicated management infrastructure), a natural consequence will be that people will want to take those same components (off the shelf) and build small configurations, e.g. with a SINGLE server, and a SINGLE target and little or no management infrastructure. Every large configuration begins as a small one. Steph From owner-ips@ece.cmu.edu Thu May 17 17:51:58 2001 Received: from ece.cmu.edu (ECE.CMU.EDU [128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id RAA27686 for ; Thu, 17 May 2001 17:51:57 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f4HJhHH26748 for ips-outgoing; Thu, 17 May 2001 15:43:17 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from lucy.trebia.com ([65.192.191.151]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f4HJhFH26744 for ; Thu, 17 May 2001 15:43:16 -0400 (EDT) Received: by lucy.TREBIA.COM with Internet Mail Service (5.5.2653.19) id ; Thu, 17 May 2001 15:47:11 -0400 Message-ID: <63616B261CEFD411834D00D0B7A86CF7048FB2@lucy.TREBIA.COM> From: Sudhir Srinivasan To: "'Venkat Rangan'" , Sudhir Srinivasan , "'ips@ece.cmu.edu'" Subject: RE: FCIP: Multiple connections Date: Thu, 17 May 2001 15:47:10 -0400 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2653.19) Content-Type: text/plain; charset="iso-8859-1" Sender: owner-ips@ece.cmu.edu Precedence: bulk > Implementing this "flow allegiance" is > not a significant incremental effort since in order to map all frames of a > single direction of a flow on to a particular connection, most of the > work is already done anyway. I disagree - mapping in one direction only is much easier than mapping consistently in both directions across multiple vendors' solutions -- both ends have to use the same mapping algorithm or have to remember a LOT of stuff to do this in an interoperable manner. Now let's look at the issues you raise. 1. Asymmetric routing: Routers won't distinguish between two FCIP connections; so I argue that using a different connection does not add routing asymmetry. 2. NIC application - Such devices should be built to ensure that the connections don't span IP addresses (NICs) and the processing should be done above the TCP port level so that using a different TCP port wouldn't matter. 3. Connection failure - even with a single connection, the issue you mention is already there to some extent. One side may terminate the connection and it will take a while for the other side to realize it. Also, ULP mechanisms should kick in to contain the damage. So, in summary, I don't believe these issues are sufficient to justify the extra burden that the rule places on devices or the potential risk of interoperability problems. On the issue of "negotiating", FCIP does not have any notion of parameter negotiation today, correct? -----Original Message----- From: Venkat Rangan [mailto:venkat@rhapsodynetworks.com] Sent: Thursday, May 17, 2001 11:37 AM To: 'Sudhir Srinivasan'; 'ips@ece.cmu.edu' Subject: RE: FCIP: Multiple connections Sudhir, For a given FC Flow (indicated by a SID, DID pair of the FC frames), both forward and reverse directions are mapped to the same connection. This ensures that: 1. Any TCP congestion related rate adjustment on one direction impacts the other direction equally. Although even a single TCP connection is subject to variations introduced due to assymmetric routing, we do not want to introduce more assymmetry by sending forward direction frames of one flow on one connection and the reverse on a different connection. 2. By allowing forward and reverse traffic flow on different connections, the protocol may exclude implementation of a class of products that rely on seeing the forward and reverse traffic on the same NIC where the connection is processed. If the connections end up on different NICs or ports and the processing is off-loaded to them, then you have the need to either exchange state or forward traffic to a common point. 3. Error recovery in the case of TCP connection failure may cause one of the two directions to be functional while the other is not. If both directions are mapped to the same connection, such recovery is more easily mapped to a link/ISL failure. If the protocol allows forward and reverse directions of a flow to be mapped on different connections, I would like to suggest that there is an option that allows a pair of FCIP Device Endpoints to negotiate that both directions must be mapped on to the same connection. Implementing this "flow allegiance" is not a significant incremental effort since in order to map all frames of a single direction of a flow on to a particular connection, most of the work is already done anyway. Regards, Venkat Rangan Rhapsody Networks Inc. http://www.rhapsodynetworks.com -----Original Message----- From: Sudhir Srinivasan [mailto:SudhirS@trebia.com] Sent: Monday, May 14, 2001 1:53 PM To: 'ips@ece.cmu.edu' Cc: Sudhir Srinivasan Subject: FCIP: Multiple connections In Section 5.5, what is the reason to include rule (2)? Is it not sufficient that frames in each direction be delivered in order? Physical analogy is to have two half-duplex lines instead of a full-duplex line. Vendor should have the choice to go either way. -Sudhir Trebia Networks, Inc. Sudhir.Srinivasan@trebia.com 978-929-0830 x139 www.trebia.com From owner-ips@ece.cmu.edu Thu May 17 18:49:04 2001 Received: from ece.cmu.edu (ECE.CMU.EDU [128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id SAA28177 for ; Thu, 17 May 2001 18:49:03 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f4HKrOl02381 for ips-outgoing; Thu, 17 May 2001 16:53:24 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from palrel2.hp.com (palrel2.hp.com [156.153.255.234]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f4HKrJH02372 for ; Thu, 17 May 2001 16:53:23 -0400 (EDT) Received: from amrelay2.boi.hp.com (amrelay2.boi.hp.com [15.56.8.41]) by palrel2.hp.com (Postfix) with ESMTP id AE2B43E4E; Thu, 17 May 2001 13:53:18 -0700 (PDT) Received: from xatlbh2.atl.hp.com (xatlbh2.atl.hp.com [15.45.89.187]) by amrelay2.boi.hp.com (8.9.3 (PHNE_18979)/8.9.3 SMKit7.02) with ESMTP id OAA03093; Thu, 17 May 2001 14:53:07 -0600 (MDT) Received: by xatlbh2.atl.hp.com with Internet Mail Service (5.5.2653.19) id ; Thu, 17 May 2001 16:53:06 -0400 Message-ID: <6BD67FFB937FD411A04F00D0B74FE87802A0906B@xrose06.rose.hp.com> From: "KRUEGER,MARJORIE (HP-Roseville,ex1)" To: "'Matt Wakeley'" , ips@ece.cmu.edu Subject: RE: TCP Framing Date: Thu, 17 May 2001 16:52:59 -0400 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2653.19) Content-Type: text/plain; charset="iso-8859-1" Sender: owner-ips@ece.cmu.edu Precedence: bulk > As I expressed in my earlier posting, the goal and purpose of > framing is to: > > 1- eliminate the huge amounts of buffering (high speed, expensive, eats up > board space, more expensive than fibre channel) that will be required by 10Gig > implementations, and > 2- provide a common feature set so that any (10Gig) implementation will be > able to interoperate with any other iSCSI implementation, be it 1Gig, 10Gig, > HW or SW implementations. > > The only way to do this is to *mandate* implementation of some kind of framing > mechanism *IN THE iSCSI SPEC*. Isn't it actually that a 10Gig implementation couldn't interoperate *at 10Gig speeds* without some sort of frame-sync method - they would be considerably "slowed-down" (or forced to have large TCP rcv buffers). This doesn't mean 'no interoperation', just no 10 Gig thruput? The reason people are working so hard on a TCP-level framing solution is: - marker scheme is so krufty and software soln's are penalized (they probably wouldn't be able to drive 10Gig speeds if 'markers' are mandated) - there are other TCP applications that benefit from a common frame-sync method. This makes these changes more likely to be incorporated into software stacks (OS releases). There's already grumbling in IETF that 'that IPS team' "seems to exhibit a strong tendency to avoid reusing existing IETF standards that reasonably fit their needs". It would benefit us to support the common TCP framing solution and contribute to it's early adoption. Even if the TCP framing RFC wasn't mandated by iSCSI, a vendor could require it's implementation end-to-end to guarantee 10Gig speeds? Marj From owner-ips@ece.cmu.edu Thu May 17 19:38:30 2001 Received: from ece.cmu.edu (ECE.CMU.EDU [128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id TAA28751 for ; Thu, 17 May 2001 19:38:30 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f4HLliG06495 for ips-outgoing; Thu, 17 May 2001 17:47:44 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from spdmraac.compuserve.com (ds-img-rel-3.compuserve.com [149.174.206.154]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f4HLlgH06486 for ; Thu, 17 May 2001 17:47:42 -0400 (EDT) Received: (from mailgate@localhost) by spdmraac.compuserve.com (8.9.3/8.9.3/SUN-REL-1.3) id RAA18851 for ips@ece.cmu.edu; Thu, 17 May 2001 17:47:40 -0400 (EDT) Received: from compuserve.com (dal-tgn-tks-vty29.as.wcom.net [216.192.230.29]) by spdmraac.compuserve.com (8.9.3/8.9.3/SUN-REL-1.3) with ESMTP id RAA18833 for ; Thu, 17 May 2001 17:47:30 -0400 (EDT) Message-ID: <3B0447B5.96F854B1@compuserve.com> Date: Thu, 17 May 2001 16:50:46 -0500 From: Ralph Weber Reply-To: ENDL_TX@computer.org X-Mailer: Mozilla 4.7 [en]C-CCK-MCD NSCPCD47 (Win95; I) X-Accept-Language: en,pdf MIME-Version: 1.0 To: IPS Reflector Subject: FCIP - Proposed changes in support of common FC encapsulation Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-ips@ece.cmu.edu Precedence: bulk Content-Transfer-Encoding: 7bit I believe the following changes need to be made in FCIP to make correct reference to the Common FC Encapsulation draft. These changes have been reviewed by Milan Merhar, Larry Lamers, and Bob Snively all of whom were also at the Nashua meeting where the basic data format issues were agreed. The changes are presented in the order that they are expected to appear in draft-ietf-ips-fcovertcpip-03.txt. In the section 3 objectives list (top of page 3 in draft-ietf-ips- fcovertcpip-02.txt), replace: 1) specify the encapsulation and mapping of FC frames using the FC encapsulation method specified in . with: 1) specify the encapsulation and mapping of FC frames employing the common FC Frame Encapsulation[23]. In the 1st paragraph of section 4.2, replace: The FCIP protocol specifies the TCP/IP encapsulation, mapping and routing of FC frames and applies these mechanisms to an FC network utilizing IP for its backbone, or more generally, between any two FC devices. with: The FCIP protocol specifies the TCP/IP encapsulation (employing the common FC Frame Encapsulation described in [23]), mapping and routing of FC frames and applies these mechanisms to an FC network utilizing IP for its backbone, or more generally, between any two FC devices. In section 4.2 list item 6, replace: 6. An FCIP device MAY send FC encapsulated TCP/IP packets to more than one FCIP device. However, these encapsulated packets are treated as separate instances and are not correlated in any way by the FCIP protocol devices. with: 6. An FCIP device MAY send encapsulated FC frames to more than one FCIP device. However, these encapsulated FC frames are treated as separate instances and are not correlated in any way by the FCIP protocol devices Delete list item 11 in section 4.2 since it is covered in the intro- ductory text at the beginning of the section. For reference, list item 11 currently reads: 11. FCIP uses the common encapsulation method as specified by . In the 3rd paragraph of section 4.3, replace: Each FCIP data frame is built by adding an FCIP header to one FC frame delivered to the FCIP endpoint for transport. The FCIP data frames are handed in their entirety to TCP; TCP is responsible for delivering the same series of FCIP data frames to the receiving side in the same order as they are transmitted by the sending FCIP device. The FCIP device MUST find the FCIP headers and deliver the FC frames wrapped inside the FCIP data frames to the correct FC ports connected to the FCIP device. with: An FCIP data frame is built by encapsulating one FC frame delivered to the FCIP endpoint for transport using the encapsulation format described in the common FC Frame Encapsulation [23] and in section 5.2 of this document. The FCIP data frames are handed in their entirety to TCP; TCP is responsible for delivering the same series of FCIP data frames to the receiving side in the same order as they are transmitted by the sending FCIP device. The FCIP receiving device MUST reverse the FC frame encapsulation process, verify the correctness of the header discarding frames with header errors, and deliver the FC frames to the correct FC ports connected to the FCIP device. In section 5.1, the last paragraph of "SOF and EOF Delimiters:", replace: When an FC frame is encapsulated and sent over a byte-oriented interface, the SOF and EOF delimiters are represented as sequences of four consecutive bytes, which carry the equivalent Class of Service and frame termination information as the FC ordered sets. This form of encoding can not provide unambiguous identification of frame beginning and end, however, and must rely on other mechanisms provided by the encapsulation protocol. With: When an FC frame is encapsulated and sent over a byte-oriented interface, the SOF and EOF delimiters are represented as sequences of four consecutive bytes, which carry the equivalent Class of Service and frame termination information as the FC ordered sets. The representation of SOF and EOF in an encapsulation FC frame is described in the common FC Frame Encapsulation [23]. Note: If the deleted sentence is considered important, then it should be moved to section 3.3 of the common encapsulation draft. Section 5.2 appears to have been omitted as a placeholder for the FCIP encapsulation details. Therefore, the following new text for section 5.2 is proposed. 5.2 FCIP Encapsulation of FC Frames The FCIP encapsulation of FC frames employs the common FC Frame Encapsulation [23]. The features from the common FC Frame Encapsulation that are unique to individual protocols SHALL be applied as follows for the FCIP encapsulation of FC frames. The Protocol# field SHALL contain 1 in accordance with the IANA Considerations annex of the common FC Frame Encapsulation [23]. The Protocol Specific field SHALL have the format shown in figure xx. Note: the word numbers in figure xx are relative to the complete FC frame encapsulation header, not to the Protocol Specific field. W|------------------------------Bit------------------------------| o| | r|3 3 2 2 2 2 2 2 2 2 2 2 1 1 1 1 1 1 1 1 1 1 | d|1 0 9 8 7 6 5 4 3 2 1 0 9 8 7 6 5 4 3 2 1 0 9 8 7 6 5 4 3 2 1 0| +---------------------------------------------------------------+ 1| replication of encapsulation word 0 | +-------------------------------+-------------------------------+ 2| reserved | -reserved | +-------------------------------+-------------------------------+ Fig. xx - FCIP Usage of common FC Frame Encapsulation Protocol Specific field Word 1 of the Protocol Specific field SHALL contain an exact copy of word 0 in the common FC Frame Encapsulation [23]. Word 2 of the Protocol Specific field is reserved for future enhancements to the FCIP protocol. The reserved field (bits 31-16 in word 2): SHALL contain 0. The -reserved field (bits 15-0 in word 2): SHALL contain 65535 (or 0xFFFF). The CRCV (CRC Valid) Flag SHALL be set to 0. The CRC field SHALL be set to 0. In section 5.3 list item 4, replace: After connection establishment, FCIP devices use the FCIP frame encapsulation as defined in [common encapsulation document]. with: After connection establishment, FCIP devices use the FCIP frame encapsulation defined in the common FC Frame Encapsulation [23] and in section 5.2 of this document. In the 1st paragraph of 5.4.1, replace: These FCIP ELS messages use the same encapsulation mechanism as described in TBD. with: These FCIP ELS messages use the same encapsulation mechanism as described in 5.2. In the 3rd paragraph of 5.4.2, replace: FCIP Encapsulated frame with FCIP frame In the 1st paragraph of 8.5, replace: FCIP data frame with FCIP frame In Section 11 (References) add the following: [23] Weber, Rajagopal, Travostino, Chau, McDonnell, Monia Merhar, "FC Frame Encapsulation", draft-ietf-ips-fcencapsulation-__.txt (RFC reference and date to be added during standards action). RFC1072, October 1988 Thanks. Ralph... From owner-ips@ece.cmu.edu Thu May 17 21:01:00 2001 Received: from ece.cmu.edu (ECE.CMU.EDU [128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id VAA29595 for ; Thu, 17 May 2001 21:00:59 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f4HN0qa11536 for ips-outgoing; Thu, 17 May 2001 19:00:52 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from pluto.he.net (pluto.he.net [216.218.167.2]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f4HN0oH11531 for ; Thu, 17 May 2001 19:00:50 -0400 (EDT) Received: from DSLmodem (63-229-221-92.customers.uswest.net [63.229.221.92]) by pluto.he.net (8.8.6/8.8.2) with SMTP id QAA32162; Thu, 17 May 2001 16:00:42 -0700 From: "Sriram Rupanagunta" To: "Sudhir Srinivasan" , "'Venkat Rangan'" , Subject: RE: FCIP: Multiple connections Date: Thu, 17 May 2001 18:06:44 -0500 Message-ID: MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 (Normal) X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook IMO, Build 9.0.2416 (9.0.2910.0) Importance: Normal X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4133.2400 In-Reply-To: <63616B261CEFD411834D00D0B7A86CF7048FB2@lucy.TREBIA.COM> Sender: owner-ips@ece.cmu.edu Precedence: bulk Content-Transfer-Encoding: 7bit I agree with Sudhir on this. It is harder to make sure that both sides adhere to same connection. - Sriram > > > Implementing this "flow allegiance" is > > not a significant incremental effort since in order to map all > frames of a > > single direction of a flow on to a particular connection, most of the > > work is already done anyway. > > I disagree - mapping in one direction only is much > easier than mapping consistently in both directions > across multiple vendors' solutions -- both ends have > to use the same mapping algorithm or have to remember > a LOT of stuff to do this in an interoperable manner. > > Now let's look at the issues you raise. > 1. Asymmetric routing: Routers won't distinguish between > two FCIP connections; so I argue that using a different > connection does not add routing asymmetry. > 2. NIC application - Such devices should be built to ensure > that the connections don't span IP addresses (NICs) and > the processing should be done above the TCP port level > so that using a different TCP port wouldn't matter. > 3. Connection failure - even with a single connection, > the issue you mention is already there to some extent. > One side may terminate the connection and it will take > a while for the other side to realize it. Also, ULP > mechanisms should kick in to contain the damage. > > So, in summary, I don't believe these issues are > sufficient to justify the extra burden that the rule > places on devices or the potential risk of interoperability > problems. > > On the issue of "negotiating", FCIP does not have any notion > of parameter negotiation today, correct? > > -----Original Message----- > From: Venkat Rangan [mailto:venkat@rhapsodynetworks.com] > Sent: Thursday, May 17, 2001 11:37 AM > To: 'Sudhir Srinivasan'; 'ips@ece.cmu.edu' > Subject: RE: FCIP: Multiple connections > > > Sudhir, > > For a given FC Flow (indicated by a SID, DID pair of the FC frames), > both forward and reverse directions are mapped to the same connection. > This ensures that: > > 1. Any TCP congestion related rate adjustment on one direction impacts the > other direction equally. Although even a single TCP connection > is subject > to > variations introduced due to assymmetric routing, we do not want to > introduce > more assymmetry by sending forward direction frames of one flow on one > connection and the reverse on a different connection. > > 2. By allowing forward and reverse traffic flow on different connections, > the protocol may exclude implementation of a class of products that > rely on seeing the forward and reverse traffic on the same NIC where > the connection is processed. If the connections end up on different > NICs or ports and the processing is off-loaded to them, then you have > the need to either exchange state or forward traffic to a common point. > > 3. Error recovery in the case of TCP connection failure may cause > one of the > two directions to be functional while the other is not. If both > directions > are mapped to the same connection, such recovery is more > easily mapped to > a link/ISL failure. > > If the protocol allows forward and reverse directions of a flow > to be mapped > on > different connections, I would like to suggest that there is an > option that > allows a pair of FCIP Device Endpoints to negotiate that both directions > must be > mapped on to the same connection. Implementing this "flow allegiance" is > not a significant incremental effort since in order to map all frames of a > single direction of a flow on to a particular connection, most of the > work is already done anyway. > > Regards, > > Venkat Rangan > Rhapsody Networks Inc. > http://www.rhapsodynetworks.com > > > > -----Original Message----- > From: Sudhir Srinivasan [mailto:SudhirS@trebia.com] > Sent: Monday, May 14, 2001 1:53 PM > To: 'ips@ece.cmu.edu' > Cc: Sudhir Srinivasan > Subject: FCIP: Multiple connections > > > > In Section 5.5, what is the reason to include rule (2)? > Is it not sufficient that frames in each direction be delivered > in order? Physical analogy is to have two half-duplex lines > instead of a full-duplex line. Vendor should have the choice > to go either way. > > -Sudhir > > Trebia Networks, Inc. Sudhir.Srinivasan@trebia.com > 978-929-0830 x139 www.trebia.com > From owner-ips@ece.cmu.edu Thu May 17 23:19:23 2001 Received: from ece.cmu.edu (ECE.CMU.EDU [128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id XAA03915 for ; Thu, 17 May 2001 23:19:22 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f4I14eH19321 for ips-outgoing; Thu, 17 May 2001 21:04:40 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from palrel2.hp.com (palrel2.hp.com [156.153.255.234]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f4I14dH19317 for ; Thu, 17 May 2001 21:04:39 -0400 (EDT) Received: from hpcuhe.cup.hp.com (hpcuhe.cup.hp.com [15.0.80.203]) by palrel2.hp.com (Postfix) with ESMTP id 597B03B61 for ; Thu, 17 May 2001 18:04:38 -0700 (PDT) Received: from cup.hp.com (santoshr@hpindhhm.cup.hp.com [15.8.80.197]) by hpcuhe.cup.hp.com (8.9.3 (PHNE_18979)/8.9.3 SMKit7.02) with ESMTP id SAA07357 for ; Thu, 17 May 2001 18:04:33 -0700 (PDT) Message-ID: <3B047533.7CB74A0E@cup.hp.com> Date: Thu, 17 May 2001 18:04:51 -0700 From: Santosh Rao Organization: Hewlett Packard, Cupertino. X-Mailer: Mozilla 4.7 [en] (X11; U; HP-UX B.11.00 9000/778) X-Accept-Language: en MIME-Version: 1.0 To: ips@ece.cmu.edu Subject: Re: iSCSI : Review Feedback on iSCSI Rev 06 References: <200105040201.TAA00579@hpcuhe.cup.hp.com> <20010507130209.0907F94006@sandmail.sandburst.com> <3AF73ED0.E6CB535F@cup.hp.com> <200105140255.f4E2tkx23004@chmls05.mediaone.net> Content-Type: multipart/mixed; boundary="------------80F8EA2A26CBAAE87731E369" Sender: owner-ips@ece.cmu.edu Precedence: bulk This is a multi-part message in MIME format. --------------80F8EA2A26CBAAE87731E369 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Stephen Bailey wrote: > > Santosh, > > > What I was pointing out was that this upper limit needs to be lower than > > DataPDULength, failing which fragmentation can occur. > > I see what you're saying now, but I don't agree that the iSCSI spec > requires this fragmentation. Data PDUs are not NOP, Text or > Login[Response] PDUs. The DataPDULength limit applies only to data > PDU length (or, we should change the name to PDULength). Steph, I've asked this question a long time back on whether DataPDULength controls the length of all Data PDUs or only the Data PDUs and have'nt heard a definitive answer on this yet. If it controls the length of all Data PDUs, it must be re-named to MaxPDULength. If it does not control the length of all PDUs, then, a seperate Max PDU Length must be negotiated for all the non-scsi PDUs. (text, login, nop). > The most important point to make is that both the initiator and the > target be able to control the total size of any PDU sequence received > (therefore, sent) during an operational session. Agreed. > > Do people actually use GID_FT? It doesn't seem wise. I always used > GA_NXT, and it seemed that all the other FCP implementors I asked did > as well (it's been a while). Well, you've just met your first GID_FT implementor in that case. It saves us a lot of on-the-wire traffic by replacing 'n' GA_NXTs with a single GID_FT as long as we can handle the buffer allocation [which is ok with us]. In any case, that's a separate discussion... :} Regards, Santosh --------------80F8EA2A26CBAAE87731E369 Content-Type: text/x-vcard; charset=us-ascii; name="santoshr.vcf" Content-Description: Card for Santosh Rao Content-Disposition: attachment; filename="santoshr.vcf" Content-Transfer-Encoding: 7bit begin:vcard n:Rao;Santosh tel;work:408-447-3751 x-mozilla-html:FALSE org:Hewlett Packard, Cupertino.;SISL adr:;;19420, Homestead Road, M\S 43LN, ;Cupertino.;CA.;95014.;USA. version:2.1 email;internet:santoshr@cup.hp.com title:Software Design Engineer x-mozilla-cpt:;21088 fn:Santosh Rao end:vcard --------------80F8EA2A26CBAAE87731E369-- From owner-ips@ece.cmu.edu Thu May 17 23:22:07 2001 Received: from ece.cmu.edu (ECE.CMU.EDU [128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id XAA03967 for ; Thu, 17 May 2001 23:22:06 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f4I1BwH19833 for ips-outgoing; Thu, 17 May 2001 21:11:58 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from palrel1.hp.com (palrel1.hp.com [156.153.255.242]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f4I1BuH19829 for ; Thu, 17 May 2001 21:11:57 -0400 (EDT) Received: from hpcuhe.cup.hp.com (hpcuhe.cup.hp.com [15.0.80.203]) by palrel1.hp.com (Postfix) with ESMTP id 372E82C63 for ; Thu, 17 May 2001 18:12:10 -0700 (PDT) Received: from cup.hp.com (santoshr@hpindhhm.cup.hp.com [15.8.80.197]) by hpcuhe.cup.hp.com (8.9.3 (PHNE_18979)/8.9.3 SMKit7.02) with ESMTP id SAA07776 for ; Thu, 17 May 2001 18:11:51 -0700 (PDT) Message-ID: <3B0476E9.F3E8BB1E@cup.hp.com> Date: Thu, 17 May 2001 18:12:09 -0700 From: Santosh Rao Organization: Hewlett Packard, Cupertino. X-Mailer: Mozilla 4.7 [en] (X11; U; HP-UX B.11.00 9000/778) X-Accept-Language: en MIME-Version: 1.0 To: ips@ece.cmu.edu Subject: Re: iSCSI : Bridging missing CmdSNs and Abort I/O Error recovery References: <3B0006EC.D304C098@cup.hp.com> Content-Type: multipart/mixed; boundary="------------C787F9DC9B9C998378B6C9D6" Sender: owner-ips@ece.cmu.edu Precedence: bulk This is a multi-part message in MIME format. --------------C787F9DC9B9C998378B6C9D6 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit All, I did'nt hear back any response on this and am raising the issue again. An Abort Task MUST be sent on the same connection as the command unless that connection was logged out due to connection recovery. If the connection was logged out for closing the connection (1), closing the session (0), or removes the connection at target's request (3), an Abort Task is meaningless since all pending tasks are cleaned up at the target by this action. In all other cases, an Abort Task cannot be used to perform a clean-up of the command and ensure deterministic release of resources, since the Abort Task response can arrive ahead of other in-flight PDUs from target to initiator on another connection. Comments ? - Santosh ps : While on the subject of connection allegiance as defined in Section 1.2.5, this definition must be extended to specify that the response for ALL iSCSI command PDUs MUST be sent on the same connection as the command. Santosh Rao wrote: > > Julian, > > I have re-read your previous answer and I may be missing something, but, > I don't see how it addresses my concern (?). > > Per your mail : > "To abort safely a task for which the task abort answer is "Command Not > Received Yet" the initiator must issue another abort command on the same > connection as the original command unless this connection was logged out > in which case it may send it on any connection." > > I'm pointing out that the above may not be sufficient since the issue > spans beyond that. If the target HAD received the command by the time it > received the task mgmt command on another connection, it would not > respond with "Command not received yet". In this case, the initiator > would not be required to re-send the Abort Task on the same connection > as the original command. > > In such a scenario, if there were PDUs in-flight from target to > initiator on the original connection, these could arrive after the task > mgmt response [which is on the 2nd connection]. Such an Abort Task > cleanup does not reliably allow the initiator to free its task tag and > other resources, since PDUs for that task continue to arrive on the > original connection after the initiator has completed task clean-up > using Abort Task & released the task resources. > > - Santosh > > julian_satran@il.ibm.com wrote: > > > > please reread carefully my previous answer. Julo > > > > Santosh Rao on 09-05-2001 19:37:41 > > > > Please respond to Santosh Rao > > > > To: IPS Reflector > > cc: > > Subject: Re: iSCSI : Bridging missing CmdSNs and Abort I/O Error recovery > > > > > julian_satran@il.ibm.com wrote: > > > > > > > We could add the RefCmdSN and it may help plug-in the hole but unless > > the > > > > > Abort is ussued on the same connection as the original command we > > can't > > > > be > > > > > sure that the old-one will not pop-up (as we enable ExpCmdSN to move > > on > > > > we > > > > > don't have even the 2**31-1 protection bracket :-)). Thus sending in > > > > > another nop/abort on the same connection is still required. > > > > > > > > > > To simplify the whole process I will: > > > > > > > > > > a - add RefCmdSN to the Task Management > > > > > b - add a command not received yet to the answers > > > > > > > > > > c - add a part to 7 reading: > > > > > > > > > > 1.1 How to Abort Safely a Command that Was Not Received > > > > > > > > > > To abort safely a task for which the task abort answer is "Command > > Not > > > > > Received Yet" the initiator must issue another abort command on > > the > > > > same > > > > > connection as the original command unless this connection was > > logged > > > > out > > > > > in which case it may send it on any connection. > > > > Julian, > > > > The above may not be sufficient, since a requirement for the Abort Task > > completion is that both the initiator & target can safely re-use their > > task tags and other resources following completion of the Abort Task. If > > the Abort Task is sent on another connection than the connection used > > for the original command [& that connection has not been logged out], > > then, there may be some in-flight PDUs from the target to the initiator > > for that command on the connection. > > > > Sending an Abort Task on another connection may cause the initiator to > > receive the Abort Task response indicating success on the other > > connection which then results in initiator freeing up its task tag and > > other resources used for that command. Thereafter, the stale PDUs on the > > original connection can show up at the initiator, causing problems. > > > > I would suggest that Abort Task be always sent on the same connection as > > the command, unless the connection used for that command has been logged > > out. Not doing so implies the initiator cannot safely re-use its task > > tag and other resources that were tied up for that I/O. > > > > Regards, > > Santosh > > - santoshr.vcf --------------C787F9DC9B9C998378B6C9D6 Content-Type: text/x-vcard; charset=us-ascii; name="santoshr.vcf" Content-Description: Card for Santosh Rao Content-Disposition: attachment; filename="santoshr.vcf" Content-Transfer-Encoding: 7bit begin:vcard n:Rao;Santosh tel;work:408-447-3751 x-mozilla-html:FALSE org:Hewlett Packard, Cupertino.;SISL adr:;;19420, Homestead Road, M\S 43LN, ;Cupertino.;CA.;95014.;USA. version:2.1 email;internet:santoshr@cup.hp.com title:Software Design Engineer x-mozilla-cpt:;21088 fn:Santosh Rao end:vcard --------------C787F9DC9B9C998378B6C9D6-- From owner-ips@ece.cmu.edu Thu May 17 23:22:26 2001 Received: from ece.cmu.edu (ECE.CMU.EDU [128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id XAA03981 for ; Thu, 17 May 2001 23:22:25 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f4I1FhQ20091 for ips-outgoing; Thu, 17 May 2001 21:15:43 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from palrel2.hp.com (palrel2.hp.com [156.153.255.234]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f4I1FfH20086 for ; Thu, 17 May 2001 21:15:42 -0400 (EDT) Received: from hpcuhe.cup.hp.com (hpcuhe.cup.hp.com [15.0.80.203]) by palrel2.hp.com (Postfix) with ESMTP id 83E4C26CD; Thu, 17 May 2001 18:15:27 -0700 (PDT) Received: from cup.hp.com (santoshr@hpindhhm.cup.hp.com [15.8.80.197]) by hpcuhe.cup.hp.com (8.9.3 (PHNE_18979)/8.9.3 SMKit7.02) with ESMTP id SAA07963; Thu, 17 May 2001 18:15:23 -0700 (PDT) Message-ID: <3B0477BD.4CE427C9@cup.hp.com> Date: Thu, 17 May 2001 18:15:41 -0700 From: Santosh Rao Organization: Hewlett Packard, Cupertino. X-Mailer: Mozilla 4.7 [en] (X11; U; HP-UX B.11.00 9000/778) X-Accept-Language: en MIME-Version: 1.0 To: ips , Julian Satran Subject: Re: iSCSI : Review Feedback on iSCSI Rev 06 References: <200105040201.TAA00579@hpcuhe.cup.hp.com> Content-Type: multipart/mixed; boundary="------------343D43994570753195BBCE30" Sender: owner-ips@ece.cmu.edu Precedence: bulk This is a multi-part message in MIME format. --------------343D43994570753195BBCE30 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Julian, I did'nt hear back on any of these review comments on Rev 06 and hence, am raising these again for comments. Could you perhaps clarify on these so that we get closure on the same. Regards, Santosh Santosh Rao wrote: > -------------------------------------------------------------------- > > 1) Suggest re-naming the following in the interests of better > clarity : > > a) Rename the ExpDataSN in SCSI Response PDU to EndDataSN as was used in > the previous version of the draft. (indicative of being the final Data sequence > number.) > > b) Rename DataSN in the R2T PDU to R2TSN. (This is really an R2T seqeunce > number.) > > c) Rename R2TExpDataSn to EndR2TSN. (indicative of being the final R2T > sequence number.) > > 2) If a target issues a REJECT response to any received PDU within a SCSI > Command operation, does this imply the target has internally aborted the I/O as > well ? Will the target send a SCSI Response in addition following the Reject ? > > What behaviour must an initiator expect when it sees a reject in response to > some PDU within a SCSI command ? Does it treat this as a termination of the I/O > or should it await a SCSI Response ? > > Also, if the target does terminate the command on sending REJECT, can a "retry" > be issued ? > > In the interests of clearing all these obscurities with the REJECT, I'd like to > suggest the following model : > > - SCSI Command be always responded with a SCSI Response. > - SCSI Task Mgmt Command be always responded with a SCSI Task Mgmt response. > - Only non-SCSI PDUs may be errored back with a REJECT PDU. > > The above is also semantically aligned with the FC LS_RJT/BA_RJT model. > > 3) Why does the new draft limit the size of text commands and nop commands to > 4096 bytes ? What is the rationale behind the magic number selection of 4096 ? > > Since DataPDULength is defined in units of 512 bytes, how does iSCSI deal with > fragmentation caused due to the usage of a DataPDULength < 4K and an attempt to > perform a 4K text , login or nop operation ? > > I believe such scenarios were discussed in an earlier thread > titled "Fragmentation & Reassembly issues in iSCSI." > (http://ips.pdl.cs.cmu.edu/mail/msg03031.html) and the resolution was to > enforce a restriction that the size of a login, nop & text operation be > limited to DataPDULength (?). > > On a seperate note, what happened to PingMaxReplyLength ? It seems to have been > removed from the draft. How can the 2 end points now negotiate a max limit on > nop ping operations ? (for ex : how can a target enforce that the nop ping > operation data buffer not exceed 512 bytes ?). > > 4) As mentioned in a seperate mail, the login or text negotiation can span > several command/response exchanges each using the same initiator task tag. The > addition of a "Target Task Tag" field in the login response, text command and > text response would allow targets to perform a quick loopup on their T.T.T for > this negotiation state information, instead of having to lookup based on > (Initiator iSCSI Name + ISID). > > 5) Section 2.8.3 > "The data length of a text command or response SHOULD be less than > 4096 bytes." > > suggest re-wording the above to : > "The data length of a login, nop & text command or response MUST be less than > DataPDULength. For the leading session login, the length of the login command > MUST be less than 512 bytes." > > The above will ensure that login, text & nop operations will not result in > fragmentation. > > 6) Section 2.8.3 > "A session may have only one outstanding text command or text command > sequence at any given time." > > The above is contradicting itself. Is it only one outstanding text command or > one outstanding text sequence. > > Suggest re-wording the above to : > " A session may have only one outstanding text command at any given time." > > 7) Section 2.11.5 > "The TSID is an initiator identifying tag set by the target. It is > valid only if the connection is accepted." > ^^^^^^^^^^ > > The above needs to be re-worded as : > "only if the login is accepted" > > 8) Section 2.13.1 > > "A target may also issue a NOP-In on its own to carry a changed CmdSN > ^^^^^ > and/or MaxCmdSN if there is no other PDU to carry them for a long > time." > > The above reference to changed CmdSN must be re-worded to "changed ExpCmdSN". > > 9) Section 2.18.1 > > "Target requests logout". > > Suggest removal of the above option. A target that needs to logout would use > either : > "Target will drop the connection" > or > "Target will drop all connections" > > The "Target requests logout" also does not allow the target to specify which > flavor of logout it wishes to receive. It is better to remove this option and > use only a single scheme. > > 10) Section 2.19 > > The section 2.19 seems out of place being in the midst of descriptions of other > iSCSI PDUs. It should be moved out to a seperate location since it has nothing > to do with PDU descriptions. > > 11) As was brought up at the Nashua interim meeting, Status SNACK must be made > optional to support. Simple implementations as also gateway and bridging > products may choose not to [or are unable to] support SNACK mechanisms. Non-use > of Status SNACK must be accompanied by setting StatSN to 0 on all status PDUs. > > 12) Suggest the addition of new login keys to negotiate support for Data SNACK > & Status SNACK. If a target does not support Status SNACK, it must set StatSN > to 0 for all Status PDUs. > > 13) Section 3.2 > " 3.2 iSCSI Logical Unit Control Mode Page > > The following outlines the iSCSI Port mode page: " > > The above needs to be re-worded to : > "The following outlines the iSCSI LUN Control Mode Page: ". > > As suggested earlier, perhaps, suggest using the SPC-2 terms > "protocol specific port page" and "protocol specific LUN page" for these mode > pages. > > 14) Section 3.3.3 > LoginLogoutMinTime > > The wording must be clarified to indicate that this delay is only in the case > the prior logout was due to a target originated event. There is no need for an > initiator to delay after a logout originated by itself. > > For example, while doing open, read/write, close type of operations on a single > thread of activity for the entire session, it may be typical to see iSCSI level > on-the-wire activity of login, read/write, logout.... > > In the above type of scenarios, LoginLogoutMinTime is not applicable and the > initiator must not delay prior to issuing logins. > > 15) Section 4.3 > "If the target responds to a text or Login command with the F bit set > to 1, with a text response with the F bit set to 0, or a login > response with the text bit set to 0, the initiator must keep sending > ^^^^^^^^ > the text command (even empty) with the F bit set to 1 until it gets > the Login Response with the F bit set to 1." > > The above must be re-worded to : > "or a login response with the F bit set to 0, ...". > > 16) Section 6 > "It is further assumed that a target will keep the "status & sense" > for a command it has executed while the total number of outstanding > commands and executed commands does not exceed its limit and status > has not been acknowledged." > > What if the target has exceeded its limit on resources ? Can it drop these > stale resources or must it close its command window and not accept further new > commands ? > > If it is the latter, then, iSCSI is optimizing for recovery paths and is > impacting performance paths by limiting the ability of the target to accept new > I/Os due to having to hold onto stale resources awaiting ExpStatSN updates. > > 17) Section 6.7.3 > "N.B. As an alternative to Logout and reissue commands, the > initiator MAY instead reset the target and terminate all > outstanding commands with a service response indicating > Delivery Subsystem Failure. The initiator MUST perform one of > the two actions." > > As was brought up at the Nashua interim meeting, suggest removal of the above > sentence since it may lead implementors to use Target Reset as a form of error > recovery while dealing with session-level error recovery. > > An alternate form of error recovery in its place would be to use : > "logout - cleans the connection" > or > "logout - cleans the session" > > 18) Section 7.3 > > This section outlines how deterministic clean-up of tasks can be achieved > during task mgmt command execution. However, in the absence of mandating this > behaviour for targets, initiators cannot rely on the same for reliable > deterministic clean-up of pending tasks and will need to go ahead and implement > their own forms of abort task based cleanup. > > This section is meaningless in the absence of a mandate that forces targets to > do as described. In its absence, initiators will be forced to implement their > own schemes for clean up. > > 19) Appendix E > > "15 AccessID > Use: ALL > Who: Initiator > AccessID= > Deliver a SCSI AccessID to the target " > > What is a SCSI Access ID ? Can further descriptions be added to this definition > and a reference be added to SAM2 or some other SCSI document, if appropriate. > > 20) Appendix E > In several places, IO seems to have been used instead of LO. Is this a typo or > a new acronym ? > > Speaking of acronyms, can we add an acronym section to the draft ? > > 21) Appendix E > the description of ImmediateData is describing the case : > "ImmediateData=YES & InitialR2T=YES" twice in differing manners. One of them > needs fixing [or removal]. > > --------------------------------------------------------------------------- --------------343D43994570753195BBCE30 Content-Type: text/x-vcard; charset=us-ascii; name="santoshr.vcf" Content-Description: Card for Santosh Rao Content-Disposition: attachment; filename="santoshr.vcf" Content-Transfer-Encoding: 7bit begin:vcard n:Rao;Santosh tel;work:408-447-3751 x-mozilla-html:FALSE org:Hewlett Packard, Cupertino.;SISL adr:;;19420, Homestead Road, M\S 43LN, ;Cupertino.;CA.;95014.;USA. version:2.1 email;internet:santoshr@cup.hp.com title:Software Design Engineer x-mozilla-cpt:;21088 fn:Santosh Rao end:vcard --------------343D43994570753195BBCE30-- From owner-ips@ece.cmu.edu Thu May 17 23:22:38 2001 Received: from ece.cmu.edu (ECE.CMU.EDU [128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id XAA03996 for ; Thu, 17 May 2001 23:22:37 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f4I1H5k20219 for ips-outgoing; Thu, 17 May 2001 21:17:05 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from palrel2.hp.com (palrel2.hp.com [156.153.255.234]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f4I1H3H20215 for ; Thu, 17 May 2001 21:17:03 -0400 (EDT) Received: from hpcuhe.cup.hp.com (hpcuhe.cup.hp.com [15.0.80.203]) by palrel2.hp.com (Postfix) with ESMTP id 6F60A3C65 for ; Thu, 17 May 2001 18:17:02 -0700 (PDT) Received: from cup.hp.com (santoshr@hpindhhm.cup.hp.com [15.8.80.197]) by hpcuhe.cup.hp.com (8.9.3 (PHNE_18979)/8.9.3 SMKit7.02) with ESMTP id SAA08030 for ; Thu, 17 May 2001 18:16:58 -0700 (PDT) Message-ID: <3B04781B.22FA2A89@cup.hp.com> Date: Thu, 17 May 2001 18:17:15 -0700 From: Santosh Rao Organization: Hewlett Packard, Cupertino. X-Mailer: Mozilla 4.7 [en] (X11; U; HP-UX B.11.00 9000/778) X-Accept-Language: en MIME-Version: 1.0 To: ips@ece.cmu.edu Subject: Re: iSCSI : Review Feedback on iSCSI Rev 06 References: <200105040201.TAA00579@hpcuhe.cup.hp.com> <20010507130209.0907F94006@sandmail.sandburst.com> <3AF73ED0.E6CB535F@cup.hp.com> <200105140255.f4E2tkx23004@chmls05.mediaone.net> <3B047533.7CB74A0E@cup.hp.com> Content-Type: multipart/mixed; boundary="------------5992C18D7DD7B5D079DE3FC8" Sender: owner-ips@ece.cmu.edu Precedence: bulk This is a multi-part message in MIME format. --------------5992C18D7DD7B5D079DE3FC8 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Santosh Rao wrote: > > I've asked this question a long time back on whether DataPDULength > controls the length of all Data PDUs or only the Data PDUs and have'nt ^^^^^^^^^^^^^^^ the above should read "of all PDUs". > heard a definitive answer on this yet. > > --------------5992C18D7DD7B5D079DE3FC8 Content-Type: text/x-vcard; charset=us-ascii; name="santoshr.vcf" Content-Description: Card for Santosh Rao Content-Disposition: attachment; filename="santoshr.vcf" Content-Transfer-Encoding: 7bit begin:vcard n:Rao;Santosh tel;work:408-447-3751 x-mozilla-html:FALSE org:Hewlett Packard, Cupertino.;SISL adr:;;19420, Homestead Road, M\S 43LN, ;Cupertino.;CA.;95014.;USA. version:2.1 email;internet:santoshr@cup.hp.com title:Software Design Engineer x-mozilla-cpt:;21088 fn:Santosh Rao end:vcard --------------5992C18D7DD7B5D079DE3FC8-- From owner-ips@ece.cmu.edu Fri May 18 00:37:53 2001 Received: from ece.cmu.edu (ECE.CMU.EDU [128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id AAA05062 for ; Fri, 18 May 2001 00:37:52 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f4I2BBK23741 for ips-outgoing; Thu, 17 May 2001 22:11:11 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from med.corp.rhapsodynetworks.com (64-160-62-201.rhapsodynetworks.com [64.160.62.201] (may be forged)) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f4I2B9H23736 for ; Thu, 17 May 2001 22:11:09 -0400 (EDT) Received: by med.corp.rhapsodynetworks.com with Internet Mail Service (5.5.2653.19) id ; Thu, 17 May 2001 19:11:03 -0700 Message-ID: <45BEF1D68145D51186C100B0D0AABE6503D401@med.corp.rhapsodynetworks.com> From: Venkat Rangan To: "'Sriram Rupanagunta'" , Sudhir Srinivasan , Venkat Rangan , ips@ece.cmu.edu Subject: RE: FCIP: Multiple connections Date: Thu, 17 May 2001 19:11:00 -0700 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2653.19) Content-Type: text/plain; charset="iso-8859-1" Sender: owner-ips@ece.cmu.edu Precedence: bulk I think another proposal could be that: - the reverse direction frames are sent to the same IP address of the source of the forward direction. - rule 1 still applies: i.e., frames of the same flow in the reverse direction can not be sent on multiple connections. - when a new connection is established to recover into, that new connection is again to the same IP address and not to a different IP address. This removes the possibility of the reverse direction frames landing on a different NIC/port. Venkat Rangan Rhapsody Networks Inc. http://www.rhapsodynetworks.com -----Original Message----- From: Sriram Rupanagunta [mailto:sriramr@aarohi-inc.com] Sent: Thursday, May 17, 2001 4:07 PM To: Sudhir Srinivasan; 'Venkat Rangan'; ips@ece.cmu.edu Subject: RE: FCIP: Multiple connections I agree with Sudhir on this. It is harder to make sure that both sides adhere to same connection. - Sriram > > > Implementing this "flow allegiance" is > > not a significant incremental effort since in order to map all > frames of a > > single direction of a flow on to a particular connection, most of the > > work is already done anyway. > > I disagree - mapping in one direction only is much > easier than mapping consistently in both directions > across multiple vendors' solutions -- both ends have > to use the same mapping algorithm or have to remember > a LOT of stuff to do this in an interoperable manner. > > Now let's look at the issues you raise. > 1. Asymmetric routing: Routers won't distinguish between > two FCIP connections; so I argue that using a different > connection does not add routing asymmetry. > 2. NIC application - Such devices should be built to ensure > that the connections don't span IP addresses (NICs) and > the processing should be done above the TCP port level > so that using a different TCP port wouldn't matter. > 3. Connection failure - even with a single connection, > the issue you mention is already there to some extent. > One side may terminate the connection and it will take > a while for the other side to realize it. Also, ULP > mechanisms should kick in to contain the damage. > > So, in summary, I don't believe these issues are > sufficient to justify the extra burden that the rule > places on devices or the potential risk of interoperability > problems. > > On the issue of "negotiating", FCIP does not have any notion > of parameter negotiation today, correct? > > -----Original Message----- > From: Venkat Rangan [mailto:venkat@rhapsodynetworks.com] > Sent: Thursday, May 17, 2001 11:37 AM > To: 'Sudhir Srinivasan'; 'ips@ece.cmu.edu' > Subject: RE: FCIP: Multiple connections > > > Sudhir, > > For a given FC Flow (indicated by a SID, DID pair of the FC frames), > both forward and reverse directions are mapped to the same connection. > This ensures that: > > 1. Any TCP congestion related rate adjustment on one direction impacts the > other direction equally. Although even a single TCP connection > is subject > to > variations introduced due to assymmetric routing, we do not want to > introduce > more assymmetry by sending forward direction frames of one flow on one > connection and the reverse on a different connection. > > 2. By allowing forward and reverse traffic flow on different connections, > the protocol may exclude implementation of a class of products that > rely on seeing the forward and reverse traffic on the same NIC where > the connection is processed. If the connections end up on different > NICs or ports and the processing is off-loaded to them, then you have > the need to either exchange state or forward traffic to a common point. > > 3. Error recovery in the case of TCP connection failure may cause > one of the > two directions to be functional while the other is not. If both > directions > are mapped to the same connection, such recovery is more > easily mapped to > a link/ISL failure. > > If the protocol allows forward and reverse directions of a flow > to be mapped > on > different connections, I would like to suggest that there is an > option that > allows a pair of FCIP Device Endpoints to negotiate that both directions > must be > mapped on to the same connection. Implementing this "flow allegiance" is > not a significant incremental effort since in order to map all frames of a > single direction of a flow on to a particular connection, most of the > work is already done anyway. > > Regards, > > Venkat Rangan > Rhapsody Networks Inc. > http://www.rhapsodynetworks.com > > > > -----Original Message----- > From: Sudhir Srinivasan [mailto:SudhirS@trebia.com] > Sent: Monday, May 14, 2001 1:53 PM > To: 'ips@ece.cmu.edu' > Cc: Sudhir Srinivasan > Subject: FCIP: Multiple connections > > > > In Section 5.5, what is the reason to include rule (2)? > Is it not sufficient that frames in each direction be delivered > in order? Physical analogy is to have two half-duplex lines > instead of a full-duplex line. Vendor should have the choice > to go either way. > > -Sudhir > > Trebia Networks, Inc. Sudhir.Srinivasan@trebia.com > 978-929-0830 x139 www.trebia.com > From owner-ips@ece.cmu.edu Fri May 18 05:52:54 2001 Received: from ece.cmu.edu (ECE.CMU.EDU [128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id FAA21263 for ; Fri, 18 May 2001 05:52:54 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f4I81Tf14655 for ips-outgoing; Fri, 18 May 2001 04:01:29 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from d12lmsgate.de.ibm.com (d12lmsgate.de.ibm.com [195.212.91.199]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f4I81QH14650 for ; Fri, 18 May 2001 04:01:26 -0400 (EDT) Received: from d12relay01.de.ibm.com (d12relay01.de.ibm.com [9.165.215.22]) by d12lmsgate.de.ibm.com (1.0.0) with ESMTP id KAA358924 for ; Fri, 18 May 2001 10:01:14 +0200 From: julian_satran@il.ibm.com Received: from d12mta02.de.ibm.com (d12mta01_cs0 [9.165.222.237]) by d12relay01.de.ibm.com (8.8.8m3/NCO v4.96) with SMTP id KAA10852 for ; Fri, 18 May 2001 10:01:13 +0200 Received: by d12mta02.de.ibm.com(Lotus SMTP MTA v4.6.5 (863.2 5-20-1999)) id C1256A50.002C0B37 ; Fri, 18 May 2001 10:01:04 +0200 X-Lotus-FromDomain: IBMIL@IBMDE To: ips@ece.cmu.edu Message-ID: Date: Fri, 18 May 2001 06:33:14 +0300 Subject: Re: iscsi: Text keys Mime-Version: 1.0 Content-type: text/plain; charset=us-ascii Content-Disposition: inline Sender: owner-ips@ece.cmu.edu Precedence: bulk Santosh, I understand your concerns. I am also looking for a decent solution that will not break too much existing software. Of of them would be stating that mode set commands, although not rejected, will have no effect on the parameters (we can't limit them to login as those are unaware of such states). As for enquiry in text form - that could be introduced with some very simple syntax - like a key followed by a question mark (?) and answered by a key followed by an exclamation mark (!) but I wonder what will this new thing buy you ? Julo Santosh Rao on 04-05-2001 02:55:21 Please respond to Santosh Rao To: ips@ece.cmu.edu cc: Subject: Re: iscsi: Text keys julian_satran@il.ibm.com wrote: > > 1) There are several text keys which have been specified > with Use=ALL. Such keys can be negotiated during > full-feature phase, which raises interesting questions > about their effect on currently outstanding tasks. > > For example, > a) maxOutstandingR2T : what is the effect on currently > outstanding R2Ts ? > b) LoginLogoutMaxTime : how does it affect connections > which just got dropped and need to be recovered ? > c) DataPDULength : effect on data PDUs in flight. > > Similar questions need to be answered for practically > all the keys which have this behaviour. For the sake of > simplicity, it may be better to forbid such usage and > allow these keys to be only negotiated in the connection > or session login phase. > > +++ some of the parameters you mentioned are SCSI mode-page parameters. > Even if we > restricted them to login they can be changed by a SCSI mode-set. I agree > that keeping them all > restricted to login would simplify implementations but I am not convince it > would make them better. Evidently a negotiated change will apply to all > packets that are sent after the negotiation ended (and a negotiation end is > known to both parties) and an initiator will avoid sending things in > violation of its last offering. > Except that we will have to find a way to reject mode set I am open to > restrict those parameters to login if that is what the majority of > implementors tells me. > +++ Julian, 1) I would like to request that login/text key negotiation be restricted to login time only and have been asking for this on the reflector earlier as well. Implementations would be simpler with fewer options, and we are better off without options like these that are prone to side-effects when changed during full feature phase. 2) Regarding the mode select setting of these parameters, this issue could not be discussed at the Nashua interim meeting for lack of time. I'd like to once again suggest that iSCSI use only 1 scheme(text/login keys) for setting these keys while allowing a query of these options from both layers. 3) Speaking of querying for options, my understanding is that the initiator sends all its options and the target responds with its chosen one from that list during login/text negotiation. Is there a mechanism by which an initiator could query the target for its supported key capabilities ? (i.e. what is the equivalent of a mode sense for these login keys ?). 4) Lastly, the login/text negotiation can comprise of several text command/response exchanges with each being based on the same Initiator Task Tag. The addition of a Target Task Tag field to the login response, text command and text response PDUs would help targets to lookup the negotiation state based on the Target Task Tag which is easier than having to perform a lookup on (Initiator iSCSI Name + ISID). Regards, Santosh - santoshr.vcf From owner-ips@ece.cmu.edu Fri May 18 05:54:56 2001 Received: from ece.cmu.edu (ECE.CMU.EDU [128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id FAA21286 for ; Fri, 18 May 2001 05:54:56 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f4I81f014673 for ips-outgoing; Fri, 18 May 2001 04:01:41 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from d12lmsgate-3.de.ibm.com (d12lmsgate-3.de.ibm.com [195.212.91.201]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f4I81TH14656 for ; Fri, 18 May 2001 04:01:29 -0400 (EDT) Received: from d12relay01.de.ibm.com (d12relay01.de.ibm.com [9.165.215.22]) by d12lmsgate-3.de.ibm.com (1.0.0) with ESMTP id KAA182542 for ; Fri, 18 May 2001 10:01:13 +0200 From: julian_satran@il.ibm.com Received: from d12mta02.de.ibm.com (d12mta01_cs0 [9.165.222.237]) by d12relay01.de.ibm.com (8.8.8m3/NCO v4.96) with SMTP id KAA10848 for ; Fri, 18 May 2001 10:01:12 +0200 Received: by d12mta02.de.ibm.com(Lotus SMTP MTA v4.6.5 (863.2 5-20-1999)) id C1256A50.002C0BC1 ; Fri, 18 May 2001 10:01:05 +0200 X-Lotus-FromDomain: IBMIL@IBMDE To: ips@ece.cmu.edu Message-ID: Date: Fri, 18 May 2001 07:44:49 +0300 Subject: Re: iSCSI : digest error handling violates EMDP/InDataOrder Mime-Version: 1.0 Content-type: text/plain; charset=us-ascii Content-Disposition: inline Sender: owner-ips@ece.cmu.edu Precedence: bulk 2.7.5 will read: The Buffer Offset field contains the offset of this PDU payload data against the complete data transfer. The sum of the buffer offset and length should not exceed the expected transfer length for the command. Data ordering is governed by a disconnect-reconnect mode page bit (EMDP). If this bit is 0 the data packets must be delivered in increasing buffer offset order and data overlays are forbidden. 3.1.2 will read: Data PDUs can be in any order (EMDP = 1) or at continuously increasing addresses (EMDP = 0). EMDP can also be set by a text-mode key=value pair (DataOrder). The appendix will have: Use: LO Who: Initiator and Target DataOrder= The default is yes but targets MAY support no. No is used by iSCSI to indicate that the data PDUs can be in any order (EMDP = 1). Yes is used to indicate that data PDUs have to be at continuously increasing addresses (EMDP = 0). R2T will contain: A R2T MAY be answered with one or more SCSI Data-out PDUs with a matching Target Transfer Tag. If a R2T is answered with a single Data PDU, the Buffer Offset in the Data PDU MUST be the same as the one specified by the R2T. The data length of the Data PDU MUST not exceed the Desired Data Length specified in R2T. If the R2T is answered with a sequence of Data PDUs the Buffer Offset and Length must be within the range of those specified by R2T, the last PDU should have the F bit set to 1, the Buffer Offsets and Lengths for consecutive PDUs should form a continuous non-overlapping range and the PDUs should be sent in increasing offset order. The target may send several R2T PDUs (up to a negotiated number) and thus have a number of data transfers pending. Within a connection, outstanding R2Ts MUST be fulfilled by the initiator in the order in which they where received. Buffer offset ordering in consecutive R2Ts is governed by EMDP. If EMDP is 0 consecutive R2Ts SHOULD refer to continuous non-overlapping ranges. However, a target MAY send out-of-order R2Ts (e.g., for recovery) and an initiator MAY choose to terminate a command when receiving an out-of-order R2T that in can't fulfill, with an appropriate response (indicating service delivery subsystem failure) and aborting the command at the target. Julo Santosh Rao on 04-05-2001 03:35:03 Please respond to Santosh Rao To: Julian Satran/Haifa/IBM@IBMIL cc: ips@ece.cmu.edu Subject: Re: iSCSI : digest error handling violates EMDP/InDataOrder In previous exchanges in this thread, it was stated that EMDP only governs data order within a sequence. This, to me [and others from FC world, as was noted by Bob] reads as the definition of Random relative Offset [as defined by bob]. However, Section 2.7.5 on Buffer Offset states that : "Output data within a burst MUST be delivered in increasing buffer offset order". This contradicts the above by indicating that iSCSI forbids Random Relative Offset. So, does EMDP apply to WRITEs or does it not (?). Also, does iSCSI allow data overlay or is this prohibited ? I'd suggest the following be clearly described in the draft : a) iSCSI defines EMDP to govern the order of data PDUs within a SCSI command for both READ and WRITE operations. An EMDP setting of 0 implies data overlay is forbidden. b) The out-of-order transmission of a given sequence of WRITE data PDUs in response to an R2T (the random relative offset property) be explicitly dis-allowed by iSCSI [which is currently stated in Section 2.7.5]. Regards, Santosh julian_satran@il.ibm.com wrote: > > Santosh, > > Let's take a systematic approach to it. > > Restriction on data ordering are required if the source or the destination > of the data is unable to deliver or take data data in any order other that > sequential. > > Semiconductor or other direct access memories don't have this restriction. > > Tapes and other sequential media do have this type of restriction and so > some streaming devices. > > If the restricted device is a target of a SCSI operation with an > unrestricted initiator then: > > a. on reads the target can always ship its data in sequential order > b. on writes the target can always request the data in sequential order > > However if the restricted device is an initiator then: > > a. on reads the initiator will request the target to send the data in order > b. on writes the restricted initiator will have to get the R2Ts in order > from the target and will be able to support data recovery through an R2T > only if it has enough buffered data. > > A restricted device will act as an initiator only if it becomes a third > part copy manager (CM) in a third party operation an does copy from one of > its devices to another device. > > Introducing a new mode bit (as Robert Snively seems to suggest) will not > change the fact that the restriction can't be upholded > and do recovery unless the restricted initiator has enough memory. > > The spec should only specify a way to terminate a command in those > conditions and leave it at that. > > I will change the wording of the DataOrder to make it clearer but I > consider the whole issue entirely academic and overblown. > > Recall also that a CM implemented which such severe buffering restrictions > violates the basic SCSI assumption that a target is the data master. > > Regards, > Julo > > Santosh Rao on 28/04/2001 00:03:26 > > Please respond to Santosh Rao > > To: Julian Satran/Haifa/IBM@IBMIL > cc: Black_David@emc.com, ips@ece.cmu.edu > Subject: Re: iSCSI : digest error handling violates EMDP/InDataOrder > > julian_satran@il.ibm.com wrote: > > > > David, > > > > I read Bob's mail and my interpretation is similar to his. However I > think > > that SPC explicitly states that different transports are free to > interpret > > and make use of this page as they find appropriate. > > > > I have a hard time understanding Santosh's objection as it does not refer > > to the reason the EMDP is there but to the way it is written in FCP (not > > iSCSI). > > Julian, > > As has been stated earlier, EMDP allows control over the order in which > the target requests outbound data or sends inbound data. EMDP can be > used by initiators to control this order and turn off out-of-order R2T > requests [as well as turn off out of order read data pdus]. > > This is a useful control option and is already provided by other SCSI > transports. What good reason exists to deny this provision in iSCSI ? > > Also, I have some concerns about the ambiguous definition of DataOrder. > > Per the spec : > "DataOrder= > > The default is yes but targets MAY support no. No is used by iSCSI to > indicate that the data PDUs can be in any order (EMDP = 1). Yes is used > to indicate that incoming data PDUs have to be at continuously > increasing addresses (EMDP = 0)." > > Based on the above definition wording : > > a) How is DataOrder interpreted for WRITE I/Os ? > b) Is the ordering across the entire SCSI command or a subset of the I/O > ? If so, what constitutes this subset ? > > Different implementors can arrive at different interpretations reading > the above definition ! > > - Santosh > - santoshr.vcf - santoshr.vcf From owner-ips@ece.cmu.edu Fri May 18 05:56:58 2001 Received: from ece.cmu.edu (ECE.CMU.EDU [128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id FAA21307 for ; Fri, 18 May 2001 05:56:53 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f4I81Pp14648 for ips-outgoing; Fri, 18 May 2001 04:01:25 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from d12lmsgate.de.ibm.com (d12lmsgate.de.ibm.com [195.212.91.199]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f4I81MH14642 for ; Fri, 18 May 2001 04:01:22 -0400 (EDT) Received: from d12relay01.de.ibm.com (d12relay01.de.ibm.com [9.165.215.22]) by d12lmsgate.de.ibm.com (1.0.0) with ESMTP id KAA276750 for ; Fri, 18 May 2001 10:01:13 +0200 From: julian_satran@il.ibm.com Received: from d12mta02.de.ibm.com (d12mta01_cs0 [9.165.222.237]) by d12relay01.de.ibm.com (8.8.8m3/NCO v4.96) with SMTP id KAA10850 for ; Fri, 18 May 2001 10:01:12 +0200 Received: by d12mta02.de.ibm.com(Lotus SMTP MTA v4.6.5 (863.2 5-20-1999)) id C1256A50.002C0B77 ; Fri, 18 May 2001 10:01:05 +0200 X-Lotus-FromDomain: IBMIL@IBMDE To: ips@ece.cmu.edu Message-ID: Date: Fri, 18 May 2001 06:59:52 +0300 Subject: Re: iscsi: comments to iSCSI rev 6 Mime-Version: 1.0 Content-type: text/plain; charset=us-ascii Content-Disposition: inline Sender: owner-ips@ece.cmu.edu Precedence: bulk Santosh, This subject was discussed several times. The reason for using the LUN is to enable iSCSI proxies (or virtualizing devices) to operate as routers in the host to LUN direction and have them possibly insert the LUN field in the LUN to host direction. It addition it enables tag allocation per LUN (not per target). Clearly the LUN could be any other routing tag but the LUN is already a well understood mechanism and will guarantee interoperability. I wonder if we should not make the LUN a mandatory field for NOP-out in general and enable "path checking" up to the LU when the using proxies. As for the tag name - it would be missleading as we do not require it to identify uniquely a task. Julo Santosh Rao on 04-05-2001 03:07:24 Please respond to Santosh Rao To: Matt Wakeley , Julian Satran/Haifa/IBM@IBMIL cc: IPS Reflector Subject: Re: iscsi: comments to iSCSI rev 6 Matt Wakeley wrote: > > Section 2.12.3: indicate that the LUN is copied from the NOP-IN. This is > > much more clear than "the correct value for the task". > > > > +++ and open it up to strange thing like LU5 asking a nop-out to LU7 ? +++ > > How will that occur? The target sent the NOP-IN ping request. The target > includes a LUN. The only "correct value" that the initiator can return for > the LUN in the NOP-OUT ping response is the value the target sent in the > original request. Julian, Re-wording the LUN descriptions in NOP-IN & NOP-OUT to read that initiators MUST echo the LUN field from a received NOP-IN into a NOP-OUT is a more clear and un-ambiguous definition than the current text. That said, I still continue to have reservations about the need for a LUN field in a non-SCSI PDU. Fibre Channel does not use the LUN field in its ELS' & BLS'. LUN is a SCSI construct and it must not be used in non-scsi PDUs. I'm yet to hear strong reasons why LUN is required in the NOP-IN or NOP-OUT. The originator of the NOP operation must generate task tags independent of LUNs for non-SCSI PDUs. Such PDUs have no scsi context. As a side cosmetic comment, can we re-name "Target Transfer Tag" to "Target Task Tag" to have symmetry with the use of Initiator Task Tag ? - Santosh - santoshr.vcf From owner-ips@ece.cmu.edu Fri May 18 14:27:09 2001 Received: from ece.cmu.edu (ECE.CMU.EDU [128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id OAA10838 for ; Fri, 18 May 2001 14:27:09 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f4IG9r327142 for ips-outgoing; Fri, 18 May 2001 12:09:53 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from chmls05.mediaone.net (chmls05.mediaone.net [24.147.1.143]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f4IFbLH24647 for ; Fri, 18 May 2001 11:37:21 -0400 (EDT) Received: from eddylaptop (h00e098880aa4.ne.mediaone.net [66.31.75.244]) by chmls05.mediaone.net (8.11.1/8.11.1) with SMTP id f4IFbGx11266; Fri, 18 May 2001 11:37:16 -0400 (EDT) From: "Eddy Quicksall" To: , Date: Fri, 18 May 2001 11:37:20 -0400 Message-ID: <000101c0dfb0$6f106f00$0100a8c0@eddylaptop> MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 (Normal) X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook CWS, Build 9.0.2416 (9.0.2911.0) Importance: Normal X-MimeOLE: Produced By Microsoft MimeOLE V5.00.3018.1300 Sender: owner-ips@ece.cmu.edu Precedence: bulk Content-Transfer-Encoding: 7bit Section 2.4 SCSI Response says: +---------------+---------------+- 36| ExpDataSN or Reserved (0) +---------------+---------------+- 40| R2TExpDataSN or Reserved (0) +---------------+---------------+- Can we add a phrase that says "This field is reserved if S is 0"? That way, it is very clear when it is reserved. mailto:Eddy@Quicksall.com From owner-ips@ece.cmu.edu Fri May 18 15:24:32 2001 Received: from ece.cmu.edu (ECE.CMU.EDU [128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id PAA13012 for ; Fri, 18 May 2001 15:24:27 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f4IHc5P04310 for ips-outgoing; Fri, 18 May 2001 13:38:05 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from hotmail.com (oe44.law11.hotmail.com [64.4.16.16]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f4IHc3H04297 for ; Fri, 18 May 2001 13:38:03 -0400 (EDT) Received: from mail pickup service by hotmail.com with Microsoft SMTPSVC; Fri, 18 May 2001 10:37:57 -0700 X-Originating-IP: [66.31.75.244] From: "Eddy Quicksall" To: , References: <000101c0dfb0$6f106f00$0100a8c0@eddylaptop> Subject: iSCSI: ExpDataSN or Reserved Date: Fri, 18 May 2001 13:38:02 -0400 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 5.00.3018.1300 X-MimeOLE: Produced By Microsoft MimeOLE V5.00.3018.1300 Message-ID: X-OriginalArrivalTime: 18 May 2001 17:37:57.0104 (UTC) FILETIME=[4682E700:01C0DFC1] Sender: owner-ips@ece.cmu.edu Precedence: bulk Content-Transfer-Encoding: 7bit I forgot to put on the subject. ----- Original Message ----- From: "Eddy Quicksall" To: ; Sent: Friday, May 18, 2001 11:37 AM > Section 2.4 SCSI Response says: > > +---------------+---------------+- > 36| ExpDataSN or Reserved (0) > +---------------+---------------+- > 40| R2TExpDataSN or Reserved (0) > +---------------+---------------+- > > Can we add a phrase that says "This field is reserved if S is 0"? That way, > it is very clear when it is reserved. > > > mailto:Eddy@Quicksall.com > > > From owner-ips@ece.cmu.edu Fri May 18 15:30:03 2001 Received: from ece.cmu.edu (ECE.CMU.EDU [128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id PAA13168 for ; Fri, 18 May 2001 15:30:03 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f4IIeLv09362 for ips-outgoing; Fri, 18 May 2001 14:40:21 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from pluto.he.net (pluto.he.net [216.218.167.2]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f4IIeJH09357 for ; Fri, 18 May 2001 14:40:19 -0400 (EDT) Received: from DSLmodem (63-229-221-92.customers.uswest.net [63.229.221.92]) by pluto.he.net (8.8.6/8.8.2) with SMTP id LAA11753 for ; Fri, 18 May 2001 11:40:13 -0700 From: "Sriram Rupanagunta" To: Subject: RE: FCIP: Multiple connections Date: Fri, 18 May 2001 13:46:19 -0500 Message-ID: MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 (Normal) X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook IMO, Build 9.0.2416 (9.0.2910.0) Importance: Normal X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4133.2400 In-reply-to: <63616B261CEFD411834D00D0B7A86CF7048FBE@lucy.TREBIA.COM> Sender: owner-ips@ece.cmu.edu Precedence: bulk Content-Transfer-Encoding: 7bit Venkat, > - the reverse direction frames are sent to the same IP address of > the source of the forward direction. > This would still not eliminate the need for the FCIP peers to maintain some sort of state information. They would still need to remember the IP addresss from which they received each flow, and make sure to use appropriate connection to send the reverse traffic. Imposing any rule other than rule 1 (Sec 5.5) would make the implementations more difficult, without any great benefit. I agree with you to some extent about the error recovery situation, but then, we need to come up with scenarios, to make sure that these additional rules solve a real problem, before adding them, and increasing the complexity of implementations. - Sriram > [Sudhir] > This precludes taking advantage of multi-homed NICs (not that > I see that as very critical, but somebody else might). > I think the real way to solve this problem is to complete > the specification of how discovery on the IP plane will happen. > If there is a way of establishing tunnel context during > connection setup, then each side can independently enforce > any property such as "NIC allegiance" as needed rather than > putting a rather arbitrary blanket requirement in the standard. > > - when a new connection is established to recover into, that new > connection is again to the same IP address and not to a different > IP address. > > [Sudhir] > What if the FCIP device reboots and gets a different IP address > from DHCP? I don't see the need for this. > > > This removes the possibility of the reverse direction frames landing > on a different NIC/port. > > Venkat Rangan > Rhapsody Networks Inc. > http://www.rhapsodynetworks.com > > > -----Original Message----- > From: Sriram Rupanagunta [mailto:sriramr@aarohi-inc.com] > Sent: Thursday, May 17, 2001 4:07 PM > To: Sudhir Srinivasan; 'Venkat Rangan'; ips@ece.cmu.edu > Subject: RE: FCIP: Multiple connections > > > I agree with Sudhir on this. It is harder to make sure that > both sides adhere to same connection. > > - Sriram > > > > > > Implementing this "flow allegiance" is > > > not a significant incremental effort since in order to map all > > frames of a > > > single direction of a flow on to a particular connection, most of the > > > work is already done anyway. > > > > I disagree - mapping in one direction only is much > > easier than mapping consistently in both directions > > across multiple vendors' solutions -- both ends have > > to use the same mapping algorithm or have to remember > > a LOT of stuff to do this in an interoperable manner. > > > > Now let's look at the issues you raise. > > 1. Asymmetric routing: Routers won't distinguish between > > two FCIP connections; so I argue that using a different > > connection does not add routing asymmetry. > > 2. NIC application - Such devices should be built to ensure > > that the connections don't span IP addresses (NICs) and > > the processing should be done above the TCP port level > > so that using a different TCP port wouldn't matter. > > 3. Connection failure - even with a single connection, > > the issue you mention is already there to some extent. > > One side may terminate the connection and it will take > > a while for the other side to realize it. Also, ULP > > mechanisms should kick in to contain the damage. > > > > So, in summary, I don't believe these issues are > > sufficient to justify the extra burden that the rule > > places on devices or the potential risk of interoperability > > problems. > > > > On the issue of "negotiating", FCIP does not have any notion > > of parameter negotiation today, correct? > > > > -----Original Message----- > > From: Venkat Rangan [mailto:venkat@rhapsodynetworks.com] > > Sent: Thursday, May 17, 2001 11:37 AM > > To: 'Sudhir Srinivasan'; 'ips@ece.cmu.edu' > > Subject: RE: FCIP: Multiple connections > > > > > > Sudhir, > > > > For a given FC Flow (indicated by a SID, DID pair of the FC frames), > > both forward and reverse directions are mapped to the same connection. > > This ensures that: > > > > 1. Any TCP congestion related rate adjustment on one direction > impacts the > > other direction equally. Although even a single TCP connection > > is subject > > to > > variations introduced due to assymmetric routing, we do not want to > > introduce > > more assymmetry by sending forward direction frames of one > flow on one > > connection and the reverse on a different connection. > > > > 2. By allowing forward and reverse traffic flow on different > connections, > > the protocol may exclude implementation of a class of products that > > rely on seeing the forward and reverse traffic on the same NIC where > > the connection is processed. If the connections end up on different > > NICs or ports and the processing is off-loaded to them, then you have > > the need to either exchange state or forward traffic to a > common point. > > > > 3. Error recovery in the case of TCP connection failure may cause > > one of the > > two directions to be functional while the other is not. If both > > directions > > are mapped to the same connection, such recovery is more > > easily mapped to > > a link/ISL failure. > > > > If the protocol allows forward and reverse directions of a flow > > to be mapped > > on > > different connections, I would like to suggest that there is an > > option that > > allows a pair of FCIP Device Endpoints to negotiate that both directions > > must be > > mapped on to the same connection. Implementing this "flow allegiance" is > > not a significant incremental effort since in order to map all > frames of a > > single direction of a flow on to a particular connection, most of the > > work is already done anyway. > > > > Regards, > > > > Venkat Rangan > > Rhapsody Networks Inc. > > http://www.rhapsodynetworks.com > > > > > > > > -----Original Message----- > > From: Sudhir Srinivasan [mailto:SudhirS@trebia.com] > > Sent: Monday, May 14, 2001 1:53 PM > > To: 'ips@ece.cmu.edu' > > Cc: Sudhir Srinivasan > > Subject: FCIP: Multiple connections > > > > > > > > In Section 5.5, what is the reason to include rule (2)? > > Is it not sufficient that frames in each direction be delivered > > in order? Physical analogy is to have two half-duplex lines > > instead of a full-duplex line. Vendor should have the choice > > to go either way. > > > > -Sudhir > > > > Trebia Networks, Inc. Sudhir.Srinivasan@trebia.com > > 978-929-0830 x139 www.trebia.com > > > From owner-ips@ece.cmu.edu Fri May 18 15:30:34 2001 Received: from ece.cmu.edu (ECE.CMU.EDU [128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id PAA13203 for ; Fri, 18 May 2001 15:30:34 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f4IICbY07087 for ips-outgoing; Fri, 18 May 2001 14:12:37 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from lucy.trebia.com ([65.192.191.151]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f4IICZH07080 for ; Fri, 18 May 2001 14:12:35 -0400 (EDT) Received: by lucy.TREBIA.COM with Internet Mail Service (5.5.2653.19) id ; Fri, 18 May 2001 14:16:27 -0400 Message-ID: <63616B261CEFD411834D00D0B7A86CF7048FBE@lucy.TREBIA.COM> From: Sudhir Srinivasan To: "'Venkat Rangan'" , "'Sriram Rupanagunta'" , Sudhir Srinivasan , ips@ece.cmu.edu Subject: RE: FCIP: Multiple connections Date: Fri, 18 May 2001 14:16:25 -0400 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2653.19) Content-Type: text/plain; charset="iso-8859-1" Sender: owner-ips@ece.cmu.edu Precedence: bulk See comments labeled [Sudhir] below: -----Original Message----- From: Venkat Rangan [mailto:venkat@rhapsodynetworks.com] Sent: Thursday, May 17, 2001 10:11 PM To: 'Sriram Rupanagunta'; Sudhir Srinivasan; Venkat Rangan; ips@ece.cmu.edu Subject: RE: FCIP: Multiple connections I think another proposal could be that: - the reverse direction frames are sent to the same IP address of the source of the forward direction. [Sudhir] This precludes taking advantage of multi-homed NICs (not that I see that as very critical, but somebody else might). I think the real way to solve this problem is to complete the specification of how discovery on the IP plane will happen. If there is a way of establishing tunnel context during connection setup, then each side can independently enforce any property such as "NIC allegiance" as needed rather than putting a rather arbitrary blanket requirement in the standard. - when a new connection is established to recover into, that new connection is again to the same IP address and not to a different IP address. [Sudhir] What if the FCIP device reboots and gets a different IP address from DHCP? I don't see the need for this. This removes the possibility of the reverse direction frames landing on a different NIC/port. Venkat Rangan Rhapsody Networks Inc. http://www.rhapsodynetworks.com -----Original Message----- From: Sriram Rupanagunta [mailto:sriramr@aarohi-inc.com] Sent: Thursday, May 17, 2001 4:07 PM To: Sudhir Srinivasan; 'Venkat Rangan'; ips@ece.cmu.edu Subject: RE: FCIP: Multiple connections I agree with Sudhir on this. It is harder to make sure that both sides adhere to same connection. - Sriram > > > Implementing this "flow allegiance" is > > not a significant incremental effort since in order to map all > frames of a > > single direction of a flow on to a particular connection, most of the > > work is already done anyway. > > I disagree - mapping in one direction only is much > easier than mapping consistently in both directions > across multiple vendors' solutions -- both ends have > to use the same mapping algorithm or have to remember > a LOT of stuff to do this in an interoperable manner. > > Now let's look at the issues you raise. > 1. Asymmetric routing: Routers won't distinguish between > two FCIP connections; so I argue that using a different > connection does not add routing asymmetry. > 2. NIC application - Such devices should be built to ensure > that the connections don't span IP addresses (NICs) and > the processing should be done above the TCP port level > so that using a different TCP port wouldn't matter. > 3. Connection failure - even with a single connection, > the issue you mention is already there to some extent. > One side may terminate the connection and it will take > a while for the other side to realize it. Also, ULP > mechanisms should kick in to contain the damage. > > So, in summary, I don't believe these issues are > sufficient to justify the extra burden that the rule > places on devices or the potential risk of interoperability > problems. > > On the issue of "negotiating", FCIP does not have any notion > of parameter negotiation today, correct? > > -----Original Message----- > From: Venkat Rangan [mailto:venkat@rhapsodynetworks.com] > Sent: Thursday, May 17, 2001 11:37 AM > To: 'Sudhir Srinivasan'; 'ips@ece.cmu.edu' > Subject: RE: FCIP: Multiple connections > > > Sudhir, > > For a given FC Flow (indicated by a SID, DID pair of the FC frames), > both forward and reverse directions are mapped to the same connection. > This ensures that: > > 1. Any TCP congestion related rate adjustment on one direction impacts the > other direction equally. Although even a single TCP connection > is subject > to > variations introduced due to assymmetric routing, we do not want to > introduce > more assymmetry by sending forward direction frames of one flow on one > connection and the reverse on a different connection. > > 2. By allowing forward and reverse traffic flow on different connections, > the protocol may exclude implementation of a class of products that > rely on seeing the forward and reverse traffic on the same NIC where > the connection is processed. If the connections end up on different > NICs or ports and the processing is off-loaded to them, then you have > the need to either exchange state or forward traffic to a common point. > > 3. Error recovery in the case of TCP connection failure may cause > one of the > two directions to be functional while the other is not. If both > directions > are mapped to the same connection, such recovery is more > easily mapped to > a link/ISL failure. > > If the protocol allows forward and reverse directions of a flow > to be mapped > on > different connections, I would like to suggest that there is an > option that > allows a pair of FCIP Device Endpoints to negotiate that both directions > must be > mapped on to the same connection. Implementing this "flow allegiance" is > not a significant incremental effort since in order to map all frames of a > single direction of a flow on to a particular connection, most of the > work is already done anyway. > > Regards, > > Venkat Rangan > Rhapsody Networks Inc. > http://www.rhapsodynetworks.com > > > > -----Original Message----- > From: Sudhir Srinivasan [mailto:SudhirS@trebia.com] > Sent: Monday, May 14, 2001 1:53 PM > To: 'ips@ece.cmu.edu' > Cc: Sudhir Srinivasan > Subject: FCIP: Multiple connections > > > > In Section 5.5, what is the reason to include rule (2)? > Is it not sufficient that frames in each direction be delivered > in order? Physical analogy is to have two half-duplex lines > instead of a full-duplex line. Vendor should have the choice > to go either way. > > -Sudhir > > Trebia Networks, Inc. Sudhir.Srinivasan@trebia.com > 978-929-0830 x139 www.trebia.com > From owner-ips@ece.cmu.edu Fri May 18 18:24:05 2001 Received: from ece.cmu.edu (ECE.CMU.EDU [128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id SAA17746 for ; Fri, 18 May 2001 18:24:05 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f4IJuQv15376 for ips-outgoing; Fri, 18 May 2001 15:56:26 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from lucy.trebia.com ([65.192.191.151]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f4IJuLH15370 for ; Fri, 18 May 2001 15:56:25 -0400 (EDT) Received: by lucy.TREBIA.COM with Internet Mail Service (5.5.2653.19) id ; Fri, 18 May 2001 16:00:17 -0400 Message-ID: <63616B261CEFD411834D00D0B7A86CF7048FC3@lucy.TREBIA.COM> From: Sudhir Srinivasan To: IPS Reflector Cc: Sudhir Srinivasan Subject: RE: FCIP - Proposed changes in support of common FC encapsulation Date: Fri, 18 May 2001 16:00:14 -0400 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2653.19) Content-Type: text/plain; charset="iso-8859-1" Sender: owner-ips@ece.cmu.edu Precedence: bulk Only one comment on this. You suggest the following text: "The FCIP receiving device MUST reverse the FC frame encapsulation process, verify the correctness of the header discarding frames with header errors, and deliver the FC frames to the correct FC ports connected to the FCIP device." The part about "discarding frames with header errors" is confusing because FCIP header errors will result in loss of synchronization and Section 8.1 will then define the action to be taken. - Sudhir -----Original Message----- From: Ralph Weber [mailto:ralphoweber@compuserve.com] Sent: Thursday, May 17, 2001 5:51 PM To: IPS Reflector Subject: FCIP - Proposed changes in support of common FC encapsulation In the 3rd paragraph of section 4.3, replace: Each FCIP data frame is built by adding an FCIP header to one FC frame delivered to the FCIP endpoint for transport. The FCIP data frames are handed in their entirety to TCP; TCP is responsible for delivering the same series of FCIP data frames to the receiving side in the same order as they are transmitted by the sending FCIP device. The FCIP device MUST find the FCIP headers and deliver the FC frames wrapped inside the FCIP data frames to the correct FC ports connected to the FCIP device. with: An FCIP data frame is built by encapsulating one FC frame delivered to the FCIP endpoint for transport using the encapsulation format described in the common FC Frame Encapsulation [23] and in section 5.2 of this document. The FCIP data frames are handed in their entirety to TCP; TCP is responsible for delivering the same series of FCIP data frames to the receiving side in the same order as they are transmitted by the sending FCIP device. The FCIP receiving device MUST reverse the FC frame encapsulation process, verify the correctness of the header discarding frames with header errors, and deliver the FC frames to the correct FC ports connected to the FCIP device. From owner-ips@ece.cmu.edu Fri May 18 20:08:12 2001 Received: from ece.cmu.edu (ECE.CMU.EDU [128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id UAA20002 for ; Fri, 18 May 2001 20:08:12 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f4ILqeI23728 for ips-outgoing; Fri, 18 May 2001 17:52:40 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from maho3msx2.isus.emc.com (maho3msx2.isus.emc.com [128.221.11.32]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f4ILqcH23723 for ; Fri, 18 May 2001 17:52:38 -0400 (EDT) Received: by maho3msx2.isus.emc.com with Internet Mail Service (5.5.2653.19) id ; Fri, 18 May 2001 17:52:32 -0400 Message-ID: <0F31E5C394DAD311B60C00E029101A07080155A0@corpmx9.isus.emc.com> From: Black_David@emc.com To: aboba@internaut.com, ips@ece.cmu.edu Subject: RE: iSCSI Security rough consensus Date: Fri, 18 May 2001 17:52:31 -0400 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2653.19) Content-Type: text/plain; charset="iso-8859-1" Sender: owner-ips@ece.cmu.edu Precedence: bulk Bernard, > Do we have a set of guidelines about what this draft is supposed to > achieve? Not yet, but this is a good place to talk about them - thanks for the prod. IMHO, I would do the following ... > For example: > > 1. Do we need to support negotiation of SRP prime modulus/generator > groups from within the standard set? Not "negotiation" per se. I'd pick a small (tasteful) number of them, make them all "MUST implement", have the Initiator pick one and announce it via iSCSI text key(s) and/or value(s) sent as part of the initial message. If the Target doesn't like it for some reason (e.g., we exercised bad taste [in 20/20 hindsight] and the announced one is insufficiently secure), it indicates its dissatisfaction by terminating the login, but "SHOULD NOT" do this without a very good reason, as a general strategy of retrying with a different modulus/generator at the Initiator in response to a Target reject of this form opens up man-in-the-middle attacks on the negotiation to force use of a "weaker" modulus/group (from the attacker's perspective). > 2. Do we need to generate keying material for Phase 1 as well as Phase 2 > SAs? Phase 2 only, but see next item for an approach to rekeying a Phase 2 SA without using a Phase 1 SA. > 3. Do we need to support rekeys? Yes, but Ted and I had talked about doing this without a key exchange. The idea is to iteratively use a secure hash on the material generated by SRP (not all of which is used to key ESP) to generate the same sequence of keys on both sides. Either side twiddles a few bits in the SPI to tell the other to go to the next key (e.g., use 2 or 3 bits in the SPI as a counter, bump the counter to go to the next key, which is used for the packet that contains the bumped counter). > 4. Do we need to support ciphersuite negotiations? Sort of, with the same approach as the SRP prime modulus/generator group case above, i.e., pick a few, make them MUST implement, rely on iSCSI text keys/values as primary negotiation mechanism, Initiator announces what is to be done, and Target can accept or terminate the login. One could alternatively use a few bits in the SPI to do this. At the moment, I'd spec 3DES with a SHA-1 HMAC and AES with an appropriate AES MAC, and look for an opportunity to reduce the spec to only require the latter. > 5. Is there a need to negotiate filters? I don't think so. A simple model of one Initiator talking to one Target over one SA per TCP connection makes the filters implicit and obvious. > 6. Is there a need for concealment of identities? I would think not because SRP passes the identity in the clear. > In other words, how much of IKE are you willing to throw away? From the above, I hope an answer of "quite a bit" is credible. Implementers who don't like the answers to questions such as 2-6 above would have the option of implementing full IKE (or son-of-IKE) to get Blowfish, Skipjack, MD5, elliptic curve, negotiated filters, concealed identities, Phase 1 SAs, etc. This would be done prior to iSCSI starting up. > While a lot > of IKE complexity is due to artifacts of now discarded layering, a lot of > it is also due to the generality of what it tries to achieve. If you want > less features, you can get a lighter weight protocol... That would be the goal. This is all IMHO/off the top of my head and comments are encouraged. Thanks, --David p.s., I saw an earlier comment that 8MB for IKE was not a big deal for an embedded system. The HBA vendors clearly weren't paying attention to that comment because 8MB is a big deal for an HBA. It's also a big deal for a disk drive, and I would suggest that it's also a visible issue for disk arrays. Now, before someone quotes the size of Symmetrix shared disk cache at me (it's measured in 10s of gigabytes), let me point out that there is *no* code or local data in that cache, and the HDS/HP high-end arrays are similar in that aspect. --------------------------------------------------- David L. Black, Senior Technologist EMC Corporation, 42 South St., Hopkinton, MA 01748 +1 (508) 435-1000 x75140 FAX: +1 (508) 497-8500 black_david@emc.com Mobile: +1 (978) 394-7754 --------------------------------------------------- From owner-ips@ece.cmu.edu Fri May 18 21:24:23 2001 Received: from ece.cmu.edu (ECE.CMU.EDU [128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id VAA21004 for ; Fri, 18 May 2001 21:24:22 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f4INNba29780 for ips-outgoing; Fri, 18 May 2001 19:23:37 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from palrel10.hp.com (palrel10.hp.com [192.151.10.58]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f4INNaH29776 for ; Fri, 18 May 2001 19:23:36 -0400 (EDT) Received: from amrelay1.boi.hp.com (amrelay1.boi.hp.com [15.56.8.24]) by palrel10.hp.com (Postfix) with ESMTP id 4FD811F67F for ; Fri, 18 May 2001 16:23:30 -0700 (PDT) Received: from xatlbh3.atl.hp.com (xatlbh3.atl.hp.com [15.45.89.188]) by amrelay1.boi.hp.com (8.9.3 (PHNE_18979)/8.9.3 SMKit7.02) with ESMTP id RAA11283 for ; Fri, 18 May 2001 17:23:29 -0600 (MDT) Received: by xatlbh3.atl.hp.com with Internet Mail Service (5.5.2653.19) id ; Fri, 18 May 2001 19:23:28 -0400 Message-ID: <6BD67FFB937FD411A04F00D0B74FE87802A0907D@xrose06.rose.hp.com> From: "KRUEGER,MARJORIE (HP-Roseville,ex1)" To: "Ips Reflector (E-mail)" Subject: Slides from Nashua Interim meeting? Date: Fri, 18 May 2001 19:23:25 -0400 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2653.19) Content-Type: text/plain; charset="iso-8859-1" Sender: owner-ips@ece.cmu.edu Precedence: bulk Where are these presentations slides posted? I couldn't find them on Julian's archive or http://www.ece.cmu.edu/~ips/ Marjorie Krueger Networked Storage Architecture Networked Storage Solutions Org. Hewlett-Packard tel: +1 916 785 2656 fax: +1 916 785 0391 email: marjorie_krueger@hp.com From owner-ips@ece.cmu.edu Fri May 18 22:49:48 2001 Received: from ece.cmu.edu (ECE.CMU.EDU [128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id WAA23600 for ; Fri, 18 May 2001 22:49:47 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f4J1DoL07264 for ips-outgoing; Fri, 18 May 2001 21:13:50 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from atlrel2.hp.com (atlrel2.hp.com [156.153.255.202]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f4J1DnH07260 for ; Fri, 18 May 2001 21:13:49 -0400 (EDT) Received: from amrelay1.boi.hp.com (amrelay1.boi.hp.com [15.56.8.24]) by atlrel2.hp.com (Postfix) with ESMTP id C0AEE1442; Fri, 18 May 2001 21:13:48 -0400 (EDT) Received: from xpabh2.corp.hp.com (xpabh2.corp.hp.com [15.58.136.192]) by amrelay1.boi.hp.com (8.9.3 (PHNE_18979)/8.9.3 SMKit7.02) with ESMTP id TAA23277; Fri, 18 May 2001 19:13:48 -0600 (MDT) Received: by xpabh2.corp.hp.com with Internet Mail Service (5.5.2653.19) id ; Fri, 18 May 2001 18:13:48 -0700 Message-ID: <6BD67FFB937FD411A04F00D0B74FE87802A0907E@xrose06.rose.hp.com> From: "KRUEGER,MARJORIE (HP-Roseville,ex1)" To: "'Black_David@emc.com'" , ips@ece.cmu.edu Subject: RE: DRAFT Nashua interim meeting minutes Date: Fri, 18 May 2001 18:13:47 -0700 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2653.19) Content-Type: text/plain; charset="iso-8859-1" Sender: owner-ips@ece.cmu.edu Precedence: bulk A few items from my notes that I couldn't find in your draft minutes: Strong consensus that header formats and opcodes should not be changed in further revisions of the draft except when there is rough concensus from the group for such changes. There are changes expected in rev. 07 to correct some things pointed out on the list. There was discussion in the security portion of the agenda to the effect that "if we mandate IPSec with it's data integrity check capabilities, is there further need to specify error recovery"? Marjorie Krueger Networked Storage Architecture Networked Storage Solutions Org. Hewlett-Packard tel: +1 916 785 2656 fax: +1 916 785 0391 email: marjorie_krueger@hp.com From owner-ips@ece.cmu.edu Fri May 18 23:43:51 2001 Received: from ece.cmu.edu (ECE.CMU.EDU [128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id XAA24336 for ; Fri, 18 May 2001 23:43:51 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f4J26Z110577 for ips-outgoing; Fri, 18 May 2001 22:06:35 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from med.corp.rhapsodynetworks.com (64-160-62-201.rhapsodynetworks.com [64.160.62.201] (may be forged)) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f4J26WH10570 for ; Fri, 18 May 2001 22:06:33 -0400 (EDT) Received: by med.corp.rhapsodynetworks.com with Internet Mail Service (5.5.2653.19) id ; Fri, 18 May 2001 19:06:23 -0700 Message-ID: <45BEF1D68145D51186C100B0D0AABE6503D40D@med.corp.rhapsodynetworks.com> From: Venkat Rangan To: "'julian_satran@il.ibm.com'" , ips@ece.cmu.edu Subject: ISCSI: Flags in header indicating that digest is present Date: Fri, 18 May 2001 19:06:21 -0700 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2653.19) Content-Type: text/plain; charset="iso-8859-1" Sender: owner-ips@ece.cmu.edu Precedence: bulk Julian, At some point, we had bits in the BHS that indicated whether Header Digest and/or Data Digest is present. They now have disappeared, perhaps when we removed the WN-based PDUs for Draft-6. Now, how do we know from the PDU itself whether it is present or not? If not, is the intention that we maintain the current state of the digest negotiation and apply it to all PDUs? From Draft 5: ---------------- 2.2.1 What's Next (WN) This is an encoded field that indicates what the next segment is. bit 7 - 1 Next is another header segment. bit 6-4 Next header type code 0 Extended CDB 1 Bi-directional read-data transfer header 2 Long Data Header 3,4,5,6,7 Reserved bit 3-0 Reserved bit 7 - 0 Next is a data segment or no additional segment (empty data segment) bit 6 Header Digest Present (1) or Not (0) bit 5 Data Digest Present (1) or Not (0) bit 4-0 Reserved ------------------ Thanks, Venkat Rangan Rhapsody Networks Inc. http://www.rhapsodynetworks.com From owner-ips@ece.cmu.edu Fri May 18 23:48:24 2001 Received: from ece.cmu.edu (ECE.CMU.EDU [128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id XAA24428 for ; Fri, 18 May 2001 23:48:23 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f4J2CPe10945 for ips-outgoing; Fri, 18 May 2001 22:12:25 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from med.corp.rhapsodynetworks.com (64-160-62-201.rhapsodynetworks.com [64.160.62.201] (may be forged)) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f4J2COH10941 for ; Fri, 18 May 2001 22:12:24 -0400 (EDT) Received: by med.corp.rhapsodynetworks.com with Internet Mail Service (5.5.2653.19) id ; Fri, 18 May 2001 19:12:18 -0700 Message-ID: <45BEF1D68145D51186C100B0D0AABE6503D40E@med.corp.rhapsodynetworks.com> From: Venkat Rangan To: "'julian_satran@il.ibm.com'" , ips@ece.cmu.edu Subject: RE: Another shot at codes and please comment Date: Fri, 18 May 2001 19:12:17 -0700 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2653.19) Content-Type: text/plain; charset="iso-8859-1" Sender: owner-ips@ece.cmu.edu Precedence: bulk Julian, What is the state of this change? Is this going to be in draft 7, or is it in draft 6? Thanks, Venkat Rangan Rhapsody Networks Inc. http://www.rhapsodynetworks.com -----Original Message----- From: julian_satran@il.ibm.com [mailto:julian_satran@il.ibm.com] Sent: Tuesday, May 01, 2001 6:50 AM To: ips@ece.cmu.edu Subject: Another shot at codes and please comment Here is a another version of the opcodes part: 1.1.1.1 Opcode The Opcode indicates what type of iSCSI PDU the header encapsulates. The Opcodes are divided into two categories: initiator opcodes and target opcodes. Initiator opcodes are in PDUs sent by the initiators (request PDUs), and target opcodes are in PDUs sent by the target (response PDUs). Initiators MUST NOT use target opcodes and targets MUST NOT use initiator opcodes. Valid initiator opcodes defined in this specification are: 0x00 NOP-Out (from initiator to target) 0x01 SCSI Command (encapsulates a SCSI Command Descriptor Block) 0x02 SCSI Task Management Command 0x03 Login Command 0x04 Text Command 0x05 SCSI Data-out (for WRITE operations) 0x06 Logout Command 0x10 SNACK Request Valid target opcodes are: 0x20 NOP-In (from target to initiator) 0x21 SCSI Response (contains SCSI status and possibly sense information or other response information) 0x22 SCSI Task Management Response 0x23 Login Response 0x24 Text Response 0x25 SCSI Data-in (for READ operations) 0x26 Logout Response 0x31 Ready To Transfer (R2T - sent by target to initiator when it is ready to receive data from initiator) 0x32 Asynchronous Message (sent by target to initiator to indicate certain special conditions) 0x3f Reject Initiator opcodes 0x1c-0x1e and target opcodes 0x3c-0x3e are vendor specific codes. Please comment, Julo From owner-ips@ece.cmu.edu Sat May 19 03:36:11 2001 Received: from ece.cmu.edu (ECE.CMU.EDU [128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id DAA09831 for ; Sat, 19 May 2001 03:36:11 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f4J5k1e23321 for ips-outgoing; Sat, 19 May 2001 01:46:01 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from d12lmsgate-2.de.ibm.com (d12lmsgate-2.de.ibm.com [195.212.91.200]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f4J5k0H23317 for ; Sat, 19 May 2001 01:46:00 -0400 (EDT) Received: from d12relay01.de.ibm.com (d12relay01.de.ibm.com [9.165.215.22]) by d12lmsgate-2.de.ibm.com (1.0.0) with ESMTP id HAA335364 for ; Sat, 19 May 2001 07:45:49 +0200 From: julian_satran@il.ibm.com Received: from d12mta02.de.ibm.com (d12mta01_cs0 [9.165.222.237]) by d12relay01.de.ibm.com (8.8.8m3/NCO v4.96) with SMTP id HAA58778 for ; Sat, 19 May 2001 07:45:45 +0200 Received: by d12mta02.de.ibm.com(Lotus SMTP MTA v4.6.5 (863.2 5-20-1999)) id C1256A51.001FA50A ; Sat, 19 May 2001 07:45:38 +0200 X-Lotus-FromDomain: IBMIL@IBMDE To: ips@ece.cmu.edu Message-ID: Date: Sat, 19 May 2001 08:51:20 +0300 Subject: RE: Another shot at codes and please comment Mime-Version: 1.0 Content-type: text/plain; charset=us-ascii Content-Disposition: inline Sender: owner-ips@ece.cmu.edu Precedence: bulk Venkat, This seems to be the popular choice for 07 (2 voices for, none against and silence from the bored majority). Julo Venkat Rangan on 19-05-2001 05:12:17 Please respond to Venkat Rangan To: Julian Satran/Haifa/IBM@IBMIL, ips@ece.cmu.edu cc: Subject: RE: Another shot at codes and please comment Julian, What is the state of this change? Is this going to be in draft 7, or is it in draft 6? Thanks, Venkat Rangan Rhapsody Networks Inc. http://www.rhapsodynetworks.com -----Original Message----- From: julian_satran@il.ibm.com [mailto:julian_satran@il.ibm.com] Sent: Tuesday, May 01, 2001 6:50 AM To: ips@ece.cmu.edu Subject: Another shot at codes and please comment Here is a another version of the opcodes part: 1.1.1.1 Opcode The Opcode indicates what type of iSCSI PDU the header encapsulates. The Opcodes are divided into two categories: initiator opcodes and target opcodes. Initiator opcodes are in PDUs sent by the initiators (request PDUs), and target opcodes are in PDUs sent by the target (response PDUs). Initiators MUST NOT use target opcodes and targets MUST NOT use initiator opcodes. Valid initiator opcodes defined in this specification are: 0x00 NOP-Out (from initiator to target) 0x01 SCSI Command (encapsulates a SCSI Command Descriptor Block) 0x02 SCSI Task Management Command 0x03 Login Command 0x04 Text Command 0x05 SCSI Data-out (for WRITE operations) 0x06 Logout Command 0x10 SNACK Request Valid target opcodes are: 0x20 NOP-In (from target to initiator) 0x21 SCSI Response (contains SCSI status and possibly sense information or other response information) 0x22 SCSI Task Management Response 0x23 Login Response 0x24 Text Response 0x25 SCSI Data-in (for READ operations) 0x26 Logout Response 0x31 Ready To Transfer (R2T - sent by target to initiator when it is ready to receive data from initiator) 0x32 Asynchronous Message (sent by target to initiator to indicate certain special conditions) 0x3f Reject Initiator opcodes 0x1c-0x1e and target opcodes 0x3c-0x3e are vendor specific codes. Please comment, Julo From owner-ips@ece.cmu.edu Sat May 19 03:39:46 2001 Received: from ece.cmu.edu (ECE.CMU.EDU [128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id DAA09873 for ; Sat, 19 May 2001 03:39:45 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f4J5gdN23168 for ips-outgoing; Sat, 19 May 2001 01:42:39 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from d12lmsgate.de.ibm.com (d12lmsgate.de.ibm.com [195.212.91.199]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f4J5gbH23164 for ; Sat, 19 May 2001 01:42:37 -0400 (EDT) Received: from d12relay01.de.ibm.com (d12relay01.de.ibm.com [9.165.215.22]) by d12lmsgate.de.ibm.com (1.0.0) with ESMTP id HAA207834 for ; Sat, 19 May 2001 07:42:31 +0200 From: julian_satran@il.ibm.com Received: from d12mta02.de.ibm.com (d12mta01_cs0 [9.165.222.237]) by d12relay01.de.ibm.com (8.8.8m3/NCO v4.96) with SMTP id HAA84968 for ; Sat, 19 May 2001 07:42:28 +0200 Received: by d12mta02.de.ibm.com(Lotus SMTP MTA v4.6.5 (863.2 5-20-1999)) id C1256A51.001F57E9 ; Sat, 19 May 2001 07:42:21 +0200 X-Lotus-FromDomain: IBMIL@IBMDE To: ips@ece.cmu.edu Message-ID: Date: Sat, 19 May 2001 08:48:04 +0300 Subject: Re: ISCSI: Flags in header indicating that digest is present Mime-Version: 1.0 Content-type: text/plain; charset=us-ascii Content-Disposition: inline Sender: owner-ips@ece.cmu.edu Precedence: bulk Venkat, Yes - the new format has a total length field that consumes too many bits to have some left for digests. Julo Venkat Rangan on 19-05-2001 05:06:21 Please respond to Venkat Rangan To: Julian Satran/Haifa/IBM@IBMIL, ips@ece.cmu.edu cc: Subject: ISCSI: Flags in header indicating that digest is present Julian, At some point, we had bits in the BHS that indicated whether Header Digest and/or Data Digest is present. They now have disappeared, perhaps when we removed the WN-based PDUs for Draft-6. Now, how do we know from the PDU itself whether it is present or not? If not, is the intention that we maintain the current state of the digest negotiation and apply it to all PDUs? From Draft 5: ---------------- 2.2.1 What's Next (WN) This is an encoded field that indicates what the next segment is. bit 7 - 1 Next is another header segment. bit 6-4 Next header type code 0 Extended CDB 1 Bi-directional read-data transfer header 2 Long Data Header 3,4,5,6,7 Reserved bit 3-0 Reserved bit 7 - 0 Next is a data segment or no additional segment (empty data segment) bit 6 Header Digest Present (1) or Not (0) bit 5 Data Digest Present (1) or Not (0) bit 4-0 Reserved ------------------ Thanks, Venkat Rangan Rhapsody Networks Inc. http://www.rhapsodynetworks.com From owner-ips@ece.cmu.edu Sat May 19 04:41:20 2001 Received: from ece.cmu.edu (ECE.CMU.EDU [128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id EAA10290 for ; Sat, 19 May 2001 04:41:20 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f4J7Xo229128 for ips-outgoing; Sat, 19 May 2001 03:33:50 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from d12lmsgate-2.de.ibm.com (d12lmsgate-2.de.ibm.com [195.212.91.200]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f4J7XmH29123 for ; Sat, 19 May 2001 03:33:49 -0400 (EDT) Received: from d12relay01.de.ibm.com (d12relay01.de.ibm.com [9.165.215.22]) by d12lmsgate-2.de.ibm.com (1.0.0) with ESMTP id JAA335430 for ; Sat, 19 May 2001 09:33:43 +0200 From: julian_satran@il.ibm.com Received: from d12mta02.de.ibm.com (d12mta01_cs0 [9.165.222.237]) by d12relay01.de.ibm.com (8.8.8m3/NCO v4.96) with SMTP id JAA14854 for ; Sat, 19 May 2001 09:33:40 +0200 Received: by d12mta02.de.ibm.com(Lotus SMTP MTA v4.6.5 (863.2 5-20-1999)) id C1256A51.00298827 ; Sat, 19 May 2001 09:33:38 +0200 X-Lotus-FromDomain: IBMIL@IBMDE To: ips@ece.cmu.edu Message-ID: Date: Sat, 19 May 2001 10:39:22 +0300 Subject: Re: iSCSI : Bridging missing CmdSNs and Abort I/O Error recovery Mime-Version: 1.0 Content-type: text/plain; charset=us-ascii Content-Disposition: inline Sender: owner-ips@ece.cmu.edu Precedence: bulk Santosh, Your scenarios can be easily handled in any implementation. A target can respond with an OK answer and the target and initiator can cooperate in removing the spurious answer if still in flight in manner similar to the one used for the task set cleanup. Julo Santosh Rao on 14-05-2001 19:25:16 Please respond to Santosh Rao To: ips@ece.cmu.edu cc: Subject: Re: iSCSI : Bridging missing CmdSNs and Abort I/O Error recovery Julian, I have re-read your previous answer and I may be missing something, but, I don't see how it addresses my concern (?). Per your mail : "To abort safely a task for which the task abort answer is "Command Not Received Yet" the initiator must issue another abort command on the same connection as the original command unless this connection was logged out in which case it may send it on any connection." I'm pointing out that the above may not be sufficient since the issue spans beyond that. If the target HAD received the command by the time it received the task mgmt command on another connection, it would not respond with "Command not received yet". In this case, the initiator would not be required to re-send the Abort Task on the same connection as the original command. In such a scenario, if there were PDUs in-flight from target to initiator on the original connection, these could arrive after the task mgmt response [which is on the 2nd connection]. Such an Abort Task cleanup does not reliably allow the initiator to free its task tag and other resources, since PDUs for that task continue to arrive on the original connection after the initiator has completed task clean-up using Abort Task & released the task resources. - Santosh julian_satran@il.ibm.com wrote: > > please reread carefully my previous answer. Julo > > Santosh Rao on 09-05-2001 19:37:41 > > Please respond to Santosh Rao > > To: IPS Reflector > cc: > Subject: Re: iSCSI : Bridging missing CmdSNs and Abort I/O Error recovery > > > julian_satran@il.ibm.com wrote: > > > > > We could add the RefCmdSN and it may help plug-in the hole but unless > the > > > > Abort is ussued on the same connection as the original command we > can't > > > be > > > > sure that the old-one will not pop-up (as we enable ExpCmdSN to move > on > > > we > > > > don't have even the 2**31-1 protection bracket :-)). Thus sending in > > > > another nop/abort on the same connection is still required. > > > > > > > > To simplify the whole process I will: > > > > > > > > a - add RefCmdSN to the Task Management > > > > b - add a command not received yet to the answers > > > > > > > > c - add a part to 7 reading: > > > > > > > > 1.1 How to Abort Safely a Command that Was Not Received > > > > > > > > To abort safely a task for which the task abort answer is "Command > Not > > > > Received Yet" the initiator must issue another abort command on > the > > > same > > > > connection as the original command unless this connection was > logged > > > out > > > > in which case it may send it on any connection. > > Julian, > > The above may not be sufficient, since a requirement for the Abort Task > completion is that both the initiator & target can safely re-use their > task tags and other resources following completion of the Abort Task. If > the Abort Task is sent on another connection than the connection used > for the original command [& that connection has not been logged out], > then, there may be some in-flight PDUs from the target to the initiator > for that command on the connection. > > Sending an Abort Task on another connection may cause the initiator to > receive the Abort Task response indicating success on the other > connection which then results in initiator freeing up its task tag and > other resources used for that command. Thereafter, the stale PDUs on the > original connection can show up at the initiator, causing problems. > > I would suggest that Abort Task be always sent on the same connection as > the command, unless the connection used for that command has been logged > out. Not doing so implies the initiator cannot safely re-use its task > tag and other resources that were tied up for that I/O. > > Regards, > Santosh > - santoshr.vcf - santoshr.vcf From owner-ips@ece.cmu.edu Sat May 19 04:41:34 2001 Received: from ece.cmu.edu (ECE.CMU.EDU [128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id EAA10301 for ; Sat, 19 May 2001 04:41:34 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f4J7L6u28410 for ips-outgoing; Sat, 19 May 2001 03:21:06 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from d12lmsgate.de.ibm.com (d12lmsgate.de.ibm.com [195.212.91.199]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f4J7L5H28405 for ; Sat, 19 May 2001 03:21:05 -0400 (EDT) Received: from d12relay01.de.ibm.com (d12relay01.de.ibm.com [9.165.215.22]) by d12lmsgate.de.ibm.com (1.0.0) with ESMTP id JAA160896 for ; Sat, 19 May 2001 09:20:58 +0200 From: julian_satran@il.ibm.com Received: from d12mta02.de.ibm.com (d12mta01_cs0 [9.165.222.237]) by d12relay01.de.ibm.com (8.8.8m3/NCO v4.96) with SMTP id JAA66560 for ; Sat, 19 May 2001 09:20:56 +0200 Received: by d12mta02.de.ibm.com(Lotus SMTP MTA v4.6.5 (863.2 5-20-1999)) id C1256A51.00285BD1 ; Sat, 19 May 2001 09:20:49 +0200 X-Lotus-FromDomain: IBMIL@IBMDE To: ips@ece.cmu.edu Message-ID: Date: Sat, 19 May 2001 10:26:29 +0300 Subject: Re: iSCSI: Canonical Targets Mime-Version: 1.0 Content-type: text/plain; charset=us-ascii Content-Disposition: inline Sender: owner-ips@ece.cmu.edu Precedence: bulk Jim, The canonical attribute is associated with strictly conforming to a rule (as in canonical form for expressions or canon in music, or canonical religious text). I would use the term generic-name or class-name for it. But again the objection on the name was only minor. I strongly believe this is a service-delivery discovery item and we are repeating an old mistake - creating a specialized solution for a general problem. Any discovery service would be better than placing this in the iSCSI realm. Julo "Jim Hafner" on 14-05-2001 18:14:31 Please respond to "Jim Hafner" To: Julian Satran/Haifa/IBM@IBMIL cc: ips@ece.cmu.edu Subject: Re: iSCSI: Canonical Targets Julo, You object to the term "canonical"? My dictionary says canonical is "conforming to a general rule", but perhaps that isn't the best term. Would "well-known" would be better? Jim Hafner Julian Satran/Haifa/IBM@IBMIL@ece.cmu.edu on 05-12-2001 09:05:00 AM Sent by: owner-ips@ece.cmu.edu To: ips@ece.cmu.edu cc: Subject: Re: iSCSI: Canonical Targets Why would any discovery service be wiser/stupider than the mechanism we are having in N&D? If the issue of ports is widespread then others have it too and discovery may include all the nice things we have in iSCSI. I don't see any reason iSNS or an LDAP repository can't contain them. My only claim is that this adds weight to iSCSI. As for interoperability - I claim that adding features makes interoperability an issue. Removing them does not. The whole "canonical" (why canonical - there is nothing canonical about it) target idea is good but in another realm - I don't think it belongs to iSCSI - it belongs to discovery be it specific or general. I can leave with the mechanism as it is - but I really feel that we are on the wrong track. Regards, Julo James Smart on 12-05-2001 15:42:09 Please respond to james.smart@trebia.com To: Julian Satran/Haifa/IBM@IBMIL, Mark Bakke cc: ips@ece.cmu.edu Subject: Re: iSCSI: Canonical Targets I'm in agreement with Mark, that canonical target support is mandatory. The key is interoperability. The initiator must be mandated to probe the canonical target and not assume a single target device. If all initiators do not support this type of probing, then interoperability goes out the window. Mark's effort, mandates the support on the target device - which indirectly puts the mandate on the initiator to check, and makes sure there are not confusing answers from devices that otherwise would not respond to the canonical target. Is this replacing discovery - perhaps in some ways. The real issue is allowing i/o traffic to be rerouted to non-IANA TCP port numbers, which is a critical need of the proxy/routers/etc, which have little left in the socket 4-tuple to classify on if rerouting is not allowed.. I would expect the "discovery" (SLP, uPnP,etc) mechanisms would contain the "targets" that the proxy/router/etc, but the expectation is that they would all point at the well-known IANA TCP port number. The canonical processing simply allows the target device to shift session-specific traffic to target-assigned port numbers that can aid it's classification schemes. -- James julian_satran@il.ibm.com wrote: > Mark, > > I am not sure that we want to have your "canonical" target as a mandatory > construct. > It is good for proxies, routers, portals etc.? Don't the have better > mechanisms for discovery. > But why should you force it on some small box that has the name wired and > printed on it's label. > To reach the canonical you have to go through all the "login etc.". > > I think that all goes back to asking - why should we use iSCSI for > discovery? > > SLP, UPnP, Salutation, Jini all attempt discovery using a a general form > (good for iSCSI, Lpr printers etc.) > Can't we take the same path? > > Julo > > Mark Bakke on 12-05-2001 00:04:24 > > Please respond to Mark Bakke > > To: IPS > cc: > Subject: iSCSI: Canonical Targets > > It was pointed out during the interim meeting that the > canonical target is defined somewhat ambiguously, and may > be a bit too flexible for interoperability purposes. This > message is a stake in the ground for defining this a little > more tightly. > > So here are the new canonical target rules. Please let me > know if there are major problems with any of them. I tried > to define them a-la-carte, to make it easier to pick out any > that might cause trouble. > > 1. Each iSCSI implementation MUST include a canonical target. > > 2. A canonical target MUST be accessible at the default, IANA- > assigned TCP port on each IP address on which the iSCSI > implementation is listening for iSCSI connections. > > 3. A canonical target MUST NOT be used for SCSI commands. > > A canonical target is an iSCSI construct only, and does not > have a corresponding SCSI device. This means it may not > be used to access Logical Units. > > A session created to a canonical target is a discovery session > only, and once in full feature phase, is used only for text > commands and asynchronous messages. (Do any other commands make > sense)? > > 4. A device containing a single target MUST provide both the > canonical target and the real target. (This is implied by the > above requirements). > > An initiator connecting to such a device using only its IP address > would first connect to the canonical target, and use SendTargets > to obtain the iSCSI name of the real target. It would then create > a separate session to the real target. Essentially, this means > there's nothing special about a single-target device. > > 5. An iSCSI device MUST provide a unique iSCSI name for each of its > targets. Using the canonical target as a nameless iSCSI target > is not supported. > > 6. We can further specify the order in which the SendTargets response > fields are returned, to simplify things further, e.g. each target > in the SendTargets response MUST return these fields in this order: > > - A TargetName= field > - A TargetAlias= field (value left blank if there's no alias) > - One or more TargetAddress= fields > - Any vendor-specific fields (ignored by standard initiators) > > The above rules will make the behavior of various target implementations > identical, regardless of the number of interfaces or targets they > support. This will reduce the number of end-cases that initiator > implementations will have to handle. > > If we agree on these, we can edit them into the iSCSI and NDT documents. > > Additional questions to send input on: > > 1. Should a non-canonical target respond to a SendTargets command? > > 2. If so, should it respond only with addresses for its own target, > or should it respond with other targets, as a canonical target > might do? > > 3. If not, the initiator must connect to a canonical target to find > the other addresses of a target to which it is already connected; > is the information it has sufficient to do so? (I think the answer > is yes, given canonical requirement #2 above). > > -- > Mark A. Bakke > Cisco Systems > mbakke@cisco.com > 763.398.1054 From owner-ips@ece.cmu.edu Sat May 19 04:45:29 2001 Received: from ece.cmu.edu (ECE.CMU.EDU [128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id EAA10335 for ; Sat, 19 May 2001 04:45:28 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f4J77CN27679 for ips-outgoing; Sat, 19 May 2001 03:07:12 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from d12lmsgate.de.ibm.com (d12lmsgate.de.ibm.com [195.212.91.199]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f4J77BH27675 for ; Sat, 19 May 2001 03:07:11 -0400 (EDT) Received: from d12relay02.de.ibm.com (d12relay02.de.ibm.com [9.165.215.23]) by d12lmsgate.de.ibm.com (1.0.0) with ESMTP id JAA70114 for ; Sat, 19 May 2001 09:07:04 +0200 From: julian_satran@il.ibm.com Received: from d12mta02.de.ibm.com (d12mta01_cs0 [9.165.222.237]) by d12relay02.de.ibm.com (8.8.8m3/NCO v4.96) with SMTP id JAA113060 for ; Sat, 19 May 2001 09:07:00 +0200 Received: by d12mta02.de.ibm.com(Lotus SMTP MTA v4.6.5 (863.2 5-20-1999)) id C1256A51.0027154F ; Sat, 19 May 2001 09:06:53 +0200 X-Lotus-FromDomain: IBMIL@IBMDE To: ips@ece.cmu.edu Message-ID: Date: Sat, 19 May 2001 10:08:16 +0300 Subject: Re: iSCSI: Canonical Targets Mime-Version: 1.0 Content-type: text/plain; charset=us-ascii Content-Disposition: inline Sender: owner-ips@ece.cmu.edu Precedence: bulk Mark, I don't see any iSCSI interoperability issue if you don't have the canonical target. This is a service discovery mechanism that we are mistakenly taking on a specialized track. I assume that many other services "hide" behind portals (print is one good example). We are not going to leverage this commonality if we are attempting a specialized interface. Perhaps the boxes should be mandated to have something to enable discovery. My only claim is that this is not an "iSCSI item" it is a service discovery issue and it should be handled in that realm and enable interoperability with other service discovery mechanisms. Julo Mark Bakke on 14-05-2001 16:13:59 Please respond to Mark Bakke To: james.smart@trebia.com cc: Julian Satran/Haifa/IBM@IBMIL, ips@ece.cmu.edu Subject: Re: iSCSI: Canonical Targets James- Your argument for interoperability is correct. Please see my comments below on the other discovery mechanisms. James Smart wrote: > > I'm in agreement with Mark, that canonical target support is mandatory. The > key is interoperability. The initiator must be mandated to probe the > canonical target and not assume a single target device. If all initiators do > not support this type of probing, then interoperability goes out the window. > Mark's effort, mandates the support on the target device - which indirectly > puts the mandate on the initiator to check, and makes sure there are not > confusing answers from devices that otherwise would not respond to the > canonical target. Correct. The danger in not supporting the canonical target on all devices is that initiators may then choose not to look at it. Even in an environment without proxies and gateways, any medium-sized disk array should be able to support more than one logical target. > Is this replacing discovery - perhaps in some ways. The real issue is > allowing i/o traffic to be rerouted to non-IANA TCP port numbers, which is a > critical need of the proxy/routers/etc, which have little left in the socket > 4-tuple to classify on if rerouting is not allowed.. I would expect the Actually, by defining several logical targets, a single IP address + IANA TCP port can be shared by all. This is a direction I suspect many implementors will take. > "discovery" (SLP, uPnP,etc) mechanisms would contain the "targets" that the > proxy/router/etc, but the expectation is that they would all point at the > well-known IANA TCP port number. The canonical processing simply allows the The SLP template includes the TCP port number on which the target is listening. If a target is discovered using SLP, the initiator could skip using SendTargets on its address. Although we are not considering using it, SSDP (used by UPnP) returns a URL, which could include a TCP port number. > target device to shift session-specific traffic to target-assigned port > numbers that can aid it's classification schemes. -- Mark > -- James > > julian_satran@il.ibm.com wrote: > > > Mark, > > > > I am not sure that we want to have your "canonical" target as a mandatory > > construct. > > It is good for proxies, routers, portals etc.? Don't the have better > > mechanisms for discovery. > > But why should you force it on some small box that has the name wired and > > printed on it's label. > > To reach the canonical you have to go through all the "login etc.". > > > > I think that all goes back to asking - why should we use iSCSI for > > discovery? > > > > SLP, UPnP, Salutation, Jini all attempt discovery using a a general form > > (good for iSCSI, Lpr printers etc.) > > Can't we take the same path? > > > > Julo > > > > Mark Bakke on 12-05-2001 00:04:24 > > > > Please respond to Mark Bakke > > > > To: IPS > > cc: > > Subject: iSCSI: Canonical Targets > > > > It was pointed out during the interim meeting that the > > canonical target is defined somewhat ambiguously, and may > > be a bit too flexible for interoperability purposes. This > > message is a stake in the ground for defining this a little > > more tightly. > > > > So here are the new canonical target rules. Please let me > > know if there are major problems with any of them. I tried > > to define them a-la-carte, to make it easier to pick out any > > that might cause trouble. > > > > 1. Each iSCSI implementation MUST include a canonical target. > > > > 2. A canonical target MUST be accessible at the default, IANA- > > assigned TCP port on each IP address on which the iSCSI > > implementation is listening for iSCSI connections. > > > > 3. A canonical target MUST NOT be used for SCSI commands. > > > > A canonical target is an iSCSI construct only, and does not > > have a corresponding SCSI device. This means it may not > > be used to access Logical Units. > > > > A session created to a canonical target is a discovery session > > only, and once in full feature phase, is used only for text > > commands and asynchronous messages. (Do any other commands make > > sense)? > > > > 4. A device containing a single target MUST provide both the > > canonical target and the real target. (This is implied by the > > above requirements). > > > > An initiator connecting to such a device using only its IP address > > would first connect to the canonical target, and use SendTargets > > to obtain the iSCSI name of the real target. It would then create > > a separate session to the real target. Essentially, this means > > there's nothing special about a single-target device. > > > > 5. An iSCSI device MUST provide a unique iSCSI name for each of its > > targets. Using the canonical target as a nameless iSCSI target > > is not supported. > > > > 6. We can further specify the order in which the SendTargets response > > fields are returned, to simplify things further, e.g. each target > > in the SendTargets response MUST return these fields in this order: > > > > - A TargetName= field > > - A TargetAlias= field (value left blank if there's no alias) > > - One or more TargetAddress= fields > > - Any vendor-specific fields (ignored by standard initiators) > > > > The above rules will make the behavior of various target implementations > > identical, regardless of the number of interfaces or targets they > > support. This will reduce the number of end-cases that initiator > > implementations will have to handle. > > > > If we agree on these, we can edit them into the iSCSI and NDT documents. > > > > Additional questions to send input on: > > > > 1. Should a non-canonical target respond to a SendTargets command? > > > > 2. If so, should it respond only with addresses for its own target, > > or should it respond with other targets, as a canonical target > > might do? > > > > 3. If not, the initiator must connect to a canonical target to find > > the other addresses of a target to which it is already connected; > > is the information it has sufficient to do so? (I think the answer > > is yes, given canonical requirement #2 above). > > > > -- > > Mark A. Bakke > > Cisco Systems > > mbakke@cisco.com > > 763.398.1054 -- Mark A. Bakke Cisco Systems mbakke@cisco.com 763.398.1054 From owner-ips@ece.cmu.edu Sat May 19 06:49:24 2001 Received: from ece.cmu.edu (ECE.CMU.EDU [128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id GAA10878 for ; Sat, 19 May 2001 06:49:24 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f4J9EF014342 for ips-outgoing; Sat, 19 May 2001 05:14:15 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from d12lmsgate-3.de.ibm.com (d12lmsgate-3.de.ibm.com [195.212.91.201]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f4J9EEH14338 for ; Sat, 19 May 2001 05:14:14 -0400 (EDT) Received: from d12relay01.de.ibm.com (d12relay01.de.ibm.com [9.165.215.22]) by d12lmsgate-3.de.ibm.com (1.0.0) with ESMTP id LAA85622; Sat, 19 May 2001 11:14:06 +0200 From: biran@il.ibm.com Received: from d12mta05.de.ibm.com (d12mta05_cs0 [9.165.222.239]) by d12relay01.de.ibm.com (8.8.8m3/NCO v4.96) with SMTP id LAA177312; Sat, 19 May 2001 11:14:03 +0200 Received: by d12mta05.de.ibm.com(Lotus SMTP MTA v4.6.5 (863.2 5-20-1999)) id C1256A51.0032B7B8 ; Sat, 19 May 2001 11:13:58 +0200 X-Lotus-FromDomain: IBMIL@IBMDE To: Black_David@emc.com cc: aboba@internaut.com, ips@ece.cmu.edu Message-ID: Date: Sat, 19 May 2001 12:08:16 +0300 Subject: RE: iSCSI Security rough consensus Mime-Version: 1.0 Content-type: text/plain; charset=us-ascii Content-Disposition: inline Sender: owner-ips@ece.cmu.edu Precedence: bulk David, >> 1. Do we need to support negotiation of SRP prime modulus/generator >> groups from within the standard set? > Not "negotiation" per se. I'd pick a small (tasteful) > number of them, make them all "MUST implement", have > the Initiator pick one and announce it via iSCSI text > key(s) and/or value(s) sent as part of the initial message. > If the Target doesn't like it for some reason (e.g., we > exercised bad taste [in 20/20 hindsight] and the announced > one is insufficiently secure), it indicates its dissatisfaction > by terminating the login, but "SHOULD NOT" do this > without a very good reason, as a general strategy of > retrying with a different modulus/generator at the Initiator > in response to a Target reject of this form opens up > man-in-the-middle attacks on the negotiation > to force use of a "weaker" modulus/group (from the > attacker's perspective). By the 06 spec for SRP (Appendix A ), the target simply sends the modulus/generator in the second message of the SRP sequence. This is very reasonable for password-based scheme where the target keeps password verifiers DB, already computed by specific modulus/generator. It's similar to how telnet is using SRP (RFC 2944). Till now I didn't hear any comment for negotiation-enhancement of the SRP sequence. >> 2. Do we need to generate keying material for Phase 1 as well as Phase 2 >> SAs? > Phase 2 only, but see next item for an approach to rekeying > a Phase 2 SA without using a Phase 1 SA. The way I see it, SRP_WITH_ESP_KEYING will be performed for each new iSCSI connection, producing keying material for that (and only that) TCP connection. So if you want to relate it to the ISAKMP/IKE notion of phases (is it necessary ?), it indeed might be just for rekeying aspect. Regards, Ofer Ofer Biran Storage and Systems Technology IBM Research Lab in Haifa biran@il.ibm.com 972-4-8296253 From owner-ips@ece.cmu.edu Sat May 19 12:08:58 2001 Received: from ece.cmu.edu (ECE.CMU.EDU [128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id MAA13778 for ; Sat, 19 May 2001 12:08:58 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f4JEHfI00405 for ips-outgoing; Sat, 19 May 2001 10:17:41 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from siaar1aa.compuserve.com (siaar1aa.compuserve.com [149.174.40.9]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f4JEHeH00401 for ; Sat, 19 May 2001 10:17:40 -0400 (EDT) Received: (from mailgate@localhost) by siaar1aa.compuserve.com (8.9.3/8.9.3/SUN-REL-1.3) id KAA02072 for ips@ece.cmu.edu; Sat, 19 May 2001 10:17:34 -0400 (EDT) Received: from compuserve.com (chi-tgn-gvq-vty135.as.wcom.net [216.192.150.135]) by siaar1aa.compuserve.com (8.9.3/8.9.3/SUN-REL-1.3) with ESMTP id KAA02063; Sat, 19 May 2001 10:17:20 -0400 (EDT) Message-ID: <3B068122.96123F80@compuserve.com> Date: Sat, 19 May 2001 09:20:18 -0500 From: Ralph Weber Reply-To: ENDL_TX@computer.org X-Mailer: Mozilla 4.7 [en]C-CCK-MCD NSCPCD47 (Win98; I) X-Accept-Language: en,pdf MIME-Version: 1.0 To: IPS Reflector CC: Sudhir Srinivasan Subject: Re: FCIP - Proposed changes in support of common FC encapsulation References: <63616B261CEFD411834D00D0B7A86CF7048FC3@lucy.TREBIA.COM> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-ips@ece.cmu.edu Precedence: bulk Content-Transfer-Encoding: 7bit Sudhir, Do you think it would be better to delete the "discarding frames with header errors" part entirely or to reword it to be more of a reference to 8.1? Thanks. Ralph... Sudhir Srinivasan wrote: > > Only one comment on this. You suggest the following text: > > "The FCIP receiving > device MUST reverse the FC frame encapsulation process, verify the > correctness of the header discarding frames with header errors, and > deliver the FC frames to the correct FC ports connected to the FCIP > device." > > The part about "discarding frames with header errors" is > confusing because FCIP header errors will result in loss of > synchronization and Section 8.1 will then define the action > to be taken. > > - Sudhir > > -----Original Message----- > From: Ralph Weber [mailto:ralphoweber@compuserve.com] > Sent: Thursday, May 17, 2001 5:51 PM > To: IPS Reflector > Subject: FCIP - Proposed changes in support of common FC encapsulation > > In the 3rd paragraph of section 4.3, replace: > > Each FCIP data frame is built by adding an FCIP header to one FC > frame delivered to the FCIP endpoint for transport. The FCIP data > frames are handed in their entirety to TCP; TCP is responsible for > delivering the same series of FCIP data frames to the receiving > side in the same order as they are transmitted by the sending FCIP > device. The FCIP device MUST find the FCIP headers and deliver > the FC frames wrapped inside the FCIP data frames to the correct > FC ports connected to the FCIP device. > > with: > > An FCIP data frame is built by encapsulating one FC frame delivered > to the FCIP endpoint for transport using the encapsulation format > described in the common FC Frame Encapsulation [23] and in section > 5.2 of this document. The FCIP data frames are handed in their > entirety to TCP; TCP is responsible for delivering the same series > of FCIP data frames to the receiving side in the same order as > they are transmitted by the sending FCIP device. The FCIP receiving > device MUST reverse the FC frame encapsulation process, verify the > correctness of the header discarding frames with header errors, and > deliver the FC frames to the correct FC ports connected to the FCIP > device. From owner-ips@ece.cmu.edu Sat May 19 16:56:36 2001 Received: from ece.cmu.edu (ECE.CMU.EDU [128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id QAA15528 for ; Sat, 19 May 2001 16:56:36 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f4JJGd317649 for ips-outgoing; Sat, 19 May 2001 15:16:39 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from d12lmsgate-2.de.ibm.com (d12lmsgate-2.de.ibm.com [195.212.91.200]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f4JJGcH17643 for ; Sat, 19 May 2001 15:16:38 -0400 (EDT) Received: from d12relay01.de.ibm.com (d12relay01.de.ibm.com [9.165.215.22]) by d12lmsgate-2.de.ibm.com (1.0.0) with ESMTP id VAA257558 for ; Sat, 19 May 2001 21:16:32 +0200 From: julian_satran@il.ibm.com Received: from d12mta02.de.ibm.com (d12mta01_cs0 [9.165.222.237]) by d12relay01.de.ibm.com (8.8.8m3/NCO v4.96) with SMTP id VAA226314 for ; Sat, 19 May 2001 21:16:28 +0200 Received: by d12mta02.de.ibm.com(Lotus SMTP MTA v4.6.5 (863.2 5-20-1999)) id C1256A51.0069DC0C ; Sat, 19 May 2001 21:16:16 +0200 X-Lotus-FromDomain: IBMIL@IBMDE To: ips@ece.cmu.edu Message-ID: Date: Sat, 19 May 2001 22:21:58 +0300 Subject: Re: iSCSI: Service unavailable Mime-Version: 1.0 Content-type: text/plain; charset=us-ascii Content-Disposition: inline Sender: owner-ips@ece.cmu.edu Precedence: bulk OK - Julo "Eddy Quicksall" on 19-05-2001 16:01:05 Please respond to "Eddy Quicksall" To: Julian Satran/Haifa/IBM@IBMIL, ips@ece.cmu.edu cc: Subject: iSCSI: Service unavailable The status of "Service unavailable" is useful for many cases. I would prefer that the statement "This is usually due to maintenance" be removed from the comment. So it would simply read: ----------------------------------------------------------------- Service | 0301 | 1 | The iSCSI service or target is not Unavailable | | | currently operational. ----------------------------------------------------------------- What do you think? mailto:Eddy@Quicksall.com From owner-ips@ece.cmu.edu Sat May 19 17:00:04 2001 Received: from ece.cmu.edu (ECE.CMU.EDU [128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id RAA15579 for ; Sat, 19 May 2001 17:00:04 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f4JJXac18776 for ips-outgoing; Sat, 19 May 2001 15:33:36 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from d12lmsgate-2.de.ibm.com (d12lmsgate-2.de.ibm.com [195.212.91.200]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f4JJXYH18765 for ; Sat, 19 May 2001 15:33:34 -0400 (EDT) Received: from d12relay01.de.ibm.com (d12relay01.de.ibm.com [9.165.215.22]) by d12lmsgate-2.de.ibm.com (1.0.0) with ESMTP id VAA129266 for ; Sat, 19 May 2001 21:33:28 +0200 From: julian_satran@il.ibm.com Received: from d12mta02.de.ibm.com (d12mta01_cs0 [9.165.222.237]) by d12relay01.de.ibm.com (8.8.8m3/NCO v4.96) with SMTP id VAA267084 for ; Sat, 19 May 2001 21:33:24 +0200 Received: by d12mta02.de.ibm.com(Lotus SMTP MTA v4.6.5 (863.2 5-20-1999)) id C1256A51.006B6BDC ; Sat, 19 May 2001 21:33:19 +0200 X-Lotus-FromDomain: IBMIL@IBMDE To: ips@ece.cmu.edu Message-ID: Date: Sat, 19 May 2001 22:39:04 +0300 Subject: Re: Mime-Version: 1.0 Content-type: text/plain; charset=us-ascii Content-Disposition: inline Sender: owner-ips@ece.cmu.edu Precedence: bulk Will do - Julo "Eddy Quicksall" on 18-05-2001 18:37:20 Please respond to "Eddy Quicksall" To: Julian Satran/Haifa/IBM@IBMIL, ips@ece.cmu.edu cc: Subject: Section 2.4 SCSI Response says: +---------------+---------------+- 36| ExpDataSN or Reserved (0) +---------------+---------------+- 40| R2TExpDataSN or Reserved (0) +---------------+---------------+- Can we add a phrase that says "This field is reserved if S is 0"? That way, it is very clear when it is reserved. mailto:Eddy@Quicksall.com From owner-ips@ece.cmu.edu Sat May 19 20:55:03 2001 Received: from ece.cmu.edu (ECE.CMU.EDU [128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id UAA17292 for ; Sat, 19 May 2001 20:55:03 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f4JN1AE01415 for ips-outgoing; Sat, 19 May 2001 19:01:10 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from chmls06.mediaone.net (chmls06.mediaone.net [24.147.1.144]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f4JD12H26202 for ; Sat, 19 May 2001 09:01:02 -0400 (EDT) Received: from eddylaptop (h00e098880aa4.ne.mediaone.net [66.31.75.244]) by chmls06.mediaone.net (8.11.1/8.11.1) with SMTP id f4JD0w810001; Sat, 19 May 2001 09:00:59 -0400 (EDT) From: "Eddy Quicksall" To: , Subject: iSCSI: Service unavailable Date: Sat, 19 May 2001 09:01:05 -0400 Message-ID: <000001c0e063$c4aadcd0$0100a8c0@eddylaptop> MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 (Normal) X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook CWS, Build 9.0.2416 (9.0.2911.0) Importance: Normal X-MimeOLE: Produced By Microsoft MimeOLE V5.00.3018.1300 Sender: owner-ips@ece.cmu.edu Precedence: bulk Content-Transfer-Encoding: 7bit The status of "Service unavailable" is useful for many cases. I would prefer that the statement "This is usually due to maintenance" be removed from the comment. So it would simply read: ----------------------------------------------------------------- Service | 0301 | 1 | The iSCSI service or target is not Unavailable | | | currently operational. ----------------------------------------------------------------- What do you think? mailto:Eddy@Quicksall.com From owner-ips@ece.cmu.edu Sun May 20 10:05:31 2001 Received: from ece.cmu.edu ([128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id KAA10517 for ; Sun, 20 May 2001 10:05:30 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f4KBw2D23309 for ips-outgoing; Sun, 20 May 2001 07:58:02 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from lucy.trebia.com ([65.192.191.151]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f4KBvtH23300 for ; Sun, 20 May 2001 07:58:00 -0400 (EDT) Received: by lucy.TREBIA.COM with Internet Mail Service (5.5.2653.19) id ; Sun, 20 May 2001 08:01:52 -0400 Message-ID: <63616B261CEFD411834D00D0B7A86CF7048FC7@lucy.TREBIA.COM> From: Sudhir Srinivasan To: "'ENDL_TX@computer.org'" , IPS Reflector Cc: Sudhir Srinivasan Subject: RE: FCIP - Proposed changes in support of common FC encapsulation Date: Sun, 20 May 2001 08:01:51 -0400 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2653.19) Content-Type: text/plain; charset="iso-8859-1" Sender: owner-ips@ece.cmu.edu Precedence: bulk Ralph, My preference is to reference Section 8.1 - here's a suggestion: "The FCIP receiving device MUST reverse the FC frame encapsulation process, verify the correctness of the header and deliver the FC frames to the correct FC ports connected to the FCIP device. When header errors are encountered in FCIP data frames, the FCIP device MUST take corrective action in accordance with Section 8.1." - Sudhir -----Original Message----- From: Ralph Weber [mailto:ralphoweber@compuserve.com] Sent: Saturday, May 19, 2001 10:20 AM To: IPS Reflector Cc: Sudhir Srinivasan Subject: Re: FCIP - Proposed changes in support of common FC encapsulation Sudhir, Do you think it would be better to delete the "discarding frames with header errors" part entirely or to reword it to be more of a reference to 8.1? Thanks. Ralph... Sudhir Srinivasan wrote: > > Only one comment on this. You suggest the following text: > > "The FCIP receiving > device MUST reverse the FC frame encapsulation process, verify the > correctness of the header discarding frames with header errors, and > deliver the FC frames to the correct FC ports connected to the FCIP > device." > > The part about "discarding frames with header errors" is > confusing because FCIP header errors will result in loss of > synchronization and Section 8.1 will then define the action > to be taken. > > - Sudhir > > -----Original Message----- > From: Ralph Weber [mailto:ralphoweber@compuserve.com] > Sent: Thursday, May 17, 2001 5:51 PM > To: IPS Reflector > Subject: FCIP - Proposed changes in support of common FC encapsulation > > In the 3rd paragraph of section 4.3, replace: > > Each FCIP data frame is built by adding an FCIP header to one FC > frame delivered to the FCIP endpoint for transport. The FCIP data > frames are handed in their entirety to TCP; TCP is responsible for > delivering the same series of FCIP data frames to the receiving > side in the same order as they are transmitted by the sending FCIP > device. The FCIP device MUST find the FCIP headers and deliver > the FC frames wrapped inside the FCIP data frames to the correct > FC ports connected to the FCIP device. > > with: > > An FCIP data frame is built by encapsulating one FC frame delivered > to the FCIP endpoint for transport using the encapsulation format > described in the common FC Frame Encapsulation [23] and in section > 5.2 of this document. The FCIP data frames are handed in their > entirety to TCP; TCP is responsible for delivering the same series > of FCIP data frames to the receiving side in the same order as > they are transmitted by the sending FCIP device. The FCIP receiving > device MUST reverse the FC frame encapsulation process, verify the > correctness of the header discarding frames with header errors, and > deliver the FC frames to the correct FC ports connected to the FCIP > device. From owner-ips@ece.cmu.edu Sun May 20 22:51:04 2001 Received: from ece.cmu.edu ([128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id WAA17139 for ; Sun, 20 May 2001 22:51:04 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f4L0sTB06477 for ips-outgoing; Sun, 20 May 2001 20:54:29 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from hotmail.com (f79.law11.hotmail.com [64.4.17.79]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f4L0sSH06472 for ; Sun, 20 May 2001 20:54:28 -0400 (EDT) Received: from mail pickup service by hotmail.com with Microsoft SMTPSVC; Sun, 20 May 2001 17:54:22 -0700 Received: from 66.31.75.244 by lw11fd.law11.hotmail.msn.com with HTTP; Mon, 21 May 2001 00:54:22 GMT X-Originating-IP: [66.31.75.244] From: "Eddy Quicksall" To: julian_satran@il.ibm.com, ips@ece.cmu.edu Subject: RE: Another shot at codes and please comment Date: Sun, 20 May 2001 20:54:22 -0400 Mime-Version: 1.0 Content-Type: text/plain; format=flowed Message-ID: X-OriginalArrivalTime: 21 May 2001 00:54:22.0138 (UTC) FILETIME=[92D581A0:01C0E190] Sender: owner-ips@ece.cmu.edu Precedence: bulk I vote ok on these OpCodes. BTW, I would prefer that the opcode field is used strictly for the opcode (i.e. not have the X and I bits). However, if this has already been discussed or if there is just no other place for the bits, I will submit. Eddy ----Original Message Follows---- From: julian_satran@il.ibm.com To: ips@ece.cmu.edu Subject: RE: Another shot at codes and please comment Date: Sat, 19 May 2001 08:51:20 +0300 Venkat, This seems to be the popular choice for 07 (2 voices for, none against and silence from the bored majority). Julo Venkat Rangan on 19-05-2001 05:12:17 Please respond to Venkat Rangan To: Julian Satran/Haifa/IBM@IBMIL, ips@ece.cmu.edu cc: Subject: RE: Another shot at codes and please comment Julian, What is the state of this change? Is this going to be in draft 7, or is it in draft 6? Thanks, Venkat Rangan Rhapsody Networks Inc. http://www.rhapsodynetworks.com -----Original Message----- From: julian_satran@il.ibm.com [mailto:julian_satran@il.ibm.com] Sent: Tuesday, May 01, 2001 6:50 AM To: ips@ece.cmu.edu Subject: Another shot at codes and please comment Here is a another version of the opcodes part: 1.1.1.1 Opcode The Opcode indicates what type of iSCSI PDU the header encapsulates. The Opcodes are divided into two categories: initiator opcodes and target opcodes. Initiator opcodes are in PDUs sent by the initiators (request PDUs), and target opcodes are in PDUs sent by the target (response PDUs). Initiators MUST NOT use target opcodes and targets MUST NOT use initiator opcodes. Valid initiator opcodes defined in this specification are: 0x00 NOP-Out (from initiator to target) 0x01 SCSI Command (encapsulates a SCSI Command Descriptor Block) 0x02 SCSI Task Management Command 0x03 Login Command 0x04 Text Command 0x05 SCSI Data-out (for WRITE operations) 0x06 Logout Command 0x10 SNACK Request Valid target opcodes are: 0x20 NOP-In (from target to initiator) 0x21 SCSI Response (contains SCSI status and possibly sense information or other response information) 0x22 SCSI Task Management Response 0x23 Login Response 0x24 Text Response 0x25 SCSI Data-in (for READ operations) 0x26 Logout Response 0x31 Ready To Transfer (R2T - sent by target to initiator when it is ready to receive data from initiator) 0x32 Asynchronous Message (sent by target to initiator to indicate certain special conditions) 0x3f Reject Initiator opcodes 0x1c-0x1e and target opcodes 0x3c-0x3e are vendor specific codes. Please comment, Julo _________________________________________________________________ Get your FREE download of MSN Explorer at http://explorer.msn.com From owner-ips@ece.cmu.edu Mon May 21 02:35:32 2001 Received: from ece.cmu.edu ([128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id CAA01165 for ; Mon, 21 May 2001 02:35:32 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f4L4eor19370 for ips-outgoing; Mon, 21 May 2001 00:40:50 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from msgbas1.cos.agilent.com (msgbas1x.cos.agilent.com [192.6.9.33]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f4L4enH19365 for ; Mon, 21 May 2001 00:40:49 -0400 (EDT) Received: from msgrel1.and.agilent.com (msgrel1.and.agilent.com [130.30.33.104]) by msgbas1.cos.agilent.com (Postfix) with ESMTP id 513E9C05 for ; Sun, 20 May 2001 22:40:48 -0600 (MDT) Received: from rtl.rose.agilent.com (rtl.rose.agilent.com [156.140.232.231]) by msgrel1.and.agilent.com (Postfix) with ESMTP id 5448F11F for ; Mon, 21 May 2001 00:40:47 -0400 (EDT) Received: from mail.rose.agilent.com (mailsrv@bellhop [156.140.233.51]) by rtl.rose.agilent.com (8.9.3 (PHNE_18979)/8.9.3 SMKit7.1.0) with ESMTP id VAA25231 for ; Sun, 20 May 2001 21:40:46 -0700 (PDT) Received: from agilent.com (cos1nai129071.cos.agilent.com [141.184.129.71]) by mail.rose.agilent.com (Netscape Messaging Server 3.6) with ESMTP id AAA1536 for ; Sun, 20 May 2001 21:40:42 -0700 Message-ID: <3B089C47.D48A5230@agilent.com> Date: Sun, 20 May 2001 21:40:39 -0700 From: "Matt Wakeley" Reply-To: Matt Wakeley Organization: Agilent Technologies X-Mailer: Mozilla 4.77 [en] (Windows NT 5.0; U) X-Accept-Language: en MIME-Version: 1.0 To: ips@ece.cmu.edu Subject: Re: TCP Framing References: <6BD67FFB937FD411A04F00D0B74FE87802A0906B@xrose06.rose.hp.com> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-ips@ece.cmu.edu Precedence: bulk Content-Transfer-Encoding: 7bit "KRUEGER,MARJORIE (HP-Roseville,ex1)" wrote: > Isn't it actually that a 10Gig implementation couldn't interoperate *at > 10Gig speeds* without some sort of frame-sync method - they would be > considerably "slowed-down" (or forced to have large TCP rcv buffers). This > doesn't mean 'no interoperation', just no 10 Gig thruput? Marj, The issue is, there will be lots of 1Gig applications out there - perhaps one on every desktop. All these 1Gig applications will be aggregated through switches to 10Gig servers. Now, suppose you are the vendor of this server. Are you going to say in your product specification that your product will only interoperate with applications running new TCP stacks with this new framing solution? I'll bet not. So, unless there is a framing solution mandated by the iSCSI spec (under the control of the iSCSI spec), you will have to implement the full TCP buffering solution, and will have lost all the cost and space benefits of a framing solution. To address your other comments: > The reason people are working so hard on a TCP-level framing solution is: > > - marker scheme is so krufty and software soln's are penalized (they > probably wouldn't be able to drive 10Gig speeds if 'markers' are mandated) 10Gig solutions will not be implemented in software, markers or not. But due to aggregation, the 10Gig solutions will require the 1Gig solutions to implement framing (markers). > - there are other TCP applications that benefit from a common frame-sync > method. This makes these changes more likely to be incorporated into > software stacks (OS releases). Yeah, but they will require further work, and will probably be adopted after the new frame-sync method. And I still question how you can require a optional enhancement to TCP to make your new protocol work. > There's already grumbling in IETF that 'that IPS team' "seems to exhibit a > strong tendency to avoid > reusing existing IETF standards that reasonably fit their needs". It would > benefit us to support the common TCP framing solution and contribute to it's > early adoption. They are only grumbling because we didn't embrace their new transport with open arms. > Even if the TCP framing RFC wasn't mandated by iSCSI, a vendor could require > it's implementation end-to-end to guarantee 10Gig speeds? See my comments above. Do you want to be the vendor that comes out with a solution that is more restrictive than your competition? If the framing solution is part of the iSCSI standard, then everyone will have to implement it to claim iSCSI compliancy. -Matt From owner-ips@ece.cmu.edu Mon May 21 05:51:33 2001 Received: from ece.cmu.edu ([128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id FAA04438 for ; Mon, 21 May 2001 05:51:33 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f4L87XN00806 for ips-outgoing; Mon, 21 May 2001 04:07:33 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from e31.bld.us.ibm.com (e31.co.us.ibm.com [32.97.110.129]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f4L87VH00793 for ; Mon, 21 May 2001 04:07:31 -0400 (EDT) Received: from westrelay02.boulder.ibm.com (westrelay02.boulder.ibm.com [9.99.140.23]) by e31.bld.us.ibm.com (8.9.3/8.9.3) with ESMTP id DAA70384 for ; Mon, 21 May 2001 03:59:55 -0400 Received: from f4n49e (d03nm065h.boulder.ibm.com [9.99.140.49]) by westrelay02.boulder.ibm.com (8.11.1/NCO v4.96.1.0) with ESMTP id f4L87UR156206 for ; Mon, 21 May 2001 02:07:30 -0600 X-Priority: 1 (High) Importance: Normal Subject: Re: TCP Framing To: ips@ece.cmu.edu X-Mailer: Lotus Notes Release 5.0.3 (Intl) 21 March 2000 Message-ID: From: "John Hufferd" Date: Mon, 21 May 2001 01:07:09 -0700 X-MIMETrack: Serialize by Router on D03NM065/03/M/IBM(Release 5.0.6 |December 14, 2000) at 05/21/2001 02:07:29 AM MIME-Version: 1.0 Content-type: text/plain; charset=us-ascii Sender: owner-ips@ece.cmu.edu Precedence: bulk I must agree with Matt. And in addition to what Matt said, about the 1 Gig Software, we must remember that many Desktops and Laptops, will be able to operate with CAT 5 Cable at 100 Mb/s, and some will be happy with that, others using the same CAT 5 cable will be able to operate comfortably at 300+ Mb/s (there will not be many that will be comfortable at 1000 Mb/s, but most will be very happy with 300 MB/S). So what you will see across a business campus is software implementations running on Desktops and Laptops at an assortments of speeds, from 100 Mb/s, through 300+ Mb/s. There of course will be Hardware HBAs in some Desktops, running on the same Networks (at Gigabit Speeds), but you should expect to see lots of SW implementations running at these lesser speeds. To be really successful and fulfill the real promise of iSCSI, we want both the HW and the SW implementations to function well. With the 100 Mb/s to 300+ Mb/s many of these Desktops and Laptops will be getting better performance from the Network then from their local Hard Disk. They will have more then enough CPU power to support 300+ Mb/s and not be noticed in the individual's system. As Matt was saying these different, systems will get multiplexed together into the target device, and it is important that the target device be able to support both HW and Software implementations with a minimum of cost, on the targets 10 Gigabit iSCSI Portals. Though I like the Warp support, I think it is unlikely that the TCP/IP stacks of all the different Desktops and Laptops, etc. will be updated to support that, in any acceptable time frame. I think we must depend on Markers to insure that everything can operate at top speed, and at the lowest cost. That is why I think that the Markers should be "Must Implement", but "Optional to use". . . . John L. Hufferd Senior Technical Staff Member (STSM) IBM/SSG San Jose Ca (408) 256-0403, Tie: 276-0403, eFax: (408) 904-4688 Internet address: hufferd@us.ibm.com "Matt Wakeley" @ece.cmu.edu on 05/20/2001 09:40:39 PM Please respond to Matt Wakeley Sent by: owner-ips@ece.cmu.edu To: ips@ece.cmu.edu cc: Subject: Re: TCP Framing "KRUEGER,MARJORIE (HP-Roseville,ex1)" wrote: > Isn't it actually that a 10Gig implementation couldn't interoperate *at > 10Gig speeds* without some sort of frame-sync method - they would be > considerably "slowed-down" (or forced to have large TCP rcv buffers). This > doesn't mean 'no interoperation', just no 10 Gig thruput? Marj, The issue is, there will be lots of 1Gig applications out there - perhaps one on every desktop. All these 1Gig applications will be aggregated through switches to 10Gig servers. Now, suppose you are the vendor of this server. Are you going to say in your product specification that your product will only interoperate with applications running new TCP stacks with this new framing solution? I'll bet not. So, unless there is a framing solution mandated by the iSCSI spec (under the control of the iSCSI spec), you will have to implement the full TCP buffering solution, and will have lost all the cost and space benefits of a framing solution. To address your other comments: > The reason people are working so hard on a TCP-level framing solution is: > > - marker scheme is so krufty and software soln's are penalized (they > probably wouldn't be able to drive 10Gig speeds if 'markers' are mandated) 10Gig solutions will not be implemented in software, markers or not. But due to aggregation, the 10Gig solutions will require the 1Gig solutions to implement framing (markers). > - there are other TCP applications that benefit from a common frame-sync > method. This makes these changes more likely to be incorporated into > software stacks (OS releases). Yeah, but they will require further work, and will probably be adopted after the new frame-sync method. And I still question how you can require a optional enhancement to TCP to make your new protocol work. > There's already grumbling in IETF that 'that IPS team' "seems to exhibit a > strong tendency to avoid > reusing existing IETF standards that reasonably fit their needs". It would > benefit us to support the common TCP framing solution and contribute to it's > early adoption. They are only grumbling because we didn't embrace their new transport with open arms. > Even if the TCP framing RFC wasn't mandated by iSCSI, a vendor could require > it's implementation end-to-end to guarantee 10Gig speeds? See my comments above. Do you want to be the vendor that comes out with a solution that is more restrictive than your competition? If the framing solution is part of the iSCSI standard, then everyone will have to implement it to claim iSCSI compliancy. -Matt From owner-ips@ece.cmu.edu Mon May 21 12:07:32 2001 Received: from ece.cmu.edu ([128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id MAA12004 for ; Mon, 21 May 2001 12:07:32 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f4LDqwG02228 for ips-outgoing; Mon, 21 May 2001 09:52:58 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from sandmail.sandburst.com (sandburst-gw.bstn-gw02.ma.us.intelilink.net [216.57.129.34]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f4LDquH02224 for ; Mon, 21 May 2001 09:52:56 -0400 (EDT) Received: from cs.uchicago.edu (dynamite-38.sandburst.com [172.16.5.38]) by sandmail.sandburst.com (Postfix) with ESMTP id CB05294006 for ; Mon, 21 May 2001 09:52:40 -0400 (EDT) To: ips@ece.cmu.edu Subject: Re: TCP Framing (considered helpful?) In-Reply-To: Message from "John Hufferd" of "Mon, 21 May 2001 01:07:09 PDT." References: Date: Mon, 21 May 2001 09:50:41 -0400 From: Stephen Bailey Message-Id: <20010521135240.CB05294006@sandmail.sandburst.com> Sender: owner-ips@ece.cmu.edu Precedence: bulk John, > I think we must depend on Markers to insure that everything can operate at > top speed, and at the lowest cost. A key question is whether markers actually ensure that everything operates at `top speed, and at the lowest cost'. Matt thinks so. I (and, presumably those who wrote the framing document) think not. My issue is not even with `lowest cost'. I don't believe markers will allow you to run at top speed. Specifically: 1) I doubt the feasibility of implementing the control required for an eddy buffer (where you store data you can't place) at 10G. Admittedly, the validity of this claim can't really be assessed without actually working the implementation, so for 99% of the list participants (myself included) this is a `yes it is, no it isn't' point. 2) an eddy buffer solution requires some substantial speed-up in both the NIC data path, and MOST IMPORTANTLY: the host bus. In order to unload the eddy buffer while still handling incoming traffic at line rate, clearly the host bus bandwidth must be > line rate. I know of at least one general purpose framed solution operating at 10G which has been available for >3 years (SGI's GSN/ST/XIO NIC). I'm sure there are others. I can't imagine there's any argument that a framed solution would be voted `most likely to run fast and be cheap'. Every storage network and cluster interconnect has been designed that way since antiquity. The key tradeoff involves the OS vendors, and I'm wondering why we're speaking for them. The question IS, how much more work is it to introduce TCP framing over and above what is required to insert iSCSI into their network framework. My experience from writing NIC and storage drivers for many commercial UNIX-family OSes is: 1) it's an easy and well defined process to insert a new SCSI transport driver into the SCSI stack. 2) it's hard and poorly defined process to insert ANYTHING into the network stack. Networking has historically been a user-mode activity. Architected services are only provided to user mode programs. Kernel clients have been few and far between and so are handled on a case-by-case basis. For example NFS. Every OS has hacks to make NFS run fast, but they are not stable interfaces for general purpose use. Even Solaris' SysV-derived STREAMS stack, which is intended precisely to provide flexible, crisp interfaces for kernel network clients, does not document the relevant (IP stack) intermodule interfaces. I know that there are more and more kernel network clients, but they are coming either on fluid platforms (e.g. linux), in which case the argument of `it'll take too long to get OS support' doesn't apply, or they are vendor-supplied, in which case a performance iSCSI solution in ANY form may take a while, and the choice of framing or markers isn't going to make a difference. I can't say squat about the architecture of Winsock, but the fact that there is a Microsoft author of the framing proposal who seems very serious about supporting framing and RDMA as quickly as possible suggests that framing support should be available on Windows very soon. Steph From owner-ips@ece.cmu.edu Mon May 21 12:55:49 2001 Received: from ece.cmu.edu ([128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id MAA13205 for ; Mon, 21 May 2001 12:55:49 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f4LFOF710044 for ips-outgoing; Mon, 21 May 2001 11:24:15 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from d12lmsgate-3.de.ibm.com (d12lmsgate-3.de.ibm.com [195.212.91.201]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f4LFODH10038 for ; Mon, 21 May 2001 11:24:13 -0400 (EDT) Received: from d12relay02.de.ibm.com (d12relay02.de.ibm.com [9.165.215.23]) by d12lmsgate-3.de.ibm.com (1.0.0) with ESMTP id RAA83348 for ; Mon, 21 May 2001 17:24:05 +0200 From: julian_satran@il.ibm.com Received: from d12mta02.de.ibm.com (d12mta01_cs0 [9.165.222.237]) by d12relay02.de.ibm.com (8.8.8m3/NCO v4.96) with SMTP id RAA81426 for ; Mon, 21 May 2001 17:23:54 +0200 Received: by d12mta02.de.ibm.com(Lotus SMTP MTA v4.6.5 (863.2 5-20-1999)) id C1256A53.00547904 ; Mon, 21 May 2001 17:22:40 +0200 X-Lotus-FromDomain: IBMIL@IBMDE To: ips@ece.cmu.edu Message-ID: Date: Mon, 21 May 2001 18:28:32 +0300 Subject: Re: iSCSI: CmdSN with Text Command Mime-Version: 1.0 Content-type: text/plain; charset=us-ascii Content-Disposition: inline Sender: owner-ips@ece.cmu.edu Precedence: bulk Eddy, There is no reason Txt commands should be excmpt from numbering. The quoted satements only say that iSCSI does the numbering. To avoid confusion I will however suggest the following rephrasing: 1.1.1.1 Command Numbering and Acknowledging iSCSI supports ordered command delivery within a session. All commands (initiator-to-target) are numbered. Any SCSI activity is related to a task (SAM-2). The task is identified by the Initiator Task Tag for the life of the task. Commands in transit from the initiator to the target layer are numbered by iSCSI; the number is carried by the iSCSI PDU as CmdSN (Command-Sequence-Number). The numbering is session-wide. All outgoing iSCSI PDUs that have a task association, except Data-Out, carry this number. CmdSNs are allocated by the initiator iSCSI within a 32-bit unsigned counter (modulo 2**32). Comparisons and arithmetic on CmdSN SHOULD use Serial Number Arithmetic as defined in [RFC1982] where SERIAL_BITS = 32. Commands meant for immediate delivery are marked as such through an immediate delivery flag. They carry the same CmdSN as that which would have been given to a non-immediate command but the CmdSN is not advanced for immediate commands. Not covered in this document are the means by which one may request immediate delivery for a command or by which iSCSI will decide by itself to mark a PDU for immediate delivery. If immediate delivery is used with task management commands, these commands may reach the SCSI target task management before the tasks they are supposed to act upon. However, their CmdSN is a good reference for what commands the immediate command was supposed to act upon. During command delivery to the target, the allocated numbers are unique session wide. Except for the commands marked for immediate delivery the iSCSI target layer MUST deliver the commands for execution in the order specified by CmdSN. Commands marked for immediate delivery may be handed over by the iSCSI target layer for execution as soon as detected. iSCSI may avoid delivering some command for execution if so required by some prior SCSI or iSCSI action (e.g., clear task set Task Management request received before all the commands it was supposed to act on). The initiator and target are assumed to have three registers that define the numbering mechanism: - CmdSN - the current command Sequence Number advanced by 1 on each command shipped except for commands marked for immediate delivery. - ExpCmdSN - the next expected command by the target. The target acknowledges all commands up to but not including this number. The target iSCSI layer sets the ExpCmdSN to the largest non-immediate CmdSN that it is able to deliver for execution plus 1 (no holes in the CmdSN sequence). - MaxCmdSN - the maximum number to be shipped. The queuing capacity of the receiving iSCSI layer is MaxCmdSN - ExpCmdSN + 1. ExpCmdSN and MaxCmdSN are derived from target-to-initiator PDU fields. MaxCmdSN and ExpCmdSN fields are processed as follows: -if the PDU MaxCmdSN is less than the PDU ExpCmdSN (in Serial Arithmetic Sense and with a difference bounded by 2**31-1), they are both ignored -if the PDU MaxCmdSN is less than the local MaxCmdSN (in Serial Arithmetic Sense and with a difference bounded by 2**31-1), it is ignored; else it updates the local MaxCmdSN -if the PDU ExpCmdSN is less than the local ExpCmdSN (in Serial Arithmetic Sense and with a difference bounded by 2**31-1), it is ignored; else it updates the local ExpCmdSN This sequence is required as updates may arrive out of order because they travel on different TCP connections. The target MUST NOT transmit a MaxCmdSN that is more than 2**31 - 1 above the last ExpCmdSN. For non-immediate commands, the CmdSN field can take any value from ExpCmdSN to MaxCmdSN. For immediate commands, the CmdSN field can take any value from ExpCmdSN to MaxCmdSN+1. The target MUST silently ignore any command outside this range or duplicates within the range that have not been flagged with the retry bit (the X bit in the opcode). iSCSI initiators and targets MUST support the command numbering scheme. Regards, Julo "Eddy Quicksall" on 16-05-2001 20:53:49 Please respond to "Eddy Quicksall" To: ips@ece.cmu.edu cc: Subject: iSCSI: CmdSN with Text Command I am confused about the use of CmdSN with the Text command. The reason is as follows: Section 1.2.2.1 says iSCSI supports ordered command delivery within a session. All commands (initiator-to-target) are numbered. Commands in transit from the initiator SCSI layer to the SCSI target layer are numbered by iSCSI; the number is carried by the iSCSI PDU as CmdSN (Command-Sequence-Number). - CmdSN - the current command Sequence Number advanced by 1 on each command shipped except for commands marked for immediate delivery. ... The target iSCSI layer sets the ExpCmdSN to the largest non-immediate CmdSN that it is able to deliver to the target SCSI layer plus 1 (no holes in the CmdSN sequence). It looks like the 1st and 3rd paragraphs are saying that the CmdSN of a Text Command would be advanced. But, the 2nd and 4th paragraphs hint that only the SCSI layers advance CmdSN. Can someone clear this up? i.e., is CmdSN advanced for Text Commands? mailto:Eddy@Quicksall.com From owner-ips@ece.cmu.edu Mon May 21 14:19:29 2001 Received: from ece.cmu.edu ([128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id OAA15656 for ; Mon, 21 May 2001 14:19:29 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f4LGFMq14431 for ips-outgoing; Mon, 21 May 2001 12:15:22 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from e31.bld.us.ibm.com (e31.co.us.ibm.com [32.97.110.129]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f4LGFLH14427 for ; Mon, 21 May 2001 12:15:21 -0400 (EDT) Received: from westrelay02.boulder.ibm.com (westrelay02.boulder.ibm.com [9.99.140.23]) by e31.bld.us.ibm.com (8.9.3/8.9.3) with ESMTP id MAB46770 for ; Mon, 21 May 2001 12:07:44 -0400 Received: from f3n42e (d03nm042h.boulder.ibm.com [9.99.140.42]) by westrelay02.boulder.ibm.com (8.11.1/NCO v4.96.1.0) with ESMTP id f4LGFFO131552 for ; Mon, 21 May 2001 10:15:16 -0600 Importance: Normal Subject: iSCSI: response to second login (with same ISID) To: ips@ece.cmu.edu From: "Jim Hafner" Date: Mon, 21 May 2001 09:15:14 -0700 Message-ID: X-MIMETrack: Serialize by Router on D03NM042/03/M/IBM(Release 5.0.6 |December 14, 2000) at 05/21/2001 09:15:15 AM MIME-Version: 1.0 Content-type: text/plain; charset=us-ascii Sender: owner-ips@ece.cmu.edu Precedence: bulk Folks, Santosh has requested a response to this question and I don't think I've noticed an answer. It's one of those "dot the i" things that we need to complete. The current thinking is that an iSCSI initiator must use a different ISID for each session it opens with a specific iSCSI target. We need to decide on the iSCSI target's response when the iSCSI initiator attempts to break this rule. The choices are: 1) REJECT the login with some error code (that the iSCSI initiator can understand or infer to mean "ISID in use") and leave the pre-existing session alone 2) CLOSE the pre-existing session (and abort all it's tasks) and then process the new login fresh. Personally, I have no strong feeling either way, but lean towards option (1). It's simple, clean and doesn't involve any interruption. Anybody else got any opinions? Jim Hafner From owner-ips@ece.cmu.edu Mon May 21 14:36:42 2001 Received: from ece.cmu.edu ([128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id OAA15921 for ; Mon, 21 May 2001 14:36:41 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f4LH3J518548 for ips-outgoing; Mon, 21 May 2001 13:03:19 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from chmls05.mediaone.net (chmls05.mediaone.net [24.147.1.143]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f4LH3HH18543 for ; Mon, 21 May 2001 13:03:18 -0400 (EDT) Received: from breinhold ([140.186.66.73]) by chmls05.mediaone.net (8.11.1/8.11.1) with SMTP id f4LH3Cx28887; Mon, 21 May 2001 13:03:13 -0400 (EDT) From: "Barry Reinhold" To: "Julian_Satran@Il. Ibm. Com" Cc: "ISCSI" Subject: iSCSI: Login State Tables Date: Mon, 21 May 2001 13:00:27 -0400 Message-ID: MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 (Normal) X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook IMO, Build 9.0.2416 (9.0.2911.0) Importance: Normal X-MimeOLE: Produced By Microsoft MimeOLE V5.00.2919.6700 Sender: owner-ips@ece.cmu.edu Precedence: bulk Content-Transfer-Encoding: 7bit Julian and interested parties: The login mechanisms as documented in 06 concerns me, as there is a lot of room for "interpretation" of the current text. I have attempted to pull the 06 statements together into a state table in order to avoid what I fear may be a bunch of interoperability headaches. The state table, however, also places some restrictions on the process that is not in the current text. In particular: 1. It forces all negotiations to go through an explicit security handshake. If security is not being used the handshake still takes place. This allows a target to demand security in a clean manner even if the initiator did not request it. 2. It explicitly places most parameters into an operation class and a security class. These parameters may only be exchanged during defined phases of the login process. (This is intended to remove ambiguity associated with the use of operational parameters that were negotiated in the security phase). There is a set of parameters that are really not in either camp. These parameters can be used in both the security phase and the operational phase. Once negotiated their values would remain established unless explicitly renegotiated. [Note: Another choice here is to place all operational parameters into this category and apply the rule that all negotiated values MUST be maintained by both parties unless renegotiated.] 3. It does not allow a target to respond to a Text PDU that has a F=1 setting with new key=value pairs in a login PDU with F=1. 4. It allows the initiator to continue negotiations if it was happy with a context but additional negotiations made it unhappy. This allows the protocol to address situations in which there is a coupling between parameters. This is intended to mostly be a clarification of what I think 06 is attempting to state, however, you may think differently. If there is basic agreement that this is what we are trying to do in 06 I would like to see something of this nature added to the draft. Parameter classification: Operational Parameters: Maxconnections, FMarker, RFMarker, RFMarkInt, SFMarkInt, IFMarkInt, InitialR2T, BidiInitialR2T, ImmediateData, DataPDULength, FirstBurstSize, LogoutLoginMinTime, LogoutLoginMaxTime, EnableACA, MaxOutstandingR2T, DataOrder, BootSession, Glen-Turner Vendor Specific Keys. Not Classified - TargetName, InitiatorName, TargetAlias, InitiatorAlias, TargetAddress, SendTargets Security - DataDigest, HeaderDigest, AuthMethod Identifiers used in login state table SecDone - An indicator that takes on the value True (T) or False (F). This indicator is when the local device is satisfied with the current security context and is ready to move to the operational parameter negotiation phase. When a PDU has been received the value of SecDone is that established after the contents of the received PDU have been evaluated. OpDone - This is the logical equivalent of SecDone for the Operational Phase. Fsent - This variable is used to indicate that the last PDU sent by this device had the F bit set to 1. [Note: The meaning of the F bit is changed by this usage. It no longer means that this is the last PDU to be sent by the initiator, in this context it means that this is the last text PDU I will send if the target doesn't continue the negotiation.] SCC - A short hand notation for the key "SecurityContextComplete" Rcv_Text- A valid Text PDU was received Rcv_Login-A login PDU was received Xmit_Text-A Text PDU is transmitted Perror - A protocol error was detected Initiator Login State Transition Table: +--------+-------------------------+--------+-------------------------+----- --+ |C State | Event(s) and conditions |N State | Actions | Notes | +--------+-------------------------+--------+-------------------------+----- --+ |START |(TCP connection Est- |SEC-END |xmit login(SCC=yes) | 1 | | |ablished) & SecDone | | | | +--------+-------------------------+--------+-------------------------+----- --+ |START |(TCP connection Est- |SEC-NEG |xmit login | | | |ablished & not(SecDone) | | | | +--------+-------------------------+--------+-------------------------+----- --+ |SEC-NEG |Rcv_Text & SecDone=F |SEC-NEG |Xmit_Text | | +--------+-------------------------+--------+-------------------------+----- --+ |SEC-NEG |Rcv_Text & SecDone=T |SEC-END |xmit_Text(SCC=yes) | | +--------+-------------------------+--------+-------------------------+----- --+ |SEC-NEG |Perror |ABORT |TCP connection terminated| 2 | +--------+-------------------------+--------+-------------------------+----- --+ |SEC-END |Rcv_Text & SecDone=F |SEC-NEG |xmit_Text | 3 | +--------+-------------------------+--------+-------------------------+----- --+ |SEC-END |Rcv_Text & SecDone=T |SEC-NEG |xmit_Text | | +--------+-------------------------+--------+-------------------------+----- --+ |SEC-END |Rcv_Text(SCC=yes) & |OPER-NEG|xmit_Text | 4 | | |SecDone = T & OpDone = F | | | | +--------+-------------------------+--------+-------------------------+----- --+ |SEC-END |Rcv_Text(SCC=yes) & |OPER-END|xmit_Text | 4 | | |SecDone = T & OpDone = T | | | | +--------+-------------------------+--------+-------------------------+----- --+ |SEC-END |Perror |ABORT |TCP connection terminated| 5 | +--------+-------------------------+--------+-------------------------+----- --+ |OPER-NEG|Rcv_Text & OpDone = F |OPER-NEG|xmit_Text | | +--------+-------------------------+--------+-------------------------+----- --+ |OPER-NEG|Rcv_Text & OpDone = T |OPER-END|xmit_Text(F=1), Fsent=1 | | +--------+-------------------------+--------+-------------------------+----- --+ |OPER-NEG|Perror |ABORT |TCP connection terminated| 6 | +--------+-------------------------+--------+-------------------------+----- --+ |OPER-END|Rcv_Text & OpDone = F |OPER-NEG|xmit_Text, Fsent=0 | | +--------+-------------------------+--------+-------------------------+----- --+ |OPER-END|Rcv_Text & OpDone = T |OPER-END|xmit_Text(F=1), Fsent=1 | | +--------+-------------------------+--------+-------------------------+----- --+ |OPER-END|Rcv_Login(F=1) & Fsent=1 |FULL | | | +--------+-------------------------+--------+-------------------------+----- --+ |OPER-END|Perror |ABORT |TCP connection terminated| 7 | +--------+-------------------------+--------+-------------------------+----- --+ Notes: All - The F bit is set to zero (F=0) unless otherwise indicated in the login state table. All - Operational parameter key=value lists SHALL NOT be allowed before the security context has been completed. All - The Key=value pair "SecurityContextComplete=yes" must always be sent by itself 1 - No key=value pairs may be included in this PDU except "SecurityContextComplete=yes" 2 - Identified protocol errors are: Rcv_Text has F=1, Rcv_Text has operational parameters 3 - If the target sends a security parameter that causes the initiator to no longer be satisfied with the current security context the initiator continues the security negotiations. 4 - This and subsequent PDUs sent on this connection SHALL be transmitted using the negotiated security context. 5 - Identified protocol errors are Rcv_Text has F=1, Rcv_Text has operational parameters 6 - Identified protocol errors are: Rcv_Text has F=1 and Fsent is not set, Rcv_Text contains a security key 7 - Identified protocol errors are: Rcv_Text has F=1, A login PDU is received with new key=value parameters, A login PDU is received with F=0. Target Login State Transition Table: +--------+-------------------------+--------+-------------------------+----- --+ |C State | Event(s) and conditions |N State | Actions | Notes | +--------+-------------------------+--------+-------------------------+----- --+ |WAIT |rcv_login(scc=yes) & |OPER |xmit_Text(scc=yes) | 1 | | |SecDone = T | | | | +--------+-------------------------+--------+-------------------------+----- --+ |WAIT |rcv_login & SecDone = T |SECURITY|xmit_Text(scc=yes) | | +--------+-------------------------+--------+-------------------------+----- --+ |WAIT |rcv_login(scc=yes) & |SECURITY|xmit_Text | | | |SecDone = F | | | | +--------+-------------------------+--------+-------------------------+----- --+ |WAIT |rcv_login & SecDone = T |SECURITY|xmit_Text | | +--------+-------------------------+--------+-------------------------+----- --+ |WAIT |Perror |ABORT |xmit_Login(F=1) | 2 | | | | |status_class != 0 | | +--------+-------------------------+--------+-------------------------+----- --+ |SECURITY|rec_Text & SecDone = F |SECURITY| xmit_Text | | +--------+-------------------------+--------+-------------------------+----- --+ |SECURITY|rec_Text & SecDone = T |SECURITY| xmit_Text(scc=yes) | | +--------+-------------------------+--------+-------------------------+----- --+ |SECURITY|rec_text(Scc=yes) & |SECURITY| xmit_Text | | | | SecDone = F | | | | +--------+-------------------------+--------+-------------------------+----- --+ |SECURITY|rec_text(Scc=yes) & |OPER | xmit_Text(SCC=yes) | 3 | | | SecDone = T | | | | +--------+-------------------------+--------+-------------------------+----- --+ |SECURITY|Perror |ABORT |xmit_Login(F=1) | 4 | | | | |status_class != 0 | | +--------+-------------------------+--------+-------------------------+----- --+ |OPER |rcv_Text & OpDone = F |OPER |xmit_Text | | +--------+-------------------------+--------+-------------------------+----- --+ |OPER |rcv_Text & OpDone = T |OPER |xmit_Text | | +--------+-------------------------+--------+-------------------------+----- --+ |OPER |rcv_Text(F=1) |OPER |xmit_Text | | | |OpDone = F | | | | +--------+-------------------------+--------+-------------------------+----- --+ |OPER |rcv_Text(F=1) |Full |xmit_Login(F=1) | 5 | | |OpDone = T | | | | +--------+-------------------------+--------+-------------------------+----- --+ |OPER |Perror |ABORT |xmit_Login(F=1) | 6 | | | | |status_class != 0 | | +--------+-------------------------+--------+-------------------------+----- --+ 1. The Target SHALL NOT send a login PDU back unless there is an error and it is ending the login process. The next PDU to be received will be using the negotiated security context (externally established or default) 2. Identified protocol errors: The F bit is set in the login PDU, Operational parameters are given as key=value lists in the PDU 3. The device SHALL use the negotiated security on the all subsequent PDUs received for this connection. 4. Identified protocol errors: The F bit is set in the text PDU, a login PDU was received, operational parameters were received in the PDU. 5. The target SHALL NOT send any additional key=value lists in this PDU. 6. Identified protocol errors: Security parameters were received in the PDU. Barry Reinhold Principal Architect Trebia Networks barry.reinhold@trebia.com 603-868-5144/603-659-0885/978-929-0830 x138 From owner-ips@ece.cmu.edu Mon May 21 15:05:18 2001 Received: from ece.cmu.edu ([128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id PAA16502 for ; Mon, 21 May 2001 15:05:18 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f4LHvig23274 for ips-outgoing; Mon, 21 May 2001 13:57:44 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from chmls05.mediaone.net (chmls05.mediaone.net [24.147.1.143]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f4LHvgH23268 for ; Mon, 21 May 2001 13:57:42 -0400 (EDT) Received: from breinhold ([140.186.66.73]) by chmls05.mediaone.net (8.11.1/8.11.1) with SMTP id f4LHvdx16938 for ; Mon, 21 May 2001 13:57:39 -0400 (EDT) From: "Barry Reinhold" To: "ISCSI" Subject: ISCSI: Login State Tables Date: Mon, 21 May 2001 13:54:58 -0400 Message-ID: MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="----=_NextPart_000_0002_01C0E1FD.9F59ECA0" X-Priority: 3 (Normal) X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook IMO, Build 9.0.2416 (9.0.2911.0) Importance: Normal X-MimeOLE: Produced By Microsoft MimeOLE V5.00.2919.6700 Sender: owner-ips@ece.cmu.edu Precedence: bulk This is a multi-part message in MIME format. ------=_NextPart_000_0002_01C0E1FD.9F59ECA0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit For those of you who end up getting the login state tables in a mangled form -- try this as an attachment. Barry Reinhold Principal Architect Trebia Networks barry.reinhold@trebia.com 603-868-5144/603-659-0885/978-929-0830 x138 ------=_NextPart_000_0002_01C0E1FD.9F59ECA0 Content-Type: text/plain; name="login proposal.txt" Content-Disposition: attachment; filename="login proposal.txt" Content-Transfer-Encoding: quoted-printable Julian and interested parties: The login mechanisms as documented in 06 concerns me, as there is a lot = of room for "interpretation" of the current text. I have attempted to pull the 06 = statements together into a state table in order to avoid what I fear may be a bunch of = interoperability headaches. The state table, however, also places some restrictions on = the process that is not in the current text. In particular: 1. It forces all negotiations to go through an explicit security = handshake. If=20 security is not being used the handshake still takes place. This allows = a target to demand security in a clean manner even if the initiator did not request = it. 2. It explicitly places most parameters into an operation class and a = security class. These parameters may only be exchanged during defined phases of the = login process. (This is intended to remove ambiguity associated with the use of = operational parameters that were negotiated in the security phase). There is a set of = parameters that are=20 really not in either camp. These parameters can be used in both the = security phase and the operational phase. Once negotiated their values would remain = established unless=20 explicitly renegotiated. [Note: Another choice here is to place all = operational parameters into this category and apply the rule that all negotiated values MUST be = maintained by both parties unless renegotiated.] 3. It does not allow a target to respond to a Text PDU that has a F=3D1 = setting with new key=3Dvalue pairs in a login PDU with F=3D1. 4. It allows the initiator to continue negotiations if it was happy with = a context but additional negotiations made it unhappy. This allows the protocol to = address situations in which there is a coupling between parameters. This is intended to mostly be a clarification of what I think 06 is = attempting to state, however, you may think differently. If there is basic agreement = that this is what we are trying to do in 06 I would like to see something of this = nature added to the draft. Parameter classification: Operational Parameters: Maxconnections, FMarker, RFMarker, RFMarkInt, SFMarkInt, IFMarkInt, = InitialR2T, BidiInitialR2T, ImmediateData, DataPDULength, FirstBurstSize, = LogoutLoginMinTime, LogoutLoginMaxTime, EnableACA, MaxOutstandingR2T, DataOrder, = BootSession,=20 Glen-Turner Vendor Specific Keys. Not Classified - TargetName, InitiatorName, TargetAlias, InitiatorAlias, TargetAddress, SendTargets Security - DataDigest, HeaderDigest, AuthMethod Identifiers used in login state table SecDone - An indicator that takes on the value True (T) or False (F). This indicator is when the local device is satisfied with the current security context and is ready to move to the operational parameter negotiation phase. When a PDU has been received the value of SecDone is that established after the contents of the received PDU have been evaluated. OpDone - This is the logical equivalent of SecDone for the Operational = Phase. Fsent - This variable is used to indicate that the last PDU sent by this device had the F bit set to 1. [Note: The meaning of the=20 F bit is changed by this usage. It no longer means that this = is the last PDU to be sent by the initiator, in this context it = means=20 that this is the last text PDU I will send if the target=20 doesn't continue the negotiation.] SCC - A short hand notation for the key "SecurityContextComplete" Rcv_Text- A valid Text PDU was received Rcv_Login-A login PDU was received Xmit_Text-A Text PDU is transmitted Perror - A protocol error was detected Initiator Login State Transition Table: +--------+-------------------------+--------+-------------------------+--= -----+ |C State | Event(s) and conditions |N State | Actions | = Notes | +--------+-------------------------+--------+-------------------------+--= -----+ |START |(TCP connection Est- |SEC-END |xmit login(SCC=3Dyes) = | 1 | | |ablished) & SecDone | | | = | +--------+-------------------------+--------+-------------------------+--= -----+ |START |(TCP connection Est- |SEC-NEG |xmit login | = | | |ablished & not(SecDone) | | | = | +--------+-------------------------+--------+-------------------------+--= -----+ |SEC-NEG |Rcv_Text & SecDone=3DF |SEC-NEG |Xmit_Text = | | +--------+-------------------------+--------+-------------------------+--= -----+ |SEC-NEG |Rcv_Text & SecDone=3DT |SEC-END |xmit_Text(SCC=3Dyes) = | | +--------+-------------------------+--------+-------------------------+--= -----+ |SEC-NEG |Perror |ABORT |TCP connection terminated| = 2 | +--------+-------------------------+--------+-------------------------+--= -----+ |SEC-END |Rcv_Text & SecDone=3DF |SEC-NEG |xmit_Text = | 3 | +--------+-------------------------+--------+-------------------------+--= -----+ |SEC-END |Rcv_Text & SecDone=3DT |SEC-NEG |xmit_Text = | | +--------+-------------------------+--------+-------------------------+--= -----+ |SEC-END |Rcv_Text(SCC=3Dyes) & |OPER-NEG|xmit_Text = | 4 | =20 | |SecDone =3D T & OpDone =3D F | | = | | +--------+-------------------------+--------+-------------------------+--= -----+ |SEC-END |Rcv_Text(SCC=3Dyes) & |OPER-END|xmit_Text = | 4 | =20 | |SecDone =3D T & OpDone =3D T | | = | | +--------+-------------------------+--------+-------------------------+--= -----+ |SEC-END |Perror |ABORT |TCP connection terminated| = 5 | +--------+-------------------------+--------+-------------------------+--= -----+ |OPER-NEG|Rcv_Text & OpDone =3D F |OPER-NEG|xmit_Text = | | +--------+-------------------------+--------+-------------------------+--= -----+ |OPER-NEG|Rcv_Text & OpDone =3D T |OPER-END|xmit_Text(F=3D1), = Fsent=3D1 | | +--------+-------------------------+--------+-------------------------+--= -----+ |OPER-NEG|Perror |ABORT |TCP connection terminated| = 6 | +--------+-------------------------+--------+-------------------------+--= -----+ |OPER-END|Rcv_Text & OpDone =3D F |OPER-NEG|xmit_Text, Fsent=3D0 = | | +--------+-------------------------+--------+-------------------------+--= -----+ |OPER-END|Rcv_Text & OpDone =3D T |OPER-END|xmit_Text(F=3D1), = Fsent=3D1 | | +--------+-------------------------+--------+-------------------------+--= -----+ |OPER-END|Rcv_Login(F=3D1) & Fsent=3D1 |FULL | = | | +--------+-------------------------+--------+-------------------------+--= -----+ |OPER-END|Perror |ABORT |TCP connection terminated| = 7 | +--------+-------------------------+--------+-------------------------+--= -----+ Notes: All - The F bit is set to zero (F=3D0) unless otherwise indicated in the login state table. All - Operational parameter key=3Dvalue lists SHALL NOT be allowed=20 before the security context has been completed. All - The Key=3Dvalue pair "SecurityContextComplete=3Dyes" must always=20 be sent by itself 1 - No key=3Dvalue pairs may be included in this PDU=20 except "SecurityContextComplete=3Dyes" 2 - Identified protocol errors are: Rcv_Text has F=3D1, Rcv_Text=20 has operational parameters 3 - If the target sends a security parameter that causes the initiator = to no longer be satisfied with the current security context the = initiator=20 continues the security negotiations. 4 - This and subsequent PDUs sent on this connection SHALL be = transmitted=20 using the negotiated security context. 5 - Identified protocol errors are Rcv_Text has F=3D1, Rcv_Text has=20 operational parameters 6 - Identified protocol errors are: Rcv_Text has F=3D1 and Fsent is = not set,=20 Rcv_Text contains a security key 7 - Identified protocol errors are: Rcv_Text has F=3D1, A login PDU is = received with new key=3Dvalue parameters, A login PDU is received = with F=3D0. Target Login State Transition Table: +--------+-------------------------+--------+-------------------------+--= -----+ |C State | Event(s) and conditions |N State | Actions | = Notes | +--------+-------------------------+--------+-------------------------+--= -----+ |WAIT |rcv_login(scc=3Dyes) & |OPER |xmit_Text(scc=3Dyes) = | 1 | | |SecDone =3D T | | = | | +--------+-------------------------+--------+-------------------------+--= -----+ |WAIT |rcv_login & SecDone =3D T |SECURITY|xmit_Text(scc=3Dyes) = | | +--------+-------------------------+--------+-------------------------+--= -----+ |WAIT |rcv_login(scc=3Dyes) & |SECURITY|xmit_Text = | | | |SecDone =3D F | | = | | +--------+-------------------------+--------+-------------------------+--= -----+ |WAIT |rcv_login & SecDone =3D T |SECURITY|xmit_Text = | | +--------+-------------------------+--------+-------------------------+--= -----+ |WAIT |Perror |ABORT |xmit_Login(F=3D1) = | 2 | | | | |status_class !=3D 0 = | | +--------+-------------------------+--------+-------------------------+--= -----+ |SECURITY|rec_Text & SecDone =3D F |SECURITY| xmit_Text = | | +--------+-------------------------+--------+-------------------------+--= -----+ |SECURITY|rec_Text & SecDone =3D T |SECURITY| xmit_Text(scc=3Dyes) = | | +--------+-------------------------+--------+-------------------------+--= -----+ |SECURITY|rec_text(Scc=3Dyes) & |SECURITY| xmit_Text = | | | | SecDone =3D F | | = | | +--------+-------------------------+--------+-------------------------+--= -----+ |SECURITY|rec_text(Scc=3Dyes) & |OPER | xmit_Text(SCC=3Dyes) = | 3 | | | SecDone =3D T | | = | | +--------+-------------------------+--------+-------------------------+--= -----+ |SECURITY|Perror |ABORT |xmit_Login(F=3D1) = | 4 | | | | |status_class !=3D 0 = | | +--------+-------------------------+--------+-------------------------+--= -----+ |OPER |rcv_Text & OpDone =3D F |OPER |xmit_Text = | | +--------+-------------------------+--------+-------------------------+--= -----+ |OPER |rcv_Text & OpDone =3D T |OPER |xmit_Text = | | +--------+-------------------------+--------+-------------------------+--= -----+ |OPER |rcv_Text(F=3D1) |OPER |xmit_Text = | | | |OpDone =3D F | | = | | +--------+-------------------------+--------+-------------------------+--= -----+ |OPER |rcv_Text(F=3D1) |Full |xmit_Login(F=3D1) = | 5 | | |OpDone =3D T | | = | | +--------+-------------------------+--------+-------------------------+--= -----+ |OPER |Perror |ABORT |xmit_Login(F=3D1) = | 6 | | | | |status_class !=3D 0 = | | +--------+-------------------------+--------+-------------------------+--= -----+ 1. The Target SHALL NOT send a login PDU back unless there is an error = and it is ending the login process. The next PDU to be received will be = using the negotiated security context (externally established or default) 2. Identified protocol errors: The F bit is set in the login PDU, = Operational parameters are given as key=3Dvalue lists in the PDU 3. The device SHALL use the negotiated security on the all subsequent = PDUs received for this connection. 4. Identified protocol errors: The F bit is set in the text PDU, a login = PDU was received, operational parameters were received in the PDU. 5. The target SHALL NOT send any additional key=3Dvalue lists in this = PDU. 6. Identified protocol errors: Security parameters were received in the = PDU. ------=_NextPart_000_0002_01C0E1FD.9F59ECA0-- From owner-ips@ece.cmu.edu Mon May 21 15:19:17 2001 Received: from ece.cmu.edu ([128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id PAA16674 for ; Mon, 21 May 2001 15:19:16 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f4LHX6H21142 for ips-outgoing; Mon, 21 May 2001 13:33:06 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from chmls06.mediaone.net (chmls06.mediaone.net [24.147.1.144]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f4LGQAH15322 for ; Mon, 21 May 2001 12:26:10 -0400 (EDT) Received: from eddylaptop (h00e098880aa4.ne.mediaone.net [66.31.75.244]) by chmls06.mediaone.net (8.11.1/8.11.1) with SMTP id f4LGQ5822114; Mon, 21 May 2001 12:26:05 -0400 (EDT) From: "Eddy Quicksall" To: , Subject: iSCSI: Final bit/status clarification Date: Mon, 21 May 2001 12:26:13 -0400 Message-ID: <001001c0e212$c13d1b70$0100a8c0@eddylaptop> MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 (Normal) X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook CWS, Build 9.0.2416 (9.0.2911.0) Importance: Normal X-MimeOLE: Produced By Microsoft MimeOLE V5.00.3018.1300 Sender: owner-ips@ece.cmu.edu Precedence: bulk Content-Transfer-Encoding: 7bit Section 2.7.1 F (Final) Bit For incoming data, this bit is 1 for the last input data PDU associated with the command (even if it includes the status). Is a clarification implied by "even if it includes the status"? I don't see the need for a clarification here because I don't see any ambiguity. Would it make sense to remove it? Or, maybe it should say "even if it does not include the status". -------- mailto:Eddy@Quicksall.com From owner-ips@ece.cmu.edu Mon May 21 17:17:54 2001 Received: from ece.cmu.edu ([128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id RAA18957 for ; Mon, 21 May 2001 17:17:54 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f4LJXM701959 for ips-outgoing; Mon, 21 May 2001 15:33:22 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from palrel2.hp.com (palrel2.hp.com [156.153.255.234]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f4LJXKH01943 for ; Mon, 21 May 2001 15:33:20 -0400 (EDT) Received: from hpcuhe.cup.hp.com (hpcuhe.cup.hp.com [15.0.80.203]) by palrel2.hp.com (Postfix) with ESMTP id 6F6C81364 for ; Mon, 21 May 2001 12:33:19 -0700 (PDT) Received: from cup.hp.com (santoshr@hpindhhm.cup.hp.com [15.8.80.197]) by hpcuhe.cup.hp.com (8.9.3 (PHNE_18979)/8.9.3 SMKit7.02) with ESMTP id MAA05390 for ; Mon, 21 May 2001 12:33:14 -0700 (PDT) Message-ID: <3B096D97.9C957568@cup.hp.com> Date: Mon, 21 May 2001 12:33:43 -0700 From: Santosh Rao Organization: Hewlett Packard, Cupertino. X-Mailer: Mozilla 4.7 [en] (X11; U; HP-UX B.11.00 9000/778) X-Accept-Language: en MIME-Version: 1.0 To: ips@ece.cmu.edu Subject: Re: iSCSI: response to second login (with same ISID) References: Content-Type: multipart/mixed; boundary="------------830ADA42D83D77976F15A829" Sender: owner-ips@ece.cmu.edu Precedence: bulk This is a multi-part message in MIME format. --------------830ADA42D83D77976F15A829 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Jim Hafner wrote: > The current thinking is that an iSCSI initiator must use a different ISID > for each session it opens with a specific iSCSI target. Jim, Are you referring to multiple concurrent sessions from an initiator node to a given target port above in saying that different ISIDs must be used ? In the case where the initiator re-establishes a session to the same target port after a power cycle, it is desirable to re-use the same ISID, to allow for tracking of persistent reservation like properties, correct ? (Just checking to make sure the ISID semantics did'nt change since the Nashua discussion.) > We need to decide on the iSCSI target's response when the iSCSI initiator > attempts to break this rule. The choices are: > 1) REJECT the login with some error code (that the iSCSI initiator can > understand or infer to mean "ISID in use") and leave the pre-existing > session alone > 2) CLOSE the pre-existing session (and abort all it's tasks) and then > process the new login fresh. > > Personally, I have no strong feeling either way, but lean towards option > (1). It's simple, clean and doesn't involve any interruption. I'd vote for (2) for the following reasons : a) It is possible for initiators to use the ISID as a scheme to establish persistence of their O.S. device files. In doing so, initiators would attempt to obtain the same ISID for a given session to a given target ports across boots. (and rightly so, for enabling targets to track persistent reservation like properties, etc). In the event where the initiator went through a non-orderly shutdown which did not close all sessions, it would attempt to re-login on re-boot with the same ISID. If the target rejected this re-login, per (1) (citing an invalid ISID), the initiator would no longer be able to maintain persistence of the session to an ISID. b) Using (2) allows the hosts to hint to the target that the previous session is now stale and thus triggers clean-up of stale session resources. Regards, Santosh --------------830ADA42D83D77976F15A829 Content-Type: text/x-vcard; charset=us-ascii; name="santoshr.vcf" Content-Description: Card for Santosh Rao Content-Disposition: attachment; filename="santoshr.vcf" Content-Transfer-Encoding: 7bit begin:vcard n:Rao;Santosh tel;work:408-447-3751 x-mozilla-html:FALSE org:Hewlett Packard, Cupertino.;SISL adr:;;19420, Homestead Road, M\S 43LN, ;Cupertino.;CA.;95014.;USA. version:2.1 email;internet:santoshr@cup.hp.com title:Software Design Engineer x-mozilla-cpt:;21088 fn:Santosh Rao end:vcard --------------830ADA42D83D77976F15A829-- From owner-ips@ece.cmu.edu Mon May 21 17:17:56 2001 Received: from ece.cmu.edu ([128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id RAA18953 for ; Mon, 21 May 2001 17:17:51 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f4LJfqx02667 for ips-outgoing; Mon, 21 May 2001 15:41:52 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from smtp02.wxs.nl (smtp02.wxs.nl [195.121.6.54]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f4LJfoH02660 for ; Mon, 21 May 2001 15:41:50 -0400 (EDT) Received: from daniel ([213.10.179.111]) by smtp02.wxs.nl (Netscape Messaging Server 4.05) with SMTP id GDPAPI02.2H6; Mon, 21 May 2001 21:41:42 +0200 Message-ID: <002701c0e22e$ee701630$9600000a@daniel> From: "Sanjeev Bhagat \(TRIPACE/Zoetermeer\)" To: , "Jim Hafner" References: Subject: Re: iSCSI: response to second login (with same ISID) Date: Mon, 21 May 2001 21:47:50 +0200 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 5.50.4133.2400 Disposition-Notification-To: "Sanjeev Bhagat \(TRIPACE/Zoetermeer\)" X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4133.2400 Sender: owner-ips@ece.cmu.edu Precedence: bulk Content-Transfer-Encoding: 7bit Hello, Before we decide one of the given 2 options we must analyze under which conditions will initiator send the same ISID in two login. If initiator sends the same ISID because of programattic error then in that case option no 1 is more relevant in which target rejects the login with some error code. But if the initiator sends because of some digest error or it is trying to open the same session which was ended because of some error before and target is still in recovery stage then option no. 2 is more relevant where target closes the pre-existing session and then proceeds for fresh login. Regards, Sanjeev ----- Original Message ----- From: "Jim Hafner" To: Sent: Monday, May 21, 2001 6:15 PM Subject: iSCSI: response to second login (with same ISID) > Folks, > > Santosh has requested a response to this question and I don't think I've > noticed an answer. It's one of those "dot the i" things that we need to > complete. > > The current thinking is that an iSCSI initiator must use a different ISID > for each session it opens with a specific iSCSI target. > > We need to decide on the iSCSI target's response when the iSCSI initiator > attempts to break this rule. The choices are: > 1) REJECT the login with some error code (that the iSCSI initiator can > understand or infer to mean "ISID in use") and leave the pre-existing > session alone > 2) CLOSE the pre-existing session (and abort all it's tasks) and then > process the new login fresh. > > Personally, I have no strong feeling either way, but lean towards option > (1). It's simple, clean and doesn't involve any interruption. > > Anybody else got any opinions? > > Jim Hafner > From owner-ips@ece.cmu.edu Mon May 21 17:23:18 2001 Received: from ece.cmu.edu ([128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id RAA19034 for ; Mon, 21 May 2001 17:23:18 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f4LJn6t03332 for ips-outgoing; Mon, 21 May 2001 15:49:06 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from e32.bld.us.ibm.com (e32.co.us.ibm.com [32.97.110.130]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f4LJn4H03325 for ; Mon, 21 May 2001 15:49:04 -0400 (EDT) Received: from westrelay02.boulder.ibm.com (westrelay02.boulder.ibm.com [9.99.140.23]) by e32.bld.us.ibm.com (8.9.3/8.9.3) with ESMTP id PAA84610; Mon, 21 May 2001 15:49:51 -0400 Received: from f3n42e (d03nm042h.boulder.ibm.com [9.99.140.42]) by westrelay02.boulder.ibm.com (8.11.1/NCO v4.96.1.0) with ESMTP id f4LJn3O27776; Mon, 21 May 2001 13:49:03 -0600 Importance: Normal Subject: Re: iSCSI: response to second login (with same ISID) To: Santosh Rao Cc: ips@ece.cmu.edu From: "Jim Hafner" Date: Mon, 21 May 2001 12:49:00 -0700 Message-ID: X-MIMETrack: Serialize by Router on D03NM042/03/M/IBM(Release 5.0.6 |December 14, 2000) at 05/21/2001 12:49:02 PM MIME-Version: 1.0 Content-type: text/plain; charset=us-ascii Sender: owner-ips@ece.cmu.edu Precedence: bulk Santosh, Yes, that is exactly the context. No concurrent sessions from the same iSCSI initiator to the same iSCSI target with the same ISID. And Yes, the recovery for persistent reservations is exactly the reason for that rule. You give valid arguments for option 2. Jim Hafner Santosh Rao @ece.cmu.edu on 05-21-2001 12:33:43 PM Sent by: owner-ips@ece.cmu.edu To: ips@ece.cmu.edu cc: Subject: Re: iSCSI: response to second login (with same ISID) Jim Hafner wrote: > The current thinking is that an iSCSI initiator must use a different ISID > for each session it opens with a specific iSCSI target. Jim, Are you referring to multiple concurrent sessions from an initiator node to a given target port above in saying that different ISIDs must be used ? In the case where the initiator re-establishes a session to the same target port after a power cycle, it is desirable to re-use the same ISID, to allow for tracking of persistent reservation like properties, correct ? (Just checking to make sure the ISID semantics did'nt change since the Nashua discussion.) > We need to decide on the iSCSI target's response when the iSCSI initiator > attempts to break this rule. The choices are: > 1) REJECT the login with some error code (that the iSCSI initiator can > understand or infer to mean "ISID in use") and leave the pre-existing > session alone > 2) CLOSE the pre-existing session (and abort all it's tasks) and then > process the new login fresh. > > Personally, I have no strong feeling either way, but lean towards option > (1). It's simple, clean and doesn't involve any interruption. I'd vote for (2) for the following reasons : a) It is possible for initiators to use the ISID as a scheme to establish persistence of their O.S. device files. In doing so, initiators would attempt to obtain the same ISID for a given session to a given target ports across boots. (and rightly so, for enabling targets to track persistent reservation like properties, etc). In the event where the initiator went through a non-orderly shutdown which did not close all sessions, it would attempt to re-login on re-boot with the same ISID. If the target rejected this re-login, per (1) (citing an invalid ISID), the initiator would no longer be able to maintain persistence of the session to an ISID. b) Using (2) allows the hosts to hint to the target that the previous session is now stale and thus triggers clean-up of stale session resources. Regards, Santosh From owner-ips@ece.cmu.edu Mon May 21 19:46:47 2001 Received: from ece.cmu.edu ([128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id TAA20670 for ; Mon, 21 May 2001 19:46:46 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f4LMlv619020 for ips-outgoing; Mon, 21 May 2001 18:47:57 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from palrel11.hp.com (palrel11.hp.com [192.151.10.59]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f4LMlsH19012 for ; Mon, 21 May 2001 18:47:55 -0400 (EDT) Received: from amrelay1.boi.hp.com (amrelay1.boi.hp.com [15.56.8.24]) by palrel11.hp.com (Postfix) with ESMTP id 1603E1F644; Mon, 21 May 2001 15:47:52 -0700 (PDT) Received: from xatlbh1.atl.hp.com (xatlbh1.atl.hp.com [15.45.89.186]) by amrelay1.boi.hp.com (8.9.3 (PHNE_18979)/8.9.3 SMKit7.02) with ESMTP id QAA07195; Mon, 21 May 2001 16:47:43 -0600 (MDT) Received: by xatlbh1.atl.hp.com with Internet Mail Service (5.5.2653.19) id ; Mon, 21 May 2001 18:47:37 -0400 Message-ID: <6BD67FFB937FD411A04F00D0B74FE87802A09088@xrose06.rose.hp.com> From: "KRUEGER,MARJORIE (HP-Roseville,ex1)" To: "'Jim Hafner'" , ips@ece.cmu.edu Subject: RE: iSCSI: response to second login (with same ISID) Date: Mon, 21 May 2001 18:47:36 -0400 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2653.19) Content-Type: text/plain; charset="iso-8859-1" Sender: owner-ips@ece.cmu.edu Precedence: bulk I can't find Santosh original mail on this subject- I assume you are talking about the case where TSID=0,ISID=dup session number? So it wouldn't be a recovery attempt. I'd think #2 would result in unintended behavior - acts like 2nd way to close a session and is a side effect that the initiator shouldn't intend. I feel #1 is the clean way to handle this case. Marj > Folks, > > Santosh has requested a response to this question and I don't > think I've > noticed an answer. It's one of those "dot the i" things that > we need to > complete. > > The current thinking is that an iSCSI initiator must use a > different ISID > for each session it opens with a specific iSCSI target. > > We need to decide on the iSCSI target's response when the > iSCSI initiator > attempts to break this rule. The choices are: > 1) REJECT the login with some error code (that the iSCSI initiator can > understand or infer to mean "ISID in use") and leave the pre-existing > session alone > 2) CLOSE the pre-existing session (and abort all it's tasks) and then > process the new login fresh. > > Personally, I have no strong feeling either way, but lean > towards option > (1). It's simple, clean and doesn't involve any interruption. > > Anybody else got any opinions? > > Jim Hafner > From owner-ips@ece.cmu.edu Mon May 21 19:47:38 2001 Received: from ece.cmu.edu ([128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id TAA20691 for ; Mon, 21 May 2001 19:47:38 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f4LM6kN15642 for ips-outgoing; Mon, 21 May 2001 18:06:46 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from pikachu.makesans.com (makesans.com [63.203.52.243]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f4LKwgH09723 for ; Mon, 21 May 2001 16:58:43 -0400 (EDT) Received: by pikachu.makesans.com with Internet Mail Service (5.5.2650.21) id ; Mon, 21 May 2001 13:56:15 -0700 Message-ID: <3FE334BDD7B4254ABF88C721B0E5AC490B2BDF@pikachu.makesans.com> From: Gary Kotzur To: "'Jim Hafner'" , ips@ece.cmu.edu Subject: RE: iSCSI: response to second login (with same ISID) Date: Mon, 21 May 2001 13:56:14 -0700 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2650.21) Content-Type: text/plain; charset="iso-8859-1" Sender: owner-ips@ece.cmu.edu Precedence: bulk I agree with your choice of number one, it is cleaner and the initiator can "correct" his login after getting the error code. Gary Kotzur Principal Member of Technical Staff MaXXan Systems, Inc. GaryKotzur@MaXXan.com Ph: 832-436-1012 Cl: 281-814-9396 Fx: 832-436-1099 This email message is for the sole use of the intended recipient(s) and may contain confidential information. Any unauthorized use; review, disclosure or distribution is prohibited. If you are not the intended recipient, please contact the sender by reply email to GaryKotzur@MaXXan.com and destroy all copies of the original message. -----Original Message----- From: Jim Hafner [mailto:hafner@almaden.ibm.com] Sent: Monday, May 21, 2001 11:15 AM To: ips@ece.cmu.edu Subject: iSCSI: response to second login (with same ISID) Folks, Santosh has requested a response to this question and I don't think I've noticed an answer. It's one of those "dot the i" things that we need to complete. The current thinking is that an iSCSI initiator must use a different ISID for each session it opens with a specific iSCSI target. We need to decide on the iSCSI target's response when the iSCSI initiator attempts to break this rule. The choices are: 1) REJECT the login with some error code (that the iSCSI initiator can understand or infer to mean "ISID in use") and leave the pre-existing session alone 2) CLOSE the pre-existing session (and abort all it's tasks) and then process the new login fresh. Personally, I have no strong feeling either way, but lean towards option (1). It's simple, clean and doesn't involve any interruption. Anybody else got any opinions? Jim Hafner From owner-ips@ece.cmu.edu Mon May 21 19:47:58 2001 Received: from ece.cmu.edu ([128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id TAA20715 for ; Mon, 21 May 2001 19:47:57 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f4LN7k620634 for ips-outgoing; Mon, 21 May 2001 19:07:46 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from maho3msx2.isus.emc.com (maho3msx2.isus.emc.com [128.221.11.32]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f4LN7jH20630 for ; Mon, 21 May 2001 19:07:45 -0400 (EDT) Received: by maho3msx2.isus.emc.com with Internet Mail Service (5.5.2653.19) id ; Mon, 21 May 2001 19:07:40 -0400 Message-ID: <0F31E5C394DAD311B60C00E029101A07080155B3@corpmx9.isus.emc.com> From: Black_David@emc.com To: marjorie_krueger@hp.com, ips@ece.cmu.edu Subject: RE: DRAFT Nashua interim meeting minutes Date: Mon, 21 May 2001 19:07:34 -0400 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2653.19) Content-Type: text/plain Sender: owner-ips@ece.cmu.edu Precedence: bulk > Strong consensus that header formats and opcodes should not be changed in > further revisions of the draft except when there is rough concensus from the > group for such changes. There are changes expected in rev. 07 to correct > some things pointed out on the list. Added the following under iSCSI Requirements Last Call Issues: The general sense of the room was that data formats and the like ought not to be changed - at this point in the process, stability matters, and further changes of this sort need to be discussed on the list prior to being made. While I don't believe consensus was called on this, I do remember it as the "general sense of the room". > There was discussion in the security portion of the agenda to the effect > that "if we mandate IPSec with it's data integrity check capabilities, is > there further need to specify error recovery"? That topic was picked up again in the error recovery session. In essence, if ESP is in use, CRC failures at the iSCSI level will essentially never happen, allowing at least within-session and within-connection (and presumably) within-command recovery to be dropped, but the HMAC in ESP is computationally expensive leading to the interest in a CRC shim. I've added explanation of this to the point in the error recovery session where the CRC shim was discussed. Thanks, --David --------------------------------------------------- David L. Black, Senior Technologist EMC Corporation, 42 South St., Hopkinton, MA 01748 +1 (508) 435-1000 x75140 FAX: +1 (508) 497-8500 black_david@emc.com Mobile: +1 (978) 394-7754 --------------------------------------------------- From owner-ips@ece.cmu.edu Mon May 21 20:13:59 2001 Received: from ece.cmu.edu ([128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id UAA21000 for ; Mon, 21 May 2001 20:13:59 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f4LF4hE08238 for ips-outgoing; Mon, 21 May 2001 11:04:43 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from e31.bld.us.ibm.com (e31.co.us.ibm.com [32.97.110.129]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f4LF4fH08229 for ; Mon, 21 May 2001 11:04:41 -0400 (EDT) Received: from westrelay02.boulder.ibm.com (westrelay02.boulder.ibm.com [9.99.140.23]) by e31.bld.us.ibm.com (8.9.3/8.9.3) with ESMTP id KAA42662 for ; Mon, 21 May 2001 10:57:05 -0400 Received: from f3n42e (d03nm042h.boulder.ibm.com [9.99.140.42]) by westrelay02.boulder.ibm.com (8.11.1/NCO v4.96.1.0) with ESMTP id f4LF4eO180838 for ; Mon, 21 May 2001 09:04:40 -0600 Importance: Normal Subject: Re: iSCSI: Canonical Targets To: "Julian Satran" Cc: ips@ece.cmu.edu From: "Jim Hafner" Date: Mon, 21 May 2001 08:04:38 -0700 Message-ID: X-MIMETrack: Serialize by Router on D03NM042/03/M/IBM(Release 5.0.6 |December 14, 2000) at 05/21/2001 08:04:39 AM MIME-Version: 1.0 Content-type: text/plain; charset=us-ascii Sender: owner-ips@ece.cmu.edu Precedence: bulk Julo, I was only teasing about the name. I don't care either, it's a nit. As for "discovery service". Here's a twist on the issue. In the case of most services, at a single ipaddress:tcpport, you'll find a single service entity (web server, ftp server, etc) and only AFTER you get through the first layer does finer granularity of discovery occur (e.g., through the query string of the url). But even in these cases, there's actually little of discovery going on. You pretty much have to know what you're going for first, or you drill manually (think of ftp server accessed through browser -- you get directory listing which you browse). What I think we're doing here is providing a "directory list" function for the single service entity that lives at the defined tcpport. This "listing" service can then be used to drill deeper. I don't see much that's different in what we're spec'ing and what needs to be spec'd for this purpose. We can rely on some mechanisms such as SLP and SNS for somethings. SLP helps to find the place to start for listing service. SNS has "all the answers". We're attempting to fill in the middle ground. Jim Hafner Julian Satran/Haifa/IBM@IBMIL@ece.cmu.edu on 05-19-2001 12:26:29 AM Sent by: owner-ips@ece.cmu.edu To: ips@ece.cmu.edu cc: Subject: Re: iSCSI: Canonical Targets Jim, The canonical attribute is associated with strictly conforming to a rule (as in canonical form for expressions or canon in music, or canonical religious text). I would use the term generic-name or class-name for it. But again the objection on the name was only minor. I strongly believe this is a service-delivery discovery item and we are repeating an old mistake - creating a specialized solution for a general problem. Any discovery service would be better than placing this in the iSCSI realm. Julo "Jim Hafner" on 14-05-2001 18:14:31 Please respond to "Jim Hafner" To: Julian Satran/Haifa/IBM@IBMIL cc: ips@ece.cmu.edu Subject: Re: iSCSI: Canonical Targets Julo, You object to the term "canonical"? My dictionary says canonical is "conforming to a general rule", but perhaps that isn't the best term. Would "well-known" would be better? Jim Hafner Julian Satran/Haifa/IBM@IBMIL@ece.cmu.edu on 05-12-2001 09:05:00 AM Sent by: owner-ips@ece.cmu.edu To: ips@ece.cmu.edu cc: Subject: Re: iSCSI: Canonical Targets Why would any discovery service be wiser/stupider than the mechanism we are having in N&D? If the issue of ports is widespread then others have it too and discovery may include all the nice things we have in iSCSI. I don't see any reason iSNS or an LDAP repository can't contain them. My only claim is that this adds weight to iSCSI. As for interoperability - I claim that adding features makes interoperability an issue. Removing them does not. The whole "canonical" (why canonical - there is nothing canonical about it) target idea is good but in another realm - I don't think it belongs to iSCSI - it belongs to discovery be it specific or general. I can leave with the mechanism as it is - but I really feel that we are on the wrong track. Regards, Julo James Smart on 12-05-2001 15:42:09 Please respond to james.smart@trebia.com To: Julian Satran/Haifa/IBM@IBMIL, Mark Bakke cc: ips@ece.cmu.edu Subject: Re: iSCSI: Canonical Targets I'm in agreement with Mark, that canonical target support is mandatory. The key is interoperability. The initiator must be mandated to probe the canonical target and not assume a single target device. If all initiators do not support this type of probing, then interoperability goes out the window. Mark's effort, mandates the support on the target device - which indirectly puts the mandate on the initiator to check, and makes sure there are not confusing answers from devices that otherwise would not respond to the canonical target. Is this replacing discovery - perhaps in some ways. The real issue is allowing i/o traffic to be rerouted to non-IANA TCP port numbers, which is a critical need of the proxy/routers/etc, which have little left in the socket 4-tuple to classify on if rerouting is not allowed.. I would expect the "discovery" (SLP, uPnP,etc) mechanisms would contain the "targets" that the proxy/router/etc, but the expectation is that they would all point at the well-known IANA TCP port number. The canonical processing simply allows the target device to shift session-specific traffic to target-assigned port numbers that can aid it's classification schemes. -- James julian_satran@il.ibm.com wrote: > Mark, > > I am not sure that we want to have your "canonical" target as a mandatory > construct. > It is good for proxies, routers, portals etc.? Don't the have better > mechanisms for discovery. > But why should you force it on some small box that has the name wired and > printed on it's label. > To reach the canonical you have to go through all the "login etc.". > > I think that all goes back to asking - why should we use iSCSI for > discovery? > > SLP, UPnP, Salutation, Jini all attempt discovery using a a general form > (good for iSCSI, Lpr printers etc.) > Can't we take the same path? > > Julo > > Mark Bakke on 12-05-2001 00:04:24 > > Please respond to Mark Bakke > > To: IPS > cc: > Subject: iSCSI: Canonical Targets > > It was pointed out during the interim meeting that the > canonical target is defined somewhat ambiguously, and may > be a bit too flexible for interoperability purposes. This > message is a stake in the ground for defining this a little > more tightly. > > So here are the new canonical target rules. Please let me > know if there are major problems with any of them. I tried > to define them a-la-carte, to make it easier to pick out any > that might cause trouble. > > 1. Each iSCSI implementation MUST include a canonical target. > > 2. A canonical target MUST be accessible at the default, IANA- > assigned TCP port on each IP address on which the iSCSI > implementation is listening for iSCSI connections. > > 3. A canonical target MUST NOT be used for SCSI commands. > > A canonical target is an iSCSI construct only, and does not > have a corresponding SCSI device. This means it may not > be used to access Logical Units. > > A session created to a canonical target is a discovery session > only, and once in full feature phase, is used only for text > commands and asynchronous messages. (Do any other commands make > sense)? > > 4. A device containing a single target MUST provide both the > canonical target and the real target. (This is implied by the > above requirements). > > An initiator connecting to such a device using only its IP address > would first connect to the canonical target, and use SendTargets > to obtain the iSCSI name of the real target. It would then create > a separate session to the real target. Essentially, this means > there's nothing special about a single-target device. > > 5. An iSCSI device MUST provide a unique iSCSI name for each of its > targets. Using the canonical target as a nameless iSCSI target > is not supported. > > 6. We can further specify the order in which the SendTargets response > fields are returned, to simplify things further, e.g. each target > in the SendTargets response MUST return these fields in this order: > > - A TargetName= field > - A TargetAlias= field (value left blank if there's no alias) > - One or more TargetAddress= fields > - Any vendor-specific fields (ignored by standard initiators) > > The above rules will make the behavior of various target implementations > identical, regardless of the number of interfaces or targets they > support. This will reduce the number of end-cases that initiator > implementations will have to handle. > > If we agree on these, we can edit them into the iSCSI and NDT documents. > > Additional questions to send input on: > > 1. Should a non-canonical target respond to a SendTargets command? > > 2. If so, should it respond only with addresses for its own target, > or should it respond with other targets, as a canonical target > might do? > > 3. If not, the initiator must connect to a canonical target to find > the other addresses of a target to which it is already connected; > is the information it has sufficient to do so? (I think the answer > is yes, given canonical requirement #2 above). > > -- > Mark A. Bakke > Cisco Systems > mbakke@cisco.com > 763.398.1054 From owner-ips@ece.cmu.edu Mon May 21 20:29:51 2001 Received: from ece.cmu.edu ([128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id UAA21170 for ; Mon, 21 May 2001 20:29:51 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f4LMxuP20007 for ips-outgoing; Mon, 21 May 2001 18:59:56 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from palrel10.hp.com (palrel10.hp.com [192.151.10.58]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f4LMxsH19997 for ; Mon, 21 May 2001 18:59:54 -0400 (EDT) Received: from hpbs5002.boi.hp.com (hpbs5002.boi.hp.com [15.2.216.28]) by palrel10.hp.com (Postfix) with ESMTP id 3EBA81F5EC for ; Mon, 21 May 2001 15:59:48 -0700 (PDT) Received: from xpabh4.corp.hp.com (xpabh4.corp.hp.com [15.58.136.1]) by hpbs5002.boi.hp.com with ESMTP (8.7.1/8.7.3 SMKit7.02) id QAA20293; Mon, 21 May 2001 16:59:47 -0600 (MDT) Received: by xpabh4.corp.hp.com with Internet Mail Service (5.5.2653.19) id ; Mon, 21 May 2001 15:58:57 -0700 Message-ID: <6BD67FFB937FD411A04F00D0B74FE87802A09089@xrose06.rose.hp.com> From: "KRUEGER,MARJORIE (HP-Roseville,ex1)" To: "'Santosh Rao'" , ips@ece.cmu.edu Subject: RE: iSCSI: response to second login (with same ISID) Date: Mon, 21 May 2001 15:58:50 -0700 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2653.19) Content-Type: text/plain; charset="iso-8859-1" Sender: owner-ips@ece.cmu.edu Precedence: bulk > I'd vote for (2) for the following reasons : > > a) It is possible for initiators to use the ISID as a scheme to > establish persistence of their O.S. device files. In doing so, > initiators would attempt to obtain the same ISID for a given > session to > a given target ports across boots. (and rightly so, for > enabling targets > to track persistent reservation like properties, etc). > > In the event where the initiator went through a non-orderly shutdown > which did not close all sessions, it would attempt to re-login on > re-boot with the same ISID. If the target rejected this re-login, per > (1) (citing an invalid ISID), the initiator would no longer be able to > maintain persistence of the session to an ISID. > > b) Using (2) allows the hosts to hint to the target that the previous > session is now stale and thus triggers clean-up of stale session > resources. This seems to me to provide another example why ISID should *not* be used for persistent reservation re-establishment. It's ambiguous whether this is a case of "defective initiator" or "initiator trying to re-claim persistent reservations" and there is no way for iSCSI to resolve this question. Since persistent reservations are SCSI layer things, lets try to find a way to keep their implementation from forcing unnecessary limitations at the iSCSI layer. The use of init. name+ISID+target name for identifying persistent reservations needs further discussion before we allow it to creep into iSCSI protocol rules. I've very uncomfortable with treating ISID like a fixed address just because SCSI persistent reservations don't have a sufficient SCSI layer mechanism. Seems like this will force all kinds of layering violations into iSCSI. Marj From owner-ips@ece.cmu.edu Mon May 21 22:23:56 2001 Received: from ece.cmu.edu ([128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id WAA22913 for ; Mon, 21 May 2001 22:23:56 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f4M0wB629345 for ips-outgoing; Mon, 21 May 2001 20:58:11 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from zmamail05.zma.compaq.com (zmamail05.zma.compaq.com [161.114.64.105]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f4M0wAH29340 for ; Mon, 21 May 2001 20:58:10 -0400 (EDT) Received: by zmamail05.zma.compaq.com (Postfix, from userid 12345) id B9FCD1DA1; Mon, 21 May 2001 20:58:00 -0400 (EDT) Received: from exchou-gh02.cca.cpqcorp.net (exchou-gh02.cca.cpqcorp.net [16.110.248.202]) by zmamail05.zma.compaq.com (Postfix) with ESMTP id 6E37D1F0E; Mon, 21 May 2001 20:58:00 -0400 (EDT) Received: by exchou-gh02.cca.cpqcorp.net with Internet Mail Service (5.5.2652.78) id ; Mon, 21 May 2001 19:57:59 -0500 Message-ID: <31891B757C09184BBFEC5275F85D5595104D62@cceexc18.americas.cpqcorp.net> From: "Martin, Nick" To: "'ips@ece.cmu.edu'" Cc: "'Jim Hafner'" Subject: RE: iSCSI: response to second login (with same ISID) Date: Mon, 21 May 2001 19:57:48 -0500 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2652.78) Content-Type: text/plain Sender: owner-ips@ece.cmu.edu Precedence: bulk All, I would hope for option 1. There is no protection to prevent a user mode program in a host from opening a TCP connection to the target and attempting an iSCSI login using any SID it likes. It could easily affect the operation of another copy of this same application or of a kernel driver, neither of which should have to suffer disruption. I would further suggest, that it would be appropriate for the target to "test" the original session to see if it is still working. If the original session turns out to be non-operational although not explicitly logged out, then it could be cleaned up. When cleanup is successfully completed, there is no conflict and the new request becomes a fresh login. I suggest that it is not required, but maybe not unresonable for the target to delay the new login request while it tests and then potentially cleans up after the previous user of this session id. In the worst case, it might require an initiator legitimately attempting to re-establish a session which to it appears non-functional, to try to re-login twice (with some reasonable delay between these). The first time should cause the target to test the original session, which it should after a short timeout detect to be non-operational, triggering a cleanup. System wide co-ordination of the use of SID within a multi-user host/initiator will sometimes be problematic. I would stand on the side of protecting the current (and presumably still active) owner of the sesssion ID from potential tresspassers. Thanks, Nick -----Original Message----- From: Jim Hafner [mailto:hafner@almaden.ibm.com] Sent: Monday, May 21, 2001 11:15 AM To: ips@ece.cmu.edu Subject: iSCSI: response to second login (with same ISID) Folks, Santosh has requested a response to this question and I don't think I've noticed an answer. It's one of those "dot the i" things that we need to complete. The current thinking is that an iSCSI initiator must use a different ISID for each session it opens with a specific iSCSI target. We need to decide on the iSCSI target's response when the iSCSI initiator attempts to break this rule. The choices are: 1) REJECT the login with some error code (that the iSCSI initiator can understand or infer to mean "ISID in use") and leave the pre-existing session alone 2) CLOSE the pre-existing session (and abort all it's tasks) and then process the new login fresh. Personally, I have no strong feeling either way, but lean towards option (1). It's simple, clean and doesn't involve any interruption. Anybody else got any opinions? Jim Hafner From owner-ips@ece.cmu.edu Mon May 21 22:25:21 2001 Received: from ece.cmu.edu ([128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id WAA22928 for ; Mon, 21 May 2001 22:25:21 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f4M1DAh00562 for ips-outgoing; Mon, 21 May 2001 21:13:10 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from palrel1.hp.com (palrel1.hp.com [156.153.255.242]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f4M1D9H00558 for ; Mon, 21 May 2001 21:13:09 -0400 (EDT) Received: from hpcuhe.cup.hp.com (hpcuhe.cup.hp.com [15.0.80.203]) by palrel1.hp.com (Postfix) with ESMTP id 2BAA716BB; Mon, 21 May 2001 18:13:08 -0700 (PDT) Received: from cup.hp.com (santoshr@hpindhhm.cup.hp.com [15.8.80.197]) by hpcuhe.cup.hp.com (8.9.3 (PHNE_18979)/8.9.3 SMKit7.02) with ESMTP id SAA03603; Mon, 21 May 2001 18:13:03 -0700 (PDT) Message-ID: <3B09BD3C.5B67DB77@cup.hp.com> Date: Mon, 21 May 2001 18:13:32 -0700 From: Santosh Rao Organization: Hewlett Packard, Cupertino. X-Mailer: Mozilla 4.7 [en] (X11; U; HP-UX B.11.00 9000/778) X-Accept-Language: en MIME-Version: 1.0 To: "Martin, Nick" Cc: "'ips@ece.cmu.edu'" , "'Jim Hafner'" Subject: Re: iSCSI: response to second login (with same ISID) References: <31891B757C09184BBFEC5275F85D5595104D62@cceexc18.americas.cpqcorp.net> Content-Type: multipart/mixed; boundary="------------995E5DD5676246D664378BD5" Sender: owner-ips@ece.cmu.edu Precedence: bulk This is a multi-part message in MIME format. --------------995E5DD5676246D664378BD5 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit "Martin, Nick" wrote: > > All, > > I would hope for option 1. There is no protection to prevent a user mode > program in a host from opening a TCP connection to the target and attempting > an iSCSI login using any SID it likes. It could easily affect the operation > of another copy of this same application or of a kernel driver, neither of > which should have to suffer disruption. If I read Steph Bailey's comments on this right (from the ERT work), the target would have to authenticate the 2nd login on the same ISID [as it does with any other login] and only take action when authentication is successful. Such a malicious user program would not pass login authentication and therefore, would be unable to trigger such a disruption. > I would further suggest, that it would be appropriate for the target to > "test" the original session to see if it is still working. If the original > session turns out to be non-operational although not explicitly logged out, > then it could be cleaned up. When cleanup is successfully completed, there > is no conflict and the new request becomes a fresh login. > > I suggest that it is not required, but maybe not unresonable for the target > to delay the new login request while it tests and then potentially cleans up > after the previous user of this session id. This approach can lead to long latencies in servicing logins, affecting initiator boot-up time. Regards, Santosh --------------995E5DD5676246D664378BD5 Content-Type: text/x-vcard; charset=us-ascii; name="santoshr.vcf" Content-Description: Card for Santosh Rao Content-Disposition: attachment; filename="santoshr.vcf" Content-Transfer-Encoding: 7bit begin:vcard n:Rao;Santosh tel;work:408-447-3751 x-mozilla-html:FALSE org:Hewlett Packard, Cupertino.;SISL adr:;;19420, Homestead Road, M\S 43LN, ;Cupertino.;CA.;95014.;USA. version:2.1 email;internet:santoshr@cup.hp.com title:Software Design Engineer x-mozilla-cpt:;21088 fn:Santosh Rao end:vcard --------------995E5DD5676246D664378BD5-- From owner-ips@ece.cmu.edu Mon May 21 22:27:47 2001 Received: from ece.cmu.edu ([128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id WAA22963 for ; Mon, 21 May 2001 22:27:46 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f4M0i0728231 for ips-outgoing; Mon, 21 May 2001 20:44:00 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from atlrel1.hp.com (atlrel1.hp.com [156.153.255.210]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f4M0hwH28226 for ; Mon, 21 May 2001 20:43:58 -0400 (EDT) Received: from hpcuhe.cup.hp.com (hpcuhe.cup.hp.com [15.0.80.203]) by atlrel1.hp.com (Postfix) with ESMTP id D18A41081 for ; Mon, 21 May 2001 20:43:57 -0400 (EDT) Received: from cup.hp.com (santoshr@hpindhhm.cup.hp.com [15.8.80.197]) by hpcuhe.cup.hp.com (8.9.3 (PHNE_18979)/8.9.3 SMKit7.02) with ESMTP id RAA01124; Mon, 21 May 2001 17:38:51 -0700 (PDT) Message-ID: <3B09B538.8561E22A@cup.hp.com> Date: Mon, 21 May 2001 17:39:20 -0700 From: Santosh Rao Organization: Hewlett Packard, Cupertino. X-Mailer: Mozilla 4.7 [en] (X11; U; HP-UX B.11.00 9000/778) X-Accept-Language: en MIME-Version: 1.0 To: "KRUEGER,MARJORIE (HP-Roseville,ex1)" Cc: ips@ece.cmu.edu Subject: Re: iSCSI: response to second login (with same ISID) References: <6BD67FFB937FD411A04F00D0B74FE87802A09089@xrose06.rose.hp.com> Content-Type: multipart/mixed; boundary="------------EE43B3BFBD86CAA7068E6B54" Sender: owner-ips@ece.cmu.edu Precedence: bulk This is a multi-part message in MIME format. --------------EE43B3BFBD86CAA7068E6B54 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit "KRUEGER,MARJORIE (HP-Roseville,ex1)" wrote: > Since > persistent reservations are SCSI layer things, lets try to find a way to > keep their implementation from forcing unnecessary limitations at the iSCSI > layer. The use of init. name+ISID+target name for identifying persistent > reservations needs further discussion before we allow it to creep into iSCSI > protocol rules. I've very uncomfortable with treating ISID like a fixed > address just because SCSI persistent reservations don't have a sufficient > SCSI layer mechanism. A fixed port identifier of some form is a basic O.S. requirement to be able to persistently bind the LUNs it sees to some form of device files. This binding is based on either a physical port identifier or name (ex : pSCSI target id, FC nport_id, FC port WWN) or in the case of logical service delivery ports like iSCSI to a logical port identifier such as the ISID. A fixed address usage is not only for persistent reservation semantics. The host O.S. SCSI stacks need *something* that allows them to persistently bind device files to the same LUNs on each reboot. The ISID is one form of achieving this. > Seems like this will force all kinds of layering > violations into iSCSI. Not clear why this is so. Consider the ISID as a logical port identifier that id considered as a SCSI transport layer endpoint. Regards, Santosh --------------EE43B3BFBD86CAA7068E6B54 Content-Type: text/x-vcard; charset=us-ascii; name="santoshr.vcf" Content-Description: Card for Santosh Rao Content-Disposition: attachment; filename="santoshr.vcf" Content-Transfer-Encoding: 7bit begin:vcard n:Rao;Santosh tel;work:408-447-3751 x-mozilla-html:FALSE org:Hewlett Packard, Cupertino.;SISL adr:;;19420, Homestead Road, M\S 43LN, ;Cupertino.;CA.;95014.;USA. version:2.1 email;internet:santoshr@cup.hp.com title:Software Design Engineer x-mozilla-cpt:;21088 fn:Santosh Rao end:vcard --------------EE43B3BFBD86CAA7068E6B54-- From owner-ips@ece.cmu.edu Mon May 21 23:26:53 2001 Received: from ece.cmu.edu ([128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id XAA24537 for ; Mon, 21 May 2001 23:26:52 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f4M2GZ405022 for ips-outgoing; Mon, 21 May 2001 22:16:35 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from palrel10.hp.com (palrel10.hp.com [192.151.10.58]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f4M2GYH05018 for ; Mon, 21 May 2001 22:16:34 -0400 (EDT) Received: from core.rose.hp.com (core.rose.hp.com [15.43.208.100]) by palrel10.hp.com (Postfix) with ESMTP id 0730C1FAC9 for ; Mon, 21 May 2001 19:16:26 -0700 (PDT) Received: (from cbm@localhost) by core.rose.hp.com (8.8.6 (PHNE_14041)/8.8.6 SMKit7.02) id TAA20432 for ips@ece.cmu.edu; Mon, 21 May 2001 19:17:33 -0700 (PDT) Message-Id: <200105220217.TAA20432@core.rose.hp.com> Subject: Re: DRAFT Nashua interim meeting minutes To: ips@ece.cmu.edu Date: Mon, 21 May 2001 19:17:33 PDT Reply-To: cbm@rose.hp.com From: "Mallikarjun C." X-Mailer: Elm [revision: 212.4] Sender: owner-ips@ece.cmu.edu Precedence: bulk David, I just have two comments on this note. >Within-connection recovery reissues command on same connection to fill > CmdSN holes. Has to be on same connection due to connection > allegiance. If there is a CmdSN hole, the command should be unacknowledged causing the initiator to re-use the same CmdSN on retry. I don't see connection allegiance here since the CmdSN was never ack'ed. IMHO, technically it should be able to be re-issued on any connection in the session (though don't know why an initiator would do that). >These errors require framing support over TCP - > in the absence of framing, the connection has to > be terminated. This is generally true, since most likely it would be a header digest error. If the CmdSN hole was due to a data digest error on immediate data, original framing is still intact. -- Mallikarjun Mallikarjun Chadalapaka Networked Storage Architecture Network Storage Solutions Organization MS 5668 Hewlett-Packard, Roseville. cbm@rose.hp.com From owner-ips@ece.cmu.edu Mon May 21 23:27:14 2001 Received: from ece.cmu.edu ([128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id XAA24548 for ; Mon, 21 May 2001 23:27:13 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f4M1x1r03825 for ips-outgoing; Mon, 21 May 2001 21:59:01 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from palrel1.hp.com (palrel1.hp.com [156.153.255.242]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f4M1x0H03821 for ; Mon, 21 May 2001 21:59:00 -0400 (EDT) Received: from core.rose.hp.com (core.rose.hp.com [15.43.208.100]) by palrel1.hp.com (Postfix) with ESMTP id 73876166E for ; Mon, 21 May 2001 18:58:59 -0700 (PDT) Received: (from cbm@localhost) by core.rose.hp.com (8.8.6 (PHNE_14041)/8.8.6 SMKit7.02) id TAA17606 for ips@ece.cmu.edu; Mon, 21 May 2001 19:00:06 -0700 (PDT) Message-Id: <200105220200.TAA17606@core.rose.hp.com> Subject: RE: DRAFT Nashua interim meeting minutes To: ips@ece.cmu.edu Date: Mon, 21 May 2001 19:00:06 PDT Reply-To: cbm@rose.hp.com From: "Mallikarjun C." X-Mailer: Elm [revision: 212.4] Sender: owner-ips@ece.cmu.edu Precedence: bulk David, >> There was discussion in the security portion of the agenda to the effect >> that "if we mandate IPSec with it's data integrity check capabilities, is >> there further need to specify error recovery"? > >That topic was picked up again in the error recovery session. In essence, >if ESP is in >use, CRC failures at the iSCSI level will essentially never happen, allowing >at least >within-session and within-connection (and presumably) within-command >recovery >to be dropped, I want to add one comment here, though the right context to discuss this would be while making a case for "CRC shim"/ESP-with-CRC. Within-session recovery should continue to live on even with perfect TCP, to transparently effect connection recovery on NIC failures. The Nashua error recovery slides explicitly make this point. It however may not have to be mandatory with a perfect/shored-up TCP. -- Mallikarjun Mallikarjun Chadalapaka Networked Storage Architecture Network Storage Solutions Organization MS 5668 Hewlett-Packard, Roseville. cbm@rose.hp.com >but the HMAC in ESP is computationally expensive leading to >the >interest in a CRC shim. I've added explanation of this to the point in the >error >recovery session where the CRC shim was discussed. > >Thanks, >--David > >--------------------------------------------------- >David L. Black, Senior Technologist >EMC Corporation, 42 South St., Hopkinton, MA 01748 >+1 (508) 435-1000 x75140 FAX: +1 (508) 497-8500 >black_david@emc.com Mobile: +1 (978) 394-7754 >--------------------------------------------------- > > From owner-ips@ece.cmu.edu Tue May 22 09:16:35 2001 Received: from ece.cmu.edu ([128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id JAA16873 for ; Tue, 22 May 2001 09:16:34 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f4MBA9w16025 for ips-outgoing; Tue, 22 May 2001 07:10:09 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from siaar1aa.compuserve.com (siaar1aa.compuserve.com [149.174.40.9]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f4MBA7H16021 for ; Tue, 22 May 2001 07:10:07 -0400 (EDT) Received: (from mailgate@localhost) by siaar1aa.compuserve.com (8.9.3/8.9.3/SUN-REL-1.3) id HAA12859 for ips@ece.cmu.edu; Tue, 22 May 2001 07:10:02 -0400 (EDT) Received: from compuserve.com (chi-tgn-goi-vty20.as.wcom.net [216.192.134.20]) by siaar1aa.compuserve.com (8.9.3/8.9.3/SUN-REL-1.3) with ESMTP id HAA12825 for ; Tue, 22 May 2001 07:09:50 -0400 (EDT) Message-ID: <3B0A33BB.6C54ED87@compuserve.com> Date: Tue, 22 May 2001 04:39:08 -0500 From: Ralph Weber Reply-To: ENDL_TX@computer.org X-Mailer: Mozilla 4.7 [en]C-CCK-MCD NSCPCD47 (Win98; I) X-Accept-Language: en,pdf MIME-Version: 1.0 To: IPS Reflector Subject: Re: FCIP - Proposed changes in support of common FC encapsulation References: <63616B261CEFD411834D00D0B7A86CF7048FC7@lucy.TREBIA.COM> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-ips@ece.cmu.edu Precedence: bulk Content-Transfer-Encoding: 7bit This is the complete list of changes need to be made in FCIP to make correctly reference to the Common FC Encapsulation draft as updated to reflect the correction proposed by Sudhir Srinivasan. I am currently investigating whether an IANA Considerations annex will be required. If one is necessary it will be proposed separately. The changes are presented in the order that they are expected to appear in draft-ietf-ips-fcovertcpip-03.txt. In the section 3 objectives list (top of page 3 in draft-ietf-ips- fcovertcpip-02.txt), replace: 1) specify the encapsulation and mapping of FC frames using the FC encapsulation method specified in . with: 1) specify the encapsulation and mapping of FC frames employing the common FC Frame Encapsulation[23]. In the 1st paragraph of section 4.2, replace: The FCIP protocol specifies the TCP/IP encapsulation, mapping and routing of FC frames and applies these mechanisms to an FC network utilizing IP for its backbone, or more generally, between any two FC devices. with: The FCIP protocol specifies the TCP/IP encapsulation (employing the common FC Frame Encapsulation described in [23]), mapping and routing of FC frames and applies these mechanisms to an FC network utilizing IP for its backbone, or more generally, between any two FC devices. In section 4.2 list item 6, replace: 6. An FCIP device MAY send FC encapsulated TCP/IP packets to more than one FCIP device. However, these encapsulated packets are treated as separate instances and are not correlated in any way by the FCIP protocol devices. with: 6. An FCIP device MAY send encapsulated FC frames to more than one FCIP device. However, these encapsulated FC frames are treated as separate instances and are not correlated in any way by the FCIP protocol devices Delete list item 11 in section 4.2 since it is covered in the intro- ductory text at the beginning of the section. For reference, list item 11 currently reads: 11. FCIP uses the common encapsulation method as specified by . In the 3rd paragraph of section 4.3, replace: Each FCIP data frame is built by adding an FCIP header to one FC frame delivered to the FCIP endpoint for transport. The FCIP data frames are handed in their entirety to TCP; TCP is responsible for delivering the same series of FCIP data frames to the receiving side in the same order as they are transmitted by the sending FCIP device. The FCIP device MUST find the FCIP headers and deliver the FC frames wrapped inside the FCIP data frames to the correct FC ports connected to the FCIP device. with: An FCIP data frame is built by encapsulating one FC frame delivered to the FCIP endpoint for transport using the encapsulation format described in the common FC Frame Encapsulation [23] and in section 5.2 of this document. The FCIP data frames are handed in their entirety to TCP; TCP is responsible for delivering the same series of FCIP data frames to the receiving side in the same order as they are transmitted by the sending FCIP device. The FCIP receiving device MUST reverse the FC frame encapsulation process, verify the correctness of the header and deliver the FC frames to the correct FC ports connected to the FCIP device. When header errors are encountered in FCIP data frames, the FCIP device MUST take corrective action in accordance with Section 8.1. In section 5.1, the last paragraph of "SOF and EOF Delimiters:", replace: When an FC frame is encapsulated and sent over a byte-oriented interface, the SOF and EOF delimiters are represented as sequences of four consecutive bytes, which carry the equivalent Class of Service and frame termination information as the FC ordered sets. This form of encoding can not provide unambiguous identification of frame beginning and end, however, and must rely on other mechanisms provided by the encapsulation protocol. With: When an FC frame is encapsulated and sent over a byte-oriented interface, the SOF and EOF delimiters are represented as sequences of four consecutive bytes, which carry the equivalent Class of Service and frame termination information as the FC ordered sets. The representation of SOF and EOF in an encapsulation FC frame is described in the common FC Frame Encapsulation [23]. Note: If the deleted sentence is considered important, then it should be moved to section 3.3 of the common encapsulation draft. Section 5.2 appears to have been omitted as a placeholder for the FCIP encapsulation details. Therefore, the following new text for section 5.2 is proposed. 5.2 FCIP Encapsulation of FC Frames The FCIP encapsulation of FC frames employs the common FC Frame Encapsulation [23]. The features from the common FC Frame Encapsulation that are unique to individual protocols SHALL be applied as follows for the FCIP encapsulation of FC frames. The Protocol# field SHALL contain 1 in accordance with the IANA Considerations annex of the common FC Frame Encapsulation [23]. The Protocol Specific field SHALL have the format shown in figure xx. Note: the word numbers in figure xx are relative to the complete FC frame encapsulation header, not to the Protocol Specific field. W|------------------------------Bit------------------------------| o| | r|3 3 2 2 2 2 2 2 2 2 2 2 1 1 1 1 1 1 1 1 1 1 | d|1 0 9 8 7 6 5 4 3 2 1 0 9 8 7 6 5 4 3 2 1 0 9 8 7 6 5 4 3 2 1 0| +---------------------------------------------------------------+ 1| replication of encapsulation word 0 | +-------------------------------+-------------------------------+ 2| reserved | -reserved | +-------------------------------+-------------------------------+ Fig. xx - FCIP Usage of common FC Frame Encapsulation Protocol Specific field Word 1 of the Protocol Specific field SHALL contain an exact copy of word 0 in the common FC Frame Encapsulation [23]. Word 2 of the Protocol Specific field is reserved for future enhancements to the FCIP protocol. The reserved field (bits 31-16 in word 2): SHALL contain 0. The -reserved field (bits 15-0 in word 2): SHALL contain 65535 (or 0xFFFF). The CRCV (CRC Valid) Flag SHALL be set to 0. The CRC field SHALL be set to 0. In section 5.3 list item 4, replace: After connection establishment, FCIP devices use the FCIP frame encapsulation as defined in [common encapsulation document]. with: After connection establishment, FCIP devices use the FCIP frame encapsulation defined in the common FC Frame Encapsulation [23] and in section 5.2 of this document. In the 1st paragraph of 5.4.1, replace: These FCIP ELS messages use the same encapsulation mechanism as described in TBD. with: These FCIP ELS messages use the same encapsulation mechanism as described in 5.2. In the 3rd paragraph of 5.4.2, replace: FCIP Encapsulated frame with FCIP frame In the 1st paragraph of 8.5, replace: FCIP data frame with FCIP frame In Section 11 (References) add the following: [23] Weber, Rajagopal, Travostino, Chau, McDonnell, Monia Merhar, "FC Frame Encapsulation", draft-ietf-ips-fcencapsulation-__.txt (RFC reference and date to be added during standards action). RFC1072, October 1988 From owner-ips@ece.cmu.edu Tue May 22 15:25:37 2001 Received: from ece.cmu.edu ([128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id PAA01805 for ; Tue, 22 May 2001 15:25:37 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f4MGrPA11112 for ips-outgoing; Tue, 22 May 2001 12:53:25 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from d06lmsgate.uk.ibm.COM (d06lmsgate.uk.ibm.com [195.212.29.1]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f4MGrLH11103 for ; Tue, 22 May 2001 12:53:22 -0400 (EDT) Received: from d06relay02.portsmouth.uk.ibm.com (d06relay02.portsmouth.uk.ibm.com [9.166.84.148]) by d06lmsgate.uk.ibm.COM (1.0.0) with ESMTP id RAA130400 for ; Tue, 22 May 2001 17:35:45 +0100 From: julian_satran@il.ibm.com Received: from d06mta03.portsmouth.uk.ibm.com (d06mta03_cs0 [9.180.35.1]) by d06relay02.portsmouth.uk.ibm.com (8.8.8m3/NCO v4.96) with SMTP id RAA96424 for ; Tue, 22 May 2001 17:53:10 +0100 Received: by d06mta03.portsmouth.uk.ibm.com(Lotus SMTP MTA v4.6.5 (863.2 5-20-1999)) id 80256A54.005CC0EE ; Tue, 22 May 2001 17:53:07 +0100 X-Lotus-FromDomain: IBMIL@IBMDE@IBMGB To: "Barry Reinhold" cc: "ISCSI" Message-ID: <80256A54.005CBB5E.00@d06mta03.portsmouth.uk.ibm.com> Date: Tue, 22 May 2001 19:58:37 +0300 Subject: Re: iSCSI: Login State Tables Mime-Version: 1.0 Content-type: text/plain; charset=us-ascii Content-Disposition: inline Sender: owner-ips@ece.cmu.edu Precedence: bulk Barry, Thanks for sharing your thoughts with us. Some of the things you mention are already in our text (like forbidding a negotiation on an F=1 response) as we agreed. We will give careful consideration to all the rest and I will get the appendix to this list soon. We are considering removing most of the parameters from the ALL category and making them unchangeable by a a MODE set (although the mode set will be accepted - for compatibility) - as some of you (most vocally Santosh Rao) have requested. Regards, Julo "Barry Reinhold" on 21-05-2001 20:00:27 Please respond to "Barry Reinhold" To: Julian Satran/Haifa/IBM@IBMIL cc: "ISCSI" Subject: iSCSI: Login State Tables Julian and interested parties: The login mechanisms as documented in 06 concerns me, as there is a lot of room for "interpretation" of the current text. I have attempted to pull the 06 statements together into a state table in order to avoid what I fear may be a bunch of interoperability headaches. The state table, however, also places some restrictions on the process that is not in the current text. In particular: 1. It forces all negotiations to go through an explicit security handshake. If security is not being used the handshake still takes place. This allows a target to demand security in a clean manner even if the initiator did not request it. 2. It explicitly places most parameters into an operation class and a security class. These parameters may only be exchanged during defined phases of the login process. (This is intended to remove ambiguity associated with the use of operational parameters that were negotiated in the security phase). There is a set of parameters that are really not in either camp. These parameters can be used in both the security phase and the operational phase. Once negotiated their values would remain established unless explicitly renegotiated. [Note: Another choice here is to place all operational parameters into this category and apply the rule that all negotiated values MUST be maintained by both parties unless renegotiated.] 3. It does not allow a target to respond to a Text PDU that has a F=1 setting with new key=value pairs in a login PDU with F=1. 4. It allows the initiator to continue negotiations if it was happy with a context but additional negotiations made it unhappy. This allows the protocol to address situations in which there is a coupling between parameters. This is intended to mostly be a clarification of what I think 06 is attempting to state, however, you may think differently. If there is basic agreement that this is what we are trying to do in 06 I would like to see something of this nature added to the draft. Parameter classification: Operational Parameters: Maxconnections, FMarker, RFMarker, RFMarkInt, SFMarkInt, IFMarkInt, InitialR2T, BidiInitialR2T, ImmediateData, DataPDULength, FirstBurstSize, LogoutLoginMinTime, LogoutLoginMaxTime, EnableACA, MaxOutstandingR2T, DataOrder, BootSession, Glen-Turner Vendor Specific Keys. Not Classified - TargetName, InitiatorName, TargetAlias, InitiatorAlias, TargetAddress, SendTargets Security - DataDigest, HeaderDigest, AuthMethod Identifiers used in login state table SecDone - An indicator that takes on the value True (T) or False (F). This indicator is when the local device is satisfied with the current security context and is ready to move to the operational parameter negotiation phase. When a PDU has been received the value of SecDone is that established after the contents of the received PDU have been evaluated. OpDone - This is the logical equivalent of SecDone for the Operational Phase. Fsent - This variable is used to indicate that the last PDU sent by this device had the F bit set to 1. [Note: The meaning of the F bit is changed by this usage. It no longer means that this is the last PDU to be sent by the initiator, in this context it means that this is the last text PDU I will send if the target doesn't continue the negotiation.] SCC - A short hand notation for the key "SecurityContextComplete" Rcv_Text- A valid Text PDU was received Rcv_Login-A login PDU was received Xmit_Text-A Text PDU is transmitted Perror - A protocol error was detected Initiator Login State Transition Table: +--------+-------------------------+--------+-------------------------+----- --+ |C State | Event(s) and conditions |N State | Actions | Notes | +--------+-------------------------+--------+-------------------------+----- --+ |START |(TCP connection Est- |SEC-END |xmit login(SCC=yes) | 1 | | |ablished) & SecDone | | | | +--------+-------------------------+--------+-------------------------+----- --+ |START |(TCP connection Est- |SEC-NEG |xmit login | | | |ablished & not(SecDone) | | | | +--------+-------------------------+--------+-------------------------+----- --+ |SEC-NEG |Rcv_Text & SecDone=F |SEC-NEG |Xmit_Text | | +--------+-------------------------+--------+-------------------------+----- --+ |SEC-NEG |Rcv_Text & SecDone=T |SEC-END |xmit_Text(SCC=yes) | | +--------+-------------------------+--------+-------------------------+----- --+ |SEC-NEG |Perror |ABORT |TCP connection terminated| 2 | +--------+-------------------------+--------+-------------------------+----- --+ |SEC-END |Rcv_Text & SecDone=F |SEC-NEG |xmit_Text | 3 | +--------+-------------------------+--------+-------------------------+----- --+ |SEC-END |Rcv_Text & SecDone=T |SEC-NEG |xmit_Text | | +--------+-------------------------+--------+-------------------------+----- --+ |SEC-END |Rcv_Text(SCC=yes) & |OPER-NEG|xmit_Text | 4 | | |SecDone = T & OpDone = F | | | | +--------+-------------------------+--------+-------------------------+----- --+ |SEC-END |Rcv_Text(SCC=yes) & |OPER-END|xmit_Text | 4 | | |SecDone = T & OpDone = T | | | | +--------+-------------------------+--------+-------------------------+----- --+ |SEC-END |Perror |ABORT |TCP connection terminated| 5 | +--------+-------------------------+--------+-------------------------+----- --+ |OPER-NEG|Rcv_Text & OpDone = F |OPER-NEG|xmit_Text | | +--------+-------------------------+--------+-------------------------+----- --+ |OPER-NEG|Rcv_Text & OpDone = T |OPER-END|xmit_Text(F=1), Fsent=1 | | +--------+-------------------------+--------+-------------------------+----- --+ |OPER-NEG|Perror |ABORT |TCP connection terminated| 6 | +--------+-------------------------+--------+-------------------------+----- --+ |OPER-END|Rcv_Text & OpDone = F |OPER-NEG|xmit_Text, Fsent=0 | | +--------+-------------------------+--------+-------------------------+----- --+ |OPER-END|Rcv_Text & OpDone = T |OPER-END|xmit_Text(F=1), Fsent=1 | | +--------+-------------------------+--------+-------------------------+----- --+ |OPER-END|Rcv_Login(F=1) & Fsent=1 |FULL | | | +--------+-------------------------+--------+-------------------------+----- --+ |OPER-END|Perror |ABORT |TCP connection terminated| 7 | +--------+-------------------------+--------+-------------------------+----- --+ Notes: All - The F bit is set to zero (F=0) unless otherwise indicated in the login state table. All - Operational parameter key=value lists SHALL NOT be allowed before the security context has been completed. All - The Key=value pair "SecurityContextComplete=yes" must always be sent by itself 1 - No key=value pairs may be included in this PDU except "SecurityContextComplete=yes" 2 - Identified protocol errors are: Rcv_Text has F=1, Rcv_Text has operational parameters 3 - If the target sends a security parameter that causes the initiator to no longer be satisfied with the current security context the initiator continues the security negotiations. 4 - This and subsequent PDUs sent on this connection SHALL be transmitted using the negotiated security context. 5 - Identified protocol errors are Rcv_Text has F=1, Rcv_Text has operational parameters 6 - Identified protocol errors are: Rcv_Text has F=1 and Fsent is not set, Rcv_Text contains a security key 7 - Identified protocol errors are: Rcv_Text has F=1, A login PDU is received with new key=value parameters, A login PDU is received with F=0. Target Login State Transition Table: +--------+-------------------------+--------+-------------------------+----- --+ |C State | Event(s) and conditions |N State | Actions | Notes | +--------+-------------------------+--------+-------------------------+----- --+ |WAIT |rcv_login(scc=yes) & |OPER |xmit_Text(scc=yes) | 1 | | |SecDone = T | | | | +--------+-------------------------+--------+-------------------------+----- --+ |WAIT |rcv_login & SecDone = T |SECURITY|xmit_Text(scc=yes) | | +--------+-------------------------+--------+-------------------------+----- --+ |WAIT |rcv_login(scc=yes) & |SECURITY|xmit_Text | | | |SecDone = F | | | | +--------+-------------------------+--------+-------------------------+----- --+ |WAIT |rcv_login & SecDone = T |SECURITY|xmit_Text | | +--------+-------------------------+--------+-------------------------+----- --+ |WAIT |Perror |ABORT |xmit_Login(F=1) | 2 | | | | |status_class != 0 | | +--------+-------------------------+--------+-------------------------+----- --+ |SECURITY|rec_Text & SecDone = F |SECURITY| xmit_Text | | +--------+-------------------------+--------+-------------------------+----- --+ |SECURITY|rec_Text & SecDone = T |SECURITY| xmit_Text(scc=yes) | | +--------+-------------------------+--------+-------------------------+----- --+ |SECURITY|rec_text(Scc=yes) & |SECURITY| xmit_Text | | | | SecDone = F | | | | +--------+-------------------------+--------+-------------------------+----- --+ |SECURITY|rec_text(Scc=yes) & |OPER | xmit_Text(SCC=yes) | 3 | | | SecDone = T | | | | +--------+-------------------------+--------+-------------------------+----- --+ |SECURITY|Perror |ABORT |xmit_Login(F=1) | 4 | | | | |status_class != 0 | | +--------+-------------------------+--------+-------------------------+----- --+ |OPER |rcv_Text & OpDone = F |OPER |xmit_Text | | +--------+-------------------------+--------+-------------------------+----- --+ |OPER |rcv_Text & OpDone = T |OPER |xmit_Text | | +--------+-------------------------+--------+-------------------------+----- --+ |OPER |rcv_Text(F=1) |OPER |xmit_Text | | | |OpDone = F | | | | +--------+-------------------------+--------+-------------------------+----- --+ |OPER |rcv_Text(F=1) |Full |xmit_Login(F=1) | 5 | | |OpDone = T | | | | +--------+-------------------------+--------+-------------------------+----- --+ |OPER |Perror |ABORT |xmit_Login(F=1) | 6 | | | | |status_class != 0 | | +--------+-------------------------+--------+-------------------------+----- --+ 1. The Target SHALL NOT send a login PDU back unless there is an error and it is ending the login process. The next PDU to be received will be using the negotiated security context (externally established or default) 2. Identified protocol errors: The F bit is set in the login PDU, Operational parameters are given as key=value lists in the PDU 3. The device SHALL use the negotiated security on the all subsequent PDUs received for this connection. 4. Identified protocol errors: The F bit is set in the text PDU, a login PDU was received, operational parameters were received in the PDU. 5. The target SHALL NOT send any additional key=value lists in this PDU. 6. Identified protocol errors: Security parameters were received in the PDU. Barry Reinhold Principal Architect Trebia Networks barry.reinhold@trebia.com 603-868-5144/603-659-0885/978-929-0830 x138 From owner-ips@ece.cmu.edu Tue May 22 16:10:23 2001 Received: from ece.cmu.edu ([128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id QAA03025 for ; Tue, 22 May 2001 16:10:23 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f4MHfm315450 for ips-outgoing; Tue, 22 May 2001 13:41:48 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from d12lmsgate-3.de.ibm.com (d12lmsgate-3.de.ibm.com [195.212.91.201]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f4MHfkH15446 for ; Tue, 22 May 2001 13:41:46 -0400 (EDT) Received: from d12relay02.de.ibm.com (d12relay02.de.ibm.com [9.165.215.23]) by d12lmsgate-3.de.ibm.com (1.0.0) with ESMTP id TAA59396 for ; Tue, 22 May 2001 19:41:40 +0200 From: julian_satran@il.ibm.com Received: from d12mta02.de.ibm.com (d12mta01_cs0 [9.165.222.237]) by d12relay02.de.ibm.com (8.8.8m3/NCO v4.96) with SMTP id TAA166072 for ; Tue, 22 May 2001 19:41:24 +0200 Received: by d12mta02.de.ibm.com(Lotus SMTP MTA v4.6.5 (863.2 5-20-1999)) id C1256A54.00612C0F ; Tue, 22 May 2001 19:41:22 +0200 X-Lotus-FromDomain: IBMIL@IBMDE To: ips@ece.cmu.edu Message-ID: Date: Tue, 22 May 2001 20:47:16 +0300 Subject: Re: iSCSI: Canonical Targets Mime-Version: 1.0 Content-type: text/plain; charset=us-ascii Content-Disposition: inline Sender: owner-ips@ece.cmu.edu Precedence: bulk Jim, Analogies may be misleading. You have no new web server behind a web server (although you may have some redirection). You are suggesting a discovery for several targets behind a portal (address). Those can be disks, tapes, changers etc. in any combination. I know this is valuable (we have it in almost from version 0!). All discovery mechanisms have all this in already in a form or other (take a look at Salutation, Jini, UpNP). To build it under SCSI we are putting in place some artifacts (a different login - perhaps secure?, some commands to be done only there or not?) that are barely useful for other services. Why shouldn't we take the road of deferring the name discovery to the specialized (iSCSI) part of a more general discovery protocol? You don't expect the management applications to start using iSCSI for discovery do you? Julo Jim Hafner@IBMUS 21-05-2001 18:04 To: Julian Satran/Haifa/IBM@IBMIL cc: ips@ece.cmu.edu From: Jim Hafner/Almaden/IBM@IBMUS Subject: Re: iSCSI: Canonical Targets (Document link: Julian Satran - Mail) Julo, I was only teasing about the name. I don't care either, it's a nit. As for "discovery service". Here's a twist on the issue. In the case of most services, at a single ipaddress:tcpport, you'll find a single service entity (web server, ftp server, etc) and only AFTER you get through the first layer does finer granularity of discovery occur (e.g., through the query string of the url). But even in these cases, there's actually little of discovery going on. You pretty much have to know what you're going for first, or you drill manually (think of ftp server accessed through browser -- you get directory listing which you browse). What I think we're doing here is providing a "directory list" function for the single service entity that lives at the defined tcpport. This "listing" service can then be used to drill deeper. I don't see much that's different in what we're spec'ing and what needs to be spec'd for this purpose. We can rely on some mechanisms such as SLP and SNS for somethings. SLP helps to find the place to start for listing service. SNS has "all the answers". We're attempting to fill in the middle ground. Jim Hafner Julian Satran/Haifa/IBM@IBMIL@ece.cmu.edu on 05-19-2001 12:26:29 AM Sent by: owner-ips@ece.cmu.edu To: ips@ece.cmu.edu cc: Subject: Re: iSCSI: Canonical Targets Jim, The canonical attribute is associated with strictly conforming to a rule (as in canonical form for expressions or canon in music, or canonical religious text). I would use the term generic-name or class-name for it. But again the objection on the name was only minor. I strongly believe this is a service-delivery discovery item and we are repeating an old mistake - creating a specialized solution for a general problem. Any discovery service would be better than placing this in the iSCSI realm. Julo "Jim Hafner" on 14-05-2001 18:14:31 Please respond to "Jim Hafner" To: Julian Satran/Haifa/IBM@IBMIL cc: ips@ece.cmu.edu Subject: Re: iSCSI: Canonical Targets Julo, You object to the term "canonical"? My dictionary says canonical is "conforming to a general rule", but perhaps that isn't the best term. Would "well-known" would be better? Jim Hafner Julian Satran/Haifa/IBM@IBMIL@ece.cmu.edu on 05-12-2001 09:05:00 AM Sent by: owner-ips@ece.cmu.edu To: ips@ece.cmu.edu cc: Subject: Re: iSCSI: Canonical Targets Why would any discovery service be wiser/stupider than the mechanism we are having in N&D? If the issue of ports is widespread then others have it too and discovery may include all the nice things we have in iSCSI. I don't see any reason iSNS or an LDAP repository can't contain them. My only claim is that this adds weight to iSCSI. As for interoperability - I claim that adding features makes interoperability an issue. Removing them does not. The whole "canonical" (why canonical - there is nothing canonical about it) target idea is good but in another realm - I don't think it belongs to iSCSI - it belongs to discovery be it specific or general. I can leave with the mechanism as it is - but I really feel that we are on the wrong track. Regards, Julo James Smart on 12-05-2001 15:42:09 Please respond to james.smart@trebia.com To: Julian Satran/Haifa/IBM@IBMIL, Mark Bakke cc: ips@ece.cmu.edu Subject: Re: iSCSI: Canonical Targets I'm in agreement with Mark, that canonical target support is mandatory. The key is interoperability. The initiator must be mandated to probe the canonical target and not assume a single target device. If all initiators do not support this type of probing, then interoperability goes out the window. Mark's effort, mandates the support on the target device - which indirectly puts the mandate on the initiator to check, and makes sure there are not confusing answers from devices that otherwise would not respond to the canonical target. Is this replacing discovery - perhaps in some ways. The real issue is allowing i/o traffic to be rerouted to non-IANA TCP port numbers, which is a critical need of the proxy/routers/etc, which have little left in the socket 4-tuple to classify on if rerouting is not allowed.. I would expect the "discovery" (SLP, uPnP,etc) mechanisms would contain the "targets" that the proxy/router/etc, but the expectation is that they would all point at the well-known IANA TCP port number. The canonical processing simply allows the target device to shift session-specific traffic to target-assigned port numbers that can aid it's classification schemes. -- James julian_satran@il.ibm.com wrote: > Mark, > > I am not sure that we want to have your "canonical" target as a mandatory > construct. > It is good for proxies, routers, portals etc.? Don't the have better > mechanisms for discovery. > But why should you force it on some small box that has the name wired and > printed on it's label. > To reach the canonical you have to go through all the "login etc.". > > I think that all goes back to asking - why should we use iSCSI for > discovery? > > SLP, UPnP, Salutation, Jini all attempt discovery using a a general form > (good for iSCSI, Lpr printers etc.) > Can't we take the same path? > > Julo > > Mark Bakke on 12-05-2001 00:04:24 > > Please respond to Mark Bakke > > To: IPS > cc: > Subject: iSCSI: Canonical Targets > > It was pointed out during the interim meeting that the > canonical target is defined somewhat ambiguously, and may > be a bit too flexible for interoperability purposes. This > message is a stake in the ground for defining this a little > more tightly. > > So here are the new canonical target rules. Please let me > know if there are major problems with any of them. I tried > to define them a-la-carte, to make it easier to pick out any > that might cause trouble. > > 1. Each iSCSI implementation MUST include a canonical target. > > 2. A canonical target MUST be accessible at the default, IANA- > assigned TCP port on each IP address on which the iSCSI > implementation is listening for iSCSI connections. > > 3. A canonical target MUST NOT be used for SCSI commands. > > A canonical target is an iSCSI construct only, and does not > have a corresponding SCSI device. This means it may not > be used to access Logical Units. > > A session created to a canonical target is a discovery session > only, and once in full feature phase, is used only for text > commands and asynchronous messages. (Do any other commands make > sense)? > > 4. A device containing a single target MUST provide both the > canonical target and the real target. (This is implied by the > above requirements). > > An initiator connecting to such a device using only its IP address > would first connect to the canonical target, and use SendTargets > to obtain the iSCSI name of the real target. It would then create > a separate session to the real target. Essentially, this means > there's nothing special about a single-target device. > > 5. An iSCSI device MUST provide a unique iSCSI name for each of its > targets. Using the canonical target as a nameless iSCSI target > is not supported. > > 6. We can further specify the order in which the SendTargets response > fields are returned, to simplify things further, e.g. each target > in the SendTargets response MUST return these fields in this order: > > - A TargetName= field > - A TargetAlias= field (value left blank if there's no alias) > - One or more TargetAddress= fields > - Any vendor-specific fields (ignored by standard initiators) > > The above rules will make the behavior of various target implementations > identical, regardless of the number of interfaces or targets they > support. This will reduce the number of end-cases that initiator > implementations will have to handle. > > If we agree on these, we can edit them into the iSCSI and NDT documents. > > Additional questions to send input on: > > 1. Should a non-canonical target respond to a SendTargets command? > > 2. If so, should it respond only with addresses for its own target, > or should it respond with other targets, as a canonical target > might do? > > 3. If not, the initiator must connect to a canonical target to find > the other addresses of a target to which it is already connected; > is the information it has sufficient to do so? (I think the answer > is yes, given canonical requirement #2 above). > > -- > Mark A. Bakke > Cisco Systems > mbakke@cisco.com > 763.398.1054 From owner-ips@ece.cmu.edu Tue May 22 16:25:24 2001 Received: from ece.cmu.edu ([128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id QAA03465 for ; Tue, 22 May 2001 16:25:23 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f4MIg5320771 for ips-outgoing; Tue, 22 May 2001 14:42:05 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from smtp013.mail.yahoo.com (smtp013.mail.yahoo.com [216.136.173.57]) by ece.cmu.edu (8.11.0/8.10.2) with SMTP id f4MIg3H20765 for ; Tue, 22 May 2001 14:42:03 -0400 (EDT) Received: from sdsl-216-36-75-164.dsl.sjc.megapath.net (HELO somesh) (216.36.75.164) by smtp.mail.vip.sc5.yahoo.com with SMTP; 22 May 2001 18:42:02 -0000 X-Apparently-From: Reply-To: From: "Somesh Gupta" To: Subject: RE: TCP Framing (considered helpful?) Date: Tue, 22 May 2001 11:34:40 -0700 Message-ID: MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 (Normal) X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook IMO, Build 9.0.2416 (9.0.2910.0) Importance: Normal In-Reply-To: <20010521135240.CB05294006@sandmail.sandburst.com> X-MimeOLE: Produced By Microsoft MimeOLE V5.00.2919.6700 Sender: owner-ips@ece.cmu.edu Precedence: bulk Content-Transfer-Encoding: 7bit I think if the framing solution finds favor with the TCP "gods", it is a much better alternative than the markers (which I was a proponent of - but only because I did not believe that a framing mechanism would be approved). Even if the framing mechanism is not approved, an alternate mechanism which provides for alignment of iSCSI headers at regular intervals (at negotiate progression in TCP sequence #s e.g. 0x12345, 0x22345, 0x32345 and so on) in the TCP stream is a much better solution. Unfortunately this would require some change to the header format (for padding) which probably no one would like to change at this time. The assumption that people will implement some change as big as iSCSI will be to the OS (in an integrated way - not as a kludgy add-on for demos) without being able to make the relatively minor change required to the API seems to be not a very convincing argument. Furthermore, at least in the Unix environment, iSCSI will be implemented in the kernel, and the accomodation for framing won't require an external API change. And as Steph says, the framing proposal has Microsoft participation. The insertion and deletion of markers seems to be painful accomodation for software based iSCSI solutions. The Unix folks might want to speak to this, but the markers will probably require more of a kludge in the kernel (to make it efficient) as compared to the changes required for framing. Somesh > -----Original Message----- > From: owner-ips@ece.cmu.edu [mailto:owner-ips@ece.cmu.edu]On Behalf Of > Stephen Bailey > Sent: Monday, May 21, 2001 6:51 AM > To: ips@ece.cmu.edu > Subject: Re: TCP Framing (considered helpful?) > > > John, > > > I think we must depend on Markers to insure that everything can > operate at > > top speed, and at the lowest cost. > > A key question is whether markers actually ensure that everything > operates at `top speed, and at the lowest cost'. > > Matt thinks so. I (and, presumably those who wrote the framing > document) think not. > > My issue is not even with `lowest cost'. I don't believe markers will > allow you to run at top speed. Specifically: > 1) I doubt the feasibility of implementing the control required for > an eddy buffer (where you store data you can't place) at 10G. > Admittedly, the validity of this claim can't really be assessed > without actually working the implementation, so for 99% of the > list participants (myself included) this is a `yes it is, no it > isn't' point. > 2) an eddy buffer solution requires some substantial speed-up in > both the NIC data path, and MOST IMPORTANTLY: the host bus. In > order to unload the eddy buffer while still handling incoming > traffic at line rate, clearly the host bus bandwidth must be > > line rate. > > I know of at least one general purpose framed solution operating at > 10G which has been available for >3 years (SGI's GSN/ST/XIO NIC). I'm > sure there are others. > > I can't imagine there's any argument that a framed solution would be > voted `most likely to run fast and be cheap'. Every storage network > and cluster interconnect has been designed that way since antiquity. > > The key tradeoff involves the OS vendors, and I'm wondering why we're > speaking for them. The question IS, how much more work is it to > introduce TCP framing over and above what is required to insert iSCSI > into their network framework. My experience from writing NIC and > storage drivers for many commercial UNIX-family OSes is: > 1) it's an easy and well defined process to insert a new SCSI > transport driver into the SCSI stack. > 2) it's hard and poorly defined process to insert ANYTHING into the > network stack. > > Networking has historically been a user-mode activity. Architected > services are only provided to user mode programs. Kernel clients have > been few and far between and so are handled on a case-by-case basis. > For example NFS. Every OS has hacks to make NFS run fast, but they > are not stable interfaces for general purpose use. > > Even Solaris' SysV-derived STREAMS stack, which is intended precisely > to provide flexible, crisp interfaces for kernel network clients, does > not document the relevant (IP stack) intermodule interfaces. > > I know that there are more and more kernel network clients, but they > are coming either on fluid platforms (e.g. linux), in which case the > argument of `it'll take too long to get OS support' doesn't apply, or > they are vendor-supplied, in which case a performance iSCSI solution > in ANY form may take a while, and the choice of framing or markers > isn't going to make a difference. > > I can't say squat about the architecture of Winsock, but the fact that > there is a Microsoft author of the framing proposal who seems very > serious about supporting framing and RDMA as quickly as possible > suggests that framing support should be available on Windows very > soon. > > Steph _________________________________________________________ Do You Yahoo!? Get your free @yahoo.com address at http://mail.yahoo.com From owner-ips@ece.cmu.edu Tue May 22 16:39:18 2001 Received: from ece.cmu.edu ([128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id QAA03464 for ; Tue, 22 May 2001 16:25:23 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f4MIx4422157 for ips-outgoing; Tue, 22 May 2001 14:59:04 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from med.corp.rhapsodynetworks.com (64-160-62-201.rhapsodynetworks.com [64.160.62.201] (may be forged)) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f4MIwrH22119 for ; Tue, 22 May 2001 14:58:53 -0400 (EDT) Received: by med.corp.rhapsodynetworks.com with Internet Mail Service (5.5.2653.19) id ; Tue, 22 May 2001 11:58:46 -0700 Message-ID: <45BEF1D68145D51186C100B0D0AABE6506CE28@med.corp.rhapsodynetworks.com> From: Dennis Young To: ips@ece.cmu.edu Subject: RE: iSCSI: Final bit/status clarification Date: Tue, 22 May 2001 11:58:45 -0700 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2653.19) Content-Type: text/plain; charset="iso-8859-1" Sender: owner-ips@ece.cmu.edu Precedence: bulk I believe there is a typo in the 3rd paragraph of sec 4.3 of iSCSI draft 06: "... or a login response with the text bit set to 0, ..." The word 'text' should be 'F', right? In general, should we use this mailing list for pointing out typos (sometimes I don't know for sure they are typos). Is there an errata somewhere for draft 06 already? Thanks. -----Original Message----- From: Eddy Quicksall [mailto:EQuicksall@mediaone.net] Sent: Monday, May 21, 2001 9:26 AM To: julian_satran@il.ibm.com; ips@ece.cmu.edu Subject: iSCSI: Final bit/status clarification Section 2.7.1 F (Final) Bit For incoming data, this bit is 1 for the last input data PDU associated with the command (even if it includes the status). Is a clarification implied by "even if it includes the status"? I don't see the need for a clarification here because I don't see any ambiguity. Would it make sense to remove it? Or, maybe it should say "even if it does not include the status". -------- mailto:Eddy@Quicksall.com From owner-ips@ece.cmu.edu Tue May 22 20:42:43 2001 Received: from ece.cmu.edu ([128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id UAA07604 for ; Tue, 22 May 2001 20:42:42 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f4MLsNi06187 for ips-outgoing; Tue, 22 May 2001 17:54:23 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from mxbh4.isus.emc.com (mxbh4.isus.emc.com [128.221.10.33]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f4MLsMH06183 for ; Tue, 22 May 2001 17:54:22 -0400 (EDT) Received: by mxbh4.isus.emc.com with Internet Mail Service (5.5.2653.19) id ; Tue, 22 May 2001 17:54:16 -0400 Message-ID: <0F31E5C394DAD311B60C00E029101A07080155D0@corpmx9.isus.emc.com> From: Black_David@emc.com To: ips@ece.cmu.edu Subject: Nashua Interim Minutes Date: Tue, 22 May 2001 17:54:15 -0400 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2653.19) Content-Type: text/plain; charset="iso-8859-1" Sender: owner-ips@ece.cmu.edu Precedence: bulk Here are the minutes in their final form, including clarifications based on Mallikarjun's comments. I have seen no objection to any of the consensus items since the posting of the interim minutes, hence I believe all items described as consensus (usually Rough Consensus) below represent the rough consensus of the WG. Note that I have revised the security consensus in the notes below to reflect subsequent discussion at the meeting and validated on the list (i.e., that use of SRP to key ESP is a promising direction, but not required). Thanks, --David IETF IP Storage (ips) Working Group Nashua Interim Meeting - April 30 and May 1, 2001 Final Minutes ----------------------------------------------------------- April 30, 2001 - FC encapsulation protocols (FCIP and iFCP) ----------------------------------------------------------- FC Common Encapsulation - draft-ietf-ips-fcencapsulation-00.txt ----------------------- CRC discussion - some protocols want to use a CRC, some don't, hence there's a CRC Valid flag in the header. Using one of the bits from the protocol number field to encode CRC Valid was discussed and rejected as too clever by half ** Rough Consensus: Retain CRC Valid as a separate flag as currently defined. -> Action: David Black to send email to list describing allocation approach that would (initially) avoid two protocol numbers that differ only in the encoded CRC Valid bit without permanently using two bits from the protocol number field to do this. ** Rough Consensus - Use different default TCP ports for FCIP and iFCP. -> Action: David Black to pursue causing these ports plus one for iSCSI to be allocated. -> Action: David Black will work with authors on revising the IANA text. ** Rough Consensus on the Reserved flag bits: (1) MUST be zero on send. (2) Protocol MAY specify MUST be zero on receive. (3) How to allow future protocols to make use of these? - Actual use requires modification of common encap. - Protocol-generic use of this encap SHOULD not assume that reserved flags are zero. Later discussion on use of inverted check fields ( followed by 1's complement of ) reached the following ** Rough Consensus: All uses of inverted check fields are to have the structure of a 32 bit word consisting of 16 bits of followed by 16 bits of the 1's complement of , where may consist of multiple fields. This applies to the common encapsulation header, and both the SOF and EOF encodings. Fibre Channel MIBs - General Discussion --------------------------------------- ** Rough consensus - iFCP and FCIP will do separate protocol-specific MIBs. Use existing FC MIBs (ipfc WG) for common stuff, similar to iSCSI's approach/direction of abstracting SCSI stuff into separate MIB. FCIP MIB - draft-anil-fcip-mib-00.txt -------- This is being developed as an extension of the FC Framework MIB (draft-ietf-ipfc-fsmgmt-int-mib-06.txt). Several comments were made on things needing attention in the MIB, including clarifying the distinction between port and endpoint, and the likely need for RowStatus support for row creation. TCP connection parameters are whether FCIP is configured to use them. TCP statistics - aggregated across connections that constitute a single FCIP link. This includes tracking of totals across closed TCP connections in a still-open link. Some more global suggestions: - Beware of just duplicationg TCP parameters from other MIBs. Aggregation across TCP connections in an FCIP link should be ok. - Look at RMON and follow its style to define an FCIP RMON MIB. FCIP use of Common Encapsulation -------------------------------- ** Rough Consensus: FCIP will not use CRC. The fact that this leaves the timestamp field unprotected was characterized as acceptable because the probability of an error causing a frame that should have been discarded to not be discarded is very small due to the timestamp needing to fall within an approximately 40 second window to cause the frame to be forwarded. 40 seconds is more than enough for satellite transmission - this won't allow FCIP to go to the moon, but that's probably out of scope. Protocol-specific fields for FCIP are not needed (protocol works fine if words 1 and 2 are zero), The decision on their use is: ** Rough Consensus: For FCIP, word 1 is a copy of 0, the first 16 bits of word 2 are reserved, and the last 16 bits are the 1's complement of the first 16 bits. Consensus called over Charles Monia's objection. Back to protocol-specific data. Bob S wants to use Word 1 as a copy of Word 0, and reserve Word 2 (reserved/-reserved) (replicate 3 into 2? - would consume all extra space). -> Action: David Black will work with the FCIP authors to provide right Quality of Service (diffserv) words. -- Model Update (Jim Nelson, Vixel) Work in progress based on direction Mike O'Donnell presented in Minneapolis, update prior to T11 June meeting (week of June 4 in Minneapolis - BB-2 is after FC-SW-2 on Monday, afternoon). Full model goes in FC-BB-2, a piece of this goes in FCIP to replace BSW (Backbone switch) discussion. Full discussion when the text becomes available (August, except that London IETF meeting conflicts w/T11) --> Action: David and Elizabeth are already in touch with the Area Directors regarding the IETF and T11 meeting conflict, and what to do about it. -- FCIP interaction with TCP Changes made in -02 version of FCIP draft: - Now distributing FC flows across multiple TCP connections (each port pair [N_Port login session] has to be limited to one TCP connection for in-order delivery, and FC ACKs need to be on same conn. as ACKed traffic). - Removed discussion of TCP parameters. - Error recovery (TCP keep-alive too large): 2 possible approaches -1- Fibre Channel ELS (Extended Link Sequence) to detect loss of connectivity. This would not propagate across endpoints, and a zero-length frame might be better as that would not be a valid frame to forward into Fibre Channel. -> Action: Bob Snively will check whether FSPF (Fibre Channel routing protocol that will always be present on an FCIP link as FCIP is currently specified) already contains heartbeats to detect a lost connection. If so, FCIP can leave this issue to FSPF to handle. -2- Check time stamps. Added text to indicate that horribly late frames get bit-bucketed. Common Encapsulation says "put timestamp here", but doesn't provide much in the way of details Resulting discussion was inconclusive. FC experts need to write the timestamp rules to make sure that timestamp processing adheres to FC requirements - these rules probably belong in the common encapsulation document as they apply to any Fibre Channel traffic. The following issues are open: (1) How do we ensure that the times at both ends of an FCIP or iFCP link are synchronized? FC-GS-4's time protocol or requiring use of NTP across the link are possibilities. FC B ports can't source or sink FC-GS-4 frames, so may have to use NTP. (2) Need a concise and strict definition of when it's ok to *not* timestamp traffic. Current text on loss of sync and related error recovery issues is incorrect in several ways and needs to be revised. Revison *must* not be susceptible to misinterpreting frame payload as a frame when sync is lost. Multiple TCP connections - iFCP classifies by destination - FCIP can classify by traffic class (frame type), and also by destination - FC class 4 is defining virtual channels (can have > 1 per destination). --> Action: David Black to run some version of the above summary past the Area Directors to make sure this doesn't run afoul of general principle of not using multiple TCP connections to increase a single application's performance. -- FCIP QoS and Performance -> Action: David Black will check QoS text in Section 9.1 of -02 draft and review first sentence of Section 9.2 -> Action: Authors will look at QoS text in iFCP draft. The TCP performance section of the draft contains some informational/advisory text on dealing with some TCP issues (e.g., long/fat networks [high bandwidth x delay product]). Flow control management (Section 10) This is about flow control mapping between FC and TCP. Section title is misleading and may be changed. -> Action: Remove reference to Fibre Channel Class 1. T11.3 is removing support for it in FC-MI. There are open issues in Section 10. This is about reflecting TCP traffic propagation issues into FC credit-based flow control. Source throttling. -- Data Integrity (Bob Snively, Brocade) This was a discussion of detailed edits to the draft to make sure that there are no serious date integrity issues. Section 3 error handling text will be deleted in favor of a pointer to 8.3. Section 4.3 on ordering will be connected this to the multi-path classification characteristics above. Section 8.1 - Bob Snbively wants to add ability to do data-based resync -> Action: Bob will write the text to explain how to do this for serious review. Will also rewrite the text to distinguish use of another framing mechanism vs. the data scan. Resync should be optional, and data scan must not be capable of misinterpreting what appears to be a Fibre Channel frame in an FCIP data payload as an actual frame. Section 8.3 - additional text on dealing with CRC errors for frames going onto Fibre Channel. Section 4.2 - use usual login procedure for fabric devices. FCIP need not do some sort of TCP/IP logins, but FC mechanisms must work unmodified. Endpoint definition - distinguish Fibre Channel port from TCP connection endpoint. Naming and Discovery for FCIP ----------------------------- -- FCIP and iSNS - Josh Tseng, Nishan Hostnames will name FCIP boxes - discussed as possibility in Orlando, presented as proposal at this time. DNS or iSNS can be used to resolve them - this is about tunnel configuration for FCIP. Simple DNS approach - have to enter hostnames of remote FCIP devices in each FCIP device. Each device resolves names with DNS and sets up tunnels. All configuration is manual. iSNS can automate configuration and discovery. Domain concept makes group-based managment of FCIP connectivity possible - scales better than manual configuration. iSNS server can be discovered via DHCP option or SLP. FCIP box identifiers don't have to be DNS hostnames, but those are convenient. Can automate hostname assignment to FCIP boxes. Management station can alias names. -- FCIP use of SLP - Dave Peterson, Cisco [used Excel spreadsheet + MS Word document] iSNS and SLPv2 use same authentication. Dave Peterson doesn't anticipate using SLP scopes for FCIP. New SLP RFC (3082) provides sufficient (somewhat timely) notification of devices appearing and disappearing. Proposal - static config, SLP for dynamic config, also iSNS, both are optional. DHCP - self-discovery mechanism, can't discover info about others. DNS - too limited, SLP improves LDAP - database interface, may be used by either SLP or iSNS Putting together FCIP discovery model for SLPv2. Suggestion - just use SLP, limit iSNS to functionality beyond SLP. Josh: RFC 3082 is Experimental, SLP scope is different from iSNS Discovery domains. SLP periodic re-registration has to be set correctly. SLP can't store non-string attributes (e.g., X.509 certificates). SLP is a good discovery mechanism, but not good for subsequent/sophisticated mgt. After much discussion, it was not possible to call consensus on the relationship between iSNS and SLP usage/requirements for FCIP. -> Action: Josh and Dave Petersen will post their positions to the reflector with David Black and Elizabeth summarizing areas of agreement and open issues. iFCP - Charles Monia, Nishan ----------------------------- Changes to draft - iFCP addressing model updated, including how to manage address tables, and incorporation of transparent mode. ELS handling changed for common encapsulation. No longer need extended header (this was discussed in Minneapolis). Lots of editorial changes. QoS text added, now 1 TCP/IP connection per N_Port session. Next version targeted for May 15, major work is to incorporate common encapsulation. Changes to incorporate Common Encapsulation: (1) Protocol-specific fields: - LS_CMD - identify command for Accepts of augmented ELS's. - iFCP flags (see below), copy of SOF, copy of EOF. iFCP uses these copies, and skips over SOF and EOF words. - CRC is always checked, CRCV field is ignored on receive, but must be set on send. (2) Flags - Compliance level (which version of document, watch out for version # change). This is a 1-bit version number. - Augmented (part of an ELS w/augmented data) - Address transparent vs. translation mode - iFCP session control frame (must be non-augmented, translation) Header CRC error causes connection close. CRC error in 7-word header is unlikely, and only a single N_port to N_port connection is affected. (3) Timestamp format and content will be same as FCIP. Likely to be able to put this syntax and semantics into the common encapsulation document. iFCP address-transparent mode. This assembles iFCP gateways into a single logical fabric, within which addresses are global and untranslated. Maximum of 239 gateways in a fabric (this is the FC domain limit for number of switches, each gateway counts as a switch). Address translation mode will probably be mandatory. Translation and Transparent mode do not interoperate. Transparent mode does not require any FC header or CRC changes. iFCP transparent mode is similar to FCIP, but implements FC services via IP equivalents rather than using FC switches. Gateways implement F or FL port, could even do an E port. For E port, gateway MUST become the principle switch. ELS modifications - out of 73 total ELSs, 14 of them, and 3 of their ACC (Accept) responses require iFCP intervention/changes. -> Action: Authors to look at FCP-2 document for more ELSs. REC now comes from there, and SRR appears to be missing from the list. One of the types of augmentation appends WWN to ELS sent between iFCP gateways. TPRLO (Third PaRty LogOut) can require a massive augmentation that can span multiple FC frames due to the ability to to specify up to 4095 devices to be logged out. Gateways have to maintain state to track augmented Accepts and handle them properly - this is for the gateway that proxies for target of ELS. Exchange identifier associates ACCs with ELSs on the FC side of that proxy. Fabric Service Profiles. This is about making sure that a iFCP-base fabric uniformly exports services to all attached devices, even if gateways are from different vendors. - Fabric services to be provided at well-known addresses (mandatory, optional, prohibited, behavior description for each service). - Transport (details of Class 2 and 3 behavior, prohibit Class 1, FC QoS mapping [when FC does QoS]). iFCP Naming and Discovery - Josh Tseng -------------------------------------- iSNS is required for iFCP - nothing else works. Overview of iSNS use by iFCP. Security for iFCP - use IPSec, probably tunnel mode. FCIP is leaning in the same direction. Security Comments - David Black ------------------------------- These comments apply to all IPS protocols, and are particularly applicable to the iSCSI security discussion scheduled for tomorrow. Confidentiality is not required. The IPS WG may have the freedom to "profile" IPSec, e.g., make AH optional to implement. The expensive parts of IPSec are Confidentiality (cycles) and negotiation/key exchange [IKE/ISAKMP] (code) - IKE/ISAKMP could be optional to implement (i.e., manual configuration, with attendant management problems). One of the ipsec WG chairs should be here tomorrow to provide additional information. --------------------------- May 1, 2001 - iSCSI --------------------------- iSCSI Naming and Discovery -------------------------- -- Overview - Kaladhar Voruganti, IBM -- Naming and Addressing - Mark Bakke, Cisco WWUI acronym in earlier versions of draft gone, set of naming types has been reduced to eui (WWN) and fqn. fqn acronyn will be changed to iqn (i = iSCSI) to avoid possibly rude (mis)-pronunciation An iSCSI name is a persistent location-independent identifier, an iSCSI address is how one gets to it, with DNS recommended in preference to specifying the IP address. There is a problem with domain names changing ownership without record of what iSCSI names iqns were created by the old owner. This risks unacceptable duplication. Suggestions for using IANA private enterprise identifier, date, and some sort of existing vendor identifier were made. The simple approach of using WWNs runs into administrative difficulties like a $1500 fee to get initial allocation. -> Action: Authors to propose resolution to this issue. Third party command authentication/authorization is an open issue. -- SAM2 and iSCSI (concept mapping) - Jim Hafner, IBM This is about how the concepts in the SCSI architecture (SAM2) map to the concepts in iSCSI, and the implications of this mapping. Basic concepts and mappings: o Network entity - has an IP network address (and TCP port) o iSCSI Node - within network entity, identified by iSCSI Name Contains at most ONE SCSI device (canonical iSCSI Target does not have a SCSI device). o SCSI Device Name (T10 is working on this) = iSCSI Node Name o SCSI Device - contains SCSI Ports = iSCSI Node Name + ISID or TSID. o SCSI Port has one or more pairs, may change dynamically. o SCSI I_T Nexus = iSCSI Session, connects two iSCSI Ports. Q: Multiple sessions per node pair? A: Yes, each session has unique SCSI Port instance on each side. Result is that SCSI Ports are dynamic entities. This is different from models used in other SCSI tranports because the SCSI Port entities are bound one-to-one by a connection. The iSCSI Name is intended to name OS image (actually SCSI instance in OS image) - goal here is to avoid problem with FC Node Name, which is associated with HBA - iSCSI Node Name is supposed to be associated with the "SCSI entity" in the OS. This requires the SCSI instance to manage ISIDs or TSIDs globally across all iSCSI interfaces. SCSI Port Address - not clear what this T10 concept maps to, could just use SCSI Port Name (iSCSI name + ISID or TSID). Mapping Consequences: - ISIDs have to be unique initiator-wide (across all NICs/Adapters connected to same target). - Persistent reservations get linked to both ISID and TSID -- reservation comes back only if initiatiator reuses ISID and target reuses TSID. This has implications on target saving/tracking of state, and requires initiator to remember which ISID was used with which target. -> Action: Next version of draft will need a table of how nexus state reacts to events (e.g., Resets). Discussion of Persistent Reservation behavior wrt this model. SCSI has things to say about Target reboot. Host reboot is generally not visible, but only exception is that if addresses are reassigned (e.g., by Fibre Channel fabric), Target has to cope with this reassignment. Names span sessions to make authentication simpler. Keeping ISID separate from the name makes the authentication stuff simpler. Q: Do persistent reservations open up a denial of service attack route? A: No - can't do this without prior authentication, and there are ways to blow away reservations from other initiators. Difference from FCP is that iSCSI persistent reservations are keyed to session IDs vs. FCP keying them to FC ports (via WWN). SCSI allows sharing of reservations. At the iSCSI level, each reservation is tied to a session. SCSI reservation sharing is how reservations get shared across more than one I_T_Nexus. T10 is considering reservation semantics that ignore ports and make reservations at SCSI Device level, which would yield reservations at iSCSI node granularity. On session establishment, TSID choice may be influenced by existence of reservation state - this results in iSCSI having to check for SCSI persistent reservations for that initiator name and ISID. Third party reservations were brought up. They're volatile, essentially obsolete, and not needed by anything in SCSI, included extended copy. There are also command format problems that would make it very difficult for iSCSI to pass a node name + session ID to the third party copy device. Persistent reservation key *does not* contain iSCSI node name or ISID. Cluster software (or whatever) creates its own reservation keys. --> Action: There may be an issue of scope of ACA with respect to the model. Jim Hafner will check into this. CA should not be an issue because iSCSI requires Autosense. Access controls need to use iSCSI Initiator node name. T10 will handle the required changes, as part of long identifier format changes (e.g., for things like SCSI alias designation), which also cover third-party commands. Additional address info for third-party commands could be definitive, or just hints. SRP and FCP use definitive addresses. -- Discovery - Mark Bakke, Cisco draft-ietf-ips-iscsi-name-disc-01.txt Review of types of configuration: - Static/manual - SLP for simple discovery - iSNS for centralized management "Send Targets" - an additional mechanism that can be used in all three cases. Send Targets is allowed to send info on targets that are elsewhere, or just targets behind its port. This can also implement some "soft zoning" (i.e. restricted what an Initiator can see), and cascading (return a target that is elsewhere to which "Send Targets" must be sent). If there's only one iSCSI target behind a port, a login to the "iscsi" target could return that name and immediately allow commands. This would lead to needing to distinguish between login only for "Send Targets" and login that results in the one and only target coming back and going into full-feature mode. It adds complexity but might simplify booting. OPEN ISSUE. There is a need for some sort of iterator interface or something else that avoids an Initiator having to allocate potentially unlimited space to hold the results of Send Targets. This issue will be taken to the mailing list. OPEN ISSUE. Will be adding tags to addresses that specify which addresses can be aggregated into a session. Absence of tag indicates that aggregation is not possible. Tag unique to a single address would indicate that multiple connections to that address are aupported. Only semantics of tag is whether these values match, and context is within a single response to Send Targets. One tag per address for now. The ability to specify another initiator in Send Targets will not be supported, as there is T10 work to be done in the access control area (this is about giving an Initiator's view to a third party copy device) and it raises security issues. -- SLP - Mark Bakke, Cisco draft-ietf-ips-iscsi-slp-00.txt SLP template has been submitted as WG draft. SLP and iSNS now do *exactly* the same authentication. Working on interoperability with iSNS. Working on host/device taxonomy to make recommendations about what should be implemented where (SLP and/or iSNS in drives, arrays, hosts, gateways, etc.?). Open source for SLP authentication is now available. Need to look at RFC 3082 for event propagation to see what it does and figure out how to use it. iSCSI may need a few minor extensions to this. The iSCSI SLP Template may need changes to better accomodate copy managers. None of the discovery mechanisms are REQUIRED at this point in time. A question was asked about how network management tools discover network configuration and devices - this is generally based on MIBs and reading information from them. One can have a conformance statement for a MIB that mandates a relatively small portion of it that is necessary for this sort of discovery. -- iSNS - Josh Tseng, Nishan draft-ietf-ips-isns-02.txt SNS requirements in naming and discovery draft are based on FC-GS-3. Discussion of iSNS functionality and alternatives: - LDAP: This is a directory service, but seems to lack info about when and where to send state change notifications. - SNMP: Discovery domains cannot be implemented, cannot do complex queries. - SLP: Now has a notification mechanism, but needs some changes. SLP depends on multicast (in absence of SLP, have to manually configure addresses of SLP directory agent). DHCP can discover SLP, hence can piggyback on DHCP infrastructure. SLP scopes aren't the same as discovery domains - have to add something. SLP requires on service agents to reregister with directory agents, iSNS does things in the reverse direction. Nishan will open source iSNS client and server within next month. iSCSI Requirements Last Call Issues - Marjorie Krueger, HP ----------------------------------- This discussion was about closing the issues raised during informal Last Call of this draft on the mailing list. ** Rough Consensus: iSCSI should not be making changes to SCSI-3, except where the relevant SCSI document says something is "transport dependent". Wording will be crafted to require "minimum changes" or something like that outside of this case. ** Rough Consensus: Layering wording will not be added to this document; the exhortation in the IPS WG charter is sufficient. ** Rough Consensus: "packet formats similar to" wording will NOT be added. The general sense of the room was that data formats and the like ought not to be changed - at this point in the process, stability matters, and further changes of this sort need to be discussed on the list prior to being made. ** Rough Consensus: Accept Bob Snively's change to require ordering only at I_T_L level, not I_T, but don't exclude implentations that do I_T ordering. After a long discussion of ordering, the conclusion was: ** Rough Consensus: Strictly ordered command delivery to target is required across a session, even in error recovery cases (this is stricter than either FCP or parallel SCSI require). Those wanting looser ordering should use multiple sessions. Rough consensus on the last point was called over visible opposition (significant minority of the room). Security -------- -- David Black (EMC, co-chair) presentation on security requirements See slides - this was an informational presentation/discussion about requirements (MUST implement authentication and cryptographic integrity, MUST have a required mechanism for each, confidentiality is OPTIONAL, IP vs. iSCSI level mechanisms [iSCSI can multiplex initiators and targets on a single pair). -- Ofer Biran (IBM, Haifa) Lots of slides - see slides for details. After much discussion ... ** Rough Consensus: (1) Need iSCSI level authentication in addition to what is done for IP or TCP/IP. (2) ESP with null encryption is mandatory to implement - TLS was considered and rejected as part of this. (3) SRP is mandatory to implement. Beyond this, the use of SRP (or another inband authentication mechanism) to key ESP was discussed as a promising direction. It's premature to require this - the minimum requirement for now is ESP with pre-shared keys and a separate implementation of SRP. -> Action: David Black, Ted Ts'o, and Ofer Biran will write up details of getting ESP key material from in-band authentication mechanism and related "Start Here" text key to turn on ESP dynamically. Error Recovery - Randy Haagens, HP and Kalman Meth, IBM ------------------------------------------------------- NOTE: Due to the late hour and diminishing attendance, no consensus calls were attempted in this area. This is a progress report from some extensive and on-going off-line work on details of errror recovery. The goal is to provide a set of mechanisms that allow implementations to choose where they want to be between "Trust TCP Explicitly" (Detect errors and flag to SCSI) and "Can't Trust TCP" (comprehensive error recovery. This work will be folded into the basic iSCSI document. Session recovery appears to be cleanly separable from finer-grained mechanisms. Not sure about separations among within-session, within-connection, and within command recovery. Not every situation will need fine-grained recovery. Session recovery will be mandatory - close session and open a new session. ** Need to make it clear that explicit logout on last connection in session. Within-session recovery reissues commands on another connection in same session - login of new connection includes instruction to logout old connection. Have to make sure that old connection is successfully logged out before issuing new commands with retry (X) bit set. This is optional. Have to use same tags and CmdSNs in doing the retry. Terminology note: This is really "resend". Whether or not a "retry" of the command is actually done is at the device's option. Need to be careful to distinguish these terms. Comment: Wording is needed to indicate that if the initiator chooses not to reissue a command that was pending on the terminated connection, it MUST terminate the session. Q: Is having both implicit and explicit logout mechanisms overkill? A: Needed to cover resend on an existing connection vs. opening a new connection for this purpose. Target can reject attempt to open new connection, and then initiator has to clean up. Comment: This portion of the draft contains a recommendation to use Target Reset that needs to be removed. Within-connection recovery reissues command on same connection to fill CmdSN holes. Should be on same connection due to connection allegiance. Recovery from header errors requires framing support over TCP - in the absence of framing, the connection has to be terminated. Comment: When there are multiple connections, target needs to delay for a bit before telling Initiator that commands are missing. On a single connection, this is easy. When CmdSN has advanced past the hole on all connections, target knows that a command is missing. A request was made for an explicit way to request a resend rather than the implicit monitoring of ExpCmdSN by the Initiator and possible use of NOP by the target. This will be considered. Discussion of (lack of) trust in TCP. Between ESP at IP level and encouraging SCTP to use a CRC, we're getting stronger integrity in the stack at a lower level. TCP (or SCTP) will retry is a wonderful answer to a whole bunch of problems. ESP has a strong but computationally expensive integrity check (HMAC), and hence can't be mandatory to use - security discussion above makes it mandatory to implement only. This lead to some interest in figuring out how to shim a CRC between TCP and IP, which would then allow some error recovery to be dropped (the ones which wouldn't be needed/useful when ESP is in use). OPEN ISSUE: for further discussion. Discussion of data resends differing between reads and writes. On read, initiator resends command causing target to resend all the data. On write, the target can issue only some of the R2Ts (for the missing data) and it should not be a big deal for the initiator to cope. OPEN ISSUE - will require confirmation/further discussion on list. Status recovery within connection is based on status SNACK. -> Action: Need to revise text to indicate that SNACK is optional. SNACK is required to do within-connection recovery of status. Comment: This may be a big deal for backup - put in a negotiation key/value pair to allow initiator to discover whether target does SNACK. Backup may want to avoid using a target that doesn't do status SNACK. OPEN ISSUE: Take to list. Within-command recovery from dropped data. If data CRC fails, issue immediate Data SNACK. If header CRC fails, find the hole, issue Data SNACK at that point. From owner-ips@ece.cmu.edu Tue May 22 22:50:57 2001 Received: from ece.cmu.edu ([128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id WAA10928 for ; Tue, 22 May 2001 22:50:57 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f4N0f1s17408 for ips-outgoing; Tue, 22 May 2001 20:41:01 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from venus.cs.uml.edu (venus.cs.uml.edu [129.63.8.51]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f4N0ewH17402 for ; Tue, 22 May 2001 20:40:59 -0400 (EDT) Received: from localhost (localhost [127.0.0.1]) by venus.cs.uml.edu (8.11.0/8.11.0) with SMTP id f4N0eVR313642; Tue, 22 May 2001 20:40:32 -0400 (EDT) Date: Tue, 22 May 2001 20:40:31 -0400 (EDT) From: Mike Brown Reply-To: Mike Brown To: Somesh Gupta cc: ips@ece.cmu.edu Subject: Re: TCP framing In-Reply-To: Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-ips@ece.cmu.edu Precedence: bulk Hi all, I am against using a tcp framing scheme for reasons previously stated by Matt Wakely and John Hufferd. I believe the assertion that iSCSI will be implmented in the kernel in the UNIX environment is too simplistic a view. I think robust implementations will likely consist of some userspace/kernel hybrid. (I am of the opinion security associations using strong cryptography should not be implemented in the kernel). While this could also avoid an external API change, requiring os vendors to change their existing TCP/IP stacks might be too idealistic a view. The insertion and deletion of markers from an implmentation prespective would not be too painful an accomodation for software based solutions. In fact, they would be simpler to implement since existing TCP/IP stacks could be used. John Hufferd's idea that "Markers should be 'Must Implement', but 'Optional to use'" would enable software solutions to ignore using them and still be efficient if their size is kept small enough. I am of the opinion that a new standard's probability of universal acceptance is inversely proportional to the number of changes it requires in other standards and existing implementations. On Tue, 22 May 2001, Somesh Gupta wrote: >I think if the framing solution finds favor with the TCP "gods", >it is a much better alternative than the markers (which I was >a proponent of - but only because I did not believe that >a framing mechanism would be approved). > >Even if the framing mechanism is not approved, an alternate >mechanism which provides for alignment of iSCSI headers at >regular intervals (at negotiate progression in TCP sequence >#s e.g. 0x12345, 0x22345, 0x32345 and so on) in the TCP stream >is a much better solution. Unfortunately this would require >some change to the header format (for padding) which probably >no one would like to change at this time. > >The assumption that people will implement some change as big >as iSCSI will be to the OS (in an integrated way - not as >a kludgy add-on for demos) without being able to make the >relatively minor change required to the API seems to be not >a very convincing argument. > >Furthermore, at least in the Unix environment, iSCSI will be >implemented in the kernel, and the accomodation for framing >won't require an external API change. And as Steph says, >the framing proposal has Microsoft participation. > >The insertion and deletion of markers seems to be painful >accomodation for software based iSCSI solutions. The Unix >folks might want to speak to this, but the markers will >probably require more of a kludge in the kernel (to make >it efficient) as compared to the changes required for framing. > >Somesh > >> -----Original Message----- >> From: owner-ips@ece.cmu.edu [mailto:owner-ips@ece.cmu.edu]On Behalf Of >> Stephen Bailey >> Sent: Monday, May 21, 2001 6:51 AM >> To: ips@ece.cmu.edu >> Subject: Re: TCP Framing (considered helpful?) >> >> >> John, >> >> > I think we must depend on Markers to insure that everything can >> operate at >> > top speed, and at the lowest cost. >> >> A key question is whether markers actually ensure that everything >> operates at `top speed, and at the lowest cost'. >> >> Matt thinks so. I (and, presumably those who wrote the framing >> document) think not. >> >> My issue is not even with `lowest cost'. I don't believe markers will >> allow you to run at top speed. Specifically: >> 1) I doubt the feasibility of implementing the control required for >> an eddy buffer (where you store data you can't place) at 10G. >> Admittedly, the validity of this claim can't really be assessed >> without actually working the implementation, so for 99% of the >> list participants (myself included) this is a `yes it is, no it >> isn't' point. >> 2) an eddy buffer solution requires some substantial speed-up in >> both the NIC data path, and MOST IMPORTANTLY: the host bus. In >> order to unload the eddy buffer while still handling incoming >> traffic at line rate, clearly the host bus bandwidth must be > >> line rate. >> >> I know of at least one general purpose framed solution operating at >> 10G which has been available for >3 years (SGI's GSN/ST/XIO NIC). I'm >> sure there are others. >> >> I can't imagine there's any argument that a framed solution would be >> voted `most likely to run fast and be cheap'. Every storage network >> and cluster interconnect has been designed that way since antiquity. >> >> The key tradeoff involves the OS vendors, and I'm wondering why we're >> speaking for them. The question IS, how much more work is it to >> introduce TCP framing over and above what is required to insert iSCSI >> into their network framework. My experience from writing NIC and >> storage drivers for many commercial UNIX-family OSes is: >> 1) it's an easy and well defined process to insert a new SCSI >> transport driver into the SCSI stack. >> 2) it's hard and poorly defined process to insert ANYTHING into the >> network stack. >> >> Networking has historically been a user-mode activity. Architected >> services are only provided to user mode programs. Kernel clients have >> been few and far between and so are handled on a case-by-case basis. >> For example NFS. Every OS has hacks to make NFS run fast, but they >> are not stable interfaces for general purpose use. >> >> Even Solaris' SysV-derived STREAMS stack, which is intended precisely >> to provide flexible, crisp interfaces for kernel network clients, does >> not document the relevant (IP stack) intermodule interfaces. >> >> I know that there are more and more kernel network clients, but they >> are coming either on fluid platforms (e.g. linux), in which case the >> argument of `it'll take too long to get OS support' doesn't apply, or >> they are vendor-supplied, in which case a performance iSCSI solution >> in ANY form may take a while, and the choice of framing or markers >> isn't going to make a difference. >> >> I can't say squat about the architecture of Winsock, but the fact that >> there is a Microsoft author of the framing proposal who seems very >> serious about supporting framing and RDMA as quickly as possible >> suggests that framing support should be available on Windows very >> soon. >> >> Steph > >_________________________________________________________ >Do You Yahoo!? >Get your free @yahoo.com address at http://mail.yahoo.com > -Michael F. Brown, UMass Lowell Computer Science phone: (978) 934-5354 email: mbrown@cs.uml.edu "In theory, there is no difference between theory and practice, but in practice, there is." - Jan L.A. van de Snepscheut From owner-ips@ece.cmu.edu Tue May 22 23:34:06 2001 Received: from ece.cmu.edu ([128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id XAA11678 for ; Tue, 22 May 2001 23:34:05 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f4N1QWd20230 for ips-outgoing; Tue, 22 May 2001 21:26:32 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from chmls06.mediaone.net (chmls06.mediaone.net [24.147.1.144]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f4N1QUH20222 for ; Tue, 22 May 2001 21:26:30 -0400 (EDT) Received: from cs.uchicago.edu (h00008649aa79.ne.mediaone.net [24.91.82.73]) by chmls06.mediaone.net (8.11.1/8.11.1) with ESMTP id f4N1QS813238 for ; Tue, 22 May 2001 21:26:28 -0400 (EDT) Message-Id: <200105230126.f4N1QS813238@chmls06.mediaone.net> To: ips@ece.cmu.edu Subject: Re: iSCSI: response to second login (with same ISID) In-Reply-To: Message from "Jim Hafner" of "Mon, 21 May 2001 09:15:14 PDT." References: Date: Tue, 22 May 2001 21:24:29 -0400 From: Stephen Bailey Sender: owner-ips@ece.cmu.edu Precedence: bulk > Anybody else got any opinions? I think it has to be: > 2) CLOSE the pre-existing session (and abort all it's tasks) and then > process the new login fresh. This provides a mechanism for rapidly reclaiming session state in the target. For example, if an iSCSI adapter loses its mind (I'm sure we all believe THAT will never happen), a login with the same SID will reclaim the target state. This capability is provided in FC, and certainly in the early days (which was the first 4+ years) it was very important. If you logged in again, you knew the target would blast anything outstanding for the initiator. Granted, in FCP, you had a relatively short RA_TOV, but in iSCSI RA_TOV could be pretty long. That said, we can provide the capability in some other way, e.g. a flag bit. I don't care one way or the other, but if we don't provide an explicit way to overload a session, then logging in with the same SID must do it. Steph From owner-ips@ece.cmu.edu Tue May 22 23:36:48 2001 Received: from ece.cmu.edu ([128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id XAA11717 for ; Tue, 22 May 2001 23:36:48 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f4N1nLi21913 for ips-outgoing; Tue, 22 May 2001 21:49:21 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from palrel13.hp.com (palrel13.hp.com [192.151.10.61]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f4N1nIH21887 for ; Tue, 22 May 2001 21:49:18 -0400 (EDT) Received: from hpcuhe.cup.hp.com (hpcuhe.cup.hp.com [15.0.80.203]) by palrel13.hp.com (Postfix) with ESMTP id EC3411F52D for ; Tue, 22 May 2001 18:49:12 -0700 (PDT) Received: from cup.hp.com (santoshr@hpindhhm.cup.hp.com [15.8.80.197]) by hpcuhe.cup.hp.com (8.9.3 (PHNE_18979)/8.9.3 SMKit7.02) with ESMTP id SAA27695 for ; Tue, 22 May 2001 18:49:08 -0700 (PDT) Message-ID: <3B0B1734.6C306F5A@cup.hp.com> Date: Tue, 22 May 2001 18:49:40 -0700 From: Santosh Rao Organization: Hewlett Packard, Cupertino. X-Mailer: Mozilla 4.7 [en] (X11; U; HP-UX B.11.00 9000/778) X-Accept-Language: en MIME-Version: 1.0 To: IPS Reflector Subject: Re: iSCSI : Review Feedback on iSCSI Rev 06 References: Content-Type: multipart/mixed; boundary="------------12125BAB172DF21D905E5410" Sender: owner-ips@ece.cmu.edu Precedence: bulk This is a multi-part message in MIME format. --------------12125BAB172DF21D905E5410 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Julian, Thanks for the clarification. A few additional queries below. Regards, Santosh ----------------------------------------------------------------------------------- julian_satran@il.ibm.com wrote: > 1) Suggest re-naming the following in the interests of better > clarity : > > a) Rename the ExpDataSN in SCSI Response PDU to EndDataSN as was used in > the previous version of the draft. (indicative of being the final Data > sequence > number.) > +++ Meaning has changed - 0 means none (it was fffffff before) +++ Have the semantics of ExpDataSN in the SCSI Response PDU changed (?). From the description, it sounds like it is a count of the no of READ Data PDUs sent by the target for the command. 0 implies none were sent. (no data xfer). Hence, the name EndDataSN would be fitting since it really is the last DataSN the target used for this READ I/O. > 3) Why does the new draft limit the size of text commands and nop commands > to > 4096 bytes ? What is the rationale behind the magic number selection of > 4096 ? > > Since DataPDULength is defined in units of 512 bytes, how does iSCSI deal > with > fragmentation caused due to the usage of a DataPDULength < 4K and an > attempt to > perform a 4K text , login or nop operation ? > > I believe such scenarios were discussed in an earlier thread > titled "Fragmentation & Reassembly issues in iSCSI." > (http://ips.pdl.cs.cmu.edu/mail/msg03031.html) and the resolution was to > enforce a restriction that the size of a login, nop & text operation be > limited to DataPDULength (?). > > +++ we decided to drop negotiating trivia. A fixed maximum is more > practical > and 4096 will no create inconvenience for a long time +++ > 5) Section 2.8.3 > "The data length of a text command or response SHOULD be less than > 4096 bytes." > > suggest re-wording the above to : > "The data length of a login, nop & text command or response MUST be less > than > DataPDULength. For the leading session login, the length of the login > command > MUST be less than 512 bytes." > > The above will ensure that login, text & nop operations will not result in > fragmentation. > > +++ I have a hard time understanding the argument +++ One basic question. Does DataPDULength govern the max length of SCSI Data PDUs only or all iSCSI PDUs ? This does'nt come across clearly from the definition. If this only governs SCSI Data PDUs, then, I'd suggest the following re-wrod to make it more clear in the definition on page 129 (Appendix E) : Change : "Initiator and target negotiate the maximum data payload supported for command or data PDUs in units of 512 bytes." to : "Initiator and target negotiate the maximum data payload supported for SCSI command or data PDUs in units of 512 bytes." The above issues (3) & (5) are relevant if DataPDULength governs the length of all iSCSI PDUs. > 9) Section 2.18.1 > > "Target requests logout". > > Suggest removal of the above option. A target that needs to logout would > use > either : > "Target will drop the connection" > or > "Target will drop all connections" > > The "Target requests logout" also does not allow the target to specify > which > flavor of logout it wishes to receive. It is better to remove this option > and > use only a single scheme. > > +++ The reason for introducing it was to allow a maintenance operation > (getting out a NIC card) to start > at the target. However only the initiator knows when the connects gets > quiet. This is why will have to keep this form. The other 2 are for > emergency handling +++ Why is "target will drop connection" or "target will drop all connections" not sufficient for this purpose ? In this case, the target would send the async message, drop the connection/session and abort all outstanding I/O. The initiator on seeing the async message would clean-up all outstanding I/O on that session/connection and would attempt to re-login after LogoutLoginMinTime. --------------12125BAB172DF21D905E5410 Content-Type: text/x-vcard; charset=us-ascii; name="santoshr.vcf" Content-Description: Card for Santosh Rao Content-Disposition: attachment; filename="santoshr.vcf" Content-Transfer-Encoding: 7bit begin:vcard n:Rao;Santosh tel;work:408-447-3751 x-mozilla-html:FALSE org:Hewlett Packard, Cupertino.;SISL adr:;;19420, Homestead Road, M\S 43LN, ;Cupertino.;CA.;95014.;USA. version:2.1 email;internet:santoshr@cup.hp.com title:Software Design Engineer x-mozilla-cpt:;21088 fn:Santosh Rao end:vcard --------------12125BAB172DF21D905E5410-- From owner-ips@ece.cmu.edu Tue May 22 23:39:20 2001 Received: from ece.cmu.edu ([128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id XAA11680 for ; Tue, 22 May 2001 23:34:06 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f4N2B2023327 for ips-outgoing; Tue, 22 May 2001 22:11:02 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from chmls06.mediaone.net (chmls06.mediaone.net [24.147.1.144]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f4N2B1H23323 for ; Tue, 22 May 2001 22:11:01 -0400 (EDT) Received: from cs.uchicago.edu (h00008649aa79.ne.mediaone.net [24.91.82.73]) by chmls06.mediaone.net (8.11.1/8.11.1) with ESMTP id f4N2Ax812121 for ; Tue, 22 May 2001 22:10:59 -0400 (EDT) Message-Id: <200105230210.f4N2Ax812121@chmls06.mediaone.net> To: ips@ece.cmu.edu Subject: Re: iSCSI: response to second login (with same ISID) In-Reply-To: Message from Santosh Rao of "Mon, 21 May 2001 18:13:32 PDT." <3B09BD3C.5B67DB77@cup.hp.com> References: <31891B757C09184BBFEC5275F85D5595104D62@cceexc18.americas.cpqcorp.net> <3B09BD3C.5B67DB77@cup.hp.com> Date: Tue, 22 May 2001 22:09:00 -0400 From: Stephen Bailey Sender: owner-ips@ece.cmu.edu Precedence: bulk Nick, > If I read Steph Bailey's comments on this right (from the ERT work), the > target would have to authenticate the 2nd login on the same ISID [as it > does with any other login] and only take action when authentication is > successful. What he said. Steph From owner-ips@ece.cmu.edu Tue May 22 23:39:21 2001 Received: from ece.cmu.edu ([128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id XAA11679 for ; Tue, 22 May 2001 23:34:06 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f4N2UiO24627 for ips-outgoing; Tue, 22 May 2001 22:30:44 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from tiny-teddy.aarnet.edu.au (IDENT:root@tiny-teddy.aarnet.edu.au [203.21.37.30]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f4N15AH18908 for ; Tue, 22 May 2001 21:05:10 -0400 (EDT) Received: from aarnet.edu.au (gt1.aarnet.adelaide.edu.au [129.127.180.236]) by tiny-teddy.aarnet.edu.au (8.11.0/8.11.0) with ESMTP id f4N153X13041 for ; Wed, 23 May 2001 10:35:03 +0930 Message-ID: <3B0B0CC2.88CB88B2@aarnet.edu.au> Date: Wed, 23 May 2001 10:35:06 +0930 From: Glen Turner Organization: Australian Academic and Research Network X-Mailer: Mozilla 4.77 [en] (X11; U; Linux 2.4.4-ac6-gdt1 i686) X-Accept-Language: en MIME-Version: 1.0 To: ips@ece.cmu.edu Subject: draft-bakke-iscsi-wwui-urn-00 internationalisation issue Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-ips@ece.cmu.edu Precedence: bulk Content-Transfer-Encoding: 7bit > Rules for Lexical Equivalence: > > The entire URN is case-insensitive. Why? This implies that devices now need to carry casing tables tables for multi-lingual support. If the field is case sensitive then adding multilingual support is simply a matter of specifiying UTF-8 and no ongoing work is needed to track multilingual DNS. Regards, Glen From owner-ips@ece.cmu.edu Tue May 22 23:39:21 2001 Received: from ece.cmu.edu ([128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id XAA11681 for ; Tue, 22 May 2001 23:34:06 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f4N2AIY23280 for ips-outgoing; Tue, 22 May 2001 22:10:18 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from chmls06.mediaone.net (chmls06.mediaone.net [24.147.1.144]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f4N2AHH23276 for ; Tue, 22 May 2001 22:10:17 -0400 (EDT) Received: from cs.uchicago.edu (h00008649aa79.ne.mediaone.net [24.91.82.73]) by chmls06.mediaone.net (8.11.1/8.11.1) with ESMTP id f4N2AG811682 for ; Tue, 22 May 2001 22:10:16 -0400 (EDT) Message-Id: <200105230210.f4N2AG811682@chmls06.mediaone.net> To: ips@ece.cmu.edu Subject: Re: iSCSI: Canonical Targets In-Reply-To: Message from julian_satran@il.ibm.com of "Tue, 22 May 2001 20:47:16 +0300." References: Date: Tue, 22 May 2001 22:08:16 -0400 From: Stephen Bailey Sender: owner-ips@ece.cmu.edu Precedence: bulk Julian, > Why shouldn't we take the road of deferring the name discovery to > the specialized (iSCSI) part of a more general discovery protocol? The reason is, for a minimal, useful iSCSI configuration, we must not require any additional pieces of infrastructure beyond (Mark's term) table stakes for IP (e.g. DNS), an iSCSI initiator and an iSCSI target. Assuming we want to allow multitarget iSCSI endpoints in minimal configurations (I'm sure we do), there MUST be a way for the initiator to learn what targets are behind that endpoint from within the initiator's machine environment. The solution to this problem can't be `walk over to the target and read a bunch of numbers and type them in to the initiator'. > You don't expect the management applications to start using iSCSI for > discovery do you? Why not? When you say `management application' you seem to imply an application which would manage an arbitrary, possibly extremely large configuration (e.g. OpenView). Actual management applications range from simple to complex, and many configurations (particularly in the pilot phase that hopefully iSCSI will be entering) start very simple. I don't think anybody would argue that FCP PL-DA provided very complicated networks, yet every initiator implementation usually provided a program or something which simply spat out the WWNs (and probably the AL_PAs) for each target. Such a simple `management application' is going to be a lot like (perhaps IS) an existing SCSI control utility that can give you a list of buses, targets, LUNs, some inquiry data, and whatnot. Clearly, iSNS has nothing to say about this scenario. Are you proposing that SLP be used to supplant the existing simple directory mechanism in the iSCSI draft (and that it be mandatory to implement)? That might work. I'm not sure it makes sense, but if you think so, let's discuss it. In this case, multicast support (a typical objection to SLP) wouldn't be necessary, because you'd already know where the target is by configuration. SLP+iSCSI mechanisms would be offering the same service as the existing SendTargets but in a more ultimately scalable way. However, this proposal doesn't appear to solve any security problems. What security provisions does SLP have? Presumably the target would still authenticate the initiator in the SLP connection to ensure the initiator had rights to get the target list. That part is the same as the current proposal, so you're not saving the target any work. Steph From owner-ips@ece.cmu.edu Wed May 23 03:22:22 2001 Received: from ece.cmu.edu ([128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id DAA28557 for ; Wed, 23 May 2001 03:22:21 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f4N5VKr06517 for ips-outgoing; Wed, 23 May 2001 01:31:20 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from e31.bld.us.ibm.com (e31.co.us.ibm.com [32.97.110.129]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f4N5VJH06510 for ; Wed, 23 May 2001 01:31:19 -0400 (EDT) Received: from westrelay02.boulder.ibm.com (westrelay02.boulder.ibm.com [9.99.140.23]) by e31.bld.us.ibm.com (8.9.3/8.9.3) with ESMTP id BAA93710 for ; Wed, 23 May 2001 01:23:42 -0400 Received: from f4n49e (d03nm065h.boulder.ibm.com [9.99.140.49]) by westrelay02.boulder.ibm.com (8.11.1/NCO v4.96.1.0) with ESMTP id f4N5VHV160894 for ; Tue, 22 May 2001 23:31:17 -0600 X-Priority: 1 (High) Importance: Normal Subject: RE: TCP Framing (considered helpful?) To: X-Mailer: Lotus Notes Release 5.0.3 (Intl) 21 March 2000 Message-ID: From: "John Hufferd" Date: Tue, 22 May 2001 22:30:54 -0700 X-MIMETrack: Serialize by Router on D03NM065/03/M/IBM(Release 5.0.6 |December 14, 2000) at 05/22/2001 11:31:16 PM MIME-Version: 1.0 Content-type: text/plain; charset=us-ascii Sender: owner-ips@ece.cmu.edu Precedence: bulk Somesh, I disagree with your statements. You should not believe that we can be separate from the standard TCP/IPs. This will not be accepted into the Business environment. Likewise, if you want to wait for Microsoft to start shipping the function, and have that function included into all the Desktops and Laptops that we want to support, it is going to be a long wait. This is not a negative statement about Microsoft, all large software projects, such as the next release of Windows, does not quickly put new things in that are not yet approved, or that are just recently approved in the standard. We have been waiting for sometime to get all the functions we wanted in Winsock Direct, and the progress there is being very measured. This is the way the real world works. They will, I think add Warpness over time, but we do not what to wait for that. Now for you to say that vendors such as SUN, HP, and IBM will be more casual about adding the function than Windows, is not the way I read the industry, in something as critical as TCP/IP. Markers, are simple, straight forward to implement, and will use very little processor power to use. I hope you agree with that the implementation is easy, and if your issue is that you disagree on my estimate on what processor power it takes to use, that is why we are suggesting it be Optional to use. The installation needs to measure a total cost performance trade off. That is, trade off more CPU cycles for Desktops and Laptops (which usually has more MIPS then they can use) vrs cost of Hardware in Targets. As for changing the PDU formats again, and adding padding, I think we have done quite enough of that, and have run down that path too many times to even discuss it again, so I think you called that idea correctly. . . . John L. Hufferd Senior Technical Staff Member (STSM) IBM/SSG San Jose Ca (408) 256-0403, Tie: 276-0403, eFax: (408) 904-4688 Internet address: hufferd@us.ibm.com "Somesh Gupta" @ece.cmu.edu on 05/22/2001 11:34:40 AM Please respond to Sent by: owner-ips@ece.cmu.edu To: cc: Subject: RE: TCP Framing (considered helpful?) I think if the framing solution finds favor with the TCP "gods", it is a much better alternative than the markers (which I was a proponent of - but only because I did not believe that a framing mechanism would be approved). Even if the framing mechanism is not approved, an alternate mechanism which provides for alignment of iSCSI headers at regular intervals (at negotiate progression in TCP sequence #s e.g. 0x12345, 0x22345, 0x32345 and so on) in the TCP stream is a much better solution. Unfortunately this would require some change to the header format (for padding) which probably no one would like to change at this time. The assumption that people will implement some change as big as iSCSI will be to the OS (in an integrated way - not as a kludgy add-on for demos) without being able to make the relatively minor change required to the API seems to be not a very convincing argument. Furthermore, at least in the Unix environment, iSCSI will be implemented in the kernel, and the accomodation for framing won't require an external API change. And as Steph says, the framing proposal has Microsoft participation. The insertion and deletion of markers seems to be painful accomodation for software based iSCSI solutions. The Unix folks might want to speak to this, but the markers will probably require more of a kludge in the kernel (to make it efficient) as compared to the changes required for framing. Somesh > -----Original Message----- > From: owner-ips@ece.cmu.edu [mailto:owner-ips@ece.cmu.edu]On Behalf Of > Stephen Bailey > Sent: Monday, May 21, 2001 6:51 AM > To: ips@ece.cmu.edu > Subject: Re: TCP Framing (considered helpful?) > > > John, > > > I think we must depend on Markers to insure that everything can > operate at > > top speed, and at the lowest cost. > > A key question is whether markers actually ensure that everything > operates at `top speed, and at the lowest cost'. > > Matt thinks so. I (and, presumably those who wrote the framing > document) think not. > > My issue is not even with `lowest cost'. I don't believe markers will > allow you to run at top speed. Specifically: > 1) I doubt the feasibility of implementing the control required for > an eddy buffer (where you store data you can't place) at 10G. > Admittedly, the validity of this claim can't really be assessed > without actually working the implementation, so for 99% of the > list participants (myself included) this is a `yes it is, no it > isn't' point. > 2) an eddy buffer solution requires some substantial speed-up in > both the NIC data path, and MOST IMPORTANTLY: the host bus. In > order to unload the eddy buffer while still handling incoming > traffic at line rate, clearly the host bus bandwidth must be > > line rate. > > I know of at least one general purpose framed solution operating at > 10G which has been available for >3 years (SGI's GSN/ST/XIO NIC). I'm > sure there are others. > > I can't imagine there's any argument that a framed solution would be > voted `most likely to run fast and be cheap'. Every storage network > and cluster interconnect has been designed that way since antiquity. > > The key tradeoff involves the OS vendors, and I'm wondering why we're > speaking for them. The question IS, how much more work is it to > introduce TCP framing over and above what is required to insert iSCSI > into their network framework. My experience from writing NIC and > storage drivers for many commercial UNIX-family OSes is: > 1) it's an easy and well defined process to insert a new SCSI > transport driver into the SCSI stack. > 2) it's hard and poorly defined process to insert ANYTHING into the > network stack. > > Networking has historically been a user-mode activity. Architected > services are only provided to user mode programs. Kernel clients have > been few and far between and so are handled on a case-by-case basis. > For example NFS. Every OS has hacks to make NFS run fast, but they > are not stable interfaces for general purpose use. > > Even Solaris' SysV-derived STREAMS stack, which is intended precisely > to provide flexible, crisp interfaces for kernel network clients, does > not document the relevant (IP stack) intermodule interfaces. > > I know that there are more and more kernel network clients, but they > are coming either on fluid platforms (e.g. linux), in which case the > argument of `it'll take too long to get OS support' doesn't apply, or > they are vendor-supplied, in which case a performance iSCSI solution > in ANY form may take a while, and the choice of framing or markers > isn't going to make a difference. > > I can't say squat about the architecture of Winsock, but the fact that > there is a Microsoft author of the framing proposal who seems very > serious about supporting framing and RDMA as quickly as possible > suggests that framing support should be available on Windows very > soon. > > Steph _________________________________________________________ Do You Yahoo!? Get your free @yahoo.com address at http://mail.yahoo.com From owner-ips@ece.cmu.edu Wed May 23 03:24:34 2001 Received: from ece.cmu.edu ([128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id DAA28585 for ; Wed, 23 May 2001 03:24:33 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f4N5PfS06111 for ips-outgoing; Wed, 23 May 2001 01:25:41 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from d12lmsgate-2.de.ibm.com (d12lmsgate-2.de.ibm.com [195.212.91.200]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f4N5PdH06106 for ; Wed, 23 May 2001 01:25:39 -0400 (EDT) Received: from d12relay02.de.ibm.com (d12relay02.de.ibm.com [9.165.215.23]) by d12lmsgate-2.de.ibm.com (1.0.0) with ESMTP id HAA283012 for ; Wed, 23 May 2001 07:25:33 +0200 From: julian_satran@il.ibm.com Received: from d12mta02.de.ibm.com (d12mta01_cs0 [9.165.222.237]) by d12relay02.de.ibm.com (8.8.8m3/NCO v4.96) with SMTP id HAA151672 for ; Wed, 23 May 2001 07:25:15 +0200 Received: by d12mta02.de.ibm.com(Lotus SMTP MTA v4.6.5 (863.2 5-20-1999)) id C1256A55.001DC5C1 ; Wed, 23 May 2001 07:25:11 +0200 X-Lotus-FromDomain: IBMIL@IBMDE To: ips@ece.cmu.edu Message-ID: Date: Wed, 23 May 2001 08:31:06 +0300 Subject: Re: iSCSI : Review Feedback on iSCSI Rev 06 Mime-Version: 1.0 Content-type: text/plain; charset=us-ascii Content-Disposition: inline Sender: owner-ips@ece.cmu.edu Precedence: bulk comments in text Julo Santosh Rao on 23-05-2001 04:49:40 Please respond to Santosh Rao To: IPS Reflector cc: Subject: Re: iSCSI : Review Feedback on iSCSI Rev 06 Julian, Thanks for the clarification. A few additional queries below. Regards, Santosh ----------------------------------------------------------------------------------- julian_satran@il.ibm.com wrote: > 1) Suggest re-naming the following in the interests of better > clarity : > > a) Rename the ExpDataSN in SCSI Response PDU to EndDataSN as was used in > the previous version of the draft. (indicative of being the final Data > sequence > number.) > +++ Meaning has changed - 0 means none (it was fffffff before) +++ Have the semantics of ExpDataSN in the SCSI Response PDU changed (?). From the description, it sounds like it is a count of the no of READ Data PDUs sent by the target for the command. 0 implies none were sent. (no data xfer). Hence, the name EndDataSN would be fitting since it really is the last DataSN the target used for this READ I/O. ---ExpDataSN is what would be the next DataSN (not the last). 0 means none expected, 1 means number 0 was sent and number 1 is expected (if at all)! --- > 3) Why does the new draft limit the size of text commands and nop commands > to > 4096 bytes ? What is the rationale behind the magic number selection of > 4096 ? > > Since DataPDULength is defined in units of 512 bytes, how does iSCSI deal > with > fragmentation caused due to the usage of a DataPDULength < 4K and an > attempt to > perform a 4K text , login or nop operation ? > > I believe such scenarios were discussed in an earlier thread > titled "Fragmentation & Reassembly issues in iSCSI." > (http://ips.pdl.cs.cmu.edu/mail/msg03031.html) and the resolution was to > enforce a restriction that the size of a login, nop & text operation be > limited to DataPDULength (?). > > +++ we decided to drop negotiating trivia. A fixed maximum is more > practical > and 4096 will no create inconvenience for a long time +++ > 5) Section 2.8.3 > "The data length of a text command or response SHOULD be less than > 4096 bytes." > > suggest re-wording the above to : > "The data length of a login, nop & text command or response MUST be less > than > DataPDULength. For the leading session login, the length of the login > command > MUST be less than 512 bytes." > > The above will ensure that login, text & nop operations will not result in > fragmentation. > > +++ I have a hard time understanding the argument +++ One basic question. Does DataPDULength govern the max length of SCSI Data PDUs only or all iSCSI PDUs ? This does'nt come across clearly from the definition. If this only governs SCSI Data PDUs, then, I'd suggest the following re-wrod to make it more clear in the definition on page 129 (Appendix E) : Change : "Initiator and target negotiate the maximum data payload supported for command or data PDUs in units of 512 bytes." to : "Initiator and target negotiate the maximum data payload supported for SCSI command or data PDUs in units of 512 bytes." The above issues (3) & (5) are relevant if DataPDULength governs the length of all iSCSI PDUs. --- OK --- > 9) Section 2.18.1 > > "Target requests logout". > > Suggest removal of the above option. A target that needs to logout would > use > either : > "Target will drop the connection" > or > "Target will drop all connections" > > The "Target requests logout" also does not allow the target to specify > which > flavor of logout it wishes to receive. It is better to remove this option > and > use only a single scheme. > > +++ The reason for introducing it was to allow a maintenance operation > (getting out a NIC card) to start > at the target. However only the initiator knows when the connects gets > quiet. This is why will have to keep this form. The other 2 are for > emergency handling +++ Why is "target will drop connection" or "target will drop all connections" not sufficient for this purpose ? In this case, the target would send the async message, drop the connection/session and abort all outstanding I/O. The initiator on seeing the async message would clean-up all outstanding I/O on that session/connection and would attempt to re-login after LogoutLoginMinTime. --- Because there might be commands in flight that the target does not know about when issuing the request --- - santoshr.vcf From owner-ips@ece.cmu.edu Wed May 23 05:16:52 2001 Received: from ece.cmu.edu ([128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id FAA29339 for ; Wed, 23 May 2001 05:16:51 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f4N6wv012030 for ips-outgoing; Wed, 23 May 2001 02:58:57 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from e31.bld.us.ibm.com (e31.co.us.ibm.com [32.97.110.129]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f4N6wtH12025 for ; Wed, 23 May 2001 02:58:56 -0400 (EDT) Received: from westrelay02.boulder.ibm.com (westrelay02.boulder.ibm.com [9.99.140.23]) by e31.bld.us.ibm.com (8.9.3/8.9.3) with ESMTP id CAA90240; Wed, 23 May 2001 02:51:19 -0400 Received: from f4n49e (d03nm065h.boulder.ibm.com [9.99.140.49]) by westrelay02.boulder.ibm.com (8.11.1/NCO v4.96.1.0) with ESMTP id f4N6wsV110706; Wed, 23 May 2001 00:58:54 -0600 X-Priority: 1 (High) Importance: Normal Subject: Re: TCP Framing (considered helpful?) To: Stephen Bailey Cc: ips@ece.cmu.edu X-Mailer: Lotus Notes Release 5.0.3 (Intl) 21 March 2000 Message-ID: From: "John Hufferd" Date: Tue, 22 May 2001 23:58:30 -0700 X-MIMETrack: Serialize by Router on D03NM065/03/M/IBM(Release 5.0.6 |December 14, 2000) at 05/23/2001 12:58:54 AM MIME-Version: 1.0 Content-type: text/plain; charset=us-ascii Sender: owner-ips@ece.cmu.edu Precedence: bulk Replies in text below (between [Huff] and [/Huff] ). . . . John L. Hufferd Senior Technical Staff Member (STSM) IBM/SSG San Jose Ca (408) 256-0403, Tie: 276-0403, eFax: (408) 904-4688 Internet address: hufferd@us.ibm.com Stephen Bailey @ece.cmu.edu on 05/21/2001 06:50:41 AM Sent by: owner-ips@ece.cmu.edu To: ips@ece.cmu.edu cc: Subject: Re: TCP Framing (considered helpful?) John, > I think we must depend on Markers to insure that everything can operate at > top speed, and at the lowest cost. A key question is whether markers actually ensure that everything operates at `top speed, and at the lowest cost'. Matt thinks so. I (and, presumably those who wrote the framing document) think not. [Huff] I do not think you can say that. I also support framing (Warp), as a much more elegant solution, but I find it inappropriate to depend on it actually happening, and being made available in the various OSs in the time frame we need. I do believe, that over time it will be made available, and is a better approach for all TCP/IP applications that can use it. [/Huff] My issue is not even with `lowest cost'. I don't believe markers will allow you to run at top speed. Specifically: 1) I doubt the feasibility of implementing the control required for an eddy buffer (where you store data you can't place) at 10G. Admittedly, the validity of this claim can't really be assessed without actually working the implementation, so for 99% of the list participants (myself included) this is a `yes it is, no it isn't' point. [Huff] I believe this has had much more work done on it then you think. I have personally stepped through the proposals from several vendors that are working on this option for their HW HBAs. Usually, because of the iSCSI PDU headers, the data/commands can be placed directly into the SCSI Host buffers, almost every time. Only when the PDU headers arrive slightly out of order (do to normal routing) are the packets unable to be placed directly into the Host buffer. And that requires some, but only a small amount, of buffering space. It is the packet drops that occur on PDU headers, and resultant error retries, that cause the need for large amounts of "on HBA/chip" buffering. So by using Markers, these HW iSCSI HBAs can limit the amount of buffering on the chip/HBAs. [/Huff] 2) an eddy buffer solution requires some substantial speed-up in both the NIC data path, and MOST IMPORTANTLY: the host bus. In order to unload the eddy buffer while still handling incoming traffic at line rate, clearly the host bus bandwidth must be > line rate. [Huff] This is not an effect of an eddy buffer solution, it is a fact that every TCP/IP NIC has to deal with. Especially at the new Speeds. Our current PCI buss will not support 10 Gigabit, further PCI-X will not support it either, even PCI-DDR does not fully support the full data rate. So it needs to rely on the TCP/IP window management. The only other thing you can do is drop the packets. this clearly makes the problem worse. [/Huff] I know of at least one general purpose framed solution operating at 10G which has been available for >3 years (SGI's GSN/ST/XIO NIC). I'm sure there are others. I can't imagine there's any argument that a framed solution would be voted `most likely to run fast and be cheap'. Every storage network and cluster interconnect has been designed that way since antiquity. The key tradeoff involves the OS vendors, and I'm wondering why we're speaking for them. The question IS, how much more work is it to introduce TCP framing over and above what is required to insert iSCSI into their network framework. My experience from writing NIC and storage drivers for many commercial UNIX-family OSes is: 1) it's an easy and well defined process to insert a new SCSI transport driver into the SCSI stack. 2) it's hard and poorly defined process to insert ANYTHING into the network stack. [Huff] I think you are making my point. This is the problem with SW Stacks. That is why I believe that it will take a very long time for the various vendors to include such changes into their "bet you business" TCP/IP SW Stacks. The point that Matt and I have been trying to make is that most OS vendors are NOT creating the iSCSI HW HBAs (NICs). These iSCSI HW HBAs (NICs) have the TCP/IP completely on the HBA, and they have added the iSCSI processing also so that they can steer the packets directly into the approprate SCSI Host buffers. Adding either Markers or Framing into the iSCSI HW HBAs is not a big problem. It is only a problem of getting Framing (timely) into Host TCP/IP Stacks. [/Huff] Networking has historically been a user-mode activity. Architected services are only provided to user mode programs. Kernel clients have been few and far between and so are handled on a case-by-case basis. For example NFS. Every OS has hacks to make NFS run fast, but they are not stable interfaces for general purpose use. Even Solaris' SysV-derived STREAMS stack, which is intended precisely to provide flexible, crisp interfaces for kernel network clients, does not document the relevant (IP stack) intermodule interfaces. I know that there are more and more kernel network clients, but they are coming either on fluid platforms (e.g. linux), in which case the argument of `it'll take too long to get OS support' doesn't apply, or they are vendor-supplied, in which case a performance iSCSI solution in ANY form may take a while, and the choice of framing or markers isn't going to make a difference. [Huff] I think you are saying something I agree with and something I do not agree with. That is, that software changes to TCP/IP in the various "Bet you Business" OSs, will take some time. However, it is not true that new iSCSI device drivers will take very long. Two types are being created today. By Cisco, IBM, Intel, etc. These types are iSCSI DD that make calls to normal TCP/IP stacks, and the DD that are being written by the iSCSI HW HBA vendors. These do not require the OS vendor to do anything special. This is happening NOW, (Check with CISCO, Intel, and IBM (me?)). The last thing we want is to depend on a TCP/IP change to get in the way of our momentum. [/Huff] I can't say squat about the architecture of Winsock, but the fact that there is a Microsoft author of the framing proposal who seems very serious about supporting framing and RDMA as quickly as possible suggests that framing support should be available on Windows very soon. [Huff] My following statements are not meant as a negative of Microsoft. However, they and all producers of Key complicated new Software do not quickly bring these to the general market in a way that is as pleasing to HW vendors as HW vendors would like. I believe that Microsoft's heart is in the right place on this issue, and that they will do the right thing with framing, over time. But it is not clear in what release that will be shipped, nor what support pack it will be included. Also it is not clear how the support will be handled for current Win2k, WinNT etc. This is why I think we should have Framing a Must implement and an Optional to use. It is the easiest thing for SW to create, and brings the needed cost reduction to iSCSI HW and it is completely under our (iSCSI protocol) control. [/Huff] Steph From owner-ips@ece.cmu.edu Wed May 23 07:22:33 2001 Received: from ece.cmu.edu ([128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id HAA00516 for ; Wed, 23 May 2001 07:22:33 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f4N9pBK01943 for ips-outgoing; Wed, 23 May 2001 05:51:11 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from bramg1.net.external.hp.com (bramg1.net.external.hp.com [192.6.126.73]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f4N9pAH01938 for ; Wed, 23 May 2001 05:51:10 -0400 (EDT) Received: from quasit.br.itc.hp.com (quasit.br.itc.hp.com [15.145.8.135]) by bramg1.net.external.hp.com (Postfix) with ESMTP id 2C56CE9 for ; Wed, 23 May 2001 11:51:08 +0200 (METDST) Received: from fowey.BR.ITC.HP.COM (fowey.br.itc.hp.com [15.145.8.186]) by quasit.br.itc.hp.com (8.9.3 (PHNE_18979)/8.9.3 SMKit6.0.6 OpenMail) with SMTP id KAA06260 for ; Wed, 23 May 2001 10:51:08 +0100 (BST) Received: from 15.145.8.186 by fowey.BR.ITC.HP.COM (InterScan E-Mail VirusWall NT); Wed, 23 May 2001 10:51:07 +0100 (GMT Daylight Time) Received: by fowey.br.itc.hp.com with Internet Mail Service (5.5.2653.19) id ; Wed, 23 May 2001 10:51:07 +0100 Message-ID: <0B9A57FF1D57D411B47500D0B73E5CC101E7A708@dickens.bri.hp.com> From: "BURBRIDGE,MATTHEW (HP-UnitedKingdom,ex2)" To: ips@ece.cmu.edu Subject: Asynchronous Message: Session Termination Date: Wed, 23 May 2001 10:51:04 +0100 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2653.19) Content-Type: text/plain; charset="iso-8859-1" Sender: owner-ips@ece.cmu.edu Precedence: bulk Hi Julian, If a target only supports Session Recovery, then when an error is detected it will terminate the session. The current Async Message (iSCSI Event "Target indicates that his will/has dropped all connections") does not necessary inform the initiator that the session has terminated and the initiator is quite within rights to establish a new connection and attempt With-in Session recovery. Can you change the spec to reflect that a value of zero in both the Parameter2 and Parameter3 would set the maximum time to reconnect to "Never" thereby informing the initiator that the session has closed? Thanks Matthew Burbridge NIS-Bristol Hewlett Packard Telnet: 312 7010 E-mail: matthewb@bri.hp.com From owner-ips@ece.cmu.edu Wed May 23 07:25:30 2001 Received: from ece.cmu.edu ([128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id HAA00566 for ; Wed, 23 May 2001 07:25:30 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f4N9Vsc00817 for ips-outgoing; Wed, 23 May 2001 05:31:54 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from d12lmsgate-2.de.ibm.com (d12lmsgate-2.de.ibm.com [195.212.91.200]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f4N9VqH00810 for ; Wed, 23 May 2001 05:31:52 -0400 (EDT) Received: from d12relay01.de.ibm.com (d12relay01.de.ibm.com [9.165.215.22]) by d12lmsgate-2.de.ibm.com (1.0.0) with ESMTP id LAA118178 for ; Wed, 23 May 2001 11:31:34 +0200 From: julian_satran@il.ibm.com Received: from d12mta02.de.ibm.com (d12mta01_cs0 [9.165.222.237]) by d12relay01.de.ibm.com (8.8.8m3/NCO v4.96) with SMTP id LAA57078 for ; Wed, 23 May 2001 11:31:34 +0200 Received: by d12mta02.de.ibm.com(Lotus SMTP MTA v4.6.5 (863.2 5-20-1999)) id C1256A55.00344CD3 ; Wed, 23 May 2001 11:31:15 +0200 X-Lotus-FromDomain: IBMIL@IBMDE To: ips@ece.cmu.edu Message-ID: Date: Wed, 23 May 2001 12:37:12 +0300 Subject: RE: iSCSI: Final bit/status clarification Mime-Version: 1.0 Content-type: text/plain; charset=us-ascii Content-Disposition: inline Sender: owner-ips@ece.cmu.edu Precedence: bulk It was a typo. Thanks, Julo Dennis Young on 22-05-2001 21:58:45 Please respond to Dennis Young To: ips@ece.cmu.edu cc: Subject: RE: iSCSI: Final bit/status clarification I believe there is a typo in the 3rd paragraph of sec 4.3 of iSCSI draft 06: "... or a login response with the text bit set to 0, ..." The word 'text' should be 'F', right? In general, should we use this mailing list for pointing out typos (sometimes I don't know for sure they are typos). Is there an errata somewhere for draft 06 already? Thanks. -----Original Message----- From: Eddy Quicksall [mailto:EQuicksall@mediaone.net] Sent: Monday, May 21, 2001 9:26 AM To: julian_satran@il.ibm.com; ips@ece.cmu.edu Subject: iSCSI: Final bit/status clarification Section 2.7.1 F (Final) Bit For incoming data, this bit is 1 for the last input data PDU associated with the command (even if it includes the status). Is a clarification implied by "even if it includes the status"? I don't see the need for a clarification here because I don't see any ambiguity. Would it make sense to remove it? Or, maybe it should say "even if it does not include the status". -------- mailto:Eddy@Quicksall.com From owner-ips@ece.cmu.edu Wed May 23 08:27:45 2001 Received: from ece.cmu.edu ([128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id IAA02391 for ; Wed, 23 May 2001 08:27:45 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f4NADZN03199 for ips-outgoing; Wed, 23 May 2001 06:13:35 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from bramg1.net.external.hp.com (bramg1.net.external.hp.com [192.6.126.73]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f4NADXH03194 for ; Wed, 23 May 2001 06:13:33 -0400 (EDT) Received: from quasit.br.itc.hp.com (quasit.br.itc.hp.com [15.145.8.135]) by bramg1.net.external.hp.com (Postfix) with ESMTP id E015DAD for ; Wed, 23 May 2001 12:13:27 +0200 (METDST) Received: from loddon.br.itc.hp.com (loddon.br.itc.hp.com [15.145.8.166]) by quasit.br.itc.hp.com (8.9.3 (PHNE_18979)/8.9.3 SMKit6.0.6 OpenMail) with SMTP id LAA13166 for ; Wed, 23 May 2001 11:13:28 +0100 (BST) Received: from 15.145.8.166 by loddon.br.itc.hp.com (InterScan E-Mail VirusWall NT); Wed, 23 May 2001 11:13:28 +0100 (GMT Daylight Time) Received: by loddon.br.itc.hp.com with Internet Mail Service (5.5.2653.19) id ; Wed, 23 May 2001 11:13:28 +0100 Message-ID: <0B9A57FF1D57D411B47500D0B73E5CC101E7A709@dickens.bri.hp.com> From: "BURBRIDGE,MATTHEW (HP-UnitedKingdom,ex2)" To: "'ips@ece.cmu.edu'" Subject: Login Rejection: Target does not support multiple addresses per S ession Date: Wed, 23 May 2001 11:13:25 +0100 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2653.19) Content-Type: text/plain; charset="ISO-8859-1" Sender: owner-ips@ece.cmu.edu Precedence: bulk Hi Julian If an initiator attempts to set up a new connection for an existing session (i.e TSID is non-zero, and Restart bit is 0) but via another physical port then if the target does not support session spanning there is currently no Login Response Status code that covers the scenario. Please can you add an additional error code which informs the initiator that the login has been rejected due to a target not being able to support session spanning (e.g. "Target does not support multiple addresses per Session"). Attempts by the initiator to connect to the session via the original port, or for Within-session recovery, or to create an completely new session, would not be rejected using this error code. Thanks Matthew Burbridge NIS-Bristol Hewlett Packard Telnet: 312 7010 E-mail: matthewb@bri.hp.com From owner-ips@ece.cmu.edu Wed May 23 12:20:36 2001 Received: from ece.cmu.edu ([128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id MAA08811 for ; Wed, 23 May 2001 12:20:35 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f4NDWI015434 for ips-outgoing; Wed, 23 May 2001 09:32:18 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from mail.wrs.com (unknown-1-11.windriver.com [147.11.1.11]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f4NDW7H15415 for ; Wed, 23 May 2001 09:32:16 -0400 (EDT) Received: from london ([192.168.1.20]) by mail.wrs.com (8.9.3/8.9.1) with SMTP id GAA05121 for ; Wed, 23 May 2001 06:31:25 -0700 (PDT) From: "Rod Harrison" To: Subject: RE: TCP Framing (considered helpful?) Date: Wed, 23 May 2001 14:31:48 +0100 Message-ID: MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit X-Priority: 3 (Normal) X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook IMO, Build 9.0.2416 (9.0.2910.0) X-MimeOLE: Produced By Microsoft MimeOLE V5.00.2314.1300 Importance: Normal In-Reply-To: Sender: owner-ips@ece.cmu.edu Precedence: bulk Content-Transfer-Encoding: 7bit I whole heartedly agree on not mandating TCP framing for all the reasons already posted. Having worked in the kernel team for an OSV for 8 years I can tell you there is a lot of inertia attached to pushing an OS out of the door. Those projects are planned a long way ahead of the ship date and putting things in late rarely, if ever, happens. Also, they are never shipped when they are ready, there are a couple of windows in the year and that's it. I've seen finished product sit on the shelf for months waiting for the non-engineering folks to decide it's time to ship. So, in summary, even if the OSVs we all care about have TCP framing done and dusted now, I think it could easily be a year or more before those stacks hit the streets. Too long in my opinion. However, since it looks like TCP framing is going to happen why don't we try and future proof ourselves and allow its use to be negotiated? Both sides of the iSCSI session will know if they have TCP framing or not so how about adding a TCP parameter to the FMarker text key. As has been suggested before we can mandate support for markers but allow their use to be optional. Something like .... FMarker= This should allow a session to use TCP markers if they are available and markers if not. - Rod -----Original Message----- From: owner-ips@ece.cmu.edu [mailto:owner-ips@ece.cmu.edu]On Behalf Of John Hufferd Sent: Wednesday, May 23, 2001 6:31 AM To: ips@ece.cmu.edu Subject: RE: TCP Framing (considered helpful?) Somesh, I disagree with your statements. You should not believe that we can be separate from the standard TCP/IPs. This will not be accepted into the Business environment. Likewise, if you want to wait for Microsoft to start shipping the function, and have that function included into all the Desktops and Laptops that we want to support, it is going to be a long wait. This is not a negative statement about Microsoft, all large software projects, such as the next release of Windows, does not quickly put new things in that are not yet approved, or that are just recently approved in the standard. We have been waiting for sometime to get all the functions we wanted in Winsock Direct, and the progress there is being very measured. This is the way the real world works. They will, I think add Warpness over time, but we do not what to wait for that. Now for you to say that vendors such as SUN, HP, and IBM will be more casual about adding the function than Windows, is not the way I read the industry, in something as critical as TCP/IP. Markers, are simple, straight forward to implement, and will use very little processor power to use. I hope you agree with that the implementation is easy, and if your issue is that you disagree on my estimate on what processor power it takes to use, that is why we are suggesting it be Optional to use. The installation needs to measure a total cost performance trade off. That is, trade off more CPU cycles for Desktops and Laptops (which usually has more MIPS then they can use) vrs cost of Hardware in Targets. As for changing the PDU formats again, and adding padding, I think we have done quite enough of that, and have run down that path too many times to even discuss it again, so I think you called that idea correctly. . . . John L. Hufferd Senior Technical Staff Member (STSM) IBM/SSG San Jose Ca (408) 256-0403, Tie: 276-0403, eFax: (408) 904-4688 Internet address: hufferd@us.ibm.com "Somesh Gupta" @ece.cmu.edu on 05/22/2001 11:34:40 AM Please respond to Sent by: owner-ips@ece.cmu.edu To: cc: Subject: RE: TCP Framing (considered helpful?) I think if the framing solution finds favor with the TCP "gods", it is a much better alternative than the markers (which I was a proponent of - but only because I did not believe that a framing mechanism would be approved). Even if the framing mechanism is not approved, an alternate mechanism which provides for alignment of iSCSI headers at regular intervals (at negotiate progression in TCP sequence #s e.g. 0x12345, 0x22345, 0x32345 and so on) in the TCP stream is a much better solution. Unfortunately this would require some change to the header format (for padding) which probably no one would like to change at this time. The assumption that people will implement some change as big as iSCSI will be to the OS (in an integrated way - not as a kludgy add-on for demos) without being able to make the relatively minor change required to the API seems to be not a very convincing argument. Furthermore, at least in the Unix environment, iSCSI will be implemented in the kernel, and the accomodation for framing won't require an external API change. And as Steph says, the framing proposal has Microsoft participation. The insertion and deletion of markers seems to be painful accomodation for software based iSCSI solutions. The Unix folks might want to speak to this, but the markers will probably require more of a kludge in the kernel (to make it efficient) as compared to the changes required for framing. Somesh > -----Original Message----- > From: owner-ips@ece.cmu.edu [mailto:owner-ips@ece.cmu.edu]On Behalf Of > Stephen Bailey > Sent: Monday, May 21, 2001 6:51 AM > To: ips@ece.cmu.edu > Subject: Re: TCP Framing (considered helpful?) > > > John, > > > I think we must depend on Markers to insure that everything can > operate at > > top speed, and at the lowest cost. > > A key question is whether markers actually ensure that everything > operates at `top speed, and at the lowest cost'. > > Matt thinks so. I (and, presumably those who wrote the framing > document) think not. > > My issue is not even with `lowest cost'. I don't believe markers will > allow you to run at top speed. Specifically: > 1) I doubt the feasibility of implementing the control required for > an eddy buffer (where you store data you can't place) at 10G. > Admittedly, the validity of this claim can't really be assessed > without actually working the implementation, so for 99% of the > list participants (myself included) this is a `yes it is, no it > isn't' point. > 2) an eddy buffer solution requires some substantial speed-up in > both the NIC data path, and MOST IMPORTANTLY: the host bus. In > order to unload the eddy buffer while still handling incoming > traffic at line rate, clearly the host bus bandwidth must be > > line rate. > > I know of at least one general purpose framed solution operating at > 10G which has been available for >3 years (SGI's GSN/ST/XIO NIC). I'm > sure there are others. > > I can't imagine there's any argument that a framed solution would be > voted `most likely to run fast and be cheap'. Every storage network > and cluster interconnect has been designed that way since antiquity. > > The key tradeoff involves the OS vendors, and I'm wondering why we're > speaking for them. The question IS, how much more work is it to > introduce TCP framing over and above what is required to insert iSCSI > into their network framework. My experience from writing NIC and > storage drivers for many commercial UNIX-family OSes is: > 1) it's an easy and well defined process to insert a new SCSI > transport driver into the SCSI stack. > 2) it's hard and poorly defined process to insert ANYTHING into the > network stack. > > Networking has historically been a user-mode activity. Architected > services are only provided to user mode programs. Kernel clients have > been few and far between and so are handled on a case-by-case basis. > For example NFS. Every OS has hacks to make NFS run fast, but they > are not stable interfaces for general purpose use. > > Even Solaris' SysV-derived STREAMS stack, which is intended precisely > to provide flexible, crisp interfaces for kernel network clients, does > not document the relevant (IP stack) intermodule interfaces. > > I know that there are more and more kernel network clients, but they > are coming either on fluid platforms (e.g. linux), in which case the > argument of `it'll take too long to get OS support' doesn't apply, or > they are vendor-supplied, in which case a performance iSCSI solution > in ANY form may take a while, and the choice of framing or markers > isn't going to make a difference. > > I can't say squat about the architecture of Winsock, but the fact that > there is a Microsoft author of the framing proposal who seems very > serious about supporting framing and RDMA as quickly as possible > suggests that framing support should be available on Windows very > soon. > > Steph _________________________________________________________ Do You Yahoo!? Get your free @yahoo.com address at http://mail.yahoo.com From owner-ips@ece.cmu.edu Wed May 23 13:41:46 2001 Received: from ece.cmu.edu ([128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id NAA10787 for ; Wed, 23 May 2001 13:41:45 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f4NFJvX24060 for ips-outgoing; Wed, 23 May 2001 11:19:57 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from dogwood.cisco.com (dogwood.cisco.com [161.44.11.19]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f4NFJtH24051 for ; Wed, 23 May 2001 11:19:55 -0400 (EDT) Received: from cisco.com (mbakke@mbakke-lnx.cisco.com [161.44.68.87]) by dogwood.cisco.com (8.8.6 (PHNE_14041)/CISCO.SERVER.1.2) with ESMTP id LAA24728; Wed, 23 May 2001 11:19:49 -0400 (EDT) Message-ID: <3B0BD463.8854907B@cisco.com> Date: Wed, 23 May 2001 10:16:51 -0500 From: Mark Bakke X-Mailer: Mozilla 4.72 [en] (X11; U; Linux 2.2.16-3.uid32 i686) X-Accept-Language: en, de MIME-Version: 1.0 To: "BURBRIDGE,MATTHEW (HP-UnitedKingdom,ex2)" CC: ips@ece.cmu.edu Subject: Re: Asynchronous Message: Session Termination References: <0B9A57FF1D57D411B47500D0B73E5CC101E7A708@dickens.bri.hp.com> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-ips@ece.cmu.edu Precedence: bulk Content-Transfer-Encoding: 7bit Matthew- The intent of these was not to connect and re-establish the same session after a set amount of time; it was to inform the initiator not to try to establish a new session until the time passes, which is I think what you want. Perhaps changing the description of #4 from: 4 Target indicates it will/has dropped all connections - the Parameter2 field indicates, in seconds, the minimum time to reconnect and Parameter3 the maximum time to reconnect. to: 4 Target indicates it will/has dropped all connections - the Parameter2 field indicates, in seconds, the minimum time to wait to establish a new session and Parameter3 the maximum time to wait to establish a new session. Actually, I had originally requested this, to allow a target to say "I'm shutting down / failing over in X seconds; and will be back again in Y seconds, so don't bother connecting until then". With this behavior, the description should read: 4 Target indicates it will/has dropped all connections - the Parameter2 field indicates, in seconds, the time until all connections will be terminated (or zero if immediately), and Parameter3 the maximum time the initiator should wait before attempting to establish a new session. If everyone agrees with this, I would rather have the above description. -- Mark "BURBRIDGE,MATTHEW (HP-UnitedKingdom,ex2)" wrote: > > Hi Julian, > > If a target only supports Session Recovery, then when an error is detected > it will terminate the session. The current Async Message (iSCSI Event > "Target indicates that his will/has dropped all connections") does not > necessary inform the initiator that the session has terminated and the > initiator is quite within rights to establish a new connection and attempt > With-in Session recovery. > > Can you change the spec to reflect that a value of zero in both the > Parameter2 and Parameter3 would set the maximum time to reconnect to "Never" > thereby informing the initiator that the session has closed? > > Thanks > > Matthew Burbridge > NIS-Bristol > Hewlett Packard > Telnet: 312 7010 > E-mail: matthewb@bri.hp.com -- Mark A. Bakke Cisco Systems mbakke@cisco.com 763.398.1054 From owner-ips@ece.cmu.edu Wed May 23 14:29:33 2001 Received: from ece.cmu.edu ([128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id OAA11712 for ; Wed, 23 May 2001 14:29:33 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f4NGJ4S28845 for ips-outgoing; Wed, 23 May 2001 12:19:04 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from bramg1.net.external.hp.com (bramg1.net.external.hp.com [192.6.126.73]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f4NGJ2H28838 for ; Wed, 23 May 2001 12:19:02 -0400 (EDT) Received: from quasit.br.itc.hp.com (quasit.br.itc.hp.com [15.145.8.135]) by bramg1.net.external.hp.com (Postfix) with ESMTP id 50F85A2; Wed, 23 May 2001 18:19:00 +0200 (METDST) Received: from loddon.br.itc.hp.com (loddon.br.itc.hp.com [15.145.8.166]) by quasit.br.itc.hp.com (8.9.3 (PHNE_18979)/8.9.3 SMKit6.0.6 OpenMail) with SMTP id RAA16072; Wed, 23 May 2001 17:18:50 +0100 (BST) Received: from 15.145.8.166 by loddon.br.itc.hp.com (InterScan E-Mail VirusWall NT); Wed, 23 May 2001 17:18:50 +0100 (GMT Daylight Time) Received: by loddon.br.itc.hp.com with Internet Mail Service (5.5.2653.19) id ; Wed, 23 May 2001 17:18:50 +0100 Message-ID: <0B9A57FF1D57D411B47500D0B73E5CC101E7A70E@dickens.bri.hp.com> From: "BURBRIDGE,MATTHEW (HP-UnitedKingdom,ex2)" To: "'Mark Bakke'" Cc: ips@ece.cmu.edu Subject: RE: Asynchronous Message: Session Termination Date: Wed, 23 May 2001 17:18:34 +0100 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2653.19) Content-Type: text/plain; charset="ISO-8859-1" Sender: owner-ips@ece.cmu.edu Precedence: bulk Mark, The last description is good and covers all situation. Presumably, if Parameter3 is zero then the initiator is able to establish a new session immediately. Can you modify the description to specify that Parameter3 must be greater than or equal to Parameter2 (it makes no sense to have a value less than Parameter2) and that zero is a valid value for Parameter3. Matthew -----Original Message----- From: Mark Bakke [mailto:mbakke@cisco.com] Sent: Wednesday, May 23, 2001 4:17 PM To: BURBRIDGE,MATTHEW (HP-UnitedKingdom,ex2) Cc: ips@ece.cmu.edu Subject: Re: Asynchronous Message: Session Termination Matthew- The intent of these was not to connect and re-establish the same session after a set amount of time; it was to inform the initiator not to try to establish a new session until the time passes, which is I think what you want. Perhaps changing the description of #4 from: 4 Target indicates it will/has dropped all connections - the Parameter2 field indicates, in seconds, the minimum time to reconnect and Parameter3 the maximum time to reconnect. to: 4 Target indicates it will/has dropped all connections - the Parameter2 field indicates, in seconds, the minimum time to wait to establish a new session and Parameter3 the maximum time to wait to establish a new session. Actually, I had originally requested this, to allow a target to say "I'm shutting down / failing over in X seconds; and will be back again in Y seconds, so don't bother connecting until then". With this behavior, the description should read: 4 Target indicates it will/has dropped all connections - the Parameter2 field indicates, in seconds, the time until all connections will be terminated (or zero if immediately), and Parameter3 the maximum time the initiator should wait before attempting to establish a new session. If everyone agrees with this, I would rather have the above description. -- Mark "BURBRIDGE,MATTHEW (HP-UnitedKingdom,ex2)" wrote: > > Hi Julian, > > If a target only supports Session Recovery, then when an error is detected > it will terminate the session. The current Async Message (iSCSI Event > "Target indicates that his will/has dropped all connections") does not > necessary inform the initiator that the session has terminated and the > initiator is quite within rights to establish a new connection and attempt > With-in Session recovery. > > Can you change the spec to reflect that a value of zero in both the > Parameter2 and Parameter3 would set the maximum time to reconnect to "Never" > thereby informing the initiator that the session has closed? > > Thanks > > Matthew Burbridge > NIS-Bristol > Hewlett Packard > Telnet: 312 7010 > E-mail: matthewb@bri.hp.com -- Mark A. Bakke Cisco Systems mbakke@cisco.com 763.398.1054 From owner-ips@ece.cmu.edu Wed May 23 15:22:11 2001 Received: from ece.cmu.edu ([128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id PAA12562 for ; Wed, 23 May 2001 15:22:10 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f4NHftg05582 for ips-outgoing; Wed, 23 May 2001 13:41:55 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from palrel2.hp.com (palrel2.hp.com [156.153.255.234]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f4NHfrH05578 for ; Wed, 23 May 2001 13:41:53 -0400 (EDT) Received: from hpcuhe.cup.hp.com (hpcuhe.cup.hp.com [15.0.80.203]) by palrel2.hp.com (Postfix) with ESMTP id B9D9E2B5D for ; Wed, 23 May 2001 10:41:51 -0700 (PDT) Received: from cup.hp.com (santoshr@hpindhhm.cup.hp.com [15.8.80.197]) by hpcuhe.cup.hp.com (8.9.3 (PHNE_18979)/8.9.3 SMKit7.02) with ESMTP id KAA11982 for ; Wed, 23 May 2001 10:41:47 -0700 (PDT) Message-ID: <3B0BF67D.797A871A@cup.hp.com> Date: Wed, 23 May 2001 10:42:21 -0700 From: Santosh Rao Organization: Hewlett Packard, Cupertino. X-Mailer: Mozilla 4.7 [en] (X11; U; HP-UX B.11.00 9000/778) X-Accept-Language: en MIME-Version: 1.0 To: ips@ece.cmu.edu Subject: Re: Asynchronous Message: Session Termination References: <0B9A57FF1D57D411B47500D0B73E5CC101E7A708@dickens.bri.hp.com> <3B0BD463.8854907B@cisco.com> Content-Type: multipart/mixed; boundary="------------4EDE2B663F7057C407C8E71A" Sender: owner-ips@ece.cmu.edu Precedence: bulk This is a multi-part message in MIME format. --------------4EDE2B663F7057C407C8E71A Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit The Async Message "target will drop all connections" should be described to imply that the session is logged out and all outstanding I/Os are going to be aborted at the target. This allows this feature to be used by targets to drive session recovery. Mark Bakke wrote: > > Actually, I had originally requested this, to allow a target to say > "I'm shutting down / failing over in X seconds; and will be back again > in Y seconds, so don't bother connecting until then". With this > behavior, the description should read: > > 4 Target indicates it will/has dropped all connections - the > Parameter2 field indicates, in seconds, the time until all > connections will be terminated (or zero if immediately), and > Parameter3 the maximum time the initiator should wait before ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Did you intend to mean "the minimum time the initiator should wait"... ? The min time before retrying login adds value to the initiator rather than the max time before retrying login in this case. In the case where "target will drop all connections" was used, the max time really holds no relevance, since the session has been logged out. Hence, I like Mark's definition on this which provides the initiator with time when session will be logged out and time to wait before attempting to retry session login. > attempting to establish a new session. > > If everyone agrees with this, I would rather have the above description. > > -- > Mark --------------4EDE2B663F7057C407C8E71A Content-Type: text/x-vcard; charset=us-ascii; name="santoshr.vcf" Content-Description: Card for Santosh Rao Content-Disposition: attachment; filename="santoshr.vcf" Content-Transfer-Encoding: 7bit begin:vcard n:Rao;Santosh tel;work:408-447-3751 x-mozilla-html:FALSE org:Hewlett Packard, Cupertino.;SISL adr:;;19420, Homestead Road, M\S 43LN, ;Cupertino.;CA.;95014.;USA. version:2.1 email;internet:santoshr@cup.hp.com title:Software Design Engineer x-mozilla-cpt:;21088 fn:Santosh Rao end:vcard --------------4EDE2B663F7057C407C8E71A-- From owner-ips@ece.cmu.edu Wed May 23 15:29:09 2001 Received: from ece.cmu.edu ([128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id PAA12741 for ; Wed, 23 May 2001 15:29:09 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f4NGwPx01912 for ips-outgoing; Wed, 23 May 2001 12:58:25 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from msgbas1.cos.agilent.com (msgbas1x.cos.agilent.com [192.6.9.33]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f4NGwNH01889 for ; Wed, 23 May 2001 12:58:23 -0400 (EDT) Received: from msgrel1.and.agilent.com (msgrel1.and.agilent.com [130.30.33.104]) by msgbas1.cos.agilent.com (Postfix) with ESMTP id E775E2B9 for ; Wed, 23 May 2001 10:58:21 -0600 (MDT) Received: from rtl.rose.agilent.com (rtl.rose.agilent.com [156.140.232.231]) by msgrel1.and.agilent.com (Postfix) with ESMTP id 17B95BD for ; Wed, 23 May 2001 12:58:21 -0400 (EDT) Received: from mail.rose.agilent.com (mailsrv@bellhop [156.140.233.51]) by rtl.rose.agilent.com (8.9.3 (PHNE_18979)/8.9.3 SMKit7.1.0) with ESMTP id JAA08402 for ; Wed, 23 May 2001 09:58:20 -0700 (PDT) Received: from agilent.com (cos1nai129158.cos.agilent.com [141.184.129.158]) by mail.rose.agilent.com (Netscape Messaging Server 3.6) with ESMTP id AAA5D52 for ; Wed, 23 May 2001 09:58:16 -0700 Message-ID: <3B0BEBFD.9347890E@agilent.com> Date: Wed, 23 May 2001 09:57:33 -0700 From: "Matt Wakeley" Reply-To: Matt Wakeley Organization: Agilent Technologies X-Mailer: Mozilla 4.77 [en] (Windows NT 5.0; U) X-Accept-Language: en MIME-Version: 1.0 To: ips@ece.cmu.edu Subject: Re: Asynchronous Message: Session Termination References: <0B9A57FF1D57D411B47500D0B73E5CC101E7A708@dickens.bri.hp.com> <3B0BD463.8854907B@cisco.com> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-ips@ece.cmu.edu Precedence: bulk Content-Transfer-Encoding: 7bit Mark Bakke wrote: > > Matthew- > > The intent of these was not to connect and re-establish the same > session after a set amount of time; it was to inform the initiator > not to try to establish a new session until the time passes, which > is I think what you want. Mark, I don't think that was the intent of this feature. Just because a target drops all it's connections, does not mean that the session is closed. In fact, the "maximum" time seems to indicate that if a connection is not tried within that amount of time, then the session will be aborted, or some other action taken. Obviously, clarification on the intent is required, and then clarification in the text. -Matt From owner-ips@ece.cmu.edu Wed May 23 16:42:54 2001 Received: from ece.cmu.edu ([128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id QAA14645 for ; Wed, 23 May 2001 16:42:54 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f4NIOtS09296 for ips-outgoing; Wed, 23 May 2001 14:24:55 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from pluto.he.net (pluto.he.net [216.218.167.2]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f4NIOqH09290 for ; Wed, 23 May 2001 14:24:53 -0400 (EDT) Received: from DSLmodem (63-229-221-92.customers.uswest.net [63.229.221.92]) by pluto.he.net (8.8.6/8.8.2) with SMTP id LAA09879 for ; Wed, 23 May 2001 11:24:49 -0700 From: "Sriram Rupanagunta" To: "IPS Reflector" Subject: RE: FCIP - Discovery should be Manual/SLP Date: Wed, 23 May 2001 13:31:15 -0500 Message-ID: MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit X-Priority: 3 (Normal) X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook IMO, Build 9.0.2416 (9.0.2910.0) In-Reply-To: <3B02F012.241E76D1@compuserve.com> X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4133.2400 Importance: Normal Sender: owner-ips@ece.cmu.edu Precedence: bulk Content-Transfer-Encoding: 7bit Looking at the requirements in FCIP world, this seems to me to be the right approach. -Sriram > > I support Dave's proposed text for FCIP Clause 4.2 item 3, with the > exception that the sentence regarding iSNS needs to be removed. > Looking at the features list, I cannot see any iSNS unique features > that are meaningful to FCIP. > > Thanks. > > Ralph... > > - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - > > Subject: Proposal to use SLPv2 for FCIP device discovery > Date: Mon, 30 Apr 2001 17:07:08 -0500 > From: "Dave Peterson" > To: "Ips@Ece. Cmu. Edu" > > > As requested here is the proposal I presented at Mondays' FCIP > meeting: > > FCIP Draft Proposal > For Clause 4.2, item 3. > > Each FCIP device may be statically or dynamically configured with > a list of IP addresses and port numbers corresponding to all the > participating FCIP devices. If dynamic discovery of participating > FCIP devices is supported this function shall be performed using the > Service Location Protocol (SLPv2). For additional FCIP device > management capability, the iSNS Internet Storage Naming Service may > be implemented. It is outside the scope of this specification to > describe any static configuration method for participating FCIP device > discovery. Refer to clause XXX for a detailed description of dynamic > discovery of participating FCIP device using SLPv2. > > Notes: > 1. DHCP: > a. Allows an entity to discovery information about itself, > not discover information about all other entities. > b. Uses a broadcast mechanism that may not work via routers > without additional configuration. But, most current router > implementations will support the forwarding of DHCP requests > across routed subnets. > c. May be used to find SLP Directory Agents and Scope Lists > allowing for a more scalable solution. > 2. DNS Services are too limiting. This is the reason why SLP was > started. > 3. LDAP is simply a database interface. SLP and iSNS may use LDAP > as a back-end store. > 4. SLP FCIP Device Type Specifics > a. IP address(es) > b. Port numbers(s) > c. Scope > d. Attributes > i. Discovery domain (i.e. a group name or zone name, don't want > to use zone name in this context though) > ii. need to start listing these (if we can think of any more) > e. Lifetime > > Work In progress: > 1. Putting together a model for FCIP device discovery using SLPv2. > 2. Implementing a "default/suggested" SLPv2 template. > 3. Reviewing RFC 3082 "Notification and Subscription for SLP for > enhanced device notification. > > Requirements SLPv2 iSNS > Discovery of FCIP targets Y Y > Discovery of FCIP device IP address(es) Y Y > Discovery of FCIP port number(s) Y Y > Attribute support Y Y > Semi-timely notification of devices coming and going Y Y > Authentication Y Y* > Minimal/no configuration Y Y(?) > Provisioning Y Y > Multicast support Y N > Publicly available source Y N > Standardized and mature Y N > Lighweight implementation Y N > > Non-Requirements > Zoning N Y > State Change Notification N Y > Naming Services N Y > > * uses SLPv2 constructs > > > David Peterson > Lead Architect - Standards Development > Cisco Systems - SRBU > 6450 Wedgwood Road > Maple Grove, MN 55311 > Office: 763-398-1007 > Cell: 612-802-3299 > Email: dap@cisco.com > From owner-ips@ece.cmu.edu Wed May 23 19:18:37 2001 Received: from ece.cmu.edu ([128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id TAA17505 for ; Wed, 23 May 2001 19:18:36 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f4NKYnc20317 for ips-outgoing; Wed, 23 May 2001 16:34:49 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from msgbas1.cos.agilent.com (msgbas1x.cos.agilent.com [192.6.9.33]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f4NKYlH20312 for ; Wed, 23 May 2001 16:34:47 -0400 (EDT) Received: from msgrel1.and.agilent.com (msgrel1.and.agilent.com [130.30.33.104]) by msgbas1.cos.agilent.com (Postfix) with ESMTP id 8300267F for ; Wed, 23 May 2001 14:34:46 -0600 (MDT) Received: from rtl.rose.agilent.com (rtl.rose.agilent.com [156.140.232.231]) by msgrel1.and.agilent.com (Postfix) with ESMTP id 72F792D8 for ; Wed, 23 May 2001 16:34:45 -0400 (EDT) Received: from mail.rose.agilent.com (mailsrv@bellhop [156.140.233.51]) by rtl.rose.agilent.com (8.9.3 (PHNE_18979)/8.9.3 SMKit7.1.0) with ESMTP id NAA00443 for ; Wed, 23 May 2001 13:34:44 -0700 (PDT) Received: from agilent.com (matt5670.rose.agilent.com [156.140.235.204]) by mail.rose.agilent.com (Netscape Messaging Server 3.6) with ESMTP id AAA5B2 for ; Wed, 23 May 2001 13:34:40 -0700 Message-ID: <3B0C1F58.9DAE3510@agilent.com> Date: Wed, 23 May 2001 13:36:40 -0700 From: "Matt Wakeley" Reply-To: Matt Wakeley Organization: Agilent Technologies X-Mailer: Mozilla 4.77 [en] (Windows NT 5.0; U) X-Accept-Language: en MIME-Version: 1.0 To: ips@ece.cmu.edu Subject: Re: iSCSI: response to second login (with same ISID) References: <3B096D97.9C957568@cup.hp.com> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-ips@ece.cmu.edu Precedence: bulk Content-Transfer-Encoding: 7bit Santosh Rao wrote: > In the event where the initiator went through a non-orderly shutdown > which did not close all sessions, it would attempt to re-login on > re-boot with the same ISID. If the target rejected this re-login, per > (1) (citing an invalid ISID), the initiator would no longer be able to > maintain persistence of the session to an ISID. In this case, the initiator is at fault (it performed a non-orderly shutdown). Hence, when it logs in with the target, the target comes back and says, "bad initiator, you're already logged in with me". The initiator can then say "oh yeah, ok, let's clean up" and perform an orderly shutdown of the session. > b) Using (2) allows the hosts to hint to the target that the previous > session is now stale and thus triggers clean-up of stale session > resources. Rather than "hint", I think explicite cleaning up is better. -Matt From owner-ips@ece.cmu.edu Wed May 23 19:53:35 2001 Received: from ece.cmu.edu ([128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id TAA17987 for ; Wed, 23 May 2001 19:53:35 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f4NLcQm25257 for ips-outgoing; Wed, 23 May 2001 17:38:26 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from dogwood.cisco.com (dogwood.cisco.com [161.44.11.19]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f4NLcPH25253 for ; Wed, 23 May 2001 17:38:25 -0400 (EDT) Received: from cisco.com (mbakke@mbakke-lnx.cisco.com [161.44.68.87]) by dogwood.cisco.com (8.8.6 (PHNE_14041)/CISCO.SERVER.1.2) with ESMTP id RAA07650; Wed, 23 May 2001 17:35:54 -0400 (EDT) Message-ID: <3B0C2C88.E1A8EEA0@cisco.com> Date: Wed, 23 May 2001 16:32:56 -0500 From: Mark Bakke X-Mailer: Mozilla 4.72 [en] (X11; U; Linux 2.2.16-3.uid32 i686) X-Accept-Language: en, de MIME-Version: 1.0 To: Santosh Rao CC: ips@ece.cmu.edu Subject: Re: Asynchronous Message: Session Termination References: <0B9A57FF1D57D411B47500D0B73E5CC101E7A708@dickens.bri.hp.com> <3B0BD463.8854907B@cisco.com> <3B0BF67D.797A871A@cup.hp.com> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-ips@ece.cmu.edu Precedence: bulk Content-Transfer-Encoding: 7bit Santosh Rao wrote: > > The Async Message "target will drop all connections" should be described > to imply that the session is logged out and all outstanding I/Os are > going to be aborted at the target. This allows this feature to be used > by targets to drive session recovery. > > Mark Bakke wrote: > > > > Actually, I had originally requested this, to allow a target to say > > "I'm shutting down / failing over in X seconds; and will be back again > > in Y seconds, so don't bother connecting until then". With this > > behavior, the description should read: > > > > 4 Target indicates it will/has dropped all connections - the > > Parameter2 field indicates, in seconds, the time until all > > connections will be terminated (or zero if immediately), and > > Parameter3 the maximum time the initiator should wait before > > ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ > Did you intend to mean "the minimum time the initiator should wait"... ? Yes, I meant minimum time; thanks. > > The min time before retrying login adds value to the initiator rather > than the max time before retrying login in this case. In the case where > "target will drop all connections" was used, the max time really holds > no relevance, since the session has been logged out. It also means that the initiator won't attempt to start a TCP connection before the target actually starts rebooting, which will proceed to back off timers as packets are dropped, and depending on exactly when the target is ready, could take a very long time to give up on the connection and try a new one. > Hence, I like Mark's definition on this which provides the initiator > with time when session will be logged out and time to wait before > attempting to retry session login. > > attempting to establish a new session. > > > > If everyone agrees with this, I would rather have the above description. > > > > -- > > Mark -- Mark A. Bakke Cisco Systems mbakke@cisco.com 763.398.1054 From owner-ips@ece.cmu.edu Wed May 23 19:53:38 2001 Received: from ece.cmu.edu ([128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id TAA17998 for ; Wed, 23 May 2001 19:53:38 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f4NLrYr26357 for ips-outgoing; Wed, 23 May 2001 17:53:34 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from atlrel1.hp.com (atlrel1.hp.com [156.153.255.210]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f4NLrXH26353 for ; Wed, 23 May 2001 17:53:33 -0400 (EDT) Received: from core.rose.hp.com (core.rose.hp.com [15.43.208.100]) by atlrel1.hp.com (Postfix) with ESMTP id BF867AC2 for ; Wed, 23 May 2001 17:53:32 -0400 (EDT) Received: (from cbm@localhost) by core.rose.hp.com (8.8.6 (PHNE_14041)/8.8.6 SMKit7.02) id OAA28571 for ips@ece.cmu.edu; Wed, 23 May 2001 14:54:39 -0700 (PDT) Message-Id: <200105232154.OAA28571@core.rose.hp.com> Subject: Re: TCP Framing (considered helpful?) To: ips@ece.cmu.edu Date: Wed, 23 May 2001 14:54:39 PDT Reply-To: cbm@rose.hp.com From: "Mallikarjun C." X-Mailer: Elm [revision: 212.4] Sender: owner-ips@ece.cmu.edu Precedence: bulk John, Two comments. I may have misunderstood what you're saying since your initial comments imply framing=warp/RDMA/messageboundary, and towards the end, you seem to imply framing=markers. - It's true that implementing iSCSI-level markers does not require modifying TCP/IP stacks. But using the markers (I mean to use it to effectively steer) requires TCP/IP changes! It is a separate matter that it is offloaded onto the NIC. - You seem to implicitly suggest that WARP (when it becomes available) must go into host software stacks to be useful, and hence cannot be done in the right timeframe for iSCSI. Since one could envision offloading WARP onto the NIC as well, I assume you're hinting a) either at interoperability issues between software and hardware iSCSI implementations relying on WARP, b) or at interoperability issues between hardware and hardware implementations. iSCSI currently mandates a "no sync and steering" mode which ensures interoperability in either case. Perhaps you were really concerned about performance in case (b) then? -- Mallikarjun Mallikarjun Chadalapaka Networked Storage Architecture Network Storage Solutions Organization MS 5668 Hewlett-Packard, Roseville. cbm@rose.hp.com >Replies in text below (between [Huff] and [/Huff] ). > >. >. >. >John L. Hufferd >Senior Technical Staff Member (STSM) >IBM/SSG San Jose Ca >(408) 256-0403, Tie: 276-0403, eFax: (408) 904-4688 >Internet address: hufferd@us.ibm.com > > >Stephen Bailey @ece.cmu.edu on 05/21/2001 06:50:41 >AM > >Sent by: owner-ips@ece.cmu.edu > > >To: ips@ece.cmu.edu >cc: >Subject: Re: TCP Framing (considered helpful?) > > > >John, > >> I think we must depend on Markers to insure that everything can operate >at >> top speed, and at the lowest cost. > >A key question is whether markers actually ensure that everything >operates at `top speed, and at the lowest cost'. > >Matt thinks so. I (and, presumably those who wrote the framing >document) think not. > >[Huff] I do not think you can say that. I also support framing (Warp), as >a much more elegant solution, but I find it inappropriate to depend on it >actually happening, and being made available in the various OSs in the time >frame we need. I do believe, that over time it will be made available, and >is a better approach for all TCP/IP applications that can use it. [/Huff] > >My issue is not even with `lowest cost'. I don't believe markers will >allow you to run at top speed. Specifically: > 1) I doubt the feasibility of implementing the control required for > an eddy buffer (where you store data you can't place) at 10G. > Admittedly, the validity of this claim can't really be assessed > without actually working the implementation, so for 99% of the > list participants (myself included) this is a `yes it is, no it > isn't' point. > > [Huff] I believe this has had much more work done on it then you > think. I have personally stepped through the proposals from > several vendors that are working on this option for their HW HBAs. > Usually, because of the iSCSI PDU headers, the data/commands > can be placed directly into the SCSI Host buffers, almost every > time. Only when the PDU headers arrive slightly out of order > (do to normal routing) are the packets unable to be placed > directly into the Host buffer. And that requires some, but only > a small amount, of buffering space. > It is the packet drops that occur on PDU headers, and resultant > error retries, that cause the need for large amounts of "on > HBA/chip" buffering. > So by using Markers, these HW iSCSI HBAs can limit the amount of > buffering on the chip/HBAs. [/Huff] > > 2) an eddy buffer solution requires some substantial speed-up in > both the NIC data path, and MOST IMPORTANTLY: the host bus. In > order to unload the eddy buffer while still handling incoming > traffic at line rate, clearly the host bus bandwidth must be > > line rate. > > [Huff] This is not an effect of an eddy buffer solution, it is a > fact that every TCP/IP NIC has to deal with. Especially at the > new Speeds. Our current PCI buss will not support 10 Gigabit, further > PCI-X will not support it either, even PCI-DDR does not fully support > the full data rate. So it needs to rely on the TCP/IP window > management. The only other thing you can do is drop the packets. > this clearly makes the problem worse. [/Huff] > > >I know of at least one general purpose framed solution operating at >10G which has been available for >3 years (SGI's GSN/ST/XIO NIC). I'm >sure there are others. > >I can't imagine there's any argument that a framed solution would be >voted `most likely to run fast and be cheap'. Every storage network >and cluster interconnect has been designed that way since antiquity. > >The key tradeoff involves the OS vendors, and I'm wondering why we're >speaking for them. The question IS, how much more work is it to >introduce TCP framing over and above what is required to insert iSCSI >into their network framework. My experience from writing NIC and >storage drivers for many commercial UNIX-family OSes is: > 1) it's an easy and well defined process to insert a new SCSI > transport driver into the SCSI stack. > 2) it's hard and poorly defined process to insert ANYTHING into the > network stack. >[Huff] I think you are making my point. This is the problem with SW >Stacks. That is why I believe that it will take a very long time for >the various vendors to include such changes into their "bet you business" >TCP/IP SW Stacks. The point that Matt and I have been trying to make >is that most OS vendors are NOT creating the iSCSI HW HBAs (NICs). >These iSCSI HW HBAs (NICs) have the TCP/IP completely on the HBA, and >they have added the iSCSI processing also so that they can steer the >packets directly into the approprate SCSI Host buffers. Adding either >Markers or Framing into the iSCSI HW HBAs is not a big problem. It is >only a problem of getting Framing (timely) into Host TCP/IP Stacks. >[/Huff] > >Networking has historically been a user-mode activity. Architected >services are only provided to user mode programs. Kernel clients have >been few and far between and so are handled on a case-by-case basis. >For example NFS. Every OS has hacks to make NFS run fast, but they >are not stable interfaces for general purpose use. > >Even Solaris' SysV-derived STREAMS stack, which is intended precisely >to provide flexible, crisp interfaces for kernel network clients, does >not document the relevant (IP stack) intermodule interfaces. > >I know that there are more and more kernel network clients, but they >are coming either on fluid platforms (e.g. linux), in which case the >argument of `it'll take too long to get OS support' doesn't apply, or >they are vendor-supplied, in which case a performance iSCSI solution >in ANY form may take a while, and the choice of framing or markers >isn't going to make a difference. > >[Huff] I think you are saying something I agree with and something I >do not agree with. That is, that software changes to TCP/IP in the >various "Bet you Business" OSs, will take some time. However, it is >not true that new iSCSI device drivers will take very long. Two types >are being created today. By Cisco, IBM, Intel, etc. These types are >iSCSI DD that make calls to normal TCP/IP stacks, and the DD that >are being written by the iSCSI HW HBA vendors. These do not require >the OS vendor to do anything special. This is happening NOW, >(Check with CISCO, Intel, and IBM (me?)). The last thing we want >is to depend on a TCP/IP change to get in the >way of our momentum. [/Huff] > >I can't say squat about the architecture of Winsock, but the fact that >there is a Microsoft author of the framing proposal who seems very >serious about supporting framing and RDMA as quickly as possible >suggests that framing support should be available on Windows very >soon. > >[Huff] My following statements are not meant as a negative of Microsoft. >However, they and all producers of Key complicated new Software do >not quickly bring these to the general market in a way that is as >pleasing to HW vendors as HW vendors would like. > >I believe that Microsoft's heart is in the right place on this issue, >and that they will do the right thing with framing, over time. >But it is not clear in what release that will be shipped, nor what support >pack it will be included. Also it is not clear how the support >will be handled for current Win2k, WinNT etc. > >This is why I think we should have Framing a Must implement >and an Optional to use. It is the easiest thing for SW to >create, and brings the needed cost reduction to iSCSI HW and >it is completely under our (iSCSI protocol) control. >[/Huff] > > > >Steph > > > > From owner-ips@ece.cmu.edu Wed May 23 20:03:26 2001 Received: from ece.cmu.edu ([128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id UAA18146 for ; Wed, 23 May 2001 20:03:26 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f4NMF1J27768 for ips-outgoing; Wed, 23 May 2001 18:15:01 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from mxic2.us.dg.com (mxic2.us.dg.com [128.221.31.40]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f4NMF0H27762 for ; Wed, 23 May 2001 18:15:00 -0400 (EDT) Received: by mxic2.us.dg.com with Internet Mail Service (5.5.2653.19) id ; Wed, 23 May 2001 18:14:23 -0400 Message-ID: <0F31E5C394DAD311B60C00E029101A07080155E3@corpmx9.isus.emc.com> From: Black_David@emc.com To: ips@ece.cmu.edu Subject: iSNS, SLP, and FCIP Date: Wed, 23 May 2001 18:14:52 -0400 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2653.19) Content-Type: text/plain; charset="iso-8859-1" Sender: owner-ips@ece.cmu.edu Precedence: bulk The interim meeting notes indicate that we need to make progress on this. Let me start by injecting a couple of things that either weren't discussed at or may not have been clear from the interim meeting: (1) SLP scopes are allowed to overlap in a fashion similar to Fibre Channel zones -- Service, User, and Directory Agents can each be in multiple scopes. (2) RFC 2610 specifies DHCP options for configuring SLP Directory Agent locations and SLP scopes for both User and Service Agents. This would allow DHCP to be used for centralized administration of an SLP infrastructure, and works even when DHCP is not used for IP address assignment. One caveat about the analogy in (1) above - Fibre Channel zones are above the level of FCIP and completely transparent to it. SLP and iSNS for FCIP would be used only for tunnel setup - telling each FCIP device about peers to whom it should establish TCP connections for FCIP. Also, both of these have some impact on iSCSI use of iSNS - I'll pick up that topic in a separate message. Going back over the meeting notes on this subject, I see the following topics: (1) Change notification (RFC 3082) (2) Scope vs. discovery domains (3) Periodic re-registration timing values (4) SLP only supports strings (5) Centralized administration Taking these up in order ... -- (1) Change notification There are two cases here, initial discovery and loss of connectivity. For initial discovery, FCIP may not need change notification *because* it's a peer to peer protocol (as opposed to client- server), so it doesn't matter who sets up the initial TCP connection. For the situation in which some FCIP devices are up and running and a new one comes up, SLP-based discovery at the new device is sufficient to set up everything. The FCIP draft doesn't appear to contain any words about how to deal with concurrent setup - i.e., both peers initiate setup at the same time, resulting in two parallel TCP connections that aren't associated with a single FC link. Some text on this topic should be added to the FCIP draft. For loss of connectivity, either FCIP or something running over it (e.g., FSPF) is going to heartbeat each connection, and so no external mechanism seems to be necessary. The one situation that doesn't seem to be covered here is a partial loss of connectivity (FCIP device loses touch with one of its peers, but not all of them). These scenarios tend to be messy, and I'm not sure how iSNS compares to SLP in dealing with them (loss of connectivity to an iSNS server seems to have more severe impacts than loss of connectivity to an SLP DA). -- (2) Scope vs. discovery domains For FCIP, these appear to be essentially the same concept since SLP scopes can overlap in the same fashion as iSNS discovery domains. iSNS is somewhat better at information hiding - with SLP, each agent knows what scope(s) it's in, whereas with iSNS, that information is private to the nameserver (similar to Fibre Channel). This may make a big difference if static configuration and more than a handful of scopes are involved, but appears to be less of an issue if DHCP is used. -- (3) Periodic re-registration SLP registrations with a Directory Agent carry a lifetime that defines both how long the DA can cache the registration and the period within which the registration must be refreshed. This results in additional traffic to do the refreshes. Lifetimes are measured in seconds, and RFC 2608 recommends a default of 5 minutes before an idle TCP connection is closed. This suggests that a few minutes for a re-registration period may be reasonable, and doesn't sound like a cause for a lot of traffic for moderate-sized configurations. iSNS usage should generate less traffic, regardless. -- (4) SLP only supports strings As does iSCSI. OTOH, turning an X.509 cert into a string is not exactly pleasant, but OTOH FCIP doesn't seem to have a requirement for distributing non-string parameters. This may change as FCIP security is more tightly specified. -- (5) Centralized administration RFC 2610 provides a way for DHCP to be used to centrally administer SLP (and do so in a fashion where the only multicast-like things are DHCP's link-level broadcasts). On the assumption that FCIP connectivity among N devices is considerably simpler in structure than typical FC zoning among N ports, this seems to scale up to reasonable-sized configurations (several dozen FCIP devices should not be a problem, I'm not sure about more than 100). This is all for further discussion and comment, although it does not appear to present a strong case for use of iSNS with FCIP. Thanks, --David --------------------------------------------------- David L. Black, Senior Technologist EMC Corporation, 42 South St., Hopkinton, MA 01748 +1 (508) 435-1000 x75140 FAX: +1 (508) 497-8500 black_david@emc.com Mobile: +1 (978) 394-7754 --------------------------------------------------- From owner-ips@ece.cmu.edu Wed May 23 20:33:07 2001 Received: from ece.cmu.edu ([128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id UAA18635 for ; Wed, 23 May 2001 20:33:07 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f4NMrZH00618 for ips-outgoing; Wed, 23 May 2001 18:53:35 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from server1.NishanSystems.COM (smtp.nishansystems.com [216.217.36.162]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f4NMrYH00614 for ; Wed, 23 May 2001 18:53:34 -0400 (EDT) Received: by smtp.nishansystems.com with Internet Mail Service (5.5.2653.19) id ; Wed, 23 May 2001 15:53:29 -0700 Message-ID: From: Charles Monia To: "Ips (E-mail)" Subject: Common Encapsulation: Stale Frame Detection in an IP fabric Date: Wed, 23 May 2001 15:53:28 -0700 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2653.19) Content-Type: text/plain; charset="iso-8859-1" Sender: owner-ips@ece.cmu.edu Precedence: bulk Hi: This note is a strawman proposal for detecting stale FC frames using the features present in the common encapsulation specification. The spec includes provisions for time tagging a frame in order to detect frames that have exceeded the lifetime guarantees expected of a Fibre Channel fabric (R_A_TOV). A fibre channel fabric must guarantee that the maximum time in flight for any frame will not exceed the limit specified by the R_A_TOV parameter. In FC fabrics, R_A_TOV is defined by the fabric and supplied to the N_PORT in the fabric login 'accept' response. The default value is 10 seconds. Failure to meet this guarantee may result in stale frames associated with defunct transactions. These frames, if received outside the R_A_TOV window, could cause failures and data corruption. In a closed FC fabric, the R_A_TOV limit is guaranteed by switch element design and control of the fabric topology. There is no enforcement mechanism. In contrast, such control may not be possible in an IP fabric. In most cases, the IP network will, almost certainly, consist of heterogeneous components and the user cannot be assumed to have full control over the infrastructure and its topology. For that reason some explicit way to enforce the flight time guarantee must be provided. The FC frame encapsulation format provides the means for this enforcement at the edges of the IP network by allocating space for a time stamp which is applied when the frame is injected into the IP network and may be checked by the receiving node. The following is a proposal for managing time base synchronization at the TCP end nodes. The protocol for synchronizing an end node time base is SNTP. In order to insure that all nodes are time-aligned, they should obtain the address of a reference SNTP server via a hard-wired address or by querying an appropriate repository via SLP or some other protocol. If multiple SNTP server addresses are provided, the servers must be synchronized. Alternatively, a multicast group address may be used in support of operation in Anycast mode. Implementation of Anycast mode is as specified in RFC 2030, including the precautions defined in that document. Multicast mode should not be used. An SNTP server may use any one of the time reference sources listed in RFC 2030. The resolution of the time reference MUST be 125 milliseconds or better. If support for stale frame detection by a node is optional, a node that does not enforce the flight time limit shall set the time stamp field in the encapsulation header of an outgoing frame to 0,0 and shall ignore the contents of the timestamp for incoming frames. A node that supports stale frame detection shall implement the time stamp with a precision of 0.125 seconds or better. With regard to the time base, the node is in either the synchronized or unsychronized state. When in the unsynhronized state, the node shall a) Set the time stamp field to 0,0 for all outgoing frames b) Ignore the time stamp field for all incoming frames. When in the synchronized state, the node shall: a) Set the time stamp field for each outgoing frame in accordance with the node's internal time base b) Check the time stamp field of each incoming frame. c) If the incoming frame has a time stamp of 0,0, the receiving node shall not test the frame to determine if it is stale. d) If the incoming frame has a non-zero time stamp, the receiving node shall compute the time in flight and compare it against the limit specified for the IP fabric. e) If the result in step (d) exceeds the enforcement time limit, the frame shall be discarded. Otherwise, the frame shall be accepted. A node enters the synchronized state upon receiving a successful response to an SNTP query. A node shall enter the unsynchronized state: a) Upon power up and before the succesful completion of an SNTP query b) When an unsuccesful SNTP query occurs. Setting the enforcement time limit. In general, 2* E_D_TOV < Stale Frame time limit < R_A_TOV Where E_D_TOV is the FC time limit between expected events such as the arrival of two successive frames in a sequence. A rule of thumb for the stale frame time limit is .5 * R_A_TOV. Charles Charles Monia Senior Technology Consultant Nishan Systems email: cmonia@nishansystems.com voice: (408) 519-3986 fax: (408) 435-8385 From owner-ips@ece.cmu.edu Wed May 23 20:33:16 2001 Received: from ece.cmu.edu ([128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id UAA18646 for ; Wed, 23 May 2001 20:33:16 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f4NMS7828713 for ips-outgoing; Wed, 23 May 2001 18:28:07 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from sandmail.sandburst.com (sandburst-gw.bstn-gw02.ma.us.intelilink.net [216.57.129.34]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f4NMS6H28709 for ; Wed, 23 May 2001 18:28:06 -0400 (EDT) Received: from cs.uchicago.edu (dynamite-38.sandburst.com [172.16.5.38]) by sandmail.sandburst.com (Postfix) with ESMTP id 5D3D694006 for ; Wed, 23 May 2001 18:28:05 -0400 (EDT) To: ips@ece.cmu.edu Subject: Re: iSCSI: response to second login (with same ISID) In-Reply-To: Message from "Matt Wakeley" of "Wed, 23 May 2001 13:36:40 PDT." <3B0C1F58.9DAE3510@agilent.com> References: <3B096D97.9C957568@cup.hp.com> <3B0C1F58.9DAE3510@agilent.com> Date: Wed, 23 May 2001 18:26:04 -0400 From: Stephen Bailey Message-Id: <20010523222805.5D3D694006@sandmail.sandburst.com> Sender: owner-ips@ece.cmu.edu Precedence: bulk Matt, > In this case, the initiator is at fault (it performed a non-orderly > shutdown). Hence, when it logs in with the target, the target comes back and > says, "bad initiator, you're already logged in with me". The initiator can > then say "oh yeah, ok, let's clean up" and perform an orderly shutdown of the > session. What if the initiator lost its mind (hence the non-orderly shutdown)? It doesn't seem to make sense to require a new instance of an initiator to clean up for an old one. What's WRONG with the FCP-style behavior (PLOGI establishes a new session, PDISC/ADISC reconnects)? I don't see that sanity checking the initiator is really necessary in this case. I know sanity checking is NICE, but getting the implicit logout seems more important. If Login with known SID results in an implicit logout, the target will terminate the existing connections for the session, and the initiator will certainly hear about this! Steph From owner-ips@ece.cmu.edu Wed May 23 20:35:16 2001 Received: from ece.cmu.edu ([128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id UAA18674 for ; Wed, 23 May 2001 20:35:15 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f4NMkLE00104 for ips-outgoing; Wed, 23 May 2001 18:46:21 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from zmamail05.zma.compaq.com (zmamail05.zma.compaq.com [161.114.64.105]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f4NMkJH00099 for ; Wed, 23 May 2001 18:46:19 -0400 (EDT) Received: by zmamail05.zma.compaq.com (Postfix, from userid 12345) id E0B031A48; Wed, 23 May 2001 18:46:13 -0400 (EDT) Received: from exchou-gh02.cca.cpqcorp.net (exchou-gh02.cca.cpqcorp.net [16.110.248.202]) by zmamail05.zma.compaq.com (Postfix) with ESMTP id 9916F18A9; Wed, 23 May 2001 18:46:13 -0400 (EDT) Received: by exchou-gh02.cca.cpqcorp.net with Internet Mail Service (5.5.2652.78) id ; Wed, 23 May 2001 17:46:13 -0500 Message-ID: <31891B757C09184BBFEC5275F85D5595104D64@cceexc18.americas.cpqcorp.net> From: "Martin, Nick" To: "'Stephen Bailey'" , ips@ece.cmu.edu Subject: RE: iSCSI: response to second login (with same ISID) Date: Wed, 23 May 2001 17:46:04 -0500 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2652.78) Content-Type: text/plain Sender: owner-ips@ece.cmu.edu Precedence: bulk Stephen, Santosh, and All, I am a bit weak in my understanding of authentication. (Is the initiator being authenticated, or the driver on the initiator, or the application on the initiator, or the user of the application on the initiator, etc.) My impression is that authentication means that something within the initiator can somehow prove its identity to the target (and vise versa). The initiator is not proving its intentions. I know from personal experience how easy it is for an authorized user to unintentionally start a second copy of a backup program, while the tape drive is still in use. This should be a harmless mistake. What kind of traffic cop would prevent this request from making it to the iSCSI target and terminating the session already in progress? (Rhetorical question, please do not describe for me such a mechanism ;) I do not dispute the need to be able to clean up stale sessions. I would prefer that this be handled by timeout and/or explicit request rather than presuming that *every* "authenticated" request to establish a session with the same ISID is an *intentional* attempt to logout any other active session with that ISID. When I attempt a duplicate login for the same ISID, I would like to be able to get an error response code for "session already logged in". Then I would have a choice of whether I will quit, try again later, try again immediately with a "login unconditionally" flag, or try again with a different ISID. In some situations, such as kernel drivers, "login unconditionally" could be used on every attempt to save time. The ISID's for kernel drivers may need to be reserved (and pre-configured) so that a kernel reboot after an ungraceful shutdown will automatically recover all the kernel's sessions. However if two copies of an application attempt to open sessions to the same iSCSI URL from the same host at the same time, the second should not disrupt the first. I'm not sure what will force them to have different ISID's, or how they would even try to each pick a unique one. I would like to somehow leave the [programmer of the] iSCSI initiator a choice. Whatever mechanism is chosen to globally assign and/or reserve ISID's within an initiator is not specified by iSCSI and so will be a local convention. Software not adhering to the local convention (perhaps unaware of the correct local convention) could at least operate non-disruptively by probing for an available ISID, but only if automatic logout of pre-existing sessions is never done, or is selectable rather than always unconditional. So I guess what I am suggesting is a bit in the Login PDU to select behavior 1) or behavior 2). "1) REJECT the login with some error code (that the iSCSI initiator can understand or infer to mean "ISID in use") and leave the pre-existing session alone 2) CLOSE the pre-existing session (and abort all it's tasks) and then process the new login fresh." I now believe both are needed. Thanks, Nick > -----Original Message----- > From: Stephen Bailey [mailto:steph@cs.uchicago.edu] > Sent: Tuesday, May 22, 2001 9:09 PM > To: ips@ece.cmu.edu > Subject: Re: iSCSI: response to second login (with same ISID) > > > Nick, > > > If I read Steph Bailey's comments on this right (from the > ERT work), the > > target would have to authenticate the 2nd login on the same > ISID [as it > > does with any other login] and only take action when > authentication is > > successful. > > What he said. > > Steph > From owner-ips@ece.cmu.edu Wed May 23 20:38:55 2001 Received: from ece.cmu.edu ([128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id UAA18763 for ; Wed, 23 May 2001 20:38:55 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f4NMhOM29874 for ips-outgoing; Wed, 23 May 2001 18:43:24 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from dogwood.cisco.com (dogwood.cisco.com [161.44.11.19]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f4NMhMH29864 for ; Wed, 23 May 2001 18:43:22 -0400 (EDT) Received: from cisco.com (mbakke@mbakke-lnx.cisco.com [161.44.68.87]) by dogwood.cisco.com (8.8.6 (PHNE_14041)/CISCO.SERVER.1.2) with ESMTP id SAA12765; Wed, 23 May 2001 18:38:27 -0400 (EDT) Message-ID: <3B0C3B31.64701CFE@cisco.com> Date: Wed, 23 May 2001 17:35:29 -0500 From: Mark Bakke X-Mailer: Mozilla 4.72 [en] (X11; U; Linux 2.2.16-3.uid32 i686) X-Accept-Language: en, de MIME-Version: 1.0 To: "BURBRIDGE,MATTHEW (HP-UnitedKingdom,ex2)" CC: ips@ece.cmu.edu Subject: Re: Asynchronous Message: Session Termination References: <0B9A57FF1D57D411B47500D0B73E5CC101E7A70E@dickens.bri.hp.com> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-ips@ece.cmu.edu Precedence: bulk Content-Transfer-Encoding: 7bit Matthew- How about we have Parameter3 be in addition to Parameter2? That would solve the same problem. Along with comments from Santosh, here's another try: 4 Target indicates it will/has dropped all connections - the Parameter2 field indicates, in seconds, the time until all connections will be terminated (or zero if immediately), and Parameter3 the minimum additional time the initiator should wait before attempting to establish a new session. If Parameter3 is zero, the initiator may attempt to reconnect as soon as the time specified in Parameter2 has passed. So if an initiator receives this with P2 == 3 and P3 == 10, it would know it has 3 seconds to try clean up nicely, and can re-connect 13 seconds from when this message was received. There is currently no way to use P3 to tell the initiator not to try connecting ever again, but this behavior can be provided by simply setting P3 to zero, and either denying the connection or returning a redirect when the initiator attempts to re-connect. -- Mark "BURBRIDGE,MATTHEW (HP-UnitedKingdom,ex2)" wrote: > > Mark, > > The last description is good and covers all situation. Presumably, if > Parameter3 is zero then the initiator is able to establish a new session > immediately. Can you modify the description to specify that Parameter3 must > be greater than or equal to Parameter2 (it makes no sense to have a value > less than Parameter2) and that zero is a valid value for Parameter3. > > Matthew > > -----Original Message----- > From: Mark Bakke [mailto:mbakke@cisco.com] > Sent: Wednesday, May 23, 2001 4:17 PM > To: BURBRIDGE,MATTHEW (HP-UnitedKingdom,ex2) > Cc: ips@ece.cmu.edu > Subject: Re: Asynchronous Message: Session Termination > > Matthew- > > The intent of these was not to connect and re-establish the same > session after a set amount of time; it was to inform the initiator > not to try to establish a new session until the time passes, which > is I think what you want. Perhaps changing the description of > #4 from: > > 4 Target indicates it will/has dropped all connections - the > Parameter2 field indicates, in seconds, the minimum time to > reconnect and Parameter3 the maximum time to reconnect. > > to: > > 4 Target indicates it will/has dropped all connections - the > Parameter2 field indicates, in seconds, the minimum time to > wait to establish a new session and Parameter3 the maximum > time to wait to establish a new session. > > Actually, I had originally requested this, to allow a target to say > "I'm shutting down / failing over in X seconds; and will be back again > in Y seconds, so don't bother connecting until then". With this > behavior, the description should read: > > 4 Target indicates it will/has dropped all connections - the > Parameter2 field indicates, in seconds, the time until all > connections will be terminated (or zero if immediately), and > Parameter3 the maximum time the initiator should wait before > attempting to establish a new session. > > If everyone agrees with this, I would rather have the above description. > > -- > Mark > > "BURBRIDGE,MATTHEW (HP-UnitedKingdom,ex2)" wrote: > > > > Hi Julian, > > > > If a target only supports Session Recovery, then when an error is detected > > it will terminate the session. The current Async Message (iSCSI Event > > "Target indicates that his will/has dropped all connections") does not > > necessary inform the initiator that the session has terminated and the > > initiator is quite within rights to establish a new connection and attempt > > With-in Session recovery. > > > > Can you change the spec to reflect that a value of zero in both the > > Parameter2 and Parameter3 would set the maximum time to reconnect to > "Never" > > thereby informing the initiator that the session has closed? > > > > Thanks > > > > Matthew Burbridge > > NIS-Bristol > > Hewlett Packard > > Telnet: 312 7010 > > E-mail: matthewb@bri.hp.com > > -- > Mark A. Bakke > Cisco Systems > mbakke@cisco.com > 763.398.1054 -- Mark A. Bakke Cisco Systems mbakke@cisco.com 763.398.1054 From owner-ips@ece.cmu.edu Wed May 23 20:40:54 2001 Received: from ece.cmu.edu ([128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id UAA18798 for ; Wed, 23 May 2001 20:40:54 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f4NMiBB29941 for ips-outgoing; Wed, 23 May 2001 18:44:11 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from dogwood.cisco.com (dogwood.cisco.com [161.44.11.19]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f4NMi9H29934 for ; Wed, 23 May 2001 18:44:09 -0400 (EDT) Received: from cisco.com (mbakke@mbakke-lnx.cisco.com [161.44.68.87]) by dogwood.cisco.com (8.8.6 (PHNE_14041)/CISCO.SERVER.1.2) with ESMTP id SAA20572; Wed, 23 May 2001 18:44:02 -0400 (EDT) Message-ID: <3B0C3C81.4BA484F8@cisco.com> Date: Wed, 23 May 2001 17:41:05 -0500 From: Mark Bakke X-Mailer: Mozilla 4.72 [en] (X11; U; Linux 2.2.16-3.uid32 i686) X-Accept-Language: en, de MIME-Version: 1.0 To: Matt Wakeley CC: ips@ece.cmu.edu Subject: Re: Asynchronous Message: Session Termination References: <0B9A57FF1D57D411B47500D0B73E5CC101E7A708@dickens.bri.hp.com> <3B0BD463.8854907B@cisco.com> <3B0BEBFD.9347890E@agilent.com> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-ips@ece.cmu.edu Precedence: bulk Content-Transfer-Encoding: 7bit Matt- I think that you can get the functionality you want by sending the other Async message type (3) on each of the connections that you want the initiator to clean up and drop. This seems sufficient. 3 Target indicates it will/has dropped the connection - the Parameter1 field will indicate on what CID while the Parameter2 field indicates, in seconds, the minimum time to reconnect and Parameter3 the maximum time to reconnect. I still think that the intent of (4) was to drop the whole session; at any rate, this is a necessary feature to allow a target to reboot or fail over reasonably cleanly. I think if we word (3) and (4) right, we will be able to support both types of intent. -- Mark Matt Wakeley wrote: > > Mark Bakke wrote: > > > > Matthew- > > > > The intent of these was not to connect and re-establish the same > > session after a set amount of time; it was to inform the initiator > > not to try to establish a new session until the time passes, which > > is I think what you want. > > Mark, > > I don't think that was the intent of this feature. Just because a target > drops all it's connections, does not mean that the session is closed. In fact, > the "maximum" time seems to indicate that if a connection is not tried within > that amount of time, then the session will be aborted, or some other action > taken. > > Obviously, clarification on the intent is required, and then clarification in > the text. > > -Matt -- Mark A. Bakke Cisco Systems mbakke@cisco.com 763.398.1054 From owner-ips@ece.cmu.edu Wed May 23 22:00:13 2001 Received: from ece.cmu.edu ([128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id WAA20607 for ; Wed, 23 May 2001 22:00:13 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f4O07Kf05387 for ips-outgoing; Wed, 23 May 2001 20:07:20 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from maho3msx2.isus.emc.com (maho3msx2.isus.emc.com [128.221.11.32]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f4O07IH05383 for ; Wed, 23 May 2001 20:07:18 -0400 (EDT) Received: by maho3msx2.isus.emc.com with Internet Mail Service (5.5.2653.19) id ; Wed, 23 May 2001 20:07:13 -0400 Received: by mxic2.us.dg.com with Internet Mail Service (5.5.2653.19) id ; Wed, 23 May 2001 20:05:49 -0400 Message-ID: <0F31E5C394DAD311B60C00E029101A07080155E5@corpmx9.isus.emc.com> From: Black_David@emc.com To: ips@ece.cmu.ediu Subject: SLP, iSNS, and iSCSI Date: Wed, 23 May 2001 20:06:18 -0400 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2653.19) Content-Type: text/plain; charset="iso-8859-1" Sender: owner-ips@ece.cmu.edu Precedence: bulk I previously noted a couple of thing about SLP that appear not to have been part of prior discussions: [A] SLP scopes are allowed to overlap in a fashion similar to Fibre Channel zones -- Service, User, and Directory Agents can each be in multiple scopes. [B] RFC 2610 specifies DHCP options for configuring SLP Directory Agent locations and SLP scopes for both User and Service Agents. This would allow DHCP to be used for centralized administration of an SLP infrastructure, and works even when DHCP is not used for IP address assignment. Going back to Josh Tseng's analysis of SLP and iSNS for iSCSI, Josh covered the following topics: (1) State Change Notification (2) Multicast (3) Zoning and Discovery Domains (4) Heartbeat (5) Non-string attributes (6) Security This is a note about the impact that [A] and [B] have on these 6 items, plus some related comments based on the SLP notification mechanism in RFC 3082 -- (1) State Change Notification [A] and [B] don't seem to affect this area. RFC 3082 provides some level of support (but see next item). From a process standpoint, RFC 3082 is experimental, and hence can't be required by a standards-track document. Unlike FCIP, iSCSI is client-server and hence has a use/need for some way of informing clients (Initiators) that a new server (Target) has appeared. -- (2) Multicast SLP requires both service and directory agents to listen for multicast requests, but can be configured in a fashion where they're never used. The RFC 3082 notification mechanism *requires* multicast, which is likely to be an issue in larger networks and some VPN situations. -- (3) Zoning and Discovery Domains Both [A] and [B] matter here, and the appropriate comparison is probably SLP + DHCP vs. iSNS. For SLP + DHCP, it looks like SLP scope is usable in the same fashion as an iSNS Zone/Discovery Domain, modulo issues with State Change Notification [(1) above]. -- (4) Heartbeat Neither [A], [B] nor RFC 3082 affect this area, but it doesn't seem to be a major issue -- the concern is how long an SLP Directory Agent may cache information for and hence how often an SLP Service Agent has to refresh its registration(s). A few minutes might be a good answer here since that's the default time for SLP to keep idle TCP connections open. -- (5) Non-string based attributes [A], [B], and RFC 3082 appear not to have any effect here. Whether we need to distribute things like X.509 certs through the SNS appears to be an open issue. -- (6) Security The fact that multiple scopes can be used and that the DEFAULT SLP scope can be removed from SLP directory agents (RFC 2608, Section 12.4) appears to remove the argument that the SLP Directory Agent is a central point of trust. DHCP doesn't need to replace it as a central point, as DHCP is readily partitionable along network topology boundaries. An important note about this area is that iSCSI access control departs from the model(s) used in Fibre Channel in an important aspect. In Fibre Channel, end systems trust the Fabric to handle certain aspects of access control - both soft and hard zoning are primarily implemented in switches (and soft zoning also trusts HBAs to not do port scans). For iSCSI, Targets are going to find themselves more involved in access control and authentication of Initiators. There will be cases such as strict VLAN configuration along with no layer 3 gateway out of the VLAN reproduce Fibre Channel-like configurations, but iSCSI will need to operate in more general networks. This is all for discussion. An initial take on this is that State Change Notification (1) is still an area in which SLP appears to fall short of iSNS, especially if RFC 3082 notification is ruled out of bounds due to its dependence on multicast. OTOH, DHCP appears to be a viable route to central administration of SLP. Comments?, --David --------------------------------------------------- David L. Black, Senior Technologist EMC Corporation, 42 South St., Hopkinton, MA 01748 +1 (508) 435-1000 x75140 FAX: +1 (508) 497-8500 black_david@emc.com Mobile: +1 (978) 394-7754 --------------------------------------------------- From owner-ips@ece.cmu.edu Wed May 23 22:00:51 2001 Received: from ece.cmu.edu ([128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id WAA20626 for ; Wed, 23 May 2001 22:00:51 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f4NNoJI04271 for ips-outgoing; Wed, 23 May 2001 19:50:19 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from palrel2.hp.com (palrel2.hp.com [156.153.255.234]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f4NNoEH04264 for ; Wed, 23 May 2001 19:50:18 -0400 (EDT) Received: from hpcuhe.cup.hp.com (hpcuhe.cup.hp.com [15.0.80.203]) by palrel2.hp.com (Postfix) with ESMTP id 94FE721E4 for ; Wed, 23 May 2001 16:50:13 -0700 (PDT) Received: from cup.hp.com (santoshr@hpindhhm.cup.hp.com [15.8.80.197]) by hpcuhe.cup.hp.com (8.9.3 (PHNE_18979)/8.9.3 SMKit7.02) with ESMTP id QAA12498 for ; Wed, 23 May 2001 16:50:08 -0700 (PDT) Message-ID: <3B0C4CD3.5FEC3618@cup.hp.com> Date: Wed, 23 May 2001 16:50:43 -0700 From: Santosh Rao Organization: Hewlett Packard, Cupertino. X-Mailer: Mozilla 4.7 [en] (X11; U; HP-UX B.11.00 9000/778) X-Accept-Language: en MIME-Version: 1.0 To: ips@ece.cmu.edu Subject: Re: iSCSI: response to second login (with same ISID) References: <3B096D97.9C957568@cup.hp.com> <3B0C1F58.9DAE3510@agilent.com> Content-Type: multipart/mixed; boundary="------------FFE03E160345A12882846426" Sender: owner-ips@ece.cmu.edu Precedence: bulk This is a multi-part message in MIME format. --------------FFE03E160345A12882846426 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Matt, The issue spans beyond just an orderly vs. non-orderly shutdown. Some user application may be performing raw I/O access to an iSCSI device when the system may be shutdown. The logout operation itself may fail (usually a don't care for the drivers.). What is wrong with FC semantics of a login causing an implicit logout. It allows for clean up of old stale sessions. Regards, Santosh Matt Wakeley wrote: > > Santosh Rao wrote: > > > In the event where the initiator went through a non-orderly shutdown > > which did not close all sessions, it would attempt to re-login on > > re-boot with the same ISID. If the target rejected this re-login, per > > (1) (citing an invalid ISID), the initiator would no longer be able to > > maintain persistence of the session to an ISID. > > In this case, the initiator is at fault (it performed a non-orderly > shutdown). Hence, when it logs in with the target, the target comes back and > says, "bad initiator, you're already logged in with me". The initiator can > then say "oh yeah, ok, let's clean up" and perform an orderly shutdown of the > session. > > > b) Using (2) allows the hosts to hint to the target that the previous > > session is now stale and thus triggers clean-up of stale session > > resources. > > Rather than "hint", I think explicite cleaning up is better. > > -Matt --------------FFE03E160345A12882846426 Content-Type: text/x-vcard; charset=us-ascii; name="santoshr.vcf" Content-Description: Card for Santosh Rao Content-Disposition: attachment; filename="santoshr.vcf" Content-Transfer-Encoding: 7bit begin:vcard n:Rao;Santosh tel;work:408-447-3751 x-mozilla-html:FALSE org:Hewlett Packard, Cupertino.;SISL adr:;;19420, Homestead Road, M\S 43LN, ;Cupertino.;CA.;95014.;USA. version:2.1 email;internet:santoshr@cup.hp.com title:Software Design Engineer x-mozilla-cpt:;21088 fn:Santosh Rao end:vcard --------------FFE03E160345A12882846426-- From owner-ips@ece.cmu.edu Wed May 23 23:25:14 2001 Received: from ece.cmu.edu ([128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id XAA22301 for ; Wed, 23 May 2001 23:25:14 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f4O224712765 for ips-outgoing; Wed, 23 May 2001 22:02:04 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from palrel2.hp.com (palrel2.hp.com [156.153.255.234]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f4O222H12756 for ; Wed, 23 May 2001 22:02:02 -0400 (EDT) Received: from hpcuhe.cup.hp.com (hpcuhe.cup.hp.com [15.0.80.203]) by palrel2.hp.com (Postfix) with ESMTP id 1F5842B11 for ; Wed, 23 May 2001 19:02:02 -0700 (PDT) Received: from cup.hp.com (santoshr@hpindhhm.cup.hp.com [15.8.80.197]) by hpcuhe.cup.hp.com (8.9.3 (PHNE_18979)/8.9.3 SMKit7.02) with ESMTP id TAA19828 for ; Wed, 23 May 2001 19:01:46 -0700 (PDT) Message-ID: <3B0C6B9C.9F062AAE@cup.hp.com> Date: Wed, 23 May 2001 19:02:05 -0700 From: Santosh Rao Organization: Hewlett Packard, Cupertino. X-Mailer: Mozilla 4.7 [en] (X11; U; HP-UX B.11.00 9000/778) X-Accept-Language: en MIME-Version: 1.0 To: ips@ece.cmu.edu Subject: Re: iSCSI: response to second login (with same ISID) References: <31891B757C09184BBFEC5275F85D5595104D64@cceexc18.americas.cpqcorp.net> Content-Type: multipart/mixed; boundary="------------02258CEA5084F75C76808FB5" Sender: owner-ips@ece.cmu.edu Precedence: bulk This is a multi-part message in MIME format. --------------02258CEA5084F75C76808FB5 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit > However if two copies of an application attempt to open sessions to the same > iSCSI URL from the same host at the same time, the second should not disrupt > the first. I'm not sure what will force them to have different ISID's, or > how they would even try to each pick a unique one. The above scenario should not typically result in the HBA driver sending out 2 logins on the wire, unless its open logic is broken. The open paths are typically single-threaded and the first of the 2 applications entering the driver open code would cause a login to go out on the wire with the 2nd one just bumping an open_count or ref_count of some kind within the HBA driver. The cases when an initiator would send out 2 logins on the wire would be ones like : a) Bug in the initiator driver open_count logic or open locking logic. b) Initiator's state out of sync with target's. (initiator thinks it is logged out with the target, but target thinks the initiator is still logged in.) c) Prior logout operation failed at the target. d) initiator failed to logout all sessions prior to reboot. In cases (b), (c) & (d), the initiator does want to re-login, and logout any stale session if active. In case (a) the driver is broken and not much forward progress will occur with such a version. The login semantics should be optimized for : a) Minimal time for initiator login operation. (Should not require some iterations of login to get a session going, while the initiator is in a "guess an ISID" game with the target. Also, should not require a login -> login reject -> logout of old session -> login of new session). b) Allow a scheme for clean up of stale session resources. Regards, Santosh --------------02258CEA5084F75C76808FB5 Content-Type: text/x-vcard; charset=us-ascii; name="santoshr.vcf" Content-Description: Card for Santosh Rao Content-Disposition: attachment; filename="santoshr.vcf" Content-Transfer-Encoding: 7bit begin:vcard n:Rao;Santosh tel;work:408-447-3751 x-mozilla-html:FALSE org:Hewlett Packard, Cupertino.;SISL adr:;;19420, Homestead Road, M\S 43LN, ;Cupertino.;CA.;95014.;USA. version:2.1 email;internet:santoshr@cup.hp.com title:Software Design Engineer x-mozilla-cpt:;21088 fn:Santosh Rao end:vcard --------------02258CEA5084F75C76808FB5-- From owner-ips@ece.cmu.edu Wed May 23 23:32:56 2001 Received: from ece.cmu.edu ([128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id XAA22705 for ; Wed, 23 May 2001 23:32:55 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f4O1Pfq10394 for ips-outgoing; Wed, 23 May 2001 21:25:41 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from palrel2.hp.com (palrel2.hp.com [156.153.255.234]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f4O1PdH10390 for ; Wed, 23 May 2001 21:25:39 -0400 (EDT) Received: from hpcuhe.cup.hp.com (hpcuhe.cup.hp.com [15.0.80.203]) by palrel2.hp.com (Postfix) with ESMTP id 01AF5D27 for ; Wed, 23 May 2001 18:25:39 -0700 (PDT) Received: from cup.hp.com (santoshr@hpindhhm.cup.hp.com [15.8.80.197]) by hpcuhe.cup.hp.com (8.9.3 (PHNE_18979)/8.9.3 SMKit7.02) with ESMTP id SAA18075 for ; Wed, 23 May 2001 18:25:34 -0700 (PDT) Message-ID: <3B0C6331.BAD09BB9@cup.hp.com> Date: Wed, 23 May 2001 18:26:09 -0700 From: Santosh Rao Organization: Hewlett Packard, Cupertino. X-Mailer: Mozilla 4.7 [en] (X11; U; HP-UX B.11.00 9000/778) X-Accept-Language: en MIME-Version: 1.0 To: ips@ece.cmu.edu Subject: Re: iSCSI: response to second login (with same ISID) References: <3B096D97.9C957568@cup.hp.com> <3B0C1F58.9DAE3510@agilent.com> Content-Type: multipart/mixed; boundary="------------A9CC02A0219A6D44D7B2FA92" Sender: owner-ips@ece.cmu.edu Precedence: bulk This is a multi-part message in MIME format. --------------A9CC02A0219A6D44D7B2FA92 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Matt Wakeley wrote: > > Santosh Rao wrote: > > > In the event where the initiator went through a non-orderly shutdown > > which did not close all sessions, it would attempt to re-login on > > re-boot with the same ISID. If the target rejected this re-login, per > > (1) (citing an invalid ISID), the initiator would no longer be able to > > maintain persistence of the session to an ISID. > > In this case, the initiator is at fault (it performed a non-orderly > shutdown). Hence, when it logs in with the target, the target comes back and > says, "bad initiator, you're already logged in with me". The initiator can > then say "oh yeah, ok, let's clean up" and perform an orderly shutdown of the > session. One problem with the above approach. Once the initiator has rebooted, it has no way of cleaning the old session, since a session logout has to be sent within the same set of TCP connections that the session was operating on. There is no provision to logout a session from a newly established set of TCP connection[s] for a new session upon the host's bootup. --------------A9CC02A0219A6D44D7B2FA92 Content-Type: text/x-vcard; charset=us-ascii; name="santoshr.vcf" Content-Description: Card for Santosh Rao Content-Disposition: attachment; filename="santoshr.vcf" Content-Transfer-Encoding: 7bit begin:vcard n:Rao;Santosh tel;work:408-447-3751 x-mozilla-html:FALSE org:Hewlett Packard, Cupertino.;SISL adr:;;19420, Homestead Road, M\S 43LN, ;Cupertino.;CA.;95014.;USA. version:2.1 email;internet:santoshr@cup.hp.com title:Software Design Engineer x-mozilla-cpt:;21088 fn:Santosh Rao end:vcard --------------A9CC02A0219A6D44D7B2FA92-- From owner-ips@ece.cmu.edu Wed May 23 23:53:59 2001 Received: from ece.cmu.edu ([128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id XAA22953 for ; Wed, 23 May 2001 23:53:58 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f4O1E0h09668 for ips-outgoing; Wed, 23 May 2001 21:14:00 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from palrel2.hp.com (palrel2.hp.com [156.153.255.234]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f4O1DxH09664 for ; Wed, 23 May 2001 21:13:59 -0400 (EDT) Received: from hpcuhe.cup.hp.com (hpcuhe.cup.hp.com [15.0.80.203]) by palrel2.hp.com (Postfix) with ESMTP id 3F70922C5 for ; Wed, 23 May 2001 18:13:58 -0700 (PDT) Received: from cup.hp.com (santoshr@hpindhhm.cup.hp.com [15.8.80.197]) by hpcuhe.cup.hp.com (8.9.3 (PHNE_18979)/8.9.3 SMKit7.02) with ESMTP id SAA17527 for ; Wed, 23 May 2001 18:13:53 -0700 (PDT) Message-ID: <3B0C6074.7AE23A07@cup.hp.com> Date: Wed, 23 May 2001 18:14:28 -0700 From: Santosh Rao Organization: Hewlett Packard, Cupertino. X-Mailer: Mozilla 4.7 [en] (X11; U; HP-UX B.11.00 9000/778) X-Accept-Language: en MIME-Version: 1.0 To: ips@ece.cmu.edu Subject: Re: Asynchronous Message: Session Termination References: <0B9A57FF1D57D411B47500D0B73E5CC101E7A70E@dickens.bri.hp.com> <3B0C3B31.64701CFE@cisco.com> Content-Type: multipart/mixed; boundary="------------8CAC9D4FC1A82A5CD593D766" Sender: owner-ips@ece.cmu.edu Precedence: bulk This is a multi-part message in MIME format. --------------8CAC9D4FC1A82A5CD593D766 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit In addition to the changes discussed in this thread so far, I'd like to suggest the following further changes to the async message PDU (Section 2.18.1 , page 72) : 1) Regarding the text : "target indicates it will/has dropped the connection" & "target indicates it will/has dropped all connections" suggest re-phrasing it to : "target indicates it will drop the connection" & "target indicates it will drop all connections" 2) State explicitly that : a) Targets must first send the async message and then close the connection after the specified time to ensure the message is pushed to the initiator. ("has dropped" means the async PDU cannot be transmitted, thereby rendering it useless.) b) The target will abort all outstanding commands on this session as a result of this operation. 3) Add a parameter to "target indicates it will drop the connection" that specifies the seconds after which the target will drop the connection (0 indicates immediately). This is consistent with the changes being proposed for the addition of a similar parameter to the "target indicates it will drop all connections". This allows initiators to attempt to switch further I/Os to alternate connections within the session accordingly. The description should also state explicitly that all outstanding I/Os on that connection are aborted by the target after the connection is terminated, unless a prior "logout for connection recovery" was received to recover the connection being dropped, in which case the connection allegiance of the outstanding I/Os is switched to the new connection specified in the Logout PDU. 4) Remove the "Target requests logout" event in the Async Message PDU. This can be achieved by using (3) in its above form, without requiring the ping pong caused by : T -> I : async PDU I -> T : Logout PDU T -> I : Logout Response PDU T -> I : closes TCP connection. Regards, Santosh Mark Bakke wrote: > > Matthew- > > How about we have Parameter3 be in addition to Parameter2? That > would solve the same problem. > > Along with comments from Santosh, here's another try: > > 4 Target indicates it will/has dropped all connections - the > Parameter2 field indicates, in seconds, the time until all > connections will be terminated (or zero if immediately), and > Parameter3 the minimum additional time the initiator should > wait before attempting to establish a new session. If > Parameter3 is zero, the initiator may attempt to reconnect as > soon as the time specified in Parameter2 has passed. > > So if an initiator receives this with P2 == 3 and P3 == 10, it > would know it has 3 seconds to try clean up nicely, and can > re-connect 13 seconds from when this message was received. > > There is currently no way to use P3 to tell the initiator not to > try connecting ever again, but this behavior can be provided by > simply setting P3 to zero, and either denying the connection or > returning a redirect when the initiator attempts to re-connect. > > -- > Mark > > "BURBRIDGE,MATTHEW (HP-UnitedKingdom,ex2)" wrote: > > > > Mark, > > > > The last description is good and covers all situation. Presumably, if > > Parameter3 is zero then the initiator is able to establish a new session > > immediately. Can you modify the description to specify that Parameter3 must > > be greater than or equal to Parameter2 (it makes no sense to have a value > > less than Parameter2) and that zero is a valid value for Parameter3. > > > > Matthew > > > > -----Original Message----- > > From: Mark Bakke [mailto:mbakke@cisco.com] > > Sent: Wednesday, May 23, 2001 4:17 PM > > To: BURBRIDGE,MATTHEW (HP-UnitedKingdom,ex2) > > Cc: ips@ece.cmu.edu > > Subject: Re: Asynchronous Message: Session Termination > > > > Matthew- > > > > The intent of these was not to connect and re-establish the same > > session after a set amount of time; it was to inform the initiator > > not to try to establish a new session until the time passes, which > > is I think what you want. Perhaps changing the description of > > #4 from: > > > > 4 Target indicates it will/has dropped all connections - the > > Parameter2 field indicates, in seconds, the minimum time to > > reconnect and Parameter3 the maximum time to reconnect. > > > > to: > > > > 4 Target indicates it will/has dropped all connections - the > > Parameter2 field indicates, in seconds, the minimum time to > > wait to establish a new session and Parameter3 the maximum > > time to wait to establish a new session. > > > > Actually, I had originally requested this, to allow a target to say > > "I'm shutting down / failing over in X seconds; and will be back again > > in Y seconds, so don't bother connecting until then". With this > > behavior, the description should read: > > > > 4 Target indicates it will/has dropped all connections - the > > Parameter2 field indicates, in seconds, the time until all > > connections will be terminated (or zero if immediately), and > > Parameter3 the maximum time the initiator should wait before > > attempting to establish a new session. > > > > If everyone agrees with this, I would rather have the above description. > > > > -- > > Mark > > > > "BURBRIDGE,MATTHEW (HP-UnitedKingdom,ex2)" wrote: > > > > > > Hi Julian, > > > > > > If a target only supports Session Recovery, then when an error is detected > > > it will terminate the session. The current Async Message (iSCSI Event > > > "Target indicates that his will/has dropped all connections") does not > > > necessary inform the initiator that the session has terminated and the > > > initiator is quite within rights to establish a new connection and attempt > > > With-in Session recovery. > > > > > > Can you change the spec to reflect that a value of zero in both the > > > Parameter2 and Parameter3 would set the maximum time to reconnect to > > "Never" > > > thereby informing the initiator that the session has closed? > > > > > > Thanks > > > > > > Matthew Burbridge > > > NIS-Bristol > > > Hewlett Packard > > > Telnet: 312 7010 > > > E-mail: matthewb@bri.hp.com > > > > -- > > Mark A. Bakke > > Cisco Systems > > mbakke@cisco.com > > 763.398.1054 > > -- > Mark A. Bakke > Cisco Systems > mbakke@cisco.com > 763.398.1054 --------------8CAC9D4FC1A82A5CD593D766 Content-Type: text/x-vcard; charset=us-ascii; name="santoshr.vcf" Content-Description: Card for Santosh Rao Content-Disposition: attachment; filename="santoshr.vcf" Content-Transfer-Encoding: 7bit begin:vcard n:Rao;Santosh tel;work:408-447-3751 x-mozilla-html:FALSE org:Hewlett Packard, Cupertino.;SISL adr:;;19420, Homestead Road, M\S 43LN, ;Cupertino.;CA.;95014.;USA. version:2.1 email;internet:santoshr@cup.hp.com title:Software Design Engineer x-mozilla-cpt:;21088 fn:Santosh Rao end:vcard --------------8CAC9D4FC1A82A5CD593D766-- From owner-ips@ece.cmu.edu Thu May 24 00:44:56 2001 Received: from ece.cmu.edu ([128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id AAA23735 for ; Thu, 24 May 2001 00:44:56 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f4O2Udb14589 for ips-outgoing; Wed, 23 May 2001 22:30:39 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from server1.NishanSystems.COM (smtp.nishansystems.com [216.217.36.162]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f4O2UcH14580 for ; Wed, 23 May 2001 22:30:38 -0400 (EDT) Received: by smtp.nishansystems.com with Internet Mail Service (5.5.2653.19) id ; Wed, 23 May 2001 19:30:32 -0700 Message-ID: From: Joshua Tseng To: ips@ece.cmu.edu Subject: RE: SLP, iSNS, and iSCSI Date: Wed, 23 May 2001 19:30:31 -0700 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2653.19) Content-Type: text/plain; charset="iso-8859-1" Sender: owner-ips@ece.cmu.edu Precedence: bulk David, > [B] RFC 2610 specifies DHCP options for configuring > SLP Directory Agent locations and SLP scopes for both > User and Service Agents. This would allow DHCP to be > used for centralized administration of an SLP > infrastructure, and works even when DHCP is not used > for IP address assignment. > I'm not a DHCP expert, but in my limited understanding of the protocol, the DHCP server is not intelligent about who is asking for IP addresses and options. There is nothing to identify or authenticate a DHCP client making a request (other than the MAC), since it doesn't even have an IP address, not to mention an FQDN or any other identity. If this is true, I am puzzled as to how you can use DHCP to manage SLP scopes. The DHCP server can't say "device A gets scopes X, Y, and Z", since it doesn't even know who device A is. All it can do is hand out the same set of scopes to everyone requesting option 79. And that's all. So how can you configure a complex set of overlapping scopes using a DHCP server??? Lastly, I want to make sure everyone understands that support for FCIP is included in the iSNS out of courtesy to the FCIP community, and after close consultation with several key individuals from that community. Since we are not implementing FCIP, I personally am neutral on whether it should be used for FCIP, although I do believe it would be of great value. Josh From owner-ips@ece.cmu.edu Thu May 24 04:43:13 2001 Received: from ece.cmu.edu ([128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id EAA09582 for ; Thu, 24 May 2001 04:43:13 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f4O6cb129673 for ips-outgoing; Thu, 24 May 2001 02:38:37 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from zmamail05.zma.compaq.com (zmamail05.zma.compaq.com [161.114.64.105]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f4O6caH29668 for ; Thu, 24 May 2001 02:38:36 -0400 (EDT) Received: by zmamail05.zma.compaq.com (Postfix, from userid 12345) id E3B9E1CA0; Thu, 24 May 2001 02:38:30 -0400 (EDT) Received: from exchou-gh02.cca.cpqcorp.net (exchou-gh02.cca.cpqcorp.net [16.110.248.202]) by zmamail05.zma.compaq.com (Postfix) with ESMTP id 99BD91C3F; Thu, 24 May 2001 02:38:30 -0400 (EDT) Received: by exchou-gh02.cca.cpqcorp.net with Internet Mail Service (5.5.2652.78) id ; Thu, 24 May 2001 01:38:30 -0500 Message-ID: <31891B757C09184BBFEC5275F85D5595104D66@cceexc18.americas.cpqcorp.net> From: "Martin, Nick" To: "'Santosh Rao'" , ips@ece.cmu.edu Subject: RE: iSCSI: response to second login (with same ISID) Date: Thu, 24 May 2001 01:38:17 -0500 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2652.78) Content-Type: text/plain Sender: owner-ips@ece.cmu.edu Precedence: bulk [The message I am replying to is below this one. A similar one was posted while I was composing this reply.] Santosh, If the iSCSI HBA is exposing SCSI targets to the OS, then the OS's SCSI target driver (or class driver if I have my Microsoft terminology correct) can support "exclusive access" semantics which can protect the current user of a tape device from other users on the same system attempting to use the same device through the same driver, or can allow a disk device (and the session for it) to be shared by multiple users when that is appropriate. If the target device supports some form of SCSI reserve/release, then this would protect the device from access via a second session from anywhere. In the target, iSCSI could also refuse a second concurrent session login to the same tape target protecting initiators which do not reserve/release. (The trick is: does the target think this is a new session using the same ISID (reject), or an attempt to clean-up and re-establish an existing session (reset, accept).) Think about an iSCSI enabled version of tar or cpio (simple applications used for tape backups). These could run from user space over a TCP connection on a standard NIC using an OS that has no awareness of iSCSI. If a second copy of the program is started, how is it to know that something is already using the same tape device? How is it to know the ISID used for that other copy? How is it to know whether ANY particular ISID is really currently in use, or what it is being used for? How can we prevent the second copy from unknowingly stealing the device from the first by simply using the same ISID? The second user thinks everything is fine until the first user restarts his command that just failed. In this model there is no HBA driver or target driver. There is nothing that knows about both sessions being attempted except the target. You may think this is a bit far fetched, but I almost have such a program. I expect other folks will be clever enough to think of other ways to use this "user mode access" capability of TCP/IP to use remote iSCSI devices without kernel co-operation. Can we enable them to peacefully co-exist with each other? Can they peacefully co-exist with kernel mode drivers? There may be other scenarios such as administrators dynamically adding and removing iSCSI devices to an OS while a system is running rather than at boot time (think 24x7 operations, what choice to they have?). If the ISID's are going to have to be manually configured, then there is a high likelihood of human error causing disruptions. If we can enable non-disruptive automatic ISID probing to find an available ISID during device configuration or initialization, this seems safer. Once we pick an ISID, we should still remember it across re-boot, to avoid repeated probing, and to assist manual clean-up after ungraceful shutdown. We also need the "logout this old session" capability both for recovery after an initiator crash or error which failed to logout, and for when the initiator realizes it has no remaining working TCP connections to an existing session and wants to be sure to free target resources and start fresh. So, I think there are valid arguments for both behaviors 1 and 2, and we need to enable the initiator to select the behavior appropriate for each application. Stated another way, we need to allow for the case when the initiator is sure that this ISID is unique and it takes full responsibility for all consequences, and for the case where the initiator can not be sure this ISID is unique and wants there to be no side effects for others if it turns out not to be unique. Thanks, Nick -----Original Message----- From: Santosh Rao [mailto:santoshr@cup.hp.com] Sent: Wednesday, May 23, 2001 8:36 PM To: Martin, Nick Cc: santoshr@cup.hp.com Subject: Re: iSCSI: response to second login (with same ISID) "Martin, Nick" wrote: > > I know from personal experience how easy it is for an authorized user to > unintentionally start a second copy of a backup program, while the tape > drive is still in use. This should be a harmless mistake. What kind of > traffic cop would prevent this request from making it to the iSCSI target > and terminating the session already in progress? (Rhetorical question, > please do not describe for me such a mechanism ;) Nick, I know you don't want an answer to your rhetorical question ;-}, but, I could'nt resist one quick query here. In the scenario you describe, is it not the case that HBA drivers would only send out a Login on the first open of that target, and on subsequent opens would only bump an open_count or a ref_count of some kind rather than send a login on the wire on every open ? If that is correct, then, the scenario you describe would still cause disruption even with the modified semantics, only, when the WRITE is issued by the 2nd execution of the backup application (?). Regards, Santosh From owner-ips@ece.cmu.edu Thu May 24 13:09:27 2001 Received: from ece.cmu.edu ([128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id NAA17365 for ; Thu, 24 May 2001 13:09:26 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f4OFDCl11575 for ips-outgoing; Thu, 24 May 2001 11:13:12 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from e31.bld.us.ibm.com (e31.co.us.ibm.com [32.97.110.129]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f4OFDAH11570 for ; Thu, 24 May 2001 11:13:10 -0400 (EDT) Received: from westrelay02.boulder.ibm.com (westrelay02.boulder.ibm.com [9.99.140.23]) by e31.bld.us.ibm.com (8.9.3/8.9.3) with ESMTP id LAA56928 for ; Thu, 24 May 2001 11:05:32 -0400 Received: from f3n42e (d03nm042h.boulder.ibm.com [9.99.140.42]) by westrelay02.boulder.ibm.com (8.11.1/NCO v4.96.1.0) with ESMTP id f4OFCCF253474 for ; Thu, 24 May 2001 09:12:12 -0600 Importance: Normal Subject: RE: iSCSI: response to second login (with same ISID) To: ips@ece.cmu.edu From: "Jim Hafner" Date: Thu, 24 May 2001 08:12:10 -0700 Message-ID: X-MIMETrack: Serialize by Router on D03NM042/03/M/IBM(Release 5.0.6 |December 14, 2000) at 05/24/2001 08:12:11 AM MIME-Version: 1.0 Content-type: text/plain; charset=us-ascii Sender: owner-ips@ece.cmu.edu Precedence: bulk Steph, Santoch, Nick, and All, As the one who posted this question in the first place, I think it's time for me to chime back in. I see valid arguments for both behaviors and Nick supporting a feature to enable that. There are two scenarios we're dealing with. (1) Initiator (for whatever reason) unknowingly reuses an active ISID. Nick's example of the user application using a generic NIC is a good one (I programmed a Java Initiator, and that's about as high up in the application stack as you can get!). (2) Error scenario of some sort, where the initiator and target don' agree on state for the "live ISID". The former begs for "login reject, ISID in use". The latter begs for error recovery. The way I see it, however, is that the second case should be dealt with in the context of error recovery (independent of the ISID question). If we don't have any way to cleanup a target's session without having a live connection in that session to do so, then we're missing a major piece of the iSCSI puzzle. We have to have a mechanism for the target to clean up session resources that are lost to the host. Isn't there some timeout the target can rely on here? Isn't there a mechanism for the initiator to say "I lost my brains on that session, so you can clean up"? (Isn't this last what Nick's Login PDU bit would say?) So, I'd recommend we use "login reject, ISID in use" for the reused ISID. We solve the "error case" as an error case (who's doing these scenarios?). In the error case, the "login reject" is one form of error detection (there are others). Santosh argues that login shouldn't take too long, but error recovery can take as long as it should to clean things up, even if it has to go through "login->reject->cleanup->login" during boot. One final note. The comparison with FCP behavior is only partially valid. The problem is that FC login occurs at an extremely low level of the wire protocol where the NICs have control of things in firmware (typically), or at least down in the lowest level of the kernel. In other words, that process is owned by one self-contained system component. For iSCSI, this login, as has been noted, can occur anywhere and everywhere (simultaneously) in the application stack. So, I wouldn't necessarily recommend that we model FC behavior here. Jim Hafner From owner-ips@ece.cmu.edu Thu May 24 13:12:02 2001 Received: from ece.cmu.edu ([128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id NAA17422 for ; Thu, 24 May 2001 13:12:02 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f4OFHGR11892 for ips-outgoing; Thu, 24 May 2001 11:17:16 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from erie.mcdata.com (mcgate.mcdata.com [144.49.1.5]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f4OFHEH11886 for ; Thu, 24 May 2001 11:17:14 -0400 (EDT) Received: by erie.mcdata.com with Internet Mail Service (5.5.2653.19) id ; Thu, 24 May 2001 09:17:06 -0600 Message-ID: From: Anil Rijhsinghani To: "'ENDL_TX@computer.org'" , IPS Reflector , "'dap@cisco.com'" Subject: RE: FCIP - Discovery should be Manual/SLP Date: Thu, 24 May 2001 09:17:06 -0600 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2653.19) Content-Type: text/plain Sender: owner-ips@ece.cmu.edu Precedence: bulk Agree with the use of SLPv2 as the only mandatory discovery mechanism for FCIP, with naming/zoning not being requirements for FCIP. Note that static configuration (as opposed to dynamic discovery method) will be specified in the FCIP MIB, so I would replace the wording below "static config is outside the scope of this specification" to "static config is specified in the FCIP MIB document". Regards, Anil -----Original Message----- From: Ralph Weber [mailto:ralphoweber@compuserve.com] Sent: Wednesday, May 16, 2001 3:25 PM To: IPS Reflector Subject: FCIP - Discovery should be Manual/SLP I support Dave's proposed text for FCIP Clause 4.2 item 3, with the exception that the sentence regarding iSNS needs to be removed. Looking at the features list, I cannot see any iSNS unique features that are meaningful to FCIP. Thanks. Ralph... - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - Subject: Proposal to use SLPv2 for FCIP device discovery Date: Mon, 30 Apr 2001 17:07:08 -0500 From: "Dave Peterson" To: "Ips@Ece. Cmu. Edu" As requested here is the proposal I presented at Mondays' FCIP meeting: FCIP Draft Proposal For Clause 4.2, item 3. Each FCIP device may be statically or dynamically configured with a list of IP addresses and port numbers corresponding to all the participating FCIP devices. If dynamic discovery of participating FCIP devices is supported this function shall be performed using the Service Location Protocol (SLPv2). For additional FCIP device management capability, the iSNS Internet Storage Naming Service may be implemented. It is outside the scope of this specification to describe any static configuration method for participating FCIP device discovery. Refer to clause XXX for a detailed description of dynamic discovery of participating FCIP device using SLPv2. Notes: 1. DHCP: a. Allows an entity to discovery information about itself, not discover information about all other entities. b. Uses a broadcast mechanism that may not work via routers without additional configuration. But, most current router implementations will support the forwarding of DHCP requests across routed subnets. c. May be used to find SLP Directory Agents and Scope Lists allowing for a more scalable solution. 2. DNS Services are too limiting. This is the reason why SLP was started. 3. LDAP is simply a database interface. SLP and iSNS may use LDAP as a back-end store. 4. SLP FCIP Device Type Specifics a. IP address(es) b. Port numbers(s) c. Scope d. Attributes i. Discovery domain (i.e. a group name or zone name, don't want to use zone name in this context though) ii. need to start listing these (if we can think of any more) e. Lifetime Work In progress: 1. Putting together a model for FCIP device discovery using SLPv2. 2. Implementing a "default/suggested" SLPv2 template. 3. Reviewing RFC 3082 "Notification and Subscription for SLP for enhanced device notification. Requirements SLPv2 iSNS Discovery of FCIP targets Y Y Discovery of FCIP device IP address(es) Y Y Discovery of FCIP port number(s) Y Y Attribute support Y Y Semi-timely notification of devices coming and going Y Y Authentication Y Y* Minimal/no configuration Y Y(?) Provisioning Y Y Multicast support Y N Publicly available source Y N Standardized and mature Y N Lighweight implementation Y N Non-Requirements Zoning N Y State Change Notification N Y Naming Services N Y * uses SLPv2 constructs David Peterson Lead Architect - Standards Development Cisco Systems - SRBU 6450 Wedgwood Road Maple Grove, MN 55311 Office: 763-398-1007 Cell: 612-802-3299 Email: dap@cisco.com From owner-ips@ece.cmu.edu Thu May 24 13:20:45 2001 Received: from ece.cmu.edu ([128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id NAA17603 for ; Thu, 24 May 2001 13:20:44 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f4OF2qK10884 for ips-outgoing; Thu, 24 May 2001 11:02:52 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from bramg1.net.external.hp.com (bramg1.net.external.hp.com [192.6.126.73]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f4OF2iH10875 for ; Thu, 24 May 2001 11:02:51 -0400 (EDT) Received: from quasit.br.itc.hp.com (quasit.br.itc.hp.com [15.145.8.135]) by bramg1.net.external.hp.com (Postfix) with ESMTP id 8BAA1186 for ; Thu, 24 May 2001 17:02:43 +0200 (METDST) Received: from loddon.br.itc.hp.com (loddon.br.itc.hp.com [15.145.8.166]) by quasit.br.itc.hp.com (8.9.3 (PHNE_18979)/8.9.3 SMKit6.0.6 OpenMail) with SMTP id QAA17047 for ; Thu, 24 May 2001 16:02:42 +0100 (BST) Received: from 15.145.8.166 by loddon.br.itc.hp.com (InterScan E-Mail VirusWall NT); Thu, 24 May 2001 16:02:42 +0100 (GMT Daylight Time) Received: by loddon.br.itc.hp.com with Internet Mail Service (5.5.2653.19) id ; Thu, 24 May 2001 16:02:42 +0100 Message-ID: <0B9A57FF1D57D411B47500D0B73E5CC101E7A713@dickens.bri.hp.com> From: "BURBRIDGE,MATTHEW (HP-UnitedKingdom,ex2)" To: ips@ece.cmu.edu Subject: RE: Login Rejection: Target does not support multiple addresses p er Session Date: Thu, 24 May 2001 16:02:18 +0100 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2653.19) Content-Type: text/plain; charset="ISO-8859-1" Sender: owner-ips@ece.cmu.edu Precedence: bulk Nick, Good point, A second port on the target. If there is a second port on the initiator and it is attempting to set up a second connection on the initiator then presumably it can handle session spanning. Matthew, Is your question below presuming a second port is being used on the initiator, or a second port is being used on the target, or both? Thanks, Nick Martin > -----Original Message----- > From: BURBRIDGE,MATTHEW (HP-UnitedKingdom,ex2) > [mailto:matthew_burbridge@hp.com] > Sent: Wednesday, May 23, 2001 5:13 AM > To: 'ips@ece.cmu.edu' > Subject: Login Rejection: Target does not support multiple > addresses per > S ession > > > Hi Julian > > If an initiator attempts to set up a new connection for an > existing session > (i.e TSID is non-zero, and Restart bit is 0) but via another > physical port > then if the target does not support session spanning there is > currently no > Login Response Status code that covers the scenario. > > Please can you add an additional error code which informs the > initiator that > the login has been rejected due to a target not being able to support > session spanning (e.g. "Target does not support multiple addresses per > Session"). Attempts by the initiator to connect to the > session via the > original port, or for Within-session recovery, or to create > an completely > new session, would not be rejected using this error code. > > Thanks > > Matthew Burbridge > NIS-Bristol > Hewlett Packard > Telnet: 312 7010 > E-mail: matthewb@bri.hp.com > From owner-ips@ece.cmu.edu Thu May 24 14:12:59 2001 Received: from ece.cmu.edu ([128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id OAA18407 for ; Thu, 24 May 2001 14:12:59 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f4OFtem15111 for ips-outgoing; Thu, 24 May 2001 11:55:40 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from sj-msg-core-2.cisco.com (sj-msg-core-2.cisco.com [171.69.24.11]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f4OFtcH15092 for ; Thu, 24 May 2001 11:55:38 -0400 (EDT) Received: from sponge.cisco.com (sponge.cisco.com [64.101.128.87]) by sj-msg-core-2.cisco.com (8.11.3/8.9.1) with ESMTP id f4OFtXU05169; Thu, 24 May 2001 08:55:33 -0700 (PDT) Received: from dap02w2k (dhcp-161-44-68-130.cisco.com [161.44.68.130]) by sponge.cisco.com (Mirapoint) with SMTP id AAZ19942; Thu, 24 May 2001 10:55:28 -0500 (CDT) From: "Dave Peterson" To: "Anil Rijhsinghani" , , "IPS Reflector" Subject: RE: FCIP - Discovery should be Manual/SLP Date: Thu, 24 May 2001 10:52:43 -0500 Message-ID: MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 (Normal) X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook IMO, Build 9.0.2416 (9.0.2911.0) In-Reply-To: Importance: Normal X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4133.2400 Sender: owner-ips@ece.cmu.edu Precedence: bulk Content-Transfer-Encoding: 7bit I am not opposed to performing static configuration using the FCIP MIB and can provide appropriate wording but I think we should also allow the capability of static configuration using other methods (e.g., command line interface). Dave > -----Original Message----- > From: owner-ips@ece.cmu.edu [mailto:owner-ips@ece.cmu.edu]On Behalf Of > Anil Rijhsinghani > Sent: Thursday, May 24, 2001 10:17 AM > To: 'ENDL_TX@computer.org'; IPS Reflector; 'dap@cisco.com' > Subject: RE: FCIP - Discovery should be Manual/SLP > > > Agree with the use of SLPv2 as the only mandatory discovery mechanism > for FCIP, with naming/zoning not being requirements for FCIP. > > Note that static configuration (as opposed to dynamic discovery method) > will be specified in the FCIP MIB, so I would replace the wording below > "static config is outside the scope of this specification" to > "static config is specified in the FCIP MIB document". > > Regards, > Anil > > -----Original Message----- > From: Ralph Weber [mailto:ralphoweber@compuserve.com] > Sent: Wednesday, May 16, 2001 3:25 PM > To: IPS Reflector > Subject: FCIP - Discovery should be Manual/SLP > > > I support Dave's proposed text for FCIP Clause 4.2 item 3, with the > exception that the sentence regarding iSNS needs to be removed. > Looking at the features list, I cannot see any iSNS unique features > that are meaningful to FCIP. > > Thanks. > > Ralph... > > - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - > > Subject: Proposal to use SLPv2 for FCIP device discovery > Date: Mon, 30 Apr 2001 17:07:08 -0500 > From: "Dave Peterson" > To: "Ips@Ece. Cmu. Edu" > > > As requested here is the proposal I presented at Mondays' FCIP > meeting: > > FCIP Draft Proposal > For Clause 4.2, item 3. > > Each FCIP device may be statically or dynamically configured with > a list of IP addresses and port numbers corresponding to all the > participating FCIP devices. If dynamic discovery of participating > FCIP devices is supported this function shall be performed using the > Service Location Protocol (SLPv2). For additional FCIP device > management capability, the iSNS Internet Storage Naming Service may > be implemented. It is outside the scope of this specification to > describe any static configuration method for participating FCIP device > discovery. Refer to clause XXX for a detailed description of dynamic > discovery of participating FCIP device using SLPv2. > > Notes: > 1. DHCP: > a. Allows an entity to discovery information about itself, > not discover information about all other entities. > b. Uses a broadcast mechanism that may not work via routers > without additional configuration. But, most current router > implementations will support the forwarding of DHCP requests > across routed subnets. > c. May be used to find SLP Directory Agents and Scope Lists > allowing for a more scalable solution. > 2. DNS Services are too limiting. This is the reason why SLP was > started. > 3. LDAP is simply a database interface. SLP and iSNS may use LDAP > as a back-end store. > 4. SLP FCIP Device Type Specifics > a. IP address(es) > b. Port numbers(s) > c. Scope > d. Attributes > i. Discovery domain (i.e. a group name or zone name, don't want > to use zone name in this context though) > ii. need to start listing these (if we can think of any more) > e. Lifetime > > Work In progress: > 1. Putting together a model for FCIP device discovery using SLPv2. > 2. Implementing a "default/suggested" SLPv2 template. > 3. Reviewing RFC 3082 "Notification and Subscription for SLP for > enhanced device notification. > > Requirements SLPv2 iSNS > Discovery of FCIP targets Y Y > Discovery of FCIP device IP address(es) Y Y > Discovery of FCIP port number(s) Y Y > Attribute support Y Y > Semi-timely notification of devices coming and going Y Y > Authentication Y Y* > Minimal/no configuration Y Y(?) > Provisioning Y Y > Multicast support Y N > Publicly available source Y N > Standardized and mature Y N > Lighweight implementation Y N > > Non-Requirements > Zoning N Y > State Change Notification N Y > Naming Services N Y > > * uses SLPv2 constructs > > > David Peterson > Lead Architect - Standards Development > Cisco Systems - SRBU > 6450 Wedgwood Road > Maple Grove, MN 55311 > Office: 763-398-1007 > Cell: 612-802-3299 > Email: dap@cisco.com From owner-ips@ece.cmu.edu Thu May 24 15:18:53 2001 Received: from ece.cmu.edu ([128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id PAA19389 for ; Thu, 24 May 2001 15:18:53 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f4OHPdm21909 for ips-outgoing; Thu, 24 May 2001 13:25:39 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from palrel2.hp.com (palrel2.hp.com [156.153.255.234]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f4OHPbH21905 for ; Thu, 24 May 2001 13:25:37 -0400 (EDT) Received: from hpcuhe.cup.hp.com (hpcuhe.cup.hp.com [15.0.80.203]) by palrel2.hp.com (Postfix) with ESMTP id C00D328B3 for ; Thu, 24 May 2001 10:25:36 -0700 (PDT) Received: from cup.hp.com (santoshr@hpindhhm.cup.hp.com [15.8.80.197]) by hpcuhe.cup.hp.com (8.9.3 (PHNE_18979)/8.9.3 SMKit7.02) with ESMTP id KAA02208 for ; Thu, 24 May 2001 10:25:31 -0700 (PDT) Message-ID: <3B0D4430.6DD6B0A9@cup.hp.com> Date: Thu, 24 May 2001 10:26:08 -0700 From: Santosh Rao Organization: Hewlett Packard, Cupertino. X-Mailer: Mozilla 4.7 [en] (X11; U; HP-UX B.11.00 9000/778) X-Accept-Language: en MIME-Version: 1.0 To: ips@ece.cmu.edu Subject: Re: iSCSI: response to second login (with same ISID) References: <31891B757C09184BBFEC5275F85D5595104D66@cceexc18.americas.cpqcorp.net> Content-Type: multipart/mixed; boundary="------------D559EBB1AC08CD2122BDF0FB" Sender: owner-ips@ece.cmu.edu Precedence: bulk This is a multi-part message in MIME format. --------------D559EBB1AC08CD2122BDF0FB Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Nick, I agree with you that there is a case for both usage models and a bit in the Login PDU that allows for both behaviours would address both sets of issues. I'm in support of adding a bit ("login unconditionally") to the Login PDU that allows the initiator to specify whether any previous session for that ISID, if it exists, is to be logged out and a new session login established. Thanks, Santosh "Martin, Nick" wrote: > > [The message I am replying to is below this one. A similar one was posted > while I was composing this reply.] > > Santosh, > > If the iSCSI HBA is exposing SCSI targets to the OS, then the OS's SCSI > target driver (or class driver if I have my Microsoft terminology correct) > can support "exclusive access" semantics which can protect the current user > of a tape device from other users on the same system attempting to use the > same device through the same driver, or can allow a disk device (and the > session for it) to be shared by multiple users when that is appropriate. If > the target device supports some form of SCSI reserve/release, then this > would protect the device from access via a second session from anywhere. In > the target, iSCSI could also refuse a second concurrent session login to the > same tape target protecting initiators which do not reserve/release. (The > trick is: does the target think this is a new session using the same ISID > (reject), or an attempt to clean-up and re-establish an existing session > (reset, accept).) > > Think about an iSCSI enabled version of tar or cpio (simple applications > used for tape backups). These could run from user space over a TCP > connection on a standard NIC using an OS that has no awareness of iSCSI. If > a second copy of the program is started, how is it to know that something is > already using the same tape device? How is it to know the ISID used for > that other copy? How is it to know whether ANY particular ISID is really > currently in use, or what it is being used for? How can we prevent the > second copy from unknowingly stealing the device from the first by simply > using the same ISID? The second user thinks everything is fine until the > first user restarts his command that just failed. In this model there is no > HBA driver or target driver. There is nothing that knows about both > sessions being attempted except the target. > > You may think this is a bit far fetched, but I almost have such a program. > I expect other folks will be clever enough to think of other ways to use > this "user mode access" capability of TCP/IP to use remote iSCSI devices > without kernel co-operation. Can we enable them to peacefully co-exist with > each other? Can they peacefully co-exist with kernel mode drivers? > > There may be other scenarios such as administrators dynamically adding and > removing iSCSI devices to an OS while a system is running rather than at > boot time (think 24x7 operations, what choice to they have?). If the ISID's > are going to have to be manually configured, then there is a high likelihood > of human error causing disruptions. If we can enable non-disruptive > automatic ISID probing to find an available ISID during device configuration > or initialization, this seems safer. Once we pick an ISID, we should still > remember it across re-boot, to avoid repeated probing, and to assist manual > clean-up after ungraceful shutdown. We also need the "logout this old > session" capability both for recovery after an initiator crash or error > which failed to logout, and for when the initiator realizes it has no > remaining working TCP connections to an existing session and wants to be > sure to free target resources and start fresh. > > So, I think there are valid arguments for both behaviors 1 and 2, and we > need to enable the initiator to select the behavior appropriate for each > application. > > Stated another way, we need to allow for the case when the initiator is sure > that this ISID is unique and it takes full responsibility for all > consequences, and for the case where the initiator can not be sure this ISID > is unique and wants there to be no side effects for others if it turns out > not to be unique. > > Thanks, > Nick > -----Original Message----- > From: Santosh Rao [mailto:santoshr@cup.hp.com] > Sent: Wednesday, May 23, 2001 8:36 PM > To: Martin, Nick > Cc: santoshr@cup.hp.com > Subject: Re: iSCSI: response to second login (with same ISID) > > "Martin, Nick" wrote: > > > > > I know from personal experience how easy it is for an authorized user to > > unintentionally start a second copy of a backup program, while the tape > > drive is still in use. This should be a harmless mistake. What kind of > > traffic cop would prevent this request from making it to the iSCSI target > > and terminating the session already in progress? (Rhetorical question, > > please do not describe for me such a mechanism ;) > > Nick, > > I know you don't want an answer to your rhetorical question ;-}, but, I > could'nt resist one quick query here. > > In the scenario you describe, is it not the case that HBA drivers would > only send out a Login on the first open of that target, and on > subsequent opens would only bump an open_count or a ref_count of some > kind rather than send a login on the wire on every open ? > > If that is correct, then, the scenario you describe would still cause > disruption even with the modified semantics, only, when the WRITE is > issued by the 2nd execution of the backup application (?). > > Regards, > Santosh --------------D559EBB1AC08CD2122BDF0FB Content-Type: text/x-vcard; charset=us-ascii; name="santoshr.vcf" Content-Description: Card for Santosh Rao Content-Disposition: attachment; filename="santoshr.vcf" Content-Transfer-Encoding: 7bit begin:vcard n:Rao;Santosh tel;work:408-447-3751 x-mozilla-html:FALSE org:Hewlett Packard, Cupertino.;SISL adr:;;19420, Homestead Road, M\S 43LN, ;Cupertino.;CA.;95014.;USA. version:2.1 email;internet:santoshr@cup.hp.com title:Software Design Engineer x-mozilla-cpt:;21088 fn:Santosh Rao end:vcard --------------D559EBB1AC08CD2122BDF0FB-- From owner-ips@ece.cmu.edu Thu May 24 15:19:23 2001 Received: from ece.cmu.edu ([128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id PAA19414 for ; Thu, 24 May 2001 15:19:22 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f4OHmFE23931 for ips-outgoing; Thu, 24 May 2001 13:48:15 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from erie.mcdata.com (mcgate.mcdata.com [144.49.1.5]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f4OHmDH23921 for ; Thu, 24 May 2001 13:48:13 -0400 (EDT) Received: by erie.mcdata.com with Internet Mail Service (5.5.2653.19) id ; Thu, 24 May 2001 11:48:06 -0600 Message-ID: From: Anil Rijhsinghani To: "'Elizabeth Rodriguez'" , Dave Peterson , Anil Rijhsinghani , ENDL_TX@computer.org, IPS Reflector Subject: RE: FCIP - Discovery should be Manual/SLP Date: Thu, 24 May 2001 11:48:06 -0600 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2653.19) Content-Type: text/plain; charset="iso-8859-1" Sender: owner-ips@ece.cmu.edu Precedence: bulk Elizabeth: I mis-spoke, what I should have said was that when dynamic discovery is provided then I believe SLPv2 should be the only required method. As you say, there will be environments in which static configuration is sufficient. Dave: True, other mechanisms could provide the same function. At some point when we get to writing the compliance section for the MIB, we will need to decide if this is a necessary minimum (the static config group). Usually implementation of the MIB itself is not mandatory, but when it is implemented then there will be specific groups which are. Anil -----Original Message----- From: Elizabeth Rodriguez [mailto:egrodriguez@lucent.com] Sent: Thursday, May 24, 2001 11:09 AM To: Dave Peterson; Anil Rijhsinghani; ENDL_TX@computer.org; IPS Reflector Subject: RE: FCIP - Discovery should be Manual/SLP (Chair hat off) Anil, Correct me here if I am wrong, but I thought no mandatory to implement discovery mechanism was going to be prescribed for FCIP. If a dynamic discovery mechanism was desired, then SLPv2 would be the method to implement. Elizabeth -----Original Message----- From: Dave Peterson [mailto:dap@cisco.com] Sent: Thursday, May 24, 2001 10:53 AM To: Anil Rijhsinghani; ENDL_TX@computer.org; IPS Reflector Subject: RE: FCIP - Discovery should be Manual/SLP I am not opposed to performing static configuration using the FCIP MIB and can provide appropriate wording but I think we should also allow the capability of static configuration using other methods (e.g., command line interface). Dave > -----Original Message----- > From: owner-ips@ece.cmu.edu [mailto:owner-ips@ece.cmu.edu]On Behalf Of > Anil Rijhsinghani > Sent: Thursday, May 24, 2001 10:17 AM > To: 'ENDL_TX@computer.org'; IPS Reflector; 'dap@cisco.com' > Subject: RE: FCIP - Discovery should be Manual/SLP > > > Agree with the use of SLPv2 as the only mandatory discovery mechanism > for FCIP, with naming/zoning not being requirements for FCIP. > > Note that static configuration (as opposed to dynamic discovery method) > will be specified in the FCIP MIB, so I would replace the wording below > "static config is outside the scope of this specification" to > "static config is specified in the FCIP MIB document". > > Regards, > Anil > > -----Original Message----- > From: Ralph Weber [mailto:ralphoweber@compuserve.com] > Sent: Wednesday, May 16, 2001 3:25 PM > To: IPS Reflector > Subject: FCIP - Discovery should be Manual/SLP > > > I support Dave's proposed text for FCIP Clause 4.2 item 3, with the > exception that the sentence regarding iSNS needs to be removed. > Looking at the features list, I cannot see any iSNS unique features > that are meaningful to FCIP. > > Thanks. > > Ralph... > > - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - > > Subject: Proposal to use SLPv2 for FCIP device discovery > Date: Mon, 30 Apr 2001 17:07:08 -0500 > From: "Dave Peterson" > To: "Ips@Ece. Cmu. Edu" > > > As requested here is the proposal I presented at Mondays' FCIP > meeting: > > FCIP Draft Proposal > For Clause 4.2, item 3. > > Each FCIP device may be statically or dynamically configured with > a list of IP addresses and port numbers corresponding to all the > participating FCIP devices. If dynamic discovery of participating > FCIP devices is supported this function shall be performed using the > Service Location Protocol (SLPv2). For additional FCIP device > management capability, the iSNS Internet Storage Naming Service may > be implemented. It is outside the scope of this specification to > describe any static configuration method for participating FCIP device > discovery. Refer to clause XXX for a detailed description of dynamic > discovery of participating FCIP device using SLPv2. > > Notes: > 1. DHCP: > a. Allows an entity to discovery information about itself, > not discover information about all other entities. > b. Uses a broadcast mechanism that may not work via routers > without additional configuration. But, most current router > implementations will support the forwarding of DHCP requests > across routed subnets. > c. May be used to find SLP Directory Agents and Scope Lists > allowing for a more scalable solution. > 2. DNS Services are too limiting. This is the reason why SLP was > started. > 3. LDAP is simply a database interface. SLP and iSNS may use LDAP > as a back-end store. > 4. SLP FCIP Device Type Specifics > a. IP address(es) > b. Port numbers(s) > c. Scope > d. Attributes > i. Discovery domain (i.e. a group name or zone name, don't want > to use zone name in this context though) > ii. need to start listing these (if we can think of any more) > e. Lifetime > > Work In progress: > 1. Putting together a model for FCIP device discovery using SLPv2. > 2. Implementing a "default/suggested" SLPv2 template. > 3. Reviewing RFC 3082 "Notification and Subscription for SLP for > enhanced device notification. > > Requirements SLPv2 iSNS > Discovery of FCIP targets Y Y > Discovery of FCIP device IP address(es) Y Y > Discovery of FCIP port number(s) Y Y > Attribute support Y Y > Semi-timely notification of devices coming and going Y Y > Authentication Y Y* > Minimal/no configuration Y Y(?) > Provisioning Y Y > Multicast support Y N > Publicly available source Y N > Standardized and mature Y N > Lighweight implementation Y N > > Non-Requirements > Zoning N Y > State Change Notification N Y > Naming Services N Y > > * uses SLPv2 constructs > > > David Peterson > Lead Architect - Standards Development > Cisco Systems - SRBU > 6450 Wedgwood Road > Maple Grove, MN 55311 > Office: 763-398-1007 > Cell: 612-802-3299 > Email: dap@cisco.com From owner-ips@ece.cmu.edu Thu May 24 15:26:10 2001 Received: from ece.cmu.edu ([128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id PAA19509 for ; Thu, 24 May 2001 15:26:10 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f4OH9LR20661 for ips-outgoing; Thu, 24 May 2001 13:09:21 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from ihemail1.firewall.lucent.com (ihemail1.lucent.com [192.11.222.161]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f4OH9JH20655 for ; Thu, 24 May 2001 13:09:19 -0400 (EDT) Received: from ihemail1.firewall.lucent.com (localhost [127.0.0.1]) by ihemail1.firewall.lucent.com (Switch-2.1.1/Switch-2.1.0) with ESMTP id f4OH9If03834 for ; Thu, 24 May 2001 13:09:18 -0400 (EDT) Received: from nc8220exchange.ral.lucent.com (h135-92-100-21.lucent.com [135.92.100.21]) by ihemail1.firewall.lucent.com (Switch-2.1.1/Switch-2.1.0) with ESMTP id f4OH9IW03820 for ; Thu, 24 May 2001 13:09:18 -0400 (EDT) Subject: RE: FCIP - Discovery should be Manual/SLP MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Date: Thu, 24 May 2001 13:09:09 -0400 Message-ID: Disposition-Notification-To: "Elizabeth Rodriguez" X-MimeOLE: Produced By Microsoft Exchange V6.0.4418.65 Content-Class: urn:content-classes:message Thread-Topic: FCIP - Discovery should be Manual/SLP Thread-Index: AcDkccIy7r3flIhRRlaLf1s6Qc8mmAAAc5YA From: "Elizabeth Rodriguez" To: "Dave Peterson" , "Anil Rijhsinghani" , , "IPS Reflector" Content-Transfer-Encoding: 8bit X-MIME-Autoconverted: from quoted-printable to 8bit by ece.cmu.edu id f4OH9JH20657 Sender: owner-ips@ece.cmu.edu Precedence: bulk Content-Transfer-Encoding: 8bit (Chair hat off) Anil, Correct me here if I am wrong, but I thought no mandatory to implement discovery mechanism was going to be prescribed for FCIP. If a dynamic discovery mechanism was desired, then SLPv2 would be the method to implement. Elizabeth -----Original Message----- From: Dave Peterson [mailto:dap@cisco.com] Sent: Thursday, May 24, 2001 10:53 AM To: Anil Rijhsinghani; ENDL_TX@computer.org; IPS Reflector Subject: RE: FCIP - Discovery should be Manual/SLP I am not opposed to performing static configuration using the FCIP MIB and can provide appropriate wording but I think we should also allow the capability of static configuration using other methods (e.g., command line interface). Dave > -----Original Message----- > From: owner-ips@ece.cmu.edu [mailto:owner-ips@ece.cmu.edu]On Behalf Of > Anil Rijhsinghani > Sent: Thursday, May 24, 2001 10:17 AM > To: 'ENDL_TX@computer.org'; IPS Reflector; 'dap@cisco.com' > Subject: RE: FCIP - Discovery should be Manual/SLP > > > Agree with the use of SLPv2 as the only mandatory discovery mechanism > for FCIP, with naming/zoning not being requirements for FCIP. > > Note that static configuration (as opposed to dynamic discovery method) > will be specified in the FCIP MIB, so I would replace the wording below > "static config is outside the scope of this specification" to > "static config is specified in the FCIP MIB document". > > Regards, > Anil > > -----Original Message----- > From: Ralph Weber [mailto:ralphoweber@compuserve.com] > Sent: Wednesday, May 16, 2001 3:25 PM > To: IPS Reflector > Subject: FCIP - Discovery should be Manual/SLP > > > I support Dave's proposed text for FCIP Clause 4.2 item 3, with the > exception that the sentence regarding iSNS needs to be removed. > Looking at the features list, I cannot see any iSNS unique features > that are meaningful to FCIP. > > Thanks. > > Ralph... > > - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - > > Subject: Proposal to use SLPv2 for FCIP device discovery > Date: Mon, 30 Apr 2001 17:07:08 -0500 > From: "Dave Peterson" > To: "Ips@Ece. Cmu. Edu" > > > As requested here is the proposal I presented at Mondays' FCIP > meeting: > > FCIP Draft Proposal > For Clause 4.2, item 3. > > Each FCIP device may be statically or dynamically configured with > a list of IP addresses and port numbers corresponding to all the > participating FCIP devices. If dynamic discovery of participating > FCIP devices is supported this function shall be performed using the > Service Location Protocol (SLPv2). For additional FCIP device > management capability, the iSNS Internet Storage Naming Service may > be implemented. It is outside the scope of this specification to > describe any static configuration method for participating FCIP device > discovery. Refer to clause XXX for a detailed description of dynamic > discovery of participating FCIP device using SLPv2. > > Notes: > 1. DHCP: > a. Allows an entity to discovery information about itself, > not discover information about all other entities. > b. Uses a broadcast mechanism that may not work via routers > without additional configuration. But, most current router > implementations will support the forwarding of DHCP requests > across routed subnets. > c. May be used to find SLP Directory Agents and Scope Lists > allowing for a more scalable solution. > 2. DNS Services are too limiting. This is the reason why SLP was > started. > 3. LDAP is simply a database interface. SLP and iSNS may use LDAP > as a back-end store. > 4. SLP FCIP Device Type Specifics > a. IP address(es) > b. Port numbers(s) > c. Scope > d. Attributes > i. Discovery domain (i.e. a group name or zone name, don't want > to use zone name in this context though) > ii. need to start listing these (if we can think of any more) > e. Lifetime > > Work In progress: > 1. Putting together a model for FCIP device discovery using SLPv2. > 2. Implementing a "default/suggested" SLPv2 template. > 3. Reviewing RFC 3082 "Notification and Subscription for SLP for > enhanced device notification. > > Requirements SLPv2 iSNS > Discovery of FCIP targets Y Y > Discovery of FCIP device IP address(es) Y Y > Discovery of FCIP port number(s) Y Y > Attribute support Y Y > Semi-timely notification of devices coming and going Y Y > Authentication Y Y* > Minimal/no configuration Y Y(?) > Provisioning Y Y > Multicast support Y N > Publicly available source Y N > Standardized and mature Y N > Lighweight implementation Y N > > Non-Requirements > Zoning N Y > State Change Notification N Y > Naming Services N Y > > * uses SLPv2 constructs > > > David Peterson > Lead Architect - Standards Development > Cisco Systems - SRBU > 6450 Wedgwood Road > Maple Grove, MN 55311 > Office: 763-398-1007 > Cell: 612-802-3299 > Email: dap@cisco.com From owner-ips@ece.cmu.edu Thu May 24 19:09:11 2001 Received: from ece.cmu.edu ([128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id TAA22474 for ; Thu, 24 May 2001 19:09:10 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f4OLQcj14466 for ips-outgoing; Thu, 24 May 2001 17:26:38 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from sandmail.sandburst.com (sandburst-gw.bstn-gw02.ma.us.intelilink.net [216.57.129.34]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f4OLQbH14462 for ; Thu, 24 May 2001 17:26:37 -0400 (EDT) Received: from cs.uchicago.edu (dynamite-38.sandburst.com [172.16.5.38]) by sandmail.sandburst.com (Postfix) with ESMTP id 92F9C94015 for ; Thu, 24 May 2001 17:26:36 -0400 (EDT) To: ips@ece.cmu.edu Subject: Re: iSCSI: response to second login (with same ISID) In-Reply-To: Message from Santosh Rao of "Thu, 24 May 2001 10:26:08 PDT." <3B0D4430.6DD6B0A9@cup.hp.com> References: <31891B757C09184BBFEC5275F85D5595104D66@cceexc18.americas.cpqcorp.net> <3B0D4430.6DD6B0A9@cup.hp.com> Date: Thu, 24 May 2001 17:24:34 -0400 From: Stephen Bailey Message-Id: <20010524212636.92F9C94015@sandmail.sandburst.com> Sender: owner-ips@ece.cmu.edu Precedence: bulk Nick & Santosh, > I expect other folks will be clever enough to think of other ways to use > this "user mode access" capability of TCP/IP to use remote iSCSI devices > without kernel co-operation. Can we enable them to peacefully co-exist with > each other? Can they peacefully co-exist with kernel mode drivers? There's a nice, elegant can of worms, I mean solution, to this problem. In fact, I'm sure this is probably just a rehash of somebody else's (Jim Hafner's) proposal, but I'm going to beat out some implications. Some people might say this is already the way things work. I don't see anything in the spec that precludes this interpretation, anyway. If a SCSI I (as in I_T) is defined by a unique Initiator Name, then you simply say that each distinct iSCSI entity within a system (and across systems too, of course) has it's own Initiator Name. That means that each entity has it's own, distinct SID space. In other words, the kernel mode iSCSI entity will have a different Initiator Name from the user-mode programs. Each iSCSI entity (user program or kernel) is responsible for producing its own Initiator Name, AND authenticating that name with the target. A user mode program could find the kernel's initiator name, but it couldn't perform the proper authentication, so it would not be able to `hijack' the TSID. Similarly noncooperative user mode programs could not authenticate themselves as each other either. > You may think this is a bit far fetched, but I almost have such a > program. Not at all. I think it's a very exciting possibility. One of the features which an iSCSI running on RDMA can offer is fully accelerated (as in, hardware handled data), user-mode (OS-bypassed) iSCSI connections. This is a major reason why I keep pushing iSCSI/RDMA as The Right Thing. > Santosh said: > I'm in support of adding a bit ("login unconditionally") to the Login > PDU that allows the initiator to specify whether any previous session > for that ISID, if it exists, is to be logged out and a new session login > established. Given the idea above, it's not necessary. Even if you discount the idea above (which you better not, or I'm going to be really MAD!), I don't see how such a bit would be used. What is the algorithm for setting the bit? If you allow multiple entities (e.g. a user-client and a kernel client) to use the same Initiator Name, one of those entities can deny service from the other, accidentally, or deliberately by having the bit set. Specifically, to allow access at all in this scenario, it must either be the case that: o two entities can authenticate with the same Initiator Name or o no authentication is required In this case, you can not force a malicious or clumsy entity not to set the bit, which means that one entity can deny service to the other by hijacking (forcing implicit logout) a session. Even if you disallow multiple entities using the same Initiator Name, the policing behavior of the `~login unconditionally' setting doesn't buy you much. The first time in your life you login with a TSID, you must assume that you may have been logged in to the same target in a previous life (reboot), within RA_TOV. You have no way (no memory of the previous life) to know that you did not do that. Therefore, clearly, the first time in your life you use a TSID, you will login with `login unconditionally'. On subsequent logins with a TSID within the same life you could set '~login unconditionally' and you might learn that you blew your TSID allocation policy (or something else), but frankly, it seems like a pretty small gain. Even if you always login with `login unconditionally', you will still certainly know if you blown your TSID allocation policy. Steph P.S. > (or class driver if I have my Microsoft terminology correct) Actually, that's VMS[++ == WNT] terminology, and yep you got it :^) From owner-ips@ece.cmu.edu Thu May 24 19:12:30 2001 Received: from ece.cmu.edu ([128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id TAA22531 for ; Thu, 24 May 2001 19:12:29 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f4OKShf09258 for ips-outgoing; Thu, 24 May 2001 16:28:43 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from mxic2.us.dg.com (mxic2.us.dg.com [128.221.31.40]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f4OKSgH09254 for ; Thu, 24 May 2001 16:28:42 -0400 (EDT) Received: by mxic2.us.dg.com with Internet Mail Service (5.5.2653.19) id ; Thu, 24 May 2001 16:28:05 -0400 Message-ID: <0F31E5C394DAD311B60C00E029101A07080155F7@corpmx9.isus.emc.com> From: Black_David@emc.com To: jtseng@NishanSystems.com, ips@ece.cmu.edu Subject: RE: SLP, iSNS, and iSCSI Date: Thu, 24 May 2001 16:28:36 -0400 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2653.19) Content-Type: text/plain Sender: owner-ips@ece.cmu.edu Precedence: bulk Josh, > I'm not a DHCP expert, but in my limited understanding > of the protocol, the DHCP server is not intelligent > about who is asking for IP addresses and options. > There is nothing to identify or authenticate a DHCP > client making a request (other than the MAC), since it > doesn't even have an IP address, not to mention an FQDN > or any other identity. I'm not a DHCP expert either, so we may need to go find one. The following is what I currently understand ... I agree with the concern about the scenario in which the DHCP server is dynamically allocating IP addresses ... > If this is true, I am puzzled as to how you can use > DHCP to manage SLP scopes. The DHCP server can't say > "device A gets scopes X, Y, and Z", since it doesn't > even know who device A is. All it can do is hand out > the same set of scopes to everyone requesting option 79. > And that's all. So how can you configure a complex set > of overlapping scopes using a DHCP server??? There are at least two existing answers to that question, both of which involve doing something different to assign IP addresses: (1) Manual Allocation. From Section 1 of RFC 2131: In "manual allocation", a client's IP address is assigned by the network administrator, and DHCP is used simply to convey the assigned address to the client. In other words, the client's IP address is bound to its MAC, and possibly additional information, such as the subnet on which that MAC is presented if a DHCP proxy is involved. A DHCP proxy gets around DHCP's use of link-level broadcasts that don't propagate through layer 3 (IP) forwarding by proxying the node issuing the broadcast to the DHCP server. The DHCP server can see the proxy identity, and hence knows which subnet is involved. (2) DHCP can be used to distribute configuration information without doing any IP address allocation. This is described starting in Section 3.4 of RFC 2131, and uses a different DHCP message from the one used to request IP address allocation. In particular, a DHCP sever receiving such a message "MUST NOT check for an existing lease" (on the IP address). Even in the dynamic allocation scenario, knowledge of the subnet/proxy involved may be enough for the DHCP server to figure out what the right config parameters are (e.g., all FCIP devices on the same subnet may be in the same SLP scope(s), i.e., intended to connect to the same set of FCIP peers). > Lastly, I want to make sure everyone understands that > support for FCIP is included in the iSNS out of courtesy > to the FCIP community, and after close consultation with > several key individuals from that community. Since we > are not implementing FCIP, I personally am neutral on > whether it should be used for FCIP, although I do > believe it would be of great value. I believe I was also a source of requests to do this, and the courtesy is appreciated. I think it's now up to the WG to decide what FCIP discovery/configuration mechanisms to REQUIRE and/or RECOMMEND in the FCIP draft. Thanks, --David --------------------------------------------------- David L. Black, Senior Technologist EMC Corporation, 42 South St., Hopkinton, MA 01748 +1 (508) 435-1000 x75140 FAX: +1 (508) 497-8500 black_david@emc.com Mobile: +1 (978) 394-7754 --------------------------------------------------- From owner-ips@ece.cmu.edu Thu May 24 19:12:34 2001 Received: from ece.cmu.edu ([128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id TAA22553 for ; Thu, 24 May 2001 19:12:34 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f4OM5cj17276 for ips-outgoing; Thu, 24 May 2001 18:05:38 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from server1.NishanSystems.COM (smtp.nishansystems.com [216.217.36.162]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f4OM5bH17272 for ; Thu, 24 May 2001 18:05:37 -0400 (EDT) Received: by smtp.nishansystems.com with Internet Mail Service (5.5.2653.19) id ; Thu, 24 May 2001 15:05:26 -0700 Message-ID: From: Joshua Tseng To: "'Black_David@emc.com'" , ips@ece.cmu.edu Subject: RE: SLP, iSNS, and iSCSI Date: Thu, 24 May 2001 15:05:25 -0700 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2653.19) Content-Type: text/plain; charset="iso-8859-1" Sender: owner-ips@ece.cmu.edu Precedence: bulk David, > I believe I was also a source of requests to do this, and the > courtesy is appreciated. I think it's now up to the WG to decide > what FCIP discovery/configuration mechanisms to REQUIRE > and/or RECOMMEND in the FCIP draft. Thank you for your explanation, and I agree that it is a WG issue now. Regards, Josh > -----Original Message----- > From: Black_David@emc.com [mailto:Black_David@emc.com] > Sent: Thursday, May 24, 2001 1:29 PM > To: jtseng@NishanSystems.com; ips@ece.cmu.edu > Subject: RE: SLP, iSNS, and iSCSI > > > Josh, > > > I'm not a DHCP expert, but in my limited understanding > > of the protocol, the DHCP server is not intelligent > > about who is asking for IP addresses and options. > > There is nothing to identify or authenticate a DHCP > > client making a request (other than the MAC), since it > > doesn't even have an IP address, not to mention an FQDN > > or any other identity. > > I'm not a DHCP expert either, so we may need to go find one. > The following is what I currently understand ... > > I agree with the concern about the scenario in which the > DHCP server is dynamically allocating IP addresses ... > > > If this is true, I am puzzled as to how you can use > > DHCP to manage SLP scopes. The DHCP server can't say > > "device A gets scopes X, Y, and Z", since it doesn't > > even know who device A is. All it can do is hand out > > the same set of scopes to everyone requesting option 79. > > And that's all. So how can you configure a complex set > > of overlapping scopes using a DHCP server??? > > There are at least two existing answers to that question, > both of which involve doing something different to assign > IP addresses: > > (1) Manual Allocation. From Section 1 of RFC 2131: > > In "manual allocation", a client's IP > address is assigned by the network administrator, and DHCP > is used simply to convey the assigned address to the client. > > In other words, the client's IP address is bound to its MAC, > and possibly additional information, such as the subnet on > which that MAC is presented if a DHCP proxy is involved. > A DHCP proxy gets around DHCP's use of link-level broadcasts > that don't propagate through layer 3 (IP) forwarding by proxying > the node issuing the broadcast to the DHCP server. The > DHCP server can see the proxy identity, and hence knows > which subnet is involved. > > (2) DHCP can be used to distribute configuration information > without doing any IP address allocation. This is described > starting in Section 3.4 of RFC 2131, and uses a different > DHCP message from the one used to request IP address > allocation. In particular, a DHCP sever receiving such a > message "MUST NOT check for an existing lease" > (on the IP address). > > Even in the dynamic allocation scenario, knowledge of the > subnet/proxy involved may be enough for the DHCP > server to figure out what the right config parameters > are (e.g., all FCIP devices on the same subnet may be in > the same SLP scope(s), i.e., intended to connect to the > same set of FCIP peers). > > > Lastly, I want to make sure everyone understands that > > support for FCIP is included in the iSNS out of courtesy > > to the FCIP community, and after close consultation with > > several key individuals from that community. Since we > > are not implementing FCIP, I personally am neutral on > > whether it should be used for FCIP, although I do > > believe it would be of great value. > > I believe I was also a source of requests to do this, and the > courtesy is appreciated. I think it's now up to the WG to decide > what FCIP discovery/configuration mechanisms to REQUIRE > and/or RECOMMEND in the FCIP draft. > > Thanks, > --David > > --------------------------------------------------- > David L. Black, Senior Technologist > EMC Corporation, 42 South St., Hopkinton, MA 01748 > +1 (508) 435-1000 x75140 FAX: +1 (508) 497-8500 > black_david@emc.com Mobile: +1 (978) 394-7754 > --------------------------------------------------- > From owner-ips@ece.cmu.edu Thu May 24 20:33:21 2001 Received: from ece.cmu.edu ([128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id UAA23236 for ; Thu, 24 May 2001 20:33:20 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f4OMeVS19817 for ips-outgoing; Thu, 24 May 2001 18:40:31 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from mail.brocade.com (asbestos.brocade.com [63.121.140.244]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f4OK3XH06303 for ; Thu, 24 May 2001 16:03:33 -0400 (EDT) Received: from hq-ex-c2.brocade.com (hq-ex-c2 [192.168.126.35]) by mail.brocade.com (8.8.8+Sun/8.8.8) with ESMTP id NAA07981; Thu, 24 May 2001 13:03:21 -0700 (PDT) Received: by hq-ex-c2.brocade.com with Internet Mail Service (5.5.2653.19) id ; Thu, 24 May 2001 13:03:20 -0700 Message-ID: From: Robert Snively To: "'Black_David@emc.com'" , ips@ece.cmu.edu Subject: Response to action item from Nashua meeting Date: Thu, 24 May 2001 13:03:18 -0700 X-Mailer: Internet Mail Service (5.5.2653.19) Sender: owner-ips@ece.cmu.edu Precedence: bulk > -> Action: Bob Snively will check whether FSPF > (Fibre Channel routing protocol > that will always be present on an FCIP link as > FCIP is currently specified) > already contains heartbeats to detect a lost > connection. If so, FCIP can leave this issue > to FSPF to handle. The answer is yes, FSPF does provide for a mechanism that verifies a connection is still present. The mechanism uses the Hello Switch Fabric Internal Link Service to verify link health. See FC-SW-2, section 6.1.7. The default checking time is 20 seconds, but can be adjusted. From owner-ips@ece.cmu.edu Thu May 24 22:44:46 2001 Received: from ece.cmu.edu ([128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id WAA26220 for ; Thu, 24 May 2001 22:44:45 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f4P0ZIn27562 for ips-outgoing; Thu, 24 May 2001 20:35:18 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from mxic2.us.dg.com (mxic2.us.dg.com [128.221.31.40]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f4P0ZGH27558 for ; Thu, 24 May 2001 20:35:16 -0400 (EDT) Received: by mxic2.us.dg.com with Internet Mail Service (5.5.2653.19) id ; Thu, 24 May 2001 20:34:39 -0400 Message-ID: <0F31E5C394DAD311B60C00E029101A07080155FE@corpmx9.isus.emc.com> From: Black_David@emc.com To: ips@ece.cmu.edu Subject: iSCSI boot Date: Thu, 24 May 2001 20:35:10 -0400 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2653.19) Content-Type: text/plain Sender: owner-ips@ece.cmu.edu Precedence: bulk Somewhere buried in my "to-do" list is to provide some guidance to the iSCSI boot draft authors about what to do next. What follows are more of suggestions than instructions and comments are welcome. - The draft will need to be split into two pieces, an informational draft describing the overall boot process and a standards-track document that standardizes the DHCP option. The informational draft might benefit from being merged into the corresponding portion of the naming and discovery draft, since much of what it covers is how to discover the boot device. - The DHCP option draft would need to be coordinated with the DHC WG. This need for a new DHCP option may be a problem, as there aren't may option codes left: (http://www.iana.org/assignments/bootp-dhcp-parameters). Using DHCP to find SLP to find the boot device seems both clumsy and an invitation to problems (one more thing that can break and prevent booting), but I wonder if there's a way to [ab]use existing DHCP option 17 (Root Path) for this purpose. In any case, the use of DHCP will need to be pursued with the DHCP WG via a document that describes only the DHCP option to be standardized. This probably shouldn't be undertaken until the iSCSI naming and discovery draft is relatively stable so that we're certain about naming formats and mechanisms (e.g., is "Send Targets" involved in boot? - this has implications for the information passed through DHCP). Comments? --David --------------------------------------------------- David L. Black, Senior Technologist EMC Corporation, 42 South St., Hopkinton, MA 01748 +1 (508) 435-1000 x75140 FAX: +1 (508) 497-8500 black_david@emc.com Mobile: +1 (978) 394-7754 --------------------------------------------------- From owner-ips@ece.cmu.edu Fri May 25 01:31:06 2001 Received: from ece.cmu.edu ([128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id BAA29704 for ; Fri, 25 May 2001 01:31:06 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f4P2lva06491 for ips-outgoing; Thu, 24 May 2001 22:47:57 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from palrel1.hp.com (palrel1.hp.com [156.153.255.242]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f4P2ltH06487 for ; Thu, 24 May 2001 22:47:55 -0400 (EDT) Received: from psalpha0.cup.hp.com (psalpha0.cup.hp.com [15.13.189.230]) by palrel1.hp.com (Postfix) with ESMTP id D75D910DC for ; Thu, 24 May 2001 19:47:54 -0700 (PDT) Received: from mk731913.cup.hp.com (mk731912.cup.hp.com [15.8.80.111]) by psalpha0.cup.hp.com (8.9.3/8.8.6) with ESMTP id TAA09641; Thu, 24 May 2001 19:47:43 -0700 (PDT) Message-Id: <5.1.0.14.0.20010524191351.02b54d78@esalpha2.cup.hp.com> X-Sender: krause@esalpha2.cup.hp.com X-Mailer: QUALCOMM Windows Eudora Version 5.1 Date: Thu, 24 May 2001 19:32:33 -0700 To: cbm@rose.hp.com From: Michael Krause Subject: Re: TCP Framing (considered helpful?) Cc: ips@ece.cmu.edu In-Reply-To: <200105232154.OAA28571@core.rose.hp.com> Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii"; format=flowed Sender: owner-ips@ece.cmu.edu Precedence: bulk At 02:54 PM 5/23/2001 -0700, Mallikarjun C. wrote: >John, > >Two comments. I may have misunderstood what you're saying since >your initial comments imply framing=warp/RDMA/messageboundary, >and towards the end, you seem to imply framing=markers. > >- It's true that implementing iSCSI-level markers does not require > modifying TCP/IP stacks. But using the markers (I mean to use it > to effectively steer) requires TCP/IP changes! It is a separate > matter that it is offloaded onto the NIC. > >- You seem to implicitly suggest that WARP (when it becomes available) > must go into host software stacks to be useful, and hence cannot be > done in the right timeframe for iSCSI. RDMA was never intended to be transparent to the world. One is explicitly exporting memory to remote endnodes to target for specific operations - in many ways, not really any different than what one does with a SCSI PDU header. What will need to occur for iSCSI to use RDMA is at a minimum to define how one translates a SCSI operation into an RDMA READ or RDMA WRITE operation akin to the SRP (T10 SCSI RDMA Protocol). In this case, one would have all of the iSCSI session management, security, login, error management, etc. as defined today but the main data movement would most likely be a simplified SRP main data movement algorithm / mapping (in essence a blend of the two technologies at least on the main paths). This would also allow iSCSI to bridge more easily into the InfiniBand-based server environment (will grow in volume over the next few years) by allowing a simplified bridging between the two technology / management domains while maintaining an end-to-end iSCSI software (kernel driver as well) solution. > Since one could envision > offloading WARP onto the NIC as well, I assume you're hinting While there is clearly a software component in terms of memory management and communication between endnodes, the actual data movement is implemented in hardware. It really isn't practical to implement this in a strictly software stack given the performance and solution resource impacts. Note: There is the issue of maintaining virtual-to-physical mappings on the NIC associated with RDMA which could simply be at a minimal a mapping of all physical memory (preferably 2x) thereby allowing one to perform RDMA for any service to any address used by kernel or user-space I/O operations. > a) either at interoperability issues between software > and hardware iSCSI implementations relying on WARP, > b) or at interoperability issues between hardware and > hardware implementations. > > iSCSI currently mandates a "no sync and steering" mode which > ensures interoperability in either case. Perhaps you were really > concerned about performance in case (b) then? It means that one has to decide during login as to whether the session should be established or not as a function of the attributes on either side of the connection. For example, a RDMA / TOE implementation may simply refuse to accept a login from a non-RDMA implementation as it may not have the resource (NIC or system) to handle a more resource intensive / lower performance remote endnode (might also revert to a software-based solution if desired which would not use really any NIC services depending upon what protocol stack off-load is provided). Mike From owner-ips@ece.cmu.edu Fri May 25 04:31:00 2001 Received: from ece.cmu.edu ([128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id EAA13485 for ; Fri, 25 May 2001 04:31:00 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f4P5EeT15559 for ips-outgoing; Fri, 25 May 2001 01:14:40 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from d12lmsgate.de.ibm.com (d12lmsgate.de.ibm.com [195.212.91.199]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f4P5EcH15555 for ; Fri, 25 May 2001 01:14:39 -0400 (EDT) Received: from d12relay01.de.ibm.com (d12relay01.de.ibm.com [9.165.215.22]) by d12lmsgate.de.ibm.com (1.0.0) with ESMTP id HAA47864 for ; Fri, 25 May 2001 07:14:31 +0200 From: julian_satran@il.ibm.com Received: from d12mta02.de.ibm.com (d12mta01_cs0 [9.165.222.237]) by d12relay01.de.ibm.com (8.8.8m3/NCO v4.96) with SMTP id HAA61852 for ; Fri, 25 May 2001 07:14:31 +0200 Received: by d12mta02.de.ibm.com(Lotus SMTP MTA v4.6.5 (863.2 5-20-1999)) id C1256A57.001CC06C ; Fri, 25 May 2001 07:14:02 +0200 X-Lotus-FromDomain: IBMIL@IBMDE To: ips@ece.cmu.edu Message-ID: Date: Fri, 25 May 2001 07:36:56 +0300 Subject: RE: iSCSI: response to second login (with same ISID) Mime-Version: 1.0 Content-type: text/plain; charset=us-ascii Content-Disposition: inline Sender: owner-ips@ece.cmu.edu Precedence: bulk An amazingly long thread. The basic assumptions of the thread have been that: a - a resource (device) is allocated by reserve/release b - reservations are associated with a INAME+ISID as the "reserver name" and therefore those have to be unique in a initiator target pair. Assuming that everything else is running fine (initiators are able to login), IMHO this looks entirely as a matter of recovery. If a target senses an attempt to create a new session with the same INAME+ISID as an existing session it should: 1-Determine if the old session is active (currently transmitting and receiving or send a polling NOP-In and wait for an answer) If it is then it has to reject the new session 2-If it does not it has to clean the old session and establish a new session. I also assume that well designed targets will determine that a session is dead well before a new login forces them to do so. The larger question (that Marjorie alluded to) is whether SCSI is designed to allow user level access and employ only reservations for coordination and if it is not conceivable that several sessions can own a reservation (as several threads in process can get the same lock). And if reservation uniqueness within a given initiator should not be maintained by some SCSI mechanism (a key that is already in SCSI) as in some networks I might have now a thing running over FCP and after a while a session running on a backup dialup iSCSI link. Julo "Martin, Nick" on 24-05-2001 09:38:17 Please respond to "Martin, Nick" To: "'Santosh Rao'" , ips@ece.cmu.edu cc: Subject: RE: iSCSI: response to second login (with same ISID) [The message I am replying to is below this one. A similar one was posted while I was composing this reply.] Santosh, If the iSCSI HBA is exposing SCSI targets to the OS, then the OS's SCSI target driver (or class driver if I have my Microsoft terminology correct) can support "exclusive access" semantics which can protect the current user of a tape device from other users on the same system attempting to use the same device through the same driver, or can allow a disk device (and the session for it) to be shared by multiple users when that is appropriate. If the target device supports some form of SCSI reserve/release, then this would protect the device from access via a second session from anywhere. In the target, iSCSI could also refuse a second concurrent session login to the same tape target protecting initiators which do not reserve/release. (The trick is: does the target think this is a new session using the same ISID (reject), or an attempt to clean-up and re-establish an existing session (reset, accept).) Think about an iSCSI enabled version of tar or cpio (simple applications used for tape backups). These could run from user space over a TCP connection on a standard NIC using an OS that has no awareness of iSCSI. If a second copy of the program is started, how is it to know that something is already using the same tape device? How is it to know the ISID used for that other copy? How is it to know whether ANY particular ISID is really currently in use, or what it is being used for? How can we prevent the second copy from unknowingly stealing the device from the first by simply using the same ISID? The second user thinks everything is fine until the first user restarts his command that just failed. In this model there is no HBA driver or target driver. There is nothing that knows about both sessions being attempted except the target. You may think this is a bit far fetched, but I almost have such a program. I expect other folks will be clever enough to think of other ways to use this "user mode access" capability of TCP/IP to use remote iSCSI devices without kernel co-operation. Can we enable them to peacefully co-exist with each other? Can they peacefully co-exist with kernel mode drivers? There may be other scenarios such as administrators dynamically adding and removing iSCSI devices to an OS while a system is running rather than at boot time (think 24x7 operations, what choice to they have?). If the ISID's are going to have to be manually configured, then there is a high likelihood of human error causing disruptions. If we can enable non-disruptive automatic ISID probing to find an available ISID during device configuration or initialization, this seems safer. Once we pick an ISID, we should still remember it across re-boot, to avoid repeated probing, and to assist manual clean-up after ungraceful shutdown. We also need the "logout this old session" capability both for recovery after an initiator crash or error which failed to logout, and for when the initiator realizes it has no remaining working TCP connections to an existing session and wants to be sure to free target resources and start fresh. So, I think there are valid arguments for both behaviors 1 and 2, and we need to enable the initiator to select the behavior appropriate for each application. Stated another way, we need to allow for the case when the initiator is sure that this ISID is unique and it takes full responsibility for all consequences, and for the case where the initiator can not be sure this ISID is unique and wants there to be no side effects for others if it turns out not to be unique. Thanks, Nick -----Original Message----- From: Santosh Rao [mailto:santoshr@cup.hp.com] Sent: Wednesday, May 23, 2001 8:36 PM To: Martin, Nick Cc: santoshr@cup.hp.com Subject: Re: iSCSI: response to second login (with same ISID) "Martin, Nick" wrote: > > I know from personal experience how easy it is for an authorized user to > unintentionally start a second copy of a backup program, while the tape > drive is still in use. This should be a harmless mistake. What kind of > traffic cop would prevent this request from making it to the iSCSI target > and terminating the session already in progress? (Rhetorical question, > please do not describe for me such a mechanism ;) Nick, I know you don't want an answer to your rhetorical question ;-}, but, I could'nt resist one quick query here. In the scenario you describe, is it not the case that HBA drivers would only send out a Login on the first open of that target, and on subsequent opens would only bump an open_count or a ref_count of some kind rather than send a login on the wire on every open ? If that is correct, then, the scenario you describe would still cause disruption even with the modified semantics, only, when the WRITE is issued by the 2nd execution of the backup application (?). Regards, Santosh From owner-ips@ece.cmu.edu Fri May 25 04:57:53 2001 Received: from ece.cmu.edu ([128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id EAA13636 for ; Fri, 25 May 2001 04:57:48 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f4P5Uf116442 for ips-outgoing; Fri, 25 May 2001 01:30:41 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from e31.bld.us.ibm.com (e31.co.us.ibm.com [32.97.110.129]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f4P5UdH16438 for ; Fri, 25 May 2001 01:30:39 -0400 (EDT) Received: from westrelay02.boulder.ibm.com (westrelay02.boulder.ibm.com [9.99.140.23]) by e31.bld.us.ibm.com (8.9.3/8.9.3) with ESMTP id BAA21772; Fri, 25 May 2001 01:23:01 -0400 Received: from f4n49e (d03nm065h.boulder.ibm.com [9.99.140.49]) by westrelay02.boulder.ibm.com (8.11.1/NCO v4.96.1.0) with ESMTP id f4P5Uct104342; Thu, 24 May 2001 23:30:38 -0600 X-Priority: 1 (High) Importance: Normal Subject: Re: TCP Framing (considered helpful?) To: cbm@rose.hp.com Cc: ips@ece.cmu.edu X-Mailer: Lotus Notes Release 5.0.3 (Intl) 21 March 2000 Message-ID: From: "John Hufferd" Date: Thu, 24 May 2001 22:30:11 -0700 X-MIMETrack: Serialize by Router on D03NM065/03/M/IBM(Release 5.0.6 |December 14, 2000) at 05/24/2001 11:30:37 PM MIME-Version: 1.0 Content-type: text/plain; charset=us-ascii Sender: owner-ips@ece.cmu.edu Precedence: bulk Mallikarjun, We are massively mis-communicating. The only issue with Markers/Framing is when SW clients use Markers/Framing. The first issue is usually performance on the SW side. I do not think that Markers are a big performance issue. However, I do not have a good handle with Framing's overhead. However, the other major consideration is -- when will the functions be supported such that we can exploit them. The Markers are a function of iSCSI, and the Framing (Warp) is a function of TCP/IP, and hence a concern about the implementation window for relying on the Framing availability. Markers and Framing were never an issue with HW, we could always implement them when we create the HBAs. However, for the function to be useful in reducing the implementation cost of HBAs, the Clients must have functions that match. Many of these clients will be SW implementation of iSCSI on Desktops and Laptops. Therefore, the HW vendors will need to insure that there is universal support for either Framing or Markers so that the implementation on the HBA can exploit that capability. If it turns out that they can not depend on rapid implementation of the function on all the SW Clients, then the HW adapter must assume that the function is not universal and therefore, they can not depend on it, and must build a more expensive HW HBA. We can cause the Marker to be Mandated as Must Implemented, so we can assume universal availability of Markers. We can not mandate the implementation of Framing in all TCP/IP environments. Therefore, the HW HBAs can not assume that they can use that feature. Hence, they must build the more expensive HBA. Yes, I support Framing and Warp, but the above shows that it can not be used to permit the building of a lower cost HBA (for some long time). On the other hand if we can mandate Markers, then the lower cost HW HBA can be built, and then they can still negotiate higher performance by negotiating for Framing/Warp, and many times they will find a compatible partner and then they will function at their optimal rate. Perhaps we are on the same page now?? . . . John L. Hufferd Senior Technical Staff Member (STSM) IBM/SSG San Jose Ca (408) 256-0403, Tie: 276-0403, eFax: (408) 904-4688 Internet address: hufferd@us.ibm.com "Mallikarjun C." @ece.cmu.edu on 05/23/2001 02:54:39 PM Please respond to cbm@rose.hp.com Sent by: owner-ips@ece.cmu.edu To: ips@ece.cmu.edu cc: Subject: Re: TCP Framing (considered helpful?) John, Two comments. I may have misunderstood what you're saying since your initial comments imply framing=warp/RDMA/messageboundary, and towards the end, you seem to imply framing=markers. - It's true that implementing iSCSI-level markers does not require modifying TCP/IP stacks. But using the markers (I mean to use it to effectively steer) requires TCP/IP changes! It is a separate matter that it is offloaded onto the NIC. - You seem to implicitly suggest that WARP (when it becomes available) must go into host software stacks to be useful, and hence cannot be done in the right timeframe for iSCSI. Since one could envision offloading WARP onto the NIC as well, I assume you're hinting a) either at interoperability issues between software and hardware iSCSI implementations relying on WARP, b) or at interoperability issues between hardware and hardware implementations. iSCSI currently mandates a "no sync and steering" mode which ensures interoperability in either case. Perhaps you were really concerned about performance in case (b) then? -- Mallikarjun Mallikarjun Chadalapaka Networked Storage Architecture Network Storage Solutions Organization MS 5668 Hewlett-Packard, Roseville. cbm@rose.hp.com >Replies in text below (between [Huff] and [/Huff] ). > >. >. >. >John L. Hufferd >Senior Technical Staff Member (STSM) >IBM/SSG San Jose Ca >(408) 256-0403, Tie: 276-0403, eFax: (408) 904-4688 >Internet address: hufferd@us.ibm.com > > >Stephen Bailey @ece.cmu.edu on 05/21/2001 06:50:41 >AM > >Sent by: owner-ips@ece.cmu.edu > > >To: ips@ece.cmu.edu >cc: >Subject: Re: TCP Framing (considered helpful?) > > > >John, > >> I think we must depend on Markers to insure that everything can operate >at >> top speed, and at the lowest cost. > >A key question is whether markers actually ensure that everything >operates at `top speed, and at the lowest cost'. > >Matt thinks so. I (and, presumably those who wrote the framing >document) think not. > >[Huff] I do not think you can say that. I also support framing (Warp), as >a much more elegant solution, but I find it inappropriate to depend on it >actually happening, and being made available in the various OSs in the time >frame we need. I do believe, that over time it will be made available, and >is a better approach for all TCP/IP applications that can use it. [/Huff] > >My issue is not even with `lowest cost'. I don't believe markers will >allow you to run at top speed. Specifically: > 1) I doubt the feasibility of implementing the control required for > an eddy buffer (where you store data you can't place) at 10G. > Admittedly, the validity of this claim can't really be assessed > without actually working the implementation, so for 99% of the > list participants (myself included) this is a `yes it is, no it > isn't' point. > > [Huff] I believe this has had much more work done on it then you > think. I have personally stepped through the proposals from > several vendors that are working on this option for their HW HBAs. > Usually, because of the iSCSI PDU headers, the data/commands > can be placed directly into the SCSI Host buffers, almost every > time. Only when the PDU headers arrive slightly out of order > (do to normal routing) are the packets unable to be placed > directly into the Host buffer. And that requires some, but only > a small amount, of buffering space. > It is the packet drops that occur on PDU headers, and resultant > error retries, that cause the need for large amounts of "on > HBA/chip" buffering. > So by using Markers, these HW iSCSI HBAs can limit the amount of > buffering on the chip/HBAs. [/Huff] > > 2) an eddy buffer solution requires some substantial speed-up in > both the NIC data path, and MOST IMPORTANTLY: the host bus. In > order to unload the eddy buffer while still handling incoming > traffic at line rate, clearly the host bus bandwidth must be > > line rate. > > [Huff] This is not an effect of an eddy buffer solution, it is a > fact that every TCP/IP NIC has to deal with. Especially at the > new Speeds. Our current PCI buss will not support 10 Gigabit, further > PCI-X will not support it either, even PCI-DDR does not fully support > the full data rate. So it needs to rely on the TCP/IP window > management. The only other thing you can do is drop the packets. > this clearly makes the problem worse. [/Huff] > > >I know of at least one general purpose framed solution operating at >10G which has been available for >3 years (SGI's GSN/ST/XIO NIC). I'm >sure there are others. > >I can't imagine there's any argument that a framed solution would be >voted `most likely to run fast and be cheap'. Every storage network >and cluster interconnect has been designed that way since antiquity. > >The key tradeoff involves the OS vendors, and I'm wondering why we're >speaking for them. The question IS, how much more work is it to >introduce TCP framing over and above what is required to insert iSCSI >into their network framework. My experience from writing NIC and >storage drivers for many commercial UNIX-family OSes is: > 1) it's an easy and well defined process to insert a new SCSI > transport driver into the SCSI stack. > 2) it's hard and poorly defined process to insert ANYTHING into the > network stack. >[Huff] I think you are making my point. This is the problem with SW >Stacks. That is why I believe that it will take a very long time for >the various vendors to include such changes into their "bet you business" >TCP/IP SW Stacks. The point that Matt and I have been trying to make >is that most OS vendors are NOT creating the iSCSI HW HBAs (NICs). >These iSCSI HW HBAs (NICs) have the TCP/IP completely on the HBA, and >they have added the iSCSI processing also so that they can steer the >packets directly into the approprate SCSI Host buffers. Adding either >Markers or Framing into the iSCSI HW HBAs is not a big problem. It is >only a problem of getting Framing (timely) into Host TCP/IP Stacks. >[/Huff] > >Networking has historically been a user-mode activity. Architected >services are only provided to user mode programs. Kernel clients have >been few and far between and so are handled on a case-by-case basis. >For example NFS. Every OS has hacks to make NFS run fast, but they >are not stable interfaces for general purpose use. > >Even Solaris' SysV-derived STREAMS stack, which is intended precisely >to provide flexible, crisp interfaces for kernel network clients, does >not document the relevant (IP stack) intermodule interfaces. > >I know that there are more and more kernel network clients, but they >are coming either on fluid platforms (e.g. linux), in which case the >argument of `it'll take too long to get OS support' doesn't apply, or >they are vendor-supplied, in which case a performance iSCSI solution >in ANY form may take a while, and the choice of framing or markers >isn't going to make a difference. > >[Huff] I think you are saying something I agree with and something I >do not agree with. That is, that software changes to TCP/IP in the >various "Bet you Business" OSs, will take some time. However, it is >not true that new iSCSI device drivers will take very long. Two types >are being created today. By Cisco, IBM, Intel, etc. These types are >iSCSI DD that make calls to normal TCP/IP stacks, and the DD that >are being written by the iSCSI HW HBA vendors. These do not require >the OS vendor to do anything special. This is happening NOW, >(Check with CISCO, Intel, and IBM (me?)). The last thing we want >is to depend on a TCP/IP change to get in the >way of our momentum. [/Huff] > >I can't say squat about the architecture of Winsock, but the fact that >there is a Microsoft author of the framing proposal who seems very >serious about supporting framing and RDMA as quickly as possible >suggests that framing support should be available on Windows very >soon. > >[Huff] My following statements are not meant as a negative of Microsoft. >However, they and all producers of Key complicated new Software do >not quickly bring these to the general market in a way that is as >pleasing to HW vendors as HW vendors would like. > >I believe that Microsoft's heart is in the right place on this issue, >and that they will do the right thing with framing, over time. >But it is not clear in what release that will be shipped, nor what support >pack it will be included. Also it is not clear how the support >will be handled for current Win2k, WinNT etc. > >This is why I think we should have Framing a Must implement >and an Optional to use. It is the easiest thing for SW to >create, and brings the needed cost reduction to iSCSI HW and >it is completely under our (iSCSI protocol) control. >[/Huff] > > > >Steph > > > > From owner-ips@ece.cmu.edu Fri May 25 05:47:50 2001 Received: from ece.cmu.edu ([128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id FAA13897 for ; Fri, 25 May 2001 05:47:50 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f4P6EtN18888 for ips-outgoing; Fri, 25 May 2001 02:14:55 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from d12lmsgate-3.de.ibm.com (d12lmsgate-3.de.ibm.com [195.212.91.201]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f4P6EmH18880 for ; Fri, 25 May 2001 02:14:53 -0400 (EDT) Received: from d12relay01.de.ibm.com (d12relay01.de.ibm.com [9.165.215.22]) by d12lmsgate-3.de.ibm.com (1.0.0) with ESMTP id IAA252630 for ; Fri, 25 May 2001 08:14:37 +0200 From: julian_satran@il.ibm.com Received: from d12mta02.de.ibm.com (d12mta01_cs0 [9.165.222.237]) by d12relay01.de.ibm.com (8.8.8m3/NCO v4.96) with SMTP id IAA100180 for ; Fri, 25 May 2001 08:14:37 +0200 Received: by d12mta02.de.ibm.com(Lotus SMTP MTA v4.6.5 (863.2 5-20-1999)) id C1256A57.00223FBC ; Fri, 25 May 2001 08:14:05 +0200 X-Lotus-FromDomain: IBMIL@IBMDE To: ips@ece.cmu.edu Message-ID: Date: Fri, 25 May 2001 09:20:07 +0300 Subject: Re: iSCSI: Canonical Targets Mime-Version: 1.0 Content-type: text/plain; charset=us-ascii Content-Disposition: inline Sender: owner-ips@ece.cmu.edu Precedence: bulk I am not concerned about the amount of work the "discovery target" will generate for a builder. If the function is needed it is needed and will be done. The context however is not right. The discovery will surely imply more than the name - and will include various device characteristics not all of them apparent today. A generic discovery will have provisions to include them. iSCSI - most probably not. If all we want is a lightweight discovery mechanism - we should choose a lightweight discovery protocol not add a "lightweight feature" to iSCSI. Julo Stephen Bailey on 23-05-2001 05:08:16 Please respond to Stephen Bailey To: ips@ece.cmu.edu cc: Subject: Re: iSCSI: Canonical Targets Julian, > Why shouldn't we take the road of deferring the name discovery to > the specialized (iSCSI) part of a more general discovery protocol? The reason is, for a minimal, useful iSCSI configuration, we must not require any additional pieces of infrastructure beyond (Mark's term) table stakes for IP (e.g. DNS), an iSCSI initiator and an iSCSI target. Assuming we want to allow multitarget iSCSI endpoints in minimal configurations (I'm sure we do), there MUST be a way for the initiator to learn what targets are behind that endpoint from within the initiator's machine environment. The solution to this problem can't be `walk over to the target and read a bunch of numbers and type them in to the initiator'. > You don't expect the management applications to start using iSCSI for > discovery do you? Why not? When you say `management application' you seem to imply an application which would manage an arbitrary, possibly extremely large configuration (e.g. OpenView). Actual management applications range from simple to complex, and many configurations (particularly in the pilot phase that hopefully iSCSI will be entering) start very simple. I don't think anybody would argue that FCP PL-DA provided very complicated networks, yet every initiator implementation usually provided a program or something which simply spat out the WWNs (and probably the AL_PAs) for each target. Such a simple `management application' is going to be a lot like (perhaps IS) an existing SCSI control utility that can give you a list of buses, targets, LUNs, some inquiry data, and whatnot. Clearly, iSNS has nothing to say about this scenario. Are you proposing that SLP be used to supplant the existing simple directory mechanism in the iSCSI draft (and that it be mandatory to implement)? That might work. I'm not sure it makes sense, but if you think so, let's discuss it. In this case, multicast support (a typical objection to SLP) wouldn't be necessary, because you'd already know where the target is by configuration. SLP+iSCSI mechanisms would be offering the same service as the existing SendTargets but in a more ultimately scalable way. However, this proposal doesn't appear to solve any security problems. What security provisions does SLP have? Presumably the target would still authenticate the initiator in the SLP connection to ensure the initiator had rights to get the target list. That part is the same as the current proposal, so you're not saving the target any work. Steph From owner-ips@ece.cmu.edu Fri May 25 10:30:42 2001 Received: from ece.cmu.edu ([128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id KAA18852 for ; Fri, 25 May 2001 10:30:37 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f4P93eW08955 for ips-outgoing; Fri, 25 May 2001 05:03:40 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from d12lmsgate.de.ibm.com (d12lmsgate.de.ibm.com [195.212.91.199]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f4P93cH08950 for ; Fri, 25 May 2001 05:03:38 -0400 (EDT) Received: from d12relay01.de.ibm.com (d12relay01.de.ibm.com [9.165.215.22]) by d12lmsgate.de.ibm.com (1.0.0) with ESMTP id LAA54512 for ; Fri, 25 May 2001 11:03:26 +0200 From: julian_satran@il.ibm.com Received: from d12mta02.de.ibm.com (d12mta01_cs0 [9.165.222.237]) by d12relay01.de.ibm.com (8.8.8m3/NCO v4.96) with SMTP id LAA96720 for ; Fri, 25 May 2001 11:03:26 +0200 Received: by d12mta02.de.ibm.com(Lotus SMTP MTA v4.6.5 (863.2 5-20-1999)) id C1256A57.0031B59A ; Fri, 25 May 2001 11:02:57 +0200 X-Lotus-FromDomain: IBMIL@IBMDE To: ips@ece.cmu.edu Message-ID: Date: Fri, 25 May 2001 12:08:58 +0300 Subject: Re: iSCSI: Final bit/status clarification Mime-Version: 1.0 Content-type: text/plain; charset=us-ascii Content-Disposition: inline Sender: owner-ips@ece.cmu.edu Precedence: bulk I intended to say even if it does not include status. Will fix. Thanks, Julo "Eddy Quicksall" on 21-05-2001 19:26:13 Please respond to "Eddy Quicksall" To: Julian Satran/Haifa/IBM@IBMIL, ips@ece.cmu.edu cc: Subject: iSCSI: Final bit/status clarification Section 2.7.1 F (Final) Bit For incoming data, this bit is 1 for the last input data PDU associated with the command (even if it includes the status). Is a clarification implied by "even if it includes the status"? I don't see the need for a clarification here because I don't see any ambiguity. Would it make sense to remove it? Or, maybe it should say "even if it does not include the status". -------- mailto:Eddy@Quicksall.com From owner-ips@ece.cmu.edu Fri May 25 17:00:11 2001 Received: from ece.cmu.edu ([128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id RAA24930 for ; Fri, 25 May 2001 17:00:10 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f4PJBEW25354 for ips-outgoing; Fri, 25 May 2001 15:11:14 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from server1.NishanSystems.COM (smtp.nishansystems.com [216.217.36.162]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f4PJBC325348 for ; Fri, 25 May 2001 15:11:12 -0400 (EDT) Received: by smtp.nishansystems.com with Internet Mail Service (5.5.2653.19) id ; Fri, 25 May 2001 12:11:02 -0700 Message-ID: From: Charles Monia To: "Ips (E-mail)" Subject: Common Encapsulation: Proposal for time stamp Flag Date: Fri, 25 May 2001 12:10:55 -0700 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2653.19) Content-Type: text/plain; charset="iso-8859-1" Sender: owner-ips@ece.cmu.edu Precedence: bulk Hi: According to the encapsulation specification, the time fields are checked to see if a time stamp is present by testing each word to see if it is zero. From an implementation standpoint, a single flag bit in the common flags byte seems the more efficient alternative. Comments? Charles From owner-ips@ece.cmu.edu Fri May 25 17:01:09 2001 Received: from ece.cmu.edu ([128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id RAA24962 for ; Fri, 25 May 2001 17:01:08 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f4PJT4r26754 for ips-outgoing; Fri, 25 May 2001 15:29:04 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from gateway.sanlight.org (adsl-63-202-160-80.dsl.snfc21.pacbell.net [63.202.160.80]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f4PJT2326748 for ; Fri, 25 May 2001 15:29:02 -0400 (EDT) Received: from ljoy (10.0.0.18.lan.sanlight.net [10.0.0.18]) by gateway.sanlight.org (8.11.0/8.11.0) with SMTP id f4PKZk185078; Fri, 25 May 2001 13:35:46 -0700 (PDT) (envelope-from dotis@sanlight.net) From: "Douglas Otis" To: , Subject: RE: iSCSI boot Date: Fri, 25 May 2001 12:26:31 -0700 Message-ID: MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 (Normal) X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook IMO, Build 9.0.2416 (9.0.2911.0) In-Reply-To: <0F31E5C394DAD311B60C00E029101A07080155FE@corpmx9.isus.emc.com> Importance: Normal X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4522.1200 Sender: owner-ips@ece.cmu.edu Precedence: bulk Content-Transfer-Encoding: 7bit David, If there already is a means of discovering either LDAP or SLP in conjunction with DCHP together will some security features defined within the PXE specification, then adding iSCSI specific information is not really required. If you assume there is security present within LDAP and there is a defined schema, the ability to retrieve information related to things like ISID, Initiator Name, Target Name, mount point, etc can be made available through those standard services with security providing the initial filter. LDAP can store state as it is commonly used to keep password counts and the like. It is not a direct property of LDAP, but a well structured schema should make this task easier. I know that I will hear, "Send in the Draft" but until there is consensus as to what is needed and how it is to be used, it would be a likely futile venture. Doug > Somewhere buried in my "to-do" list is to provide some > guidance to the iSCSI boot draft authors about what > to do next. What follows are more of suggestions than > instructions and comments are welcome. > > - The draft will need to be split into two pieces, > an informational draft describing the overall > boot process and a standards-track document > that standardizes the DHCP option. The > informational draft might benefit from being > merged into the corresponding portion of the > naming and discovery draft, since much of what > it covers is how to discover the boot device. > > - The DHCP option draft would need to be coordinated > with the DHC WG. This need for a new DHCP option > may be a problem, as there aren't may option codes left: > (http://www.iana.org/assignments/bootp-dhcp-parameters). > Using DHCP to find SLP to find the boot device seems > both clumsy and an invitation to problems (one more > thing that can break and prevent booting), but I wonder > if there's a way to [ab]use existing DHCP option 17 > (Root Path) for this purpose. > > In any case, the use of DHCP will need to be pursued > with the DHCP WG via a document that describes only the > DHCP option to be standardized. This probably shouldn't > be undertaken until the iSCSI naming and discovery draft > is relatively stable so that we're certain about naming > formats and mechanisms (e.g., is "Send Targets" involved > in boot? - this has implications for the information passed > through DHCP). > > Comments? > --David > > --------------------------------------------------- > David L. Black, Senior Technologist > EMC Corporation, 42 South St., Hopkinton, MA 01748 > +1 (508) 435-1000 x75140 FAX: +1 (508) 497-8500 > black_david@emc.com Mobile: +1 (978) 394-7754 > --------------------------------------------------- > > From owner-ips@ece.cmu.edu Fri May 25 17:11:42 2001 Received: from ece.cmu.edu ([128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id RAA25105 for ; Fri, 25 May 2001 17:11:41 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f4PIVjE22349 for ips-outgoing; Fri, 25 May 2001 14:31:45 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from palrel2.hp.com (palrel2.hp.com [156.153.255.234]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f4PIVg322327 for ; Fri, 25 May 2001 14:31:42 -0400 (EDT) Received: from hpcuhe.cup.hp.com (hpcuhe.cup.hp.com [15.0.80.203]) by palrel2.hp.com (Postfix) with ESMTP id AB6FD1E78 for ; Fri, 25 May 2001 11:31:41 -0700 (PDT) Received: from cup.hp.com (santoshr@hpindhhm.cup.hp.com [15.8.80.197]) by hpcuhe.cup.hp.com (8.9.3 (PHNE_18979)/8.9.3 SMKit7.02) with ESMTP id LAA29532 for ; Fri, 25 May 2001 11:31:37 -0700 (PDT) Message-ID: <3B0EA531.E0E9F69B@cup.hp.com> Date: Fri, 25 May 2001 11:32:17 -0700 From: Santosh Rao Organization: Hewlett Packard, Cupertino. X-Mailer: Mozilla 4.7 [en] (X11; U; HP-UX B.11.00 9000/778) X-Accept-Language: en MIME-Version: 1.0 To: ips@ece.cmu.edu Subject: Re: Asynchronous Message: Session Termination References: Content-Type: multipart/mixed; boundary="------------73070B88D9D779DE583CA70B" Sender: owner-ips@ece.cmu.edu Precedence: bulk This is a multi-part message in MIME format. --------------73070B88D9D779DE583CA70B Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Julian, The modified text on "Async Message" seems to have missed out several aspects of the discussion on this thread : 1) It is useful to provide a parameter for the "target will drop this connection" and "target will drop all connections" to indicate the time, in seconds, after which the target would drop the connection[s]. This allows : - the target to warn the initiator ahead of time about an upcoming disruption so that initiator can take action accordingly. - Allows elimination of the "target requests logout" option since the "will drop connection/all connections" flavor provides the initiator an upper time bound after which the connection[s] are going to be dropped. This allows the initiator to attempt to quiesce the connection[s] prior to the expiry of that time period. The "target requests logout" option should be removed, if its equivalent functionality or better can be provided by the "will drop connection/all connections" flavors. 2) Suggest modification of the below wording : " If the initiator does not attempt to reconnect within the time specified by Parameter 3 or Parameter 3 is 0 the session is terminated - in which case no other responses should be expected from the target for outstanding commands on this session and the initiator should terminate them appropriately." to indicate that upon expiry of the LogoutLoginMaxTime (parameter 3), the target internally aborts all outstanding commands on that connection and the initiator should consider those I/Os implicitly aborted. No explicit Abort mechanisms are required from the initiator to the target upon expiry of LogoutLoginMaxTime to terminate these commands. 3) The "target will drop all connections" flavor description should indicate that the target is performing a session logout. I like the parameter descriptions provided by Mark [Bakke] for this flavor, namely : - Parameter2 indicates the time, in seconds, after which the target will drop all connections. - Parameter3 indicates the additional time, in seconds, after the expiry of Parameter2, that the initiator must wait before attempting to re-establish a session. The LogoutLoginMaxTime in the current defn of "target will drop all connections" (Parameter 3) provides little value, since the target should be aborting all oustanding I/O as a part of that session logout and not wait until LogoutLoginMaxTime. 4) Regarding : "Initiators can attempt to set/modify the values a target will use for Parameter2 and Parameter3. However - I am unclear if we should leave the initiator the degree of control implied by (innocuous looking) last statement." Since, LogoutLoginMaxTime & LogoutLoginMinTime are only applicable in the context of a target originated "connection drop" as indicated by the async events, these keys add little or no value in the login negotiation. In all cases, it is the time values provided by the target in the async events that are applicable and over-ride any login negotiated values for these keys. Hence, should these keys be removed from the login negotiation ? Regards, Santosh julian_satran@il.ibm.com wrote: > > Mathhew, Mark, Matt & Santosh: > > Here is the new text for 2.17.1 > > 1.1.1 iSCSI Event > > The codes used for iSCSI Asynchronous Messages (Events) are: > > 0 No iSCSI Event > 1 Target is being reset. > 2 Target requests Logout - the Parameter1 field indicates on what > CID. > 3 Target indicates it will drop the connection - the Parameter1 > field will indicate on what CID while the Parameter2 field indicates, > in seconds, the minimum time to wait before attempting to reconnect > and Parameter3 the maximum time to reconnect after the initial wait > (Parameter2). If the initiator does not attempt to reconnect within > the time specified by Parameter 3 or Parameter 3 is 0 no other > responses should be expected from the target for outstanding commands > on this connection and the initiator should terminate them > appropriately. A value of 0 for Parameter 2 indicates that reconnect > can be attempted immediately. > 4 Target indicates it will drop all connections - the Parameter2 > field indicates, in seconds, the minimum time to wait before > attempting to reconnect and Parameter3 the maximum time to reconnect > after the initial wait (Parameter2). If the initiator does not > attempt to reconnect within the time specified by Parameter 3 or > Parameter 3 is 0 the session is terminated - in which case no other > responses should be expected from the target for outstanding commands > on this session and the initiator should terminate them > appropriately. A value of 0 for Parameter 2 indicates that reconnect > can be attempted immediately. > > Initiators can attempt to set/modify the values a target will use for > Parameter2 and Parameter3 > > However - I am unclear if we should leave the initiator the degree of > control implied by (innocuous looking) last statement. > > Thanks for helping make this text better. > > Julo --------------73070B88D9D779DE583CA70B Content-Type: text/x-vcard; charset=us-ascii; name="santoshr.vcf" Content-Description: Card for Santosh Rao Content-Disposition: attachment; filename="santoshr.vcf" Content-Transfer-Encoding: 7bit begin:vcard n:Rao;Santosh tel;work:408-447-3751 x-mozilla-html:FALSE org:Hewlett Packard, Cupertino.;SISL adr:;;19420, Homestead Road, M\S 43LN, ;Cupertino.;CA.;95014.;USA. version:2.1 email;internet:santoshr@cup.hp.com title:Software Design Engineer x-mozilla-cpt:;21088 fn:Santosh Rao end:vcard --------------73070B88D9D779DE583CA70B-- From owner-ips@ece.cmu.edu Fri May 25 17:12:31 2001 Received: from ece.cmu.edu ([128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id RAA25127 for ; Fri, 25 May 2001 17:12:31 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f4PIsXq24009 for ips-outgoing; Fri, 25 May 2001 14:54:33 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from e31.bld.us.ibm.com (e31.co.us.ibm.com [32.97.110.129]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f4PIsR324002 for ; Fri, 25 May 2001 14:54:32 -0400 (EDT) Received: from westrelay02.boulder.ibm.com (westrelay02.boulder.ibm.com [9.99.140.23]) by e31.bld.us.ibm.com (8.9.3/8.9.3) with ESMTP id OAA84098 for ; Fri, 25 May 2001 14:46:49 -0400 Received: from f4n49e (d03nm065h.boulder.ibm.com [9.99.140.49]) by westrelay02.boulder.ibm.com (8.11.1/NCO v4.96.1.0) with ESMTP id f4PIsP3200954 for ; Fri, 25 May 2001 12:54:25 -0600 X-Priority: 1 (High) Importance: Normal Subject: Re: iSCSI: Canonical Targets To: "Julian Satran" Cc: ips@ece.cmu.edu X-Mailer: Lotus Notes Release 5.0.3 (Intl) 21 March 2000 Message-ID: From: "John Hufferd" Date: Fri, 25 May 2001 09:59:27 -0700 X-MIMETrack: Serialize by Router on D03NM065/03/M/IBM(Release 5.0.6 |December 14, 2000) at 05/25/2001 12:54:25 PM MIME-Version: 1.0 Content-type: text/plain; charset=us-ascii Sender: owner-ips@ece.cmu.edu Precedence: bulk Julian, I do not agree with you on the issue of Send Targets, it is not a discovery function as much as it is a reporting function (perhaps we should have called it Report Targets). It performs a function that is similar to Report LUNs. If you are saying that Send Targets should not be there, then that is an argument against Report LUNs also. I do not think that is a meaningful discussion. We have had to use the LUN 0 to get the LUN information which was a bit more clumsy than the way we are going about Send Targets, which will have an explicit way of telling the authorized requester, what the target knows, which is directly applicable to the conduct of the Initiator (It needs to know that it can reach this Target, and what portals it can use and which portals can be used for Multiple Connections per Session, etc). This is so important and so primitive of a function, I do not get the issue about having to have support for some other variety of Discovery functions, just to get an approprate Report on the Targets supported, and the Portal on which they can be reached. . . . John L. Hufferd Senior Technical Staff Member (STSM) IBM/SSG San Jose Ca (408) 256-0403, Tie: 276-0403, eFax: (408) 904-4688 Internet address: hufferd@us.ibm.com Julian Satran/Haifa/IBM@IBMIL@ece.cmu.edu on 05/24/2001 11:20:07 PM Sent by: owner-ips@ece.cmu.edu To: ips@ece.cmu.edu cc: Subject: Re: iSCSI: Canonical Targets I am not concerned about the amount of work the "discovery target" will generate for a builder. If the function is needed it is needed and will be done. The context however is not right. The discovery will surely imply more than the name - and will include various device characteristics not all of them apparent today. A generic discovery will have provisions to include them. iSCSI - most probably not. If all we want is a lightweight discovery mechanism - we should choose a lightweight discovery protocol not add a "lightweight feature" to iSCSI. Julo Stephen Bailey on 23-05-2001 05:08:16 Please respond to Stephen Bailey To: ips@ece.cmu.edu cc: Subject: Re: iSCSI: Canonical Targets Julian, > Why shouldn't we take the road of deferring the name discovery to > the specialized (iSCSI) part of a more general discovery protocol? The reason is, for a minimal, useful iSCSI configuration, we must not require any additional pieces of infrastructure beyond (Mark's term) table stakes for IP (e.g. DNS), an iSCSI initiator and an iSCSI target. Assuming we want to allow multitarget iSCSI endpoints in minimal configurations (I'm sure we do), there MUST be a way for the initiator to learn what targets are behind that endpoint from within the initiator's machine environment. The solution to this problem can't be `walk over to the target and read a bunch of numbers and type them in to the initiator'. > You don't expect the management applications to start using iSCSI for > discovery do you? Why not? When you say `management application' you seem to imply an application which would manage an arbitrary, possibly extremely large configuration (e.g. OpenView). Actual management applications range from simple to complex, and many configurations (particularly in the pilot phase that hopefully iSCSI will be entering) start very simple. I don't think anybody would argue that FCP PL-DA provided very complicated networks, yet every initiator implementation usually provided a program or something which simply spat out the WWNs (and probably the AL_PAs) for each target. Such a simple `management application' is going to be a lot like (perhaps IS) an existing SCSI control utility that can give you a list of buses, targets, LUNs, some inquiry data, and whatnot. Clearly, iSNS has nothing to say about this scenario. Are you proposing that SLP be used to supplant the existing simple directory mechanism in the iSCSI draft (and that it be mandatory to implement)? That might work. I'm not sure it makes sense, but if you think so, let's discuss it. In this case, multicast support (a typical objection to SLP) wouldn't be necessary, because you'd already know where the target is by configuration. SLP+iSCSI mechanisms would be offering the same service as the existing SendTargets but in a more ultimately scalable way. However, this proposal doesn't appear to solve any security problems. What security provisions does SLP have? Presumably the target would still authenticate the initiator in the SLP connection to ensure the initiator had rights to get the target list. That part is the same as the current proposal, so you're not saving the target any work. Steph From owner-ips@ece.cmu.edu Fri May 25 17:13:28 2001 Received: from ece.cmu.edu ([128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id RAA25158 for ; Fri, 25 May 2001 17:13:27 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f4PJ4j024871 for ips-outgoing; Fri, 25 May 2001 15:04:45 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from dogwood.cisco.com (dogwood.cisco.com [161.44.11.19]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f4PJ4g324865 for ; Fri, 25 May 2001 15:04:42 -0400 (EDT) Received: from cisco.com (mbakke@mbakke-lnx.cisco.com [161.44.68.87]) by dogwood.cisco.com (8.8.6 (PHNE_14041)/CISCO.SERVER.1.2) with ESMTP id PAA12421; Fri, 25 May 2001 15:00:44 -0400 (EDT) Message-ID: <3B0EAB25.7AAE223D@cisco.com> Date: Fri, 25 May 2001 13:57:41 -0500 From: Mark Bakke X-Mailer: Mozilla 4.72 [en] (X11; U; Linux 2.2.16-3.uid32 i686) X-Accept-Language: en, de MIME-Version: 1.0 To: Santosh Rao CC: ips@ece.cmu.edu Subject: Re: Asynchronous Message: Session Termination References: <3B0EA531.E0E9F69B@cup.hp.com> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-ips@ece.cmu.edu Precedence: bulk Content-Transfer-Encoding: 7bit Santosh- I agree with all of your proposed modifications; that's what I had in mind as well. Codes 3 and 4 will then do a better job of code 2 (target requests logout), so 2 can be removed. I also think that we should be more specific on "... will drop all connections" and say "... will drop all connections within the current session". I think that was implied, but it is probably better to explicitly state it. I agree with both you and Julian that the statement about initiators modifying these time values should be removed; I think that these time values are really up to the target, and we should remove them from the negotiation. -- Mark Santosh Rao wrote: > > Julian, > > The modified text on "Async Message" seems to have missed out several > aspects of the discussion on this thread : > > 1) It is useful to provide a parameter for the "target will drop this > connection" and "target will drop all connections" to indicate the time, > in seconds, after which the target would drop the connection[s]. > > This allows : > > - the target to warn the initiator ahead of time about an upcoming > disruption so that initiator can take action accordingly. > > - Allows elimination of the "target requests logout" option since the > "will drop connection/all connections" flavor provides the initiator an > upper time bound after which the connection[s] are going to be dropped. > This allows the initiator to attempt to quiesce the connection[s] prior > to the expiry of that time period. The "target requests logout" option > should be removed, if its equivalent functionality or better can be > provided by the "will drop connection/all connections" flavors. > > 2) Suggest modification of the below wording : > " If the initiator does not attempt to reconnect within the time > specified by Parameter 3 or Parameter 3 is 0 the session is terminated - > in which case no other responses should be expected from the target for > outstanding commands on this session and the initiator should terminate > them appropriately." > > to indicate that upon expiry of the LogoutLoginMaxTime (parameter 3), > the target internally aborts all outstanding commands on that connection > and the initiator should consider those I/Os implicitly aborted. No > explicit Abort mechanisms are required from the initiator to the target > upon expiry of LogoutLoginMaxTime to terminate these commands. > > 3) The "target will drop all connections" flavor description should > indicate that the target is performing a session logout. I like the > parameter descriptions provided by Mark [Bakke] for this flavor, namely > : > > - Parameter2 indicates the time, in seconds, after which the target will > drop all connections. > - Parameter3 indicates the additional time, in seconds, after the expiry > of Parameter2, that the initiator must wait before attempting to > re-establish a session. > > The LogoutLoginMaxTime in the current defn of "target will drop all > connections" (Parameter 3) provides little value, since the target > should be aborting all oustanding I/O as a part of that session logout > and not wait until LogoutLoginMaxTime. > > 4) Regarding : > "Initiators can attempt to set/modify the values a target will use for > Parameter2 and Parameter3. However - I am unclear if we should leave the > initiator the degree of control implied by (innocuous looking) last > statement." > > Since, LogoutLoginMaxTime & LogoutLoginMinTime are only applicable in > the context of a target originated "connection drop" as indicated by the > async events, these keys add little or no value in the login > negotiation. In all cases, it is the time values provided by the target > in the async events that are applicable and over-ride any login > negotiated values for these keys. Hence, should these keys be removed > from the login negotiation ? > > Regards, > Santosh > > julian_satran@il.ibm.com wrote: > > > > Mathhew, Mark, Matt & Santosh: > > > > Here is the new text for 2.17.1 > > > > 1.1.1 iSCSI Event > > > > The codes used for iSCSI Asynchronous Messages (Events) are: > > > > 0 No iSCSI Event > > 1 Target is being reset. > > 2 Target requests Logout - the Parameter1 field indicates on what > > CID. > > 3 Target indicates it will drop the connection - the Parameter1 > > field will indicate on what CID while the Parameter2 field indicates, > > in seconds, the minimum time to wait before attempting to reconnect > > and Parameter3 the maximum time to reconnect after the initial wait > > (Parameter2). If the initiator does not attempt to reconnect within > > the time specified by Parameter 3 or Parameter 3 is 0 no other > > responses should be expected from the target for outstanding commands > > on this connection and the initiator should terminate them > > appropriately. A value of 0 for Parameter 2 indicates that reconnect > > can be attempted immediately. > > 4 Target indicates it will drop all connections - the Parameter2 > > field indicates, in seconds, the minimum time to wait before > > attempting to reconnect and Parameter3 the maximum time to reconnect > > after the initial wait (Parameter2). If the initiator does not > > attempt to reconnect within the time specified by Parameter 3 or > > Parameter 3 is 0 the session is terminated - in which case no other > > responses should be expected from the target for outstanding commands > > on this session and the initiator should terminate them > > appropriately. A value of 0 for Parameter 2 indicates that reconnect > > can be attempted immediately. > > > > Initiators can attempt to set/modify the values a target will use for > > Parameter2 and Parameter3 > > > > However - I am unclear if we should leave the initiator the degree of > > control implied by (innocuous looking) last statement. > > > > Thanks for helping make this text better. > > > > Julo -- Mark A. Bakke Cisco Systems mbakke@cisco.com 763.398.1054 From owner-ips@ece.cmu.edu Fri May 25 17:24:54 2001 Received: from ece.cmu.edu ([128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id RAA25353 for ; Fri, 25 May 2001 17:24:54 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f4PFpwY09319 for ips-outgoing; Fri, 25 May 2001 11:51:58 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from d12lmsgate-2.de.ibm.com (d12lmsgate-2.de.ibm.com [195.212.91.200]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f4PFpu309314 for ; Fri, 25 May 2001 11:51:56 -0400 (EDT) Received: from d12relay01.de.ibm.com (d12relay01.de.ibm.com [9.165.215.22]) by d12lmsgate-2.de.ibm.com (1.0.0) with ESMTP id RAA170840 for ; Fri, 25 May 2001 17:51:44 +0200 From: julian_satran@il.ibm.com Received: from d12mta02.de.ibm.com (d12mta01_cs0 [9.165.222.237]) by d12relay01.de.ibm.com (8.8.8m3/NCO v4.96) with SMTP id RAA100340 for ; Fri, 25 May 2001 17:51:44 +0200 Received: by d12mta02.de.ibm.com(Lotus SMTP MTA v4.6.5 (863.2 5-20-1999)) id C1256A57.005716CC ; Fri, 25 May 2001 17:51:14 +0200 X-Lotus-FromDomain: IBMIL@IBMDE To: ips@ece.cmu.edu Message-ID: Date: Fri, 25 May 2001 18:57:11 +0300 Subject: Re: Asynchronous Message: Session Termination Mime-Version: 1.0 Content-type: text/plain; charset=us-ascii Content-Disposition: inline Sender: owner-ips@ece.cmu.edu Precedence: bulk Mathhew, Mark, Matt & Santosh: Here is the new text for 2.17.1 1.1.1 iSCSI Event The codes used for iSCSI Asynchronous Messages (Events) are: 0 No iSCSI Event 1 Target is being reset. 2 Target requests Logout - the Parameter1 field indicates on what CID. 3 Target indicates it will drop the connection - the Parameter1 field will indicate on what CID while the Parameter2 field indicates, in seconds, the minimum time to wait before attempting to reconnect and Parameter3 the maximum time to reconnect after the initial wait (Parameter2). If the initiator does not attempt to reconnect within the time specified by Parameter 3 or Parameter 3 is 0 no other responses should be expected from the target for outstanding commands on this connection and the initiator should terminate them appropriately. A value of 0 for Parameter 2 indicates that reconnect can be attempted immediately. 4 Target indicates it will drop all connections - the Parameter2 field indicates, in seconds, the minimum time to wait before attempting to reconnect and Parameter3 the maximum time to reconnect after the initial wait (Parameter2). If the initiator does not attempt to reconnect within the time specified by Parameter 3 or Parameter 3 is 0 the session is terminated - in which case no other responses should be expected from the target for outstanding commands on this session and the initiator should terminate them appropriately. A value of 0 for Parameter 2 indicates that reconnect can be attempted immediately. Initiators can attempt to set/modify the values a target will use for Parameter2 and Parameter3 However - I am unclear if we should leave the initiator the degree of control implied by (innocuous looking) last statement. Thanks for helping make this text better. Julo From owner-ips@ece.cmu.edu Fri May 25 17:29:05 2001 Received: from ece.cmu.edu ([128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id RAA25434 for ; Fri, 25 May 2001 17:29:04 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f4PGe5I13514 for ips-outgoing; Fri, 25 May 2001 12:40:05 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from d12lmsgate.de.ibm.com (d12lmsgate.de.ibm.com [195.212.91.199]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f4PGe3313509 for ; Fri, 25 May 2001 12:40:03 -0400 (EDT) Received: from d12relay02.de.ibm.com (d12relay02.de.ibm.com [9.165.215.23]) by d12lmsgate.de.ibm.com (1.0.0) with ESMTP id SAA289412 for ; Fri, 25 May 2001 18:39:56 +0200 From: julian_satran@il.ibm.com Received: from d12mta02.de.ibm.com (d12mta01_cs0 [9.165.222.237]) by d12relay02.de.ibm.com (8.8.8m3/NCO v4.96) with SMTP id SAA137240 for ; Fri, 25 May 2001 18:39:31 +0200 Received: by d12mta02.de.ibm.com(Lotus SMTP MTA v4.6.5 (863.2 5-20-1999)) id C1256A57.005B813C ; Fri, 25 May 2001 18:39:28 +0200 X-Lotus-FromDomain: IBMIL@IBMDE To: ips@ece.cmu.edu Message-ID: Date: Fri, 25 May 2001 19:45:34 +0300 Subject: Re: data steering Mime-Version: 1.0 Content-type: text/plain; charset=us-ascii Content-Disposition: inline Sender: owner-ips@ece.cmu.edu Precedence: bulk It outlines a general framework for synch and steering in which several options can fit (markers is the one already contained). They by no means meant to be only vendor-specific - on the contrary the more universal they are the better. Julo "Eddy Quicksall" on 21-05-2001 17:35:21 Please respond to "Eddy Quicksall" To: Julian Satran/Haifa/IBM@IBMIL, ips@ece.cmu.edu cc: Subject: data steering Section 1.2.8 Message Synchronization and Steering Using the suggestions in this section seems to mean vendor unique operation since, I think, it is saying that I should add addition information to the stream in order to steer it. Is that correct? mailto:Eddy@Quicksall.com From owner-ips@ece.cmu.edu Fri May 25 17:47:12 2001 Received: from ece.cmu.edu ([128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id RAA25750 for ; Fri, 25 May 2001 17:47:12 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f4PKLXV00843 for ips-outgoing; Fri, 25 May 2001 16:21:33 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from lightsand.com (alsv1-blk3-hfc-0251-d1db1add.rdc2.occa.coxatwork.com [209.219.26.221]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f4PKLW300839 for ; Fri, 25 May 2001 16:21:32 -0400 (EDT) Received: from muralipc (dhcp094.lightsand.com [192.168.1.94]) by lightsand.com (8.9.3+Sun/8.9.1) with SMTP id NAA04295; Fri, 25 May 2001 13:21:24 -0700 (PDT) From: "Murali Rajagopal" To: "Charles Monia" , "Ips \(E-mail\)" Subject: RE: Common Encapsulation: Proposal for time stamp Flag Date: Fri, 25 May 2001 13:22:48 -0700 Message-ID: MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 (Normal) X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook IMO, Build 9.0.2416 (9.0.2911.0) In-Reply-To: X-MimeOLE: Produced By Microsoft MimeOLE V5.00.2919.6600 Importance: Normal Sender: owner-ips@ece.cmu.edu Precedence: bulk Content-Transfer-Encoding: 7bit Charles: Your requirement stems from a ease of implementation and a flag bit although redundant would be usedful. Correct? -Murali -----Original Message----- From: owner-ips@ece.cmu.edu [mailto:owner-ips@ece.cmu.edu]On Behalf Of Charles Monia Sent: Friday, May 25, 2001 12:11 PM To: Ips (E-mail) Subject: Common Encapsulation: Proposal for time stamp Flag Hi: According to the encapsulation specification, the time fields are checked to see if a time stamp is present by testing each word to see if it is zero. From an implementation standpoint, a single flag bit in the common flags byte seems the more efficient alternative. Comments? Charles From owner-ips@ece.cmu.edu Fri May 25 17:48:19 2001 Received: from ece.cmu.edu ([128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id RAA25772 for ; Fri, 25 May 2001 17:48:19 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f4PKMUK00918 for ips-outgoing; Fri, 25 May 2001 16:22:30 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from ztxmail05.ztx.compaq.com (ztxmail05.ztx.compaq.com [161.114.1.209]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f4PKMS300910 for ; Fri, 25 May 2001 16:22:28 -0400 (EDT) Received: by ztxmail05.ztx.compaq.com (Postfix, from userid 12345) id 82F1B1DEE; Fri, 25 May 2001 15:22:22 -0500 (CDT) Received: from exchou-gh02.cca.cpqcorp.net (exchou-gh02.cca.cpqcorp.net [16.110.248.202]) by ztxmail05.ztx.compaq.com (Postfix) with ESMTP id 6807D1F3A; Fri, 25 May 2001 15:22:22 -0500 (CDT) Received: by exchou-gh02.cca.cpqcorp.net with Internet Mail Service (5.5.2652.78) id ; Fri, 25 May 2001 15:22:22 -0500 Message-ID: <31891B757C09184BBFEC5275F85D5595104D67@cceexc18.americas.cpqcorp.net> From: "Martin, Nick" To: "'julian_satran@il.ibm.com'" , ips@ece.cmu.edu Subject: RE: iSCSI: response to second login (with same ISID) Date: Fri, 25 May 2001 15:22:11 -0500 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2652.78) Content-Type: text/plain Sender: owner-ips@ece.cmu.edu Precedence: bulk Julian, I do not know how long it would take a target using a ping to determine that an old session is really dead. Some folks think it would be too long, thus they want to be able to "login unconditionally". Primarily I thought this would be used for initiators rebooting after ungraceful shutdown. (For those who can otherwise reboot quickly, but would stall on each iSCSI session not already recovered by the target.) I do not expect to want to use a "login unconditionally" option. (I can handle rejection ;) The scary thing to me was if an initiator could not try an ISID which was in use (intentionally, accidentally, or erroneously) without disrupting (logging out) its current user (a.k.a. option 2). It appears that if given a "login unconditionally" option, some folks would use it exclusively. If the initiator can specify the maximum time it would take a target to notice dead connections and sessions, then the argument that waiting for target recovery of the session would take too long presumably goes away. I am thinking about login negotiated parameters like MinIdleTime and MaxIdleTime values in seconds. These could specify respectively the connection idle interval after which a NO-OP (keep-alive) should be sent and expected, and the time with no traffic received on this connection which should be regarded as an implicit logout request due to connection failure. (This could cause an async event if detected by the target and a working connection remains in this session.) Better names can be chosen. Reasonable defaults might be 10 seconds and 60 seconds. There should be a recommendation like MaxIdleTime should always be at least 3 times MinIdleTime. Such parameters would probably exist within the Target and the Initiator, even if they are not negotiated or exchanged. The intent is that an Initiator that cares, can limit Target connection recovery delays. These parameters set by the Initiator for the Target, would be known to both. A broken connection could be detected by each at approximately the same time. The keep-alive NO-OP's could be pings so that both directions would be kept alive by a single exchange. If the Initiator wants to be responsible for keep-alives, it should do them before the specified MinIdleTime, if it wants the Target to do them, it may need to do them also, slightly after the specified MinIdleTime, but well before MaxIdleTime. My networking is weak, so I am not sure whether there should be a separate (shorter) time specified for the maximum time to wait for the reply to a ping, or how long to wait to retry it. I am not clear on the interactions with TCP and its timeouts and retries. Is there a maximum time a ping could take and still get through? Is there a maximum time a busy iSCSI target or initiator would be expected to delay before sending a reply to a ping? Does this sound like a good idea at all or not? Thanks, Nick From owner-ips@ece.cmu.edu Fri May 25 17:50:36 2001 Received: from ece.cmu.edu ([128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id RAA25790 for ; Fri, 25 May 2001 17:50:35 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f4PKQ0E01179 for ips-outgoing; Fri, 25 May 2001 16:26:00 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from server1.NishanSystems.COM (smtp.nishansystems.com [216.217.36.162]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f4PKPw301175 for ; Fri, 25 May 2001 16:25:58 -0400 (EDT) Received: by smtp.nishansystems.com with Internet Mail Service (5.5.2653.19) id ; Fri, 25 May 2001 13:25:53 -0700 Message-ID: From: Charles Monia To: "Ips (E-mail)" Subject: RE: Common Encapsulation: Proposal for time stamp Flag Date: Fri, 25 May 2001 13:25:50 -0700 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2653.19) Content-Type: text/plain; charset="iso-8859-1" Sender: owner-ips@ece.cmu.edu Precedence: bulk > -----Original Message----- > From: Murali Rajagopal [mailto:muralir@lightsand.com] > Sent: Friday, May 25, 2001 1:23 PM > To: Charles Monia; Ips (E-mail) > Subject: RE: Common Encapsulation: Proposal for time stamp Flag > > > Charles: > > Your requirement stems from a ease of implementation and a > flag bit although > redundant would be usedful. Correct? > Yes. I see it as an optimization. Charles > -Murali > > -----Original Message----- > From: owner-ips@ece.cmu.edu [mailto:owner-ips@ece.cmu.edu]On Behalf Of > Charles Monia > Sent: Friday, May 25, 2001 12:11 PM > To: Ips (E-mail) > Subject: Common Encapsulation: Proposal for time stamp Flag > > > Hi: > > According to the encapsulation specification, the time fields > are checked to > see if a time stamp is present by testing each word to see if > it is zero. > From an implementation standpoint, a single flag bit in the > common flags > byte seems the more efficient alternative. > > Comments? > > Charles > From owner-ips@ece.cmu.edu Fri May 25 18:01:11 2001 Received: from ece.cmu.edu ([128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id SAA25986 for ; Fri, 25 May 2001 18:01:11 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f4PJsXB28827 for ips-outgoing; Fri, 25 May 2001 15:54:33 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from mercury.Sun.COM (mercury.Sun.COM [192.9.25.1]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f4PJsV328823 for ; Fri, 25 May 2001 15:54:32 -0400 (EDT) Received: from phys-ha6nwka.ebay.sun.com ([129.149.1.82]) by mercury.Sun.COM (8.9.3+Sun/8.9.3) with ESMTP id MAA14998 for ; Fri, 25 May 2001 12:54:30 -0700 (PDT) Received: from ebay.sun.com (jetsun [129.149.169.37]) by phys-ha6nwka.ebay.sun.com (8.9.3+Sun/8.9.3/ENSMAIL,v2.1p1) with ESMTP id MAA02831 for ; Fri, 25 May 2001 12:54:29 -0700 (PDT) Message-ID: <3B0EB875.DAE93D98@ebay.sun.com> Date: Fri, 25 May 2001 12:54:29 -0700 From: David Robinson Organization: Sun Microsystems, Inc. X-Mailer: Mozilla 4.7 [en] (X11; I; SunOS 5.8 sun4u) X-Accept-Language: en MIME-Version: 1.0 To: ips@ece.cmu.edu Subject: Re: iSCSI boot References: Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-ips@ece.cmu.edu Precedence: bulk Content-Transfer-Encoding: 7bit Douglas Otis wrote: > If there already is a means of discovering either LDAP or SLP in conjunction > with DCHP together will some security features defined within the PXE > specification, then adding iSCSI specific information is not really > required. If you assume there is security present within LDAP and there is > a defined schema, the ability to retrieve information related to things like > ISID, Initiator Name, Target Name, mount point, etc can be made available > through those standard services with security providing the initial filter. > LDAP can store state as it is commonly used to keep password counts and the > like. It is not a direct property of LDAP, but a well structured schema > should make this task easier. I know that I will hear, "Send in the Draft" > but until there is consensus as to what is needed and how it is to be used, > it would be a likely futile venture. I think you are over complicating things Doug. We already have a well defined standard for Network Adapters to discover their identity and their "root" storage device using DHCP. All that is really needed by the IPS WG is to define the syntax and semantics of the string that indicates where the iSCSI target is. While LDAP provides a lot of features and can easily be used as the directory service behind a DHCP server (and in fact is often is), it is highly unlikely that vendors will embed LDAP into the PROMs of their adapters to retrieve a simple string that can just as easily be served using their existing DHCP/PXE PROMs. Security is actively being worked on the the DHCP community so that is something that iSCSI can leverage. (draft-ietf-dhc-authentication-16.txt) So I won't say "Send in a Draft" but instead "The IESG won't let us reinvent existing protocols". -David From owner-ips@ece.cmu.edu Fri May 25 18:33:37 2001 Received: from ece.cmu.edu ([128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id SAA26484 for ; Fri, 25 May 2001 18:33:36 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f4PKbKS02100 for ips-outgoing; Fri, 25 May 2001 16:37:20 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from dogwood.cisco.com (dogwood.cisco.com [161.44.11.19]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f4PKbJ302095 for ; Fri, 25 May 2001 16:37:19 -0400 (EDT) Received: from cisco.com (mbakke@mbakke-lnx.cisco.com [161.44.68.87]) by dogwood.cisco.com (8.8.6 (PHNE_14041)/CISCO.SERVER.1.2) with ESMTP id QAA06775; Fri, 25 May 2001 16:37:11 -0400 (EDT) Message-ID: <3B0EC1BF.576E00D@cisco.com> Date: Fri, 25 May 2001 15:34:07 -0500 From: Mark Bakke X-Mailer: Mozilla 4.72 [en] (X11; U; Linux 2.2.16-3.uid32 i686) X-Accept-Language: en, de MIME-Version: 1.0 To: Julian Satran , IPS Subject: iSCSI: More draft-06 comments Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-ips@ece.cmu.edu Precedence: bulk Content-Transfer-Encoding: 7bit Julian- Here are a few more comments and proposed clarifications on draft-06. I apologize if some of them overlap with comments made already. -- Mark 1.2.4 - We'd like to add a clarification on the use of the keyword "none" versus silently ignoring the key. If a target does not explicitly send back =none, the initiator will not be sure whether the target selected none, or didn't understand the option. On each offered key, there are four choices a target can make: = =none =reject =NotUnderstood Given these options, the responding party MUST NOT silently ignore a key. The alternative is to remove the NotUnderstood value and have silently ignoring a key to mean the same thing, although it's probably best to be explicit. BTW, are the reserved words case-sensitive? The following is our understanding of what this section means: 1.2.4 Text Mode Negotiation During login and thereafter some session or connection parameters are negotiated through an exchange of textual information. In "list" negotiation, the offering party sends a list of values for a key in its order of preference. The general format of text negotiation is: Offer-> =,,..., Answer-> = | "none" | "reject" | "NotUnderstood" There are four possible answers: = The responding party answers with the first value from the list it supports and is allowed to use for the specific initiator. =none The responding party supports none of the values in the list. "none" is also a valid keyword within the Offer, for example: DataDigest=crc-32C,none means that the offering party would prefer using crc-32c, but will accept "none" as well. =reject If a target is not supporting, or not allowed to use with a specific initiator, any of the offered options, it may use the value "reject". The values "none" and "reject" are reserved and must be used only as described here. =NotUnderstood If the responding party did not understand or recognize a key, it must indicate this with NotUnderstood. The values "none", "reject", and "NotUnderstood" are reserved and must be used only as described here. They are case-insensitive. The responding party MUST NOT silently ignore any keys sent by the offering party, and MUST respond with one of the four above options. In "numerical" negotiations, the offering and responding party state a numerical value. The result of the negotiation is key dependent; usually the lower or the higher of the two values is used. 2.1 - Need to say that padding is not included in the length field. Should also specify a MUST for the pad byte value. How about: iSCSI PDUs are padded to an integer number of 4 byte words. The value of the pad bytes must be 0. The pad bytes are not included in the DataSegmentLength field. 2.2.1 - Also say here that the length fields do not include the digests. How about: Optional header and data digests protect the integrity and authenticity of header and data, respectively. The digests, if present, are located, respectively, after the header and PDU-specific data and include the padding bytes. The lengths of the digests are not included in the header or data length fields. 2.2.2.5 - TotalAHSLength - How about: Total length of all AHS header segments in 4 byte words including padding if any. This length field does not include header digests. 2.2.2.6 - Need to specify whether DataSegmentLength includes the data digest. How about: This is the data segment payload length in bytes. This length does not include either padding or data digests. 2.2.3.1 - Need clearer description of drop bit: B7 - Drop Bit - if set to one, the implementation may ignore this AHS if it is not understood; if set to zero, the implementation must reject this PDU if this AHS type is not understood. 2.8.3 - Need maximum size for a key name, and a harder maximum for the value; how about adding: Key names MUST NOT exceed 64 characters; key values MUST NOT exceed 255 characters. This would replace the sentence "No key SHOULD contain..." 2.8.3 - I don't think that we need or want a lot of flexibility with regard to text value representation. In particular, I think that it's up to each key definition whether to use character, decimal, or hexadecimal representations, and that each key definition must pick just one of these for everyone to use. There's just no reason for one implementation to be able to send a key in one format, and another implementation to be able to send the same key in another format; that will just make interoperability harder. We would like to see he statement about characters strings and binary numbers tightened up to read: Character strings are represented as plain UTF-8 text. Numeric values with maximum values less than 32 bits should be represented as decimal numbers, but may be represented in hex if documented for the particular key. All binary values greater than 32 bits must be represented in hexadecimal. Hexadecimal numbers are case-insensitive; either upper or lower case must be accepted. The use of either decimal or hexadecimal (but not both) is specified separately for each key. As this format is then implied, no prefixes such as "0x" are necessary. 2.8.3 - (not its string representation) - I think that the limit should be on the string representation, not on the converted value. 2.8.3 - We need to have the discussion on whether text commands and responses are done all in software, and whether the size of a text response really matters. We'll bring this up on the list separately. 2.10 - Login command - Need to put an X in the first bit of the message header for the restart bit. 2.18.2 - If you want a useful example of a SCSI event, here's one that could be added: One example using the SCSI Asynchronous Event would report that LUN data has changed if, for instance, a new LUN is added to the target. Here is what the sense data would contain: senseData[0] = 0x71; // Error code. senseData[2] = 0x06; // Sense key: UNIT ATTENTION senseData[12]= 0x3F; // ASC (REPORTED LUNS DATA HAS CHANGED) senseData[13]= 0x0E; // ASCQ The DataSegmentLength in this example would be 14 (decimal). Appendices - It would be nice to use a different numbering scheme that starts over with each appendix; such as A.1, A.2, B.1, etc. Is this a limitation of the word doc? B.01 - Need to specify bit reflection and byte ordering for the CRC-32C. Should add an example message and the expected CRC for interoperability purposes. I can put together a message if you like. B.03 - Should have an example for a target that doesn't support authentication and an initiator that insists on authentication. B.03 - This section should be broken into separate headings for SRP, CHAP, etc. to make them easier to find. B.03 - (p 108) A lot of fields are shown as numbers; this section should specify which numbers are in hex and which (if any) are in decimal. B.03 - SRP uses "U=" for the name key; CHAP uses "N=" for its name key. They could be made the same for consistency, but it's not really a problem. E - Should explicitly define what "Who" means in the intro to appendix E. (e.g. who == which role can send the key). Perhaps "role" would be a better word. E - The login and digest keys are missing from appendix E; I think it would be worth describing all of the keys here. E - Each key should describe the exact format of its value, and state whether it is a hex, decimal, UTF-8 string, etc. E.23 - DataPDULength - we need to clarify that if a session has negotiated a DataPDULength, that the initiator and target MUST NOT send data PDUs of less than this length, except for the last PDU (or only PDU) of a command. This is what an implementation is likely to do anyway, and will make it practical to keep the iSCSI data CRC end-to-end, since the data CRC covers a PDU, and not a command's worth of data. -- Mark A. Bakke Cisco Systems mbakke@cisco.com 763.398.1054 From owner-ips@ece.cmu.edu Fri May 25 19:09:01 2001 Received: from ece.cmu.edu ([128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id TAA26933 for ; Fri, 25 May 2001 19:09:00 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f4PLKoB05132 for ips-outgoing; Fri, 25 May 2001 17:20:50 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from dogwood.cisco.com (dogwood.cisco.com [161.44.11.19]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f4PLKm305127 for ; Fri, 25 May 2001 17:20:48 -0400 (EDT) Received: from cisco.com (mbakke@mbakke-lnx.cisco.com [161.44.68.87]) by dogwood.cisco.com (8.8.6 (PHNE_14041)/CISCO.SERVER.1.2) with ESMTP id RAA24430; Fri, 25 May 2001 17:20:34 -0400 (EDT) Message-ID: <3B0ECBEB.D9A5D63F@cisco.com> Date: Fri, 25 May 2001 16:17:31 -0500 From: Mark Bakke X-Mailer: Mozilla 4.72 [en] (X11; U; Linux 2.2.16-3.uid32 i686) X-Accept-Language: en, de MIME-Version: 1.0 To: Glen Turner CC: ips@ece.cmu.edu Subject: Re: Internationalisation issue References: <3B0B0CC2.88CB88B2@aarnet.edu.au> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-ips@ece.cmu.edu Precedence: bulk Content-Transfer-Encoding: 7bit Glen- The IESG wouldn't let us specify an iSCSI name as a URN, so we are just going to let this draft expire. Anyway, the case-insentive rule still applies, and is specified in the naming & discovery document. Basically, an iSCSI name is treated as an opaque string; the only valid operation on an iSCSI name (other than display) is comparison. We had to specify a method to compare iSCSI names. Since they are text strings, and since text strings are transcribable, it is less problematic for users if they are case-insensitive. However, you are correct that once we have gone from ASCII to UTF-8, comparison is not as simple. On my linux system, it looks like I have some wide character toupper() and tolower() routines which would probably do the trick after converting the UTF-8 into 32-bit Unicode, but I haven't looked at the code that goes with them or its complexity. Our possible solutions are: - Make iSCSI names case-sensitive This is easy, but not as nice for the user, especially when manual configuration is performed. - Make iSCSI names case-insensitive for all character sets. This is probably the ideal from a user's point of view. I would be interested in comments as to how much this is needed (it's what is currently specified) or even if it is needed, who plans to support it. This is the hardest one to implement. Some character sets have three cases (lower, upper and title) as well. Jim H had written up some stuff on how this works in the current Naming & Discovery document. I guess the question here is whether it's worth going to the trouble to give the user the ultimate solution. The comparison method would be: 1. If the high bit is not set on any of the characters in one of the iSCSI names to be compared, just do a strcasecmp(). 2. Convert the iSCSI names from UTF-8 to 32-bit Unicode. 3. Use wcscasecmp() to compare the names. The above assumes the GNU/Linux library calls. Again, I haven't looked at how much code this is, but it's freely available and would work. - Make iSCSI names case-insensitive within the ASCII character range, and case-senstive for everything else. This means that non-ASCII characters have to be made case- sensitive, which is fine for many users, but may not be as nice for those using other character sets. It would be easy to implement, since normal toupper() and tolower() functions can be used. - Make iSCSI names case-sensitive, but mandate that they are all transmitted and stored in lower case. This makes it the user interface's problem to do the tolower() on whatever character sets it supports. It's easy for the iSCSI implementation. However, an iSCSI name displayed on "the other side" of a connection would lose any case typed by the user. This might be the best compromise. - Make iSCSI names ASCII This eliminates many character sets, and will restrict methods of producing unique names in various languages. It would also not handle the multilingual DNS names when they happen. However, since we do have an Alias that is a UTF-8 string and doesn't have to be compared, perhaps nobody plans to generate an iSCSI name based on non-ASCII characters? Any thoughts? I would be interested in the following opinions from the list: 1. Do you or your customers see value in using non-ASCII character sets to generate iSCSI names, or is the use of Alias enough? 2. Do you or your customers see value in preserving the original case in which an iSCSI name was generated, or would it be fine just to specify that they are always transferred in lower case? 3. Is the procedure for comparing these names too much to expect from a small target? -- Mark Glen Turner wrote: > > > Rules for Lexical Equivalence: > > > > The entire URN is case-insensitive. > > Why? This implies that devices now need to carry casing tables > tables for multi-lingual support. > > If the field is case sensitive then adding multilingual support > is simply a matter of specifiying UTF-8 and no ongoing work is > needed to track multilingual DNS. > > Regards, > Glen -- Mark A. Bakke Cisco Systems mbakke@cisco.com 763.398.1054 From owner-ips@ece.cmu.edu Fri May 25 19:11:20 2001 Received: from ece.cmu.edu ([128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id TAA26982 for ; Fri, 25 May 2001 19:11:20 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f4PLkeZ07142 for ips-outgoing; Fri, 25 May 2001 17:46:40 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from dogwood.cisco.com (dogwood.cisco.com [161.44.11.19]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f4PLkd307138 for ; Fri, 25 May 2001 17:46:39 -0400 (EDT) Received: from cisco.com (mbakke@mbakke-lnx.cisco.com [161.44.68.87]) by dogwood.cisco.com (8.8.6 (PHNE_14041)/CISCO.SERVER.1.2) with ESMTP id RAA19087 for ; Fri, 25 May 2001 17:46:33 -0400 (EDT) Message-ID: <3B0ED202.3E0ED5E4@cisco.com> Date: Fri, 25 May 2001 16:43:30 -0500 From: Mark Bakke X-Mailer: Mozilla 4.72 [en] (X11; U; Linux 2.2.16-3.uid32 i686) X-Accept-Language: en, de MIME-Version: 1.0 To: IPS Subject: iSCSI: Logout command, or The Initiator's new close() Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-ips@ece.cmu.edu Precedence: bulk Content-Transfer-Encoding: 7bit Section 2.14 (the logout command) is not clear on how the logout command and response work when the logout request is send on a connection for which it requests termination. We should probably specify the TCP close behavior of the initiator and target. The initiator can send the logout request, and either close the current connection, half-close the connection and wait for the logout response (like HTTP/1.0), or leave it open, and close the connection upon receiving the response or the FIN from the target. Upon receiving the logout request, the target can either close the connection immediately, send the logout response and then close the connection, or send the logout response and wait for the initiator to close the connection upon its receipt of the logout response. Logout would work best if the initiators and targets all did this in the same manner, choosing one of the above behaviors for the initiator, and a compatible one for the target. The most straight-forward choice might be: Initiator sends the logout request, and simply waits for the response without closing the connection. The target sends the logout response, and closes its end of the connection. The initiator receives the logout response, and the FIN from the target, and closes its end of the connection. Any opinions on whether this is best? This choice does not require the initiator to know that the connection that is being closed is the one it sent the logout response on, but does require the target to send the logout response before it actually closes the connection. Another behavior that might work would be to say that the target closes all of its connections and does cleanup before sending the logout response, and if the connection on which the request came in is gone, it does not send a response. An initiator whose connection is closed after sending the logoout request can assume that the request worked. Anyway, I think that it would be good to clarify this. -- Mark A. Bakke Cisco Systems mbakke@cisco.com 763.398.1054 From owner-ips@ece.cmu.edu Fri May 25 19:11:39 2001 Received: from ece.cmu.edu ([128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id TAA26993 for ; Fri, 25 May 2001 19:11:38 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f4PLh1706833 for ips-outgoing; Fri, 25 May 2001 17:43:01 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from mxic2.us.dg.com (mxic2.us.dg.com [128.221.31.40]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f4PLgx306824 for ; Fri, 25 May 2001 17:42:59 -0400 (EDT) Received: by mxic2.us.dg.com with Internet Mail Service (5.5.2653.19) id ; Fri, 25 May 2001 17:42:21 -0400 Message-ID: <0F31E5C394DAD311B60C00E029101A070801560B@corpmx9.isus.emc.com> From: Black_David@emc.com To: cmonia@NishanSystems.com, ips@ece.cmu.edu Subject: RE: Common Encapsulation: Stale Frame Detection in an IP fabric Date: Fri, 25 May 2001 17:42:53 -0400 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2653.19) Content-Type: text/plain; charset="iso-8859-1" Sender: owner-ips@ece.cmu.edu Precedence: bulk Question for Charles and the WG: This proposal is based on synchronizing each node to the same wall clock reference (i.e., SNTP server). Is there any interest in just synchronizing the timestamps between the two ends of an FCIP link without requiring interaction with an external time source? A possible downside is that an implementation that can support multiple FCIP links may need to maintain a separate time offset (from its internal time source) for each FCIP link. NOTE: This is a question and not to be taken as a suggestion from a co-chair. Thanks, --David > -----Original Message----- > From: Charles Monia [SMTP:cmonia@NishanSystems.com] > Sent: Wednesday, May 23, 2001 6:53 PM > To: Ips (E-mail) > Subject: Common Encapsulation: Stale Frame Detection in an IP fabric > > Hi: > > This note is a strawman proposal for detecting stale FC frames using the > features present in the common encapsulation specification. > > The spec includes provisions for time tagging a frame in order to detect > frames that have exceeded the lifetime guarantees expected of a Fibre > Channel fabric (R_A_TOV). > > A fibre channel fabric must guarantee that the maximum time in flight for > any frame will not exceed the limit specified by the R_A_TOV parameter. In > FC fabrics, R_A_TOV is defined by the fabric and supplied to the N_PORT in > the fabric login 'accept' response. The default value is 10 seconds. > Failure to meet this guarantee may result in stale frames associated with > defunct transactions. These frames, if received outside the R_A_TOV > window, > could cause failures and data corruption. > > In a closed FC fabric, the R_A_TOV limit is guaranteed by switch element > design and control of the fabric topology. There is no enforcement > mechanism. In contrast, such control may not be possible in an IP fabric. > In most cases, the IP network will, almost certainly, consist of > heterogeneous components and the user cannot be assumed to have full > control > over the infrastructure and its topology. For that reason some explicit > way > to enforce the flight time guarantee must be provided. The FC frame > encapsulation format provides the means for this enforcement at the edges > of > the IP network by allocating space for a time stamp which is applied when > the frame is injected into the IP network and may be checked by the > receiving node. > > The following is a proposal for managing time base synchronization at the > TCP end nodes. > > The protocol for synchronizing an end node time base is SNTP. In order to > insure that all nodes are time-aligned, they should obtain the address of > a > reference SNTP server via a hard-wired address or by querying an > appropriate > repository via SLP or some other protocol. If multiple SNTP server > addresses are provided, the servers must be synchronized. Alternatively, a > multicast group address may be used in support of operation in Anycast > mode. > Implementation of Anycast mode is as specified in RFC 2030, including the > precautions defined in that document. Multicast mode should not be used. > > An SNTP server may use any one of the time reference sources listed in RFC > 2030. The resolution of the time reference MUST be 125 milliseconds or > better. > > If support for stale frame detection by a node is optional, a node that > does not enforce the flight time limit shall set the time stamp field in > the > encapsulation header of an outgoing frame to 0,0 and shall ignore the > contents of the timestamp for incoming frames. A node that supports stale > frame detection shall implement the time stamp with a precision of 0.125 > seconds or better. > > With regard to the time base, the node is in either the synchronized or > unsychronized state. When in the unsynhronized state, the node shall > > a) Set the time stamp field to 0,0 for all outgoing frames > b) Ignore the time stamp field for all incoming frames. > > When in the synchronized state, the node shall: > > a) Set the time stamp field for each outgoing frame in accordance with > the > node's internal time base > b) Check the time stamp field of each incoming frame. > c) If the incoming frame has a time stamp of 0,0, the receiving node > shall > not test the frame to determine if it is stale. > d) If the incoming frame has a non-zero time stamp, the receiving node > shall compute the time in flight and compare it against the limit > specified > for the IP fabric. > e) If the result in step (d) exceeds the enforcement time limit, the > frame > shall be discarded. Otherwise, the frame shall be accepted. > > A node enters the synchronized state upon receiving a successful response > to > an SNTP query. > > A node shall enter the unsynchronized state: > > a) Upon power up and before the succesful completion of an SNTP query > b) When an unsuccesful SNTP query occurs. > > > Setting the enforcement time limit. > > In general, > > 2* E_D_TOV < Stale Frame time limit < R_A_TOV > > Where E_D_TOV is the FC time limit between expected events such as the > arrival of two successive frames in a sequence. > > A rule of thumb for the stale frame time limit is .5 * R_A_TOV. > > Charles > > Charles Monia > Senior Technology Consultant > Nishan Systems > email: cmonia@nishansystems.com > voice: (408) 519-3986 > fax: (408) 435-8385 From owner-ips@ece.cmu.edu Fri May 25 19:18:50 2001 Received: from ece.cmu.edu ([128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id TAA27064 for ; Fri, 25 May 2001 19:18:50 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f4PLXpT06130 for ips-outgoing; Fri, 25 May 2001 17:33:51 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from gateway.sanlight.org (adsl-63-202-160-80.dsl.snfc21.pacbell.net [63.202.160.80]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f4PLXn306126 for ; Fri, 25 May 2001 17:33:50 -0400 (EDT) Received: from ljoy (10.0.0.18.lan.sanlight.net [10.0.0.18]) by gateway.sanlight.org (8.11.0/8.11.0) with SMTP id f4PMee185167; Fri, 25 May 2001 15:40:40 -0700 (PDT) (envelope-from dotis@sanlight.net) From: "Douglas Otis" To: "David Robinson" , Subject: RE: iSCSI boot Date: Fri, 25 May 2001 14:31:23 -0700 Message-ID: MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 (Normal) X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook IMO, Build 9.0.2416 (9.0.2911.0) In-Reply-To: <3B0EB875.DAE93D98@ebay.sun.com> Importance: Normal X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4522.1200 Sender: owner-ips@ece.cmu.edu Precedence: bulk Content-Transfer-Encoding: 7bit David, I think you misunderstand where this code would exist. It would not be within the DHCP/PXE PROM code. Use the existing environment and previously defined parameters in conjunction with existing servers and existing protocols. Doug > Douglas Otis wrote: > > If there already is a means of discovering either LDAP or SLP > in conjunction > > with DCHP together will some security features defined within the PXE > > specification, then adding iSCSI specific information is not really > > required. If you assume there is security present within LDAP > and there is > > a defined schema, the ability to retrieve information related > to things like > > ISID, Initiator Name, Target Name, mount point, etc can be made > available > > through those standard services with security providing the > initial filter. > > LDAP can store state as it is commonly used to keep password > counts and the > > like. It is not a direct property of LDAP, but a well structured schema > > should make this task easier. I know that I will hear, "Send > in the Draft" > > but until there is consensus as to what is needed and how it is > to be used, > > it would be a likely futile venture. > > I think you are over complicating things Doug. We already have a well > defined > standard for Network Adapters to discover their identity and their > "root" > storage device using DHCP. All that is really needed by the IPS WG is > to define > the syntax and semantics of the string that indicates where the iSCSI > target is. > > While LDAP provides a lot of features and can easily be used as the > directory > service behind a DHCP server (and in fact is often is), it is highly > unlikely > that vendors will embed LDAP into the PROMs of their adapters to > retrieve > a simple string that can just as easily be served using their existing > DHCP/PXE > PROMs. > > Security is actively being worked on the the DHCP community so that > is something that iSCSI can leverage. > (draft-ietf-dhc-authentication-16.txt) > > So I won't say "Send in a Draft" but instead "The IESG won't let us > reinvent existing protocols". > > -David > From owner-ips@ece.cmu.edu Fri May 25 19:21:58 2001 Received: from ece.cmu.edu ([128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id TAA27106 for ; Fri, 25 May 2001 19:21:57 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f4PLauF06362 for ips-outgoing; Fri, 25 May 2001 17:36:56 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from mail2.microsoft.com (mail2.microsoft.com [131.107.3.124]) by ece.cmu.edu (8.11.0/8.10.2) with SMTP id f4PLC1304521 for ; Fri, 25 May 2001 17:12:02 -0400 (EDT) Received: from 157.54.9.104 by mail2.microsoft.com (InterScan E-Mail VirusWall NT); Fri, 25 May 2001 13:49:06 -0700 (Pacific Daylight Time) Received: from red-imc-01.redmond.corp.microsoft.com ([157.54.9.102]) by inet-imc-02.redmond.corp.microsoft.com with Microsoft SMTPSVC(5.0.2195.2883); Fri, 25 May 2001 13:49:16 -0700 Received: from win-imc-01.wingroup.windeploy.ntdev.microsoft.com ([157.54.0.39]) by red-imc-01.redmond.corp.microsoft.com with Microsoft SMTPSVC(5.0.2195.2883); Fri, 25 May 2001 13:49:13 -0700 Received: from win-msg-02.wingroup.windeploy.ntdev.microsoft.com ([157.54.0.134]) by win-imc-01.wingroup.windeploy.ntdev.microsoft.com with Microsoft SMTPSVC(5.0.2195.2966); Fri, 25 May 2001 13:48:37 -0700 content-class: urn:content-classes:message MIME-Version: 1.0 Content-Type: text/plain; charset="US-ASCII" Subject: RE: iSCSI: response to second login (with same ISID) X-MimeOLE: Produced By Microsoft Exchange V6.0.4712.0 Date: Fri, 25 May 2001 13:48:37 -0700 Message-ID: <8CC03F47967BF14D9710887723FADDB62DCF61@win-msg-02.wingroup.windeploy.ntdev.microsoft.com> Thread-Topic: iSCSI: response to second login (with same ISID) Thread-Index: AcDiZ5M+nhQM07N7SOuOmTiqkocMiQC8/oWw From: "Lakshmi Ramasubramanian" To: "Santosh Rao" , "KRUEGER,MARJORIE (HP-Roseville,ex1)" Cc: X-OriginalArrivalTime: 25 May 2001 20:48:37.0569 (UTC) FILETIME=[12739310:01C0E55C] Content-Transfer-Encoding: 8bit X-MIME-Autoconverted: from quoted-printable to 8bit by ece.cmu.edu id f4PLC2304522 Sender: owner-ips@ece.cmu.edu Precedence: bulk Content-Transfer-Encoding: 8bit How is a target driver suppose to respond to a login from an initiator, when another initiator already has a session going on with the given target? Is the target suppose to reject the second login (say, with an error "device busy" or something simialar)? Thanks! -lakshmi -----Original Message----- From: Santosh Rao [mailto:santoshr@cup.hp.com] Sent: Monday, May 21, 2001 5:39 PM To: KRUEGER,MARJORIE (HP-Roseville,ex1) Cc: ips@ece.cmu.edu Subject: Re: iSCSI: response to second login (with same ISID) "KRUEGER,MARJORIE (HP-Roseville,ex1)" wrote: > Since > persistent reservations are SCSI layer things, lets try to find a way > to keep their implementation from forcing unnecessary limitations at > the iSCSI layer. The use of init. name+ISID+target name for > identifying persistent reservations needs further discussion before we > allow it to creep into iSCSI protocol rules. I've very uncomfortable > with treating ISID like a fixed address just because SCSI persistent > reservations don't have a sufficient SCSI layer mechanism. A fixed port identifier of some form is a basic O.S. requirement to be able to persistently bind the LUNs it sees to some form of device files. This binding is based on either a physical port identifier or name (ex : pSCSI target id, FC nport_id, FC port WWN) or in the case of logical service delivery ports like iSCSI to a logical port identifier such as the ISID. A fixed address usage is not only for persistent reservation semantics. The host O.S. SCSI stacks need *something* that allows them to persistently bind device files to the same LUNs on each reboot. The ISID is one form of achieving this. > Seems like this will force all kinds of layering > violations into iSCSI. Not clear why this is so. Consider the ISID as a logical port identifier that id considered as a SCSI transport layer endpoint. Regards, Santosh From owner-ips@ece.cmu.edu Fri May 25 20:28:42 2001 Received: from ece.cmu.edu ([128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id UAA27864 for ; Fri, 25 May 2001 20:28:42 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f4PM6ND08564 for ips-outgoing; Fri, 25 May 2001 18:06:23 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from lightsand.com (alsv1-blk3-hfc-0251-d1db1add.rdc2.occa.coxatwork.com [209.219.26.221]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f4PM6L308557 for ; Fri, 25 May 2001 18:06:21 -0400 (EDT) Received: from muralipc (dhcp094.lightsand.com [192.168.1.94]) by lightsand.com (8.9.3+Sun/8.9.1) with SMTP id PAA07520 for ; Fri, 25 May 2001 15:06:14 -0700 (PDT) From: "Murali Rajagopal" To: "Ips \(E-mail\)" Subject: FCIP Device Discovery Date: Fri, 25 May 2001 15:07:38 -0700 Message-ID: MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 (Normal) X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook IMO, Build 9.0.2416 (9.0.2911.0) In-Reply-To: X-MimeOLE: Produced By Microsoft MimeOLE V5.00.2919.6600 Importance: Normal Sender: owner-ips@ece.cmu.edu Precedence: bulk Content-Transfer-Encoding: 7bit The consensus within the IPS FCIP sub WG for device discovery is to allow a FCIP device to be statically or dynamically configured with the IP addresses of other partcipating FCIP devices. For dynamic discovery the FCIP shall use SLPv2. -Murali Rajagopal Technical Coordinator IPS FCIP WG From owner-ips@ece.cmu.edu Fri May 25 20:34:44 2001 Received: from ece.cmu.edu ([128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id UAA27954 for ; Fri, 25 May 2001 20:34:43 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f4PMhgI10950 for ips-outgoing; Fri, 25 May 2001 18:43:42 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from mxic1.isus.emc.com ([168.159.129.100]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f4PMhf310946 for ; Fri, 25 May 2001 18:43:41 -0400 (EDT) Received: by mxic1.isus.emc.com with Internet Mail Service (5.5.2653.19) id ; Fri, 25 May 2001 18:45:14 -0400 Message-ID: <0F31E5C394DAD311B60C00E029101A0708015612@corpmx9.isus.emc.com> From: Black_David@emc.com To: muralir@lightsand.com, ips@ece.cmu.edu Subject: RE: FCIP Device Discovery Date: Fri, 25 May 2001 18:43:31 -0400 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2653.19) Content-Type: text/plain Sender: owner-ips@ece.cmu.edu Precedence: bulk For procedural reasons, a WG chair needs to do consensus calls, so let me state that I believe Murali's statement reflects WG consensus and ask those who disagree to comment on the list. Thanks, --David > -----Original Message----- > From: Murali Rajagopal [SMTP:muralir@lightsand.com] > Sent: Friday, May 25, 2001 6:08 PM > To: Ips (E-mail) > Subject: FCIP Device Discovery > > The consensus within the IPS FCIP sub WG for device discovery is to allow > a > FCIP device to be statically or dynamically configured with the IP > addresses > of other partcipating FCIP devices. For dynamic discovery the FCIP shall > use > SLPv2. > > -Murali Rajagopal > > Technical Coordinator > IPS FCIP WG > From owner-ips@ece.cmu.edu Fri May 25 23:20:48 2001 Received: from ece.cmu.edu ([128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id XAA01150 for ; Fri, 25 May 2001 23:20:48 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f4Q17wi19389 for ips-outgoing; Fri, 25 May 2001 21:07:58 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from palrel2.hp.com (palrel2.hp.com [156.153.255.234]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f4Q17u319385 for ; Fri, 25 May 2001 21:07:56 -0400 (EDT) Received: from hpcuhe.cup.hp.com (hpcuhe.cup.hp.com [15.0.80.203]) by palrel2.hp.com (Postfix) with ESMTP id E2D971EDD; Fri, 25 May 2001 18:07:55 -0700 (PDT) Received: from cup.hp.com (santoshr@hpindhhm.cup.hp.com [15.8.80.197]) by hpcuhe.cup.hp.com (8.9.3 (PHNE_18979)/8.9.3 SMKit7.02) with ESMTP id SAA04331; Fri, 25 May 2001 18:07:51 -0700 (PDT) Message-ID: <3B0F020F.76D933B7@cup.hp.com> Date: Fri, 25 May 2001 18:08:31 -0700 From: Santosh Rao Organization: Hewlett Packard, Cupertino. X-Mailer: Mozilla 4.7 [en] (X11; U; HP-UX B.11.00 9000/778) X-Accept-Language: en MIME-Version: 1.0 To: Lakshmi Ramasubramanian Cc: ips@ece.cmu.edu Subject: Re: iSCSI: response to second login (with same ISID) References: <8CC03F47967BF14D9710887723FADDB62DCF61@win-msg-02.wingroup.windeploy.ntdev.microsoft.com> Content-Type: multipart/mixed; boundary="------------18B4D3D173648901FCB4C6BA" Sender: owner-ips@ece.cmu.edu Precedence: bulk This is a multi-part message in MIME format. --------------18B4D3D173648901FCB4C6BA Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Why would the target want to reject the login, unless it is not capable of [or does not wish to] speak to multiple initiators ? In the case of most targets, they should be capable of handling multiple initiators and therefore, should accept the login. If the target is incapable of supporting multiple initiators or does not wish to allow multiple initiators (ex : iSCSI tape drive preventing concurrent access), then, it can reject the login with a status class/response of 0x0205. (Target Conflict). - Santosh Lakshmi Ramasubramanian wrote: > > How is a target driver suppose to respond to a login from an initiator, > when > another initiator already has a session going on with the given target? > Is the target > suppose to reject the second login (say, with an error "device busy" or > something simialar)? --------------18B4D3D173648901FCB4C6BA Content-Type: text/x-vcard; charset=us-ascii; name="santoshr.vcf" Content-Description: Card for Santosh Rao Content-Disposition: attachment; filename="santoshr.vcf" Content-Transfer-Encoding: 7bit begin:vcard n:Rao;Santosh tel;work:408-447-3751 x-mozilla-html:FALSE org:Hewlett Packard, Cupertino.;SISL adr:;;19420, Homestead Road, M\S 43LN, ;Cupertino.;CA.;95014.;USA. version:2.1 email;internet:santoshr@cup.hp.com title:Software Design Engineer x-mozilla-cpt:;21088 fn:Santosh Rao end:vcard --------------18B4D3D173648901FCB4C6BA-- From owner-ips@ece.cmu.edu Fri May 25 23:22:31 2001 Received: from ece.cmu.edu ([128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id XAA01161 for ; Fri, 25 May 2001 23:22:31 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f4Q1nFI21853 for ips-outgoing; Fri, 25 May 2001 21:49:15 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from server1.NishanSystems.COM (smtp.nishansystems.com [216.217.36.162]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f4Q1nE321849 for ; Fri, 25 May 2001 21:49:14 -0400 (EDT) Received: by smtp.nishansystems.com with Internet Mail Service (5.5.2653.19) id ; Fri, 25 May 2001 18:49:09 -0700 Message-ID: From: Charles Monia To: ips@ece.cmu.edu Subject: RE: Common Encapsulation: Stale Frame Detection in an IP fabric Date: Fri, 25 May 2001 18:49:06 -0700 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2653.19) Content-Type: text/plain; charset="iso-8859-1" Sender: owner-ips@ece.cmu.edu Precedence: bulk Hi David: How do you see clock synchronization happening? Does the FCIP end point use an SNTP handshake? Charles > -----Original Message----- > From: Black_David@emc.com [mailto:Black_David@emc.com] > Sent: Friday, May 25, 2001 2:43 PM > To: cmonia@NishanSystems.com; ips@ece.cmu.edu > Subject: RE: Common Encapsulation: Stale Frame Detection in > an IP fabric > > > Question for Charles and the WG: > > This proposal is based on synchronizing each node > to the same wall clock reference (i.e., SNTP server). > Is there any interest in just synchronizing the > timestamps between the two ends of an FCIP link > without requiring interaction with an external > time source? > > A possible downside is that an implementation > that can support multiple FCIP links may need > to maintain a separate time offset (from its > internal time source) for each FCIP link. > > NOTE: This is a question and not to be taken > as a suggestion from a co-chair. > > Thanks, > --David > From owner-ips@ece.cmu.edu Sat May 26 08:45:33 2001 Received: from ece.cmu.edu ([128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id IAA18769 for ; Sat, 26 May 2001 08:45:33 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f4Q9Wtq27490 for ips-outgoing; Sat, 26 May 2001 05:32:55 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from d12lmsgate-3.de.ibm.com (d12lmsgate-3.de.ibm.com [195.212.91.201]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f4Q9Wr327486 for ; Sat, 26 May 2001 05:32:53 -0400 (EDT) Received: from d12relay01.de.ibm.com (d12relay01.de.ibm.com [9.165.215.22]) by d12lmsgate-3.de.ibm.com (1.0.0) with ESMTP id LAA32766 for ; Sat, 26 May 2001 11:32:47 +0200 From: julian_satran@il.ibm.com Received: from d12mta02.de.ibm.com (d12mta01_cs0 [9.165.222.237]) by d12relay01.de.ibm.com (8.8.8m3/NCO v4.96) with SMTP id LAA87966 for ; Sat, 26 May 2001 11:32:47 +0200 Received: by d12mta02.de.ibm.com(Lotus SMTP MTA v4.6.5 (863.2 5-20-1999)) id C1256A58.0034646A ; Sat, 26 May 2001 11:32:15 +0200 X-Lotus-FromDomain: IBMIL@IBMDE To: ips@ece.cmu.edu Message-ID: Date: Sat, 26 May 2001 12:38:18 +0300 Subject: Re: iSCSI: More draft-06 comments Mime-Version: 1.0 Content-type: text/plain; charset=us-ascii Content-Disposition: inline Sender: owner-ips@ece.cmu.edu Precedence: bulk comments in text - Thanks, Julo Mark Bakke on 25-05-2001 23:34:07 Please respond to Mark Bakke To: Julian Satran/Haifa/IBM@IBMIL, IPS cc: Subject: iSCSI: More draft-06 comments Julian- Here are a few more comments and proposed clarifications on draft-06. I apologize if some of them overlap with comments made already. -- Mark 1.2.4 - We'd like to add a clarification on the use of the keyword "none" versus silently ignoring the key. If a target does not explicitly send back =none, the initiator will not be sure whether the target selected none, or didn't understand the option. On each offered key, there are four choices a target can make: = =none =reject =NotUnderstood Given these options, the responding party MUST NOT silently ignore a key. The alternative is to remove the NotUnderstood value and have silently ignoring a key to mean the same thing, although it's probably best to be explicit. BTW, are the reserved words case-sensitive? The following is our understanding of what this section means: 1.2.4 Text Mode Negotiation During login and thereafter some session or connection parameters are negotiated through an exchange of textual information. In "list" negotiation, the offering party sends a list of values for a key in its order of preference. The general format of text negotiation is: Offer-> =,,..., Answer-> = | "none" | "reject" | "NotUnderstood" There are four possible answers: = The responding party answers with the first value from the list it supports and is allowed to use for the specific initiator. =none The responding party supports none of the values in the list. "none" is also a valid keyword within the Offer, for example: DataDigest=crc-32C,none means that the offering party would prefer using crc-32c, but will accept "none" as well. =reject If a target is not supporting, or not allowed to use with a specific initiator, any of the offered options, it may use the value "reject". The values "none" and "reject" are reserved and must be used only as described here. =NotUnderstood If the responding party did not understand or recognize a key, it must indicate this with NotUnderstood. The values "none", "reject", and "NotUnderstood" are reserved and must be used only as described here. They are case-insensitive. The responding party MUST NOT silently ignore any keys sent by the offering party, and MUST respond with one of the four above options. In "numerical" negotiations, the offering and responding party state a numerical value. The result of the negotiation is key dependent; usually the lower or the higher of the two values is used. +++ in fact (after many debates) the current text reads: 1.1.1 1.1.1 Text Mode Negotiation During login and thereafter some session or connection parameters are negotiated through an exchange of textual information. In "list" negotiation, the offering party sends a list of values for a key in its order of preference. The responding party answers with the first value from the list it supports and is allowed to use for the specific initiator. The value "none" MUST always be used to indicate a missing function. However, none is a valid selection only if it is explicitly offered and MAY be selected by omission (i.e., =none MAY be omitted). If a target is not supporting, or not allowed to use with a specific initiator, any of the offered options, it may use the value "reject". The values "none" and "reject" are reserved and must be used only as described here. Any key not understood is answered with "NotUnderstood". The general format of text negotiation is: Offer-> =,,..., Answer-> =|reject|NotUnderstood In "numerical" negotiations, the offering and responding party state a numerical value. The result of the negotiation is key dependent; frequently the lower or the higher of the two values is used. Although the initiator is the requesting party and controls the request-response initiation and termination the target can offer key=value pairs of its own as part of a sequence and not only in response to an identical key=value pair offered by the initiator. ++++ although this text has no ambiguity there is still a raging debate between the proponents of an always explicit response (Matt and - in some covoluted form Steph - and those like Steve Senum who whould like a hiatus signaling a default. The current text let you set none by default. This can be changed by removing the " and MAY be selected by omission... " +++ 2.1 - Need to say that padding is not included in the length field. Should also specify a MUST for the pad byte value. How about: iSCSI PDUs are padded to an integer number of 4 byte words. The value of the pad bytes must be 0. The pad bytes are not included in the DataSegmentLength field. +++ 2.1 reads now: 1.1 iSCSI PDU Length and Padding iSCSI PDUs are padded to an integer number of 4 byte words. The padding bytes should be 0. The length is handled separately is the length parts: 1.1.1.1 TotalAHSLength Total length of all AHS header segments in 4 byte words including padding if any. 1.1.1.2 DataSegmentLength This is the data segment payload length in bytes (excluding padding). 2.2.1 - Also say here that the length fields do not include the digests. How about: Optional header and data digests protect the integrity and authenticity of header and data, respectively. The digests, if present, are located, respectively, after the header and PDU-specific data and include the padding bytes. The lengths of the digests are not included in the header or data length fields. +++ the PDU outline in 2.2 is clear enough. The negative specification (does not include like) is unbounded. I will include some words in 2.2.1 +++ 2.2.2.5 - TotalAHSLength - How about: Total length of all AHS header segments in 4 byte words including padding if any. This length field does not include header digests. 2.2.2.6 - Need to specify whether DataSegmentLength includes the data digest. How about: This is the data segment payload length in bytes. This length does not include either padding or data digests. 2.2.3.1 - Need clearer description of drop bit: B7 - Drop Bit - if set to one, the implementation may ignore this AHS if it is not understood; if set to zero, the implementation must reject this PDU if this AHS type is not understood. +++ will fix +++ 2.8.3 - Need maximum size for a key name, and a harder maximum for the value; how about adding: Key names MUST NOT exceed 64 characters; key values MUST NOT exceed 255 characters. This would replace the sentence "No key SHOULD contain..." +++ OK except that i'll make it bytes to account (limit) for I18N too. It will read: The data length of a text command or response SHOULD be less than 4096 bytes. Key names MUST NOT exceed 64 bytes. Key values MUST NOT exceed 255 characters. ++++ 2.8.3 - I don't think that we need or want a lot of flexibility with regard to text value representation. In particular, I think that it's up to each key definition whether to use character, decimal, or hexadecimal representations, and that each key definition must pick just one of these for everyone to use. There's just no reason for one implementation to be able to send a key in one format, and another implementation to be able to send the same key in another format; that will just make interoperability harder. We would like to see he statement about characters strings and binary numbers tightened up to read: Character strings are represented as plain UTF-8 text. Numeric values with maximum values less than 32 bits should be represented as decimal numbers, but may be represented in hex if documented for the particular key. All binary values greater than 32 bits must be represented in hexadecimal. Hexadecimal numbers are case-insensitive; either upper or lower case must be accepted. The use of either decimal or hexadecimal (but not both) is specified separately for each key. As this format is then implied, no prefixes such as "0x" are necessary. +++ Numbers are often copied from various sources. Why should we limit them? There is no real gain and implementations will have anyhow to have both conversions+++ 2.8.3 - (not its string representation) - I think that the limit should be on the string representation, not on the converted value. ++???+++ 2.8.3 - We need to have the discussion on whether text commands and responses are done all in software, and whether the size of a text response really matters. We'll bring this up on the list separately. +++ go ahead +++ 2.10 - Login command - Need to put an X in the first bit of the message header for the restart bit. +++ will fix +++ 2.18.2 - If you want a useful example of a SCSI event, here's one that could be added: One example using the SCSI Asynchronous Event would report that LUN data has changed if, for instance, a new LUN is added to the target. Here is what the sense data would contain: senseData[0] = 0x71; // Error code. senseData[2] = 0x06; // Sense key: UNIT ATTENTION senseData[12]= 0x3F; // ASC (REPORTED LUNS DATA HAS CHANGED) senseData[13]= 0x0E; // ASCQ The DataSegmentLength in this example would be 14 (decimal). +++ do we need an example? ++++ Appendices - It would be nice to use a different numbering scheme that starts over with each appendix; such as A.1, A.2, B.1, etc. Is this a limitation of the word doc? +++ I have a tool problem here ... and was unable to fix it up to now (Word!) +++ B.01 - Need to specify bit reflection and byte ordering for the CRC-32C. Should add an example message and the expected CRC for interoperability purposes. I can put together a message if you like. +++ I don't know what reflection is. If bit and byte ordering are specified do we need something more or is it a software specific issue +++ B.03 - Should have an example for a target that doesn't support authentication and an initiator that insists on authentication. +++ Will consider - however it looks like a case of reject for which there are examples +++ B.03 - This section should be broken into separate headings for SRP, CHAP, etc. to make them easier to find. +++ I will try but I am not sure I'll manage an unfriendly tool +++ B.03 - (p 108) A lot of fields are shown as numbers; this section should specify which numbers are in hex and which (if any) are in decimal. +++ ??? +++ B.03 - SRP uses "U=" for the name key; CHAP uses "N=" for its name key. They could be made the same for consistency, but it's not really a problem. +++ I think we took the names from the referred RFCs - but we will check again +++ E - Should explicitly define what "Who" means in the intro to appendix E. (e.g. who == which role can send the key). Perhaps "role" would be a better word. +++ I'll replace it with "Who can send:" +++ E - The login and digest keys are missing from appendix E; I think it would be worth describing all of the keys here. +++ They are all in the security appendix +++ E - Each key should describe the exact format of its value, and state whether it is a hex, decimal, UTF-8 string, etc. +++ I really don't see a need to limit further freedom of expression :-) +++ E.23 - DataPDULength - we need to clarify that if a session has negotiated a DataPDULength, that the initiator and target MUST NOT send data PDUs of less than this length, except for the last PDU (or only PDU) of a command. This is what an implementation is likely to do anyway, and will make it practical to keep the iSCSI data CRC end-to-end, since the data CRC covers a PDU, and not a command's worth of data. +++ you mean a data sequence (on R2T there can be several)? I assume most of implementation will do that for efficiency reasons but why should we mandate it? +++ -- Mark A. Bakke Cisco Systems mbakke@cisco.com 763.398.1054 From owner-ips@ece.cmu.edu Sat May 26 09:45:56 2001 Received: from ece.cmu.edu ([128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id JAA19170 for ; Sat, 26 May 2001 09:45:55 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f4QCr1s08256 for ips-outgoing; Sat, 26 May 2001 08:53:01 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from d12lmsgate.de.ibm.com (d12lmsgate.de.ibm.com [195.212.91.199]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f4QCr0308252 for ; Sat, 26 May 2001 08:53:00 -0400 (EDT) Received: from d12relay02.de.ibm.com (d12relay02.de.ibm.com [9.165.215.23]) by d12lmsgate.de.ibm.com (1.0.0) with ESMTP id OAA214996 for ; Sat, 26 May 2001 14:52:52 +0200 From: julian_satran@il.ibm.com Received: from d12mta02.de.ibm.com (d12mta01_cs0 [9.165.222.237]) by d12relay02.de.ibm.com (8.8.8m3/NCO v4.96) with SMTP id OAA72060 for ; Sat, 26 May 2001 14:52:25 +0200 Received: by d12mta02.de.ibm.com(Lotus SMTP MTA v4.6.5 (863.2 5-20-1999)) id C1256A58.0046B6BE ; Sat, 26 May 2001 14:52:22 +0200 X-Lotus-FromDomain: IBMIL@IBMDE To: ips@ece.cmu.edu Message-ID: Date: Sat, 26 May 2001 15:58:26 +0300 Subject: Re: iSCSI: Logout command, or The Initiator's new close() Mime-Version: 1.0 Content-type: text/plain; charset=us-ascii Content-Disposition: inline Sender: owner-ips@ece.cmu.edu Precedence: bulk Mark, I don't think that we should dictate implementation. As we assume that nothing is sent after Logout (should we spell this out) and we have to wait for the logout response we can do a half close and wait or wait and do a full close after seeing the Logout response. The first is faster the second is simpler. Regards, Julo Mark Bakke on 26-05-2001 00:43:30 Please respond to Mark Bakke To: IPS cc: Subject: iSCSI: Logout command, or The Initiator's new close() Section 2.14 (the logout command) is not clear on how the logout command and response work when the logout request is send on a connection for which it requests termination. We should probably specify the TCP close behavior of the initiator and target. The initiator can send the logout request, and either close the current connection, half-close the connection and wait for the logout response (like HTTP/1.0), or leave it open, and close the connection upon receiving the response or the FIN from the target. Upon receiving the logout request, the target can either close the connection immediately, send the logout response and then close the connection, or send the logout response and wait for the initiator to close the connection upon its receipt of the logout response. Logout would work best if the initiators and targets all did this in the same manner, choosing one of the above behaviors for the initiator, and a compatible one for the target. The most straight-forward choice might be: Initiator sends the logout request, and simply waits for the response without closing the connection. The target sends the logout response, and closes its end of the connection. The initiator receives the logout response, and the FIN from the target, and closes its end of the connection. Any opinions on whether this is best? This choice does not require the initiator to know that the connection that is being closed is the one it sent the logout response on, but does require the target to send the logout response before it actually closes the connection. Another behavior that might work would be to say that the target closes all of its connections and does cleanup before sending the logout response, and if the connection on which the request came in is gone, it does not send a response. An initiator whose connection is closed after sending the logoout request can assume that the request worked. Anyway, I think that it would be good to clarify this. -- Mark A. Bakke Cisco Systems mbakke@cisco.com 763.398.1054 From owner-ips@ece.cmu.edu Sat May 26 09:46:19 2001 Received: from ece.cmu.edu ([128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id JAA19181 for ; Sat, 26 May 2001 09:46:19 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f4QBeLD04297 for ips-outgoing; Sat, 26 May 2001 07:40:21 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from patan.sun.com (patan.Sun.COM [192.18.98.43]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f4Q6RO307459 for ; Sat, 26 May 2001 02:27:24 -0400 (EDT) Received: from phys-ha6nwka.ebay.sun.com ([129.149.1.82]) by patan.sun.com (8.9.3+Sun/8.9.3) with ESMTP id AAA23811 for ; Sat, 26 May 2001 00:27:22 -0600 (MDT) Received: from ebay.sun.com (dsl197-15.Eng.Sun.COM [129.146.197.15]) by phys-ha6nwka.ebay.sun.com (8.9.3+Sun/8.9.3/ENSMAIL,v2.1p1) with ESMTP id XAA18911 for ; Fri, 25 May 2001 23:27:21 -0700 (PDT) Message-ID: <3B0F4C5E.9F79DABB@ebay.sun.com> Date: Fri, 25 May 2001 23:25:34 -0700 From: David Robinson X-Mailer: Mozilla 4.74 [en] (Win98; U) X-Accept-Language: en MIME-Version: 1.0 To: ips@ece.cmu.edu Subject: Re: iSCSI boot References: Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-ips@ece.cmu.edu Precedence: bulk Content-Transfer-Encoding: 7bit Douglas Otis wrote: > I think you misunderstand where this code would exist. It would not be > within the DHCP/PXE PROM code. > > Use the existing environment and previously defined parameters in > conjunction with existing servers and existing protocols. I clearly don't understand your point. The purpose of the iSCSI boot draft is to define a mechanism to determine where the "root" disk is using standard protocols, most notable DHCP. Once the root disk is accessed and the operating system is loaded, everything is done using the naming and discovery mechanisms that will be defined. Using sophisticated protocols like LDAP may be appropriate once a kernel is loaded and should be defined as part of the naming and discovery design, but it is not a boot issue. The richness of LDAP is complete overkill for the initial boot environment. As a practical example, NFS clients today can boot using either bootparams or DHCP to load the initial root partition, once the kernel is loaded it is common to "remount" the root partition using the full NFS mount mechanisms. An iSCSI initiator could do the same thing to "relogin" and use the full set of options that may not be present in the boot environment. -David From owner-ips@ece.cmu.edu Sat May 26 09:47:02 2001 Received: from ece.cmu.edu ([128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id JAA19192 for ; Sat, 26 May 2001 09:47:02 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f4QCUwh06945 for ips-outgoing; Sat, 26 May 2001 08:30:58 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from d12lmsgate-3.de.ibm.com (d12lmsgate-3.de.ibm.com [195.212.91.201]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f4QCUv306941 for ; Sat, 26 May 2001 08:30:57 -0400 (EDT) Received: from d12relay01.de.ibm.com (d12relay01.de.ibm.com [9.165.215.22]) by d12lmsgate-3.de.ibm.com (1.0.0) with ESMTP id OAA50588 for ; Sat, 26 May 2001 14:30:50 +0200 From: julian_satran@il.ibm.com Received: from d12mta02.de.ibm.com (d12mta01_cs0 [9.165.222.237]) by d12relay01.de.ibm.com (8.8.8m3/NCO v4.96) with SMTP id OAA35376 for ; Sat, 26 May 2001 14:30:50 +0200 Received: by d12mta02.de.ibm.com(Lotus SMTP MTA v4.6.5 (863.2 5-20-1999)) id C1256A58.0044B071 ; Sat, 26 May 2001 14:30:16 +0200 X-Lotus-FromDomain: IBMIL@IBMDE To: ips@ece.cmu.edu Message-ID: Date: Sat, 26 May 2001 15:36:18 +0300 Subject: Re: Asynchronous Message: Session Termination Mime-Version: 1.0 Content-type: text/plain; charset=us-ascii Content-Disposition: inline Sender: owner-ips@ece.cmu.edu Precedence: bulk Santosh, Answers in text - Julo Santosh Rao on 25-05-2001 21:32:17 Please respond to Santosh Rao To: ips@ece.cmu.edu cc: Subject: Re: Asynchronous Message: Session Termination Julian, The modified text on "Async Message" seems to have missed out several aspects of the discussion on this thread : 1) It is useful to provide a parameter for the "target will drop this connection" and "target will drop all connections" to indicate the time, in seconds, after which the target would drop the connection[s]. This allows : - the target to warn the initiator ahead of time about an upcoming disruption so that initiator can take action accordingly. - Allows elimination of the "target requests logout" option since the "will drop connection/all connections" flavor provides the initiator an upper time bound after which the connection[s] are going to be dropped. This allows the initiator to attempt to quiesce the connection[s] prior to the expiry of that time period. The "target requests logout" option should be removed, if its equivalent functionality or better can be provided by the "will drop connection/all connections" flavors. +++ the request logout was intended to allow an initiator to quisce activity. Having a target - that is unaware of the speed or needs of an initiator dictate a timeout does not look like a good solution for a non-urgent case. Request logout was meant to enable channels to go into maintenance +++ 2) Suggest modification of the below wording : " If the initiator does not attempt to reconnect within the time specified by Parameter 3 or Parameter 3 is 0 the session is terminated - in which case no other responses should be expected from the target for outstanding commands on this session and the initiator should terminate them appropriately." to indicate that upon expiry of the LogoutLoginMaxTime (parameter 3), the target internally aborts all outstanding commands on that connection and the initiator should consider those I/Os implicitly aborted. No explicit Abort mechanisms are required from the initiator to the target upon expiry of LogoutLoginMaxTime to terminate these commands. +++ that is the exact meaning of the current wording and the statement that no other responses should be expected conveys the behaviour as seen by the initiator. However since your reaction indicates that this might still be misinterpreted i'll change the text to: 3 Target indicates it will drop the connection - the Parameter1 field will indicate on what CID while the Parameter2 field indicates, in seconds, the minimum time to wait before attempting to reconnect and Parameter3 the maximum time to reconnect after the initial wait (Parameter2). If the initiator does not attempt to reconnect within the time specified by Parameter 3 or Parameter 3 is 0 the target will terminate all outstanding commands on this connection, no other responses should be expected from the target for the outstanding commands on this connection and the initiator should generate the appropriate responses. A value of 0 for Parameter 2 indicates that reconnect can be attempted immediately. 4 Target indicates it will drop all connections - the Parameter2 field indicates, in seconds, the minimum time to wait before attempting to reconnect and Parameter3 the maximum time to reconnect after the initial wait (Parameter2). If the initiator does not attempt to reconnect within the time specified by Parameter 3 or Parameter 3 is 0 the session is terminated. In this case the target will terminate all outstanding commands in this session, no other responses should be expected from the target for the outstanding commands in this session and the initiator should generate the appropriate responses. A value of 0 for Parameter 2 indicates that reconnect can be attempted immediately. 3) The "target will drop all connections" flavor description should indicate that the target is performing a session logout. I like the parameter descriptions provided by Mark [Bakke] for this flavor, namely : - Parameter2 indicates the time, in seconds, after which the target will drop all connections. - Parameter3 indicates the additional time, in seconds, after the expiry of Parameter2, that the initiator must wait before attempting to re-establish a session. The LogoutLoginMaxTime in the current defn of "target will drop all connections" (Parameter 3) provides little value, since the target should be aborting all oustanding I/O as a part of that session logout and not wait until LogoutLoginMaxTime. +++ the point is that the target is not performing a session logout but is rather droping all connections and may wish to establish new ones without droping the session - that is the meaning of Parameter3+++ 4) Regarding : "Initiators can attempt to set/modify the values a target will use for Parameter2 and Parameter3. However - I am unclear if we should leave the initiator the degree of control implied by (innocuous looking) last statement." Since, LogoutLoginMaxTime & LogoutLoginMinTime are only applicable in the context of a target originated "connection drop" as indicated by the async events, these keys add little or no value in the login negotiation. In all cases, it is the time values provided by the target in the async events that are applicable and over-ride any login negotiated values for these keys. Hence, should these keys be removed from the login negotiation ? +++ they are meant to convey the target what the specific initiator considers "enough time" and that varies by constituency. ++++ Regards, Santosh julian_satran@il.ibm.com wrote: > > Mathhew, Mark, Matt & Santosh: > > Here is the new text for 2.17.1 > > 1.1.1 iSCSI Event > > The codes used for iSCSI Asynchronous Messages (Events) are: > > 0 No iSCSI Event > 1 Target is being reset. > 2 Target requests Logout - the Parameter1 field indicates on what > CID. > 3 Target indicates it will drop the connection - the Parameter1 > field will indicate on what CID while the Parameter2 field indicates, > in seconds, the minimum time to wait before attempting to reconnect > and Parameter3 the maximum time to reconnect after the initial wait > (Parameter2). If the initiator does not attempt to reconnect within > the time specified by Parameter 3 or Parameter 3 is 0 no other > responses should be expected from the target for outstanding commands > on this connection and the initiator should terminate them > appropriately. A value of 0 for Parameter 2 indicates that reconnect > can be attempted immediately. > 4 Target indicates it will drop all connections - the Parameter2 > field indicates, in seconds, the minimum time to wait before > attempting to reconnect and Parameter3 the maximum time to reconnect > after the initial wait (Parameter2). If the initiator does not > attempt to reconnect within the time specified by Parameter 3 or > Parameter 3 is 0 the session is terminated - in which case no other > responses should be expected from the target for outstanding commands > on this session and the initiator should terminate them > appropriately. A value of 0 for Parameter 2 indicates that reconnect > can be attempted immediately. > > Initiators can attempt to set/modify the values a target will use for > Parameter2 and Parameter3 > > However - I am unclear if we should leave the initiator the degree of > control implied by (innocuous looking) last statement. > > Thanks for helping make this text better. > > Julo - santoshr.vcf From owner-ips@ece.cmu.edu Sat May 26 09:49:14 2001 Received: from ece.cmu.edu ([128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id JAA19207 for ; Sat, 26 May 2001 09:49:14 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f4QBgp404461 for ips-outgoing; Sat, 26 May 2001 07:42:51 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from web14403.mail.yahoo.com (web14403.mail.yahoo.com [216.136.174.60]) by ece.cmu.edu (8.11.0/8.10.2) with SMTP id f4Q8Gl313386 for ; Sat, 26 May 2001 04:16:47 -0400 (EDT) Message-ID: <20010526081646.81776.qmail@web14403.mail.yahoo.com> Received: from [203.145.156.59] by web14403.mail.yahoo.com; Sat, 26 May 2001 01:16:46 PDT Date: Sat, 26 May 2001 01:16:46 -0700 (PDT) From: Rishabh Srivastav Subject: iSCSI: markers To: ips@ece.cmu.edu MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Sender: owner-ips@ece.cmu.edu Precedence: bulk Hi, In the current marker scheme, the distance is given from the current position to the beginning of the next PDU. Why does the marker scheme not consider specifying the distance from the beginning of the current PDU itself ? Thanks, Rishabh __________________________________________________ Do You Yahoo!? Yahoo! Auctions - buy the things you want at great prices http://auctions.yahoo.com/ From owner-ips@ece.cmu.edu Sat May 26 09:49:53 2001 Received: from ece.cmu.edu ([128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id JAA19219 for ; Sat, 26 May 2001 09:49:52 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f4QClF207855 for ips-outgoing; Sat, 26 May 2001 08:47:15 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from d12lmsgate-2.de.ibm.com (d12lmsgate-2.de.ibm.com [195.212.91.200]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f4QClD307851 for ; Sat, 26 May 2001 08:47:13 -0400 (EDT) Received: from d12relay01.de.ibm.com (d12relay01.de.ibm.com [9.165.215.22]) by d12lmsgate-2.de.ibm.com (1.0.0) with ESMTP id OAA197938 for ; Sat, 26 May 2001 14:47:07 +0200 From: julian_satran@il.ibm.com Received: from d12mta02.de.ibm.com (d12mta01_cs0 [9.165.222.237]) by d12relay01.de.ibm.com (8.8.8m3/NCO v4.96) with SMTP id OAA46982 for ; Sat, 26 May 2001 14:47:07 +0200 Received: by d12mta02.de.ibm.com(Lotus SMTP MTA v4.6.5 (863.2 5-20-1999)) id C1256A58.0046310C ; Sat, 26 May 2001 14:46:40 +0200 X-Lotus-FromDomain: IBMIL@IBMDE To: ips@ece.cmu.edu Message-ID: Date: Sat, 26 May 2001 15:52:43 +0300 Subject: Re: Internationalisation issue Mime-Version: 1.0 Content-type: text/plain; charset=us-ascii Content-Disposition: inline Sender: owner-ips@ece.cmu.edu Precedence: bulk Mark, I think that Glen's proposal makes sense. Names in the SCSI space are not manually manipulated too often and mistakes are bound to be corrected fast. You would not believe this from a person using a language that hase no case :-) Julo Mark Bakke on 26-05-2001 00:17:31 Please respond to Mark Bakke To: Glen Turner cc: ips@ece.cmu.edu Subject: Re: Internationalisation issue Glen- The IESG wouldn't let us specify an iSCSI name as a URN, so we are just going to let this draft expire. Anyway, the case-insentive rule still applies, and is specified in the naming & discovery document. Basically, an iSCSI name is treated as an opaque string; the only valid operation on an iSCSI name (other than display) is comparison. We had to specify a method to compare iSCSI names. Since they are text strings, and since text strings are transcribable, it is less problematic for users if they are case-insensitive. However, you are correct that once we have gone from ASCII to UTF-8, comparison is not as simple. On my linux system, it looks like I have some wide character toupper() and tolower() routines which would probably do the trick after converting the UTF-8 into 32-bit Unicode, but I haven't looked at the code that goes with them or its complexity. Our possible solutions are: - Make iSCSI names case-sensitive This is easy, but not as nice for the user, especially when manual configuration is performed. - Make iSCSI names case-insensitive for all character sets. This is probably the ideal from a user's point of view. I would be interested in comments as to how much this is needed (it's what is currently specified) or even if it is needed, who plans to support it. This is the hardest one to implement. Some character sets have three cases (lower, upper and title) as well. Jim H had written up some stuff on how this works in the current Naming & Discovery document. I guess the question here is whether it's worth going to the trouble to give the user the ultimate solution. The comparison method would be: 1. If the high bit is not set on any of the characters in one of the iSCSI names to be compared, just do a strcasecmp(). 2. Convert the iSCSI names from UTF-8 to 32-bit Unicode. 3. Use wcscasecmp() to compare the names. The above assumes the GNU/Linux library calls. Again, I haven't looked at how much code this is, but it's freely available and would work. - Make iSCSI names case-insensitive within the ASCII character range, and case-senstive for everything else. This means that non-ASCII characters have to be made case- sensitive, which is fine for many users, but may not be as nice for those using other character sets. It would be easy to implement, since normal toupper() and tolower() functions can be used. - Make iSCSI names case-sensitive, but mandate that they are all transmitted and stored in lower case. This makes it the user interface's problem to do the tolower() on whatever character sets it supports. It's easy for the iSCSI implementation. However, an iSCSI name displayed on "the other side" of a connection would lose any case typed by the user. This might be the best compromise. - Make iSCSI names ASCII This eliminates many character sets, and will restrict methods of producing unique names in various languages. It would also not handle the multilingual DNS names when they happen. However, since we do have an Alias that is a UTF-8 string and doesn't have to be compared, perhaps nobody plans to generate an iSCSI name based on non-ASCII characters? Any thoughts? I would be interested in the following opinions from the list: 1. Do you or your customers see value in using non-ASCII character sets to generate iSCSI names, or is the use of Alias enough? 2. Do you or your customers see value in preserving the original case in which an iSCSI name was generated, or would it be fine just to specify that they are always transferred in lower case? 3. Is the procedure for comparing these names too much to expect from a small target? -- Mark Glen Turner wrote: > > > Rules for Lexical Equivalence: > > > > The entire URN is case-insensitive. > > Why? This implies that devices now need to carry casing tables > tables for multi-lingual support. > > If the field is case sensitive then adding multilingual support > is simply a matter of specifiying UTF-8 and no ongoing work is > needed to track multilingual DNS. > > Regards, > Glen -- Mark A. Bakke Cisco Systems mbakke@cisco.com 763.398.1054 From owner-ips@ece.cmu.edu Sat May 26 09:52:42 2001 Received: from ece.cmu.edu ([128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id JAA19253 for ; Sat, 26 May 2001 09:52:41 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f4QCafE07292 for ips-outgoing; Sat, 26 May 2001 08:36:41 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from d12lmsgate-3.de.ibm.com (d12lmsgate-3.de.ibm.com [195.212.91.201]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f4QCad307286 for ; Sat, 26 May 2001 08:36:40 -0400 (EDT) Received: from d12relay02.de.ibm.com (d12relay02.de.ibm.com [9.165.215.23]) by d12lmsgate-3.de.ibm.com (1.0.0) with ESMTP id OAA32042 for ; Sat, 26 May 2001 14:36:33 +0200 From: julian_satran@il.ibm.com Received: from d12mta02.de.ibm.com (d12mta01_cs0 [9.165.222.237]) by d12relay02.de.ibm.com (8.8.8m3/NCO v4.96) with SMTP id OAA71936 for ; Sat, 26 May 2001 14:36:05 +0200 Received: by d12mta02.de.ibm.com(Lotus SMTP MTA v4.6.5 (863.2 5-20-1999)) id C1256A58.0045382A ; Sat, 26 May 2001 14:36:03 +0200 X-Lotus-FromDomain: IBMIL@IBMDE To: ips@ece.cmu.edu Message-ID: Date: Sat, 26 May 2001 15:42:08 +0300 Subject: RE: iSCSI: response to second login (with same ISID) Mime-Version: 1.0 Content-type: text/plain; charset=us-ascii Content-Disposition: inline Sender: owner-ips@ece.cmu.edu Precedence: bulk Martin, I agree that it would be bad if at login an initiator would have to wait for a long time for the target to detect if the old session has gone away. But there is nothing in the current spec that will prevent a good target implementation doing what you describe whithout any negotiation. Regards, Julo "Martin, Nick" on 25-05-2001 23:22:11 Please respond to "Martin, Nick" To: Julian Satran/Haifa/IBM@IBMIL, ips@ece.cmu.edu cc: Subject: RE: iSCSI: response to second login (with same ISID) Julian, I do not know how long it would take a target using a ping to determine that an old session is really dead. Some folks think it would be too long, thus they want to be able to "login unconditionally". Primarily I thought this would be used for initiators rebooting after ungraceful shutdown. (For those who can otherwise reboot quickly, but would stall on each iSCSI session not already recovered by the target.) I do not expect to want to use a "login unconditionally" option. (I can handle rejection ;) The scary thing to me was if an initiator could not try an ISID which was in use (intentionally, accidentally, or erroneously) without disrupting (logging out) its current user (a.k.a. option 2). It appears that if given a "login unconditionally" option, some folks would use it exclusively. If the initiator can specify the maximum time it would take a target to notice dead connections and sessions, then the argument that waiting for target recovery of the session would take too long presumably goes away. I am thinking about login negotiated parameters like MinIdleTime and MaxIdleTime values in seconds. These could specify respectively the connection idle interval after which a NO-OP (keep-alive) should be sent and expected, and the time with no traffic received on this connection which should be regarded as an implicit logout request due to connection failure. (This could cause an async event if detected by the target and a working connection remains in this session.) Better names can be chosen. Reasonable defaults might be 10 seconds and 60 seconds. There should be a recommendation like MaxIdleTime should always be at least 3 times MinIdleTime. Such parameters would probably exist within the Target and the Initiator, even if they are not negotiated or exchanged. The intent is that an Initiator that cares, can limit Target connection recovery delays. These parameters set by the Initiator for the Target, would be known to both. A broken connection could be detected by each at approximately the same time. The keep-alive NO-OP's could be pings so that both directions would be kept alive by a single exchange. If the Initiator wants to be responsible for keep-alives, it should do them before the specified MinIdleTime, if it wants the Target to do them, it may need to do them also, slightly after the specified MinIdleTime, but well before MaxIdleTime. My networking is weak, so I am not sure whether there should be a separate (shorter) time specified for the maximum time to wait for the reply to a ping, or how long to wait to retry it. I am not clear on the interactions with TCP and its timeouts and retries. Is there a maximum time a ping could take and still get through? Is there a maximum time a busy iSCSI target or initiator would be expected to delay before sending a reply to a ping? Does this sound like a good idea at all or not? Thanks, Nick From owner-ips@ece.cmu.edu Sat May 26 10:32:21 2001 Received: from ece.cmu.edu ([128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id KAA19535 for ; Sat, 26 May 2001 10:32:21 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f4QD73f09077 for ips-outgoing; Sat, 26 May 2001 09:07:03 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from d12lmsgate-2.de.ibm.com (d12lmsgate-2.de.ibm.com [195.212.91.200]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f4QD71309072 for ; Sat, 26 May 2001 09:07:02 -0400 (EDT) Received: from d12relay01.de.ibm.com (d12relay01.de.ibm.com [9.165.215.22]) by d12lmsgate-2.de.ibm.com (1.0.0) with ESMTP id PAA264874 for ; Sat, 26 May 2001 15:06:55 +0200 From: julian_satran@il.ibm.com Received: from d12mta02.de.ibm.com (d12mta01_cs0 [9.165.222.237]) by d12relay01.de.ibm.com (8.8.8m3/NCO v4.96) with SMTP id PAA23554 for ; Sat, 26 May 2001 15:06:55 +0200 Received: by d12mta02.de.ibm.com(Lotus SMTP MTA v4.6.5 (863.2 5-20-1999)) id C1256A58.0048005B ; Sat, 26 May 2001 15:06:26 +0200 X-Lotus-FromDomain: IBMIL@IBMDE To: ips@ece.cmu.edu Message-ID: Date: Sat, 26 May 2001 16:12:29 +0300 Subject: Re: iSCSI: Logout command, or The Initiator's new close() Mime-Version: 1.0 Content-type: text/plain; charset=us-ascii Content-Disposition: inline Sender: owner-ips@ece.cmu.edu Precedence: bulk Mark, If that helps understanding we might add the following text: After sending the Logout PDU, an initiator MUST NOT send any iSCSI PDU on the closing connection and MAY immediately initiate a half-close while continuing to receive on the referred connection. After receiving the Login response the initiator MUST close completely the referred to connection. Regards, Julo Mark Bakke on 26-05-2001 00:43:30 Please respond to Mark Bakke To: IPS cc: Subject: iSCSI: Logout command, or The Initiator's new close() Section 2.14 (the logout command) is not clear on how the logout command and response work when the logout request is send on a connection for which it requests termination. We should probably specify the TCP close behavior of the initiator and target. The initiator can send the logout request, and either close the current connection, half-close the connection and wait for the logout response (like HTTP/1.0), or leave it open, and close the connection upon receiving the response or the FIN from the target. Upon receiving the logout request, the target can either close the connection immediately, send the logout response and then close the connection, or send the logout response and wait for the initiator to close the connection upon its receipt of the logout response. Logout would work best if the initiators and targets all did this in the same manner, choosing one of the above behaviors for the initiator, and a compatible one for the target. The most straight-forward choice might be: Initiator sends the logout request, and simply waits for the response without closing the connection. The target sends the logout response, and closes its end of the connection. The initiator receives the logout response, and the FIN from the target, and closes its end of the connection. Any opinions on whether this is best? This choice does not require the initiator to know that the connection that is being closed is the one it sent the logout response on, but does require the target to send the logout response before it actually closes the connection. Another behavior that might work would be to say that the target closes all of its connections and does cleanup before sending the logout response, and if the connection on which the request came in is gone, it does not send a response. An initiator whose connection is closed after sending the logoout request can assume that the request worked. Anyway, I think that it would be good to clarify this. -- Mark A. Bakke Cisco Systems mbakke@cisco.com 763.398.1054 From owner-ips@ece.cmu.edu Sat May 26 12:28:48 2001 Received: from ece.cmu.edu ([128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id MAA19980 for ; Sat, 26 May 2001 12:28:48 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f4QEoMs14713 for ips-outgoing; Sat, 26 May 2001 10:50:22 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from d12lmsgate.de.ibm.com (d12lmsgate.de.ibm.com [195.212.91.199]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f4QEoK314709 for ; Sat, 26 May 2001 10:50:21 -0400 (EDT) Received: from d12relay01.de.ibm.com (d12relay01.de.ibm.com [9.165.215.22]) by d12lmsgate.de.ibm.com (1.0.0) with ESMTP id QAA87652 for ; Sat, 26 May 2001 16:50:13 +0200 From: julian_satran@il.ibm.com Received: from d12mta05.de.ibm.com (d12mta05_cs0 [9.165.222.239]) by d12relay01.de.ibm.com (8.8.8m3/NCO v4.96) with SMTP id QAA45888 for ; Sat, 26 May 2001 16:50:14 +0200 Received: by d12mta05.de.ibm.com(Lotus SMTP MTA v4.6.5 (863.2 5-20-1999)) id C1256A58.00517CC5 ; Sat, 26 May 2001 16:50:03 +0200 X-Lotus-FromDomain: IBMIL@IBMDE To: "'ips@ece.cmu.edu'" Message-ID: Date: Sat, 26 May 2001 17:55:44 +0300 Subject: Re: Login Rejection: Target does not support multiple addresses per S ession Mime-Version: 1.0 Content-type: text/plain; charset=us-ascii Content-Disposition: inline Sender: owner-ips@ece.cmu.edu Precedence: bulk Matthew, Assuming that we want the target to control the addresses a session spans I will add No multiple | 0208 | 1 | Target does not support session Addresses | | | spanning target addresses Julo "BURBRIDGE,MATTHEW (HP-UnitedKingdom,ex2)" on 23-05-2001 13:13:25 Please respond to "BURBRIDGE,MATTHEW (HP-UnitedKingdom,ex2)" To: "'ips@ece.cmu.edu'" cc: Subject: Login Rejection: Target does not support multiple addresses per S ession Hi Julian If an initiator attempts to set up a new connection for an existing session (i.e TSID is non-zero, and Restart bit is 0) but via another physical port then if the target does not support session spanning there is currently no Login Response Status code that covers the scenario. Please can you add an additional error code which informs the initiator that the login has been rejected due to a target not being able to support session spanning (e.g. "Target does not support multiple addresses per Session"). Attempts by the initiator to connect to the session via the original port, or for Within-session recovery, or to create an completely new session, would not be rejected using this error code. Thanks Matthew Burbridge NIS-Bristol Hewlett Packard Telnet: 312 7010 E-mail: matthewb@bri.hp.com From owner-ips@ece.cmu.edu Sat May 26 12:33:27 2001 Received: from ece.cmu.edu ([128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id MAA20012 for ; Sat, 26 May 2001 12:33:27 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f4QEqIo14853 for ips-outgoing; Sat, 26 May 2001 10:52:18 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from d12lmsgate-3.de.ibm.com (d12lmsgate-3.de.ibm.com [195.212.91.201]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f4QEqG314849 for ; Sat, 26 May 2001 10:52:17 -0400 (EDT) Received: from d12relay01.de.ibm.com (d12relay01.de.ibm.com [9.165.215.22]) by d12lmsgate-3.de.ibm.com (1.0.0) with ESMTP id QAA96078 for ; Sat, 26 May 2001 16:52:10 +0200 From: julian_satran@il.ibm.com Received: from d12mta02.de.ibm.com (d12mta01_cs0 [9.165.222.237]) by d12relay01.de.ibm.com (8.8.8m3/NCO v4.96) with SMTP id QAA68376 for ; Sat, 26 May 2001 16:52:10 +0200 Received: by d12mta02.de.ibm.com(Lotus SMTP MTA v4.6.5 (863.2 5-20-1999)) id C1256A58.0051A3E5 ; Sat, 26 May 2001 16:51:43 +0200 X-Lotus-FromDomain: IBMIL@IBMDE To: ips@ece.cmu.edu Message-ID: Date: Sat, 26 May 2001 17:57:46 +0300 Subject: Re: iSCSI: markers Mime-Version: 1.0 Content-type: text/plain; charset=us-ascii Content-Disposition: inline Sender: owner-ips@ece.cmu.edu Precedence: bulk The marker is supposed to help you get to your future not past :-) Julo Rishabh Srivastav on 26-05-2001 16:42:18 Please respond to Rishabh Srivastav To: ips@ece.cmu.edu cc: Subject: iSCSI: markers Julo, Current marker scheme in the draft specifies a distance from current position to the beginning of the next PDU. Why does it not consider the distance from the beginning of the current PDU itself ? Thanks, Rishabh __________________________________________________ Do You Yahoo!? Yahoo! Auctions - buy the things you want at great prices http://auctions.yahoo.com/ From owner-ips@ece.cmu.edu Sat May 26 12:34:15 2001 Received: from ece.cmu.edu ([128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id MAA20036 for ; Sat, 26 May 2001 12:34:15 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f4QEXEd13763 for ips-outgoing; Sat, 26 May 2001 10:33:14 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from web14405.mail.yahoo.com (web14405.mail.yahoo.com [216.136.174.62]) by ece.cmu.edu (8.11.0/8.10.2) with SMTP id f4QDgX311046 for ; Sat, 26 May 2001 09:42:33 -0400 (EDT) Message-ID: <20010526134218.61007.qmail@web14405.mail.yahoo.com> Received: from [203.145.156.245] by web14405.mail.yahoo.com; Sat, 26 May 2001 06:42:18 PDT Date: Sat, 26 May 2001 06:42:18 -0700 (PDT) From: Rishabh Srivastav Subject: iSCSI: markers To: ips@ece.cmu.edu MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Sender: owner-ips@ece.cmu.edu Precedence: bulk Julo, Current marker scheme in the draft specifies a distance from current position to the beginning of the next PDU. Why does it not consider the distance from the beginning of the current PDU itself ? Thanks, Rishabh __________________________________________________ Do You Yahoo!? Yahoo! Auctions - buy the things you want at great prices http://auctions.yahoo.com/ From owner-ips@ece.cmu.edu Sat May 26 12:34:17 2001 Received: from ece.cmu.edu ([128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id MAA20047 for ; Sat, 26 May 2001 12:34:16 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f4QEYxJ13874 for ips-outgoing; Sat, 26 May 2001 10:34:59 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from d12lmsgate-2.de.ibm.com (d12lmsgate-2.de.ibm.com [195.212.91.200]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f4QEYw313869 for ; Sat, 26 May 2001 10:34:58 -0400 (EDT) Received: from d12relay01.de.ibm.com (d12relay01.de.ibm.com [9.165.215.22]) by d12lmsgate-2.de.ibm.com (1.0.0) with ESMTP id QAA104504 for ; Sat, 26 May 2001 16:34:52 +0200 From: julian_satran@il.ibm.com Received: from d12mta02.de.ibm.com (d12mta01_cs0 [9.165.222.237]) by d12relay01.de.ibm.com (8.8.8m3/NCO v4.96) with SMTP id QAA58042 for ; Sat, 26 May 2001 16:34:52 +0200 Received: by d12mta02.de.ibm.com(Lotus SMTP MTA v4.6.5 (863.2 5-20-1999)) id C1256A58.00500A7D ; Sat, 26 May 2001 16:34:15 +0200 X-Lotus-FromDomain: IBMIL@IBMDE To: ips@ece.cmu.edu Message-ID: Date: Sat, 26 May 2001 17:40:20 +0300 Subject: Re: Asynchronous Message: Session Termination Mime-Version: 1.0 Content-type: text/plain; charset=us-ascii Content-Disposition: inline Sender: owner-ips@ece.cmu.edu Precedence: bulk The value 0 for parameter3 will indicate this. Julo "BURBRIDGE,MATTHEW (HP-UnitedKingdom,ex2)" on 23-05-2001 12:51:04 Please respond to "BURBRIDGE,MATTHEW (HP-UnitedKingdom,ex2)" To: ips@ece.cmu.edu cc: Subject: Asynchronous Message: Session Termination Hi Julian, If a target only supports Session Recovery, then when an error is detected it will terminate the session. The current Async Message (iSCSI Event "Target indicates that his will/has dropped all connections") does not necessary inform the initiator that the session has terminated and the initiator is quite within rights to establish a new connection and attempt With-in Session recovery. Can you change the spec to reflect that a value of zero in both the Parameter2 and Parameter3 would set the maximum time to reconnect to "Never" thereby informing the initiator that the session has closed? Thanks Matthew Burbridge NIS-Bristol Hewlett Packard Telnet: 312 7010 E-mail: matthewb@bri.hp.com From owner-ips@ece.cmu.edu Sat May 26 14:02:09 2001 Received: from ece.cmu.edu ([128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id OAA20471 for ; Sat, 26 May 2001 14:02:09 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f4QG8Hs18973 for ips-outgoing; Sat, 26 May 2001 12:08:17 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from c007.snv.cp.net (c007-h000.c007.snv.cp.net [209.228.33.206]) by ece.cmu.edu (8.11.0/8.10.2) with SMTP id f4QG8G318969 for ; Sat, 26 May 2001 12:08:16 -0400 (EDT) Received: (cpmta 18583 invoked from network); 26 May 2001 09:08:05 -0700 Received: from unknown (HELO ljoy) (66.1.139.185) by smtp.telocity.com (209.228.33.206) with SMTP; 26 May 2001 09:08:05 -0700 X-Sent: 26 May 2001 16:08:05 GMT From: "Douglas Otis" To: "David Robinson" , Subject: RE: iSCSI boot Date: Sat, 26 May 2001 09:05:41 -0700 Message-ID: MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 (Normal) X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook IMO, Build 9.0.2416 (9.0.2911.0) In-Reply-To: <3B0F4C5E.9F79DABB@ebay.sun.com> X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4522.1200 Importance: Normal Sender: owner-ips@ece.cmu.edu Precedence: bulk Content-Transfer-Encoding: 7bit David, Rather than attempting to replicate a micro NFS client level of complexity in a boot NAS, A DUA to obtain parameters should allow immediate use of a SAN. As far as the level of complexity, the means of obtaining the required access together with code exists today. It is simply a matter of defining the structures. Yes, it is different from starting with a NAS. DHCP passes a limited number of parameters and will be overwhelmed by the demands for starting an iSCSI session. LDAP can easily accommodate these demands without one bit of the protocol modified or new DHCP parameters defined. There could simply be a table defined in the boot image for that matter if LDAP seems to difficult but that seems difficult to manage. Doug > Douglas Otis wrote: > > I think you misunderstand where this code would exist. It would not be > > within the DHCP/PXE PROM code. > > > > Use the existing environment and previously defined parameters in > > conjunction with existing servers and existing protocols. > > I clearly don't understand your point. The purpose of the iSCSI boot > draft is to define a mechanism to determine where the "root" disk is > using standard protocols, most notable DHCP. Once the root disk is > accessed and the operating system is loaded, everything is done > using the naming and discovery mechanisms that will be defined. > > Using sophisticated protocols like LDAP may be appropriate once a > kernel is loaded and should be defined as part of the naming and > discovery design, but it is not a boot issue. The richness of LDAP > is complete overkill for the initial boot environment. > > As a practical example, NFS clients today can boot using either > bootparams or DHCP to load the initial root partition, once the > kernel is loaded it is common to "remount" the root partition > using the full NFS mount mechanisms. An iSCSI initiator could > do the same thing to "relogin" and use the full set of options > that may not be present in the boot environment. > > -David > From owner-ips@ece.cmu.edu Sat May 26 14:04:27 2001 Received: from ece.cmu.edu ([128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id OAA20486 for ; Sat, 26 May 2001 14:04:27 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f4QGJII19594 for ips-outgoing; Sat, 26 May 2001 12:19:18 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from dogwood.cisco.com (dogwood.cisco.com [161.44.11.19]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f4QGJH319590 for ; Sat, 26 May 2001 12:19:17 -0400 (EDT) Received: from ahganemw2k ([10.21.58.83] (may be forged)) by dogwood.cisco.com (8.8.6 (PHNE_14041)/CISCO.SERVER.1.2) with SMTP id MAA11288 for ; Sat, 26 May 2001 12:19:08 -0400 (EDT) From: "Ayman Ghanem" To: Subject: iSCSI: Logout response Date: Sat, 26 May 2001 11:18:03 -0500 Message-ID: MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit X-Priority: 3 (Normal) X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook IMO, Build 9.0.2416 (9.0.2910.0) In-Reply-To: X-MimeOLE: Produced By Microsoft MimeOLE V5.00.2919.6700 Importance: Normal Sender: owner-ips@ece.cmu.edu Precedence: bulk Content-Transfer-Encoding: 7bit I think we need an additional response code for a logout response to inform the initiator if the connection was not found on the target. Also, a minor editing of text for response code 0: 0 - connection (or session) closed successfully -Ayman From owner-ips@ece.cmu.edu Sat May 26 14:08:35 2001 Received: from ece.cmu.edu ([128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id OAA20509 for ; Sat, 26 May 2001 14:08:35 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f4QG0UH18559 for ips-outgoing; Sat, 26 May 2001 12:00:30 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from d12lmsgate-2.de.ibm.com (d12lmsgate-2.de.ibm.com [195.212.91.200]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f4QG0J318520 for ; Sat, 26 May 2001 12:00:19 -0400 (EDT) Received: from d12relay01.de.ibm.com (d12relay01.de.ibm.com [9.165.215.22]) by d12lmsgate-2.de.ibm.com (1.0.0) with ESMTP id SAA104518 for ; Sat, 26 May 2001 18:00:13 +0200 From: julian_satran@il.ibm.com Received: from d12mta02.de.ibm.com (d12mta01_cs0 [9.165.222.237]) by d12relay01.de.ibm.com (8.8.8m3/NCO v4.96) with SMTP id SAA32704 for ; Sat, 26 May 2001 18:00:13 +0200 Received: by d12mta02.de.ibm.com(Lotus SMTP MTA v4.6.5 (863.2 5-20-1999)) id C1256A58.0057DD1D ; Sat, 26 May 2001 17:59:42 +0200 X-Lotus-FromDomain: IBMIL@IBMDE To: hafner@almaden.ibm.com cc: ips@ece.cmu.edu Message-ID: Date: Sat, 26 May 2001 19:05:44 +0300 Subject: iSCSI-Nashua presentation Mime-Version: 1.0 Content-type: text/plain; charset=us-ascii Content-Disposition: inline Sender: owner-ips@ece.cmu.edu Precedence: bulk Jim, Your Nashua presentation is available on my site. Regards, Julo From owner-ips@ece.cmu.edu Sun May 27 03:13:59 2001 Received: from ece.cmu.edu (ECE.CMU.EDU [128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id DAA01058 for ; Sun, 27 May 2001 03:13:59 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f4R4jtC00554 for ips-outgoing; Sun, 27 May 2001 00:45:55 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from d12lmsgate-3.de.ibm.com (d12lmsgate-3.de.ibm.com [195.212.91.201]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f4R4jr300547 for ; Sun, 27 May 2001 00:45:53 -0400 (EDT) Received: from d12relay01.de.ibm.com (d12relay01.de.ibm.com [9.165.215.22]) by d12lmsgate-3.de.ibm.com (1.0.0) with ESMTP id GAA96196 for ; Sun, 27 May 2001 06:45:47 +0200 From: julian_satran@il.ibm.com Received: from d12mta02.de.ibm.com (d12mta01_cs0 [9.165.222.237]) by d12relay01.de.ibm.com (8.8.8m3/NCO v4.96) with SMTP id GAA83304 for ; Sun, 27 May 2001 06:45:46 +0200 Received: by d12mta02.de.ibm.com(Lotus SMTP MTA v4.6.5 (863.2 5-20-1999)) id C1256A59.001A1C3E ; Sun, 27 May 2001 06:45:11 +0200 X-Lotus-FromDomain: IBMIL@IBMDE To: ips@ece.cmu.edu Message-ID: Date: Sun, 27 May 2001 07:51:23 +0300 Subject: Re: iSCSI: Logout response Mime-Version: 1.0 Content-type: text/plain; charset=us-ascii Content-Disposition: inline Sender: owner-ips@ece.cmu.edu Precedence: bulk I'll fix the text, Response 1 was supposed to cover all errors. Newertheless I'll make it: 1.1.1 Response Logout response: 0 - connection or session closed successfully 1 - CID not found 2 - cleanup failed for various reasons "Ayman Ghanem" on 26-05-2001 19:18:03 Please respond to "Ayman Ghanem" To: ips@ece.cmu.edu cc: Subject: iSCSI: Logout response I think we need an additional response code for a logout response to inform the initiator if the connection was not found on the target. Also, a minor editing of text for response code 0: 0 - connection (or session) closed successfully -Ayman From owner-ips@ece.cmu.edu Sun May 27 15:05:54 2001 Received: from ece.cmu.edu (ECE.CMU.EDU [128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id PAA11469 for ; Sun, 27 May 2001 15:05:54 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f4RGWiL15346 for ips-outgoing; Sun, 27 May 2001 12:32:44 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from internaut.com ([64.38.134.99]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f4RGWb315336 for ; Sun, 27 May 2001 12:32:38 -0400 (EDT) Received: from localhost (aboba@localhost) by internaut.com (8.9.3/8.9.3) with ESMTP id JAA46674; Sun, 27 May 2001 09:22:47 -0700 (PDT) (envelope-from aboba@internaut.com) Date: Sun, 27 May 2001 09:22:47 -0700 (PDT) From: Bernard Aboba To: Douglas Otis cc: David Robinson , ips@ece.cmu.edu, narten@raleigh.ibm.com Subject: iSCSI and secure boot In-Reply-To: Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-ips@ece.cmu.edu Precedence: bulk > Security is actively being worked on the the DHCP community so that > is something that iSCSI can leverage. > (draft-ietf-dhc-authentication-16.txt) Unfortunately, it's not clear to me that draft-ietf-dhc-authentication-16.txt is viable for use in securing the boot process without some additional work. As written, the draft assumes that the adapter has been seeded with a DHCP authentication key tied to the DHCP client identifier (e.g. htype/MAC address), computed from the master key. As I understand it, PXE/BIS also assumes the ability to store a public key validating the boot image. Neither spec really provides much insight on how one might obtain proper keying/authentication material to secure the iSCSI boot process. While it might be reasonable to assume that a manufacturer could supply a set of machines programmed with the correct public key to validate the boot image, it seems somewhat of a stretch that the adapters could be programmed on a large scale according to the technique described in -16. Also, in both cases, it would appear that revocation/key change is a huge headache. Note that the master secret described in -16 is not be provided to the individual stations; this is held in confidence by the DHCP server. The upshot is that I would not necessarily assume that we in the IETF really have a good handle on secure boot at this point. From owner-ips@ece.cmu.edu Sun May 27 17:39:26 2001 Received: from ece.cmu.edu (ECE.CMU.EDU [128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id RAA12070 for ; Sun, 27 May 2001 17:39:26 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f4RIfaq22265 for ips-outgoing; Sun, 27 May 2001 14:41:36 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from atlrel2.hp.com (atlrel2.hp.com [156.153.255.202]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f4RIfZ322261 for ; Sun, 27 May 2001 14:41:35 -0400 (EDT) Received: from core.rose.hp.com (core.rose.hp.com [15.43.208.100]) by atlrel2.hp.com (Postfix) with ESMTP id 4FAA5802 for ; Sun, 27 May 2001 14:41:34 -0400 (EDT) Received: (from cbm@localhost) by core.rose.hp.com (8.8.6 (PHNE_14041)/8.8.6 SMKit7.02) id LAA00958 for ips@ece.cmu.edu; Sun, 27 May 2001 11:42:41 -0700 (PDT) Message-Id: <200105271842.LAA00958@core.rose.hp.com> Subject: Re: TCP Framing (considered helpful?) To: ips@ece.cmu.edu Date: Sun, 27 May 2001 11:42:41 PDT Reply-To: cbm@rose.hp.com From: "Mallikarjun C." X-Mailer: Elm [revision: 212.4] Sender: owner-ips@ece.cmu.edu Precedence: bulk John, Thanks for reiterating your concerns. You make a case that non-marker software implementations can push marker-enabled hardware HBAs into becoming more expensive. May be, I would have thought that a non-marker hardware sender is more likely to push a marker-enabled hardware receiver to be expensive... Anyway, isn't this ultimately a performance-cost tradeoff though? That was the point of my message, in particular pointing out that there are no interoperability concerns. -- Mallikarjun Mallikarjun Chadalapaka Networked Storage Architecture Network Storage Solutions Organization MS 5668 Hewlett-Packard, Roseville. cbm@rose.hp.com >Mallikarjun, >We are massively mis-communicating. > >The only issue with Markers/Framing is when SW clients use Markers/Framing. >The first issue is usually performance on the SW side. I do not think that >Markers are a big performance issue. However, I do not have a good handle >with Framing's overhead. However, the other major consideration is -- >when will the functions be supported such that we can exploit them. The >Markers are a function of iSCSI, and the Framing (Warp) is a function of >TCP/IP, and hence a concern about the implementation window for relying on >the Framing availability. > >Markers and Framing were never an issue with HW, we could always implement >them when we create the HBAs. However, for the function to be useful in >reducing the implementation cost of HBAs, the Clients must have functions >that match. Many of these clients will be SW implementation of iSCSI on >Desktops and Laptops. Therefore, the HW vendors will need to insure that >there is universal support for either Framing or Markers so that the >implementation on the HBA can exploit that capability. If it turns out >that they can not depend on rapid implementation of the function on all the >SW Clients, then the HW adapter must assume that the function is not >universal and therefore, they can not depend on it, and must build a more >expensive HW HBA. > >We can cause the Marker to be Mandated as Must Implemented, so we can >assume universal availability of Markers. We can not mandate the >implementation of Framing in all TCP/IP environments. Therefore, the HW >HBAs can not assume that they can use that feature. Hence, they must build >the more expensive HBA. > >Yes, I support Framing and Warp, but the above shows that it can not be >used to permit the building of a lower cost HBA (for some long time). On >the other hand if we can mandate Markers, then the lower cost HW HBA can be >built, and then they can still negotiate higher performance by negotiating >for Framing/Warp, and many times they will find a compatible partner and >then they will function at their optimal rate. > >Perhaps we are on the same page now?? > >. >. >. >John L. Hufferd >Senior Technical Staff Member (STSM) >IBM/SSG San Jose Ca >(408) 256-0403, Tie: 276-0403, eFax: (408) 904-4688 >Internet address: hufferd@us.ibm.com > > >"Mallikarjun C." @ece.cmu.edu on 05/23/2001 02:54:39 PM > >Please respond to cbm@rose.hp.com > >Sent by: owner-ips@ece.cmu.edu > > >To: ips@ece.cmu.edu >cc: >Subject: Re: TCP Framing (considered helpful?) > > > >John, > >Two comments. I may have misunderstood what you're saying since >your initial comments imply framing=warp/RDMA/messageboundary, >and towards the end, you seem to imply framing=markers. > >- It's true that implementing iSCSI-level markers does not require > modifying TCP/IP stacks. But using the markers (I mean to use it > to effectively steer) requires TCP/IP changes! It is a separate > matter that it is offloaded onto the NIC. > >- You seem to implicitly suggest that WARP (when it becomes available) > must go into host software stacks to be useful, and hence cannot be > done in the right timeframe for iSCSI. Since one could envision > offloading WARP onto the NIC as well, I assume you're hinting > a) either at interoperability issues between software > and hardware iSCSI implementations relying on WARP, > b) or at interoperability issues between hardware and > hardware implementations. > > iSCSI currently mandates a "no sync and steering" mode which > ensures interoperability in either case. Perhaps you were really > concerned about performance in case (b) then? >-- >Mallikarjun > > >Mallikarjun Chadalapaka >Networked Storage Architecture >Network Storage Solutions Organization >MS 5668 Hewlett-Packard, Roseville. >cbm@rose.hp.com > > >>Replies in text below (between [Huff] and [/Huff] ). >> >>. >>. >>. >>John L. Hufferd >>Senior Technical Staff Member (STSM) >>IBM/SSG San Jose Ca >>(408) 256-0403, Tie: 276-0403, eFax: (408) 904-4688 >>Internet address: hufferd@us.ibm.com >> >> >>Stephen Bailey @ece.cmu.edu on 05/21/2001 06:50:41 >>AM >> >>Sent by: owner-ips@ece.cmu.edu >> >> >>To: ips@ece.cmu.edu >>cc: >>Subject: Re: TCP Framing (considered helpful?) >> >> >> >>John, >> >>> I think we must depend on Markers to insure that everything can operate >>at >>> top speed, and at the lowest cost. >> >>A key question is whether markers actually ensure that everything >>operates at `top speed, and at the lowest cost'. >> >>Matt thinks so. I (and, presumably those who wrote the framing >>document) think not. >> >>[Huff] I do not think you can say that. I also support framing (Warp), as >>a much more elegant solution, but I find it inappropriate to depend on it >>actually happening, and being made available in the various OSs in the >time >>frame we need. I do believe, that over time it will be made available, >and >>is a better approach for all TCP/IP applications that can use it. [/Huff] >> >>My issue is not even with `lowest cost'. I don't believe markers will >>allow you to run at top speed. Specifically: >> 1) I doubt the feasibility of implementing the control required for >> an eddy buffer (where you store data you can't place) at 10G. >> Admittedly, the validity of this claim can't really be assessed >> without actually working the implementation, so for 99% of the >> list participants (myself included) this is a `yes it is, no it >> isn't' point. >> >> [Huff] I believe this has had much more work done on it then you >> think. I have personally stepped through the proposals from >> several vendors that are working on this option for their HW HBAs. >> Usually, because of the iSCSI PDU headers, the data/commands >> can be placed directly into the SCSI Host buffers, almost every >> time. Only when the PDU headers arrive slightly out of order >> (do to normal routing) are the packets unable to be placed >> directly into the Host buffer. And that requires some, but only >> a small amount, of buffering space. >> It is the packet drops that occur on PDU headers, and resultant >> error retries, that cause the need for large amounts of "on >> HBA/chip" buffering. >> So by using Markers, these HW iSCSI HBAs can limit the amount of >> buffering on the chip/HBAs. [/Huff] >> >> 2) an eddy buffer solution requires some substantial speed-up in >> both the NIC data path, and MOST IMPORTANTLY: the host bus. In >> order to unload the eddy buffer while still handling incoming >> traffic at line rate, clearly the host bus bandwidth must be > >> line rate. >> >> [Huff] This is not an effect of an eddy buffer solution, it is a >> fact that every TCP/IP NIC has to deal with. Especially at the >> new Speeds. Our current PCI buss will not support 10 Gigabit, >further >> PCI-X will not support it either, even PCI-DDR does not fully support >> the full data rate. So it needs to rely on the TCP/IP window >> management. The only other thing you can do is drop the packets. >> this clearly makes the problem worse. [/Huff] >> >> >>I know of at least one general purpose framed solution operating at >>10G which has been available for >3 years (SGI's GSN/ST/XIO NIC). I'm >>sure there are others. >> >>I can't imagine there's any argument that a framed solution would be >>voted `most likely to run fast and be cheap'. Every storage network >>and cluster interconnect has been designed that way since antiquity. >> >>The key tradeoff involves the OS vendors, and I'm wondering why we're >>speaking for them. The question IS, how much more work is it to >>introduce TCP framing over and above what is required to insert iSCSI >>into their network framework. My experience from writing NIC and >>storage drivers for many commercial UNIX-family OSes is: >> 1) it's an easy and well defined process to insert a new SCSI >> transport driver into the SCSI stack. >> 2) it's hard and poorly defined process to insert ANYTHING into the >> network stack. >>[Huff] I think you are making my point. This is the problem with SW >>Stacks. That is why I believe that it will take a very long time for >>the various vendors to include such changes into their "bet you business" >>TCP/IP SW Stacks. The point that Matt and I have been trying to make >>is that most OS vendors are NOT creating the iSCSI HW HBAs (NICs). >>These iSCSI HW HBAs (NICs) have the TCP/IP completely on the HBA, and >>they have added the iSCSI processing also so that they can steer the >>packets directly into the approprate SCSI Host buffers. Adding either >>Markers or Framing into the iSCSI HW HBAs is not a big problem. It is >>only a problem of getting Framing (timely) into Host TCP/IP Stacks. >>[/Huff] >> >>Networking has historically been a user-mode activity. Architected >>services are only provided to user mode programs. Kernel clients have >>been few and far between and so are handled on a case-by-case basis. >>For example NFS. Every OS has hacks to make NFS run fast, but they >>are not stable interfaces for general purpose use. >> >>Even Solaris' SysV-derived STREAMS stack, which is intended precisely >>to provide flexible, crisp interfaces for kernel network clients, does >>not document the relevant (IP stack) intermodule interfaces. >> >>I know that there are more and more kernel network clients, but they >>are coming either on fluid platforms (e.g. linux), in which case the >>argument of `it'll take too long to get OS support' doesn't apply, or >>they are vendor-supplied, in which case a performance iSCSI solution >>in ANY form may take a while, and the choice of framing or markers >>isn't going to make a difference. >> >>[Huff] I think you are saying something I agree with and something I >>do not agree with. That is, that software changes to TCP/IP in the >>various "Bet you Business" OSs, will take some time. However, it is >>not true that new iSCSI device drivers will take very long. Two types >>are being created today. By Cisco, IBM, Intel, etc. These types are >>iSCSI DD that make calls to normal TCP/IP stacks, and the DD that >>are being written by the iSCSI HW HBA vendors. These do not require >>the OS vendor to do anything special. This is happening NOW, >>(Check with CISCO, Intel, and IBM (me?)). The last thing we want >>is to depend on a TCP/IP change to get in the >>way of our momentum. [/Huff] >> >>I can't say squat about the architecture of Winsock, but the fact that >>there is a Microsoft author of the framing proposal who seems very >>serious about supporting framing and RDMA as quickly as possible >>suggests that framing support should be available on Windows very >>soon. >> >>[Huff] My following statements are not meant as a negative of Microsoft. >>However, they and all producers of Key complicated new Software do >>not quickly bring these to the general market in a way that is as >>pleasing to HW vendors as HW vendors would like. >> >>I believe that Microsoft's heart is in the right place on this issue, >>and that they will do the right thing with framing, over time. >>But it is not clear in what release that will be shipped, nor what support >>pack it will be included. Also it is not clear how the support >>will be handled for current Win2k, WinNT etc. >> >>This is why I think we should have Framing a Must implement >>and an Optional to use. It is the easiest thing for SW to >>create, and brings the needed cost reduction to iSCSI HW and >>it is completely under our (iSCSI protocol) control. >>[/Huff] >> >> >> >>Steph >> >> >> >> > > > > > > From owner-ips@ece.cmu.edu Sun May 27 21:00:09 2001 Received: from ece.cmu.edu (ECE.CMU.EDU [128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id VAA12723 for ; Sun, 27 May 2001 21:00:08 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f4RMMtZ04149 for ips-outgoing; Sun, 27 May 2001 18:22:55 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from mail5.microsoft.com (mail5.microsoft.com [131.107.3.121]) by ece.cmu.edu (8.11.0/8.10.2) with SMTP id f4QH4k322378 for ; Sat, 26 May 2001 13:04:46 -0400 (EDT) Received: from 157.54.9.100 by mail5.microsoft.com (InterScan E-Mail VirusWall NT); Sat, 26 May 2001 09:28:24 -0700 (Pacific Daylight Time) Received: from red-imc-02.redmond.corp.microsoft.com ([157.54.9.107]) by inet-imc-03.redmond.corp.microsoft.com with Microsoft SMTPSVC(5.0.2195.2883); Sat, 26 May 2001 09:28:35 -0700 Received: from win-imc-02.wingroup.windeploy.ntdev.microsoft.com ([157.54.0.82]) by red-imc-02.redmond.corp.microsoft.com with Microsoft SMTPSVC(5.0.2195.2883); Sat, 26 May 2001 09:27:45 -0700 Received: from win-msg-02.wingroup.windeploy.ntdev.microsoft.com ([157.54.0.134]) by win-imc-02.wingroup.windeploy.ntdev.microsoft.com with Microsoft SMTPSVC(5.0.2195.1600); Sat, 26 May 2001 09:27:54 -0700 X-MimeOLE: Produced By Microsoft Exchange V6.0.4712.0 content-class: urn:content-classes:message MIME-Version: 1.0 Content-Type: text/plain; charset="US-ASCII" Subject: RE: iSCSI: response to second login (with same ISID) Date: Sat, 26 May 2001 09:27:54 -0700 Message-ID: <8CC03F47967BF14D9710887723FADDB62F5222@win-msg-02.wingroup.windeploy.ntdev.microsoft.com> Thread-Topic: iSCSI: response to second login (with same ISID) Thread-Index: AcDlgE4BWPopcbbmR5yPqt5Y7gIsbAAfvdCw From: "Lakshmi Ramasubramanian" To: "Santosh Rao" Cc: X-OriginalArrivalTime: 26 May 2001 16:27:54.0853 (UTC) FILETIME=[D1144950:01C0E600] Content-Transfer-Encoding: 8bit X-MIME-Autoconverted: from quoted-printable to 8bit by ece.cmu.edu id f4QH4k322380 Sender: owner-ips@ece.cmu.edu Precedence: bulk Content-Transfer-Encoding: 8bit Even if the target is capable of supporting multiple initiators, won't it cause problems with devices such as disk - say, filesystem from the two initiators' side attempt to write to the same sectors (even not intentionally) would cause data corruption. Is iSCSI layer suppose to guard against this type of device sharing? But, if the target serializes access to the device by reserve\release, it would avoid conflict. Is that how it's suppose to work? Thanks! -lakshmi -----Original Message----- From: Santosh Rao [mailto:santoshr@cup.hp.com] Sent: Friday, May 25, 2001 6:09 PM To: Lakshmi Ramasubramanian Cc: ips@ece.cmu.edu Subject: Re: iSCSI: response to second login (with same ISID) Why would the target want to reject the login, unless it is not capable of [or does not wish to] speak to multiple initiators ? In the case of most targets, they should be capable of handling multiple initiators and therefore, should accept the login. If the target is incapable of supporting multiple initiators or does not wish to allow multiple initiators (ex : iSCSI tape drive preventing concurrent access), then, it can reject the login with a status class/response of 0x0205. (Target Conflict). - Santosh Lakshmi Ramasubramanian wrote: > > How is a target driver suppose to respond to a login from an > initiator, when another initiator already has a session going on with > the given target? Is the target > suppose to reject the second login (say, with an error "device busy" or > something simialar)? From owner-ips@ece.cmu.edu Mon May 28 10:05:50 2001 Received: from ece.cmu.edu (ECE.CMU.EDU [128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id KAA04744 for ; Mon, 28 May 2001 10:05:50 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f4SBRNI26245 for ips-outgoing; Mon, 28 May 2001 07:27:23 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from zoetermeer.tripace.com ([217.17.136.66]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f4SBR7326208 for ; Mon, 28 May 2001 07:27:07 -0400 (EDT) Received: by ZOETERMEER with Internet Mail Service (5.5.2653.19) id ; Mon, 28 May 2001 13:24:01 +0200 Message-ID: <8C59010722BBD31194640050DA6EC69711F2D4@ZOETERMEER> From: "Sanjeev Bhagat (TRIPACE/Zoetermeer)" To: "'Lakshmi Ramasubramanian'" , Santosh Rao Cc: ips@ece.cmu.edu Subject: RE: iSCSI: response to second login (with same ISID) Date: Mon, 28 May 2001 13:23:59 +0200 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2653.19) Content-Type: text/plain; charset="iso-8859-1" Sender: owner-ips@ece.cmu.edu Precedence: bulk Hello, Well the conflict of writing to same sector can still be avoided by implementaion of some locks in the target but I wonder what will happen in this case where 2 inititators are connected to a target. Assume this sequence: Initiator 1 issues a read command to check the existance of file location in the iscsi target Assume the command goes with success Initiator 2 issues a write command and changes the location of the file Assume this command also ends with success Initiator 1 tries to read the file from the old location. In this case the command will end up with success but the data that goes to the initiator 1 will be wrong. Regards, Sanjeev -----Original Message----- From: Lakshmi Ramasubramanian [mailto:nramas@windows.microsoft.com] Sent: Saturday, May 26, 2001 9:28 AM To: Santosh Rao Cc: ips@ece.cmu.edu Subject: RE: iSCSI: response to second login (with same ISID) Even if the target is capable of supporting multiple initiators, won't it cause problems with devices such as disk - say, filesystem from the two initiators' side attempt to write to the same sectors (even not intentionally) would cause data corruption. Is iSCSI layer suppose to guard against this type of device sharing? But, if the target serializes access to the device by reserve\release, it would avoid conflict. Is that how it's suppose to work? Thanks! -lakshmi -----Original Message----- From: Santosh Rao [mailto:santoshr@cup.hp.com] Sent: Friday, May 25, 2001 6:09 PM To: Lakshmi Ramasubramanian Cc: ips@ece.cmu.edu Subject: Re: iSCSI: response to second login (with same ISID) Why would the target want to reject the login, unless it is not capable of [or does not wish to] speak to multiple initiators ? In the case of most targets, they should be capable of handling multiple initiators and therefore, should accept the login. If the target is incapable of supporting multiple initiators or does not wish to allow multiple initiators (ex : iSCSI tape drive preventing concurrent access), then, it can reject the login with a status class/response of 0x0205. (Target Conflict). - Santosh Lakshmi Ramasubramanian wrote: > > How is a target driver suppose to respond to a login from an > initiator, when another initiator already has a session going on with > the given target? Is the target > suppose to reject the second login (say, with an error "device busy" or > something simialar)? From owner-ips@ece.cmu.edu Mon May 28 12:08:22 2001 Received: from ece.cmu.edu (ECE.CMU.EDU [128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id MAA05672 for ; Mon, 28 May 2001 12:08:21 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f4SEBpV04521 for ips-outgoing; Mon, 28 May 2001 10:11:51 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from siaar1aa.compuserve.com (siaar1aa.compuserve.com [149.174.40.9]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f4SEBn304509 for ; Mon, 28 May 2001 10:11:49 -0400 (EDT) Received: (from mailgate@localhost) by siaar1aa.compuserve.com (8.9.3/8.9.3/SUN-REL-1.3) id KAA28456 for ips@ece.cmu.edu; Mon, 28 May 2001 10:10:25 -0400 (EDT) Received: from compuserve.com (dal-tgn-tky-vty24.as.wcom.net [216.192.234.24]) by siaar1aa.compuserve.com (8.9.3/8.9.3/SUN-REL-1.3) with ESMTP id KAA28426 for ; Mon, 28 May 2001 10:10:12 -0400 (EDT) Message-ID: <3B125D38.BFF18A8D@compuserve.com> Date: Mon, 28 May 2001 09:14:17 -0500 From: Ralph Weber Reply-To: ENDL_TX@computer.org X-Mailer: Mozilla 4.7 [en]C-CCK-MCD NSCPCD47 (Win95; I) X-Accept-Language: en,pdf MIME-Version: 1.0 To: IPS Reflector Subject: Re: Nashua Interim Minutes References: <0F31E5C394DAD311B60C00E029101A07080155D0@corpmx9.isus.emc.com> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-ips@ece.cmu.edu Precedence: bulk Content-Transfer-Encoding: 7bit I am unclear about this part of the minutes: } Security } -------- } } -- David Black (EMC, co-chair) presentation on security requirements } } See slides - this was an informational presentation/discussion about } requirements (MUST implement authentication and cryptographic } integrity, MUST have a required mechanism for each, } confidentiality is OPTIONAL, IP vs. iSCSI level mechanisms } [iSCSI can multiplex initiators and targets on a single pair). } } -- Ofer Biran (IBM, Haifa) } } Lots of slides - see slides for details. Were there two sets of slides or just the one in Julian's web archive? Thanks. Ralph... From owner-ips@ece.cmu.edu Mon May 28 14:42:17 2001 Received: from ece.cmu.edu (ECE.CMU.EDU [128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id OAA06414 for ; Mon, 28 May 2001 14:42:16 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f4SGefG12792 for ips-outgoing; Mon, 28 May 2001 12:40:41 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from d12lmsgate.de.ibm.com (d12lmsgate.de.ibm.com [195.212.91.199]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f4SGeZ312781 for ; Mon, 28 May 2001 12:40:35 -0400 (EDT) Received: from d12relay01.de.ibm.com (d12relay01.de.ibm.com [9.165.215.22]) by d12lmsgate.de.ibm.com (1.0.0) with ESMTP id SAA343486 for ; Mon, 28 May 2001 18:40:29 +0200 From: julian_satran@il.ibm.com Received: from d12mta02.de.ibm.com (d12mta01_cs0 [9.165.222.237]) by d12relay01.de.ibm.com (8.8.8m3/NCO v4.96) with SMTP id SAA78432 for ; Mon, 28 May 2001 18:40:28 +0200 Received: by d12mta02.de.ibm.com(Lotus SMTP MTA v4.6.5 (863.2 5-20-1999)) id C1256A5A.005B8A84 ; Mon, 28 May 2001 18:39:52 +0200 X-Lotus-FromDomain: IBMIL@IBMDE To: ips@ece.cmu.edu Message-ID: Date: Mon, 28 May 2001 14:53:54 +0300 Subject: RE: iSCSI - opcodes Mime-Version: 1.0 Content-type: text/plain; charset=us-ascii Content-Disposition: inline Sender: owner-ips@ece.cmu.edu Precedence: bulk The new codes are in the 07 draft. Although I've put back the bit that shows direction I seriously doubt that it's original purpose (to make life easier for analyzers) is as relevant as it was. Julo "Martin, Nick" on 01-05-2001 01:48:00 Please respond to "Martin, Nick" To: Julian Satran/Haifa/IBM@IBMIL cc: Subject: RE: iSCSI - opcodes Julian, I was not among the incensed. I would say puzzled. Although the bit may not be not critical to initiators or targets who presumably know what they are, it is handy at least for network analyzer displays. Since it now still consumes a bit, I would have made it the MSB of the Opcode field as in previous drafts (currently bit 5 or 0x20). However I can cope with your selection ((Opcode & 0x3e)>>1) :). The 2F reject instead of 3F (or 1f) for reject was another puzzler. Presumably you wanted a fence between unused reserved opcodes and vendor specific codes. Thanks, Nick Martin -----Original Message----- From: julian_satran@il.ibm.com [mailto:julian_satran@il.ibm.com] Sent: Friday, April 27, 2001 12:38 AM To: ips@ece.cmu.edu Cc: matt_wakeley@agilent.com Subject: iSCSI - opcodes Fellow iSCSI fans, Some were incesed by the lak of a "direction bit" in the opcodes in draft 06. Here is an attempt to a new list (having a bit for direction back - as the LSB ). To gain some reserved space I've curtailed the vendo-specific codes to 4 in each direction. Please comment, Julo 1.1.1.1 Opcode The Opcode indicates what type of iSCSI PDU the header encapsulates. The Opcodes are divided into two categories: initiator opcodes and target opcodes. Initiator opcodes are in PDUs sent by the initiators (request PDUs), and target opcodes are in PDUs sent by the target (response PDUs). Initiators MUST NOT use target opcodes and targets MUST NOT use initiator opcodes. Valid initiator opcodes defined in this specification are: 0x00 NOP-Out (from initiator to target) 0x02 SCSI Command (encapsulates a SCSI Command Descriptor Block) 0x04 SCSI Task Management Command 0x06 Login Command 0x08 Text Command 0x0a SCSI Data-out (for WRITE operations) 0x0c Logout Command 0x10 SNACK Request Valid target opcodes are: 0x01 NOP-In (from target to initiator) 0x03 SCSI Response (contains SCSI status and possibly sense information or other response information) 0x05 SCSI Task Management Response 0x07 Login Response 0x09 Text Response 0x0b SCSI Data-in (for READ operations) 0x0d Logout Response 0x11 Ready To Transfer (R2T - sent by target to initiator when it is ready to receive data from initiator) 0x13 Asynchronous Message (sent by target to initiator to indicate certain special conditions) 0x2f Reject Initiator opcodes 0x38, 0x3a, 0x3c and 0x3e and target opcodes 0x39, 0x3b, 0x3d and 0x3f are vendor specific codes. From owner-ips@ece.cmu.edu Mon May 28 14:43:04 2001 Received: from ece.cmu.edu (ECE.CMU.EDU [128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id OAA06430 for ; Mon, 28 May 2001 14:43:04 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f4SGebp12790 for ips-outgoing; Mon, 28 May 2001 12:40:37 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from d12lmsgate-2.de.ibm.com (d12lmsgate-2.de.ibm.com [195.212.91.200]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f4SGeZ312782 for ; Mon, 28 May 2001 12:40:35 -0400 (EDT) Received: from d12relay01.de.ibm.com (d12relay01.de.ibm.com [9.165.215.22]) by d12lmsgate-2.de.ibm.com (1.0.0) with ESMTP id SAA213624; Mon, 28 May 2001 18:40:28 +0200 From: julian_satran@il.ibm.com Received: from d12mta02.de.ibm.com (d12mta01_cs0 [9.165.222.237]) by d12relay01.de.ibm.com (8.8.8m3/NCO v4.96) with SMTP id SAA83398; Mon, 28 May 2001 18:40:22 +0200 Received: by d12mta02.de.ibm.com(Lotus SMTP MTA v4.6.5 (863.2 5-20-1999)) id C1256A5A.005B8837 ; Mon, 28 May 2001 18:39:46 +0200 X-Lotus-FromDomain: IBMIL@IBMDE To: Bernard Aboba cc: ips@ece.cmu.edu Message-ID: Date: Mon, 28 May 2001 01:01:12 +0300 Subject: Re: iSCSI and secure boot Mime-Version: 1.0 Content-type: text/plain; charset=us-ascii Content-Disposition: inline Sender: owner-ips@ece.cmu.edu Precedence: bulk Bernard, As iSCSI is transporting SCSI and SCSI has a different boot paradigm than Netboot can you please elaborate on what exactly should be an authenticated boot image in this (SCSI) context. Please take into consideration that unlike netboot - the SCSI boot is not a clearly bounded process (not even in PXE - an "open" proprietary scheme). Julo Bernard Aboba on 27-05-2001 19:22:47 Please respond to Bernard Aboba To: Douglas Otis cc: David Robinson , ips@ece.cmu.edu, narten@raleigh.ibm.com Subject: iSCSI and secure boot > Security is actively being worked on the the DHCP community so that > is something that iSCSI can leverage. > (draft-ietf-dhc-authentication-16.txt) Unfortunately, it's not clear to me that draft-ietf-dhc-authentication-16.txt is viable for use in securing the boot process without some additional work. As written, the draft assumes that the adapter has been seeded with a DHCP authentication key tied to the DHCP client identifier (e.g. htype/MAC address), computed from the master key. As I understand it, PXE/BIS also assumes the ability to store a public key validating the boot image. Neither spec really provides much insight on how one might obtain proper keying/authentication material to secure the iSCSI boot process. While it might be reasonable to assume that a manufacturer could supply a set of machines programmed with the correct public key to validate the boot image, it seems somewhat of a stretch that the adapters could be programmed on a large scale according to the technique described in -16. Also, in both cases, it would appear that revocation/key change is a huge headache. Note that the master secret described in -16 is not be provided to the individual stations; this is held in confidence by the DHCP server. The upshot is that I would not necessarily assume that we in the IETF really have a good handle on secure boot at this point. From owner-ips@ece.cmu.edu Mon May 28 17:45:30 2001 Received: from ece.cmu.edu (ECE.CMU.EDU [128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id RAA07462 for ; Mon, 28 May 2001 17:45:29 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f4SJpiJ23713 for ips-outgoing; Mon, 28 May 2001 15:51:44 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from e31.bld.us.ibm.com (e31.co.us.ibm.com [32.97.110.129]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f4SJpT323705 for ; Mon, 28 May 2001 15:51:29 -0400 (EDT) Received: from westrelay02.boulder.ibm.com (westrelay02.boulder.ibm.com [9.99.140.23]) by e31.bld.us.ibm.com (8.9.3/8.9.3) with ESMTP id PAA53894; Mon, 28 May 2001 15:43:25 -0400 Received: from f4n49e (d03nm065h.boulder.ibm.com [9.99.140.49]) by westrelay02.boulder.ibm.com (8.11.1/NCO v4.96.1.0) with ESMTP id f4SJowY181148; Mon, 28 May 2001 13:51:03 -0600 X-Priority: 1 (High) Importance: Normal Subject: Re: TCP Framing (considered helpful?) To: cbm@rose.hp.com Cc: ips@ece.cmu.edu X-Mailer: Lotus Notes Release 5.0.3 (Intl) 21 March 2000 Message-ID: From: "John Hufferd" Date: Mon, 28 May 2001 12:50:25 -0700 X-MIMETrack: Serialize by Router on D03NM065/03/M/IBM(Release 5.0.6 |December 14, 2000) at 05/28/2001 01:51:02 PM MIME-Version: 1.0 Content-type: text/plain; charset=us-ascii Sender: owner-ips@ece.cmu.edu Precedence: bulk Mallikarjun, I think that both are the case. But, the probable reason for the Non-Marker HW to exist is that it needs to work with other Non-Marker Stuff (HW/SW). So since not being able to have Marker Support assumed to be present on the Session partner (HW or SW) causes HW HBAs to be more expensive. I think our goal should be to put the "Must Implement" stuff in place so that the right execution choices can be made, and that the protocol permits inexpensive products. The point is, you can only make a reasonable performance cost trade off -- if -- you have a choice at execution time. The only way this will work is to have Markers as a "Must Implement" and Optional to use. . . . John L. Hufferd Senior Technical Staff Member (STSM) IBM/SSG San Jose Ca (408) 256-0403, Tie: 276-0403, eFax: (408) 904-4688 Internet address: hufferd@us.ibm.com "Mallikarjun C." @ece.cmu.edu on 05/27/2001 11:42:41 AM Please respond to cbm@rose.hp.com Sent by: owner-ips@ece.cmu.edu To: ips@ece.cmu.edu cc: Subject: Re: TCP Framing (considered helpful?) John, Thanks for reiterating your concerns. You make a case that non-marker software implementations can push marker-enabled hardware HBAs into becoming more expensive. May be, I would have thought that a non-marker hardware sender is more likely to push a marker-enabled hardware receiver to be expensive... Anyway, isn't this ultimately a performance-cost tradeoff though? That was the point of my message, in particular pointing out that there are no interoperability concerns. -- Mallikarjun Mallikarjun Chadalapaka Networked Storage Architecture Network Storage Solutions Organization MS 5668 Hewlett-Packard, Roseville. cbm@rose.hp.com >Mallikarjun, >We are massively mis-communicating. > >The only issue with Markers/Framing is when SW clients use Markers/Framing. >The first issue is usually performance on the SW side. I do not think that >Markers are a big performance issue. However, I do not have a good handle >with Framing's overhead. However, the other major consideration is -- >when will the functions be supported such that we can exploit them. The >Markers are a function of iSCSI, and the Framing (Warp) is a function of >TCP/IP, and hence a concern about the implementation window for relying on >the Framing availability. > >Markers and Framing were never an issue with HW, we could always implement >them when we create the HBAs. However, for the function to be useful in >reducing the implementation cost of HBAs, the Clients must have functions >that match. Many of these clients will be SW implementation of iSCSI on >Desktops and Laptops. Therefore, the HW vendors will need to insure that >there is universal support for either Framing or Markers so that the >implementation on the HBA can exploit that capability. If it turns out >that they can not depend on rapid implementation of the function on all the >SW Clients, then the HW adapter must assume that the function is not >universal and therefore, they can not depend on it, and must build a more >expensive HW HBA. > >We can cause the Marker to be Mandated as Must Implemented, so we can >assume universal availability of Markers. We can not mandate the >implementation of Framing in all TCP/IP environments. Therefore, the HW >HBAs can not assume that they can use that feature. Hence, they must build >the more expensive HBA. > >Yes, I support Framing and Warp, but the above shows that it can not be >used to permit the building of a lower cost HBA (for some long time). On >the other hand if we can mandate Markers, then the lower cost HW HBA can be >built, and then they can still negotiate higher performance by negotiating >for Framing/Warp, and many times they will find a compatible partner and >then they will function at their optimal rate. > >Perhaps we are on the same page now?? > >. >. >. >John L. Hufferd >Senior Technical Staff Member (STSM) >IBM/SSG San Jose Ca >(408) 256-0403, Tie: 276-0403, eFax: (408) 904-4688 >Internet address: hufferd@us.ibm.com > > >"Mallikarjun C." @ece.cmu.edu on 05/23/2001 02:54:39 PM > >Please respond to cbm@rose.hp.com > >Sent by: owner-ips@ece.cmu.edu > > >To: ips@ece.cmu.edu >cc: >Subject: Re: TCP Framing (considered helpful?) > > > >John, > >Two comments. I may have misunderstood what you're saying since >your initial comments imply framing=warp/RDMA/messageboundary, >and towards the end, you seem to imply framing=markers. > >- It's true that implementing iSCSI-level markers does not require > modifying TCP/IP stacks. But using the markers (I mean to use it > to effectively steer) requires TCP/IP changes! It is a separate > matter that it is offloaded onto the NIC. > >- You seem to implicitly suggest that WARP (when it becomes available) > must go into host software stacks to be useful, and hence cannot be > done in the right timeframe for iSCSI. Since one could envision > offloading WARP onto the NIC as well, I assume you're hinting > a) either at interoperability issues between software > and hardware iSCSI implementations relying on WARP, > b) or at interoperability issues between hardware and > hardware implementations. > > iSCSI currently mandates a "no sync and steering" mode which > ensures interoperability in either case. Perhaps you were really > concerned about performance in case (b) then? >-- >Mallikarjun > > >Mallikarjun Chadalapaka >Networked Storage Architecture >Network Storage Solutions Organization >MS 5668 Hewlett-Packard, Roseville. >cbm@rose.hp.com > > >>Replies in text below (between [Huff] and [/Huff] ). >> >>. >>. >>. >>John L. Hufferd >>Senior Technical Staff Member (STSM) >>IBM/SSG San Jose Ca >>(408) 256-0403, Tie: 276-0403, eFax: (408) 904-4688 >>Internet address: hufferd@us.ibm.com >> >> >>Stephen Bailey @ece.cmu.edu on 05/21/2001 06:50:41 >>AM >> >>Sent by: owner-ips@ece.cmu.edu >> >> >>To: ips@ece.cmu.edu >>cc: >>Subject: Re: TCP Framing (considered helpful?) >> >> >> >>John, >> >>> I think we must depend on Markers to insure that everything can operate >>at >>> top speed, and at the lowest cost. >> >>A key question is whether markers actually ensure that everything >>operates at `top speed, and at the lowest cost'. >> >>Matt thinks so. I (and, presumably those who wrote the framing >>document) think not. >> >>[Huff] I do not think you can say that. I also support framing (Warp), as >>a much more elegant solution, but I find it inappropriate to depend on it >>actually happening, and being made available in the various OSs in the >time >>frame we need. I do believe, that over time it will be made available, >and >>is a better approach for all TCP/IP applications that can use it. [/Huff] >> >>My issue is not even with `lowest cost'. I don't believe markers will >>allow you to run at top speed. Specifically: >> 1) I doubt the feasibility of implementing the control required for >> an eddy buffer (where you store data you can't place) at 10G. >> Admittedly, the validity of this claim can't really be assessed >> without actually working the implementation, so for 99% of the >> list participants (myself included) this is a `yes it is, no it >> isn't' point. >> >> [Huff] I believe this has had much more work done on it then you >> think. I have personally stepped through the proposals from >> several vendors that are working on this option for their HW HBAs. >> Usually, because of the iSCSI PDU headers, the data/commands >> can be placed directly into the SCSI Host buffers, almost every >> time. Only when the PDU headers arrive slightly out of order >> (do to normal routing) are the packets unable to be placed >> directly into the Host buffer. And that requires some, but only >> a small amount, of buffering space. >> It is the packet drops that occur on PDU headers, and resultant >> error retries, that cause the need for large amounts of "on >> HBA/chip" buffering. >> So by using Markers, these HW iSCSI HBAs can limit the amount of >> buffering on the chip/HBAs. [/Huff] >> >> 2) an eddy buffer solution requires some substantial speed-up in >> both the NIC data path, and MOST IMPORTANTLY: the host bus. In >> order to unload the eddy buffer while still handling incoming >> traffic at line rate, clearly the host bus bandwidth must be > >> line rate. >> >> [Huff] This is not an effect of an eddy buffer solution, it is a >> fact that every TCP/IP NIC has to deal with. Especially at the >> new Speeds. Our current PCI buss will not support 10 Gigabit, >further >> PCI-X will not support it either, even PCI-DDR does not fully support >> the full data rate. So it needs to rely on the TCP/IP window >> management. The only other thing you can do is drop the packets. >> this clearly makes the problem worse. [/Huff] >> >> >>I know of at least one general purpose framed solution operating at >>10G which has been available for >3 years (SGI's GSN/ST/XIO NIC). I'm >>sure there are others. >> >>I can't imagine there's any argument that a framed solution would be >>voted `most likely to run fast and be cheap'. Every storage network >>and cluster interconnect has been designed that way since antiquity. >> >>The key tradeoff involves the OS vendors, and I'm wondering why we're >>speaking for them. The question IS, how much more work is it to >>introduce TCP framing over and above what is required to insert iSCSI >>into their network framework. My experience from writing NIC and >>storage drivers for many commercial UNIX-family OSes is: >> 1) it's an easy and well defined process to insert a new SCSI >> transport driver into the SCSI stack. >> 2) it's hard and poorly defined process to insert ANYTHING into the >> network stack. >>[Huff] I think you are making my point. This is the problem with SW >>Stacks. That is why I believe that it will take a very long time for >>the various vendors to include such changes into their "bet you business" >>TCP/IP SW Stacks. The point that Matt and I have been trying to make >>is that most OS vendors are NOT creating the iSCSI HW HBAs (NICs). >>These iSCSI HW HBAs (NICs) have the TCP/IP completely on the HBA, and >>they have added the iSCSI processing also so that they can steer the >>packets directly into the approprate SCSI Host buffers. Adding either >>Markers or Framing into the iSCSI HW HBAs is not a big problem. It is >>only a problem of getting Framing (timely) into Host TCP/IP Stacks. >>[/Huff] >> >>Networking has historically been a user-mode activity. Architected >>services are only provided to user mode programs. Kernel clients have >>been few and far between and so are handled on a case-by-case basis. >>For example NFS. Every OS has hacks to make NFS run fast, but they >>are not stable interfaces for general purpose use. >> >>Even Solaris' SysV-derived STREAMS stack, which is intended precisely >>to provide flexible, crisp interfaces for kernel network clients, does >>not document the relevant (IP stack) intermodule interfaces. >> >>I know that there are more and more kernel network clients, but they >>are coming either on fluid platforms (e.g. linux), in which case the >>argument of `it'll take too long to get OS support' doesn't apply, or >>they are vendor-supplied, in which case a performance iSCSI solution >>in ANY form may take a while, and the choice of framing or markers >>isn't going to make a difference. >> >>[Huff] I think you are saying something I agree with and something I >>do not agree with. That is, that software changes to TCP/IP in the >>various "Bet you Business" OSs, will take some time. However, it is >>not true that new iSCSI device drivers will take very long. Two types >>are being created today. By Cisco, IBM, Intel, etc. These types are >>iSCSI DD that make calls to normal TCP/IP stacks, and the DD that >>are being written by the iSCSI HW HBA vendors. These do not require >>the OS vendor to do anything special. This is happening NOW, >>(Check with CISCO, Intel, and IBM (me?)). The last thing we want >>is to depend on a TCP/IP change to get in the >>way of our momentum. [/Huff] >> >>I can't say squat about the architecture of Winsock, but the fact that >>there is a Microsoft author of the framing proposal who seems very >>serious about supporting framing and RDMA as quickly as possible >>suggests that framing support should be available on Windows very >>soon. >> >>[Huff] My following statements are not meant as a negative of Microsoft. >>However, they and all producers of Key complicated new Software do >>not quickly bring these to the general market in a way that is as >>pleasing to HW vendors as HW vendors would like. >> >>I believe that Microsoft's heart is in the right place on this issue, >>and that they will do the right thing with framing, over time. >>But it is not clear in what release that will be shipped, nor what support >>pack it will be included. Also it is not clear how the support >>will be handled for current Win2k, WinNT etc. >> >>This is why I think we should have Framing a Must implement >>and an Optional to use. It is the easiest thing for SW to >>create, and brings the needed cost reduction to iSCSI HW and >>it is completely under our (iSCSI protocol) control. >>[/Huff] >> >> >> >>Steph >> >> >> >> > > > > > > From owner-ips@ece.cmu.edu Mon May 28 20:39:23 2001 Received: from ece.cmu.edu (ECE.CMU.EDU [128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id UAA08313 for ; Mon, 28 May 2001 20:39:23 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f4SMjOT03249 for ips-outgoing; Mon, 28 May 2001 18:45:24 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from bacchus-int.veritas.com (bacchus.veritas.com [204.177.156.37]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f4SMjN303245 for ; Mon, 28 May 2001 18:45:23 -0400 (EDT) Received: from megami.veritas.com (urd.veritas.com [192.203.47.101]) by bacchus-int.veritas.com (8.11.2/8.11.2) with SMTP id f4SMjHt00420; Mon, 28 May 2001 15:45:17 -0700 (PDT) Received: from localhost (1776 bytes) by megami.veritas.com via sendmail with P:esmtp/R:smart_host/T:smtp (sender: ) id for ; Mon, 28 May 2001 15:45:13 -0700 (PDT) (Smail-3.2.0.101 1997-Dec-17 #4 built 1999-Aug-24) Date: Mon, 28 May 2001 15:45:13 -0700 (PDT) From: Patrick Stirling To: "Sanjeev Bhagat (TRIPACE/Zoetermeer)" cc: "'Lakshmi Ramasubramanian'" , Santosh Rao , ips@ece.cmu.edu Subject: RE: iSCSI: response to second login (with same ISID) In-Reply-To: <8C59010722BBD31194640050DA6EC69711F2D4@ZOETERMEER> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-ips@ece.cmu.edu Precedence: bulk On Mon, 28 May 2001, Sanjeev Bhagat (TRIPACE/Zoetermeer) wrote: > Well the conflict of writing to same sector can still be avoided by > implementaion of some locks in the target but I wonder what will happen in > this case where 2 inititators are connected to a target. > From: Lakshmi Ramasubramanian [mailto:nramas@windows.microsoft.com] > > Even if the target is capable of supporting multiple initiators, won't > it cause problems with devices such as disk - say, filesystem from the > two initiators' side attempt to write > to the same sectors (even not intentionally) would cause data > corruption. Is iSCSI layer suppose to guard against this type of > device sharing? Umm, am I missing something here? If 2 initiators want to access the same SCSI device, they must coordinate to avoid clashing with each other. It's not the device's responsibility to prevent this. This is one reason several companies have complicated clustering software. patrick Patrick Stirling VERITAS Software (vendor of complicated clustering software!) From owner-ips@ece.cmu.edu Mon May 28 23:06:15 2001 Received: from ece.cmu.edu (ECE.CMU.EDU [128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id XAA10855 for ; Mon, 28 May 2001 23:06:15 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f4T1PX612067 for ips-outgoing; Mon, 28 May 2001 21:25:33 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from ztxmail05.ztx.compaq.com (ztxmail05.ztx.compaq.com [161.114.1.209]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f4T1PW312063 for ; Mon, 28 May 2001 21:25:32 -0400 (EDT) Received: by ztxmail05.ztx.compaq.com (Postfix, from userid 12345) id 2580C2D3; Mon, 28 May 2001 20:25:27 -0500 (CDT) Received: from exchou-gh02.cca.cpqcorp.net (exchou-gh02.cca.cpqcorp.net [16.110.248.202]) by ztxmail05.ztx.compaq.com (Postfix) with ESMTP id 1BF9E1CE for ; Mon, 28 May 2001 20:25:27 -0500 (CDT) Received: by exchou-gh02.cca.cpqcorp.net with Internet Mail Service (5.5.2652.78) id ; Mon, 28 May 2001 20:25:26 -0500 Message-ID: <78AF3C342AEAEF4BA33B35A8A15668C659C1EA@cceexc17.americas.cpqcorp.net> From: "Elliott, Robert" To: ips@ece.cmu.edu Subject: RE: iSCSI: multiple intiaitor conflicting with target Date: Mon, 28 May 2001 20:25:25 -0500 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2652.78) Content-Type: text/plain; charset="iso-8859-1" Sender: owner-ips@ece.cmu.edu Precedence: bulk This is a general SCSI issue, not just an iSCSI issue. If you're sharing a target but not a logical unit, then your only concern is TARGET RESET. Access Controls (see T10/99-245r9 and other documents on http://www.t10.org) helps with this level of sharing. If multiple initiators try to use the same iSCSI Name you'll create extra trouble at the iSCSI level, but that's not supposed to happen. If you're sharing a logical unit, you need to use tools like reservations (see SPC-2), persistent reservations (see SPC-2), device locks/memory export commands (see T10/00-312), and/or back-channel clustering coordination algorithms (Veritas Clustering, Microsoft Clustering, VMS Clustering, etc). --- Rob Elliott, Compaq Server Storage Robert.Elliott@compaq.com > -----Original Message----- > From: Sanjeev Bhagat (TRIPACE/Zoetermeer) [mailto:sbhagat@tripace.com] > Sent: Monday, May 28, 2001 7:58 PM > To: Patrick Stirling > Cc: 'Lakshmi Ramasubramanian'; Santosh Rao; ips@ece.cmu.edu > Subject: Re: iSCSI: multiple intiaitor conflicting with > target (was Re: > iSCSI: response to second login (with same ISID) > > > Hello Patrick, > > Since the initiators are two different entities working completely > independent of each other they cannot co-ordinate with each > other. iSCSI > Target also sees these two different initiators in different > sessions as > seperate entities. In the current implementation of iSCSI I > guess it is not > possible for two different initiators to know about each > other and so it is > difficult to resolve this conflicting situation. > > The answer to this problem might be lying in SAM-2 > specifications. May be I > am missing something there. Can somebody comment? > > Sanjeev > > > > > On Mon, 28 May 2001, Sanjeev Bhagat (TRIPACE/Zoetermeer) wrote: > > > > > Well the conflict of writing to same sector can still be > avoided by > > > implementaion of some locks in the target but I wonder > what will happen > in > > > this case where 2 inititators are connected to a target. > > > > > From: Lakshmi Ramasubramanian > [mailto:nramas@windows.microsoft.com] > > > > > > Even if the target is capable of supporting multiple > initiators, won't > > > it cause problems with devices such as disk - say, > filesystem from the > > > two initiators' side attempt to write > > > to the same sectors (even not intentionally) would cause data > > > corruption. Is iSCSI layer suppose to guard against this type of > > > device sharing? > > > > Umm, am I missing something here? > > > > If 2 initiators want to access the same SCSI device, they > must coordinate > > to avoid clashing with each other. It's not the device's > responsibility to > > prevent this. This is one reason several companies have complicated > > clustering software. > > > > patrick > > Patrick Stirling > > VERITAS Software (vendor of complicated clustering software!) > > > From owner-ips@ece.cmu.edu Mon May 28 23:10:04 2001 Received: from ece.cmu.edu (ECE.CMU.EDU [128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id XAA10915 for ; Mon, 28 May 2001 23:10:04 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f4T0p7T10099 for ips-outgoing; Mon, 28 May 2001 20:51:07 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from smtp02.wxs.nl (smtp02.wxs.nl [195.121.6.54]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f4T0p5310094 for ; Mon, 28 May 2001 20:51:05 -0400 (EDT) Received: from daniel ([213.10.179.111]) by smtp02.wxs.nl (Netscape Messaging Server 4.05) with SMTP id GE2NOY01.IR3; Tue, 29 May 2001 02:50:58 +0200 Message-ID: <002b01c0e7da$5bf8c250$9600000a@daniel> From: "Sanjeev Bhagat \(TRIPACE/Zoetermeer\)" To: "Patrick Stirling" Cc: "'Lakshmi Ramasubramanian'" , "Santosh Rao" , References: Subject: Re: iSCSI: multiple intiaitor conflicting with target (was Re: iSCSI: response to second login (with same ISID) Date: Tue, 29 May 2001 02:57:39 +0200 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 5.50.4133.2400 Disposition-Notification-To: "Sanjeev Bhagat \(TRIPACE/Zoetermeer\)" X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4133.2400 Sender: owner-ips@ece.cmu.edu Precedence: bulk Content-Transfer-Encoding: 7bit Hello Patrick, Since the initiators are two different entities working completely independent of each other they cannot co-ordinate with each other. iSCSI Target also sees these two different initiators in different sessions as seperate entities. In the current implementation of iSCSI I guess it is not possible for two different initiators to know about each other and so it is difficult to resolve this conflicting situation. The answer to this problem might be lying in SAM-2 specifications. May be I am missing something there. Can somebody comment? Sanjeev > > On Mon, 28 May 2001, Sanjeev Bhagat (TRIPACE/Zoetermeer) wrote: > > > Well the conflict of writing to same sector can still be avoided by > > implementaion of some locks in the target but I wonder what will happen in > > this case where 2 inititators are connected to a target. > > > From: Lakshmi Ramasubramanian [mailto:nramas@windows.microsoft.com] > > > > Even if the target is capable of supporting multiple initiators, won't > > it cause problems with devices such as disk - say, filesystem from the > > two initiators' side attempt to write > > to the same sectors (even not intentionally) would cause data > > corruption. Is iSCSI layer suppose to guard against this type of > > device sharing? > > Umm, am I missing something here? > > If 2 initiators want to access the same SCSI device, they must coordinate > to avoid clashing with each other. It's not the device's responsibility to > prevent this. This is one reason several companies have complicated > clustering software. > > patrick > Patrick Stirling > VERITAS Software (vendor of complicated clustering software!) > From owner-ips@ece.cmu.edu Tue May 29 00:21:54 2001 Received: from ece.cmu.edu (ECE.CMU.EDU [128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id AAA11657 for ; Tue, 29 May 2001 00:21:53 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f4T2SaI15565 for ips-outgoing; Mon, 28 May 2001 22:28:36 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from bacchus-int.veritas.com (bacchus.veritas.com [204.177.156.37]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f4T2SZ315560 for ; Mon, 28 May 2001 22:28:35 -0400 (EDT) Received: from megami.veritas.com (megami-13.veritas.com [166.98.13.101]) by bacchus-int.veritas.com (8.11.2/8.11.2) with SMTP id f4T2SVt13774; Mon, 28 May 2001 19:28:31 -0700 (PDT) Received: from localhost (2133 bytes) by megami.veritas.com via sendmail with P:esmtp/R:smart_host/T:smtp (sender: ) id for ; Mon, 28 May 2001 19:28:27 -0700 (PDT) (Smail-3.2.0.101 1997-Dec-17 #4 built 1999-Aug-24) Date: Mon, 28 May 2001 19:28:27 -0700 (PDT) From: Patrick Stirling To: "Sanjeev Bhagat (TRIPACE/Zoetermeer)" cc: "'Lakshmi Ramasubramanian'" , Santosh Rao , ips@ece.cmu.edu Subject: Re: iSCSI: multiple intiaitor conflicting with target (was Re: iSCSI: response to second login (with same ISID) In-Reply-To: <002b01c0e7da$5bf8c250$9600000a@daniel> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-ips@ece.cmu.edu Precedence: bulk On Tue, 29 May 2001, Sanjeev Bhagat (TRIPACE/Zoetermeer) wrote: > Since the initiators are two different entities working completely > independent of each other they cannot co-ordinate with each other. iSCSI > Target also sees these two different initiators in different sessions as > seperate entities. In the current implementation of iSCSI I guess it is not > possible for two different initiators to know about each other and so it is > difficult to resolve this conflicting situation. > > The answer to this problem might be lying in SAM-2 specifications. May be I > am missing something there. Can somebody comment? As far as I'm aware, you have two options: - the initiators co-operate to share the device - the initiators make reservations to lock each other out (in which case one could assert that they have indirect knowledge of each other) BTW this is a general SCSI issue, not confined to iSCSI. Note that you must also be careful about target resets & other task management commands here. Also note that when devices are shared it's usually done above the initiator level, i.e. there's an application using the initiator to access the device, and it's aware of other initiators. One might say "it's a layering issue" - SCSI provides reservations to help share devices, but it's up to higher layers to coordinate the sharing. patrick From owner-ips@ece.cmu.edu Tue May 29 04:45:27 2001 Received: from ece.cmu.edu (ECE.CMU.EDU [128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id EAA27473 for ; Tue, 29 May 2001 04:45:27 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f4T6WRe29724 for ips-outgoing; Tue, 29 May 2001 02:32:27 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from ztxmail05.ztx.compaq.com (ztxmail05.ztx.compaq.com [161.114.1.209]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f4T6WQ329720 for ; Tue, 29 May 2001 02:32:26 -0400 (EDT) Received: by ztxmail05.ztx.compaq.com (Postfix, from userid 12345) id 7D20336D; Tue, 29 May 2001 01:32:20 -0500 (CDT) Received: from exchou-gh02.cca.cpqcorp.net (exchou-gh02.cca.cpqcorp.net [16.110.248.202]) by ztxmail05.ztx.compaq.com (Postfix) with ESMTP id 71347329; Tue, 29 May 2001 01:32:20 -0500 (CDT) Received: by exchou-gh02.cca.cpqcorp.net with Internet Mail Service (5.5.2652.78) id ; Tue, 29 May 2001 01:32:20 -0500 Message-ID: <31891B757C09184BBFEC5275F85D5595104D68@cceexc18.americas.cpqcorp.net> From: "Martin, Nick" To: "'julian_satran@il.ibm.com'" Cc: ips@ece.cmu.edu Subject: RE: iSCSI: response to second login (with same ISID) Date: Tue, 29 May 2001 01:32:22 -0500 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2652.78) Content-Type: text/plain; charset="iso-8859-1" Sender: owner-ips@ece.cmu.edu Precedence: bulk Julian, Yes, there is nothing preventing a target from being "well behaved", there is also nothing to assure the initiator that any particular target will be "well behaved". At the moment, I think we do not have a definition of "well behaved" in terms of how long it should take a target to discover loss of connection to an initiator. Thus some folks are sure it will be "too long" and that we need a way to circumvent waiting for it. IMHO We should either agree in advance on such parameters for all targets (if we could), or allow all initiators to set, or at least query some behavior defining parameter values used by their targets. How often should a "well behaved" iSCSI target test each of its connections? How long should a target wait for the response to a "NOP-OUT" (a.k.a. iSCSI ping) before presuming the connection failed? Is more than one attempt required? If this behavior should be configurable, should it be configurable by the administrator of the target, or the administrator of the initiator, or negotiated? (For leased storage, these two administrators may not even work for the same employer.) To whom should it be visible/known? I presume any connection is considered tested&OK for N seconds after the latest valid PDU arrives. Is a value for N discussed? How long before or after N seconds of inactivity should we begin testing the connection? Does this apply equally to targets and initiators listening on their iSCSI connections? If I interpret correctly, we presently have some parameters (negotiated) for how long after detecting a connection failure, the target must preserve the session state to allow the initiator to attempt to reconnect. It may be that something already dictates "it will take 2.5 to 3.0 minutes" (seconds? hours?) from the moment of initiator cease function until target detection of connection failure, and this is an inherent characteristic of iSCSI. If this should be the case, then some folks will say those times are fine with me, others may say that's too long. Those who can not live with our protocol can deploy something else. However, I do not see what is currently dictating how long this should or could take. I hope to not exclude applications with reasonably strict recovery time requirements. I also hope not to ping every second on every idle iSCSI connection only to hasten recovery for a few applications. Thanks, Nick -----Original Message----- From: julian_satran@il.ibm.com [mailto:julian_satran@il.ibm.com] Sent: Saturday, May 26, 2001 7:42 AM To: ips@ece.cmu.edu Subject: RE: iSCSI: response to second login (with same ISID) Martin, I agree that it would be bad if at login an initiator would have to wait for a long time for the target to detect if the old session has gone away. But there is nothing in the current spec that will prevent a good target implementation doing what you describe whithout any negotiation. Regards, Julo "Martin, Nick" on 25-05-2001 23:22:11 Please respond to "Martin, Nick" To: Julian Satran/Haifa/IBM@IBMIL, ips@ece.cmu.edu cc: Subject: RE: iSCSI: response to second login (with same ISID) Julian, I do not know how long it would take a target using a ping to determine that an old session is really dead. Some folks think it would be too long, thus they want to be able to "login unconditionally". Primarily I thought this would be used for initiators rebooting after ungraceful shutdown. (For those who can otherwise reboot quickly, but would stall on each iSCSI session not already recovered by the target.) I do not expect to want to use a "login unconditionally" option. (I can handle rejection ;) The scary thing to me was if an initiator could not try an ISID which was in use (intentionally, accidentally, or erroneously) without disrupting (logging out) its current user (a.k.a. option 2). It appears that if given a "login unconditionally" option, some folks would use it exclusively. If the initiator can specify the maximum time it would take a target to notice dead connections and sessions, then the argument that waiting for target recovery of the session would take too long presumably goes away. I am thinking about login negotiated parameters like MinIdleTime and MaxIdleTime values in seconds. These could specify respectively the connection idle interval after which a NO-OP (keep-alive) should be sent and expected, and the time with no traffic received on this connection which should be regarded as an implicit logout request due to connection failure. (This could cause an async event if detected by the target and a working connection remains in this session.) Better names can be chosen. Reasonable defaults might be 10 seconds and 60 seconds. There should be a recommendation like MaxIdleTime should always be at least 3 times MinIdleTime. Such parameters would probably exist within the Target and the Initiator, even if they are not negotiated or exchanged. The intent is that an Initiator that cares, can limit Target connection recovery delays. These parameters set by the Initiator for the Target, would be known to both. A broken connection could be detected by each at approximately the same time. The keep-alive NO-OP's could be pings so that both directions would be kept alive by a single exchange. If the Initiator wants to be responsible for keep-alives, it should do them before the specified MinIdleTime, if it wants the Target to do them, it may need to do them also, slightly after the specified MinIdleTime, but well before MaxIdleTime. My networking is weak, so I am not sure whether there should be a separate (shorter) time specified for the maximum time to wait for the reply to a ping, or how long to wait to retry it. I am not clear on the interactions with TCP and its timeouts and retries. Is there a maximum time a ping could take and still get through? Is there a maximum time a busy iSCSI target or initiator would be expected to delay before sending a reply to a ping? Does this sound like a good idea at all or not? Thanks, Nick From owner-ips@ece.cmu.edu Tue May 29 10:55:40 2001 Received: from ece.cmu.edu (ECE.CMU.EDU [128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id KAA02940 for ; Tue, 29 May 2001 10:55:39 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f4TCZbf00219 for ips-outgoing; Tue, 29 May 2001 08:35:37 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from d12lmsgate-3.de.ibm.com (d12lmsgate-3.de.ibm.com [195.212.91.201]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f4TCZZ300214 for ; Tue, 29 May 2001 08:35:35 -0400 (EDT) Received: from d12relay01.de.ibm.com (d12relay01.de.ibm.com [9.165.215.22]) by d12lmsgate-3.de.ibm.com (1.0.0) with ESMTP id OAA197684 for ; Tue, 29 May 2001 14:35:28 +0200 From: julian_satran@il.ibm.com Received: from d12mta02.de.ibm.com (d12mta01_cs0 [9.165.222.237]) by d12relay01.de.ibm.com (8.8.8m3/NCO v4.96) with SMTP id OAA41984 for ; Tue, 29 May 2001 14:35:28 +0200 Received: by d12mta02.de.ibm.com(Lotus SMTP MTA v4.6.5 (863.2 5-20-1999)) id C1256A5B.00451C2F ; Tue, 29 May 2001 14:34:51 +0200 X-Lotus-FromDomain: IBMIL@IBMDE To: ips@ece.cmu.edu Message-ID: Date: Tue, 29 May 2001 15:38:44 +0300 Subject: Re: iSCSI - opcodes Mime-Version: 1.0 Content-type: text/plain; charset=us-ascii Content-Disposition: inline Sender: owner-ips@ece.cmu.edu Precedence: bulk Brian, Vendors can negotiate the use of their specific opcodes at login through ... vendor specific text-keys. I am not a big fan of vendor-specific op-codes or keys but they give people an opportunity to experiment with new things without having to get to a "rough consensus". Julo Brian Pawlowski on 28-05-2001 22:39:02 Please respond to Brian Pawlowski To: Julian Satran/Haifa/IBM@IBMIL cc: ips@ece.cmu.edu Subject: Re: iSCSI - opcodes I have a problem with this statement. Vendor specific opcodes in a shared opcode space are subject to collision unless a management framework wi specified to deal with the opcode space. What is the intention here? Is there a precedent in SCSI (FCP or whatever) for such use? If they are just unused codes, I would propose they be left RESERVED. If you intend to use them, I would like to hear how it will be managed to ensure interoperability. > Initiator opcodes 0x38, 0x3a, 0x3c and 0x3e and target opcodes 0x39, > 0x3b, 0x3d and 0x3f are vendor specific codes. From owner-ips@ece.cmu.edu Tue May 29 19:44:33 2001 Received: from ece.cmu.edu (ECE.CMU.EDU [128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id TAA12840 for ; Tue, 29 May 2001 19:44:33 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f4TLW4O13366 for ips-outgoing; Tue, 29 May 2001 17:32:04 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from c007.snv.cp.net (c007-h008.c007.snv.cp.net [209.228.33.214]) by ece.cmu.edu (8.11.0/8.10.2) with SMTP id f4TLW2313362 for ; Tue, 29 May 2001 17:32:02 -0400 (EDT) Received: (cpmta 13273 invoked from network); 29 May 2001 14:31:56 -0700 Received: from adsl-63-202-160-80.dsl.snfc21.pacbell.net (HELO ljoy) (63.202.160.80) by smtp.telocity.com (209.228.33.214) with SMTP; 29 May 2001 14:31:56 -0700 X-Sent: 29 May 2001 21:31:56 GMT From: "Douglas Otis" To: , "Bernard Aboba" Cc: Subject: RE: iSCSI and secure boot Date: Tue, 29 May 2001 14:29:35 -0700 Message-ID: MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 (Normal) X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook IMO, Build 9.0.2416 (9.0.2911.0) In-Reply-To: X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4522.1200 Importance: Normal Sender: owner-ips@ece.cmu.edu Precedence: bulk Content-Transfer-Encoding: 7bit Julian, The PXE documents indicate a method of using standard protocols. The reason for this additional level of detail was to provide a consistent set of utilities for use by the PROM code and to recommend specific options use. I see little that suggests that this is a proprietary scheme as such standard protocols do not dwell on system related issues. This provided some clarification as to what could be expected from a system that supported this environment. Doug > Bernard, > > As iSCSI is transporting SCSI and SCSI has a different boot paradigm than > Netboot can you please elaborate on what exactly should be an > authenticated > boot image in this (SCSI) context. > > Please take into consideration that unlike netboot - the SCSI > boot is not a > clearly bounded process (not even in PXE - an "open" proprietary scheme). > > Julo From owner-ips@ece.cmu.edu Tue May 29 19:47:35 2001 Received: from ece.cmu.edu (ECE.CMU.EDU [128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id TAA12858 for ; Tue, 29 May 2001 19:47:34 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f4TM6Ca15894 for ips-outgoing; Tue, 29 May 2001 18:06:12 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from d12lmsgate.de.ibm.com (d12lmsgate.de.ibm.com [195.212.91.199]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f4TM69315877 for ; Tue, 29 May 2001 18:06:09 -0400 (EDT) Received: from d12relay01.de.ibm.com (d12relay01.de.ibm.com [9.165.215.22]) by d12lmsgate.de.ibm.com (1.0.0) with ESMTP id AAA31846 for ; Wed, 30 May 2001 00:05:59 +0200 From: julian_satran@il.ibm.com Received: from d12mta02.de.ibm.com (d12mta01_cs0 [9.165.222.237]) by d12relay01.de.ibm.com (8.8.8m3/NCO v4.96) with SMTP id AAA66638 for ; Wed, 30 May 2001 00:06:00 +0200 Received: by d12mta02.de.ibm.com(Lotus SMTP MTA v4.6.5 (863.2 5-20-1999)) id C1256A5B.007953A9 ; Wed, 30 May 2001 00:05:12 +0200 X-Lotus-FromDomain: IBMIL@IBMDE To: ips@ece.cmu.edu Message-ID: Date: Wed, 30 May 2001 01:00:48 +0300 Subject: Re: iSCSI: multiple initiator conflicting with target (was Re: iSCSI: response to second login (with same ISID) Mime-Version: 1.0 Content-type: text/plain; charset=us-ascii Content-Disposition: inline Sender: owner-ips@ece.cmu.edu Precedence: bulk As far as I know this is a "primal" issue with all cooperating systems. SCSI has attempted to handle it with very granular reserve/release. However those where never very popular with vendors and users (except for exoteric systems block level reserve/release where not implemented) as they carry a high price tag and performance is quite low. "Complicated software vendors" have fared slightly better as initiators that "cannot cooperate" can in fact and more effectively than targets - as they have a good handle on the lock semantics. However this is hardly an iSCSI issue and I suggest we take this off the IPS List. Julo "Sanjeev Bhagat \(TRIPACE/Zoetermeer\)" on 29-05-2001 03:57:39 Please respond to "Sanjeev Bhagat \(TRIPACE/Zoetermeer\)" To: "Patrick Stirling" cc: "'Lakshmi Ramasubramanian'" , "Santosh Rao" , ips@ece.cmu.edu Subject: Re: iSCSI: multiple intiaitor conflicting with target (was Re: iSCSI: response to second login (with same ISID) Hello Patrick, Since the initiators are two different entities working completely independent of each other they cannot co-ordinate with each other. iSCSI Target also sees these two different initiators in different sessions as seperate entities. In the current implementation of iSCSI I guess it is not possible for two different initiators to know about each other and so it is difficult to resolve this conflicting situation. The answer to this problem might be lying in SAM-2 specifications. May be I am missing something there. Can somebody comment? Sanjeev > > On Mon, 28 May 2001, Sanjeev Bhagat (TRIPACE/Zoetermeer) wrote: > > > Well the conflict of writing to same sector can still be avoided by > > implementaion of some locks in the target but I wonder what will happen in > > this case where 2 inititators are connected to a target. > > > From: Lakshmi Ramasubramanian [mailto:nramas@windows.microsoft.com] > > > > Even if the target is capable of supporting multiple initiators, won't > > it cause problems with devices such as disk - say, filesystem from the > > two initiators' side attempt to write > > to the same sectors (even not intentionally) would cause data > > corruption. Is iSCSI layer suppose to guard against this type of > > device sharing? > > Umm, am I missing something here? > > If 2 initiators want to access the same SCSI device, they must coordinate > to avoid clashing with each other. It's not the device's responsibility to > prevent this. This is one reason several companies have complicated > clustering software. > > patrick > Patrick Stirling > VERITAS Software (vendor of complicated clustering software!) > From owner-ips@ece.cmu.edu Tue May 29 23:14:05 2001 Received: from ece.cmu.edu (ECE.CMU.EDU [128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id XAA16538 for ; Tue, 29 May 2001 23:14:05 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f4U1MjH29274 for ips-outgoing; Tue, 29 May 2001 21:22:45 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from earth.vicom.com (206-14-133-15.ncal.verio.com [206.14.133.15] (may be forged)) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f4U1Mh329269 for ; Tue, 29 May 2001 21:22:43 -0400 (EDT) Received: by earth.vicom.com with Internet Mail Service (5.5.2650.21) id ; Tue, 29 May 2001 18:24:05 -0700 Message-ID: From: David Lee To: "'ips@ece.cmu.edu'" Subject: Using SNACK to request retransmission of R2T Date: Tue, 29 May 2001 18:24:01 -0700 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2650.21) Content-Type: text/plain Sender: owner-ips@ece.cmu.edu Precedence: bulk Hi all, Please forgive me if this has already been addressed. In at least one place in the Rev. 6 draft it states that SNACK may be used by an initiator to request a resend by a target of a missing R2T PDU. However, the format of the SNACK PDU does not seem to cover this case. David Lee Vicom Systems, Inc. From owner-ips@ece.cmu.edu Wed May 30 01:01:20 2001 Received: from ece.cmu.edu (ECE.CMU.EDU [128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id BAA17852 for ; Wed, 30 May 2001 01:01:20 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f4U3a0f07813 for ips-outgoing; Tue, 29 May 2001 23:36:00 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from internaut.com ([64.38.134.99]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f4U3Zt307800 for ; Tue, 29 May 2001 23:35:56 -0400 (EDT) Received: from localhost (aboba@localhost) by internaut.com (8.9.3/8.9.3) with ESMTP id UAA49784; Tue, 29 May 2001 20:25:53 -0700 (PDT) (envelope-from aboba@internaut.com) Date: Tue, 29 May 2001 20:25:53 -0700 (PDT) From: Bernard Aboba To: julian_satran@il.ibm.com cc: ips@ece.cmu.edu, aboba@internaut.com Subject: Re: iSCSI and secure boot In-Reply-To: Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-ips@ece.cmu.edu Precedence: bulk > As iSCSI is transporting SCSI and SCSI has a different boot paradigm than > Netboot can you please elaborate on what exactly should be an authenticated > boot image in this (SCSI) context. > > Please take into consideration that unlike netboot - the SCSI boot is not a > clearly bounded process (not even in PXE - an "open" proprietary scheme). > I'm no PXE expert, but here is my (admittedly limited) understanding of what goes on: 1. Many OEMs ship PXE-enabled PCs, and in most of these PXE cannot be turned off, though it can be set lower in the boot order. In some percentage of machines PXE will even be higher in the boot order than the hard disk (bad idea, for reasons I'll describe). 2. Machine boots and if it reaches PXE in the boot order, it sends a DHCPDISCOVER. If responded to with a DHCPOFFER including Option 60 ("PXECLIENT") and TFTP server, it will finish obtaining an address via DHCP and then pull a boot image from the TFTP server. This is typically startrom.com, a 16-bit executable. Note that this is often *not* a full OS image; for example Ghost supports PXE and uses this early load in order to reformat the hard disk and start the download of the new "cloned" disk image. Other PXE supporting-apps may do other things with this early boot image. 3. If the machine is PXE 2.0 enabled, it will support signed boot images, and will check the integrity of the boot image prior to executing it. This part of the spec is called BIS -- and requires the public key of the boot image signer to be pre-loaded on the PC. My understanding is that PXE 2.0 and BIS are not widely implemented and as a result, PXE 1.0 systems are most widespread and they execute whatever boot image they are fed. This is one reason why putting PXE high in the boot order is a bad idea. This would enable a rogue ghost server to reformat your hard drive. Ouch! (this happens in real life, unfortunately). 4. PXE 2.0 does not support authenticated DHCP nor any TFTP security, so that it is vulnerable to rogue DHCP servers, offers no support for client authentication, and only checks the integrity of the boot image once it has completed the download of it. However, since it also requires pre-loading the boot image signing cert on the PC, and doesn't support revocation of invalidated boot images (which can be endlessly replayed), in practice PXE 2.0 is not widely deployed and doesn't do a good job in boot image validation. 5. The small startrom.com loader is then used to pull down a more complete OS image, either installing it on the hard drive, or pulling it into memory. At this point, for example, you may have the appropriate drivers loaded, be operating in 32-bit mode, etc. As far as how iSCSI fits in to all this, I'd hope you would enlighten me ;) As has been suggested, the iSCSI target address could be obtained via DHCP, and suitable initial iSCSI drivers provided via the initial boot load obtained via TFTP. Given the security headaches inherent in PXE, I am really interested in how all of this could be done in a deployable, secure manner. The picture is somewhat confusing today, and the following questions come to mind: a. What credentials are needed to secure iSCSI boot? 1. Kerberos machine credentials? 2. Identity/shared keys for IPSEC? b. What credentials are needed to secure DHCP? 1. According to -16, you need the authenticated DHCP key (unique per htype/MAC address) 2. According to draft-hornstein-dhc-kerbauth-0x.txt you need Kerb credentials. 3. Do we care? (see below) c. What credentials are needed to verify the boot image? 1. The boot image signing certificate (BIS) 2. Something else? (Shared secret, Kerb credentials?) Storing several identities and credential sets in PC NVRAM is difficult to administer and deploy. It makes sense to boil this mess down to a small list of items. But how? Some thoughts: * Kerberos clients are small, and I am told, romable. Might it not make sense to use Kerberos to secure PXE? Could we kill multiple birds (iSCSI security, 802.1X, NFS, TFTP, boot image security) by doing this? * Do we really need DHCP authentication for secure boot? Interface-specific authentication credentials strike me as painful to maintain, plus it requires substantial administration. From owner-ips@ece.cmu.edu Wed May 30 01:11:43 2001 Received: from ece.cmu.edu (ECE.CMU.EDU [128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id BAA18294 for ; Wed, 30 May 2001 01:11:43 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f4U36YK06026 for ips-outgoing; Tue, 29 May 2001 23:06:34 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from d12lmsgate.de.ibm.com (d12lmsgate.de.ibm.com [195.212.91.199]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f4U36V306021 for ; Tue, 29 May 2001 23:06:31 -0400 (EDT) Received: from d12relay02.de.ibm.com (d12relay02.de.ibm.com [9.165.215.23]) by d12lmsgate.de.ibm.com (1.0.0) with ESMTP id FAA177528 for ; Wed, 30 May 2001 05:06:23 +0200 From: julian_satran@il.ibm.com Received: from d12mta02.de.ibm.com (d12mta01_cs0 [9.165.222.237]) by d12relay02.de.ibm.com (8.8.8m3/NCO v4.96) with SMTP id FAA55954 for ; Wed, 30 May 2001 05:05:45 +0200 Received: by d12mta02.de.ibm.com(Lotus SMTP MTA v4.6.5 (863.2 5-20-1999)) id C1256A5C.001100A0 ; Wed, 30 May 2001 05:05:42 +0200 X-Lotus-FromDomain: IBMIL@IBMDE To: ips@ece.cmu.edu Message-ID: Date: Wed, 30 May 2001 06:07:15 +0300 Subject: Re: Using SNACK to request retransmission of R2T Mime-Version: 1.0 Content-type: text/plain; charset=us-ascii Content-Disposition: inline Sender: owner-ips@ece.cmu.edu Precedence: bulk The SNACK is just a data SNACK (R2T are treated as input data for the command). Julo David Lee on 30-05-2001 04:24:01 Please respond to David Lee To: "'ips@ece.cmu.edu'" cc: Subject: Using SNACK to request retransmission of R2T Hi all, Please forgive me if this has already been addressed. In at least one place in the Rev. 6 draft it states that SNACK may be used by an initiator to request a resend by a target of a missing R2T PDU. However, the format of the SNACK PDU does not seem to cover this case. David Lee Vicom Systems, Inc. From owner-ips@ece.cmu.edu Wed May 30 08:20:14 2001 Received: from ece.cmu.edu (ECE.CMU.EDU [128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id IAA07461 for ; Wed, 30 May 2001 08:20:14 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f4UAoIC13901 for ips-outgoing; Wed, 30 May 2001 06:50:19 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from d12lmsgate-2.de.ibm.com (d12lmsgate-2.de.ibm.com [195.212.91.200]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f4UAoEt13895 for ; Wed, 30 May 2001 06:50:14 -0400 (EDT) Received: from d12relay01.de.ibm.com (d12relay01.de.ibm.com [9.165.215.22]) by d12lmsgate-2.de.ibm.com (1.0.0) with ESMTP id MAA102944; Wed, 30 May 2001 12:50:03 +0200 From: julian_satran@il.ibm.com Received: from d12mta02.de.ibm.com (d12mta01_cs0 [9.165.222.237]) by d12relay01.de.ibm.com (8.8.8m3/NCO v4.96) with SMTP id MAA60800; Wed, 30 May 2001 12:50:03 +0200 Received: by d12mta02.de.ibm.com(Lotus SMTP MTA v4.6.5 (863.2 5-20-1999)) id C1256A5C.003B7393 ; Wed, 30 May 2001 12:49:22 +0200 X-Lotus-FromDomain: IBMIL@IBMDE To: Bernard Aboba cc: ips@ece.cmu.edu Message-ID: Date: Wed, 30 May 2001 13:54:24 +0300 Subject: Re: iSCSI and secure boot Mime-Version: 1.0 Content-type: text/plain; charset=us-ascii Content-Disposition: inline Sender: owner-ips@ece.cmu.edu Precedence: bulk Bernard, PXE is assuming that the boot image is a clearly defined (bounded) piece of data. SCSI booting does not assume this. Signing the total boot image is something you do in the regular SCSI (local) load process. All your other points (securing DHCP, securing a miniboot loader) are valid and should be considered by all vendors. I wonder however if we should consider standardizing all this behaviour before we some more hands-on experience in what is really needed and which of the solution is viable. If we take the PXE path we might be tempted to mandate a "mini-TFTP" in each target to be able to load a "startboot". Would this be wise to this now? For all those reasons I would be reluctant to add too much to the current boot draft beyond perhaps a simple authentication as mandatory. The only provision we have made in the main draft is the "Boot" key to indicate to the target that it can limit the access of an initiator doing this type of logon. Julo Bernard Aboba on 30-05-2001 06:25:53 Please respond to Bernard Aboba To: Julian Satran/Haifa/IBM@IBMIL cc: ips@ece.cmu.edu, aboba@internaut.com Subject: Re: iSCSI and secure boot > As iSCSI is transporting SCSI and SCSI has a different boot paradigm than > Netboot can you please elaborate on what exactly should be an authenticated > boot image in this (SCSI) context. > > Please take into consideration that unlike netboot - the SCSI boot is not a > clearly bounded process (not even in PXE - an "open" proprietary scheme). > I'm no PXE expert, but here is my (admittedly limited) understanding of what goes on: 1. Many OEMs ship PXE-enabled PCs, and in most of these PXE cannot be turned off, though it can be set lower in the boot order. In some percentage of machines PXE will even be higher in the boot order than the hard disk (bad idea, for reasons I'll describe). 2. Machine boots and if it reaches PXE in the boot order, it sends a DHCPDISCOVER. If responded to with a DHCPOFFER including Option 60 ("PXECLIENT") and TFTP server, it will finish obtaining an address via DHCP and then pull a boot image from the TFTP server. This is typically startrom.com, a 16-bit executable. Note that this is often *not* a full OS image; for example Ghost supports PXE and uses this early load in order to reformat the hard disk and start the download of the new "cloned" disk image. Other PXE supporting-apps may do other things with this early boot image. 3. If the machine is PXE 2.0 enabled, it will support signed boot images, and will check the integrity of the boot image prior to executing it. This part of the spec is called BIS -- and requires the public key of the boot image signer to be pre-loaded on the PC. My understanding is that PXE 2.0 and BIS are not widely implemented and as a result, PXE 1.0 systems are most widespread and they execute whatever boot image they are fed. This is one reason why putting PXE high in the boot order is a bad idea. This would enable a rogue ghost server to reformat your hard drive. Ouch! (this happens in real life, unfortunately). 4. PXE 2.0 does not support authenticated DHCP nor any TFTP security, so that it is vulnerable to rogue DHCP servers, offers no support for client authentication, and only checks the integrity of the boot image once it has completed the download of it. However, since it also requires pre-loading the boot image signing cert on the PC, and doesn't support revocation of invalidated boot images (which can be endlessly replayed), in practice PXE 2.0 is not widely deployed and doesn't do a good job in boot image validation. 5. The small startrom.com loader is then used to pull down a more complete OS image, either installing it on the hard drive, or pulling it into memory. At this point, for example, you may have the appropriate drivers loaded, be operating in 32-bit mode, etc. As far as how iSCSI fits in to all this, I'd hope you would enlighten me ;) As has been suggested, the iSCSI target address could be obtained via DHCP, and suitable initial iSCSI drivers provided via the initial boot load obtained via TFTP. Given the security headaches inherent in PXE, I am really interested in how all of this could be done in a deployable, secure manner. The picture is somewhat confusing today, and the following questions come to mind: a. What credentials are needed to secure iSCSI boot? 1. Kerberos machine credentials? 2. Identity/shared keys for IPSEC? b. What credentials are needed to secure DHCP? 1. According to -16, you need the authenticated DHCP key (unique per htype/MAC address) 2. According to draft-hornstein-dhc-kerbauth-0x.txt you need Kerb credentials. 3. Do we care? (see below) c. What credentials are needed to verify the boot image? 1. The boot image signing certificate (BIS) 2. Something else? (Shared secret, Kerb credentials?) Storing several identities and credential sets in PC NVRAM is difficult to administer and deploy. It makes sense to boil this mess down to a small list of items. But how? Some thoughts: * Kerberos clients are small, and I am told, romable. Might it not make sense to use Kerberos to secure PXE? Could we kill multiple birds (iSCSI security, 802.1X, NFS, TFTP, boot image security) by doing this? * Do we really need DHCP authentication for secure boot? Interface-specific authentication credentials strike me as painful to maintain, plus it requires substantial administration. From owner-ips@ece.cmu.edu Wed May 30 08:23:56 2001 Received: from ece.cmu.edu (ECE.CMU.EDU [128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id IAA07635 for ; Wed, 30 May 2001 08:23:56 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f4UARMw12555 for ips-outgoing; Wed, 30 May 2001 06:27:22 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from d12lmsgate-2.de.ibm.com (d12lmsgate-2.de.ibm.com [195.212.91.200]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f4UARLt12551 for ; Wed, 30 May 2001 06:27:21 -0400 (EDT) Received: from d12relay02.de.ibm.com (d12relay02.de.ibm.com [9.165.215.23]) by d12lmsgate-2.de.ibm.com (1.0.0) with ESMTP id MAA131504 for ; Wed, 30 May 2001 12:27:11 +0200 From: julian_satran@il.ibm.com Received: from d12mta02.de.ibm.com (d12mta01_cs0 [9.165.222.237]) by d12relay02.de.ibm.com (8.8.8m3/NCO v4.96) with SMTP id MAA106062 for ; Wed, 30 May 2001 12:26:30 +0200 Received: by d12mta02.de.ibm.com(Lotus SMTP MTA v4.6.5 (863.2 5-20-1999)) id C1256A5C.00395BB3 ; Wed, 30 May 2001 12:26:30 +0200 X-Lotus-FromDomain: IBMIL@IBMDE To: ips@ece.cmu.edu Message-ID: Date: Wed, 30 May 2001 06:07:15 +0300 Subject: Re: Using SNACK to request retransmission of R2T Mime-Version: 1.0 Content-type: text/plain; charset=us-ascii Content-Disposition: inline Sender: owner-ips@ece.cmu.edu Precedence: bulk The SNACK is just a data SNACK (R2T are treated as input data for the command). Julo David Lee on 30-05-2001 04:24:01 Please respond to David Lee To: "'ips@ece.cmu.edu'" cc: Subject: Using SNACK to request retransmission of R2T Hi all, Please forgive me if this has already been addressed. In at least one place in the Rev. 6 draft it states that SNACK may be used by an initiator to request a resend by a target of a missing R2T PDU. However, the format of the SNACK PDU does not seem to cover this case. David Lee Vicom Systems, Inc. From owner-ips@ece.cmu.edu Wed May 30 13:02:13 2001 Received: from ece.cmu.edu (ECE.CMU.EDU [128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id NAA17980 for ; Wed, 30 May 2001 13:02:13 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f4UETWl28964 for ips-outgoing; Wed, 30 May 2001 10:29:32 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from earth.vicom.com (206-14-133-15.ncal.verio.com [206.14.133.15] (may be forged)) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f4UETUt28957 for ; Wed, 30 May 2001 10:29:30 -0400 (EDT) Received: by earth.vicom.com with Internet Mail Service (5.5.2650.21) id ; Wed, 30 May 2001 07:30:49 -0700 Message-ID: From: David Lee To: "'julian_satran@il.ibm.com '" , "'ips@ece.cmu.edu '" Subject: RE: Using SNACK to request retransmission of R2T Date: Wed, 30 May 2001 07:30:42 -0700 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2650.21) Content-Type: text/plain; charset="iso-8859-1" Sender: owner-ips@ece.cmu.edu Precedence: bulk I'm sorry, but I did not understand this response. The case I am inquiring about involves a missing R2T PDU (detected as a result of receiving another R2T with a higher DataSn). The following excerp is from section 6.7.1: (1)Lost data PDU or lost R2T - a data PDU or R2T may be lost due to a header digest error or a data digest error. In case of a data digest error, the error is recognized immediately and the initiator MAY request the missing data through SNACK. In case of a header digest error, the initiator recognizes the missing data or R2T either when receiving a subsequent piece out of sequence or by a timeout in completing a sequence (no status). In this case, the initiator MAY request the missing data or R2T through a SNACK. The description of the SNACK PDU does not appear to include a form for requesting retransmission of R2T. David Lee -----Original Message----- From: julian_satran@il.ibm.com To: ips@ece.cmu.edu Sent: 5/29/01 8:07 PM Subject: Re: Using SNACK to request retransmission of R2T The SNACK is just a data SNACK (R2T are treated as input data for the command). Julo David Lee on 30-05-2001 04:24:01 Please respond to David Lee To: "'ips@ece.cmu.edu'" cc: Subject: Using SNACK to request retransmission of R2T Hi all, Please forgive me if this has already been addressed. In at least one place in the Rev. 6 draft it states that SNACK may be used by an initiator to request a resend by a target of a missing R2T PDU. However, the format of the SNACK PDU does not seem to cover this case. David Lee Vicom Systems, Inc. From owner-ips@ece.cmu.edu Wed May 30 13:40:46 2001 Received: from ece.cmu.edu (ECE.CMU.EDU [128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id NAA19290 for ; Wed, 30 May 2001 13:40:45 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f4UFZM604234 for ips-outgoing; Wed, 30 May 2001 11:35:22 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from ietf.org (odin.ietf.org [132.151.1.176]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f4UAkJt13653 for ; Wed, 30 May 2001 06:46:19 -0400 (EDT) Received: from CNRI.Reston.VA.US (localhost [127.0.0.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id GAA04701; Wed, 30 May 2001 06:45:59 -0400 (EDT) Message-Id: <200105301045.GAA04701@ietf.org> Mime-Version: 1.0 Content-Type: Multipart/Mixed; Boundary="NextPart" To: IETF-Announce: ; Cc: ips@ece.cmu.edu From: Internet-Drafts@ietf.org Reply-to: Internet-Drafts@ietf.org Subject: I-D ACTION:draft-ietf-ips-ifcp-02.txt Date: Wed, 30 May 2001 06:45:58 -0400 Sender: owner-ips@ece.cmu.edu Precedence: bulk --NextPart A New Internet-Draft is available from the on-line Internet-Drafts directories. This draft is a work item of the IP Storage Working Group of the IETF. Title : iFCP - A Protocol for Internet Fibre Channel Storage Networking Author(s) : C. Monia et al. Filename : draft-ietf-ips-ifcp-02.txt Pages : 67 Date : 29-May-01 This document specifies an architecture and gateway-to-gateway protocol for the implementation of Fibre Channel fabric functionality on a network in which TCP/IP switching and routing elements replace Fibre Channel components. The protocol enables the attachment of existing Fibre Channel storage products to an IP network by supporting the fabric services required by such devices. A URL for this Internet-Draft is: http://www.ietf.org/internet-drafts/draft-ietf-ips-ifcp-02.txt Internet-Drafts are also available by anonymous FTP. Login with the username "anonymous" and a password of your e-mail address. After logging in, type "cd internet-drafts" and then "get draft-ietf-ips-ifcp-02.txt". A list of Internet-Drafts directories can be found in http://www.ietf.org/shadow.html or ftp://ftp.ietf.org/ietf/1shadow-sites.txt Internet-Drafts can also be obtained by e-mail. Send a message to: mailserv@ietf.org. In the body type: "FILE /internet-drafts/draft-ietf-ips-ifcp-02.txt". NOTE: The mail server at ietf.org can return the document in MIME-encoded form by using the "mpack" utility. To use this feature, insert the command "ENCODING mime" before the "FILE" command. To decode the response(s), you will need "munpack" or a MIME-compliant mail reader. Different MIME-compliant mail readers exhibit different behavior, especially when dealing with "multipart" MIME messages (i.e. documents which have been split up into multiple messages), so check your local documentation on how to manipulate these messages. Below is the data which will enable a MIME compliant mail reader implementation to automatically retrieve the ASCII version of the Internet-Draft. --NextPart Content-Type: Multipart/Alternative; Boundary="OtherAccess" --OtherAccess Content-Type: Message/External-body; access-type="mail-server"; server="mailserv@ietf.org" Content-Type: text/plain Content-ID: <20010529142037.I-D@ietf.org> ENCODING mime FILE /internet-drafts/draft-ietf-ips-ifcp-02.txt --OtherAccess Content-Type: Message/External-body; name="draft-ietf-ips-ifcp-02.txt"; site="ftp.ietf.org"; access-type="anon-ftp"; directory="internet-drafts" Content-Type: text/plain Content-ID: <20010529142037.I-D@ietf.org> --OtherAccess-- --NextPart-- From owner-ips@ece.cmu.edu Wed May 30 17:07:50 2001 Received: from ece.cmu.edu (ECE.CMU.EDU [128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id RAA24780 for ; Wed, 30 May 2001 17:07:49 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f4UIipI19262 for ips-outgoing; Wed, 30 May 2001 14:44:51 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from c007.snv.cp.net (c007-h011.c007.snv.cp.net [209.228.33.217]) by ece.cmu.edu (8.11.0/8.10.2) with SMTP id f4UIiot19258 for ; Wed, 30 May 2001 14:44:50 -0400 (EDT) Received: (cpmta 9272 invoked from network); 30 May 2001 11:41:08 -0700 Received: from adsl-63-202-160-80.dsl.snfc21.pacbell.net (HELO ljoy) (63.202.160.80) by smtp.telocity.com (209.228.33.217) with SMTP; 30 May 2001 11:41:08 -0700 X-Sent: 30 May 2001 18:41:08 GMT From: "Douglas Otis" To: , "Bernard Aboba" Cc: Subject: RE: iSCSI and secure boot Date: Wed, 30 May 2001 11:38:48 -0700 Message-ID: MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 (Normal) X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook IMO, Build 9.0.2416 (9.0.2911.0) X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4522.1200 In-Reply-To: Importance: Normal Sender: owner-ips@ece.cmu.edu Precedence: bulk Content-Transfer-Encoding: 7bit Julian, If the results are confirmed, then the process is confirmed. There should be no need to validate every step in the process, just the last step as done with PXE 2. As far as TFTP, it lives up to its name. A simple TFTP stack can be done in a few dozen lines of code. It is really a minimum amount of effort. Doug > Bernard, > > PXE is assuming that the boot image is a clearly defined > (bounded) piece of > data. > SCSI booting does not assume this. Signing the total boot image is > something you do in the regular SCSI (local) load process. All your other > points (securing DHCP, securing a miniboot loader) are valid and should be > considered > by all vendors. > > I wonder however if we should consider standardizing all this behaviour > before we some more hands-on experience > in what is really needed and which of the solution is viable. > > If we take the PXE path we might be tempted to mandate a "mini-TFTP" in > each target to be able to load a "startboot". > > Would this be wise to this now? > > For all those reasons I would be reluctant to add too much to the current > boot draft beyond perhaps a simple authentication as mandatory. > > The only provision we have made in the main draft is the "Boot" key to > indicate to the target that it can limit the > access of an initiator doing this type of logon. > > Julo > > Bernard Aboba on 30-05-2001 06:25:53 > > Please respond to Bernard Aboba > > To: Julian Satran/Haifa/IBM@IBMIL > cc: ips@ece.cmu.edu, aboba@internaut.com > Subject: Re: iSCSI and secure boot > > > > > > As iSCSI is transporting SCSI and SCSI has a different boot > paradigm than > > Netboot can you please elaborate on what exactly should be an > authenticated > > boot image in this (SCSI) context. > > > > Please take into consideration that unlike netboot - the SCSI > boot is not > a > > clearly bounded process (not even in PXE - an "open" > proprietary scheme). > > > I'm no PXE expert, but here is my (admittedly limited) understanding of > what goes on: > > 1. Many OEMs ship PXE-enabled PCs, and in most of these PXE cannot be > turned off, though it can be set lower in the boot order. In some > percentage of machines PXE will even be higher in the boot order than the > hard disk (bad idea, for reasons I'll describe). > > 2. Machine boots and if it reaches PXE in the boot order, it sends a > DHCPDISCOVER. If responded to with a DHCPOFFER including Option 60 > ("PXECLIENT") and TFTP server, it will finish obtaining an address via > DHCP and then pull a boot image from the TFTP server. This is typically > startrom.com, a 16-bit executable. Note that this is often *not* a full OS > image; for example Ghost supports PXE and uses this early load in order to > reformat the hard disk and start the download of the new "cloned" disk > image. Other PXE supporting-apps may do other things with this early boot > image. > > 3. If the machine is PXE 2.0 enabled, it will support signed boot images, > and will check the integrity of the boot image prior to executing it. This > part of the spec is called BIS -- and requires the public key of the boot > image signer to be pre-loaded on the PC. My understanding is that PXE 2.0 > and BIS are not widely implemented and as a result, PXE 1.0 systems > are most widespread and they execute whatever boot image they are > fed. This is one reason why putting PXE high in the boot order is a bad > idea. This would enable a rogue ghost server to reformat your hard > drive. Ouch! (this happens in real life, unfortunately). > > 4. PXE 2.0 does not support authenticated DHCP nor any TFTP security, so > that it is vulnerable to rogue DHCP servers, offers no support for client > authentication, and only checks the integrity of the boot image once it > has completed the download of it. However, since it also requires > pre-loading the boot image signing cert on the PC, and doesn't support > revocation of invalidated boot images (which can be endlessly replayed), > in practice PXE 2.0 is not widely deployed and doesn't do a good job in > boot image validation. > > 5. The small startrom.com loader is then used to pull down a more complete > OS image, either installing it on the hard drive, or pulling it into > memory. At this point, for example, you may have the appropriate drivers > loaded, be operating in 32-bit mode, etc. > > As far as how iSCSI fits in to all this, I'd hope you would enlighten me > ;) As has been suggested, the iSCSI target address could be obtained via > DHCP, and suitable initial iSCSI drivers provided via the initial boot > load obtained via TFTP. > > Given the security headaches inherent in PXE, I am really interested in > how all of this could be done in a deployable, secure manner. The picture > is somewhat confusing today, and the following questions come to mind: > > a. What credentials are needed to secure iSCSI boot? > 1. Kerberos machine credentials? > 2. Identity/shared keys for IPSEC? > b. What credentials are needed to secure DHCP? > 1. According to -16, you need the authenticated DHCP key > (unique per htype/MAC address) > 2. According to draft-hornstein-dhc-kerbauth-0x.txt you > need Kerb credentials. > 3. Do we care? (see below) > c. What credentials are needed to verify the boot image? > 1. The boot image signing certificate (BIS) > 2. Something else? (Shared secret, Kerb credentials?) > > Storing several identities and credential sets in PC NVRAM is difficult to > administer and deploy. It makes sense to boil this mess down > to a small list of items. But how? > > Some thoughts: > > * Kerberos clients are small, and I am told, romable. Might it not make > sense to use Kerberos to secure PXE? Could we kill multiple birds > (iSCSI security, 802.1X, NFS, TFTP, boot image security) by doing this? > > * Do we really need DHCP authentication for secure boot? > Interface-specific > authentication credentials strike me as painful to maintain, plus it > requires substantial administration. > > > > > From owner-ips@ece.cmu.edu Wed May 30 18:01:08 2001 Received: from ece.cmu.edu (ECE.CMU.EDU [128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id SAA25886 for ; Wed, 30 May 2001 18:01:08 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f4UJdhL23868 for ips-outgoing; Wed, 30 May 2001 15:39:43 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from internaut.com ([64.38.134.99]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f4UJdet23859 for ; Wed, 30 May 2001 15:39:40 -0400 (EDT) Received: from localhost (aboba@localhost) by internaut.com (8.9.3/8.9.3) with ESMTP id MAA51009; Wed, 30 May 2001 12:29:39 -0700 (PDT) (envelope-from aboba@internaut.com) Date: Wed, 30 May 2001 12:29:39 -0700 (PDT) From: Bernard Aboba To: Douglas Otis cc: julian_satran@il.ibm.com, ips@ece.cmu.edu Subject: RE: iSCSI and secure boot In-Reply-To: Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-ips@ece.cmu.edu Precedence: bulk > If the results are confirmed, then the process is confirmed. There should > be no need to validate every step in the process, just the last step as done > with PXE 2. As far as TFTP, it lives up to its name. A simple TFTP stack > can be done in a few dozen lines of code. It is really a minimum amount of > effort. Problem with PXE 2 is that it requires touching every PC (to install the boot image cert) and so isn't widely implemented. Also, if implemented doesn't support basic capabilities such as revocation. So most PCs implementing PXE are doing PXE 1 or PXE 2 with no boot image cert installed. Therefore for all practical purposes PXE = no security. From owner-ips@ece.cmu.edu Wed May 30 19:10:45 2001 Received: from ece.cmu.edu (ECE.CMU.EDU [128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id TAA27050 for ; Wed, 30 May 2001 19:10:43 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f4UKnff29653 for ips-outgoing; Wed, 30 May 2001 16:49:41 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from server1.NishanSystems.COM (smtp.nishansystems.com [216.217.36.162]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f4UKnet29648 for ; Wed, 30 May 2001 16:49:40 -0400 (EDT) Received: by smtp.nishansystems.com with Internet Mail Service (5.5.2653.19) id ; Wed, 30 May 2001 13:49:33 -0700 Message-ID: From: Joshua Tseng To: ips@ece.cmu.edu Subject: iSNS revision -03 available Date: Wed, 30 May 2001 13:49:26 -0700 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2653.19) Content-Type: text/plain; charset="iso-8859-1" Sender: owner-ips@ece.cmu.edu Precedence: bulk Folks, iSNS revision -03 document has been submitted to IETF. Until it is available on the IETF web site, it can be retrieved here: http://www.nishansystems.com/ietf/draft-ietf-ips-isns-03.txt The iSNS source code will be available within the next 36 hours, and it conforms to this revision of iSNS. The following are major changes for revision -03 of iSNS: 1) FCIP has been removed from the iSNS spec and the open source code. 2) iSCSI Portal Groups added for aggregating iSCSI interfaces 3) Added separate optional UDP port for ESI messages 4) Updated DDReg and DDSReg messages Regards, Josh Tseng From owner-ips@ece.cmu.edu Wed May 30 20:14:48 2001 Received: from ece.cmu.edu (ECE.CMU.EDU [128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id UAA27831 for ; Wed, 30 May 2001 20:14:48 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f4ULxiG05354 for ips-outgoing; Wed, 30 May 2001 17:59:44 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from c007.snv.cp.net (c007-h008.c007.snv.cp.net [209.228.33.214]) by ece.cmu.edu (8.11.0/8.10.2) with SMTP id f4ULxgt05350 for ; Wed, 30 May 2001 17:59:42 -0400 (EDT) Received: (cpmta 15769 invoked from network); 30 May 2001 14:59:36 -0700 Received: from adsl-63-202-160-80.dsl.snfc21.pacbell.net (HELO ljoy) (63.202.160.80) by smtp.telocity.com (209.228.33.214) with SMTP; 30 May 2001 14:59:36 -0700 X-Sent: 30 May 2001 21:59:36 GMT From: "Douglas Otis" To: "Bernard Aboba" Cc: , Subject: RE: iSCSI and secure boot Date: Wed, 30 May 2001 14:57:16 -0700 Message-ID: MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 (Normal) X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook IMO, Build 9.0.2416 (9.0.2911.0) Importance: Normal X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4522.1200 In-Reply-To: Sender: owner-ips@ece.cmu.edu Precedence: bulk Content-Transfer-Encoding: 7bit Bernard, Can you describe a secure diskless boot that does not require individual setup of each? The clever thing to do would be to ensure this process is extremely basic and does not include anything highly variable such as iSCSI. Use a second step to then obtain anything that may require extensive updates and variables such as an iSCSI kernel. This could be essentially a replication of the first boot but with some management added perhaps via a DUA. Make sure this is indeed a one-time setup. Keep it simple and there should be no problem with revoking or needing to update various options. Doug > Problem with PXE 2 is that it requires touching every PC (to install > the boot image cert) and so isn't widely implemented. Also, if > implemented doesn't support basic capabilities such as revocation. So most > PCs implementing PXE are doing PXE 1 or PXE 2 with no boot image cert > installed. Therefore for all practical purposes PXE = no security. From owner-ips@ece.cmu.edu Wed May 30 21:58:05 2001 Received: from ece.cmu.edu (ECE.CMU.EDU [128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id VAA29686 for ; Wed, 30 May 2001 21:58:05 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f4V07VG14509 for ips-outgoing; Wed, 30 May 2001 20:07:31 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from d12lmsgate-2.de.ibm.com (d12lmsgate-2.de.ibm.com [195.212.91.200]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f4V07Tt14502 for ; Wed, 30 May 2001 20:07:29 -0400 (EDT) Received: from d12relay01.de.ibm.com (d12relay01.de.ibm.com [9.165.215.22]) by d12lmsgate-2.de.ibm.com (1.0.0) with ESMTP id CAA173054 for ; Thu, 31 May 2001 02:07:23 +0200 From: julian_satran@il.ibm.com Received: from d12mta02.de.ibm.com (d12mta01_cs0 [9.165.222.237]) by d12relay01.de.ibm.com (8.8.8m3/NCO v4.96) with SMTP id CAA52808 for ; Thu, 31 May 2001 02:07:23 +0200 Received: by d12mta02.de.ibm.com(Lotus SMTP MTA v4.6.5 (863.2 5-20-1999)) id C1256A5D.00008A2C ; Thu, 31 May 2001 02:05:53 +0200 X-Lotus-FromDomain: IBMIL@IBMDE To: ips@ece.cmu.edu Message-ID: Date: Thu, 31 May 2001 01:26:12 +0300 Subject: RE: Using SNACK to request retransmission of R2T Mime-Version: 1.0 Content-type: text/plain; charset=us-ascii Content-Disposition: inline Sender: owner-ips@ece.cmu.edu Precedence: bulk David, R2Ts - the PDUs by which a target asks for data are treated as "incoming data". They have a DataSN field (I've renamed it R2TSN in 07). Their retransmission is requested by a Data SNACK. To avoid any more confusion the text for 07 reads: 1.1 SNACK Request Byte / 0 | 1 | 2 | 3 | / | | | | |7 6 5 4 3 2 1 0|7 6 5 4 3 2 1 0|7 6 5 4 3 2 1 0|7 6 5 4 3 2 1 0| +---------------+---------------+---------------+---------------+ 0|0|1| 0x10 |1|Reserved(0)|S| Reserved (0) | +---------------+---------------+---------------+---------------+ 4/ Reserved (0) / +/ / +---------------+---------------+---------------+---------------+ 16| Initiator Task Tag or Reserved (0x'ffffffff') | +---------------+---------------+---------------+---------------+ 20| BegRun | +---------------+---------------+---------------+---------------+ 24| RunLength | +---------------+---------------+---------------+---------------+ 28| ExpStatSN/ExpDataSN | +---------------+---------------+---------------+---------------+ 32/ Reserved (0) / +/ / +---------------+---------------+---------------+---------------+ 48 SNACK request is used to request retransmission of status, data or R2T PDUs from the target. The SNACK request indicates to the target the missed status or data run, where the run is composed of an initial missed StatSN, DataSN or R2TSN and the number of additional missed Status, Data or R2T PDUs (0 means only the initial). 1.1.1 S If 1, indicates that this is a Status SNACK; otherwise it is a Data or R2T SNACK. Data/R2T SNACK for a command MUST precede implicit or explicit status acknowledgement for the given command. For a Data/R2T SNACK, the Initiator Task Tag MUST be set to the Initiator Task Tag of the referenced Command. Otherwise, it is reserved. iSCSI targets MUST support Status SNACK and MAY support Data/R2T SNACK. 1.1.2 BegRun First missed DataSN, R2TSN or StatSN 1.1.3 RunLength Number of additional sequential missed DataSN or StatSN. If BegRun is the only one missing, RunLength MUST be 0. 1.1.4 ExpStatSN/ExpDataSN ExpStatSN if S is 1 otherwise ExpDataSN. 1.1.5 Additional Runs Contains 0, 1 or 2 additional double words - the first being the BegRun and the second the RunLength of the additional run. Julo David Lee on 30-05-2001 17:30:42 Please respond to David Lee To: Julian Satran/Haifa/IBM@IBMIL, "'ips@ece.cmu.edu '" cc: Subject: RE: Using SNACK to request retransmission of R2T I'm sorry, but I did not understand this response. The case I am inquiring about involves a missing R2T PDU (detected as a result of receiving another R2T with a higher DataSn). The following excerp is from section 6.7.1: (1)Lost data PDU or lost R2T - a data PDU or R2T may be lost due to a header digest error or a data digest error. In case of a data digest error, the error is recognized immediately and the initiator MAY request the missing data through SNACK. In case of a header digest error, the initiator recognizes the missing data or R2T either when receiving a subsequent piece out of sequence or by a timeout in completing a sequence (no status). In this case, the initiator MAY request the missing data or R2T through a SNACK. The description of the SNACK PDU does not appear to include a form for requesting retransmission of R2T. David Lee -----Original Message----- From: julian_satran@il.ibm.com To: ips@ece.cmu.edu Sent: 5/29/01 8:07 PM Subject: Re: Using SNACK to request retransmission of R2T The SNACK is just a data SNACK (R2T are treated as input data for the command). Julo David Lee on 30-05-2001 04:24:01 Please respond to David Lee To: "'ips@ece.cmu.edu'" cc: Subject: Using SNACK to request retransmission of R2T Hi all, Please forgive me if this has already been addressed. In at least one place in the Rev. 6 draft it states that SNACK may be used by an initiator to request a resend by a target of a missing R2T PDU. However, the format of the SNACK PDU does not seem to cover this case. David Lee Vicom Systems, Inc. From owner-ips@ece.cmu.edu Wed May 30 22:00:03 2001 Received: from ece.cmu.edu (ECE.CMU.EDU [128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id WAA29716 for ; Wed, 30 May 2001 22:00:03 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f4V07WM14510 for ips-outgoing; Wed, 30 May 2001 20:07:32 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from d12lmsgate-2.de.ibm.com (d12lmsgate-2.de.ibm.com [195.212.91.200]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f4V07Tt14501 for ; Wed, 30 May 2001 20:07:29 -0400 (EDT) Received: from d12relay01.de.ibm.com (d12relay01.de.ibm.com [9.165.215.22]) by d12lmsgate-2.de.ibm.com (1.0.0) with ESMTP id CAA316884 for ; Thu, 31 May 2001 02:07:23 +0200 From: julian_satran@il.ibm.com Received: from d12mta02.de.ibm.com (d12mta01_cs0 [9.165.222.237]) by d12relay01.de.ibm.com (8.8.8m3/NCO v4.96) with SMTP id CAA52806 for ; Thu, 31 May 2001 02:07:23 +0200 Received: by d12mta02.de.ibm.com(Lotus SMTP MTA v4.6.5 (863.2 5-20-1999)) id C1256A5D.00008A94 ; Thu, 31 May 2001 02:05:54 +0200 X-Lotus-FromDomain: IBMIL@IBMDE To: ips@ece.cmu.edu Message-ID: Date: Thu, 31 May 2001 01:43:24 +0300 Subject: RE: iSCSI and secure boot Mime-Version: 1.0 Content-type: text/plain; charset=us-ascii Content-Disposition: inline Sender: owner-ips@ece.cmu.edu Precedence: bulk Doug, I don't understand your statement at all. Please translate in simple English - simple enough for a non-native speaker as myself. Julo "Douglas Otis" on 30-05-2001 21:38:48 Please respond to "Douglas Otis" To: Julian Satran/Haifa/IBM@IBMIL, "Bernard Aboba" cc: ips@ece.cmu.edu Subject: RE: iSCSI and secure boot Julian, If the results are confirmed, then the process is confirmed. There should be no need to validate every step in the process, just the last step as done with PXE 2. As far as TFTP, it lives up to its name. A simple TFTP stack can be done in a few dozen lines of code. It is really a minimum amount of effort. Doug > Bernard, > > PXE is assuming that the boot image is a clearly defined > (bounded) piece of > data. > SCSI booting does not assume this. Signing the total boot image is > something you do in the regular SCSI (local) load process. All your other > points (securing DHCP, securing a miniboot loader) are valid and should be > considered > by all vendors. > > I wonder however if we should consider standardizing all this behaviour > before we some more hands-on experience > in what is really needed and which of the solution is viable. > > If we take the PXE path we might be tempted to mandate a "mini-TFTP" in > each target to be able to load a "startboot". > > Would this be wise to this now? > > For all those reasons I would be reluctant to add too much to the current > boot draft beyond perhaps a simple authentication as mandatory. > > The only provision we have made in the main draft is the "Boot" key to > indicate to the target that it can limit the > access of an initiator doing this type of logon. > > Julo > > Bernard Aboba on 30-05-2001 06:25:53 > > Please respond to Bernard Aboba > > To: Julian Satran/Haifa/IBM@IBMIL > cc: ips@ece.cmu.edu, aboba@internaut.com > Subject: Re: iSCSI and secure boot > > > > > > As iSCSI is transporting SCSI and SCSI has a different boot > paradigm than > > Netboot can you please elaborate on what exactly should be an > authenticated > > boot image in this (SCSI) context. > > > > Please take into consideration that unlike netboot - the SCSI > boot is not > a > > clearly bounded process (not even in PXE - an "open" > proprietary scheme). > > > I'm no PXE expert, but here is my (admittedly limited) understanding of > what goes on: > > 1. Many OEMs ship PXE-enabled PCs, and in most of these PXE cannot be > turned off, though it can be set lower in the boot order. In some > percentage of machines PXE will even be higher in the boot order than the > hard disk (bad idea, for reasons I'll describe). > > 2. Machine boots and if it reaches PXE in the boot order, it sends a > DHCPDISCOVER. If responded to with a DHCPOFFER including Option 60 > ("PXECLIENT") and TFTP server, it will finish obtaining an address via > DHCP and then pull a boot image from the TFTP server. This is typically > startrom.com, a 16-bit executable. Note that this is often *not* a full OS > image; for example Ghost supports PXE and uses this early load in order to > reformat the hard disk and start the download of the new "cloned" disk > image. Other PXE supporting-apps may do other things with this early boot > image. > > 3. If the machine is PXE 2.0 enabled, it will support signed boot images, > and will check the integrity of the boot image prior to executing it. This > part of the spec is called BIS -- and requires the public key of the boot > image signer to be pre-loaded on the PC. My understanding is that PXE 2.0 > and BIS are not widely implemented and as a result, PXE 1.0 systems > are most widespread and they execute whatever boot image they are > fed. This is one reason why putting PXE high in the boot order is a bad > idea. This would enable a rogue ghost server to reformat your hard > drive. Ouch! (this happens in real life, unfortunately). > > 4. PXE 2.0 does not support authenticated DHCP nor any TFTP security, so > that it is vulnerable to rogue DHCP servers, offers no support for client > authentication, and only checks the integrity of the boot image once it > has completed the download of it. However, since it also requires > pre-loading the boot image signing cert on the PC, and doesn't support > revocation of invalidated boot images (which can be endlessly replayed), > in practice PXE 2.0 is not widely deployed and doesn't do a good job in > boot image validation. > > 5. The small startrom.com loader is then used to pull down a more complete > OS image, either installing it on the hard drive, or pulling it into > memory. At this point, for example, you may have the appropriate drivers > loaded, be operating in 32-bit mode, etc. > > As far as how iSCSI fits in to all this, I'd hope you would enlighten me > ;) As has been suggested, the iSCSI target address could be obtained via > DHCP, and suitable initial iSCSI drivers provided via the initial boot > load obtained via TFTP. > > Given the security headaches inherent in PXE, I am really interested in > how all of this could be done in a deployable, secure manner. The picture > is somewhat confusing today, and the following questions come to mind: > > a. What credentials are needed to secure iSCSI boot? > 1. Kerberos machine credentials? > 2. Identity/shared keys for IPSEC? > b. What credentials are needed to secure DHCP? > 1. According to -16, you need the authenticated DHCP key > (unique per htype/MAC address) > 2. According to draft-hornstein-dhc-kerbauth-0x.txt you > need Kerb credentials. > 3. Do we care? (see below) > c. What credentials are needed to verify the boot image? > 1. The boot image signing certificate (BIS) > 2. Something else? (Shared secret, Kerb credentials?) > > Storing several identities and credential sets in PC NVRAM is difficult to > administer and deploy. It makes sense to boil this mess down > to a small list of items. But how? > > Some thoughts: > > * Kerberos clients are small, and I am told, romable. Might it not make > sense to use Kerberos to secure PXE? Could we kill multiple birds > (iSCSI security, 802.1X, NFS, TFTP, boot image security) by doing this? > > * Do we really need DHCP authentication for secure boot? > Interface-specific > authentication credentials strike me as painful to maintain, plus it > requires substantial administration. > > > > > From owner-ips@ece.cmu.edu Wed May 30 23:43:18 2001 Received: from ece.cmu.edu (ECE.CMU.EDU [128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id XAA02205 for ; Wed, 30 May 2001 23:43:18 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f4V1c0j20280 for ips-outgoing; Wed, 30 May 2001 21:38:00 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from mxbh4.isus.emc.com (mxbh4.isus.emc.com [128.221.10.33]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f4V1bxt20276 for ; Wed, 30 May 2001 21:37:59 -0400 (EDT) Received: by mxbh4.isus.emc.com with Internet Mail Service (5.5.2653.19) id ; Wed, 30 May 2001 21:37:53 -0400 Message-ID: <0F31E5C394DAD311B60C00E029101A070801562F@corpmx9.isus.emc.com> From: Black_David@emc.com To: cmonia@NishanSystems.com, ips@ece.cmu.edu Subject: RE: Common Encapsulation: Stale Frame Detection in an IP fabric Date: Wed, 30 May 2001 21:37:07 -0400 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2653.19) Content-Type: text/plain Sender: owner-ips@ece.cmu.edu Precedence: bulk Something like that. If the two endpoints can agree on a common timebase and control their relative drift, that's good enough to detect stale FC frames even if the timebase doesn't correspond to a wall clock. I'm wondering about situations in which an FCIP box is deployed into a situation where there isn't a good SNTP source available. Thanks, --David --------------------------------------------------- David L. Black, Senior Technologist EMC Corporation, 42 South St., Hopkinton, MA 01748 +1 (508) 435-1000 x75140 FAX: +1 (508) 497-8500 black_david@emc.com Mobile: +1 (978) 394-7754 --------------------------------------------------- > -----Original Message----- > From: Charles Monia [SMTP:cmonia@NishanSystems.com] > Sent: Friday, May 25, 2001 9:49 PM > To: ips@ece.cmu.edu > Subject: RE: Common Encapsulation: Stale Frame Detection in an IP > fabric > > Hi David: > > How do you see clock synchronization happening? Does the FCIP end point > use > an SNTP handshake? > > Charles > > -----Original Message----- > > From: Black_David@emc.com [mailto:Black_David@emc.com] > > Sent: Friday, May 25, 2001 2:43 PM > > To: cmonia@NishanSystems.com; ips@ece.cmu.edu > > Subject: RE: Common Encapsulation: Stale Frame Detection in > > an IP fabric > > > > > > Question for Charles and the WG: > > > > This proposal is based on synchronizing each node > > to the same wall clock reference (i.e., SNTP server). > > Is there any interest in just synchronizing the > > timestamps between the two ends of an FCIP link > > without requiring interaction with an external > > time source? > > > > A possible downside is that an implementation > > that can support multiple FCIP links may need > > to maintain a separate time offset (from its > > internal time source) for each FCIP link. > > > > NOTE: This is a question and not to be taken > > as a suggestion from a co-chair. > > > > Thanks, > > --David > > From owner-ips@ece.cmu.edu Thu May 31 00:50:45 2001 Received: from ece.cmu.edu (ECE.CMU.EDU [128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id AAA02846 for ; Thu, 31 May 2001 00:50:44 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f4V36OQ26129 for ips-outgoing; Wed, 30 May 2001 23:06:24 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from d12lmsgate.de.ibm.com (d12lmsgate.de.ibm.com [195.212.91.199]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f4V36Mt26124 for ; Wed, 30 May 2001 23:06:22 -0400 (EDT) Received: from d12relay02.de.ibm.com (d12relay02.de.ibm.com [9.165.215.23]) by d12lmsgate.de.ibm.com (1.0.0) with ESMTP id FAA199412 for ; Thu, 31 May 2001 05:06:15 +0200 From: julian_satran@il.ibm.com Received: from d12mta02.de.ibm.com (d12mta01_cs0 [9.165.222.237]) by d12relay02.de.ibm.com (8.8.8m3/NCO v4.96) with SMTP id FAA133272 for ; Thu, 31 May 2001 05:05:33 +0200 Received: by d12mta02.de.ibm.com(Lotus SMTP MTA v4.6.5 (863.2 5-20-1999)) id C1256A5D.0010FC30 ; Thu, 31 May 2001 05:05:31 +0200 X-Lotus-FromDomain: IBMIL@IBMDE To: ips@ece.cmu.edu Message-ID: Date: Thu, 31 May 2001 06:08:12 +0300 Subject: Re: More on iSCSI boot Mime-Version: 1.0 Content-type: text/plain; charset=us-ascii Content-Disposition: inline Sender: owner-ips@ece.cmu.edu Precedence: bulk David, The trouble with our boot is two-fold: -the SCSI boot - unlike the network boot is unbounded (there is no such thing as an image). -even if we would like to standardize a "primary" or "minimal" boot we have no good understanding (experience) of how this will interact with the iSCSI security mechanisms. Julo Black_David@emc.com on 31-05-2001 05:49:57 Please respond to Black_David@emc.com To: ips@ece.cmu.edu cc: Subject: More on iSCSI boot Let's start with a simple boot from a disk - the system BIOS reads the boot sector(s) off of the disk drive, loads and runs it/them. That primary bootloader then handles whatever else is necessary based on the ability to do disk reads (a secondary bootloader and/or other things may be involved). On Intel systems, it's generally a combination of the system BIOS and card BIOS that make the disk reads work. Simplifying assumptions are common (e.g., boot from LUN 0 on the first SCSI target found). The goal of the iSCSI boot draft is to explain how to make this "simple boot from a disk" mechanism work when the boot disk is attached via iSCSI. An iSCSI adapter has some things to do in order to make this work, e.g., it has to log into the target before disk reads can be issued. I tend to believe that "Don't do this (i.e., try to boot over iSCSI)" is not an acceptable answer. In the message that started this whole boot discussion, I suggested that Using DHCP to find SLP to find the boot device seems both clumsy and an invitation to problems (one more thing that can break and prevent booting), I think that's doubly true if LDAP is used instead of SLP. David Robinson's messages support my inclination to reuse DHCP option 17 (Root Path) by defining iSCSI syntax for it. Between that, any TCP parameters that one wants to set through DHCP, and iSCSI parameters that can be defaulted, it should be possible to get the first disk reads done through iSCSI. As has been pointed out, booting is often poorly secured in general. While it'd be nice to change this, iSCSI will face the same pressures that other boot mechanisms face - keep it simple, get the boot image loaded, and let it do something fancier. Any signature/validation of the boot image will be above the level of iSCSI. Somehow, I don't expect to find implementations of ESP in card BIOSes anytime soon. If implementers want to use DHCP and TFTP to boot, I don't see any point in stopping them, but I don't think either should be mandated. DHCP has centralized administration advantages, and TFTP is a simple way to download code, but both are "one more thing that can break and prevent booting" and hence may not be used all the time. Comments? --David --------------------------------------------------- David L. Black, Senior Technologist EMC Corporation, 42 South St., Hopkinton, MA 01748 +1 (508) 435-1000 x75140 FAX: +1 (508) 497-8500 black_david@emc.com Mobile: +1 (978) 394-7754 --------------------------------------------------- From owner-ips@ece.cmu.edu Thu May 31 00:55:17 2001 Received: from ece.cmu.edu (ECE.CMU.EDU [128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id AAA02921 for ; Thu, 31 May 2001 00:55:17 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f4V2o8P25107 for ips-outgoing; Wed, 30 May 2001 22:50:08 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from maho3msx2.isus.emc.com (maho3msx2.isus.emc.com [128.221.11.32]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f4V2o5t25094 for ; Wed, 30 May 2001 22:50:05 -0400 (EDT) Received: by maho3msx2.isus.emc.com with Internet Mail Service (5.5.2653.19) id ; Wed, 30 May 2001 22:49:59 -0400 Message-ID: <0F31E5C394DAD311B60C00E029101A0708015630@corpmx9.isus.emc.com> From: Black_David@emc.com To: ips@ece.cmu.edu Subject: More on iSCSI boot Date: Wed, 30 May 2001 22:49:57 -0400 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2653.19) Content-Type: text/plain Sender: owner-ips@ece.cmu.edu Precedence: bulk Let's start with a simple boot from a disk - the system BIOS reads the boot sector(s) off of the disk drive, loads and runs it/them. That primary bootloader then handles whatever else is necessary based on the ability to do disk reads (a secondary bootloader and/or other things may be involved). On Intel systems, it's generally a combination of the system BIOS and card BIOS that make the disk reads work. Simplifying assumptions are common (e.g., boot from LUN 0 on the first SCSI target found). The goal of the iSCSI boot draft is to explain how to make this "simple boot from a disk" mechanism work when the boot disk is attached via iSCSI. An iSCSI adapter has some things to do in order to make this work, e.g., it has to log into the target before disk reads can be issued. I tend to believe that "Don't do this (i.e., try to boot over iSCSI)" is not an acceptable answer. In the message that started this whole boot discussion, I suggested that Using DHCP to find SLP to find the boot device seems both clumsy and an invitation to problems (one more thing that can break and prevent booting), I think that's doubly true if LDAP is used instead of SLP. David Robinson's messages support my inclination to reuse DHCP option 17 (Root Path) by defining iSCSI syntax for it. Between that, any TCP parameters that one wants to set through DHCP, and iSCSI parameters that can be defaulted, it should be possible to get the first disk reads done through iSCSI. As has been pointed out, booting is often poorly secured in general. While it'd be nice to change this, iSCSI will face the same pressures that other boot mechanisms face - keep it simple, get the boot image loaded, and let it do something fancier. Any signature/validation of the boot image will be above the level of iSCSI. Somehow, I don't expect to find implementations of ESP in card BIOSes anytime soon. If implementers want to use DHCP and TFTP to boot, I don't see any point in stopping them, but I don't think either should be mandated. DHCP has centralized administration advantages, and TFTP is a simple way to download code, but both are "one more thing that can break and prevent booting" and hence may not be used all the time. Comments? --David --------------------------------------------------- David L. Black, Senior Technologist EMC Corporation, 42 South St., Hopkinton, MA 01748 +1 (508) 435-1000 x75140 FAX: +1 (508) 497-8500 black_david@emc.com Mobile: +1 (978) 394-7754 --------------------------------------------------- From owner-ips@ece.cmu.edu Thu May 31 02:00:12 2001 Received: from ece.cmu.edu (ECE.CMU.EDU [128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id CAA06492 for ; Thu, 31 May 2001 02:00:12 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f4V3wn129517 for ips-outgoing; Wed, 30 May 2001 23:58:49 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from patan.sun.com (patan.Sun.COM [192.18.98.43]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f4V3wlt29512 for ; Wed, 30 May 2001 23:58:47 -0400 (EDT) Received: from phys-ha6nwka.ebay.sun.com ([129.149.1.82]) by patan.sun.com (8.9.3+Sun/8.9.3) with ESMTP id VAA15338 for ; Wed, 30 May 2001 21:58:45 -0600 (MDT) Received: from ebay.sun.com (dsl197-15.Eng.Sun.COM [129.146.197.15]) by phys-ha6nwka.ebay.sun.com (8.9.3+Sun/8.9.3/ENSMAIL,v2.1p1) with ESMTP id UAA24951 for ; Wed, 30 May 2001 20:58:46 -0700 (PDT) Message-ID: <3B15C1C0.82895D8F@ebay.sun.com> Date: Wed, 30 May 2001 21:00:00 -0700 From: David Robinson Organization: Sun Microsystems, Inc. X-Mailer: Mozilla 4.77 [en] (Win98; U) X-Accept-Language: en MIME-Version: 1.0 To: ips@ece.cmu.edu Subject: Re: More on iSCSI boot References: Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-ips@ece.cmu.edu Precedence: bulk Content-Transfer-Encoding: 7bit Julian, I think you are simply confused on terminology. I think when David says "boot image" he is using in generally to mean transfering enough data to provide a self supporting operating system. In the case of SCSI the "image" is set of sectors necessary to start the kernel, similarily in NFS it is typically the "vmunix" file(s). It is not the full LUN representing the root filesystem. As Bernard has shown, a flexible and scaleable mechanism to bootstrap a security system without a disk is just very hard. We need to do a reality check on the environment that iSCSI boot will use. I don't think there will be a large market for booting over the Internet, but boot disks will be physically co-located in constrained environments where DHCP will be "good enough". While the IESG may not like the weak security, I would be surprised if it insisted that the ips WG take on the task of building a better DHCP. -David julian_satran@il.ibm.com wrote: > > David, > > The trouble with our boot is two-fold: > > -the SCSI boot - unlike the network boot is unbounded (there is no such > thing as an image). > -even if we would like to standardize a "primary" or "minimal" boot we have > no good understanding (experience) > of how this will interact with the iSCSI security mechanisms. > > Julo > > Black_David@emc.com on 31-05-2001 05:49:57 > > Please respond to Black_David@emc.com > > To: ips@ece.cmu.edu > cc: > Subject: More on iSCSI boot > > Let's start with a simple boot from a disk - the system > BIOS reads the boot sector(s) off of the disk drive, > loads and runs it/them. That primary bootloader then > handles whatever else is necessary based on the ability > to do disk reads (a secondary bootloader and/or other > things may be involved). On Intel systems, it's generally > a combination of the system BIOS and card BIOS that > make the disk reads work. Simplifying assumptions > are common (e.g., boot from LUN 0 on the first SCSI > target found). The goal of the iSCSI boot draft is > to explain how to make this "simple boot from a disk" > mechanism work when the boot disk is attached via iSCSI. > > An iSCSI adapter has some things to do in order to make > this work, e.g., it has to log into the target before disk > reads can be issued. I tend to believe that "Don't do this > (i.e., try to boot over iSCSI)" is not an acceptable answer. > > In the message that started this whole boot discussion, I > suggested that > > Using DHCP to find SLP to find the boot device seems > both clumsy and an invitation to problems (one more > thing that can break and prevent booting), > > I think that's doubly true if LDAP is used instead of SLP. > David Robinson's messages support my inclination to reuse > DHCP option 17 (Root Path) by defining iSCSI syntax > for it. Between that, any TCP parameters that one wants to > set through DHCP, and iSCSI parameters that can be defaulted, > it should be possible to get the first disk reads done > through iSCSI. > > As has been pointed out, booting is often poorly secured in > general. While it'd be nice to change this, iSCSI will face > the same pressures that other boot mechanisms face - keep > it simple, get the boot image loaded, and let it do something > fancier. Any signature/validation of the boot image will be > above the level of iSCSI. Somehow, I don't expect to find > implementations of ESP in card BIOSes anytime soon. > > If implementers want to use DHCP and TFTP to boot, I don't > see any point in stopping them, but I don't think either should > be mandated. DHCP has centralized administration advantages, > and TFTP is a simple way to download code, but both are "one > more thing that can break and prevent booting" and hence may > not be used all the time. > > Comments? > --David > > --------------------------------------------------- > David L. Black, Senior Technologist > EMC Corporation, 42 South St., Hopkinton, MA 01748 > +1 (508) 435-1000 x75140 FAX: +1 (508) 497-8500 > black_david@emc.com Mobile: +1 (978) 394-7754 > --------------------------------------------------- From owner-ips@ece.cmu.edu Thu May 31 03:41:50 2001 Received: from ece.cmu.edu (ECE.CMU.EDU [128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id DAA18338 for ; Thu, 31 May 2001 03:41:50 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f4V6V3N08892 for ips-outgoing; Thu, 31 May 2001 02:31:03 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from homa.asicdesigners.com (ip-64-63-65-105.reverse.mobilenetics.com [64.63.65.105] (may be forged)) by ece.cmu.edu (8.11.0/8.10.2) with SMTP id f4V6V1t08888 for ; Thu, 31 May 2001 02:31:02 -0400 (EDT) Received: from homa.asicdesigners.com ([10.1.1.5]) by homa.asicdesigners.com with Microsoft SMTPSVC(5.0.2195.2966); Wed, 30 May 2001 23:30:46 -0700 Received: by homa.asicdesigners.com (Microsoft Connector for POP3 Mailboxes 5.00.2195) with SMTP (Global POP3 Download) id MSG05302001-233039-11.MMD@asicdesigners.com; Wed, 30 May 2001 23:30:39 -0700 Received: from ece.cmu.edu (ECE.CMU.EDU [128.2.236.200]) by mail14.netservers.net (8.9.3/8.9.3) with ESMTP id XAA09744 for ; Wed, 30 May 2001 23:24:17 -0700 Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f4V3wn129517 for ips-outgoing; Wed, 30 May 2001 23:58:49 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from patan.sun.com (patan.Sun.COM [192.18.98.43]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f4V3wlt29512 for ; Wed, 30 May 2001 23:58:47 -0400 (EDT) Received: from phys-ha6nwka.ebay.sun.com ([129.149.1.82]) by patan.sun.com (8.9.3+Sun/8.9.3) with ESMTP id VAA15338 for ; Wed, 30 May 2001 21:58:45 -0600 (MDT) Received: from ebay.sun.com (dsl197-15.Eng.Sun.COM [129.146.197.15]) by phys-ha6nwka.ebay.sun.com (8.9.3+Sun/8.9.3/ENSMAIL,v2.1p1) with ESMTP id UAA24951 for ; Wed, 30 May 2001 20:58:46 -0700 (PDT) Message-ID: <3B15C1C0.82895D8F@ebay.sun.com> Date: Wed, 30 May 2001 21:00:00 -0700 From: David Robinson Organization: Sun Microsystems, Inc. X-Mailer: Mozilla 4.77 [en] (Win98; U) X-Accept-Language: en MIME-Version: 1.0 To: ips@ece.cmu.edu Subject: Re: More on iSCSI boot References: Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-UIDL: b4182fa012ef75b0bc9a62b969be91f4 X-OriginalArrivalTime: 31 May 2001 06:30:46.0750 (UTC) FILETIME=[39EE5FE0:01C0E99B] Sender: owner-ips@ece.cmu.edu Precedence: bulk Content-Transfer-Encoding: 7bit Julian, I think you are simply confused on terminology. I think when David says "boot image" he is using in generally to mean transfering enough data to provide a self supporting operating system. In the case of SCSI the "image" is set of sectors necessary to start the kernel, similarily in NFS it is typically the "vmunix" file(s). It is not the full LUN representing the root filesystem. As Bernard has shown, a flexible and scaleable mechanism to bootstrap a security system without a disk is just very hard. We need to do a reality check on the environment that iSCSI boot will use. I don't think there will be a large market for booting over the Internet, but boot disks will be physically co-located in constrained environments where DHCP will be "good enough". While the IESG may not like the weak security, I would be surprised if it insisted that the ips WG take on the task of building a better DHCP. -David julian_satran@il.ibm.com wrote: > > David, > > The trouble with our boot is two-fold: > > -the SCSI boot - unlike the network boot is unbounded (there is no such > thing as an image). > -even if we would like to standardize a "primary" or "minimal" boot we have > no good understanding (experience) > of how this will interact with the iSCSI security mechanisms. > > Julo > > Black_David@emc.com on 31-05-2001 05:49:57 > > Please respond to Black_David@emc.com > > To: ips@ece.cmu.edu > cc: > Subject: More on iSCSI boot > > Let's start with a simple boot from a disk - the system > BIOS reads the boot sector(s) off of the disk drive, > loads and runs it/them. That primary bootloader then > handles whatever else is necessary based on the ability > to do disk reads (a secondary bootloader and/or other > things may be involved). On Intel systems, it's generally > a combination of the system BIOS and card BIOS that > make the disk reads work. Simplifying assumptions > are common (e.g., boot from LUN 0 on the first SCSI > target found). The goal of the iSCSI boot draft is > to explain how to make this "simple boot from a disk" > mechanism work when the boot disk is attached via iSCSI. > > An iSCSI adapter has some things to do in order to make > this work, e.g., it has to log into the target before disk > reads can be issued. I tend to believe that "Don't do this > (i.e., try to boot over iSCSI)" is not an acceptable answer. > > In the message that started this whole boot discussion, I > suggested that > > Using DHCP to find SLP to find the boot device seems > both clumsy and an invitation to problems (one more > thing that can break and prevent booting), > > I think that's doubly true if LDAP is used instead of SLP. > David Robinson's messages support my inclination to reuse > DHCP option 17 (Root Path) by defining iSCSI syntax > for it. Between that, any TCP parameters that one wants to > set through DHCP, and iSCSI parameters that can be defaulted, > it should be possible to get the first disk reads done > through iSCSI. > > As has been pointed out, booting is often poorly secured in > general. While it'd be nice to change this, iSCSI will face > the same pressures that other boot mechanisms face - keep > it simple, get the boot image loaded, and let it do something > fancier. Any signature/validation of the boot image will be > above the level of iSCSI. Somehow, I don't expect to find > implementations of ESP in card BIOSes anytime soon. > > If implementers want to use DHCP and TFTP to boot, I don't > see any point in stopping them, but I don't think either should > be mandated. DHCP has centralized administration advantages, > and TFTP is a simple way to download code, but both are "one > more thing that can break and prevent booting" and hence may > not be used all the time. > > Comments? > --David > > --------------------------------------------------- > David L. Black, Senior Technologist > EMC Corporation, 42 South St., Hopkinton, MA 01748 > +1 (508) 435-1000 x75140 FAX: +1 (508) 497-8500 > black_david@emc.com Mobile: +1 (978) 394-7754 > --------------------------------------------------- From owner-ips@ece.cmu.edu Thu May 31 03:42:34 2001 Received: from ece.cmu.edu (ECE.CMU.EDU [128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id DAA18350 for ; Thu, 31 May 2001 03:42:33 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f4V5teX06863 for ips-outgoing; Thu, 31 May 2001 01:55:40 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from homa.asicdesigners.com (ip-64-63-65-105.reverse.mobilenetics.com [64.63.65.105] (may be forged)) by ece.cmu.edu (8.11.0/8.10.2) with SMTP id f4V5tZt06856 for ; Thu, 31 May 2001 01:55:35 -0400 (EDT) Received: from homa.asicdesigners.com ([10.1.1.5]) by homa.asicdesigners.com with Microsoft SMTPSVC(5.0.2195.2966); Wed, 30 May 2001 22:55:53 -0700 Received: by homa.asicdesigners.com (Microsoft Connector for POP3 Mailboxes 5.00.2195) with SMTP (Global POP3 Download) id MSG05302001-225537-2.MMD@asicdesigners.com; Wed, 30 May 2001 22:55:37 -0700 Received: from ece.cmu.edu (ECE.CMU.EDU [128.2.236.200]) by mail14.netservers.net (8.9.3/8.9.3) with ESMTP id WAA07741 for ; Wed, 30 May 2001 22:08:29 -0700 Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f4V2o8P25107 for ips-outgoing; Wed, 30 May 2001 22:50:08 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from maho3msx2.isus.emc.com (maho3msx2.isus.emc.com [128.221.11.32]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f4V2o5t25094 for ; Wed, 30 May 2001 22:50:05 -0400 (EDT) Received: by maho3msx2.isus.emc.com with Internet Mail Service (5.5.2653.19) id ; Wed, 30 May 2001 22:49:59 -0400 Message-ID: <0F31E5C394DAD311B60C00E029101A0708015630@corpmx9.isus.emc.com> From: Black_David@emc.com To: ips@ece.cmu.edu Subject: More on iSCSI boot Date: Wed, 30 May 2001 22:49:57 -0400 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2653.19) Content-Type: text/plain X-UIDL: 4973d51a27ae9898dfbcb7f2bb959dfa X-OriginalArrivalTime: 31 May 2001 05:55:53.0671 (UTC) FILETIME=[5A5BDD70:01C0E996] Sender: owner-ips@ece.cmu.edu Precedence: bulk Let's start with a simple boot from a disk - the system BIOS reads the boot sector(s) off of the disk drive, loads and runs it/them. That primary bootloader then handles whatever else is necessary based on the ability to do disk reads (a secondary bootloader and/or other things may be involved). On Intel systems, it's generally a combination of the system BIOS and card BIOS that make the disk reads work. Simplifying assumptions are common (e.g., boot from LUN 0 on the first SCSI target found). The goal of the iSCSI boot draft is to explain how to make this "simple boot from a disk" mechanism work when the boot disk is attached via iSCSI. An iSCSI adapter has some things to do in order to make this work, e.g., it has to log into the target before disk reads can be issued. I tend to believe that "Don't do this (i.e., try to boot over iSCSI)" is not an acceptable answer. In the message that started this whole boot discussion, I suggested that Using DHCP to find SLP to find the boot device seems both clumsy and an invitation to problems (one more thing that can break and prevent booting), I think that's doubly true if LDAP is used instead of SLP. David Robinson's messages support my inclination to reuse DHCP option 17 (Root Path) by defining iSCSI syntax for it. Between that, any TCP parameters that one wants to set through DHCP, and iSCSI parameters that can be defaulted, it should be possible to get the first disk reads done through iSCSI. As has been pointed out, booting is often poorly secured in general. While it'd be nice to change this, iSCSI will face the same pressures that other boot mechanisms face - keep it simple, get the boot image loaded, and let it do something fancier. Any signature/validation of the boot image will be above the level of iSCSI. Somehow, I don't expect to find implementations of ESP in card BIOSes anytime soon. If implementers want to use DHCP and TFTP to boot, I don't see any point in stopping them, but I don't think either should be mandated. DHCP has centralized administration advantages, and TFTP is a simple way to download code, but both are "one more thing that can break and prevent booting" and hence may not be used all the time. Comments? --David --------------------------------------------------- David L. Black, Senior Technologist EMC Corporation, 42 South St., Hopkinton, MA 01748 +1 (508) 435-1000 x75140 FAX: +1 (508) 497-8500 black_david@emc.com Mobile: +1 (978) 394-7754 --------------------------------------------------- From owner-ips@ece.cmu.edu Thu May 31 03:44:10 2001 Received: from ece.cmu.edu (ECE.CMU.EDU [128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id DAA18373 for ; Thu, 31 May 2001 03:44:10 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f4V5tbQ06860 for ips-outgoing; Thu, 31 May 2001 01:55:37 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from homa.asicdesigners.com (ip-64-63-65-105.reverse.mobilenetics.com [64.63.65.105] (may be forged)) by ece.cmu.edu (8.11.0/8.10.2) with SMTP id f4V5tZt06852 for ; Thu, 31 May 2001 01:55:35 -0400 (EDT) Received: from homa.asicdesigners.com ([10.1.1.5]) by homa.asicdesigners.com with Microsoft SMTPSVC(5.0.2195.2966); Wed, 30 May 2001 22:55:53 -0700 Received: by homa.asicdesigners.com (Microsoft Connector for POP3 Mailboxes 5.00.2195) with SMTP (Global POP3 Download) id MSG05302001-225536-1.MMD@asicdesigners.com; Wed, 30 May 2001 22:55:36 -0700 Received: from ece.cmu.edu (ECE.CMU.EDU [128.2.236.200]) by mail14.netservers.net (8.9.3/8.9.3) with ESMTP id WAA07694 for ; Wed, 30 May 2001 22:07:12 -0700 Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f4V36OQ26129 for ips-outgoing; Wed, 30 May 2001 23:06:24 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from d12lmsgate.de.ibm.com (d12lmsgate.de.ibm.com [195.212.91.199]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f4V36Mt26124 for ; Wed, 30 May 2001 23:06:22 -0400 (EDT) Received: from d12relay02.de.ibm.com (d12relay02.de.ibm.com [9.165.215.23]) by d12lmsgate.de.ibm.com (1.0.0) with ESMTP id FAA199412 for ; Thu, 31 May 2001 05:06:15 +0200 From: julian_satran@il.ibm.com Received: from d12mta02.de.ibm.com (d12mta01_cs0 [9.165.222.237]) by d12relay02.de.ibm.com (8.8.8m3/NCO v4.96) with SMTP id FAA133272 for ; Thu, 31 May 2001 05:05:33 +0200 Received: by d12mta02.de.ibm.com(Lotus SMTP MTA v4.6.5 (863.2 5-20-1999)) id C1256A5D.0010FC30 ; Thu, 31 May 2001 05:05:31 +0200 X-Lotus-FromDomain: IBMIL@IBMDE To: ips@ece.cmu.edu Message-ID: Date: Thu, 31 May 2001 06:08:12 +0300 Subject: Re: More on iSCSI boot Mime-Version: 1.0 Content-type: text/plain; charset=us-ascii Content-Disposition: inline X-UIDL: f4b8ca262df59261676610fd260e044a X-OriginalArrivalTime: 31 May 2001 05:55:53.0609 (UTC) FILETIME=[5A526790:01C0E996] Sender: owner-ips@ece.cmu.edu Precedence: bulk David, The trouble with our boot is two-fold: -the SCSI boot - unlike the network boot is unbounded (there is no such thing as an image). -even if we would like to standardize a "primary" or "minimal" boot we have no good understanding (experience) of how this will interact with the iSCSI security mechanisms. Julo Black_David@emc.com on 31-05-2001 05:49:57 Please respond to Black_David@emc.com To: ips@ece.cmu.edu cc: Subject: More on iSCSI boot Let's start with a simple boot from a disk - the system BIOS reads the boot sector(s) off of the disk drive, loads and runs it/them. That primary bootloader then handles whatever else is necessary based on the ability to do disk reads (a secondary bootloader and/or other things may be involved). On Intel systems, it's generally a combination of the system BIOS and card BIOS that make the disk reads work. Simplifying assumptions are common (e.g., boot from LUN 0 on the first SCSI target found). The goal of the iSCSI boot draft is to explain how to make this "simple boot from a disk" mechanism work when the boot disk is attached via iSCSI. An iSCSI adapter has some things to do in order to make this work, e.g., it has to log into the target before disk reads can be issued. I tend to believe that "Don't do this (i.e., try to boot over iSCSI)" is not an acceptable answer. In the message that started this whole boot discussion, I suggested that Using DHCP to find SLP to find the boot device seems both clumsy and an invitation to problems (one more thing that can break and prevent booting), I think that's doubly true if LDAP is used instead of SLP. David Robinson's messages support my inclination to reuse DHCP option 17 (Root Path) by defining iSCSI syntax for it. Between that, any TCP parameters that one wants to set through DHCP, and iSCSI parameters that can be defaulted, it should be possible to get the first disk reads done through iSCSI. As has been pointed out, booting is often poorly secured in general. While it'd be nice to change this, iSCSI will face the same pressures that other boot mechanisms face - keep it simple, get the boot image loaded, and let it do something fancier. Any signature/validation of the boot image will be above the level of iSCSI. Somehow, I don't expect to find implementations of ESP in card BIOSes anytime soon. If implementers want to use DHCP and TFTP to boot, I don't see any point in stopping them, but I don't think either should be mandated. DHCP has centralized administration advantages, and TFTP is a simple way to download code, but both are "one more thing that can break and prevent booting" and hence may not be used all the time. Comments? --David --------------------------------------------------- David L. Black, Senior Technologist EMC Corporation, 42 South St., Hopkinton, MA 01748 +1 (508) 435-1000 x75140 FAX: +1 (508) 497-8500 black_david@emc.com Mobile: +1 (978) 394-7754 --------------------------------------------------- From owner-ips@ece.cmu.edu Thu May 31 05:51:18 2001 Received: from ece.cmu.edu (ECE.CMU.EDU [128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id FAA19060 for ; Thu, 31 May 2001 05:51:18 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f4V81IM14196 for ips-outgoing; Thu, 31 May 2001 04:01:18 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from homa.asicdesigners.com (ip-64-63-65-105.reverse.mobilenetics.com [64.63.65.105] (may be forged)) by ece.cmu.edu (8.11.0/8.10.2) with SMTP id f4V81Ft14186 for ; Thu, 31 May 2001 04:01:15 -0400 (EDT) Received: from homa.asicdesigners.com ([10.1.1.5]) by homa.asicdesigners.com with Microsoft SMTPSVC(5.0.2195.2966); Thu, 31 May 2001 01:00:46 -0700 Received: by homa.asicdesigners.com (Microsoft Connector for POP3 Mailboxes 5.00.2195) with SMTP (Global POP3 Download) id MSG05312001-010039-63.MMD@asicdesigners.com; Thu, 31 May 2001 01:00:39 -0700 Received: from ece.cmu.edu (ECE.CMU.EDU [128.2.236.200]) by mail14.netservers.net (8.9.3/8.9.3) with ESMTP id AAA11915 for ; Thu, 31 May 2001 00:52:54 -0700 Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f4V5teX06863 for ips-outgoing; Thu, 31 May 2001 01:55:40 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from homa.asicdesigners.com (ip-64-63-65-105.reverse.mobilenetics.com [64.63.65.105] (may be forged)) by ece.cmu.edu (8.11.0/8.10.2) with SMTP id f4V5tZt06856 for ; Thu, 31 May 2001 01:55:35 -0400 (EDT) Received: from homa.asicdesigners.com ([10.1.1.5]) by homa.asicdesigners.com with Microsoft SMTPSVC(5.0.2195.2966); Wed, 30 May 2001 22:55:53 -0700 Received: by homa.asicdesigners.com (Microsoft Connector for POP3 Mailboxes 5.00.2195) with SMTP (Global POP3 Download) id MSG05302001-225537-2.MMD@asicdesigners.com; Wed, 30 May 2001 22:55:37 -0700 Received: from ece.cmu.edu (ECE.CMU.EDU [128.2.236.200]) by mail14.netservers.net (8.9.3/8.9.3) with ESMTP id WAA07741 for ; Wed, 30 May 2001 22:08:29 -0700 Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f4V2o8P25107 for ips-outgoing; Wed, 30 May 2001 22:50:08 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from maho3msx2.isus.emc.com (maho3msx2.isus.emc.com [128.221.11.32]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f4V2o5t25094 for ; Wed, 30 May 2001 22:50:05 -0400 (EDT) Received: by maho3msx2.isus.emc.com with Internet Mail Service (5.5.2653.19) id ; Wed, 30 May 2001 22:49:59 -0400 Message-ID: <0F31E5C394DAD311B60C00E029101A0708015630@corpmx9.isus.emc.com> From: Black_David@emc.com To: ips@ece.cmu.edu Subject: More on iSCSI boot Date: Wed, 30 May 2001 22:49:57 -0400 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2653.19) Content-Type: text/plain X-OriginalArrivalTime: 31 May 2001 05:55:53.0671 (UTC) FILETIME=[5A5BDD70:01C0E996] X-UIDL: 4973d51a27ae9898dfbcb7f2bb959dfa Sender: owner-ips@ece.cmu.edu Precedence: bulk Let's start with a simple boot from a disk - the system BIOS reads the boot sector(s) off of the disk drive, loads and runs it/them. That primary bootloader then handles whatever else is necessary based on the ability to do disk reads (a secondary bootloader and/or other things may be involved). On Intel systems, it's generally a combination of the system BIOS and card BIOS that make the disk reads work. Simplifying assumptions are common (e.g., boot from LUN 0 on the first SCSI target found). The goal of the iSCSI boot draft is to explain how to make this "simple boot from a disk" mechanism work when the boot disk is attached via iSCSI. An iSCSI adapter has some things to do in order to make this work, e.g., it has to log into the target before disk reads can be issued. I tend to believe that "Don't do this (i.e., try to boot over iSCSI)" is not an acceptable answer. In the message that started this whole boot discussion, I suggested that Using DHCP to find SLP to find the boot device seems both clumsy and an invitation to problems (one more thing that can break and prevent booting), I think that's doubly true if LDAP is used instead of SLP. David Robinson's messages support my inclination to reuse DHCP option 17 (Root Path) by defining iSCSI syntax for it. Between that, any TCP parameters that one wants to set through DHCP, and iSCSI parameters that can be defaulted, it should be possible to get the first disk reads done through iSCSI. As has been pointed out, booting is often poorly secured in general. While it'd be nice to change this, iSCSI will face the same pressures that other boot mechanisms face - keep it simple, get the boot image loaded, and let it do something fancier. Any signature/validation of the boot image will be above the level of iSCSI. Somehow, I don't expect to find implementations of ESP in card BIOSes anytime soon. If implementers want to use DHCP and TFTP to boot, I don't see any point in stopping them, but I don't think either should be mandated. DHCP has centralized administration advantages, and TFTP is a simple way to download code, but both are "one more thing that can break and prevent booting" and hence may not be used all the time. Comments? --David --------------------------------------------------- David L. Black, Senior Technologist EMC Corporation, 42 South St., Hopkinton, MA 01748 +1 (508) 435-1000 x75140 FAX: +1 (508) 497-8500 black_david@emc.com Mobile: +1 (978) 394-7754 --------------------------------------------------- From owner-ips@ece.cmu.edu Thu May 31 05:51:21 2001 Received: from ece.cmu.edu (ECE.CMU.EDU [128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id FAA19071 for ; Thu, 31 May 2001 05:51:20 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f4V81IX14197 for ips-outgoing; Thu, 31 May 2001 04:01:18 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from homa.asicdesigners.com (ip-64-63-65-105.reverse.mobilenetics.com [64.63.65.105] (may be forged)) by ece.cmu.edu (8.11.0/8.10.2) with SMTP id f4V81Ft14190 for ; Thu, 31 May 2001 04:01:15 -0400 (EDT) Received: from homa.asicdesigners.com ([10.1.1.5]) by homa.asicdesigners.com with Microsoft SMTPSVC(5.0.2195.2966); Thu, 31 May 2001 01:00:46 -0700 Received: by homa.asicdesigners.com (Microsoft Connector for POP3 Mailboxes 5.00.2195) with SMTP (Global POP3 Download) id MSG05312001-010039-65.MMD@asicdesigners.com; Thu, 31 May 2001 01:00:39 -0700 Received: from ece.cmu.edu (ECE.CMU.EDU [128.2.236.200]) by mail14.netservers.net (8.9.3/8.9.3) with ESMTP id AAA11991 for ; Thu, 31 May 2001 00:55:02 -0700 Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f4V5tbQ06860 for ips-outgoing; Thu, 31 May 2001 01:55:37 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from homa.asicdesigners.com (ip-64-63-65-105.reverse.mobilenetics.com [64.63.65.105] (may be forged)) by ece.cmu.edu (8.11.0/8.10.2) with SMTP id f4V5tZt06852 for ; Thu, 31 May 2001 01:55:35 -0400 (EDT) Received: from homa.asicdesigners.com ([10.1.1.5]) by homa.asicdesigners.com with Microsoft SMTPSVC(5.0.2195.2966); Wed, 30 May 2001 22:55:53 -0700 Received: by homa.asicdesigners.com (Microsoft Connector for POP3 Mailboxes 5.00.2195) with SMTP (Global POP3 Download) id MSG05302001-225536-1.MMD@asicdesigners.com; Wed, 30 May 2001 22:55:36 -0700 Received: from ece.cmu.edu (ECE.CMU.EDU [128.2.236.200]) by mail14.netservers.net (8.9.3/8.9.3) with ESMTP id WAA07694 for ; Wed, 30 May 2001 22:07:12 -0700 Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f4V36OQ26129 for ips-outgoing; Wed, 30 May 2001 23:06:24 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from d12lmsgate.de.ibm.com (d12lmsgate.de.ibm.com [195.212.91.199]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f4V36Mt26124 for ; Wed, 30 May 2001 23:06:22 -0400 (EDT) Received: from d12relay02.de.ibm.com (d12relay02.de.ibm.com [9.165.215.23]) by d12lmsgate.de.ibm.com (1.0.0) with ESMTP id FAA199412 for ; Thu, 31 May 2001 05:06:15 +0200 From: julian_satran@il.ibm.com Received: from d12mta02.de.ibm.com (d12mta01_cs0 [9.165.222.237]) by d12relay02.de.ibm.com (8.8.8m3/NCO v4.96) with SMTP id FAA133272 for ; Thu, 31 May 2001 05:05:33 +0200 Received: by d12mta02.de.ibm.com(Lotus SMTP MTA v4.6.5 (863.2 5-20-1999)) id C1256A5D.0010FC30 ; Thu, 31 May 2001 05:05:31 +0200 X-Lotus-FromDomain: IBMIL@IBMDE To: ips@ece.cmu.edu Message-ID: Date: Thu, 31 May 2001 06:08:12 +0300 Subject: Re: More on iSCSI boot Mime-Version: 1.0 Content-type: text/plain; charset=us-ascii Content-Disposition: inline X-OriginalArrivalTime: 31 May 2001 05:55:53.0609 (UTC) FILETIME=[5A526790:01C0E996] X-UIDL: f4b8ca262df59261676610fd260e044a Sender: owner-ips@ece.cmu.edu Precedence: bulk David, The trouble with our boot is two-fold: -the SCSI boot - unlike the network boot is unbounded (there is no such thing as an image). -even if we would like to standardize a "primary" or "minimal" boot we have no good understanding (experience) of how this will interact with the iSCSI security mechanisms. Julo Black_David@emc.com on 31-05-2001 05:49:57 Please respond to Black_David@emc.com To: ips@ece.cmu.edu cc: Subject: More on iSCSI boot Let's start with a simple boot from a disk - the system BIOS reads the boot sector(s) off of the disk drive, loads and runs it/them. That primary bootloader then handles whatever else is necessary based on the ability to do disk reads (a secondary bootloader and/or other things may be involved). On Intel systems, it's generally a combination of the system BIOS and card BIOS that make the disk reads work. Simplifying assumptions are common (e.g., boot from LUN 0 on the first SCSI target found). The goal of the iSCSI boot draft is to explain how to make this "simple boot from a disk" mechanism work when the boot disk is attached via iSCSI. An iSCSI adapter has some things to do in order to make this work, e.g., it has to log into the target before disk reads can be issued. I tend to believe that "Don't do this (i.e., try to boot over iSCSI)" is not an acceptable answer. In the message that started this whole boot discussion, I suggested that Using DHCP to find SLP to find the boot device seems both clumsy and an invitation to problems (one more thing that can break and prevent booting), I think that's doubly true if LDAP is used instead of SLP. David Robinson's messages support my inclination to reuse DHCP option 17 (Root Path) by defining iSCSI syntax for it. Between that, any TCP parameters that one wants to set through DHCP, and iSCSI parameters that can be defaulted, it should be possible to get the first disk reads done through iSCSI. As has been pointed out, booting is often poorly secured in general. While it'd be nice to change this, iSCSI will face the same pressures that other boot mechanisms face - keep it simple, get the boot image loaded, and let it do something fancier. Any signature/validation of the boot image will be above the level of iSCSI. Somehow, I don't expect to find implementations of ESP in card BIOSes anytime soon. If implementers want to use DHCP and TFTP to boot, I don't see any point in stopping them, but I don't think either should be mandated. DHCP has centralized administration advantages, and TFTP is a simple way to download code, but both are "one more thing that can break and prevent booting" and hence may not be used all the time. Comments? --David --------------------------------------------------- David L. Black, Senior Technologist EMC Corporation, 42 South St., Hopkinton, MA 01748 +1 (508) 435-1000 x75140 FAX: +1 (508) 497-8500 black_david@emc.com Mobile: +1 (978) 394-7754 --------------------------------------------------- From owner-ips@ece.cmu.edu Thu May 31 05:53:35 2001 Received: from ece.cmu.edu (ECE.CMU.EDU [128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id FAA19105 for ; Thu, 31 May 2001 05:53:34 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f4V81HF14195 for ips-outgoing; Thu, 31 May 2001 04:01:17 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from homa.asicdesigners.com (ip-64-63-65-105.reverse.mobilenetics.com [64.63.65.105] (may be forged)) by ece.cmu.edu (8.11.0/8.10.2) with SMTP id f4V81Et14182 for ; Thu, 31 May 2001 04:01:14 -0400 (EDT) Received: from homa.asicdesigners.com ([10.1.1.5]) by homa.asicdesigners.com with Microsoft SMTPSVC(5.0.2195.2966); Thu, 31 May 2001 01:00:46 -0700 Received: by homa.asicdesigners.com (Microsoft Connector for POP3 Mailboxes 5.00.2195) with SMTP (Global POP3 Download) id MSG05312001-010038-62.MMD@asicdesigners.com; Thu, 31 May 2001 01:00:38 -0700 Received: from ece.cmu.edu (ECE.CMU.EDU [128.2.236.200]) by mail14.netservers.net (8.9.3/8.9.3) with ESMTP id AAA11905 for ; Thu, 31 May 2001 00:52:35 -0700 Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f4V6V3N08892 for ips-outgoing; Thu, 31 May 2001 02:31:03 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from homa.asicdesigners.com (ip-64-63-65-105.reverse.mobilenetics.com [64.63.65.105] (may be forged)) by ece.cmu.edu (8.11.0/8.10.2) with SMTP id f4V6V1t08888 for ; Thu, 31 May 2001 02:31:02 -0400 (EDT) Received: from homa.asicdesigners.com ([10.1.1.5]) by homa.asicdesigners.com with Microsoft SMTPSVC(5.0.2195.2966); Wed, 30 May 2001 23:30:46 -0700 Received: by homa.asicdesigners.com (Microsoft Connector for POP3 Mailboxes 5.00.2195) with SMTP (Global POP3 Download) id MSG05302001-233039-11.MMD@asicdesigners.com; Wed, 30 May 2001 23:30:39 -0700 Received: from ece.cmu.edu (ECE.CMU.EDU [128.2.236.200]) by mail14.netservers.net (8.9.3/8.9.3) with ESMTP id XAA09744 for ; Wed, 30 May 2001 23:24:17 -0700 Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f4V3wn129517 for ips-outgoing; Wed, 30 May 2001 23:58:49 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from patan.sun.com (patan.Sun.COM [192.18.98.43]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f4V3wlt29512 for ; Wed, 30 May 2001 23:58:47 -0400 (EDT) Received: from phys-ha6nwka.ebay.sun.com ([129.149.1.82]) by patan.sun.com (8.9.3+Sun/8.9.3) with ESMTP id VAA15338 for ; Wed, 30 May 2001 21:58:45 -0600 (MDT) Received: from ebay.sun.com (dsl197-15.Eng.Sun.COM [129.146.197.15]) by phys-ha6nwka.ebay.sun.com (8.9.3+Sun/8.9.3/ENSMAIL,v2.1p1) with ESMTP id UAA24951 for ; Wed, 30 May 2001 20:58:46 -0700 (PDT) Message-ID: <3B15C1C0.82895D8F@ebay.sun.com> Date: Wed, 30 May 2001 21:00:00 -0700 From: David Robinson Organization: Sun Microsystems, Inc. X-Mailer: Mozilla 4.77 [en] (Win98; U) X-Accept-Language: en MIME-Version: 1.0 To: ips@ece.cmu.edu Subject: Re: More on iSCSI boot References: Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-OriginalArrivalTime: 31 May 2001 06:30:46.0750 (UTC) FILETIME=[39EE5FE0:01C0E99B] X-UIDL: b4182fa012ef75b0bc9a62b969be91f4 Sender: owner-ips@ece.cmu.edu Precedence: bulk Content-Transfer-Encoding: 7bit Julian, I think you are simply confused on terminology. I think when David says "boot image" he is using in generally to mean transfering enough data to provide a self supporting operating system. In the case of SCSI the "image" is set of sectors necessary to start the kernel, similarily in NFS it is typically the "vmunix" file(s). It is not the full LUN representing the root filesystem. As Bernard has shown, a flexible and scaleable mechanism to bootstrap a security system without a disk is just very hard. We need to do a reality check on the environment that iSCSI boot will use. I don't think there will be a large market for booting over the Internet, but boot disks will be physically co-located in constrained environments where DHCP will be "good enough". While the IESG may not like the weak security, I would be surprised if it insisted that the ips WG take on the task of building a better DHCP. -David julian_satran@il.ibm.com wrote: > > David, > > The trouble with our boot is two-fold: > > -the SCSI boot - unlike the network boot is unbounded (there is no such > thing as an image). > -even if we would like to standardize a "primary" or "minimal" boot we have > no good understanding (experience) > of how this will interact with the iSCSI security mechanisms. > > Julo > > Black_David@emc.com on 31-05-2001 05:49:57 > > Please respond to Black_David@emc.com > > To: ips@ece.cmu.edu > cc: > Subject: More on iSCSI boot > > Let's start with a simple boot from a disk - the system > BIOS reads the boot sector(s) off of the disk drive, > loads and runs it/them. That primary bootloader then > handles whatever else is necessary based on the ability > to do disk reads (a secondary bootloader and/or other > things may be involved). On Intel systems, it's generally > a combination of the system BIOS and card BIOS that > make the disk reads work. Simplifying assumptions > are common (e.g., boot from LUN 0 on the first SCSI > target found). The goal of the iSCSI boot draft is > to explain how to make this "simple boot from a disk" > mechanism work when the boot disk is attached via iSCSI. > > An iSCSI adapter has some things to do in order to make > this work, e.g., it has to log into the target before disk > reads can be issued. I tend to believe that "Don't do this > (i.e., try to boot over iSCSI)" is not an acceptable answer. > > In the message that started this whole boot discussion, I > suggested that > > Using DHCP to find SLP to find the boot device seems > both clumsy and an invitation to problems (one more > thing that can break and prevent booting), > > I think that's doubly true if LDAP is used instead of SLP. > David Robinson's messages support my inclination to reuse > DHCP option 17 (Root Path) by defining iSCSI syntax > for it. Between that, any TCP parameters that one wants to > set through DHCP, and iSCSI parameters that can be defaulted, > it should be possible to get the first disk reads done > through iSCSI. > > As has been pointed out, booting is often poorly secured in > general. While it'd be nice to change this, iSCSI will face > the same pressures that other boot mechanisms face - keep > it simple, get the boot image loaded, and let it do something > fancier. Any signature/validation of the boot image will be > above the level of iSCSI. Somehow, I don't expect to find > implementations of ESP in card BIOSes anytime soon. > > If implementers want to use DHCP and TFTP to boot, I don't > see any point in stopping them, but I don't think either should > be mandated. DHCP has centralized administration advantages, > and TFTP is a simple way to download code, but both are "one > more thing that can break and prevent booting" and hence may > not be used all the time. > > Comments? > --David > > --------------------------------------------------- > David L. Black, Senior Technologist > EMC Corporation, 42 South St., Hopkinton, MA 01748 > +1 (508) 435-1000 x75140 FAX: +1 (508) 497-8500 > black_david@emc.com Mobile: +1 (978) 394-7754 > --------------------------------------------------- From owner-ips@ece.cmu.edu Thu May 31 08:27:54 2001 Received: from ece.cmu.edu (ECE.CMU.EDU [128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id IAA22002 for ; Thu, 31 May 2001 08:27:54 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f4VAOh503125 for ips-outgoing; Thu, 31 May 2001 06:24:43 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from d12lmsgate-2.de.ibm.com (d12lmsgate-2.de.ibm.com [195.212.91.200]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f4VAOct03119 for ; Thu, 31 May 2001 06:24:38 -0400 (EDT) Received: from d12relay01.de.ibm.com (d12relay01.de.ibm.com [9.165.215.22]) by d12lmsgate-2.de.ibm.com (1.0.0) with ESMTP id MAA11000 for ; Thu, 31 May 2001 12:24:30 +0200 From: julian_satran@il.ibm.com Received: from d12mta02.de.ibm.com (d12mta01_cs0 [9.165.222.237]) by d12relay01.de.ibm.com (8.8.8m3/NCO v4.96) with SMTP id MAA98240 for ; Thu, 31 May 2001 12:24:30 +0200 Received: by d12mta02.de.ibm.com(Lotus SMTP MTA v4.6.5 (863.2 5-20-1999)) id C1256A5D.00391B72 ; Thu, 31 May 2001 12:23:45 +0200 X-Lotus-FromDomain: IBMIL@IBMDE To: ips@ece.cmu.edu Message-ID: Date: Thu, 31 May 2001 13:17:29 +0300 Subject: Re: More on iSCSI boot Mime-Version: 1.0 Content-type: text/plain; charset=us-ascii Content-Disposition: inline Sender: owner-ips@ece.cmu.edu Precedence: bulk David, I don't think I am confused by terminology (although I might be!). In PXE as in all network boots the boot image is a file and you can do with it all sorts of authentication (that is what I call a bounded process). In a SCSI boot the target has no way of distinguishing a read for boot from a read for a fully operational initiator (the boot is unbounded). Authenticating this type of "unbounded image" is hard. You can imagine various schemes but all of them are hard and many of them require heavy administration and cooperating third parties (DHCP and others). An authentic DHCP is but a small item in the foolproofing boot. Considering what an "infected" boot can do to an organization standardizing a 2 phase process might be the way to go (very little external support is required) but we lack the required experience to attempt even this now. We some patience we might get a secure boot even over an unsecured DHCP (as we get secure communication over unsecured channels). Regards, Julo David Robinson on 31-05-2001 07:00:00 Please respond to David Robinson To: ips@ece.cmu.edu cc: Subject: Re: More on iSCSI boot Julian, I think you are simply confused on terminology. I think when David says "boot image" he is using in generally to mean transfering enough data to provide a self supporting operating system. In the case of SCSI the "image" is set of sectors necessary to start the kernel, similarily in NFS it is typically the "vmunix" file(s). It is not the full LUN representing the root filesystem. As Bernard has shown, a flexible and scaleable mechanism to bootstrap a security system without a disk is just very hard. We need to do a reality check on the environment that iSCSI boot will use. I don't think there will be a large market for booting over the Internet, but boot disks will be physically co-located in constrained environments where DHCP will be "good enough". While the IESG may not like the weak security, I would be surprised if it insisted that the ips WG take on the task of building a better DHCP. -David julian_satran@il.ibm.com wrote: > > David, > > The trouble with our boot is two-fold: > > -the SCSI boot - unlike the network boot is unbounded (there is no such > thing as an image). > -even if we would like to standardize a "primary" or "minimal" boot we have > no good understanding (experience) > of how this will interact with the iSCSI security mechanisms. > > Julo > > Black_David@emc.com on 31-05-2001 05:49:57 > > Please respond to Black_David@emc.com > > To: ips@ece.cmu.edu > cc: > Subject: More on iSCSI boot > > Let's start with a simple boot from a disk - the system > BIOS reads the boot sector(s) off of the disk drive, > loads and runs it/them. That primary bootloader then > handles whatever else is necessary based on the ability > to do disk reads (a secondary bootloader and/or other > things may be involved). On Intel systems, it's generally > a combination of the system BIOS and card BIOS that > make the disk reads work. Simplifying assumptions > are common (e.g., boot from LUN 0 on the first SCSI > target found). The goal of the iSCSI boot draft is > to explain how to make this "simple boot from a disk" > mechanism work when the boot disk is attached via iSCSI. > > An iSCSI adapter has some things to do in order to make > this work, e.g., it has to log into the target before disk > reads can be issued. I tend to believe that "Don't do this > (i.e., try to boot over iSCSI)" is not an acceptable answer. > > In the message that started this whole boot discussion, I > suggested that > > Using DHCP to find SLP to find the boot device seems > both clumsy and an invitation to problems (one more > thing that can break and prevent booting), > > I think that's doubly true if LDAP is used instead of SLP. > David Robinson's messages support my inclination to reuse > DHCP option 17 (Root Path) by defining iSCSI syntax > for it. Between that, any TCP parameters that one wants to > set through DHCP, and iSCSI parameters that can be defaulted, > it should be possible to get the first disk reads done > through iSCSI. > > As has been pointed out, booting is often poorly secured in > general. While it'd be nice to change this, iSCSI will face > the same pressures that other boot mechanisms face - keep > it simple, get the boot image loaded, and let it do something > fancier. Any signature/validation of the boot image will be > above the level of iSCSI. Somehow, I don't expect to find > implementations of ESP in card BIOSes anytime soon. > > If implementers want to use DHCP and TFTP to boot, I don't > see any point in stopping them, but I don't think either should > be mandated. DHCP has centralized administration advantages, > and TFTP is a simple way to download code, but both are "one > more thing that can break and prevent booting" and hence may > not be used all the time. > > Comments? > --David > > --------------------------------------------------- > David L. Black, Senior Technologist > EMC Corporation, 42 South St., Hopkinton, MA 01748 > +1 (508) 435-1000 x75140 FAX: +1 (508) 497-8500 > black_david@emc.com Mobile: +1 (978) 394-7754 > --------------------------------------------------- From owner-ips@ece.cmu.edu Thu May 31 11:14:54 2001 Received: from ece.cmu.edu (ECE.CMU.EDU [128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id LAA28194 for ; Thu, 31 May 2001 11:14:54 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f4VCjVb11807 for ips-outgoing; Thu, 31 May 2001 08:45:31 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from homa.asicdesigners.com (ip-64-63-65-105.reverse.mobilenetics.com [64.63.65.105] (may be forged)) by ece.cmu.edu (8.11.0/8.10.2) with SMTP id f4VCjTt11803 for ; Thu, 31 May 2001 08:45:30 -0400 (EDT) Received: from homa.asicdesigners.com ([10.1.1.5]) by homa.asicdesigners.com with Microsoft SMTPSVC(5.0.2195.2966); Thu, 31 May 2001 05:45:48 -0700 Received: by homa.asicdesigners.com (Microsoft Connector for POP3 Mailboxes 5.00.2195) with SMTP (Global POP3 Download) id MSG05312001-054538-184.MMD@asicdesigners.com; Thu, 31 May 2001 05:45:38 -0700 Received: from ece.cmu.edu (ECE.CMU.EDU [128.2.236.200]) by mail14.netservers.net (8.9.3/8.9.3) with ESMTP id FAA18978 for ; Thu, 31 May 2001 05:43:50 -0700 Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f4VAOh503125 for ips-outgoing; Thu, 31 May 2001 06:24:43 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from d12lmsgate-2.de.ibm.com (d12lmsgate-2.de.ibm.com [195.212.91.200]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f4VAOct03119 for ; Thu, 31 May 2001 06:24:38 -0400 (EDT) Received: from d12relay01.de.ibm.com (d12relay01.de.ibm.com [9.165.215.22]) by d12lmsgate-2.de.ibm.com (1.0.0) with ESMTP id MAA11000 for ; Thu, 31 May 2001 12:24:30 +0200 From: julian_satran@il.ibm.com Received: from d12mta02.de.ibm.com (d12mta01_cs0 [9.165.222.237]) by d12relay01.de.ibm.com (8.8.8m3/NCO v4.96) with SMTP id MAA98240 for ; Thu, 31 May 2001 12:24:30 +0200 Received: by d12mta02.de.ibm.com(Lotus SMTP MTA v4.6.5 (863.2 5-20-1999)) id C1256A5D.00391B72 ; Thu, 31 May 2001 12:23:45 +0200 X-Lotus-FromDomain: IBMIL@IBMDE To: ips@ece.cmu.edu Message-ID: Date: Thu, 31 May 2001 13:17:29 +0300 Subject: Re: More on iSCSI boot Mime-Version: 1.0 Content-type: text/plain; charset=us-ascii Content-Disposition: inline X-UIDL: 30b4676697a43b1f7e7f0e8fc4d160af X-OriginalArrivalTime: 31 May 2001 12:45:48.0546 (UTC) FILETIME=[9E0BF620:01C0E9CF] Sender: owner-ips@ece.cmu.edu Precedence: bulk David, I don't think I am confused by terminology (although I might be!). In PXE as in all network boots the boot image is a file and you can do with it all sorts of authentication (that is what I call a bounded process). In a SCSI boot the target has no way of distinguishing a read for boot from a read for a fully operational initiator (the boot is unbounded). Authenticating this type of "unbounded image" is hard. You can imagine various schemes but all of them are hard and many of them require heavy administration and cooperating third parties (DHCP and others). An authentic DHCP is but a small item in the foolproofing boot. Considering what an "infected" boot can do to an organization standardizing a 2 phase process might be the way to go (very little external support is required) but we lack the required experience to attempt even this now. We some patience we might get a secure boot even over an unsecured DHCP (as we get secure communication over unsecured channels). Regards, Julo David Robinson on 31-05-2001 07:00:00 Please respond to David Robinson To: ips@ece.cmu.edu cc: Subject: Re: More on iSCSI boot Julian, I think you are simply confused on terminology. I think when David says "boot image" he is using in generally to mean transfering enough data to provide a self supporting operating system. In the case of SCSI the "image" is set of sectors necessary to start the kernel, similarily in NFS it is typically the "vmunix" file(s). It is not the full LUN representing the root filesystem. As Bernard has shown, a flexible and scaleable mechanism to bootstrap a security system without a disk is just very hard. We need to do a reality check on the environment that iSCSI boot will use. I don't think there will be a large market for booting over the Internet, but boot disks will be physically co-located in constrained environments where DHCP will be "good enough". While the IESG may not like the weak security, I would be surprised if it insisted that the ips WG take on the task of building a better DHCP. -David julian_satran@il.ibm.com wrote: > > David, > > The trouble with our boot is two-fold: > > -the SCSI boot - unlike the network boot is unbounded (there is no such > thing as an image). > -even if we would like to standardize a "primary" or "minimal" boot we have > no good understanding (experience) > of how this will interact with the iSCSI security mechanisms. > > Julo > > Black_David@emc.com on 31-05-2001 05:49:57 > > Please respond to Black_David@emc.com > > To: ips@ece.cmu.edu > cc: > Subject: More on iSCSI boot > > Let's start with a simple boot from a disk - the system > BIOS reads the boot sector(s) off of the disk drive, > loads and runs it/them. That primary bootloader then > handles whatever else is necessary based on the ability > to do disk reads (a secondary bootloader and/or other > things may be involved). On Intel systems, it's generally > a combination of the system BIOS and card BIOS that > make the disk reads work. Simplifying assumptions > are common (e.g., boot from LUN 0 on the first SCSI > target found). The goal of the iSCSI boot draft is > to explain how to make this "simple boot from a disk" > mechanism work when the boot disk is attached via iSCSI. > > An iSCSI adapter has some things to do in order to make > this work, e.g., it has to log into the target before disk > reads can be issued. I tend to believe that "Don't do this > (i.e., try to boot over iSCSI)" is not an acceptable answer. > > In the message that started this whole boot discussion, I > suggested that > > Using DHCP to find SLP to find the boot device seems > both clumsy and an invitation to problems (one more > thing that can break and prevent booting), > > I think that's doubly true if LDAP is used instead of SLP. > David Robinson's messages support my inclination to reuse > DHCP option 17 (Root Path) by defining iSCSI syntax > for it. Between that, any TCP parameters that one wants to > set through DHCP, and iSCSI parameters that can be defaulted, > it should be possible to get the first disk reads done > through iSCSI. > > As has been pointed out, booting is often poorly secured in > general. While it'd be nice to change this, iSCSI will face > the same pressures that other boot mechanisms face - keep > it simple, get the boot image loaded, and let it do something > fancier. Any signature/validation of the boot image will be > above the level of iSCSI. Somehow, I don't expect to find > implementations of ESP in card BIOSes anytime soon. > > If implementers want to use DHCP and TFTP to boot, I don't > see any point in stopping them, but I don't think either should > be mandated. DHCP has centralized administration advantages, > and TFTP is a simple way to download code, but both are "one > more thing that can break and prevent booting" and hence may > not be used all the time. > > Comments? > --David > > --------------------------------------------------- > David L. Black, Senior Technologist > EMC Corporation, 42 South St., Hopkinton, MA 01748 > +1 (508) 435-1000 x75140 FAX: +1 (508) 497-8500 > black_david@emc.com Mobile: +1 (978) 394-7754 > --------------------------------------------------- From owner-ips@ece.cmu.edu Thu May 31 11:52:35 2001 Received: from ece.cmu.edu (ECE.CMU.EDU [128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id LAA29525 for ; Thu, 31 May 2001 11:52:35 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f4VDpiQ16621 for ips-outgoing; Thu, 31 May 2001 09:51:44 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from mxbh4.isus.emc.com (mxbh4.isus.emc.com [128.221.10.33]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f4VDpht16617 for ; Thu, 31 May 2001 09:51:43 -0400 (EDT) Received: by mxbh4.isus.emc.com with Internet Mail Service (5.5.2653.19) id ; Thu, 31 May 2001 09:51:37 -0400 Message-ID: <0F31E5C394DAD311B60C00E029101A0708015636@corpmx9.isus.emc.com> From: Black_David@emc.com To: ENDL_TX@computer.org, ips@ece.cmu.edu Subject: RE: Nashua Interim Minutes Date: Thu, 31 May 2001 09:51:30 -0400 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2653.19) Content-Type: text/plain Sender: owner-ips@ece.cmu.edu Precedence: bulk There are two sets of slides. In the interim until the IETF proceedings pick them up, I've sent mine to Julian to put on his web site along with Ofer Biran's slides so that both sets of security slides are in one place. Thanks, --David > -----Original Message----- > From: Ralph Weber [SMTP:ralphoweber@compuserve.com] > Sent: Monday, May 28, 2001 10:14 AM > To: IPS Reflector > Subject: Re: Nashua Interim Minutes > > I am unclear about this part of the minutes: > > } Security > } -------- > } > } -- David Black (EMC, co-chair) presentation on security requirements > } > } See slides - this was an informational presentation/discussion about > } requirements (MUST implement authentication and cryptographic > } integrity, MUST have a required mechanism for each, > } confidentiality is OPTIONAL, IP vs. iSCSI level mechanisms > } [iSCSI can multiplex initiators and targets on a single } address, TCP port> pair). > } > } -- Ofer Biran (IBM, Haifa) > } > } Lots of slides - see slides for details. > > Were there two sets of slides or just the one in Julian's web > archive? > > Thanks. > > Ralph... > From owner-ips@ece.cmu.edu Thu May 31 12:41:15 2001 Received: from ece.cmu.edu (ECE.CMU.EDU [128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id MAA01427 for ; Thu, 31 May 2001 12:41:15 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f4VEdsu20271 for ips-outgoing; Thu, 31 May 2001 10:39:54 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from internaut.com ([64.38.134.99]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f4VEdqt20266 for ; Thu, 31 May 2001 10:39:52 -0400 (EDT) Received: from localhost (aboba@localhost) by internaut.com (8.9.3/8.9.3) with ESMTP id HAA52319; Thu, 31 May 2001 07:29:34 -0700 (PDT) (envelope-from aboba@internaut.com) Date: Thu, 31 May 2001 07:29:34 -0700 (PDT) From: Bernard Aboba To: Douglas Otis cc: julian_satran@il.ibm.com, ips@ece.cmu.edu Subject: RE: iSCSI and secure boot In-Reply-To: Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-ips@ece.cmu.edu Precedence: bulk > Can you describe a secure diskless boot that does not require individual > setup of each? Some touching of each PC is required. However, the question is whether: a. The setup requires setup of multiple credentials, or just one. Note that BIS requires each PC to be configured with certificate of the boot image signing authority. However, this does not provide client authentication capabilities - so if you want to authenticate iSCSI or DHCP, or anything else, then you'd need additional credentials. b. Whether *per-interface* credentials are needed (e.g. authenticated DHCP draft -16), or *per-machine*. Per-interface credentials require the machine to be touched every time an interface is added or removed, not just when the machine is shipped by the OEM. From owner-ips@ece.cmu.edu Thu May 31 12:43:11 2001 Received: from ece.cmu.edu (ECE.CMU.EDU [128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id MAA01490 for ; Thu, 31 May 2001 12:43:11 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f4VEwjG21901 for ips-outgoing; Thu, 31 May 2001 10:58:45 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from internaut.com ([64.38.134.99]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f4VEwgt21896 for ; Thu, 31 May 2001 10:58:43 -0400 (EDT) Received: from localhost (aboba@localhost) by internaut.com (8.9.3/8.9.3) with ESMTP id HAA52339; Thu, 31 May 2001 07:48:40 -0700 (PDT) (envelope-from aboba@internaut.com) Date: Thu, 31 May 2001 07:48:40 -0700 (PDT) From: Bernard Aboba To: David Robinson cc: ips@ece.cmu.edu Subject: Re: More on iSCSI boot In-Reply-To: <3B15C1C0.82895D8F@ebay.sun.com> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-ips@ece.cmu.edu Precedence: bulk > I would be surprised if it insisted that > the ips WG take on the task of building a better DHCP. > Agree. Boot security in general is not the problem of the IPS WG. However, I do think that the WG should think through the implications of iSCSI security on the potential for a secure boot process. For example, are we assuming that it would be impossible to implement iSCSI security during boot? Also, I'd note that between BIS and authenticated DHCP we already are pushing the limits of credential storage and administrative complexity. It's not entirely clear to me that this passes the "laugh test" - particularly when the need to secure other components (NFS, iSCSI, etc.) is added to the mix. If that's true then even a "better DHCP" (e.g. authenticated DHCP) may not be the right answer. From owner-ips@ece.cmu.edu Thu May 31 12:43:34 2001 Received: from ece.cmu.edu (ECE.CMU.EDU [128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id MAA01501 for ; Thu, 31 May 2001 12:43:34 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f4VEmlB20996 for ips-outgoing; Thu, 31 May 2001 10:48:47 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from internaut.com ([64.38.134.99]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f4VEmjt20991 for ; Thu, 31 May 2001 10:48:45 -0400 (EDT) Received: from localhost (aboba@localhost) by internaut.com (8.9.3/8.9.3) with ESMTP id HAA52329; Thu, 31 May 2001 07:38:33 -0700 (PDT) (envelope-from aboba@internaut.com) Date: Thu, 31 May 2001 07:38:32 -0700 (PDT) From: Bernard Aboba To: julian_satran@il.ibm.com cc: ips@ece.cmu.edu Subject: Re: More on iSCSI boot In-Reply-To: Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-ips@ece.cmu.edu Precedence: bulk > -even if we would like to standardize a "primary" or "minimal" boot we have > no good understanding (experience) > of how this will interact with the iSCSI security mechanisms. > Agree with this. But I think the lesson is (as David mentioned) that you need a (perhaps non-mandatory) security method that can be executed in the boot rom. Personally, I'm not that convinced that boot image signing as presently conceived is all that useful anyway, since it isn't widely deployed and there's no way to revoke the signature of a bad boot image (e.g. no certificate chain support). From owner-ips@ece.cmu.edu Thu May 31 13:41:23 2001 Received: from ece.cmu.edu (ECE.CMU.EDU [128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id NAA03393 for ; Thu, 31 May 2001 13:41:23 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f4VFUQL24758 for ips-outgoing; Thu, 31 May 2001 11:30:26 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from homa.asicdesigners.com (ip-64-63-65-105.reverse.mobilenetics.com [64.63.65.105] (may be forged)) by ece.cmu.edu (8.11.0/8.10.2) with SMTP id f4VFUNt24752 for ; Thu, 31 May 2001 11:30:23 -0400 (EDT) Received: from homa.asicdesigners.com ([10.1.1.5]) by homa.asicdesigners.com with Microsoft SMTPSVC(5.0.2195.2966); Thu, 31 May 2001 08:30:42 -0700 Received: by homa.asicdesigners.com (Microsoft Connector for POP3 Mailboxes 5.00.2195) with SMTP (Global POP3 Download) id MSG05312001-083038-251.MMD@asicdesigners.com; Thu, 31 May 2001 08:30:38 -0700 Received: from ece.cmu.edu (ECE.CMU.EDU [128.2.236.200]) by mail14.netservers.net (8.9.3/8.9.3) with ESMTP id IAA27044 for ; Thu, 31 May 2001 08:27:40 -0700 Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f4VCjVb11807 for ips-outgoing; Thu, 31 May 2001 08:45:31 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from homa.asicdesigners.com (ip-64-63-65-105.reverse.mobilenetics.com [64.63.65.105] (may be forged)) by ece.cmu.edu (8.11.0/8.10.2) with SMTP id f4VCjTt11803 for ; Thu, 31 May 2001 08:45:30 -0400 (EDT) Received: from homa.asicdesigners.com ([10.1.1.5]) by homa.asicdesigners.com with Microsoft SMTPSVC(5.0.2195.2966); Thu, 31 May 2001 05:45:48 -0700 Received: by homa.asicdesigners.com (Microsoft Connector for POP3 Mailboxes 5.00.2195) with SMTP (Global POP3 Download) id MSG05312001-054538-184.MMD@asicdesigners.com; Thu, 31 May 2001 05:45:38 -0700 Received: from ece.cmu.edu (ECE.CMU.EDU [128.2.236.200]) by mail14.netservers.net (8.9.3/8.9.3) with ESMTP id FAA18978 for ; Thu, 31 May 2001 05:43:50 -0700 Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f4VAOh503125 for ips-outgoing; Thu, 31 May 2001 06:24:43 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from d12lmsgate-2.de.ibm.com (d12lmsgate-2.de.ibm.com [195.212.91.200]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f4VAOct03119 for ; Thu, 31 May 2001 06:24:38 -0400 (EDT) Received: from d12relay01.de.ibm.com (d12relay01.de.ibm.com [9.165.215.22]) by d12lmsgate-2.de.ibm.com (1.0.0) with ESMTP id MAA11000 for ; Thu, 31 May 2001 12:24:30 +0200 From: julian_satran@il.ibm.com Received: from d12mta02.de.ibm.com (d12mta01_cs0 [9.165.222.237]) by d12relay01.de.ibm.com (8.8.8m3/NCO v4.96) with SMTP id MAA98240 for ; Thu, 31 May 2001 12:24:30 +0200 Received: by d12mta02.de.ibm.com(Lotus SMTP MTA v4.6.5 (863.2 5-20-1999)) id C1256A5D.00391B72 ; Thu, 31 May 2001 12:23:45 +0200 X-Lotus-FromDomain: IBMIL@IBMDE To: ips@ece.cmu.edu Message-ID: Date: Thu, 31 May 2001 13:17:29 +0300 Subject: Re: More on iSCSI boot Mime-Version: 1.0 Content-type: text/plain; charset=us-ascii Content-Disposition: inline X-OriginalArrivalTime: 31 May 2001 12:45:48.0546 (UTC) FILETIME=[9E0BF620:01C0E9CF] X-UIDL: 30b4676697a43b1f7e7f0e8fc4d160af Sender: owner-ips@ece.cmu.edu Precedence: bulk David, I don't think I am confused by terminology (although I might be!). In PXE as in all network boots the boot image is a file and you can do with it all sorts of authentication (that is what I call a bounded process). In a SCSI boot the target has no way of distinguishing a read for boot from a read for a fully operational initiator (the boot is unbounded). Authenticating this type of "unbounded image" is hard. You can imagine various schemes but all of them are hard and many of them require heavy administration and cooperating third parties (DHCP and others). An authentic DHCP is but a small item in the foolproofing boot. Considering what an "infected" boot can do to an organization standardizing a 2 phase process might be the way to go (very little external support is required) but we lack the required experience to attempt even this now. We some patience we might get a secure boot even over an unsecured DHCP (as we get secure communication over unsecured channels). Regards, Julo David Robinson on 31-05-2001 07:00:00 Please respond to David Robinson To: ips@ece.cmu.edu cc: Subject: Re: More on iSCSI boot Julian, I think you are simply confused on terminology. I think when David says "boot image" he is using in generally to mean transfering enough data to provide a self supporting operating system. In the case of SCSI the "image" is set of sectors necessary to start the kernel, similarily in NFS it is typically the "vmunix" file(s). It is not the full LUN representing the root filesystem. As Bernard has shown, a flexible and scaleable mechanism to bootstrap a security system without a disk is just very hard. We need to do a reality check on the environment that iSCSI boot will use. I don't think there will be a large market for booting over the Internet, but boot disks will be physically co-located in constrained environments where DHCP will be "good enough". While the IESG may not like the weak security, I would be surprised if it insisted that the ips WG take on the task of building a better DHCP. -David julian_satran@il.ibm.com wrote: > > David, > > The trouble with our boot is two-fold: > > -the SCSI boot - unlike the network boot is unbounded (there is no such > thing as an image). > -even if we would like to standardize a "primary" or "minimal" boot we have > no good understanding (experience) > of how this will interact with the iSCSI security mechanisms. > > Julo > > Black_David@emc.com on 31-05-2001 05:49:57 > > Please respond to Black_David@emc.com > > To: ips@ece.cmu.edu > cc: > Subject: More on iSCSI boot > > Let's start with a simple boot from a disk - the system > BIOS reads the boot sector(s) off of the disk drive, > loads and runs it/them. That primary bootloader then > handles whatever else is necessary based on the ability > to do disk reads (a secondary bootloader and/or other > things may be involved). On Intel systems, it's generally > a combination of the system BIOS and card BIOS that > make the disk reads work. Simplifying assumptions > are common (e.g., boot from LUN 0 on the first SCSI > target found). The goal of the iSCSI boot draft is > to explain how to make this "simple boot from a disk" > mechanism work when the boot disk is attached via iSCSI. > > An iSCSI adapter has some things to do in order to make > this work, e.g., it has to log into the target before disk > reads can be issued. I tend to believe that "Don't do this > (i.e., try to boot over iSCSI)" is not an acceptable answer. > > In the message that started this whole boot discussion, I > suggested that > > Using DHCP to find SLP to find the boot device seems > both clumsy and an invitation to problems (one more > thing that can break and prevent booting), > > I think that's doubly true if LDAP is used instead of SLP. > David Robinson's messages support my inclination to reuse > DHCP option 17 (Root Path) by defining iSCSI syntax > for it. Between that, any TCP parameters that one wants to > set through DHCP, and iSCSI parameters that can be defaulted, > it should be possible to get the first disk reads done > through iSCSI. > > As has been pointed out, booting is often poorly secured in > general. While it'd be nice to change this, iSCSI will face > the same pressures that other boot mechanisms face - keep > it simple, get the boot image loaded, and let it do something > fancier. Any signature/validation of the boot image will be > above the level of iSCSI. Somehow, I don't expect to find > implementations of ESP in card BIOSes anytime soon. > > If implementers want to use DHCP and TFTP to boot, I don't > see any point in stopping them, but I don't think either should > be mandated. DHCP has centralized administration advantages, > and TFTP is a simple way to download code, but both are "one > more thing that can break and prevent booting" and hence may > not be used all the time. > > Comments? > --David > > --------------------------------------------------- > David L. Black, Senior Technologist > EMC Corporation, 42 South St., Hopkinton, MA 01748 > +1 (508) 435-1000 x75140 FAX: +1 (508) 497-8500 > black_david@emc.com Mobile: +1 (978) 394-7754 > --------------------------------------------------- From owner-ips@ece.cmu.edu Thu May 31 14:30:29 2001 Received: from ece.cmu.edu (ECE.CMU.EDU [128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id OAA05299 for ; Thu, 31 May 2001 14:30:29 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f4VH22u02555 for ips-outgoing; Thu, 31 May 2001 13:02:02 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from homa.asicdesigners.com (ip-64-63-65-105.reverse.mobilenetics.com [64.63.65.105] (may be forged)) by ece.cmu.edu (8.11.0/8.10.2) with SMTP id f4VH1vt02537 for ; Thu, 31 May 2001 13:01:57 -0400 (EDT) Received: from homa.asicdesigners.com ([10.1.1.5]) by homa.asicdesigners.com with Microsoft SMTPSVC(5.0.2195.2966); Thu, 31 May 2001 10:00:44 -0700 Received: by homa.asicdesigners.com (Microsoft Connector for POP3 Mailboxes 5.00.2195) with SMTP (Global POP3 Download) id MSG05312001-100039-309.MMD@asicdesigners.com; Thu, 31 May 2001 10:00:39 -0700 Received: from ece.cmu.edu (ECE.CMU.EDU [128.2.236.200]) by mail13.netservers.net (8.9.3/8.9.3) with ESMTP id JAA17087 for ; Thu, 31 May 2001 09:53:55 -0700 Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f4VEmlB20996 for ips-outgoing; Thu, 31 May 2001 10:48:47 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from internaut.com ([64.38.134.99]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f4VEmjt20991 for ; Thu, 31 May 2001 10:48:45 -0400 (EDT) Received: from localhost (aboba@localhost) by internaut.com (8.9.3/8.9.3) with ESMTP id HAA52329; Thu, 31 May 2001 07:38:33 -0700 (PDT) (envelope-from aboba@internaut.com) Date: Thu, 31 May 2001 07:38:32 -0700 (PDT) From: Bernard Aboba To: julian_satran@il.ibm.com cc: ips@ece.cmu.edu Subject: Re: More on iSCSI boot In-Reply-To: Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII X-UIDL: 8e430255b8d93a713e0bcd90f0fd288d X-OriginalArrivalTime: 31 May 2001 17:00:44.0562 (UTC) FILETIME=[3B2EC720:01C0E9F3] Sender: owner-ips@ece.cmu.edu Precedence: bulk > -even if we would like to standardize a "primary" or "minimal" boot we have > no good understanding (experience) > of how this will interact with the iSCSI security mechanisms. > Agree with this. But I think the lesson is (as David mentioned) that you need a (perhaps non-mandatory) security method that can be executed in the boot rom. Personally, I'm not that convinced that boot image signing as presently conceived is all that useful anyway, since it isn't widely deployed and there's no way to revoke the signature of a bad boot image (e.g. no certificate chain support). From owner-ips@ece.cmu.edu Thu May 31 14:33:25 2001 Received: from ece.cmu.edu (ECE.CMU.EDU [128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id OAA05414 for ; Thu, 31 May 2001 14:33:25 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f4VGGBK28726 for ips-outgoing; Thu, 31 May 2001 12:16:11 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from homa.asicdesigners.com (ip-64-63-65-105.reverse.mobilenetics.com [64.63.65.105] (may be forged)) by ece.cmu.edu (8.11.0/8.10.2) with SMTP id f4VGG9t28722 for ; Thu, 31 May 2001 12:16:10 -0400 (EDT) Received: from homa.asicdesigners.com ([10.1.1.5]) by homa.asicdesigners.com with Microsoft SMTPSVC(5.0.2195.2966); Thu, 31 May 2001 09:15:42 -0700 Received: by homa.asicdesigners.com (Microsoft Connector for POP3 Mailboxes 5.00.2195) with SMTP (Global POP3 Download) id MSG05312001-091538-271.MMD@asicdesigners.com; Thu, 31 May 2001 09:15:38 -0700 Received: from ece.cmu.edu (ECE.CMU.EDU [128.2.236.200]) by mail13.netservers.net (8.9.3/8.9.3) with ESMTP id JAA14122 for ; Thu, 31 May 2001 09:12:49 -0700 Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f4VDpiQ16621 for ips-outgoing; Thu, 31 May 2001 09:51:44 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from mxbh4.isus.emc.com (mxbh4.isus.emc.com [128.221.10.33]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f4VDpht16617 for ; Thu, 31 May 2001 09:51:43 -0400 (EDT) Received: by mxbh4.isus.emc.com with Internet Mail Service (5.5.2653.19) id ; Thu, 31 May 2001 09:51:37 -0400 Message-ID: <0F31E5C394DAD311B60C00E029101A0708015636@corpmx9.isus.emc.com> From: Black_David@emc.com To: ENDL_TX@computer.org, ips@ece.cmu.edu Subject: RE: Nashua Interim Minutes Date: Thu, 31 May 2001 09:51:30 -0400 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2653.19) Content-Type: text/plain X-UIDL: 675f648135d60d6a2bc345484130ebae X-OriginalArrivalTime: 31 May 2001 16:15:42.0812 (UTC) FILETIME=[F0D071C0:01C0E9EC] Sender: owner-ips@ece.cmu.edu Precedence: bulk There are two sets of slides. In the interim until the IETF proceedings pick them up, I've sent mine to Julian to put on his web site along with Ofer Biran's slides so that both sets of security slides are in one place. Thanks, --David > -----Original Message----- > From: Ralph Weber [SMTP:ralphoweber@compuserve.com] > Sent: Monday, May 28, 2001 10:14 AM > To: IPS Reflector > Subject: Re: Nashua Interim Minutes > > I am unclear about this part of the minutes: > > } Security > } -------- > } > } -- David Black (EMC, co-chair) presentation on security requirements > } > } See slides - this was an informational presentation/discussion about > } requirements (MUST implement authentication and cryptographic > } integrity, MUST have a required mechanism for each, > } confidentiality is OPTIONAL, IP vs. iSCSI level mechanisms > } [iSCSI can multiplex initiators and targets on a single } address, TCP port> pair). > } > } -- Ofer Biran (IBM, Haifa) > } > } Lots of slides - see slides for details. > > Were there two sets of slides or just the one in Julian's web > archive? > > Thanks. > > Ralph... > From owner-ips@ece.cmu.edu Thu May 31 15:13:13 2001 Received: from ece.cmu.edu (ECE.CMU.EDU [128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id PAA06559 for ; Thu, 31 May 2001 15:13:13 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f4VH22002557 for ips-outgoing; Thu, 31 May 2001 13:02:02 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from homa.asicdesigners.com (ip-64-63-65-105.reverse.mobilenetics.com [64.63.65.105] (may be forged)) by ece.cmu.edu (8.11.0/8.10.2) with SMTP id f4VH1wt02541 for ; Thu, 31 May 2001 13:01:58 -0400 (EDT) Received: from homa.asicdesigners.com ([10.1.1.5]) by homa.asicdesigners.com with Microsoft SMTPSVC(5.0.2195.2966); Thu, 31 May 2001 10:00:44 -0700 Received: by homa.asicdesigners.com (Microsoft Connector for POP3 Mailboxes 5.00.2195) with SMTP (Global POP3 Download) id MSG05312001-100039-310.MMD@asicdesigners.com; Thu, 31 May 2001 10:00:39 -0700 Received: from ece.cmu.edu (ECE.CMU.EDU [128.2.236.200]) by mail13.netservers.net (8.9.3/8.9.3) with ESMTP id JAA17223 for ; Thu, 31 May 2001 09:55:34 -0700 Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f4VEdsu20271 for ips-outgoing; Thu, 31 May 2001 10:39:54 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from internaut.com ([64.38.134.99]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f4VEdqt20266 for ; Thu, 31 May 2001 10:39:52 -0400 (EDT) Received: from localhost (aboba@localhost) by internaut.com (8.9.3/8.9.3) with ESMTP id HAA52319; Thu, 31 May 2001 07:29:34 -0700 (PDT) (envelope-from aboba@internaut.com) Date: Thu, 31 May 2001 07:29:34 -0700 (PDT) From: Bernard Aboba To: Douglas Otis cc: julian_satran@il.ibm.com, ips@ece.cmu.edu Subject: RE: iSCSI and secure boot In-Reply-To: Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII X-UIDL: 8c32e4565d50dcb6344cbc56d2f2be3d X-OriginalArrivalTime: 31 May 2001 17:00:44.0609 (UTC) FILETIME=[3B35F310:01C0E9F3] Sender: owner-ips@ece.cmu.edu Precedence: bulk > Can you describe a secure diskless boot that does not require individual > setup of each? Some touching of each PC is required. However, the question is whether: a. The setup requires setup of multiple credentials, or just one. Note that BIS requires each PC to be configured with certificate of the boot image signing authority. However, this does not provide client authentication capabilities - so if you want to authenticate iSCSI or DHCP, or anything else, then you'd need additional credentials. b. Whether *per-interface* credentials are needed (e.g. authenticated DHCP draft -16), or *per-machine*. Per-interface credentials require the machine to be touched every time an interface is added or removed, not just when the machine is shipped by the OEM. From owner-ips@ece.cmu.edu Thu May 31 15:17:38 2001 Received: from ece.cmu.edu (ECE.CMU.EDU [128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id PAA06645 for ; Thu, 31 May 2001 15:17:37 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f4VH22902556 for ips-outgoing; Thu, 31 May 2001 13:02:02 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from homa.asicdesigners.com (ip-64-63-65-105.reverse.mobilenetics.com [64.63.65.105] (may be forged)) by ece.cmu.edu (8.11.0/8.10.2) with SMTP id f4VH1wt02546 for ; Thu, 31 May 2001 13:01:59 -0400 (EDT) Received: from homa.asicdesigners.com ([10.1.1.5]) by homa.asicdesigners.com with Microsoft SMTPSVC(5.0.2195.2966); Thu, 31 May 2001 10:00:44 -0700 Received: by homa.asicdesigners.com (Microsoft Connector for POP3 Mailboxes 5.00.2195) with SMTP (Global POP3 Download) id MSG05312001-100039-311.MMD@asicdesigners.com; Thu, 31 May 2001 10:00:39 -0700 Received: from ece.cmu.edu (ECE.CMU.EDU [128.2.236.200]) by mail13.netservers.net (8.9.3/8.9.3) with ESMTP id JAA17216 for ; Thu, 31 May 2001 09:55:31 -0700 Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f4VEwjG21901 for ips-outgoing; Thu, 31 May 2001 10:58:45 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from internaut.com ([64.38.134.99]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f4VEwgt21896 for ; Thu, 31 May 2001 10:58:43 -0400 (EDT) Received: from localhost (aboba@localhost) by internaut.com (8.9.3/8.9.3) with ESMTP id HAA52339; Thu, 31 May 2001 07:48:40 -0700 (PDT) (envelope-from aboba@internaut.com) Date: Thu, 31 May 2001 07:48:40 -0700 (PDT) From: Bernard Aboba To: David Robinson cc: ips@ece.cmu.edu Subject: Re: More on iSCSI boot In-Reply-To: <3B15C1C0.82895D8F@ebay.sun.com> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII X-UIDL: 1214215f79a8746e8688e1c5bebf6d1d X-OriginalArrivalTime: 31 May 2001 17:00:44.0671 (UTC) FILETIME=[3B3F68F0:01C0E9F3] Sender: owner-ips@ece.cmu.edu Precedence: bulk > I would be surprised if it insisted that > the ips WG take on the task of building a better DHCP. > Agree. Boot security in general is not the problem of the IPS WG. However, I do think that the WG should think through the implications of iSCSI security on the potential for a secure boot process. For example, are we assuming that it would be impossible to implement iSCSI security during boot? Also, I'd note that between BIS and authenticated DHCP we already are pushing the limits of credential storage and administrative complexity. It's not entirely clear to me that this passes the "laugh test" - particularly when the need to secure other components (NFS, iSCSI, etc.) is added to the mix. If that's true then even a "better DHCP" (e.g. authenticated DHCP) may not be the right answer. From owner-ips@ece.cmu.edu Thu May 31 16:05:03 2001 Received: from ece.cmu.edu (ECE.CMU.EDU [128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id QAA07371 for ; Thu, 31 May 2001 16:05:03 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f4VI6Uh07769 for ips-outgoing; Thu, 31 May 2001 14:06:30 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from sandmail.sandburst.com (sandburst-gw.bstn-gw02.ma.us.intelilink.net [216.57.129.34]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f4VI6Tt07765 for ; Thu, 31 May 2001 14:06:29 -0400 (EDT) Received: from cs.uchicago.edu (dynamite-38.sandburst.com [172.16.5.38]) by sandmail.sandburst.com (Postfix) with ESMTP id 5F88094009 for ; Thu, 31 May 2001 14:06:28 -0400 (EDT) To: ips@ece.cmu.edu Subject: Re: More on iSCSI boot In-Reply-To: Message from Black_David@emc.com of "Wed, 30 May 2001 22:49:57 EDT." <0F31E5C394DAD311B60C00E029101A0708015630@corpmx9.isus.emc.com> References: <0F31E5C394DAD311B60C00E029101A0708015630@corpmx9.isus.emc.com> Date: Thu, 31 May 2001 14:04:21 -0400 From: Stephen Bailey Message-Id: <20010531180628.5F88094009@sandmail.sandburst.com> Sender: owner-ips@ece.cmu.edu Precedence: bulk Just one nit. > On Intel systems, it's generally a combination of the system BIOS > and card BIOS that make the disk reads work. It this really true in practice? Once upon a time, it seemed like card-specific BIOS extension were becoming deprecated. Probably all BIOSes still support BIOS extensions, but I don't think current adapters actually provide them. My understanding is that if you want to boot off a disk controller, you either have to have specific BIOS support for it, or you have to emulate/implement INT13. This appeared to be conscious choice of the platform (BIOS) vendors to only support a bounded set of boot alternatives (since boot should be a bounded process) on each particular platform. If this is the case, it doesn't make sense to provide for iSCSI boot on a platform that is not conscious of that choice. This implies that the resources available for iSCSI boot are the complete resources of the platform's BIOS environment. This doesn't seem to substantively change the discussion, but it does remove one particular scenario from consideration. Steph From owner-ips@ece.cmu.edu Thu May 31 16:06:09 2001 Received: from ece.cmu.edu (ECE.CMU.EDU [128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id QAA07399 for ; Thu, 31 May 2001 16:06:08 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f4VJ2ca12361 for ips-outgoing; Thu, 31 May 2001 15:02:38 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from homa.asicdesigners.com (ip-64-63-65-105.reverse.mobilenetics.com [64.63.65.105] (may be forged)) by ece.cmu.edu (8.11.0/8.10.2) with SMTP id f4VJ2bt12356 for ; Thu, 31 May 2001 15:02:37 -0400 (EDT) Received: from homa.asicdesigners.com ([10.1.1.5]) by homa.asicdesigners.com with Microsoft SMTPSVC(5.0.2195.2966); Thu, 31 May 2001 12:00:44 -0700 Received: by homa.asicdesigners.com (Microsoft Connector for POP3 Mailboxes 5.00.2195) with SMTP (Global POP3 Download) id MSG05312001-120041-436.MMD@asicdesigners.com; Thu, 31 May 2001 12:00:41 -0700 Received: from ece.cmu.edu (ECE.CMU.EDU [128.2.236.200]) by mail14.netservers.net (8.9.3/8.9.3) with ESMTP id LAA05872 for ; Thu, 31 May 2001 11:46:14 -0700 Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f4VGGBK28726 for ips-outgoing; Thu, 31 May 2001 12:16:11 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from homa.asicdesigners.com (ip-64-63-65-105.reverse.mobilenetics.com [64.63.65.105] (may be forged)) by ece.cmu.edu (8.11.0/8.10.2) with SMTP id f4VGG9t28722 for ; Thu, 31 May 2001 12:16:10 -0400 (EDT) Received: from homa.asicdesigners.com ([10.1.1.5]) by homa.asicdesigners.com with Microsoft SMTPSVC(5.0.2195.2966); Thu, 31 May 2001 09:15:42 -0700 Received: by homa.asicdesigners.com (Microsoft Connector for POP3 Mailboxes 5.00.2195) with SMTP (Global POP3 Download) id MSG05312001-091538-271.MMD@asicdesigners.com; Thu, 31 May 2001 09:15:38 -0700 Received: from ece.cmu.edu (ECE.CMU.EDU [128.2.236.200]) by mail13.netservers.net (8.9.3/8.9.3) with ESMTP id JAA14122 for ; Thu, 31 May 2001 09:12:49 -0700 Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f4VDpiQ16621 for ips-outgoing; Thu, 31 May 2001 09:51:44 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from mxbh4.isus.emc.com (mxbh4.isus.emc.com [128.221.10.33]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f4VDpht16617 for ; Thu, 31 May 2001 09:51:43 -0400 (EDT) Received: by mxbh4.isus.emc.com with Internet Mail Service (5.5.2653.19) id ; Thu, 31 May 2001 09:51:37 -0400 Message-ID: <0F31E5C394DAD311B60C00E029101A0708015636@corpmx9.isus.emc.com> From: Black_David@emc.com To: ENDL_TX@computer.org, ips@ece.cmu.edu Subject: RE: Nashua Interim Minutes Date: Thu, 31 May 2001 09:51:30 -0400 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2653.19) Content-Type: text/plain X-OriginalArrivalTime: 31 May 2001 16:15:42.0812 (UTC) FILETIME=[F0D071C0:01C0E9EC] X-UIDL: 675f648135d60d6a2bc345484130ebae Sender: owner-ips@ece.cmu.edu Precedence: bulk There are two sets of slides. In the interim until the IETF proceedings pick them up, I've sent mine to Julian to put on his web site along with Ofer Biran's slides so that both sets of security slides are in one place. Thanks, --David > -----Original Message----- > From: Ralph Weber [SMTP:ralphoweber@compuserve.com] > Sent: Monday, May 28, 2001 10:14 AM > To: IPS Reflector > Subject: Re: Nashua Interim Minutes > > I am unclear about this part of the minutes: > > } Security > } -------- > } > } -- David Black (EMC, co-chair) presentation on security requirements > } > } See slides - this was an informational presentation/discussion about > } requirements (MUST implement authentication and cryptographic > } integrity, MUST have a required mechanism for each, > } confidentiality is OPTIONAL, IP vs. iSCSI level mechanisms > } [iSCSI can multiplex initiators and targets on a single } address, TCP port> pair). > } > } -- Ofer Biran (IBM, Haifa) > } > } Lots of slides - see slides for details. > > Were there two sets of slides or just the one in Julian's web > archive? > > Thanks. > > Ralph... > From owner-ips@ece.cmu.edu Thu May 31 16:22:23 2001 Received: from ece.cmu.edu (ECE.CMU.EDU [128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id QAA07562 for ; Thu, 31 May 2001 16:22:23 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f4VIlAL11117 for ips-outgoing; Thu, 31 May 2001 14:47:10 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from homa.asicdesigners.com (ip-64-63-65-105.reverse.mobilenetics.com [64.63.65.105] (may be forged)) by ece.cmu.edu (8.11.0/8.10.2) with SMTP id f4VIl7t11112 for ; Thu, 31 May 2001 14:47:08 -0400 (EDT) Received: from homa.asicdesigners.com ([10.1.1.5]) by homa.asicdesigners.com with Microsoft SMTPSVC(5.0.2195.2966); Thu, 31 May 2001 11:46:40 -0700 Received: by homa.asicdesigners.com (Microsoft Connector for POP3 Mailboxes 5.00.2195) with SMTP (Global POP3 Download) id MSG05312001-114539-416.MMD@asicdesigners.com; Thu, 31 May 2001 11:45:39 -0700 Received: from ece.cmu.edu (ECE.CMU.EDU [128.2.236.200]) by mail14.netservers.net (8.9.3/8.9.3) with ESMTP id LAA05714 for ; Thu, 31 May 2001 11:43:39 -0700 Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f4VH22u02555 for ips-outgoing; Thu, 31 May 2001 13:02:02 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from homa.asicdesigners.com (ip-64-63-65-105.reverse.mobilenetics.com [64.63.65.105] (may be forged)) by ece.cmu.edu (8.11.0/8.10.2) with SMTP id f4VH1vt02537 for ; Thu, 31 May 2001 13:01:57 -0400 (EDT) Received: from homa.asicdesigners.com ([10.1.1.5]) by homa.asicdesigners.com with Microsoft SMTPSVC(5.0.2195.2966); Thu, 31 May 2001 10:00:44 -0700 Received: by homa.asicdesigners.com (Microsoft Connector for POP3 Mailboxes 5.00.2195) with SMTP (Global POP3 Download) id MSG05312001-100039-309.MMD@asicdesigners.com; Thu, 31 May 2001 10:00:39 -0700 Received: from ece.cmu.edu (ECE.CMU.EDU [128.2.236.200]) by mail13.netservers.net (8.9.3/8.9.3) with ESMTP id JAA17087 for ; Thu, 31 May 2001 09:53:55 -0700 Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f4VEmlB20996 for ips-outgoing; Thu, 31 May 2001 10:48:47 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from internaut.com ([64.38.134.99]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f4VEmjt20991 for ; Thu, 31 May 2001 10:48:45 -0400 (EDT) Received: from localhost (aboba@localhost) by internaut.com (8.9.3/8.9.3) with ESMTP id HAA52329; Thu, 31 May 2001 07:38:33 -0700 (PDT) (envelope-from aboba@internaut.com) Date: Thu, 31 May 2001 07:38:32 -0700 (PDT) From: Bernard Aboba To: julian_satran@il.ibm.com cc: ips@ece.cmu.edu Subject: Re: More on iSCSI boot In-Reply-To: Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII X-OriginalArrivalTime: 31 May 2001 17:00:44.0562 (UTC) FILETIME=[3B2EC720:01C0E9F3] X-UIDL: 8e430255b8d93a713e0bcd90f0fd288d Sender: owner-ips@ece.cmu.edu Precedence: bulk > -even if we would like to standardize a "primary" or "minimal" boot we have > no good understanding (experience) > of how this will interact with the iSCSI security mechanisms. > Agree with this. But I think the lesson is (as David mentioned) that you need a (perhaps non-mandatory) security method that can be executed in the boot rom. Personally, I'm not that convinced that boot image signing as presently conceived is all that useful anyway, since it isn't widely deployed and there's no way to revoke the signature of a bad boot image (e.g. no certificate chain support). From owner-ips@ece.cmu.edu Thu May 31 17:07:25 2001 Received: from ece.cmu.edu (ECE.CMU.EDU [128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id RAA08127 for ; Thu, 31 May 2001 17:07:25 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f4VJl4l16306 for ips-outgoing; Thu, 31 May 2001 15:47:04 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from homa.asicdesigners.com (ip-64-63-65-105.reverse.mobilenetics.com [64.63.65.105] (may be forged)) by ece.cmu.edu (8.11.0/8.10.2) with SMTP id f4VJl1t16298 for ; Thu, 31 May 2001 15:47:01 -0400 (EDT) Received: from homa.asicdesigners.com ([10.1.1.5]) by homa.asicdesigners.com with Microsoft SMTPSVC(5.0.2195.2966); Thu, 31 May 2001 12:46:33 -0700 Received: by homa.asicdesigners.com (Microsoft Connector for POP3 Mailboxes 5.00.2195) with SMTP (Global POP3 Download) id MSG05312001-124539-472.MMD@asicdesigners.com; Thu, 31 May 2001 12:45:39 -0700 Received: from ece.cmu.edu (ECE.CMU.EDU [128.2.236.200]) by mail14.netservers.net (8.9.3/8.9.3) with ESMTP id MAA08339 for ; Thu, 31 May 2001 12:32:10 -0700 Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f4VH22002557 for ips-outgoing; Thu, 31 May 2001 13:02:02 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from homa.asicdesigners.com (ip-64-63-65-105.reverse.mobilenetics.com [64.63.65.105] (may be forged)) by ece.cmu.edu (8.11.0/8.10.2) with SMTP id f4VH1wt02541 for ; Thu, 31 May 2001 13:01:58 -0400 (EDT) Received: from homa.asicdesigners.com ([10.1.1.5]) by homa.asicdesigners.com with Microsoft SMTPSVC(5.0.2195.2966); Thu, 31 May 2001 10:00:44 -0700 Received: by homa.asicdesigners.com (Microsoft Connector for POP3 Mailboxes 5.00.2195) with SMTP (Global POP3 Download) id MSG05312001-100039-310.MMD@asicdesigners.com; Thu, 31 May 2001 10:00:39 -0700 Received: from ece.cmu.edu (ECE.CMU.EDU [128.2.236.200]) by mail13.netservers.net (8.9.3/8.9.3) with ESMTP id JAA17223 for ; Thu, 31 May 2001 09:55:34 -0700 Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f4VEdsu20271 for ips-outgoing; Thu, 31 May 2001 10:39:54 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from internaut.com ([64.38.134.99]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f4VEdqt20266 for ; Thu, 31 May 2001 10:39:52 -0400 (EDT) Received: from localhost (aboba@localhost) by internaut.com (8.9.3/8.9.3) with ESMTP id HAA52319; Thu, 31 May 2001 07:29:34 -0700 (PDT) (envelope-from aboba@internaut.com) Date: Thu, 31 May 2001 07:29:34 -0700 (PDT) From: Bernard Aboba To: Douglas Otis cc: julian_satran@il.ibm.com, ips@ece.cmu.edu Subject: RE: iSCSI and secure boot In-Reply-To: Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII X-OriginalArrivalTime: 31 May 2001 17:00:44.0609 (UTC) FILETIME=[3B35F310:01C0E9F3] X-UIDL: 8c32e4565d50dcb6344cbc56d2f2be3d Sender: owner-ips@ece.cmu.edu Precedence: bulk > Can you describe a secure diskless boot that does not require individual > setup of each? Some touching of each PC is required. However, the question is whether: a. The setup requires setup of multiple credentials, or just one. Note that BIS requires each PC to be configured with certificate of the boot image signing authority. However, this does not provide client authentication capabilities - so if you want to authenticate iSCSI or DHCP, or anything else, then you'd need additional credentials. b. Whether *per-interface* credentials are needed (e.g. authenticated DHCP draft -16), or *per-machine*. Per-interface credentials require the machine to be touched every time an interface is added or removed, not just when the machine is shipped by the OEM. From owner-ips@ece.cmu.edu Thu May 31 17:08:29 2001 Received: from ece.cmu.edu (ECE.CMU.EDU [128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id RAA08139 for ; Thu, 31 May 2001 17:08:28 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f4VJELF13380 for ips-outgoing; Thu, 31 May 2001 15:14:21 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from mxic2.us.dg.com (mxic2.us.dg.com [128.221.31.40]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f4VJEKt13371 for ; Thu, 31 May 2001 15:14:20 -0400 (EDT) Received: by mxic2.us.dg.com with Internet Mail Service (5.5.2653.19) id ; Thu, 31 May 2001 15:13:35 -0400 Message-ID: <0F31E5C394DAD311B60C00E029101A070801563E@corpmx9.isus.emc.com> From: Black_David@emc.com To: ips@ece.cmu.edu Subject: iSCSI Security mechanisms Date: Thu, 31 May 2001 15:14:11 -0400 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2653.19) Content-Type: text/plain Sender: owner-ips@ece.cmu.edu Precedence: bulk At the risk of exciting this nest of hornets, here's a summary of the current WG rough consensus on iSCSI security requirements for implementations. I'm doing this both because I've received some off-line indications of confusion and because the Nashua minutes aren't as clear about this as they could be: - In-band iSCSI authentication SRP - REQUIRED all other mechanisms - OPTIONAL - Cryptographic communication integrity (these are all IPSec components): ESP with null encryption - REQUIRED ESP with non-null encryption - OPTIONAL AH - OPTIONAL IKE - OPTIONAL I would note that anyone considering encryption ought to be working on/with AES, not just 3DES. This leaves open the issue of where the key(s) for ESP come from. IKE is OPTIONAL, and use of SRP to supply keys for ESP is NOT REQUIRED (not even specified - I need to find the time to work on writing this up). This leaves pre-shared keys as the minimum mechanism, and hence I believe that a suitably secured administrative interface to supply pre-shared keys to ESP will have to be REQUIRED for interoperability even if a dynamic keying mechanism like IKE is implemented. Thanks, --David --------------------------------------------------- David L. Black, Senior Technologist EMC Corporation, 42 South St., Hopkinton, MA 01748 +1 (508) 435-1000 x75140 FAX: +1 (508) 497-8500 black_david@emc.com Mobile: +1 (978) 394-7754 --------------------------------------------------- From owner-ips@ece.cmu.edu Thu May 31 17:16:11 2001 Received: from ece.cmu.edu (ECE.CMU.EDU [128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id RAA08234 for ; Thu, 31 May 2001 17:16:11 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f4VJl4T16305 for ips-outgoing; Thu, 31 May 2001 15:47:04 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from homa.asicdesigners.com (ip-64-63-65-105.reverse.mobilenetics.com [64.63.65.105] (may be forged)) by ece.cmu.edu (8.11.0/8.10.2) with SMTP id f4VJl0t16294 for ; Thu, 31 May 2001 15:47:01 -0400 (EDT) Received: from homa.asicdesigners.com ([10.1.1.5]) by homa.asicdesigners.com with Microsoft SMTPSVC(5.0.2195.2966); Thu, 31 May 2001 12:46:33 -0700 Received: by homa.asicdesigners.com (Microsoft Connector for POP3 Mailboxes 5.00.2195) with SMTP (Global POP3 Download) id MSG05312001-124538-471.MMD@asicdesigners.com; Thu, 31 May 2001 12:45:38 -0700 Received: from ece.cmu.edu (ECE.CMU.EDU [128.2.236.200]) by mail14.netservers.net (8.9.3/8.9.3) with ESMTP id MAA08299 for ; Thu, 31 May 2001 12:31:05 -0700 Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f4VH22902556 for ips-outgoing; Thu, 31 May 2001 13:02:02 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from homa.asicdesigners.com (ip-64-63-65-105.reverse.mobilenetics.com [64.63.65.105] (may be forged)) by ece.cmu.edu (8.11.0/8.10.2) with SMTP id f4VH1wt02546 for ; Thu, 31 May 2001 13:01:59 -0400 (EDT) Received: from homa.asicdesigners.com ([10.1.1.5]) by homa.asicdesigners.com with Microsoft SMTPSVC(5.0.2195.2966); Thu, 31 May 2001 10:00:44 -0700 Received: by homa.asicdesigners.com (Microsoft Connector for POP3 Mailboxes 5.00.2195) with SMTP (Global POP3 Download) id MSG05312001-100039-311.MMD@asicdesigners.com; Thu, 31 May 2001 10:00:39 -0700 Received: from ece.cmu.edu (ECE.CMU.EDU [128.2.236.200]) by mail13.netservers.net (8.9.3/8.9.3) with ESMTP id JAA17216 for ; Thu, 31 May 2001 09:55:31 -0700 Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f4VEwjG21901 for ips-outgoing; Thu, 31 May 2001 10:58:45 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from internaut.com ([64.38.134.99]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f4VEwgt21896 for ; Thu, 31 May 2001 10:58:43 -0400 (EDT) Received: from localhost (aboba@localhost) by internaut.com (8.9.3/8.9.3) with ESMTP id HAA52339; Thu, 31 May 2001 07:48:40 -0700 (PDT) (envelope-from aboba@internaut.com) Date: Thu, 31 May 2001 07:48:40 -0700 (PDT) From: Bernard Aboba To: David Robinson cc: ips@ece.cmu.edu Subject: Re: More on iSCSI boot In-Reply-To: <3B15C1C0.82895D8F@ebay.sun.com> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII X-OriginalArrivalTime: 31 May 2001 17:00:44.0671 (UTC) FILETIME=[3B3F68F0:01C0E9F3] X-UIDL: 1214215f79a8746e8688e1c5bebf6d1d Sender: owner-ips@ece.cmu.edu Precedence: bulk > I would be surprised if it insisted that > the ips WG take on the task of building a better DHCP. > Agree. Boot security in general is not the problem of the IPS WG. However, I do think that the WG should think through the implications of iSCSI security on the potential for a secure boot process. For example, are we assuming that it would be impossible to implement iSCSI security during boot? Also, I'd note that between BIS and authenticated DHCP we already are pushing the limits of credential storage and administrative complexity. It's not entirely clear to me that this passes the "laugh test" - particularly when the need to secure other components (NFS, iSCSI, etc.) is added to the mix. If that's true then even a "better DHCP" (e.g. authenticated DHCP) may not be the right answer. From owner-ips@ece.cmu.edu Thu May 31 18:36:40 2001 Received: from ece.cmu.edu (ECE.CMU.EDU [128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id SAA09548 for ; Thu, 31 May 2001 18:36:39 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f4VKUp520085 for ips-outgoing; Thu, 31 May 2001 16:30:51 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from homa.asicdesigners.com (ip-64-63-65-105.reverse.mobilenetics.com [64.63.65.105] (may be forged)) by ece.cmu.edu (8.11.0/8.10.2) with SMTP id f4VKUmt20078 for ; Thu, 31 May 2001 16:30:48 -0400 (EDT) Received: from homa.asicdesigners.com ([10.1.1.5]) by homa.asicdesigners.com with Microsoft SMTPSVC(5.0.2195.2966); Thu, 31 May 2001 13:31:06 -0700 Received: by homa.asicdesigners.com (Microsoft Connector for POP3 Mailboxes 5.00.2195) with SMTP (Global POP3 Download) id MSG05312001-133038-517.MMD@asicdesigners.com; Thu, 31 May 2001 13:30:38 -0700 Received: from ece.cmu.edu (ECE.CMU.EDU [128.2.236.200]) by mail13.netservers.net (8.9.3/8.9.3) with ESMTP id NAA30577 for ; Thu, 31 May 2001 13:21:04 -0700 Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f4VI6Uh07769 for ips-outgoing; Thu, 31 May 2001 14:06:30 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from sandmail.sandburst.com (sandburst-gw.bstn-gw02.ma.us.intelilink.net [216.57.129.34]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f4VI6Tt07765 for ; Thu, 31 May 2001 14:06:29 -0400 (EDT) Received: from cs.uchicago.edu (dynamite-38.sandburst.com [172.16.5.38]) by sandmail.sandburst.com (Postfix) with ESMTP id 5F88094009 for ; Thu, 31 May 2001 14:06:28 -0400 (EDT) To: ips@ece.cmu.edu Subject: Re: More on iSCSI boot In-Reply-To: Message from Black_David@emc.com of "Wed, 30 May 2001 22:49:57 EDT." <0F31E5C394DAD311B60C00E029101A0708015630@corpmx9.isus.emc.com> References: <0F31E5C394DAD311B60C00E029101A0708015630@corpmx9.isus.emc.com> Date: Thu, 31 May 2001 14:04:21 -0400 From: Stephen Bailey Message-Id: <20010531180628.5F88094009@sandmail.sandburst.com> X-UIDL: 5d3549bb155e69e847366e5a8c429845 X-OriginalArrivalTime: 31 May 2001 20:31:06.0859 (UTC) FILETIME=[9EA873B0:01C0EA10] Sender: owner-ips@ece.cmu.edu Precedence: bulk Just one nit. > On Intel systems, it's generally a combination of the system BIOS > and card BIOS that make the disk reads work. It this really true in practice? Once upon a time, it seemed like card-specific BIOS extension were becoming deprecated. Probably all BIOSes still support BIOS extensions, but I don't think current adapters actually provide them. My understanding is that if you want to boot off a disk controller, you either have to have specific BIOS support for it, or you have to emulate/implement INT13. This appeared to be conscious choice of the platform (BIOS) vendors to only support a bounded set of boot alternatives (since boot should be a bounded process) on each particular platform. If this is the case, it doesn't make sense to provide for iSCSI boot on a platform that is not conscious of that choice. This implies that the resources available for iSCSI boot are the complete resources of the platform's BIOS environment. This doesn't seem to substantively change the discussion, but it does remove one particular scenario from consideration. Steph From owner-ips@ece.cmu.edu Thu May 31 19:10:26 2001 Received: from ece.cmu.edu (ECE.CMU.EDU [128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id TAA09884 for ; Thu, 31 May 2001 19:10:26 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f4VLief26005 for ips-outgoing; Thu, 31 May 2001 17:44:40 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from opus.ece.cmu.edu (root@OPUS.ECE.CMU.EDU [128.2.134.91]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f4VLidt26001 for ; Thu, 31 May 2001 17:44:39 -0400 (EDT) Received: from opus.ece.cmu.edu (bassoon@localhost [127.0.0.1]) by opus.ece.cmu.edu (8.9.3/8.9.3/SuSE Linux 8.9.3-0.1) with ESMTP id RAA09497 for ; Thu, 31 May 2001 17:44:39 -0400 Message-Id: <200105312144.RAA09497@opus.ece.cmu.edu> To: ips@ece.cmu.edu subject: iSNS Source Code Date: Thu, 31 May 2001 17:44:39 -0400 From: Dave Nagle Sender: owner-ips@ece.cmu.edu Precedence: bulk Annoucement ----------- The Internet Storage Name Service (iSNS) source code is now available from the IPS web site. The client code is posted and the server code will be available in mid june. You'll find the code at: http://www.ece.cmu.edu/~ips/IPS_Projects/ips_projects.html Also, after tonight, there will be a mailing list to discuss source code issues. The list will be called ips_source and you can subscribe by consulting the ips web pages. From owner-ips@ece.cmu.edu Thu May 31 19:11:39 2001 Received: from ece.cmu.edu (ECE.CMU.EDU [128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id TAA09908 for ; Thu, 31 May 2001 19:11:39 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f4VLJES23945 for ips-outgoing; Thu, 31 May 2001 17:19:14 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from c007.snv.cp.net (c007-h011.c007.snv.cp.net [209.228.33.217]) by ece.cmu.edu (8.11.0/8.10.2) with SMTP id f4VLJCt23940 for ; Thu, 31 May 2001 17:19:12 -0400 (EDT) Received: (cpmta 10930 invoked from network); 31 May 2001 14:06:33 -0700 Received: from adsl-63-202-160-80.dsl.snfc21.pacbell.net (HELO ljoy) (63.202.160.80) by smtp.telocity.com (209.228.33.217) with SMTP; 31 May 2001 14:06:33 -0700 X-Sent: 31 May 2001 21:06:33 GMT From: "Douglas Otis" To: "Bernard Aboba" Cc: , Subject: RE: iSCSI and secure boot Date: Thu, 31 May 2001 14:04:13 -0700 Message-ID: MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 (Normal) X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook IMO, Build 9.0.2416 (9.0.2911.0) In-Reply-To: X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4522.1200 Importance: Normal Sender: owner-ips@ece.cmu.edu Precedence: bulk Content-Transfer-Encoding: 7bit Bernard, Julian, > > Can you describe a secure diskless boot that does not require individual > > setup of each? > > Some touching of each PC is required. However, the question is whether: It would be impossible to make a stable boot image that depended on revision 1 of a protocol about to hit the market. As such, it is misguided to consider using DHCP options to allow direct booting of iSCSI. Once security is considered, it becomes even more obvious that a two step process is required to allow needed flexibility and manageability. Once a management scheme is selected that is suitable for an enterprise deployment, LDAP or commercial equivalents are a good candidate. Attempting to place this management function on the iSCSI server complicates iSCSI and ensures no common method of promulgating management. In this respect, I differ from the opinion of David Black. David likes to construe an LDAP server as too difficult and wishes to fulfill this management need with various other inventions. > a. The setup requires setup of multiple credentials, or just > one. Note that BIS requires each PC to be configured with certificate of > the boot image signing authority. However, this does not provide client > authentication capabilities - so if you want to authenticate iSCSI or > DHCP, or anything else, then you'd need additional credentials. This second step could depend fully on TFTP and a DUA for LDAP. This second layer should divorce itself from iSCSI other than to establish an environment suitable for individualized booting using iSCSI. This seems quite possible to implement and to promote as a reference implementation once a schema is defined for LDAP. It would not require changes to existing system efforts but instead build upon them. The goal would be to provide a single simple boot that would *not* require change and yet allow the passing of variables and images required for the evolution of iSCSI. Just this initial boot would be accommodated by the DHCP, TFTP, and booting system. DHCP already provides a significant amount of flexibility. > b. Whether *per-interface* credentials are needed (e.g. authenticated DHCP > draft -16), or *per-machine*. Per-interface credentials require the > machine to be touched every time an interface is added or removed, not > just when the machine is shipped by the OEM. LDAP could provide the needed database required to provide the correct images in a highly flexible manner. As this type of server is often a critical server in an enterprise environment, it seems like a very safe choice. Julian's concern about not understanding this environment should encourage the use of existing schemes rather than reinventing new ones. Think of booting as a minimum of a two step process. A simple secure image coupled with information from a secure LDAP server to then obtain then next step. The only code that would need to be learned would be the DUA, and TFTP. Doug From owner-ips@ece.cmu.edu Thu May 31 19:15:01 2001 Received: from ece.cmu.edu (ECE.CMU.EDU [128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id TAA09940 for ; Thu, 31 May 2001 19:15:01 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f4VLmiR26337 for ips-outgoing; Thu, 31 May 2001 17:48:44 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from ntmail.qlc.com (216-231-2-8.qlc.com [216.231.2.8]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f4VLmht26333 for ; Thu, 31 May 2001 17:48:43 -0400 (EDT) Received: by ntmail.qlc.com with Internet Mail Service (5.5.2653.19) id ; Thu, 31 May 2001 14:47:37 -0700 Message-ID: <88B878891850D511BA6A0003470D72E426D7E8@ntmail.qlc.com> From: Chuck Micalizzi To: "'julian_satran@il.ibm.com'" Cc: ips@ece.cmu.edu Subject: iSCSI rev. 7 status Date: Thu, 31 May 2001 14:47:36 -0700 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2653.19) Content-Type: text/plain; charset="iso-8859-1" Sender: owner-ips@ece.cmu.edu Precedence: bulk Julian, Do you expect to be posting version 7 anytime soon? thanks chuck From owner-ips@ece.cmu.edu Thu May 31 19:15:43 2001 Received: from ece.cmu.edu (ECE.CMU.EDU [128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id TAA09967 for ; Thu, 31 May 2001 19:15:43 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f4VLUU524857 for ips-outgoing; Thu, 31 May 2001 17:30:30 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from homa.asicdesigners.com (ip-64-63-65-105.reverse.mobilenetics.com [64.63.65.105] (may be forged)) by ece.cmu.edu (8.11.0/8.10.2) with SMTP id f4VLUTt24853 for ; Thu, 31 May 2001 17:30:29 -0400 (EDT) Received: from homa.asicdesigners.com ([10.1.1.5]) by homa.asicdesigners.com with Microsoft SMTPSVC(5.0.2195.2966); Thu, 31 May 2001 14:30:47 -0700 Received: by homa.asicdesigners.com (Microsoft Connector for POP3 Mailboxes 5.00.2195) with SMTP (Global POP3 Download) id MSG05312001-143039-574.MMD@asicdesigners.com; Thu, 31 May 2001 14:30:39 -0700 Received: from ece.cmu.edu (ECE.CMU.EDU [128.2.236.200]) by mail14.netservers.net (8.9.3/8.9.3) with ESMTP id OAA13639 for ; Thu, 31 May 2001 14:18:40 -0700 Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f4VJELF13380 for ips-outgoing; Thu, 31 May 2001 15:14:21 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from mxic2.us.dg.com (mxic2.us.dg.com [128.221.31.40]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f4VJEKt13371 for ; Thu, 31 May 2001 15:14:20 -0400 (EDT) Received: by mxic2.us.dg.com with Internet Mail Service (5.5.2653.19) id ; Thu, 31 May 2001 15:13:35 -0400 Message-ID: <0F31E5C394DAD311B60C00E029101A070801563E@corpmx9.isus.emc.com> From: Black_David@emc.com To: ips@ece.cmu.edu Subject: iSCSI Security mechanisms Date: Thu, 31 May 2001 15:14:11 -0400 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2653.19) Content-Type: text/plain X-UIDL: 1f3db25ecfd772cf029aeb5ad31c45fc X-OriginalArrivalTime: 31 May 2001 21:30:47.0812 (UTC) FILETIME=[F5128440:01C0EA18] Sender: owner-ips@ece.cmu.edu Precedence: bulk At the risk of exciting this nest of hornets, here's a summary of the current WG rough consensus on iSCSI security requirements for implementations. I'm doing this both because I've received some off-line indications of confusion and because the Nashua minutes aren't as clear about this as they could be: - In-band iSCSI authentication SRP - REQUIRED all other mechanisms - OPTIONAL - Cryptographic communication integrity (these are all IPSec components): ESP with null encryption - REQUIRED ESP with non-null encryption - OPTIONAL AH - OPTIONAL IKE - OPTIONAL I would note that anyone considering encryption ought to be working on/with AES, not just 3DES. This leaves open the issue of where the key(s) for ESP come from. IKE is OPTIONAL, and use of SRP to supply keys for ESP is NOT REQUIRED (not even specified - I need to find the time to work on writing this up). This leaves pre-shared keys as the minimum mechanism, and hence I believe that a suitably secured administrative interface to supply pre-shared keys to ESP will have to be REQUIRED for interoperability even if a dynamic keying mechanism like IKE is implemented. Thanks, --David --------------------------------------------------- David L. Black, Senior Technologist EMC Corporation, 42 South St., Hopkinton, MA 01748 +1 (508) 435-1000 x75140 FAX: +1 (508) 497-8500 black_david@emc.com Mobile: +1 (978) 394-7754 --------------------------------------------------- From owner-ips@ece.cmu.edu Thu May 31 20:35:43 2001 Received: from ece.cmu.edu (ECE.CMU.EDU [128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id UAA10802 for ; Thu, 31 May 2001 20:35:43 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f4VMjZ200514 for ips-outgoing; Thu, 31 May 2001 18:45:35 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from mercury.Sun.COM (mercury.Sun.COM [192.9.25.1]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f4VMjYt00509 for ; Thu, 31 May 2001 18:45:34 -0400 (EDT) Received: from phys-ha6nwkb.ebay.sun.com ([129.149.1.83]) by mercury.Sun.COM (8.9.3+Sun/8.9.3) with ESMTP id PAA01229; Thu, 31 May 2001 15:45:30 -0700 (PDT) Received: from ebay.sun.com (d-nwk16-169-196 [129.149.169.196]) by phys-ha6nwkb.ebay.sun.com (8.9.3+Sun/8.9.3/ENSMAIL,v2.1p1) with ESMTP id PAA09957; Thu, 31 May 2001 15:45:30 -0700 (PDT) Message-ID: <3B16C9D3.AB7FACBC@ebay.sun.com> Date: Thu, 31 May 2001 15:46:43 -0700 From: David Robinson Organization: Sun Microsystems, Inc. X-Mailer: Mozilla 4.77 [en] (Win98; U) X-Accept-Language: en MIME-Version: 1.0 To: Douglas Otis CC: Bernard Aboba , julian_satran@il.ibm.com, ips@ece.cmu.edu Subject: Re: iSCSI and secure boot References: Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-ips@ece.cmu.edu Precedence: bulk Content-Transfer-Encoding: 7bit > It would be impossible to make a stable boot image that depended on revision > 1 of a protocol about to hit the market. As such, it is misguided to > consider using DHCP options to allow direct booting of iSCSI. Once security > is considered, it becomes even more obvious that a two step process is > required to allow needed flexibility and manageability. Once a management > scheme is selected that is suitable for an enterprise deployment, LDAP or > commercial equivalents are a good candidate. Attempting to place this > management function on the iSCSI server complicates iSCSI and ensures no > common method of promulgating management. In this respect, I differ from > the opinion of David Black. David likes to construe an LDAP server as too > difficult and wishes to fulfill this management need with various other > inventions. While we have been using DHCP as the example of how hard it is to securely boot over a network, switching to LDAP does nothing to make it easier. Every initiator still needs to have a key stored securely in nonvolitile memory and mechanisms to initially create the key as well as update or revoke it. LDAP doesn't solve this problem. > This second step could depend fully on TFTP and a DUA for LDAP. This second > layer should divorce itself from iSCSI other than to establish an > environment suitable for individualized booting using iSCSI. This seems > quite possible to implement and to promote as a reference implementation > once a schema is defined for LDAP. It would not require changes to existing > system efforts but instead build upon them. The goal would be to provide a > single simple boot that would *not* require change and yet allow the passing > of variables and images required for the evolution of iSCSI. Just this > initial boot would be accommodated by the DHCP, TFTP, and booting system. > DHCP already provides a significant amount of flexibility. Regardless of if you used TFTP, DHCP, or LDAP as the secondary boot platform, it must be one that can be trusted and the trust must come from the physical hardware and not over a network. They] same key management issues still exist. > > b. Whether *per-interface* credentials are needed (e.g. authenticated DHCP > > draft -16), or *per-machine*. Per-interface credentials require the > > machine to be touched every time an interface is added or removed, not > > just when the machine is shipped by the OEM. > > LDAP could provide the needed database required to provide the correct > images in a highly flexible manner. As this type of server is often a > critical server in an enterprise environment, it seems like a very safe > choice. Julian's concern about not understanding this environment should > encourage the use of existing schemes rather than reinventing new ones. > Think of booting as a minimum of a two step process. A simple secure image > coupled with information from a secure LDAP server to then obtain then next > step. The only code that would need to be learned would be the DUA, and > TFTP. But the problem is that no matter how secure the LDAP server is, the client must be secure or all bets are off. The only way to make the client secure is to have some form of credential on the client before any network traffic is initiated. Bernard was simply describing that having a per-machine credential won't scale and the proposal to have a per-interface credential will have orders of magnitude worse scaling. Again LDAP can't be used to manage this as the credential must be on the client *before* it issues any requests. -David From owner-ips@ece.cmu.edu Thu May 31 20:37:54 2001 Received: from ece.cmu.edu (ECE.CMU.EDU [128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id UAA10888 for ; Thu, 31 May 2001 20:37:54 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f4VN0T001600 for ips-outgoing; Thu, 31 May 2001 19:00:29 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from homa.asicdesigners.com (ip-64-63-65-105.reverse.mobilenetics.com [64.63.65.105] (may be forged)) by ece.cmu.edu (8.11.0/8.10.2) with SMTP id f4VN0Rt01593 for ; Thu, 31 May 2001 19:00:27 -0400 (EDT) Received: from homa.asicdesigners.com ([10.1.1.5]) by homa.asicdesigners.com with Microsoft SMTPSVC(5.0.2195.2966); Thu, 31 May 2001 16:00:46 -0700 Received: by homa.asicdesigners.com (Microsoft Connector for POP3 Mailboxes 5.00.2195) with SMTP (Global POP3 Download) id MSG05312001-160038-648.MMD@asicdesigners.com; Thu, 31 May 2001 16:00:38 -0700 Received: from ece.cmu.edu (ECE.CMU.EDU [128.2.236.200]) by mail13.netservers.net (8.9.3/8.9.3) with ESMTP id PAA07124 for ; Thu, 31 May 2001 15:50:02 -0700 Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f4VKUp520085 for ips-outgoing; Thu, 31 May 2001 16:30:51 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from homa.asicdesigners.com (ip-64-63-65-105.reverse.mobilenetics.com [64.63.65.105] (may be forged)) by ece.cmu.edu (8.11.0/8.10.2) with SMTP id f4VKUmt20078 for ; Thu, 31 May 2001 16:30:48 -0400 (EDT) Received: from homa.asicdesigners.com ([10.1.1.5]) by homa.asicdesigners.com with Microsoft SMTPSVC(5.0.2195.2966); Thu, 31 May 2001 13:31:06 -0700 Received: by homa.asicdesigners.com (Microsoft Connector for POP3 Mailboxes 5.00.2195) with SMTP (Global POP3 Download) id MSG05312001-133038-517.MMD@asicdesigners.com; Thu, 31 May 2001 13:30:38 -0700 Received: from ece.cmu.edu (ECE.CMU.EDU [128.2.236.200]) by mail13.netservers.net (8.9.3/8.9.3) with ESMTP id NAA30577 for ; Thu, 31 May 2001 13:21:04 -0700 Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f4VI6Uh07769 for ips-outgoing; Thu, 31 May 2001 14:06:30 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from sandmail.sandburst.com (sandburst-gw.bstn-gw02.ma.us.intelilink.net [216.57.129.34]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f4VI6Tt07765 for ; Thu, 31 May 2001 14:06:29 -0400 (EDT) Received: from cs.uchicago.edu (dynamite-38.sandburst.com [172.16.5.38]) by sandmail.sandburst.com (Postfix) with ESMTP id 5F88094009 for ; Thu, 31 May 2001 14:06:28 -0400 (EDT) To: ips@ece.cmu.edu Subject: Re: More on iSCSI boot In-Reply-To: Message from Black_David@emc.com of "Wed, 30 May 2001 22:49:57 EDT." <0F31E5C394DAD311B60C00E029101A0708015630@corpmx9.isus.emc.com> References: <0F31E5C394DAD311B60C00E029101A0708015630@corpmx9.isus.emc.com> Date: Thu, 31 May 2001 14:04:21 -0400 From: Stephen Bailey Message-Id: <20010531180628.5F88094009@sandmail.sandburst.com> X-OriginalArrivalTime: 31 May 2001 20:31:06.0859 (UTC) FILETIME=[9EA873B0:01C0EA10] X-UIDL: 5d3549bb155e69e847366e5a8c429845 Sender: owner-ips@ece.cmu.edu Precedence: bulk Just one nit. > On Intel systems, it's generally a combination of the system BIOS > and card BIOS that make the disk reads work. It this really true in practice? Once upon a time, it seemed like card-specific BIOS extension were becoming deprecated. Probably all BIOSes still support BIOS extensions, but I don't think current adapters actually provide them. My understanding is that if you want to boot off a disk controller, you either have to have specific BIOS support for it, or you have to emulate/implement INT13. This appeared to be conscious choice of the platform (BIOS) vendors to only support a bounded set of boot alternatives (since boot should be a bounded process) on each particular platform. If this is the case, it doesn't make sense to provide for iSCSI boot on a platform that is not conscious of that choice. This implies that the resources available for iSCSI boot are the complete resources of the platform's BIOS environment. This doesn't seem to substantively change the discussion, but it does remove one particular scenario from consideration. Steph From owner-ips@ece.cmu.edu Thu May 31 21:19:19 2001 Received: from ece.cmu.edu (ECE.CMU.EDU [128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id VAA11204 for ; Thu, 31 May 2001 21:19:19 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f4VNbcW04159 for ips-outgoing; Thu, 31 May 2001 19:37:38 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from homa.asicdesigners.com (ip-64-63-65-105.reverse.mobilenetics.com [64.63.65.105] (may be forged)) by ece.cmu.edu (8.11.0/8.10.2) with SMTP id f4VNbat04151 for ; Thu, 31 May 2001 19:37:36 -0400 (EDT) Received: from homa.asicdesigners.com ([10.1.1.5]) by homa.asicdesigners.com with Microsoft SMTPSVC(5.0.2195.2966); Thu, 31 May 2001 16:36:33 -0700 Received: by homa.asicdesigners.com (Microsoft Connector for POP3 Mailboxes 5.00.2195) with SMTP (Global POP3 Download) id MSG05312001-163628-2.MMD@asicdesigners.com; Thu, 31 May 2001 16:36:28 -0700 Received: from ece.cmu.edu (ECE.CMU.EDU [128.2.236.200]) by mail14.netservers.net (8.9.3/8.9.3) with ESMTP id QAA20156 for ; Thu, 31 May 2001 16:33:33 -0700 Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f4VLUU524857 for ips-outgoing; Thu, 31 May 2001 17:30:30 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from homa.asicdesigners.com (ip-64-63-65-105.reverse.mobilenetics.com [64.63.65.105] (may be forged)) by ece.cmu.edu (8.11.0/8.10.2) with SMTP id f4VLUTt24853 for ; Thu, 31 May 2001 17:30:29 -0400 (EDT) Received: from homa.asicdesigners.com ([10.1.1.5]) by homa.asicdesigners.com with Microsoft SMTPSVC(5.0.2195.2966); Thu, 31 May 2001 14:30:47 -0700 Received: by homa.asicdesigners.com (Microsoft Connector for POP3 Mailboxes 5.00.2195) with SMTP (Global POP3 Download) id MSG05312001-143039-574.MMD@asicdesigners.com; Thu, 31 May 2001 14:30:39 -0700 Received: from ece.cmu.edu (ECE.CMU.EDU [128.2.236.200]) by mail14.netservers.net (8.9.3/8.9.3) with ESMTP id OAA13639 for ; Thu, 31 May 2001 14:18:40 -0700 Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f4VJELF13380 for ips-outgoing; Thu, 31 May 2001 15:14:21 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from mxic2.us.dg.com (mxic2.us.dg.com [128.221.31.40]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f4VJEKt13371 for ; Thu, 31 May 2001 15:14:20 -0400 (EDT) Received: by mxic2.us.dg.com with Internet Mail Service (5.5.2653.19) id ; Thu, 31 May 2001 15:13:35 -0400 Message-ID: <0F31E5C394DAD311B60C00E029101A070801563E@corpmx9.isus.emc.com> From: Black_David@emc.com To: ips@ece.cmu.edu Subject: iSCSI Security mechanisms Date: Thu, 31 May 2001 15:14:11 -0400 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2653.19) Content-Type: text/plain X-OriginalArrivalTime: 31 May 2001 21:30:47.0812 (UTC) FILETIME=[F5128440:01C0EA18] X-UIDL: 1f3db25ecfd772cf029aeb5ad31c45fc Sender: owner-ips@ece.cmu.edu Precedence: bulk At the risk of exciting this nest of hornets, here's a summary of the current WG rough consensus on iSCSI security requirements for implementations. I'm doing this both because I've received some off-line indications of confusion and because the Nashua minutes aren't as clear about this as they could be: - In-band iSCSI authentication SRP - REQUIRED all other mechanisms - OPTIONAL - Cryptographic communication integrity (these are all IPSec components): ESP with null encryption - REQUIRED ESP with non-null encryption - OPTIONAL AH - OPTIONAL IKE - OPTIONAL I would note that anyone considering encryption ought to be working on/with AES, not just 3DES. This leaves open the issue of where the key(s) for ESP come from. IKE is OPTIONAL, and use of SRP to supply keys for ESP is NOT REQUIRED (not even specified - I need to find the time to work on writing this up). This leaves pre-shared keys as the minimum mechanism, and hence I believe that a suitably secured administrative interface to supply pre-shared keys to ESP will have to be REQUIRED for interoperability even if a dynamic keying mechanism like IKE is implemented. Thanks, --David --------------------------------------------------- David L. Black, Senior Technologist EMC Corporation, 42 South St., Hopkinton, MA 01748 +1 (508) 435-1000 x75140 FAX: +1 (508) 497-8500 black_david@emc.com Mobile: +1 (978) 394-7754 --------------------------------------------------- From owner-ips@ece.cmu.edu Thu May 31 21:24:25 2001 Received: from ece.cmu.edu (ECE.CMU.EDU [128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id VAA11256 for ; Thu, 31 May 2001 21:24:25 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f4VNX2803814 for ips-outgoing; Thu, 31 May 2001 19:33:02 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from homa.asicdesigners.com (ip-64-63-65-105.reverse.mobilenetics.com [64.63.65.105] (may be forged)) by ece.cmu.edu (8.11.0/8.10.2) with SMTP id f4VNWxt03803 for ; Thu, 31 May 2001 19:33:00 -0400 (EDT) Received: from homa.asicdesigners.com ([10.1.1.5]) by homa.asicdesigners.com with Microsoft SMTPSVC(5.0.2195.2966); Thu, 31 May 2001 16:30:28 -0700 Received: by homa.asicdesigners.com (Microsoft Connector for POP3 Mailboxes 5.00.2195) with SMTP (Global POP3 Download) id MSG05312001-163020-1.MMD@asicdesigners.com; Thu, 31 May 2001 16:30:20 -0700 Received: from ece.cmu.edu (ECE.CMU.EDU [128.2.236.200]) by mail14.netservers.net (8.9.3/8.9.3) with ESMTP id QAA19205 for ; Thu, 31 May 2001 16:14:31 -0700 Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f4VLJES23945 for ips-outgoing; Thu, 31 May 2001 17:19:14 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from c007.snv.cp.net (c007-h011.c007.snv.cp.net [209.228.33.217]) by ece.cmu.edu (8.11.0/8.10.2) with SMTP id f4VLJCt23940 for ; Thu, 31 May 2001 17:19:12 -0400 (EDT) Received: (cpmta 10930 invoked from network); 31 May 2001 14:06:33 -0700 Received: from adsl-63-202-160-80.dsl.snfc21.pacbell.net (HELO ljoy) (63.202.160.80) by smtp.telocity.com (209.228.33.217) with SMTP; 31 May 2001 14:06:33 -0700 X-Sent: 31 May 2001 21:06:33 GMT From: "Douglas Otis" To: "Bernard Aboba" Cc: , Subject: RE: iSCSI and secure boot Date: Thu, 31 May 2001 14:04:13 -0700 Message-ID: MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 (Normal) X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook IMO, Build 9.0.2416 (9.0.2911.0) In-Reply-To: X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4522.1200 Importance: Normal X-UIDL: ce708398070926097490020edc7fc975 X-OriginalArrivalTime: 31 May 2001 23:30:28.0734 (UTC) FILETIME=[AD3C41E0:01C0EA29] Sender: owner-ips@ece.cmu.edu Precedence: bulk Content-Transfer-Encoding: 7bit Bernard, Julian, > > Can you describe a secure diskless boot that does not require individual > > setup of each? > > Some touching of each PC is required. However, the question is whether: It would be impossible to make a stable boot image that depended on revision 1 of a protocol about to hit the market. As such, it is misguided to consider using DHCP options to allow direct booting of iSCSI. Once security is considered, it becomes even more obvious that a two step process is required to allow needed flexibility and manageability. Once a management scheme is selected that is suitable for an enterprise deployment, LDAP or commercial equivalents are a good candidate. Attempting to place this management function on the iSCSI server complicates iSCSI and ensures no common method of promulgating management. In this respect, I differ from the opinion of David Black. David likes to construe an LDAP server as too difficult and wishes to fulfill this management need with various other inventions. > a. The setup requires setup of multiple credentials, or just > one. Note that BIS requires each PC to be configured with certificate of > the boot image signing authority. However, this does not provide client > authentication capabilities - so if you want to authenticate iSCSI or > DHCP, or anything else, then you'd need additional credentials. This second step could depend fully on TFTP and a DUA for LDAP. This second layer should divorce itself from iSCSI other than to establish an environment suitable for individualized booting using iSCSI. This seems quite possible to implement and to promote as a reference implementation once a schema is defined for LDAP. It would not require changes to existing system efforts but instead build upon them. The goal would be to provide a single simple boot that would *not* require change and yet allow the passing of variables and images required for the evolution of iSCSI. Just this initial boot would be accommodated by the DHCP, TFTP, and booting system. DHCP already provides a significant amount of flexibility. > b. Whether *per-interface* credentials are needed (e.g. authenticated DHCP > draft -16), or *per-machine*. Per-interface credentials require the > machine to be touched every time an interface is added or removed, not > just when the machine is shipped by the OEM. LDAP could provide the needed database required to provide the correct images in a highly flexible manner. As this type of server is often a critical server in an enterprise environment, it seems like a very safe choice. Julian's concern about not understanding this environment should encourage the use of existing schemes rather than reinventing new ones. Think of booting as a minimum of a two step process. A simple secure image coupled with information from a secure LDAP server to then obtain then next step. The only code that would need to be learned would be the DUA, and TFTP. Doug From owner-ips@ece.cmu.edu Thu May 31 21:24:41 2001 Received: from ece.cmu.edu (ECE.CMU.EDU [128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id VAA11272 for ; Thu, 31 May 2001 21:24:40 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f4VNbbC04156 for ips-outgoing; Thu, 31 May 2001 19:37:37 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from homa.asicdesigners.com (ip-64-63-65-105.reverse.mobilenetics.com [64.63.65.105] (may be forged)) by ece.cmu.edu (8.11.0/8.10.2) with SMTP id f4VNbZt04147 for ; Thu, 31 May 2001 19:37:35 -0400 (EDT) Received: from homa.asicdesigners.com ([10.1.1.5]) by homa.asicdesigners.com with Microsoft SMTPSVC(5.0.2195.2966); Thu, 31 May 2001 16:36:33 -0700 Received: by homa.asicdesigners.com (Microsoft Connector for POP3 Mailboxes 5.00.2195) with SMTP (Global POP3 Download) id MSG05312001-163628-1.MMD@asicdesigners.com; Thu, 31 May 2001 16:36:28 -0700 Received: from ece.cmu.edu (ECE.CMU.EDU [128.2.236.200]) by mail14.netservers.net (8.9.3/8.9.3) with ESMTP id QAA20136 for ; Thu, 31 May 2001 16:33:18 -0700 Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f4VLmiR26337 for ips-outgoing; Thu, 31 May 2001 17:48:44 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from ntmail.qlc.com (216-231-2-8.qlc.com [216.231.2.8]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f4VLmht26333 for ; Thu, 31 May 2001 17:48:43 -0400 (EDT) Received: by ntmail.qlc.com with Internet Mail Service (5.5.2653.19) id ; Thu, 31 May 2001 14:47:37 -0700 Message-ID: <88B878891850D511BA6A0003470D72E426D7E8@ntmail.qlc.com> From: Chuck Micalizzi To: "'julian_satran@il.ibm.com'" Cc: ips@ece.cmu.edu Subject: iSCSI rev. 7 status Date: Thu, 31 May 2001 14:47:36 -0700 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2653.19) Content-Type: text/plain; charset="iso-8859-1" X-UIDL: fa3381b42e889146c2ea7b3c78b4af49 X-OriginalArrivalTime: 31 May 2001 23:36:33.0500 (UTC) FILETIME=[86A721C0:01C0EA2A] Sender: owner-ips@ece.cmu.edu Precedence: bulk Julian, Do you expect to be posting version 7 anytime soon? thanks chuck From owner-ips@ece.cmu.edu Thu May 31 21:25:37 2001 Received: from ece.cmu.edu (ECE.CMU.EDU [128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id VAA11299 for ; Thu, 31 May 2001 21:25:36 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f4VNX2h03813 for ips-outgoing; Thu, 31 May 2001 19:33:02 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from homa.asicdesigners.com (ip-64-63-65-105.reverse.mobilenetics.com [64.63.65.105] (may be forged)) by ece.cmu.edu (8.11.0/8.10.2) with SMTP id f4VNX0t03809 for ; Thu, 31 May 2001 19:33:00 -0400 (EDT) Received: from homa.asicdesigners.com ([10.1.1.5]) by homa.asicdesigners.com with Microsoft SMTPSVC(5.0.2195.2966); Thu, 31 May 2001 16:30:29 -0700 Received: by homa.asicdesigners.com (Microsoft Connector for POP3 Mailboxes 5.00.2195) with SMTP (Global POP3 Download) id MSG05312001-163020-2.MMD@asicdesigners.com; Thu, 31 May 2001 16:30:20 -0700 Received: from ece.cmu.edu (ECE.CMU.EDU [128.2.236.200]) by mail14.netservers.net (8.9.3/8.9.3) with ESMTP id QAA19210 for ; Thu, 31 May 2001 16:14:42 -0700 Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f4VLief26005 for ips-outgoing; Thu, 31 May 2001 17:44:40 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from opus.ece.cmu.edu (root@OPUS.ECE.CMU.EDU [128.2.134.91]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f4VLidt26001 for ; Thu, 31 May 2001 17:44:39 -0400 (EDT) Received: from opus.ece.cmu.edu (bassoon@localhost [127.0.0.1]) by opus.ece.cmu.edu (8.9.3/8.9.3/SuSE Linux 8.9.3-0.1) with ESMTP id RAA09497 for ; Thu, 31 May 2001 17:44:39 -0400 Message-Id: <200105312144.RAA09497@opus.ece.cmu.edu> To: ips@ece.cmu.edu subject: iSNS Source Code Date: Thu, 31 May 2001 17:44:39 -0400 From: Dave Nagle X-UIDL: 8f78b794ca63436fd27a8ffd5191b7b9 X-OriginalArrivalTime: 31 May 2001 23:30:29.0062 (UTC) FILETIME=[AD6E4E60:01C0EA29] Sender: owner-ips@ece.cmu.edu Precedence: bulk Annoucement ----------- The Internet Storage Name Service (iSNS) source code is now available from the IPS web site. The client code is posted and the server code will be available in mid june. You'll find the code at: http://www.ece.cmu.edu/~ips/IPS_Projects/ips_projects.html Also, after tonight, there will be a mailing list to discuss source code issues. The list will be called ips_source and you can subscribe by consulting the ips web pages. From owner-ips@ece.cmu.edu Thu May 31 22:13:00 2001 Received: from ece.cmu.edu (ECE.CMU.EDU [128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id WAA12827 for ; Thu, 31 May 2001 22:12:59 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f510MtZ07213 for ips-outgoing; Thu, 31 May 2001 20:22:55 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from internaut.com ([64.38.134.99]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f510Mrt07208 for ; Thu, 31 May 2001 20:22:53 -0400 (EDT) Received: from localhost (aboba@localhost) by internaut.com (8.9.3/8.9.3) with ESMTP id RAA52985; Thu, 31 May 2001 17:12:30 -0700 (PDT) (envelope-from aboba@internaut.com) Date: Thu, 31 May 2001 17:12:30 -0700 (PDT) From: Bernard Aboba To: Black_David@emc.com cc: ips@ece.cmu.edu Subject: Re: iSCSI Security mechanisms In-Reply-To: <0F31E5C394DAD311B60C00E029101A070801563E@corpmx9.isus.emc.com> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-ips@ece.cmu.edu Precedence: bulk > This leaves open the issue of where the key(s) for > ESP come from. IKE is OPTIONAL, and use of SRP to > supply keys for ESP is NOT REQUIRED (not even > specified - I need to find the time to work on > writing this up). This leaves pre-shared keys as > the minimum mechanism, and hence I believe that > a suitably secured administrative interface to > supply pre-shared keys to ESP will have to be > REQUIRED for interoperability even if a dynamic > keying mechanism like IKE is implemented. > Let me make sure I understand this: a. You use SRP (which derives a key) only for authentication, but then throw the derived keying material away. b. You then use IPSEC manual keying, along with an unspecified transform, and key distribution mechanism. Note that what is described above is not really pre-shared secrets, because those are used to authenticate DH in order to derive keying material. Since you're not using derived keying material, we're really talking about manual keying. I would suggest that the above manages to combine the computational demands of IKE (and then some, since SRP is slower than ordinary DH) the administrative headaches of manual keying and the interoperability problems of specifying nothing. Ancient proverb: "The first step in removing yourself from a deep hole is to stop digging." If you're going to do SRP for authentication, you might as well add in key derivation, and negotiation of things like ciphersuites, prime modulus/generator groups, etc. so you can use it for IPSEC keying. A rather small spec could accomplish this IF we decide it is necessary. If don't want to use the SRP keying material, you might as well use a slimmed down version of IKE (say, aggressive mode only) to address the code size concern. From owner-ips@ece.cmu.edu Thu May 31 23:01:17 2001 Received: from ece.cmu.edu (ECE.CMU.EDU [128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id XAA14267 for ; Thu, 31 May 2001 23:01:16 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f511kaV12780 for ips-outgoing; Thu, 31 May 2001 21:46:36 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from homa.asicdesigners.com (ip-64-63-65-105.reverse.mobilenetics.com [64.63.65.105] (may be forged)) by ece.cmu.edu (8.11.0/8.10.2) with SMTP id f511kWt12763 for ; Thu, 31 May 2001 21:46:32 -0400 (EDT) Received: from homa.asicdesigners.com ([10.1.1.5]) by homa.asicdesigners.com with Microsoft SMTPSVC(5.0.2195.2966); Thu, 31 May 2001 18:46:05 -0700 Received: by homa.asicdesigners.com (Microsoft Connector for POP3 Mailboxes 5.00.2195) with SMTP (Global POP3 Download) id MSG05312001-184557-6.MMD@asicdesigners.com; Thu, 31 May 2001 18:45:57 -0700 Received: from ece.cmu.edu (ECE.CMU.EDU [128.2.236.200]) by mail13.netservers.net (8.9.3/8.9.3) with ESMTP id SAA14541 for ; Thu, 31 May 2001 18:39:28 -0700 Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f4VNbbC04156 for ips-outgoing; Thu, 31 May 2001 19:37:37 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from homa.asicdesigners.com (ip-64-63-65-105.reverse.mobilenetics.com [64.63.65.105] (may be forged)) by ece.cmu.edu (8.11.0/8.10.2) with SMTP id f4VNbZt04147 for ; Thu, 31 May 2001 19:37:35 -0400 (EDT) Received: from homa.asicdesigners.com ([10.1.1.5]) by homa.asicdesigners.com with Microsoft SMTPSVC(5.0.2195.2966); Thu, 31 May 2001 16:36:33 -0700 Received: by homa.asicdesigners.com (Microsoft Connector for POP3 Mailboxes 5.00.2195) with SMTP (Global POP3 Download) id MSG05312001-163628-1.MMD@asicdesigners.com; Thu, 31 May 2001 16:36:28 -0700 Received: from ece.cmu.edu (ECE.CMU.EDU [128.2.236.200]) by mail14.netservers.net (8.9.3/8.9.3) with ESMTP id QAA20136 for ; Thu, 31 May 2001 16:33:18 -0700 Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f4VLmiR26337 for ips-outgoing; Thu, 31 May 2001 17:48:44 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from ntmail.qlc.com (216-231-2-8.qlc.com [216.231.2.8]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f4VLmht26333 for ; Thu, 31 May 2001 17:48:43 -0400 (EDT) Received: by ntmail.qlc.com with Internet Mail Service (5.5.2653.19) id ; Thu, 31 May 2001 14:47:37 -0700 Message-ID: <88B878891850D511BA6A0003470D72E426D7E8@ntmail.qlc.com> From: Chuck Micalizzi To: "'julian_satran@il.ibm.com'" Cc: ips@ece.cmu.edu Subject: iSCSI rev. 7 status Date: Thu, 31 May 2001 14:47:36 -0700 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2653.19) Content-Type: text/plain; charset="iso-8859-1" X-OriginalArrivalTime: 31 May 2001 23:36:33.0500 (UTC) FILETIME=[86A721C0:01C0EA2A] X-UIDL: fa3381b42e889146c2ea7b3c78b4af49 Sender: owner-ips@ece.cmu.edu Precedence: bulk Julian, Do you expect to be posting version 7 anytime soon? thanks chuck From owner-ips@ece.cmu.edu Thu May 31 23:02:15 2001 Received: from ece.cmu.edu (ECE.CMU.EDU [128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id XAA14308 for ; Thu, 31 May 2001 23:02:15 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f511v6K13521 for ips-outgoing; Thu, 31 May 2001 21:57:06 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from tiny-teddy.aarnet.edu.au (IDENT:root@tiny-teddy.aarnet.edu.au [203.21.37.30]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f511sYt13329 for ; Thu, 31 May 2001 21:54:34 -0400 (EDT) Received: from aarnet.edu.au (gt1.aarnet.adelaide.edu.au [129.127.180.236]) by tiny-teddy.aarnet.edu.au (8.11.0/8.11.0) with ESMTP id f511sPX01219; Fri, 1 Jun 2001 11:24:25 +0930 Message-ID: <3B16F5CC.5E7C6B31@aarnet.edu.au> Date: Fri, 01 Jun 2001 11:24:20 +0930 From: Glen Turner Organization: Australian Academic and Research Network X-Mailer: Mozilla 4.77 [en] (X11; U; Linux 2.4.4-ac11-mpls0.990-gdt1 i686) X-Accept-Language: en MIME-Version: 1.0 To: Black_David@emc.com CC: ips@ece.cmu.edu Subject: Re: More on iSCSI boot References: <0F31E5C394DAD311B60C00E029101A0708015630@corpmx9.isus.emc.com> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-ips@ece.cmu.edu Precedence: bulk Content-Transfer-Encoding: 7bit Black_David@emc.com wrote: > > David Robinson's messages support my inclination to reuse > DHCP option 17 (Root Path) by defining iSCSI syntax > for it. The "root path" is still useful if the image has been loaded using iSCSI or TFTP. An alternative would be to request an option for "image load protocol" and redefine DHCP's "sname" and "file" and options 66 "Server name" and option 67 "bootfile name" for interpretation with iSCSI. Without an allocated option, it might be possible to simply redefine the sname/66 and file/67 so that sname/66 beginning with "iscsi:" defines the iSCSI boot method. GENERAL COMMENTS ON DRAFT-02 I'm not at all keen on the mechanism in draft-*-boot-02. The DHCP option should be more generic, as in : and defined for iSCSI as iscsi:[:[:[:[:]]]] This would limit future options growth. Even better would be to define a DHCP option "boot protocol" and reuse the sname/66 and file/67 fields. In the absence of the "boot protocol" option TFTP would be assumed. Other issues with the draft: - if is a domain name then - the "name server" DHCP option SHOULD be provided - SHOULD be a FQDN unless the "domain name" option DHCP option is provided - may be better to be textual, as this allows variants on TCP (such as framing). Default value is "tcp" with "tcp-*", "udp" and "sctp" intended for future use. Syntax is: ::= ::= ::= ::= ::= ::= ::= - | . ::= a | b | ... | z The syntax of all textual options should be defined using IETF BNF rather than in the text. - The "LUN" field is a 16 byte hexadecimal representation of the 8-byte LU number in hex. does not allow for SCSI to redefine or expand the LUN. Although this is unlikely, I'd suggest text like: is the SCSI logical unit number in hexadecimal. The syntax is: ::= ::= ::= ::= 0 | 1 | ... | 9 | A | B | ... | F | a | b | ... | f ::= | If you do want a fixed-length field, then using BNF can replace the somewhat unwieldy text ::= : | ::= ::= ::= 0 | 1 | ... | 9 | A | B | ... | F | a | b | ... | f -- Glen Turner Network Engineer (08) 8303 3936 Australian Academic and Research Network glen.turner@aarnet.edu.au http://www.aarnet.edu.au/ -- The revolution will not be televised, it will be digitised From owner-ips@ece.cmu.edu Thu May 31 23:04:07 2001 Received: from ece.cmu.edu (ECE.CMU.EDU [128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id XAA14324 for ; Thu, 31 May 2001 23:04:07 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f511JSq11016 for ips-outgoing; Thu, 31 May 2001 21:19:28 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from c007.snv.cp.net (c007-h000.c007.snv.cp.net [209.228.33.206]) by ece.cmu.edu (8.11.0/8.10.2) with SMTP id f511JQt11011 for ; Thu, 31 May 2001 21:19:26 -0400 (EDT) Received: (cpmta 22702 invoked from network); 31 May 2001 18:19:20 -0700 Received: from adsl-63-202-160-80.dsl.snfc21.pacbell.net (HELO ljoy) (63.202.160.80) by smtp.telocity.com (209.228.33.206) with SMTP; 31 May 2001 18:19:20 -0700 X-Sent: 1 Jun 2001 01:19:20 GMT From: "Douglas Otis" To: "David Robinson" Cc: "Bernard Aboba" , , Subject: RE: iSCSI and secure boot Date: Thu, 31 May 2001 18:16:59 -0700 Message-ID: MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 (Normal) X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook IMO, Build 9.0.2416 (9.0.2911.0) In-Reply-To: <3B16C9D3.AB7FACBC@ebay.sun.com> X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4522.1200 Importance: Normal Sender: owner-ips@ece.cmu.edu Precedence: bulk Content-Transfer-Encoding: 7bit David, > While we have been using DHCP as the example of how hard it is to > securely boot over a network, switching to LDAP does nothing to > make it easier. Every initiator still needs to have a key stored > securely in nonvolitile memory and mechanisms to initially > create the key as well as update or revoke it. LDAP doesn't > solve this problem. LDAP does solve the problem. If you have a simple boot that is indeed stable, you do not have the issues of updating a key or how it is used. The initial image should utilize LDAP as a means of applying extensions that can be updated or revoked by means of modifying the information presented or pointed to by LDAP. Keep this boot simple and within a stable environment. That alone demands that new protocols not be used. You would want a highly stable server and protocol to base this simple boot managed in a secure manner. > Regardless of if you used TFTP, DHCP, or LDAP as the secondary boot > platform, it must be one that can be trusted and the trust must > come from the physical hardware and not over a network. The > same key management issues still exist. You already have initial image that can be trusted based on stored information. This information does not go away. Use this stored information as a shared secret merged with information within the boot image to then continue the next step of the process. > But the problem is that no matter how secure the LDAP server is, > the client must be secure or all bets are off. The only way to make > the client secure is to have some form of credential on the client > before any network traffic is initiated. Bernard was simply describing > that having a per-machine credential won't scale and the proposal > to have a per-interface credential will have orders of magnitude > worse scaling. Again LDAP can't be used to manage this as the > credential > must be on the client *before* it issues any requests. No, the problem is not that it will not scale. This is not analogous to a login. If you wanted a secure agent that could remotely modify these keys, it could be done, but I do not think that to be a good thing. What problem should this solve? It will not scale if each machine must be individually handled every few weeks when a protocol changes or an update goes awry. Security is simply a matter of bootstrapping using stored information to establish a trusted image to build an environment using simple routines in a highly stable environment. The next step can leverage this same information. To use a new protocol within the initial booting, there will be a great deal of difficulty created for all. Doug > -David From owner-ips@ece.cmu.edu Thu May 31 23:07:35 2001 Received: from ece.cmu.edu (ECE.CMU.EDU [128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id XAA14337 for ; Thu, 31 May 2001 23:07:35 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f511kb412781 for ips-outgoing; Thu, 31 May 2001 21:46:37 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from homa.asicdesigners.com (ip-64-63-65-105.reverse.mobilenetics.com [64.63.65.105] (may be forged)) by ece.cmu.edu (8.11.0/8.10.2) with SMTP id f511kXt12771 for ; Thu, 31 May 2001 21:46:33 -0400 (EDT) Received: from homa.asicdesigners.com ([10.1.1.5]) by homa.asicdesigners.com with Microsoft SMTPSVC(5.0.2195.2966); Thu, 31 May 2001 18:46:05 -0700 Received: by homa.asicdesigners.com (Microsoft Connector for POP3 Mailboxes 5.00.2195) with SMTP (Global POP3 Download) id MSG05312001-184557-8.MMD@asicdesigners.com; Thu, 31 May 2001 18:45:57 -0700 Received: from ece.cmu.edu (ECE.CMU.EDU [128.2.236.200]) by mail14.netservers.net (8.9.3/8.9.3) with ESMTP id SAA24900 for ; Thu, 31 May 2001 18:40:33 -0700 Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f4VNX2h03813 for ips-outgoing; Thu, 31 May 2001 19:33:02 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from homa.asicdesigners.com (ip-64-63-65-105.reverse.mobilenetics.com [64.63.65.105] (may be forged)) by ece.cmu.edu (8.11.0/8.10.2) with SMTP id f4VNX0t03809 for ; Thu, 31 May 2001 19:33:00 -0400 (EDT) Received: from homa.asicdesigners.com ([10.1.1.5]) by homa.asicdesigners.com with Microsoft SMTPSVC(5.0.2195.2966); Thu, 31 May 2001 16:30:29 -0700 Received: by homa.asicdesigners.com (Microsoft Connector for POP3 Mailboxes 5.00.2195) with SMTP (Global POP3 Download) id MSG05312001-163020-2.MMD@asicdesigners.com; Thu, 31 May 2001 16:30:20 -0700 Received: from ece.cmu.edu (ECE.CMU.EDU [128.2.236.200]) by mail14.netservers.net (8.9.3/8.9.3) with ESMTP id QAA19210 for ; Thu, 31 May 2001 16:14:42 -0700 Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f4VLief26005 for ips-outgoing; Thu, 31 May 2001 17:44:40 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from opus.ece.cmu.edu (root@OPUS.ECE.CMU.EDU [128.2.134.91]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f4VLidt26001 for ; Thu, 31 May 2001 17:44:39 -0400 (EDT) Received: from opus.ece.cmu.edu (bassoon@localhost [127.0.0.1]) by opus.ece.cmu.edu (8.9.3/8.9.3/SuSE Linux 8.9.3-0.1) with ESMTP id RAA09497 for ; Thu, 31 May 2001 17:44:39 -0400 Message-Id: <200105312144.RAA09497@opus.ece.cmu.edu> To: ips@ece.cmu.edu subject: iSNS Source Code Date: Thu, 31 May 2001 17:44:39 -0400 From: Dave Nagle X-OriginalArrivalTime: 31 May 2001 23:30:29.0062 (UTC) FILETIME=[AD6E4E60:01C0EA29] X-UIDL: 8f78b794ca63436fd27a8ffd5191b7b9 Sender: owner-ips@ece.cmu.edu Precedence: bulk Annoucement ----------- The Internet Storage Name Service (iSNS) source code is now available from the IPS web site. The client code is posted and the server code will be available in mid june. You'll find the code at: http://www.ece.cmu.edu/~ips/IPS_Projects/ips_projects.html Also, after tonight, there will be a mailing list to discuss source code issues. The list will be called ips_source and you can subscribe by consulting the ips web pages. From owner-ips@ece.cmu.edu Thu May 31 23:08:22 2001 Received: from ece.cmu.edu (ECE.CMU.EDU [128.2.236.200]) by ietf.org (8.9.1a/8.9.1a) with SMTP id XAA14357 for ; Thu, 31 May 2001 23:08:22 -0400 (EDT) Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f511kb112782 for ips-outgoing; Thu, 31 May 2001 21:46:37 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from homa.asicdesigners.com (ip-64-63-65-105.reverse.mobilenetics.com [64.63.65.105] (may be forged)) by ece.cmu.edu (8.11.0/8.10.2) with SMTP id f511kWt12767 for ; Thu, 31 May 2001 21:46:33 -0400 (EDT) Received: from homa.asicdesigners.com ([10.1.1.5]) by homa.asicdesigners.com with Microsoft SMTPSVC(5.0.2195.2966); Thu, 31 May 2001 18:46:05 -0700 Received: by homa.asicdesigners.com (Microsoft Connector for POP3 Mailboxes 5.00.2195) with SMTP (Global POP3 Download) id MSG05312001-184557-7.MMD@asicdesigners.com; Thu, 31 May 2001 18:45:57 -0700 Received: from ece.cmu.edu (ECE.CMU.EDU [128.2.236.200]) by mail14.netservers.net (8.9.3/8.9.3) with ESMTP id SAA24843 for ; Thu, 31 May 2001 18:39:24 -0700 Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f4VNX2803814 for ips-outgoing; Thu, 31 May 2001 19:33:02 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from homa.asicdesigners.com (ip-64-63-65-105.reverse.mobilenetics.com [64.63.65.105] (may be forged)) by ece.cmu.edu (8.11.0/8.10.2) with SMTP id f4VNWxt03803 for ; Thu, 31 May 2001 19:33:00 -0400 (EDT) Received: from homa.asicdesigners.com ([10.1.1.5]) by homa.asicdesigners.com with Microsoft SMTPSVC(5.0.2195.2966); Thu, 31 May 2001 16:30:28 -0700 Received: by homa.asicdesigners.com (Microsoft Connector for POP3 Mailboxes 5.00.2195) with SMTP (Global POP3 Download) id MSG05312001-163020-1.MMD@asicdesigners.com; Thu, 31 May 2001 16:30:20 -0700 Received: from ece.cmu.edu (ECE.CMU.EDU [128.2.236.200]) by mail14.netservers.net (8.9.3/8.9.3) with ESMTP id QAA19205 for ; Thu, 31 May 2001 16:14:31 -0700 Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id f4VLJES23945 for ips-outgoing; Thu, 31 May 2001 17:19:14 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from c007.snv.cp.net (c007-h011.c007.snv.cp.net [209.228.33.217]) by ece.cmu.edu (8.11.0/8.10.2) with SMTP id f4VLJCt23940 for ; Thu, 31 May 2001 17:19:12 -0400 (EDT) Received: (cpmta 10930 invoked from network); 31 May 2001 14:06:33 -0700 Received: from adsl-63-202-160-80.dsl.snfc21.pacbell.net (HELO ljoy) (63.202.160.80) by smtp.telocity.com (209.228.33.217) with SMTP; 31 May 2001 14:06:33 -0700 X-Sent: 31 May 2001 21:06:33 GMT From: "Douglas Otis" To: "Bernard Aboba" Cc: , Subject: RE: iSCSI and secure boot Date: Thu, 31 May 2001 14:04:13 -0700 Message-ID: MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 (Normal) X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook IMO, Build 9.0.2416 (9.0.2911.0) In-Reply-To: X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4522.1200 Importance: Normal X-OriginalArrivalTime: 31 May 2001 23:30:28.0734 (UTC) FILETIME=[AD3C41E0:01C0EA29] X-UIDL: ce708398070926097490020edc7fc975 Sender: owner-ips@ece.cmu.edu Precedence: bulk Content-Transfer-Encoding: 7bit Bernard, Julian, > > Can you describe a secure diskless boot that does not require individual > > setup of each? > > Some touching of each PC is required. However, the question is whether: It would be impossible to make a stable boot image that depended on revision 1 of a protocol about to hit the market. As such, it is misguided to consider using DHCP options to allow direct booting of iSCSI. Once security is considered, it becomes even more obvious that a two step process is required to allow needed flexibility and manageability. Once a management scheme is selected that is suitable for an enterprise deployment, LDAP or commercial equivalents are a good candidate. Attempting to place this management function on the iSCSI server complicates iSCSI and ensures no common method of promulgating management. In this respect, I differ from the opinion of David Black. David likes to construe an LDAP server as too difficult and wishes to fulfill this management need with various other inventions. > a. The setup requires setup of multiple credentials, or just > one. Note that BIS requires each PC to be configured with certificate of > the boot image signing authority. However, this does not provide client > authentication capabilities - so if you want to authenticate iSCSI or > DHCP, or anything else, then you'd need additional credentials. This second step could depend fully on TFTP and a DUA for LDAP. This second layer should divorce itself from iSCSI other than to establish an environment suitable for individualized booting using iSCSI. This seems quite possible to implement and to promote as a reference implementation once a schema is defined for LDAP. It would not require changes to existing system efforts but instead build upon them. The goal would be to provide a single simple boot that would *not* require change and yet allow the passing of variables and images required for the evolution of iSCSI. Just this initial boot would be accommodated by the DHCP, TFTP, and booting system. DHCP already provides a significant amount of flexibility. > b. Whether *per-interface* credentials are needed (e.g. authenticated DHCP > draft -16), or *per-machine*. Per-interface credentials require the > machine to be touched every time an interface is added or removed, not > just when the machine is shipped by the OEM. LDAP could provide the needed database required to provide the correct images in a highly flexible manner. As this type of server is often a critical server in an enterprise environment, it seems like a very safe choice. Julian's concern about not understanding this environment should encourage the use of existing schemes rather than reinventing new ones. Think of booting as a minimum of a two step process. A simple secure image coupled with information from a secure LDAP server to then obtain then next step. The only code that would need to be learned would be the DUA, and TFTP. Doug