From david.black@emc.com Sun Apr 3 07:33:11 2011 Return-Path: X-Original-To: storm@core3.amsl.com Delivered-To: storm@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id BB75F3A67F5 for ; Sun, 3 Apr 2011 07:33:11 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -106.472 X-Spam-Level: X-Spam-Status: No, score=-106.472 tagged_above=-999 required=5 tests=[AWL=0.126, BAYES_00=-2.599, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_MED=-4, USER_IN_WHITELIST=-100] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 0g3aPe-2GoWw for ; Sun, 3 Apr 2011 07:33:09 -0700 (PDT) Received: from mexforward.lss.emc.com (mexforward.lss.emc.com [128.222.32.20]) by core3.amsl.com (Postfix) with ESMTP id 8338B3A681B for ; Sun, 3 Apr 2011 07:33:09 -0700 (PDT) Received: from hop04-l1d11-si01.isus.emc.com (HOP04-L1D11-SI01.isus.emc.com [10.254.111.54]) by mexforward.lss.emc.com (Switch-3.4.3/Switch-3.4.3) with ESMTP id p33EYljj031155 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Sun, 3 Apr 2011 10:34:47 -0400 Received: from mailhub.lss.emc.com (mailhub.lss.emc.com [10.254.221.145]) by hop04-l1d11-si01.isus.emc.com (RSA Interceptor); Sun, 3 Apr 2011 10:34:40 -0400 Received: from mxhub03.corp.emc.com (mxhub03.corp.emc.com [10.254.141.105]) by mailhub.lss.emc.com (Switch-3.4.3/Switch-3.4.3) with ESMTP id p33EYbnL025296; Sun, 3 Apr 2011 10:34:37 -0400 Received: from mx14a.corp.emc.com ([169.254.1.143]) by mxhub03.corp.emc.com ([10.254.141.105]) with mapi; Sun, 3 Apr 2011 10:34:37 -0400 From: To: , Date: Sun, 3 Apr 2011 10:34:35 -0400 Thread-Topic: [storm] MPA Draft - Review Thread-Index: Acvxn6ZJgaQgCHvjST65raWYhZpz2wAbJWtw Message-ID: <7C4DFCE962635144B8FAE8CA11D0BF1E03E5EF1C51@MX14A.corp.emc.com> References: <7C4DFCE962635144B8FAE8CA11D0BF1E03D74C7183@MX14A.corp.emc.com> <7C4DFCE962635144B8FAE8CA11D0BF1E03E5E9530C@MX14A.corp.emc.com> <76DBE161893C324BA6D4BB507EE4C3849510935E96@IRVEXCHCCR02.corp.ad.broadcom.com> In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: acceptlanguage: en-US Content-Type: multipart/alternative; boundary="_000_7C4DFCE962635144B8FAE8CA11D0BF1E03E5EF1C51MX14Acorpemcc_" MIME-Version: 1.0 X-EMM-MHVC: 1 Cc: storm@ietf.org Subject: Re: [storm] MPA Draft - Review X-BeenThere: storm@ietf.org X-Mailman-Version: 2.1.9 Precedence: list List-Id: Storage Maintenance WG List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 03 Apr 2011 14:33:11 -0000 --_000_7C4DFCE962635144B8FAE8CA11D0BF1E03E5EF1C51MX14Acorpemcc_ Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable Looks good to me. Thanks, --David From: arkady kanevsky [mailto:arkady.kanevsky@gmail.com] Sent: Saturday, April 02, 2011 9:36 PM To: Hemal Shah Cc: Black, David; storm@ietf.org Subject: Re: [storm] MPA Draft - Review All, updated version 04 is attached. Hemal, Thanks for catching it. I had fixed the first issue. I had added reference to FPDU in the FULPDU de= finition for the second. David, Please, check to see that you comments are addressed. Steve and Robert, please, check that you comment is fixed correctly. Once I get positive feedback from all of you, I will submit the version. Thanks, Arkady On Thu, Mar 31, 2011 at 2:43 PM, Hemal Shah > wrote: I have some comments on -03 draft: 1. In section 10, it is written that "Enhanced MPA Initiator and Responde= r: If a responder receives an enhanced MPA message, it MUST respond with a= n unenhanced MPA message." I think it should be written that the responder = must respond with an enhanced MPA message. It appears like a typo to me. 2. I find the use of FULPDU confusing in this draft. RFC5044 does not def= ine term FULPDU. RFC5044 uses term FPDU to refer to Framed Protocol Data Un= it. I suggest that we use term FPDU instead of FULPDU in the draft. Hemal -----Original Message----- From: storm-bounces@ietf.org [mailto:storm-b= ounces@ietf.org] On Behalf Of david.black@emc.com Sent: Thursday, March 31, 2011 7:48 AM To: storm@ietf.org Subject: Re: [storm] MPA Draft - Review Importance: High The -03 version of the MPA draft has addressed all of the issues from my re= view, and . Unfortunately, I need some minor edits for clarity before I ca= n send this on to our AD with a publication request. Would the authors ple= ase submit a -04 version with the following two changes quickly. Section 9 (end) OLD The peer-to-peer negotiation for the RTR message follows the following order: Initiator -->: Sets Control Flags it is capable to send for RTR Responder <--: Sets Control Flags it is capable to receive for RTR Initiator -->: The first message send MUST be a negotiated RTR NEW The peer-to-peer negotiation for the RTR message follows the following order: Initiator -->: Sets Control Flags to indicate Initiator-supported forms = of RTR Responder <--: Sets Control Flags to indicate Responder-supported forms = of RTR Initiator -->: If at least one form of RTR is supported by both Initiato= r and Responder, then the first message sent MUST be an RTR using a form = supported by both the Initiator and Responder. Section 10 OLD In this case initiator CAN attempt to establish RDMA connection using unenhanced MPA protocol as defined in [RFC5044] and let ULP deal with ORD and IRD, and peer-to-peer negotiations. NEW In this case initiator MAY attempt to establish RDMA connection using ------------------------->^^^ unenhanced MPA protocol as defined in [RFC5044] if this protocol is compatible with the application, and let ULP deal with ORD and IRD, and peer-to-peer negotiations. Ordinarily, I'd write an RFC Editor Note for small changes like these, but = they're sufficiently critical to interoperability that I'd prefer to have a= new draft version that contains them. Thanks, --David > -----Original Message----- > From: Black, David > Sent: Wednesday, December 22, 2010 9:26 PM > To: storm@ietf.org > Cc: Black, David > Subject: MPA Draft - Review > > WG Last Call on this draft has run its course: > > Enhanced RDMA Connection Establishment > draft-ietf-storm-mpa-peer-connect-02 > > I've done my review as a WG chair (and the person who will be shepherding= this draft to the ADs and > IESG): > > - This draft is on the right track, but has open issues. > - Another version of the draft will be needed. > > Also, it would be greatly appreciated if a few people other than the auth= ors could take a look at > this draft. We have a very good author team on this draft, whose experti= se is beyond doubt, but > more eyes on this draft would help. > > [1] My primary concern is that Section 9 on interoperability is inadequat= e: > > An initiator SHOULD NOT use the Enhanced DDP Connection Establishment > formats or function codes when no enhanced functionality is desired. > > A responder SHOULD continue to accept the unenhanced connection > requests. > > The good news is that the first sentence is ok. > The bad news is that the second sentence has significant problems: > - It uses SHOULD instead of MUST. > - It doesn't lay out behavior for initiator and responder > Revision mixes. > IETF interoperability requirements are usually expressed with MUST, inclu= ding backwards > compatibility. If interop with unenhanced implementations is only a SHOU= LD, that will need a > convincing explanation. > > There are 3 Initiator/Responder cases that need attention (New/Old, Old/N= ew and New/New). I think > they lead to roughly the following: > > New/Old: > - Explain error or failure that the New Initiator will see because the Ol= d responder > doesn't support Revision 2 of the MPA protocol. > - Explain what the Initiator does when it sees that error or failure. Th= e > easiest approach is to always retry with Revision 1, but that won'= t work > if the Initiator has to send an RTR (that's the "convincing explan= ation" > for why backwards compatibility is not always possible). The resu= lt > might be two requirements: > - If the Initiator has data to send, it MUST retry with Revision 1= . > - If the Initiator has no data to send, and hence has to send an R= TR, > the connection setup fails, the TCP connection closes and = that > failure MUST to be reported to the application. > > Old/New: > - If a responder receives a Revision 1 message, it MUST respond with a Re= vision 1 message. > > New/New: > - If a responder receives a Revision 2 message, it MUST respond with a Re= vision 2 message. > > I found a few other concerns: > > [B]In Section 7, we need to get the listing of all the SCTP function code= s into one place. Either > repeat the definitions of codes 1-4 from RFC 5043, or create an IANA regi= stry in Section 10 and list > all 7 codes as its initial contents. > > [C] In Section 8, what happens if the responder sends an IRD or ORD value= that's different from the > corresponding initiator value? Is the responder allowed to increase the = value that was sent? An > important case to cover is that the initiator sends a valid value (e.g., = 0x2000) but the responder > returns the 0x3FFF value indicating that negotiation is not supported. A= lso, what is the behavior > of an IRD or ORD that is set to 0x0000? > > [D] In contrast, the Section 8 discussion of Control Flag functionality i= s in better shape. It > would be helpful to add a sentence or two indicating when the RTR occurs = (Request ->, <- Reply, RTR > ->), even though that is discussed earlier in the draft. Also, it's nece= ssary to state whether > negotiation of RTR functionality commits the Initiator to using an RTR (e= .g., suppose the initiator > negotiates control flags to allow an RTR and instead sends an FULPDU with= payload data after > receiving the Reply - is that ok or is it an error?). > > [E] Nit: In the definition of Control Flag A: ULPDU -> FULPDU > > Thanks, > --David > ---------------------------------------------------- > David L. Black, Distinguished Engineer > EMC Corporation, 176 South St., Hopkinton, MA 01748 > +1 (508) 293-7953 FAX: +1 (5= 08) 293-7786 > david.black@emc.com Mobile: +1 (978) 3= 94-7754 > ---------------------------------------------------- _______________________________________________ storm mailing list storm@ietf.org https://www.ietf.org/mailman/listinfo/storm _______________________________________________ storm mailing list storm@ietf.org https://www.ietf.org/mailman/listinfo/storm -- Cheers, Arkady Kanevsky --_000_7C4DFCE962635144B8FAE8CA11D0BF1E03E5EF1C51MX14Acorpemcc_ Content-Type: text/html; charset="us-ascii" Content-Transfer-Encoding: quoted-printable

Looks good to me.

 

Thanks,
--David

&nbs= p;

From: arkady kan= evsky [mailto:arkady.kanevsky@gmail.com]
Sent: Saturday, April 0= 2, 2011 9:36 PM
To: Hemal Shah
Cc: Black, David; storm@= ietf.org
Subject: Re: [storm] MPA Draft - Review

 

All,
updated version 04 is attached.=

Hemal,
Thanks for catching it.
I had fixed the first issue. I= had added reference to FPDU in the FULPDU definition for the second.
David,
Please, check to see that you comments are addressed.

St= eve and Robert,
please, check that you comment is fixed correctly.
Once I get positive feedback from all of you, I will submit the version.<= br>
Thanks,
Arkady

On Thu, Ma= r 31, 2011 at 2:43 PM, Hemal Shah <hemal@broadcom.com> wrote:

I have some comm= ents on -03 draft:

 =

  1. In section 10, it is written th= at "Enhanced MPA Initiator and Responder:  If a responder receive= s an enhanced MPA message, it MUST respond with an unenhanced MPA message.&= quot; I think it should be written that the responder must respond with an = enhanced MPA message. It appears like a typo to me.
    2. <= li class=3DMsoNormal style=3D'mso-margin-top-alt:auto;mso-margin-bottom-alt= :auto;mso-list:l0 level1 lfo1'>I find the use of FULPDU confusing in this draft. RFC5044 does no= t define term FULPDU. RFC5044 uses term FPDU to refer to Framed Protocol Da= ta Unit. I suggest that we use term FPDU instead of FULPDU in the draft.

 

Hemal =

 

-----Original Message-----
From: storm-bounces@ietf.org [mailto:storm-bounces@ietf.org] On B= ehalf Of david.bla= ck@emc.com
Sent: Thursday, March 31, 2011 7:48 AM
To: storm@ietf.org
Subject: Re:= [storm] MPA Draft - Review

Importance: High

 

The -03 version of the MPA draft has addressed all of the issues from my= review, and .  Unfortunately, I need some minor edits for clarity bef= ore I can send this on to our AD with a publication request.  Would th= e authors please submit a -04 version with the following two changes quickl= y.

 

<= p class=3DMsoNormal>S= ection 9 (end)

 

=

OLD

 

=

   The peer-to-peer negotiation for the RTR message fo= llows the

   following order:&nb= sp;   

   &n= bsp;            = ;            &n= bsp;  

   Initiator --= >: Sets Control Flags it is capable to send for RTR   &nb= sp; 

     &n= bsp;            = ;            &n= bsp;

   Responder <--: Sets C= ontrol Flags it is capable to receive for RTR  

          = ;            &n= bsp;        

=

   Initiator -->: The first message send MUST be a = negotiated RTR

 

=

NEW

 

=

   The peer-to-peer negotiation for the RTR message fo= llows the

   following order:&nb= sp;   

   &n= bsp;            = ;            &n= bsp;  

   Initiator --= >: Sets Control Flags to indicate Initiator-supported forms of RTR =      

  = ;            &n= bsp;            = ;    

   Res= ponder <--: Sets Control Flags to indicate Responder-supported forms of = RTR      

&n= bsp;            = ;            &n= bsp;     

 &= nbsp; Initiator -->: If at least one form of RTR is supported by both In= itiator and

     &= nbsp;  Responder, then the first message sent MUST be an RTR using a f= orm supported

     = ;   by both the Initiator and Responder.

 

Section 10=

 

= OLD=

      In

      this case initiator CAN attempt to est= ablish RDMA connection using

  &n= bsp;   unenhanced MPA protocol as defined in [RFC5044] and let UL= P deal

      with = ORD and IRD, and peer-to-peer negotiations.

 

NEW

 

      In

      this case ini= tiator MAY attempt to establish RDMA connection using

=

------------------------->^^^

      unenhanced MPA protocol as defined in [RFC5= 044] if this protocol is

   =      compatible with the application, and let ULP deal = with ORD and IRD,

    &= nbsp; and peer-to-peer negotiations.

 =

Ordinarily, I'd write an RFC Editor Note f= or small changes like these, but they're sufficiently critical to interoper= ability that I'd prefer to have a new draft version that contains them.

 

Thanks,=

--David

&n= bsp;

 

> -----Original Message-----

> Fro= m: Black, David

> Sent: Wednesday, Decem= ber 22, 2010 9:26 PM

<= span style=3D'font-size:10.0pt;font-family:Consolas'>> To: storm@ietf.org=

> Cc: Black, David

> S= ubject: MPA Draft - Review

> =

> WG Last Call on this draft has run its course:

>

>&n= bsp;            = ;     Enhanced RDMA Connection Establishment=

>        &= nbsp;          draft-ietf-stor= m-mpa-peer-connect-02

= >

> I've done my review as a WG chair (and the person who= will be shepherding this draft to the ADs and

<= div>

> IESG):

>

=

> - This draft is on the right track, but has open issues.

> - Another version of the draft will be n= eeded.

>

> Also, it would be greatly appreciated if a few people other than the= authors could take a look at

> this dra= ft.  We have a very good author team on this draft, whose expertise is= beyond doubt, but

> more eyes on this d= raft would help.

>

> [1] My primary concern is that Section 9 on interoperabili= ty is inadequate:

>

>    An initiator SHOULD NOT use the Enhance= d DDP Connection Establishment

> &n= bsp;  formats or function codes when no enhanced functionality is desi= red.

>

=

= >    A responder SHOULD continue to accept the unenhanced= connection

>    requests= .

>

>= ; The good news is that the first sentence is ok.

> The bad news is that the second sentence has significant proble= ms:

>      = ;  - It uses SHOULD instead of MUST.

<= p class=3DMsoNormal>&= gt;        - It doesn't lay out behavior= for initiator and responder

> &nbs= p;            &= nbsp; Revision mixes.

= > IETF interoperab= ility requirements are usually expressed with MUST, including backwards

> compatibility.  If interop with unen= hanced implementations is only a SHOULD, that will need a=

> convincing explanation.

>

> There are 3 Initiator/Responder= cases that need attention (New/Old, Old/New and New/New).  I think

> they lead to roughly the following:=

>

> New= /Old:

> - Explain error or failure that = the New Initiator will see because the Old responder

<= /div>

>        doesn't support Re= vision 2 of the MPA protocol.

> - Explai= n what the Initiator does when it sees that error or failure.  The

>       = easiest approach is to always retry with Revision 1, but that won't work

>      &nbs= p; if the Initiator has to send an RTR (that's the "convincing explana= tion"

>    &nbs= p;   for why backwards compatibility is not always possible).&nbs= p; The result

>    &= nbsp;   might be two requirements:

>        - If the Initiator has dat= a to send, it MUST retry with Revision 1.

<= p class=3DMsoNormal>&= gt;        - If the Initiator has no dat= a to send, and hence has to send an RTR,

&g= t;            &= nbsp;   the connection setup fails, the TCP connection closes and= that

>     &nb= sp;          failure MUST to b= e reported to the application.

>

> Old/New:

>= ; - If a responder receives a Revision 1 message, it MUST respond with a Re= vision 1 message.

>

> New/New:

> - If a resp= onder receives a Revision 2 message, it MUST respond with a Revision 2 mess= age.

>

=

= > I found a few other concerns:

>

> [B]In Section 7, we need to get the list= ing of all the SCTP function codes into one place.  Either<= /span>

> repeat the definitions of codes 1-4 from RFC 5043= , or create an IANA registry in Section 10 and list

> all 7 codes as its initial contents.

>

> [C] In Section 8, what h= appens if the responder sends an IRD or ORD value that's different from the=

> corresponding initiator value?  = Is the responder allowed to increase the value that was sent?  An=

> important case to cover is that the initia= tor sends a valid value (e.g., 0x2000) but the responder<= /p>

> returns the 0x3FFF value indicating that negotiation is = not supported.  Also, what is the behavior

=

> of an IRD or ORD that is set to 0x0000?

>

> [D] In contrast, the Sect= ion 8 discussion of Control Flag functionality is in better shape.  It=

> would be helpful to add a sentence or= two indicating when the RTR occurs (Request ->, <- Reply, RTR

> ->), even though that is discussed earlie= r in the draft.  Also, it's necessary to state whether

> negotiation of RTR functionality commits the Initiato= r to using an RTR (e.g., suppose the initiator

<= div>

> negotiates control flags to allow an RTR and instead sends an FULP= DU with payload data after

> receiving t= he Reply - is that ok or is it an error?).

=

= >

> [E] Nit: In the definition of Co= ntrol Flag A: ULPDU -> FULPDU

>

> Thanks,

=

= > --David

> -------------------------= ---------------------------

> David L. B= lack, Distinguished Engineer

> EMC Corpo= ration, 176 South St., Hopkinton, MA  01748

> +1= (508) 293-7953         &n= bsp;   FAX: +1 (508) 293-7786

> david.black@emc.com&n= bsp;       Mobile: +1 (978) 394-7754=

> ----------------------------------------------------

 

______= _________________________________________

<= p class=3DMsoNormal>s= torm mailing list

https://www.ietf.org/mailman/listinfo/storm

 

 

<= /div>

=
_______________________________________________
storm mailing liststorm@ietf.org
https://www.ietf.o= rg/mailman/listinfo/storm


=

--
Cheers,
Arkady Kanevsky

<= /div>= --_000_7C4DFCE962635144B8FAE8CA11D0BF1E03E5EF1C51MX14Acorpemcc_-- From arkady.kanevsky@gmail.com Sat Apr 2 18:34:15 2011 Return-Path: X-Original-To: storm@core3.amsl.com Delivered-To: storm@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id C831128C0D0 for ; Sat, 2 Apr 2011 18:34:15 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -3.598 X-Spam-Level: X-Spam-Status: No, score=-3.598 tagged_above=-999 required=5 tests=[BAYES_00=-2.599, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_LOW=-1] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id zXZYqMjd7-s7 for ; Sat, 2 Apr 2011 18:34:08 -0700 (PDT) Received: from mail-pw0-f44.google.com (mail-pw0-f44.google.com [209.85.160.44]) by core3.amsl.com (Postfix) with ESMTP id 8C2A13A68FD for ; Sat, 2 Apr 2011 18:34:08 -0700 (PDT) Received: by pwi5 with SMTP id 5so1255587pwi.31 for ; Sat, 02 Apr 2011 18:35:50 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:in-reply-to:references:date :message-id:subject:from:to:cc:content-type; bh=gJeDg4j0sKkxZGHtKwgxid9WToGbLu0CoGfRWvLJuEw=; b=uxIKXQgxb7XRaenb0wLKgUTzCoSPv+lUgksa1+yLi4hVgSrqC4YutUxTWV9wvdGV/w 1vYFPWoIUzornZ/lpKJpwKIw/nsGjuySmBJNOJ1HkWJDOI1Ar4VICwtBXHxMfCAF2st7 qHrmpb3MBJ2oC9ddIWrSEkKIQZhMNixR31Org= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; b=v2Xguzv5p+uw5zlvE6cYTL5fSbEDxZOrXsvo0G57eZp0yw2Cb/GyXKfLW69b8SxqWH GabNyVxVpNzxb8ldmiPrFktQL7Dr7+TMlxKJ4wxeue6EBnu+SoQB4HuNY1TemPYp9G/K NfJTj7BtkKqTe/I94+A6vd4wZYryRj1ZHYs5g= MIME-Version: 1.0 Received: by 10.143.26.4 with SMTP id d4mr4566766wfj.243.1301794549976; Sat, 02 Apr 2011 18:35:49 -0700 (PDT) Received: by 10.142.185.20 with HTTP; Sat, 2 Apr 2011 18:35:49 -0700 (PDT) In-Reply-To: <76DBE161893C324BA6D4BB507EE4C3849510935E96@IRVEXCHCCR02.corp.ad.broadcom.com> References: <7C4DFCE962635144B8FAE8CA11D0BF1E03D74C7183@MX14A.corp.emc.com> <7C4DFCE962635144B8FAE8CA11D0BF1E03E5E9530C@MX14A.corp.emc.com> <76DBE161893C324BA6D4BB507EE4C3849510935E96@IRVEXCHCCR02.corp.ad.broadcom.com> Date: Sat, 2 Apr 2011 21:35:49 -0400 Message-ID: From: arkady kanevsky To: Hemal Shah Content-Type: multipart/mixed; boundary=001636e0a9bfb8d0e9049ff9a967 X-Mailman-Approved-At: Sun, 03 Apr 2011 13:12:01 -0700 Cc: "storm@ietf.org" Subject: Re: [storm] MPA Draft - Review X-BeenThere: storm@ietf.org X-Mailman-Version: 2.1.9 Precedence: list List-Id: Storage Maintenance WG List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 03 Apr 2011 01:34:16 -0000 --001636e0a9bfb8d0e9049ff9a967 Content-Type: multipart/alternative; boundary=001636e0a9bfb8d0d5049ff9a965 --001636e0a9bfb8d0d5049ff9a965 Content-Type: text/plain; charset=ISO-8859-1 All, updated version 04 is attached. Hemal, Thanks for catching it. I had fixed the first issue. I had added reference to FPDU in the FULPDU definition for the second. David, Please, check to see that you comments are addressed. Steve and Robert, please, check that you comment is fixed correctly. Once I get positive feedback from all of you, I will submit the version. Thanks, Arkady On Thu, Mar 31, 2011 at 2:43 PM, Hemal Shah wrote: > I have some comments on -03 draft: > > > 1. In section 10, it is written that "Enhanced MPA Initiator and > Responder: If a responder receives an enhanced MPA message, it MUST respond > with an unenhanced MPA message." I think it should be written that the > responder must respond with an enhanced MPA message. It appears like a typo > to me. > 2. I find the use of FULPDU confusing in this draft. RFC5044 does not > define term FULPDU. RFC5044 uses term FPDU to refer to Framed Protocol Data > Unit. I suggest that we use term FPDU instead of FULPDU in the draft. > > > Hemal > > -----Original Message----- > From: storm-bounces@ietf.org [mailto:storm-bounces@ietf.org] > On Behalf Of david.black@emc.com > Sent: Thursday, March 31, 2011 7:48 AM > To: storm@ietf.org > Subject: Re: [storm] MPA Draft - Review > Importance: High > > The -03 version of the MPA draft has addressed all of the issues from my > review, and . Unfortunately, I need some minor edits for clarity before I > can send this on to our AD with a publication request. Would the authors > please submit a -04 version with the following two changes quickly. > > Section 9 (end) > > OLD > > The peer-to-peer negotiation for the RTR message follows the > following order: > > Initiator -->: Sets Control Flags it is capable to send for RTR > > Responder <--: Sets Control Flags it is capable to receive for RTR > > Initiator -->: The first message send MUST be a negotiated RTR > > NEW > > The peer-to-peer negotiation for the RTR message follows the > following order: > > Initiator -->: Sets Control Flags to indicate Initiator-supported forms > of RTR > > Responder <--: Sets Control Flags to indicate Responder-supported forms > of RTR > > Initiator -->: If at least one form of RTR is supported by both > Initiator and > Responder, then the first message sent MUST be an RTR using a form > supported > by both the Initiator and Responder. > > Section 10 > > OLD > In > this case initiator CAN attempt to establish RDMA connection using > unenhanced MPA protocol as defined in [RFC5044] and let ULP deal > with ORD and IRD, and peer-to-peer negotiations. > > NEW > > In > this case initiator MAY attempt to establish RDMA connection using > ------------------------->^^^ > unenhanced MPA protocol as defined in [RFC5044] if this protocol is > compatible with the application, and let ULP deal with ORD and IRD, > and peer-to-peer negotiations. > > Ordinarily, I'd write an RFC Editor Note for small changes like these, but > they're sufficiently critical to interoperability that I'd prefer to have a > new draft version that contains them. > > Thanks, > --David > > > > -----Original Message----- > > From: Black, David > > Sent: Wednesday, December 22, 2010 9:26 PM > > To: storm@ietf.org > > Cc: Black, David > > Subject: MPA Draft - Review > > > > WG Last Call on this draft has run its course: > > > > Enhanced RDMA Connection Establishment > > draft-ietf-storm-mpa-peer-connect-02 > > > > I've done my review as a WG chair (and the person who will be shepherding > this draft to the ADs and > > IESG): > > > > - This draft is on the right track, but has open issues. > > - Another version of the draft will be needed. > > > > Also, it would be greatly appreciated if a few people other than the > authors could take a look at > > this draft. We have a very good author team on this draft, whose > expertise is beyond doubt, but > > more eyes on this draft would help. > > > > [1] My primary concern is that Section 9 on interoperability is > inadequate: > > > > An initiator SHOULD NOT use the Enhanced DDP Connection Establishment > > formats or function codes when no enhanced functionality is desired. > > > > A responder SHOULD continue to accept the unenhanced connection > > requests. > > > > The good news is that the first sentence is ok. > > The bad news is that the second sentence has significant problems: > > - It uses SHOULD instead of MUST. > > - It doesn't lay out behavior for initiator and responder > > Revision mixes. > > IETF interoperability requirements are usually expressed with MUST, > including backwards > > compatibility. If interop with unenhanced implementations is only a > SHOULD, that will need a > > convincing explanation. > > > > There are 3 Initiator/Responder cases that need attention (New/Old, > Old/New and New/New). I think > > they lead to roughly the following: > > > > New/Old: > > - Explain error or failure that the New Initiator will see because the > Old responder > > doesn't support Revision 2 of the MPA protocol. > > - Explain what the Initiator does when it sees that error or failure. > The > > easiest approach is to always retry with Revision 1, but that > won't work > > if the Initiator has to send an RTR (that's the "convincing > explanation" > > for why backwards compatibility is not always possible). The > result > > might be two requirements: > > - If the Initiator has data to send, it MUST retry with Revision > 1. > > - If the Initiator has no data to send, and hence has to send an > RTR, > > the connection setup fails, the TCP connection closes and > that > > failure MUST to be reported to the application. > > > > Old/New: > > - If a responder receives a Revision 1 message, it MUST respond with a > Revision 1 message. > > > > New/New: > > - If a responder receives a Revision 2 message, it MUST respond with a > Revision 2 message. > > > > I found a few other concerns: > > > > [B]In Section 7, we need to get the listing of all the SCTP function > codes into one place. Either > > repeat the definitions of codes 1-4 from RFC 5043, or create an IANA > registry in Section 10 and list > > all 7 codes as its initial contents. > > > > [C] In Section 8, what happens if the responder sends an IRD or ORD value > that's different from the > > corresponding initiator value? Is the responder allowed to increase the > value that was sent? An > > important case to cover is that the initiator sends a valid value (e.g., > 0x2000) but the responder > > returns the 0x3FFF value indicating that negotiation is not supported. > Also, what is the behavior > > of an IRD or ORD that is set to 0x0000? > > > > [D] In contrast, the Section 8 discussion of Control Flag functionality > is in better shape. It > > would be helpful to add a sentence or two indicating when the RTR occurs > (Request ->, <- Reply, RTR > > ->), even though that is discussed earlier in the draft. Also, it's > necessary to state whether > > negotiation of RTR functionality commits the Initiator to using an RTR > (e.g., suppose the initiator > > negotiates control flags to allow an RTR and instead sends an FULPDU with > payload data after > > receiving the Reply - is that ok or is it an error?). > > > > [E] Nit: In the definition of Control Flag A: ULPDU -> FULPDU > > > > Thanks, > > --David > > ---------------------------------------------------- > > David L. Black, Distinguished Engineer > > EMC Corporation, 176 South St., Hopkinton, MA 01748 > > +1 (508) 293-7953 FAX: +1 (508) 293-7786 > > david.black@emc.com Mobile: +1 (978) 394-7754 > > ---------------------------------------------------- > > _______________________________________________ > storm mailing list > storm@ietf.org > https://www.ietf.org/mailman/listinfo/storm > > > > _______________________________________________ > storm mailing list > storm@ietf.org > https://www.ietf.org/mailman/listinfo/storm > > -- Cheers, Arkady Kanevsky --001636e0a9bfb8d0d5049ff9a965 Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable All,
updated version 04 is attached.

Hemal,
Thanks for catchin= g it.
I had fixed the first issue. I had added reference to FPDU in the = FULPDU definition for the second.

David,
Please, check to see tha= t you comments are addressed.

Steve and Robert,
please, check that you comment is fixed correctly.=

Once I get positive feedback from all of you, I will submit the ver= sion.

Thanks,
Arkady

On Thu, Ma= r 31, 2011 at 2:43 PM, Hemal Shah <hemal@broadcom.com> wrote:
I have some comments on -03 draft:
=A0
  1. In section 10, it is written that "Enhanced MPA Initiator and Resp= onder:=A0 If a responder receives an enhanced MPA message, it MUST respond = with an unenhanced MPA message." I think it should be written that the= responder must respond with an enhanced MPA message. It appears like a typo to me.
  2. I find the use of FULPDU con= fusing in this draft. RFC5044 does not define term FULPDU. RFC5044 uses ter= m FPDU to refer to Framed Protocol Data Unit. I suggest that we use term FP= DU instead of FULPDU in the draft.
=A0
Hemal
=A0
-----Original Message-----
From: storm-bou= nces@ietf.org [mailto:storm-bounces@ietf.org] On Behalf Of david.black@emc.com
Sent: Thursday, March 31, 2011 7:48 AM
To: storm@ietf.org<= br> Subject: Re: [storm] MPA Draft - Review
Importance: High
=A0
The -03 version of the MPA draft has addressed all of the issues from = my review, and .=A0 Unfortunately, I need some minor edits for clarity befo= re I can send this on to our AD with a publication request.=A0 Would the au= thors please submit a -04 version with the following two changes quickly.
=A0
Section 9 (end)
=A0
OLD
=A0
=A0=A0 The peer-to-peer negotiation for the RTR message follows the
=A0=A0 following order:=A0=A0=A0=A0
=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0= =A0=A0=A0=A0=A0=A0=A0=A0
=A0=A0 Initiator -->: Sets Control Flags it is capable to send for = RTR=A0=A0=A0=A0=A0
=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0= =A0=A0=A0=A0=A0=A0=A0=A0
=A0=A0 Responder <--: Sets Control Flags it is capable to receive f= or RTR=A0=A0
=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0= =A0=A0=A0=A0=A0=A0=A0=A0
=A0=A0 Initiator -->: The first message send MUST be a negotiated R= TR
=A0
NEW
=A0
=A0=A0 The peer-to-peer negotiation for the RTR message follows the
=A0=A0 following order:=A0=A0=A0=A0
=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0= =A0=A0=A0=A0=A0=A0=A0=A0
=A0=A0 Initiator -->: Sets Control Flags to indicate Initiator-supp= orted forms of RTR=A0=A0=A0=A0=A0=A0
=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0= =A0=A0=A0=A0=A0=A0=A0=A0
=A0=A0 Responder <--: Sets Control Flags to indicate Responder-supp= orted forms of RTR=A0=A0=A0=A0=A0=A0
=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0= =A0=A0=A0=A0=A0=A0=A0=A0
=A0=A0 Initiator -->: If at least one form of RTR is supported by b= oth Initiator and
=A0=A0=A0=A0=A0=A0=A0 Responder, then the first message sent MUST be a= n RTR using a form supported
=A0=A0=A0=A0=A0=A0=A0 by both the Initiator and Responder.
=A0
Section 10
=A0
OLD
=A0=A0=A0=A0=A0 In
=A0=A0=A0=A0=A0 this case initiator CAN attempt to establish RDMA conn= ection using
=A0=A0=A0=A0=A0 unenhanced MPA protocol as defined in [RFC5044] and le= t ULP deal
=A0=A0=A0=A0=A0 with ORD and IRD, and peer-to-peer negotiations.
=A0
NEW
=A0
=A0=A0=A0=A0=A0 In
=A0=A0=A0=A0=A0 this case initiator MAY attempt to establish RDMA conn= ection using
------------------------->^^^
=A0=A0=A0=A0=A0 unenhanced MPA protocol as defined in [RFC5044] if thi= s protocol is
=A0=A0=A0=A0=A0=A0=A0 compatible with the application, and let ULP dea= l with ORD and IRD,
=A0=A0=A0=A0=A0 and peer-to-peer negotiations.
=A0
Ordinarily, I'd write an RFC Editor Note for small changes like th= ese, but they're sufficiently critical to interoperability that I'd= prefer to have a new draft version that contains them.
=A0
Thanks,
--David
=A0
=A0
> -----Original Message-----
> From: Black, David
> Sent: Wednesday, December 22, 2010 9:26 PM
> Cc: Black, David
> Subject: MPA Draft - Review
>
> WG Last Call on this draft has run its course:
>
>=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 Enhanced RDMA = Connection Establishment
>=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 draft-ietf-= storm-mpa-peer-connect-02
>
> I've done my review as a WG chair (and the person who will be= shepherding this draft to the ADs and
> IESG):
>
> - This draft is on the right track, but has open issues.
> - Another version of the draft will be needed.
>
> Also, it would be greatly appreciated if a few people other than = the authors could take a look at
> this draft.=A0 We have a very good author team on this draft, who= se expertise is beyond doubt, but
> more eyes on this draft would help.
>
> [1] My primary concern is that Section 9 on interoperability is i= nadequate:
>
>=A0=A0=A0 An initiator SHOULD NOT use the Enhanced DDP Connection = Establishment
>=A0=A0=A0 formats or function codes when no enhanced functionality= is desired.
>
>=A0=A0=A0 A responder SHOULD continue to accept the unenhanced con= nection
>=A0=A0=A0 requests.
>
> The good news is that the first sentence is ok.
> The bad news is that the second sentence has significant problems= :
>=A0=A0=A0=A0=A0=A0=A0 - It uses SHOULD instead of MUST.
>=A0=A0=A0=A0=A0=A0=A0 - It doesn't lay out behavior for initia= tor and responder
>=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 Revision mixes.
> IETF interoperability requirements are usually expressed with MUS= T, including backwards
> compatibility.=A0 If interop with unenhanced implementations is o= nly a SHOULD, that will need a
> convincing explanation.
>
> There are 3 Initiator/Responder cases that need attention (New/Ol= d, Old/New and New/New).=A0 I think
> they lead to roughly the following:
>
> New/Old:
> - Explain error or failure that the New Initiator will see becaus= e the Old responder
>=A0=A0=A0=A0=A0=A0=A0 doesn't support Revision 2 of the MPA pr= otocol.
> - Explain what the Initiator does when it sees that error or fail= ure.=A0 The
>=A0=A0=A0=A0=A0=A0=A0 easiest approach is to always retry with Rev= ision 1, but that won't work
>=A0=A0=A0=A0=A0=A0=A0 if the Initiator has to send an RTR (that= 9;s the "convincing explanation"
>=A0=A0=A0=A0=A0=A0=A0 for why backwards compatibility is not alway= s possible).=A0 The result
>=A0=A0=A0=A0=A0=A0=A0 might be two requirements:
>=A0=A0=A0=A0=A0=A0=A0 - If the Initiator has data to send, it MUST= retry with Revision 1.
>=A0=A0=A0=A0=A0=A0=A0 - If the Initiator has no data to send, and = hence has to send an RTR,
>=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 the connection setup= fails, the TCP connection closes and that
>=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 failure MUST to be r= eported to the application.
>
> Old/New:
> - If a responder receives a Revision 1 message, it MUST respond w= ith a Revision 1 message.
>
> New/New:
> - If a responder receives a Revision 2 message, it MUST respond w= ith a Revision 2 message.
>
> I found a few other concerns:
>
> [B]In Section 7, we need to get the listing of all the SCTP funct= ion codes into one place.=A0 Either
> repeat the definitions of codes 1-4 from RFC 5043, or create an I= ANA registry in Section 10 and list
> all 7 codes as its initial contents.
>
> [C] In Section 8, what happens if the responder sends an IRD or O= RD value that's different from the
> corresponding initiator value?=A0 Is the responder allowed to inc= rease the value that was sent?=A0 An
> important case to cover is that the initiator sends a valid value= (e.g., 0x2000) but the responder
> returns the 0x3FFF value indicating that negotiation is not suppo= rted.=A0 Also, what is the behavior
> of an IRD or ORD that is set to 0x0000?
>
> [D] In contrast, the Section 8 discussion of Control Flag functio= nality is in better shape.=A0 It
> would be helpful to add a sentence or two indicating when the RTR= occurs (Request ->, <- Reply, RTR
> ->), even though that is discussed earlier in the draft.=A0 Al= so, it's necessary to state whether
> negotiation of RTR functionality commits the Initiator to using a= n RTR (e.g., suppose the initiator
> negotiates control flags to allow an RTR and instead sends an FUL= PDU with payload data after
> receiving the Reply - is that ok or is it an error?).
>
> [E] Nit: In the definition of Control Flag A: ULPDU -> FULPDU<= /div>
>
> Thanks,
> --David
> ----------------------------------------------------
> David L. Black, Distinguished Engineer
> EMC Corporation, 176 South St., Hopkinton, MA=A0 01748
> +1 = (508) 293-7953=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 FAX: +1 (508) 293-7786
> david.bl= ack@emc.com=A0=A0=A0=A0=A0=A0=A0 Mobile: +1 (978) 394-7754
> ----------------------------------------------------
=A0
_______________________________________________
storm mailing list
=A0
=A0

_______________________________________________
storm mailing list
storm@ietf.org
h= ttps://www.ietf.org/mailman/listinfo/storm




--
Cheers,
Arkady K= anevsky
--001636e0a9bfb8d0d5049ff9a965-- --001636e0a9bfb8d0e9049ff9a967 Content-Type: text/plain; charset=US-ASCII; name="draft-ietf-storm-mpa-peer-connect-04.txt" Content-Disposition: attachment; filename="draft-ietf-storm-mpa-peer-connect-04.txt" Content-Transfer-Encoding: base64 X-Attachment-Id: f_gm1anu8f0 CgoKU1RPUk0gICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg ICBBLiBLYW5ldnNreSwgRWQuCkludGVybmV0LURyYWZ0ICAgICAgICAgICAgICAgICAgICAgICAg ICAgICAgICAgICAgICAgICAgICAgICAgICAgIFZNd2FyZQpVcGRhdGVzOiA1MDQzLCA1MDQ0IChp ZiBhcHByb3ZlZCkgICAgICAgICAgICAgICAgICAgICAgICBDLiBCZXN0bGVyLCBFZC4KSW50ZW5k ZWQgc3RhdHVzOiBTdGFuZGFyZHMgVHJhY2sgICAgICAgICAgICAgICAgICAgICAgICAgICAgICBD b25zdWx0YW50CkV4cGlyZXM6IE9jdG9iZXIgNSwgMjAxMSAgICAgICAgICAgICAgICAgICAgICAg ICAgICAgICAgICAgICAgICBSLiBTaGFycAogICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgSW50ZWwKICAgICAgICAgICAgICAg ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICBTLiBXaXNl CiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgT3Bl biBHcmlkIENvbXB1dGluZwogICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg ICAgICAgICAgICAgICAgICAgIEFwcmlsIDMsIDIwMTEKCgogICAgICAgICAgICAgICAgIEVuaGFu Y2VkIFJETUEgQ29ubmVjdGlvbiBFc3RhYmxpc2htZW50CiAgICAgICAgICAgICAgICAgIGRyYWZ0 LWlldGYtc3Rvcm0tbXBhLXBlZXItY29ubmVjdC0wNAoKQWJzdHJhY3QKCiAgIFRoaXMgZG9jdW1l bnQgdXBkYXRlcyBbUkZDNTA0M10gYW5kIFtSRkM1MDQ0XSBieSBleHRlbmRpbmcgTVBBCiAgIG5l Z290aWF0aW9uIGZvciBSRE1BIENvbm5lY3Rpb24gZXN0YWJsaXNobWVudC4gIFRoZSBmaXJzdCBl eHRlbnNpb24KICAgZXh0ZW5kcyBbUkZDNTA0M10sIGVuYWJsaW5nIHBlZXItdG8tcGVlciBjb25u ZWN0aW9uIGVzdGFibGlzaG1lbnQKICAgb3ZlciBNUEEvVENQLiAgVGhlIHNlY29uZCBleHRlbnNp b24gZXh0ZW5kcyBib3RoIFtSRkM1MDQzXSBhbmQKICAgW1JGQzUwNDRdLCBieSBwcm92aWRpbmcg YW4gb3B0aW9uIGZvciBzdGFuZGFyZGl6ZWQgZXhjaGFuZ2Ugb2YgUkRNQS0KICAgbGF5ZXIgY29u bmVjdGlvbiBjb25maWd1cmF0aW9uLgoKU3RhdHVzIG9mIHRoaXMgTWVtbwoKICAgVGhpcyBJbnRl cm5ldC1EcmFmdCBpcyBzdWJtaXR0ZWQgaW4gZnVsbCBjb25mb3JtYW5jZSB3aXRoIHRoZQogICBw cm92aXNpb25zIG9mIEJDUCA3OCBhbmQgQkNQIDc5LgoKICAgSW50ZXJuZXQtRHJhZnRzIGFyZSB3 b3JraW5nIGRvY3VtZW50cyBvZiB0aGUgSW50ZXJuZXQgRW5naW5lZXJpbmcKICAgVGFzayBGb3Jj ZSAoSUVURikuICBOb3RlIHRoYXQgb3RoZXIgZ3JvdXBzIG1heSBhbHNvIGRpc3RyaWJ1dGUKICAg d29ya2luZyBkb2N1bWVudHMgYXMgSW50ZXJuZXQtRHJhZnRzLiAgVGhlIGxpc3Qgb2YgY3VycmVu dCBJbnRlcm5ldC0KICAgRHJhZnRzIGlzIGF0IGh0dHA6Ly9kYXRhdHJhY2tlci5pZXRmLm9yZy9k cmFmdHMvY3VycmVudC8uCgogICBJbnRlcm5ldC1EcmFmdHMgYXJlIGRyYWZ0IGRvY3VtZW50cyB2 YWxpZCBmb3IgYSBtYXhpbXVtIG9mIHNpeCBtb250aHMKICAgYW5kIG1heSBiZSB1cGRhdGVkLCBy ZXBsYWNlZCwgb3Igb2Jzb2xldGVkIGJ5IG90aGVyIGRvY3VtZW50cyBhdCBhbnkKICAgdGltZS4g IEl0IGlzIGluYXBwcm9wcmlhdGUgdG8gdXNlIEludGVybmV0LURyYWZ0cyBhcyByZWZlcmVuY2UK ICAgbWF0ZXJpYWwgb3IgdG8gY2l0ZSB0aGVtIG90aGVyIHRoYW4gYXMgIndvcmsgaW4gcHJvZ3Jl c3MuIgoKICAgVGhpcyBJbnRlcm5ldC1EcmFmdCB3aWxsIGV4cGlyZSBvbiBPY3RvYmVyIDUsIDIw MTEuCgpDb3B5cmlnaHQgTm90aWNlCgogICBDb3B5cmlnaHQgKGMpIDIwMTEgSUVURiBUcnVzdCBh bmQgdGhlIHBlcnNvbnMgaWRlbnRpZmllZCBhcyB0aGUKICAgZG9jdW1lbnQgYXV0aG9ycy4gIEFs bCByaWdodHMgcmVzZXJ2ZWQuCgogICBUaGlzIGRvY3VtZW50IGlzIHN1YmplY3QgdG8gQkNQIDc4 IGFuZCB0aGUgSUVURiBUcnVzdCdzIExlZ2FsCiAgIFByb3Zpc2lvbnMgUmVsYXRpbmcgdG8gSUVU RiBEb2N1bWVudHMKICAgKGh0dHA6Ly90cnVzdGVlLmlldGYub3JnL2xpY2Vuc2UtaW5mbykgaW4g ZWZmZWN0IG9uIHRoZSBkYXRlIG9mCgoKCkthbmV2c2t5LCBldCBhbC4gICAgICAgICBFeHBpcmVz IE9jdG9iZXIgNSwgMjAxMSAgICAgICAgICAgICAgICBbUGFnZSAxXQoMCkludGVybmV0LURyYWZ0 ICAgRW5oYW5jZWQgUkRNQSBDb25uZWN0aW9uIEVzdGFibGlzaG1lbnQgICAgICAgQXByaWwgMjAx MQoKCiAgIHB1YmxpY2F0aW9uIG9mIHRoaXMgZG9jdW1lbnQuICBQbGVhc2UgcmV2aWV3IHRoZXNl IGRvY3VtZW50cwogICBjYXJlZnVsbHksIGFzIHRoZXkgZGVzY3JpYmUgeW91ciByaWdodHMgYW5k IHJlc3RyaWN0aW9ucyB3aXRoIHJlc3BlY3QKICAgdG8gdGhpcyBkb2N1bWVudC4gIENvZGUgQ29t cG9uZW50cyBleHRyYWN0ZWQgZnJvbSB0aGlzIGRvY3VtZW50IG11c3QKICAgaW5jbHVkZSBTaW1w bGlmaWVkIEJTRCBMaWNlbnNlIHRleHQgYXMgZGVzY3JpYmVkIGluIFNlY3Rpb24gNC5lIG9mCiAg IHRoZSBUcnVzdCBMZWdhbCBQcm92aXNpb25zIGFuZCBhcmUgcHJvdmlkZWQgd2l0aG91dCB3YXJy YW50eSBhcwogICBkZXNjcmliZWQgaW4gdGhlIFNpbXBsaWZpZWQgQlNEIExpY2Vuc2UuCgoKVGFi bGUgb2YgQ29udGVudHMKCiAgIDEuICBJbnRyb2R1Y3Rpb24gLiAuIC4gLiAuIC4gLiAuIC4gLiAu IC4gLiAuIC4gLiAuIC4gLiAuIC4gLiAuIC4gLiAgMwogICAgIDEuMS4gIFN1bW1hcnkgb2YgY2hh bmdlcyBmcm9tIFJGQyA1MDQ0IC4gLiAuIC4gLiAuIC4gLiAuIC4gLiAuIC4gIDMKICAgICAxLjIu ICBTdW1tYXJ5IG9mIGNoYW5nZXMgZnJvbSBSRkMgNTA0MyAuIC4gLiAuIC4gLiAuIC4gLiAuIC4g LiAuICAzCiAgIDIuICBSZXF1aXJlbWVudHMgTGFuZ3VhZ2UgIC4gLiAuIC4gLiAuIC4gLiAuIC4g LiAuIC4gLiAuIC4gLiAuIC4gLiAgMwogICAzLiAgRGVmaW5pdGlvbnMgIC4gLiAuIC4gLiAuIC4g LiAuIC4gLiAuIC4gLiAuIC4gLiAuIC4gLiAuIC4gLiAuIC4gIDMKICAgNC4gIE1vdGl2YXRpb25z ICAuIC4gLiAuIC4gLiAuIC4gLiAuIC4gLiAuIC4gLiAuIC4gLiAuIC4gLiAuIC4gLiAuICA0CiAg ICAgNC4xLiAgRW5hYmxpbmcgTVBBIE1vZGUgIC4gLiAuIC4gLiAuIC4gLiAuIC4gLiAuIC4gLiAu IC4gLiAuIC4gLiAgNQogICAgIDQuMi4gIExhY2sgb2YgRXhwbGljaXQgUlRSIGluIE1QQSBSZXF1 ZXN0L1JlcGx5IEV4Y2hhbmdlIC4gLiAuIC4gIDUKICAgICA0LjMuICBMaW1pdGF0aW9ucyBvbiBV TFAgV29ya2Fyb3VuZCAgLiAuIC4gLiAuIC4gLiAuIC4gLiAuIC4gLiAuICA2CiAgICAgICA0LjMu MS4gIFRyYW5zcG9ydCBOZXV0cmFsIEFQSXMgLiAuIC4gLiAuIC4gLiAuIC4gLiAuIC4gLiAuIC4g LiAgNwogICAgICAgNC4zLjIuICBXb3JrL0NvbXBsZXRpb24gUXVldWUgQWNjb3VudGluZyAuIC4g LiAuIC4gLiAuIC4gLiAuIC4gIDcKICAgICAgIDQuMy4zLiAgSG9zdC1iYXNlZCBJbXBsZW1lbnRh dGlvbiBvZiBNUEEgRmVuY2luZyAuIC4gLiAuIC4gLiAuICA4CiAgICAgNC40LiAgU3RhbmRhcmRp emVkIFJETUEgUGFyYW1ldGVyIE5lZ290aWF0aW9uICAuIC4gLiAuIC4gLiAuIC4gLiAgOAogICA1 LiAgTVBBIENvbm5lY3Rpb24gU2V0dXAgLiAuIC4gLiAuIC4gLiAuIC4gLiAuIC4gLiAuIC4gLiAu IC4gLiAuIC4gIDkKICAgNi4gIEVuaGFuY2VkIE1QQSBSZXF1ZXN0L1JlcGx5IEZyYW1lcyAgLiAu IC4gLiAuIC4gLiAuIC4gLiAuIC4gLiAuIDEwCiAgIDcuICBFbmhhbmNlZCBTQ1RQIFNlc3Npb24g Q29udHJvbCBDaHVua3MgLiAuIC4gLiAuIC4gLiAuIC4gLiAuIC4gLiAxMQogICA4LiAgTVBBIEVy cm9yIFJlcG9ydGluZyAgLiAuIC4gLiAuIC4gLiAuIC4gLiAuIC4gLiAuIC4gLiAuIC4gLiAuIC4g MTMKICAgOS4gIEVuaGFuY2VkIFJETUEgQ29ubmVjdGlvbiBFc3RhYmxpc2htZW50IERhdGEgIC4g LiAuIC4gLiAuIC4gLiAuIDEzCiAgIDEwLiBJbnRlcm9wZXJhYmlsaXR5IC4gLiAuIC4gLiAuIC4g LiAuIC4gLiAuIC4gLiAuIC4gLiAuIC4gLiAuIC4gLiAxNQogICAxMS4gSUFOQSBDb25zaWRlcmF0 aW9ucyAgLiAuIC4gLiAuIC4gLiAuIC4gLiAuIC4gLiAuIC4gLiAuIC4gLiAuIC4gMTYKICAgMTIu IFNlY3VyaXR5IENvbnNpZGVyYXRpb25zICAuIC4gLiAuIC4gLiAuIC4gLiAuIC4gLiAuIC4gLiAu IC4gLiAuIDE2CiAgIDEzLiBBY2tub3dsZWRnZW1lbnRzIC4gLiAuIC4gLiAuIC4gLiAuIC4gLiAu IC4gLiAuIC4gLiAuIC4gLiAuIC4gLiAxNgogICAxNC4gUmVmZXJlbmNlcyAuIC4gLiAuIC4gLiAu IC4gLiAuIC4gLiAuIC4gLiAuIC4gLiAuIC4gLiAuIC4gLiAuIC4gMTYKICAgICAxNC4xLiBOb3Jt YXRpdmUgUmVmZXJlbmNlcyAuIC4gLiAuIC4gLiAuIC4gLiAuIC4gLiAuIC4gLiAuIC4gLiAuIDE2 CiAgICAgMTQuMi4gSW5mb3JtYXRpdmUgUmVmZXJlbmNlcyAuIC4gLiAuIC4gLiAuIC4gLiAuIC4g LiAuIC4gLiAuIC4gLiAxNgogICBBdXRob3JzJyBBZGRyZXNzZXMgLiAuIC4gLiAuIC4gLiAuIC4g LiAuIC4gLiAuIC4gLiAuIC4gLiAuIC4gLiAuIC4gMTcKCgoKCgoKCgoKCgoKCgoKS2FuZXZza3ks IGV0IGFsLiAgICAgICAgIEV4cGlyZXMgT2N0b2JlciA1LCAyMDExICAgICAgICAgICAgICAgIFtQ YWdlIDJdCgwKSW50ZXJuZXQtRHJhZnQgICBFbmhhbmNlZCBSRE1BIENvbm5lY3Rpb24gRXN0YWJs aXNobWVudCAgICAgICBBcHJpbCAyMDExCgoKMS4gIEludHJvZHVjdGlvbgoKICAgV2hlbiB1c2Vk IG92ZXIgVENQLCB0aGUgY3VycmVudCBSRERQIHN1aXRlIG9mIHByb3RvY29scyByZWxpZXMgb24K ICAgTWFya2VyIFBEVSBBbGlnbm1lbnQgKE1QQSkgW1JGQzUwNDRdIHByb3RvY29sIGZvciBib3Ro IGNvbm5lY3Rpb24KICAgZXN0YWJsaXNobWVudCBhbmQgZm9yIG1hcmtlcnMgZm9yIFRDUCBsYXll cmluZy4gIEN1cnJlbnRseSBNUEEgb25seQogICBzdXBwb3J0cyBhIGNsaWVudC1zZXJ2ZXIgbW9k ZWwgZm9yIGNvbm5lY3Rpb24gZXN0YWJsaXNobWVudCwgZm9yY2luZwogICBwZWVyLXRvLXBlZXIg YXBwbGljYXRpb25zIHRvIGludGVyYWN0IGFzIHRob3VnaCB0aGV5IGhhZCBhIGNsaWVudC8KICAg c2VydmVyIHJlbGF0aW9uc2hpcC4gIEluIGFkZGl0aW9uIG5lZ290aWF0aW9uIG9mIHNvbWUgb2Yg UmVtb3RlCiAgIERpcmVjdCBNZW1vcnkgQWNjZXNzIFByb3RvY29sIChSRE1BUCkgW1JGQzUwNDBd IHNwZWNpZmljIHBhcmFtZXRlcnMKICAgYXJlIGxlZnQgdG8gVUxQIG5lZ290aWF0aW9uLiAgUHJv dmlkaW5nIGFuIG9wdGlvbmFsIFVMUC1pbmRlcGVuZGVudAogICBmb3JtYXQgZm9yIGV4Y2hhbmdp bmcgdGhlc2UgcGFyYW1ldGVycyB3b3VsZCBiZSBvZiBiZW5lZml0IHRvCiAgIHRyYW5zcG9ydCBu ZXV0cmFsIFJETUEgYXBwbGljYXRpb25zLgoKMS4xLiAgU3VtbWFyeSBvZiBjaGFuZ2VzIGZyb20g UkZDIDUwNDQKCiAgIFRoaXMgZHJhZnQgZXh0ZW5kcyBbUkZDNTA0NF0gTVBBIGNvbm5lY3Rpb24g c2V0dXAgcHJvdG9jb2wuICBGaXJzdCwKICAgaXQgYWRkIGV4Y2hhbmdlIGFuZCBuZWdvdGlhdGlv biBvZiBtYXhpbXVtIG51bWJlciBvZiBSRE1BIFJlYWQKICAgSW5jb21pbmcgKElSRCkgYW5kIE91 dGdvaW5nIG1lc3NhZ2VzIChPUkQpLiAgU2Vjb25kLCBpdCBhZGRzIG9uZSBtb3JlCiAgIFJlYWR5 IHRvIFJlY2VpdmUgKFJUUikgZnJhbWUgZnJvbSByZXF1ZXN0b3IgdG8gcmVzcG9uZGVyIGFzIHRo ZSBsYXN0CiAgIG1lc3NhZ2Ugb2YgY29ubmVjdGlvbiBlc3RhYmxpc2htZW50IGFuZCBhZGRzIG5l Z290aWF0aW9uIG9mIFJUUiBmcmFtZQogICBtZXNzYWdlIHR5cGUgaW50byBNUEEgcmVxdWVzdC9y ZXNwb25zZSBmcmFtZXMuCgoxLjIuICBTdW1tYXJ5IG9mIGNoYW5nZXMgZnJvbSBSRkMgNTA0MwoK ICAgVGhpcyBkcmFmdCBleHRlbmRzIFtSRkM1MDQzXSBieSBhZGRpbmcgbmV3IEVuaGFuY2VkIFNl c3Npb24gQ29udHJvbAogICBDaHVua3MgdGhhdCBleHRlbmQgdGhlIGN1cnJlbnRseSBkZWZpbmVk IENodW5rcyB3aXRoIHRoZSBhZGRpdGlvbiBvZgogICBJUkQgYW5kIE9SRCBuZWdvdGlhdGlvbi4K CgoyLiAgUmVxdWlyZW1lbnRzIExhbmd1YWdlCgogICBUaGUga2V5IHdvcmRzICJNVVNUIiwgIk1V U1QgTk9UIiwgIlJFUVVJUkVEIiwgIlNIQUxMIiwgIlNIQUxMIE5PVCIsCiAgICJTSE9VTEQiLCAi U0hPVUxEIE5PVCIsICJSRUNPTU1FTkRFRCIsICJNQVkiLCBhbmQgIk9QVElPTkFMIiBpbiB0aGlz CiAgIGRvY3VtZW50IGFyZSB0byBiZSBpbnRlcnByZXRlZCBhcyBkZXNjcmliZWQgaW4gW1JGQzIx MTldLgoKCjMuICBEZWZpbml0aW9ucwoKICAgRlVMUERVOiAgRnJhbWVkIFVwcGVyIExheWVyIFBy b3RvY29sIFBEVS4gIFNlZSBGUERVIG9mIFtSRkM1MDQ0XS4KCiAgIENvbXBsZXRpb24gUXVldWUg KENRKTogIEEgY29uc3VtZXIgYWNjZXNzaWJsZSBxdWV1ZSB3aGVyZSB0aGUgUkRNQQogICAgICBk ZXZpY2UgcmVwb3J0cyBjb21wbGV0aW9ucyBvZiBXb3JrIFJlcXVlc3RzLiAgQSBDb25zdW1lciBp cyBhYmxlCiAgICAgIHRvIHJlYXAgY29tcGxldGlvbnMgZnJvbSBhIENRIHdpdGhvdXQgcmVxdWly aW5nIHBlciB0cmFuc2FjdGlvbgogICAgICBzdXBwb3J0IGZyb20gdGhlIGtlcm5lbCBvciBvdGhl ciBwcml2aWxlZ2VkIGVudGl0eS4gIFNlZSBbUkRNQUNdLgoKCgoKCgoKS2FuZXZza3ksIGV0IGFs LiAgICAgICAgIEV4cGlyZXMgT2N0b2JlciA1LCAyMDExICAgICAgICAgICAgICAgIFtQYWdlIDNd CgwKSW50ZXJuZXQtRHJhZnQgICBFbmhhbmNlZCBSRE1BIENvbm5lY3Rpb24gRXN0YWJsaXNobWVu dCAgICAgICBBcHJpbCAyMDExCgoKICAgQ29tcGxldGlvbiBRdWV1ZSBFbnRyeSAoQ1FFKTogIFRy YW5zcG9ydCBhbmQgZGV2aWNlIHNwZWNpZmljCiAgICAgIHJlcHJlc2VudGF0aW9uIG9mIGEgV29y ayBDb21wbGV0aW9uLiAgQSBDb21wbGV0aW9uIFF1ZXVlIGhvbGRzCiAgICAgIENRRXMuICBTZWUg W1JETUFDXS4KCiAgIEluYm91bmQgUkRNQSBSZWFkIFF1ZXVlIERlcHRoIChJUkQpOiAgVGhlIG1h eGltdW0gbnVtYmVyIG9mIGluY29taW5nCiAgICAgIG91dHN0YW5kaW5nIFJETUEgUmVhZCBSZXF1 ZXN0IE1lc3NhZ2VzIGFuIFJETUEgY29ubmVjdGlvbiBjYW4KICAgICAgaGFuZGxlLiAgU2VlIFtS RE1BQ10uCgogICBJUkQ6ICBTZWUgSW5ib3VuZCBSRE1BIFJlYWQgUXVldWUgRGVwdGguCgogICBP UkQ6ICBTZWUgT3V0Ym91bmQgUkRNQSBSZWFkIFF1ZXVlIERlcHRoLgoKICAgT3V0Ym91bmQgUkRN QSBSZWFkIFF1ZXVlIERlcHRoIChPUkQpOiAgVGhlIG1heGltdW0gbnVtYmVyIG9mCiAgICAgIG91 dHN0YW5kaW5nIFJETUEgUmVhZCBSZXF1ZXN0cyB0aGF0IGNhbiBiZSBpc3N1ZWQgZm9yIHRoZSBS RE1BCiAgICAgIGNvbm5lY3Rpb24uICBUaGlzIHNob3VsZCBiZSBsZXNzIHRoYW4gb3IgZXF1YWwg dG8gdGhlIHBlZXIncyBJUkQuCiAgICAgIFNlZSBbUkRNQUNdLgoKICAgUXVldWUgUGFpciAoUVAp OiAgVGhlIHRyYWRpdGlvbmFsIG5hbWUgZm9yIGEgbG9jYWwgRW5kcG9pbnQgaW4gYQogICAgICBb VklBXSBkZXJpdmVkIGxvY2FsIGludGVyZmFjZS4gIEEgUXVldWUgUGFpciBpcyB0aGUgc2V0IG9m IFdvcmsKICAgICAgUXVldWVzIGFzc29jaWF0ZWQgZXhjbHVzaXZlbHkgd2l0aCBhIHNpbmdsZSBF bmRwb2ludC4gIFRoZSBTZW5kCiAgICAgIFF1ZXVlIChTUSksIFJlY2VpdmUgUXVldWUgKFJRKSBh bmQgSW5ib3VuZCBSRE1BIFJlYWQgUXVldWUgKElSUSkKICAgICAgYXJlIGNvbnNpZGVyZWQgdG8g YmUgcGFydCBvZiB0aGUgUXVldWUgUGFpci4gIFRoZSBwb3RlbnRpYWxseQogICAgICBzaGFyZWQg Q29tcGxldGlvbiBRdWV1ZSAoQ1EpIGFuZCBTaGFyZWQgUmVjZWl2ZSBRdWV1ZSAoU1JRKSBhcmUK ICAgICAgbm90LiAgU2VlIFtSRE1BQ10uCgogICBTaGFyZWQgUmVjZWl2ZSBRdWV1ZShTUlEpOiAg QSBzaGFyZWQgcG9vbCBvZiBSZWNlaXZlIFdvcmsgUmVxdWVzdHMKICAgICAgcG9zdGVkIGJ5IHRo ZSBDb25zdW1lciB0aGF0IGNhbiBiZSBhbGxvY2F0ZWQgYnkgbXVsdGlwbGUgUkRNQQogICAgICBl bmRwb2ludHMgKFF1ZXVlIFBhaXIpLiAgU2VlIFtEQVBMXS4KCiAgIFdvcmsgUXVldWU6ICBBbiBl bGVtZW50IG9mIGEgW1ZJQV0gZGVyaXZlZCBsb2NhbCBpbnRlcmZhY2UgdGhhdAogICAgICBhbGxv d3MgdXNlci1zcGFjZSBhcHBsaWNhdGlvbnMgdG8gc3VibWl0IFdvcmsgUmVxdWVzdHMgZGlyZWN0 bHkgdG8KICAgICAgbmV0d29yayBoYXJkd2FyZS4gIFNwZWNpZmljIFdvcmsgUXVldWVzIGluY2x1 ZGUgdGhlIFNlbmQgUXVldWUKICAgICAgKFNRKSBmb3IgdHJhbnNtaXQgcmVxdWVzdHMsIFJlY2Vp dmUgUXVldWUgKFJRKSBmb3IgcmVjZWl2ZQogICAgICByZXF1ZXN0cyBzcGVjaWZpYyB0byBhIHNp bmdsZSBFbmRwb2ludCBhbmQgU2hhcmVkIFJlY2VpdmUgUXVldWVzCiAgICAgIChTUlFzKSBmb3Ig cmVjZWl2ZSByZXF1ZXN0cyB0aGF0IGNhbiBiZSBhbGxvY2F0ZWQgYnkgb25lIG9yIG1vcmUKICAg ICAgRW5kcG9pbnRzLiAgU2VlIFtSRE1BQ10uCgogICBXb3JrIFF1ZXVlIEVsZW1lbnQgKFdRRSk6 ICBUcmFuc3BvcnQgYW5kIGRldmljZSBzcGVjaWZpYwogICAgICByZXByZXNlbnRhdGlvbiBvZiBh IFdvcmsgUmVxdWVzdC4gIFNlZSBbUkRNQUNdCgogICBXb3JrIFJlcXVlc3Q6ICBBbiBlbGVtZW50 YXJ5IG9iamVjdCB1c2VkIGJ5IENvbnN1bWVycyB0byBlbnF1ZXVlIGEKICAgICAgcmVxdWVzdGVk IG9wZXJhdGlvbiAoV1FFcykgb250byBhIFdvcmsgUXVldWUuICBTZWUgW1JETUFDXS4KCgo0LiAg TW90aXZhdGlvbnMKCiAgIFRoZSBnb2FsIG9mIHRoaXMgZHJhZnQgaXMgdHdvZm9sZC4gIE9uZSBp cyB0byBleHRlbmQgc3VwcG9ydCBmcm9tIHRoZQogICBjdXJyZW50IGNsaWVudC1zZXJ2ZXIgbW9k ZWwgZm9yIFJETUEgY29ubmVjdGlvbiBzZXR1cCB0byBhIHBlZXItdG8tCgoKCkthbmV2c2t5LCBl dCBhbC4gICAgICAgICBFeHBpcmVzIE9jdG9iZXIgNSwgMjAxMSAgICAgICAgICAgICAgICBbUGFn ZSA0XQoMCkludGVybmV0LURyYWZ0ICAgRW5oYW5jZWQgUkRNQSBDb25uZWN0aW9uIEVzdGFibGlz aG1lbnQgICAgICAgQXByaWwgMjAxMQoKCiAgIHBlZXIgbW9kZWwuICBUaGUgc2Vjb25kIGlzIHRv IGFkZCBuZWdvdGlhdGlvbiBvZiBSRE1BIFJlYWQgcXVldWUgc2l6ZQogICBmb3IgYm90aCBzaWRl cyBvZiBhbiBSRE1BIGNvbm5lY3Rpb24uCgo0LjEuICBFbmFibGluZyBNUEEgTW9kZQoKICAgTVBB IGRlZmluZXMgZW5jb2Rpbmcgb2YgRERQIFNlZ21lbnRzIGluIEZVTFBEVXMgKEZyYW1lZCBVcHBl ciBMYXllcgogICBQcm90b2NvbCBQRFVzKS4gIEdlbmVyYXRpb24gb2YgRlVMUERVcyByZXF1aXJl cyB0aGUgYWJpbGl0eSB0bwogICBwZXJpb2RpY2FsbHkgaW5zZXJ0IE1QQSBNYXJrZXJzIGFuZCB0 byBnZW5lcmF0ZSB0aGUgTVBBIENSQy0zMmMgZm9yCiAgIGVhY2ggZnJhbWUuICBSZWNlcHRpb24g bWF5IHJlcXVpcmUgcGFyc2luZy9yZW1vdmluZyB0aGUgbWFya2VycyBhZnRlcgogICB1c2luZyB0 aGVtIHRvIGlkZW50aWZ5IE1QQSBGcmFtZSBib3VuZGFyaWVzLCBhbmQgdmFsaWRhdGlvbiBvZiB0 aGUKICAgTVBBLUNSQzMyYy4KCiAgIEEgbWFqb3IgZGVzaWduIG9iamVjdGl2ZSBmb3IgTVBBIHdh cyB0byBlbnN1cmUgdGhhdCB0aGUgcmVzdWx0aW5nIFRDUAogICBzdHJlYW0gd291bGQgYmUgYSBm dWxseSBjb21wbGlhbnQgVENQIHN0cmVhbSBmb3IgYW55IGFuZCBhbGwgVENQLQogICBhd2FyZSBt aWRkbGUtYm94ZXMuICBUaGUgY2hhbGxlbmdlIGlzIHRoYXQgd2hpbGUgb25seSBzb21lIFRDUAog ICBwYXlsb2FkIHN0cmVhbXMgYXJlIGEgdmFsaWQgc3RyZWFtIG9mIE1QQSBGVUxQRFVzLCBhbnkg c2VxdWVuY2Ugb2YKICAgYnl0ZXMgaXMgYSB2YWxpZCBUQ1AgcGF5bG9hZCBzdHJlYW0uICBUaGUg ZGV0ZXJtaW5hdGlvbiB0aGF0IGEgZ2l2ZW4KICAgc3RyZWFtIGlzIGluIGEgc3BlY2lmaWMgTVBB IG1vZGUgY2Fubm90IGJlIG1hZGUgYXQgdGhlIE1QQSBvciBUQ1AKICAgbGF5ZXIuICBUaGVyZWZv cmUgZW5hYmxpbmcgb2YgTVBBIG1vZGUgaXMgaGFuZGxlZCBieSB0aGUgVUxQLgoKICAgVGhlIE1Q QSBwcm90b2NvbCBjYW4gYmUgdmlld2VkIGFzIGhhdmluZyB0d28gcGFydHMuCgogICBvICBhIHNw ZWNpZmljYXRpb24gb2YgZ2VuZXJhdGlvbiBhbmQgcmVjZXB0aW9uIG9mIE1QQSBGVUxQRFVzLiAg VGhpcwogICAgICBpcyB1bmNoYW5nZWQgYnkgRW5oYW5jZWQgUkRNQSBDb25uZWN0aW9uIEVzdGFi bGlzaG1lbnQuCgogICBvICBhIHByZS1NUEEgZXhjaGFuZ2Ugb2YgbWVzc2FnZXMgdG8gZW5hYmxl IGEgc3BlY2lmaWMgTVBBIG1vZGUgZm9yCiAgICAgIHRoZSBUQ1AgY29ubmVjdGlvbi4gIEVuaGFu Y2VkIFJETUEgQ29ubmVjdGlvbiBFc3RhYmxpc2htZW50CiAgICAgIGV4dGVuZHMgdGhpcyBwcm90 b2NvbCB3aXRoIHR3byBuZXcgZmVhdHVyZXMuCgogICBJbiB0eXBpY2FsIGltcGxlbWVudGF0aW9u cywgZ2VuZXJhdGlvbiBhbmQgcmVjZXB0aW9uIG9mIE1QQSBGVUxQRFVzCiAgIGlzIGhhbmRsZWQg YnkgaGFyZHdhcmUuICBUaGUgZXhjaGFuZ2Ugb2YgdGhlIE1QQSBSZXF1ZXN0IGFuZCBSZXBseQog ICBmcmFtZXMgaXMgdGhlbiBoYW5kbGVkIGJ5IGhvc3Qgc29mdHdhcmUuICBBcyB3aWxsIGJlIGV4 cGxhaW5lZCwgdGhpcwogICBpbXBsZW1lbnRhdGlvbiBzcGxpdCBwcmV2ZW50cyBhcHBsaWNhdGlv bnMgZnJvbSB3b3JraW5nIGFyb3VuZCB0aGUKICAgY2xpZW50LXNlcnZlciBhc3N1bXB0aW9ucyBp biB0aGUgY3VycmVudCBNUEEgUmVxdWVzdC9SZXBseSBleGNoYW5nZS4KCjQuMi4gIExhY2sgb2Yg RXhwbGljaXQgUlRSIGluIE1QQSBSZXF1ZXN0L1JlcGx5IEV4Y2hhbmdlCgogICBUaGUgZXhjaGFu Z2Ugb2YgTVBBIFJlcXVlc3QgYW5kIFJlcGx5IG1lc3NhZ2VzIHRvIHBsYWNlIGEgVENQCiAgIGNv bm5lY3Rpb24gaW4gTVBBIG1vZGUgaXMgc3BlY2lmaWVkIGluIFtSRkM1MDQ0XS4gIFRoaXMgcHJv dG9jb2wKICAgcHJvdmlkZXMgbWFueSBiZW5lZml0cyB0byB0aGUgZGVzaWduIG9mIE1QQSBGVUxQ RFUgaGFyZHdhcmU6CgogICBvICBUaGUgVUxQIGlzIHJlc3BvbnNpYmxlIGZvciBzcGVjaWZ5aW5n IHRoZSBleGFjdCBNUEEgTW9kZSAoTWFya2VycwogICAgICBlbmFibGVkIG9yIGRpc2FibGVkLCBD UkMtMzJjIGVuYWJsZWQgb3Igc3VwcHJlc3NlZCkgYW5kIHRoZSBwb2ludAogICAgICBpbiB0aGUg VENQIHN0cmVhbXMgKGluYm91bmQgYW5kIG91dGJvdW5kKSB3aGVyZSBNUEEgZnJhbWVzIHdpbGwK ICAgICAgYmVnaW4uCgogICBvICBCZWZvcmUgdGhlIGZpcnN0IE1QQSBmcmFtZSBpcyB0cmFuc21p dHRlZCwgYWxsIHByZS1NUEEgbW9kZSBUQ1AKICAgICAgcGF5bG9hZCB3aWxsIGhhdmUgYmVlbiBh Y2tub3dsZWRnZWQgYnkgdGhlIHBlZXIuICBUaGVyZWZvcmUgaXQgaXMKCgoKS2FuZXZza3ksIGV0 IGFsLiAgICAgICAgIEV4cGlyZXMgT2N0b2JlciA1LCAyMDExICAgICAgICAgICAgICAgIFtQYWdl IDVdCgwKSW50ZXJuZXQtRHJhZnQgICBFbmhhbmNlZCBSRE1BIENvbm5lY3Rpb24gRXN0YWJsaXNo bWVudCAgICAgICBBcHJpbCAyMDExCgoKICAgICAgbmV2ZXIgbmVjZXNzYXJ5IHRvIGdlbmVyYXRl IGEgcmV0cmFuc21pc3Npb24gdGhhdCBtaXhlcyBwcmUtTVBBCiAgICAgIGFuZCBNUEEgcGF5bG9h ZC4KCiAgIG8gIEJlZm9yZSBNUEEgcmVjZXB0aW9uIGlzIGVuYWJsZWQsIGFsbCBpbmNvbWluZyBw cmUtTVBBIG1vZGUgVENQCiAgICAgIHBheWxvYWQgd2lsbCBoYXZlIGJlZW4gYWNrbm93bGVkZ2Vk LiAgVGhlcmVmb3JlIHRoZSBob3N0IHdpbGwKICAgICAgbmV2ZXIgcmVjZWl2ZSBhIFRDUCBzZWdt ZW50IHRoYXQgbWl4ZXMgcHJlLU1QQSBhbmQgTVBBIHBheWxvYWQuCgogICBUaGUgbGltaXRhdGlv biBvZiB0aGUgY3VycmVudCBNUEEgUmVxdWVzdC9SZXBseSBleGNoYW5nZSBpcyB0aGF0IGl0CiAg IGRvZXMgbm90IGRlZmluZSBhIFJlYWR5IHRvIFJlY2VpdmUgKFJUUikgbWVzc2FnZSB0aGF0IHRo ZSBhY3RpdmUgc2lkZQogICB3b3VsZCBzZW5kLCBzbyB0aGF0IHRoZSBwYXNzaXZlIHNpZGUgY2Fu IGtub3cgdGhhdCB0aGUgbGFzdCBub24tTVBBCiAgIHBheWxvYWQgKHRoZSBNUEEgUmVwbHkpIGhh ZCBiZWVuIHJlY2VpdmVkLgoKICAgSW5zdGVhZCwgdGhlIHJvbGUgb2YgYW4gUlRSIG1lc3NhZ2Ug aXMgcGlnZ3ktYmFja2VkIG9uIHRoZSBmaXJzdCBNUEEKICAgRlVMUERVIHNlbnQgYnkgdGhlIGFj dGl2ZSBzaWRlLiAgVGhpcyBpcyBhY3R1YWxseSBhIHZhbHVhYmxlCiAgIG9wdGltaXphdGlvbiBm b3IgYWxsIGFwcGxpY2F0aW9ucyB0aGF0IGZpdCB0aGUgY2xhc3NpYyBjbGllbnQvc2VydmVyCiAg IG1vZGVsLiAgVGhlIGNsaWVudCBvbmx5IGluaXRpYXRlcyB0aGUgY29ubmVjdGlvbiB3aGVuIGl0 IGhhcyBhCiAgIHJlcXVlc3QgdG8gc2VuZCB0byB0aGUgc2VydmVyLCBhbmQgdGhlIHNlcnZlciBo YXMgbm90aGluZyB0byBzZW5kCiAgIHVudGlsIGl0IGhhcyByZWNlaXZlZCBhbmQgcHJvY2Vzc2Vk IHRoZSBjbGllbnQgcmVxdWVzdC4KCiAgIEV2ZW4gYXBwbGljYXRpb25zIHdoZXJlIHRoZSBzZXJ2 ZXIgc2VuZHMgc29tZSBjb25maWd1cmF0aW9uIGRhdGEKICAgaW1tZWRpYXRlbHkgY2FuIGVhc2ls eSBzZW5kIHRoZSBzYW1lIGluZm9ybWF0aW9uIGFzIGFwcGxpY2F0aW9uCiAgIHByaXZhdGUgZGF0 YSBpbiB0aGUgTVBBIFJlcGx5LiAgU28gdGhlIGN1cnJlbnRseSBkZWZpbmVkIGV4Y2hhbmdlCiAg IHdvcmtzIGZvciBhbG1vc3QgYWxsIGFwcGxpY2F0aW9ucy4KCiAgIE1hbnkgcGVlci10by1wZWVy IGFwcGxpY2F0aW9ucywgZXNwZWNpYWxseSB0aG9zZSBpbnZvbHZpbmcgY2x1c3RlcgogICBjYWxj dWxhdGlvbnMgKGZyZXF1ZW50bHkgdXNpbmcgTVBJIFtVc2luZ01QSV0sIG9yIFtSRFNdKSwgaGF2 ZSBubwogICBuYXR1cmFsIGNsaWVudCBvciBzZXJ2ZXIgcm9sZXMgKFtQUE1QSV0sIFtPcGVuTVBd KS4gIFR5cGljYWxseSBvbmUKICAgbWVtYmVyIG9mIHRoZSBjbHVzdGVyIGlzIGFyYml0cmFyaWx5 IHNlbGVjdGVkIHRvIGluaXRpYXRlIHRoZQogICBjb25uZWN0aW9uIHdoZW4gdGhlIGRpc3RyaWJ1 dGVkIHRhc2sgaXMgbGF1bmNoZWQsIHdoaWxlIHRoZSBvdGhlcgogICBhY2NlcHRzIGl0LiAgQXQg c3RhcnR1cCB0aW1lLCBob3dldmVyLCB0aGVyZSBpcyBubyB3YXkgdG8gcHJlZGljdAogICB3aGlj aCBub2RlIHdpbGwgaGF2ZSB0aGUgZmlyc3QgbWVzc2FnZSB0byBhY3R1YWxseSBzZW5kLgogICBF c3RhYmxpc2hpbmcgdGhlIGNvbm5lY3Rpb25zIGltbWVkaWF0ZWx5LCBob3dldmVyLCBpcyB2YWx1 YWJsZQogICBiZWNhdXNlIGl0IHJlZHVjZXMgbGF0ZW5jeSBvbmNlIHJlc3VsdHMgYXJlIHJlYWR5 IHRvIHRyYW5zbWl0IGFuZCBpdAogICB2YWxpZGF0ZXMgY29ubmVjdGl2aXR5IHRocm91Z2hvdXQg dGhlIGNsdXN0ZXIuCgogICBUaGUgbGFjayBvZiBhbiBleHBsaWNpdCBSVFIgbWVzc2FnZSBpbiB0 aGUgTVBBIFJlcXVlc3QvUmVwbHkgZXhjaGFuZ2UKICAgZm9yY2VzIGFsbCBhcHBsaWNhdGlvbnMg dG8gaGF2ZSBhIGZpcnN0IG1lc3NhZ2UgZnJvbSB0aGUgY29ubmVjdGlvbgogICBpbml0aWF0b3Is IHdoZXRoZXIgdGhpcyBtYXRjaGVzIHRoZSBhcHBsaWNhdGlvbiBjb21tdW5pY2F0aW9uIG1vZGVs CiAgIG9yIG5vdC4KCjQuMy4gIExpbWl0YXRpb25zIG9uIFVMUCBXb3JrYXJvdW5kCgogICBUaGUg cmVxdWlyZW1lbnQgdGhhdCB0aGUgUkRNQSBjb25uZWN0aW9uIGluaXRpYXRvciBzZW5kcyB0aGUg Zmlyc3QKICAgbWVzc2FnZSBkb2VzIG5vdCBhcHBlYXIgdG8gYmUgdGhhdCBvbmVyb3VzIG9uIGZp cnN0IGV4YW1pbmF0aW9uLiAgVGhlCiAgIG5hdHVyYWwgcXVlc3Rpb24gaXMgd2h5IHRoZSBhcHBs aWNhdGlvbiBsYXllciB3b3VsZCBub3Qgc2ltcGx5CiAgIGdlbmVyYXRlIGEgIm5vcCIgbWVzc2Fn ZSB3aGVuIHRoZXJlIHdhcyBubyBvdGhlciBtZXNzYWdlIHRvIHN1Ym1pdC4KCiAgIFRoZXJlIGFy ZSB0aHJlZSBmYWN0b3JzIHRoYXQgbWFrZSB0aGlzIHdvcmthcm91bmQgdW5zdWl0YWJsZSBmb3Ig bWFueQoKCgpLYW5ldnNreSwgZXQgYWwuICAgICAgICAgRXhwaXJlcyBPY3RvYmVyIDUsIDIwMTEg ICAgICAgICAgICAgICAgW1BhZ2UgNl0KDApJbnRlcm5ldC1EcmFmdCAgIEVuaGFuY2VkIFJETUEg Q29ubmVjdGlvbiBFc3RhYmxpc2htZW50ICAgICAgIEFwcmlsIDIwMTEKCgogICBwZWVyLXRvLXBl ZXIgYXBwbGljYXRpb25zLgoKICAgbyAgVHJhbnNwb3J0IE5ldXRyYWwgQVBJcy4KCiAgIG8gIFdv cmsvQ29tcGxldGlvbiBRdWV1ZSBBY2NvdW50aW5nLgoKICAgbyAgSG9zdC1iYXNlZCBpbXBsZW1l bnRhdGlvbiBvZiBNUEEgRmVuY2luZy4KCjQuMy4xLiAgVHJhbnNwb3J0IE5ldXRyYWwgQVBJcwoK ICAgTWFueSBvZiB0aGVzZSBhcHBsaWNhdGlvbnMgYWNjZXNzIFJETUEgc2VydmljZXMgdXNpbmcg YSB0cmFuc3BvcnQKICAgbmV1dHJhbCBBUEkgc3VjaCBhcyBbREFQTF0gb3IgW09GQV0uICBPbmx5 IE1QQSBoYXMgYSBmaXJzdCBtZXNzYWdlCiAgIHJlcXVpcmVtZW50LiAgT3RoZXIgUkRNQSB0cmFu c3BvcnRzLCBpbmNsdWRpbmcgU0NUUCBhbmQgSW5maW5pQmFuZCwKICAgZG8gbm90LgoKICAgQXBw bGljYXRpb24gb3IgbWlkZGxld2FyZSBjb21tdW5pY2F0aW9ucyBjYW4gYmUgZXhwcmVzc2VkIGFz CiAgIHRyYW5zcG9ydCBuZXV0cmFsIFJETUEgb3BlcmF0aW9ucywgYWxsb3dpbmcgbG93ZXIgc29m dHdhcmUgbGF5ZXJzIHRvCiAgIHRyYW5zbGF0ZSB0byB0cmFuc3BvcnQgYW5kIGRldmljZSBzcGVj aWZpY3MuICBIYXZpbmcgYSBkaXN0aW5jdCBleHRyYQogICBtZXNzYWdlIHRoYXQgaXMgcmVxdWly ZWQgb25seSBmb3Igb25lIHRyYW5zcG9ydCB1bmRlcm1pbmVzIHRoZQogICBhcHBsaWNhdGlvbidz IGdvYWwgb2YgYmVpbmcgdHJhbnNwb3J0IG5ldXRyYWwuCgo0LjMuMi4gIFdvcmsvQ29tcGxldGlv biBRdWV1ZSBBY2NvdW50aW5nCgogICBSRE1BIGxvY2FsIEFQSXMgY29udmVudGlvbmFsbHkgdXNl IHdvcmsgcXVldWVzIHRvIHN1Ym1pdCByZXF1ZXN0cwogICAod29yayBxdWV1ZSBlbGVtZW50cyBv ciBXUUVzKSBhbmQgdG8gYXN5bmNocm9ub3VzbHkgcmVjZWl2ZQogICBjb21wbGV0aW9ucyAoaW4g Y29tcGxldGlvbiBxdWV1ZXMgb3IgQ1FzKS4KCiAgIEVhY2ggd29yayByZXF1ZXN0IGNhbiBnZW5l cmF0ZSBhIGNvbXBsZXRpb24gcXVldWUgZW50cnkgKENRRSkuCiAgIENvbXBsZXRpb25zIGZvciBz dWNjZXNzZnVsIHRyYW5zbWl0IHdvcmsgcmVxdWVzdHMgYXJlIGZyZXF1ZW50bHkKICAgc3VwcHJl c3NlZCwgYnV0IHRoZSBjb21wbGV0aW9uIHF1ZXVlIGNhcGFjaXR5IG11c3QgYWNjb3VudCBmb3Ig dGhlCiAgIHBvc3NpYmlsaXR5IHRoYXQgZWFjaCB3aWxsIGNvbXBsZXRlIGluIGVycm9yLiAgQSBj b21wbGV0aW9uIHF1ZXVlIGNhbgogICByZWNlaXZlIGNvbXBsZXRpb25zIGZyb20gbXVsdGlwbGUg d29yayBxdWV1ZXMuCgogICBDb21wbGV0aW9uIFF1ZXVlcyBhcmUgZGVmaW5lZCBzbyBhcyB0byBh bGxvdyBoYXJkd2FyZSBSRE1BCiAgIGltcGxlbWVudGF0aW9ucyB0byBnZW5lcmF0ZSBDUUVzIGRp cmVjdGx5IHRvIGEgdXNlci1zcGFjZSBtYXBwZWQKICAgYnVmZmVyLiAgVGhpcyBlbmFibGVzIGEg dXNlci1zcGFjZSBSRE1BIGNvbnN1bWVyIHRvIHJlYXAgY29tcGxldGlvbnMKICAgd2l0aG91dCBy ZXF1aXJpbmcga2VybmVsIGludGVydmVudGlvbi4KCiAgIEEgaGFyZHdhcmUgUkRNQSBpbXBsZW1l bnRhdGlvbiBjYW5ub3QgcmVhc29uYWJseSB3YWl0IGZvciBhbgogICBhdmFpbGFibGUgc2xvdCBp biB0aGUgY29tcGxldGlvbiBxdWV1ZS4gIFRoZSBxdWV1ZSBtdXN0IGJlIHNpemVkIHN1Y2gKICAg dGhhdCBhbiBvdmVyZmxvdyB3aWxsIG5vdCBvY2N1ci4gIFdoZW4gYW4gb3ZlcmZsb3cgZG9lcyBv Y2N1ciBpdCBpcwogICBjb25zaWRlcmVkIGNhdGFzdHJvcGhpYyBhbmQgd2lsbCB0eXBpY2FsbHkg cmVxdWlyZSB0ZWFyaW5nIGRvd24gYWxsCiAgIFJETUEgY29ubmVjdGlvbnMgdXNpbmcgdGhhdCBD US4KCiAgIFRoaXMgc3R5bGUgb2YgaW50ZXJmYWNlIGlzIHZlcnkgZWZmaWNpZW50LCBidXQgcGxh Y2VzIGEgYnVyZGVuIG9uIHRoZQogICBhcHBsaWNhdGlvbiB0byBwcm9wZXJseSBzaXplIGVhY2gg Q29tcGxldGlvbiBRdWV1ZSB0byBtYXRjaCB0aGUgV29yawogICBRdWV1ZXMgdGhhdCBmZWVkIGl0 LgoKCgoKS2FuZXZza3ksIGV0IGFsLiAgICAgICAgIEV4cGlyZXMgT2N0b2JlciA1LCAyMDExICAg ICAgICAgICAgICAgIFtQYWdlIDddCgwKSW50ZXJuZXQtRHJhZnQgICBFbmhhbmNlZCBSRE1BIENv bm5lY3Rpb24gRXN0YWJsaXNobWVudCAgICAgICBBcHJpbCAyMDExCgoKICAgV2hpbGUgdGhlIGZv cm1hdCBvZiBib3RoIFdRRXMgYW5kIENRRXMgaXMgdHJhbnNwb3J0IGFuZCBkZXZpY2UKICAgZGVw ZW5kZW50LCBhIHRyYW5zcG9ydCBuZXV0cmFsIEFQSSBjYW4gZGVhbCB3aXRoIFdRRXMgYW5kIENR RXMgYXMKICAgYWJzdHJhY3QgdHJhbnNwb3J0IGFuZCBkZXZpY2UgbmV1dHJhbCBvYmplY3RzLiAg VGhlcmVmb3JlIHRoZSBudW1iZXIKICAgb2YgV1FFcyBhbmQgQ1FFcyByZXF1aXJlZCBmb3IgYW4g YXBwbGljYXRpb24gY2FuIGJlIHRyYW5zcG9ydCBhbmQKICAgZGV2aWNlIG5ldXRyYWwuCgogICBU aGUgY2FwYWNpdHkgb2YgdGhlIHdvcmsgcXVldWVzIGFuZCBjb21wbGV0aW9uIHF1ZXVlcyBjYW4g YmUKICAgY2FsY3VsYXRlZCBpbiBhbiBhYnN0cmFjdCB0cmFuc3BvcnQvZGV2aWNlIG5ldXRyYWwg ZmFzaGlvbi4gIExvd2VyCiAgIGxheWVycyBvZiB0aGUgcHJvdG9jb2wgc3RhY2sgY2Fubm90IGRp c3J1cHQgdGhlc2UgY2FsY3VsYXRpb25zIGJ5CiAgIGluc2VydGluZyBhIGR1bW15ICJub3AiIFdv cmsgUmVxdWVzdCBhbmQgZmlsdGVyaW5nIG91dCB0aGUgbWF0Y2hpbmcKICAgY29tcGxldGlvbi4g IFRoZSBsb3dlciBsYXllciBkb2VzIG5vdCBrbm93IHRoZSB1c2FnZSBtb2RlbCBvbiB3aGljaAog ICB0aGUgcXVldWUgc2l6ZXMgYXJlIGJ1aWx0LCBub3IgZG9lcyBpdCBrbm93IGhvdyBmcmVxdWVu dGx5IGFuCiAgIGluc2VydGlvbiB3aWxsIGJlIHJlcXVpcmVkLgoKNC4zLjMuICBIb3N0LWJhc2Vk IEltcGxlbWVudGF0aW9uIG9mIE1QQSBGZW5jaW5nCgogICBNYW55IGhhcmR3YXJlIGltcGxlbWVu dGF0aW9ucyBvZiBpV0FSUCB1c2luZyBNUEEvVENQIGRvIG5vdCBoYW5kbGUKICAgdGhlIE1QQSBS ZXF1ZXN0L1JlcGx5IGV4Y2hhbmdlIGluIGhhcmR3YXJlLCByYXRoZXIgdGhleSBhcmUgaGFuZGxl ZAogICBieSB0aGUgaG9zdCBwcm9jZXNzb3IgaW4gc29mdHdhcmUuICBXaXRoIHN1Y2ggZGVzaWdu cyBpdCBpcyBjb21tb24KICAgZm9yIHRoZSBNUEEgRmVuY2luZyB0byBiZSBpbXBsZW1lbnRlZCBp biB0aGUgdXNlci1zcGFjZSBkZXZpY2UtCiAgIHNwZWNpZmljIGxpYnJhcnkgKGNvbW1vbmx5IHJl ZmVycmVkIHRvIGFzIGEgJ1VzZXIgVmVyYnMnIGxpYnJhcnkgb3IKICAgbW9kdWxlKS4KCiAgIFdo ZW4gdGhlIGdlbmVyYXRpb24gYW5kIHJlY2VwdGlvbiBvZiBNUEEgRlVMUERVcyBpcyBhbHJlYWR5 IGRlZGljYXRlZAogICB0byBoYXJkd2FyZSwgYSBXb3JrIENvbXBsZXRpb24gY2FuIG9ubHkgYmUg Z2VuZXJhdGVkIGJ5IGFuIHVudGFnZ2VkCiAgIG1lc3NhZ2UuCgo0LjQuICBTdGFuZGFyZGl6ZWQg UkRNQSBQYXJhbWV0ZXIgTmVnb3RpYXRpb24KCiAgIE1vc3QgUkRNQSBhcHBsaWNhdGlvbnMgYXJl IGRldmVsb3BlZCB1c2luZyBhIHRyYW5zcG9ydCBuZXV0cmFsIEFQSSB0bwogICBhY2Nlc3MgUkRN QSBzZXJ2aWNlcyBiYXNlZCBvbiBhICJxdWV1ZSBwYWlyIiBwYXJhZGlnbSBhcyBvcmlnaW5hbGx5 CiAgIGRlZmluZWQgYnkgdGhlIFZpcnR1YWwgSW50ZXJmYWNlIEFyY2hpdGVjdHVyZSBbVklBXSwg cmVmaW5lZCBieSB0aGUKICAgRGlyZWN0IEFjY2VzcyBQcm9ncmFtbWluZyBMaWJyYXJ5IFtEQVBM XSBhbmQgbW9zdCBjb21tb25seSBkZXBsb3llZAogICB3aXRoIHRoZSBPcGVuRmFicmljcyBBUEkg W09GQV0uCgogICBUaGVzZSB0cmFuc3BvcnQgbmV1dHJhbCBBUElzIHNlZWsgdG8gcHJvdmlkZSBh IGNvbW1vbiBzZXQgb2YgUkRNQQogICBzZXJ2aWNlcyB3aGV0aGVyIHRoZSB1bmRlcmx5aW5nIHRy YW5zcG9ydCBpcywgZm9yIGV4YW1wbGUsIGlXQVJQIG92ZXIKICAgTVBBLCBpV0FSUCBvdmVyIFND VFAgb3IgSW5maW5pQmFuZC4KCiAgIFRoZSBjb21tb24gbW9kZWwgZm9yIGVzdGFibGlzaGluZyBh biBSRE1BIGNvbm5lY3Rpb24gaGFzIHRoZQogICBmb2xsb3dpbmcgc3RlcHM6CgogICBvICBUaGUg cGFzc2l2ZSBzaWRlIFVMUCBsaXN0ZW5zIGZvciBjb25uZWN0aW9uIHJlcXVlc3RzLgoKICAgbyAg VGhlIGFjdGl2ZSBzaWRlIFVMUCBzdWJtaXRzIGEgY29ubmVjdGlvbiByZXF1ZXN0IHVzaW5nIGFu IFJETUEKICAgICAgZW5kcG9pbnQgKCJxdWV1ZSBwYWlyIiksIHRoZSBkZXNpcmVkIGRlc3RpbmF0 aW9uIGFuZCB0aGUKICAgICAgcGFyYW1ldGVycyB0byBiZSB1c2VkIGZvciB0aGUgY29ubmVjdGlv bi4gIFRob3NlIHBhcmFtZXRlcnMKICAgICAgaW5jbHVkZSBib3RoIFJETUEgbGF5ZXIgY2hhcmFj dGVyaXN0aWNzLCBzdWNoIGFzIHRoZSBSRE1BIFJlYWQKCgoKS2FuZXZza3ksIGV0IGFsLiAgICAg ICAgIEV4cGlyZXMgT2N0b2JlciA1LCAyMDExICAgICAgICAgICAgICAgIFtQYWdlIDhdCgwKSW50 ZXJuZXQtRHJhZnQgICBFbmhhbmNlZCBSRE1BIENvbm5lY3Rpb24gRXN0YWJsaXNobWVudCAgICAg ICBBcHJpbCAyMDExCgoKICAgICAgY3JlZGl0cyB0byBiZSBhbGxvd2VkIGFuZCBhcHBsaWNhdGlv biBzcGVjaWZpYyBkYXRhICh0eXBpY2FsbHkKICAgICAgcmVmZXJyZWQgdG8gYXMgInByaXZhdGUg ZGF0YSIpLgoKICAgbyAgVGhlIHBhc3NpdmUgc2lkZSBVTFAgcmVjZWl2ZXMgdGhlIENvbm5lY3Rp b24gUmVxdWVzdCwgd2hpY2gKICAgICAgaW5jbHVkZXMgdGhlIGlkZW50aXR5IG9mIHRoZSBhY3Rp dmUgc2lkZSBhbmQgdGhlIHJlcXVlc3RlZAogICAgICBjb25uZWN0aW9uIGNoYXJhY3RlcmlzdGlj cy4gIFRoZSBwYXNzaXZlIHNpZGUgVUxQIHVzZXMgdGhpcwogICAgICBpbmZvcm1hdGlvbiB0byBk ZWNpZGUgd2hldGhlciB0byBhY2NlcHQgdGhlIGNvbm5lY3Rpb24sIGFuZCBpZiBpdAogICAgICBp cyB0byBiZSBhY2NlcHRlZCwgaG93IHRvIGNyZWF0ZSBhbmQvb3IgY29uZmlndXJlIHRoZSBSRE1B CiAgICAgIGVuZHBvaW50LgoKICAgbyAgSWYgYWNjZXB0aW5nLCB0aGUgcGFzc2l2ZSBzaWRlIFVM UCBzdWJtaXRzIGl0cyBhY2NlcHRhbmNlIG9mIHRoZQogICAgICBDb25uZWN0aW9uIFJlcXVlc3Qu ICBUaGlzIGxvY2FsIGFjY2VwdCBvcGVyYXRpb24gaW5jbHVkZXMgdGhlIFJETUEKICAgICAgZW5k cG9pbnQgdG8gYmUgdXNlZCBhbmQgdGhlIGNvbm5lY3Rpb24gY2hhcmFjdGVyaXN0aWNzIChib3Ro IHRoZQogICAgICBSRE1BIGNvbmZpZ3VyYXRpb24gYW5kIGFueSBhcHBsaWNhdGlvbiBzcGVjaWZp YyBwcml2YXRlIGRhdGEgdG8gYmUKICAgICAgcmV0dXJuZWQpLgoKICAgbyAgVGhlIGFjdGl2ZSBz aWRlIHJlY2VpdmVzIGNvbmZpcm1hdGlvbiB0aGF0IHRoZSBjb25uZWN0aW9uIGhhcyBiZWVuCiAg ICAgIGFjY2VwdGVkLCB3aGF0IHRoZSBjb25maWd1cmVkIGNvbm5lY3Rpb24gY2hhcmFjdGVyaXN0 aWNzIGFyZSwgYW5kCiAgICAgIGFueSBhcHBsaWNhdGlvbiBzdXBwbGllZCBwcml2YXRlIGRhdGEu CgogICBBcyBjdXJyZW50bHkgZGVmaW5lZCwgRERQIGNvbm5lY3Rpb24gZXN0YWJsaXNobWVudCBy ZXF1aXJlcyB0aGUgVUxQCiAgIHRvIGVuY29kZSB0aGUgUkRNQSBjb25maWd1cmF0aW9uIGluIHRo ZSBhcHBsaWNhdGlvbiBzcGVjaWZpYyBwcml2YXRlCiAgIGRhdGEuICBUaGlzIHJlc3VsdHMgdW5k ZXNpcmFibGUgZHVwbGljYXRpb24gb2YgbG9naWMgdG8gY292ZXIgYm90aAogICBJbmZpbmlCYW5k IGFuZCBpV0FSUCwgYW5kIHRvIHNwZWNpZnkgdGhlIGV4dHJhY3Rpb24gb2YgdGhlIFJETUEKICAg Y2hhcmFjdGVyaXN0aWNzIGZyb20gdGhlIFVMUCBmb3IgZWFjaCBzcGVjaWZpYyBVcHBlciBMYXll ciBQcm90b2NvbC4KCiAgIEEgc3RhbmRhcmQgZGVmaW5pdGlvbiBvZiB0aGUgUkRNQSBjaGFyYWN0 ZXJpc3RpY3Mgd2l0aGluIHRoZSBwcml2YXRlCiAgIGRhdGEgc2VjdGlvbiB3b3VsZCBlbmFibGUg Y29tbW9uIGNvbm5lY3Rpb24gZXN0YWJsaXNobWVudCBBUElzIHRvCiAgIGZvcm1hdCB0aGUgUkRN QSBjaGFyYWN0ZXJpc3RpY3MgYmFzZWQgb24gdGhlIHNhbWUgQVBJIGluZm9ybWF0aW9uCiAgIHVz ZWQgd2hlbiBlc3RhYmxpc2hpbmcgYW4gSW5maW5pQmFuZCBjb25uZWN0aW9uLiAgVGhlIGFwcGxp Y2F0aW9uCiAgIHdvdWxkIHRoZW4gb25seSBoYXZlIHRvIGluZGljYXRlIHRoYXQgaXQgd2FzIHVz aW5nIHRoaXMgc3RhbmRhcmQKICAgZm9ybWF0IHRvIGVuYWJsZSBjb21tb24gY29ubmVjdGlvbiBl c3RhYmxpc2htZW50IHByb2NlZHVyZXMgdG8gYXBwbHkKICAgY29tbW9uIGNvZGUgdG8gcHJvcGVy bHkgcGFyc2UgdGhlc2UgZmllbGRzIGFuZCBjb25maWd1cmUgdGhlIFJETUEKICAgZW5kcG9pbnRz IGFjY29yZGluZ2x5LgoKCjUuICBNUEEgQ29ubmVjdGlvbiBTZXR1cAoKICAgQmVsb3cgd2UgcHJv dmlkZSBvdmVydmlldyBvZiBFbmhhbmNlZCBDb25uZWN0aW9uIFNldHVwLiAgVGhlIGdvYWwgaXMK ICAgdG8gYWxsb3cgc3RhbmRhcmQgbmVnb3RpYXRpb24gb2YgT1JEL0lSRCBzZXR0aW5nIG9uIGJv dGggc2lkZXMgb2YgdGhlCiAgIFJETUEgY29ubmVjdGlvbiBhbmQvb3IgdG8gbmVnb3RpYXRlIHRo ZSBpbml0aWFsIGRhdGEgdHJhbnNmZXIKICAgb3BlcmF0aW9uIGJ5IGFuIGluaXRpYXRvciB3aGVu IHRoZSBleGlzdGluZyAnY2xpZW50IHNlbmRzIGZpcnN0JyBydWxlCiAgIGRvZXMgbm90IG1hdGNo IGFwcGxpY2F0aW9uIHJlcXVpcmVtZW50cy4KCiAgIFRoZSBSRE1BIGNvbm5lY3Rpb24gaW5pdGlh dG9yIHNlbmRzIGFuIE1QQSBSZXF1ZXN0LCBhcyBzcGVjaWZpZWQgaW4KICAgW1JGQzUwNDRdOyB0 aGUgbmV3IGZvcm1hdCBkZWZpbmVkIGhlcmUgYWxsb3dzIGZvcjoKCgoKCgpLYW5ldnNreSwgZXQg YWwuICAgICAgICAgRXhwaXJlcyBPY3RvYmVyIDUsIDIwMTEgICAgICAgICAgICAgICAgW1BhZ2Ug OV0KDApJbnRlcm5ldC1EcmFmdCAgIEVuaGFuY2VkIFJETUEgQ29ubmVjdGlvbiBFc3RhYmxpc2ht ZW50ICAgICAgIEFwcmlsIDIwMTEKCgogICBvICBTdGFuZGFyZGl6ZWQgbmVnb3RpYXRpb24gb2Yg T1JEIGFuZCBJUkQuCgogICBvICBOZWdvdGlhdGlvbiBvZiBhbiBSVFIgbWVzc2FnZS4KCiAgIFRo ZSBSRE1BIGNvbm5lY3Rpb24gcmVzcG9uZGVyIHByb2Nlc3NlcyB0aGUgTVBBIFJlcXVlc3QgYW5k IGdlbmVyYXRlcwogICBhbiBNUEEgUmVwbHksIGFzIHNwZWNpZmllZCBpbiBbUkZDNTA0NF07IHRo ZSBuZXcgZm9ybWF0IGNvbXBsZXRlcyB0aGUKICAgbmVnb3RpYXRpb24uCgogICBUaGUgbG9jYWwg aW50ZXJmYWNlIFNIT1VMRCByZXF1aXJlIHRoZSBVTFAgdG8gZXhwbGljaXRseSBjb25maWd1cmUg b24KICAgYSBwZXItYXBwbGljYXRpb24gb3IgcGVyLWNvbm5lY3Rpb24gYmFzaXMgd2hlbiBhbiBl eHBsaWNpdCBSVFIKICAgbWVzc2FnZSB3aWxsIGJlIHJlcXVpcmVkLiAgUGlnZ3ktYmFja2luZyB0 aGUgUlRSIG9uIGEgQ2xpZW50J3MgZmlyc3QKICAgbWVzc2FnZSBpcyBhIHZhbHVhYmxlIG9wdGlt aXphdGlvbiBmb3IgbW9zdCBjb25uZWN0aW9ucy4KCiAgIFRoZSBSRE1BIGNvbm5lY3Rpb24gaW5p dGlhdG9yIE1VU1QgTk9UIGFsbG93IGFueSBsYXRlciBGVUxQRFVzIHRvIGJlCiAgIHRyYW5zbWl0 dGVkIGJlZm9yZSB0aGUgUlRSIG1lc3NhZ2UuICBPbmUgbWV0aG9kIHRvIGFjaGlldmUgdGhhdCBp cyB0bwogICBkZWxheSBub3RpZnlpbmcgdGhlIFVMUCB0aGF0IHRoZSBSRE1BIGNvbm5lY3Rpb24g aGFzIGJlZW4gZXN0YWJsaXNoZWQKICAgdW50aWwgYWZ0ZXIgYW55IHJlcXVpcmVkIFJUUiBNZXNz YWdlIGhhcyBiZWVuIHRyYW5zbWl0dGVkLgoKICAgQWxsIE1QQSBleGNoYW5nZXMgYXJlIHBlcmZv cm1lZCB2aWEgVENQIHByaW9yIHRvIFJETUEgZXN0YWJsaXNobWVudCwKICAgYW5kIGFyZSB0aGVy ZWZvcmUgc2lnbmFsZWQgdmlhIFRDUCBhbmQgbm90IHZpYSBSRE1BIGNvbXBsZXRpb24uCgoKNi4g IEVuaGFuY2VkIE1QQSBSZXF1ZXN0L1JlcGx5IEZyYW1lcwoKICAgRW5oYW5jZWQgUkRNQSBDb25u ZWN0aW9uIEVzdGFibGlzaG1lbnQgdXNlcyBhbiBhbHRlcm5hdGUgZm9ybWF0IGZvcgogICBNUEEg UmVxdWVzdHMgYW5kIFJlcGxpZXMsIGFzIGZvbGxvd3M6CgogICAgICAgMCAgICAgICAgICAgICAg ICAgICAxICAgICAgICAgICAgICAgICAgIDIgICAgICAgICAgICAgICAgICAgMwogICAgICAgIDAg MSAyIDMgNCA1IDYgNyA4IDkgMCAxIDIgMyA0IDUgNiA3IDggOSAwIDEgMiAzIDQgNSA2IDcgOCA5 IDAgMQogICAgICAgKy0rLSstKy0rLSstKy0rLSstKy0rLSstKy0rLSstKy0rLSstKy0rLSstKy0r LSstKy0rLSstKy0rLSstKy0rLSsKICAgIDAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAg ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB8CiAgICAgICArICAgICAgICAgS2V5 ICgxNiBieXRlcyBjb250YWluaW5nICJNUEEgSUQgUmVxIEZyYW1lIikgICAgICAgICAgKwogICAg NCAgfCAgICAgICg0RCA1MCA0MSAyMCA0OSA0NCAyMCA1MiA2NSA3MSAyMCA0NiA3MiA2MSA2RCA2 NSkgICAgICAgIHwKICAgICAgICsgICAgICAgICBPciAgKDE2IGJ5dGVzIGNvbnRhaW5pbmcgIk1Q QSBJRCBSZXAgRnJhbWUiKSAgICAgICAgICArCiAgICA4ICB8ICAgICAgKDREIDUwIDQxIDIwIDQ5 IDQ0IDIwIDUyIDY1IDcwIDIwIDQ2IDcyIDYxIDZEIDY1KSAgICAgICAgfAogICAgICAgKyAgICAg ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg ICsKICAgIDEyIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg ICAgICAgICAgICAgICAgICB8CiAgICAgICArLSstKy0rLSstKy0rLSstKy0rLSstKy0rLSstKy0r LSstKy0rLSstKy0rLSstKy0rLSstKy0rLSstKy0rLSstKwogICAgMTYgfE18Q3xSfFN8IFJlcyAg IHwgICAgIFJldiAgICAgICB8ICAgICAgICAgIFBEX0xlbmd0aCAgICAgICAgICAgIHwKICAgICAg ICstKy0rLSstKy0rLSstKy0rLSstKy0rLSstKy0rLSstKy0rLSstKy0rLSstKy0rLSstKy0rLSst Ky0rLSstKy0rCiAgICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg ICAgICAgICAgICAgICAgICAgICAgICAgfAogICAgICAgfiAgICAgICAgICAgICAgICAgICAgICAg ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH4KICAgICAgIH4gICAgICAg ICAgICAgICAgICAgUHJpdmF0ZSBEYXRhICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+ CiAgICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg ICAgICAgICAgICAgICAgfAogICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAr LSstKy0rLSstKy0rLSstKy0rLSstKy0rLSstKy0rLSsKICAgICAgIHwgICAgICAgICAgICAgICAg ICAgICAgICAgICAgICAgfAogICAgICAgKy0rLSstKy0rLSstKy0rLSstKy0rLSstKy0rLSstKy0r CgoKCgpLYW5ldnNreSwgZXQgYWwuICAgICAgICAgRXhwaXJlcyBPY3RvYmVyIDUsIDIwMTEgICAg ICAgICAgICAgICBbUGFnZSAxMF0KDApJbnRlcm5ldC1EcmFmdCAgIEVuaGFuY2VkIFJETUEgQ29u bmVjdGlvbiBFc3RhYmxpc2htZW50ICAgICAgIEFwcmlsIDIwMTEKCgogICBLZXk6ICBVbmNoYW5n ZWQgZnJvbSBbUkZDNTA0NF0uCgogICBNOiBVbmNoYW5nZWQgZnJvbSBbUkZDNTA0NF0uCgogICBD OiBVbmNoYW5nZWQgZnJvbSBbUkZDNTA0NF0uCgogICBSOiBVbmNoYW5nZWQgZnJvbSBbUkZDNTA0 NF0uCgogICBTOiBPbmUgaWYgdGhlIFByaXZhdGUgRGF0YSBiZWdpbnMgd2l0aCB0aGUgRW5oYW5j ZWQgUkRNQSBDb25uZWN0aW9uCiAgICAgIEVzdGFibGlzaG1lbnQgRGF0YS4gIFplcm8gb3RoZXJ3 aXNlLgoKICAgUmVzOiAgT25lIGJpdCBzbWFsbGVyIHRoYW4gaW4gW1JGQzUwNDRdLCBvdGhlcndp c2UgdW5jaGFuZ2VkLgoKICAgUmV2OiAgVGhpcyBmaWVsZCBjb250YWlucyB0aGUgcmV2aXNpb24g b2YgTVBBLiAgVG8gdXNlIGFueSBFbmhhbmNlZAogICAgICBDb25uZWN0aW9uIEVzdGFibGlzaG1l bnQgZmVhdHVyZSB0aGlzIE1VU1QgYmUgc2V0IHRvIHR3bywgSWYgbm8KICAgICAgRW5oYW5jZWQg Q29ubmVjdGlvbiBFc3RhYmxpc2htZW50IGZlYXR1cmVzIGFyZSBkZXNpcmVkIGl0IE1BWSBiZQog ICAgICBzZXQgdG8gb25lLiAgQSBob3N0IGFjY2VwdGluZyBNUEEgY29ubmVjdGlvbnMgTVVTVCBj b250aW51ZSB0bwogICAgICBhY2NlcHQgTVBBIFJlcXVlc3RzIHdpdGggdmVyc2lvbiBvbmUgZXZl biBpZiBpdCBzdXBwb3J0cyB2ZXJzaW9uCiAgICAgIHR3by4KCiAgIFBEX0xlbmd0aDogIFVuY2hh bmdlZCBmcm9tIFtSRkM1MDQ0XS4gIFRoaXMgaXMgdGhlIHRvdGFsIGxlbmd0aCBvZgogICAgICB0 aGUgUHJpdmF0ZSBEYXRhIGZpZWxkLCBpbmNsdWRpbmcgdGhlIEVuaGFuY2VkIFJETUEgQ29ubmVj dGlvbgogICAgICBFc3RhYmxpc2htZW50IERhdGEgaWYgcHJlc2VudC4KCiAgIFByaXZhdGUgRGF0 YTogIFVuY2hhbmdlZCBmcm9tIFtSRkM1MDQ0XS4gIEhvd2V2ZXIsIGlmIHRoZSAnUycgZmxhZyBp cwogICAgICBzZXQsIFByaXZhdGUgRGF0YSBiZWdpbnMgd2l0aCBFbmhhbmNlZCBSRE1BIENvbm5l Y3Rpb24KICAgICAgRXN0YWJsaXNobWVudCBEYXRhLgoKCjcuICBFbmhhbmNlZCBTQ1RQIFNlc3Np b24gQ29udHJvbCBDaHVua3MKCiAgIFRoZSB0eXBlIG9mIHRoZSBTQ1RQIFNlc3Npb24gQ29udHJv bCBDaHVuayBpcyBkZWZpbmVkIGJ5IGEgRnVuY3Rpb24KICAgQ29kZS4gIFtSRkM1MDQzXSBhbHJl YWR5IGRlZmluZXMgY29kZXMgZm9yICdERFAgU3RyZWFtIFNlc3Npb24KICAgSW5pdGlhdGUnIGFu ZCAnRERQIFN0cmVhbSBTZXNzaW9uIEFjY2VwdCcsIHdoaWNoIGFyZSBlcXVpdmFsZW50IHRvIGEK ICAgTVBBIFJlcXVlc3QgRnJhbWUgYW5kIGFuIGFjY2VwdGluZyBNUEEgUmVwbHkgRnJhbWUuCgog ICBFbmhhbmNlZCBSRE1BIENvbm5lY3Rpb24gRXN0YWJsaXNobWVudCByZXF1aXJlcyB0aHJlZSBh ZGRpdGlvbmFsCiAgIEZ1bmN0aW9uIGNvZGVzLiAgQWxsIEREUCBTdHJlYW0gU2Vzc2lvbiBGdW5j dGlvbmFsIENvZGVzIGFyZSBsaXN0ZWQKICAgYmVsb3c6CgogICBERFAgU3RyZWFtIFNlc3Npb24g SW5pdGlhdGU6ICAweDAwMQoKICAgRERQIFN0cmVhbSBTZXNzaW9uIEFjY2VwdDogIDB4MDAyCgog ICBERFAgU3RyZWFtIFNlc3Npb24gUmVqZWN0OiAgMHgwMDMKCgoKCgoKS2FuZXZza3ksIGV0IGFs LiAgICAgICAgIEV4cGlyZXMgT2N0b2JlciA1LCAyMDExICAgICAgICAgICAgICAgW1BhZ2UgMTFd CgwKSW50ZXJuZXQtRHJhZnQgICBFbmhhbmNlZCBSRE1BIENvbm5lY3Rpb24gRXN0YWJsaXNobWVu dCAgICAgICBBcHJpbCAyMDExCgoKICAgRERQIFN0cmVhbSBTZXNzaW9uIFRlcm1pbmF0ZTogIDB4 MDA0CgogICBFbmhhbmNlZCBERFAgU3RyZWFtIFNlc3Npb24gSW5pdGlhdGU6ICAweDAwNQoKICAg RW5oYW5jZWQgRERQIFN0cmVhbSBTZXNzaW9uIEFjY2VwdDogIDB4MDA2CgogICBFbmhhbmNlZCBE RFAgU3RyZWFtIFNlc3Npb24gUmVqZWN0OiAgMHgwMDcKCiAgIFRoZSBFbmhhbmNlZCBSZWplY3Qg ZnVuY3Rpb24gY29kZSBTSE9VTEQgYmUgdXNlZCB0byBpbmRpY2F0ZSBhCiAgIGNvbmZpZ3VyYXRp b24gdGhhdCB3b3VsZCBoYXZlIGJlZW4gYWNjZXB0ZWQuCgogICBUaGUgZXh0ZW5kZWQgRERQIHN0 cmVhbSBzZXNzaW9uIGVzdGFibGlzaG1lbnQgZm9sbG93cyB0aGUgc2FtZSBydWxlcwogICBhcyBy ZWd1bGFyIEREUCBzdHJlYW0gc2Vzc2lvbiBlc3RhYmxpc2htZW50IGFzIGRlZmluZWQgaW4gW1JG QzUwNDNdLgogICBVTFAtc3VwcGxpZWQgUHJpdmF0ZSBEYXRhIE1VU1QgYmUgaW5jbHVkZWQgZm9y IGV4dGVuZGVkIEREUCBTdHJlYW0KICAgU2Vzc2lvbiBJbml0aWF0ZSwgZXh0ZW5kZWQgRERQIFN0 cmVhbSBTZXNzaW9uIEFjY2VwdCwgYW5kIGV4dGVuZGVkCiAgIEREUCBTdHJlYW0gU2Vzc2lvbiBS ZWplY3QgbWVzc2FnZXMuICBIb3dldmVyLCB0aGUgVUxQIHN1cHBsaWVkCiAgIFByaXZhdGUgRGF0 YSBNQVkgYmUgb2YgemVybyBsZW5ndGguCgogICBQcml2YXRlIERhdGEgbGVuZ3RoIE1VU1QgTk9U IGV4Y2VlZCA1MTIgYnl0ZXMgaW4gYW55IG1lc3NhZ2UsCiAgIGluY2x1ZGluZyBlbmhhbmNlZCBS RE1BIGNvbm5lY3Rpb24gZXN0YWJsaXNobWVudCBkYXRhLgoKICAgUHJpdmF0ZSBEYXRhIE1VU1Qg Tk9UIGJlIGluY2x1ZGVkIGluIHRoZSBERFAgU3RyZWFtIFNlc3Npb24gVGVybWluYXRlCiAgIG1l c3NhZ2UuCgogICBSZWNlaXZlZCBleHRlbmRlZCBERFAgU3RyZWFtIFNlc3Npb24gQ29udHJvbCBt ZXNzYWdlcyBTSE9VTEQgYmUKICAgcmVwb3J0ZWQgdG8gdGhlIFVMUC4gIElmIHJlcG9ydGVkLCBh bnkgc3VwcGxpZWQgUHJpdmF0ZSBEYXRhIE1VU1QgYmUKICAgYXZhaWxhYmxlIGZvciB0aGUgVUxQ IHRvIGV4YW1pbmUuCgogICBUaGUgZW5oYW5jZWQgRERQIHN0cmVhbSBtYW5hZ2VtZW50IE1VU1Qg dXNlIEREUCBzdHJlYW0gc2Vzc2lvbgogICB0ZXJtaW5hdGlvbiBmdW5jdGlvbiBjb2RlIHRvIHRl cm1pbmF0ZSBhIHN0cmVhbSBlc3RhYmxpc2hlZCB1c2luZwogICBlbmhhbmNlZCBERFAgc3RyZWFt IHNlc3Npb24gZnVuY3Rpb24gY29kZXMuCgogICBJdCBzaG91bGQgYmUgbm90ZWQgdGhhdCBbUkZD NTA0M10gYWxyZWFkeSBzdXBwb3J0cyBlaXRoZXIgc2lkZQogICBzZW5kaW5nIHRoZSBmaXJzdCBE RFAgTWVzc2FnZS4gIFRoZSBQYXlsb2FkIFByb3RvY29sIElkZW50aWZpZXIKICAgKFBQSUQpIGFs cmVhZHkgZGlzdGluZ3Vpc2hlcyBiZXR3ZWVuIFNlc3Npb24gRXN0YWJsaXNobWVudCBhbmQgRERQ CiAgIFNlZ21lbnRzLgoKICAgVGhlIGZvbGxvd2luZyBhZGRpdGlvbmFsIExlZ2FsIFNlcXVlbmNl cyBvZiBERFAgU3RyZWFtIFNlc3Npb24KICAgbWVzc2FnZXMgYXJlIGRlZmluZWQ6CgogICBvICBF eHRlbmRlZCBBY3RpdmUvUGFzc2l2ZSBTZXNzaW9uIEFjY2VwdGVkOiBhcyB3aXRoIHNlY3Rpb24g Ni4yIG9mCiAgICAgIFtSRkM1MDQzXSwgYnV0IHdpdGggdGhlIGV4dGVuZGVkIG9wY29kZXMgYXMg ZGVmaW5lZCBpbiB0aGlzCiAgICAgIGRvY3VtZW50LgoKICAgbyAgRXh0ZW5kZWQgQWN0aXZlL1Bh c3NpdmUgU2Vzc2lvbiBSZWplY3RlZDogYXMgd2l0aCBzZWN0aW9uIDYuMyBvZgogICAgICBbUkZD NTA0M10sIGJ1dCB3aXRoIHRoZSBleHRlbmRlZCBvcGNvZGVzIGFzIGRlZmluZWQgaW4gdGhpcwog ICAgICBkb2N1bWVudC4KCgoKCkthbmV2c2t5LCBldCBhbC4gICAgICAgICBFeHBpcmVzIE9jdG9i ZXIgNSwgMjAxMSAgICAgICAgICAgICAgIFtQYWdlIDEyXQoMCkludGVybmV0LURyYWZ0ICAgRW5o YW5jZWQgUkRNQSBDb25uZWN0aW9uIEVzdGFibGlzaG1lbnQgICAgICAgQXByaWwgMjAxMQoKCiAg IG8gIEV4dGVuZGVkIEFjdGl2ZS9QYXNzaXZlIFNlc3Npb24gTm9uLVVMUCBSZWplY3RlZDogYXMg d2l0aCBzZWN0aW9uCiAgICAgIDYuNCBvZiBbUkZDNTA0M10sIGJ1dCB3aXRoIHRoZSBleHRlbmRl ZCBvcGNvZGVzIGFzIGRlZmluZWQgaW4gdGhpcwogICAgICBkb2N1bWVudC4KCgo4LiAgTVBBIEVy cm9yIFJlcG9ydGluZwoKICAgVGhlIFtSRkM1MDQzXSBhbmQgW1JGQzUwNDRdIGRvIG5vdCBkZWZp bmUgZXJyb3IgY29kZXMuICBUaGUgcHJvdG9jb2wKICAgbGF5ZXJzIG9uIHdoaWNoIFJETUEgY29u bmVjdGlvbiBlc3RhYmxpc2htZW50IGlzIGxheWVyZWQgdXBvbgogICBbUkZDNTA0MF0gYW5kIFtS RkM1MDQxXSBkZWZpbmUgbGF5ZXJzIGFuZCBlcnJvciB0eXBlcy4gIFNwZWNpZmljYWxseSwKICAg TVBBIG5lZ290aWF0aW9uIGZvciBSRE1BIGNvbm5lY3Rpb24gZXN0YWJsaXNobWVudCB1c2VzOgoK ICAgTGF5ZXI6ICAgICAgMDAxMGIgLSBMTFAvTVBBCiAgIEVycm9yIFR5cGU6IDB4MyAgIC0gTExQ CgogICBUaGUgZm9sbG93aW5nIGVycm9yIGNvZGVzIGFyZSBkZWZpbmVkIGZvciBNUEEgbmVnb3Rp YXRpb246CgogICBFcnJvciBDb2RlICAgICAgICAgRGVzY3JpcHRpb24KICAgLS0tLS0tLS0tLS0t LS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0KICAgMHgxICAgICAg ICAgICAgICAgIExvY2FsIENhdGFzdHJvcGhpYwogICAweDIgICAgICAgICAgICAgICAgSW5zdWZm aWNpZW50IElSRCByZXNvdXJjZXMKCgo5LiAgRW5oYW5jZWQgUkRNQSBDb25uZWN0aW9uIEVzdGFi bGlzaG1lbnQgRGF0YQoKICAgICAgIDAgICAgICAgICAgICAgICAgICAgMSAgICAgICAgICAgICAg ICAgICAyICAgICAgICAgICAgICAgICAgIDMKICAgICAgICAwIDEgMiAzIDQgNSA2IDcgOCA5IDAg MSAyIDMgNCA1IDYgNyA4IDkgMCAxIDIgMyA0IDUgNiA3IDggOSAwIDEKICAgICAgICstKy0rLSst Ky0rLSstKy0rLSstKy0rLSstKy0rLSstKy0rLSstKy0rLSstKy0rLSstKy0rLSstKy0rLSstKy0r CiAgICAwICB8QXxCfCAgICAgICAgSVJEICAgICAgICAgICAgICAgIHxDfER8ICAgICAgICBPUkQg ICAgICAgICAgICAgICAgfAogICAgNCAgKy0rLSstKy0rLSstKy0rLSstKy0rLSstKy0rLSstKy0r LSstKy0rLSstKy0rLSstKy0rLSstKy0rLSstKy0rLSsKCgoKICAgSVJEOiAgSW4gcmVxdWVzdDog dGhlIEluaXRpYXRvciBkZXNpcmVkIHJlc3BvbmRlciBJUkQgZm9yIHRoZQogICAgICBjb25uZWN0 aW9uLiAgSW4gcmVwbHk6IHRoZSBkZXB0aCB0aGUgUmVzcG9uZGVyIHdpbGwgc3VwcG9ydC4KICAg ICAgUmVzcG9uZGVyIFNIT1VMRCBOT1Qgc2V0IGl0cyBJUkQgaGlnaGVyIHRoYW4gSW5pdGlhdG9y IE9SRC4gIFVwb24KICAgICAgcmVjZWl2aW5nIGFjY2VwdCBjb25uZWN0aW9uIG1lc3NhZ2UgZnJv bSB0aGUgUmVzcG9uZGVyLCB0aGUKICAgICAgSW5pdGlhdG9yIE1VU1Qgc2V0IGl0cyBPUkQgdG8g dGhlIHJlc3BvbmRlciBJUkQuICBCb3RoIFJlc3BvbmRlcgogICAgICBhbmQgSW5pdGlhdG9yIE1V U1QgcGFzcyB0aGUgcmVtb3RlIHNpZGUgcHJvdmlkZWQgSVJEIHRvIFVMUC4gIEFuCiAgICAgIGFs bCBvbmVzIHZhbHVlICgweDNGRkYpIGluZGljYXRlcyB0aGF0IGF1dG9tYXRpYyBuZWdvdGlhdGlv biBvZgogICAgICB0aGUgSVJEIGlzIG5vdCBkZXNpcmVkLCBhbmQgdGhhdCB0aGUgVUxQIHdpbGwg YmUgcmVzcG9uc2libGUgZm9yCiAgICAgIGRvaW5nIHRoaXMgY29uZmlndXJhdGlvbi4KCiAgIE9S RDogIEluIHJlcXVlc3Q6IHRoZSBJbml0aWF0b3IgaW5pdGlhbCBPUkQgc2V0dGluZyBmb3IgdGhl CiAgICAgIGNvbm5lY3Rpb24uICBJbiByZXBseTogdGhlIGRlcHRoIHRoZSBSZXNwb25kZXIgd2ls bCBzdXBwb3J0LgogICAgICBSZXNwb25kZXIgU0hPVUxEIHNldCBpdHMgT1JEIHRvIGEgdmFsdWUg dGhhdCBpcyBsZXNzIHRoYW4gb3IgZXF1YWwKICAgICAgdG8gdGhlIHJlcXVlc3RlZCBJbml0aWF0 b3IgSVJELiAgVXBvbiByZWNlaXZpbmcgYWNjZXB0IGNvbm5lY3Rpb24KICAgICAgZnJvbSB0aGUg UmVzcG9uZGVyLCB0aGUgSW5pdGlhdG9yIE1VU1Qgc2V0IGl0cyBJUkQgdG8gYSB2YWx1ZSBhdAoK CgpLYW5ldnNreSwgZXQgYWwuICAgICAgICAgRXhwaXJlcyBPY3RvYmVyIDUsIDIwMTEgICAgICAg ICAgICAgICBbUGFnZSAxM10KDApJbnRlcm5ldC1EcmFmdCAgIEVuaGFuY2VkIFJETUEgQ29ubmVj dGlvbiBFc3RhYmxpc2htZW50ICAgICAgIEFwcmlsIDIwMTEKCgogICAgICBsZWFzdCBhcyBsYXJn ZSBhcyB0aGUgcmVzcG9uZGVyIE9SRCBpZiBpdCBoYXMgc3VmZmljaWVudCByZXNvdXJjZXMKICAg ICAgZm9yIGl0LiAgSWYgdGhlIEluaXRpYXRvciBkb2VzIG5vdCBoYXZlIHN1ZmZpY2llbnQgcmVz b3VyY2VzIHRvCiAgICAgIHNhdGlzZnkgdGhlIFJlc3BvbmRlciBPUkQgaXQgTVVTVCB0ZXJtaW5h dGUgdGhlIGNvbm5lY3Rpb24gYW5kCiAgICAgIHJlcG9ydCB0byBsb2NhbCBVTFAgdGhlIFJlc3Bv bmRlciBPUkQgYW5kIElSRCB3aXRoIGFuIGluZGljYXRvciBvZgogICAgICBpbnN1ZmZpY2llbnQg cmVzb3VyY2VzIHRvIHNhdGlzZnkgUmVzcG9uZGVyIE9SRCwgYW5kIE1VU1Qgc2VuZAogICAgICB0 ZXJtaW5hdGlvbiBtZXNzYWdlIHRvIHRoZSBSZXNwb25kZXIgaW5kaWNhdGluZyBpbnN1ZmZpY2ll bnQKICAgICAgcmVzb3VyY2VzLiAgVGh1cywgdGhlIFRFUk0gbWVzc2FnZSBNVVNUIGNvbnRhaW4g TGF5ZXIgMiwgRXJyb3IKICAgICAgVHlwZSAzLCBFcnJvciBDb2RlIDIuICBCb3RoIFJlc3BvbmRl ciBhbmQgSW5pdGlhdG9yIE1VU1QgcGFzcyB0aGUKICAgICAgcmVtb3RlIHNpZGUgcHJvdmlkZWQg T1JEIHRvIFVMUC4gIEFuIGFsbCBvbmVzIHZhbHVlICgweDNGRkYpCiAgICAgIGluZGljYXRlcyB0 aGF0IGF1dG9tYXRpYyBuZWdvdGlhdGlvbiBvZiB0aGUgT1JEIGlzIG5vdCBkZXNpcmVkLAogICAg ICBhbmQgdGhhdCB0aGUgVUxQIHdpbGwgYmUgcmVzcG9uc2libGUgZm9yIGRvaW5nIHRoaXMgY29u ZmlndXJhdGlvbi4KCiAgIEE6IENvbnRyb2wgRmxhZyBmb3IgdXNpbmcgYSB6ZXJvIGxlbmd0aCBG VUxQRFUgYXMgdGhlIFJUUiBtZXNzYWdlLgoKICAgQjogQ29udHJvbCBGbGFnIGZvciB1c2luZyBh IHplcm8gbGVuZ3RoIFJETUEgV3JpdGUgYXMgdGhlIFJUUgogICAgICBtZXNzYWdlLgoKICAgQzog Q29udHJvbCBGbGFnIGZvciB1c2luZyBhIHplcm8gbGVuZ3RoIFJETUEgUmVhZCBhcyB0aGUgUlRS IG1lc3NhZ2UuCgogICBEOiBSZXNlcnZlZC4gIE11c3QgYmUgc2VudCBhcyB6ZXJvIGFuZCBpZ25v cmVkIHdoZW4gcmVjZWl2ZWQuCgogICBJbiB0aGUgTVBBIFJlcXVlc3QsIHRoZSBJbml0aWF0b3Ig U0hPVUxEIHNldCB0aGUgQSwgQiBhbmQgQyBDb250cm9sCiAgIEZsYWdzIHJlc3BlY3RpdmVseSB0 byBUUlVFIGZvciBlYWNoIG9mIHRoZSBvcHRpb25zIGl0IHN1cHBvcnRzLgoKICAgSW4gdGhlIE1Q QSBSZXBseSwgdGhlIENvbnRyb2wgRmxhZyBpcyBzZXQgZm9yIHRoZSBzZXQgb2Ygb3B0aW9ucyB0 aGF0CiAgIHRoZSBwYXNzaXZlIHNpZGUgd2lsbCBhY2NlcHQgYXMgYW4gUlRSIG1lc3NhZ2UuICBU aGlzIHJlc3BvbnNlIE1VU1QKICAgaW5jbHVkZSBhbGwgb3B0aW9ucyB0aGF0IHRoZSByZXNwb25k ZXIgd2lsbCBzdXBwb3J0IHdpdGhvdXQgcmVxdWlyaW5nCiAgIGEgY29ubmVjdGlvbiBzcGVjaWZp YyBlbmFibGluZyBhY3Rpb24uICBGb3IgZXhhbXBsZSwgaWYgdGhlIHJlc3BvbmRlcgogICB3aWxs IGFsd2F5cyB1bmJsb2NrIGFuIE1QQSBjb25uZWN0aW9uIHdoZW4gaXQgcmVjZWl2ZXMgYSB6ZXJv IGxlbmd0aAogICBNUEEgV3JpdGUsIGl0IHNob3VsZCBpbmRpY2F0ZSBzbyB3aXRob3V0IHJlZ2Fy ZCB0byB3aGF0IHdhcyBpbiB0aGUKICAgTVBBIFJlcXVlc3QuICBPcHRpb25zIHdoaWNoIHJlcXVp cmUgY29ubmVjdGlvbiBzcGVjaWZpYyBlbmFibGluZwogICBhY3Rpb25zIFNIT1VMRCBOT1QgYmUg c2V0IHVubGVzcyB0aGUgY29ycmVzcG9uZGluZyBmbGFnIHdhcyBzZXQgaW4KICAgdGhlIE1QQSBS ZXF1ZXN0LiAgVGhlIHJlc3BvbmRlciBNQVkgY2hvb3NlIHRvIGxpbWl0IHRoZSBudW1iZXIgb2YK ICAgbW9kZXMgdGhhdCBpdCBlbmFibGVzLgoKICAgSWYgdGhlcmUgaXMgbm8gU3RhbmRhcmQgUkRN QVAgQ29uZmlndXJhdGlvbiBEYXRhIGluIHRoZSBNUEEgUmVwbHkKICAgRnJhbWUsIGFuZCB0aGUg RW5oYW5jZWQgQ29ubmVjdGlvbiBFc3RhYmxpc2htZW50IHZlcnNpb24gbnVtYmVyIGlzCiAgIHVz ZWQsIGl0IGlzIHRoZSBlcXVpdmFsZW50IG9mIHNldHRpbmcgJ0EnLCAnQicgYW5kICdDJy4KCiAg IFNldHRpbmcgbm8gQ29udHJvbCBGbGFncyBpbiB0aGUgTVBBIFJlcGx5IGluZGljYXRlcyB0aGF0 IGFuIFJETUEgU2VuZAogICBtZXNzYWdlIHdpbGwgYmUgcmVxdWlyZWQuICBBcyB0aGlzIG9wdGlv biB3aWxsIHJlcXVpcmUgdGhlIGluaXRpYXRvcgogICBVTFAgdG8gYmUgaW52b2x2ZWQgaXQgU0hP VUxEIE5PVCBiZSB1c2VkIHVubGVzcyBuZWNlc3NhcnkuCgogICBUaGUgcGVlci10by1wZWVyIG5l Z290aWF0aW9uIGZvciB0aGUgUlRSIG1lc3NhZ2UgZm9sbG93cyB0aGUKICAgZm9sbG93aW5nIG9y ZGVyOgoKCgoKCgpLYW5ldnNreSwgZXQgYWwuICAgICAgICAgRXhwaXJlcyBPY3RvYmVyIDUsIDIw MTEgICAgICAgICAgICAgICBbUGFnZSAxNF0KDApJbnRlcm5ldC1EcmFmdCAgIEVuaGFuY2VkIFJE TUEgQ29ubmVjdGlvbiBFc3RhYmxpc2htZW50ICAgICAgIEFwcmlsIDIwMTEKCgogICBJbml0aWF0 b3IgLS0+OiBTZXRzIENvbnRyb2wgRmxhZ3MgdG8gaW5kaWNhdGUgSW5pdGlhdG9yLXN1cHBvcnRl ZAogICBmb3JtcyBvZiBSVFIKCiAgIFJlc3BvbmRlciA8LS06IFNldHMgQ29udHJvbCBGbGFncyB0 byBpbmRpY2F0ZSBSZXNwb25kZXItc3VwcG9ydGVkCiAgIGZvcm1zIG9mIFJUUgoKICAgSW5pdGlh dG9yIC0tPjogSWYgYXQgbGVhc3Qgb25lIGZvcm0gb2YgUlRSIGlzIHN1cHBvcnRlZCBieSBib3Ro CiAgIEluaXRpYXRvciBhbmQgUmVzcG9uZGVyLCB0aGVuIHRoZSBmaXJzdCBtZXNzYWdlIHNlbnQg TVVTVCBiZSBhbiBSVFIKICAgdXNpbmcgYSBmb3JtIHN1cHBvcnRlZCBieSBib3RoIHRoZSBJbml0 aWF0b3IgYW5kIFJlc3BvbmRlci4KCiAgIEluaXRpYXRvciBvciBSZXNwb25kZXIgU0hPVUxEIGdl bmVyYXRlIHRoZSBURVJNIG1lc3NhZ2UgdGhhdCBjb250YWlucwogICBMYXllciAyLCBFcnJvciBU eXBlIDMsIEVycm9yIENvZGUgMSB3aGVuIGl0IGVuY291bnRlcnMgYW55IGVycm9yCiAgIGxvY2Fs bHkgZm9yIHdoaWNoIHRoZSBzcGVjaWFsIEVycm9yIENvZGUgaXMgbm90IGRlZmluZWQgaW4gc2Vj dGlvbgogICBTZWN0aW9uIDggYmVmb3JlIHJlc2V0dGluZyB0aGUgY29ubmVjdGlvbi4KCgoxMC4g IEludGVyb3BlcmFiaWxpdHkKCiAgIEFuIGluaXRpYXRvciBTSE9VTEQgTk9UIHVzZSB0aGUgRW5o YW5jZWQgRERQIENvbm5lY3Rpb24gRXN0YWJsaXNobWVudAogICBmb3JtYXRzIG9yIGZ1bmN0aW9u IGNvZGVzIHdoZW4gbm8gZW5oYW5jZWQgZnVuY3Rpb25hbGl0eSBpcyBkZXNpcmVkLgoKICAgQSBy ZXNwb25kZXIgTVVTVCBjb250aW51ZSB0byBhY2NlcHQgdGhlIHVuZW5oYW5jZWQgY29ubmVjdGlv bgogICByZXF1ZXN0cy4KCiAgIFRoZXJlIGFyZSB0aHJlZSBJbml0aWF0b3IvUmVzcG9uZGVyIGNh c2VzIHRoYXQgaW52b2x2ZSBlbmhhbmNlZCBNUEE6CiAgIGJvdGggaW5pdGlhdG9yIGFuZCByZXNw b25kZXIsIG9ubHkgcmVzcG9uZGVyLCBhbmQgb25seSBpbml0aWF0b3IuCiAgIFRoZSBlbmhhbmNl ZCBNUEEgZnJhbWUgaXMgZGVmaW5lZCBieSBmaWVsZCAnUycgc2V0IHRvIDEuCgogICBFbmhhbmNl ZCBNUEEgSW5pdGlhdG9yIGFuZCBSZXNwb25kZXI6ICBJZiBhIHJlc3BvbmRlciByZWNlaXZlcyBh bgogICAgICBlbmhhbmNlZCBNUEEgbWVzc2FnZSwgaXQgTVVTVCByZXNwb25kIHdpdGggYW4gZW5o YW5jZWQgTVBBCiAgICAgIG1lc3NhZ2UuCgogICBFbmhhbmNlZCBNUEEgUmVzcG9uZGVyIG9ubHk6 ICBJZiBhIHJlc3BvbmRlciByZWNlaXZlcyBhbiB1bmVuaGFuY2VkCiAgICAgIE1QQSBtZXNzYWdl ICgnUycgaXMgc2V0IHRvIDApLCBpdCBNVVNUIHJlc3BvbmQgd2l0aCBhbiB1bmVuaGFuY2VkCiAg ICAgIE1QQSBtZXNzYWdlLgoKICAgRW5oYW5jZWQgTVBBIEluaXRpYXRvciBvbmx5OiAgSWYgYSBy ZXNwb25kZXIgZG9lcyBub3Qgc3VwcG9ydAogICAgICByZWNlaXZlZCBleHRlbmRlZCBNUEEgbWVz c2FnZSwgdGhlbiBpdCBNVVNUIGNsb3NlIHRoZSBUQ1AKICAgICAgY29ubmVjdGlvbiBhbmQgZXhp dCBNUEEgc2luY2UgTVBBIGZyYW1lIGlzIGltcHJvcGVybHkgZm9ybWF0dGVkCiAgICAgIGZvciBp dCBhcyBzdGF0ZWQgaW4gW1JGQzUwNDRdLiAgVGh1cywgYm90aCBpbml0aWF0b3IgYW5kIHJlc3Bv bmRlcgogICAgICByZXBvcnQgVENQIGNvbm5lY3Rpb24gdGVybWluYXRpb24gdG8gYW4gYXBwbGlj YXRpb24gbG9jYWxseS4gIEluCiAgICAgIHRoaXMgY2FzZSBpbml0aWF0b3IgTUFZIGF0dGVtcHQg dG8gZXN0YWJsaXNoIFJETUEgY29ubmVjdGlvbiB1c2luZwogICAgICB1bmVuaGFuY2VkIE1QQSBw cm90b2NvbCBhcyBkZWZpbmVkIGluIFtSRkM1MDQ0XSBpZiB0aGlzIHByb3RvY29sCiAgICAgIGlz IGNvbXBhdGlibGUgd2l0aCB0aGUgYXBwbGljYXRpb24sIGFuZCBsZXQgVUxQIGRlYWwgd2l0aCBP UkQgYW5kCiAgICAgIElSRCwgYW5kIHBlZXItdG8tcGVlciBuZWdvdGlhdGlvbnMuCgoKCgoKCkth bmV2c2t5LCBldCBhbC4gICAgICAgICBFeHBpcmVzIE9jdG9iZXIgNSwgMjAxMSAgICAgICAgICAg ICAgIFtQYWdlIDE1XQoMCkludGVybmV0LURyYWZ0ICAgRW5oYW5jZWQgUkRNQSBDb25uZWN0aW9u IEVzdGFibGlzaG1lbnQgICAgICAgQXByaWwgMjAxMQoKCjExLiAgSUFOQSBDb25zaWRlcmF0aW9u cwoKICAgVGhpcyBkb2N1bWVudCBoYXMgbm8gSUFOQSBjb25zaWRlcmF0aW9ucy4KCgoxMi4gIFNl Y3VyaXR5IENvbnNpZGVyYXRpb25zCgogICBUaGUgc2VjdXJpdHkgY29uc2lkZXJhdGlvbnMgZnJv bSBSRkMgNTA0NCBhcHBseSBhbmQgdGhlIGNoYW5nZXMgaW4KICAgdGhpcyBkb2N1bWVudCBkbyBu b3QgaW50cm9kdWNlIG5ldyBzZWN1cml0eSBjb25zaWRlcmF0aW9ucy4KCgoxMy4gIEFja25vd2xl ZGdlbWVudHMKCiAgIFRoZSBhdXRob3JzIHdpc2ggdG8gdGhhbmsgU2VhbiBIZWZ0eSwgRGF2ZSBN aW50dXJuLCBUb20gVGFscGV5IGFuZAogICBEYXZpZCBCbGFjayBmb3IgdGhlaXIgdmFsdWFibGUg Y29udHJpYnV0aW9ucyBhbmQgcmV2aWV3cyBvZiB0aGlzCiAgIGRvY3VtZW50LgoKCjE0LiAgUmVm ZXJlbmNlcwoKMTQuMS4gIE5vcm1hdGl2ZSBSZWZlcmVuY2VzCgogICBbUkZDMjExOV0gIEJyYWRu ZXIsIFMuLCAiS2V5IHdvcmRzIGZvciB1c2UgaW4gUkZDcyB0byBJbmRpY2F0ZQogICAgICAgICAg ICAgIFJlcXVpcmVtZW50IExldmVscyIsIEJDUCAxNCwgUkZDIDIxMTksIE1hcmNoIDE5OTcuCgog ICBbUkZDNTA0MF0gIFJlY2lvLCBSLiwgTWV0emxlciwgQi4sIEN1bGxleSwgUC4sIEhpbGxhbmQs IEouLCBhbmQgRC4KICAgICAgICAgICAgICBHYXJjaWEsICJBIFJlbW90ZSBEaXJlY3QgTWVtb3J5 IEFjY2VzcyBQcm90b2NvbAogICAgICAgICAgICAgIFNwZWNpZmljYXRpb24iLCBSRkMgNTA0MCwg T2N0b2JlciAyMDA3LgoKICAgW1JGQzUwNDFdICBTaGFoLCBILiwgUGlua2VydG9uLCBKLiwgUmVj aW8sIFIuLCBhbmQgUC4gQ3VsbGV5LCAiRGlyZWN0CiAgICAgICAgICAgICAgRGF0YSBQbGFjZW1l bnQgb3ZlciBSZWxpYWJsZSBUcmFuc3BvcnRzIiwgUkZDIDUwNDEsCiAgICAgICAgICAgICAgT2N0 b2JlciAyMDA3LgoKICAgW1JGQzUwNDNdICBCZXN0bGVyLCBDLiBhbmQgUi4gU3Rld2FydCwgIlN0 cmVhbSBDb250cm9sIFRyYW5zbWlzc2lvbgogICAgICAgICAgICAgIFByb3RvY29sIChTQ1RQKSBE aXJlY3QgRGF0YSBQbGFjZW1lbnQgKEREUCkgQWRhcHRhdGlvbiIsCiAgICAgICAgICAgICAgUkZD IDUwNDMsIE9jdG9iZXIgMjAwNy4KCiAgIFtSRkM1MDQ0XSAgQ3VsbGV5LCBQLiwgRWx6dXIsIFUu LCBSZWNpbywgUi4sIEJhaWxleSwgUy4sIGFuZCBKLgogICAgICAgICAgICAgIENhcnJpZXIsICJN YXJrZXIgUERVIEFsaWduZWQgRnJhbWluZyBmb3IgVENQCiAgICAgICAgICAgICAgU3BlY2lmaWNh dGlvbiIsIFJGQyA1MDQ0LCBPY3RvYmVyIDIwMDcuCgoxNC4yLiAgSW5mb3JtYXRpdmUgUmVmZXJl bmNlcwoKICAgW0RBUExdICAgICAiRGlyZWN0IEFjY2VzcyBQcm9ncmFtbWluZyBMaWJyYXJ5IiwK ICAgICAgICAgICAgICA8aHR0cDovL3d3dy5kYXRjb2xsYWJvcmF0aXZlLm9yZz4uCgogICBbT0ZB XSAgICAgICJPRkEgdmVyYnMgJiBBUElzIiwgPGh0dHA6Ly93d3cub3BlbmZhYnJpY3Mub3JnLz4u CgoKCgpLYW5ldnNreSwgZXQgYWwuICAgICAgICAgRXhwaXJlcyBPY3RvYmVyIDUsIDIwMTEgICAg ICAgICAgICAgICBbUGFnZSAxNl0KDApJbnRlcm5ldC1EcmFmdCAgIEVuaGFuY2VkIFJETUEgQ29u bmVjdGlvbiBFc3RhYmxpc2htZW50ICAgICAgIEFwcmlsIDIwMTEKCgogICBbT3Blbk1QXSAgIE1j R3Jhdy1IaWxsLCAiUGFyYWxsZWwgUHJvZ3JhbW1pbmcgaW4gQyB3aXRoIE1QSSBhbmQKICAgICAg ICAgICAgICBPcGVuTVAiLCAyMDAzLgoKICAgW1BQTVBJXSAgICBNb3JnYW4gS2F1Zm1hbm4gUHVi bGlzaGVycyBJbmMuLCAiUGFyYWxsZWwgUHJvZ3JhbW1pbmcKICAgICAgICAgICAgICB3aXRoIE1Q SSIsIDIwMDguCgogICBbUkRNQUNdICAgICJSRE1BIFByb3RvY29sIFZlcmJzIFNwZWNpZmljYXRp b24gKFZlcnNpb24gMS4wKSIsIDxodHRwOi8KICAgICAgICAgICAgICAvd3d3LnJkbWFjb25zb3J0 aXVtLm9yZy9ob21lLwogICAgICAgICAgICAgIGRyYWZ0LWhpbGxhbmQtaXdhcnAtdmVyYnMtdjEu MC1SRE1BLnBkZj4uCgogICBbUkRTXSAgICAgIE9wZW4gRmFicmljcyBBc3NvY2lhdGlvbiwgIlJl bGlhYmxlIERhdGFncmFtIFNvY2tldCIsCiAgICAgICAgICAgICAgMjAwOCwgPGh0dHA6Ly93d3cu b3BlbmZhYnJpY3Mub3JnL2FyY2hpdmVzLwogICAgICAgICAgICAgIHNwcmluZzIwMDhzb25vbWEv VHVlc2RheS9zb25vbWFfMjAwOF8wNDA4JTIwT3JhY2xlLnBwdD4uCgogICBbVXNpbmdNUEldCiAg ICAgICAgICAgICAgTUlUIFByZXNzLCAiVXNpbmcgTVBJLTI6IEFkdmFuY2VkIEZlYXR1cmVzIG9m IHRoZSBNZXNzYWdlCiAgICAgICAgICAgICAgUGFzc2luZyBJbnRlcmZhY2UiLCAxOTk5LgoKICAg W1ZJQV0gICAgICBDb21wYXEsIEludGVsLCBNaWNyb3NvZnQsICJWaXJ0dWFsIEludGVyZmFjZSBB cmNoaXRlY3R1cmUKICAgICAgICAgICAgICBTcGVjaWZpY2F0aW9uIiwgMTk5NywgPGh0dHA6Ly9w bGxhYi5jcy5udGh1LmVkdS50dy9jczU0MDMvCiAgICAgICAgICAgICAgUmVhZGluZ3MvRUpCL3Nh bl8xMC5wZGY+LgoKCkF1dGhvcnMnIEFkZHJlc3NlcwoKICAgQXJrYWR5IEthbmV2c2t5IChlZGl0 b3IpCiAgIFZNd2FyZQogICA1IENhbWJyaWRnZSBDZW50ZXIKICAgQ2FtYnJpZGdlLCBNQSAgMDIx NDIKICAgVVNBCgogICBQaG9uZTogKzEtNjE3LTUyOC03NzIxCiAgIEVtYWlsOiBhcmthZHlAdm13 YXJlLmNvbQoKCiAgIENhaXRsaW4gQmVzdGxlciAoZWRpdG9yKQogICBDb25zdWx0YW50CiAgIDU1 NSBFIEVsIENhbWlubyBSZWFsICMxMDQKICAgU3Vubnl2YWxlLCBDQSAgOTQwODcKICAgVVNBCgog ICBQaG9uZTogKzEtOTQ5LTUyOC0zMDg1CiAgIEVtYWlsOiBjYWl0QGFzb21pLmNvbQoKCgoKCgoK CkthbmV2c2t5LCBldCBhbC4gICAgICAgICBFeHBpcmVzIE9jdG9iZXIgNSwgMjAxMSAgICAgICAg ICAgICAgIFtQYWdlIDE3XQoMCkludGVybmV0LURyYWZ0ICAgRW5oYW5jZWQgUkRNQSBDb25uZWN0 aW9uIEVzdGFibGlzaG1lbnQgICAgICAgQXByaWwgMjAxMQoKCiAgIFJvYmVydCBTaGFycAogICBJ bnRlbAogICBMQUQgSGlnaCBQZXJmb3JtYW5jZSBNZXNzYWdlIFBhc3NpbmcsIE1haWxzdG9wOiBB TjEtV1RSMQogICAxNTAxIFNvdXRoIE1vcGFjLCBTdWl0ZSA0MDAKICAgQXVzdGluLCBUWCAgNzg3 NDYKICAgVVNBCgogICBQaG9uZTogKzEtNTEyLTQ5My0zMjQyCiAgIEVtYWlsOiByb2JlcnQuby5z aGFycEBpbnRlbC5jb20KCgogICBTdGV2ZSBXaXNlCiAgIE9wZW4gR3JpZCBDb21wdXRpbmcKICAg NDAzMCBCcmFrZXIgTGFuZSBTVEUgMTMwCiAgIEF1c3RpbiwgVFggIDc4NzU5CiAgIFVTQQoKICAg UGhvbmU6ICsxLTUxMi0zNDMtOTE5NiB4MTAxCiAgIEVtYWlsOiBzd2lzZUBvcGVuZ3JpZGNvbXB1 dGluZy5jb20KCgoKCgoKCgoKCgoKCgoKCgoKCgoKCgoKCgoKCgoKCgpLYW5ldnNreSwgZXQgYWwu ICAgICAgICAgRXhwaXJlcyBPY3RvYmVyIDUsIDIwMTEgICAgICAgICAgICAgICBbUGFnZSAxOF0K DAo= --001636e0a9bfb8d0e9049ff9a967-- From swise@opengridcomputing.com Sun Apr 3 14:34:59 2011 Return-Path: X-Original-To: storm@core3.amsl.com Delivered-To: storm@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id 3752E3A689E for ; Sun, 3 Apr 2011 14:34:59 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.264 X-Spam-Level: X-Spam-Status: No, score=-2.264 tagged_above=-999 required=5 tests=[BAYES_00=-2.599, HTML_MESSAGE=0.001, IP_NOT_FRIENDLY=0.334] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id qO28chwsmrdl for ; Sun, 3 Apr 2011 14:34:56 -0700 (PDT) Received: from linode.aoot.com (linode.aoot.com [69.164.194.13]) by core3.amsl.com (Postfix) with ESMTP id A1DBA3A6882 for ; Sun, 3 Apr 2011 14:34:56 -0700 (PDT) Received: from [172.16.0.97] (pool-71-114-244-108.austtx.dsl-w.verizon.net [71.114.244.108]) by linode.aoot.com (Postfix) with ESMTP id 8235B8223; Sun, 3 Apr 2011 16:36:37 -0500 (CDT) Message-ID: <4D98E86A.60403@opengridcomputing.com> Date: Sun, 03 Apr 2011 16:36:42 -0500 From: Steve Wise User-Agent: Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US; rv:1.9.2.15) Gecko/20110303 Lightning/1.0b2 Thunderbird/3.1.9 MIME-Version: 1.0 To: arkady kanevsky References: <7C4DFCE962635144B8FAE8CA11D0BF1E03D74C7183@MX14A.corp.emc.com> <7C4DFCE962635144B8FAE8CA11D0BF1E03E5E9530C@MX14A.corp.emc.com> <76DBE161893C324BA6D4BB507EE4C3849510935E96@IRVEXCHCCR02.corp.ad.broadcom.com> In-Reply-To: Content-Type: multipart/alternative; boundary="------------050809050500050409070704" Cc: storm@ietf.org Subject: Re: [storm] MPA Draft - Review X-BeenThere: storm@ietf.org X-Mailman-Version: 2.1.9 Precedence: list List-Id: Storage Maintenance WG List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 03 Apr 2011 21:34:59 -0000 This is a multi-part message in MIME format. --------------050809050500050409070704 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Hey Arkady, It does seem like you did the section 9 changes Bob and I requested: ---- Change the IRD definition on the request from "In request: the Initiator requested responder IRD for the connection." to "In request: the Initiator initial IRD setting for the connection." ---- Thanks, Steve. On 4/2/2011 8:35 PM, arkady kanevsky wrote: > All, > updated version 04 is attached. > > Hemal, > Thanks for catching it. > I had fixed the first issue. I had added reference to FPDU in the > FULPDU definition for the second. > > David, > Please, check to see that you comments are addressed. > > Steve and Robert, > please, check that you comment is fixed correctly. > > Once I get positive feedback from all of you, I will submit the version. > > Thanks, > Arkady > > On Thu, Mar 31, 2011 at 2:43 PM, Hemal Shah > wrote: > > I have some comments on -03 draft: > > 1. In section 10, it is written that "Enhanced MPA Initiator > and Responder: If a responder receives an enhanced MPA > message, it MUST respond with an unenhanced MPA message." I > think it should be written that the responder must respond > with an enhanced MPA message. It appears like a typo to me. > 2. I find the use of FULPDU confusing in this draft. RFC5044 > does not define term FULPDU. RFC5044 uses term FPDU to refer > to Framed Protocol Data Unit. I suggest that we use term > FPDU instead of FULPDU in the draft. > > Hemal > -----Original Message----- > From: storm-bounces@ietf.org > [mailto:storm-bounces@ietf.org] On Behalf Of david.black@emc.com > > Sent: Thursday, March 31, 2011 7:48 AM > To: storm@ietf.org > Subject: Re: [storm] MPA Draft - Review > Importance: High > The -03 version of the MPA draft has addressed all of the issues > from my review, and . Unfortunately, I need some minor edits for > clarity before I can send this on to our AD with a publication > request. Would the authors please submit a -04 version with the > following two changes quickly. > Section 9 (end) > OLD > The peer-to-peer negotiation for the RTR message follows the > following order: > Initiator -->: Sets Control Flags it is capable to send for RTR > Responder <--: Sets Control Flags it is capable to receive for RTR > Initiator -->: The first message send MUST be a negotiated RTR > NEW > The peer-to-peer negotiation for the RTR message follows the > following order: > Initiator -->: Sets Control Flags to indicate > Initiator-supported forms of RTR > Responder <--: Sets Control Flags to indicate > Responder-supported forms of RTR > Initiator -->: If at least one form of RTR is supported by both > Initiator and > Responder, then the first message sent MUST be an RTR > using a form supported > by both the Initiator and Responder. > Section 10 > OLD > In > this case initiator CAN attempt to establish RDMA connection > using > unenhanced MPA protocol as defined in [RFC5044] and let ULP deal > with ORD and IRD, and peer-to-peer negotiations. > NEW > In > this case initiator MAY attempt to establish RDMA connection > using > ------------------------->^^^ > unenhanced MPA protocol as defined in [RFC5044] if this > protocol is > compatible with the application, and let ULP deal with ORD > and IRD, > and peer-to-peer negotiations. > Ordinarily, I'd write an RFC Editor Note for small changes like > these, but they're sufficiently critical to interoperability that > I'd prefer to have a new draft version that contains them. > Thanks, > --David > > -----Original Message----- > > From: Black, David > > Sent: Wednesday, December 22, 2010 9:26 PM > > To: storm@ietf.org > > Cc: Black, David > > Subject: MPA Draft - Review > > > > WG Last Call on this draft has run its course: > > > > Enhanced RDMA Connection Establishment > > draft-ietf-storm-mpa-peer-connect-02 > > > > I've done my review as a WG chair (and the person who will be > shepherding this draft to the ADs and > > IESG): > > > > - This draft is on the right track, but has open issues. > > - Another version of the draft will be needed. > > > > Also, it would be greatly appreciated if a few people other than > the authors could take a look at > > this draft. We have a very good author team on this draft, > whose expertise is beyond doubt, but > > more eyes on this draft would help. > > > > [1] My primary concern is that Section 9 on interoperability is > inadequate: > > > > An initiator SHOULD NOT use the Enhanced DDP Connection > Establishment > > formats or function codes when no enhanced functionality is > desired. > > > > A responder SHOULD continue to accept the unenhanced connection > > requests. > > > > The good news is that the first sentence is ok. > > The bad news is that the second sentence has significant problems: > > - It uses SHOULD instead of MUST. > > - It doesn't lay out behavior for initiator and responder > > Revision mixes. > > IETF interoperability requirements are usually expressed with > MUST, including backwards > > compatibility. If interop with unenhanced implementations is > only a SHOULD, that will need a > > convincing explanation. > > > > There are 3 Initiator/Responder cases that need attention > (New/Old, Old/New and New/New). I think > > they lead to roughly the following: > > > > New/Old: > > - Explain error or failure that the New Initiator will see > because the Old responder > > doesn't support Revision 2 of the MPA protocol. > > - Explain what the Initiator does when it sees that error or > failure. The > > easiest approach is to always retry with Revision 1, but > that won't work > > if the Initiator has to send an RTR (that's the > "convincing explanation" > > for why backwards compatibility is not always possible). > The result > > might be two requirements: > > - If the Initiator has data to send, it MUST retry with > Revision 1. > > - If the Initiator has no data to send, and hence has to > send an RTR, > > the connection setup fails, the TCP connection > closes and that > > failure MUST to be reported to the application. > > > > Old/New: > > - If a responder receives a Revision 1 message, it MUST respond > with a Revision 1 message. > > > > New/New: > > - If a responder receives a Revision 2 message, it MUST respond > with a Revision 2 message. > > > > I found a few other concerns: > > > > [B]In Section 7, we need to get the listing of all the SCTP > function codes into one place. Either > > repeat the definitions of codes 1-4 from RFC 5043, or create an > IANA registry in Section 10 and list > > all 7 codes as its initial contents. > > > > [C] In Section 8, what happens if the responder sends an IRD or > ORD value that's different from the > > corresponding initiator value? Is the responder allowed to > increase the value that was sent? An > > important case to cover is that the initiator sends a valid > value (e.g., 0x2000) but the responder > > returns the 0x3FFF value indicating that negotiation is not > supported. Also, what is the behavior > > of an IRD or ORD that is set to 0x0000? > > > > [D] In contrast, the Section 8 discussion of Control Flag > functionality is in better shape. It > > would be helpful to add a sentence or two indicating when the > RTR occurs (Request ->, <- Reply, RTR > > ->), even though that is discussed earlier in the draft. Also, > it's necessary to state whether > > negotiation of RTR functionality commits the Initiator to using > an RTR (e.g., suppose the initiator > > negotiates control flags to allow an RTR and instead sends an > FULPDU with payload data after > > receiving the Reply - is that ok or is it an error?). > > > > [E] Nit: In the definition of Control Flag A: ULPDU -> FULPDU > > > > Thanks, > > --David > > ---------------------------------------------------- > > David L. Black, Distinguished Engineer > > EMC Corporation, 176 South St., Hopkinton, MA 01748 > > +1 (508) 293-7953 > FAX: +1 (508) 293-7786 > > david.black@emc.com Mobile: > +1 (978) 394-7754 > > ---------------------------------------------------- > _______________________________________________ > storm mailing list > storm@ietf.org > https://www.ietf.org/mailman/listinfo/storm > > _______________________________________________ > storm mailing list > storm@ietf.org > https://www.ietf.org/mailman/listinfo/storm > > > > > -- > Cheers, > Arkady Kanevsky > > > _______________________________________________ > storm mailing list > storm@ietf.org > https://www.ietf.org/mailman/listinfo/storm --------------050809050500050409070704 Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Hey Arkady,

It does seem like you did the section 9 changes Bob and I requested:

----
  Change the IRD definition on the request from "In request: the Initiator requested responder IRD for the
  connection." to "In request: the Initiator initial IRD setting for the connection." 
----

Thanks,

Steve.

On 4/2/2011 8:35 PM, arkady kanevsky wrote:
All,
updated version 04 is attached.

Hemal,
Thanks for catching it.
I had fixed the first issue. I had added reference to FPDU in the FULPDU definition for the second.

David,
Please, check to see that you comments are addressed.

Steve and Robert,
please, check that you comment is fixed correctly.

Once I get positive feedback from all of you, I will submit the version.

Thanks,
Arkady

On Thu, Mar 31, 2011 at 2:43 PM, Hemal Shah <hemal@broadcom.com> wrote:
I have some comments on -03 draft:
 
  1. In section 10, it is written that "Enhanced MPA Initiator and Responder:  If a responder receives an enhanced MPA message, it MUST respond with an unenhanced MPA message." I think it should be written that the responder must respond with an enhanced MPA message. It appears like a typo to me.
  2. I find the use of FULPDU confusing in this draft. RFC5044 does not define term FULPDU. RFC5044 uses term FPDU to refer to Framed Protocol Data Unit. I suggest that we use term FPDU instead of FULPDU in the draft.
 
Hemal
 
-----Original Message-----
From: storm-bounces@ietf.org [mailto:storm-bounces@ietf.org] On Behalf Of david.black@emc.com
Sent: Thursday, March 31, 2011 7:48 AM
To: storm@ietf.org
Subject: Re: [storm] MPA Draft - Review
Importance: High
 
The -03 version of the MPA draft has addressed all of the issues from my review, and .  Unfortunately, I need some minor edits for clarity before I can send this on to our AD with a publication request.  Would the authors please submit a -04 version with the following two changes quickly.
 
Section 9 (end)
 
OLD
 
   The peer-to-peer negotiation for the RTR message follows the
   following order:    
                               
   Initiator -->: Sets Control Flags it is capable to send for RTR     
                               
   Responder <--: Sets Control Flags it is capable to receive for RTR  
                               
   Initiator -->: The first message send MUST be a negotiated RTR
 
NEW
 
   The peer-to-peer negotiation for the RTR message follows the
   following order:    
                               
   Initiator -->: Sets Control Flags to indicate Initiator-supported forms of RTR      
                               
   Responder <--: Sets Control Flags to indicate Responder-supported forms of RTR      
                               
   Initiator -->: If at least one form of RTR is supported by both Initiator and
        Responder, then the first message sent MUST be an RTR using a form supported
        by both the Initiator and Responder.
 
Section 10
 
OLD
      In
      this case initiator CAN attempt to establish RDMA connection using
      unenhanced MPA protocol as defined in [RFC5044] and let ULP deal
      with ORD and IRD, and peer-to-peer negotiations.
 
NEW
 
      In
      this case initiator MAY attempt to establish RDMA connection using
------------------------->^^^
      unenhanced MPA protocol as defined in [RFC5044] if this protocol is
        compatible with the application, and let ULP deal with ORD and IRD,
      and peer-to-peer negotiations.
 
Ordinarily, I'd write an RFC Editor Note for small changes like these, but they're sufficiently critical to interoperability that I'd prefer to have a new draft version that contains them.
 
Thanks,
--David
 
 
> -----Original Message-----
> From: Black, David
> Sent: Wednesday, December 22, 2010 9:26 PM
> Cc: Black, David
> Subject: MPA Draft - Review
>
> WG Last Call on this draft has run its course:
>
>                  Enhanced RDMA Connection Establishment
>                   draft-ietf-storm-mpa-peer-connect-02
>
> I've done my review as a WG chair (and the person who will be shepherding this draft to the ADs and
> IESG):
>
> - This draft is on the right track, but has open issues.
> - Another version of the draft will be needed.
>
> Also, it would be greatly appreciated if a few people other than the authors could take a look at
> this draft.  We have a very good author team on this draft, whose expertise is beyond doubt, but
> more eyes on this draft would help.
>
> [1] My primary concern is that Section 9 on interoperability is inadequate:
>
>    An initiator SHOULD NOT use the Enhanced DDP Connection Establishment
>    formats or function codes when no enhanced functionality is desired.
>
>    A responder SHOULD continue to accept the unenhanced connection
>    requests.
>
> The good news is that the first sentence is ok.
> The bad news is that the second sentence has significant problems:
>        - It uses SHOULD instead of MUST.
>        - It doesn't lay out behavior for initiator and responder
>                Revision mixes.
> IETF interoperability requirements are usually expressed with MUST, including backwards
> compatibility.  If interop with unenhanced implementations is only a SHOULD, that will need a
> convincing explanation.
>
> There are 3 Initiator/Responder cases that need attention (New/Old, Old/New and New/New).  I think
> they lead to roughly the following:
>
> New/Old:
> - Explain error or failure that the New Initiator will see because the Old responder
>        doesn't support Revision 2 of the MPA protocol.
> - Explain what the Initiator does when it sees that error or failure.  The
>        easiest approach is to always retry with Revision 1, but that won't work
>        if the Initiator has to send an RTR (that's the "convincing explanation"
>        for why backwards compatibility is not always possible).  The result
>        might be two requirements:
>        - If the Initiator has data to send, it MUST retry with Revision 1.
>        - If the Initiator has no data to send, and hence has to send an RTR,
>                the connection setup fails, the TCP connection closes and that
>                failure MUST to be reported to the application.
>
> Old/New:
> - If a responder receives a Revision 1 message, it MUST respond with a Revision 1 message.
>
> New/New:
> - If a responder receives a Revision 2 message, it MUST respond with a Revision 2 message.
>
> I found a few other concerns:
>
> [B]In Section 7, we need to get the listing of all the SCTP function codes into one place.  Either
> repeat the definitions of codes 1-4 from RFC 5043, or create an IANA registry in Section 10 and list
> all 7 codes as its initial contents.
>
> [C] In Section 8, what happens if the responder sends an IRD or ORD value that's different from the
> corresponding initiator value?  Is the responder allowed to increase the value that was sent?  An
> important case to cover is that the initiator sends a valid value (e.g., 0x2000) but the responder
> returns the 0x3FFF value indicating that negotiation is not supported.  Also, what is the behavior
> of an IRD or ORD that is set to 0x0000?
>
> [D] In contrast, the Section 8 discussion of Control Flag functionality is in better shape.  It
> would be helpful to add a sentence or two indicating when the RTR occurs (Request ->, <- Reply, RTR
> ->), even though that is discussed earlier in the draft.  Also, it's necessary to state whether
> negotiation of RTR functionality commits the Initiator to using an RTR (e.g., suppose the initiator
> negotiates control flags to allow an RTR and instead sends an FULPDU with payload data after
> receiving the Reply - is that ok or is it an error?).
>
> [E] Nit: In the definition of Control Flag A: ULPDU -> FULPDU
>
> Thanks,
> --David
> ----------------------------------------------------
> David L. Black, Distinguished Engineer
> EMC Corporation, 176 South St., Hopkinton, MA  01748
> +1 (508) 293-7953             FAX: +1 (508) 293-7786
> david.black@emc.com        Mobile: +1 (978) 394-7754
> ----------------------------------------------------
 
_______________________________________________
storm mailing list
 
 

_______________________________________________
storm mailing list
storm@ietf.org
https://www.ietf.org/mailman/listinfo/storm




--
Cheers,
Arkady Kanevsky

_______________________________________________
storm mailing list
storm@ietf.org
https://www.ietf.org/mailman/listinfo/storm

--------------050809050500050409070704-- From swise@opengridcomputing.com Sun Apr 3 17:47:08 2011 Return-Path: X-Original-To: storm@core3.amsl.com Delivered-To: storm@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id BF49B28C0CE for ; Sun, 3 Apr 2011 17:47:08 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.264 X-Spam-Level: X-Spam-Status: No, score=-2.264 tagged_above=-999 required=5 tests=[BAYES_00=-2.599, HTML_MESSAGE=0.001, IP_NOT_FRIENDLY=0.334] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id EHEHR4ugYeK6 for ; Sun, 3 Apr 2011 17:47:06 -0700 (PDT) Received: from linode.aoot.com (linode.aoot.com [69.164.194.13]) by core3.amsl.com (Postfix) with ESMTP id C965C28B56A for ; Sun, 3 Apr 2011 17:47:06 -0700 (PDT) Received: from [172.16.0.97] (pool-71-114-244-108.austtx.dsl-w.verizon.net [71.114.244.108]) by linode.aoot.com (Postfix) with ESMTP id EBFAC81FD; Sun, 3 Apr 2011 19:48:47 -0500 (CDT) Message-ID: <4D991573.2050400@opengridcomputing.com> Date: Sun, 03 Apr 2011 19:48:51 -0500 From: Steve Wise User-Agent: Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US; rv:1.9.2.15) Gecko/20110303 Lightning/1.0b2 Thunderbird/3.1.9 MIME-Version: 1.0 To: arkady kanevsky References: <7C4DFCE962635144B8FAE8CA11D0BF1E03D74C7183@MX14A.corp.emc.com> <7C4DFCE962635144B8FAE8CA11D0BF1E03E5E9530C@MX14A.corp.emc.com> <76DBE161893C324BA6D4BB507EE4C3849510935E96@IRVEXCHCCR02.corp.ad.broadcom.com> <4D98E86A.60403@opengridcomputing.com> In-Reply-To: <4D98E86A.60403@opengridcomputing.com> Content-Type: multipart/alternative; boundary="------------050503080708010708030508" Cc: storm@ietf.org Subject: Re: [storm] MPA Draft - Review X-BeenThere: storm@ietf.org X-Mailman-Version: 2.1.9 Precedence: list List-Id: Storage Maintenance WG List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 04 Apr 2011 00:47:08 -0000 This is a multi-part message in MIME format. --------------050503080708010708030508 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit On 4/3/2011 4:36 PM, Steve Wise wrote: > Hey Arkady, > > It does seem like you did the section 9 changes Bob and I requested: To clarify: please modify section 9 as I've shown below.. > ---- > Change the IRD definition on the request from "In request: the > Initiator requested responder IRD for the > connection." to "In request: the Initiator initial IRD setting for > the connection." > ---- > > Thanks, > > Steve. > > On 4/2/2011 8:35 PM, arkady kanevsky wrote: >> All, >> updated version 04 is attached. >> >> Hemal, >> Thanks for catching it. >> I had fixed the first issue. I had added reference to FPDU in the >> FULPDU definition for the second. >> >> David, >> Please, check to see that you comments are addressed. >> >> Steve and Robert, >> please, check that you comment is fixed correctly. >> >> Once I get positive feedback from all of you, I will submit the version. >> >> Thanks, >> Arkady >> >> On Thu, Mar 31, 2011 at 2:43 PM, Hemal Shah > > wrote: >> >> I have some comments on -03 draft: >> >> 1. In section 10, it is written that "Enhanced MPA Initiator >> and Responder: If a responder receives an enhanced MPA >> message, it MUST respond with an unenhanced MPA message." I >> think it should be written that the responder must respond >> with an enhanced MPA message. It appears like a typo to me. >> 2. I find the use of FULPDU confusing in this draft. RFC5044 >> does not define term FULPDU. RFC5044 uses term FPDU to >> refer to Framed Protocol Data Unit. I suggest that we use >> term FPDU instead of FULPDU in the draft. >> >> Hemal >> -----Original Message----- >> From: storm-bounces@ietf.org >> [mailto:storm-bounces@ietf.org] On Behalf Of david.black@emc.com >> >> Sent: Thursday, March 31, 2011 7:48 AM >> To: storm@ietf.org >> Subject: Re: [storm] MPA Draft - Review >> Importance: High >> The -03 version of the MPA draft has addressed all of the issues >> from my review, and . Unfortunately, I need some minor edits for >> clarity before I can send this on to our AD with a publication >> request. Would the authors please submit a -04 version with the >> following two changes quickly. >> Section 9 (end) >> OLD >> The peer-to-peer negotiation for the RTR message follows the >> following order: >> Initiator -->: Sets Control Flags it is capable to send for RTR >> Responder <--: Sets Control Flags it is capable to receive for >> RTR >> Initiator -->: The first message send MUST be a negotiated RTR >> NEW >> The peer-to-peer negotiation for the RTR message follows the >> following order: >> Initiator -->: Sets Control Flags to indicate >> Initiator-supported forms of RTR >> Responder <--: Sets Control Flags to indicate >> Responder-supported forms of RTR >> Initiator -->: If at least one form of RTR is supported by >> both Initiator and >> Responder, then the first message sent MUST be an RTR >> using a form supported >> by both the Initiator and Responder. >> Section 10 >> OLD >> In >> this case initiator CAN attempt to establish RDMA >> connection using >> unenhanced MPA protocol as defined in [RFC5044] and let ULP >> deal >> with ORD and IRD, and peer-to-peer negotiations. >> NEW >> In >> this case initiator MAY attempt to establish RDMA >> connection using >> ------------------------->^^^ >> unenhanced MPA protocol as defined in [RFC5044] if this >> protocol is >> compatible with the application, and let ULP deal with >> ORD and IRD, >> and peer-to-peer negotiations. >> Ordinarily, I'd write an RFC Editor Note for small changes like >> these, but they're sufficiently critical to interoperability that >> I'd prefer to have a new draft version that contains them. >> Thanks, >> --David >> > -----Original Message----- >> > From: Black, David >> > Sent: Wednesday, December 22, 2010 9:26 PM >> > To: storm@ietf.org >> > Cc: Black, David >> > Subject: MPA Draft - Review >> > >> > WG Last Call on this draft has run its course: >> > >> > Enhanced RDMA Connection Establishment >> > draft-ietf-storm-mpa-peer-connect-02 >> > >> > I've done my review as a WG chair (and the person who will be >> shepherding this draft to the ADs and >> > IESG): >> > >> > - This draft is on the right track, but has open issues. >> > - Another version of the draft will be needed. >> > >> > Also, it would be greatly appreciated if a few people other >> than the authors could take a look at >> > this draft. We have a very good author team on this draft, >> whose expertise is beyond doubt, but >> > more eyes on this draft would help. >> > >> > [1] My primary concern is that Section 9 on interoperability is >> inadequate: >> > >> > An initiator SHOULD NOT use the Enhanced DDP Connection >> Establishment >> > formats or function codes when no enhanced functionality is >> desired. >> > >> > A responder SHOULD continue to accept the unenhanced connection >> > requests. >> > >> > The good news is that the first sentence is ok. >> > The bad news is that the second sentence has significant problems: >> > - It uses SHOULD instead of MUST. >> > - It doesn't lay out behavior for initiator and responder >> > Revision mixes. >> > IETF interoperability requirements are usually expressed with >> MUST, including backwards >> > compatibility. If interop with unenhanced implementations is >> only a SHOULD, that will need a >> > convincing explanation. >> > >> > There are 3 Initiator/Responder cases that need attention >> (New/Old, Old/New and New/New). I think >> > they lead to roughly the following: >> > >> > New/Old: >> > - Explain error or failure that the New Initiator will see >> because the Old responder >> > doesn't support Revision 2 of the MPA protocol. >> > - Explain what the Initiator does when it sees that error or >> failure. The >> > easiest approach is to always retry with Revision 1, but >> that won't work >> > if the Initiator has to send an RTR (that's the >> "convincing explanation" >> > for why backwards compatibility is not always >> possible). The result >> > might be two requirements: >> > - If the Initiator has data to send, it MUST retry with >> Revision 1. >> > - If the Initiator has no data to send, and hence has to >> send an RTR, >> > the connection setup fails, the TCP connection >> closes and that >> > failure MUST to be reported to the application. >> > >> > Old/New: >> > - If a responder receives a Revision 1 message, it MUST respond >> with a Revision 1 message. >> > >> > New/New: >> > - If a responder receives a Revision 2 message, it MUST respond >> with a Revision 2 message. >> > >> > I found a few other concerns: >> > >> > [B]In Section 7, we need to get the listing of all the SCTP >> function codes into one place. Either >> > repeat the definitions of codes 1-4 from RFC 5043, or create an >> IANA registry in Section 10 and list >> > all 7 codes as its initial contents. >> > >> > [C] In Section 8, what happens if the responder sends an IRD or >> ORD value that's different from the >> > corresponding initiator value? Is the responder allowed to >> increase the value that was sent? An >> > important case to cover is that the initiator sends a valid >> value (e.g., 0x2000) but the responder >> > returns the 0x3FFF value indicating that negotiation is not >> supported. Also, what is the behavior >> > of an IRD or ORD that is set to 0x0000? >> > >> > [D] In contrast, the Section 8 discussion of Control Flag >> functionality is in better shape. It >> > would be helpful to add a sentence or two indicating when the >> RTR occurs (Request ->, <- Reply, RTR >> > ->), even though that is discussed earlier in the draft. Also, >> it's necessary to state whether >> > negotiation of RTR functionality commits the Initiator to using >> an RTR (e.g., suppose the initiator >> > negotiates control flags to allow an RTR and instead sends an >> FULPDU with payload data after >> > receiving the Reply - is that ok or is it an error?). >> > >> > [E] Nit: In the definition of Control Flag A: ULPDU -> FULPDU >> > >> > Thanks, >> > --David >> > ---------------------------------------------------- >> > David L. Black, Distinguished Engineer >> > EMC Corporation, 176 South St., Hopkinton, MA 01748 >> > +1 (508) 293-7953 >> FAX: +1 (508) 293-7786 >> > david.black@emc.com Mobile: >> +1 (978) 394-7754 >> > ---------------------------------------------------- >> _______________________________________________ >> storm mailing list >> storm@ietf.org >> https://www.ietf.org/mailman/listinfo/storm >> >> _______________________________________________ >> storm mailing list >> storm@ietf.org >> https://www.ietf.org/mailman/listinfo/storm >> >> >> >> >> -- >> Cheers, >> Arkady Kanevsky >> >> >> _______________________________________________ >> storm mailing list >> storm@ietf.org >> https://www.ietf.org/mailman/listinfo/storm > > > _______________________________________________ > storm mailing list > storm@ietf.org > https://www.ietf.org/mailman/listinfo/storm --------------050503080708010708030508 Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit On 4/3/2011 4:36 PM, Steve Wise wrote:
Hey Arkady,

It does seem like you did the section 9 changes Bob and I requested:
 
To clarify: please modify section 9 as I've shown below..


----
  Change the IRD definition on the request from "In request: the Initiator requested responder IRD for the
  connection." to "In request: the Initiator initial IRD setting for the connection." 
----

Thanks,

Steve.

On 4/2/2011 8:35 PM, arkady kanevsky wrote:
All,
updated version 04 is attached.

Hemal,
Thanks for catching it.
I had fixed the first issue. I had added reference to FPDU in the FULPDU definition for the second.

David,
Please, check to see that you comments are addressed.

Steve and Robert,
please, check that you comment is fixed correctly.

Once I get positive feedback from all of you, I will submit the version.

Thanks,
Arkady

On Thu, Mar 31, 2011 at 2:43 PM, Hemal Shah <hemal@broadcom.com> wrote:
I have some comments on -03 draft:
 
  1. In section 10, it is written that "Enhanced MPA Initiator and Responder:  If a responder receives an enhanced MPA message, it MUST respond with an unenhanced MPA message." I think it should be written that the responder must respond with an enhanced MPA message. It appears like a typo to me.
  2. I find the use of FULPDU confusing in this draft. RFC5044 does not define term FULPDU. RFC5044 uses term FPDU to refer to Framed Protocol Data Unit. I suggest that we use term FPDU instead of FULPDU in the draft.
 
Hemal
 
-----Original Message-----
From: storm-bounces@ietf.org [mailto:storm-bounces@ietf.org] On Behalf Of david.black@emc.com
Sent: Thursday, March 31, 2011 7:48 AM
To: storm@ietf.org
Subject: Re: [storm] MPA Draft - Review
Importance: High
 
The -03 version of the MPA draft has addressed all of the issues from my review, and .  Unfortunately, I need some minor edits for clarity before I can send this on to our AD with a publication request.  Would the authors please submit a -04 version with the following two changes quickly.
 
Section 9 (end)
 
OLD
 
   The peer-to-peer negotiation for the RTR message follows the
   following order:    
                               
   Initiator -->: Sets Control Flags it is capable to send for RTR     
                               
   Responder <--: Sets Control Flags it is capable to receive for RTR  
                               
   Initiator -->: The first message send MUST be a negotiated RTR
 
NEW
 
   The peer-to-peer negotiation for the RTR message follows the
   following order:    
                               
   Initiator -->: Sets Control Flags to indicate Initiator-supported forms of RTR      
                               
   Responder <--: Sets Control Flags to indicate Responder-supported forms of RTR      
                               
   Initiator -->: If at least one form of RTR is supported by both Initiator and
        Responder, then the first message sent MUST be an RTR using a form supported
        by both the Initiator and Responder.
 
Section 10
 
OLD
      In
      this case initiator CAN attempt to establish RDMA connection using
      unenhanced MPA protocol as defined in [RFC5044] and let ULP deal
      with ORD and IRD, and peer-to-peer negotiations.
 
NEW
 
      In
      this case initiator MAY attempt to establish RDMA connection using
------------------------->^^^
      unenhanced MPA protocol as defined in [RFC5044] if this protocol is
        compatible with the application, and let ULP deal with ORD and IRD,
      and peer-to-peer negotiations.
 
Ordinarily, I'd write an RFC Editor Note for small changes like these, but they're sufficiently critical to interoperability that I'd prefer to have a new draft version that contains them.
 
Thanks,
--David
 
 
> -----Original Message-----
> From: Black, David
> Sent: Wednesday, December 22, 2010 9:26 PM
> Cc: Black, David
> Subject: MPA Draft - Review
>
> WG Last Call on this draft has run its course:
>
>                  Enhanced RDMA Connection Establishment
>                   draft-ietf-storm-mpa-peer-connect-02
>
> I've done my review as a WG chair (and the person who will be shepherding this draft to the ADs and
> IESG):
>
> - This draft is on the right track, but has open issues.
> - Another version of the draft will be needed.
>
> Also, it would be greatly appreciated if a few people other than the authors could take a look at
> this draft.  We have a very good author team on this draft, whose expertise is beyond doubt, but
> more eyes on this draft would help.
>
> [1] My primary concern is that Section 9 on interoperability is inadequate:
>
>    An initiator SHOULD NOT use the Enhanced DDP Connection Establishment
>    formats or function codes when no enhanced functionality is desired.
>
>    A responder SHOULD continue to accept the unenhanced connection
>    requests.
>
> The good news is that the first sentence is ok.
> The bad news is that the second sentence has significant problems:
>        - It uses SHOULD instead of MUST.
>        - It doesn't lay out behavior for initiator and responder
>                Revision mixes.
> IETF interoperability requirements are usually expressed with MUST, including backwards
> compatibility.  If interop with unenhanced implementations is only a SHOULD, that will need a
> convincing explanation.
>
> There are 3 Initiator/Responder cases that need attention (New/Old, Old/New and New/New).  I think
> they lead to roughly the following:
>
> New/Old:
> - Explain error or failure that the New Initiator will see because the Old responder
>        doesn't support Revision 2 of the MPA protocol.
> - Explain what the Initiator does when it sees that error or failure.  The
>        easiest approach is to always retry with Revision 1, but that won't work
>        if the Initiator has to send an RTR (that's the "convincing explanation"
>        for why backwards compatibility is not always possible).  The result
>        might be two requirements:
>        - If the Initiator has data to send, it MUST retry with Revision 1.
>        - If the Initiator has no data to send, and hence has to send an RTR,
>                the connection setup fails, the TCP connection closes and that
>                failure MUST to be reported to the application.
>
> Old/New:
> - If a responder receives a Revision 1 message, it MUST respond with a Revision 1 message.
>
> New/New:
> - If a responder receives a Revision 2 message, it MUST respond with a Revision 2 message.
>
> I found a few other concerns:
>
> [B]In Section 7, we need to get the listing of all the SCTP function codes into one place.  Either
> repeat the definitions of codes 1-4 from RFC 5043, or create an IANA registry in Section 10 and list
> all 7 codes as its initial contents.
>
> [C] In Section 8, what happens if the responder sends an IRD or ORD value that's different from the
> corresponding initiator value?  Is the responder allowed to increase the value that was sent?  An
> important case to cover is that the initiator sends a valid value (e.g., 0x2000) but the responder
> returns the 0x3FFF value indicating that negotiation is not supported.  Also, what is the behavior
> of an IRD or ORD that is set to 0x0000?
>
> [D] In contrast, the Section 8 discussion of Control Flag functionality is in better shape.  It
> would be helpful to add a sentence or two indicating when the RTR occurs (Request ->, <- Reply, RTR
> ->), even though that is discussed earlier in the draft.  Also, it's necessary to state whether
> negotiation of RTR functionality commits the Initiator to using an RTR (e.g., suppose the initiator
> negotiates control flags to allow an RTR and instead sends an FULPDU with payload data after
> receiving the Reply - is that ok or is it an error?).
>
> [E] Nit: In the definition of Control Flag A: ULPDU -> FULPDU
>
> Thanks,
> --David
> ----------------------------------------------------
> David L. Black, Distinguished Engineer
> EMC Corporation, 176 South St., Hopkinton, MA  01748
> +1 (508) 293-7953             FAX: +1 (508) 293-7786
> david.black@emc.com        Mobile: +1 (978) 394-7754
> ----------------------------------------------------
 
_______________________________________________
storm mailing list
 
 

_______________________________________________
storm mailing list
storm@ietf.org
https://www.ietf.org/mailman/listinfo/storm




--
Cheers,
Arkady Kanevsky

_______________________________________________
storm mailing list
storm@ietf.org
https://www.ietf.org/mailman/listinfo/storm



_______________________________________________
storm mailing list
storm@ietf.org
https://www.ietf.org/mailman/listinfo/storm

--------------050503080708010708030508-- From arkady.kanevsky@gmail.com Mon Apr 4 16:24:20 2011 Return-Path: X-Original-To: storm@core3.amsl.com Delivered-To: storm@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id 580ED3A6805 for ; Mon, 4 Apr 2011 16:24:20 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -3.598 X-Spam-Level: X-Spam-Status: No, score=-3.598 tagged_above=-999 required=5 tests=[BAYES_00=-2.599, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_LOW=-1] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 0OK-YoMNTNpw for ; Mon, 4 Apr 2011 16:24:18 -0700 (PDT) Received: from mail-pv0-f172.google.com (mail-pv0-f172.google.com [74.125.83.172]) by core3.amsl.com (Postfix) with ESMTP id 4FB8D3A67ED for ; Mon, 4 Apr 2011 16:24:18 -0700 (PDT) Received: by pvh1 with SMTP id 1so1044559pvh.31 for ; Mon, 04 Apr 2011 16:26:01 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:in-reply-to:references:date :message-id:subject:from:to:cc:content-type; bh=YCTPdUYGPgPIE4fY9BNVdHVUlY0PPYzw2U8QDDjsoLE=; b=nDuKY+LYAqk7YWQbkxDI7KEW/p++MrBdqJddhQowYRnL/dlg1Qo8Z/AF/h2SQ9JEWC 5BbN6OiEyKXGvwSLUQeBwBzKN6H4CjoAVI1KrZeZBUibJsxhsONdn9h8NDReMLsv6pP9 QXz8c3/UWdlm9Yq0CxIvlL7b5QsBr5A8oUFNs= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; b=nd+PtRnn3bolRFzY4f/SQ0xIDsX4kiDmhdywuD2OfdMSHMc8WEEMACBTF+bGZLjydu VYUCVG3Y6VRXT81RWj/ajnAGIdGBPrHwCh9Tro7PkcaeXQv2yH5C0GAGUlOXb8rtXBs2 Ka1VGL2XUpGBTFn50LQ/Hwv65bOHdox0Drh44= MIME-Version: 1.0 Received: by 10.142.44.5 with SMTP id r5mr6683117wfr.300.1301959561096; Mon, 04 Apr 2011 16:26:01 -0700 (PDT) Received: by 10.142.185.20 with HTTP; Mon, 4 Apr 2011 16:26:00 -0700 (PDT) In-Reply-To: <4D98E86A.60403@opengridcomputing.com> References: <7C4DFCE962635144B8FAE8CA11D0BF1E03D74C7183@MX14A.corp.emc.com> <7C4DFCE962635144B8FAE8CA11D0BF1E03E5E9530C@MX14A.corp.emc.com> <76DBE161893C324BA6D4BB507EE4C3849510935E96@IRVEXCHCCR02.corp.ad.broadcom.com> <4D98E86A.60403@opengridcomputing.com> Date: Mon, 4 Apr 2011 19:26:00 -0400 Message-ID: From: arkady kanevsky To: Steve Wise , "Sharp, Robert O" Content-Type: multipart/alternative; boundary=000e0cd2e00e26b05704a020151d Cc: storm@ietf.org Subject: Re: [storm] MPA Draft - Review X-BeenThere: storm@ietf.org X-Mailman-Version: 2.1.9 Precedence: list List-Id: Storage Maintenance WG List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 04 Apr 2011 23:24:20 -0000 --000e0cd2e00e26b05704a020151d Content-Type: text/plain; charset=ISO-8859-1 Steve and Bob, I changed it to "In request: the Initiator desired responder IRD for the connection." as you asked. I can change it to "initial" instead of "desired". Arkady On Sun, Apr 3, 2011 at 5:36 PM, Steve Wise wrote: > Hey Arkady, > > It does seem like you did the section 9 changes Bob and I requested: > > ---- > > Change the IRD definition on the request from "In request: the Initiator > requested responder IRD for the > connection." to "In request: the Initiator initial IRD setting for the > connection." > ---- > > Thanks, > > Steve. > > > On 4/2/2011 8:35 PM, arkady kanevsky wrote: > > All, > updated version 04 is attached. > > Hemal, > Thanks for catching it. > I had fixed the first issue. I had added reference to FPDU in the FULPDU > definition for the second. > > David, > Please, check to see that you comments are addressed. > > Steve and Robert, > please, check that you comment is fixed correctly. > > Once I get positive feedback from all of you, I will submit the version. > > Thanks, > Arkady > > On Thu, Mar 31, 2011 at 2:43 PM, Hemal Shah wrote: > >> I have some comments on -03 draft: >> >> >> 1. In section 10, it is written that "Enhanced MPA Initiator and >> Responder: If a responder receives an enhanced MPA message, it MUST respond >> with an unenhanced MPA message." I think it should be written that the >> responder must respond with an enhanced MPA message. It appears like a typo >> to me. >> 2. I find the use of FULPDU confusing in this draft. RFC5044 does not >> define term FULPDU. RFC5044 uses term FPDU to refer to Framed Protocol Data >> Unit. I suggest that we use term FPDU instead of FULPDU in the draft. >> >> >> Hemal >> >> -----Original Message----- >> From: storm-bounces@ietf.org [mailto:storm-bounces@ietf.org] >> On Behalf Of david.black@emc.com >> Sent: Thursday, March 31, 2011 7:48 AM >> To: storm@ietf.org >> Subject: Re: [storm] MPA Draft - Review >> Importance: High >> >> The -03 version of the MPA draft has addressed all of the issues from my >> review, and . Unfortunately, I need some minor edits for clarity before I >> can send this on to our AD with a publication request. Would the authors >> please submit a -04 version with the following two changes quickly. >> >> Section 9 (end) >> >> OLD >> >> The peer-to-peer negotiation for the RTR message follows the >> following order: >> >> Initiator -->: Sets Control Flags it is capable to send for RTR >> >> Responder <--: Sets Control Flags it is capable to receive for RTR >> >> Initiator -->: The first message send MUST be a negotiated RTR >> >> NEW >> >> The peer-to-peer negotiation for the RTR message follows the >> following order: >> >> Initiator -->: Sets Control Flags to indicate Initiator-supported forms >> of RTR >> >> Responder <--: Sets Control Flags to indicate Responder-supported forms >> of RTR >> >> Initiator -->: If at least one form of RTR is supported by both >> Initiator and >> Responder, then the first message sent MUST be an RTR using a form >> supported >> by both the Initiator and Responder. >> >> Section 10 >> >> OLD >> In >> this case initiator CAN attempt to establish RDMA connection using >> unenhanced MPA protocol as defined in [RFC5044] and let ULP deal >> with ORD and IRD, and peer-to-peer negotiations. >> >> NEW >> >> In >> this case initiator MAY attempt to establish RDMA connection using >> ------------------------->^^^ >> unenhanced MPA protocol as defined in [RFC5044] if this protocol is >> compatible with the application, and let ULP deal with ORD and >> IRD, >> and peer-to-peer negotiations. >> >> Ordinarily, I'd write an RFC Editor Note for small changes like these, but >> they're sufficiently critical to interoperability that I'd prefer to have a >> new draft version that contains them. >> >> Thanks, >> --David >> >> >> > -----Original Message----- >> > From: Black, David >> > Sent: Wednesday, December 22, 2010 9:26 PM >> > To: storm@ietf.org >> > Cc: Black, David >> > Subject: MPA Draft - Review >> > >> > WG Last Call on this draft has run its course: >> > >> > Enhanced RDMA Connection Establishment >> > draft-ietf-storm-mpa-peer-connect-02 >> > >> > I've done my review as a WG chair (and the person who will be >> shepherding this draft to the ADs and >> > IESG): >> > >> > - This draft is on the right track, but has open issues. >> > - Another version of the draft will be needed. >> > >> > Also, it would be greatly appreciated if a few people other than the >> authors could take a look at >> > this draft. We have a very good author team on this draft, whose >> expertise is beyond doubt, but >> > more eyes on this draft would help. >> > >> > [1] My primary concern is that Section 9 on interoperability is >> inadequate: >> > >> > An initiator SHOULD NOT use the Enhanced DDP Connection Establishment >> > formats or function codes when no enhanced functionality is desired. >> > >> > A responder SHOULD continue to accept the unenhanced connection >> > requests. >> > >> > The good news is that the first sentence is ok. >> > The bad news is that the second sentence has significant problems: >> > - It uses SHOULD instead of MUST. >> > - It doesn't lay out behavior for initiator and responder >> > Revision mixes. >> > IETF interoperability requirements are usually expressed with MUST, >> including backwards >> > compatibility. If interop with unenhanced implementations is only a >> SHOULD, that will need a >> > convincing explanation. >> > >> > There are 3 Initiator/Responder cases that need attention (New/Old, >> Old/New and New/New). I think >> > they lead to roughly the following: >> > >> > New/Old: >> > - Explain error or failure that the New Initiator will see because the >> Old responder >> > doesn't support Revision 2 of the MPA protocol. >> > - Explain what the Initiator does when it sees that error or failure. >> The >> > easiest approach is to always retry with Revision 1, but that >> won't work >> > if the Initiator has to send an RTR (that's the "convincing >> explanation" >> > for why backwards compatibility is not always possible). The >> result >> > might be two requirements: >> > - If the Initiator has data to send, it MUST retry with Revision >> 1. >> > - If the Initiator has no data to send, and hence has to send an >> RTR, >> > the connection setup fails, the TCP connection closes and >> that >> > failure MUST to be reported to the application. >> > >> > Old/New: >> > - If a responder receives a Revision 1 message, it MUST respond with a >> Revision 1 message. >> > >> > New/New: >> > - If a responder receives a Revision 2 message, it MUST respond with a >> Revision 2 message. >> > >> > I found a few other concerns: >> > >> > [B]In Section 7, we need to get the listing of all the SCTP function >> codes into one place. Either >> > repeat the definitions of codes 1-4 from RFC 5043, or create an IANA >> registry in Section 10 and list >> > all 7 codes as its initial contents. >> > >> > [C] In Section 8, what happens if the responder sends an IRD or ORD >> value that's different from the >> > corresponding initiator value? Is the responder allowed to increase the >> value that was sent? An >> > important case to cover is that the initiator sends a valid value (e.g., >> 0x2000) but the responder >> > returns the 0x3FFF value indicating that negotiation is not supported. >> Also, what is the behavior >> > of an IRD or ORD that is set to 0x0000? >> > >> > [D] In contrast, the Section 8 discussion of Control Flag functionality >> is in better shape. It >> > would be helpful to add a sentence or two indicating when the RTR occurs >> (Request ->, <- Reply, RTR >> > ->), even though that is discussed earlier in the draft. Also, it's >> necessary to state whether >> > negotiation of RTR functionality commits the Initiator to using an RTR >> (e.g., suppose the initiator >> > negotiates control flags to allow an RTR and instead sends an FULPDU >> with payload data after >> > receiving the Reply - is that ok or is it an error?). >> > >> > [E] Nit: In the definition of Control Flag A: ULPDU -> FULPDU >> > >> > Thanks, >> > --David >> > ---------------------------------------------------- >> > David L. Black, Distinguished Engineer >> > EMC Corporation, 176 South St., Hopkinton, MA 01748 >> > +1 (508) 293-7953 FAX: +1 (508) 293-7786 >> > david.black@emc.com Mobile: +1 (978) 394-7754 >> > ---------------------------------------------------- >> >> _______________________________________________ >> storm mailing list >> storm@ietf.org >> https://www.ietf.org/mailman/listinfo/storm >> >> >> >> _______________________________________________ >> storm mailing list >> storm@ietf.org >> https://www.ietf.org/mailman/listinfo/storm >> >> > > > -- > Cheers, > Arkady Kanevsky > > > _______________________________________________ > storm mailing liststorm@ietf.orghttps://www.ietf.org/mailman/listinfo/storm > > > -- Cheers, Arkady Kanevsky --000e0cd2e00e26b05704a020151d Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Steve and Bob,
I changed it to
"In request: the Initiator desire= d responder IRD
for the connection." as you asked.
I can change = it to "initial" instead of "desired".
Arkady

On Sun, Apr 3, 2011 at 5:36 PM, Steve Wise <swise@openg= ridcomputing.com> wrote:
=20 =20 =20 =20
Hey Arkady,

It does seem like you did the section 9 changes Bob and I requested:
----

=A0 Change the IRD definition on the request from "In request: the Initiator requested responder IRD for the
=A0 connection." to "In request: the Initiator initial IRD se= tting for the connection."=A0
----

Thanks,

Steve.


On 4/2/2011 8:35 PM, arkady kanevsky wrote:
All,
updated version 04 is attached.

Hemal,
Thanks for catching it.
I had fixed the first issue. I had added reference to FPDU in the FULPDU definition for the second.

David,
Please, check to see that you comments are addressed.

Steve and Robert,
please, check that you comment is fixed correctly.

Once I get positive feedback from all of you, I will submit the version.

Thanks,
Arkady

On Thu, Mar 31, 2011 at 2:43 PM, Hemal Shah <hemal@broadcom.com> wrote:
I have some comments on -03 draft:
=A0
  1. In section 10, it is written that "Enhanced MPA Initiator and Responder:=A0 If a responder receives an enhanced MPA message, it MUST respond with an unenhanced MPA message." I think it should be writte= n that the responder must respond with an enhanced MPA message. It appears like a typo to me.
  2. I find the use of FULPDU confusing in this draft. RFC5044 does not define term FULPDU. RFC5044 uses term FPDU to refer to Framed Protocol Data Unit. I suggest that we use term FPDU instead of FULPDU in the draft.
=A0
Hemal
=A0
-----Original Message-----
From: storm-bounces@ietf.org [mailto:storm-bounces@ietf.org] On Behalf Of david.black@emc.com
Sent: Thursday, March 31, 2011 7:48 AM
To: s= torm@ietf.org
Subject: Re: [storm] MPA Draft - Review
Importance: High
=A0
The -03 version of the MPA draft has addressed all of the issues from my review, and .=A0 Unfortunately, I need some minor edits for clarity before I can send this on to our AD with a publication request.=A0 Would the authors please submit a -04 version with the following two changes quickly.
=A0
Section 9 (end)
=A0
OLD
=A0
=A0=A0 The peer-to-peer negotiation for the RTR message follows the
=A0=A0 following order:=A0=A0=A0=A0
=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0= =A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0
=A0=A0 Initiator -->: Sets Control Flags it is capable to send for RTR=A0=A0=A0=A0=A0
=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0= =A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0
=A0=A0 Responder <--: Sets Control Flags it is capable to receive for RTR=A0=A0
=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0= =A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0
=A0=A0 Initiator -->: The first message send MUST be a negotiated RTR
=A0
NEW
=A0
=A0=A0 The peer-to-peer negotiation for the RTR message follows the
=A0=A0 following order:=A0=A0=A0=A0
=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0= =A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0
=A0=A0 Initiator -->: Sets Control Flags to indicate Initiator-supported forms of RTR=A0=A0=A0=A0= =A0=A0
=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0= =A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0
=A0=A0 Responder <--: Sets Control Flags to indicate Responder-supported forms of RTR=A0=A0=A0=A0= =A0=A0
=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0= =A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0
=A0=A0 Initiator -->: If at least one form of RTR is supported by both Initiator and
=A0=A0=A0=A0=A0=A0=A0 Responder, then the first mess= age sent MUST be an RTR using a form supported
=A0=A0=A0=A0=A0=A0=A0 by both the Initiator and Resp= onder.
=A0
Section 10
=A0
OLD
=A0=A0=A0=A0=A0 In
=A0=A0=A0=A0=A0 this case initiator CAN attempt to establish RDMA connection using
=A0=A0=A0=A0=A0 unenhanced MPA protocol as defined i= n [RFC5044] and let ULP deal
=A0=A0=A0=A0=A0 with ORD and IRD, and peer-to-peer negotiations.
=A0
NEW
=A0
=A0=A0=A0=A0=A0 In
=A0=A0=A0=A0=A0 this case initiator MAY attempt to establish RDMA connection using
------------------------->^^^
=A0=A0=A0=A0=A0 unenhanced MPA protocol as defined i= n [RFC5044] if this protocol is
=A0=A0=A0=A0=A0=A0=A0 compatible with the applicatio= n, and let ULP deal with ORD and IRD,
=A0=A0=A0=A0=A0 and peer-to-peer negotiations.
=A0
Ordinarily, I'd write an RFC Editor Note for small changes like these, but they're sufficiently critical to interoperability that I'd prefer to hav= e a new draft version that contains them.
=A0
Thanks,
--David
=A0
=A0
> -----Original Message-----
> From: Black, David
> Sent: Wednesday, December 22, 2010 9:26 PM
> Cc: Black, David
> Subject: MPA Draft - Review
>
> WG Last Call on this draft has run its course:
>
>=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0= =A0 Enhanced RDMA Connection Establishment
>=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0= =A0=A0 draft-ietf-storm-mpa-peer-connect-02
>
> I've done my review as a WG chair (and the person who will be shepherding this draft to the ADs and
> IESG):
>
> - This draft is on the right track, but has open issues.
> - Another version of the draft will be needed.
>
> Also, it would be greatly appreciated if a few people other than the authors could take a look at
> this draft.=A0 We have a very good author team on this draft, whose expertise is beyond doubt, but
> more eyes on this draft would help.
>
> [1] My primary concern is that Section 9 on interoperability is inadequate:
>
>=A0=A0=A0 An initiator SHOULD NOT use the Enhanc= ed DDP Connection Establishment
>=A0=A0=A0 formats or function codes when no enhanced functionality is desired.
>
>=A0=A0=A0 A responder SHOULD continue to accept = the unenhanced connection
>=A0=A0=A0 requests.
>
> The good news is that the first sentence is ok.
> The bad news is that the second sentence has significant problems:
>=A0=A0=A0=A0=A0=A0=A0 - It uses SHOULD instead o= f MUST.
>=A0=A0=A0=A0=A0=A0=A0 - It doesn't lay out b= ehavior for initiator and responder
>=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 Re= vision mixes.
> IETF interoperability requirements are usually expressed with MUST, including backwards
> compatibility.=A0 If interop with unenhanced implementations is only a SHOULD, that will need a
> convincing explanation.
>
> There are 3 Initiator/Responder cases that need attention (New/Old, Old/New and New/New).=A0 I think
> they lead to roughly the following:
>
> New/Old:
> - Explain error or failure that the New Initiator will see because the Old responder
>=A0=A0=A0=A0=A0=A0=A0 doesn't support Revisi= on 2 of the MPA protocol.
> - Explain what the Initiator does when it sees that error or failure.=A0 The
>=A0=A0=A0=A0=A0=A0=A0 easiest approach is to alw= ays retry with Revision 1, but that won't work
>=A0=A0=A0=A0=A0=A0=A0 if the Initiator has to se= nd an RTR (that's the "convincing explanation"
>=A0=A0=A0=A0=A0=A0=A0 for why backwards compatib= ility is not always possible).=A0 The result
>=A0=A0=A0=A0=A0=A0=A0 might be two requirements:=
>=A0=A0=A0=A0=A0=A0=A0 - If the Initiator has dat= a to send, it MUST retry with Revision 1.
>=A0=A0=A0=A0=A0=A0=A0 - If the Initiator has no = data to send, and hence has to send an RTR,
>=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 th= e connection setup fails, the TCP connection closes and that
>=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 fa= ilure MUST to be reported to the application.
>
> Old/New:
> - If a responder receives a Revision 1 message, it MUST respond with a Revision 1 message.
>
> New/New:
> - If a responder receives a Revision 2 message, it MUST respond with a Revision 2 message.
>
> I found a few other concerns:
>
> [B]In Section 7, we need to get the listing of all the SCTP function codes into one place.=A0 Either
> repeat the definitions of codes 1-4 from RFC 5043, or create an IANA registry in Section 10 and list
> all 7 codes as its initial contents.
>
> [C] In Section 8, what happens if the responder sends an IRD or ORD value that's differen= t from the
> corresponding initiator value?=A0 Is the responder allowed to increase the value that was sent?=A0 An
> important case to cover is that the initiator sends a valid value (e.g., 0x2000) but the responder
> returns the 0x3FFF value indicating that negotiation is not supported.=A0 Also, what is the behavior
> of an IRD or ORD that is set to 0x0000?
>
> [D] In contrast, the Section 8 discussion of Control Flag functionality is in better shape.=A0 It
> would be helpful to add a sentence or two indicating when the RTR occurs (Request ->, <- Reply, RTR
> ->), even though that is discussed earlier in the draft.=A0 Also, it's necessary to st= ate whether
> negotiation of RTR functionality commits the Initiator to using an RTR (e.g., suppose the initiator
> negotiates control flags to allow an RTR and instead sends an FULPDU with payload data after
> receiving the Reply - is that ok or is it an error?).
>
> [E] Nit: In the definition of Control Flag A: ULPDU -> FULPDU
>
> Thanks,
> --David
> ----------------------------------------------------
> David L. Black, Distinguished Engineer
> EMC Corporation, 176 South St., Hopkinton, MA=A0 01748
> +1 (508) 293-7953=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 FAX: +1 (508) 293-7786
> david.black@emc.com=A0=A0=A0=A0=A0=A0=A0 Mobile: +1 (978) 394-7754
> ----------------------------------------------------
=A0
_______________________________________________
storm mailing list
=A0
=A0

_______________________________________________
storm mailing list
storm@ietf.or= g
https://www.ietf.org/mailman/listinfo/storm




--
Cheers,
Arkady Kanevsky

_______________________________________________
storm mailing list
storm@ietf.org
h=
ttps://www.ietf.org/mailman/listinfo/storm




--
Cheers,
Arkady Kanev= sky
--000e0cd2e00e26b05704a020151d-- From swise@opengridcomputing.com Mon Apr 4 18:39:30 2011 Return-Path: X-Original-To: storm@core3.amsl.com Delivered-To: storm@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id A7FC73A683B for ; Mon, 4 Apr 2011 18:39:30 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.264 X-Spam-Level: X-Spam-Status: No, score=-2.264 tagged_above=-999 required=5 tests=[BAYES_00=-2.599, HTML_MESSAGE=0.001, IP_NOT_FRIENDLY=0.334] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id c-QEJgueZWNA for ; Mon, 4 Apr 2011 18:39:28 -0700 (PDT) Received: from linode.aoot.com (linode.aoot.com [69.164.194.13]) by core3.amsl.com (Postfix) with ESMTP id 6625E3A6821 for ; Mon, 4 Apr 2011 18:39:28 -0700 (PDT) Received: from [172.16.0.97] (pool-71-114-244-108.austtx.dsl-w.verizon.net [71.114.244.108]) by linode.aoot.com (Postfix) with ESMTP id 563D08223; Mon, 4 Apr 2011 20:41:10 -0500 (CDT) Message-ID: <4D9A733B.9050503@opengridcomputing.com> Date: Mon, 04 Apr 2011 20:41:15 -0500 From: Steve Wise User-Agent: Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US; rv:1.9.2.15) Gecko/20110303 Lightning/1.0b2 Thunderbird/3.1.9 MIME-Version: 1.0 To: arkady kanevsky References: <7C4DFCE962635144B8FAE8CA11D0BF1E03D74C7183@MX14A.corp.emc.com> <7C4DFCE962635144B8FAE8CA11D0BF1E03E5E9530C@MX14A.corp.emc.com> <76DBE161893C324BA6D4BB507EE4C3849510935E96@IRVEXCHCCR02.corp.ad.broadcom.com> <4D98E86A.60403@opengridcomputing.com> In-Reply-To: Content-Type: multipart/alternative; boundary="------------010702010001070200090400" Cc: storm@ietf.org Subject: Re: [storm] MPA Draft - Review X-BeenThere: storm@ietf.org X-Mailman-Version: 2.1.9 Precedence: list List-Id: Storage Maintenance WG List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 05 Apr 2011 01:39:30 -0000 This is a multi-part message in MIME format. --------------010702010001070200090400 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit The key word to remove here is "responder". The IRD in the MPA start request is the initiators desired IRD _of the initiator_. Not the initiators desired _responder_ IRD. If you want to change "desiired" to "initial" thats ok with me. But the key is to get rid of the word "responder" in that sentence. Steve. On 4/4/2011 6:26 PM, arkady kanevsky wrote: > Steve and Bob, > I changed it to > "In request: the Initiator desired responder IRD > for the connection." as you asked. > I can change it to "initial" instead of "desired". > Arkady > > On Sun, Apr 3, 2011 at 5:36 PM, Steve Wise > > wrote: > > Hey Arkady, > > It does seem like you did the section 9 changes Bob and I requested: > > ---- > > Change the IRD definition on the request from "In request: the > Initiator requested responder IRD for the > connection." to "In request: the Initiator initial IRD setting > for the connection." > ---- > > Thanks, > > Steve. > > > On 4/2/2011 8:35 PM, arkady kanevsky wrote: >> All, >> updated version 04 is attached. >> >> Hemal, >> Thanks for catching it. >> I had fixed the first issue. I had added reference to FPDU in the >> FULPDU definition for the second. >> >> David, >> Please, check to see that you comments are addressed. >> >> Steve and Robert, >> please, check that you comment is fixed correctly. >> >> Once I get positive feedback from all of you, I will submit the >> version. >> >> Thanks, >> Arkady >> >> On Thu, Mar 31, 2011 at 2:43 PM, Hemal Shah > > wrote: >> >> I have some comments on -03 draft: >> >> 1. In section 10, it is written that "Enhanced MPA >> Initiator and Responder: If a responder receives an >> enhanced MPA message, it MUST respond with an >> unenhanced MPA message." I think it should be written >> that the responder must respond with an enhanced MPA >> message. It appears like a typo to me. >> 2. I find the use of FULPDU confusing in this draft. >> RFC5044 does not define term FULPDU. RFC5044 uses term >> FPDU to refer to Framed Protocol Data Unit. I suggest >> that we use term FPDU instead of FULPDU in the draft. >> >> Hemal >> -----Original Message----- >> From: storm-bounces@ietf.org >> [mailto:storm-bounces@ietf.org] On Behalf Of >> david.black@emc.com >> Sent: Thursday, March 31, 2011 7:48 AM >> To: storm@ietf.org >> Subject: Re: [storm] MPA Draft - Review >> Importance: High >> The -03 version of the MPA draft has addressed all of the >> issues from my review, and . Unfortunately, I need some >> minor edits for clarity before I can send this on to our AD >> with a publication request. Would the authors please submit >> a -04 version with the following two changes quickly. >> Section 9 (end) >> OLD >> The peer-to-peer negotiation for the RTR message follows the >> following order: >> Initiator -->: Sets Control Flags it is capable to send >> for RTR >> Responder <--: Sets Control Flags it is capable to receive >> for RTR >> Initiator -->: The first message send MUST be a negotiated RTR >> NEW >> The peer-to-peer negotiation for the RTR message follows the >> following order: >> Initiator -->: Sets Control Flags to indicate >> Initiator-supported forms of RTR >> Responder <--: Sets Control Flags to indicate >> Responder-supported forms of RTR >> Initiator -->: If at least one form of RTR is supported by >> both Initiator and >> Responder, then the first message sent MUST be an RTR >> using a form supported >> by both the Initiator and Responder. >> Section 10 >> OLD >> In >> this case initiator CAN attempt to establish RDMA >> connection using >> unenhanced MPA protocol as defined in [RFC5044] and let >> ULP deal >> with ORD and IRD, and peer-to-peer negotiations. >> NEW >> In >> this case initiator MAY attempt to establish RDMA >> connection using >> ------------------------->^^^ >> unenhanced MPA protocol as defined in [RFC5044] if this >> protocol is >> compatible with the application, and let ULP deal >> with ORD and IRD, >> and peer-to-peer negotiations. >> Ordinarily, I'd write an RFC Editor Note for small changes >> like these, but they're sufficiently critical to >> interoperability that I'd prefer to have a new draft version >> that contains them. >> Thanks, >> --David >> > -----Original Message----- >> > From: Black, David >> > Sent: Wednesday, December 22, 2010 9:26 PM >> > To: storm@ietf.org >> > Cc: Black, David >> > Subject: MPA Draft - Review >> > >> > WG Last Call on this draft has run its course: >> > >> > Enhanced RDMA Connection Establishment >> > draft-ietf-storm-mpa-peer-connect-02 >> > >> > I've done my review as a WG chair (and the person who will >> be shepherding this draft to the ADs and >> > IESG): >> > >> > - This draft is on the right track, but has open issues. >> > - Another version of the draft will be needed. >> > >> > Also, it would be greatly appreciated if a few people other >> than the authors could take a look at >> > this draft. We have a very good author team on this draft, >> whose expertise is beyond doubt, but >> > more eyes on this draft would help. >> > >> > [1] My primary concern is that Section 9 on >> interoperability is inadequate: >> > >> > An initiator SHOULD NOT use the Enhanced DDP Connection >> Establishment >> > formats or function codes when no enhanced functionality >> is desired. >> > >> > A responder SHOULD continue to accept the unenhanced >> connection >> > requests. >> > >> > The good news is that the first sentence is ok. >> > The bad news is that the second sentence has significant >> problems: >> > - It uses SHOULD instead of MUST. >> > - It doesn't lay out behavior for initiator and >> responder >> > Revision mixes. >> > IETF interoperability requirements are usually expressed >> with MUST, including backwards >> > compatibility. If interop with unenhanced implementations >> is only a SHOULD, that will need a >> > convincing explanation. >> > >> > There are 3 Initiator/Responder cases that need attention >> (New/Old, Old/New and New/New). I think >> > they lead to roughly the following: >> > >> > New/Old: >> > - Explain error or failure that the New Initiator will see >> because the Old responder >> > doesn't support Revision 2 of the MPA protocol. >> > - Explain what the Initiator does when it sees that error >> or failure. The >> > easiest approach is to always retry with Revision 1, >> but that won't work >> > if the Initiator has to send an RTR (that's the >> "convincing explanation" >> > for why backwards compatibility is not always >> possible). The result >> > might be two requirements: >> > - If the Initiator has data to send, it MUST retry >> with Revision 1. >> > - If the Initiator has no data to send, and hence >> has to send an RTR, >> > the connection setup fails, the TCP >> connection closes and that >> > failure MUST to be reported to the application. >> > >> > Old/New: >> > - If a responder receives a Revision 1 message, it MUST >> respond with a Revision 1 message. >> > >> > New/New: >> > - If a responder receives a Revision 2 message, it MUST >> respond with a Revision 2 message. >> > >> > I found a few other concerns: >> > >> > [B]In Section 7, we need to get the listing of all the SCTP >> function codes into one place. Either >> > repeat the definitions of codes 1-4 from RFC 5043, or >> create an IANA registry in Section 10 and list >> > all 7 codes as its initial contents. >> > >> > [C] In Section 8, what happens if the responder sends an >> IRD or ORD value that's different from the >> > corresponding initiator value? Is the responder allowed to >> increase the value that was sent? An >> > important case to cover is that the initiator sends a valid >> value (e.g., 0x2000) but the responder >> > returns the 0x3FFF value indicating that negotiation is not >> supported. Also, what is the behavior >> > of an IRD or ORD that is set to 0x0000? >> > >> > [D] In contrast, the Section 8 discussion of Control Flag >> functionality is in better shape. It >> > would be helpful to add a sentence or two indicating when >> the RTR occurs (Request ->, <- Reply, RTR >> > ->), even though that is discussed earlier in the draft. >> Also, it's necessary to state whether >> > negotiation of RTR functionality commits the Initiator to >> using an RTR (e.g., suppose the initiator >> > negotiates control flags to allow an RTR and instead sends >> an FULPDU with payload data after >> > receiving the Reply - is that ok or is it an error?). >> > >> > [E] Nit: In the definition of Control Flag A: ULPDU -> FULPDU >> > >> > Thanks, >> > --David >> > ---------------------------------------------------- >> > David L. Black, Distinguished Engineer >> > EMC Corporation, 176 South St., Hopkinton, MA 01748 >> > +1 (508) 293-7953 >> FAX: +1 (508) >> 293-7786 >> > david.black@emc.com >> Mobile: +1 (978) 394-7754 >> > ---------------------------------------------------- >> _______________________________________________ >> storm mailing list >> storm@ietf.org >> https://www.ietf.org/mailman/listinfo/storm >> >> _______________________________________________ >> storm mailing list >> storm@ietf.org >> https://www.ietf.org/mailman/listinfo/storm >> >> >> >> >> -- >> Cheers, >> Arkady Kanevsky >> >> >> _______________________________________________ >> storm mailing list >> storm@ietf.org >> https://www.ietf.org/mailman/listinfo/storm > > > > > -- > Cheers, > Arkady Kanevsky --------------010702010001070200090400 Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit The key word to remove here is "responder".  The IRD in the MPA start request is the initiators desired IRD _of the initiator_.  Not the initiators desired _responder_ IRD.

If you want to change "desiired" to "initial" thats ok with me.  But the key is to get rid of the word "responder" in that sentence.

Steve.

On 4/4/2011 6:26 PM, arkady kanevsky wrote:
Steve and Bob,
I changed it to
"In request: the Initiator desired responder IRD
for the connection." as you asked.
I can change it to "initial" instead of "desired".
Arkady

On Sun, Apr 3, 2011 at 5:36 PM, Steve Wise <swise@opengridcomputing.com> wrote:
Hey Arkady,

It does seem like you did the section 9 changes Bob and I requested:

----

  Change the IRD definition on the request from "In request: the Initiator requested responder IRD for the
  connection." to "In request: the Initiator initial IRD setting for the connection." 
----

Thanks,

Steve.


On 4/2/2011 8:35 PM, arkady kanevsky wrote:
All,
updated version 04 is attached.

Hemal,
Thanks for catching it.
I had fixed the first issue. I had added reference to FPDU in the FULPDU definition for the second.

David,
Please, check to see that you comments are addressed.

Steve and Robert,
please, check that you comment is fixed correctly.

Once I get positive feedback from all of you, I will submit the version.

Thanks,
Arkady

On Thu, Mar 31, 2011 at 2:43 PM, Hemal Shah <hemal@broadcom.com> wrote:
I have some comments on -03 draft:
 
  1. In section 10, it is written that "Enhanced MPA Initiator and Responder:  If a responder receives an enhanced MPA message, it MUST respond with an unenhanced MPA message." I think it should be written that the responder must respond with an enhanced MPA message. It appears like a typo to me.
  2. I find the use of FULPDU confusing in this draft. RFC5044 does not define term FULPDU. RFC5044 uses term FPDU to refer to Framed Protocol Data Unit. I suggest that we use term FPDU instead of FULPDU in the draft.
 
Hemal
 
-----Original Message-----
From: storm-bounces@ietf.org [mailto:storm-bounces@ietf.org] On Behalf Of david.black@emc.com
Sent: Thursday, March 31, 2011 7:48 AM
To: storm@ietf.org
Subject: Re: [storm] MPA Draft - Review
Importance: High
 
The -03 version of the MPA draft has addressed all of the issues from my review, and .  Unfortunately, I need some minor edits for clarity before I can send this on to our AD with a publication request.  Would the authors please submit a -04 version with the following two changes quickly.
 
Section 9 (end)
 
OLD
 
   The peer-to-peer negotiation for the RTR message follows the
   following order:    
                               
   Initiator -->: Sets Control Flags it is capable to send for RTR     
                               
   Responder <--: Sets Control Flags it is capable to receive for RTR  
                               
   Initiator -->: The first message send MUST be a negotiated RTR
 
NEW
 
   The peer-to-peer negotiation for the RTR message follows the
   following order:    
                               
   Initiator -->: Sets Control Flags to indicate Initiator-supported forms of RTR      
                               
   Responder <--: Sets Control Flags to indicate Responder-supported forms of RTR      
                               
   Initiator -->: If at least one form of RTR is supported by both Initiator and
        Responder, then the first message sent MUST be an RTR using a form supported
        by both the Initiator and Responder.
 
Section 10
 
OLD
      In
      this case initiator CAN attempt to establish RDMA connection using
      unenhanced MPA protocol as defined in [RFC5044] and let ULP deal
      with ORD and IRD, and peer-to-peer negotiations.
 
NEW
 
      In
      this case initiator MAY attempt to establish RDMA connection using
------------------------->^^^
      unenhanced MPA protocol as defined in [RFC5044] if this protocol is
        compatible with the application, and let ULP deal with ORD and IRD,
      and peer-to-peer negotiations.
 
Ordinarily, I'd write an RFC Editor Note for small changes like these, but they're sufficiently critical to interoperability that I'd prefer to have a new draft version that contains them.
 
Thanks,
--David
 
 
> -----Original Message-----
> From: Black, David
> Sent: Wednesday, December 22, 2010 9:26 PM
> Cc: Black, David
> Subject: MPA Draft - Review
>
> WG Last Call on this draft has run its course:
>
>                  Enhanced RDMA Connection Establishment
>                   draft-ietf-storm-mpa-peer-connect-02
>
> I've done my review as a WG chair (and the person who will be shepherding this draft to the ADs and
> IESG):
>
> - This draft is on the right track, but has open issues.
> - Another version of the draft will be needed.
>
> Also, it would be greatly appreciated if a few people other than the authors could take a look at
> this draft.  We have a very good author team on this draft, whose expertise is beyond doubt, but
> more eyes on this draft would help.
>
> [1] My primary concern is that Section 9 on interoperability is inadequate:
>
>    An initiator SHOULD NOT use the Enhanced DDP Connection Establishment
>    formats or function codes when no enhanced functionality is desired.
>
>    A responder SHOULD continue to accept the unenhanced connection
>    requests.
>
> The good news is that the first sentence is ok.
> The bad news is that the second sentence has significant problems:
>        - It uses SHOULD instead of MUST.
>        - It doesn't lay out behavior for initiator and responder
>                Revision mixes.
> IETF interoperability requirements are usually expressed with MUST, including backwards
> compatibility.  If interop with unenhanced implementations is only a SHOULD, that will need a
> convincing explanation.
>
> There are 3 Initiator/Responder cases that need attention (New/Old, Old/New and New/New).  I think
> they lead to roughly the following:
>
> New/Old:
> - Explain error or failure that the New Initiator will see because the Old responder
>        doesn't support Revision 2 of the MPA protocol.
> - Explain what the Initiator does when it sees that error or failure.  The
>        easiest approach is to always retry with Revision 1, but that won't work
>        if the Initiator has to send an RTR (that's the "convincing explanation"
>        for why backwards compatibility is not always possible).  The result
>        might be two requirements:
>        - If the Initiator has data to send, it MUST retry with Revision 1.
>        - If the Initiator has no data to send, and hence has to send an RTR,
>                the connection setup fails, the TCP connection closes and that
>                failure MUST to be reported to the application.
>
> Old/New:
> - If a responder receives a Revision 1 message, it MUST respond with a Revision 1 message.
>
> New/New:
> - If a responder receives a Revision 2 message, it MUST respond with a Revision 2 message.
>
> I found a few other concerns:
>
> [B]In Section 7, we need to get the listing of all the SCTP function codes into one place.  Either
> repeat the definitions of codes 1-4 from RFC 5043, or create an IANA registry in Section 10 and list
> all 7 codes as its initial contents.
>
> [C] In Section 8, what happens if the responder sends an IRD or ORD value that's different from the
> corresponding initiator value?  Is the responder allowed to increase the value that was sent?  An
> important case to cover is that the initiator sends a valid value (e.g., 0x2000) but the responder
> returns the 0x3FFF value indicating that negotiation is not supported.  Also, what is the behavior
> of an IRD or ORD that is set to 0x0000?
>
> [D] In contrast, the Section 8 discussion of Control Flag functionality is in better shape.  It
> would be helpful to add a sentence or two indicating when the RTR occurs (Request ->, <- Reply, RTR
> ->), even though that is discussed earlier in the draft.  Also, it's necessary to state whether
> negotiation of RTR functionality commits the Initiator to using an RTR (e.g., suppose the initiator
> negotiates control flags to allow an RTR and instead sends an FULPDU with payload data after
> receiving the Reply - is that ok or is it an error?).
>
> [E] Nit: In the definition of Control Flag A: ULPDU -> FULPDU
>
> Thanks,
> --David
> ----------------------------------------------------
> David L. Black, Distinguished Engineer
> EMC Corporation, 176 South St., Hopkinton, MA  01748
> +1 (508) 293-7953             FAX: +1 (508) 293-7786
> david.black@emc.com        Mobile: +1 (978) 394-7754
> ----------------------------------------------------
 
_______________________________________________
storm mailing list
 
 

_______________________________________________
storm mailing list
storm@ietf.org
https://www.ietf.org/mailman/listinfo/storm




--
Cheers,
Arkady Kanevsky

_______________________________________________
storm mailing list
storm@ietf.org
https://www.ietf.org/mailman/listinfo/storm




--
Cheers,
Arkady Kanevsky

--------------010702010001070200090400-- From robert.o.sharp@intel.com Tue Apr 5 07:36:18 2011 Return-Path: X-Original-To: storm@core3.amsl.com Delivered-To: storm@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id 4FA283A693F for ; Tue, 5 Apr 2011 07:36:18 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -6.598 X-Spam-Level: X-Spam-Status: No, score=-6.598 tagged_above=-999 required=5 tests=[BAYES_00=-2.599, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_MED=-4] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id q0+0i9J3-pd6 for ; Tue, 5 Apr 2011 07:36:10 -0700 (PDT) Received: from mga14.intel.com (mga14.intel.com [143.182.124.37]) by core3.amsl.com (Postfix) with ESMTP id BC1203A693D for ; Tue, 5 Apr 2011 07:36:09 -0700 (PDT) Received: from azsmga001.ch.intel.com ([10.2.17.19]) by azsmga102.ch.intel.com with ESMTP; 05 Apr 2011 07:37:52 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="4.63,304,1299484800"; d="scan'208,217";a="414107244" Received: from azsmsx601.amr.corp.intel.com ([10.2.121.193]) by azsmga001.ch.intel.com with ESMTP; 05 Apr 2011 07:37:52 -0700 Received: from azsmsx602.amr.corp.intel.com (10.2.121.201) by azsmsx601.amr.corp.intel.com (10.2.121.193) with Microsoft SMTP Server (TLS) id 8.2.255.0; Tue, 5 Apr 2011 07:37:52 -0700 Received: from azsmsx506.amr.corp.intel.com ([10.2.121.72]) by azsmsx602.amr.corp.intel.com ([10.2.121.201]) with mapi; Tue, 5 Apr 2011 07:37:51 -0700 From: "Sharp, Robert O" To: Steve Wise , arkady kanevsky Date: Tue, 5 Apr 2011 07:37:51 -0700 Thread-Topic: [storm] MPA Draft - Review Thread-Index: AcvzN8DFKRXk0BDTSWiXjksuIS/4uQAZzxeA Message-ID: <2EFBCAEF10980645BBCFB605689E08E904D7F61C5B@azsmsx506.amr.corp.intel.com> References: <7C4DFCE962635144B8FAE8CA11D0BF1E03D74C7183@MX14A.corp.emc.com> <7C4DFCE962635144B8FAE8CA11D0BF1E03E5E9530C@MX14A.corp.emc.com> <76DBE161893C324BA6D4BB507EE4C3849510935E96@IRVEXCHCCR02.corp.ad.broadcom.com> <4D98E86A.60403@opengridcomputing.com> <4D9A733B.9050503@opengridcomputing.com> <4D9A7BF7.3040201@opengridcomputing.com> In-Reply-To: <4D9A7BF7.3040201@opengridcomputing.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: acceptlanguage: en-US Content-Type: multipart/alternative; boundary="_000_2EFBCAEF10980645BBCFB605689E08E904D7F61C5Bazsmsx506amrc_" MIME-Version: 1.0 Cc: "storm@ietf.org" Subject: Re: [storm] MPA Draft - Review X-BeenThere: storm@ietf.org X-Mailman-Version: 2.1.9 Precedence: list List-Id: Storage Maintenance WG List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 05 Apr 2011 14:36:18 -0000 --_000_2EFBCAEF10980645BBCFB605689E08E904D7F61C5Bazsmsx506amrc_ Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable Looks good to me as well. Thanks, Bob From: Steve Wise [mailto:swise@opengridcomputing.com] Sent: Monday, April 04, 2011 9:19 PM To: arkady kanevsky Cc: Sharp, Robert O; storm@ietf.org Subject: Re: [storm] MPA Draft - Review Looks good. Bob Sharp should ACK this too though. Thanks, Steve. On 4/4/2011 8:50 PM, arkady kanevsky wrote: Got it. Thanks Steve. How about now? Arkady On Mon, Apr 4, 2011 at 9:41 PM, Steve Wise > wrote: The key word to remove here is "responder". The IRD in the MPA start reque= st is the initiators desired IRD _of the initiator_. Not the initiators de= sired _responder_ IRD. If you want to change "desiired" to "initial" thats ok with me. But the ke= y is to get rid of the word "responder" in that sentence. Steve. On 4/4/2011 6:26 PM, arkady kanevsky wrote: Steve and Bob, I changed it to "In request: the Initiator desired responder IRD for the connection." as you asked. I can change it to "initial" instead of "desired". Arkady On Sun, Apr 3, 2011 at 5:36 PM, Steve Wise > wrote: Hey Arkady, It does seem like you did the section 9 changes Bob and I requested: ---- Change the IRD definition on the request from "In request: the Initiator = requested responder IRD for the connection." to "In request: the Initiator initial IRD setting for the co= nnection." ---- Thanks, Steve. On 4/2/2011 8:35 PM, arkady kanevsky wrote: All, updated version 04 is attached. Hemal, Thanks for catching it. I had fixed the first issue. I had added reference to FPDU in the FULPDU de= finition for the second. David, Please, check to see that you comments are addressed. Steve and Robert, please, check that you comment is fixed correctly. Once I get positive feedback from all of you, I will submit the version. Thanks, Arkady On Thu, Mar 31, 2011 at 2:43 PM, Hemal Shah > wrote: I have some comments on -03 draft: 1. In section 10, it is written that "Enhanced MPA Initiator and Responde= r: If a responder receives an enhanced MPA message, it MUST respond with a= n unenhanced MPA message." I think it should be written that the responder = must respond with an enhanced MPA message. It appears like a typo to me. 2. I find the use of FULPDU confusing in this draft. RFC5044 does not def= ine term FULPDU. RFC5044 uses term FPDU to refer to Framed Protocol Data Un= it. I suggest that we use term FPDU instead of FULPDU in the draft. Hemal -----Original Message----- From: storm-bounces@ietf.org [mailto:storm-b= ounces@ietf.org] On Behalf Of david.black@emc.com Sent: Thursday, March 31, 2011 7:48 AM To: storm@ietf.org Subject: Re: [storm] MPA Draft - Review Importance: High The -03 version of the MPA draft has addressed all of the issues from my re= view, and . Unfortunately, I need some minor edits for clarity before I ca= n send this on to our AD with a publication request. Would the authors ple= ase submit a -04 version with the following two changes quickly. Section 9 (end) OLD The peer-to-peer negotiation for the RTR message follows the following order: Initiator -->: Sets Control Flags it is capable to send for RTR Responder <--: Sets Control Flags it is capable to receive for RTR Initiator -->: The first message send MUST be a negotiated RTR NEW The peer-to-peer negotiation for the RTR message follows the following order: Initiator -->: Sets Control Flags to indicate Initiator-supported forms = of RTR Responder <--: Sets Control Flags to indicate Responder-supported forms = of RTR Initiator -->: If at least one form of RTR is supported by both Initiato= r and Responder, then the first message sent MUST be an RTR using a form = supported by both the Initiator and Responder. Section 10 OLD In this case initiator CAN attempt to establish RDMA connection using unenhanced MPA protocol as defined in [RFC5044] and let ULP deal with ORD and IRD, and peer-to-peer negotiations. NEW In this case initiator MAY attempt to establish RDMA connection using ------------------------->^^^ unenhanced MPA protocol as defined in [RFC5044] if this protocol is compatible with the application, and let ULP deal with ORD and IRD, and peer-to-peer negotiations. Ordinarily, I'd write an RFC Editor Note for small changes like these, but = they're sufficiently critical to interoperability that I'd prefer to have a= new draft version that contains them. Thanks, --David > -----Original Message----- > From: Black, David > Sent: Wednesday, December 22, 2010 9:26 PM > To: storm@ietf.org > Cc: Black, David > Subject: MPA Draft - Review > > WG Last Call on this draft has run its course: > > Enhanced RDMA Connection Establishment > draft-ietf-storm-mpa-peer-connect-02 > > I've done my review as a WG chair (and the person who will be shepherding= this draft to the ADs and > IESG): > > - This draft is on the right track, but has open issues. > - Another version of the draft will be needed. > > Also, it would be greatly appreciated if a few people other than the auth= ors could take a look at > this draft. We have a very good author team on this draft, whose experti= se is beyond doubt, but > more eyes on this draft would help. > > [1] My primary concern is that Section 9 on interoperability is inadequat= e: > > An initiator SHOULD NOT use the Enhanced DDP Connection Establishment > formats or function codes when no enhanced functionality is desired. > > A responder SHOULD continue to accept the unenhanced connection > requests. > > The good news is that the first sentence is ok. > The bad news is that the second sentence has significant problems: > - It uses SHOULD instead of MUST. > - It doesn't lay out behavior for initiator and responder > Revision mixes. > IETF interoperability requirements are usually expressed with MUST, inclu= ding backwards > compatibility. If interop with unenhanced implementations is only a SHOU= LD, that will need a > convincing explanation. > > There are 3 Initiator/Responder cases that need attention (New/Old, Old/N= ew and New/New). I think > they lead to roughly the following: > > New/Old: > - Explain error or failure that the New Initiator will see because the Ol= d responder > doesn't support Revision 2 of the MPA protocol. > - Explain what the Initiator does when it sees that error or failure. Th= e > easiest approach is to always retry with Revision 1, but that won'= t work > if the Initiator has to send an RTR (that's the "convincing explan= ation" > for why backwards compatibility is not always possible). The resu= lt > might be two requirements: > - If the Initiator has data to send, it MUST retry with Revision 1= . > - If the Initiator has no data to send, and hence has to send an R= TR, > the connection setup fails, the TCP connection closes and = that > failure MUST to be reported to the application. > > Old/New: > - If a responder receives a Revision 1 message, it MUST respond with a Re= vision 1 message. > > New/New: > - If a responder receives a Revision 2 message, it MUST respond with a Re= vision 2 message. > > I found a few other concerns: > > [B]In Section 7, we need to get the listing of all the SCTP function code= s into one place. Either > repeat the definitions of codes 1-4 from RFC 5043, or create an IANA regi= stry in Section 10 and list > all 7 codes as its initial contents. > > [C] In Section 8, what happens if the responder sends an IRD or ORD value= that's different from the > corresponding initiator value? Is the responder allowed to increase the = value that was sent? An > important case to cover is that the initiator sends a valid value (e.g., = 0x2000) but the responder > returns the 0x3FFF value indicating that negotiation is not supported. A= lso, what is the behavior > of an IRD or ORD that is set to 0x0000? > > [D] In contrast, the Section 8 discussion of Control Flag functionality i= s in better shape. It > would be helpful to add a sentence or two indicating when the RTR occurs = (Request ->, <- Reply, RTR > ->), even though that is discussed earlier in the draft. Also, it's nece= ssary to state whether > negotiation of RTR functionality commits the Initiator to using an RTR (e= .g., suppose the initiator > negotiates control flags to allow an RTR and instead sends an FULPDU with= payload data after > receiving the Reply - is that ok or is it an error?). > > [E] Nit: In the definition of Control Flag A: ULPDU -> FULPDU > > Thanks, > --David > ---------------------------------------------------- > David L. Black, Distinguished Engineer > EMC Corporation, 176 South St., Hopkinton, MA 01748 > +1 (508) 293-7953 FAX: +1 (5= 08) 293-7786 > david.black@emc.com Mobile: +1 (978) 3= 94-7754 > ---------------------------------------------------- _______________________________________________ storm mailing list storm@ietf.org https://www.ietf.org/mailman/listinfo/storm _______________________________________________ storm mailing list storm@ietf.org https://www.ietf.org/mailman/listinfo/storm -- Cheers, Arkady Kanevsky _______________________________________________ storm mailing list storm@ietf.org https://www.ietf.org/mailman/listinfo/storm -- Cheers, Arkady Kanevsky -- Cheers, Arkady Kanevsky --_000_2EFBCAEF10980645BBCFB605689E08E904D7F61C5Bazsmsx506amrc_ Content-Type: text/html; charset="us-ascii" Content-Transfer-Encoding: quoted-printable

Looks good to me as well.

 

Thanks,

Bob

 

From: Steve Wise [mailto:swise@opengridcomputing.com]
Sent: Monday, April 04, 2011 9:19 PM
To: arkady kanevsky
Cc: Sharp, Robert O; storm@ietf.org
Subject: Re: [storm] MPA Draft - Review

 

Looks good.

Bob Sharp should ACK this too though.

Thanks,

Steve.


On 4/4/2011 8:50 PM, arkady kanevsky wrote:

Got it. Thanks Steve. How about now?
Arkady

On Mon, Apr 4, 2011 at 9:41 PM, Steve Wise <swise@opengridcomputing.com= > wrote:

The key word to remove here is "responder".&= nbsp; The IRD in the MPA start request is the initiators desired IRD _of the initiator_.  Not the initiators desired _responder_ IRD.

If you want to change "desiired" to "initial" thats ok = with me.  But the key is to get rid of the word "responder" in th= at sentence.

Steve.



On 4/4/2011 6:26 PM, arkady kanevsky wrote:

Steve and Bob,
I changed it to
"In request: the Initiator desired responder IRD
for the connection." as you asked.
I can change it to "initial" instead of "desired".
Arkady

On Sun, Apr 3, 2011 at 5:36 PM, Steve Wise <swise@opengri= dcomputing.com> wrote:

Hey Arkady,

It does seem like you did the section 9 changes Bob and I requested:

----


  Change the IRD definition on the request from "In request: the Initiator requested responder IRD for the
  connection." to "In request: the Initiator initial IRD set= ting for the connection." 

----

Thanks,

Steve.



On 4/2/2011 8:35 PM, arkady kanevsky wrote:

All,
updated version 04 is attached.

Hemal,
Thanks for catching it.
I had fixed the first issue. I had added reference to FPDU in the FULPDU definition for the second.

David,
Please, check to see that you comments are addressed.

Steve and Robert,
please, check that you comment is fixed correctly.

Once I get positive feedback from all of you, I will submit the version.
Thanks,
Arkady

On Thu, Mar 31, 2011 at 2:43 PM, Hemal Shah <hemal@broadcom.com= > wrote:

= I have some comments on -03 draft:

=  

  1. In section 10, it is written that "Enhanced MPA Initiator and Responder:  If a responder receives an enhanced MPA message, it M= UST respond with an unenhanced MPA message." I think it should be wri= tten that the responder must respond with an enhanced MPA message. It appea= rs like a typo to me.
  2. I find the use of FULPDU confusing in this draft. RFC5044 does not defin= e term FULPDU. RFC5044 uses term FPDU to refer to Framed Protocol Data U= nit. I suggest that we use term FPDU instead of FULPDU in the draft.

=  

= Hemal

=  

= -----Original Message-----
From: storm-bou= nces@ietf.org [mailto:storm-b= ounces@ietf.org] On Behalf Of david= .black@emc.com
Sent: Thursday, March 31, 2011 7:48 AM
To: storm@ietf.org<= br> Subject: Re: [storm] MPA Draft - Review

= Importance: High

=  

= The -03 version of the MPA draft has addressed all of the issues from my review, an= d .  Unfortunately, I need some minor edits for clarity before I can sen= d this on to our AD with a publication request.  Would the authors pleas= e submit a -04 version with the following two changes quickly.

=  

= Section 9 (end)

=  

= OLD

=  

=    The peer-to-peer negotiation for the RTR message follows the

=    following order:    

=             &nb= sp;            =       

=    Initiator -->: Sets Control Flags it is capable to send for RTR     

=             &nb= sp;            =       

=    Responder <--: Sets Control Flags it is capable to receive for RTR  

=             &nb= sp;            =       

=    Initiator -->: The first message send MUST be a negotiated RTR

=  

= NEW

=  

=    The peer-to-peer negotiation for the RTR message follows the

=    following order:    

=             &nb= sp;            =       

=    Initiator -->: Sets Control Flags to indicate Initiator-supported forms = of RTR      

=             &nb= sp;            =       

=    Responder <--: Sets Control Flags to indicate Responder-supported forms = of RTR      

=             &nb= sp;            =       

=    Initiator -->: If at least one form of RTR is supported by both Initiato= r and

=         Responder, then the first message sent MUST be an RTR using a form supporte= d

=         by both the Initiator and Responder.

=  

= Section 10

=  

= OLD

=       In

=       this case initiator CAN attempt to establish RDMA connection using

=       unenhanced MPA protocol as defined in [RFC5044] and let ULP deal=

=       with ORD and IRD, and peer-to-peer negotiations.

=  

= NEW

=  

=       In

=       this case initiator MAY attempt to establish RDMA connection using

= ------------------------->^^^

=       unenhanced MPA protocol as defined in [RFC5044] if this protocol is

=         compatible with the application, and let ULP deal with ORD and IRD,

=       and peer-to-peer negotiations.

=  

= Ordinarily, I'd write an RFC Editor Note for small changes like these, but they're sufficiently critical to interoperability that I'd prefer to have a new dra= ft version that contains them.

=  

= Thanks,

= --David

=  

=  

= > -----Original Message-----

= > From: Black, David

= > Sent: Wednesday, December 22, 2010 9:26 PM

= > To: storm@ietf.org=

= > Cc: Black, David

= > Subject: MPA Draft - Review

= >

= > WG Last Call on this draft has run its course:

= >

= >            = ;      Enhanced RDMA Connection Establishment

= >            = ;       draft-ietf-storm-mpa-peer-connect-02

= >

= > I've done my review as a WG chair (and the person who will be shepherding t= his draft to the ADs and

= > IESG):

= >

= > - This draft is on the right track, but has open issues.

= > - Another version of the draft will be needed.

= >

= > Also, it would be greatly appreciated if a few people other than the author= s could take a look at

= > this draft.  We have a very good author team on this draft, whose expertise is beyond doubt, but

= > more eyes on this draft would help.

= >

= > [1] My primary concern is that Section 9 on interoperability is inadequate:

= >

= >    An initiator SHOULD NOT use the Enhanced DDP Connection Establishment<= /o:p>

= >    formats or function codes when no enhanced functionality is desired.

= >

= >    A responder SHOULD continue to accept the unenhanced connection<= /span>

= >    requests.

= >

= > The good news is that the first sentence is ok.

= > The bad news is that the second sentence has significant problems:

= >        - It uses SHOULD instead of MUST.

= >        - It doesn't lay out behavior for initiator and responder=

= >            = ;    Revision mixes.

= > IETF interoperability requirements are usually expressed with MUST, includi= ng backwards

= > compatibility.  If interop with unenhanced implementations is only a SHOULD, that will need a

= > convincing explanation.

= >

= > There are 3 Initiator/Responder cases that need attention (New/Old, Old/New= and New/New).  I think

= > they lead to roughly the following:

= >

= > New/Old:

= > - Explain error or failure that the New Initiator will see because the Old responder

= >        doesn't support Revision 2 of the MPA protocol.

= > - Explain what the Initiator does when it sees that error or failure.  T= he

= >        easiest approach is to always retry with Revision 1, but that won't work

= >        if the Initiator has to send an RTR (that's the "convincing explanation"

= >        for why backwards compatibility is not always possible).  The result

= >        might be two requirements:

= >        - If the Initiator has data to send, it MUST retry with Revision 1.

= >        - If the Initiator has no data to send, and hence has to send an RTR,<= /o:p>

= >            = ;    the connection setup fails, the TCP connection closes and that

= >            = ;    failure MUST to be reported to the application.

= >

= > Old/New:

= > - If a responder receives a Revision 1 message, it MUST respond with a Revisi= on 1 message.

= >

= > New/New:

= > - If a responder receives a Revision 2 message, it MUST respond with a Revisi= on 2 message.

= >

= > I found a few other concerns:

= >

= > [B]In Section 7, we need to get the listing of all the SCTP function codes = into one place.  Either

= > repeat the definitions of codes 1-4 from RFC 5043, or create an IANA regist= ry in Section 10 and list

= > all 7 codes as its initial contents.

= >

= > [C] In Section 8, what happens if the responder sends an IRD or ORD value that'= s different from the

= > corresponding initiator value?  Is the responder allowed to increase t= he value that was sent?  An

= > important case to cover is that the initiator sends a valid value (e.g., 0x2000) but the responder

= > returns the 0x3FFF value indicating that negotiation is not supported. = ; Also, what is the behavior

= > of an IRD or ORD that is set to 0x0000?

= >

= > [D] In contrast, the Section 8 discussion of Control Flag functionality is in better shape.  It

= > would be helpful to add a sentence or two indicating when the RTR occurs (Request ->, <- Reply, RTR

= > ->), even though that is discussed earlier in the draft.  Also, it'= s necessary to state whether

= > negotiation of RTR functionality commits the Initiator to using an RTR (e.g= ., suppose the initiator

= > negotiates control flags to allow an RTR and instead sends an FULPDU with payload data after

= > receiving the Reply - is that ok or is it an error?).

= >

= > [E] Nit: In the definition of Control Flag A: ULPDU -> FULPDU

= >

= > Thanks,

= > --David

= > ----------------------------------------------------

= > David L. Black, Distinguished Engineer

= > EMC Corporation, 176 South St., Hopkinton, MA  01748

= > +1 (508) 293-795= 3           &nbs= p; FAX: +1 (508) 293-7786

= > ----------------------------------------------------

=  

= _______________________________________________

= storm mailing list

=  

=  


_______________________________________________
storm mailing list
storm@ietf.org
h= ttps://www.ietf.org/mailman/listinfo/storm




--
Cheers,
Arkady Kanevsky

 
__________________________________________=
_____
storm mailing list
storm@ietf.org
http=
s://www.ietf.org/mailman/listinfo/storm

 




--
Cheers,
Arkady Kanevsky

 




--
Cheers,
Arkady Kanevsky

 

--_000_2EFBCAEF10980645BBCFB605689E08E904D7F61C5Bazsmsx506amrc_-- From arkady.kanevsky@gmail.com Mon Apr 4 16:32:42 2011 Return-Path: X-Original-To: storm@core3.amsl.com Delivered-To: storm@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id 384743A6821 for ; Mon, 4 Apr 2011 16:32:42 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -3.598 X-Spam-Level: X-Spam-Status: No, score=-3.598 tagged_above=-999 required=5 tests=[BAYES_00=-2.599, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_LOW=-1] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id BRj-AlrBsq5r for ; Mon, 4 Apr 2011 16:32:36 -0700 (PDT) Received: from mail-pw0-f44.google.com (mail-pw0-f44.google.com [209.85.160.44]) by core3.amsl.com (Postfix) with ESMTP id 865B13A6822 for ; Mon, 4 Apr 2011 16:32:36 -0700 (PDT) Received: by pwi5 with SMTP id 5so2066746pwi.31 for ; Mon, 04 Apr 2011 16:34:19 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:in-reply-to:references:date :message-id:subject:from:to:cc:content-type; bh=EjApPcS0vSFq/I2P3J62xfuciQ9K9oSBjymajfXEntM=; b=oEZ7NAuRvogJmm28kIjuZMCV0RL8Z+uh8PLQnXiJk7t9xHpCgY7J6T+VR7ZuqyMirZ 8oG8rgJVu3U22SD31DTSgj3WdzJonVIE0dsfncXzLATN3R2D5azoLpYgsMoorfiaWcIe ZrImtvB1JyUE1xUsSc4hM5s20xaI7u0Gjr7tQ= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; b=DOxue3Dtniw0irYqOmvxaLET8lu9NVjxeJxulYnDWpEIUVjeUW7xxCYQOEfAMZBZgR +UGotV5ms9fg6DAStMNDH8wZpgSolfJ9rEDHKJm8Z0QjxUttizVAxNhwNV4uBa6p+NTb ZGihDixEU+PcmSg1VPxbYUlLzvFuhdxJT17tM= MIME-Version: 1.0 Received: by 10.142.140.10 with SMTP id n10mr6888521wfd.72.1301960059284; Mon, 04 Apr 2011 16:34:19 -0700 (PDT) Received: by 10.142.185.20 with HTTP; Mon, 4 Apr 2011 16:34:19 -0700 (PDT) In-Reply-To: References: <7C4DFCE962635144B8FAE8CA11D0BF1E03D74C7183@MX14A.corp.emc.com> <7C4DFCE962635144B8FAE8CA11D0BF1E03E5E9530C@MX14A.corp.emc.com> <76DBE161893C324BA6D4BB507EE4C3849510935E96@IRVEXCHCCR02.corp.ad.broadcom.com> <4D98E86A.60403@opengridcomputing.com> Date: Mon, 4 Apr 2011 19:34:19 -0400 Message-ID: From: arkady kanevsky To: Steve Wise , "Sharp, Robert O" Content-Type: multipart/mixed; boundary=000e0cd2dc3ed86f6204a02032b8 X-Mailman-Approved-At: Tue, 05 Apr 2011 08:26:32 -0700 Cc: storm@ietf.org Subject: Re: [storm] MPA Draft - Review X-BeenThere: storm@ietf.org X-Mailman-Version: 2.1.9 Precedence: list List-Id: Storage Maintenance WG List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 04 Apr 2011 23:32:42 -0000 --000e0cd2dc3ed86f6204a02032b8 Content-Type: multipart/alternative; boundary=000e0cd2dc3ed86f3104a02032b6 --000e0cd2dc3ed86f3104a02032b6 Content-Type: text/plain; charset=ISO-8859-1 Steve and Bob, Here is updated draft. Thanks, Arkady On Mon, Apr 4, 2011 at 7:26 PM, arkady kanevsky wrote: > Steve and Bob, > I changed it to > "In request: the Initiator desired responder IRD > for the connection." as you asked. > I can change it to "initial" instead of "desired". > Arkady > > > On Sun, Apr 3, 2011 at 5:36 PM, Steve Wise wrote: > >> Hey Arkady, >> >> It does seem like you did the section 9 changes Bob and I requested: >> >> ---- >> >> Change the IRD definition on the request from "In request: the Initiator >> requested responder IRD for the >> connection." to "In request: the Initiator initial IRD setting for the >> connection." >> ---- >> >> Thanks, >> >> Steve. >> >> >> On 4/2/2011 8:35 PM, arkady kanevsky wrote: >> >> All, >> updated version 04 is attached. >> >> Hemal, >> Thanks for catching it. >> I had fixed the first issue. I had added reference to FPDU in the FULPDU >> definition for the second. >> >> David, >> Please, check to see that you comments are addressed. >> >> Steve and Robert, >> please, check that you comment is fixed correctly. >> >> Once I get positive feedback from all of you, I will submit the version. >> >> Thanks, >> Arkady >> >> On Thu, Mar 31, 2011 at 2:43 PM, Hemal Shah wrote: >> >>> I have some comments on -03 draft: >>> >>> >>> 1. In section 10, it is written that "Enhanced MPA Initiator and >>> Responder: If a responder receives an enhanced MPA message, it MUST respond >>> with an unenhanced MPA message." I think it should be written that the >>> responder must respond with an enhanced MPA message. It appears like a typo >>> to me. >>> 2. I find the use of FULPDU confusing in this draft. RFC5044 does not >>> define term FULPDU. RFC5044 uses term FPDU to refer to Framed Protocol Data >>> Unit. I suggest that we use term FPDU instead of FULPDU in the draft. >>> >>> >>> Hemal >>> >>> -----Original Message----- >>> From: storm-bounces@ietf.org [mailto:storm-bounces@ietf.org] >>> On Behalf Of david.black@emc.com >>> Sent: Thursday, March 31, 2011 7:48 AM >>> To: storm@ietf.org >>> Subject: Re: [storm] MPA Draft - Review >>> Importance: High >>> >>> The -03 version of the MPA draft has addressed all of the issues from my >>> review, and . Unfortunately, I need some minor edits for clarity before I >>> can send this on to our AD with a publication request. Would the authors >>> please submit a -04 version with the following two changes quickly. >>> >>> Section 9 (end) >>> >>> OLD >>> >>> The peer-to-peer negotiation for the RTR message follows the >>> following order: >>> >>> Initiator -->: Sets Control Flags it is capable to send for RTR >>> >>> Responder <--: Sets Control Flags it is capable to receive for RTR >>> >>> Initiator -->: The first message send MUST be a negotiated RTR >>> >>> NEW >>> >>> The peer-to-peer negotiation for the RTR message follows the >>> following order: >>> >>> Initiator -->: Sets Control Flags to indicate Initiator-supported >>> forms of RTR >>> >>> Responder <--: Sets Control Flags to indicate Responder-supported >>> forms of RTR >>> >>> Initiator -->: If at least one form of RTR is supported by both >>> Initiator and >>> Responder, then the first message sent MUST be an RTR using a >>> form supported >>> by both the Initiator and Responder. >>> >>> Section 10 >>> >>> OLD >>> In >>> this case initiator CAN attempt to establish RDMA connection using >>> unenhanced MPA protocol as defined in [RFC5044] and let ULP deal >>> with ORD and IRD, and peer-to-peer negotiations. >>> >>> NEW >>> >>> In >>> this case initiator MAY attempt to establish RDMA connection using >>> ------------------------->^^^ >>> unenhanced MPA protocol as defined in [RFC5044] if this protocol is >>> compatible with the application, and let ULP deal with ORD and >>> IRD, >>> and peer-to-peer negotiations. >>> >>> Ordinarily, I'd write an RFC Editor Note for small changes like these, >>> but they're sufficiently critical to interoperability that I'd prefer to >>> have a new draft version that contains them. >>> >>> Thanks, >>> --David >>> >>> >>> > -----Original Message----- >>> > From: Black, David >>> > Sent: Wednesday, December 22, 2010 9:26 PM >>> > To: storm@ietf.org >>> > Cc: Black, David >>> > Subject: MPA Draft - Review >>> > >>> > WG Last Call on this draft has run its course: >>> > >>> > Enhanced RDMA Connection Establishment >>> > draft-ietf-storm-mpa-peer-connect-02 >>> > >>> > I've done my review as a WG chair (and the person who will be >>> shepherding this draft to the ADs and >>> > IESG): >>> > >>> > - This draft is on the right track, but has open issues. >>> > - Another version of the draft will be needed. >>> > >>> > Also, it would be greatly appreciated if a few people other than the >>> authors could take a look at >>> > this draft. We have a very good author team on this draft, whose >>> expertise is beyond doubt, but >>> > more eyes on this draft would help. >>> > >>> > [1] My primary concern is that Section 9 on interoperability is >>> inadequate: >>> > >>> > An initiator SHOULD NOT use the Enhanced DDP Connection >>> Establishment >>> > formats or function codes when no enhanced functionality is desired. >>> > >>> > A responder SHOULD continue to accept the unenhanced connection >>> > requests. >>> > >>> > The good news is that the first sentence is ok. >>> > The bad news is that the second sentence has significant problems: >>> > - It uses SHOULD instead of MUST. >>> > - It doesn't lay out behavior for initiator and responder >>> > Revision mixes. >>> > IETF interoperability requirements are usually expressed with MUST, >>> including backwards >>> > compatibility. If interop with unenhanced implementations is only a >>> SHOULD, that will need a >>> > convincing explanation. >>> > >>> > There are 3 Initiator/Responder cases that need attention (New/Old, >>> Old/New and New/New). I think >>> > they lead to roughly the following: >>> > >>> > New/Old: >>> > - Explain error or failure that the New Initiator will see because the >>> Old responder >>> > doesn't support Revision 2 of the MPA protocol. >>> > - Explain what the Initiator does when it sees that error or failure. >>> The >>> > easiest approach is to always retry with Revision 1, but that >>> won't work >>> > if the Initiator has to send an RTR (that's the "convincing >>> explanation" >>> > for why backwards compatibility is not always possible). The >>> result >>> > might be two requirements: >>> > - If the Initiator has data to send, it MUST retry with Revision >>> 1. >>> > - If the Initiator has no data to send, and hence has to send an >>> RTR, >>> > the connection setup fails, the TCP connection closes >>> and that >>> > failure MUST to be reported to the application. >>> > >>> > Old/New: >>> > - If a responder receives a Revision 1 message, it MUST respond with a >>> Revision 1 message. >>> > >>> > New/New: >>> > - If a responder receives a Revision 2 message, it MUST respond with a >>> Revision 2 message. >>> > >>> > I found a few other concerns: >>> > >>> > [B]In Section 7, we need to get the listing of all the SCTP function >>> codes into one place. Either >>> > repeat the definitions of codes 1-4 from RFC 5043, or create an IANA >>> registry in Section 10 and list >>> > all 7 codes as its initial contents. >>> > >>> > [C] In Section 8, what happens if the responder sends an IRD or ORD >>> value that's different from the >>> > corresponding initiator value? Is the responder allowed to increase >>> the value that was sent? An >>> > important case to cover is that the initiator sends a valid value >>> (e.g., 0x2000) but the responder >>> > returns the 0x3FFF value indicating that negotiation is not supported. >>> Also, what is the behavior >>> > of an IRD or ORD that is set to 0x0000? >>> > >>> > [D] In contrast, the Section 8 discussion of Control Flag functionality >>> is in better shape. It >>> > would be helpful to add a sentence or two indicating when the RTR >>> occurs (Request ->, <- Reply, RTR >>> > ->), even though that is discussed earlier in the draft. Also, it's >>> necessary to state whether >>> > negotiation of RTR functionality commits the Initiator to using an RTR >>> (e.g., suppose the initiator >>> > negotiates control flags to allow an RTR and instead sends an FULPDU >>> with payload data after >>> > receiving the Reply - is that ok or is it an error?). >>> > >>> > [E] Nit: In the definition of Control Flag A: ULPDU -> FULPDU >>> > >>> > Thanks, >>> > --David >>> > ---------------------------------------------------- >>> > David L. Black, Distinguished Engineer >>> > EMC Corporation, 176 South St., Hopkinton, MA 01748 >>> > +1 (508) 293-7953 FAX: +1 (508) 293-7786 >>> > david.black@emc.com Mobile: +1 (978) 394-7754 >>> > ---------------------------------------------------- >>> >>> _______________________________________________ >>> storm mailing list >>> storm@ietf.org >>> https://www.ietf.org/mailman/listinfo/storm >>> >>> >>> >>> _______________________________________________ >>> storm mailing list >>> storm@ietf.org >>> https://www.ietf.org/mailman/listinfo/storm >>> >>> >> >> >> -- >> Cheers, >> Arkady Kanevsky >> >> >> _______________________________________________ >> storm mailing liststorm@ietf.orghttps://www.ietf.org/mailman/listinfo/storm >> >> >> > > > -- > Cheers, > Arkady Kanevsky > -- Cheers, Arkady Kanevsky --000e0cd2dc3ed86f3104a02032b6 Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Steve and Bob,
Here is updated draft.
Thanks,
Arkady

On Mon, Apr 4, 2011 at 7:26 PM, arkady kanevsky <arkady.kanevsk= y@gmail.com> wrote:
Steve and Bob,I changed it to
"In request: the Initiator desired responder IRD for the connection." as you asked.
I can change it to "initial= " instead of "desired".
Arkady


On Sun, Apr 3, 2011 at 5:36 PM, Steve Wise <swise@opengridcomputing.com> wrote:
=20 =20 =20 =20
Hey Arkady,

It does seem like you did the section 9 changes Bob and I requested:
----

=A0 Change the IRD definition on the request from "In request: the Initiator requested responder IRD for the
=A0 connection." to "In request: the Initiator initial IRD se= tting for the connection."=A0
----

Thanks,

Steve.


On 4/2/2011 8:35 PM, arkady kanevsky wrote:
All,
updated version 04 is attached.

Hemal,
Thanks for catching it.
I had fixed the first issue. I had added reference to FPDU in the FULPDU definition for the second.

David,
Please, check to see that you comments are addressed.

Steve and Robert,
please, check that you comment is fixed correctly.

Once I get positive feedback from all of you, I will submit the version.

Thanks,
Arkady

On Thu, Mar 31, 2011 at 2:43 PM, Hemal Shah <hemal@broadcom.com> wrote:
I have some comments on -03 draft:
=A0
  1. In section 10, it is written that "Enhanced MPA Initiator and Responder:=A0 If a responder receives an enhanced MPA message, it MUST respond with an unenhanced MPA message." I think it should be writte= n that the responder must respond with an enhanced MPA message. It appears like a typo to me.
  2. I find the use of FULPDU confusing in this draft. RFC5044 does not define term FULPDU. RFC5044 uses term FPDU to refer to Framed Protocol Data Unit. I suggest that we use term FPDU instead of FULPDU in the draft.
=A0
Hemal
=A0
-----Original Message-----
From: storm-bounces@ietf.org [mailto:storm-bounces@ietf.org] On Behalf Of david.black@emc.com
Sent: Thursday, March 31, 2011 7:48 AM
To: s= torm@ietf.org
Subject: Re: [storm] MPA Draft - Review
Importance: High
=A0
The -03 version of the MPA draft has addressed all of the issues from my review, and .=A0 Unfortunately, I need some minor edits for clarity before I can send this on to our AD with a publication request.=A0 Would the authors please submit a -04 version with the following two changes quickly.
=A0
Section 9 (end)
=A0
OLD
=A0
=A0=A0 The peer-to-peer negotiation for the RTR message follows the
=A0=A0 following order:=A0=A0=A0=A0
=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0= =A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0
=A0=A0 Initiator -->: Sets Control Flags it is capable to send for RTR=A0=A0=A0=A0=A0
=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0= =A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0
=A0=A0 Responder <--: Sets Control Flags it is capable to receive for RTR=A0=A0
=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0= =A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0
=A0=A0 Initiator -->: The first message send MUST be a negotiated RTR
=A0
NEW
=A0
=A0=A0 The peer-to-peer negotiation for the RTR message follows the
=A0=A0 following order:=A0=A0=A0=A0
=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0= =A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0
=A0=A0 Initiator -->: Sets Control Flags to indicate Initiator-supported forms of RTR=A0=A0=A0=A0= =A0=A0
=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0= =A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0
=A0=A0 Responder <--: Sets Control Flags to indicate Responder-supported forms of RTR=A0=A0=A0=A0= =A0=A0
=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0= =A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0
=A0=A0 Initiator -->: If at least one form of RTR is supported by both Initiator and
=A0=A0=A0=A0=A0=A0=A0 Responder, then the first mess= age sent MUST be an RTR using a form supported
=A0=A0=A0=A0=A0=A0=A0 by both the Initiator and Resp= onder.
=A0
Section 10
=A0
OLD
=A0=A0=A0=A0=A0 In
=A0=A0=A0=A0=A0 this case initiator CAN attempt to establish RDMA connection using
=A0=A0=A0=A0=A0 unenhanced MPA protocol as defined i= n [RFC5044] and let ULP deal
=A0=A0=A0=A0=A0 with ORD and IRD, and peer-to-peer negotiations.
=A0
NEW
=A0
=A0=A0=A0=A0=A0 In
=A0=A0=A0=A0=A0 this case initiator MAY attempt to establish RDMA connection using
------------------------->^^^
=A0=A0=A0=A0=A0 unenhanced MPA protocol as defined i= n [RFC5044] if this protocol is
=A0=A0=A0=A0=A0=A0=A0 compatible with the applicatio= n, and let ULP deal with ORD and IRD,
=A0=A0=A0=A0=A0 and peer-to-peer negotiations.
=A0
Ordinarily, I'd write an RFC Editor Note for small changes like these, but they're sufficiently critical to interoperability that I'd prefer to hav= e a new draft version that contains them.
=A0
Thanks,
--David
=A0
=A0
> -----Original Message-----
> From: Black, David
> Sent: Wednesday, December 22, 2010 9:26 PM
> Cc: Black, David
> Subject: MPA Draft - Review
>
> WG Last Call on this draft has run its course:
>
>=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0= =A0 Enhanced RDMA Connection Establishment
>=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0= =A0=A0 draft-ietf-storm-mpa-peer-connect-02
>
> I've done my review as a WG chair (and the person who will be shepherding this draft to the ADs and
> IESG):
>
> - This draft is on the right track, but has open issues.
> - Another version of the draft will be needed.
>
> Also, it would be greatly appreciated if a few people other than the authors could take a look at
> this draft.=A0 We have a very good author team on this draft, whose expertise is beyond doubt, but
> more eyes on this draft would help.
>
> [1] My primary concern is that Section 9 on interoperability is inadequate:
>
>=A0=A0=A0 An initiator SHOULD NOT use the Enhanc= ed DDP Connection Establishment
>=A0=A0=A0 formats or function codes when no enhanced functionality is desired.
>
>=A0=A0=A0 A responder SHOULD continue to accept = the unenhanced connection
>=A0=A0=A0 requests.
>
> The good news is that the first sentence is ok.
> The bad news is that the second sentence has significant problems:
>=A0=A0=A0=A0=A0=A0=A0 - It uses SHOULD instead o= f MUST.
>=A0=A0=A0=A0=A0=A0=A0 - It doesn't lay out b= ehavior for initiator and responder
>=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 Re= vision mixes.
> IETF interoperability requirements are usually expressed with MUST, including backwards
> compatibility.=A0 If interop with unenhanced implementations is only a SHOULD, that will need a
> convincing explanation.
>
> There are 3 Initiator/Responder cases that need attention (New/Old, Old/New and New/New).=A0 I think
> they lead to roughly the following:
>
> New/Old:
> - Explain error or failure that the New Initiator will see because the Old responder
>=A0=A0=A0=A0=A0=A0=A0 doesn't support Revisi= on 2 of the MPA protocol.
> - Explain what the Initiator does when it sees that error or failure.=A0 The
>=A0=A0=A0=A0=A0=A0=A0 easiest approach is to alw= ays retry with Revision 1, but that won't work
>=A0=A0=A0=A0=A0=A0=A0 if the Initiator has to se= nd an RTR (that's the "convincing explanation"
>=A0=A0=A0=A0=A0=A0=A0 for why backwards compatib= ility is not always possible).=A0 The result
>=A0=A0=A0=A0=A0=A0=A0 might be two requirements:=
>=A0=A0=A0=A0=A0=A0=A0 - If the Initiator has dat= a to send, it MUST retry with Revision 1.
>=A0=A0=A0=A0=A0=A0=A0 - If the Initiator has no = data to send, and hence has to send an RTR,
>=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 th= e connection setup fails, the TCP connection closes and that
>=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 fa= ilure MUST to be reported to the application.
>
> Old/New:
> - If a responder receives a Revision 1 message, it MUST respond with a Revision 1 message.
>
> New/New:
> - If a responder receives a Revision 2 message, it MUST respond with a Revision 2 message.
>
> I found a few other concerns:
>
> [B]In Section 7, we need to get the listing of all the SCTP function codes into one place.=A0 Either
> repeat the definitions of codes 1-4 from RFC 5043, or create an IANA registry in Section 10 and list
> all 7 codes as its initial contents.
>
> [C] In Section 8, what happens if the responder sends an IRD or ORD value that's differen= t from the
> corresponding initiator value?=A0 Is the responder allowed to increase the value that was sent?=A0 An
> important case to cover is that the initiator sends a valid value (e.g., 0x2000) but the responder
> returns the 0x3FFF value indicating that negotiation is not supported.=A0 Also, what is the behavior
> of an IRD or ORD that is set to 0x0000?
>
> [D] In contrast, the Section 8 discussion of Control Flag functionality is in better shape.=A0 It
> would be helpful to add a sentence or two indicating when the RTR occurs (Request ->, <- Reply, RTR
> ->), even though that is discussed earlier in the draft.=A0 Also, it's necessary to st= ate whether
> negotiation of RTR functionality commits the Initiator to using an RTR (e.g., suppose the initiator
> negotiates control flags to allow an RTR and instead sends an FULPDU with payload data after
> receiving the Reply - is that ok or is it an error?).
>
> [E] Nit: In the definition of Control Flag A: ULPDU -> FULPDU
>
> Thanks,
> --David
> ----------------------------------------------------
> David L. Black, Distinguished Engineer
> EMC Corporation, 176 South St., Hopkinton, MA=A0 01748
> +1 (508) 293-7953=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 FAX: +1 (508) 293-7786
> david.black@emc.com=A0=A0=A0=A0=A0=A0=A0 Mobile: +1 (978) 394-7754
> ----------------------------------------------------
=A0
_______________________________________________
storm mailing list
=A0
=A0

_______________________________________________
storm mailing list
storm@ietf.or= g
https://www.ietf.org/mailman/listinfo/storm




--
Cheers,
Arkady Kanevsky

_______________________________________________
storm mailing list
storm@ietf.org
h=
ttps://www.ietf.org/mailman/listinfo/storm




--
Cheers,
= Arkady Kanevsky



--
Cheers,
Arkad= y Kanevsky
--000e0cd2dc3ed86f3104a02032b6-- --000e0cd2dc3ed86f6204a02032b8 Content-Type: text/plain; charset=US-ASCII; name="draft-ietf-storm-mpa-peer-connect-04.txt" Content-Disposition: attachment; filename="draft-ietf-storm-mpa-peer-connect-04.txt" Content-Transfer-Encoding: base64 X-Attachment-Id: f_gm41ak010 CgoKU1RPUk0gICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg ICBBLiBLYW5ldnNreSwgRWQuCkludGVybmV0LURyYWZ0ICAgICAgICAgICAgICAgICAgICAgICAg ICAgICAgICAgICAgICAgICAgICAgICAgICAgIFZNd2FyZQpVcGRhdGVzOiA1MDQzLCA1MDQ0IChp ZiBhcHByb3ZlZCkgICAgICAgICAgICAgICAgICAgICAgICBDLiBCZXN0bGVyLCBFZC4KSW50ZW5k ZWQgc3RhdHVzOiBTdGFuZGFyZHMgVHJhY2sgICAgICAgICAgICAgICAgICAgICAgICAgICAgICBD b25zdWx0YW50CkV4cGlyZXM6IE9jdG9iZXIgNywgMjAxMSAgICAgICAgICAgICAgICAgICAgICAg ICAgICAgICAgICAgICAgICBSLiBTaGFycAogICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgSW50ZWwKICAgICAgICAgICAgICAg ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICBTLiBXaXNl CiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgT3Bl biBHcmlkIENvbXB1dGluZwogICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg ICAgICAgICAgICAgICAgICAgIEFwcmlsIDUsIDIwMTEKCgogICAgICAgICAgICAgICAgIEVuaGFu Y2VkIFJETUEgQ29ubmVjdGlvbiBFc3RhYmxpc2htZW50CiAgICAgICAgICAgICAgICAgIGRyYWZ0 LWlldGYtc3Rvcm0tbXBhLXBlZXItY29ubmVjdC0wNAoKQWJzdHJhY3QKCiAgIFRoaXMgZG9jdW1l bnQgdXBkYXRlcyBbUkZDNTA0M10gYW5kIFtSRkM1MDQ0XSBieSBleHRlbmRpbmcgTVBBCiAgIG5l Z290aWF0aW9uIGZvciBSRE1BIENvbm5lY3Rpb24gZXN0YWJsaXNobWVudC4gIFRoZSBmaXJzdCBl eHRlbnNpb24KICAgZXh0ZW5kcyBbUkZDNTA0M10sIGVuYWJsaW5nIHBlZXItdG8tcGVlciBjb25u ZWN0aW9uIGVzdGFibGlzaG1lbnQKICAgb3ZlciBNUEEvVENQLiAgVGhlIHNlY29uZCBleHRlbnNp b24gZXh0ZW5kcyBib3RoIFtSRkM1MDQzXSBhbmQKICAgW1JGQzUwNDRdLCBieSBwcm92aWRpbmcg YW4gb3B0aW9uIGZvciBzdGFuZGFyZGl6ZWQgZXhjaGFuZ2Ugb2YgUkRNQS0KICAgbGF5ZXIgY29u bmVjdGlvbiBjb25maWd1cmF0aW9uLgoKU3RhdHVzIG9mIHRoaXMgTWVtbwoKICAgVGhpcyBJbnRl cm5ldC1EcmFmdCBpcyBzdWJtaXR0ZWQgaW4gZnVsbCBjb25mb3JtYW5jZSB3aXRoIHRoZQogICBw cm92aXNpb25zIG9mIEJDUCA3OCBhbmQgQkNQIDc5LgoKICAgSW50ZXJuZXQtRHJhZnRzIGFyZSB3 b3JraW5nIGRvY3VtZW50cyBvZiB0aGUgSW50ZXJuZXQgRW5naW5lZXJpbmcKICAgVGFzayBGb3Jj ZSAoSUVURikuICBOb3RlIHRoYXQgb3RoZXIgZ3JvdXBzIG1heSBhbHNvIGRpc3RyaWJ1dGUKICAg d29ya2luZyBkb2N1bWVudHMgYXMgSW50ZXJuZXQtRHJhZnRzLiAgVGhlIGxpc3Qgb2YgY3VycmVu dCBJbnRlcm5ldC0KICAgRHJhZnRzIGlzIGF0IGh0dHA6Ly9kYXRhdHJhY2tlci5pZXRmLm9yZy9k cmFmdHMvY3VycmVudC8uCgogICBJbnRlcm5ldC1EcmFmdHMgYXJlIGRyYWZ0IGRvY3VtZW50cyB2 YWxpZCBmb3IgYSBtYXhpbXVtIG9mIHNpeCBtb250aHMKICAgYW5kIG1heSBiZSB1cGRhdGVkLCBy ZXBsYWNlZCwgb3Igb2Jzb2xldGVkIGJ5IG90aGVyIGRvY3VtZW50cyBhdCBhbnkKICAgdGltZS4g IEl0IGlzIGluYXBwcm9wcmlhdGUgdG8gdXNlIEludGVybmV0LURyYWZ0cyBhcyByZWZlcmVuY2UK ICAgbWF0ZXJpYWwgb3IgdG8gY2l0ZSB0aGVtIG90aGVyIHRoYW4gYXMgIndvcmsgaW4gcHJvZ3Jl c3MuIgoKICAgVGhpcyBJbnRlcm5ldC1EcmFmdCB3aWxsIGV4cGlyZSBvbiBPY3RvYmVyIDcsIDIw MTEuCgpDb3B5cmlnaHQgTm90aWNlCgogICBDb3B5cmlnaHQgKGMpIDIwMTEgSUVURiBUcnVzdCBh bmQgdGhlIHBlcnNvbnMgaWRlbnRpZmllZCBhcyB0aGUKICAgZG9jdW1lbnQgYXV0aG9ycy4gIEFs bCByaWdodHMgcmVzZXJ2ZWQuCgogICBUaGlzIGRvY3VtZW50IGlzIHN1YmplY3QgdG8gQkNQIDc4 IGFuZCB0aGUgSUVURiBUcnVzdCdzIExlZ2FsCiAgIFByb3Zpc2lvbnMgUmVsYXRpbmcgdG8gSUVU RiBEb2N1bWVudHMKICAgKGh0dHA6Ly90cnVzdGVlLmlldGYub3JnL2xpY2Vuc2UtaW5mbykgaW4g ZWZmZWN0IG9uIHRoZSBkYXRlIG9mCgoKCkthbmV2c2t5LCBldCBhbC4gICAgICAgICBFeHBpcmVz IE9jdG9iZXIgNywgMjAxMSAgICAgICAgICAgICAgICBbUGFnZSAxXQoMCkludGVybmV0LURyYWZ0 ICAgRW5oYW5jZWQgUkRNQSBDb25uZWN0aW9uIEVzdGFibGlzaG1lbnQgICAgICAgQXByaWwgMjAx MQoKCiAgIHB1YmxpY2F0aW9uIG9mIHRoaXMgZG9jdW1lbnQuICBQbGVhc2UgcmV2aWV3IHRoZXNl IGRvY3VtZW50cwogICBjYXJlZnVsbHksIGFzIHRoZXkgZGVzY3JpYmUgeW91ciByaWdodHMgYW5k IHJlc3RyaWN0aW9ucyB3aXRoIHJlc3BlY3QKICAgdG8gdGhpcyBkb2N1bWVudC4gIENvZGUgQ29t cG9uZW50cyBleHRyYWN0ZWQgZnJvbSB0aGlzIGRvY3VtZW50IG11c3QKICAgaW5jbHVkZSBTaW1w bGlmaWVkIEJTRCBMaWNlbnNlIHRleHQgYXMgZGVzY3JpYmVkIGluIFNlY3Rpb24gNC5lIG9mCiAg IHRoZSBUcnVzdCBMZWdhbCBQcm92aXNpb25zIGFuZCBhcmUgcHJvdmlkZWQgd2l0aG91dCB3YXJy YW50eSBhcwogICBkZXNjcmliZWQgaW4gdGhlIFNpbXBsaWZpZWQgQlNEIExpY2Vuc2UuCgoKVGFi bGUgb2YgQ29udGVudHMKCiAgIDEuICBJbnRyb2R1Y3Rpb24gLiAuIC4gLiAuIC4gLiAuIC4gLiAu IC4gLiAuIC4gLiAuIC4gLiAuIC4gLiAuIC4gLiAgMwogICAgIDEuMS4gIFN1bW1hcnkgb2YgY2hh bmdlcyBmcm9tIFJGQyA1MDQ0IC4gLiAuIC4gLiAuIC4gLiAuIC4gLiAuIC4gIDMKICAgICAxLjIu ICBTdW1tYXJ5IG9mIGNoYW5nZXMgZnJvbSBSRkMgNTA0MyAuIC4gLiAuIC4gLiAuIC4gLiAuIC4g LiAuICAzCiAgIDIuICBSZXF1aXJlbWVudHMgTGFuZ3VhZ2UgIC4gLiAuIC4gLiAuIC4gLiAuIC4g LiAuIC4gLiAuIC4gLiAuIC4gLiAgMwogICAzLiAgRGVmaW5pdGlvbnMgIC4gLiAuIC4gLiAuIC4g LiAuIC4gLiAuIC4gLiAuIC4gLiAuIC4gLiAuIC4gLiAuIC4gIDMKICAgNC4gIE1vdGl2YXRpb25z ICAuIC4gLiAuIC4gLiAuIC4gLiAuIC4gLiAuIC4gLiAuIC4gLiAuIC4gLiAuIC4gLiAuICA0CiAg ICAgNC4xLiAgRW5hYmxpbmcgTVBBIE1vZGUgIC4gLiAuIC4gLiAuIC4gLiAuIC4gLiAuIC4gLiAu IC4gLiAuIC4gLiAgNQogICAgIDQuMi4gIExhY2sgb2YgRXhwbGljaXQgUlRSIGluIE1QQSBSZXF1 ZXN0L1JlcGx5IEV4Y2hhbmdlIC4gLiAuIC4gIDUKICAgICA0LjMuICBMaW1pdGF0aW9ucyBvbiBV TFAgV29ya2Fyb3VuZCAgLiAuIC4gLiAuIC4gLiAuIC4gLiAuIC4gLiAuICA2CiAgICAgICA0LjMu MS4gIFRyYW5zcG9ydCBOZXV0cmFsIEFQSXMgLiAuIC4gLiAuIC4gLiAuIC4gLiAuIC4gLiAuIC4g LiAgNwogICAgICAgNC4zLjIuICBXb3JrL0NvbXBsZXRpb24gUXVldWUgQWNjb3VudGluZyAuIC4g LiAuIC4gLiAuIC4gLiAuIC4gIDcKICAgICAgIDQuMy4zLiAgSG9zdC1iYXNlZCBJbXBsZW1lbnRh dGlvbiBvZiBNUEEgRmVuY2luZyAuIC4gLiAuIC4gLiAuICA4CiAgICAgNC40LiAgU3RhbmRhcmRp emVkIFJETUEgUGFyYW1ldGVyIE5lZ290aWF0aW9uICAuIC4gLiAuIC4gLiAuIC4gLiAgOAogICA1 LiAgTVBBIENvbm5lY3Rpb24gU2V0dXAgLiAuIC4gLiAuIC4gLiAuIC4gLiAuIC4gLiAuIC4gLiAu IC4gLiAuIC4gIDkKICAgNi4gIEVuaGFuY2VkIE1QQSBSZXF1ZXN0L1JlcGx5IEZyYW1lcyAgLiAu IC4gLiAuIC4gLiAuIC4gLiAuIC4gLiAuIDEwCiAgIDcuICBFbmhhbmNlZCBTQ1RQIFNlc3Npb24g Q29udHJvbCBDaHVua3MgLiAuIC4gLiAuIC4gLiAuIC4gLiAuIC4gLiAxMQogICA4LiAgTVBBIEVy cm9yIFJlcG9ydGluZyAgLiAuIC4gLiAuIC4gLiAuIC4gLiAuIC4gLiAuIC4gLiAuIC4gLiAuIC4g MTMKICAgOS4gIEVuaGFuY2VkIFJETUEgQ29ubmVjdGlvbiBFc3RhYmxpc2htZW50IERhdGEgIC4g LiAuIC4gLiAuIC4gLiAuIDEzCiAgIDEwLiBJbnRlcm9wZXJhYmlsaXR5IC4gLiAuIC4gLiAuIC4g LiAuIC4gLiAuIC4gLiAuIC4gLiAuIC4gLiAuIC4gLiAxNQogICAxMS4gSUFOQSBDb25zaWRlcmF0 aW9ucyAgLiAuIC4gLiAuIC4gLiAuIC4gLiAuIC4gLiAuIC4gLiAuIC4gLiAuIC4gMTYKICAgMTIu IFNlY3VyaXR5IENvbnNpZGVyYXRpb25zICAuIC4gLiAuIC4gLiAuIC4gLiAuIC4gLiAuIC4gLiAu IC4gLiAuIDE2CiAgIDEzLiBBY2tub3dsZWRnZW1lbnRzIC4gLiAuIC4gLiAuIC4gLiAuIC4gLiAu IC4gLiAuIC4gLiAuIC4gLiAuIC4gLiAxNgogICAxNC4gUmVmZXJlbmNlcyAuIC4gLiAuIC4gLiAu IC4gLiAuIC4gLiAuIC4gLiAuIC4gLiAuIC4gLiAuIC4gLiAuIC4gMTYKICAgICAxNC4xLiBOb3Jt YXRpdmUgUmVmZXJlbmNlcyAuIC4gLiAuIC4gLiAuIC4gLiAuIC4gLiAuIC4gLiAuIC4gLiAuIDE2 CiAgICAgMTQuMi4gSW5mb3JtYXRpdmUgUmVmZXJlbmNlcyAuIC4gLiAuIC4gLiAuIC4gLiAuIC4g LiAuIC4gLiAuIC4gLiAxNgogICBBdXRob3JzJyBBZGRyZXNzZXMgLiAuIC4gLiAuIC4gLiAuIC4g LiAuIC4gLiAuIC4gLiAuIC4gLiAuIC4gLiAuIC4gMTcKCgoKCgoKCgoKCgoKCgoKS2FuZXZza3ks IGV0IGFsLiAgICAgICAgIEV4cGlyZXMgT2N0b2JlciA3LCAyMDExICAgICAgICAgICAgICAgIFtQ YWdlIDJdCgwKSW50ZXJuZXQtRHJhZnQgICBFbmhhbmNlZCBSRE1BIENvbm5lY3Rpb24gRXN0YWJs aXNobWVudCAgICAgICBBcHJpbCAyMDExCgoKMS4gIEludHJvZHVjdGlvbgoKICAgV2hlbiB1c2Vk IG92ZXIgVENQLCB0aGUgY3VycmVudCBSRERQIHN1aXRlIG9mIHByb3RvY29scyByZWxpZXMgb24K ICAgTWFya2VyIFBEVSBBbGlnbm1lbnQgKE1QQSkgW1JGQzUwNDRdIHByb3RvY29sIGZvciBib3Ro IGNvbm5lY3Rpb24KICAgZXN0YWJsaXNobWVudCBhbmQgZm9yIG1hcmtlcnMgZm9yIFRDUCBsYXll cmluZy4gIEN1cnJlbnRseSBNUEEgb25seQogICBzdXBwb3J0cyBhIGNsaWVudC1zZXJ2ZXIgbW9k ZWwgZm9yIGNvbm5lY3Rpb24gZXN0YWJsaXNobWVudCwgZm9yY2luZwogICBwZWVyLXRvLXBlZXIg YXBwbGljYXRpb25zIHRvIGludGVyYWN0IGFzIHRob3VnaCB0aGV5IGhhZCBhIGNsaWVudC8KICAg c2VydmVyIHJlbGF0aW9uc2hpcC4gIEluIGFkZGl0aW9uIG5lZ290aWF0aW9uIG9mIHNvbWUgb2Yg UmVtb3RlCiAgIERpcmVjdCBNZW1vcnkgQWNjZXNzIFByb3RvY29sIChSRE1BUCkgW1JGQzUwNDBd IHNwZWNpZmljIHBhcmFtZXRlcnMKICAgYXJlIGxlZnQgdG8gVUxQIG5lZ290aWF0aW9uLiAgUHJv dmlkaW5nIGFuIG9wdGlvbmFsIFVMUC1pbmRlcGVuZGVudAogICBmb3JtYXQgZm9yIGV4Y2hhbmdp bmcgdGhlc2UgcGFyYW1ldGVycyB3b3VsZCBiZSBvZiBiZW5lZml0IHRvCiAgIHRyYW5zcG9ydCBu ZXV0cmFsIFJETUEgYXBwbGljYXRpb25zLgoKMS4xLiAgU3VtbWFyeSBvZiBjaGFuZ2VzIGZyb20g UkZDIDUwNDQKCiAgIFRoaXMgZHJhZnQgZXh0ZW5kcyBbUkZDNTA0NF0gTVBBIGNvbm5lY3Rpb24g c2V0dXAgcHJvdG9jb2wuICBGaXJzdCwKICAgaXQgYWRkIGV4Y2hhbmdlIGFuZCBuZWdvdGlhdGlv biBvZiBtYXhpbXVtIG51bWJlciBvZiBSRE1BIFJlYWQKICAgSW5jb21pbmcgKElSRCkgYW5kIE91 dGdvaW5nIG1lc3NhZ2VzIChPUkQpLiAgU2Vjb25kLCBpdCBhZGRzIG9uZSBtb3JlCiAgIFJlYWR5 IHRvIFJlY2VpdmUgKFJUUikgZnJhbWUgZnJvbSByZXF1ZXN0b3IgdG8gcmVzcG9uZGVyIGFzIHRo ZSBsYXN0CiAgIG1lc3NhZ2Ugb2YgY29ubmVjdGlvbiBlc3RhYmxpc2htZW50IGFuZCBhZGRzIG5l Z290aWF0aW9uIG9mIFJUUiBmcmFtZQogICBtZXNzYWdlIHR5cGUgaW50byBNUEEgcmVxdWVzdC9y ZXNwb25zZSBmcmFtZXMuCgoxLjIuICBTdW1tYXJ5IG9mIGNoYW5nZXMgZnJvbSBSRkMgNTA0MwoK ICAgVGhpcyBkcmFmdCBleHRlbmRzIFtSRkM1MDQzXSBieSBhZGRpbmcgbmV3IEVuaGFuY2VkIFNl c3Npb24gQ29udHJvbAogICBDaHVua3MgdGhhdCBleHRlbmQgdGhlIGN1cnJlbnRseSBkZWZpbmVk IENodW5rcyB3aXRoIHRoZSBhZGRpdGlvbiBvZgogICBJUkQgYW5kIE9SRCBuZWdvdGlhdGlvbi4K CgoyLiAgUmVxdWlyZW1lbnRzIExhbmd1YWdlCgogICBUaGUga2V5IHdvcmRzICJNVVNUIiwgIk1V U1QgTk9UIiwgIlJFUVVJUkVEIiwgIlNIQUxMIiwgIlNIQUxMIE5PVCIsCiAgICJTSE9VTEQiLCAi U0hPVUxEIE5PVCIsICJSRUNPTU1FTkRFRCIsICJNQVkiLCBhbmQgIk9QVElPTkFMIiBpbiB0aGlz CiAgIGRvY3VtZW50IGFyZSB0byBiZSBpbnRlcnByZXRlZCBhcyBkZXNjcmliZWQgaW4gW1JGQzIx MTldLgoKCjMuICBEZWZpbml0aW9ucwoKICAgRlVMUERVOiAgRnJhbWVkIFVwcGVyIExheWVyIFBy b3RvY29sIFBEVS4gIFNlZSBGUERVIG9mIFtSRkM1MDQ0XS4KCiAgIENvbXBsZXRpb24gUXVldWUg KENRKTogIEEgY29uc3VtZXIgYWNjZXNzaWJsZSBxdWV1ZSB3aGVyZSB0aGUgUkRNQQogICAgICBk ZXZpY2UgcmVwb3J0cyBjb21wbGV0aW9ucyBvZiBXb3JrIFJlcXVlc3RzLiAgQSBDb25zdW1lciBp cyBhYmxlCiAgICAgIHRvIHJlYXAgY29tcGxldGlvbnMgZnJvbSBhIENRIHdpdGhvdXQgcmVxdWly aW5nIHBlciB0cmFuc2FjdGlvbgogICAgICBzdXBwb3J0IGZyb20gdGhlIGtlcm5lbCBvciBvdGhl ciBwcml2aWxlZ2VkIGVudGl0eS4gIFNlZSBbUkRNQUNdLgoKCgoKCgoKS2FuZXZza3ksIGV0IGFs LiAgICAgICAgIEV4cGlyZXMgT2N0b2JlciA3LCAyMDExICAgICAgICAgICAgICAgIFtQYWdlIDNd CgwKSW50ZXJuZXQtRHJhZnQgICBFbmhhbmNlZCBSRE1BIENvbm5lY3Rpb24gRXN0YWJsaXNobWVu dCAgICAgICBBcHJpbCAyMDExCgoKICAgQ29tcGxldGlvbiBRdWV1ZSBFbnRyeSAoQ1FFKTogIFRy YW5zcG9ydCBhbmQgZGV2aWNlIHNwZWNpZmljCiAgICAgIHJlcHJlc2VudGF0aW9uIG9mIGEgV29y ayBDb21wbGV0aW9uLiAgQSBDb21wbGV0aW9uIFF1ZXVlIGhvbGRzCiAgICAgIENRRXMuICBTZWUg W1JETUFDXS4KCiAgIEluYm91bmQgUkRNQSBSZWFkIFF1ZXVlIERlcHRoIChJUkQpOiAgVGhlIG1h eGltdW0gbnVtYmVyIG9mIGluY29taW5nCiAgICAgIG91dHN0YW5kaW5nIFJETUEgUmVhZCBSZXF1 ZXN0IE1lc3NhZ2VzIGFuIFJETUEgY29ubmVjdGlvbiBjYW4KICAgICAgaGFuZGxlLiAgU2VlIFtS RE1BQ10uCgogICBJUkQ6ICBTZWUgSW5ib3VuZCBSRE1BIFJlYWQgUXVldWUgRGVwdGguCgogICBP UkQ6ICBTZWUgT3V0Ym91bmQgUkRNQSBSZWFkIFF1ZXVlIERlcHRoLgoKICAgT3V0Ym91bmQgUkRN QSBSZWFkIFF1ZXVlIERlcHRoIChPUkQpOiAgVGhlIG1heGltdW0gbnVtYmVyIG9mCiAgICAgIG91 dHN0YW5kaW5nIFJETUEgUmVhZCBSZXF1ZXN0cyB0aGF0IGNhbiBiZSBpc3N1ZWQgZm9yIHRoZSBS RE1BCiAgICAgIGNvbm5lY3Rpb24uICBUaGlzIHNob3VsZCBiZSBsZXNzIHRoYW4gb3IgZXF1YWwg dG8gdGhlIHBlZXIncyBJUkQuCiAgICAgIFNlZSBbUkRNQUNdLgoKICAgUXVldWUgUGFpciAoUVAp OiAgVGhlIHRyYWRpdGlvbmFsIG5hbWUgZm9yIGEgbG9jYWwgRW5kcG9pbnQgaW4gYQogICAgICBb VklBXSBkZXJpdmVkIGxvY2FsIGludGVyZmFjZS4gIEEgUXVldWUgUGFpciBpcyB0aGUgc2V0IG9m IFdvcmsKICAgICAgUXVldWVzIGFzc29jaWF0ZWQgZXhjbHVzaXZlbHkgd2l0aCBhIHNpbmdsZSBF bmRwb2ludC4gIFRoZSBTZW5kCiAgICAgIFF1ZXVlIChTUSksIFJlY2VpdmUgUXVldWUgKFJRKSBh bmQgSW5ib3VuZCBSRE1BIFJlYWQgUXVldWUgKElSUSkKICAgICAgYXJlIGNvbnNpZGVyZWQgdG8g YmUgcGFydCBvZiB0aGUgUXVldWUgUGFpci4gIFRoZSBwb3RlbnRpYWxseQogICAgICBzaGFyZWQg Q29tcGxldGlvbiBRdWV1ZSAoQ1EpIGFuZCBTaGFyZWQgUmVjZWl2ZSBRdWV1ZSAoU1JRKSBhcmUK ICAgICAgbm90LiAgU2VlIFtSRE1BQ10uCgogICBTaGFyZWQgUmVjZWl2ZSBRdWV1ZShTUlEpOiAg QSBzaGFyZWQgcG9vbCBvZiBSZWNlaXZlIFdvcmsgUmVxdWVzdHMKICAgICAgcG9zdGVkIGJ5IHRo ZSBDb25zdW1lciB0aGF0IGNhbiBiZSBhbGxvY2F0ZWQgYnkgbXVsdGlwbGUgUkRNQQogICAgICBl bmRwb2ludHMgKFF1ZXVlIFBhaXIpLiAgU2VlIFtEQVBMXS4KCiAgIFdvcmsgUXVldWU6ICBBbiBl bGVtZW50IG9mIGEgW1ZJQV0gZGVyaXZlZCBsb2NhbCBpbnRlcmZhY2UgdGhhdAogICAgICBhbGxv d3MgdXNlci1zcGFjZSBhcHBsaWNhdGlvbnMgdG8gc3VibWl0IFdvcmsgUmVxdWVzdHMgZGlyZWN0 bHkgdG8KICAgICAgbmV0d29yayBoYXJkd2FyZS4gIFNwZWNpZmljIFdvcmsgUXVldWVzIGluY2x1 ZGUgdGhlIFNlbmQgUXVldWUKICAgICAgKFNRKSBmb3IgdHJhbnNtaXQgcmVxdWVzdHMsIFJlY2Vp dmUgUXVldWUgKFJRKSBmb3IgcmVjZWl2ZQogICAgICByZXF1ZXN0cyBzcGVjaWZpYyB0byBhIHNp bmdsZSBFbmRwb2ludCBhbmQgU2hhcmVkIFJlY2VpdmUgUXVldWVzCiAgICAgIChTUlFzKSBmb3Ig cmVjZWl2ZSByZXF1ZXN0cyB0aGF0IGNhbiBiZSBhbGxvY2F0ZWQgYnkgb25lIG9yIG1vcmUKICAg ICAgRW5kcG9pbnRzLiAgU2VlIFtSRE1BQ10uCgogICBXb3JrIFF1ZXVlIEVsZW1lbnQgKFdRRSk6 ICBUcmFuc3BvcnQgYW5kIGRldmljZSBzcGVjaWZpYwogICAgICByZXByZXNlbnRhdGlvbiBvZiBh IFdvcmsgUmVxdWVzdC4gIFNlZSBbUkRNQUNdCgogICBXb3JrIFJlcXVlc3Q6ICBBbiBlbGVtZW50 YXJ5IG9iamVjdCB1c2VkIGJ5IENvbnN1bWVycyB0byBlbnF1ZXVlIGEKICAgICAgcmVxdWVzdGVk IG9wZXJhdGlvbiAoV1FFcykgb250byBhIFdvcmsgUXVldWUuICBTZWUgW1JETUFDXS4KCgo0LiAg TW90aXZhdGlvbnMKCiAgIFRoZSBnb2FsIG9mIHRoaXMgZHJhZnQgaXMgdHdvZm9sZC4gIE9uZSBp cyB0byBleHRlbmQgc3VwcG9ydCBmcm9tIHRoZQogICBjdXJyZW50IGNsaWVudC1zZXJ2ZXIgbW9k ZWwgZm9yIFJETUEgY29ubmVjdGlvbiBzZXR1cCB0byBhIHBlZXItdG8tCgoKCkthbmV2c2t5LCBl dCBhbC4gICAgICAgICBFeHBpcmVzIE9jdG9iZXIgNywgMjAxMSAgICAgICAgICAgICAgICBbUGFn ZSA0XQoMCkludGVybmV0LURyYWZ0ICAgRW5oYW5jZWQgUkRNQSBDb25uZWN0aW9uIEVzdGFibGlz aG1lbnQgICAgICAgQXByaWwgMjAxMQoKCiAgIHBlZXIgbW9kZWwuICBUaGUgc2Vjb25kIGlzIHRv IGFkZCBuZWdvdGlhdGlvbiBvZiBSRE1BIFJlYWQgcXVldWUgc2l6ZQogICBmb3IgYm90aCBzaWRl cyBvZiBhbiBSRE1BIGNvbm5lY3Rpb24uCgo0LjEuICBFbmFibGluZyBNUEEgTW9kZQoKICAgTVBB IGRlZmluZXMgZW5jb2Rpbmcgb2YgRERQIFNlZ21lbnRzIGluIEZVTFBEVXMgKEZyYW1lZCBVcHBl ciBMYXllcgogICBQcm90b2NvbCBQRFVzKS4gIEdlbmVyYXRpb24gb2YgRlVMUERVcyByZXF1aXJl cyB0aGUgYWJpbGl0eSB0bwogICBwZXJpb2RpY2FsbHkgaW5zZXJ0IE1QQSBNYXJrZXJzIGFuZCB0 byBnZW5lcmF0ZSB0aGUgTVBBIENSQy0zMmMgZm9yCiAgIGVhY2ggZnJhbWUuICBSZWNlcHRpb24g bWF5IHJlcXVpcmUgcGFyc2luZy9yZW1vdmluZyB0aGUgbWFya2VycyBhZnRlcgogICB1c2luZyB0 aGVtIHRvIGlkZW50aWZ5IE1QQSBGcmFtZSBib3VuZGFyaWVzLCBhbmQgdmFsaWRhdGlvbiBvZiB0 aGUKICAgTVBBLUNSQzMyYy4KCiAgIEEgbWFqb3IgZGVzaWduIG9iamVjdGl2ZSBmb3IgTVBBIHdh cyB0byBlbnN1cmUgdGhhdCB0aGUgcmVzdWx0aW5nIFRDUAogICBzdHJlYW0gd291bGQgYmUgYSBm dWxseSBjb21wbGlhbnQgVENQIHN0cmVhbSBmb3IgYW55IGFuZCBhbGwgVENQLQogICBhd2FyZSBt aWRkbGUtYm94ZXMuICBUaGUgY2hhbGxlbmdlIGlzIHRoYXQgd2hpbGUgb25seSBzb21lIFRDUAog ICBwYXlsb2FkIHN0cmVhbXMgYXJlIGEgdmFsaWQgc3RyZWFtIG9mIE1QQSBGVUxQRFVzLCBhbnkg c2VxdWVuY2Ugb2YKICAgYnl0ZXMgaXMgYSB2YWxpZCBUQ1AgcGF5bG9hZCBzdHJlYW0uICBUaGUg ZGV0ZXJtaW5hdGlvbiB0aGF0IGEgZ2l2ZW4KICAgc3RyZWFtIGlzIGluIGEgc3BlY2lmaWMgTVBB IG1vZGUgY2Fubm90IGJlIG1hZGUgYXQgdGhlIE1QQSBvciBUQ1AKICAgbGF5ZXIuICBUaGVyZWZv cmUgZW5hYmxpbmcgb2YgTVBBIG1vZGUgaXMgaGFuZGxlZCBieSB0aGUgVUxQLgoKICAgVGhlIE1Q QSBwcm90b2NvbCBjYW4gYmUgdmlld2VkIGFzIGhhdmluZyB0d28gcGFydHMuCgogICBvICBhIHNw ZWNpZmljYXRpb24gb2YgZ2VuZXJhdGlvbiBhbmQgcmVjZXB0aW9uIG9mIE1QQSBGVUxQRFVzLiAg VGhpcwogICAgICBpcyB1bmNoYW5nZWQgYnkgRW5oYW5jZWQgUkRNQSBDb25uZWN0aW9uIEVzdGFi bGlzaG1lbnQuCgogICBvICBhIHByZS1NUEEgZXhjaGFuZ2Ugb2YgbWVzc2FnZXMgdG8gZW5hYmxl IGEgc3BlY2lmaWMgTVBBIG1vZGUgZm9yCiAgICAgIHRoZSBUQ1AgY29ubmVjdGlvbi4gIEVuaGFu Y2VkIFJETUEgQ29ubmVjdGlvbiBFc3RhYmxpc2htZW50CiAgICAgIGV4dGVuZHMgdGhpcyBwcm90 b2NvbCB3aXRoIHR3byBuZXcgZmVhdHVyZXMuCgogICBJbiB0eXBpY2FsIGltcGxlbWVudGF0aW9u cywgZ2VuZXJhdGlvbiBhbmQgcmVjZXB0aW9uIG9mIE1QQSBGVUxQRFVzCiAgIGlzIGhhbmRsZWQg YnkgaGFyZHdhcmUuICBUaGUgZXhjaGFuZ2Ugb2YgdGhlIE1QQSBSZXF1ZXN0IGFuZCBSZXBseQog ICBmcmFtZXMgaXMgdGhlbiBoYW5kbGVkIGJ5IGhvc3Qgc29mdHdhcmUuICBBcyB3aWxsIGJlIGV4 cGxhaW5lZCwgdGhpcwogICBpbXBsZW1lbnRhdGlvbiBzcGxpdCBwcmV2ZW50cyBhcHBsaWNhdGlv bnMgZnJvbSB3b3JraW5nIGFyb3VuZCB0aGUKICAgY2xpZW50LXNlcnZlciBhc3N1bXB0aW9ucyBp biB0aGUgY3VycmVudCBNUEEgUmVxdWVzdC9SZXBseSBleGNoYW5nZS4KCjQuMi4gIExhY2sgb2Yg RXhwbGljaXQgUlRSIGluIE1QQSBSZXF1ZXN0L1JlcGx5IEV4Y2hhbmdlCgogICBUaGUgZXhjaGFu Z2Ugb2YgTVBBIFJlcXVlc3QgYW5kIFJlcGx5IG1lc3NhZ2VzIHRvIHBsYWNlIGEgVENQCiAgIGNv bm5lY3Rpb24gaW4gTVBBIG1vZGUgaXMgc3BlY2lmaWVkIGluIFtSRkM1MDQ0XS4gIFRoaXMgcHJv dG9jb2wKICAgcHJvdmlkZXMgbWFueSBiZW5lZml0cyB0byB0aGUgZGVzaWduIG9mIE1QQSBGVUxQ RFUgaGFyZHdhcmU6CgogICBvICBUaGUgVUxQIGlzIHJlc3BvbnNpYmxlIGZvciBzcGVjaWZ5aW5n IHRoZSBleGFjdCBNUEEgTW9kZSAoTWFya2VycwogICAgICBlbmFibGVkIG9yIGRpc2FibGVkLCBD UkMtMzJjIGVuYWJsZWQgb3Igc3VwcHJlc3NlZCkgYW5kIHRoZSBwb2ludAogICAgICBpbiB0aGUg VENQIHN0cmVhbXMgKGluYm91bmQgYW5kIG91dGJvdW5kKSB3aGVyZSBNUEEgZnJhbWVzIHdpbGwK ICAgICAgYmVnaW4uCgogICBvICBCZWZvcmUgdGhlIGZpcnN0IE1QQSBmcmFtZSBpcyB0cmFuc21p dHRlZCwgYWxsIHByZS1NUEEgbW9kZSBUQ1AKICAgICAgcGF5bG9hZCB3aWxsIGhhdmUgYmVlbiBh Y2tub3dsZWRnZWQgYnkgdGhlIHBlZXIuICBUaGVyZWZvcmUgaXQgaXMKCgoKS2FuZXZza3ksIGV0 IGFsLiAgICAgICAgIEV4cGlyZXMgT2N0b2JlciA3LCAyMDExICAgICAgICAgICAgICAgIFtQYWdl IDVdCgwKSW50ZXJuZXQtRHJhZnQgICBFbmhhbmNlZCBSRE1BIENvbm5lY3Rpb24gRXN0YWJsaXNo bWVudCAgICAgICBBcHJpbCAyMDExCgoKICAgICAgbmV2ZXIgbmVjZXNzYXJ5IHRvIGdlbmVyYXRl IGEgcmV0cmFuc21pc3Npb24gdGhhdCBtaXhlcyBwcmUtTVBBCiAgICAgIGFuZCBNUEEgcGF5bG9h ZC4KCiAgIG8gIEJlZm9yZSBNUEEgcmVjZXB0aW9uIGlzIGVuYWJsZWQsIGFsbCBpbmNvbWluZyBw cmUtTVBBIG1vZGUgVENQCiAgICAgIHBheWxvYWQgd2lsbCBoYXZlIGJlZW4gYWNrbm93bGVkZ2Vk LiAgVGhlcmVmb3JlIHRoZSBob3N0IHdpbGwKICAgICAgbmV2ZXIgcmVjZWl2ZSBhIFRDUCBzZWdt ZW50IHRoYXQgbWl4ZXMgcHJlLU1QQSBhbmQgTVBBIHBheWxvYWQuCgogICBUaGUgbGltaXRhdGlv biBvZiB0aGUgY3VycmVudCBNUEEgUmVxdWVzdC9SZXBseSBleGNoYW5nZSBpcyB0aGF0IGl0CiAg IGRvZXMgbm90IGRlZmluZSBhIFJlYWR5IHRvIFJlY2VpdmUgKFJUUikgbWVzc2FnZSB0aGF0IHRo ZSBhY3RpdmUgc2lkZQogICB3b3VsZCBzZW5kLCBzbyB0aGF0IHRoZSBwYXNzaXZlIHNpZGUgY2Fu IGtub3cgdGhhdCB0aGUgbGFzdCBub24tTVBBCiAgIHBheWxvYWQgKHRoZSBNUEEgUmVwbHkpIGhh ZCBiZWVuIHJlY2VpdmVkLgoKICAgSW5zdGVhZCwgdGhlIHJvbGUgb2YgYW4gUlRSIG1lc3NhZ2Ug aXMgcGlnZ3ktYmFja2VkIG9uIHRoZSBmaXJzdCBNUEEKICAgRlVMUERVIHNlbnQgYnkgdGhlIGFj dGl2ZSBzaWRlLiAgVGhpcyBpcyBhY3R1YWxseSBhIHZhbHVhYmxlCiAgIG9wdGltaXphdGlvbiBm b3IgYWxsIGFwcGxpY2F0aW9ucyB0aGF0IGZpdCB0aGUgY2xhc3NpYyBjbGllbnQvc2VydmVyCiAg IG1vZGVsLiAgVGhlIGNsaWVudCBvbmx5IGluaXRpYXRlcyB0aGUgY29ubmVjdGlvbiB3aGVuIGl0 IGhhcyBhCiAgIHJlcXVlc3QgdG8gc2VuZCB0byB0aGUgc2VydmVyLCBhbmQgdGhlIHNlcnZlciBo YXMgbm90aGluZyB0byBzZW5kCiAgIHVudGlsIGl0IGhhcyByZWNlaXZlZCBhbmQgcHJvY2Vzc2Vk IHRoZSBjbGllbnQgcmVxdWVzdC4KCiAgIEV2ZW4gYXBwbGljYXRpb25zIHdoZXJlIHRoZSBzZXJ2 ZXIgc2VuZHMgc29tZSBjb25maWd1cmF0aW9uIGRhdGEKICAgaW1tZWRpYXRlbHkgY2FuIGVhc2ls eSBzZW5kIHRoZSBzYW1lIGluZm9ybWF0aW9uIGFzIGFwcGxpY2F0aW9uCiAgIHByaXZhdGUgZGF0 YSBpbiB0aGUgTVBBIFJlcGx5LiAgU28gdGhlIGN1cnJlbnRseSBkZWZpbmVkIGV4Y2hhbmdlCiAg IHdvcmtzIGZvciBhbG1vc3QgYWxsIGFwcGxpY2F0aW9ucy4KCiAgIE1hbnkgcGVlci10by1wZWVy IGFwcGxpY2F0aW9ucywgZXNwZWNpYWxseSB0aG9zZSBpbnZvbHZpbmcgY2x1c3RlcgogICBjYWxj dWxhdGlvbnMgKGZyZXF1ZW50bHkgdXNpbmcgTVBJIFtVc2luZ01QSV0sIG9yIFtSRFNdKSwgaGF2 ZSBubwogICBuYXR1cmFsIGNsaWVudCBvciBzZXJ2ZXIgcm9sZXMgKFtQUE1QSV0sIFtPcGVuTVBd KS4gIFR5cGljYWxseSBvbmUKICAgbWVtYmVyIG9mIHRoZSBjbHVzdGVyIGlzIGFyYml0cmFyaWx5 IHNlbGVjdGVkIHRvIGluaXRpYXRlIHRoZQogICBjb25uZWN0aW9uIHdoZW4gdGhlIGRpc3RyaWJ1 dGVkIHRhc2sgaXMgbGF1bmNoZWQsIHdoaWxlIHRoZSBvdGhlcgogICBhY2NlcHRzIGl0LiAgQXQg c3RhcnR1cCB0aW1lLCBob3dldmVyLCB0aGVyZSBpcyBubyB3YXkgdG8gcHJlZGljdAogICB3aGlj aCBub2RlIHdpbGwgaGF2ZSB0aGUgZmlyc3QgbWVzc2FnZSB0byBhY3R1YWxseSBzZW5kLgogICBF c3RhYmxpc2hpbmcgdGhlIGNvbm5lY3Rpb25zIGltbWVkaWF0ZWx5LCBob3dldmVyLCBpcyB2YWx1 YWJsZQogICBiZWNhdXNlIGl0IHJlZHVjZXMgbGF0ZW5jeSBvbmNlIHJlc3VsdHMgYXJlIHJlYWR5 IHRvIHRyYW5zbWl0IGFuZCBpdAogICB2YWxpZGF0ZXMgY29ubmVjdGl2aXR5IHRocm91Z2hvdXQg dGhlIGNsdXN0ZXIuCgogICBUaGUgbGFjayBvZiBhbiBleHBsaWNpdCBSVFIgbWVzc2FnZSBpbiB0 aGUgTVBBIFJlcXVlc3QvUmVwbHkgZXhjaGFuZ2UKICAgZm9yY2VzIGFsbCBhcHBsaWNhdGlvbnMg dG8gaGF2ZSBhIGZpcnN0IG1lc3NhZ2UgZnJvbSB0aGUgY29ubmVjdGlvbgogICBpbml0aWF0b3Is IHdoZXRoZXIgdGhpcyBtYXRjaGVzIHRoZSBhcHBsaWNhdGlvbiBjb21tdW5pY2F0aW9uIG1vZGVs CiAgIG9yIG5vdC4KCjQuMy4gIExpbWl0YXRpb25zIG9uIFVMUCBXb3JrYXJvdW5kCgogICBUaGUg cmVxdWlyZW1lbnQgdGhhdCB0aGUgUkRNQSBjb25uZWN0aW9uIGluaXRpYXRvciBzZW5kcyB0aGUg Zmlyc3QKICAgbWVzc2FnZSBkb2VzIG5vdCBhcHBlYXIgdG8gYmUgdGhhdCBvbmVyb3VzIG9uIGZp cnN0IGV4YW1pbmF0aW9uLiAgVGhlCiAgIG5hdHVyYWwgcXVlc3Rpb24gaXMgd2h5IHRoZSBhcHBs aWNhdGlvbiBsYXllciB3b3VsZCBub3Qgc2ltcGx5CiAgIGdlbmVyYXRlIGEgIm5vcCIgbWVzc2Fn ZSB3aGVuIHRoZXJlIHdhcyBubyBvdGhlciBtZXNzYWdlIHRvIHN1Ym1pdC4KCiAgIFRoZXJlIGFy ZSB0aHJlZSBmYWN0b3JzIHRoYXQgbWFrZSB0aGlzIHdvcmthcm91bmQgdW5zdWl0YWJsZSBmb3Ig bWFueQoKCgpLYW5ldnNreSwgZXQgYWwuICAgICAgICAgRXhwaXJlcyBPY3RvYmVyIDcsIDIwMTEg ICAgICAgICAgICAgICAgW1BhZ2UgNl0KDApJbnRlcm5ldC1EcmFmdCAgIEVuaGFuY2VkIFJETUEg Q29ubmVjdGlvbiBFc3RhYmxpc2htZW50ICAgICAgIEFwcmlsIDIwMTEKCgogICBwZWVyLXRvLXBl ZXIgYXBwbGljYXRpb25zLgoKICAgbyAgVHJhbnNwb3J0IE5ldXRyYWwgQVBJcy4KCiAgIG8gIFdv cmsvQ29tcGxldGlvbiBRdWV1ZSBBY2NvdW50aW5nLgoKICAgbyAgSG9zdC1iYXNlZCBpbXBsZW1l bnRhdGlvbiBvZiBNUEEgRmVuY2luZy4KCjQuMy4xLiAgVHJhbnNwb3J0IE5ldXRyYWwgQVBJcwoK ICAgTWFueSBvZiB0aGVzZSBhcHBsaWNhdGlvbnMgYWNjZXNzIFJETUEgc2VydmljZXMgdXNpbmcg YSB0cmFuc3BvcnQKICAgbmV1dHJhbCBBUEkgc3VjaCBhcyBbREFQTF0gb3IgW09GQV0uICBPbmx5 IE1QQSBoYXMgYSBmaXJzdCBtZXNzYWdlCiAgIHJlcXVpcmVtZW50LiAgT3RoZXIgUkRNQSB0cmFu c3BvcnRzLCBpbmNsdWRpbmcgU0NUUCBhbmQgSW5maW5pQmFuZCwKICAgZG8gbm90LgoKICAgQXBw bGljYXRpb24gb3IgbWlkZGxld2FyZSBjb21tdW5pY2F0aW9ucyBjYW4gYmUgZXhwcmVzc2VkIGFz CiAgIHRyYW5zcG9ydCBuZXV0cmFsIFJETUEgb3BlcmF0aW9ucywgYWxsb3dpbmcgbG93ZXIgc29m dHdhcmUgbGF5ZXJzIHRvCiAgIHRyYW5zbGF0ZSB0byB0cmFuc3BvcnQgYW5kIGRldmljZSBzcGVj aWZpY3MuICBIYXZpbmcgYSBkaXN0aW5jdCBleHRyYQogICBtZXNzYWdlIHRoYXQgaXMgcmVxdWly ZWQgb25seSBmb3Igb25lIHRyYW5zcG9ydCB1bmRlcm1pbmVzIHRoZQogICBhcHBsaWNhdGlvbidz IGdvYWwgb2YgYmVpbmcgdHJhbnNwb3J0IG5ldXRyYWwuCgo0LjMuMi4gIFdvcmsvQ29tcGxldGlv biBRdWV1ZSBBY2NvdW50aW5nCgogICBSRE1BIGxvY2FsIEFQSXMgY29udmVudGlvbmFsbHkgdXNl IHdvcmsgcXVldWVzIHRvIHN1Ym1pdCByZXF1ZXN0cwogICAod29yayBxdWV1ZSBlbGVtZW50cyBv ciBXUUVzKSBhbmQgdG8gYXN5bmNocm9ub3VzbHkgcmVjZWl2ZQogICBjb21wbGV0aW9ucyAoaW4g Y29tcGxldGlvbiBxdWV1ZXMgb3IgQ1FzKS4KCiAgIEVhY2ggd29yayByZXF1ZXN0IGNhbiBnZW5l cmF0ZSBhIGNvbXBsZXRpb24gcXVldWUgZW50cnkgKENRRSkuCiAgIENvbXBsZXRpb25zIGZvciBz dWNjZXNzZnVsIHRyYW5zbWl0IHdvcmsgcmVxdWVzdHMgYXJlIGZyZXF1ZW50bHkKICAgc3VwcHJl c3NlZCwgYnV0IHRoZSBjb21wbGV0aW9uIHF1ZXVlIGNhcGFjaXR5IG11c3QgYWNjb3VudCBmb3Ig dGhlCiAgIHBvc3NpYmlsaXR5IHRoYXQgZWFjaCB3aWxsIGNvbXBsZXRlIGluIGVycm9yLiAgQSBj b21wbGV0aW9uIHF1ZXVlIGNhbgogICByZWNlaXZlIGNvbXBsZXRpb25zIGZyb20gbXVsdGlwbGUg d29yayBxdWV1ZXMuCgogICBDb21wbGV0aW9uIFF1ZXVlcyBhcmUgZGVmaW5lZCBzbyBhcyB0byBh bGxvdyBoYXJkd2FyZSBSRE1BCiAgIGltcGxlbWVudGF0aW9ucyB0byBnZW5lcmF0ZSBDUUVzIGRp cmVjdGx5IHRvIGEgdXNlci1zcGFjZSBtYXBwZWQKICAgYnVmZmVyLiAgVGhpcyBlbmFibGVzIGEg dXNlci1zcGFjZSBSRE1BIGNvbnN1bWVyIHRvIHJlYXAgY29tcGxldGlvbnMKICAgd2l0aG91dCBy ZXF1aXJpbmcga2VybmVsIGludGVydmVudGlvbi4KCiAgIEEgaGFyZHdhcmUgUkRNQSBpbXBsZW1l bnRhdGlvbiBjYW5ub3QgcmVhc29uYWJseSB3YWl0IGZvciBhbgogICBhdmFpbGFibGUgc2xvdCBp biB0aGUgY29tcGxldGlvbiBxdWV1ZS4gIFRoZSBxdWV1ZSBtdXN0IGJlIHNpemVkIHN1Y2gKICAg dGhhdCBhbiBvdmVyZmxvdyB3aWxsIG5vdCBvY2N1ci4gIFdoZW4gYW4gb3ZlcmZsb3cgZG9lcyBv Y2N1ciBpdCBpcwogICBjb25zaWRlcmVkIGNhdGFzdHJvcGhpYyBhbmQgd2lsbCB0eXBpY2FsbHkg cmVxdWlyZSB0ZWFyaW5nIGRvd24gYWxsCiAgIFJETUEgY29ubmVjdGlvbnMgdXNpbmcgdGhhdCBD US4KCiAgIFRoaXMgc3R5bGUgb2YgaW50ZXJmYWNlIGlzIHZlcnkgZWZmaWNpZW50LCBidXQgcGxh Y2VzIGEgYnVyZGVuIG9uIHRoZQogICBhcHBsaWNhdGlvbiB0byBwcm9wZXJseSBzaXplIGVhY2gg Q29tcGxldGlvbiBRdWV1ZSB0byBtYXRjaCB0aGUgV29yawogICBRdWV1ZXMgdGhhdCBmZWVkIGl0 LgoKCgoKS2FuZXZza3ksIGV0IGFsLiAgICAgICAgIEV4cGlyZXMgT2N0b2JlciA3LCAyMDExICAg ICAgICAgICAgICAgIFtQYWdlIDddCgwKSW50ZXJuZXQtRHJhZnQgICBFbmhhbmNlZCBSRE1BIENv bm5lY3Rpb24gRXN0YWJsaXNobWVudCAgICAgICBBcHJpbCAyMDExCgoKICAgV2hpbGUgdGhlIGZv cm1hdCBvZiBib3RoIFdRRXMgYW5kIENRRXMgaXMgdHJhbnNwb3J0IGFuZCBkZXZpY2UKICAgZGVw ZW5kZW50LCBhIHRyYW5zcG9ydCBuZXV0cmFsIEFQSSBjYW4gZGVhbCB3aXRoIFdRRXMgYW5kIENR RXMgYXMKICAgYWJzdHJhY3QgdHJhbnNwb3J0IGFuZCBkZXZpY2UgbmV1dHJhbCBvYmplY3RzLiAg VGhlcmVmb3JlIHRoZSBudW1iZXIKICAgb2YgV1FFcyBhbmQgQ1FFcyByZXF1aXJlZCBmb3IgYW4g YXBwbGljYXRpb24gY2FuIGJlIHRyYW5zcG9ydCBhbmQKICAgZGV2aWNlIG5ldXRyYWwuCgogICBU aGUgY2FwYWNpdHkgb2YgdGhlIHdvcmsgcXVldWVzIGFuZCBjb21wbGV0aW9uIHF1ZXVlcyBjYW4g YmUKICAgY2FsY3VsYXRlZCBpbiBhbiBhYnN0cmFjdCB0cmFuc3BvcnQvZGV2aWNlIG5ldXRyYWwg ZmFzaGlvbi4gIExvd2VyCiAgIGxheWVycyBvZiB0aGUgcHJvdG9jb2wgc3RhY2sgY2Fubm90IGRp c3J1cHQgdGhlc2UgY2FsY3VsYXRpb25zIGJ5CiAgIGluc2VydGluZyBhIGR1bW15ICJub3AiIFdv cmsgUmVxdWVzdCBhbmQgZmlsdGVyaW5nIG91dCB0aGUgbWF0Y2hpbmcKICAgY29tcGxldGlvbi4g IFRoZSBsb3dlciBsYXllciBkb2VzIG5vdCBrbm93IHRoZSB1c2FnZSBtb2RlbCBvbiB3aGljaAog ICB0aGUgcXVldWUgc2l6ZXMgYXJlIGJ1aWx0LCBub3IgZG9lcyBpdCBrbm93IGhvdyBmcmVxdWVu dGx5IGFuCiAgIGluc2VydGlvbiB3aWxsIGJlIHJlcXVpcmVkLgoKNC4zLjMuICBIb3N0LWJhc2Vk IEltcGxlbWVudGF0aW9uIG9mIE1QQSBGZW5jaW5nCgogICBNYW55IGhhcmR3YXJlIGltcGxlbWVu dGF0aW9ucyBvZiBpV0FSUCB1c2luZyBNUEEvVENQIGRvIG5vdCBoYW5kbGUKICAgdGhlIE1QQSBS ZXF1ZXN0L1JlcGx5IGV4Y2hhbmdlIGluIGhhcmR3YXJlLCByYXRoZXIgdGhleSBhcmUgaGFuZGxl ZAogICBieSB0aGUgaG9zdCBwcm9jZXNzb3IgaW4gc29mdHdhcmUuICBXaXRoIHN1Y2ggZGVzaWdu cyBpdCBpcyBjb21tb24KICAgZm9yIHRoZSBNUEEgRmVuY2luZyB0byBiZSBpbXBsZW1lbnRlZCBp biB0aGUgdXNlci1zcGFjZSBkZXZpY2UtCiAgIHNwZWNpZmljIGxpYnJhcnkgKGNvbW1vbmx5IHJl ZmVycmVkIHRvIGFzIGEgJ1VzZXIgVmVyYnMnIGxpYnJhcnkgb3IKICAgbW9kdWxlKS4KCiAgIFdo ZW4gdGhlIGdlbmVyYXRpb24gYW5kIHJlY2VwdGlvbiBvZiBNUEEgRlVMUERVcyBpcyBhbHJlYWR5 IGRlZGljYXRlZAogICB0byBoYXJkd2FyZSwgYSBXb3JrIENvbXBsZXRpb24gY2FuIG9ubHkgYmUg Z2VuZXJhdGVkIGJ5IGFuIHVudGFnZ2VkCiAgIG1lc3NhZ2UuCgo0LjQuICBTdGFuZGFyZGl6ZWQg UkRNQSBQYXJhbWV0ZXIgTmVnb3RpYXRpb24KCiAgIE1vc3QgUkRNQSBhcHBsaWNhdGlvbnMgYXJl IGRldmVsb3BlZCB1c2luZyBhIHRyYW5zcG9ydCBuZXV0cmFsIEFQSSB0bwogICBhY2Nlc3MgUkRN QSBzZXJ2aWNlcyBiYXNlZCBvbiBhICJxdWV1ZSBwYWlyIiBwYXJhZGlnbSBhcyBvcmlnaW5hbGx5 CiAgIGRlZmluZWQgYnkgdGhlIFZpcnR1YWwgSW50ZXJmYWNlIEFyY2hpdGVjdHVyZSBbVklBXSwg cmVmaW5lZCBieSB0aGUKICAgRGlyZWN0IEFjY2VzcyBQcm9ncmFtbWluZyBMaWJyYXJ5IFtEQVBM XSBhbmQgbW9zdCBjb21tb25seSBkZXBsb3llZAogICB3aXRoIHRoZSBPcGVuRmFicmljcyBBUEkg W09GQV0uCgogICBUaGVzZSB0cmFuc3BvcnQgbmV1dHJhbCBBUElzIHNlZWsgdG8gcHJvdmlkZSBh IGNvbW1vbiBzZXQgb2YgUkRNQQogICBzZXJ2aWNlcyB3aGV0aGVyIHRoZSB1bmRlcmx5aW5nIHRy YW5zcG9ydCBpcywgZm9yIGV4YW1wbGUsIGlXQVJQIG92ZXIKICAgTVBBLCBpV0FSUCBvdmVyIFND VFAgb3IgSW5maW5pQmFuZC4KCiAgIFRoZSBjb21tb24gbW9kZWwgZm9yIGVzdGFibGlzaGluZyBh biBSRE1BIGNvbm5lY3Rpb24gaGFzIHRoZQogICBmb2xsb3dpbmcgc3RlcHM6CgogICBvICBUaGUg cGFzc2l2ZSBzaWRlIFVMUCBsaXN0ZW5zIGZvciBjb25uZWN0aW9uIHJlcXVlc3RzLgoKICAgbyAg VGhlIGFjdGl2ZSBzaWRlIFVMUCBzdWJtaXRzIGEgY29ubmVjdGlvbiByZXF1ZXN0IHVzaW5nIGFu IFJETUEKICAgICAgZW5kcG9pbnQgKCJxdWV1ZSBwYWlyIiksIHRoZSBkZXNpcmVkIGRlc3RpbmF0 aW9uIGFuZCB0aGUKICAgICAgcGFyYW1ldGVycyB0byBiZSB1c2VkIGZvciB0aGUgY29ubmVjdGlv bi4gIFRob3NlIHBhcmFtZXRlcnMKICAgICAgaW5jbHVkZSBib3RoIFJETUEgbGF5ZXIgY2hhcmFj dGVyaXN0aWNzLCBzdWNoIGFzIHRoZSBSRE1BIFJlYWQKCgoKS2FuZXZza3ksIGV0IGFsLiAgICAg ICAgIEV4cGlyZXMgT2N0b2JlciA3LCAyMDExICAgICAgICAgICAgICAgIFtQYWdlIDhdCgwKSW50 ZXJuZXQtRHJhZnQgICBFbmhhbmNlZCBSRE1BIENvbm5lY3Rpb24gRXN0YWJsaXNobWVudCAgICAg ICBBcHJpbCAyMDExCgoKICAgICAgY3JlZGl0cyB0byBiZSBhbGxvd2VkIGFuZCBhcHBsaWNhdGlv biBzcGVjaWZpYyBkYXRhICh0eXBpY2FsbHkKICAgICAgcmVmZXJyZWQgdG8gYXMgInByaXZhdGUg ZGF0YSIpLgoKICAgbyAgVGhlIHBhc3NpdmUgc2lkZSBVTFAgcmVjZWl2ZXMgdGhlIENvbm5lY3Rp b24gUmVxdWVzdCwgd2hpY2gKICAgICAgaW5jbHVkZXMgdGhlIGlkZW50aXR5IG9mIHRoZSBhY3Rp dmUgc2lkZSBhbmQgdGhlIHJlcXVlc3RlZAogICAgICBjb25uZWN0aW9uIGNoYXJhY3RlcmlzdGlj cy4gIFRoZSBwYXNzaXZlIHNpZGUgVUxQIHVzZXMgdGhpcwogICAgICBpbmZvcm1hdGlvbiB0byBk ZWNpZGUgd2hldGhlciB0byBhY2NlcHQgdGhlIGNvbm5lY3Rpb24sIGFuZCBpZiBpdAogICAgICBp cyB0byBiZSBhY2NlcHRlZCwgaG93IHRvIGNyZWF0ZSBhbmQvb3IgY29uZmlndXJlIHRoZSBSRE1B CiAgICAgIGVuZHBvaW50LgoKICAgbyAgSWYgYWNjZXB0aW5nLCB0aGUgcGFzc2l2ZSBzaWRlIFVM UCBzdWJtaXRzIGl0cyBhY2NlcHRhbmNlIG9mIHRoZQogICAgICBDb25uZWN0aW9uIFJlcXVlc3Qu ICBUaGlzIGxvY2FsIGFjY2VwdCBvcGVyYXRpb24gaW5jbHVkZXMgdGhlIFJETUEKICAgICAgZW5k cG9pbnQgdG8gYmUgdXNlZCBhbmQgdGhlIGNvbm5lY3Rpb24gY2hhcmFjdGVyaXN0aWNzIChib3Ro IHRoZQogICAgICBSRE1BIGNvbmZpZ3VyYXRpb24gYW5kIGFueSBhcHBsaWNhdGlvbiBzcGVjaWZp YyBwcml2YXRlIGRhdGEgdG8gYmUKICAgICAgcmV0dXJuZWQpLgoKICAgbyAgVGhlIGFjdGl2ZSBz aWRlIHJlY2VpdmVzIGNvbmZpcm1hdGlvbiB0aGF0IHRoZSBjb25uZWN0aW9uIGhhcyBiZWVuCiAg ICAgIGFjY2VwdGVkLCB3aGF0IHRoZSBjb25maWd1cmVkIGNvbm5lY3Rpb24gY2hhcmFjdGVyaXN0 aWNzIGFyZSwgYW5kCiAgICAgIGFueSBhcHBsaWNhdGlvbiBzdXBwbGllZCBwcml2YXRlIGRhdGEu CgogICBBcyBjdXJyZW50bHkgZGVmaW5lZCwgRERQIGNvbm5lY3Rpb24gZXN0YWJsaXNobWVudCBy ZXF1aXJlcyB0aGUgVUxQCiAgIHRvIGVuY29kZSB0aGUgUkRNQSBjb25maWd1cmF0aW9uIGluIHRo ZSBhcHBsaWNhdGlvbiBzcGVjaWZpYyBwcml2YXRlCiAgIGRhdGEuICBUaGlzIHJlc3VsdHMgdW5k ZXNpcmFibGUgZHVwbGljYXRpb24gb2YgbG9naWMgdG8gY292ZXIgYm90aAogICBJbmZpbmlCYW5k IGFuZCBpV0FSUCwgYW5kIHRvIHNwZWNpZnkgdGhlIGV4dHJhY3Rpb24gb2YgdGhlIFJETUEKICAg Y2hhcmFjdGVyaXN0aWNzIGZyb20gdGhlIFVMUCBmb3IgZWFjaCBzcGVjaWZpYyBVcHBlciBMYXll ciBQcm90b2NvbC4KCiAgIEEgc3RhbmRhcmQgZGVmaW5pdGlvbiBvZiB0aGUgUkRNQSBjaGFyYWN0 ZXJpc3RpY3Mgd2l0aGluIHRoZSBwcml2YXRlCiAgIGRhdGEgc2VjdGlvbiB3b3VsZCBlbmFibGUg Y29tbW9uIGNvbm5lY3Rpb24gZXN0YWJsaXNobWVudCBBUElzIHRvCiAgIGZvcm1hdCB0aGUgUkRN QSBjaGFyYWN0ZXJpc3RpY3MgYmFzZWQgb24gdGhlIHNhbWUgQVBJIGluZm9ybWF0aW9uCiAgIHVz ZWQgd2hlbiBlc3RhYmxpc2hpbmcgYW4gSW5maW5pQmFuZCBjb25uZWN0aW9uLiAgVGhlIGFwcGxp Y2F0aW9uCiAgIHdvdWxkIHRoZW4gb25seSBoYXZlIHRvIGluZGljYXRlIHRoYXQgaXQgd2FzIHVz aW5nIHRoaXMgc3RhbmRhcmQKICAgZm9ybWF0IHRvIGVuYWJsZSBjb21tb24gY29ubmVjdGlvbiBl c3RhYmxpc2htZW50IHByb2NlZHVyZXMgdG8gYXBwbHkKICAgY29tbW9uIGNvZGUgdG8gcHJvcGVy bHkgcGFyc2UgdGhlc2UgZmllbGRzIGFuZCBjb25maWd1cmUgdGhlIFJETUEKICAgZW5kcG9pbnRz IGFjY29yZGluZ2x5LgoKCjUuICBNUEEgQ29ubmVjdGlvbiBTZXR1cAoKICAgQmVsb3cgd2UgcHJv dmlkZSBvdmVydmlldyBvZiBFbmhhbmNlZCBDb25uZWN0aW9uIFNldHVwLiAgVGhlIGdvYWwgaXMK ICAgdG8gYWxsb3cgc3RhbmRhcmQgbmVnb3RpYXRpb24gb2YgT1JEL0lSRCBzZXR0aW5nIG9uIGJv dGggc2lkZXMgb2YgdGhlCiAgIFJETUEgY29ubmVjdGlvbiBhbmQvb3IgdG8gbmVnb3RpYXRlIHRo ZSBpbml0aWFsIGRhdGEgdHJhbnNmZXIKICAgb3BlcmF0aW9uIGJ5IGFuIGluaXRpYXRvciB3aGVu IHRoZSBleGlzdGluZyAnY2xpZW50IHNlbmRzIGZpcnN0JyBydWxlCiAgIGRvZXMgbm90IG1hdGNo IGFwcGxpY2F0aW9uIHJlcXVpcmVtZW50cy4KCiAgIFRoZSBSRE1BIGNvbm5lY3Rpb24gaW5pdGlh dG9yIHNlbmRzIGFuIE1QQSBSZXF1ZXN0LCBhcyBzcGVjaWZpZWQgaW4KICAgW1JGQzUwNDRdOyB0 aGUgbmV3IGZvcm1hdCBkZWZpbmVkIGhlcmUgYWxsb3dzIGZvcjoKCgoKCgpLYW5ldnNreSwgZXQg YWwuICAgICAgICAgRXhwaXJlcyBPY3RvYmVyIDcsIDIwMTEgICAgICAgICAgICAgICAgW1BhZ2Ug OV0KDApJbnRlcm5ldC1EcmFmdCAgIEVuaGFuY2VkIFJETUEgQ29ubmVjdGlvbiBFc3RhYmxpc2ht ZW50ICAgICAgIEFwcmlsIDIwMTEKCgogICBvICBTdGFuZGFyZGl6ZWQgbmVnb3RpYXRpb24gb2Yg T1JEIGFuZCBJUkQuCgogICBvICBOZWdvdGlhdGlvbiBvZiBhbiBSVFIgbWVzc2FnZS4KCiAgIFRo ZSBSRE1BIGNvbm5lY3Rpb24gcmVzcG9uZGVyIHByb2Nlc3NlcyB0aGUgTVBBIFJlcXVlc3QgYW5k IGdlbmVyYXRlcwogICBhbiBNUEEgUmVwbHksIGFzIHNwZWNpZmllZCBpbiBbUkZDNTA0NF07IHRo ZSBuZXcgZm9ybWF0IGNvbXBsZXRlcyB0aGUKICAgbmVnb3RpYXRpb24uCgogICBUaGUgbG9jYWwg aW50ZXJmYWNlIFNIT1VMRCByZXF1aXJlIHRoZSBVTFAgdG8gZXhwbGljaXRseSBjb25maWd1cmUg b24KICAgYSBwZXItYXBwbGljYXRpb24gb3IgcGVyLWNvbm5lY3Rpb24gYmFzaXMgd2hlbiBhbiBl eHBsaWNpdCBSVFIKICAgbWVzc2FnZSB3aWxsIGJlIHJlcXVpcmVkLiAgUGlnZ3ktYmFja2luZyB0 aGUgUlRSIG9uIGEgQ2xpZW50J3MgZmlyc3QKICAgbWVzc2FnZSBpcyBhIHZhbHVhYmxlIG9wdGlt aXphdGlvbiBmb3IgbW9zdCBjb25uZWN0aW9ucy4KCiAgIFRoZSBSRE1BIGNvbm5lY3Rpb24gaW5p dGlhdG9yIE1VU1QgTk9UIGFsbG93IGFueSBsYXRlciBGVUxQRFVzIHRvIGJlCiAgIHRyYW5zbWl0 dGVkIGJlZm9yZSB0aGUgUlRSIG1lc3NhZ2UuICBPbmUgbWV0aG9kIHRvIGFjaGlldmUgdGhhdCBp cyB0bwogICBkZWxheSBub3RpZnlpbmcgdGhlIFVMUCB0aGF0IHRoZSBSRE1BIGNvbm5lY3Rpb24g aGFzIGJlZW4gZXN0YWJsaXNoZWQKICAgdW50aWwgYWZ0ZXIgYW55IHJlcXVpcmVkIFJUUiBNZXNz YWdlIGhhcyBiZWVuIHRyYW5zbWl0dGVkLgoKICAgQWxsIE1QQSBleGNoYW5nZXMgYXJlIHBlcmZv cm1lZCB2aWEgVENQIHByaW9yIHRvIFJETUEgZXN0YWJsaXNobWVudCwKICAgYW5kIGFyZSB0aGVy ZWZvcmUgc2lnbmFsZWQgdmlhIFRDUCBhbmQgbm90IHZpYSBSRE1BIGNvbXBsZXRpb24uCgoKNi4g IEVuaGFuY2VkIE1QQSBSZXF1ZXN0L1JlcGx5IEZyYW1lcwoKICAgRW5oYW5jZWQgUkRNQSBDb25u ZWN0aW9uIEVzdGFibGlzaG1lbnQgdXNlcyBhbiBhbHRlcm5hdGUgZm9ybWF0IGZvcgogICBNUEEg UmVxdWVzdHMgYW5kIFJlcGxpZXMsIGFzIGZvbGxvd3M6CgogICAgICAgMCAgICAgICAgICAgICAg ICAgICAxICAgICAgICAgICAgICAgICAgIDIgICAgICAgICAgICAgICAgICAgMwogICAgICAgIDAg MSAyIDMgNCA1IDYgNyA4IDkgMCAxIDIgMyA0IDUgNiA3IDggOSAwIDEgMiAzIDQgNSA2IDcgOCA5 IDAgMQogICAgICAgKy0rLSstKy0rLSstKy0rLSstKy0rLSstKy0rLSstKy0rLSstKy0rLSstKy0r LSstKy0rLSstKy0rLSstKy0rLSsKICAgIDAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAg ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB8CiAgICAgICArICAgICAgICAgS2V5 ICgxNiBieXRlcyBjb250YWluaW5nICJNUEEgSUQgUmVxIEZyYW1lIikgICAgICAgICAgKwogICAg NCAgfCAgICAgICg0RCA1MCA0MSAyMCA0OSA0NCAyMCA1MiA2NSA3MSAyMCA0NiA3MiA2MSA2RCA2 NSkgICAgICAgIHwKICAgICAgICsgICAgICAgICBPciAgKDE2IGJ5dGVzIGNvbnRhaW5pbmcgIk1Q QSBJRCBSZXAgRnJhbWUiKSAgICAgICAgICArCiAgICA4ICB8ICAgICAgKDREIDUwIDQxIDIwIDQ5 IDQ0IDIwIDUyIDY1IDcwIDIwIDQ2IDcyIDYxIDZEIDY1KSAgICAgICAgfAogICAgICAgKyAgICAg ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg ICsKICAgIDEyIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg ICAgICAgICAgICAgICAgICB8CiAgICAgICArLSstKy0rLSstKy0rLSstKy0rLSstKy0rLSstKy0r LSstKy0rLSstKy0rLSstKy0rLSstKy0rLSstKy0rLSstKwogICAgMTYgfE18Q3xSfFN8IFJlcyAg IHwgICAgIFJldiAgICAgICB8ICAgICAgICAgIFBEX0xlbmd0aCAgICAgICAgICAgIHwKICAgICAg ICstKy0rLSstKy0rLSstKy0rLSstKy0rLSstKy0rLSstKy0rLSstKy0rLSstKy0rLSstKy0rLSst Ky0rLSstKy0rCiAgICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg ICAgICAgICAgICAgICAgICAgICAgICAgfAogICAgICAgfiAgICAgICAgICAgICAgICAgICAgICAg ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH4KICAgICAgIH4gICAgICAg ICAgICAgICAgICAgUHJpdmF0ZSBEYXRhICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+ CiAgICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg ICAgICAgICAgICAgICAgfAogICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAr LSstKy0rLSstKy0rLSstKy0rLSstKy0rLSstKy0rLSsKICAgICAgIHwgICAgICAgICAgICAgICAg ICAgICAgICAgICAgICAgfAogICAgICAgKy0rLSstKy0rLSstKy0rLSstKy0rLSstKy0rLSstKy0r CgoKCgpLYW5ldnNreSwgZXQgYWwuICAgICAgICAgRXhwaXJlcyBPY3RvYmVyIDcsIDIwMTEgICAg ICAgICAgICAgICBbUGFnZSAxMF0KDApJbnRlcm5ldC1EcmFmdCAgIEVuaGFuY2VkIFJETUEgQ29u bmVjdGlvbiBFc3RhYmxpc2htZW50ICAgICAgIEFwcmlsIDIwMTEKCgogICBLZXk6ICBVbmNoYW5n ZWQgZnJvbSBbUkZDNTA0NF0uCgogICBNOiBVbmNoYW5nZWQgZnJvbSBbUkZDNTA0NF0uCgogICBD OiBVbmNoYW5nZWQgZnJvbSBbUkZDNTA0NF0uCgogICBSOiBVbmNoYW5nZWQgZnJvbSBbUkZDNTA0 NF0uCgogICBTOiBPbmUgaWYgdGhlIFByaXZhdGUgRGF0YSBiZWdpbnMgd2l0aCB0aGUgRW5oYW5j ZWQgUkRNQSBDb25uZWN0aW9uCiAgICAgIEVzdGFibGlzaG1lbnQgRGF0YS4gIFplcm8gb3RoZXJ3 aXNlLgoKICAgUmVzOiAgT25lIGJpdCBzbWFsbGVyIHRoYW4gaW4gW1JGQzUwNDRdLCBvdGhlcndp c2UgdW5jaGFuZ2VkLgoKICAgUmV2OiAgVGhpcyBmaWVsZCBjb250YWlucyB0aGUgcmV2aXNpb24g b2YgTVBBLiAgVG8gdXNlIGFueSBFbmhhbmNlZAogICAgICBDb25uZWN0aW9uIEVzdGFibGlzaG1l bnQgZmVhdHVyZSB0aGlzIE1VU1QgYmUgc2V0IHRvIHR3bywgSWYgbm8KICAgICAgRW5oYW5jZWQg Q29ubmVjdGlvbiBFc3RhYmxpc2htZW50IGZlYXR1cmVzIGFyZSBkZXNpcmVkIGl0IE1BWSBiZQog ICAgICBzZXQgdG8gb25lLiAgQSBob3N0IGFjY2VwdGluZyBNUEEgY29ubmVjdGlvbnMgTVVTVCBj b250aW51ZSB0bwogICAgICBhY2NlcHQgTVBBIFJlcXVlc3RzIHdpdGggdmVyc2lvbiBvbmUgZXZl biBpZiBpdCBzdXBwb3J0cyB2ZXJzaW9uCiAgICAgIHR3by4KCiAgIFBEX0xlbmd0aDogIFVuY2hh bmdlZCBmcm9tIFtSRkM1MDQ0XS4gIFRoaXMgaXMgdGhlIHRvdGFsIGxlbmd0aCBvZgogICAgICB0 aGUgUHJpdmF0ZSBEYXRhIGZpZWxkLCBpbmNsdWRpbmcgdGhlIEVuaGFuY2VkIFJETUEgQ29ubmVj dGlvbgogICAgICBFc3RhYmxpc2htZW50IERhdGEgaWYgcHJlc2VudC4KCiAgIFByaXZhdGUgRGF0 YTogIFVuY2hhbmdlZCBmcm9tIFtSRkM1MDQ0XS4gIEhvd2V2ZXIsIGlmIHRoZSAnUycgZmxhZyBp cwogICAgICBzZXQsIFByaXZhdGUgRGF0YSBiZWdpbnMgd2l0aCBFbmhhbmNlZCBSRE1BIENvbm5l Y3Rpb24KICAgICAgRXN0YWJsaXNobWVudCBEYXRhLgoKCjcuICBFbmhhbmNlZCBTQ1RQIFNlc3Np b24gQ29udHJvbCBDaHVua3MKCiAgIFRoZSB0eXBlIG9mIHRoZSBTQ1RQIFNlc3Npb24gQ29udHJv bCBDaHVuayBpcyBkZWZpbmVkIGJ5IGEgRnVuY3Rpb24KICAgQ29kZS4gIFtSRkM1MDQzXSBhbHJl YWR5IGRlZmluZXMgY29kZXMgZm9yICdERFAgU3RyZWFtIFNlc3Npb24KICAgSW5pdGlhdGUnIGFu ZCAnRERQIFN0cmVhbSBTZXNzaW9uIEFjY2VwdCcsIHdoaWNoIGFyZSBlcXVpdmFsZW50IHRvIGEK ICAgTVBBIFJlcXVlc3QgRnJhbWUgYW5kIGFuIGFjY2VwdGluZyBNUEEgUmVwbHkgRnJhbWUuCgog ICBFbmhhbmNlZCBSRE1BIENvbm5lY3Rpb24gRXN0YWJsaXNobWVudCByZXF1aXJlcyB0aHJlZSBh ZGRpdGlvbmFsCiAgIEZ1bmN0aW9uIGNvZGVzLiAgQWxsIEREUCBTdHJlYW0gU2Vzc2lvbiBGdW5j dGlvbmFsIENvZGVzIGFyZSBsaXN0ZWQKICAgYmVsb3c6CgogICBERFAgU3RyZWFtIFNlc3Npb24g SW5pdGlhdGU6ICAweDAwMQoKICAgRERQIFN0cmVhbSBTZXNzaW9uIEFjY2VwdDogIDB4MDAyCgog ICBERFAgU3RyZWFtIFNlc3Npb24gUmVqZWN0OiAgMHgwMDMKCgoKCgoKS2FuZXZza3ksIGV0IGFs LiAgICAgICAgIEV4cGlyZXMgT2N0b2JlciA3LCAyMDExICAgICAgICAgICAgICAgW1BhZ2UgMTFd CgwKSW50ZXJuZXQtRHJhZnQgICBFbmhhbmNlZCBSRE1BIENvbm5lY3Rpb24gRXN0YWJsaXNobWVu dCAgICAgICBBcHJpbCAyMDExCgoKICAgRERQIFN0cmVhbSBTZXNzaW9uIFRlcm1pbmF0ZTogIDB4 MDA0CgogICBFbmhhbmNlZCBERFAgU3RyZWFtIFNlc3Npb24gSW5pdGlhdGU6ICAweDAwNQoKICAg RW5oYW5jZWQgRERQIFN0cmVhbSBTZXNzaW9uIEFjY2VwdDogIDB4MDA2CgogICBFbmhhbmNlZCBE RFAgU3RyZWFtIFNlc3Npb24gUmVqZWN0OiAgMHgwMDcKCiAgIFRoZSBFbmhhbmNlZCBSZWplY3Qg ZnVuY3Rpb24gY29kZSBTSE9VTEQgYmUgdXNlZCB0byBpbmRpY2F0ZSBhCiAgIGNvbmZpZ3VyYXRp b24gdGhhdCB3b3VsZCBoYXZlIGJlZW4gYWNjZXB0ZWQuCgogICBUaGUgZXh0ZW5kZWQgRERQIHN0 cmVhbSBzZXNzaW9uIGVzdGFibGlzaG1lbnQgZm9sbG93cyB0aGUgc2FtZSBydWxlcwogICBhcyBy ZWd1bGFyIEREUCBzdHJlYW0gc2Vzc2lvbiBlc3RhYmxpc2htZW50IGFzIGRlZmluZWQgaW4gW1JG QzUwNDNdLgogICBVTFAtc3VwcGxpZWQgUHJpdmF0ZSBEYXRhIE1VU1QgYmUgaW5jbHVkZWQgZm9y IGV4dGVuZGVkIEREUCBTdHJlYW0KICAgU2Vzc2lvbiBJbml0aWF0ZSwgZXh0ZW5kZWQgRERQIFN0 cmVhbSBTZXNzaW9uIEFjY2VwdCwgYW5kIGV4dGVuZGVkCiAgIEREUCBTdHJlYW0gU2Vzc2lvbiBS ZWplY3QgbWVzc2FnZXMuICBIb3dldmVyLCB0aGUgVUxQIHN1cHBsaWVkCiAgIFByaXZhdGUgRGF0 YSBNQVkgYmUgb2YgemVybyBsZW5ndGguCgogICBQcml2YXRlIERhdGEgbGVuZ3RoIE1VU1QgTk9U IGV4Y2VlZCA1MTIgYnl0ZXMgaW4gYW55IG1lc3NhZ2UsCiAgIGluY2x1ZGluZyBlbmhhbmNlZCBS RE1BIGNvbm5lY3Rpb24gZXN0YWJsaXNobWVudCBkYXRhLgoKICAgUHJpdmF0ZSBEYXRhIE1VU1Qg Tk9UIGJlIGluY2x1ZGVkIGluIHRoZSBERFAgU3RyZWFtIFNlc3Npb24gVGVybWluYXRlCiAgIG1l c3NhZ2UuCgogICBSZWNlaXZlZCBleHRlbmRlZCBERFAgU3RyZWFtIFNlc3Npb24gQ29udHJvbCBt ZXNzYWdlcyBTSE9VTEQgYmUKICAgcmVwb3J0ZWQgdG8gdGhlIFVMUC4gIElmIHJlcG9ydGVkLCBh bnkgc3VwcGxpZWQgUHJpdmF0ZSBEYXRhIE1VU1QgYmUKICAgYXZhaWxhYmxlIGZvciB0aGUgVUxQ IHRvIGV4YW1pbmUuCgogICBUaGUgZW5oYW5jZWQgRERQIHN0cmVhbSBtYW5hZ2VtZW50IE1VU1Qg dXNlIEREUCBzdHJlYW0gc2Vzc2lvbgogICB0ZXJtaW5hdGlvbiBmdW5jdGlvbiBjb2RlIHRvIHRl cm1pbmF0ZSBhIHN0cmVhbSBlc3RhYmxpc2hlZCB1c2luZwogICBlbmhhbmNlZCBERFAgc3RyZWFt IHNlc3Npb24gZnVuY3Rpb24gY29kZXMuCgogICBJdCBzaG91bGQgYmUgbm90ZWQgdGhhdCBbUkZD NTA0M10gYWxyZWFkeSBzdXBwb3J0cyBlaXRoZXIgc2lkZQogICBzZW5kaW5nIHRoZSBmaXJzdCBE RFAgTWVzc2FnZS4gIFRoZSBQYXlsb2FkIFByb3RvY29sIElkZW50aWZpZXIKICAgKFBQSUQpIGFs cmVhZHkgZGlzdGluZ3Vpc2hlcyBiZXR3ZWVuIFNlc3Npb24gRXN0YWJsaXNobWVudCBhbmQgRERQ CiAgIFNlZ21lbnRzLgoKICAgVGhlIGZvbGxvd2luZyBhZGRpdGlvbmFsIExlZ2FsIFNlcXVlbmNl cyBvZiBERFAgU3RyZWFtIFNlc3Npb24KICAgbWVzc2FnZXMgYXJlIGRlZmluZWQ6CgogICBvICBF eHRlbmRlZCBBY3RpdmUvUGFzc2l2ZSBTZXNzaW9uIEFjY2VwdGVkOiBhcyB3aXRoIHNlY3Rpb24g Ni4yIG9mCiAgICAgIFtSRkM1MDQzXSwgYnV0IHdpdGggdGhlIGV4dGVuZGVkIG9wY29kZXMgYXMg ZGVmaW5lZCBpbiB0aGlzCiAgICAgIGRvY3VtZW50LgoKICAgbyAgRXh0ZW5kZWQgQWN0aXZlL1Bh c3NpdmUgU2Vzc2lvbiBSZWplY3RlZDogYXMgd2l0aCBzZWN0aW9uIDYuMyBvZgogICAgICBbUkZD NTA0M10sIGJ1dCB3aXRoIHRoZSBleHRlbmRlZCBvcGNvZGVzIGFzIGRlZmluZWQgaW4gdGhpcwog ICAgICBkb2N1bWVudC4KCgoKCkthbmV2c2t5LCBldCBhbC4gICAgICAgICBFeHBpcmVzIE9jdG9i ZXIgNywgMjAxMSAgICAgICAgICAgICAgIFtQYWdlIDEyXQoMCkludGVybmV0LURyYWZ0ICAgRW5o YW5jZWQgUkRNQSBDb25uZWN0aW9uIEVzdGFibGlzaG1lbnQgICAgICAgQXByaWwgMjAxMQoKCiAg IG8gIEV4dGVuZGVkIEFjdGl2ZS9QYXNzaXZlIFNlc3Npb24gTm9uLVVMUCBSZWplY3RlZDogYXMg d2l0aCBzZWN0aW9uCiAgICAgIDYuNCBvZiBbUkZDNTA0M10sIGJ1dCB3aXRoIHRoZSBleHRlbmRl ZCBvcGNvZGVzIGFzIGRlZmluZWQgaW4gdGhpcwogICAgICBkb2N1bWVudC4KCgo4LiAgTVBBIEVy cm9yIFJlcG9ydGluZwoKICAgVGhlIFtSRkM1MDQzXSBhbmQgW1JGQzUwNDRdIGRvIG5vdCBkZWZp bmUgZXJyb3IgY29kZXMuICBUaGUgcHJvdG9jb2wKICAgbGF5ZXJzIG9uIHdoaWNoIFJETUEgY29u bmVjdGlvbiBlc3RhYmxpc2htZW50IGlzIGxheWVyZWQgdXBvbgogICBbUkZDNTA0MF0gYW5kIFtS RkM1MDQxXSBkZWZpbmUgbGF5ZXJzIGFuZCBlcnJvciB0eXBlcy4gIFNwZWNpZmljYWxseSwKICAg TVBBIG5lZ290aWF0aW9uIGZvciBSRE1BIGNvbm5lY3Rpb24gZXN0YWJsaXNobWVudCB1c2VzOgoK ICAgTGF5ZXI6ICAgICAgMDAxMGIgLSBMTFAvTVBBCiAgIEVycm9yIFR5cGU6IDB4MyAgIC0gTExQ CgogICBUaGUgZm9sbG93aW5nIGVycm9yIGNvZGVzIGFyZSBkZWZpbmVkIGZvciBNUEEgbmVnb3Rp YXRpb246CgogICBFcnJvciBDb2RlICAgICAgICAgRGVzY3JpcHRpb24KICAgLS0tLS0tLS0tLS0t LS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0KICAgMHgxICAgICAg ICAgICAgICAgIExvY2FsIENhdGFzdHJvcGhpYwogICAweDIgICAgICAgICAgICAgICAgSW5zdWZm aWNpZW50IElSRCByZXNvdXJjZXMKCgo5LiAgRW5oYW5jZWQgUkRNQSBDb25uZWN0aW9uIEVzdGFi bGlzaG1lbnQgRGF0YQoKICAgICAgIDAgICAgICAgICAgICAgICAgICAgMSAgICAgICAgICAgICAg ICAgICAyICAgICAgICAgICAgICAgICAgIDMKICAgICAgICAwIDEgMiAzIDQgNSA2IDcgOCA5IDAg MSAyIDMgNCA1IDYgNyA4IDkgMCAxIDIgMyA0IDUgNiA3IDggOSAwIDEKICAgICAgICstKy0rLSst Ky0rLSstKy0rLSstKy0rLSstKy0rLSstKy0rLSstKy0rLSstKy0rLSstKy0rLSstKy0rLSstKy0r CiAgICAwICB8QXxCfCAgICAgICAgSVJEICAgICAgICAgICAgICAgIHxDfER8ICAgICAgICBPUkQg ICAgICAgICAgICAgICAgfAogICAgNCAgKy0rLSstKy0rLSstKy0rLSstKy0rLSstKy0rLSstKy0r LSstKy0rLSstKy0rLSstKy0rLSstKy0rLSstKy0rLSsKCgoKICAgSVJEOiAgSW4gcmVxdWVzdDog dGhlIEluaXRpYXRvciBpbml0aWFsIHJlc3BvbmRlciBJUkQgZm9yIHRoZQogICAgICBjb25uZWN0 aW9uLiAgSW4gcmVwbHk6IHRoZSBkZXB0aCB0aGUgUmVzcG9uZGVyIHdpbGwgc3VwcG9ydC4KICAg ICAgUmVzcG9uZGVyIFNIT1VMRCBOT1Qgc2V0IGl0cyBJUkQgaGlnaGVyIHRoYW4gSW5pdGlhdG9y IE9SRC4gIFVwb24KICAgICAgcmVjZWl2aW5nIGFjY2VwdCBjb25uZWN0aW9uIG1lc3NhZ2UgZnJv bSB0aGUgUmVzcG9uZGVyLCB0aGUKICAgICAgSW5pdGlhdG9yIE1VU1Qgc2V0IGl0cyBPUkQgdG8g dGhlIHJlc3BvbmRlciBJUkQuICBCb3RoIFJlc3BvbmRlcgogICAgICBhbmQgSW5pdGlhdG9yIE1V U1QgcGFzcyB0aGUgcmVtb3RlIHNpZGUgcHJvdmlkZWQgSVJEIHRvIFVMUC4gIEFuCiAgICAgIGFs bCBvbmVzIHZhbHVlICgweDNGRkYpIGluZGljYXRlcyB0aGF0IGF1dG9tYXRpYyBuZWdvdGlhdGlv biBvZgogICAgICB0aGUgSVJEIGlzIG5vdCBkZXNpcmVkLCBhbmQgdGhhdCB0aGUgVUxQIHdpbGwg YmUgcmVzcG9uc2libGUgZm9yCiAgICAgIGRvaW5nIHRoaXMgY29uZmlndXJhdGlvbi4KCiAgIE9S RDogIEluIHJlcXVlc3Q6IHRoZSBJbml0aWF0b3IgaW5pdGlhbCBPUkQgc2V0dGluZyBmb3IgdGhl CiAgICAgIGNvbm5lY3Rpb24uICBJbiByZXBseTogdGhlIGRlcHRoIHRoZSBSZXNwb25kZXIgd2ls bCBzdXBwb3J0LgogICAgICBSZXNwb25kZXIgU0hPVUxEIHNldCBpdHMgT1JEIHRvIGEgdmFsdWUg dGhhdCBpcyBsZXNzIHRoYW4gb3IgZXF1YWwKICAgICAgdG8gdGhlIHJlcXVlc3RlZCBJbml0aWF0 b3IgSVJELiAgVXBvbiByZWNlaXZpbmcgYWNjZXB0IGNvbm5lY3Rpb24KICAgICAgZnJvbSB0aGUg UmVzcG9uZGVyLCB0aGUgSW5pdGlhdG9yIE1VU1Qgc2V0IGl0cyBJUkQgdG8gYSB2YWx1ZSBhdAoK CgpLYW5ldnNreSwgZXQgYWwuICAgICAgICAgRXhwaXJlcyBPY3RvYmVyIDcsIDIwMTEgICAgICAg ICAgICAgICBbUGFnZSAxM10KDApJbnRlcm5ldC1EcmFmdCAgIEVuaGFuY2VkIFJETUEgQ29ubmVj dGlvbiBFc3RhYmxpc2htZW50ICAgICAgIEFwcmlsIDIwMTEKCgogICAgICBsZWFzdCBhcyBsYXJn ZSBhcyB0aGUgcmVzcG9uZGVyIE9SRCBpZiBpdCBoYXMgc3VmZmljaWVudCByZXNvdXJjZXMKICAg ICAgZm9yIGl0LiAgSWYgdGhlIEluaXRpYXRvciBkb2VzIG5vdCBoYXZlIHN1ZmZpY2llbnQgcmVz b3VyY2VzIHRvCiAgICAgIHNhdGlzZnkgdGhlIFJlc3BvbmRlciBPUkQgaXQgTVVTVCB0ZXJtaW5h dGUgdGhlIGNvbm5lY3Rpb24gYW5kCiAgICAgIHJlcG9ydCB0byBsb2NhbCBVTFAgdGhlIFJlc3Bv bmRlciBPUkQgYW5kIElSRCB3aXRoIGFuIGluZGljYXRvciBvZgogICAgICBpbnN1ZmZpY2llbnQg cmVzb3VyY2VzIHRvIHNhdGlzZnkgUmVzcG9uZGVyIE9SRCwgYW5kIE1VU1Qgc2VuZAogICAgICB0 ZXJtaW5hdGlvbiBtZXNzYWdlIHRvIHRoZSBSZXNwb25kZXIgaW5kaWNhdGluZyBpbnN1ZmZpY2ll bnQKICAgICAgcmVzb3VyY2VzLiAgVGh1cywgdGhlIFRFUk0gbWVzc2FnZSBNVVNUIGNvbnRhaW4g TGF5ZXIgMiwgRXJyb3IKICAgICAgVHlwZSAzLCBFcnJvciBDb2RlIDIuICBCb3RoIFJlc3BvbmRl ciBhbmQgSW5pdGlhdG9yIE1VU1QgcGFzcyB0aGUKICAgICAgcmVtb3RlIHNpZGUgcHJvdmlkZWQg T1JEIHRvIFVMUC4gIEFuIGFsbCBvbmVzIHZhbHVlICgweDNGRkYpCiAgICAgIGluZGljYXRlcyB0 aGF0IGF1dG9tYXRpYyBuZWdvdGlhdGlvbiBvZiB0aGUgT1JEIGlzIG5vdCBkZXNpcmVkLAogICAg ICBhbmQgdGhhdCB0aGUgVUxQIHdpbGwgYmUgcmVzcG9uc2libGUgZm9yIGRvaW5nIHRoaXMgY29u ZmlndXJhdGlvbi4KCiAgIEE6IENvbnRyb2wgRmxhZyBmb3IgdXNpbmcgYSB6ZXJvIGxlbmd0aCBG VUxQRFUgYXMgdGhlIFJUUiBtZXNzYWdlLgoKICAgQjogQ29udHJvbCBGbGFnIGZvciB1c2luZyBh IHplcm8gbGVuZ3RoIFJETUEgV3JpdGUgYXMgdGhlIFJUUgogICAgICBtZXNzYWdlLgoKICAgQzog Q29udHJvbCBGbGFnIGZvciB1c2luZyBhIHplcm8gbGVuZ3RoIFJETUEgUmVhZCBhcyB0aGUgUlRS IG1lc3NhZ2UuCgogICBEOiBSZXNlcnZlZC4gIE11c3QgYmUgc2VudCBhcyB6ZXJvIGFuZCBpZ25v cmVkIHdoZW4gcmVjZWl2ZWQuCgogICBJbiB0aGUgTVBBIFJlcXVlc3QsIHRoZSBJbml0aWF0b3Ig U0hPVUxEIHNldCB0aGUgQSwgQiBhbmQgQyBDb250cm9sCiAgIEZsYWdzIHJlc3BlY3RpdmVseSB0 byBUUlVFIGZvciBlYWNoIG9mIHRoZSBvcHRpb25zIGl0IHN1cHBvcnRzLgoKICAgSW4gdGhlIE1Q QSBSZXBseSwgdGhlIENvbnRyb2wgRmxhZyBpcyBzZXQgZm9yIHRoZSBzZXQgb2Ygb3B0aW9ucyB0 aGF0CiAgIHRoZSBwYXNzaXZlIHNpZGUgd2lsbCBhY2NlcHQgYXMgYW4gUlRSIG1lc3NhZ2UuICBU aGlzIHJlc3BvbnNlIE1VU1QKICAgaW5jbHVkZSBhbGwgb3B0aW9ucyB0aGF0IHRoZSByZXNwb25k ZXIgd2lsbCBzdXBwb3J0IHdpdGhvdXQgcmVxdWlyaW5nCiAgIGEgY29ubmVjdGlvbiBzcGVjaWZp YyBlbmFibGluZyBhY3Rpb24uICBGb3IgZXhhbXBsZSwgaWYgdGhlIHJlc3BvbmRlcgogICB3aWxs IGFsd2F5cyB1bmJsb2NrIGFuIE1QQSBjb25uZWN0aW9uIHdoZW4gaXQgcmVjZWl2ZXMgYSB6ZXJv IGxlbmd0aAogICBNUEEgV3JpdGUsIGl0IHNob3VsZCBpbmRpY2F0ZSBzbyB3aXRob3V0IHJlZ2Fy ZCB0byB3aGF0IHdhcyBpbiB0aGUKICAgTVBBIFJlcXVlc3QuICBPcHRpb25zIHdoaWNoIHJlcXVp cmUgY29ubmVjdGlvbiBzcGVjaWZpYyBlbmFibGluZwogICBhY3Rpb25zIFNIT1VMRCBOT1QgYmUg c2V0IHVubGVzcyB0aGUgY29ycmVzcG9uZGluZyBmbGFnIHdhcyBzZXQgaW4KICAgdGhlIE1QQSBS ZXF1ZXN0LiAgVGhlIHJlc3BvbmRlciBNQVkgY2hvb3NlIHRvIGxpbWl0IHRoZSBudW1iZXIgb2YK ICAgbW9kZXMgdGhhdCBpdCBlbmFibGVzLgoKICAgSWYgdGhlcmUgaXMgbm8gU3RhbmRhcmQgUkRN QVAgQ29uZmlndXJhdGlvbiBEYXRhIGluIHRoZSBNUEEgUmVwbHkKICAgRnJhbWUsIGFuZCB0aGUg RW5oYW5jZWQgQ29ubmVjdGlvbiBFc3RhYmxpc2htZW50IHZlcnNpb24gbnVtYmVyIGlzCiAgIHVz ZWQsIGl0IGlzIHRoZSBlcXVpdmFsZW50IG9mIHNldHRpbmcgJ0EnLCAnQicgYW5kICdDJy4KCiAg IFNldHRpbmcgbm8gQ29udHJvbCBGbGFncyBpbiB0aGUgTVBBIFJlcGx5IGluZGljYXRlcyB0aGF0 IGFuIFJETUEgU2VuZAogICBtZXNzYWdlIHdpbGwgYmUgcmVxdWlyZWQuICBBcyB0aGlzIG9wdGlv biB3aWxsIHJlcXVpcmUgdGhlIGluaXRpYXRvcgogICBVTFAgdG8gYmUgaW52b2x2ZWQgaXQgU0hP VUxEIE5PVCBiZSB1c2VkIHVubGVzcyBuZWNlc3NhcnkuCgogICBUaGUgcGVlci10by1wZWVyIG5l Z290aWF0aW9uIGZvciB0aGUgUlRSIG1lc3NhZ2UgZm9sbG93cyB0aGUKICAgZm9sbG93aW5nIG9y ZGVyOgoKCgoKCgpLYW5ldnNreSwgZXQgYWwuICAgICAgICAgRXhwaXJlcyBPY3RvYmVyIDcsIDIw MTEgICAgICAgICAgICAgICBbUGFnZSAxNF0KDApJbnRlcm5ldC1EcmFmdCAgIEVuaGFuY2VkIFJE TUEgQ29ubmVjdGlvbiBFc3RhYmxpc2htZW50ICAgICAgIEFwcmlsIDIwMTEKCgogICBJbml0aWF0 b3IgLS0+OiBTZXRzIENvbnRyb2wgRmxhZ3MgdG8gaW5kaWNhdGUgSW5pdGlhdG9yLXN1cHBvcnRl ZAogICBmb3JtcyBvZiBSVFIKCiAgIFJlc3BvbmRlciA8LS06IFNldHMgQ29udHJvbCBGbGFncyB0 byBpbmRpY2F0ZSBSZXNwb25kZXItc3VwcG9ydGVkCiAgIGZvcm1zIG9mIFJUUgoKICAgSW5pdGlh dG9yIC0tPjogSWYgYXQgbGVhc3Qgb25lIGZvcm0gb2YgUlRSIGlzIHN1cHBvcnRlZCBieSBib3Ro CiAgIEluaXRpYXRvciBhbmQgUmVzcG9uZGVyLCB0aGVuIHRoZSBmaXJzdCBtZXNzYWdlIHNlbnQg TVVTVCBiZSBhbiBSVFIKICAgdXNpbmcgYSBmb3JtIHN1cHBvcnRlZCBieSBib3RoIHRoZSBJbml0 aWF0b3IgYW5kIFJlc3BvbmRlci4KCiAgIEluaXRpYXRvciBvciBSZXNwb25kZXIgU0hPVUxEIGdl bmVyYXRlIHRoZSBURVJNIG1lc3NhZ2UgdGhhdCBjb250YWlucwogICBMYXllciAyLCBFcnJvciBU eXBlIDMsIEVycm9yIENvZGUgMSB3aGVuIGl0IGVuY291bnRlcnMgYW55IGVycm9yCiAgIGxvY2Fs bHkgZm9yIHdoaWNoIHRoZSBzcGVjaWFsIEVycm9yIENvZGUgaXMgbm90IGRlZmluZWQgaW4gc2Vj dGlvbgogICBTZWN0aW9uIDggYmVmb3JlIHJlc2V0dGluZyB0aGUgY29ubmVjdGlvbi4KCgoxMC4g IEludGVyb3BlcmFiaWxpdHkKCiAgIEFuIGluaXRpYXRvciBTSE9VTEQgTk9UIHVzZSB0aGUgRW5o YW5jZWQgRERQIENvbm5lY3Rpb24gRXN0YWJsaXNobWVudAogICBmb3JtYXRzIG9yIGZ1bmN0aW9u IGNvZGVzIHdoZW4gbm8gZW5oYW5jZWQgZnVuY3Rpb25hbGl0eSBpcyBkZXNpcmVkLgoKICAgQSBy ZXNwb25kZXIgTVVTVCBjb250aW51ZSB0byBhY2NlcHQgdGhlIHVuZW5oYW5jZWQgY29ubmVjdGlv bgogICByZXF1ZXN0cy4KCiAgIFRoZXJlIGFyZSB0aHJlZSBJbml0aWF0b3IvUmVzcG9uZGVyIGNh c2VzIHRoYXQgaW52b2x2ZSBlbmhhbmNlZCBNUEE6CiAgIGJvdGggaW5pdGlhdG9yIGFuZCByZXNw b25kZXIsIG9ubHkgcmVzcG9uZGVyLCBhbmQgb25seSBpbml0aWF0b3IuCiAgIFRoZSBlbmhhbmNl ZCBNUEEgZnJhbWUgaXMgZGVmaW5lZCBieSBmaWVsZCAnUycgc2V0IHRvIDEuCgogICBFbmhhbmNl ZCBNUEEgSW5pdGlhdG9yIGFuZCBSZXNwb25kZXI6ICBJZiBhIHJlc3BvbmRlciByZWNlaXZlcyBh bgogICAgICBlbmhhbmNlZCBNUEEgbWVzc2FnZSwgaXQgTVVTVCByZXNwb25kIHdpdGggYW4gZW5o YW5jZWQgTVBBCiAgICAgIG1lc3NhZ2UuCgogICBFbmhhbmNlZCBNUEEgUmVzcG9uZGVyIG9ubHk6 ICBJZiBhIHJlc3BvbmRlciByZWNlaXZlcyBhbiB1bmVuaGFuY2VkCiAgICAgIE1QQSBtZXNzYWdl ICgnUycgaXMgc2V0IHRvIDApLCBpdCBNVVNUIHJlc3BvbmQgd2l0aCBhbiB1bmVuaGFuY2VkCiAg ICAgIE1QQSBtZXNzYWdlLgoKICAgRW5oYW5jZWQgTVBBIEluaXRpYXRvciBvbmx5OiAgSWYgYSBy ZXNwb25kZXIgZG9lcyBub3Qgc3VwcG9ydAogICAgICByZWNlaXZlZCBleHRlbmRlZCBNUEEgbWVz c2FnZSwgdGhlbiBpdCBNVVNUIGNsb3NlIHRoZSBUQ1AKICAgICAgY29ubmVjdGlvbiBhbmQgZXhp dCBNUEEgc2luY2UgTVBBIGZyYW1lIGlzIGltcHJvcGVybHkgZm9ybWF0dGVkCiAgICAgIGZvciBp dCBhcyBzdGF0ZWQgaW4gW1JGQzUwNDRdLiAgVGh1cywgYm90aCBpbml0aWF0b3IgYW5kIHJlc3Bv bmRlcgogICAgICByZXBvcnQgVENQIGNvbm5lY3Rpb24gdGVybWluYXRpb24gdG8gYW4gYXBwbGlj YXRpb24gbG9jYWxseS4gIEluCiAgICAgIHRoaXMgY2FzZSBpbml0aWF0b3IgTUFZIGF0dGVtcHQg dG8gZXN0YWJsaXNoIFJETUEgY29ubmVjdGlvbiB1c2luZwogICAgICB1bmVuaGFuY2VkIE1QQSBw cm90b2NvbCBhcyBkZWZpbmVkIGluIFtSRkM1MDQ0XSBpZiB0aGlzIHByb3RvY29sCiAgICAgIGlz IGNvbXBhdGlibGUgd2l0aCB0aGUgYXBwbGljYXRpb24sIGFuZCBsZXQgVUxQIGRlYWwgd2l0aCBP UkQgYW5kCiAgICAgIElSRCwgYW5kIHBlZXItdG8tcGVlciBuZWdvdGlhdGlvbnMuCgoKCgoKCkth bmV2c2t5LCBldCBhbC4gICAgICAgICBFeHBpcmVzIE9jdG9iZXIgNywgMjAxMSAgICAgICAgICAg ICAgIFtQYWdlIDE1XQoMCkludGVybmV0LURyYWZ0ICAgRW5oYW5jZWQgUkRNQSBDb25uZWN0aW9u IEVzdGFibGlzaG1lbnQgICAgICAgQXByaWwgMjAxMQoKCjExLiAgSUFOQSBDb25zaWRlcmF0aW9u cwoKICAgVGhpcyBkb2N1bWVudCBoYXMgbm8gSUFOQSBjb25zaWRlcmF0aW9ucy4KCgoxMi4gIFNl Y3VyaXR5IENvbnNpZGVyYXRpb25zCgogICBUaGUgc2VjdXJpdHkgY29uc2lkZXJhdGlvbnMgZnJv bSBSRkMgNTA0NCBhcHBseSBhbmQgdGhlIGNoYW5nZXMgaW4KICAgdGhpcyBkb2N1bWVudCBkbyBu b3QgaW50cm9kdWNlIG5ldyBzZWN1cml0eSBjb25zaWRlcmF0aW9ucy4KCgoxMy4gIEFja25vd2xl ZGdlbWVudHMKCiAgIFRoZSBhdXRob3JzIHdpc2ggdG8gdGhhbmsgU2VhbiBIZWZ0eSwgRGF2ZSBN aW50dXJuLCBUb20gVGFscGV5IGFuZAogICBEYXZpZCBCbGFjayBmb3IgdGhlaXIgdmFsdWFibGUg Y29udHJpYnV0aW9ucyBhbmQgcmV2aWV3cyBvZiB0aGlzCiAgIGRvY3VtZW50LgoKCjE0LiAgUmVm ZXJlbmNlcwoKMTQuMS4gIE5vcm1hdGl2ZSBSZWZlcmVuY2VzCgogICBbUkZDMjExOV0gIEJyYWRu ZXIsIFMuLCAiS2V5IHdvcmRzIGZvciB1c2UgaW4gUkZDcyB0byBJbmRpY2F0ZQogICAgICAgICAg ICAgIFJlcXVpcmVtZW50IExldmVscyIsIEJDUCAxNCwgUkZDIDIxMTksIE1hcmNoIDE5OTcuCgog ICBbUkZDNTA0MF0gIFJlY2lvLCBSLiwgTWV0emxlciwgQi4sIEN1bGxleSwgUC4sIEhpbGxhbmQs IEouLCBhbmQgRC4KICAgICAgICAgICAgICBHYXJjaWEsICJBIFJlbW90ZSBEaXJlY3QgTWVtb3J5 IEFjY2VzcyBQcm90b2NvbAogICAgICAgICAgICAgIFNwZWNpZmljYXRpb24iLCBSRkMgNTA0MCwg T2N0b2JlciAyMDA3LgoKICAgW1JGQzUwNDFdICBTaGFoLCBILiwgUGlua2VydG9uLCBKLiwgUmVj aW8sIFIuLCBhbmQgUC4gQ3VsbGV5LCAiRGlyZWN0CiAgICAgICAgICAgICAgRGF0YSBQbGFjZW1l bnQgb3ZlciBSZWxpYWJsZSBUcmFuc3BvcnRzIiwgUkZDIDUwNDEsCiAgICAgICAgICAgICAgT2N0 b2JlciAyMDA3LgoKICAgW1JGQzUwNDNdICBCZXN0bGVyLCBDLiBhbmQgUi4gU3Rld2FydCwgIlN0 cmVhbSBDb250cm9sIFRyYW5zbWlzc2lvbgogICAgICAgICAgICAgIFByb3RvY29sIChTQ1RQKSBE aXJlY3QgRGF0YSBQbGFjZW1lbnQgKEREUCkgQWRhcHRhdGlvbiIsCiAgICAgICAgICAgICAgUkZD IDUwNDMsIE9jdG9iZXIgMjAwNy4KCiAgIFtSRkM1MDQ0XSAgQ3VsbGV5LCBQLiwgRWx6dXIsIFUu LCBSZWNpbywgUi4sIEJhaWxleSwgUy4sIGFuZCBKLgogICAgICAgICAgICAgIENhcnJpZXIsICJN YXJrZXIgUERVIEFsaWduZWQgRnJhbWluZyBmb3IgVENQCiAgICAgICAgICAgICAgU3BlY2lmaWNh dGlvbiIsIFJGQyA1MDQ0LCBPY3RvYmVyIDIwMDcuCgoxNC4yLiAgSW5mb3JtYXRpdmUgUmVmZXJl bmNlcwoKICAgW0RBUExdICAgICAiRGlyZWN0IEFjY2VzcyBQcm9ncmFtbWluZyBMaWJyYXJ5IiwK ICAgICAgICAgICAgICA8aHR0cDovL3d3dy5kYXRjb2xsYWJvcmF0aXZlLm9yZz4uCgogICBbT0ZB XSAgICAgICJPRkEgdmVyYnMgJiBBUElzIiwgPGh0dHA6Ly93d3cub3BlbmZhYnJpY3Mub3JnLz4u CgoKCgpLYW5ldnNreSwgZXQgYWwuICAgICAgICAgRXhwaXJlcyBPY3RvYmVyIDcsIDIwMTEgICAg ICAgICAgICAgICBbUGFnZSAxNl0KDApJbnRlcm5ldC1EcmFmdCAgIEVuaGFuY2VkIFJETUEgQ29u bmVjdGlvbiBFc3RhYmxpc2htZW50ICAgICAgIEFwcmlsIDIwMTEKCgogICBbT3Blbk1QXSAgIE1j R3Jhdy1IaWxsLCAiUGFyYWxsZWwgUHJvZ3JhbW1pbmcgaW4gQyB3aXRoIE1QSSBhbmQKICAgICAg ICAgICAgICBPcGVuTVAiLCAyMDAzLgoKICAgW1BQTVBJXSAgICBNb3JnYW4gS2F1Zm1hbm4gUHVi bGlzaGVycyBJbmMuLCAiUGFyYWxsZWwgUHJvZ3JhbW1pbmcKICAgICAgICAgICAgICB3aXRoIE1Q SSIsIDIwMDguCgogICBbUkRNQUNdICAgICJSRE1BIFByb3RvY29sIFZlcmJzIFNwZWNpZmljYXRp b24gKFZlcnNpb24gMS4wKSIsIDxodHRwOi8KICAgICAgICAgICAgICAvd3d3LnJkbWFjb25zb3J0 aXVtLm9yZy9ob21lLwogICAgICAgICAgICAgIGRyYWZ0LWhpbGxhbmQtaXdhcnAtdmVyYnMtdjEu MC1SRE1BLnBkZj4uCgogICBbUkRTXSAgICAgIE9wZW4gRmFicmljcyBBc3NvY2lhdGlvbiwgIlJl bGlhYmxlIERhdGFncmFtIFNvY2tldCIsCiAgICAgICAgICAgICAgMjAwOCwgPGh0dHA6Ly93d3cu b3BlbmZhYnJpY3Mub3JnL2FyY2hpdmVzLwogICAgICAgICAgICAgIHNwcmluZzIwMDhzb25vbWEv VHVlc2RheS9zb25vbWFfMjAwOF8wNDA4JTIwT3JhY2xlLnBwdD4uCgogICBbVXNpbmdNUEldCiAg ICAgICAgICAgICAgTUlUIFByZXNzLCAiVXNpbmcgTVBJLTI6IEFkdmFuY2VkIEZlYXR1cmVzIG9m IHRoZSBNZXNzYWdlCiAgICAgICAgICAgICAgUGFzc2luZyBJbnRlcmZhY2UiLCAxOTk5LgoKICAg W1ZJQV0gICAgICBDb21wYXEsIEludGVsLCBNaWNyb3NvZnQsICJWaXJ0dWFsIEludGVyZmFjZSBB cmNoaXRlY3R1cmUKICAgICAgICAgICAgICBTcGVjaWZpY2F0aW9uIiwgMTk5NywgPGh0dHA6Ly9w bGxhYi5jcy5udGh1LmVkdS50dy9jczU0MDMvCiAgICAgICAgICAgICAgUmVhZGluZ3MvRUpCL3Nh bl8xMC5wZGY+LgoKCkF1dGhvcnMnIEFkZHJlc3NlcwoKICAgQXJrYWR5IEthbmV2c2t5IChlZGl0 b3IpCiAgIFZNd2FyZQogICA1IENhbWJyaWRnZSBDZW50ZXIKICAgQ2FtYnJpZGdlLCBNQSAgMDIx NDIKICAgVVNBCgogICBQaG9uZTogKzEtNjE3LTUyOC03NzIxCiAgIEVtYWlsOiBhcmthZHlAdm13 YXJlLmNvbQoKCiAgIENhaXRsaW4gQmVzdGxlciAoZWRpdG9yKQogICBDb25zdWx0YW50CiAgIDU1 NSBFIEVsIENhbWlubyBSZWFsICMxMDQKICAgU3Vubnl2YWxlLCBDQSAgOTQwODcKICAgVVNBCgog ICBQaG9uZTogKzEtOTQ5LTUyOC0zMDg1CiAgIEVtYWlsOiBjYWl0QGFzb21pLmNvbQoKCgoKCgoK CkthbmV2c2t5LCBldCBhbC4gICAgICAgICBFeHBpcmVzIE9jdG9iZXIgNywgMjAxMSAgICAgICAg ICAgICAgIFtQYWdlIDE3XQoMCkludGVybmV0LURyYWZ0ICAgRW5oYW5jZWQgUkRNQSBDb25uZWN0 aW9uIEVzdGFibGlzaG1lbnQgICAgICAgQXByaWwgMjAxMQoKCiAgIFJvYmVydCBTaGFycAogICBJ bnRlbAogICBMQUQgSGlnaCBQZXJmb3JtYW5jZSBNZXNzYWdlIFBhc3NpbmcsIE1haWxzdG9wOiBB TjEtV1RSMQogICAxNTAxIFNvdXRoIE1vcGFjLCBTdWl0ZSA0MDAKICAgQXVzdGluLCBUWCAgNzg3 NDYKICAgVVNBCgogICBQaG9uZTogKzEtNTEyLTQ5My0zMjQyCiAgIEVtYWlsOiByb2JlcnQuby5z aGFycEBpbnRlbC5jb20KCgogICBTdGV2ZSBXaXNlCiAgIE9wZW4gR3JpZCBDb21wdXRpbmcKICAg NDAzMCBCcmFrZXIgTGFuZSBTVEUgMTMwCiAgIEF1c3RpbiwgVFggIDc4NzU5CiAgIFVTQQoKICAg UGhvbmU6ICsxLTUxMi0zNDMtOTE5NiB4MTAxCiAgIEVtYWlsOiBzd2lzZUBvcGVuZ3JpZGNvbXB1 dGluZy5jb20KCgoKCgoKCgoKCgoKCgoKCgoKCgoKCgoKCgoKCgoKCgpLYW5ldnNreSwgZXQgYWwu ICAgICAgICAgRXhwaXJlcyBPY3RvYmVyIDcsIDIwMTEgICAgICAgICAgICAgICBbUGFnZSAxOF0K DAo= --000e0cd2dc3ed86f6204a02032b8-- From arkady.kanevsky@gmail.com Mon Apr 4 18:48:33 2011 Return-Path: X-Original-To: storm@core3.amsl.com Delivered-To: storm@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id 2A4BF3A6840 for ; Mon, 4 Apr 2011 18:48:33 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -3.598 X-Spam-Level: X-Spam-Status: No, score=-3.598 tagged_above=-999 required=5 tests=[AWL=-0.000, BAYES_00=-2.599, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_LOW=-1] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id Y5u6pPu5pwU3 for ; Mon, 4 Apr 2011 18:48:28 -0700 (PDT) Received: from mail-pz0-f44.google.com (mail-pz0-f44.google.com [209.85.210.44]) by core3.amsl.com (Postfix) with ESMTP id 807773A6842 for ; Mon, 4 Apr 2011 18:48:28 -0700 (PDT) Received: by pzk30 with SMTP id 30so1890903pzk.31 for ; Mon, 04 Apr 2011 18:50:11 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:in-reply-to:references:date :message-id:subject:from:to:cc:content-type; bh=0TtfF4VlsAn9GD5dDdUrB2eb4py0VTkuHlezDpt3tUE=; b=AbVXhXel5/NF8Y7Xav69HzqNcmtxDaI1mx0ZpulrPNcJo65QvMZPDRXJ28mpuBdk6u 9Q02WfgUeQl9UqHAZ+k6F0dcGXF+gYacKUeH1bbTyF9KyGGYFnZzsdWvbXCaG84d8S26 yqpHTctT1FA/mIJXp0n/NtxXcLt8jTyNfWT9o= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; b=AN6rFav2kRJeLlDRX47ZzEVu8UX0aNoGEmd5NQQ4sgidbM3NGjX9PyZPaDNQKtXh/G QtTty95PHO0rwYgNWvOC4N7WvhDPOf3PzPdiq9egd8KfwRqTp0mYlKDg93TVOxbcGV/G Fh+yjC6oIEoYFLfSMCtadOW4Ukjfho5VYbvXA= MIME-Version: 1.0 Received: by 10.142.247.10 with SMTP id u10mr6685917wfh.414.1301968211098; Mon, 04 Apr 2011 18:50:11 -0700 (PDT) Received: by 10.142.185.20 with HTTP; Mon, 4 Apr 2011 18:50:10 -0700 (PDT) In-Reply-To: <4D9A733B.9050503@opengridcomputing.com> References: <7C4DFCE962635144B8FAE8CA11D0BF1E03D74C7183@MX14A.corp.emc.com> <7C4DFCE962635144B8FAE8CA11D0BF1E03E5E9530C@MX14A.corp.emc.com> <76DBE161893C324BA6D4BB507EE4C3849510935E96@IRVEXCHCCR02.corp.ad.broadcom.com> <4D98E86A.60403@opengridcomputing.com> <4D9A733B.9050503@opengridcomputing.com> Date: Mon, 4 Apr 2011 21:50:10 -0400 Message-ID: From: arkady kanevsky To: Steve Wise Content-Type: multipart/mixed; boundary=00504502cc56bb471204a0221808 X-Mailman-Approved-At: Tue, 05 Apr 2011 08:26:32 -0700 Cc: storm@ietf.org Subject: Re: [storm] MPA Draft - Review X-BeenThere: storm@ietf.org X-Mailman-Version: 2.1.9 Precedence: list List-Id: Storage Maintenance WG List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 05 Apr 2011 01:48:33 -0000 --00504502cc56bb471204a0221808 Content-Type: multipart/alternative; boundary=00504502cc56bb46ed04a0221806 --00504502cc56bb46ed04a0221806 Content-Type: text/plain; charset=ISO-8859-1 Got it. Thanks Steve. How about now? Arkady On Mon, Apr 4, 2011 at 9:41 PM, Steve Wise wrote: > The key word to remove here is "responder". The IRD in the MPA start > request is the initiators desired IRD _of the initiator_. Not the > initiators desired _responder_ IRD. > > If you want to change "desiired" to "initial" thats ok with me. But the > key is to get rid of the word "responder" in that sentence. > > Steve. > > > On 4/4/2011 6:26 PM, arkady kanevsky wrote: > > Steve and Bob, > I changed it to > "In request: the Initiator desired responder IRD > for the connection." as you asked. > I can change it to "initial" instead of "desired". > Arkady > > On Sun, Apr 3, 2011 at 5:36 PM, Steve Wise wrote: > >> Hey Arkady, >> >> It does seem like you did the section 9 changes Bob and I requested: >> >> ---- >> >> Change the IRD definition on the request from "In request: the Initiator >> requested responder IRD for the >> connection." to "In request: the Initiator initial IRD setting for the >> connection." >> ---- >> >> Thanks, >> >> Steve. >> >> >> On 4/2/2011 8:35 PM, arkady kanevsky wrote: >> >> All, >> updated version 04 is attached. >> >> Hemal, >> Thanks for catching it. >> I had fixed the first issue. I had added reference to FPDU in the FULPDU >> definition for the second. >> >> David, >> Please, check to see that you comments are addressed. >> >> Steve and Robert, >> please, check that you comment is fixed correctly. >> >> Once I get positive feedback from all of you, I will submit the version. >> >> Thanks, >> Arkady >> >> On Thu, Mar 31, 2011 at 2:43 PM, Hemal Shah wrote: >> >>> I have some comments on -03 draft: >>> >>> >>> 1. In section 10, it is written that "Enhanced MPA Initiator and >>> Responder: If a responder receives an enhanced MPA message, it MUST respond >>> with an unenhanced MPA message." I think it should be written that the >>> responder must respond with an enhanced MPA message. It appears like a typo >>> to me. >>> 2. I find the use of FULPDU confusing in this draft. RFC5044 does not >>> define term FULPDU. RFC5044 uses term FPDU to refer to Framed Protocol Data >>> Unit. I suggest that we use term FPDU instead of FULPDU in the draft. >>> >>> >>> Hemal >>> >>> -----Original Message----- >>> From: storm-bounces@ietf.org [mailto:storm-bounces@ietf.org] >>> On Behalf Of david.black@emc.com >>> Sent: Thursday, March 31, 2011 7:48 AM >>> To: storm@ietf.org >>> Subject: Re: [storm] MPA Draft - Review >>> Importance: High >>> >>> The -03 version of the MPA draft has addressed all of the issues from my >>> review, and . Unfortunately, I need some minor edits for clarity before I >>> can send this on to our AD with a publication request. Would the authors >>> please submit a -04 version with the following two changes quickly. >>> >>> Section 9 (end) >>> >>> OLD >>> >>> The peer-to-peer negotiation for the RTR message follows the >>> following order: >>> >>> Initiator -->: Sets Control Flags it is capable to send for RTR >>> >>> Responder <--: Sets Control Flags it is capable to receive for RTR >>> >>> Initiator -->: The first message send MUST be a negotiated RTR >>> >>> NEW >>> >>> The peer-to-peer negotiation for the RTR message follows the >>> following order: >>> >>> Initiator -->: Sets Control Flags to indicate Initiator-supported >>> forms of RTR >>> >>> Responder <--: Sets Control Flags to indicate Responder-supported >>> forms of RTR >>> >>> Initiator -->: If at least one form of RTR is supported by both >>> Initiator and >>> Responder, then the first message sent MUST be an RTR using a >>> form supported >>> by both the Initiator and Responder. >>> >>> Section 10 >>> >>> OLD >>> In >>> this case initiator CAN attempt to establish RDMA connection using >>> unenhanced MPA protocol as defined in [RFC5044] and let ULP deal >>> with ORD and IRD, and peer-to-peer negotiations. >>> >>> NEW >>> >>> In >>> this case initiator MAY attempt to establish RDMA connection using >>> ------------------------->^^^ >>> unenhanced MPA protocol as defined in [RFC5044] if this protocol is >>> compatible with the application, and let ULP deal with ORD and >>> IRD, >>> and peer-to-peer negotiations. >>> >>> Ordinarily, I'd write an RFC Editor Note for small changes like these, >>> but they're sufficiently critical to interoperability that I'd prefer to >>> have a new draft version that contains them. >>> >>> Thanks, >>> --David >>> >>> >>> > -----Original Message----- >>> > From: Black, David >>> > Sent: Wednesday, December 22, 2010 9:26 PM >>> > To: storm@ietf.org >>> > Cc: Black, David >>> > Subject: MPA Draft - Review >>> > >>> > WG Last Call on this draft has run its course: >>> > >>> > Enhanced RDMA Connection Establishment >>> > draft-ietf-storm-mpa-peer-connect-02 >>> > >>> > I've done my review as a WG chair (and the person who will be >>> shepherding this draft to the ADs and >>> > IESG): >>> > >>> > - This draft is on the right track, but has open issues. >>> > - Another version of the draft will be needed. >>> > >>> > Also, it would be greatly appreciated if a few people other than the >>> authors could take a look at >>> > this draft. We have a very good author team on this draft, whose >>> expertise is beyond doubt, but >>> > more eyes on this draft would help. >>> > >>> > [1] My primary concern is that Section 9 on interoperability is >>> inadequate: >>> > >>> > An initiator SHOULD NOT use the Enhanced DDP Connection >>> Establishment >>> > formats or function codes when no enhanced functionality is desired. >>> > >>> > A responder SHOULD continue to accept the unenhanced connection >>> > requests. >>> > >>> > The good news is that the first sentence is ok. >>> > The bad news is that the second sentence has significant problems: >>> > - It uses SHOULD instead of MUST. >>> > - It doesn't lay out behavior for initiator and responder >>> > Revision mixes. >>> > IETF interoperability requirements are usually expressed with MUST, >>> including backwards >>> > compatibility. If interop with unenhanced implementations is only a >>> SHOULD, that will need a >>> > convincing explanation. >>> > >>> > There are 3 Initiator/Responder cases that need attention (New/Old, >>> Old/New and New/New). I think >>> > they lead to roughly the following: >>> > >>> > New/Old: >>> > - Explain error or failure that the New Initiator will see because the >>> Old responder >>> > doesn't support Revision 2 of the MPA protocol. >>> > - Explain what the Initiator does when it sees that error or failure. >>> The >>> > easiest approach is to always retry with Revision 1, but that >>> won't work >>> > if the Initiator has to send an RTR (that's the "convincing >>> explanation" >>> > for why backwards compatibility is not always possible). The >>> result >>> > might be two requirements: >>> > - If the Initiator has data to send, it MUST retry with Revision >>> 1. >>> > - If the Initiator has no data to send, and hence has to send an >>> RTR, >>> > the connection setup fails, the TCP connection closes >>> and that >>> > failure MUST to be reported to the application. >>> > >>> > Old/New: >>> > - If a responder receives a Revision 1 message, it MUST respond with a >>> Revision 1 message. >>> > >>> > New/New: >>> > - If a responder receives a Revision 2 message, it MUST respond with a >>> Revision 2 message. >>> > >>> > I found a few other concerns: >>> > >>> > [B]In Section 7, we need to get the listing of all the SCTP function >>> codes into one place. Either >>> > repeat the definitions of codes 1-4 from RFC 5043, or create an IANA >>> registry in Section 10 and list >>> > all 7 codes as its initial contents. >>> > >>> > [C] In Section 8, what happens if the responder sends an IRD or ORD >>> value that's different from the >>> > corresponding initiator value? Is the responder allowed to increase >>> the value that was sent? An >>> > important case to cover is that the initiator sends a valid value >>> (e.g., 0x2000) but the responder >>> > returns the 0x3FFF value indicating that negotiation is not supported. >>> Also, what is the behavior >>> > of an IRD or ORD that is set to 0x0000? >>> > >>> > [D] In contrast, the Section 8 discussion of Control Flag functionality >>> is in better shape. It >>> > would be helpful to add a sentence or two indicating when the RTR >>> occurs (Request ->, <- Reply, RTR >>> > ->), even though that is discussed earlier in the draft. Also, it's >>> necessary to state whether >>> > negotiation of RTR functionality commits the Initiator to using an RTR >>> (e.g., suppose the initiator >>> > negotiates control flags to allow an RTR and instead sends an FULPDU >>> with payload data after >>> > receiving the Reply - is that ok or is it an error?). >>> > >>> > [E] Nit: In the definition of Control Flag A: ULPDU -> FULPDU >>> > >>> > Thanks, >>> > --David >>> > ---------------------------------------------------- >>> > David L. Black, Distinguished Engineer >>> > EMC Corporation, 176 South St., Hopkinton, MA 01748 >>> > +1 (508) 293-7953 FAX: +1 (508) 293-7786 >>> > david.black@emc.com Mobile: +1 (978) 394-7754 >>> > ---------------------------------------------------- >>> >>> _______________________________________________ >>> storm mailing list >>> storm@ietf.org >>> https://www.ietf.org/mailman/listinfo/storm >>> >>> >>> >>> _______________________________________________ >>> storm mailing list >>> storm@ietf.org >>> https://www.ietf.org/mailman/listinfo/storm >>> >>> >> >> >> -- >> Cheers, >> Arkady Kanevsky >> >> >> _______________________________________________ >> storm mailing liststorm@ietf.orghttps://www.ietf.org/mailman/listinfo/storm >> >> >> > > > -- > Cheers, > Arkady Kanevsky > > > -- Cheers, Arkady Kanevsky --00504502cc56bb46ed04a0221806 Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Got it. Thanks Steve.
How about now?
Arkady

On Mon, Apr 4, 2011 at 9:41 PM, Steve Wise <<= a href=3D"mailto:swise@opengridcomputing.com">swise@opengridcomputing.com> wrote:
=20 =20 =20 =20
The key word to remove here is "responder".=A0 The IRD in the= MPA start request is the initiators desired IRD _of the initiator_.=A0 Not the initiators desired _responder_ IRD.

If you want to change "desiired" to "initial" thats= ok with me.=A0 But the key is to get rid of the word "responder" in that sentenc= e.

Steve.


On 4/4/2011 6:26 PM, arkady kanevsky wrote:
Steve and Bob,
I changed it to
"In request: the Initiator desired responder IRD
for the connection." as you asked.
I can change it to "initial" instead of "desired"= .
Arkady

On Sun, Apr 3, 2011 at 5:36 PM, Steve Wise <swise@opengridcomputing.com> wrote:
Hey Arkady,

It does seem like you did the section 9 changes Bob and I requested:

----

=A0 Change the IRD definition on the request from "In request: the Initiator requested responder IRD for the
=A0 connection." to "In request: the Initiator init= ial IRD setting for the connection."=A0
----

Thanks,

Steve.


On 4/2/2011 8:35 PM, arkady kanevsky wrote:
All,
updated version 04 is attached.

Hemal,
Thanks for catching it.
I had fixed the first issue. I had added reference to FPDU in the FULPDU definition for the second.

David,
Please, check to see that you comments are addressed.

Steve and Robert,
please, check that you comment is fixed correctly.

Once I get positive feedback from all of you, I will submit the version.

Thanks,
Arkady

On Thu, Mar 31, 2011 at 2:43 PM, Hemal Shah <hemal@broadcom.com> wrote:
I have some comments on -03 draft:
=A0
  1. In section 10, it is written that "Enhanced MPA Initiator and Responder:= =A0 If a responder receives an enhanced MPA message, it MUST respond with an unenhanced MPA message." I think it shou= ld be written that the responder must respond with an enhanced MPA message. It appears like a typo to me.
  2. I find the use of FULPDU confusing in this draft. RFC5044 does not define term FULPDU. RFC5044 uses term FPDU to refer to Framed Protocol Data Unit. I suggest that we use term FPDU instead of FULPDU in the draft.
=A0
Hemal
=A0
-----Original Message-----
From: storm-bounces@ietf.org [mailto:storm-bounces@ietf.org] On Behalf Of david.black@emc.com
Sent: Thursday, March 31, 2011 7:48 AM
To: storm@ietf.org
Subject: Re: [storm] MPA Draft - Review
Importance: High
=A0
The -03 version of the MPA draft has addressed all of the issues from my review, and .=A0 Unfortunately, I need some minor edits for clarity before I can send this on to our AD with a publication request.=A0 Would the authors please submit a -04 version with the following two changes quickly.
=A0
Section 9 (end)
=A0
OLD
=A0
=A0=A0 The peer-to-peer negotiation for the RTR message follows the
=A0=A0 following order:=A0=A0=A0=A0
=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0= =A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0
=A0=A0 Initiator -->: Sets Control Flags it is capable to send for RTR=A0=A0= =A0=A0=A0
=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0= =A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0
=A0=A0 Responder <--: Sets Control Flags it is capable to receive for RTR=A0= =A0
=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0= =A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0
=A0=A0 Initiator -->: The first message send MUST be a negotiated RTR
=A0
NEW
=A0
=A0=A0 The peer-to-peer negotiation for the RTR message follows the
=A0=A0 following order:=A0=A0=A0=A0
=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0= =A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0
=A0=A0 Initiator -->: Sets Control Flags to indicate Initiator-supported forms of RTR=A0=A0=A0=A0=A0=A0
=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0= =A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0
=A0=A0 Responder <--: Sets Control Flags to indicate Responder-supported forms of RTR=A0=A0=A0=A0=A0=A0
=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0= =A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0
=A0=A0 Initiator -->: If at least one form of RTR is supported by both Initiator and
=A0=A0=A0=A0=A0=A0=A0 Responder, then th= e first message sent MUST be an RTR using a form supported
=A0=A0=A0=A0=A0=A0=A0 by both the Initia= tor and Responder.
=A0
Section 10
=A0
OLD
=A0=A0=A0=A0=A0 In
=A0=A0=A0=A0=A0 this case initiator CAN = attempt to establish RDMA connection using
=A0=A0=A0=A0=A0 unenhanced MPA protocol = as defined in [RFC5044] and let ULP deal
=A0=A0=A0=A0=A0 with ORD and IRD, and peer-to-peer negotiations.
=A0
NEW
=A0
=A0=A0=A0=A0=A0 In
=A0=A0=A0=A0=A0 this case initiator MAY = attempt to establish RDMA connection using
------------------------->^^^
=A0=A0=A0=A0=A0 unenhanced MPA protocol = as defined in [RFC5044] if this protocol is
=A0=A0=A0=A0=A0=A0=A0 compatible with th= e application, and let ULP deal with ORD and IRD,
=A0=A0=A0=A0=A0 and peer-to-peer negotia= tions.
=A0
Ordinarily, I'd write an RFC Editor Note for small changes like these, but they're sufficiently critical to interoperability that I'd prefer to hav= e a new draft version that contains them.
=A0
Thanks,
--David
=A0
=A0
> -----Original Message-----
> From: Black, David
> Sent: Wednesday, December 22, 2010 9:26 PM
> Cc: Black, David
> Subject: MPA Draft - Review
>
> WG Last Call on this draft has run its course:
>
>=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0= =A0=A0=A0=A0=A0 Enhanced RDMA Connection Establishment
>=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0= =A0=A0=A0=A0=A0=A0 draft-ietf-storm-mpa-peer-connect-02
>
> I've done my review as a WG chair (and the person who will be shepherding this draft to the ADs and
> IESG):
>
> - This draft is on the right track, but has open issues.
> - Another version of the draft will be needed.
>
> Also, it would be greatly appreciated if a few people other than the authors could take a look at
> this draft.=A0 We have a very good author team on this draft, whose expertise is beyond doubt, but
> more eyes on this draft would help.
>
> [1] My primary concern is that Section 9 on interoperability is inadequate:
>
>=A0=A0=A0 An initiator SHOULD NOT us= e the Enhanced DDP Connection Establishment
>=A0=A0=A0 formats or function codes when no enhanced functionality is desired.
>
>=A0=A0=A0 A responder SHOULD continu= e to accept the unenhanced connection
>=A0=A0=A0 requests.
>
> The good news is that the first sentence is ok.
> The bad news is that the second sentence has significant problems:
>=A0=A0=A0=A0=A0=A0=A0 - It uses SHOU= LD instead of MUST.
>=A0=A0=A0=A0=A0=A0=A0 - It doesn'= ;t lay out behavior for initiator and responder
>=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0= =A0=A0=A0 Revision mixes.
> IETF interoperability requirements are usually expressed with MUST, including backwards
> compatibility.=A0 If interop with unenhanced implementations is only a SHOULD, that will need a
> convincing explanation.
>
> There are 3 Initiator/Responder cases that need attention (New/Old, Old/New and New/New).=A0 I think
> they lead to roughly the following:
>
> New/Old:
> - Explain error or failure that the New Initiator will see because the Old responder
>=A0=A0=A0=A0=A0=A0=A0 doesn't su= pport Revision 2 of the MPA protocol.
> - Explain what the Initiator does when it sees that error or failure.=A0 The
>=A0=A0=A0=A0=A0=A0=A0 easiest approa= ch is to always retry with Revision 1, but that won't work
>=A0=A0=A0=A0=A0=A0=A0 if the Initiat= or has to send an RTR (that's the "convincin= g explanation"
>=A0=A0=A0=A0=A0=A0=A0 for why backwa= rds compatibility is not always possible).=A0 The result
>=A0=A0=A0=A0=A0=A0=A0 might be two requirements:
>=A0=A0=A0=A0=A0=A0=A0 - If the Initi= ator has data to send, it MUST retry with Revision 1.
>=A0=A0=A0=A0=A0=A0=A0 - If the Initi= ator has no data to send, and hence has to send an RTR,
>=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0= =A0=A0=A0 the connection setup fails, the TCP connection closes and that
>=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0= =A0=A0=A0 failure MUST to be reported to the application.
>
> Old/New:
> - If a responder receives a Revision 1 message, it MUST respond with a Revision 1 message.
>
> New/New:
> - If a responder receives a Revision 2 message, it MUST respond with a Revision 2 message.
>
> I found a few other concerns:
>
> [B]In Section 7, we need to get the listing of all the SCTP function codes into one place.=A0 Either
> repeat the definitions of codes 1-4 from RFC 5043, or create an IANA registry in Section 10 and list
> all 7 codes as its initial contents.
>
> [C] In Section 8, what happens if the responder sends an IRD or ORD value that's different from the
> corresponding initiator value?=A0 Is the responder allowed to increase the value that was sent?=A0 An
> important case to cover is that the initiator sends a valid value (e.g., 0x2000) but the responder
> returns the 0x3FFF value indicating that negotiation is not supported.=A0 Also, what is the behavior
> of an IRD or ORD that is set to 0x0000?
>
> [D] In contrast, the Section 8 discussion of Control Flag functionality is in better shape.=A0 It
> would be helpful to add a sentence or two indicating when the RTR occurs (Request ->, <- Reply, RTR
> ->), even though that is discussed earlier in the draft.=A0 Also, it's necessary to state whether
> negotiation of RTR functionality commits the Initiator to using an RTR (e.g., suppose the initiator
> negotiates control flags to allow an RTR and instead sends an FULPDU with payload data after
> receiving the Reply - is that ok or is it an error?).
>
> [E] Nit: In the definition of Control Flag A: ULPDU -> FULPDU
>
> Thanks,
> --David
> -------------------------------------------= ---------
> David L. Black, Distinguished Engineer
> EMC Corporation, 176 South St., Hopkinton, MA=A0 01748
> +1 (508) 293-7953=A0=A0=A0=A0=A0=A0=A0=A0=A0= =A0=A0=A0 FAX: +1 (508) 293-7786
> david.black@emc.com=A0=A0=A0=A0=A0=A0=A0 Mobile: +1 (978) 394-7754
> -------------------------------------------= ---------
=A0
________________________________________= _______
storm mailing list
=A0
=A0

_______________________________________________
storm mailing list
s= torm@ietf.org
https://www.ietf.org/mailman/listinfo/storm




--
Cheers,
Arkady Kanevsky

_______________________________________________
storm mailing list
storm@ietf.org
h=
ttps://www.ietf.org/mailman/listinfo/storm




--
Cheers,
Arkady Kanevsky




--
Cheers,
Arkady Kanev= sky
--00504502cc56bb46ed04a0221806-- --00504502cc56bb471204a0221808 Content-Type: text/plain; charset=US-ASCII; name="draft-ietf-storm-mpa-peer-connect-04.txt" Content-Disposition: attachment; filename="draft-ietf-storm-mpa-peer-connect-04.txt" Content-Transfer-Encoding: base64 X-Attachment-Id: f_gm465sl00 CgoKU1RPUk0gICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg ICBBLiBLYW5ldnNreSwgRWQuCkludGVybmV0LURyYWZ0ICAgICAgICAgICAgICAgICAgICAgICAg ICAgICAgICAgICAgICAgICAgICAgICAgICAgIFZNd2FyZQpVcGRhdGVzOiA1MDQzLCA1MDQ0IChp ZiBhcHByb3ZlZCkgICAgICAgICAgICAgICAgICAgICAgICBDLiBCZXN0bGVyLCBFZC4KSW50ZW5k ZWQgc3RhdHVzOiBTdGFuZGFyZHMgVHJhY2sgICAgICAgICAgICAgICAgICAgICAgICAgICAgICBD b25zdWx0YW50CkV4cGlyZXM6IE9jdG9iZXIgNywgMjAxMSAgICAgICAgICAgICAgICAgICAgICAg ICAgICAgICAgICAgICAgICBSLiBTaGFycAogICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgSW50ZWwKICAgICAgICAgICAgICAg ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICBTLiBXaXNl CiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgT3Bl biBHcmlkIENvbXB1dGluZwogICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg ICAgICAgICAgICAgICAgICAgIEFwcmlsIDUsIDIwMTEKCgogICAgICAgICAgICAgICAgIEVuaGFu Y2VkIFJETUEgQ29ubmVjdGlvbiBFc3RhYmxpc2htZW50CiAgICAgICAgICAgICAgICAgIGRyYWZ0 LWlldGYtc3Rvcm0tbXBhLXBlZXItY29ubmVjdC0wNAoKQWJzdHJhY3QKCiAgIFRoaXMgZG9jdW1l bnQgdXBkYXRlcyBbUkZDNTA0M10gYW5kIFtSRkM1MDQ0XSBieSBleHRlbmRpbmcgTVBBCiAgIG5l Z290aWF0aW9uIGZvciBSRE1BIENvbm5lY3Rpb24gZXN0YWJsaXNobWVudC4gIFRoZSBmaXJzdCBl eHRlbnNpb24KICAgZXh0ZW5kcyBbUkZDNTA0M10sIGVuYWJsaW5nIHBlZXItdG8tcGVlciBjb25u ZWN0aW9uIGVzdGFibGlzaG1lbnQKICAgb3ZlciBNUEEvVENQLiAgVGhlIHNlY29uZCBleHRlbnNp b24gZXh0ZW5kcyBib3RoIFtSRkM1MDQzXSBhbmQKICAgW1JGQzUwNDRdLCBieSBwcm92aWRpbmcg YW4gb3B0aW9uIGZvciBzdGFuZGFyZGl6ZWQgZXhjaGFuZ2Ugb2YgUkRNQS0KICAgbGF5ZXIgY29u bmVjdGlvbiBjb25maWd1cmF0aW9uLgoKU3RhdHVzIG9mIHRoaXMgTWVtbwoKICAgVGhpcyBJbnRl cm5ldC1EcmFmdCBpcyBzdWJtaXR0ZWQgaW4gZnVsbCBjb25mb3JtYW5jZSB3aXRoIHRoZQogICBw cm92aXNpb25zIG9mIEJDUCA3OCBhbmQgQkNQIDc5LgoKICAgSW50ZXJuZXQtRHJhZnRzIGFyZSB3 b3JraW5nIGRvY3VtZW50cyBvZiB0aGUgSW50ZXJuZXQgRW5naW5lZXJpbmcKICAgVGFzayBGb3Jj ZSAoSUVURikuICBOb3RlIHRoYXQgb3RoZXIgZ3JvdXBzIG1heSBhbHNvIGRpc3RyaWJ1dGUKICAg d29ya2luZyBkb2N1bWVudHMgYXMgSW50ZXJuZXQtRHJhZnRzLiAgVGhlIGxpc3Qgb2YgY3VycmVu dCBJbnRlcm5ldC0KICAgRHJhZnRzIGlzIGF0IGh0dHA6Ly9kYXRhdHJhY2tlci5pZXRmLm9yZy9k cmFmdHMvY3VycmVudC8uCgogICBJbnRlcm5ldC1EcmFmdHMgYXJlIGRyYWZ0IGRvY3VtZW50cyB2 YWxpZCBmb3IgYSBtYXhpbXVtIG9mIHNpeCBtb250aHMKICAgYW5kIG1heSBiZSB1cGRhdGVkLCBy ZXBsYWNlZCwgb3Igb2Jzb2xldGVkIGJ5IG90aGVyIGRvY3VtZW50cyBhdCBhbnkKICAgdGltZS4g IEl0IGlzIGluYXBwcm9wcmlhdGUgdG8gdXNlIEludGVybmV0LURyYWZ0cyBhcyByZWZlcmVuY2UK ICAgbWF0ZXJpYWwgb3IgdG8gY2l0ZSB0aGVtIG90aGVyIHRoYW4gYXMgIndvcmsgaW4gcHJvZ3Jl c3MuIgoKICAgVGhpcyBJbnRlcm5ldC1EcmFmdCB3aWxsIGV4cGlyZSBvbiBPY3RvYmVyIDcsIDIw MTEuCgpDb3B5cmlnaHQgTm90aWNlCgogICBDb3B5cmlnaHQgKGMpIDIwMTEgSUVURiBUcnVzdCBh bmQgdGhlIHBlcnNvbnMgaWRlbnRpZmllZCBhcyB0aGUKICAgZG9jdW1lbnQgYXV0aG9ycy4gIEFs bCByaWdodHMgcmVzZXJ2ZWQuCgogICBUaGlzIGRvY3VtZW50IGlzIHN1YmplY3QgdG8gQkNQIDc4 IGFuZCB0aGUgSUVURiBUcnVzdCdzIExlZ2FsCiAgIFByb3Zpc2lvbnMgUmVsYXRpbmcgdG8gSUVU RiBEb2N1bWVudHMKICAgKGh0dHA6Ly90cnVzdGVlLmlldGYub3JnL2xpY2Vuc2UtaW5mbykgaW4g ZWZmZWN0IG9uIHRoZSBkYXRlIG9mCgoKCkthbmV2c2t5LCBldCBhbC4gICAgICAgICBFeHBpcmVz IE9jdG9iZXIgNywgMjAxMSAgICAgICAgICAgICAgICBbUGFnZSAxXQoMCkludGVybmV0LURyYWZ0 ICAgRW5oYW5jZWQgUkRNQSBDb25uZWN0aW9uIEVzdGFibGlzaG1lbnQgICAgICAgQXByaWwgMjAx MQoKCiAgIHB1YmxpY2F0aW9uIG9mIHRoaXMgZG9jdW1lbnQuICBQbGVhc2UgcmV2aWV3IHRoZXNl IGRvY3VtZW50cwogICBjYXJlZnVsbHksIGFzIHRoZXkgZGVzY3JpYmUgeW91ciByaWdodHMgYW5k IHJlc3RyaWN0aW9ucyB3aXRoIHJlc3BlY3QKICAgdG8gdGhpcyBkb2N1bWVudC4gIENvZGUgQ29t cG9uZW50cyBleHRyYWN0ZWQgZnJvbSB0aGlzIGRvY3VtZW50IG11c3QKICAgaW5jbHVkZSBTaW1w bGlmaWVkIEJTRCBMaWNlbnNlIHRleHQgYXMgZGVzY3JpYmVkIGluIFNlY3Rpb24gNC5lIG9mCiAg IHRoZSBUcnVzdCBMZWdhbCBQcm92aXNpb25zIGFuZCBhcmUgcHJvdmlkZWQgd2l0aG91dCB3YXJy YW50eSBhcwogICBkZXNjcmliZWQgaW4gdGhlIFNpbXBsaWZpZWQgQlNEIExpY2Vuc2UuCgoKVGFi bGUgb2YgQ29udGVudHMKCiAgIDEuICBJbnRyb2R1Y3Rpb24gLiAuIC4gLiAuIC4gLiAuIC4gLiAu IC4gLiAuIC4gLiAuIC4gLiAuIC4gLiAuIC4gLiAgMwogICAgIDEuMS4gIFN1bW1hcnkgb2YgY2hh bmdlcyBmcm9tIFJGQyA1MDQ0IC4gLiAuIC4gLiAuIC4gLiAuIC4gLiAuIC4gIDMKICAgICAxLjIu ICBTdW1tYXJ5IG9mIGNoYW5nZXMgZnJvbSBSRkMgNTA0MyAuIC4gLiAuIC4gLiAuIC4gLiAuIC4g LiAuICAzCiAgIDIuICBSZXF1aXJlbWVudHMgTGFuZ3VhZ2UgIC4gLiAuIC4gLiAuIC4gLiAuIC4g LiAuIC4gLiAuIC4gLiAuIC4gLiAgMwogICAzLiAgRGVmaW5pdGlvbnMgIC4gLiAuIC4gLiAuIC4g LiAuIC4gLiAuIC4gLiAuIC4gLiAuIC4gLiAuIC4gLiAuIC4gIDMKICAgNC4gIE1vdGl2YXRpb25z ICAuIC4gLiAuIC4gLiAuIC4gLiAuIC4gLiAuIC4gLiAuIC4gLiAuIC4gLiAuIC4gLiAuICA0CiAg ICAgNC4xLiAgRW5hYmxpbmcgTVBBIE1vZGUgIC4gLiAuIC4gLiAuIC4gLiAuIC4gLiAuIC4gLiAu IC4gLiAuIC4gLiAgNQogICAgIDQuMi4gIExhY2sgb2YgRXhwbGljaXQgUlRSIGluIE1QQSBSZXF1 ZXN0L1JlcGx5IEV4Y2hhbmdlIC4gLiAuIC4gIDUKICAgICA0LjMuICBMaW1pdGF0aW9ucyBvbiBV TFAgV29ya2Fyb3VuZCAgLiAuIC4gLiAuIC4gLiAuIC4gLiAuIC4gLiAuICA2CiAgICAgICA0LjMu MS4gIFRyYW5zcG9ydCBOZXV0cmFsIEFQSXMgLiAuIC4gLiAuIC4gLiAuIC4gLiAuIC4gLiAuIC4g LiAgNwogICAgICAgNC4zLjIuICBXb3JrL0NvbXBsZXRpb24gUXVldWUgQWNjb3VudGluZyAuIC4g LiAuIC4gLiAuIC4gLiAuIC4gIDcKICAgICAgIDQuMy4zLiAgSG9zdC1iYXNlZCBJbXBsZW1lbnRh dGlvbiBvZiBNUEEgRmVuY2luZyAuIC4gLiAuIC4gLiAuICA4CiAgICAgNC40LiAgU3RhbmRhcmRp emVkIFJETUEgUGFyYW1ldGVyIE5lZ290aWF0aW9uICAuIC4gLiAuIC4gLiAuIC4gLiAgOAogICA1 LiAgTVBBIENvbm5lY3Rpb24gU2V0dXAgLiAuIC4gLiAuIC4gLiAuIC4gLiAuIC4gLiAuIC4gLiAu IC4gLiAuIC4gIDkKICAgNi4gIEVuaGFuY2VkIE1QQSBSZXF1ZXN0L1JlcGx5IEZyYW1lcyAgLiAu IC4gLiAuIC4gLiAuIC4gLiAuIC4gLiAuIDEwCiAgIDcuICBFbmhhbmNlZCBTQ1RQIFNlc3Npb24g Q29udHJvbCBDaHVua3MgLiAuIC4gLiAuIC4gLiAuIC4gLiAuIC4gLiAxMQogICA4LiAgTVBBIEVy cm9yIFJlcG9ydGluZyAgLiAuIC4gLiAuIC4gLiAuIC4gLiAuIC4gLiAuIC4gLiAuIC4gLiAuIC4g MTMKICAgOS4gIEVuaGFuY2VkIFJETUEgQ29ubmVjdGlvbiBFc3RhYmxpc2htZW50IERhdGEgIC4g LiAuIC4gLiAuIC4gLiAuIDEzCiAgIDEwLiBJbnRlcm9wZXJhYmlsaXR5IC4gLiAuIC4gLiAuIC4g LiAuIC4gLiAuIC4gLiAuIC4gLiAuIC4gLiAuIC4gLiAxNQogICAxMS4gSUFOQSBDb25zaWRlcmF0 aW9ucyAgLiAuIC4gLiAuIC4gLiAuIC4gLiAuIC4gLiAuIC4gLiAuIC4gLiAuIC4gMTYKICAgMTIu IFNlY3VyaXR5IENvbnNpZGVyYXRpb25zICAuIC4gLiAuIC4gLiAuIC4gLiAuIC4gLiAuIC4gLiAu IC4gLiAuIDE2CiAgIDEzLiBBY2tub3dsZWRnZW1lbnRzIC4gLiAuIC4gLiAuIC4gLiAuIC4gLiAu IC4gLiAuIC4gLiAuIC4gLiAuIC4gLiAxNgogICAxNC4gUmVmZXJlbmNlcyAuIC4gLiAuIC4gLiAu IC4gLiAuIC4gLiAuIC4gLiAuIC4gLiAuIC4gLiAuIC4gLiAuIC4gMTYKICAgICAxNC4xLiBOb3Jt YXRpdmUgUmVmZXJlbmNlcyAuIC4gLiAuIC4gLiAuIC4gLiAuIC4gLiAuIC4gLiAuIC4gLiAuIDE2 CiAgICAgMTQuMi4gSW5mb3JtYXRpdmUgUmVmZXJlbmNlcyAuIC4gLiAuIC4gLiAuIC4gLiAuIC4g LiAuIC4gLiAuIC4gLiAxNgogICBBdXRob3JzJyBBZGRyZXNzZXMgLiAuIC4gLiAuIC4gLiAuIC4g LiAuIC4gLiAuIC4gLiAuIC4gLiAuIC4gLiAuIC4gMTcKCgoKCgoKCgoKCgoKCgoKS2FuZXZza3ks IGV0IGFsLiAgICAgICAgIEV4cGlyZXMgT2N0b2JlciA3LCAyMDExICAgICAgICAgICAgICAgIFtQ YWdlIDJdCgwKSW50ZXJuZXQtRHJhZnQgICBFbmhhbmNlZCBSRE1BIENvbm5lY3Rpb24gRXN0YWJs aXNobWVudCAgICAgICBBcHJpbCAyMDExCgoKMS4gIEludHJvZHVjdGlvbgoKICAgV2hlbiB1c2Vk IG92ZXIgVENQLCB0aGUgY3VycmVudCBSRERQIHN1aXRlIG9mIHByb3RvY29scyByZWxpZXMgb24K ICAgTWFya2VyIFBEVSBBbGlnbm1lbnQgKE1QQSkgW1JGQzUwNDRdIHByb3RvY29sIGZvciBib3Ro IGNvbm5lY3Rpb24KICAgZXN0YWJsaXNobWVudCBhbmQgZm9yIG1hcmtlcnMgZm9yIFRDUCBsYXll cmluZy4gIEN1cnJlbnRseSBNUEEgb25seQogICBzdXBwb3J0cyBhIGNsaWVudC1zZXJ2ZXIgbW9k ZWwgZm9yIGNvbm5lY3Rpb24gZXN0YWJsaXNobWVudCwgZm9yY2luZwogICBwZWVyLXRvLXBlZXIg YXBwbGljYXRpb25zIHRvIGludGVyYWN0IGFzIHRob3VnaCB0aGV5IGhhZCBhIGNsaWVudC8KICAg c2VydmVyIHJlbGF0aW9uc2hpcC4gIEluIGFkZGl0aW9uIG5lZ290aWF0aW9uIG9mIHNvbWUgb2Yg UmVtb3RlCiAgIERpcmVjdCBNZW1vcnkgQWNjZXNzIFByb3RvY29sIChSRE1BUCkgW1JGQzUwNDBd IHNwZWNpZmljIHBhcmFtZXRlcnMKICAgYXJlIGxlZnQgdG8gVUxQIG5lZ290aWF0aW9uLiAgUHJv dmlkaW5nIGFuIG9wdGlvbmFsIFVMUC1pbmRlcGVuZGVudAogICBmb3JtYXQgZm9yIGV4Y2hhbmdp bmcgdGhlc2UgcGFyYW1ldGVycyB3b3VsZCBiZSBvZiBiZW5lZml0IHRvCiAgIHRyYW5zcG9ydCBu ZXV0cmFsIFJETUEgYXBwbGljYXRpb25zLgoKMS4xLiAgU3VtbWFyeSBvZiBjaGFuZ2VzIGZyb20g UkZDIDUwNDQKCiAgIFRoaXMgZHJhZnQgZXh0ZW5kcyBbUkZDNTA0NF0gTVBBIGNvbm5lY3Rpb24g c2V0dXAgcHJvdG9jb2wuICBGaXJzdCwKICAgaXQgYWRkIGV4Y2hhbmdlIGFuZCBuZWdvdGlhdGlv biBvZiBtYXhpbXVtIG51bWJlciBvZiBSRE1BIFJlYWQKICAgSW5jb21pbmcgKElSRCkgYW5kIE91 dGdvaW5nIG1lc3NhZ2VzIChPUkQpLiAgU2Vjb25kLCBpdCBhZGRzIG9uZSBtb3JlCiAgIFJlYWR5 IHRvIFJlY2VpdmUgKFJUUikgZnJhbWUgZnJvbSByZXF1ZXN0b3IgdG8gcmVzcG9uZGVyIGFzIHRo ZSBsYXN0CiAgIG1lc3NhZ2Ugb2YgY29ubmVjdGlvbiBlc3RhYmxpc2htZW50IGFuZCBhZGRzIG5l Z290aWF0aW9uIG9mIFJUUiBmcmFtZQogICBtZXNzYWdlIHR5cGUgaW50byBNUEEgcmVxdWVzdC9y ZXNwb25zZSBmcmFtZXMuCgoxLjIuICBTdW1tYXJ5IG9mIGNoYW5nZXMgZnJvbSBSRkMgNTA0MwoK ICAgVGhpcyBkcmFmdCBleHRlbmRzIFtSRkM1MDQzXSBieSBhZGRpbmcgbmV3IEVuaGFuY2VkIFNl c3Npb24gQ29udHJvbAogICBDaHVua3MgdGhhdCBleHRlbmQgdGhlIGN1cnJlbnRseSBkZWZpbmVk IENodW5rcyB3aXRoIHRoZSBhZGRpdGlvbiBvZgogICBJUkQgYW5kIE9SRCBuZWdvdGlhdGlvbi4K CgoyLiAgUmVxdWlyZW1lbnRzIExhbmd1YWdlCgogICBUaGUga2V5IHdvcmRzICJNVVNUIiwgIk1V U1QgTk9UIiwgIlJFUVVJUkVEIiwgIlNIQUxMIiwgIlNIQUxMIE5PVCIsCiAgICJTSE9VTEQiLCAi U0hPVUxEIE5PVCIsICJSRUNPTU1FTkRFRCIsICJNQVkiLCBhbmQgIk9QVElPTkFMIiBpbiB0aGlz CiAgIGRvY3VtZW50IGFyZSB0byBiZSBpbnRlcnByZXRlZCBhcyBkZXNjcmliZWQgaW4gW1JGQzIx MTldLgoKCjMuICBEZWZpbml0aW9ucwoKICAgRlVMUERVOiAgRnJhbWVkIFVwcGVyIExheWVyIFBy b3RvY29sIFBEVS4gIFNlZSBGUERVIG9mIFtSRkM1MDQ0XS4KCiAgIENvbXBsZXRpb24gUXVldWUg KENRKTogIEEgY29uc3VtZXIgYWNjZXNzaWJsZSBxdWV1ZSB3aGVyZSB0aGUgUkRNQQogICAgICBk ZXZpY2UgcmVwb3J0cyBjb21wbGV0aW9ucyBvZiBXb3JrIFJlcXVlc3RzLiAgQSBDb25zdW1lciBp cyBhYmxlCiAgICAgIHRvIHJlYXAgY29tcGxldGlvbnMgZnJvbSBhIENRIHdpdGhvdXQgcmVxdWly aW5nIHBlciB0cmFuc2FjdGlvbgogICAgICBzdXBwb3J0IGZyb20gdGhlIGtlcm5lbCBvciBvdGhl ciBwcml2aWxlZ2VkIGVudGl0eS4gIFNlZSBbUkRNQUNdLgoKCgoKCgoKS2FuZXZza3ksIGV0IGFs LiAgICAgICAgIEV4cGlyZXMgT2N0b2JlciA3LCAyMDExICAgICAgICAgICAgICAgIFtQYWdlIDNd CgwKSW50ZXJuZXQtRHJhZnQgICBFbmhhbmNlZCBSRE1BIENvbm5lY3Rpb24gRXN0YWJsaXNobWVu dCAgICAgICBBcHJpbCAyMDExCgoKICAgQ29tcGxldGlvbiBRdWV1ZSBFbnRyeSAoQ1FFKTogIFRy YW5zcG9ydCBhbmQgZGV2aWNlIHNwZWNpZmljCiAgICAgIHJlcHJlc2VudGF0aW9uIG9mIGEgV29y ayBDb21wbGV0aW9uLiAgQSBDb21wbGV0aW9uIFF1ZXVlIGhvbGRzCiAgICAgIENRRXMuICBTZWUg W1JETUFDXS4KCiAgIEluYm91bmQgUkRNQSBSZWFkIFF1ZXVlIERlcHRoIChJUkQpOiAgVGhlIG1h eGltdW0gbnVtYmVyIG9mIGluY29taW5nCiAgICAgIG91dHN0YW5kaW5nIFJETUEgUmVhZCBSZXF1 ZXN0IE1lc3NhZ2VzIGFuIFJETUEgY29ubmVjdGlvbiBjYW4KICAgICAgaGFuZGxlLiAgU2VlIFtS RE1BQ10uCgogICBJUkQ6ICBTZWUgSW5ib3VuZCBSRE1BIFJlYWQgUXVldWUgRGVwdGguCgogICBP UkQ6ICBTZWUgT3V0Ym91bmQgUkRNQSBSZWFkIFF1ZXVlIERlcHRoLgoKICAgT3V0Ym91bmQgUkRN QSBSZWFkIFF1ZXVlIERlcHRoIChPUkQpOiAgVGhlIG1heGltdW0gbnVtYmVyIG9mCiAgICAgIG91 dHN0YW5kaW5nIFJETUEgUmVhZCBSZXF1ZXN0cyB0aGF0IGNhbiBiZSBpc3N1ZWQgZm9yIHRoZSBS RE1BCiAgICAgIGNvbm5lY3Rpb24uICBUaGlzIHNob3VsZCBiZSBsZXNzIHRoYW4gb3IgZXF1YWwg dG8gdGhlIHBlZXIncyBJUkQuCiAgICAgIFNlZSBbUkRNQUNdLgoKICAgUXVldWUgUGFpciAoUVAp OiAgVGhlIHRyYWRpdGlvbmFsIG5hbWUgZm9yIGEgbG9jYWwgRW5kcG9pbnQgaW4gYQogICAgICBb VklBXSBkZXJpdmVkIGxvY2FsIGludGVyZmFjZS4gIEEgUXVldWUgUGFpciBpcyB0aGUgc2V0IG9m IFdvcmsKICAgICAgUXVldWVzIGFzc29jaWF0ZWQgZXhjbHVzaXZlbHkgd2l0aCBhIHNpbmdsZSBF bmRwb2ludC4gIFRoZSBTZW5kCiAgICAgIFF1ZXVlIChTUSksIFJlY2VpdmUgUXVldWUgKFJRKSBh bmQgSW5ib3VuZCBSRE1BIFJlYWQgUXVldWUgKElSUSkKICAgICAgYXJlIGNvbnNpZGVyZWQgdG8g YmUgcGFydCBvZiB0aGUgUXVldWUgUGFpci4gIFRoZSBwb3RlbnRpYWxseQogICAgICBzaGFyZWQg Q29tcGxldGlvbiBRdWV1ZSAoQ1EpIGFuZCBTaGFyZWQgUmVjZWl2ZSBRdWV1ZSAoU1JRKSBhcmUK ICAgICAgbm90LiAgU2VlIFtSRE1BQ10uCgogICBTaGFyZWQgUmVjZWl2ZSBRdWV1ZShTUlEpOiAg QSBzaGFyZWQgcG9vbCBvZiBSZWNlaXZlIFdvcmsgUmVxdWVzdHMKICAgICAgcG9zdGVkIGJ5IHRo ZSBDb25zdW1lciB0aGF0IGNhbiBiZSBhbGxvY2F0ZWQgYnkgbXVsdGlwbGUgUkRNQQogICAgICBl bmRwb2ludHMgKFF1ZXVlIFBhaXIpLiAgU2VlIFtEQVBMXS4KCiAgIFdvcmsgUXVldWU6ICBBbiBl bGVtZW50IG9mIGEgW1ZJQV0gZGVyaXZlZCBsb2NhbCBpbnRlcmZhY2UgdGhhdAogICAgICBhbGxv d3MgdXNlci1zcGFjZSBhcHBsaWNhdGlvbnMgdG8gc3VibWl0IFdvcmsgUmVxdWVzdHMgZGlyZWN0 bHkgdG8KICAgICAgbmV0d29yayBoYXJkd2FyZS4gIFNwZWNpZmljIFdvcmsgUXVldWVzIGluY2x1 ZGUgdGhlIFNlbmQgUXVldWUKICAgICAgKFNRKSBmb3IgdHJhbnNtaXQgcmVxdWVzdHMsIFJlY2Vp dmUgUXVldWUgKFJRKSBmb3IgcmVjZWl2ZQogICAgICByZXF1ZXN0cyBzcGVjaWZpYyB0byBhIHNp bmdsZSBFbmRwb2ludCBhbmQgU2hhcmVkIFJlY2VpdmUgUXVldWVzCiAgICAgIChTUlFzKSBmb3Ig cmVjZWl2ZSByZXF1ZXN0cyB0aGF0IGNhbiBiZSBhbGxvY2F0ZWQgYnkgb25lIG9yIG1vcmUKICAg ICAgRW5kcG9pbnRzLiAgU2VlIFtSRE1BQ10uCgogICBXb3JrIFF1ZXVlIEVsZW1lbnQgKFdRRSk6 ICBUcmFuc3BvcnQgYW5kIGRldmljZSBzcGVjaWZpYwogICAgICByZXByZXNlbnRhdGlvbiBvZiBh IFdvcmsgUmVxdWVzdC4gIFNlZSBbUkRNQUNdCgogICBXb3JrIFJlcXVlc3Q6ICBBbiBlbGVtZW50 YXJ5IG9iamVjdCB1c2VkIGJ5IENvbnN1bWVycyB0byBlbnF1ZXVlIGEKICAgICAgcmVxdWVzdGVk IG9wZXJhdGlvbiAoV1FFcykgb250byBhIFdvcmsgUXVldWUuICBTZWUgW1JETUFDXS4KCgo0LiAg TW90aXZhdGlvbnMKCiAgIFRoZSBnb2FsIG9mIHRoaXMgZHJhZnQgaXMgdHdvZm9sZC4gIE9uZSBp cyB0byBleHRlbmQgc3VwcG9ydCBmcm9tIHRoZQogICBjdXJyZW50IGNsaWVudC1zZXJ2ZXIgbW9k ZWwgZm9yIFJETUEgY29ubmVjdGlvbiBzZXR1cCB0byBhIHBlZXItdG8tCgoKCkthbmV2c2t5LCBl dCBhbC4gICAgICAgICBFeHBpcmVzIE9jdG9iZXIgNywgMjAxMSAgICAgICAgICAgICAgICBbUGFn ZSA0XQoMCkludGVybmV0LURyYWZ0ICAgRW5oYW5jZWQgUkRNQSBDb25uZWN0aW9uIEVzdGFibGlz aG1lbnQgICAgICAgQXByaWwgMjAxMQoKCiAgIHBlZXIgbW9kZWwuICBUaGUgc2Vjb25kIGlzIHRv IGFkZCBuZWdvdGlhdGlvbiBvZiBSRE1BIFJlYWQgcXVldWUgc2l6ZQogICBmb3IgYm90aCBzaWRl cyBvZiBhbiBSRE1BIGNvbm5lY3Rpb24uCgo0LjEuICBFbmFibGluZyBNUEEgTW9kZQoKICAgTVBB IGRlZmluZXMgZW5jb2Rpbmcgb2YgRERQIFNlZ21lbnRzIGluIEZVTFBEVXMgKEZyYW1lZCBVcHBl ciBMYXllcgogICBQcm90b2NvbCBQRFVzKS4gIEdlbmVyYXRpb24gb2YgRlVMUERVcyByZXF1aXJl cyB0aGUgYWJpbGl0eSB0bwogICBwZXJpb2RpY2FsbHkgaW5zZXJ0IE1QQSBNYXJrZXJzIGFuZCB0 byBnZW5lcmF0ZSB0aGUgTVBBIENSQy0zMmMgZm9yCiAgIGVhY2ggZnJhbWUuICBSZWNlcHRpb24g bWF5IHJlcXVpcmUgcGFyc2luZy9yZW1vdmluZyB0aGUgbWFya2VycyBhZnRlcgogICB1c2luZyB0 aGVtIHRvIGlkZW50aWZ5IE1QQSBGcmFtZSBib3VuZGFyaWVzLCBhbmQgdmFsaWRhdGlvbiBvZiB0 aGUKICAgTVBBLUNSQzMyYy4KCiAgIEEgbWFqb3IgZGVzaWduIG9iamVjdGl2ZSBmb3IgTVBBIHdh cyB0byBlbnN1cmUgdGhhdCB0aGUgcmVzdWx0aW5nIFRDUAogICBzdHJlYW0gd291bGQgYmUgYSBm dWxseSBjb21wbGlhbnQgVENQIHN0cmVhbSBmb3IgYW55IGFuZCBhbGwgVENQLQogICBhd2FyZSBt aWRkbGUtYm94ZXMuICBUaGUgY2hhbGxlbmdlIGlzIHRoYXQgd2hpbGUgb25seSBzb21lIFRDUAog ICBwYXlsb2FkIHN0cmVhbXMgYXJlIGEgdmFsaWQgc3RyZWFtIG9mIE1QQSBGVUxQRFVzLCBhbnkg c2VxdWVuY2Ugb2YKICAgYnl0ZXMgaXMgYSB2YWxpZCBUQ1AgcGF5bG9hZCBzdHJlYW0uICBUaGUg ZGV0ZXJtaW5hdGlvbiB0aGF0IGEgZ2l2ZW4KICAgc3RyZWFtIGlzIGluIGEgc3BlY2lmaWMgTVBB IG1vZGUgY2Fubm90IGJlIG1hZGUgYXQgdGhlIE1QQSBvciBUQ1AKICAgbGF5ZXIuICBUaGVyZWZv cmUgZW5hYmxpbmcgb2YgTVBBIG1vZGUgaXMgaGFuZGxlZCBieSB0aGUgVUxQLgoKICAgVGhlIE1Q QSBwcm90b2NvbCBjYW4gYmUgdmlld2VkIGFzIGhhdmluZyB0d28gcGFydHMuCgogICBvICBhIHNw ZWNpZmljYXRpb24gb2YgZ2VuZXJhdGlvbiBhbmQgcmVjZXB0aW9uIG9mIE1QQSBGVUxQRFVzLiAg VGhpcwogICAgICBpcyB1bmNoYW5nZWQgYnkgRW5oYW5jZWQgUkRNQSBDb25uZWN0aW9uIEVzdGFi bGlzaG1lbnQuCgogICBvICBhIHByZS1NUEEgZXhjaGFuZ2Ugb2YgbWVzc2FnZXMgdG8gZW5hYmxl IGEgc3BlY2lmaWMgTVBBIG1vZGUgZm9yCiAgICAgIHRoZSBUQ1AgY29ubmVjdGlvbi4gIEVuaGFu Y2VkIFJETUEgQ29ubmVjdGlvbiBFc3RhYmxpc2htZW50CiAgICAgIGV4dGVuZHMgdGhpcyBwcm90 b2NvbCB3aXRoIHR3byBuZXcgZmVhdHVyZXMuCgogICBJbiB0eXBpY2FsIGltcGxlbWVudGF0aW9u cywgZ2VuZXJhdGlvbiBhbmQgcmVjZXB0aW9uIG9mIE1QQSBGVUxQRFVzCiAgIGlzIGhhbmRsZWQg YnkgaGFyZHdhcmUuICBUaGUgZXhjaGFuZ2Ugb2YgdGhlIE1QQSBSZXF1ZXN0IGFuZCBSZXBseQog ICBmcmFtZXMgaXMgdGhlbiBoYW5kbGVkIGJ5IGhvc3Qgc29mdHdhcmUuICBBcyB3aWxsIGJlIGV4 cGxhaW5lZCwgdGhpcwogICBpbXBsZW1lbnRhdGlvbiBzcGxpdCBwcmV2ZW50cyBhcHBsaWNhdGlv bnMgZnJvbSB3b3JraW5nIGFyb3VuZCB0aGUKICAgY2xpZW50LXNlcnZlciBhc3N1bXB0aW9ucyBp biB0aGUgY3VycmVudCBNUEEgUmVxdWVzdC9SZXBseSBleGNoYW5nZS4KCjQuMi4gIExhY2sgb2Yg RXhwbGljaXQgUlRSIGluIE1QQSBSZXF1ZXN0L1JlcGx5IEV4Y2hhbmdlCgogICBUaGUgZXhjaGFu Z2Ugb2YgTVBBIFJlcXVlc3QgYW5kIFJlcGx5IG1lc3NhZ2VzIHRvIHBsYWNlIGEgVENQCiAgIGNv bm5lY3Rpb24gaW4gTVBBIG1vZGUgaXMgc3BlY2lmaWVkIGluIFtSRkM1MDQ0XS4gIFRoaXMgcHJv dG9jb2wKICAgcHJvdmlkZXMgbWFueSBiZW5lZml0cyB0byB0aGUgZGVzaWduIG9mIE1QQSBGVUxQ RFUgaGFyZHdhcmU6CgogICBvICBUaGUgVUxQIGlzIHJlc3BvbnNpYmxlIGZvciBzcGVjaWZ5aW5n IHRoZSBleGFjdCBNUEEgTW9kZSAoTWFya2VycwogICAgICBlbmFibGVkIG9yIGRpc2FibGVkLCBD UkMtMzJjIGVuYWJsZWQgb3Igc3VwcHJlc3NlZCkgYW5kIHRoZSBwb2ludAogICAgICBpbiB0aGUg VENQIHN0cmVhbXMgKGluYm91bmQgYW5kIG91dGJvdW5kKSB3aGVyZSBNUEEgZnJhbWVzIHdpbGwK ICAgICAgYmVnaW4uCgogICBvICBCZWZvcmUgdGhlIGZpcnN0IE1QQSBmcmFtZSBpcyB0cmFuc21p dHRlZCwgYWxsIHByZS1NUEEgbW9kZSBUQ1AKICAgICAgcGF5bG9hZCB3aWxsIGhhdmUgYmVlbiBh Y2tub3dsZWRnZWQgYnkgdGhlIHBlZXIuICBUaGVyZWZvcmUgaXQgaXMKCgoKS2FuZXZza3ksIGV0 IGFsLiAgICAgICAgIEV4cGlyZXMgT2N0b2JlciA3LCAyMDExICAgICAgICAgICAgICAgIFtQYWdl IDVdCgwKSW50ZXJuZXQtRHJhZnQgICBFbmhhbmNlZCBSRE1BIENvbm5lY3Rpb24gRXN0YWJsaXNo bWVudCAgICAgICBBcHJpbCAyMDExCgoKICAgICAgbmV2ZXIgbmVjZXNzYXJ5IHRvIGdlbmVyYXRl IGEgcmV0cmFuc21pc3Npb24gdGhhdCBtaXhlcyBwcmUtTVBBCiAgICAgIGFuZCBNUEEgcGF5bG9h ZC4KCiAgIG8gIEJlZm9yZSBNUEEgcmVjZXB0aW9uIGlzIGVuYWJsZWQsIGFsbCBpbmNvbWluZyBw cmUtTVBBIG1vZGUgVENQCiAgICAgIHBheWxvYWQgd2lsbCBoYXZlIGJlZW4gYWNrbm93bGVkZ2Vk LiAgVGhlcmVmb3JlIHRoZSBob3N0IHdpbGwKICAgICAgbmV2ZXIgcmVjZWl2ZSBhIFRDUCBzZWdt ZW50IHRoYXQgbWl4ZXMgcHJlLU1QQSBhbmQgTVBBIHBheWxvYWQuCgogICBUaGUgbGltaXRhdGlv biBvZiB0aGUgY3VycmVudCBNUEEgUmVxdWVzdC9SZXBseSBleGNoYW5nZSBpcyB0aGF0IGl0CiAg IGRvZXMgbm90IGRlZmluZSBhIFJlYWR5IHRvIFJlY2VpdmUgKFJUUikgbWVzc2FnZSB0aGF0IHRo ZSBhY3RpdmUgc2lkZQogICB3b3VsZCBzZW5kLCBzbyB0aGF0IHRoZSBwYXNzaXZlIHNpZGUgY2Fu IGtub3cgdGhhdCB0aGUgbGFzdCBub24tTVBBCiAgIHBheWxvYWQgKHRoZSBNUEEgUmVwbHkpIGhh ZCBiZWVuIHJlY2VpdmVkLgoKICAgSW5zdGVhZCwgdGhlIHJvbGUgb2YgYW4gUlRSIG1lc3NhZ2Ug aXMgcGlnZ3ktYmFja2VkIG9uIHRoZSBmaXJzdCBNUEEKICAgRlVMUERVIHNlbnQgYnkgdGhlIGFj dGl2ZSBzaWRlLiAgVGhpcyBpcyBhY3R1YWxseSBhIHZhbHVhYmxlCiAgIG9wdGltaXphdGlvbiBm b3IgYWxsIGFwcGxpY2F0aW9ucyB0aGF0IGZpdCB0aGUgY2xhc3NpYyBjbGllbnQvc2VydmVyCiAg IG1vZGVsLiAgVGhlIGNsaWVudCBvbmx5IGluaXRpYXRlcyB0aGUgY29ubmVjdGlvbiB3aGVuIGl0 IGhhcyBhCiAgIHJlcXVlc3QgdG8gc2VuZCB0byB0aGUgc2VydmVyLCBhbmQgdGhlIHNlcnZlciBo YXMgbm90aGluZyB0byBzZW5kCiAgIHVudGlsIGl0IGhhcyByZWNlaXZlZCBhbmQgcHJvY2Vzc2Vk IHRoZSBjbGllbnQgcmVxdWVzdC4KCiAgIEV2ZW4gYXBwbGljYXRpb25zIHdoZXJlIHRoZSBzZXJ2 ZXIgc2VuZHMgc29tZSBjb25maWd1cmF0aW9uIGRhdGEKICAgaW1tZWRpYXRlbHkgY2FuIGVhc2ls eSBzZW5kIHRoZSBzYW1lIGluZm9ybWF0aW9uIGFzIGFwcGxpY2F0aW9uCiAgIHByaXZhdGUgZGF0 YSBpbiB0aGUgTVBBIFJlcGx5LiAgU28gdGhlIGN1cnJlbnRseSBkZWZpbmVkIGV4Y2hhbmdlCiAg IHdvcmtzIGZvciBhbG1vc3QgYWxsIGFwcGxpY2F0aW9ucy4KCiAgIE1hbnkgcGVlci10by1wZWVy IGFwcGxpY2F0aW9ucywgZXNwZWNpYWxseSB0aG9zZSBpbnZvbHZpbmcgY2x1c3RlcgogICBjYWxj dWxhdGlvbnMgKGZyZXF1ZW50bHkgdXNpbmcgTVBJIFtVc2luZ01QSV0sIG9yIFtSRFNdKSwgaGF2 ZSBubwogICBuYXR1cmFsIGNsaWVudCBvciBzZXJ2ZXIgcm9sZXMgKFtQUE1QSV0sIFtPcGVuTVBd KS4gIFR5cGljYWxseSBvbmUKICAgbWVtYmVyIG9mIHRoZSBjbHVzdGVyIGlzIGFyYml0cmFyaWx5 IHNlbGVjdGVkIHRvIGluaXRpYXRlIHRoZQogICBjb25uZWN0aW9uIHdoZW4gdGhlIGRpc3RyaWJ1 dGVkIHRhc2sgaXMgbGF1bmNoZWQsIHdoaWxlIHRoZSBvdGhlcgogICBhY2NlcHRzIGl0LiAgQXQg c3RhcnR1cCB0aW1lLCBob3dldmVyLCB0aGVyZSBpcyBubyB3YXkgdG8gcHJlZGljdAogICB3aGlj aCBub2RlIHdpbGwgaGF2ZSB0aGUgZmlyc3QgbWVzc2FnZSB0byBhY3R1YWxseSBzZW5kLgogICBF c3RhYmxpc2hpbmcgdGhlIGNvbm5lY3Rpb25zIGltbWVkaWF0ZWx5LCBob3dldmVyLCBpcyB2YWx1 YWJsZQogICBiZWNhdXNlIGl0IHJlZHVjZXMgbGF0ZW5jeSBvbmNlIHJlc3VsdHMgYXJlIHJlYWR5 IHRvIHRyYW5zbWl0IGFuZCBpdAogICB2YWxpZGF0ZXMgY29ubmVjdGl2aXR5IHRocm91Z2hvdXQg dGhlIGNsdXN0ZXIuCgogICBUaGUgbGFjayBvZiBhbiBleHBsaWNpdCBSVFIgbWVzc2FnZSBpbiB0 aGUgTVBBIFJlcXVlc3QvUmVwbHkgZXhjaGFuZ2UKICAgZm9yY2VzIGFsbCBhcHBsaWNhdGlvbnMg dG8gaGF2ZSBhIGZpcnN0IG1lc3NhZ2UgZnJvbSB0aGUgY29ubmVjdGlvbgogICBpbml0aWF0b3Is IHdoZXRoZXIgdGhpcyBtYXRjaGVzIHRoZSBhcHBsaWNhdGlvbiBjb21tdW5pY2F0aW9uIG1vZGVs CiAgIG9yIG5vdC4KCjQuMy4gIExpbWl0YXRpb25zIG9uIFVMUCBXb3JrYXJvdW5kCgogICBUaGUg cmVxdWlyZW1lbnQgdGhhdCB0aGUgUkRNQSBjb25uZWN0aW9uIGluaXRpYXRvciBzZW5kcyB0aGUg Zmlyc3QKICAgbWVzc2FnZSBkb2VzIG5vdCBhcHBlYXIgdG8gYmUgdGhhdCBvbmVyb3VzIG9uIGZp cnN0IGV4YW1pbmF0aW9uLiAgVGhlCiAgIG5hdHVyYWwgcXVlc3Rpb24gaXMgd2h5IHRoZSBhcHBs aWNhdGlvbiBsYXllciB3b3VsZCBub3Qgc2ltcGx5CiAgIGdlbmVyYXRlIGEgIm5vcCIgbWVzc2Fn ZSB3aGVuIHRoZXJlIHdhcyBubyBvdGhlciBtZXNzYWdlIHRvIHN1Ym1pdC4KCiAgIFRoZXJlIGFy ZSB0aHJlZSBmYWN0b3JzIHRoYXQgbWFrZSB0aGlzIHdvcmthcm91bmQgdW5zdWl0YWJsZSBmb3Ig bWFueQoKCgpLYW5ldnNreSwgZXQgYWwuICAgICAgICAgRXhwaXJlcyBPY3RvYmVyIDcsIDIwMTEg ICAgICAgICAgICAgICAgW1BhZ2UgNl0KDApJbnRlcm5ldC1EcmFmdCAgIEVuaGFuY2VkIFJETUEg Q29ubmVjdGlvbiBFc3RhYmxpc2htZW50ICAgICAgIEFwcmlsIDIwMTEKCgogICBwZWVyLXRvLXBl ZXIgYXBwbGljYXRpb25zLgoKICAgbyAgVHJhbnNwb3J0IE5ldXRyYWwgQVBJcy4KCiAgIG8gIFdv cmsvQ29tcGxldGlvbiBRdWV1ZSBBY2NvdW50aW5nLgoKICAgbyAgSG9zdC1iYXNlZCBpbXBsZW1l bnRhdGlvbiBvZiBNUEEgRmVuY2luZy4KCjQuMy4xLiAgVHJhbnNwb3J0IE5ldXRyYWwgQVBJcwoK ICAgTWFueSBvZiB0aGVzZSBhcHBsaWNhdGlvbnMgYWNjZXNzIFJETUEgc2VydmljZXMgdXNpbmcg YSB0cmFuc3BvcnQKICAgbmV1dHJhbCBBUEkgc3VjaCBhcyBbREFQTF0gb3IgW09GQV0uICBPbmx5 IE1QQSBoYXMgYSBmaXJzdCBtZXNzYWdlCiAgIHJlcXVpcmVtZW50LiAgT3RoZXIgUkRNQSB0cmFu c3BvcnRzLCBpbmNsdWRpbmcgU0NUUCBhbmQgSW5maW5pQmFuZCwKICAgZG8gbm90LgoKICAgQXBw bGljYXRpb24gb3IgbWlkZGxld2FyZSBjb21tdW5pY2F0aW9ucyBjYW4gYmUgZXhwcmVzc2VkIGFz CiAgIHRyYW5zcG9ydCBuZXV0cmFsIFJETUEgb3BlcmF0aW9ucywgYWxsb3dpbmcgbG93ZXIgc29m dHdhcmUgbGF5ZXJzIHRvCiAgIHRyYW5zbGF0ZSB0byB0cmFuc3BvcnQgYW5kIGRldmljZSBzcGVj aWZpY3MuICBIYXZpbmcgYSBkaXN0aW5jdCBleHRyYQogICBtZXNzYWdlIHRoYXQgaXMgcmVxdWly ZWQgb25seSBmb3Igb25lIHRyYW5zcG9ydCB1bmRlcm1pbmVzIHRoZQogICBhcHBsaWNhdGlvbidz IGdvYWwgb2YgYmVpbmcgdHJhbnNwb3J0IG5ldXRyYWwuCgo0LjMuMi4gIFdvcmsvQ29tcGxldGlv biBRdWV1ZSBBY2NvdW50aW5nCgogICBSRE1BIGxvY2FsIEFQSXMgY29udmVudGlvbmFsbHkgdXNl IHdvcmsgcXVldWVzIHRvIHN1Ym1pdCByZXF1ZXN0cwogICAod29yayBxdWV1ZSBlbGVtZW50cyBv ciBXUUVzKSBhbmQgdG8gYXN5bmNocm9ub3VzbHkgcmVjZWl2ZQogICBjb21wbGV0aW9ucyAoaW4g Y29tcGxldGlvbiBxdWV1ZXMgb3IgQ1FzKS4KCiAgIEVhY2ggd29yayByZXF1ZXN0IGNhbiBnZW5l cmF0ZSBhIGNvbXBsZXRpb24gcXVldWUgZW50cnkgKENRRSkuCiAgIENvbXBsZXRpb25zIGZvciBz dWNjZXNzZnVsIHRyYW5zbWl0IHdvcmsgcmVxdWVzdHMgYXJlIGZyZXF1ZW50bHkKICAgc3VwcHJl c3NlZCwgYnV0IHRoZSBjb21wbGV0aW9uIHF1ZXVlIGNhcGFjaXR5IG11c3QgYWNjb3VudCBmb3Ig dGhlCiAgIHBvc3NpYmlsaXR5IHRoYXQgZWFjaCB3aWxsIGNvbXBsZXRlIGluIGVycm9yLiAgQSBj b21wbGV0aW9uIHF1ZXVlIGNhbgogICByZWNlaXZlIGNvbXBsZXRpb25zIGZyb20gbXVsdGlwbGUg d29yayBxdWV1ZXMuCgogICBDb21wbGV0aW9uIFF1ZXVlcyBhcmUgZGVmaW5lZCBzbyBhcyB0byBh bGxvdyBoYXJkd2FyZSBSRE1BCiAgIGltcGxlbWVudGF0aW9ucyB0byBnZW5lcmF0ZSBDUUVzIGRp cmVjdGx5IHRvIGEgdXNlci1zcGFjZSBtYXBwZWQKICAgYnVmZmVyLiAgVGhpcyBlbmFibGVzIGEg dXNlci1zcGFjZSBSRE1BIGNvbnN1bWVyIHRvIHJlYXAgY29tcGxldGlvbnMKICAgd2l0aG91dCBy ZXF1aXJpbmcga2VybmVsIGludGVydmVudGlvbi4KCiAgIEEgaGFyZHdhcmUgUkRNQSBpbXBsZW1l bnRhdGlvbiBjYW5ub3QgcmVhc29uYWJseSB3YWl0IGZvciBhbgogICBhdmFpbGFibGUgc2xvdCBp biB0aGUgY29tcGxldGlvbiBxdWV1ZS4gIFRoZSBxdWV1ZSBtdXN0IGJlIHNpemVkIHN1Y2gKICAg dGhhdCBhbiBvdmVyZmxvdyB3aWxsIG5vdCBvY2N1ci4gIFdoZW4gYW4gb3ZlcmZsb3cgZG9lcyBv Y2N1ciBpdCBpcwogICBjb25zaWRlcmVkIGNhdGFzdHJvcGhpYyBhbmQgd2lsbCB0eXBpY2FsbHkg cmVxdWlyZSB0ZWFyaW5nIGRvd24gYWxsCiAgIFJETUEgY29ubmVjdGlvbnMgdXNpbmcgdGhhdCBD US4KCiAgIFRoaXMgc3R5bGUgb2YgaW50ZXJmYWNlIGlzIHZlcnkgZWZmaWNpZW50LCBidXQgcGxh Y2VzIGEgYnVyZGVuIG9uIHRoZQogICBhcHBsaWNhdGlvbiB0byBwcm9wZXJseSBzaXplIGVhY2gg Q29tcGxldGlvbiBRdWV1ZSB0byBtYXRjaCB0aGUgV29yawogICBRdWV1ZXMgdGhhdCBmZWVkIGl0 LgoKCgoKS2FuZXZza3ksIGV0IGFsLiAgICAgICAgIEV4cGlyZXMgT2N0b2JlciA3LCAyMDExICAg ICAgICAgICAgICAgIFtQYWdlIDddCgwKSW50ZXJuZXQtRHJhZnQgICBFbmhhbmNlZCBSRE1BIENv bm5lY3Rpb24gRXN0YWJsaXNobWVudCAgICAgICBBcHJpbCAyMDExCgoKICAgV2hpbGUgdGhlIGZv cm1hdCBvZiBib3RoIFdRRXMgYW5kIENRRXMgaXMgdHJhbnNwb3J0IGFuZCBkZXZpY2UKICAgZGVw ZW5kZW50LCBhIHRyYW5zcG9ydCBuZXV0cmFsIEFQSSBjYW4gZGVhbCB3aXRoIFdRRXMgYW5kIENR RXMgYXMKICAgYWJzdHJhY3QgdHJhbnNwb3J0IGFuZCBkZXZpY2UgbmV1dHJhbCBvYmplY3RzLiAg VGhlcmVmb3JlIHRoZSBudW1iZXIKICAgb2YgV1FFcyBhbmQgQ1FFcyByZXF1aXJlZCBmb3IgYW4g YXBwbGljYXRpb24gY2FuIGJlIHRyYW5zcG9ydCBhbmQKICAgZGV2aWNlIG5ldXRyYWwuCgogICBU aGUgY2FwYWNpdHkgb2YgdGhlIHdvcmsgcXVldWVzIGFuZCBjb21wbGV0aW9uIHF1ZXVlcyBjYW4g YmUKICAgY2FsY3VsYXRlZCBpbiBhbiBhYnN0cmFjdCB0cmFuc3BvcnQvZGV2aWNlIG5ldXRyYWwg ZmFzaGlvbi4gIExvd2VyCiAgIGxheWVycyBvZiB0aGUgcHJvdG9jb2wgc3RhY2sgY2Fubm90IGRp c3J1cHQgdGhlc2UgY2FsY3VsYXRpb25zIGJ5CiAgIGluc2VydGluZyBhIGR1bW15ICJub3AiIFdv cmsgUmVxdWVzdCBhbmQgZmlsdGVyaW5nIG91dCB0aGUgbWF0Y2hpbmcKICAgY29tcGxldGlvbi4g IFRoZSBsb3dlciBsYXllciBkb2VzIG5vdCBrbm93IHRoZSB1c2FnZSBtb2RlbCBvbiB3aGljaAog ICB0aGUgcXVldWUgc2l6ZXMgYXJlIGJ1aWx0LCBub3IgZG9lcyBpdCBrbm93IGhvdyBmcmVxdWVu dGx5IGFuCiAgIGluc2VydGlvbiB3aWxsIGJlIHJlcXVpcmVkLgoKNC4zLjMuICBIb3N0LWJhc2Vk IEltcGxlbWVudGF0aW9uIG9mIE1QQSBGZW5jaW5nCgogICBNYW55IGhhcmR3YXJlIGltcGxlbWVu dGF0aW9ucyBvZiBpV0FSUCB1c2luZyBNUEEvVENQIGRvIG5vdCBoYW5kbGUKICAgdGhlIE1QQSBS ZXF1ZXN0L1JlcGx5IGV4Y2hhbmdlIGluIGhhcmR3YXJlLCByYXRoZXIgdGhleSBhcmUgaGFuZGxl ZAogICBieSB0aGUgaG9zdCBwcm9jZXNzb3IgaW4gc29mdHdhcmUuICBXaXRoIHN1Y2ggZGVzaWdu cyBpdCBpcyBjb21tb24KICAgZm9yIHRoZSBNUEEgRmVuY2luZyB0byBiZSBpbXBsZW1lbnRlZCBp biB0aGUgdXNlci1zcGFjZSBkZXZpY2UtCiAgIHNwZWNpZmljIGxpYnJhcnkgKGNvbW1vbmx5IHJl ZmVycmVkIHRvIGFzIGEgJ1VzZXIgVmVyYnMnIGxpYnJhcnkgb3IKICAgbW9kdWxlKS4KCiAgIFdo ZW4gdGhlIGdlbmVyYXRpb24gYW5kIHJlY2VwdGlvbiBvZiBNUEEgRlVMUERVcyBpcyBhbHJlYWR5 IGRlZGljYXRlZAogICB0byBoYXJkd2FyZSwgYSBXb3JrIENvbXBsZXRpb24gY2FuIG9ubHkgYmUg Z2VuZXJhdGVkIGJ5IGFuIHVudGFnZ2VkCiAgIG1lc3NhZ2UuCgo0LjQuICBTdGFuZGFyZGl6ZWQg UkRNQSBQYXJhbWV0ZXIgTmVnb3RpYXRpb24KCiAgIE1vc3QgUkRNQSBhcHBsaWNhdGlvbnMgYXJl IGRldmVsb3BlZCB1c2luZyBhIHRyYW5zcG9ydCBuZXV0cmFsIEFQSSB0bwogICBhY2Nlc3MgUkRN QSBzZXJ2aWNlcyBiYXNlZCBvbiBhICJxdWV1ZSBwYWlyIiBwYXJhZGlnbSBhcyBvcmlnaW5hbGx5 CiAgIGRlZmluZWQgYnkgdGhlIFZpcnR1YWwgSW50ZXJmYWNlIEFyY2hpdGVjdHVyZSBbVklBXSwg cmVmaW5lZCBieSB0aGUKICAgRGlyZWN0IEFjY2VzcyBQcm9ncmFtbWluZyBMaWJyYXJ5IFtEQVBM XSBhbmQgbW9zdCBjb21tb25seSBkZXBsb3llZAogICB3aXRoIHRoZSBPcGVuRmFicmljcyBBUEkg W09GQV0uCgogICBUaGVzZSB0cmFuc3BvcnQgbmV1dHJhbCBBUElzIHNlZWsgdG8gcHJvdmlkZSBh IGNvbW1vbiBzZXQgb2YgUkRNQQogICBzZXJ2aWNlcyB3aGV0aGVyIHRoZSB1bmRlcmx5aW5nIHRy YW5zcG9ydCBpcywgZm9yIGV4YW1wbGUsIGlXQVJQIG92ZXIKICAgTVBBLCBpV0FSUCBvdmVyIFND VFAgb3IgSW5maW5pQmFuZC4KCiAgIFRoZSBjb21tb24gbW9kZWwgZm9yIGVzdGFibGlzaGluZyBh biBSRE1BIGNvbm5lY3Rpb24gaGFzIHRoZQogICBmb2xsb3dpbmcgc3RlcHM6CgogICBvICBUaGUg cGFzc2l2ZSBzaWRlIFVMUCBsaXN0ZW5zIGZvciBjb25uZWN0aW9uIHJlcXVlc3RzLgoKICAgbyAg VGhlIGFjdGl2ZSBzaWRlIFVMUCBzdWJtaXRzIGEgY29ubmVjdGlvbiByZXF1ZXN0IHVzaW5nIGFu IFJETUEKICAgICAgZW5kcG9pbnQgKCJxdWV1ZSBwYWlyIiksIHRoZSBkZXNpcmVkIGRlc3RpbmF0 aW9uIGFuZCB0aGUKICAgICAgcGFyYW1ldGVycyB0byBiZSB1c2VkIGZvciB0aGUgY29ubmVjdGlv bi4gIFRob3NlIHBhcmFtZXRlcnMKICAgICAgaW5jbHVkZSBib3RoIFJETUEgbGF5ZXIgY2hhcmFj dGVyaXN0aWNzLCBzdWNoIGFzIHRoZSBSRE1BIFJlYWQKCgoKS2FuZXZza3ksIGV0IGFsLiAgICAg ICAgIEV4cGlyZXMgT2N0b2JlciA3LCAyMDExICAgICAgICAgICAgICAgIFtQYWdlIDhdCgwKSW50 ZXJuZXQtRHJhZnQgICBFbmhhbmNlZCBSRE1BIENvbm5lY3Rpb24gRXN0YWJsaXNobWVudCAgICAg ICBBcHJpbCAyMDExCgoKICAgICAgY3JlZGl0cyB0byBiZSBhbGxvd2VkIGFuZCBhcHBsaWNhdGlv biBzcGVjaWZpYyBkYXRhICh0eXBpY2FsbHkKICAgICAgcmVmZXJyZWQgdG8gYXMgInByaXZhdGUg ZGF0YSIpLgoKICAgbyAgVGhlIHBhc3NpdmUgc2lkZSBVTFAgcmVjZWl2ZXMgdGhlIENvbm5lY3Rp b24gUmVxdWVzdCwgd2hpY2gKICAgICAgaW5jbHVkZXMgdGhlIGlkZW50aXR5IG9mIHRoZSBhY3Rp dmUgc2lkZSBhbmQgdGhlIHJlcXVlc3RlZAogICAgICBjb25uZWN0aW9uIGNoYXJhY3RlcmlzdGlj cy4gIFRoZSBwYXNzaXZlIHNpZGUgVUxQIHVzZXMgdGhpcwogICAgICBpbmZvcm1hdGlvbiB0byBk ZWNpZGUgd2hldGhlciB0byBhY2NlcHQgdGhlIGNvbm5lY3Rpb24sIGFuZCBpZiBpdAogICAgICBp cyB0byBiZSBhY2NlcHRlZCwgaG93IHRvIGNyZWF0ZSBhbmQvb3IgY29uZmlndXJlIHRoZSBSRE1B CiAgICAgIGVuZHBvaW50LgoKICAgbyAgSWYgYWNjZXB0aW5nLCB0aGUgcGFzc2l2ZSBzaWRlIFVM UCBzdWJtaXRzIGl0cyBhY2NlcHRhbmNlIG9mIHRoZQogICAgICBDb25uZWN0aW9uIFJlcXVlc3Qu ICBUaGlzIGxvY2FsIGFjY2VwdCBvcGVyYXRpb24gaW5jbHVkZXMgdGhlIFJETUEKICAgICAgZW5k cG9pbnQgdG8gYmUgdXNlZCBhbmQgdGhlIGNvbm5lY3Rpb24gY2hhcmFjdGVyaXN0aWNzIChib3Ro IHRoZQogICAgICBSRE1BIGNvbmZpZ3VyYXRpb24gYW5kIGFueSBhcHBsaWNhdGlvbiBzcGVjaWZp YyBwcml2YXRlIGRhdGEgdG8gYmUKICAgICAgcmV0dXJuZWQpLgoKICAgbyAgVGhlIGFjdGl2ZSBz aWRlIHJlY2VpdmVzIGNvbmZpcm1hdGlvbiB0aGF0IHRoZSBjb25uZWN0aW9uIGhhcyBiZWVuCiAg ICAgIGFjY2VwdGVkLCB3aGF0IHRoZSBjb25maWd1cmVkIGNvbm5lY3Rpb24gY2hhcmFjdGVyaXN0 aWNzIGFyZSwgYW5kCiAgICAgIGFueSBhcHBsaWNhdGlvbiBzdXBwbGllZCBwcml2YXRlIGRhdGEu CgogICBBcyBjdXJyZW50bHkgZGVmaW5lZCwgRERQIGNvbm5lY3Rpb24gZXN0YWJsaXNobWVudCBy ZXF1aXJlcyB0aGUgVUxQCiAgIHRvIGVuY29kZSB0aGUgUkRNQSBjb25maWd1cmF0aW9uIGluIHRo ZSBhcHBsaWNhdGlvbiBzcGVjaWZpYyBwcml2YXRlCiAgIGRhdGEuICBUaGlzIHJlc3VsdHMgdW5k ZXNpcmFibGUgZHVwbGljYXRpb24gb2YgbG9naWMgdG8gY292ZXIgYm90aAogICBJbmZpbmlCYW5k IGFuZCBpV0FSUCwgYW5kIHRvIHNwZWNpZnkgdGhlIGV4dHJhY3Rpb24gb2YgdGhlIFJETUEKICAg Y2hhcmFjdGVyaXN0aWNzIGZyb20gdGhlIFVMUCBmb3IgZWFjaCBzcGVjaWZpYyBVcHBlciBMYXll ciBQcm90b2NvbC4KCiAgIEEgc3RhbmRhcmQgZGVmaW5pdGlvbiBvZiB0aGUgUkRNQSBjaGFyYWN0 ZXJpc3RpY3Mgd2l0aGluIHRoZSBwcml2YXRlCiAgIGRhdGEgc2VjdGlvbiB3b3VsZCBlbmFibGUg Y29tbW9uIGNvbm5lY3Rpb24gZXN0YWJsaXNobWVudCBBUElzIHRvCiAgIGZvcm1hdCB0aGUgUkRN QSBjaGFyYWN0ZXJpc3RpY3MgYmFzZWQgb24gdGhlIHNhbWUgQVBJIGluZm9ybWF0aW9uCiAgIHVz ZWQgd2hlbiBlc3RhYmxpc2hpbmcgYW4gSW5maW5pQmFuZCBjb25uZWN0aW9uLiAgVGhlIGFwcGxp Y2F0aW9uCiAgIHdvdWxkIHRoZW4gb25seSBoYXZlIHRvIGluZGljYXRlIHRoYXQgaXQgd2FzIHVz aW5nIHRoaXMgc3RhbmRhcmQKICAgZm9ybWF0IHRvIGVuYWJsZSBjb21tb24gY29ubmVjdGlvbiBl c3RhYmxpc2htZW50IHByb2NlZHVyZXMgdG8gYXBwbHkKICAgY29tbW9uIGNvZGUgdG8gcHJvcGVy bHkgcGFyc2UgdGhlc2UgZmllbGRzIGFuZCBjb25maWd1cmUgdGhlIFJETUEKICAgZW5kcG9pbnRz IGFjY29yZGluZ2x5LgoKCjUuICBNUEEgQ29ubmVjdGlvbiBTZXR1cAoKICAgQmVsb3cgd2UgcHJv dmlkZSBvdmVydmlldyBvZiBFbmhhbmNlZCBDb25uZWN0aW9uIFNldHVwLiAgVGhlIGdvYWwgaXMK ICAgdG8gYWxsb3cgc3RhbmRhcmQgbmVnb3RpYXRpb24gb2YgT1JEL0lSRCBzZXR0aW5nIG9uIGJv dGggc2lkZXMgb2YgdGhlCiAgIFJETUEgY29ubmVjdGlvbiBhbmQvb3IgdG8gbmVnb3RpYXRlIHRo ZSBpbml0aWFsIGRhdGEgdHJhbnNmZXIKICAgb3BlcmF0aW9uIGJ5IGFuIGluaXRpYXRvciB3aGVu IHRoZSBleGlzdGluZyAnY2xpZW50IHNlbmRzIGZpcnN0JyBydWxlCiAgIGRvZXMgbm90IG1hdGNo IGFwcGxpY2F0aW9uIHJlcXVpcmVtZW50cy4KCiAgIFRoZSBSRE1BIGNvbm5lY3Rpb24gaW5pdGlh dG9yIHNlbmRzIGFuIE1QQSBSZXF1ZXN0LCBhcyBzcGVjaWZpZWQgaW4KICAgW1JGQzUwNDRdOyB0 aGUgbmV3IGZvcm1hdCBkZWZpbmVkIGhlcmUgYWxsb3dzIGZvcjoKCgoKCgpLYW5ldnNreSwgZXQg YWwuICAgICAgICAgRXhwaXJlcyBPY3RvYmVyIDcsIDIwMTEgICAgICAgICAgICAgICAgW1BhZ2Ug OV0KDApJbnRlcm5ldC1EcmFmdCAgIEVuaGFuY2VkIFJETUEgQ29ubmVjdGlvbiBFc3RhYmxpc2ht ZW50ICAgICAgIEFwcmlsIDIwMTEKCgogICBvICBTdGFuZGFyZGl6ZWQgbmVnb3RpYXRpb24gb2Yg T1JEIGFuZCBJUkQuCgogICBvICBOZWdvdGlhdGlvbiBvZiBhbiBSVFIgbWVzc2FnZS4KCiAgIFRo ZSBSRE1BIGNvbm5lY3Rpb24gcmVzcG9uZGVyIHByb2Nlc3NlcyB0aGUgTVBBIFJlcXVlc3QgYW5k IGdlbmVyYXRlcwogICBhbiBNUEEgUmVwbHksIGFzIHNwZWNpZmllZCBpbiBbUkZDNTA0NF07IHRo ZSBuZXcgZm9ybWF0IGNvbXBsZXRlcyB0aGUKICAgbmVnb3RpYXRpb24uCgogICBUaGUgbG9jYWwg aW50ZXJmYWNlIFNIT1VMRCByZXF1aXJlIHRoZSBVTFAgdG8gZXhwbGljaXRseSBjb25maWd1cmUg b24KICAgYSBwZXItYXBwbGljYXRpb24gb3IgcGVyLWNvbm5lY3Rpb24gYmFzaXMgd2hlbiBhbiBl eHBsaWNpdCBSVFIKICAgbWVzc2FnZSB3aWxsIGJlIHJlcXVpcmVkLiAgUGlnZ3ktYmFja2luZyB0 aGUgUlRSIG9uIGEgQ2xpZW50J3MgZmlyc3QKICAgbWVzc2FnZSBpcyBhIHZhbHVhYmxlIG9wdGlt aXphdGlvbiBmb3IgbW9zdCBjb25uZWN0aW9ucy4KCiAgIFRoZSBSRE1BIGNvbm5lY3Rpb24gaW5p dGlhdG9yIE1VU1QgTk9UIGFsbG93IGFueSBsYXRlciBGVUxQRFVzIHRvIGJlCiAgIHRyYW5zbWl0 dGVkIGJlZm9yZSB0aGUgUlRSIG1lc3NhZ2UuICBPbmUgbWV0aG9kIHRvIGFjaGlldmUgdGhhdCBp cyB0bwogICBkZWxheSBub3RpZnlpbmcgdGhlIFVMUCB0aGF0IHRoZSBSRE1BIGNvbm5lY3Rpb24g aGFzIGJlZW4gZXN0YWJsaXNoZWQKICAgdW50aWwgYWZ0ZXIgYW55IHJlcXVpcmVkIFJUUiBNZXNz YWdlIGhhcyBiZWVuIHRyYW5zbWl0dGVkLgoKICAgQWxsIE1QQSBleGNoYW5nZXMgYXJlIHBlcmZv cm1lZCB2aWEgVENQIHByaW9yIHRvIFJETUEgZXN0YWJsaXNobWVudCwKICAgYW5kIGFyZSB0aGVy ZWZvcmUgc2lnbmFsZWQgdmlhIFRDUCBhbmQgbm90IHZpYSBSRE1BIGNvbXBsZXRpb24uCgoKNi4g IEVuaGFuY2VkIE1QQSBSZXF1ZXN0L1JlcGx5IEZyYW1lcwoKICAgRW5oYW5jZWQgUkRNQSBDb25u ZWN0aW9uIEVzdGFibGlzaG1lbnQgdXNlcyBhbiBhbHRlcm5hdGUgZm9ybWF0IGZvcgogICBNUEEg UmVxdWVzdHMgYW5kIFJlcGxpZXMsIGFzIGZvbGxvd3M6CgogICAgICAgMCAgICAgICAgICAgICAg ICAgICAxICAgICAgICAgICAgICAgICAgIDIgICAgICAgICAgICAgICAgICAgMwogICAgICAgIDAg MSAyIDMgNCA1IDYgNyA4IDkgMCAxIDIgMyA0IDUgNiA3IDggOSAwIDEgMiAzIDQgNSA2IDcgOCA5 IDAgMQogICAgICAgKy0rLSstKy0rLSstKy0rLSstKy0rLSstKy0rLSstKy0rLSstKy0rLSstKy0r LSstKy0rLSstKy0rLSstKy0rLSsKICAgIDAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAg ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB8CiAgICAgICArICAgICAgICAgS2V5 ICgxNiBieXRlcyBjb250YWluaW5nICJNUEEgSUQgUmVxIEZyYW1lIikgICAgICAgICAgKwogICAg NCAgfCAgICAgICg0RCA1MCA0MSAyMCA0OSA0NCAyMCA1MiA2NSA3MSAyMCA0NiA3MiA2MSA2RCA2 NSkgICAgICAgIHwKICAgICAgICsgICAgICAgICBPciAgKDE2IGJ5dGVzIGNvbnRhaW5pbmcgIk1Q QSBJRCBSZXAgRnJhbWUiKSAgICAgICAgICArCiAgICA4ICB8ICAgICAgKDREIDUwIDQxIDIwIDQ5 IDQ0IDIwIDUyIDY1IDcwIDIwIDQ2IDcyIDYxIDZEIDY1KSAgICAgICAgfAogICAgICAgKyAgICAg ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg ICsKICAgIDEyIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg ICAgICAgICAgICAgICAgICB8CiAgICAgICArLSstKy0rLSstKy0rLSstKy0rLSstKy0rLSstKy0r LSstKy0rLSstKy0rLSstKy0rLSstKy0rLSstKy0rLSstKwogICAgMTYgfE18Q3xSfFN8IFJlcyAg IHwgICAgIFJldiAgICAgICB8ICAgICAgICAgIFBEX0xlbmd0aCAgICAgICAgICAgIHwKICAgICAg ICstKy0rLSstKy0rLSstKy0rLSstKy0rLSstKy0rLSstKy0rLSstKy0rLSstKy0rLSstKy0rLSst Ky0rLSstKy0rCiAgICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg ICAgICAgICAgICAgICAgICAgICAgICAgfAogICAgICAgfiAgICAgICAgICAgICAgICAgICAgICAg ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIH4KICAgICAgIH4gICAgICAg ICAgICAgICAgICAgUHJpdmF0ZSBEYXRhICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB+ CiAgICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg ICAgICAgICAgICAgICAgfAogICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAr LSstKy0rLSstKy0rLSstKy0rLSstKy0rLSstKy0rLSsKICAgICAgIHwgICAgICAgICAgICAgICAg ICAgICAgICAgICAgICAgfAogICAgICAgKy0rLSstKy0rLSstKy0rLSstKy0rLSstKy0rLSstKy0r CgoKCgpLYW5ldnNreSwgZXQgYWwuICAgICAgICAgRXhwaXJlcyBPY3RvYmVyIDcsIDIwMTEgICAg ICAgICAgICAgICBbUGFnZSAxMF0KDApJbnRlcm5ldC1EcmFmdCAgIEVuaGFuY2VkIFJETUEgQ29u bmVjdGlvbiBFc3RhYmxpc2htZW50ICAgICAgIEFwcmlsIDIwMTEKCgogICBLZXk6ICBVbmNoYW5n ZWQgZnJvbSBbUkZDNTA0NF0uCgogICBNOiBVbmNoYW5nZWQgZnJvbSBbUkZDNTA0NF0uCgogICBD OiBVbmNoYW5nZWQgZnJvbSBbUkZDNTA0NF0uCgogICBSOiBVbmNoYW5nZWQgZnJvbSBbUkZDNTA0 NF0uCgogICBTOiBPbmUgaWYgdGhlIFByaXZhdGUgRGF0YSBiZWdpbnMgd2l0aCB0aGUgRW5oYW5j ZWQgUkRNQSBDb25uZWN0aW9uCiAgICAgIEVzdGFibGlzaG1lbnQgRGF0YS4gIFplcm8gb3RoZXJ3 aXNlLgoKICAgUmVzOiAgT25lIGJpdCBzbWFsbGVyIHRoYW4gaW4gW1JGQzUwNDRdLCBvdGhlcndp c2UgdW5jaGFuZ2VkLgoKICAgUmV2OiAgVGhpcyBmaWVsZCBjb250YWlucyB0aGUgcmV2aXNpb24g b2YgTVBBLiAgVG8gdXNlIGFueSBFbmhhbmNlZAogICAgICBDb25uZWN0aW9uIEVzdGFibGlzaG1l bnQgZmVhdHVyZSB0aGlzIE1VU1QgYmUgc2V0IHRvIHR3bywgSWYgbm8KICAgICAgRW5oYW5jZWQg Q29ubmVjdGlvbiBFc3RhYmxpc2htZW50IGZlYXR1cmVzIGFyZSBkZXNpcmVkIGl0IE1BWSBiZQog ICAgICBzZXQgdG8gb25lLiAgQSBob3N0IGFjY2VwdGluZyBNUEEgY29ubmVjdGlvbnMgTVVTVCBj b250aW51ZSB0bwogICAgICBhY2NlcHQgTVBBIFJlcXVlc3RzIHdpdGggdmVyc2lvbiBvbmUgZXZl biBpZiBpdCBzdXBwb3J0cyB2ZXJzaW9uCiAgICAgIHR3by4KCiAgIFBEX0xlbmd0aDogIFVuY2hh bmdlZCBmcm9tIFtSRkM1MDQ0XS4gIFRoaXMgaXMgdGhlIHRvdGFsIGxlbmd0aCBvZgogICAgICB0 aGUgUHJpdmF0ZSBEYXRhIGZpZWxkLCBpbmNsdWRpbmcgdGhlIEVuaGFuY2VkIFJETUEgQ29ubmVj dGlvbgogICAgICBFc3RhYmxpc2htZW50IERhdGEgaWYgcHJlc2VudC4KCiAgIFByaXZhdGUgRGF0 YTogIFVuY2hhbmdlZCBmcm9tIFtSRkM1MDQ0XS4gIEhvd2V2ZXIsIGlmIHRoZSAnUycgZmxhZyBp cwogICAgICBzZXQsIFByaXZhdGUgRGF0YSBiZWdpbnMgd2l0aCBFbmhhbmNlZCBSRE1BIENvbm5l Y3Rpb24KICAgICAgRXN0YWJsaXNobWVudCBEYXRhLgoKCjcuICBFbmhhbmNlZCBTQ1RQIFNlc3Np b24gQ29udHJvbCBDaHVua3MKCiAgIFRoZSB0eXBlIG9mIHRoZSBTQ1RQIFNlc3Npb24gQ29udHJv bCBDaHVuayBpcyBkZWZpbmVkIGJ5IGEgRnVuY3Rpb24KICAgQ29kZS4gIFtSRkM1MDQzXSBhbHJl YWR5IGRlZmluZXMgY29kZXMgZm9yICdERFAgU3RyZWFtIFNlc3Npb24KICAgSW5pdGlhdGUnIGFu ZCAnRERQIFN0cmVhbSBTZXNzaW9uIEFjY2VwdCcsIHdoaWNoIGFyZSBlcXVpdmFsZW50IHRvIGEK ICAgTVBBIFJlcXVlc3QgRnJhbWUgYW5kIGFuIGFjY2VwdGluZyBNUEEgUmVwbHkgRnJhbWUuCgog ICBFbmhhbmNlZCBSRE1BIENvbm5lY3Rpb24gRXN0YWJsaXNobWVudCByZXF1aXJlcyB0aHJlZSBh ZGRpdGlvbmFsCiAgIEZ1bmN0aW9uIGNvZGVzLiAgQWxsIEREUCBTdHJlYW0gU2Vzc2lvbiBGdW5j dGlvbmFsIENvZGVzIGFyZSBsaXN0ZWQKICAgYmVsb3c6CgogICBERFAgU3RyZWFtIFNlc3Npb24g SW5pdGlhdGU6ICAweDAwMQoKICAgRERQIFN0cmVhbSBTZXNzaW9uIEFjY2VwdDogIDB4MDAyCgog ICBERFAgU3RyZWFtIFNlc3Npb24gUmVqZWN0OiAgMHgwMDMKCgoKCgoKS2FuZXZza3ksIGV0IGFs LiAgICAgICAgIEV4cGlyZXMgT2N0b2JlciA3LCAyMDExICAgICAgICAgICAgICAgW1BhZ2UgMTFd CgwKSW50ZXJuZXQtRHJhZnQgICBFbmhhbmNlZCBSRE1BIENvbm5lY3Rpb24gRXN0YWJsaXNobWVu dCAgICAgICBBcHJpbCAyMDExCgoKICAgRERQIFN0cmVhbSBTZXNzaW9uIFRlcm1pbmF0ZTogIDB4 MDA0CgogICBFbmhhbmNlZCBERFAgU3RyZWFtIFNlc3Npb24gSW5pdGlhdGU6ICAweDAwNQoKICAg RW5oYW5jZWQgRERQIFN0cmVhbSBTZXNzaW9uIEFjY2VwdDogIDB4MDA2CgogICBFbmhhbmNlZCBE RFAgU3RyZWFtIFNlc3Npb24gUmVqZWN0OiAgMHgwMDcKCiAgIFRoZSBFbmhhbmNlZCBSZWplY3Qg ZnVuY3Rpb24gY29kZSBTSE9VTEQgYmUgdXNlZCB0byBpbmRpY2F0ZSBhCiAgIGNvbmZpZ3VyYXRp b24gdGhhdCB3b3VsZCBoYXZlIGJlZW4gYWNjZXB0ZWQuCgogICBUaGUgZXh0ZW5kZWQgRERQIHN0 cmVhbSBzZXNzaW9uIGVzdGFibGlzaG1lbnQgZm9sbG93cyB0aGUgc2FtZSBydWxlcwogICBhcyBy ZWd1bGFyIEREUCBzdHJlYW0gc2Vzc2lvbiBlc3RhYmxpc2htZW50IGFzIGRlZmluZWQgaW4gW1JG QzUwNDNdLgogICBVTFAtc3VwcGxpZWQgUHJpdmF0ZSBEYXRhIE1VU1QgYmUgaW5jbHVkZWQgZm9y IGV4dGVuZGVkIEREUCBTdHJlYW0KICAgU2Vzc2lvbiBJbml0aWF0ZSwgZXh0ZW5kZWQgRERQIFN0 cmVhbSBTZXNzaW9uIEFjY2VwdCwgYW5kIGV4dGVuZGVkCiAgIEREUCBTdHJlYW0gU2Vzc2lvbiBS ZWplY3QgbWVzc2FnZXMuICBIb3dldmVyLCB0aGUgVUxQIHN1cHBsaWVkCiAgIFByaXZhdGUgRGF0 YSBNQVkgYmUgb2YgemVybyBsZW5ndGguCgogICBQcml2YXRlIERhdGEgbGVuZ3RoIE1VU1QgTk9U IGV4Y2VlZCA1MTIgYnl0ZXMgaW4gYW55IG1lc3NhZ2UsCiAgIGluY2x1ZGluZyBlbmhhbmNlZCBS RE1BIGNvbm5lY3Rpb24gZXN0YWJsaXNobWVudCBkYXRhLgoKICAgUHJpdmF0ZSBEYXRhIE1VU1Qg Tk9UIGJlIGluY2x1ZGVkIGluIHRoZSBERFAgU3RyZWFtIFNlc3Npb24gVGVybWluYXRlCiAgIG1l c3NhZ2UuCgogICBSZWNlaXZlZCBleHRlbmRlZCBERFAgU3RyZWFtIFNlc3Npb24gQ29udHJvbCBt ZXNzYWdlcyBTSE9VTEQgYmUKICAgcmVwb3J0ZWQgdG8gdGhlIFVMUC4gIElmIHJlcG9ydGVkLCBh bnkgc3VwcGxpZWQgUHJpdmF0ZSBEYXRhIE1VU1QgYmUKICAgYXZhaWxhYmxlIGZvciB0aGUgVUxQ IHRvIGV4YW1pbmUuCgogICBUaGUgZW5oYW5jZWQgRERQIHN0cmVhbSBtYW5hZ2VtZW50IE1VU1Qg dXNlIEREUCBzdHJlYW0gc2Vzc2lvbgogICB0ZXJtaW5hdGlvbiBmdW5jdGlvbiBjb2RlIHRvIHRl cm1pbmF0ZSBhIHN0cmVhbSBlc3RhYmxpc2hlZCB1c2luZwogICBlbmhhbmNlZCBERFAgc3RyZWFt IHNlc3Npb24gZnVuY3Rpb24gY29kZXMuCgogICBJdCBzaG91bGQgYmUgbm90ZWQgdGhhdCBbUkZD NTA0M10gYWxyZWFkeSBzdXBwb3J0cyBlaXRoZXIgc2lkZQogICBzZW5kaW5nIHRoZSBmaXJzdCBE RFAgTWVzc2FnZS4gIFRoZSBQYXlsb2FkIFByb3RvY29sIElkZW50aWZpZXIKICAgKFBQSUQpIGFs cmVhZHkgZGlzdGluZ3Vpc2hlcyBiZXR3ZWVuIFNlc3Npb24gRXN0YWJsaXNobWVudCBhbmQgRERQ CiAgIFNlZ21lbnRzLgoKICAgVGhlIGZvbGxvd2luZyBhZGRpdGlvbmFsIExlZ2FsIFNlcXVlbmNl cyBvZiBERFAgU3RyZWFtIFNlc3Npb24KICAgbWVzc2FnZXMgYXJlIGRlZmluZWQ6CgogICBvICBF eHRlbmRlZCBBY3RpdmUvUGFzc2l2ZSBTZXNzaW9uIEFjY2VwdGVkOiBhcyB3aXRoIHNlY3Rpb24g Ni4yIG9mCiAgICAgIFtSRkM1MDQzXSwgYnV0IHdpdGggdGhlIGV4dGVuZGVkIG9wY29kZXMgYXMg ZGVmaW5lZCBpbiB0aGlzCiAgICAgIGRvY3VtZW50LgoKICAgbyAgRXh0ZW5kZWQgQWN0aXZlL1Bh c3NpdmUgU2Vzc2lvbiBSZWplY3RlZDogYXMgd2l0aCBzZWN0aW9uIDYuMyBvZgogICAgICBbUkZD NTA0M10sIGJ1dCB3aXRoIHRoZSBleHRlbmRlZCBvcGNvZGVzIGFzIGRlZmluZWQgaW4gdGhpcwog ICAgICBkb2N1bWVudC4KCgoKCkthbmV2c2t5LCBldCBhbC4gICAgICAgICBFeHBpcmVzIE9jdG9i ZXIgNywgMjAxMSAgICAgICAgICAgICAgIFtQYWdlIDEyXQoMCkludGVybmV0LURyYWZ0ICAgRW5o YW5jZWQgUkRNQSBDb25uZWN0aW9uIEVzdGFibGlzaG1lbnQgICAgICAgQXByaWwgMjAxMQoKCiAg IG8gIEV4dGVuZGVkIEFjdGl2ZS9QYXNzaXZlIFNlc3Npb24gTm9uLVVMUCBSZWplY3RlZDogYXMg d2l0aCBzZWN0aW9uCiAgICAgIDYuNCBvZiBbUkZDNTA0M10sIGJ1dCB3aXRoIHRoZSBleHRlbmRl ZCBvcGNvZGVzIGFzIGRlZmluZWQgaW4gdGhpcwogICAgICBkb2N1bWVudC4KCgo4LiAgTVBBIEVy cm9yIFJlcG9ydGluZwoKICAgVGhlIFtSRkM1MDQzXSBhbmQgW1JGQzUwNDRdIGRvIG5vdCBkZWZp bmUgZXJyb3IgY29kZXMuICBUaGUgcHJvdG9jb2wKICAgbGF5ZXJzIG9uIHdoaWNoIFJETUEgY29u bmVjdGlvbiBlc3RhYmxpc2htZW50IGlzIGxheWVyZWQgdXBvbgogICBbUkZDNTA0MF0gYW5kIFtS RkM1MDQxXSBkZWZpbmUgbGF5ZXJzIGFuZCBlcnJvciB0eXBlcy4gIFNwZWNpZmljYWxseSwKICAg TVBBIG5lZ290aWF0aW9uIGZvciBSRE1BIGNvbm5lY3Rpb24gZXN0YWJsaXNobWVudCB1c2VzOgoK ICAgTGF5ZXI6ICAgICAgMDAxMGIgLSBMTFAvTVBBCiAgIEVycm9yIFR5cGU6IDB4MyAgIC0gTExQ CgogICBUaGUgZm9sbG93aW5nIGVycm9yIGNvZGVzIGFyZSBkZWZpbmVkIGZvciBNUEEgbmVnb3Rp YXRpb246CgogICBFcnJvciBDb2RlICAgICAgICAgRGVzY3JpcHRpb24KICAgLS0tLS0tLS0tLS0t LS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0KICAgMHgxICAgICAg ICAgICAgICAgIExvY2FsIENhdGFzdHJvcGhpYwogICAweDIgICAgICAgICAgICAgICAgSW5zdWZm aWNpZW50IElSRCByZXNvdXJjZXMKCgo5LiAgRW5oYW5jZWQgUkRNQSBDb25uZWN0aW9uIEVzdGFi bGlzaG1lbnQgRGF0YQoKICAgICAgIDAgICAgICAgICAgICAgICAgICAgMSAgICAgICAgICAgICAg ICAgICAyICAgICAgICAgICAgICAgICAgIDMKICAgICAgICAwIDEgMiAzIDQgNSA2IDcgOCA5IDAg MSAyIDMgNCA1IDYgNyA4IDkgMCAxIDIgMyA0IDUgNiA3IDggOSAwIDEKICAgICAgICstKy0rLSst Ky0rLSstKy0rLSstKy0rLSstKy0rLSstKy0rLSstKy0rLSstKy0rLSstKy0rLSstKy0rLSstKy0r CiAgICAwICB8QXxCfCAgICAgICAgSVJEICAgICAgICAgICAgICAgIHxDfER8ICAgICAgICBPUkQg ICAgICAgICAgICAgICAgfAogICAgNCAgKy0rLSstKy0rLSstKy0rLSstKy0rLSstKy0rLSstKy0r LSstKy0rLSstKy0rLSstKy0rLSstKy0rLSstKy0rLSsKCgoKICAgSVJEOiAgSW4gcmVxdWVzdDog dGhlIEluaXRpYXRvciBpbml0aWFsIElSRCBmb3IgdGhlIGNvbm5lY3Rpb24uICBJbgogICAgICBy ZXBseTogdGhlIGRlcHRoIHRoZSBSZXNwb25kZXIgd2lsbCBzdXBwb3J0LiAgUmVzcG9uZGVyIFNI T1VMRCBOT1QKICAgICAgc2V0IGl0cyBJUkQgaGlnaGVyIHRoYW4gSW5pdGlhdG9yIE9SRC4gIFVw b24gcmVjZWl2aW5nIGFjY2VwdAogICAgICBjb25uZWN0aW9uIG1lc3NhZ2UgZnJvbSB0aGUgUmVz cG9uZGVyLCB0aGUgSW5pdGlhdG9yIE1VU1Qgc2V0IGl0cwogICAgICBPUkQgdG8gdGhlIHJlc3Bv bmRlciBJUkQuICBCb3RoIFJlc3BvbmRlciBhbmQgSW5pdGlhdG9yIE1VU1QgcGFzcwogICAgICB0 aGUgcmVtb3RlIHNpZGUgcHJvdmlkZWQgSVJEIHRvIFVMUC4gIEFuIGFsbCBvbmVzIHZhbHVlICgw eDNGRkYpCiAgICAgIGluZGljYXRlcyB0aGF0IGF1dG9tYXRpYyBuZWdvdGlhdGlvbiBvZiB0aGUg SVJEIGlzIG5vdCBkZXNpcmVkLAogICAgICBhbmQgdGhhdCB0aGUgVUxQIHdpbGwgYmUgcmVzcG9u c2libGUgZm9yIGRvaW5nIHRoaXMgY29uZmlndXJhdGlvbi4KCiAgIE9SRDogIEluIHJlcXVlc3Q6 IHRoZSBJbml0aWF0b3IgaW5pdGlhbCBPUkQgc2V0dGluZyBmb3IgdGhlCiAgICAgIGNvbm5lY3Rp b24uICBJbiByZXBseTogdGhlIGRlcHRoIHRoZSBSZXNwb25kZXIgd2lsbCBzdXBwb3J0LgogICAg ICBSZXNwb25kZXIgU0hPVUxEIHNldCBpdHMgT1JEIHRvIGEgdmFsdWUgdGhhdCBpcyBsZXNzIHRo YW4gb3IgZXF1YWwKICAgICAgdG8gdGhlIHJlcXVlc3RlZCBJbml0aWF0b3IgSVJELiAgVXBvbiBy ZWNlaXZpbmcgYWNjZXB0IGNvbm5lY3Rpb24KICAgICAgZnJvbSB0aGUgUmVzcG9uZGVyLCB0aGUg SW5pdGlhdG9yIE1VU1Qgc2V0IGl0cyBJUkQgdG8gYSB2YWx1ZSBhdAogICAgICBsZWFzdCBhcyBs YXJnZSBhcyB0aGUgcmVzcG9uZGVyIE9SRCBpZiBpdCBoYXMgc3VmZmljaWVudCByZXNvdXJjZXMK CgoKS2FuZXZza3ksIGV0IGFsLiAgICAgICAgIEV4cGlyZXMgT2N0b2JlciA3LCAyMDExICAgICAg ICAgICAgICAgW1BhZ2UgMTNdCgwKSW50ZXJuZXQtRHJhZnQgICBFbmhhbmNlZCBSRE1BIENvbm5l Y3Rpb24gRXN0YWJsaXNobWVudCAgICAgICBBcHJpbCAyMDExCgoKICAgICAgZm9yIGl0LiAgSWYg dGhlIEluaXRpYXRvciBkb2VzIG5vdCBoYXZlIHN1ZmZpY2llbnQgcmVzb3VyY2VzIHRvCiAgICAg IHNhdGlzZnkgdGhlIFJlc3BvbmRlciBPUkQgaXQgTVVTVCB0ZXJtaW5hdGUgdGhlIGNvbm5lY3Rp b24gYW5kCiAgICAgIHJlcG9ydCB0byBsb2NhbCBVTFAgdGhlIFJlc3BvbmRlciBPUkQgYW5kIElS RCB3aXRoIGFuIGluZGljYXRvciBvZgogICAgICBpbnN1ZmZpY2llbnQgcmVzb3VyY2VzIHRvIHNh dGlzZnkgUmVzcG9uZGVyIE9SRCwgYW5kIE1VU1Qgc2VuZAogICAgICB0ZXJtaW5hdGlvbiBtZXNz YWdlIHRvIHRoZSBSZXNwb25kZXIgaW5kaWNhdGluZyBpbnN1ZmZpY2llbnQKICAgICAgcmVzb3Vy Y2VzLiAgVGh1cywgdGhlIFRFUk0gbWVzc2FnZSBNVVNUIGNvbnRhaW4gTGF5ZXIgMiwgRXJyb3IK ICAgICAgVHlwZSAzLCBFcnJvciBDb2RlIDIuICBCb3RoIFJlc3BvbmRlciBhbmQgSW5pdGlhdG9y IE1VU1QgcGFzcyB0aGUKICAgICAgcmVtb3RlIHNpZGUgcHJvdmlkZWQgT1JEIHRvIFVMUC4gIEFu IGFsbCBvbmVzIHZhbHVlICgweDNGRkYpCiAgICAgIGluZGljYXRlcyB0aGF0IGF1dG9tYXRpYyBu ZWdvdGlhdGlvbiBvZiB0aGUgT1JEIGlzIG5vdCBkZXNpcmVkLAogICAgICBhbmQgdGhhdCB0aGUg VUxQIHdpbGwgYmUgcmVzcG9uc2libGUgZm9yIGRvaW5nIHRoaXMgY29uZmlndXJhdGlvbi4KCiAg IEE6IENvbnRyb2wgRmxhZyBmb3IgdXNpbmcgYSB6ZXJvIGxlbmd0aCBGVUxQRFUgYXMgdGhlIFJU UiBtZXNzYWdlLgoKICAgQjogQ29udHJvbCBGbGFnIGZvciB1c2luZyBhIHplcm8gbGVuZ3RoIFJE TUEgV3JpdGUgYXMgdGhlIFJUUgogICAgICBtZXNzYWdlLgoKICAgQzogQ29udHJvbCBGbGFnIGZv ciB1c2luZyBhIHplcm8gbGVuZ3RoIFJETUEgUmVhZCBhcyB0aGUgUlRSIG1lc3NhZ2UuCgogICBE OiBSZXNlcnZlZC4gIE11c3QgYmUgc2VudCBhcyB6ZXJvIGFuZCBpZ25vcmVkIHdoZW4gcmVjZWl2 ZWQuCgogICBJbiB0aGUgTVBBIFJlcXVlc3QsIHRoZSBJbml0aWF0b3IgU0hPVUxEIHNldCB0aGUg QSwgQiBhbmQgQyBDb250cm9sCiAgIEZsYWdzIHJlc3BlY3RpdmVseSB0byBUUlVFIGZvciBlYWNo IG9mIHRoZSBvcHRpb25zIGl0IHN1cHBvcnRzLgoKICAgSW4gdGhlIE1QQSBSZXBseSwgdGhlIENv bnRyb2wgRmxhZyBpcyBzZXQgZm9yIHRoZSBzZXQgb2Ygb3B0aW9ucyB0aGF0CiAgIHRoZSBwYXNz aXZlIHNpZGUgd2lsbCBhY2NlcHQgYXMgYW4gUlRSIG1lc3NhZ2UuICBUaGlzIHJlc3BvbnNlIE1V U1QKICAgaW5jbHVkZSBhbGwgb3B0aW9ucyB0aGF0IHRoZSByZXNwb25kZXIgd2lsbCBzdXBwb3J0 IHdpdGhvdXQgcmVxdWlyaW5nCiAgIGEgY29ubmVjdGlvbiBzcGVjaWZpYyBlbmFibGluZyBhY3Rp b24uICBGb3IgZXhhbXBsZSwgaWYgdGhlIHJlc3BvbmRlcgogICB3aWxsIGFsd2F5cyB1bmJsb2Nr IGFuIE1QQSBjb25uZWN0aW9uIHdoZW4gaXQgcmVjZWl2ZXMgYSB6ZXJvIGxlbmd0aAogICBNUEEg V3JpdGUsIGl0IHNob3VsZCBpbmRpY2F0ZSBzbyB3aXRob3V0IHJlZ2FyZCB0byB3aGF0IHdhcyBp biB0aGUKICAgTVBBIFJlcXVlc3QuICBPcHRpb25zIHdoaWNoIHJlcXVpcmUgY29ubmVjdGlvbiBz cGVjaWZpYyBlbmFibGluZwogICBhY3Rpb25zIFNIT1VMRCBOT1QgYmUgc2V0IHVubGVzcyB0aGUg Y29ycmVzcG9uZGluZyBmbGFnIHdhcyBzZXQgaW4KICAgdGhlIE1QQSBSZXF1ZXN0LiAgVGhlIHJl c3BvbmRlciBNQVkgY2hvb3NlIHRvIGxpbWl0IHRoZSBudW1iZXIgb2YKICAgbW9kZXMgdGhhdCBp dCBlbmFibGVzLgoKICAgSWYgdGhlcmUgaXMgbm8gU3RhbmRhcmQgUkRNQVAgQ29uZmlndXJhdGlv biBEYXRhIGluIHRoZSBNUEEgUmVwbHkKICAgRnJhbWUsIGFuZCB0aGUgRW5oYW5jZWQgQ29ubmVj dGlvbiBFc3RhYmxpc2htZW50IHZlcnNpb24gbnVtYmVyIGlzCiAgIHVzZWQsIGl0IGlzIHRoZSBl cXVpdmFsZW50IG9mIHNldHRpbmcgJ0EnLCAnQicgYW5kICdDJy4KCiAgIFNldHRpbmcgbm8gQ29u dHJvbCBGbGFncyBpbiB0aGUgTVBBIFJlcGx5IGluZGljYXRlcyB0aGF0IGFuIFJETUEgU2VuZAog ICBtZXNzYWdlIHdpbGwgYmUgcmVxdWlyZWQuICBBcyB0aGlzIG9wdGlvbiB3aWxsIHJlcXVpcmUg dGhlIGluaXRpYXRvcgogICBVTFAgdG8gYmUgaW52b2x2ZWQgaXQgU0hPVUxEIE5PVCBiZSB1c2Vk IHVubGVzcyBuZWNlc3NhcnkuCgogICBUaGUgcGVlci10by1wZWVyIG5lZ290aWF0aW9uIGZvciB0 aGUgUlRSIG1lc3NhZ2UgZm9sbG93cyB0aGUKICAgZm9sbG93aW5nIG9yZGVyOgoKCgoKCgoKS2Fu ZXZza3ksIGV0IGFsLiAgICAgICAgIEV4cGlyZXMgT2N0b2JlciA3LCAyMDExICAgICAgICAgICAg ICAgW1BhZ2UgMTRdCgwKSW50ZXJuZXQtRHJhZnQgICBFbmhhbmNlZCBSRE1BIENvbm5lY3Rpb24g RXN0YWJsaXNobWVudCAgICAgICBBcHJpbCAyMDExCgoKICAgSW5pdGlhdG9yIC0tPjogU2V0cyBD b250cm9sIEZsYWdzIHRvIGluZGljYXRlIEluaXRpYXRvci1zdXBwb3J0ZWQKICAgZm9ybXMgb2Yg UlRSCgogICBSZXNwb25kZXIgPC0tOiBTZXRzIENvbnRyb2wgRmxhZ3MgdG8gaW5kaWNhdGUgUmVz cG9uZGVyLXN1cHBvcnRlZAogICBmb3JtcyBvZiBSVFIKCiAgIEluaXRpYXRvciAtLT46IElmIGF0 IGxlYXN0IG9uZSBmb3JtIG9mIFJUUiBpcyBzdXBwb3J0ZWQgYnkgYm90aAogICBJbml0aWF0b3Ig YW5kIFJlc3BvbmRlciwgdGhlbiB0aGUgZmlyc3QgbWVzc2FnZSBzZW50IE1VU1QgYmUgYW4gUlRS CiAgIHVzaW5nIGEgZm9ybSBzdXBwb3J0ZWQgYnkgYm90aCB0aGUgSW5pdGlhdG9yIGFuZCBSZXNw b25kZXIuCgogICBJbml0aWF0b3Igb3IgUmVzcG9uZGVyIFNIT1VMRCBnZW5lcmF0ZSB0aGUgVEVS TSBtZXNzYWdlIHRoYXQgY29udGFpbnMKICAgTGF5ZXIgMiwgRXJyb3IgVHlwZSAzLCBFcnJvciBD b2RlIDEgd2hlbiBpdCBlbmNvdW50ZXJzIGFueSBlcnJvcgogICBsb2NhbGx5IGZvciB3aGljaCB0 aGUgc3BlY2lhbCBFcnJvciBDb2RlIGlzIG5vdCBkZWZpbmVkIGluIHNlY3Rpb24KICAgU2VjdGlv biA4IGJlZm9yZSByZXNldHRpbmcgdGhlIGNvbm5lY3Rpb24uCgoKMTAuICBJbnRlcm9wZXJhYmls aXR5CgogICBBbiBpbml0aWF0b3IgU0hPVUxEIE5PVCB1c2UgdGhlIEVuaGFuY2VkIEREUCBDb25u ZWN0aW9uIEVzdGFibGlzaG1lbnQKICAgZm9ybWF0cyBvciBmdW5jdGlvbiBjb2RlcyB3aGVuIG5v IGVuaGFuY2VkIGZ1bmN0aW9uYWxpdHkgaXMgZGVzaXJlZC4KCiAgIEEgcmVzcG9uZGVyIE1VU1Qg Y29udGludWUgdG8gYWNjZXB0IHRoZSB1bmVuaGFuY2VkIGNvbm5lY3Rpb24KICAgcmVxdWVzdHMu CgogICBUaGVyZSBhcmUgdGhyZWUgSW5pdGlhdG9yL1Jlc3BvbmRlciBjYXNlcyB0aGF0IGludm9s dmUgZW5oYW5jZWQgTVBBOgogICBib3RoIGluaXRpYXRvciBhbmQgcmVzcG9uZGVyLCBvbmx5IHJl c3BvbmRlciwgYW5kIG9ubHkgaW5pdGlhdG9yLgogICBUaGUgZW5oYW5jZWQgTVBBIGZyYW1lIGlz IGRlZmluZWQgYnkgZmllbGQgJ1MnIHNldCB0byAxLgoKICAgRW5oYW5jZWQgTVBBIEluaXRpYXRv ciBhbmQgUmVzcG9uZGVyOiAgSWYgYSByZXNwb25kZXIgcmVjZWl2ZXMgYW4KICAgICAgZW5oYW5j ZWQgTVBBIG1lc3NhZ2UsIGl0IE1VU1QgcmVzcG9uZCB3aXRoIGFuIGVuaGFuY2VkIE1QQQogICAg ICBtZXNzYWdlLgoKICAgRW5oYW5jZWQgTVBBIFJlc3BvbmRlciBvbmx5OiAgSWYgYSByZXNwb25k ZXIgcmVjZWl2ZXMgYW4gdW5lbmhhbmNlZAogICAgICBNUEEgbWVzc2FnZSAoJ1MnIGlzIHNldCB0 byAwKSwgaXQgTVVTVCByZXNwb25kIHdpdGggYW4gdW5lbmhhbmNlZAogICAgICBNUEEgbWVzc2Fn ZS4KCiAgIEVuaGFuY2VkIE1QQSBJbml0aWF0b3Igb25seTogIElmIGEgcmVzcG9uZGVyIGRvZXMg bm90IHN1cHBvcnQKICAgICAgcmVjZWl2ZWQgZXh0ZW5kZWQgTVBBIG1lc3NhZ2UsIHRoZW4gaXQg TVVTVCBjbG9zZSB0aGUgVENQCiAgICAgIGNvbm5lY3Rpb24gYW5kIGV4aXQgTVBBIHNpbmNlIE1Q QSBmcmFtZSBpcyBpbXByb3Blcmx5IGZvcm1hdHRlZAogICAgICBmb3IgaXQgYXMgc3RhdGVkIGlu IFtSRkM1MDQ0XS4gIFRodXMsIGJvdGggaW5pdGlhdG9yIGFuZCByZXNwb25kZXIKICAgICAgcmVw b3J0IFRDUCBjb25uZWN0aW9uIHRlcm1pbmF0aW9uIHRvIGFuIGFwcGxpY2F0aW9uIGxvY2FsbHku ICBJbgogICAgICB0aGlzIGNhc2UgaW5pdGlhdG9yIE1BWSBhdHRlbXB0IHRvIGVzdGFibGlzaCBS RE1BIGNvbm5lY3Rpb24gdXNpbmcKICAgICAgdW5lbmhhbmNlZCBNUEEgcHJvdG9jb2wgYXMgZGVm aW5lZCBpbiBbUkZDNTA0NF0gaWYgdGhpcyBwcm90b2NvbAogICAgICBpcyBjb21wYXRpYmxlIHdp dGggdGhlIGFwcGxpY2F0aW9uLCBhbmQgbGV0IFVMUCBkZWFsIHdpdGggT1JEIGFuZAogICAgICBJ UkQsIGFuZCBwZWVyLXRvLXBlZXIgbmVnb3RpYXRpb25zLgoKCgoKCgpLYW5ldnNreSwgZXQgYWwu ICAgICAgICAgRXhwaXJlcyBPY3RvYmVyIDcsIDIwMTEgICAgICAgICAgICAgICBbUGFnZSAxNV0K DApJbnRlcm5ldC1EcmFmdCAgIEVuaGFuY2VkIFJETUEgQ29ubmVjdGlvbiBFc3RhYmxpc2htZW50 ICAgICAgIEFwcmlsIDIwMTEKCgoxMS4gIElBTkEgQ29uc2lkZXJhdGlvbnMKCiAgIFRoaXMgZG9j dW1lbnQgaGFzIG5vIElBTkEgY29uc2lkZXJhdGlvbnMuCgoKMTIuICBTZWN1cml0eSBDb25zaWRl cmF0aW9ucwoKICAgVGhlIHNlY3VyaXR5IGNvbnNpZGVyYXRpb25zIGZyb20gUkZDIDUwNDQgYXBw bHkgYW5kIHRoZSBjaGFuZ2VzIGluCiAgIHRoaXMgZG9jdW1lbnQgZG8gbm90IGludHJvZHVjZSBu ZXcgc2VjdXJpdHkgY29uc2lkZXJhdGlvbnMuCgoKMTMuICBBY2tub3dsZWRnZW1lbnRzCgogICBU aGUgYXV0aG9ycyB3aXNoIHRvIHRoYW5rIFNlYW4gSGVmdHksIERhdmUgTWludHVybiwgVG9tIFRh bHBleSBhbmQKICAgRGF2aWQgQmxhY2sgZm9yIHRoZWlyIHZhbHVhYmxlIGNvbnRyaWJ1dGlvbnMg YW5kIHJldmlld3Mgb2YgdGhpcwogICBkb2N1bWVudC4KCgoxNC4gIFJlZmVyZW5jZXMKCjE0LjEu ICBOb3JtYXRpdmUgUmVmZXJlbmNlcwoKICAgW1JGQzIxMTldICBCcmFkbmVyLCBTLiwgIktleSB3 b3JkcyBmb3IgdXNlIGluIFJGQ3MgdG8gSW5kaWNhdGUKICAgICAgICAgICAgICBSZXF1aXJlbWVu dCBMZXZlbHMiLCBCQ1AgMTQsIFJGQyAyMTE5LCBNYXJjaCAxOTk3LgoKICAgW1JGQzUwNDBdICBS ZWNpbywgUi4sIE1ldHpsZXIsIEIuLCBDdWxsZXksIFAuLCBIaWxsYW5kLCBKLiwgYW5kIEQuCiAg ICAgICAgICAgICAgR2FyY2lhLCAiQSBSZW1vdGUgRGlyZWN0IE1lbW9yeSBBY2Nlc3MgUHJvdG9j b2wKICAgICAgICAgICAgICBTcGVjaWZpY2F0aW9uIiwgUkZDIDUwNDAsIE9jdG9iZXIgMjAwNy4K CiAgIFtSRkM1MDQxXSAgU2hhaCwgSC4sIFBpbmtlcnRvbiwgSi4sIFJlY2lvLCBSLiwgYW5kIFAu IEN1bGxleSwgIkRpcmVjdAogICAgICAgICAgICAgIERhdGEgUGxhY2VtZW50IG92ZXIgUmVsaWFi bGUgVHJhbnNwb3J0cyIsIFJGQyA1MDQxLAogICAgICAgICAgICAgIE9jdG9iZXIgMjAwNy4KCiAg IFtSRkM1MDQzXSAgQmVzdGxlciwgQy4gYW5kIFIuIFN0ZXdhcnQsICJTdHJlYW0gQ29udHJvbCBU cmFuc21pc3Npb24KICAgICAgICAgICAgICBQcm90b2NvbCAoU0NUUCkgRGlyZWN0IERhdGEgUGxh Y2VtZW50IChERFApIEFkYXB0YXRpb24iLAogICAgICAgICAgICAgIFJGQyA1MDQzLCBPY3RvYmVy IDIwMDcuCgogICBbUkZDNTA0NF0gIEN1bGxleSwgUC4sIEVsenVyLCBVLiwgUmVjaW8sIFIuLCBC YWlsZXksIFMuLCBhbmQgSi4KICAgICAgICAgICAgICBDYXJyaWVyLCAiTWFya2VyIFBEVSBBbGln bmVkIEZyYW1pbmcgZm9yIFRDUAogICAgICAgICAgICAgIFNwZWNpZmljYXRpb24iLCBSRkMgNTA0 NCwgT2N0b2JlciAyMDA3LgoKMTQuMi4gIEluZm9ybWF0aXZlIFJlZmVyZW5jZXMKCiAgIFtEQVBM XSAgICAgIkRpcmVjdCBBY2Nlc3MgUHJvZ3JhbW1pbmcgTGlicmFyeSIsCiAgICAgICAgICAgICAg PGh0dHA6Ly93d3cuZGF0Y29sbGFib3JhdGl2ZS5vcmc+LgoKICAgW09GQV0gICAgICAiT0ZBIHZl cmJzICYgQVBJcyIsIDxodHRwOi8vd3d3Lm9wZW5mYWJyaWNzLm9yZy8+LgoKCgoKS2FuZXZza3ks IGV0IGFsLiAgICAgICAgIEV4cGlyZXMgT2N0b2JlciA3LCAyMDExICAgICAgICAgICAgICAgW1Bh Z2UgMTZdCgwKSW50ZXJuZXQtRHJhZnQgICBFbmhhbmNlZCBSRE1BIENvbm5lY3Rpb24gRXN0YWJs aXNobWVudCAgICAgICBBcHJpbCAyMDExCgoKICAgW09wZW5NUF0gICBNY0dyYXctSGlsbCwgIlBh cmFsbGVsIFByb2dyYW1taW5nIGluIEMgd2l0aCBNUEkgYW5kCiAgICAgICAgICAgICAgT3Blbk1Q IiwgMjAwMy4KCiAgIFtQUE1QSV0gICAgTW9yZ2FuIEthdWZtYW5uIFB1Ymxpc2hlcnMgSW5jLiwg IlBhcmFsbGVsIFByb2dyYW1taW5nCiAgICAgICAgICAgICAgd2l0aCBNUEkiLCAyMDA4LgoKICAg W1JETUFDXSAgICAiUkRNQSBQcm90b2NvbCBWZXJicyBTcGVjaWZpY2F0aW9uIChWZXJzaW9uIDEu MCkiLCA8aHR0cDovCiAgICAgICAgICAgICAgL3d3dy5yZG1hY29uc29ydGl1bS5vcmcvaG9tZS8K ICAgICAgICAgICAgICBkcmFmdC1oaWxsYW5kLWl3YXJwLXZlcmJzLXYxLjAtUkRNQS5wZGY+LgoK ICAgW1JEU10gICAgICBPcGVuIEZhYnJpY3MgQXNzb2NpYXRpb24sICJSZWxpYWJsZSBEYXRhZ3Jh bSBTb2NrZXQiLAogICAgICAgICAgICAgIDIwMDgsIDxodHRwOi8vd3d3Lm9wZW5mYWJyaWNzLm9y Zy9hcmNoaXZlcy8KICAgICAgICAgICAgICBzcHJpbmcyMDA4c29ub21hL1R1ZXNkYXkvc29ub21h XzIwMDhfMDQwOCUyME9yYWNsZS5wcHQ+LgoKICAgW1VzaW5nTVBJXQogICAgICAgICAgICAgIE1J VCBQcmVzcywgIlVzaW5nIE1QSS0yOiBBZHZhbmNlZCBGZWF0dXJlcyBvZiB0aGUgTWVzc2FnZQog ICAgICAgICAgICAgIFBhc3NpbmcgSW50ZXJmYWNlIiwgMTk5OS4KCiAgIFtWSUFdICAgICAgQ29t cGFxLCBJbnRlbCwgTWljcm9zb2Z0LCAiVmlydHVhbCBJbnRlcmZhY2UgQXJjaGl0ZWN0dXJlCiAg ICAgICAgICAgICAgU3BlY2lmaWNhdGlvbiIsIDE5OTcsIDxodHRwOi8vcGxsYWIuY3MubnRodS5l ZHUudHcvY3M1NDAzLwogICAgICAgICAgICAgIFJlYWRpbmdzL0VKQi9zYW5fMTAucGRmPi4KCgpB dXRob3JzJyBBZGRyZXNzZXMKCiAgIEFya2FkeSBLYW5ldnNreSAoZWRpdG9yKQogICBWTXdhcmUK ICAgNSBDYW1icmlkZ2UgQ2VudGVyCiAgIENhbWJyaWRnZSwgTUEgIDAyMTQyCiAgIFVTQQoKICAg UGhvbmU6ICsxLTYxNy01MjgtNzcyMQogICBFbWFpbDogYXJrYWR5QHZtd2FyZS5jb20KCgogICBD YWl0bGluIEJlc3RsZXIgKGVkaXRvcikKICAgQ29uc3VsdGFudAogICA1NTUgRSBFbCBDYW1pbm8g UmVhbCAjMTA0CiAgIFN1bm55dmFsZSwgQ0EgIDk0MDg3CiAgIFVTQQoKICAgUGhvbmU6ICsxLTk0 OS01MjgtMzA4NQogICBFbWFpbDogY2FpdEBhc29taS5jb20KCgoKCgoKCgpLYW5ldnNreSwgZXQg YWwuICAgICAgICAgRXhwaXJlcyBPY3RvYmVyIDcsIDIwMTEgICAgICAgICAgICAgICBbUGFnZSAx N10KDApJbnRlcm5ldC1EcmFmdCAgIEVuaGFuY2VkIFJETUEgQ29ubmVjdGlvbiBFc3RhYmxpc2ht ZW50ICAgICAgIEFwcmlsIDIwMTEKCgogICBSb2JlcnQgU2hhcnAKICAgSW50ZWwKICAgTEFEIEhp Z2ggUGVyZm9ybWFuY2UgTWVzc2FnZSBQYXNzaW5nLCBNYWlsc3RvcDogQU4xLVdUUjEKICAgMTUw MSBTb3V0aCBNb3BhYywgU3VpdGUgNDAwCiAgIEF1c3RpbiwgVFggIDc4NzQ2CiAgIFVTQQoKICAg UGhvbmU6ICsxLTUxMi00OTMtMzI0MgogICBFbWFpbDogcm9iZXJ0Lm8uc2hhcnBAaW50ZWwuY29t CgoKICAgU3RldmUgV2lzZQogICBPcGVuIEdyaWQgQ29tcHV0aW5nCiAgIDQwMzAgQnJha2VyIExh bmUgU1RFIDEzMAogICBBdXN0aW4sIFRYICA3ODc1OQogICBVU0EKCiAgIFBob25lOiArMS01MTIt MzQzLTkxOTYgeDEwMQogICBFbWFpbDogc3dpc2VAb3BlbmdyaWRjb21wdXRpbmcuY29tCgoKCgoK CgoKCgoKCgoKCgoKCgoKCgoKCgoKCgoKCgoKS2FuZXZza3ksIGV0IGFsLiAgICAgICAgIEV4cGly ZXMgT2N0b2JlciA3LCAyMDExICAgICAgICAgICAgICAgW1BhZ2UgMThdCgwK --00504502cc56bb471204a0221808-- From swise@opengridcomputing.com Mon Apr 4 19:16:46 2011 Return-Path: X-Original-To: storm@core3.amsl.com Delivered-To: storm@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id 492573A682E for ; Mon, 4 Apr 2011 19:16:46 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.264 X-Spam-Level: X-Spam-Status: No, score=-2.264 tagged_above=-999 required=5 tests=[BAYES_00=-2.599, HTML_MESSAGE=0.001, IP_NOT_FRIENDLY=0.334] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id MEvJuuyT3hsH for ; Mon, 4 Apr 2011 19:16:44 -0700 (PDT) Received: from linode.aoot.com (linode.aoot.com [69.164.194.13]) by core3.amsl.com (Postfix) with ESMTP id C69913A6821 for ; Mon, 4 Apr 2011 19:16:43 -0700 (PDT) Received: from [172.16.0.97] (pool-71-114-244-108.austtx.dsl-w.verizon.net [71.114.244.108]) by linode.aoot.com (Postfix) with ESMTP id D0DDA8223; Mon, 4 Apr 2011 21:18:25 -0500 (CDT) Message-ID: <4D9A7BF7.3040201@opengridcomputing.com> Date: Mon, 04 Apr 2011 21:18:31 -0500 From: Steve Wise User-Agent: Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US; rv:1.9.2.15) Gecko/20110303 Lightning/1.0b2 Thunderbird/3.1.9 MIME-Version: 1.0 To: arkady kanevsky References: <7C4DFCE962635144B8FAE8CA11D0BF1E03D74C7183@MX14A.corp.emc.com> <7C4DFCE962635144B8FAE8CA11D0BF1E03E5E9530C@MX14A.corp.emc.com> <76DBE161893C324BA6D4BB507EE4C3849510935E96@IRVEXCHCCR02.corp.ad.broadcom.com> <4D98E86A.60403@opengridcomputing.com> <4D9A733B.9050503@opengridcomputing.com> In-Reply-To: Content-Type: multipart/alternative; boundary="------------030703020404010007080804" X-Mailman-Approved-At: Tue, 05 Apr 2011 08:26:32 -0700 Cc: storm@ietf.org Subject: Re: [storm] MPA Draft - Review X-BeenThere: storm@ietf.org X-Mailman-Version: 2.1.9 Precedence: list List-Id: Storage Maintenance WG List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 05 Apr 2011 02:16:46 -0000 This is a multi-part message in MIME format. --------------030703020404010007080804 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Looks good. Bob Sharp should ACK this too though. Thanks, Steve. On 4/4/2011 8:50 PM, arkady kanevsky wrote: > Got it. Thanks Steve. > How about now? > Arkady > > On Mon, Apr 4, 2011 at 9:41 PM, Steve Wise > > wrote: > > The key word to remove here is "responder". The IRD in the MPA > start request is the initiators desired IRD _of the initiator_. > Not the initiators desired _responder_ IRD. > > If you want to change "desiired" to "initial" thats ok with me. > But the key is to get rid of the word "responder" in that sentence. > > Steve. > > > On 4/4/2011 6:26 PM, arkady kanevsky wrote: >> Steve and Bob, >> I changed it to >> "In request: the Initiator desired responder IRD >> for the connection." as you asked. >> I can change it to "initial" instead of "desired". >> Arkady >> >> On Sun, Apr 3, 2011 at 5:36 PM, Steve Wise >> > > wrote: >> >> Hey Arkady, >> >> It does seem like you did the section 9 changes Bob and I >> requested: >> >> ---- >> >> Change the IRD definition on the request from "In request: >> the Initiator requested responder IRD for the >> connection." to "In request: the Initiator initial IRD >> setting for the connection." >> ---- >> >> Thanks, >> >> Steve. >> >> >> On 4/2/2011 8:35 PM, arkady kanevsky wrote: >>> All, >>> updated version 04 is attached. >>> >>> Hemal, >>> Thanks for catching it. >>> I had fixed the first issue. I had added reference to FPDU >>> in the FULPDU definition for the second. >>> >>> David, >>> Please, check to see that you comments are addressed. >>> >>> Steve and Robert, >>> please, check that you comment is fixed correctly. >>> >>> Once I get positive feedback from all of you, I will submit >>> the version. >>> >>> Thanks, >>> Arkady >>> >>> On Thu, Mar 31, 2011 at 2:43 PM, Hemal Shah >>> > wrote: >>> >>> I have some comments on -03 draft: >>> >>> 1. In section 10, it is written that "Enhanced MPA >>> Initiator and Responder: If a responder receives >>> an enhanced MPA message, it MUST respond with an >>> unenhanced MPA message." I think it should be >>> written that the responder must respond with an >>> enhanced MPA message. It appears like a typo to me. >>> 2. I find the use of FULPDU confusing in this draft. >>> RFC5044 does not define term FULPDU. RFC5044 uses >>> term FPDU to refer to Framed Protocol Data Unit. I >>> suggest that we use term FPDU instead of FULPDU in >>> the draft. >>> >>> Hemal >>> -----Original Message----- >>> From: storm-bounces@ietf.org >>> >>> [mailto:storm-bounces@ietf.org] On Behalf Of >>> david.black@emc.com >>> Sent: Thursday, March 31, 2011 7:48 AM >>> To: storm@ietf.org >>> Subject: Re: [storm] MPA Draft - Review >>> Importance: High >>> The -03 version of the MPA draft has addressed all of >>> the issues from my review, and . Unfortunately, I need >>> some minor edits for clarity before I can send this on >>> to our AD with a publication request. Would the authors >>> please submit a -04 version with the following two >>> changes quickly. >>> Section 9 (end) >>> OLD >>> The peer-to-peer negotiation for the RTR message >>> follows the >>> following order: >>> Initiator -->: Sets Control Flags it is capable to >>> send for RTR >>> Responder <--: Sets Control Flags it is capable to >>> receive for RTR >>> Initiator -->: The first message send MUST be a >>> negotiated RTR >>> NEW >>> The peer-to-peer negotiation for the RTR message >>> follows the >>> following order: >>> Initiator -->: Sets Control Flags to indicate >>> Initiator-supported forms of RTR >>> Responder <--: Sets Control Flags to indicate >>> Responder-supported forms of RTR >>> Initiator -->: If at least one form of RTR is >>> supported by both Initiator and >>> Responder, then the first message sent MUST be >>> an RTR using a form supported >>> by both the Initiator and Responder. >>> Section 10 >>> OLD >>> In >>> this case initiator CAN attempt to establish RDMA >>> connection using >>> unenhanced MPA protocol as defined in [RFC5044] >>> and let ULP deal >>> with ORD and IRD, and peer-to-peer negotiations. >>> NEW >>> In >>> this case initiator MAY attempt to establish RDMA >>> connection using >>> ------------------------->^^^ >>> unenhanced MPA protocol as defined in [RFC5044] if >>> this protocol is >>> compatible with the application, and let ULP >>> deal with ORD and IRD, >>> and peer-to-peer negotiations. >>> Ordinarily, I'd write an RFC Editor Note for small >>> changes like these, but they're sufficiently critical to >>> interoperability that I'd prefer to have a new draft >>> version that contains them. >>> Thanks, >>> --David >>> > -----Original Message----- >>> > From: Black, David >>> > Sent: Wednesday, December 22, 2010 9:26 PM >>> > To: storm@ietf.org >>> > Cc: Black, David >>> > Subject: MPA Draft - Review >>> > >>> > WG Last Call on this draft has run its course: >>> > >>> > Enhanced RDMA Connection Establishment >>> > draft-ietf-storm-mpa-peer-connect-02 >>> > >>> > I've done my review as a WG chair (and the person who >>> will be shepherding this draft to the ADs and >>> > IESG): >>> > >>> > - This draft is on the right track, but has open issues. >>> > - Another version of the draft will be needed. >>> > >>> > Also, it would be greatly appreciated if a few people >>> other than the authors could take a look at >>> > this draft. We have a very good author team on this >>> draft, whose expertise is beyond doubt, but >>> > more eyes on this draft would help. >>> > >>> > [1] My primary concern is that Section 9 on >>> interoperability is inadequate: >>> > >>> > An initiator SHOULD NOT use the Enhanced DDP >>> Connection Establishment >>> > formats or function codes when no enhanced >>> functionality is desired. >>> > >>> > A responder SHOULD continue to accept the >>> unenhanced connection >>> > requests. >>> > >>> > The good news is that the first sentence is ok. >>> > The bad news is that the second sentence has >>> significant problems: >>> > - It uses SHOULD instead of MUST. >>> > - It doesn't lay out behavior for initiator and >>> responder >>> > Revision mixes. >>> > IETF interoperability requirements are usually >>> expressed with MUST, including backwards >>> > compatibility. If interop with unenhanced >>> implementations is only a SHOULD, that will need a >>> > convincing explanation. >>> > >>> > There are 3 Initiator/Responder cases that need >>> attention (New/Old, Old/New and New/New). I think >>> > they lead to roughly the following: >>> > >>> > New/Old: >>> > - Explain error or failure that the New Initiator will >>> see because the Old responder >>> > doesn't support Revision 2 of the MPA protocol. >>> > - Explain what the Initiator does when it sees that >>> error or failure. The >>> > easiest approach is to always retry with >>> Revision 1, but that won't work >>> > if the Initiator has to send an RTR (that's the >>> "convincing explanation" >>> > for why backwards compatibility is not always >>> possible). The result >>> > might be two requirements: >>> > - If the Initiator has data to send, it MUST >>> retry with Revision 1. >>> > - If the Initiator has no data to send, and >>> hence has to send an RTR, >>> > the connection setup fails, the TCP >>> connection closes and that >>> > failure MUST to be reported to the >>> application. >>> > >>> > Old/New: >>> > - If a responder receives a Revision 1 message, it >>> MUST respond with a Revision 1 message. >>> > >>> > New/New: >>> > - If a responder receives a Revision 2 message, it >>> MUST respond with a Revision 2 message. >>> > >>> > I found a few other concerns: >>> > >>> > [B]In Section 7, we need to get the listing of all the >>> SCTP function codes into one place. Either >>> > repeat the definitions of codes 1-4 from RFC 5043, or >>> create an IANA registry in Section 10 and list >>> > all 7 codes as its initial contents. >>> > >>> > [C] In Section 8, what happens if the responder sends >>> an IRD or ORD value that's different from the >>> > corresponding initiator value? Is the responder >>> allowed to increase the value that was sent? An >>> > important case to cover is that the initiator sends a >>> valid value (e.g., 0x2000) but the responder >>> > returns the 0x3FFF value indicating that negotiation >>> is not supported. Also, what is the behavior >>> > of an IRD or ORD that is set to 0x0000? >>> > >>> > [D] In contrast, the Section 8 discussion of Control >>> Flag functionality is in better shape. It >>> > would be helpful to add a sentence or two indicating >>> when the RTR occurs (Request ->, <- Reply, RTR >>> > ->), even though that is discussed earlier in the >>> draft. Also, it's necessary to state whether >>> > negotiation of RTR functionality commits the Initiator >>> to using an RTR (e.g., suppose the initiator >>> > negotiates control flags to allow an RTR and instead >>> sends an FULPDU with payload data after >>> > receiving the Reply - is that ok or is it an error?). >>> > >>> > [E] Nit: In the definition of Control Flag A: ULPDU -> >>> FULPDU >>> > >>> > Thanks, >>> > --David >>> > ---------------------------------------------------- >>> > David L. Black, Distinguished Engineer >>> > EMC Corporation, 176 South St., Hopkinton, MA 01748 >>> > +1 (508) 293-7953 >>> FAX: +1 >>> (508) 293-7786 >>> > david.black@emc.com >>> Mobile: +1 (978) >>> 394-7754 >>> > ---------------------------------------------------- >>> _______________________________________________ >>> storm mailing list >>> storm@ietf.org >>> https://www.ietf.org/mailman/listinfo/storm >>> >>> _______________________________________________ >>> storm mailing list >>> storm@ietf.org >>> https://www.ietf.org/mailman/listinfo/storm >>> >>> >>> >>> >>> -- >>> Cheers, >>> Arkady Kanevsky >>> >>> >>> _______________________________________________ >>> storm mailing list >>> storm@ietf.org >>> https://www.ietf.org/mailman/listinfo/storm >> >> >> >> >> -- >> Cheers, >> Arkady Kanevsky > > > > > -- > Cheers, > Arkady Kanevsky --------------030703020404010007080804 Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Looks good.

Bob Sharp should ACK this too though.

Thanks,

Steve.


On 4/4/2011 8:50 PM, arkady kanevsky wrote:
Got it. Thanks Steve.
How about now?
Arkady

On Mon, Apr 4, 2011 at 9:41 PM, Steve Wise <swise@opengridcomputing.com> wrote:
The key word to remove here is "responder".  The IRD in the MPA start request is the initiators desired IRD _of the initiator_.  Not the initiators desired _responder_ IRD.

If you want to change "desiired" to "initial" thats ok with me.  But the key is to get rid of the word "responder" in that sentence.

Steve.


On 4/4/2011 6:26 PM, arkady kanevsky wrote:
Steve and Bob,
I changed it to
"In request: the Initiator desired responder IRD
for the connection." as you asked.
I can change it to "initial" instead of "desired".
Arkady

On Sun, Apr 3, 2011 at 5:36 PM, Steve Wise <swise@opengridcomputing.com> wrote:
Hey Arkady,

It does seem like you did the section 9 changes Bob and I requested:

----

  Change the IRD definition on the request from "In request: the Initiator requested responder IRD for the
  connection." to "In request: the Initiator initial IRD setting for the connection." 
----

Thanks,

Steve.


On 4/2/2011 8:35 PM, arkady kanevsky wrote:
All,
updated version 04 is attached.

Hemal,
Thanks for catching it.
I had fixed the first issue. I had added reference to FPDU in the FULPDU definition for the second.

David,
Please, check to see that you comments are addressed.

Steve and Robert,
please, check that you comment is fixed correctly.

Once I get positive feedback from all of you, I will submit the version.

Thanks,
Arkady

On Thu, Mar 31, 2011 at 2:43 PM, Hemal Shah <hemal@broadcom.com> wrote:
I have some comments on -03 draft:
 
  1. In section 10, it is written that "Enhanced MPA Initiator and Responder:  If a responder receives an enhanced MPA message, it MUST respond with an unenhanced MPA message." I think it should be written that the responder must respond with an enhanced MPA message. It appears like a typo to me.
  2. I find the use of FULPDU confusing in this draft. RFC5044 does not define term FULPDU. RFC5044 uses term FPDU to refer to Framed Protocol Data Unit. I suggest that we use term FPDU instead of FULPDU in the draft.
 
Hemal
 
-----Original Message-----
From: storm-bounces@ietf.org [mailto:storm-bounces@ietf.org] On Behalf Of david.black@emc.com
Sent: Thursday, March 31, 2011 7:48 AM
To: storm@ietf.org
Subject: Re: [storm] MPA Draft - Review
Importance: High
 
The -03 version of the MPA draft has addressed all of the issues from my review, and .  Unfortunately, I need some minor edits for clarity before I can send this on to our AD with a publication request.  Would the authors please submit a -04 version with the following two changes quickly.
 
Section 9 (end)
 
OLD
 
   The peer-to-peer negotiation for the RTR message follows the
   following order:    
                               
   Initiator -->: Sets Control Flags it is capable to send for RTR     
                               
   Responder <--: Sets Control Flags it is capable to receive for RTR  
                               
   Initiator -->: The first message send MUST be a negotiated RTR
 
NEW
 
   The peer-to-peer negotiation for the RTR message follows the
   following order:    
                               
   Initiator -->: Sets Control Flags to indicate Initiator-supported forms of RTR      
                               
   Responder <--: Sets Control Flags to indicate Responder-supported forms of RTR      
                               
   Initiator -->: If at least one form of RTR is supported by both Initiator and
        Responder, then the first message sent MUST be an RTR using a form supported
        by both the Initiator and Responder.
 
Section 10
 
OLD
      In
      this case initiator CAN attempt to establish RDMA connection using
      unenhanced MPA protocol as defined in [RFC5044] and let ULP deal
      with ORD and IRD, and peer-to-peer negotiations.
 
NEW
 
      In
      this case initiator MAY attempt to establish RDMA connection using
------------------------->^^^
      unenhanced MPA protocol as defined in [RFC5044] if this protocol is
        compatible with the application, and let ULP deal with ORD and IRD,
      and peer-to-peer negotiations.
 
Ordinarily, I'd write an RFC Editor Note for small changes like these, but they're sufficiently critical to interoperability that I'd prefer to have a new draft version that contains them.
 
Thanks,
--David
 
 
> -----Original Message-----
> From: Black, David
> Sent: Wednesday, December 22, 2010 9:26 PM
> Cc: Black, David
> Subject: MPA Draft - Review
>
> WG Last Call on this draft has run its course:
>
>                  Enhanced RDMA Connection Establishment
>                   draft-ietf-storm-mpa-peer-connect-02
>
> I've done my review as a WG chair (and the person who will be shepherding this draft to the ADs and
> IESG):
>
> - This draft is on the right track, but has open issues.
> - Another version of the draft will be needed.
>
> Also, it would be greatly appreciated if a few people other than the authors could take a look at
> this draft.  We have a very good author team on this draft, whose expertise is beyond doubt, but
> more eyes on this draft would help.
>
> [1] My primary concern is that Section 9 on interoperability is inadequate:
>
>    An initiator SHOULD NOT use the Enhanced DDP Connection Establishment
>    formats or function codes when no enhanced functionality is desired.
>
>    A responder SHOULD continue to accept the unenhanced connection
>    requests.
>
> The good news is that the first sentence is ok.
> The bad news is that the second sentence has significant problems:
>        - It uses SHOULD instead of MUST.
>        - It doesn't lay out behavior for initiator and responder
>                Revision mixes.
> IETF interoperability requirements are usually expressed with MUST, including backwards
> compatibility.  If interop with unenhanced implementations is only a SHOULD, that will need a
> convincing explanation.
>
> There are 3 Initiator/Responder cases that need attention (New/Old, Old/New and New/New).  I think
> they lead to roughly the following:
>
> New/Old:
> - Explain error or failure that the New Initiator will see because the Old responder
>        doesn't support Revision 2 of the MPA protocol.
> - Explain what the Initiator does when it sees that error or failure.  The
>        easiest approach is to always retry with Revision 1, but that won't work
>        if the Initiator has to send an RTR (that's the "convincing explanation"
>        for why backwards compatibility is not always possible).  The result
>        might be two requirements:
>        - If the Initiator has data to send, it MUST retry with Revision 1.
>        - If the Initiator has no data to send, and hence has to send an RTR,
>                the connection setup fails, the TCP connection closes and that
>                failure MUST to be reported to the application.
>
> Old/New:
> - If a responder receives a Revision 1 message, it MUST respond with a Revision 1 message.
>
> New/New:
> - If a responder receives a Revision 2 message, it MUST respond with a Revision 2 message.
>
> I found a few other concerns:
>
> [B]In Section 7, we need to get the listing of all the SCTP function codes into one place.  Either
> repeat the definitions of codes 1-4 from RFC 5043, or create an IANA registry in Section 10 and list
> all 7 codes as its initial contents.
>
> [C] In Section 8, what happens if the responder sends an IRD or ORD value that's different from the
> corresponding initiator value?  Is the responder allowed to increase the value that was sent?  An
> important case to cover is that the initiator sends a valid value (e.g., 0x2000) but the responder
> returns the 0x3FFF value indicating that negotiation is not supported.  Also, what is the behavior
> of an IRD or ORD that is set to 0x0000?
>
> [D] In contrast, the Section 8 discussion of Control Flag functionality is in better shape.  It
> would be helpful to add a sentence or two indicating when the RTR occurs (Request ->, <- Reply, RTR
> ->), even though that is discussed earlier in the draft.  Also, it's necessary to state whether
> negotiation of RTR functionality commits the Initiator to using an RTR (e.g., suppose the initiator
> negotiates control flags to allow an RTR and instead sends an FULPDU with payload data after
> receiving the Reply - is that ok or is it an error?).
>
> [E] Nit: In the definition of Control Flag A: ULPDU -> FULPDU
>
> Thanks,
> --David
> ----------------------------------------------------
> David L. Black, Distinguished Engineer
> EMC Corporation, 176 South St., Hopkinton, MA  01748
> +1 (508) 293-7953             FAX: +1 (508) 293-7786
> david.black@emc.com        Mobile: +1 (978) 394-7754
> ----------------------------------------------------
 
_______________________________________________
storm mailing list
 
 

_______________________________________________
storm mailing list
storm@ietf.org
https://www.ietf.org/mailman/listinfo/storm




--
Cheers,
Arkady Kanevsky

_______________________________________________
storm mailing list
storm@ietf.org
https://www.ietf.org/mailman/listinfo/storm




--
Cheers,
Arkady Kanevsky




--
Cheers,
Arkady Kanevsky

--------------030703020404010007080804-- From arkady.kanevsky@gmail.com Tue Apr 5 17:39:04 2011 Return-Path: X-Original-To: storm@core3.amsl.com Delivered-To: storm@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id AEF323A69A0 for ; Tue, 5 Apr 2011 17:39:04 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -3.598 X-Spam-Level: X-Spam-Status: No, score=-3.598 tagged_above=-999 required=5 tests=[BAYES_00=-2.599, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_LOW=-1] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id bUBdqCAisPH6 for ; Tue, 5 Apr 2011 17:39:02 -0700 (PDT) Received: from mail-pw0-f44.google.com (mail-pw0-f44.google.com [209.85.160.44]) by core3.amsl.com (Postfix) with ESMTP id E0F703A682D for ; Tue, 5 Apr 2011 17:38:57 -0700 (PDT) Received: by pwi5 with SMTP id 5so478339pwi.31 for ; Tue, 05 Apr 2011 17:40:41 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:in-reply-to:references:date :message-id:subject:from:to:cc:content-type; bh=u8E2wtkeKEix5JHF/4Ab8fK0TNAGFaHJVr/1bpezYCA=; b=wwLo+A9tUh1tozaoWe9Jc1GW2UAjg97EBBuKE7OKZ8U7UwcZ1vky5ztl+6A/W9nQJO QtmstHoQzu38leLex8DJJoz/peB3gkXqDr7D+xvBFTOa/dfPVr8/0LtSEAVE8Vx8OoXv GamCmTXZamR6G58J1DZonHkOVPUvSMiAAjnp0= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; b=ao0sWsRW0BJtO4O+og+oqFe98oXreZfMHdaJxN/5/0Gqq1lhmx6J2DdyVWXa3VBbCG Kr5c1xEEfWwhkDyzqZ20CSqr2BuxE2ZlcjhsCIFjnLMqCXGvJIhBPKKTX+hOMkJwIyFX vgHbR7WhJSat8QvGKaZ+pwManw103ut7g+7O4= MIME-Version: 1.0 Received: by 10.142.140.10 with SMTP id n10mr319032wfd.72.1302050441300; Tue, 05 Apr 2011 17:40:41 -0700 (PDT) Received: by 10.142.185.20 with HTTP; Tue, 5 Apr 2011 17:40:41 -0700 (PDT) In-Reply-To: <2EFBCAEF10980645BBCFB605689E08E904D7F61C5B@azsmsx506.amr.corp.intel.com> References: <7C4DFCE962635144B8FAE8CA11D0BF1E03D74C7183@MX14A.corp.emc.com> <7C4DFCE962635144B8FAE8CA11D0BF1E03E5E9530C@MX14A.corp.emc.com> <76DBE161893C324BA6D4BB507EE4C3849510935E96@IRVEXCHCCR02.corp.ad.broadcom.com> <4D98E86A.60403@opengridcomputing.com> <4D9A733B.9050503@opengridcomputing.com> <4D9A7BF7.3040201@opengridcomputing.com> <2EFBCAEF10980645BBCFB605689E08E904D7F61C5B@azsmsx506.amr.corp.intel.com> Date: Tue, 5 Apr 2011 20:40:41 -0400 Message-ID: From: arkady kanevsky To: "Sharp, Robert O" Content-Type: multipart/alternative; boundary=000e0cd2dc3e088d6504a0353e0a Cc: "storm@ietf.org" Subject: Re: [storm] MPA Draft - Review X-BeenThere: storm@ietf.org X-Mailman-Version: 2.1.9 Precedence: list List-Id: Storage Maintenance WG List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 06 Apr 2011 00:39:04 -0000 --000e0cd2dc3e088d6504a0353e0a Content-Type: text/plain; charset=ISO-8859-1 Submitted! On Tue, Apr 5, 2011 at 10:37 AM, Sharp, Robert O wrote: > Looks good to me as well. > > > > Thanks, > > Bob > > > > *From:* Steve Wise [mailto:swise@opengridcomputing.com] > *Sent:* Monday, April 04, 2011 9:19 PM > *To:* arkady kanevsky > *Cc:* Sharp, Robert O; storm@ietf.org > > *Subject:* Re: [storm] MPA Draft - Review > > > > Looks good. > > Bob Sharp should ACK this too though. > > Thanks, > > Steve. > > > On 4/4/2011 8:50 PM, arkady kanevsky wrote: > > Got it. Thanks Steve. > How about now? > Arkady > > On Mon, Apr 4, 2011 at 9:41 PM, Steve Wise > wrote: > > The key word to remove here is "responder". The IRD in the MPA start > request is the initiators desired IRD _of the initiator_. Not the > initiators desired _responder_ IRD. > > If you want to change "desiired" to "initial" thats ok with me. But the > key is to get rid of the word "responder" in that sentence. > > Steve. > > > > On 4/4/2011 6:26 PM, arkady kanevsky wrote: > > Steve and Bob, > I changed it to > "In request: the Initiator desired responder IRD > for the connection." as you asked. > I can change it to "initial" instead of "desired". > Arkady > > On Sun, Apr 3, 2011 at 5:36 PM, Steve Wise > wrote: > > Hey Arkady, > > It does seem like you did the section 9 changes Bob and I requested: > > ---- > > > Change the IRD definition on the request from "In request: the Initiator > requested responder IRD for the > connection." to "In request: the Initiator initial IRD setting for the > connection." > > ---- > > Thanks, > > Steve. > > > > On 4/2/2011 8:35 PM, arkady kanevsky wrote: > > All, > updated version 04 is attached. > > Hemal, > Thanks for catching it. > I had fixed the first issue. I had added reference to FPDU in the FULPDU > definition for the second. > > David, > Please, check to see that you comments are addressed. > > Steve and Robert, > please, check that you comment is fixed correctly. > > Once I get positive feedback from all of you, I will submit the version. > > Thanks, > Arkady > > On Thu, Mar 31, 2011 at 2:43 PM, Hemal Shah wrote: > > I have some comments on -03 draft: > > > > 1. In section 10, it is written that "Enhanced MPA Initiator and > Responder: If a responder receives an enhanced MPA message, it MUST respond > with an unenhanced MPA message." I think it should be written that the > responder must respond with an enhanced MPA message. It appears like a typo > to me. > 2. I find the use of FULPDU confusing in this draft. RFC5044 does not > define term FULPDU. RFC5044 uses term FPDU to refer to Framed Protocol Data > Unit. I suggest that we use term FPDU instead of FULPDU in the draft. > > > > Hemal > > > > -----Original Message----- > From: storm-bounces@ietf.org [mailto:storm-bounces@ietf.org] > On Behalf Of david.black@emc.com > Sent: Thursday, March 31, 2011 7:48 AM > To: storm@ietf.org > Subject: Re: [storm] MPA Draft - Review > > Importance: High > > > > The -03 version of the MPA draft has addressed all of the issues from my > review, and . Unfortunately, I need some minor edits for clarity before I > can send this on to our AD with a publication request. Would the authors > please submit a -04 version with the following two changes quickly. > > > > Section 9 (end) > > > > OLD > > > > The peer-to-peer negotiation for the RTR message follows the > > following order: > > > > Initiator -->: Sets Control Flags it is capable to send for RTR > > > > Responder <--: Sets Control Flags it is capable to receive for RTR > > > > Initiator -->: The first message send MUST be a negotiated RTR > > > > NEW > > > > The peer-to-peer negotiation for the RTR message follows the > > following order: > > > > Initiator -->: Sets Control Flags to indicate Initiator-supported forms > of RTR > > > > Responder <--: Sets Control Flags to indicate Responder-supported forms > of RTR > > > > Initiator -->: If at least one form of RTR is supported by both > Initiator and > > Responder, then the first message sent MUST be an RTR using a form > supported > > by both the Initiator and Responder. > > > > Section 10 > > > > OLD > > In > > this case initiator CAN attempt to establish RDMA connection using > > unenhanced MPA protocol as defined in [RFC5044] and let ULP deal > > with ORD and IRD, and peer-to-peer negotiations. > > > > NEW > > > > In > > this case initiator MAY attempt to establish RDMA connection using > > ------------------------->^^^ > > unenhanced MPA protocol as defined in [RFC5044] if this protocol is > > compatible with the application, and let ULP deal with ORD and IRD, > > and peer-to-peer negotiations. > > > > Ordinarily, I'd write an RFC Editor Note for small changes like these, but > they're sufficiently critical to interoperability that I'd prefer to have a > new draft version that contains them. > > > > Thanks, > > --David > > > > > > > -----Original Message----- > > > From: Black, David > > > Sent: Wednesday, December 22, 2010 9:26 PM > > > To: storm@ietf.org > > > Cc: Black, David > > > Subject: MPA Draft - Review > > > > > > WG Last Call on this draft has run its course: > > > > > > Enhanced RDMA Connection Establishment > > > draft-ietf-storm-mpa-peer-connect-02 > > > > > > I've done my review as a WG chair (and the person who will be shepherding > this draft to the ADs and > > > IESG): > > > > > > - This draft is on the right track, but has open issues. > > > - Another version of the draft will be needed. > > > > > > Also, it would be greatly appreciated if a few people other than the > authors could take a look at > > > this draft. We have a very good author team on this draft, whose > expertise is beyond doubt, but > > > more eyes on this draft would help. > > > > > > [1] My primary concern is that Section 9 on interoperability is > inadequate: > > > > > > An initiator SHOULD NOT use the Enhanced DDP Connection Establishment > > > formats or function codes when no enhanced functionality is desired. > > > > > > A responder SHOULD continue to accept the unenhanced connection > > > requests. > > > > > > The good news is that the first sentence is ok. > > > The bad news is that the second sentence has significant problems: > > > - It uses SHOULD instead of MUST. > > > - It doesn't lay out behavior for initiator and responder > > > Revision mixes. > > > IETF interoperability requirements are usually expressed with MUST, > including backwards > > > compatibility. If interop with unenhanced implementations is only a > SHOULD, that will need a > > > convincing explanation. > > > > > > There are 3 Initiator/Responder cases that need attention (New/Old, > Old/New and New/New). I think > > > they lead to roughly the following: > > > > > > New/Old: > > > - Explain error or failure that the New Initiator will see because the > Old responder > > > doesn't support Revision 2 of the MPA protocol. > > > - Explain what the Initiator does when it sees that error or failure. > The > > > easiest approach is to always retry with Revision 1, but that > won't work > > > if the Initiator has to send an RTR (that's the "convincing > explanation" > > > for why backwards compatibility is not always possible). The > result > > > might be two requirements: > > > - If the Initiator has data to send, it MUST retry with Revision > 1. > > > - If the Initiator has no data to send, and hence has to send an > RTR, > > > the connection setup fails, the TCP connection closes and > that > > > failure MUST to be reported to the application. > > > > > > Old/New: > > > - If a responder receives a Revision 1 message, it MUST respond with a > Revision 1 message. > > > > > > New/New: > > > - If a responder receives a Revision 2 message, it MUST respond with a > Revision 2 message. > > > > > > I found a few other concerns: > > > > > > [B]In Section 7, we need to get the listing of all the SCTP function > codes into one place. Either > > > repeat the definitions of codes 1-4 from RFC 5043, or create an IANA > registry in Section 10 and list > > > all 7 codes as its initial contents. > > > > > > [C] In Section 8, what happens if the responder sends an IRD or ORD value > that's different from the > > > corresponding initiator value? Is the responder allowed to increase the > value that was sent? An > > > important case to cover is that the initiator sends a valid value (e.g., > 0x2000) but the responder > > > returns the 0x3FFF value indicating that negotiation is not supported. > Also, what is the behavior > > > of an IRD or ORD that is set to 0x0000? > > > > > > [D] In contrast, the Section 8 discussion of Control Flag functionality > is in better shape. It > > > would be helpful to add a sentence or two indicating when the RTR occurs > (Request ->, <- Reply, RTR > > > ->), even though that is discussed earlier in the draft. Also, it's > necessary to state whether > > > negotiation of RTR functionality commits the Initiator to using an RTR > (e.g., suppose the initiator > > > negotiates control flags to allow an RTR and instead sends an FULPDU with > payload data after > > > receiving the Reply - is that ok or is it an error?). > > > > > > [E] Nit: In the definition of Control Flag A: ULPDU -> FULPDU > > > > > > Thanks, > > > --David > > > ---------------------------------------------------- > > > David L. Black, Distinguished Engineer > > > EMC Corporation, 176 South St., Hopkinton, MA 01748 > > > +1 (508) 293-7953 FAX: +1 (508) 293-7786<%2B1%20%28508%29%20293-7786> > > > david.black@emc.com Mobile: +1 (978) 394-7754<%2B1%20%28978%29%20394-7754> > > > ---------------------------------------------------- > > > > _______________________________________________ > > storm mailing list > > storm@ietf.org > > https://www.ietf.org/mailman/listinfo/storm > > > > > > > _______________________________________________ > storm mailing list > storm@ietf.org > https://www.ietf.org/mailman/listinfo/storm > > > > > -- > Cheers, > Arkady Kanevsky > > > > _______________________________________________ > > storm mailing list > > storm@ietf.org > > https://www.ietf.org/mailman/listinfo/storm > > > > > > > -- > Cheers, > Arkady Kanevsky > > > > > > > -- > Cheers, > Arkady Kanevsky > > > -- Cheers, Arkady Kanevsky --000e0cd2dc3e088d6504a0353e0a Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Submitted!

On Tue, Apr 5, 2011 at 10:37 A= M, Sharp, Robert O <robert.o.sharp@intel.com> wrote:

Looks good to me as well.

=A0

Thanks,

Bob

=A0

From: Ste= ve Wise [mailto:sw= ise@opengridcomputing.com]
Sent: Monday, April 04, 2011 9:19 PM
To: arkady kanevsky
Cc: Sharp, Robert O; storm@ietf.org


Subject: Re: [storm] MPA Draft - Review

=A0

Looks good.

Bob Sharp should ACK this too though.

Thanks,

Steve.


On 4/4/2011 8:50 PM, arkady kanevsky wrote:

Got it. Thanks Steve.=
How about now?
Arkady

On Mon, Apr 4, 2011 at 9:41 PM, Steve Wise <swise@opengridc= omputing.com> wrote:

The key word to remove here is "responder"= .=A0 The IRD in the MPA start request is the initiators desired IRD _of the initiator_.=A0 Not the initiators desired _responder_ IRD.

If you want to change "desiired" to "initial" thats ok = with me.=A0 But the key is to get rid of the word "responder" in that sentence.

Steve.



On 4/4/2011 6:26 PM, arkady kanevsky wrote:

Steve and Bob,
I changed it to
"In request: the Initiator desired responder IRD
for the connection." as you asked.
I can change it to "initial" instead of "desired".
Arkady

On Sun, Apr 3, 2011 at 5:36 PM, Steve Wise <swise@opengridc= omputing.com> wrote:

Hey Arkady,

It does seem like you did the section 9 changes Bob and I requested:

----


=A0 Change the IRD definition on the request from "In request: the Initiator requested responder IRD for the
=A0 connection." to "In request: the Initiator initial IRD settin= g for the connection."=A0

----

Thanks,

Steve.



On 4/2/2011 8:35 PM, arkady kanevsky wrote:

All,
updated version 04 is attached.

Hemal,
Thanks for catching it.
I had fixed the first issue. I had added reference to FPDU in the FULPDU definition for the second.

David,
Please, check to see that you comments are addressed.

Steve and Robert,
please, check that you comment is fixed correctly.

Once I get positive feedback from all of you, I will submit the version.
Thanks,
Arkady

On Thu, Mar 31, 2011 at 2:43 PM, Hemal Shah <hemal@broadcom.com&= gt; wrote:

I have some comments on -03 draft:

=A0

  1. In section 10, it is written that "Enhanced MPA Initiator and Responder:=A0 If a responder receives an enhanced MPA message, it MUST respond with an unenhanced MPA message." I think it should be wri= tten that the responder must respond with an enhanced MPA message. It appea= rs like a typo to me.
  2. I find the use of FULPDU confusing in this draft. RFC5044 does not defin= e term FULPDU. RFC5044 uses term FPDU to refer to Framed Protocol Data U= nit. I suggest that we use term FPDU instead of FULPDU in the draft.=

=A0

Hemal

=A0

-----Original Message-----
From: storm-bou= nces@ietf.org [mailto:storm-b= ounces@ietf.org] On Behalf Of david= .black@emc.com
Sent: Thursday, March 31, 2011 7:48 AM
To: storm@ietf.org<= br> Subject: Re: [storm] MPA Draft - Review

Importance: High

=A0

The -03 version of the MPA draft has addressed all of the issues from my review, an= d .=A0 Unfortunately, I need some minor edits for clarity before I can send this on to our AD with a publication request.=A0 Would the authors please submit a -04 version with the following two changes quickly.

=A0

Section 9 (end)

=A0

OLD

=A0

=A0=A0 The peer-to-peer negotiation for the RTR message follows the

=A0=A0 following order:=A0=A0=A0=A0

=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0= =A0=A0=A0=A0=A0=A0=A0=A0

=A0=A0 Initiator -->: Sets Control Flags it is capable to send for RTR=A0=A0=A0=A0=A0

=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0= =A0=A0=A0=A0=A0=A0=A0=A0

=A0=A0 Responder <--: Sets Control Flags it is capable to receive for RTR=A0=A0

=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0= =A0=A0=A0=A0=A0=A0=A0=A0

=A0=A0 Initiator -->: The first message send MUST be a negotiated RTR

=A0

NEW

=A0

=A0=A0 The peer-to-peer negotiation for the RTR message follows the

=A0=A0 following order:=A0=A0=A0=A0

=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0= =A0=A0=A0=A0=A0=A0=A0=A0

=A0=A0 Initiator -->: Sets Control Flags to indicate Initiator-supported forms = of RTR=A0=A0=A0=A0=A0=A0

=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0= =A0=A0=A0=A0=A0=A0=A0=A0

=A0=A0 Responder <--: Sets Control Flags to indicate Responder-supported forms = of RTR=A0=A0=A0=A0=A0=A0

=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0= =A0=A0=A0=A0=A0=A0=A0=A0

=A0=A0 Initiator -->: If at least one form of RTR is supported by both Initiato= r and

=A0=A0=A0=A0=A0=A0=A0 Responder, then the first message sent MUST be an RTR using a form supporte= d

=A0=A0=A0=A0=A0=A0=A0 by both the Initiator and Responder.

=A0

Section 10

=A0

OLD

=A0=A0=A0=A0=A0 In

=A0=A0=A0=A0=A0 this case initiator CAN attempt to establish RDMA connection using

=A0=A0=A0=A0=A0 unenhanced MPA protocol as defined in [RFC5044] and let ULP deal

=A0=A0=A0=A0=A0 with ORD and IRD, and peer-to-peer negotiations.

=A0

NEW

=A0

=A0=A0=A0=A0=A0 In

=A0=A0=A0=A0=A0 this case initiator MAY attempt to establish RDMA connection using

------------------------->^^^

=A0=A0=A0=A0=A0 unenhanced MPA protocol as defined in [RFC5044] if this protocol is<= /p>

=A0=A0=A0=A0=A0=A0=A0 compatible with the application, and let ULP deal with ORD and IRD,<= /p>

=A0=A0=A0=A0=A0 and peer-to-peer negotiations.

=A0

Ordinarily, I'd write an RFC Editor Note for small changes like these, but they'= ;re sufficiently critical to interoperability that I'd prefer to have a new= draft version that contains them.

=A0

Thanks,

--David

=A0

=A0

> -----Original Message-----

> From: Black, David

> Sent: Wednesday, December 22, 2010 9:26 PM

> Cc: Black, David

> Subject: MPA Draft - Review

>

> WG Last Call on this draft has run its course:

>

>=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 Enhanced RDMA Connection Establishment

>=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 draft-ietf-storm-mpa-peer-connect-02

>

> I've done my review as a WG chair (and the person who will be shepherdi= ng this draft to the ADs and

> IESG):

>

> - This draft is on the right track, but has open issues.

> - Another version of the draft will be needed.

>

> Also, it would be greatly appreciated if a few people other than the author= s could take a look at

> this draft.=A0 We have a very good author team on this draft, whose expertise is beyond doubt, but

> more eyes on this draft would help.

>

> [1] My primary concern is that Section 9 on interoperability is inadequate:

>

>=A0=A0=A0 An initiator SHOULD NOT use the Enhanced DDP Connection Establishment

>=A0=A0=A0 formats or function codes when no enhanced functionality is desired.=

>

>=A0=A0=A0 A responder SHOULD continue to accept the unenhanced connection

>=A0=A0=A0 requests.

>

> The good news is that the first sentence is ok.

> The bad news is that the second sentence has significant problems:

>=A0=A0=A0=A0=A0=A0=A0 - It uses SHOULD instead of MUST.

>=A0=A0=A0=A0=A0=A0=A0 - It doesn't lay out behavior for initiator and responder

>=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 Revision mixes.

> IETF interoperability requirements are usually expressed with MUST, includi= ng backwards

> compatibility.=A0 If interop with unenhanced implementations is only a SHOULD, that will need a

> convincing explanation.

>

> There are 3 Initiator/Responder cases that need attention (New/Old, Old/New= and New/New).=A0 I think

> they lead to roughly the following:

>

> New/Old:

> - Explain error or failure that the New Initiator will see because the Old responder

>=A0=A0=A0=A0=A0=A0=A0 doesn't support Revision 2 of the MPA protocol.

> - Explain what the Initiator does when it sees that error or failure.=A0 The<= /span>

>=A0=A0=A0=A0=A0=A0=A0 easiest approach is to always retry with Revision 1, but that won't wor= k

>=A0=A0=A0=A0=A0=A0=A0 if the Initiator has to send an RTR (that's the "convincing explanation"

>=A0=A0=A0=A0=A0=A0=A0 for why backwards compatibility is not always possible).=A0 The result

>=A0=A0=A0=A0=A0=A0=A0 might be two requirements:

>=A0=A0=A0=A0=A0=A0=A0 - If the Initiator has data to send, it MUST retry with Revision 1.<= /p>

>=A0=A0=A0=A0=A0=A0=A0 - If the Initiator has no data to send, and hence has to send an RTR,

>=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 the connection setup fails, the TCP connection closes and that

>=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 failure MUST to be reported to the application.

>

> Old/New:

> - If a responder receives a Revision 1 message, it MUST respond with a Revisi= on 1 message.

>

> New/New:

> - If a responder receives a Revision 2 message, it MUST respond with a Revisi= on 2 message.

>

> I found a few other concerns:

>

> [B]In Section 7, we need to get the listing of all the SCTP function codes = into one place.=A0 Either

> repeat the definitions of codes 1-4 from RFC 5043, or create an IANA regist= ry in Section 10 and list

> all 7 codes as its initial contents.

>

> [C] In Section 8, what happens if the responder sends an IRD or ORD value that&= #39;s different from the

> corresponding initiator value?=A0 Is the responder allowed to increase the value that was sent?=A0 An

> important case to cover is that the initiator sends a valid value (e.g., 0x2000) but the responder

> returns the 0x3FFF value indicating that negotiation is not supported.=A0 A= lso, what is the behavior

> of an IRD or ORD that is set to 0x0000?

>

> [D] In contrast, the Section 8 discussion of Control Flag functionality is in better shape.=A0 It

> would be helpful to add a sentence or two indicating when the RTR occurs (Request ->, <- Reply, RTR

> ->), even though that is discussed earlier in the draft.=A0 Also, it'= ;s necessary to state whether

> negotiation of RTR functionality commits the Initiator to using an RTR (e.g= ., suppose the initiator

> negotiates control flags to allow an RTR and instead sends an FULPDU with payload data after

> receiving the Reply - is that ok or is it an error?).

>

> [E] Nit: In the definition of Control Flag A: ULPDU -> FULPDU

>

> Thanks,

> --David

> ----------------------------------------------------

> David L. Black, Distinguished Engineer

> EMC Corporation, 176 South St., Hopkinton, MA=A0 01748

> +1 (= 508) 293-7953=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 FAX: +1 (508) 293-7786

> david.bla= ck@emc.com=A0=A0=A0=A0=A0=A0=A0 Mobile: +1 (9= 78) 394-7754

> ----------------------------------------------------

=A0

_______________________________________________

storm mailing list

storm@ietf.org<= /span>

=A0

=A0


_______________________________________________
storm mailing list
storm@ietf.org
h= ttps://www.ietf.org/mailman/listinfo/storm




--
Cheers,
Arkady Kanevsky

=A0
_______________________________________________
storm mailing list
storm@ietf.org
https://www.ietf.org/mailman/listinfo/s=
torm

=A0




--
Cheers,
Arkady Kanevsky

=A0




--
Cheers,
Arkady Kanevsky

=A0




--
Cheers,
Arkady Kanev= sky
--000e0cd2dc3e088d6504a0353e0a-- From Internet-Drafts@ietf.org Tue Apr 5 17:45:02 2011 Return-Path: X-Original-To: storm@core3.amsl.com Delivered-To: storm@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id A830D3A6834; Tue, 5 Apr 2011 17:45:02 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -102.584 X-Spam-Level: X-Spam-Status: No, score=-102.584 tagged_above=-999 required=5 tests=[AWL=0.015, BAYES_00=-2.599, USER_IN_WHITELIST=-100] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id Umd6gOpnC-Sn; Tue, 5 Apr 2011 17:45:02 -0700 (PDT) Received: from [127.0.0.1] (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id F400D3A682B; Tue, 5 Apr 2011 17:45:01 -0700 (PDT) MIME-Version: 1.0 Content-Type: Multipart/Mixed; Boundary="NextPart" From: Internet-Drafts@ietf.org To: i-d-announce@ietf.org X-Test-IDTracker: no X-IETF-IDTracker: 3.14 Message-ID: <20110406004501.29262.65325.idtracker@localhost> Date: Tue, 05 Apr 2011 17:45:01 -0700 Cc: storm@ietf.org Subject: [storm] I-D Action:draft-ietf-storm-mpa-peer-connect-04.txt X-BeenThere: storm@ietf.org X-Mailman-Version: 2.1.9 Precedence: list List-Id: Storage Maintenance WG List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 06 Apr 2011 00:45:02 -0000 --NextPart A New Internet-Draft is available from the on-line Internet-Drafts directories. This draft is a work item of the STORage Maintenance Working Group of the IETF. Title : Enhanced RDMA Connection Establishment Author(s) : A. Kanevsky, et al. Filename : draft-ietf-storm-mpa-peer-connect-04.txt Pages : 18 Date : 2011-04-05 This document updates [RFC5043] and [RFC5044] by extending MPA negotiation for RDMA Connection establishment. The first extension extends [RFC5043], enabling peer-to-peer connection establishment over MPA/TCP. The second extension extends both [RFC5043] and [RFC5044], by providing an option for standardized exchange of RDMA- layer connection configuration. A URL for this Internet-Draft is: http://www.ietf.org/internet-drafts/draft-ietf-storm-mpa-peer-connect-04.txt Internet-Drafts are also available by anonymous FTP at: ftp://ftp.ietf.org/internet-drafts/ Below is the data which will enable a MIME compliant mail reader implementation to automatically retrieve the ASCII version of the Internet-Draft. --NextPart Content-Type: Message/External-body; name="draft-ietf-storm-mpa-peer-connect-04.txt"; site="ftp.ietf.org"; access-type="anon-ftp"; directory="internet-drafts" Content-Type: text/plain Content-ID: <2011-04-05173806.I-D@ietf.org> --NextPart-- From david.black@emc.com Wed Apr 6 10:03:09 2011 Return-Path: X-Original-To: storm@core3.amsl.com Delivered-To: storm@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id DF29E3A6955 for ; Wed, 6 Apr 2011 10:03:08 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -106.583 X-Spam-Level: X-Spam-Status: No, score=-106.583 tagged_above=-999 required=5 tests=[AWL=0.016, BAYES_00=-2.599, RCVD_IN_DNSWL_MED=-4, USER_IN_WHITELIST=-100] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id m50I2D7sDe1j for ; Wed, 6 Apr 2011 10:03:08 -0700 (PDT) Received: from mexforward.lss.emc.com (mexforward.lss.emc.com [128.222.32.20]) by core3.amsl.com (Postfix) with ESMTP id 2AE003A68E0 for ; Wed, 6 Apr 2011 10:03:07 -0700 (PDT) Received: from hop04-l1d11-si02.isus.emc.com (HOP04-L1D11-SI02.isus.emc.com [10.254.111.55]) by mexforward.lss.emc.com (Switch-3.4.3/Switch-3.4.3) with ESMTP id p36H4mcX031800 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Wed, 6 Apr 2011 13:04:48 -0400 Received: from mailhub.lss.emc.com (mailhub.lss.emc.com [10.254.222.226]) by hop04-l1d11-si02.isus.emc.com (RSA Interceptor); Wed, 6 Apr 2011 13:04:37 -0400 Received: from mxhub13.corp.emc.com (mxhub13.corp.emc.com [128.221.56.102]) by mailhub.lss.emc.com (Switch-3.4.3/Switch-3.4.3) with ESMTP id p36H4TlX026510; Wed, 6 Apr 2011 13:04:29 -0400 Received: from mx14a.corp.emc.com ([169.254.1.143]) by mxhub13.corp.emc.com ([128.221.56.102]) with mapi; Wed, 6 Apr 2011 13:04:29 -0400 From: To: , Date: Wed, 6 Apr 2011 13:04:27 -0400 Thread-Topic: MPA Draft - Implementations? Thread-Index: Acvz9BtYjFkH5NwBQqmQHAlKbuc54QAh8yNA Message-ID: <7C4DFCE962635144B8FAE8CA11D0BF1E03E5EF25EA@MX14A.corp.emc.com> References: <7C4DFCE962635144B8FAE8CA11D0BF1E03D74C7183@MX14A.corp.emc.com> <7C4DFCE962635144B8FAE8CA11D0BF1E03E5E9530C@MX14A.corp.emc.com> <76DBE161893C324BA6D4BB507EE4C3849510935E96@IRVEXCHCCR02.corp.ad.broadcom.com> <4D98E86A.60403@opengridcomputing.com> <4D9A733B.9050503@opengridcomputing.com> <4D9A7BF7.3040201@opengridcomputing.com> <2EFBCAEF10980645BBCFB605689E08E904D7F61C5B@azsmsx506.amr.corp.intel.com> In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: acceptlanguage: en-US Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-EMM-MHVC: 1 Cc: storm@ietf.org Subject: [storm] MPA Draft - Implementations? X-BeenThere: storm@ietf.org X-Mailman-Version: 2.1.9 Precedence: list List-Id: Storage Maintenance WG List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 06 Apr 2011 17:03:09 -0000 As part of requesting RFC publication of the MPA peer connect draft, I need= to provide answers to the following questions: Are there existing implementations of the protocol? Have a=20 significant number of vendors indicated their plan to=20 implement the specification? WG members should feel free to send information directly to me if there's a= ny reluctance to posting it to the mailing list. I don't need to identify = the vendors or projects who have implementations or plans to implement; inf= ormation about the approximate number of implementations and/or plans is al= l that I plan to supply (this info will be part of the IETF document shephe= rd questionnaire, and hence my answers become public). Thanks, --David (storm WG co-chair) ---------------------------------------------------- David L. Black, Distinguished Engineer EMC Corporation, 176 South St., Hopkinton, MA=A0 01748 +1 (508) 293-7953=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 FAX: +1 (508) 293-778= 6 david.black@emc.com=A0=A0=A0=A0=A0=A0=A0 Mobile: +1 (978) 394-7754 ---------------------------------------------------- From arkady.kanevsky@gmail.com Wed Apr 6 16:24:50 2011 Return-Path: X-Original-To: storm@core3.amsl.com Delivered-To: storm@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id CA5F23A6855 for ; Wed, 6 Apr 2011 16:24:50 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -3.598 X-Spam-Level: X-Spam-Status: No, score=-3.598 tagged_above=-999 required=5 tests=[BAYES_00=-2.599, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_LOW=-1] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id i-U66dqvXD5l for ; Wed, 6 Apr 2011 16:24:48 -0700 (PDT) Received: from mail-pv0-f172.google.com (mail-pv0-f172.google.com [74.125.83.172]) by core3.amsl.com (Postfix) with ESMTP id 7681E3A6810 for ; Wed, 6 Apr 2011 16:24:48 -0700 (PDT) Received: by pvh1 with SMTP id 1so914295pvh.31 for ; Wed, 06 Apr 2011 16:26:32 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:in-reply-to:references:date :message-id:subject:from:to:cc:content-type; bh=Cjz+VAP4YNo8D8WB7yAYoTx0ZqEEPTCuv7rNhvYjYVs=; b=w7X9SPzd6GZn63CGwpcgThxHzKigoEEMJFSDANyStY3GeJSTi3mywmq8zxQWK/Po4h 9P3A1P4TTnFFVdaVfanjEpa5+uB5WpP0lc+Sv/Axjb4B+c+Pwx8jNKZCSeMEN/LuJjK0 EVn8qfIfaUcMm1dDARIEWxTrvTRCUycrsV1i4= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; b=xm6q8U8jeY5ySExuQyaohfMaeP6+SmrW9GpEEFxUyuhfbiLKD2ajuIUWjafmL3j00D Trm2CCqSsjC9Q2EQzLmNzFH+1N/+t0YeBK5deqJb8B3NB5K5VnF/miKUFtYo0q7emdHy vVtBoMyPthr1Et+WQhwuykDj+7pVK/yw7K/FU= MIME-Version: 1.0 Received: by 10.142.44.5 with SMTP id r5mr177508wfr.300.1302132392256; Wed, 06 Apr 2011 16:26:32 -0700 (PDT) Received: by 10.142.185.20 with HTTP; Wed, 6 Apr 2011 16:26:32 -0700 (PDT) In-Reply-To: <7C4DFCE962635144B8FAE8CA11D0BF1E03E5EF2584@MX14A.corp.emc.com> References: <7C4DFCE962635144B8FAE8CA11D0BF1E03D74C7183@MX14A.corp.emc.com> <7C4DFCE962635144B8FAE8CA11D0BF1E03E5E9530C@MX14A.corp.emc.com> <76DBE161893C324BA6D4BB507EE4C3849510935E96@IRVEXCHCCR02.corp.ad.broadcom.com> <4D98E86A.60403@opengridcomputing.com> <4D9A733B.9050503@opengridcomputing.com> <4D9A7BF7.3040201@opengridcomputing.com> <2EFBCAEF10980645BBCFB605689E08E904D7F61C5B@azsmsx506.amr.corp.intel.com> <7C4DFCE962635144B8FAE8CA11D0BF1E03E5EF2584@MX14A.corp.emc.com> Date: Wed, 6 Apr 2011 19:26:32 -0400 Message-ID: From: arkady kanevsky To: david.black@emc.com Content-Type: multipart/alternative; boundary=000e0cd2e00eb0e45a04a048520c Cc: storm@ietf.org Subject: Re: [storm] MPA Draft - Implementations? X-BeenThere: storm@ietf.org X-Mailman-Version: 2.1.9 Precedence: list List-Id: Storage Maintenance WG List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 06 Apr 2011 23:24:51 -0000 --000e0cd2e00eb0e45a04a048520c Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: quoted-printable Robert and Steve, are the right people to provide answer to this question. Thanks, Arkady On Wed, Apr 6, 2011 at 11:51 AM, wrote: > As part of submitting this draft to our AD to request publication, I need > to provide answers to the following questions: > > > > Are there existing implementations of the protocol? Have a > > significant number of vendors indicated their plan to > > implement the specification? > > > > Please send info on implementations and/or plans to me directly if you > don=92t want to share on the list. The questions need to be answered in = terms > of numbers of implementations =96 I do not need to identify specific vend= ors > or projects who have implemented or plan to implement the enhancements. > > > > Thanks, > --David > > > > *From:* storm-bounces@ietf.org [mailto:storm-bounces@ietf.org] *On Behalf > Of *arkady kanevsky > *Sent:* Tuesday, April 05, 2011 8:41 PM > *To:* Sharp, Robert O > *Cc:* storm@ietf.org > *Subject:* Re: [storm] MPA Draft - Review > > > > Submitted! > > On Tue, Apr 5, 2011 at 10:37 AM, Sharp, Robert O > wrote: > > Looks good to me as well. > > > > Thanks, > > Bob > > > > *From:* Steve Wise [mailto:swise@opengridcomputing.com] > *Sent:* Monday, April 04, 2011 9:19 PM > *To:* arkady kanevsky > *Cc:* Sharp, Robert O; storm@ietf.org > > > *Subject:* Re: [storm] MPA Draft - Review > > > > Looks good. > > Bob Sharp should ACK this too though. > > Thanks, > > Steve. > > > On 4/4/2011 8:50 PM, arkady kanevsky wrote: > > Got it. Thanks Steve. > How about now? > Arkady > > On Mon, Apr 4, 2011 at 9:41 PM, Steve Wise > wrote: > > The key word to remove here is "responder". The IRD in the MPA start > request is the initiators desired IRD _of the initiator_. Not the > initiators desired _responder_ IRD. > > If you want to change "desiired" to "initial" thats ok with me. But the > key is to get rid of the word "responder" in that sentence. > > Steve. > > > > On 4/4/2011 6:26 PM, arkady kanevsky wrote: > > Steve and Bob, > I changed it to > "In request: the Initiator desired responder IRD > for the connection." as you asked. > I can change it to "initial" instead of "desired". > Arkady > > On Sun, Apr 3, 2011 at 5:36 PM, Steve Wise > wrote: > > Hey Arkady, > > It does seem like you did the section 9 changes Bob and I requested: > > ---- > > > Change the IRD definition on the request from "In request: the Initiato= r > requested responder IRD for the > connection." to "In request: the Initiator initial IRD setting for the > connection." > > ---- > > Thanks, > > Steve. > > > > On 4/2/2011 8:35 PM, arkady kanevsky wrote: > > All, > updated version 04 is attached. > > Hemal, > Thanks for catching it. > I had fixed the first issue. I had added reference to FPDU in the FULPDU > definition for the second. > > David, > Please, check to see that you comments are addressed. > > Steve and Robert, > please, check that you comment is fixed correctly. > > Once I get positive feedback from all of you, I will submit the version. > > Thanks, > Arkady > > On Thu, Mar 31, 2011 at 2:43 PM, Hemal Shah wrote: > > I have some comments on -03 draft: > > > > 1. In section 10, it is written that "Enhanced MPA Initiator and > Responder: If a responder receives an enhanced MPA message, it MUST r= espond > with an unenhanced MPA message." I think it should be written that the > responder must respond with an enhanced MPA message. It appears like a= typo > to me. > 2. I find the use of FULPDU confusing in this draft. RFC5044 does not > define term FULPDU. RFC5044 uses term FPDU to refer to Framed Protocol= Data > Unit. I suggest that we use term FPDU instead of FULPDU in the draft. > > > > Hemal > > > > -----Original Message----- > From: storm-bounces@ietf.org [mailto:storm-bounces@ietf.org] > On Behalf Of david.black@emc.com > Sent: Thursday, March 31, 2011 7:48 AM > To: storm@ietf.org > Subject: Re: [storm] MPA Draft - Review > > Importance: High > > > > The -03 version of the MPA draft has addressed all of the issues from my > review, and . Unfortunately, I need some minor edits for clarity before = I > can send this on to our AD with a publication request. Would the authors > please submit a -04 version with the following two changes quickly. > > > > Section 9 (end) > > > > OLD > > > > The peer-to-peer negotiation for the RTR message follows the > > following order: > > > > Initiator -->: Sets Control Flags it is capable to send for RTR > > > > Responder <--: Sets Control Flags it is capable to receive for RTR > > > > Initiator -->: The first message send MUST be a negotiated RTR > > > > NEW > > > > The peer-to-peer negotiation for the RTR message follows the > > following order: > > > > Initiator -->: Sets Control Flags to indicate Initiator-supported form= s > of RTR > > > > Responder <--: Sets Control Flags to indicate Responder-supported form= s > of RTR > > > > Initiator -->: If at least one form of RTR is supported by both > Initiator and > > Responder, then the first message sent MUST be an RTR using a for= m > supported > > by both the Initiator and Responder. > > > > Section 10 > > > > OLD > > In > > this case initiator CAN attempt to establish RDMA connection using > > unenhanced MPA protocol as defined in [RFC5044] and let ULP deal > > with ORD and IRD, and peer-to-peer negotiations. > > > > NEW > > > > In > > this case initiator MAY attempt to establish RDMA connection using > > ------------------------->^^^ > > unenhanced MPA protocol as defined in [RFC5044] if this protocol is > > compatible with the application, and let ULP deal with ORD and IR= D, > > and peer-to-peer negotiations. > > > > Ordinarily, I'd write an RFC Editor Note for small changes like these, bu= t > they're sufficiently critical to interoperability that I'd prefer to have= a > new draft version that contains them. > > > > Thanks, > > --David > > > > > > > -----Original Message----- > > > From: Black, David > > > Sent: Wednesday, December 22, 2010 9:26 PM > > > To: storm@ietf.org > > > Cc: Black, David > > > Subject: MPA Draft - Review > > > > > > WG Last Call on this draft has run its course: > > > > > > Enhanced RDMA Connection Establishment > > > draft-ietf-storm-mpa-peer-connect-02 > > > > > > I've done my review as a WG chair (and the person who will be shepherdi= ng > this draft to the ADs and > > > IESG): > > > > > > - This draft is on the right track, but has open issues. > > > - Another version of the draft will be needed. > > > > > > Also, it would be greatly appreciated if a few people other than the > authors could take a look at > > > this draft. We have a very good author team on this draft, whose > expertise is beyond doubt, but > > > more eyes on this draft would help. > > > > > > [1] My primary concern is that Section 9 on interoperability is > inadequate: > > > > > > An initiator SHOULD NOT use the Enhanced DDP Connection Establishmen= t > > > formats or function codes when no enhanced functionality is desired. > > > > > > A responder SHOULD continue to accept the unenhanced connection > > > requests. > > > > > > The good news is that the first sentence is ok. > > > The bad news is that the second sentence has significant problems: > > > - It uses SHOULD instead of MUST. > > > - It doesn't lay out behavior for initiator and responder > > > Revision mixes. > > > IETF interoperability requirements are usually expressed with MUST, > including backwards > > > compatibility. If interop with unenhanced implementations is only a > SHOULD, that will need a > > > convincing explanation. > > > > > > There are 3 Initiator/Responder cases that need attention (New/Old, > Old/New and New/New). I think > > > they lead to roughly the following: > > > > > > New/Old: > > > - Explain error or failure that the New Initiator will see because the > Old responder > > > doesn't support Revision 2 of the MPA protocol. > > > - Explain what the Initiator does when it sees that error or failure. > The > > > easiest approach is to always retry with Revision 1, but that > won't work > > > if the Initiator has to send an RTR (that's the "convincing > explanation" > > > for why backwards compatibility is not always possible). The > result > > > might be two requirements: > > > - If the Initiator has data to send, it MUST retry with Revision > 1. > > > - If the Initiator has no data to send, and hence has to send an > RTR, > > > the connection setup fails, the TCP connection closes an= d > that > > > failure MUST to be reported to the application. > > > > > > Old/New: > > > - If a responder receives a Revision 1 message, it MUST respond with a > Revision 1 message. > > > > > > New/New: > > > - If a responder receives a Revision 2 message, it MUST respond with a > Revision 2 message. > > > > > > I found a few other concerns: > > > > > > [B]In Section 7, we need to get the listing of all the SCTP function > codes into one place. Either > > > repeat the definitions of codes 1-4 from RFC 5043, or create an IANA > registry in Section 10 and list > > > all 7 codes as its initial contents. > > > > > > [C] In Section 8, what happens if the responder sends an IRD or ORD val= ue > that's different from the > > > corresponding initiator value? Is the responder allowed to increase th= e > value that was sent? An > > > important case to cover is that the initiator sends a valid value (e.g.= , > 0x2000) but the responder > > > returns the 0x3FFF value indicating that negotiation is not supported. > Also, what is the behavior > > > of an IRD or ORD that is set to 0x0000? > > > > > > [D] In contrast, the Section 8 discussion of Control Flag functionality > is in better shape. It > > > would be helpful to add a sentence or two indicating when the RTR occur= s > (Request ->, <- Reply, RTR > > > ->), even though that is discussed earlier in the draft. Also, it's > necessary to state whether > > > negotiation of RTR functionality commits the Initiator to using an RTR > (e.g., suppose the initiator > > > negotiates control flags to allow an RTR and instead sends an FULPDU wi= th > payload data after > > > receiving the Reply - is that ok or is it an error?). > > > > > > [E] Nit: In the definition of Control Flag A: ULPDU -> FULPDU > > > > > > Thanks, > > > --David > > > ---------------------------------------------------- > > > David L. Black, Distinguished Engineer > > > EMC Corporation, 176 South St., Hopkinton, MA 01748 > > > +1 (508) 293-7953 FAX: +1 (508) 293-7786 > > > david.black@emc.com Mobile: +1 (978) 394-7754 > > > ---------------------------------------------------- > > > > _______________________________________________ > > storm mailing list > > storm@ietf.org > > https://www.ietf.org/mailman/listinfo/storm > > > > > > > _______________________________________________ > storm mailing list > storm@ietf.org > https://www.ietf.org/mailman/listinfo/storm > > > > > -- > Cheers, > Arkady Kanevsky > > > > _______________________________________________ > > storm mailing list > > storm@ietf.org > > https://www.ietf.org/mailman/listinfo/storm > > > > > > > -- > Cheers, > Arkady Kanevsky > > > > > > > -- > Cheers, > Arkady Kanevsky > > > > > > > -- > Cheers, > Arkady Kanevsky > --=20 Cheers, Arkady Kanevsky --000e0cd2e00eb0e45a04a048520c Content-Type: text/html; charset=windows-1252 Content-Transfer-Encoding: quoted-printable Robert and Steve,
are the right people to provide answer to this questio= n.
Thanks,
Arkady

On Wed, Apr 6, 20= 11 at 11:51 AM, <david.black@emc.com> wrote:

As par= t of submitting this draft to our AD to request publication, I need to prov= ide answers to the following questions:

=A0

= =A0=A0=A0=A0=A0=A0=A0 Are there existing implementations of the protocol? H= ave a

=A0=A0=A0=A0=A0=A0=A0=A0significant number of= vendors indicated their plan to

=A0=A0=A0=A0= =A0=A0=A0=A0implement the specification?

=A0

Please send info on imp= lementations and/or plans to me directly if you don=92t want to share on th= e list.=A0 The questions need to be answered in terms of numbers of impleme= ntations =96 I do not need to identify specific vendors or projects who hav= e implemented or plan to implement the enhancements.

=A0

Thanks,
--David

=A0

From:= storm-bounces@ietf.org [mailto:storm-bounces@ietf.org] On Be= half Of arkady kanevsky
Sent: Tuesday, April 05, 2011 8:41 PM
To: Sharp, Robert O<= br>Cc: storm@iet= f.org
Subject: Re: [storm] MPA Draft - Review

=A0

Submitted!

On Tue, Apr 5, 201= 1 at 10:37 AM, Sharp, Robert O <robert.o.sharp@intel.com> wrote:

Looks good to me as well.

=A0

Thanks,

Bob

=A0

From: Steve Wise [mailto:swise@opengridcomputing.com]
Sent: Monday, April 04, 2011 9:19 PM
To: arkady kanevskyCc: Sharp, Robert O; storm@ietf.org


Subject: Re: [storm] MPA Draft - Review

=

=A0

Looks = good.

Bob Sharp should ACK this too though.

Thanks,

St= eve.


On 4/4/2011 8:50 PM, arkady kanevsky wrote:

Got it. Thanks Steve.
How about now?<= br>Arkady

On Mon, Apr 4, 2011 at 9:41 PM, St= eve Wise <swise@opengridcomputing.com> wrote:

The key word to remove here is "responder&= quot;.=A0 The IRD in the MPA start request is the initiators desired IRD _o= f the initiator_.=A0 Not the initiators desired _responder_ IRD.

If = you want to change "desiired" to "initial" thats ok wit= h me.=A0 But the key is to get rid of the word "responder" in tha= t sentence.

Steve.

=



On 4/4/2011 6:26 PM, arkady kanevsky wrote: =

Steve and Bob,I changed it to
"In request: the Initiator desired responder IRD
for the connection= ." as you asked.
I can change it to "initial" instead of = "desired".
Arkady

On Sun, Apr 3, 2011 at 5:36 PM, Steve Wise <swise@opengridcomputing.com> wr= ote:

Hey Arkady,

It does seem like yo= u did the section 9 changes Bob and I requested:

----


=A0 Change the IRD definition = on the request from "In request: the Initiator requested responder IRD= for the
=A0 connection." to "In request: the Initiator initi= al IRD setting for the connection."=A0

----

Thanks,

Steve.

=

On 4/2/2011 8:35 PM, arkady kanevsky wrote:

All,
updated version 04 is attached.

Hemal,
Thanks for catchin= g it.
I had fixed the first issue. I had added reference to FPDU in the = FULPDU definition for the second.

David,
Please, check to see tha= t you comments are addressed.

Steve and Robert,
please, check that you comment is fixed correctly.=

Once I get positive feedback from all of you, I will submit the ver= sion.

Thanks,
Arkady

On Thu, Mar 31, 2011 at 2:43 PM, Hemal Shah <hemal@broadcom.com> wrote:

I have some comments on -03 draft:

=A0

  1. In secti= on 10, it is written that "Enhanced MPA Initiator and Responder:=A0 If= a responder receives an enhanced MPA message, it MUST respond with an unen= hanced MPA message." I think it should be written that the responder m= ust respond with an enhanced MPA message. It appears like a typo to me.
  2. I find the use of FULPDU confusing in this draft. RFC5044 does not def= ine term FULPDU. RFC5044 uses term FPDU to refer to Framed Protocol Data Un= it. I suggest that we use term FPDU instead of FULPDU in the draft.<= /li>

=A0

Hemal

=A0

-----Original Message-----
From: storm-bounces@ietf.org [mailto:storm-bounces@ietf.= org] On Behalf Of david.black@emc.com
Sent: Thursday, March 31, 2011 7:48 AM
To: storm@ietf.org
Subject: Re: [storm] MPA Draft= - Review

Importance: High

=A0

The -03 version of th= e MPA draft has addressed all of the issues from my review, and .=A0 Unfort= unately, I need some minor edits for clarity before I can send this on to o= ur AD with a publication request.=A0 Would the authors please submit a -04 = version with the following two changes quickly.

=A0

Section 9 (end)

=A0

OLD

=A0

=A0=A0 The peer-to-peer negotiation for the RTR message foll= ows the

=A0=A0 following order:=A0=A0=A0=A0

=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0= =A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0

=A0=A0 Initiato= r -->: Sets Control Flags it is capable to send for RTR=A0=A0=A0=A0=A0 <= /span>

=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0= =A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0

=A0=A0 Responde= r <--: Sets Control Flags it is capable to receive for RTR=A0=A0 =

=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0= =A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0

=A0=A0 Initiato= r -->: The first message send MUST be a negotiated RTR

=A0

NEW

=A0

=A0=A0 The peer-to-peer negotiation for the RTR message follows t= he

=A0=A0 following order:=A0=A0=A0=A0

=A0=A0 Initiator -->: Sets Control Flags to indicate Init= iator-supported forms of RTR=A0=A0=A0=A0=A0=A0

= =A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0= =A0=A0=A0=A0=A0=A0

=A0=A0 Responder <--: Sets Control Flags to indicate Resp= onder-supported forms of RTR=A0=A0=A0=A0=A0=A0

= =A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0= =A0=A0=A0=A0=A0=A0

=A0=A0 Initiator -->: If at least one form of RTR is supp= orted by both Initiator and

=A0=A0=A0=A0=A0=A0=A0 = Responder, then the first message sent MUST be an RTR using a form supporte= d

=A0=A0=A0=A0=A0=A0=A0 by both the Initiator and Responder.

=A0

Section 10

=A0

OLD

=A0=A0=A0=A0=A0 In

=A0=A0=A0=A0=A0 this = case initiator CAN attempt to establish RDMA connection using

=A0=A0=A0=A0=A0 unenhanced MPA protocol as defined in [RFC50= 44] and let ULP deal

=A0=A0=A0=A0=A0 with ORD and = IRD, and peer-to-peer negotiations.

=A0

NEW

=A0

=A0=A0=A0=A0=A0 In

=A0=A0=A0=A0=A0 this = case initiator MAY attempt to establish RDMA connection using

------------------------->^^^

= =A0=A0=A0=A0=A0 unenhanced MPA protocol as defined in [RFC5044] if this pro= tocol is

=A0=A0=A0=A0=A0=A0=A0 compatible with the application, and l= et ULP deal with ORD and IRD,

<= span style=3D"font-size: 10pt; font-family: Consolas;">=A0=A0=A0=A0=A0 and = peer-to-peer negotiations.

=A0

Ordinarily, I'd write an R= FC Editor Note for small changes like these, but they're sufficiently c= ritical to interoperability that I'd prefer to have a new draft version= that contains them.

=A0

Thanks,

<= p class=3D"MsoNormal"> --David

<= /div>

=A0

=A0

> -----Original Message-----

>= ; From: Black, David

> Sent: Wednesday, December 22, 2010 9:26 PM

> Cc: Black, David

> Subject:= MPA Draft - Review

>

> WG Last Call on this dra= ft has run its course:

>

>=A0=A0=A0=A0=A0=A0=A0=A0= =A0=A0=A0=A0=A0=A0=A0=A0=A0 Enhanced RDMA Connection Establishment

>=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 d= raft-ietf-storm-mpa-peer-connect-02

>

> I've done my review as a WG chair (and the person w= ho will be shepherding this draft to the ADs and

> IESG):<= /p>

>

> - This draft is on th= e right track, but has open issues.

> - Another version of the draft will be needed.

>

> Also, it would be greatly appreciated if a few people o= ther than the authors could take a look at

> this draft.=A0= We have a very good author team on this draft, whose expertise is beyond d= oubt, but

> more eyes on this draft would help.=

>

> [1] My primary concern i= s that Section 9 on interoperability is inadequate:

>

>=A0=A0=A0 An initiator SH= OULD NOT use the Enhanced DDP Connection Establishment

>=A0=A0=A0 formats or function codes when no enhanced fun= ctionality is desired.

>

>=A0=A0=A0 A responder SHOULD continue to accept the unen= hanced connection

>=A0=A0=A0 requests.=

>

> The good news is that th= e first sentence is ok.

> The bad news is that the second sentence has significan= t problems:

>=A0=A0=A0=A0=A0=A0=A0 - It uses SH= OULD instead of MUST.

>=A0=A0=A0=A0=A0=A0=A0 - It doesn't lay out behavior = for initiator and responder

>=A0=A0=A0=A0=A0=A0= =A0=A0=A0=A0=A0=A0=A0=A0=A0 Revision mixes.

> IETF interoperability requirements are usually expresse= d with MUST, including backwards

> compatibility.= =A0 If interop with unenhanced implementations is only a SHOULD, that will = need a

> convincing explanation.

>

> There are 3 Initiator/Re= sponder cases that need attention (New/Old, Old/New and New/New).=A0 I thin= k

> they lead to roughly the following:

>

> New/Old:

> - Explain error = or failure that the New Initiator will see because the Old responder=

>=A0=A0=A0=A0=A0=A0=A0 doesn't support Revision 2 of = the MPA protocol.

> - Explain what the Initia= tor does when it sees that error or failure.=A0 The

>=A0=A0=A0=A0=A0=A0=A0 easiest approach is to always retr= y with Revision 1, but that won't work

>=A0= =A0=A0=A0=A0=A0=A0 if the Initiator has to send an RTR (that's the &quo= t;convincing explanation"

>=A0=A0=A0=A0=A0=A0=A0 for why backwards compatibility is= not always possible).=A0 The result

>=A0=A0=A0= =A0=A0=A0=A0 might be two requirements:

>=A0=A0=A0=A0=A0=A0=A0 - If the Initiator has data to sen= d, it MUST retry with Revision 1.

>=A0=A0=A0=A0= =A0=A0=A0 - If the Initiator has no data to send, and hence has to send an = RTR,

>=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 the connec= tion setup fails, the TCP connection closes and that

<= p class=3D"MsoNormal">>=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 failure MUST to be rep= orted to the application.

>

> Old/New:

> - If a responder receives a Revision 1 message, it MUST respond wi= th a Revision 1 message.

>

> New/New:

> - If a responder= receives a Revision 2 message, it MUST respond with a Revision 2 message.<= /span>

>

> I found a few other conc= erns:

>

> [B]In Section 7, we need= to get the listing of all the SCTP function codes into one place.=A0 Eithe= r

> repeat the definitions of codes 1-4 from RFC 5043, or c= reate an IANA registry in Section 10 and list

> all 7 codes as= its initial contents.

>

=

> [C] In Section= 8, what happens if the responder sends an IRD or ORD value that's diff= erent from the

> corresponding initiator value?= =A0 Is the responder allowed to increase the value that was sent?=A0 An

> important case to cover is that the initiator sends a v= alid value (e.g., 0x2000) but the responder

> returns the 0x= 3FFF value indicating that negotiation is not supported.=A0 Also, what is t= he behavior

> of an IRD or ORD that is set to 0= x0000?

>

> [D] In contrast, the Sec= tion 8 discussion of Control Flag functionality is in better shape.=A0 It

> would be helpful to add a sentence or two indicating wh= en the RTR occurs (Request ->, <- Reply, RTR

> ->), even t= hough that is discussed earlier in the draft.=A0 Also, it's necessary t= o state whether

> negotiation of RTR functional= ity commits the Initiator to using an RTR (e.g., suppose the initiator

> negotiates control flags to allow an RTR and instead se= nds an FULPDU with payload data after

> receiving the = Reply - is that ok or is it an error?).

>

> [E] Nit: In the definition of Control Flag A: ULPDU -&g= t; FULPDU

>

> Thanks,

> --David

> ----------------------------------------------------

> David L. Black, Distinguished Engineer

> EMC Corporation, 176 South St., Hopkinton, MA=A0 01748<= /span>

> +1 (508) 293-7953=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 = FAX: +1 (508)= 293-7786

> david.black@emc.com=A0=A0=A0=A0=A0=A0=A0 Mobile: +1 (978) 394-7754

> ----------------------------------------------------

=A0

_______________________________________________

storm mailing list

=A0

=A0


_______________________________________________
storm mailing liststorm@ietf.org= https://www.ietf.org/mailman/listinfo/storm




--
Cheers,
Ar= kady Kanevsky

=A0
_______________________________________=
________
storm mailing list
storm@ietf.org
https://www.ietf.org/mailman/listinfo/storm

=A0




--
Cheers,
Arkady Kanevsky

=A0




--
Che= ers,
Arkady Kanevsky

=A0

=




--=
Cheers,
Arkady Kanevsky




--
Cheers,
Arkady Kanevsky
--000e0cd2e00eb0e45a04a048520c-- From david.black@emc.com Fri Apr 8 14:35:50 2011 Return-Path: X-Original-To: storm@core3.amsl.com Delivered-To: storm@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id EC7643A6905 for ; Fri, 8 Apr 2011 14:35:50 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -106.588 X-Spam-Level: X-Spam-Status: No, score=-106.588 tagged_above=-999 required=5 tests=[AWL=0.011, BAYES_00=-2.599, RCVD_IN_DNSWL_MED=-4, USER_IN_WHITELIST=-100] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id apMiSJE2Xr2A for ; Fri, 8 Apr 2011 14:35:49 -0700 (PDT) Received: from mexforward.lss.emc.com (mexforward.lss.emc.com [128.222.32.20]) by core3.amsl.com (Postfix) with ESMTP id 8A2343A688A for ; Fri, 8 Apr 2011 14:35:49 -0700 (PDT) Received: from hop04-l1d11-si03.isus.emc.com (HOP04-L1D11-SI03.isus.emc.com [10.254.111.23]) by mexforward.lss.emc.com (Switch-3.4.3/Switch-3.4.3) with ESMTP id p38LbVY3014838 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO) for ; Fri, 8 Apr 2011 17:37:32 -0400 Received: from mailhub.lss.emc.com (mailhubhoprd01.lss.emc.com [10.254.221.251]) by hop04-l1d11-si03.isus.emc.com (RSA Interceptor) for ; Fri, 8 Apr 2011 17:37:19 -0400 Received: from mxhub31.corp.emc.com (mxhub31.corp.emc.com [128.221.47.160]) by mailhub.lss.emc.com (Switch-3.4.3/Switch-3.4.3) with ESMTP id p38La4ct010463 for ; Fri, 8 Apr 2011 17:36:05 -0400 Received: from mx14a.corp.emc.com ([169.254.2.150]) by mxhub31.corp.emc.com ([128.221.47.160]) with mapi; Fri, 8 Apr 2011 17:36:04 -0400 From: To: Date: Fri, 8 Apr 2011 17:36:03 -0400 Thread-Topic: MPA Peer Connect: Publication Requested Thread-Index: Acv2NPYoO8LwgecNTt2x5FmPIA012Q== Message-ID: <7C4DFCE962635144B8FAE8CA11D0BF1E041664F53C@MX14A.corp.emc.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: acceptlanguage: en-US Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-EMM-MHVC: 1 Subject: [storm] MPA Peer Connect: Publication Requested X-BeenThere: storm@ietf.org X-Mailman-Version: 2.1.9 Precedence: list List-Id: Storage Maintenance WG List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 08 Apr 2011 21:35:51 -0000 I have just requested publication of the MPA Peer Connect draft as a Propos= ed Standard RFC. The next process steps will be Area Director review (by o= ur AD, Dave Harrington) followed by IETF Last Call. The PROTO writeup that= is part of the publication request follows. Enjoy, --David PROTO writeup:=20 Enhanced RDMA Connection Establishment draft-ietf-storm-mpa-peer-connect-04.txt Requested Publication Status: Proposed Standard PROTO shepherd: David L. Black (STORM WG Co-Chair) ------------------------------------------------------------------------ (1.a) Who is the Document Shepherd for this document? Has the Document Shepherd personally reviewed this version of the=20 document and, in particular, does he or she believe this=20 version is ready for forwarding to the IESG for publication? David L. Black (david.black@emc.com) is the Document Shepherd. The Document Shepherd has reviewed this version of the document and believes that it is ready for forwarding to the IESG for publication. (1.b) Has the document had adequate review both from key WG members=20 and from key non-WG members? Does the Document Shepherd have=20 any concerns about the depth or breadth of the reviews that=20 have been performed? The document has had sufficient review from key WG members. The Document Shepherd is satisfied that this document has been sufficiently reviewed by members of the community that uses this protocol. (1.c) Does the Document Shepherd have concerns that the document=20 needs more review from a particular or broader perspective,=20 e.g., security, operational complexity, someone familiar with=20 AAA, internationalization or XML? No. (1.d) Does the Document Shepherd have any specific concerns or=20 issues with this document that the Responsible Area Director and/or the IESG should be aware of? For example, perhaps he=20 or she is uncomfortable with certain parts of the document, or=20 has concerns whether there really is a need for it. In any=20 event, if the WG has discussed those issues and has indicated=20 that it still wishes to advance the document, detail those=20 concerns here. Has an IPR disclosure related to this document=20 been filed? If so, please include a reference to the=20 disclosure and summarize the WG discussion and conclusion on=20 this issue. No. (1.e) How solid is the WG consensus behind this document? Does it=20 represent the strong concurrence of a few individuals, with=20 others being silent, or does the WG as a whole understand and=20 agree with it? The WG consensus of the WG members who are familiar with this technology is solid. The storm (STORage Maintenance) WG conducts maintenance on multiple storage protocols, and different WG members have differing levels of interest and expertise across the protocols that the WG maintains. (1.f) Has anyone threatened an appeal or otherwise indicated extreme=20 discontent? If so, please summarise the areas of conflict in=20 separate email messages to the Responsible Area Director. (It=20 should be in a separate email because this questionnaire is=20 entered into the ID Tracker.) No. (1.g) Has the Document Shepherd personally verified that the=20 document satisfies all ID nits? (See the Internet-Drafts Checklist= =20 and http://tools.ietf.org/tools/idnits/). Boilerplate checks are=20 not enough; this check needs to be thorough. Has the document=20 met all formal review criteria it needs to, such as the MIB=20 Doctor, media type and URI type reviews? Yes, the document satisfies ID nits. No other formal review criteria apply= . (1.h) Has the document split its references into normative and=20 informative? Are there normative references to documents that=20 are not ready for advancement or are otherwise in an unclear=20 state? If such normative references exist, what is the=20 strategy for their completion? Are there normative references=20 that are downward references, as described in [RFC3967]? If=20 so, list these downward references to support the Area=20 Director in the Last Call procedure for them [RFC3967]. The references have been split, and there are no downward references or normative references to work-in-progress documents. (1.i) Has the Document Shepherd verified that the document IANA=20 consideration section exists and is consistent with the body=20 of the document? If the document specifies protocol=20 extensions, are reservations requested in appropriate IANA=20 registries? Are the IANA registries clearly identified? If=20 the document creates a new registry, does it define the=20 proposed initial contents of the registry and an allocation=20 procedure for future registrations? Does it suggest a=20 reasonable name for the new registry? See [RFC5226]. If the=20 document describes an Expert Review process has Shepherd=20 conferred with the Responsible Area Director so that the IESG=20 can appoint the needed Expert during the IESG Evaluation? The IANA Considerations section exists and states that no IANA actions are required by this document. There are some values defined in this document that may be appropriate to be move into IANA registries if future extensions should occur, but creation of IANA registries is not necessary at this juncture (this document suffices as a reference). (1.j) Has the Document Shepherd verified that sections of the=20 document that are written in a formal language, such as XML=20 code, BNF rules, MIB definitions, etc., validate correctly in=20 an automated checker? N/A. (1.k) The IESG approval announcement includes a Document=20 Announcement Write-Up. Please provide such a Document=20 Announcement Write-Up? Recent examples can be found in the "Action" announcements for approved documents. The approval=20 announcement contains the following sections:=20 Technical Summary This document extends iWARP (rddp) RDMA connection establishment with two functions that apply to the adaptation layer between RDMA functionality and the transport protocol. The first extension broadens MPA (adaptation to TCP) to enable connection establishment without initial data to send in support of applications structured as a collection of peers. The second extension improves connection setup for both MPA/TCP and the SCTP adaptation by adding support for standardized exchange of resource availability (queue depth) information. Working Group Summary This document makes small additions to existing protocols. There has been clear WG recognition that this functionality is needed to=20 match the usage of these protocols by an important class of applications, and no significant WG dissent from the design in this document. Document Quality There are multiple existing implementations of the iWARP (rddp) RDMA protocols that have plans to add the functionality specified in this document. Hemal Shah reviewed the near-final version of this draft and found some important corrections that needed to be made. From arkady.kanevsky@gmail.com Sat Apr 9 06:54:37 2011 Return-Path: X-Original-To: storm@core3.amsl.com Delivered-To: storm@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id E54E83A6A9E for ; Sat, 9 Apr 2011 06:54:37 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -3.598 X-Spam-Level: X-Spam-Status: No, score=-3.598 tagged_above=-999 required=5 tests=[BAYES_00=-2.599, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_LOW=-1] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 7EdGyR6L0+oE for ; Sat, 9 Apr 2011 06:54:35 -0700 (PDT) Received: from mail-pz0-f44.google.com (mail-pz0-f44.google.com [209.85.210.44]) by core3.amsl.com (Postfix) with ESMTP id 5A79B3A6932 for ; Sat, 9 Apr 2011 06:54:35 -0700 (PDT) Received: by pzk30 with SMTP id 30so1915390pzk.31 for ; Sat, 09 Apr 2011 06:56:21 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:in-reply-to:references:date :message-id:subject:from:to:cc:content-type; bh=JdDcBLCJA+D0t52qLELqkZDaNLxG2N+kSxyrLzQ5Pgs=; b=KH2sN6veffazDqkky3d2TEJm/vmC1LQpoMsq6OYW/UbHqLadVZQvj5fgsHvfgKes4s /UqVCEUu2SPUMiLCtEcXW+rvdKetXh2MWDiQPHu+3FOmlwLpN2B0LWTqm/pks+6XxWpa PS83U8zZfFtumSElEeroXsN+Q13/T7LecL2UM= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; b=UbJ7/Hslh61juqr1YcZozoGQHMWWRMGmb7C78WZ+khPH7VJVW4G1RR8/UpYXz89As9 /qSlNlSRLZXHmvP+ORi9z/0gjvheDjkJoXTWxhP6TejvXeFZQaEpj//+DBD83rWDkq03 xFedW1IW8WLEKOeJXLbT9oukEr1lLKCE4jSHs= MIME-Version: 1.0 Received: by 10.143.169.7 with SMTP id w7mr2807894wfo.186.1302357381306; Sat, 09 Apr 2011 06:56:21 -0700 (PDT) Received: by 10.142.185.20 with HTTP; Sat, 9 Apr 2011 06:56:21 -0700 (PDT) In-Reply-To: <7C4DFCE962635144B8FAE8CA11D0BF1E041664F53C@MX14A.corp.emc.com> References: <7C4DFCE962635144B8FAE8CA11D0BF1E041664F53C@MX14A.corp.emc.com> Date: Sat, 9 Apr 2011 09:56:21 -0400 Message-ID: From: arkady kanevsky To: david.black@emc.com Content-Type: multipart/alternative; boundary=000e0cd51a54155b4504a07cb50e Cc: storm@ietf.org Subject: Re: [storm] MPA Peer Connect: Publication Requested X-BeenThere: storm@ietf.org X-Mailman-Version: 2.1.9 Precedence: list List-Id: Storage Maintenance WG List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 09 Apr 2011 13:54:38 -0000 --000e0cd51a54155b4504a07cb50e Content-Type: text/plain; charset=ISO-8859-1 Hurray. Thanks David. Arkady On Fri, Apr 8, 2011 at 5:36 PM, wrote: > > I have just requested publication of the MPA Peer Connect draft as a > Proposed Standard RFC. The next process steps will be Area Director review > (by our AD, Dave Harrington) followed by IETF Last Call. The PROTO writeup > that is part of the publication request follows. > > Enjoy, > --David > > PROTO writeup: > Enhanced RDMA Connection Establishment > draft-ietf-storm-mpa-peer-connect-04.txt > > Requested Publication Status: Proposed Standard > PROTO shepherd: David L. Black (STORM WG Co-Chair) > ------------------------------------------------------------------------ > > (1.a) Who is the Document Shepherd for this document? Has the > Document Shepherd personally reviewed this version of the > document and, in particular, does he or she believe this > version is ready for forwarding to the IESG for publication? > > David L. Black (david.black@emc.com) is the Document Shepherd. The > Document Shepherd has reviewed this version of the document and believes > that it is ready for forwarding to the IESG for publication. > > (1.b) Has the document had adequate review both from key WG members > and from key non-WG members? Does the Document Shepherd have > any concerns about the depth or breadth of the reviews that > have been performed? > > The document has had sufficient review from key WG members. The Document > Shepherd is satisfied that this document has been sufficiently reviewed > by members of the community that uses this protocol. > > (1.c) Does the Document Shepherd have concerns that the document > needs more review from a particular or broader perspective, > e.g., security, operational complexity, someone familiar with > AAA, internationalization or XML? > > No. > > (1.d) Does the Document Shepherd have any specific concerns or > issues with this document that the Responsible Area Director > and/or the IESG should be aware of? For example, perhaps he > or she is uncomfortable with certain parts of the document, or > has concerns whether there really is a need for it. In any > event, if the WG has discussed those issues and has indicated > that it still wishes to advance the document, detail those > concerns here. Has an IPR disclosure related to this document > been filed? If so, please include a reference to the > disclosure and summarize the WG discussion and conclusion on > this issue. > > No. > > (1.e) How solid is the WG consensus behind this document? Does it > represent the strong concurrence of a few individuals, with > others being silent, or does the WG as a whole understand and > agree with it? > > The WG consensus of the WG members who are familiar with this technology is > solid. The storm (STORage Maintenance) WG conducts maintenance on multiple > storage protocols, and different WG members have differing levels of > interest and expertise across the protocols that the WG maintains. > > (1.f) Has anyone threatened an appeal or otherwise indicated extreme > discontent? If so, please summarise the areas of conflict in > separate email messages to the Responsible Area Director. (It > should be in a separate email because this questionnaire is > entered into the ID Tracker.) > > No. > > (1.g) Has the Document Shepherd personally verified that the > document satisfies all ID nits? (See the Internet-Drafts Checklist > and http://tools.ietf.org/tools/idnits/). Boilerplate checks are > not enough; this check needs to be thorough. Has the document > met all formal review criteria it needs to, such as the MIB > Doctor, media type and URI type reviews? > > Yes, the document satisfies ID nits. No other formal review criteria > apply. > > (1.h) Has the document split its references into normative and > informative? Are there normative references to documents that > are not ready for advancement or are otherwise in an unclear > state? If such normative references exist, what is the > strategy for their completion? Are there normative references > that are downward references, as described in [RFC3967]? If > so, list these downward references to support the Area > Director in the Last Call procedure for them [RFC3967]. > > The references have been split, and there are no downward references or > normative references to work-in-progress documents. > > (1.i) Has the Document Shepherd verified that the document IANA > consideration section exists and is consistent with the body > of the document? If the document specifies protocol > extensions, are reservations requested in appropriate IANA > registries? Are the IANA registries clearly identified? If > the document creates a new registry, does it define the > proposed initial contents of the registry and an allocation > procedure for future registrations? Does it suggest a > reasonable name for the new registry? See [RFC5226]. If the > document describes an Expert Review process has Shepherd > conferred with the Responsible Area Director so that the IESG > can appoint the needed Expert during the IESG Evaluation? > > The IANA Considerations section exists and states that no IANA actions > are required by this document. There are some values defined in this > document that may be appropriate to be move into IANA registries > if future extensions should occur, but creation of IANA registries is > not necessary at this juncture (this document suffices as a reference). > > (1.j) Has the Document Shepherd verified that sections of the > document that are written in a formal language, such as XML > code, BNF rules, MIB definitions, etc., validate correctly in > an automated checker? > > N/A. > > (1.k) The IESG approval announcement includes a Document > Announcement Write-Up. Please provide such a Document > Announcement Write-Up? Recent examples can be found in the > "Action" announcements for approved documents. The approval > announcement contains the following sections: > > Technical Summary > > This document extends iWARP (rddp) RDMA connection establishment > with two functions that apply to the adaptation layer between RDMA > functionality and the transport protocol. The first extension > broadens > MPA (adaptation to TCP) to enable connection establishment without > initial data to send in support of applications structured as a > collection of peers. The second extension improves connection setup > for both MPA/TCP and the SCTP adaptation by adding support for > standardized exchange of resource availability (queue depth) > information. > > Working Group Summary > > This document makes small additions to existing protocols. There > has been clear WG recognition that this functionality is needed to > match the usage of these protocols by an important class of > applications, > and no significant WG dissent from the design in this document. > > Document Quality > > There are multiple existing implementations of the iWARP (rddp) RDMA > protocols that have plans to add the functionality specified in > this document. Hemal Shah reviewed the near-final version of this > draft and found some important corrections that needed to be made. > > > _______________________________________________ > storm mailing list > storm@ietf.org > https://www.ietf.org/mailman/listinfo/storm > -- Cheers, Arkady Kanevsky --000e0cd51a54155b4504a07cb50e Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Hurray.
Thanks David.
Arkady

On Fri= , Apr 8, 2011 at 5:36 PM, <david.black@emc.com> wrote:

I have just requested publication of the MPA Peer Connect draft as a Propos= ed Standard RFC. =A0The next process steps will be Area Director review (by= our AD, Dave Harrington) followed by IETF Last Call. =A0The PROTO writeup = that is part of the publication request follows.

Enjoy,
--David

PROTO writeup:
=A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 Enhanced RDMA Connection Establishment
=A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0draft-ietf-storm-mpa-peer-connect-04.txt
Requested Publication Status: Proposed Standard
PROTO shepherd: David L. Black (STORM WG Co-Chair)
------------------------------------------------------------------------
=A0(1.a) Who is the Document Shepherd for this document? Has the
=A0 =A0 =A0 =A0Document Shepherd personally reviewed this version of the =A0 =A0 =A0 =A0document and, in particular, does he or she believe this =A0 =A0 =A0 =A0version is ready for forwarding to the IESG for publication= ?

David L. Black (david.black@emc.com<= /a>) is the Document Shepherd. =A0The
Document Shepherd has reviewed this version of the document and believes that it is ready for forwarding to the IESG for publication.

=A0(1.b) Has the document had adequate review both from key WG members
=A0 =A0 =A0 =A0and from key non-WG members? Does the Document Shepherd hav= e
=A0 =A0 =A0 =A0any concerns about the depth or breadth of the reviews that=
=A0 =A0 =A0 =A0have been performed?

The document has had sufficient review from key WG members. =A0The Document=
Shepherd is satisfied that this document has been sufficiently reviewed
by members of the community that uses this protocol.

=A0(1.c) Does the Document Shepherd have concerns that the document
=A0 =A0 =A0 =A0needs more review from a particular or broader perspective,=
=A0 =A0 =A0 =A0e.g., security, operational complexity, someone familiar wi= th
=A0 =A0 =A0 =A0AAA, internationalization or XML?

No.

=A0(1.d) Does the Document Shepherd have any specific concerns or
=A0 =A0 =A0 =A0issues with this document that the Responsible Area Directo= r
=A0 =A0 =A0 =A0and/or the IESG should be aware of? For example, perhaps he=
=A0 =A0 =A0 =A0or she is uncomfortable with certain parts of the document,= or
=A0 =A0 =A0 =A0has concerns whether there really is a need for it. In any<= br> =A0 =A0 =A0 =A0event, if the WG has discussed those issues and has indicat= ed
=A0 =A0 =A0 =A0that it still wishes to advance the document, detail those<= br> =A0 =A0 =A0 =A0concerns here. Has an IPR disclosure related to this docume= nt
=A0 =A0 =A0 =A0been filed? If so, please include a reference to the
=A0 =A0 =A0 =A0disclosure and summarize the WG discussion and conclusion o= n
=A0 =A0 =A0 =A0this issue.

No.

=A0(1.e) How solid is the WG consensus behind this document? Does it
=A0 =A0 =A0 =A0represent the strong concurrence of a few individuals, with=
=A0 =A0 =A0 =A0others being silent, or does the WG as a whole understand a= nd
=A0 =A0 =A0 =A0agree with it?

The WG consensus of the WG members who are familiar with this technology is=
solid. =A0The storm (STORage Maintenance) WG conducts maintenance on multip= le
storage protocols, and different WG members have differing levels of
interest and expertise across the protocols that the WG maintains.

=A0(1.f) Has anyone threatened an appeal or otherwise indicated extreme =A0 =A0 =A0 =A0discontent? If so, please summarise the areas of conflict i= n
=A0 =A0 =A0 =A0separate email messages to the Responsible Area Director. (= It
=A0 =A0 =A0 =A0should be in a separate email because this questionnaire is=
=A0 =A0 =A0 =A0entered into the ID Tracker.)

No.

=A0(1.g) Has the Document Shepherd personally verified that the
=A0 =A0 =A0 =A0document satisfies all ID nits? (See the Internet-Drafts Ch= ecklist
=A0 =A0 =A0 =A0and http://tools.ietf.org/tools/idnits/). Boilerplate checks ar= e
=A0 =A0 =A0 =A0not enough; this check needs to be thorough. Has the docume= nt
=A0 =A0 =A0 =A0met all formal review criteria it needs to, such as the MIB=
=A0 =A0 =A0 =A0Doctor, media type and URI type reviews?

Yes, the document satisfies ID nits. =A0No other formal review criteria app= ly.

=A0(1.h) Has the document split its references into normative and
=A0 =A0 =A0 =A0informative? Are there normative references to documents th= at
=A0 =A0 =A0 =A0are not ready for advancement or are otherwise in an unclea= r
=A0 =A0 =A0 =A0state? If such normative references exist, what is the
=A0 =A0 =A0 =A0strategy for their completion? Are there normative referenc= es
=A0 =A0 =A0 =A0that are downward references, as described in [RFC3967]? If=
=A0 =A0 =A0 =A0so, list these downward references to support the Area
=A0 =A0 =A0 =A0Director in the Last Call procedure for them [RFC3967].

The references have been split, and there are no downward references or
normative references to work-in-progress documents.

=A0(1.i) Has the Document Shepherd verified that the document IANA
=A0 =A0 =A0 =A0consideration section exists and is consistent with the bod= y
=A0 =A0 =A0 =A0of the document? If the document specifies protocol
=A0 =A0 =A0 =A0extensions, are reservations requested in appropriate IANA<= br> =A0 =A0 =A0 =A0registries? Are the IANA registries clearly identified? If<= br> =A0 =A0 =A0 =A0the document creates a new registry, does it define the
=A0 =A0 =A0 =A0proposed initial contents of the registry and an allocation=
=A0 =A0 =A0 =A0procedure for future registrations? Does it suggest a
=A0 =A0 =A0 =A0reasonable name for the new registry? See [RFC5226]. If the=
=A0 =A0 =A0 =A0document describes an Expert Review process has Shepherd =A0 =A0 =A0 =A0conferred with the Responsible Area Director so that the IE= SG
=A0 =A0 =A0 =A0can appoint the needed Expert during the IESG Evaluation?
The IANA Considerations section exists and states that no IANA actions
are required by this document. =A0There are some values defined in this
document that may be appropriate to be move into IANA registries
if future extensions should occur, but creation of IANA registries is
not necessary at this juncture (this document suffices as a reference).

=A0(1.j) Has the Document Shepherd verified that sections of the
=A0 =A0 =A0 =A0document that are written in a formal language, such as XML=
=A0 =A0 =A0 =A0code, BNF rules, MIB definitions, etc., validate correctly = in
=A0 =A0 =A0 =A0an automated checker?

N/A.

=A0(1.k) The IESG approval announcement includes a Document
=A0 =A0 =A0 =A0Announcement Write-Up. Please provide such a Document
=A0 =A0 =A0 =A0Announcement Write-Up? Recent examples can be found in the<= br> =A0 =A0 =A0 =A0"Action" announcements for approved documents. Th= e approval
=A0 =A0 =A0 =A0announcement contains the following sections:

=A0 =A0 Technical Summary

=A0 =A0 =A0 =A0This document extends iWARP (rddp) RDMA connection establis= hment
=A0 =A0 =A0 =A0with two functions that apply to the adaptation layer betwe= en RDMA
=A0 =A0 =A0 =A0functionality and the transport protocol. =A0The first exte= nsion broadens
=A0 =A0 =A0 =A0MPA (adaptation to TCP) to enable connection establishment = without
=A0 =A0 =A0 =A0initial data to send in support of applications structured = as a
=A0 =A0 =A0 =A0collection of peers. =A0The second extension improves conne= ction setup
=A0 =A0 =A0 =A0for both MPA/TCP and the SCTP adaptation by adding support = for
=A0 =A0 =A0 =A0standardized exchange of resource availability (queue depth= ) information.

=A0 =A0 Working Group Summary

=A0 =A0 =A0 =A0This document makes small additions to existing protocols. = =A0There
=A0 =A0 =A0 =A0has been clear WG recognition that this functionality is ne= eded to
=A0 =A0 =A0 =A0match the usage of these protocols by an important class of= applications,
=A0 =A0 =A0 =A0and no significant WG dissent from the design in this docum= ent.

=A0 =A0 Document Quality

=A0 =A0 =A0 =A0There are multiple existing implementations of the iWARP (r= ddp) RDMA
=A0 =A0 =A0 =A0protocols that have plans to add the functionality specifie= d in
=A0 =A0 =A0 =A0this document. =A0Hemal Shah reviewed the near-final versio= n of this
=A0 =A0 =A0 =A0draft and found some important corrections that needed to b= e made.


_______________________________________________
storm mailing list
storm@ietf.org
h= ttps://www.ietf.org/mailman/listinfo/storm



--
Cheers,
Arkady Kanev= sky
--000e0cd51a54155b4504a07cb50e--