Subject: Re: [storm] Proposed text changes for the connection allocation problem

From: "Black, David" <[email-masked]>
Date: Mon, 13 Aug 2012 18:27:35 -0400

Mike,

Thanks for posting this new text, and I apologize for the delayed response -
too much "stuff" going on in Vancouver ...

I think we could also use some additional text that summarizes the new/old, old/new
and new/new implementation interactions.  Some comments on your posted text ...

> If the iSCSI Layer on the initiator side allocates the connection resources
> to support RCaP only after it receives the final Login Response PDU from the
> target, then it may not be able to handle the number of unexpected iSCSI
> control-type PDUs (as declared by the MaxOutstandingUnexpectedPDUs key from
> the initiator) that can be sent by the target before the buffer resources
> are allocated at the initiator side.  In this case the iSERHelloRequired key
> MUST be negotiated to Yes so that the initiator can allocate the connection
> resources before sending the iSER Hello Message.  See section 5.1.3 for more
> details.

That "MUST be negotiated to Yes" seems wrong, as that can't be done if the target
responds with NotUnderstood.  Also, that text doesn't provide a sufficiently blunt
warning, IMHO - it ought to state that the consequences of "may not be able to
handle" include failure of the iSER session caused by closure of the underlying
RCaP connection.

> As the iSERHelloRequired key is not defined in [RFC5046], an initiator/target
> must be able to deal with a target/initiator that does not understand the key
> or support the iSER Hello exchanges.

That's correct, although I would change:
"must be able to deal with" ->"needs to be able to deal with".

> For a target, if the iSERHelloRequired key is not negotiated, it MUST still
> respond with the iSER HelloReply Message when it receives the iSER Hello Message.
> If the iSERHelloRequired key is negotiated to No or NotUnderstood, a target MAY
> choose to terminate the connection; otherwise it SHOULD delay sending any
> unexpected control-type PDUs until one of the following events has occurred:
>
> 1. A PDU is received from the initiator after it sends the final Login Response PDU.
>
> 2. A system configurable timeout period, say one second, has expired.

That's ok, I think additional text is needed somewhere to point out that there is
known target code that sends one PDU and then delays - that might go in the additional
summary text that I suggest above, and also should be included as part of the
rationale for:

> For an initiator, if the iSERHelloRequired key is negotiated to No or NotUnderstood,
> it MAY choose to terminate the connection; otherwise it SHOULD do one of the following:
>
> 1. Allocate the connection resources before sending the final Login Request PDU.
>
> 2. Allocate one or more buffers for receiving unexpected control-type PDUs from
> the target before sending the final Login Request PDU.  This reduces the possibility
> of the unexpected control-type PDUs causing the RCaP connection to close before
> the connection resources have been allocated.

Thanks,
--David

From: [email-masked] [mailto:[email-masked]] On Behalf Of Michael Ko
Sent: Tuesday, July 24, 2012 8:27 PM
To: [email-masked]
Subject: [storm] Proposed text changes for the connection allocation problem

These are the changes I intend to incorporate into the existing draft to solve the connection allocation problem by using the iSER Hello exchanges.  Please comment.

5.1.1  Initiator Behavior

original:

If the outcome of the iSCSI negotiation is to enable iSER-assisted mode, then on the initiator side, prior to sending the Login Request with the T (Transit) bit set to 1 and the NSG (Next Stage) field set to FullFeaturePhase, the iSCSI Layer MUST request the iSER Layer to allocate the connection resources necessary to support RCaP by invoking the Allocate_Connection_Resources Operational Primitive.

replacement:

If the outcome of the iSCSI negotiation is to enable iSER-assisted mode, then on the initiator side, prior to sending the Login Request with the T (Transit) bit set to 1 and the NSG (Next Stage) field set to FullFeaturePhase, the iSCSI Layer SHOULD request the iSER Layer to allocate the connection resources necessary to support RCaP by invoking the Allocate_Connection_Resources Operational Primitive.

deleted:

If the iSER Layer at the initiator is successful in allocating the connection resources necessary to support RCaP, the following events MUST occur in the specified sequence:  [The numbered items will be retained but without the numbering.]

new:

If the iSCSI Layer on the initiator side allocates the connection resources to support RCaP only after it receives the final Login Response PDU from the target, then it may not be able to handle the number of unexpected iSCSI control-type PDUs (as declared by the MaxOutstandingUnexpectedPDUs key from the initiator) that can be sent by the target before the buffer resources are allocated at the initiator side.  In this case the iSERHelloRequired key MUST be negotiated to Yes so that the initiator can allocate the connection resources before sending the iSER Hello Message.  See section 5.1.3 for more details.

5.1.3  iSER Hello Exchange

new:

The connection resources can be allocated at the initiator side after it has received the final Login Response PDU from the target.  To prevent the potential problem caused by the target sending the number of unexpected iSCSI control-type PDUs as determined by the MaxOustandingUnexpectedPDUs key allowed in the FullFeaturePhase, the iSER Hello exchanges can be used.  This allows the initiator to allocate the connection resources before sending the iSER Hello Message.  The iSERHelloRequired key is used by the initiator to determine if it is dealing with a target that supports the iSER Hello exchanges.

As the iSERHelloRequired key is not defined in [RFC5046], an initiator/target must be able to deal with a target/initiator that does not understand the key or support the iSER Hello exchanges.

For a target, if the iSERHelloRequired key is not negotiated, it MUST still respond with the iSER HelloReply Message when it recieves the iSER Hello Message.  If the iSERHelloRequired key is negotiated to No or NotUnderstood, a target MAY choose to terminate the connection; otherwise it SHOULD delay sending any unexpected control-type PDUs until one of the following events has occurred:

1. A PDU is received from the initiator after it sends the final Login Response PDU.

2. A system configurable timeout period, say one second, has expired.

For an initiator, if the iSERHelloRequired key is negotiated to No or NotUnderstood, it MAY choose to terminate the connection; otherwise it SHOULD do one of the following:

1. Allocate the connection resources before sending the final Login Request PDU.

2. Allocate one or more buffers for receiving unexpected control-type PDUs from the target before sending the final Login Request PDU.  This reduces the possibility of the unexpected control-type PDUs causing the RCaP connection to close before the connection resources have been allocated.

10.1.3.4 iSER Protocol Errors

original:

Conversely, it is a protocol error if the iSER Hello Message is sent by the iSER Layer at the initiator when iSERHelloRequired is negotiated to "No".

new:

Conversely, if the iSER Hello Message is sent by the iSER Layer at the initiator when iSERHelloRequired is negotiated to "No", the iSER Layer at the target MAY treat this as a protocol error or respond with an iSER HelloReply Message.

Mike

---

From: Michael Ko <[email-masked]>
Date: Sun, 19 Aug 2012 18:48:36 -0700

Changed “must be able to deal with” to “needs to be able to deal with” as
suggested.

For the rest of the suggestions, I have created new text that replaces the
"new" text in section 5.1.3:

An initiator that conforms to [RFC5046] allocates connection resources
before seding the Login Request with the T (Transit) bit set to 1 and the
NSG (Next Stage) field set to FullFeaturePhase.  (For brevity, this is
referred to as "early" connection allocation.)  The current iSER
specification relaxes this requirement to allow an initiator to allocate
connection resources after it receives the final Login Response PDU from
the target.  (For brevity, this is referred to as "late" connection
allocation.)  To prevent the potential problem caused by the target sending
the number of unexpected iSCSI control-type PDUs as determined by the
MaxOustandingUnexpectedPDUs key allowed in the FullFeaturePhase when "late"
connection allocation is practised, the iSER Hello exchanges MUST be used.
This allows the initiator to allocate the connection resources before
sending the iSER Hello Messages.  The iSERHelloRequired key is used by the
initiator to determine if it is dealing with a target that supports the
iSER Hello exchanges.

In the following summary where "late" connection allocation is practised,
an initiator that follows [RFC5046] is referred to as an "old" initiator;
otherwise it is referred to as a "new" initiator.  Similarly, a target that
does not support the iSERHelloRequired key (and responds with
"NotUnderstood" when negotiating the iSERHelloRequired key) is referred to
as an "old" target; otherwise it is referred to as a "new" target.  Note
that an "old" target can still support the iSER Hello exchanges but this
fact is not known by the initiator.  A "new" target can also respond with
"No" when negotiating the iSERHelloREquired key.  In this case its beahvior
with respect to "late" connection allocation is similar to an "old" target.

A "new" initiator will work fine with a "new" target.

For an "old" initiator and an "old" target, the failure by the initiator to
handle the number of unexpected iSCSI control-type PDUs that are sent by
the target before the buffer resources are allocated at the initiator can
result in the failure of the iSER session caused by closure of the
underlying RCaP connection.  For the "old" target, there is known
implementation that sends one unexpected iSCSI control-type PDU after
sending the final Login Response and then waits awhile before sending the
next one.  This tends to alleviate somewhat the buffer allocation problem
at the initiator.

For a "new" initiator and an "old" target, the failure by the initiator to
handle the number of unexpected iSCSI control-type PDUs that are sent by
the target before the buffer resources are allocated at the initiator can
result in the failure of the iSER session caused by closure of the
underlying RCaP connection.  A "new" initiator MAY choose to terminate the
connection; otherwise it SHOULD do one of the following:

1. Allocate the connection resources before sending the final Login Request
PDU.

2. Allocate one or more buffers for receiving unexpected control-type PDUs
from the target before sending the final Login Request PDU.  This reduces
the possibility of the unexpected control-type PDUs causing the RCaP
connection to close before the connection resources have been allocated.

For an "old" initiator and a "new" target, if the iSERHelloRequired key is
not negotiated, a "new" target MUST still respond with the iSER HelloReply
Message when it receives the iSER Hello Message.  If the iSERHelloRequired
key is negotiated to "No" or "NotUnderstood", a "new" target MAY choose to
terminate the connection; otherwise it SHOULD delay sending any unexpected
control-type PDUs until one of the following events has occurred:

1. A PDU is received from the initiator after it sends the final Login
Response PDU.

Mike

On Mon, Aug 13, 2012 at 3:27 PM, Black, David <[email-masked]> wrote:

> Mike,****
>
> ** **
>
> Thanks for posting this new text, and I apologize for the delayed response
> -****
>
> too much “stuff” going on in Vancouver ...****
>
> ** **
>
> I think we could also use some additional text that summarizes the
> new/old, old/new****
>
> and new/new implementation interactions.  Some comments on your posted
> text ...****
>
> ** **
>
> > If the iSCSI Layer on the initiator side allocates the connection
> resources****
>
> > to support RCaP only after it receives the final Login Response PDU from
> the****
>
> ****
> > control-type PDUs (as declared by the MaxOutstandingUnexpectedPDUs key
> from****
>
> ****
> > are allocated at the initiator side.  In this case the iSERHelloRequired
> key****
>
> > MUST be negotiated to Yes so that the initiator can allocate the
> connection****
>
> > resources before sending the iSER Hello Message.  See section 5.1.3 for
> more****
>
> > details.****
>
> ** **
>
> That “MUST be negotiated to Yes” seems wrong, as that can’t be done if the
> target****
>
> responds with NotUnderstood.  Also, that text doesn’t provide a
> sufficiently blunt****
>
> warning, IMHO - it ought to state that the consequences of “may not be
> able to****
>
> handle” include failure of the iSER session caused by closure of the
> underlying****
>
> RCaP connection.****
>
> ** **
>
> > As the iSERHelloRequired key is not defined in [RFC5046], an
> initiator/target****
>
> > must be able to deal with a target/initiator that does not understand
> the key****
>
> ****
> That’s correct, although I would change:****
>
>  “must be able to deal with” ->“needs to be able to deal with”.****
>
> ** **
>
> > respond with the iSER HelloReply Message when it receives the iSER Hello
> Message.****
>
> > If the iSERHelloRequired key is negotiated to No or NotUnderstood, a
> target MAY****
>
> ****
> > unexpected control-type PDUs until one of the following events has
> occurred:****
>
> >** **
>
> > 2. A system configurable timeout period, say one second, has expired.***
> *
>
> ** **
>
> That’s ok, I think additional text is needed somewhere to point out that
> there is****
>
> known target code that sends one PDU and then delays - that might go in
> the additional****
>
> summary text that I suggest above, and also should be included as part of
> the****
>
> rationale for:****
>
> ** **
>
> > For an initiator, if the iSERHelloRequired key is negotiated to No or
> NotUnderstood,****
>
> > 2. Allocate one or more buffers for receiving unexpected control-type
> PDUs from****
>
> > of the unexpected control-type PDUs causing the RCaP connection to close
> before****
>
> ****
> ** **
> --David****
> ** **
> *From:* [email-masked] [mailto:[email-masked]] *On Behalf
> Of *Michael Ko
> *Sent:* Tuesday, July 24, 2012 8:27 PM
> *To:* [email-masked]
> *Subject:* [storm] Proposed text changes for the connection allocation
> problem****
>
> ** **
>
> These are the changes I intend to incorporate into the existing draft to
> solve the connection allocation problem by using the iSER Hello exchanges.
> Please comment.
>
> 5.1.1  Initiator Behavior
>
> original:
>
> If the outcome of the iSCSI negotiation is to enable iSER-assisted mode,
> then on the initiator side, prior to sending the Login Request with the T
> (Transit) bit set to 1 and the NSG (Next Stage) field set to
> FullFeaturePhase, the iSCSI Layer MUST request the iSER Layer to allocate
> the connection resources necessary to support RCaP by invoking the
> Allocate_Connection_Resources Operational Primitive.
>
> replacement:
>
> If the outcome of the iSCSI negotiation is to enable iSER-assisted mode,
> then on the initiator side, prior to sending the Login Request with the T
> (Transit) bit set to 1 and the NSG (Next Stage) field set to
> FullFeaturePhase, the iSCSI Layer SHOULD request the iSER Layer to allocate
> the connection resources necessary to support RCaP by invoking the
> Allocate_Connection_Resources Operational Primitive.
>
> deleted:
>
> If the iSER Layer at the initiator is successful in allocating the
> connection resources necessary to support RCaP, the following events MUST
> occur in the specified sequence:  [The numbered items will be retained but
> without the numbering.]
>
> new:
>
> If the iSCSI Layer on the initiator side allocates the connection
> resources to support RCaP only after it receives the final Login Response
> PDU from the target, then it may not be able to handle the number of
> unexpected iSCSI control-type PDUs (as declared by the
> MaxOutstandingUnexpectedPDUs key from the initiator) that can be sent by
> the target before the buffer resources are allocated at the initiator
> side.  In this case the iSERHelloRequired key MUST be negotiated to Yes so
> that the initiator can allocate the connection resources before sending the
> iSER Hello Message.  See section 5.1.3 for more details.
>
> The connection resources can be allocated at the initiator side after it
> has received the final Login Response PDU from the target.  To prevent the
> potential problem caused by the target sending the number of unexpected
> iSCSI control-type PDUs as determined by the MaxOustandingUnexpectedPDUs
> key allowed in the FullFeaturePhase, the iSER Hello exchanges can be used.
> This allows the initiator to allocate the connection resources before
> sending the iSER Hello Message.  The iSERHelloRequired key is used by the
> initiator to determine if it is dealing with a target that supports the
> iSER Hello exchanges.
>
> As the iSERHelloRequired key is not defined in [RFC5046], an
> initiator/target must be able to deal with a target/initiator that does not
> understand the key or support the iSER Hello exchanges.
>
> For a target, if the iSERHelloRequired key is not negotiated, it MUST
> still respond with the iSER HelloReply Message when it recieves the iSER
> Hello Message.  If the iSERHelloRequired key is negotiated to No or
> NotUnderstood, a target MAY choose to terminate the connection; otherwise
> it SHOULD delay sending any unexpected control-type PDUs until one of the
> following events has occurred:
>
> For an initiator, if the iSERHelloRequired key is negotiated to No or
> NotUnderstood, it MAY choose to terminate the connection; otherwise it
> SHOULD do one of the following:
>
> 2. Allocate one or more buffers for receiving unexpected control-type PDUs
> from the target before sending the final Login Request PDU.  This reduces
> the possibility of the unexpected control-type PDUs causing the RCaP
> connection to close before the connection resources have been allocated.
>
> Conversely, it is a protocol error if the iSER Hello Message is sent by
> the iSER Layer at the initiator when iSERHelloRequired is negotiated to
> "No".
>
> new:
>
> Conversely, if the iSER Hello Message is sent by the iSER Layer at the
> initiator when iSERHelloRequired is negotiated to "No", the iSER Layer at
> the target MAY treat this as a protocol error or respond with an iSER
> HelloReply Message.
>
> Mike****
>