Subject: Re: iscsi : default iscsi mode page settings.

From: "Julian Satran" <[email-masked]>
Date: Mon, 24 Sep 2001 21:03:21 +0300

I can sympathize with you wanting to use most of the parameters in iSCSI -
but the values are in fact restrictions that SCSI places on iSCSI.
Nevertheless the discussion is rather academic because SCSI can hand those
parameters to iSCSI.

SCSI can handle those parameters dynamically. iSCSI may have trouble
handling this type of negotiation dynamically over several connections.

Resource-wise (as Bob Snively has pointed out) those are SCSI issues.

A nice way out would be to ask T10 for a text mode negotiaton :-)

Julo

                    Santosh Rao                                                            
                    <santoshr@cup.       To:     [email-masked]                           
                    hp.com>              cc:                                               
                    Sent by:             Subject:     Re: iscsi : default iscsi mode page  
                    owner-ips@ece.        settings.                                        
                    cmu.edu                                                                

                    24-09-01 18:38                                                         
                    Please respond                                                         
                    to Santosh Rao                                                         

Julian,

The FirstBurstSize is an iscsi property and not an attribute of the scsi
ULP.
Evidence of this lies in its location itself, the disconnect-reconnect mode
page. This mode page is used to tune the parameters of the service delivery
subsystem (in our case, this would be iscsi).

I would like to re-iterate my request that FirstBurstSize be placed as a
login
key.

Regards,
Santosh

> -----Original Message-----
> From: Julian Satran [mailto:[email-masked]]
> Sent: Friday, September 21, 2001 11:06 PM
> To: [email-masked]
> Subject: Re: iscsi : default iscsi mode page settings.
>
> That is not my reading of the map. The unsolicited payload buffer size is
a
> "data sink" attribute. The data sink is the SCSI engine and not the iSCSI
> layer.
>
> Julo
>
>
>
>                     Santosh Rao
>
>                     <santoshr@cup.       To:     [email-masked]
>
>                     hp.com>              cc:
>
>                     Sent by:             Subject:     Re: iscsi : default
> iscsi mode
>                     owner-ips@ece.        page settings.
>
>                     cmu.edu
>
>
>
>
>
>                     21-09-01 21:50
>
>                     Please respond
>
>                     to Santosh Rao
>
>
>
>
>
> Julian,
>
> The FirstBurstSize is governing an iscsi parameter which is : "how much
> un-solicited
> data can the initiator send" ? I concur with Charles' suggestion that
> FirstBurstSize be
> moved to a login key.
>
> By doing this, we can improve scan times, due to the elimination of the
> mode
> sense/select & avoid the side effects of shared mode pages. This allows
> easier use of
> un-solicited data.
>
> Regards,
> Santosh

#### santoshr.vcf has been removed from this note on September 24 2001 by
Julian Satran

---

From: Santosh Rao <[email-masked]>
Date: Mon, 24 Sep 2001 12:28:43 -0700

Julian,

I'm confused by your response.

The SPC-2 description of Disconnect-Reconnect mode page indicates that :
"The parameters appropriate to each protocol and their interpretation for that protocol may
be specified in the individual protocol documents".

FYI, SPI[-4] has chosen not to attach any semantics to FirstBurstSize for the pSCSI
transport. Thus, iscsi is within its rights to declare this field as reserved and attach no
meaning to it in the mode page. The FirstBurstSize can be negotiated during iscsi login
through a login key.

> Nevertheless the discussion is rather academic because SCSI can hand those

Again, I'm confused by your response. The reasons I'm suggesting the use of a login key
instead of the mode page method are :

   * More accurate scope (applies only to this I-T nexus).

   * More optimal negotiation and reduced overhead in the establishment of the I-T nexus. (2
     less SCSI commands per I-T nexus establishment.).

   * Enables faster I/O scan times due to lesser on-the-wire activity during I-T nexus
     establishment.

   * Allows less room for error in the I-T nexus establishment (no possiblity of failure to
     establish I-T nexus due to mode sense/select command failure).

   * Avoids mode select wars that can occur when target uses shared mode pages.

   * Simpler initiator implementations since they can avoid embedding SCSI command set
     knowledge as well as code to build/parse SCSI commands. Also, they can avoid extra code
     that is required to snoop for CHECK CONDITION with (sense key=UA, ASC="mode parameters
     changed") in order to re-issue a mode sense to determine new values for FirstBurstSize.

   * Less code to interact with SCSI ULP application client to co-ordinate the mode page
     values b/n the ULP & LLP.

   * Can use un-solicited data from the very first scsi command in the session.

I don't consider any of the above reasons to be academic and would like to know which ones
among the above do you believe are academic and why ?

> handling this type of negotiation dynamically over several connections.

This is exactly the kind of stuff we don't need and should actually be trying to avoid. What
good does dynamically changing FirstBurstSize serve ? Dynamically changing FirstBurstSize
would only be achieved with least side-effects if :
1) The mode select implementation on target is not using shared mode pages.
2) The initiator has quiesced I/O prior to issuing the mode select for the change.

Neither of the above 2 conditions would typically apply and any dynamic change of
FirstBurstSize would only cause initiators to see a bunch of side-effects such as :
a) Active outbound I/Os aborted by the target with a CHECK CONDITION due to "not enough
un-solicited data".
b) UA CHECK CONDITION for "mode parameters changed".

In the interests of simplification and avoiding disruption of active I/O, such modifications
must be avoided as far as possible. One way to achieve that is to use a login key and make
it LO.

> A nice way out would be to ask T10 for a text mode negotiaton :-)

Once again, I'm perplexed by your response. I'm not saying that text mode negotiation is the
reason I suggest moving this to a login key. The main objective is to isolate such
negotiation within the iscsi layer in an iscsi specific PDU that is a part of the iscsi
login process.

Hope you will consider all of the above factors.

Thanks,
Santosh

ps : [I wonder if there are any others on this list who care to voice their opinion on this
issue. (??). ]

---

From: Santosh Rao <[email-masked]>
Date: Tue, 25 Sep 2001 10:36:18 -0700

Julian,

I did not hear back from you on my comments below.

What is the closure on this issue ? I hear at least 4 people who have concerns regarding the
use of mode pages for FirstBurstSize, if I understood John Hufferd's comments right. (Charles
Binford, myself, Sanjeev, John Hufferd). At least 3 of them would prefer to see login keys
being used.

There are the separate issues of :

   * iscsi's specification of defaults for its mode pages which is not in line with mode page
     usage. This impacts the target's ability to enforce values other than the protocol
     specified default, if the initiator were to not use mode sense/select.

   * default settings for login keys.

   * Is there a need for the "LUN Control Mode Page" and whether "Enable CRN" should be in a
     transport specific mode page ?

which need to be driven to closure as well.

Regards,
Santosh

---

From: "Mallikarjun C." <[email-masked]>
Date: Tue, 25 Sep 2001 15:10:52 -0700

Julian and Santosh,

Whether FirstBurstSize should be a text key or not,
IMHO, is (mostly) dependent on the current implementations.
Santosh rightly pointed out several drawbacks of 
mandating a new set of mode select/sense operations
that are totally iSCSI-specific since the legacy SCSI
stack (class drivers, services/midlayer) will have no
clue about them.  OTOH, if most of today's SCSI stacks
_do_ perform this mode parameter manipulation in disconnect-
reconnect mode page, that would be a strong argument to 
leave FirstBurstSize as a mode page parameter.

A somewhat larger issue is whether this should be a nexus
parameter of a given I-T nexus (which login keys enable),
or not (which a *typical* shared mode page would preclude).
While I cannot identify an obvious advantage of using
a different FirstBurstSize for each I-T nexus in a 
target, the fact that the login key enables either shared
or per-nexus parameter in a given target implementation 
seems to tilt the balance in favor of a login key.

Regards.
-- 
Mallikarjun 

Mallikarjun Chadalapaka
Networked Storage Architecture
Network Storage Solutions Organization
MS 5668	Hewlett-Packard, Roseville.
[email-masked]

---

From: Santosh Rao <[email-masked]>
Date: Tue, 25 Sep 2001 16:37:10 -0700

All,

In order to give the group some further perspective on the in-effeciencies of using a mode
sense/select based mechanism for negotiating iscsi parameters (especially, numerical parameters
like MaximumBurstSize & FirstBurstSize), I've outlined further below the steps an initiator would
need to take to negotiate iscsi numerical parameters through mode select/sense.

Note that performance aggresive initiators would like to negotiate FirstBurstSize to as high a
value as possible in order to boost performance on their WRITE I/Os. Thus, the below negotiation
would be required, for at least, FirstBurstSize, if one were to use mode select/sense.

Steps involved in mode select/sense negotiation :
====================================

When initiator needs to only read the target's mode page settings
-----------------------------------------------
1) Issue a mode sense(page control = "current values") for the desired mode page.

When initiator needs to negotiate mode page settings (i.e. read and modify settings)
--------------------------------------------------------
1) Issue a mode sense(page control = "changable values") for the desired mode page. This allows
the initiator to determine which parameters it is allowed to change. If none of the parameters
are changable, then, the target responds with a CHECK CONDITION scsi status to the mode sense.
(sense key = "illegal request", asc = "invalid field in cdb"). In this case, initiator issues a
mode sense(page control = "current values") to determine the target's current settings for those
parameters.

2) If the initiator could determine that the parameters are changable, it then issues a
mode select(sp = 1) or mode select(sp = 0) for the desired mode page. (sp = save page. depending
on whether the initiators wishes to save the mode page change).

3) The target may not support the exact value requested by the initiator. It may then choose to
perform parameter rounding and sets the parameter to a rounded value. It then fails the mode
select with a CHECK CONDITION scsi status (sense key = "recovered error", asc = "rounded
parameter").

The initiator on seeing the above CHECK CONDITION would need to perform another mode sense("page
control = current values") to determine the final negotiated value chosen by the target.

The target may also choose not to implement parameter rounding. In this case, it would respond to
the mode select from the initiator with a CHECK CONDITION scsi status with (sense key = "illegal
request", asc = "invalid field in parameter list"). In this case, the initiator has failed to
change the target's current parameters and it would need to perform a mode sense(page control =
"current values") to determine the current page setting.

As a short-cut, the initiator may choose to skip step (1) and attempt to change a parameter and
then figure out the result the hard way. Note that in this case if any 1 field was non-changable,
then, all other parameter negotiations within that page would also be failed.

As you can see for yourself from the above, to negotiate any numerical parameter, the initiator
will have to perform 2 mode sense operations and 1 mode select operation per mode page. The above
steps are a very cumbersome & ineffecient negotiation technique, compared to the alternative of a
simple 1-step handshake that the login key negotiation provides. An added advantage is that usage
of login keys implies no additional on-the-wire SCSI commands for the purpose of parameter
negotiation.

Given the above inefficiencies, I suggest that iscsi skip the use of mode select/sense technique
for negotiation and stick to login keys as far as possible. In any case, all of these mode pages
have transport specific semantics and iscsi is within its rights to not use these mode pages.

Comments ?

Thanks,
Santosh

---

From: Santosh Rao <[email-masked]>
Date: Wed, 26 Sep 2001 19:28:55 -0700

All,

With the issue of mode page vs. login keys having [almost] drawn to a close, I would like to
raise the below issues again, on the subject of default values for login keys and other iscsi
parameters :

   * In keeping with traditional use of scsi mode pages, iscsi should not specify any default
     settings for any mode pages that continue to exist for iscsi. "Default values" for mode
     pages are target specific and should not be bound to the protocol draft.

   * MORE IMPORTANTLY, I read the default for EMDP as being set to 1  :-<  This implies that
     targets must be always prepared to deal with out of order data and initiators must be
     prepared to deal with out of order data, unless the initiator performs a mode select to
     disable it. [which defeats all the previous advantages gained thru mandating use of login
     keys for other negotiations.]. A default, if it were to exist, should be 0. (in-order, by
     default).

   * Conservative specification of defaults for login keys along the following lines :
                            MaxConnections = 1
                            FMarker = "no"
                            InitialR2T = "yes"
                            BidiInitialR2T = "yes"
                            ImmediateData = "no"
                            DataPDULength = 16
                            MaxOutstandingR2T = 1
                            DataPDUInOrder = "yes"
                            ErrorRecoveryLevel = 0
                            SessionType = "normal"

   * Should the iscsi protocol require a "Lun Control Mode Page"? IOW, is an EnableCRN bit
     required at the transport layer ? If the device server capability is to be negotiated , I
     suggest this bit be moved to a SCSI ULP Mode Page such as the "Control Mode Page", through a
     T10 change as a part of the SCSI changes being driven by iscsi.

Comments ?

Thanks,
Santosh

---

From: "Eddy Quicksall" <[email-masked]>
Date: Thu, 27 Sep 2001 11:22:23 -0400

I like your defaults below.

But, section 5 says:

 The initial Login request MUST include the InitiatorName and
 SessionType key=value pairs.

Since SessionType is REQUIRED, naming a default would imply a possible typo
in the spec.

Eddy

-----Original Message-----
From: Santosh Rao [mailto:[email-masked]]
Sent: Wednesday, September 26, 2001 10:29 PM
To: [email-masked]
Subject: iscsi : iscsi parameter default values

All,

With the issue of mode page vs. login keys having [almost] drawn to a
close, I would like to
raise the below issues again, on the subject of default values for login
keys and other iscsi
parameters :

   * In keeping with traditional use of scsi mode pages, iscsi should
not specify any default
     settings for any mode pages that continue to exist for iscsi.
"Default values" for mode
     pages are target specific and should not be bound to the protocol
draft.

   * MORE IMPORTANTLY, I read the default for EMDP as being set to 1
:-<  This implies that
     targets must be always prepared to deal with out of order data and
initiators must be
     prepared to deal with out of order data, unless the initiator
performs a mode select to
     disable it. [which defeats all the previous advantages gained thru
mandating use of login
     keys for other negotiations.]. A default, if it were to exist,
should be 0. (in-order, by
     default).

   * Conservative specification of defaults for login keys along the
following lines :
                            MaxConnections = 1
                            FMarker = "no"
                            InitialR2T = "yes"
                            BidiInitialR2T = "yes"
                            ImmediateData = "no"
                            DataPDULength = 16
                            MaxOutstandingR2T = 1
                            DataPDUInOrder = "yes"
                            ErrorRecoveryLevel = 0
                            SessionType = "normal"

   * Should the iscsi protocol require a "Lun Control Mode Page"? IOW,
is an EnableCRN bit
     required at the transport layer ? If the device server capability
is to be negotiated , I
     suggest this bit be moved to a SCSI ULP Mode Page such as the
"Control Mode Page", through a
     T10 change as a part of the SCSI changes being driven by iscsi.

Comments ?

Thanks,
Santosh