Subject: iSCSI - bookmarks change

From: "Julian Satran" <[email-masked]>
Date: Tue, 4 Sep 2001 12:19:20 +0300

Now that text commands are out of login (and we don't have to care too much
about the immediate version) we can rationalize the bookmarks.

Here is my suggestion for the new text request/response:

1.1  Text Command

   The Text Command is provided to allow the exchange of information and
   for future extensions. It permits the initiator to inform a target of
   its capabilities or to request some special operations.

   A connection SHOULD have only one outstanding text command at any given
   time.

   Byte /    0       |       1       |       2       |       3       |
      /              |               |               |               |
     |7 6 5 4 3 2 1 0|7 6 5 4 3 2 1 0|7 6 5 4 3 2 1 0|7 6 5 4 3 2 1 0|
     +---------------+---------------+---------------+---------------+
    0|X|I| 0x04      |F|B| Rsvd      | Reserved                      |
     +---------------+---------------+---------------+---------------+
    4| Reserved      | DataSegmentLength                             |
     +---------------+---------------+---------------+---------------+
    8| Reserved                                                      |
     +                                                               +
   12|                                                               |
     +---------------+---------------+---------------+---------------+
   16| Initiator Task Tag                                            |
     +---------------+---------------+---------------+---------------+
   20| Reserved                                                      |
     +---------------+---------------+---------------+---------------+
   24| CmdSN                                                         |
     +---------------+---------------+---------------+---------------+
   28| ExpStatSN                                                     |
     +---------------+---------------+---------------+---------------+
   32/ Reserved                                                      /
    +/                                                               /
     +---------------+---------------+---------------+---------------+
   48| Digests if any...                                             |
     +---------------+---------------+---------------+---------------+
     / DataSegment (Text)                                            /
    +/                                                               /
     +---------------+---------------+---------------+---------------+

1.1.1     F (Final) Bit

   When set to 1 it indicates that this is the last or only text command in
   a sequence of commands; otherwise it indicates that more commands will
   follow.

1.1.2     I - Immediate

   During the Login Phase (the current stage part of CNxSG is either
   SecurityNegotiation or LoginOperationalNegotiation), the Text command
   SHOULD be issued as an immediate command (I=1).

1.1.3     B - bookmark bit

   The bookmark bit set to 1 tells the target to continue from its last
   bookmark for the specific Initiator Task Tag and thus allows a target to
   transfer a large amount of textual data over a sequence of
   text-command/text-response exchanges.

   The bookmark bit set to 0 tells the target that this is a new Text
   request; the target should reset any bookmark it holds on the specific
   Initiator Task Tag.

   A target MAY reject an old Bookmark.

   Bookmark generation at target is detailed in 2.11.2.

   Long text responses are handled as in the following example:

      I->T Text SendTargets=all (F=1,B=0)
      T->I Text <part 1> (F=0,B=1)
      I->T Text <empty> (F=1,B=1)
      T->I Text <part 2> (F=0,B=1)
      I->T Text <empty> (F=1,B=1)
      ...
      T->I Text <part n> (F=1,B=0)

1.1.4     Initiator Task Tag

   The initiator assigned identifier for this Text Command.
   If the command is sent as part of a sequence of commands (e.g., the
   Login Phase or a sequence of Text commands) the Initiator Task Tag MUST
   be the same for all the commands within the sequence (similar to linked
   SCSI commands).

1.1.5     Text

   The initiator sends the target a set of key=value or key=list pairs
   encoded in UTF-8 Unicode. All the text keys and text values specified in
   this document are to be presented and interpreted in the case they
   appear in this document (they are case sensitive). The key and value are
   separated by a '=' (0x3d) delimiter. Every key=value pair (including the
   last or only pair) MUST be followed by null (0x00) delimiter.  A list is
   a set of values separated by comma (0x2c). Large binary items can be
   encoded using their hexadecimal representation (e.g., 8190 is 0x1ffe) or
   decimal representation. The maximum length of an individual value (not
   its string representation) is 255 bytes.

   The data lengths of a text command or response MUST NOT exceed 4096
   bytes.  Key names MUST NOT exceed 63 bytes. Key values MUST NOT exceed
   255 bytes.

   Character strings are represented as plain text. Numeric and binary
   values are represented using either decimal numbers or the hexadecimal
   0xffff notation. Upper and lower case letters may be used
   interchangeably in hexadecimal notation (i.e., 0x1aBc, 0x1AbC and 0x1ABC
   are equivalent).

   The target responds by sending its response back to the initiator. The
   response text format is similar to the request text format.

   Some basic key=value pairs are described in Appendix A and Appendix D.
   All keys in Appendix D, except for the X- extension format, MUST be
   supported by iSCSI initiators and targets. Keys in Appendix A MUST be
   supported only when the function they refer to is mandatory to
   implement.

   Manufacturers may introduce new keys by prefixing them with X- followed
   by their (reversed) domain name, for example the company owning the
   domain acme.com can issue:

      X-com.acme.bar.foo.do_something=0000000000000003

   Any other key not understood by the target may be ignored by the target
   without affecting basic function. However the Text Response for a key
   that was not understood MUST be key=NotUnderstood.

   Text operations are usually meant for parameter setting/negotiations but
   can be used also to perform some long lasting operations.

   Text operations that will take a long time should be placed in their own
   Text command.

1.2

Text Response

   The Text Response PDU contains the target's responses to the initiator's
   Text Command. The format of the Text field matches that of the Text
   Command.

   Byte /    0       |       1       |       2       |       3       |
      /              |               |               |               |
     |7 6 5 4 3 2 1 0|7 6 5 4 3 2 1 0|7 6 5 4 3 2 1 0|7 6 5 4 3 2 1 0|
     +---------------+---------------+---------------+---------------+
    0|1|1| 0x24      |F|B| Rsvd      | Reserved                      |
     +---------------+---------------+---------------+---------------+
    4| Reserved      | DataSegmentLength                             |
     +---------------+---------------+---------------+---------------+
    8| Reserved                                                      |
     +                                                               +
   12|                                                               |
     +---------------+---------------+---------------+---------------+
   16| Initiator Task Tag                                            |
     +---------------+---------------+---------------+---------------+
   20| Reserved                                                      |
     +---------------+---------------+---------------+---------------+
   24| StatSN                                                        |
     +---------------+---------------+---------------+---------------+
   28| ExpCmdSN                                                      |
     +---------------+---------------+---------------+---------------+
   32| MaxCmdSN                                                      |
     +---------------+---------------+---------------+---------------+
   36/ Reserved                                                      /
    +/                                                               /
     +---------------+---------------+---------------+---------------+
   48| Digests if any...                                             |
     +---------------+---------------+---------------+---------------+
     / DataSegment (Text)                                            /
    +/                                                               /
     +---------------+---------------+---------------+---------------+

1.2.1     F (Final) Bit

   When set to 1 in response to a text command with the Final bit set to 1
   the F bit indicates that the target has finished the current stage of
   the operation or the whole operation.  Otherwise if set to 0 in response
   to a text command with the Final Bit set to 1 it indicates that the
   target has more work to do (invites a follow-on text command).  A text
   response with the F bit set to 1 in response to a text command with the
   F bit set to 0 is a protocol error.

   A text response with a F bit set to 1 MUST NOT contain key=value pairs
   that may require additional answers from the initiator.

1.2.2     B - bookmark Bit

   Whenever a target can't transfer all the remaining text data in a single
   Text response, it attempts to set an internal bookmark. If successful,
   the target associates the bookmark with the Initiator Task Tag.  The
   target indicates that it holds a bookmark for the specific Initiator
   Task Tag by setting the bookmark (B) bit to 1 in the Text response. The
   target resets the internal bookmark associated with a given Initiator
   Task Tag if it receives a Text request with the specified Initiator Task
   Tag with the bookmark bit set to 0.  When a target can't transfer all
   the text data in a single text response and it cannot set an internal
   bookmark it rejects the Text request with an appropriate Reject code. A
   target may reset its internal bookmark(s) after some time in order to
   reclaim resources associated with the bookmark and reject subsequent
   Text requests with the bookmark bit set to 1.

   When all the text data fit in a single Text response the bookmark bit of
   the response is set to 0 and the bookmark associated with the Initiator
   Task Tag is reset.

1.2.3     Initiator Task Tag

   The Initiator Task Tag matches the tag used in the initial Text Command
   or the Login Initiator Task Tag.

1.2.4     Text Response Data

   The Text Response Data Segment contains responses in the same key=value
   format as the Text Command and with the same length and coding
   constraints. Appendix A and Appendix D lists some basic Text Commands
   and their Responses.

   Although the initiator is the requesting party and controls the
   request-response initiation and termination the target can offer
   key=value pairs of its own as part of a sequence and not only in
   response to an identical key=value pair offered by the initiator.

   A Key=value pair must be confined to a given text response even in the
   presence of bookmark - i.e., it must start and end within one Text
   Response.

The Reject Reasons table looks like:

1.1.1     Reason

   The reject Reason is coded as follows:

   +------+-----------------------------------------+------------------+
   | Code | Explanation                             | Can the original |
   | (hex)|                                         | PDU be re-sent?  |
   +------+-----------------------------------------+------------------+
   | 0x01 | Format Error                            | no               |
   |      |                                         |                  |
   | 0x02 | Data (payload) Digest Error             | yes  (Note 1)    |
   |      |                                         |                  |
   | 0x03 | Data-SNACK Reject                       | yes              |
   |      |                                         |                  |
   | 0x04 | Protocol Error (e.g., SNACK request for | no               |
   |      | a status that was already acknowledged) |                  |
   |      |                                         |                  |
   | 0x05 | Command not supported in this session   | no               |
   |      | type                                    |                  |
   |      |                                         |                  |
   | 0x06 | Immediate Command Reject - too many     | yes              |
   |      | immediate commands                      |                  |
   |      |                                         |                  |
   | 0x07 | Bookmark Reject - No bookmark for this  | no               |
   |      | Initiator Task Tag                      |                  |
   |      |                                         |                  |
   | 0x08 | Bookmark Reject - Can't generate        | yes              |
   |      | bookmark - out of resources             |                  |
   |      |                                         |                  |
   | 0x0f | Full Feature Phase Command before login | no               |
   +------+-----------------------------------------+------------------+

 Comments?

 Julo

---

From: "Ayman Ghanem" <[email-masked]>
Date: Tue, 4 Sep 2001 09:05:21 -0500

Julian,

The revised text command looks fine. There are a couple of sections that
need cleanup:

1- Section 1.1.2
2- The reject message should no longer have "bookmark rejected" reasons

-Ayman

> -----Original Message-----
> From: [email-masked] [mailto:[email-masked]]On Behalf Of
> Julian Satran
> Sent: Tuesday, September 04, 2001 4:19 AM
> To: [email-masked]
> Subject: iSCSI - bookmarks change
>
>

---

From: "Eddy Quicksall" <[email-masked]>
Date: Wed, 5 Sep 2001 09:36:34 -0400

Section "4. Login Phase" of your previous EMAIL (iSCSI - Login changes and
the latest agreements) says:

	The login phase is implemented via login request and responses only.

But below you say:

	During the Login Phase ... the Text command
   SHOULD be issued as an immediate command (I=1).

These seem contradictory.

Eddy

-----Original Message-----
From: Julian Satran [mailto:[email-masked]]
Sent: Tuesday, September 04, 2001 5:19 AM
To: [email-masked]
Subject: iSCSI - bookmarks change

1.1  Text Command

   Byte /    0       |       1       |       2       |       3       |
      /              |               |               |               |

1.1.1     F (Final) Bit

1.1.2     I - Immediate

1.1.3     B - bookmark bit

1.1.4     Initiator Task Tag

1.1.5     Text

1.2

Text Response

   Byte /    0       |       1       |       2       |       3       |
      /              |               |               |               |

1.2.1     F (Final) Bit

1.2.2     B - bookmark Bit

1.2.3     Initiator Task Tag

1.2.4     Text Response Data

1.1.1     Reason

 Comments?

 Julo

---

From: "Eddy Quicksall" <[email-masked]>
Date: Wed, 5 Sep 2001 11:05:08 -0400

Some suggestions:

1.1.3 says the following:

 ... may I suggest that we say "Test Response" for T->I cases?

1.2 Text Response says:

 ... can we document what to do about it? e.g., drop the connection?

-----Original Message-----
From: Julian Satran [mailto:[email-masked]]
Sent: Tuesday, September 04, 2001 5:19 AM
To: [email-masked]
Subject: iSCSI - bookmarks change

1.1  Text Command

   Byte /    0       |       1       |       2       |       3       |
      /              |               |               |               |

1.1.1     F (Final) Bit

1.1.2     I - Immediate

1.1.3     B - bookmark bit

1.1.4     Initiator Task Tag

1.1.5     Text

1.2

Text Response

   Byte /    0       |       1       |       2       |       3       |
      /              |               |               |               |

1.2.1     F (Final) Bit

1.2.2     B - bookmark Bit

1.2.3     Initiator Task Tag

1.2.4     Text Response Data

1.1.1     Reason

 Comments?

 Julo

---

From: Mark Bakke <[email-masked]>
Date: Wed, 05 Sep 2001 10:29:49 -0500

Julian-

This is getting a lot simpler.  We can go even further by turning
the SHOULD below into a MUST.

If we say:

   A connection MUST have only one outstanding text command at
   any given time.

This means that the bookmark flag references the connection on
which the text command is done, and makes the Initiator Task
Tag unnecessary in both the text command and response.  We can
turn these into reserve fields, and reference the connection
instead of the ITT when discussing bookmarked commands.

--
Mark

-- 
Mark A. Bakke
Cisco Systems
[email-masked]
763.398.1054

---

From: "Rod Harrison" <[email-masked]>
Date: Wed, 5 Sep 2001 19:20:20 +0100

Mark,

	If we do this it will require the initiator to do some
special case handling of text responses. The Initiator Task
Tag is the initiators only way of finding its data
structures pertaining to the command, removing it will
require the initiator to "know" about the location of the
text command. This in itself is not terribly difficult, but
it is a special case. I would prefer we leave the ITT where
it is and allow initiators to use a common method for
decoding responses. We can still mandate only one text
command per connection.

	- Rod

-----Original Message-----
From: [email-masked] [mailto:[email-masked]]On
Behalf Of
Mark Bakke
Sent: Wednesday, September 05, 2001 4:30 PM
To: Julian Satran
Cc: [email-masked]
Subject: Re: iSCSI - bookmarks change

Julian-

This is getting a lot simpler.  We can go even further by
turning
the SHOULD below into a MUST.

If we say:

   A connection MUST have only one outstanding text command
at
   any given time.

This means that the bookmark flag references the connection
on
which the text command is done, and makes the Initiator Task
Tag unnecessary in both the text command and response.  We
can
turn these into reserve fields, and reference the connection
instead of the ITT when discussing bookmarked commands.

--
Mark

Julian Satran wrote:
>    Byte /    0       |       1       |       2       |
3       |
>       /              |               |               |
|
+---------------+---------------+---------------+-----------
----+
>     0|X|I| 0x04      |F|B| Rsvd      | Reserved
|
+---------------+---------------+---------------+-----------
----+
>     4| Reserved      | DataSegmentLength
|
+---------------+---------------+---------------+-----------
----+
>     8| Reserved
|
>      +
+
>    12|
|
+---------------+---------------+---------------+-----------
----+
>    16| Initiator Task Tag
|
+---------------+---------------+---------------+-----------
----+
>    20| Reserved
|
+---------------+---------------+---------------+-----------
----+
>    24| CmdSN
|
+---------------+---------------+---------------+-----------
----+
>    28| ExpStatSN
|
+---------------+---------------+---------------+-----------
----+
>    32/ Reserved
/
>     +/
/
+---------------+---------------+---------------+-----------
----+
>    48| Digests if any...
|
+---------------+---------------+---------------+-----------
----+
>      / DataSegment (Text)
/
>     +/
/
+---------------+---------------+---------------+-----------
----+
>    Byte /    0       |       1       |       2       |
3       |
>       /              |               |               |
|
+---------------+---------------+---------------+-----------
----+
>     0|1|1| 0x24      |F|B| Rsvd      | Reserved
|
+---------------+---------------+---------------+-----------
----+
>     4| Reserved      | DataSegmentLength
|
+---------------+---------------+---------------+-----------
----+
>     8| Reserved
|
>      +
+
>    12|
|
+---------------+---------------+---------------+-----------
----+
>    16| Initiator Task Tag
|
+---------------+---------------+---------------+-----------
----+
>    20| Reserved
|
+---------------+---------------+---------------+-----------
----+
>    24| StatSN
|
+---------------+---------------+---------------+-----------
----+
>    28| ExpCmdSN
|
+---------------+---------------+---------------+-----------
----+
>    32| MaxCmdSN
|
+---------------+---------------+---------------+-----------
----+
>    36/ Reserved
/
>     +/
/
+---------------+---------------+---------------+-----------
----+
>    48| Digests if any...
|
+---------------+---------------+---------------+-----------
----+
>      / DataSegment (Text)
/
>     +/
/
+---------------+---------------+---------------+-----------
----+
>    The reject Reason is coded as follows:
+------+-----------------------------------------+----------
--------+
>    | Code | Explanation                             | Can
the original |
+------+-----------------------------------------+----------
--------+
>    |      |                                         |
|
>    |      |                                         |
|
>    |      |                                         |
|
>    |      |                                         |
|
>    |      | type                                    |
|
>    |      |                                         |
|
>    |      |                                         |
|
>    |      |                                         |
|
>    |      |                                         |
|
>    | 0x0f | Full Feature Phase Command before login | no
|
+------+-----------------------------------------+----------
--------+

--
Mark A. Bakke
Cisco Systems
[email-masked]
763.398.1054

---

From: Mark Bakke <[email-masked]>
Date: Wed, 05 Sep 2001 15:01:46 -0500

Rod-

I like this solution; the intiator would still send a valid
ITT, but with one text command per connection, the target
could use either the ITT or the connection itself to find
the bookmark, whichever was more convenient.

Is there anyone out there that still wants more than one
outstanding text command on a connection?

--
Mark

-- 
Mark A. Bakke
Cisco Systems
[email-masked]
763.398.1054