Subject: iSCSI:Target Centric ISID assignments

From: "John Hufferd" <[email-masked]>
Date: Fri, 7 Sep 2001 14:29:32 -0700

(Same note as before with Subject line included.)
Let me see if I have everything together in the following regarding the
TSID centric assignment of ISID.

First Login:
1. Initiator contacts the target with TSID of zero, and ISID of zero
2. Target assigns the SSID by filling in the ISID and its own TSID.
3. Initiator remembers the ISID somehow.
4. If a parallel Session is establish (perhaps under a wedge driver) the
same thing will happen -- Initiator sends TSID=0 and ISID=0, except the
Target will assign a different ISID, since it is keeping state on the ISIDs
it has handed out to a specific iSCSI Initiator Node.  This will be a
different NEXUS not associated with any other session on that iSCSI
Initiator Node.
5. If a Multiple Connection per Session was established, the NON leading
connection will have the SSID (with TSID and ISID filled in), and things
will continue as currently documented.
6. If the Initiator goes down, and the parallel sessions need to
re-establish the NEXUS they come back with the TSID but may or may not have
the approprate  ISID (if they did not have a way to save it).
6a. If no ISID was used in the Initiator Login, the Target will
re-establish a new session.  The NEXUS that might have the Persistent
Reserves is still bound to the previous Session.
6b. If the New session wants to keep working with the old Persistent
Reserves, it needs explicitly blow-away the Reserves and Reclaim them (per
normal) using the Persistent Reserves Command Set.
6c. If the ISID had been saved by the Initiator, and a logon is reissued,
it can specify the ISID, and NO TSID, and the Target should then know that
a new session is starting.  The target should match the ISID with any
outstanding session it has.  That is, the Target should either match it up
the ISID with an old session (in some way), and continue, or it should Blow
the OLD session away.  David suggested a "Blow it away Bit". (This sounds
like option A and B all over again.)
6d. If the ISID was saved but it took so long that the Target's session
cleanup process, had thrown away the Old session, the Login just continues
as it does today.  The Initiator Session can not really determine if the
session continued, and it has it previous reserves, or if a new session was
created without the Reserves.  (Potential hitch.)  So the Option here is to
return an error message if there is no existing session.  The Initiator
will then need to understand this and somehow cause the reserves to be
issued again.  (Folks will say, that is what they do anyway so they will
always do that, so don't worry about it.)
(Comment:  This is now starting to get complicated again.)
7. In the case that the Initiator has "lost its way" and want to start
again,  the initiator will Login with either the current ISID, and TSID =0
or with both =0.
7a. Using the current ISID should cause the Initiator to handle it like 6c
above.  (Again, it sounds like the A, and B options).
7b. If both ISID and TSID are zero, it looks like 6a above.

OK, after looking at the above.  It seems to me that we can have this
assignment of ISID by the Target System. (It is kind of acting like a third
party ISID assigner.)  The rest of the documentation should be the same.
The issue of Option A or B is still with us. However, the issue of the
Rouge Initiator, that can not find its approprate ISID is solved. (Don't
you just love the emotive way I put that.) Also the assignment of ISIDs is
made easier for some implementations. (However, the approach suggested by
Jim Hafner seems simple enough.)

Now the discussion should be, did I miss something important, and is the
change worth it

.
.
.
John L. Hufferd
Senior Technical Staff Member (STSM)
IBM/SSG San Jose Ca
Main Office (408) 256-0403, Tie: 276-0403,  eFax: (408) 904-4688
Home Office (408) 997-6136
Internet address: [email-masked]

---

From: [email-masked]
Date: Fri, 7 Sep 2001 17:46:21 -0400

John's description is actually over-complicated because:
- The ISID either goes away or becomes a Target portal group number.
- The session recovery mechanism that he portrays as added complexity
	already exists and isn't affected by presence or absence of
	the ISID.  It doesn't seem to be well-documented in -07.
Partitioning the TSID into Target Portal Group + ID within group
makes it easy to figure out where to go to recover or blow away
a session.  Whether or not the Target Portal Group is used doesn't
matter to the following description - the ISID vanishes from John's
description, making things simpler, viz.:

First Login:
1. Initiator contacts the target with TSID of zero.
2. Target assigns the SSID by filling in the TSID.
3. Initiator remembers the SSID=TSID somehow.
4. If a parallel Session is established (perhaps under a wedge driver) the
	same thing will happen -- Initiator sends TSID=0, and the
	Target will assign a different TSID, since it is keeping 
	state on the TSIDs it has handed out to a specific iSCSI Initiator
Node.
	This will be a different NEXUS not associated with any other session
on
	that iSCSI Initiator Node.

Note that the target has the option of keeping TSID state globally, so that
TSID values are unique within a Target or a Target portal group.  This is
the Target implementer's choice, and can simplify session lookup.  This
particular comment applies independent of ISID removal, but is the reason
why I mentioned the idea of expanding the TSID, as I can foresee problems
with a 16 bit counter, but not with 24 or 32 bits.

5. If a Multiple Connection per Session was established, the  NON leading
	connection will have the TSID filled in, and things
	will continue as currently documented.
6. If the Initiator goes down, and the parallel sessions need to
	re-establish the NEXUS they supply the TSID.
6a. If they don't have the original TSID, they send TSID=0, and the Target
	will establish a new session.  The NEXUS that might have the
Persistent
	Reserves is still bound to the previous Session.
6b. If the New session wants to keep working with the old Persistent
	Reserves, it needs explicitly blow-away the Reserves and Reclaim
them (per
	normal) using the Persistent Reserves Command Set.
6c. If the TSID had been saved by the Initiator, and a logon is reissued,
	it can specify the TSID, and the Target should then know that
	a new session is starting.  The target should match the TSID with
	any outstanding session it has.  That is, the Target should either
	match it up the TSID with an old session (in some way), and
continue,
	or  it should Blow the OLD session away.  David suggested a "Blow it
	away Bit". (This sounds like option A and B all over again.)

Whether or not to have a "Blow it away Bit" vs. implicitly blowing away
the old session vs. failing the login is Option A/B/C all over again.
The X bit already has meaning in this context (for the 6c case the X
bit has to be zero in this case because 6d uses it), and hence can't be
reused here.

It's now considerably safer to blow away old connections because a non-zero
TSID only occurs on login when there was a previous session with that TSID
and some sort of error recovery or action on that previous session is
intended.
All the scenarios in which a "wrong" ISID causes a session to be blown away
incorrectly can't happen because the Initiator doesn't pick ISID values.

6d. If the ISID was saved but it took so long that the Target's session
	cleanup process, had thrown away the Old session, the Login just
continues
	as it does today.  The Initiator Session can determine if the
session
	continued, and it has it previous reserves, or if a new session was
	created without the Reserves.  

Actual recovery and continuation of the session has to set the X bit.  If
X is set on a Login command, and there's no existing session to log into,
then the Login has to fail in some fashion - I'm not sure that's documented
at the moment, and I think it needs to be regardless of the course we take.

	So the Option here is to return an error message if there is no
	existing session.  The Initiator will then need to understand this
	and somehow cause the reserves to be issued again.  (Folks will say,
	that is what they do anyway so they will always do that, so don't
	worry about it.)

	(John's Comment:  This is now starting to get complicated again.)

Correct, but not relevant.  This set of complications exists regardless of
what we do about the ISID as they're inherent in the session case of Error
Recovery and the use of the X-bit on Login.  This is an existing session
recovery mechanism, not something that gets added as a consequence of ISID
removal.

> 7. In the case that the Initiator has "lost its way" and want to start
	again,  the initiator will Login with TSID=0.

This is exactly case 6a above.  John's case 7a can't happen.

> party ISID assigner.)  The rest of the documentation should be the same.

Well, actually, it's simpler to get rid of the ISID and just use the TSID.
For reasons that Jim Hafner can probably explain better than I can, using
the ISID field to hold the Target portal group number on login helps tidy
up some issues in that area (and it also makes it easy for an Initiator
trying to recover or destroy an existing session to figure out exactly
which target interfaces will know about that session vs. be clueless).

> you just love the emotive way I put that.)

Keep in mind that the Rogue Initiator sure looks like a valid technical
objection to the Option A that seems to be otherwise favored in list
discussion.

> Also the assignment of ISIDs is made easier for some implementations.
> (However, the approach suggested by Jim Hafner seems simple enough.)

It's certainly simple logic.  The problem is that ISID assignment has to
be coordinated across all interfaces on a single Initiator - between code
from different vendors, administrative tools, and interaction with
logic in the OS platform that may be trying to coordinate ISID assignment,
there are many opportunities to get things wrong.

If this doesn't scare you, John's notion of having to coordinate ISID
assignment across all systems in a cluster because they share a common
iSCSI Initiator Name should.  FWIW, if the cluster example wants to
recover session state (e.g., persistent reserves) across host system
failures, it needs dynamic fault-tolerant ISID assignment across the
entire cluster (that should really scare you) or elimination of ISIDs
and ISID assignment (which is much simpler ;-) ).

Thanks,
--David

> -----Original Message-----
> From: John Hufferd [mailto:[email-masked]]
> Sent: Friday, September 07, 2001 4:54 PM
> To: [email-masked]
> Subject: 
> 
>

---

From: "John Hufferd" <[email-masked]>
Date: Fri, 7 Sep 2001 17:09:35 -0700

David,
I think you got the x bit wrong.  It is used in error recovery when the
session restart is needed.  Not when you boot up.

You also over started the case for CROSS system Recovery of Persistent
Reserves.  All that works as it does today.
The only thing that is different is that the Cluster has a single Node
Name.  That said nothing about the Persistent Reserves, unless your
proposal make that complicated in some way.   Setting up a cluster to have
a common iSCSI Initiator Node Name should work in your proposal and in the
current way we do things.

I do not think the TSID only is any simpler then the ISID and TSID, just
different, and will cause more rework of the documentation.

I am sure that Jim want to say why the Portal Tag is needed, so I will wait
for that, but it sounds like target implementation on how it creates its
TSID.

.
.
.
John L. Hufferd

[email-masked] on 09/07/2001 02:46:21 PM

To:   John Hufferd/San Jose/IBM@IBMUS, [email-masked]
cc:
Subject:  RE: iSCSI:Target Centric ISID assignments

First Login:

Thanks,
--David

---

From: "Julian Satran" <[email-masked]>
Date: Sat, 8 Sep 2001 13:21:31 +0300

Blowing away or not is again with us. A careless guy will set always the
blow-away bit.
You have just moved the management responsibility to the target under the
assumption that the target
is "monolithic". But, as you well know, this is not true for many targets.
You have to have an allocation scheme at
target and a garbage collection.

I suggest we give this to the ND team and let them debate for a while
before getting back to us.
We may want to have them look at all aspects of SSID allocation and use.

Julo

[email-masked]@ece.cmu.edu on 08-09-2001 00:46:21

Please respond to [email-masked]

Sent by:  [email-masked]

First Login:

Thanks,
--David

---

From: [email-masked] (Sandeep Joshi)
Date: Sun, 9 Sep 2001 12:47:44 -0400 (EDT)

Agree with the extra responsibility added on the target.

But one point here ..the target may not be monolithic
but one assumes it would atleast be "monogenic" (single-vendor)
thereby enabling it to disallow multiple nexuses being
started with the same <initiatorName,ISID>

The monogenic property may not hold for initiators so 
a scheme which works without HBA cooperation is preferred
over one which requires cooperation.   

The schemes presented by Jim in a previous email (a windows 
registry counter) still assume all HBAs follow that 
assignment model.

The ISID namespace seems too small to partition among
vendors.  If it were possible to add a vendor-unique
identifier to the login command/phase (and maybe nexus), 
we may solve the problem without burdening the target.

-Sandeep

---

From: "Jim Hafner" <[email-masked]>
Date: Mon, 10 Sep 2001 08:46:11 -0700

Sandeep,

The vendor-unique id is yet another scheme to add another layer of
identifying information (beyond InitiatorName and ISID).  As I've mentioned
in other posts, I don't see structural reason for it (unless the ISID space
is just too small, then we can argue about where to put and what to put in
the additional space we create).

However, one problem with the vendor-unique suggestion is the same problem
we in N&D wrestled with, namely, not every component has an identifiable
vendor and that vendor have an unique identifier to go with it.  It's true
for HW components, but all those fancy SW versions of iSCSI initiators
don't.  That's what lead us to the iqn naming scheme for InitiatorName.

Jim Hafner

[email-masked] (Sandeep Joshi)@ece.cmu.edu on 09/09/2001
09:47:44 am

To:   Julian Satran/Haifa/IBM@IBMIL, [email-masked]
cc:
Subject:  RE: iSCSI:Target Centric ISID assignments

-Sandeep