Subject: RE: iscsi : target port definition

From: "Jim Hafner" <[email-masked]>
Date: Fri, 14 Sep 2001 10:17:06 -0700

Eddy,

That certainly fits within the scope of what I described.

However, I don't think that's the "main purpose".  As I understand it, the
whole notion of target portal group was suggested to enable HW
implementations with multiple iSCSI HBAs that (a) had multiple network
interfaces/nics (b) could coordinate sessions across the network interfaces
*within their own HBA* but (c) didn't have an additional (complex) software
layer that could coordinate sessions across HBAs.   In that case, we wanted
a simple way to describe to the initiator (in SendTargets), that even
though this target may have 4 ipaddress/tcpports available, they only work
in pairs (say) as far as multi-connection sessions (one pair of addresses
per HBA).  That information would help the initiator from generating a lot
of failed "add this connection to existing session" requests.   This model
turned out to have additional value in the SCSI modeling, so we took
advantage of it.

Jim Hafner

"Eddy Quicksall" <[email-masked]>@ece.cmu.edu on 09/14/2001
09:50:33 am

Please respond to <[email-masked]>

Sent by:  [email-masked]

To:   "'Santosh Rao'" <[email-masked]>, "'IPS Reflector'"
      <[email-masked]>
cc:
Subject:  RE: iscsi : target port definition

They way I see it is the Target Portal Group is a way for target
administrators to logically group things so the initiator knows which
addresses:ports to use when establishing a nexus.

Isn't that the main purpose?

Eddy

-----Original Message-----
From: Santosh Rao [mailto:[email-masked]]
Sent: Thursday, September 13, 2001 8:38 PM
To: IPS Reflector
Subject: iscsi : target port definition

Hello,

I have a question on the interpretation of the iscsi target port
definition. The iscsi rev 08 defines the iscsi target port to map to an
iscsi target portal group.

Thus, any iscsi target that wishes to allow multiple SCSI paths to be
established to the target node MUST provide at least 2 iscsi target
portal groups.

The above definition of an iscsi target port somewhat alters the
semantics of a target portal group. A target portal group, by
definition, is a collection of a set of network portals within the
target across which a session can be spanned.

Thus, if a target supports a multi-connection session spanning across
all its network portals, such a target would use a single target portal
group to indicate that 1 big fat session pipe could be established to
all its network portals. This, in turn, would have the side effect of
only providing 1 scsi path to the upper layer wedge drivers, if the
iscsi initiators establish a session per target portal group. [which is
the target port].

From an initiator's perspective, what should be the target side
end-point of an initiator's sessions when it may need to support upper
layer wedge drivers ? Should the initiator establish a session per
target
portal group [, in which case the above issue exists] ? Or, should it
establish a session per TargetAddress ??

Regards,
Santosh

---

From: "John Hufferd" <[email-masked]>
Date: Fri, 14 Sep 2001 12:35:27 -0700

What Jim said is correct, however, there will also be HBAs that know how to
coordinate their Multi Connection Sessions across Multiple HBAs.  However,
there may still be a limit to the number of such HBAs that are supported,
like that, in a single iSCSI Target Device (like a Max of 8 HBAs working
together etc.).  Or there might be other reasons not to make the whole unit
into a single Portal Group.

.
.
.
John L. Hufferd
Senior Technical Staff Member (STSM)
IBM/SSG San Jose Ca
Main Office (408) 256-0403, Tie: 276-0403,  eFax: (408) 904-4688
Home Office (408) 997-6136
Internet address: [email-masked]

Jim Hafner/Almaden/IBM@[email-masked] on 09/14/2001 10:17:06 AM

To:   <[email-masked]>, [email-masked]
cc:
Subject:  RE: iscsi : target port definition

Eddy,

Jim Hafner

Eddy

-----Original Message-----
From: Santosh Rao [mailto:[email-masked]]
Sent: Thursday, September 13, 2001 8:38 PM
To: IPS Reflector
Subject: iscsi : target port definition

Hello,

Regards,
Santosh

---

From: "KRUEGER,MARJORIE (HP-Roseville,ex1)" <[email-masked]>
Date: Fri, 14 Sep 2001 16:05:16 -0400

> Per the above definition, the I-T nexus target end point is that target
portal
> group, which is not the case when initiators choose to establish I-T nexi
> (sessions) to subsets of the target portal group, in order to export
multiple
> scsi paths to upper layer wedge drivers.

I think Jim is suggesting that the initiator would have to establish two
"SCSI initiator ports" to the same target portal group, which would
translate to two different ISIDs for this SCSI initiator node.  How does
that conflict with this definition?  Upper layer wedge drivers shouldn't
know the difference between two separate initiator ports providing the
multiple paths or one initiator port connecting to two different target
ports, right?

> Do you see any reasons why the definition of a target port should not be
> symmetric with the definition of the initiator port ? i.e. (iscsi target
name
> + TSID) = target port. (= both port name & port identifier). This would
more
> accurately model the target port to be the end point of the I-T nexus
(session).

Of course there are a number of combinations of possibilities for defining
what comprises the SCSI port within iSCSI.  Initially the model did specify
TSID as part of the target port identifier, but this seemed to create more
"rules" to enforce at the target side (more restrictive than necessary) and
the benefits of TSID vs target portal group were not compelling.  The idea
of target portal group as the target port endpoint seemed to provide the
necessary protection (against parallel nexus, etc) with the least amount of
enforcement rules.

Marj

---

From: "Jim Hafner" <[email-masked]>
Date: Fri, 14 Sep 2001 14:05:18 -0700

Eddy, Santosh,

Marj is correct in what I'm suggesting.

If the initiator only used some of the ipaddress/tcpport (what we called
Network Portals) in a Target Portal group for one session, then it is *not*
prohibited from requesting a second session to the remaing network portals
(or any other subset of the full target portal group) within that same
target portal group *provided it used a different ISID for that session*.
The ISID RULE as stated in the draft doesn't allow it. The point being that
once the ISID is generated on the initiator, a virtual SCSI Initiator Port
is created.  That virtual SCSI Initiator Port can use any and all network
paths/bandwidth/whatever available to it to from the lower layers to
connect to a SCSI Target Port (aka target portal group).  There is no
requirement to use all the resources of the portal group.

The reason for the asymmetry has been layed out a couple of times on this
reflector.  Marj reiterates it below.   I'll rephrase it more verbosely.
The primary reason we moved from the symmetric model was to enable simpler
implementations on the target, where an implementation may have to span
multiple HW components.  The TSID RULE (which effectively is the uniqueness
of SSIDs between iSCSI Initiator and iSCSI Target) can be enforced in this
model by an implementation that is totally local to a target portal group
(partition the TSIDs across the portal groups - so they each have a piece
of the TSID namespace).

Jim Hafner

"KRUEGER,MARJORIE (HP-Roseville,ex1)" <[email-masked]> on
09/14/2001 01:05:16 pm

To:   "'Santosh Rao'" <[email-masked]>, Jim Hafner/Almaden/IBM@IBMUS
cc:   [email-masked]
Subject:  RE: iscsi : target port definition

portal
multiple

name
more
(session).

Marj

---

From: "Jim Hafner" <[email-masked]>
Date: Mon, 17 Sep 2001 08:09:29 -0700

Eddy,

Let me restate:
1) I'm opposed to a *requirement* that from a target's point of view every
session it has active has a unique TSID; that is, for all sessions for all
initiators, there is only one occurence of a particular TSID.  This is too
strong a requirement and not needed. So, what I meant here by "independent
of initiator" is really "over all initiators".

2) Using a target portal group tag as a TSID is an *implementation* option
that fits the model.  You're right that this is "independent of initiator"
but in a different sense than I intended above.  But the point of that
example is that it is possible for the target to reuse the same TSID in
lots of different ways and still meet the minimum requirements.

So, I guess I'm failing to find the right language here.  The focus I'm
trying to put on this is that a requirement should only concern itself with
the relationship between a given iSCSI initiator and iSCSI target.
Requirements on ISID and TSID concerning simultaneous relationships between
one initiator and two or more targets and one target and two or more
initiators is not necessary.

I'm I getting clearer?

Jim Hafner

"Eddy Quicksall" <[email-masked]> on 09/15/2001 05:27:50 pm

To:   Jim Hafner/Almaden/IBM@IBMUS
cc:
Subject:  RE: iscsi : target port definition (private)

Jim,

Sorry but I don't understand what you are saying...

In one case, you say "opposed to making this [TSID unique independent of
initiator] the requirement".

But on the other hand, you say "each target portal group use its target
portal group tag as the value of the TSID".

Since there can't be repeated Target Portal Group tags within an iSCSI
Target Node, you have in fact created "unique independent of initiator"
TSIDs.

What am I missing?

Eddy

-----Original Message-----
From: Jim Hafner [mailto:[email-masked]]
Sent: Friday, September 14, 2001 5:24 PM
To: [email-masked]; [email-masked]
Subject: RE: iscsi : target port definition (private)

Santosh,

I'm not sure I quite caught the thrust of your note.  Are you supportive of
the model or questioning?

I get thrown by your comment:
 "If we change the definition of TSID to be unique within the target, then
I
think this is a good idea."

If I read this as "unique independent of initiator", then, as we've
discussed before, that's an implementation choice that goes far beyond what
any part of the model says (however we define SSID).   I'd be VERY opposed
to making this the requirement.    I'll explain the reasons if (a) my
interpretation of your quote is correct and (b) it is a formal
recommendation to this group.  If not, I'll be quiet on this point.  But I
have no objection to *you* implementing that way. :-{)

You say "the TSID can be the same for different initiators".  In fact, it
can be the same for the same initiator provided the ISID from that
initiator is different (so that the resulting SSID is unique)!  So, I could
implement my target by having each target portal group use its target
portal group tag as the value of the TSID for every session regardless of
initiator.   (That's a very simple partition logic!).  By enforcing the
ISID RULE, every session would get a unique SSID (ergo, I've met the TSID
RULE by default).  [Note: I could do the exact same thing on the initiator,
that is, make every initiator portal group use its portal group tag as its
ISID -- I wouldn't be able to use multiple sessions to the same target
portal group. but that's an implementation choice!]

Of course, such a target implementation wouldn't be able to use the TSID as
a handle to some session reference (though it could use the entire SSID),
but the spec doesn't say how one uses (internally) the TSID, just what
properties it has to have externally.

Are we all on the same page yet?

Jim Hafner

"Eddy Quicksall" <[email-masked]> on 09/14/2001 01:23:56 pm

cc:   <[email-masked]>

Currently the TSID can be the same for different initiators (now, in my
case, they'll all be unique).

If we change the definition of TSID to be unique within the target, then I
think this is a good idea.

But, I like the idea of Portal Groups as being a nice way for an
administrator to allocate the Network Portals and the Portal Group
properties.

TSID RULE: The iSCSI Target SHALL NOT select a TSID for a given login
request if the resulting SSID is already in use by an existing
session between that the target and the requesting iSCSI Initiator.
See 8.1.1.

Eddy

-----Original Message-----
From: Santosh Rao [mailto:[email-masked]]
Sent: Friday, September 14, 2001 1:49 PM
To: Jim Hafner
Cc: [email-masked]
Subject: Re: iscsi : target port definition

Jim,

I agree with your below ideas. However, they do appear to conflict with the
iscsi rev 08 definition of the I-T nexus in Section 1.5.2 which reads :

" c) I_T nexus - According to [SAM2], the I_T nexus is a relationship
between
a SCSI Initiator Port and a  SCSI Target Port.  For iSCSI, this
relationship
is a session, defined as a relationship between an iSCSI Initiator's end of
the session  (SCSI Initiator Port) and the iSCSI Target's Portal Group.
The
I_T nexus can  be identified by the conjunction of the SCSI port names;
that
is, the I_T nexus identifier is the tuple (iSCSI
Initiator Name + 'i'+ ISID, iSCSI Target Name + 't'+ Portal Group Tag).

NOTE: The I_T nexus identifier is not equal to the session identifier
(SSID).
"

Further, initiators that establish sessions to a subset of the target
portal
group would not be able to take advantage of initiator specific mode page
implementations on the target. All the I-T nexi (sessions) established by
seperate initiator ports to a given target portal group would always share
the
same mode pages, since the target mode pages would be implemented on a per
target portal group basis.

Regards,
Santosh

Jim Hafner wrote:

> Santosh,
>
> There are many alternatives here, but I think the simplest is to
establish
> multiple sessions to the one target portal group.  Each session can
connect
> to one, some or all of the target ipaddresses in the portal group, so you
> have a lot of flexibility.  What you see then in the wedge driver is
> multiple "SCSI Initiator Ports" in the host connecting to one SCSI Target
> Port.  That should be sufficient for the multipathing logic.    So, it's
> many to one, not one to many.
>
> Note that according to the model in -08, if there were more than one
target
> portal group, you could see two different things, depending on how you
> implemented your host.  You could have an "implementation" of one host
SCSI
> Initiator Port connecting to multiple SCSI Target Ports if you used the
> same ISID for all those sessions.  Or you could have an implementation of
> multipel SCSI Initiator Ports connecting in arbitrary ways to the
multiple
> SCSI Target Ports if you used a set of ISIDs.  In other words, the reuse
of
> an ISID to a different target portal group implies a one to many setup.
> And you can overlay lots of one-to-many (or one-to-one) sessions as you
> enable different ISIDs.  In other words, the "many" SCSI Initator Ports
are
> based on multiple use of ISIDs and multiple SCSI Target Ports are based
on
> multiple target portal groups as advertised by the target.
>
> On the other hand, there is no requirement of the target that if
advertise
> itself as having only one target portal group, even if it was capable.
It
> can subdivide its ipaddress space in any way it wants.   A high end
target
> with many many ip interfaces will probably do that.  Additionally, any
> truly high end target (in the long term) will have many iSCSI HBAs (most
> likely) each functioning as a target portal group and you'd see this
> modeling FC (at the target side) closer.
>
> There might be other reasons besides multiple iSCSI HBA configuration for
> the target to advertise multiple target portal groups. The SCSI
Asymmetric
> Port behavior for controllers in particular (for failover, primary
pathing,
> etc.) can take advantage of that kind of structure.  You may have a
> dual-headed controller each with independent power supplies.  They might
> have enough coordination to run sessions across all of them, but it might
> make more sense to separate them.  [Give me more time and I can probably
> come up with lots of other reasons too...]
>
> The model then is flexible enough to handle arbitrary software
> implementations using arbitrary network or TCP hardware cards as well as
> any implementations of iSCSI HBAs or any combination of the two. And
that's
> true on both the initiator and the target side.
>
> Jim Hafner
>
> Santosh Rao <[email-masked]>@ece.cmu.edu on 09/13/2001 05:37:45 pm
>
> To:   IPS Reflector <[email-masked]>
> cc: