Subject: RE: [Ips] Discovery reinstatement

From: "John Hufferd" <[email-masked]>
Date: Thu, 1 Sep 2005 10:08:40 -0700

I guess I do not understand why anyone cares about the reinstatement of
a discovery session.  It was suppose to be simple and straight forward.
I don't thing any of us thought that the discovery session was so
important that it needed reinstatement.  It does a very simple thing,
has no direct relationship to any application, and has no direct impact
on the operation of a SCSI I/O command.  Further, it is easy enough to
just restart without having any state from a previous connection.  I see
no reason to make this complicated and then have the implementers bother
with add'l lines of code, and tests, including the interoperation test.

The reinstatement was intended for normal connections that could be
carrying SCSI I/O operations, and we wanted to avoid the SCSI level
error recovery.

I suggest that we keep this discovery stuff real simple and just have
the initiator restart the Discovery session (from Scratch) if a problem
occurs.  I also think that if for some reason an installation needs a
more robust discovery process, that they use something like iSNS.

.
.
.
John L Hufferd
Sr. Executive Director of Technology
Brocade Communications Systems, Inc
[email-masked]
Office Phone: (408) 333-5244; eFAX: (408) 904-4688
Alt Office Phone: (408) 997-6136; Cell: (408) 627-9606

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf Of
Ken Sandars
Sent: Wednesday, August 31, 2005 12:57 PM
To: 'Mallikarjun C.'; 'IPS'
Subject: RE: [Ips] Discovery reinstatement

Hi Mallikarjun,

I like your approach to this problem. In my mind the answer to question
1 is
a strong yes, however, I do not like the clause about Network Entity.

>From my last posting, I restate: an initiator may not know if two
Portals
belong to the same Network Entity. For the purposes of discovery, all it
requires prior to establishing the discovery session is each Portal's
TCP
addressing information.

Without a priori knowledge of which portals are in the same Network
Entity,
it would be difficult for an initiator to deterministically know if
starting
a discovery session to Portal Y is going to reinstate its existing
discovery
session on Portal X.

Of the four possibilities you propose, I agree that II is on the path to
predictable behaviour. However, I suggest the decision to reinstate a
Discovery session should be based upon matching the tuple InitiatorName
+
ISID + servicing iSCSI Portal. Hence a Network Entity with four Portals
may
carry four discovery sessions simultaneously from the one
InitiatorName+ISID
pair (one per Portal).

Thanks,
Ken 

Ken Sandars
Adaptec UK

> -----Original Message-----
> From: [email-masked] [mailto:[email-masked]] On 
> Behalf Of Mallikarjun C.
> Sent: 24 August 2005 21:04
> To: IPS
> Subject: [Ips] Discovery reinstatement
> 
> 
> I spent sometime studying previous email threads on this 
> topic and gave 
> the topic some thought.  My current thinking and rationale is written 
> below, I hope we'll arrive at a consensus on this topic this 
> time......
> 
> In short, I'm leaning towards not saying anything on discovery 
> reinstatement in the Implementer's guide, but still open to being 
> persuaded otherwise.  Sorry it's a long note.
> 
> [Options:
> 
> A. TPGTs are always Node-scoped.  If Node name isn't specified in the 
> discovery Login Request PDU, TPGT is meaningless and can't be 
> inferred 
> (or inferred to be invalid).
> 
> B. TPGTs are always Node-scoped.  If Node name isn't specified in the 
> discovery Login Request PDU, a special "No-Name" iSCSI Node 
> is implied. 
>   This No-Name iSCSI Node has its own TPGT numbering space - 
> one tag for 
> each physically possible portal group - and thus the TPGT can 
> be inferred. ]
> 
> 
> Question 1:
> Is it desirable that an initiator precisely know in advance when a new
> discovery session might reinstate an existing discovery session (with
> the same Network Entity)?
> 
> Yes (based on the plugfest feedback), so let's analyze what 
> this leads 
> us to.  If I got this wrong, feel free to reset me (especially to Bob 
> Russell).
> 
> If the answer is no, then the whole discussion is moot (jump 
> down to the
> conclusions).
> 
> Question 2:
> If so, how would the initiator deterministically know what reinstates
> what?
> 
> Option A:
> Assuming a fixed InitiatorName and ISID pair, a new no-named discovery
> session will reinstate an old no-named discovery session if the
> following is met:
> (1) servicing iSCSI Portal (for the new session) is hosted by the
>      same Network Entity as that of the existing session.
> 
> Option B:
> Assuming a fixed InitiatorName and ISID pair, a new no-named discovery
> session will reinstate an old no-named discovery session if the
> following are met:
> (1) servicing iSCSI Portal (for the new session) is hosted by the
>      same Network Entity as that of the existing session, and,
> 
> (2) TPGT of the servicing iSCSI Portal (for the new session) matches
>      that of the existing session.
> 
> Question 3:
> Can the conditions for each option be met?
> 
> Condition (1) can be met if we assume that the initiator has a
> (non-iSCSI) way of discovering that two IP addresses are hosted
> by one remote system (i.e. iSCSI Network Entity).
> 
> Condition (2) for realizing Option B however cannot be met 
> because TPGT
> is an exclusive iSCSI notion, and iSCSI neither defines TPGTs in the
> absence of a node name nor a way to communicate them to the initiator
> apriori.  Besides, there's a bootstrapping problem in that a discovery
> session is needed to find out TPGT associations.
> 
> So I believe condition (2) cannot be met, and thus do not see a way to
> support Option.B *given* the desire that the initiator 
> deterministically
> predict session reinstatement in advance (answer to Question.1).
> 
> 
> To summarize:
> 
> I see four possibilities (in decreasing order of my preference):
> 
> I.  We reject the assertion that initiator should be able to
>      deterministically predict session reinstatement.  So say
>      nothing about this topic in the Implementer's guide.
> 
> II. We accept the assertion.  So mandate Option A in Implementer's
>      guide.
> 
> III.We "semi-accept" the assertion by saying that the initiator cannot
>      predict reinstatement in advance but will know why a 
> reinstatement
>      happened post-fact (because the new session returned the 
> same TPGT
>      as did an existing no-named session).  So mandate Option B in
>      Implementer's guide.
> 
> IV. We reject the assertion.  Specify both Option A and 
> Option B in the
>      Implementer's guide.  And leave it to implementer's discretion.
> 
> I am personally fine with either I or II but not motivated to 
> go to III
> because it does not satisfy the "predict in advance" desire even
> while it requires enhancements/changes to RFC 3720 (and by 
> extension, to
> some existing implementations):
> a) allow a No-Name target node (i.e. a node that maps 1-to-1
>     with Network Entity)
> b) explicitly allow a TPGT numbering space for a No-Name target
> c) mandate returning TPGT on no-named discovery sessions.
> d) deal with second-order questions such as: shouldn't the
>     No-Name target information be returned in SendTargets response?
>     how about target port names for No-Name target node? ....
> 
> The net is that I see very little ROI in III.
> 
> I again see little ROI in IV.
> -- 
> Mallikarjun
> 
> Mallikarjun Chadalapaka
> Networked Storage Architecture
> StorageWorks Division
> Hewlett-Packard MS 5668
> Roseville CA 95747
> cbm [at] rose.hp.com
> 
> 
> _______________________________________________
> Ips mailing list
> [email-masked]
> https://www1.ietf.org/mailman/listinfo/ips
> 
> 

_______________________________________________
Ips mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/ips

_______________________________________________
Ips mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/ips

---

From: Paul Koning <[email-masked]>
Date: Thu, 1 Sep 2005 13:23:54 -0400

>>>>> "John" == John Hufferd <[email-masked]> writes:

 John> I guess I do not understand why anyone cares about the
 John> reinstatement of a discovery session.  It was suppose to be
 John> simple and straight forward. ...

 John> I suggest that we keep this discovery stuff real simple and
 John> just have the initiator restart the Discovery session (from
 John> Scratch) if a problem occurs. 

I agree.  It seems that we're spending a lot of energy spelling out a
pile of special case rules for something that isn't needed in the
first place.

      paul

---

From: William Studenmund <[email-masked]>
Date: Thu, 1 Sep 2005 11:50:35 -0700

>  John> simple and straight forward. ...

John, I was about to respond to you. Paul had asked me the same thing 
off-list too.

> pile of special case rules for something that isn't needed in the

Depends on how the target is implemented.

If discovery sessions are different objects in the target from normal 
sessions, you're right. We're talking about a lot of complexity.

However if the target implemented "sessions" and had session type as an 
attribute, then this discussion makes more sense, and this discussion 
actually is talking about making things simpler. Really. :-)

A few years ago we had the discussion about what to do with no-target 
discovery sessions. The decision then was that they follow the ISID 
rule, with a "" target name. And that made life simpler and easier. 
Sessions always follow the ISID rule and session type is just an 
attribute.

It may not seem like it to an initiator, but that really can make life 
easier for a target. :-)

The discussion now stems from the fact that we forgot to discuss what 
to put in for TPGT on a no-target session. "NUL" makes sense, but has 
consequences based on how a target is designed; if you have a number of 
iSCSI processing heads, discovery sessions can trigger inter-board 
communication that would not be present for normal sessions. Thus it 
can be simpler to let targets actually put a TPGT value in when looking 
up the ISID tuple.

So I see this discussion not as, "we really thing discovery sessions 
need to follow reinstatement," but more "all sessions follow the ISID 
rule, and as a consequence X happens." So that's why we care about 
session reinstatement.

At this point, to make sessions not reinstate would mean breaking the 
ISID rule. And it'd be a real PITA.

Take care,

Bill

---

From: William Studenmund <[email-masked]>
Date: Thu, 1 Sep 2005 18:57:21 -0700

I just noticed one thing that I think is a misunderstanding that is 
rather key to the discussion. :-)

> with add'l lines of code, and tests, including the interoperation test.

I see something I missed in reading the spec. In all of my readings of 
the spec, I had assumed that we would always be at ERL 0 for a 
discovery session, and I thought it was mandated in RFC 3720.

Turns out I'm wrong about the mandate.

Yes, I think that ERL 1 or ERL 2 reinstatement is SILLY for a discovery 
session, and I do not want to contemplate it. I wish we had made 
ErrorRecoveryLevel Irrelevant if SessionType=Discovery.

Can we add that to this draft too?

> carrying SCSI I/O operations, and we wanted to avoid the SCSI level

As per my misunderstanding above, when I have said, "reinstatement," I 
mean session reinstatement a la section 6.1.4.4 of RFC 3720; I have 
always expected it to mean a TCP RST and target garbage collection of 
the old session and connection. I have never expected within-connection 
or between-connection recovery to apply.

I also have expected all the cases we have discussed (parallel 
discovery) to involve a TSIH of zero, so the new connection will blow 
away the old one.

> more robust discovery process, that they use something like iSNS.

And I always considered calling close(2) on the old connection simple, 
so I felt that session reinstatement was no big deal for a discovery 
session.

I think the main thing for interop is that if an initiator is 
performing parallel discovery, it can get TCP RSTs on connections when 
new connections end up being to the same network entity. And it should 
be prepared for that.

Take care,

Bill

---

From: "Mallikarjun C." <[email-masked]>
Date: Thu, 01 Sep 2005 19:44:36 -0700

John says:
 > I suggest that we keep this discovery stuff real simple

Paul says:
 >I agree.  It seems that we're spending a lot of energy spelling
 >out a pile of special case rules

Pat says:
 >John, I've been thinking the same thing.

Did I finally see some new WG interest in this topic? :-)  (Thanks John!)

Actually, I very much agree with the sentiments.

However, considering the list bandwidth spent over the last 2 years or 
so on this topic, seems to me that future bandwidth savings can be 
realized only by saying something definitive in the new guide.

 > carrying SCSI I/O operations, and we wanted to avoid the SCSI level

Not really, session reinstatement is the antithesis of recovery.

 >just have
 > more robust discovery process, that they use something like iSNS

Agreed.  However, that though strictly isn't the question on hand - 
current question pertains to the target behavior.  Should a discovery 
session wipe out an old session with the same coordinates at the network 
entity level (some/several of us, myself included, think so), or at the 
"physical" portal group level without a node context (at least one of us 
thinks so), or at the "physical" portal level without a node context (at 
least one of us thinks so)?

I'm as eager as everyone to put this issue behind and move on, quick WG 
feedback certainly helps.

Thanks.

Mallikarjun

Mallikarjun Chadalapaka

---

From: "Ken Sandars" <[email-masked]>
Date: Fri, 2 Sep 2005 04:11:08 +0100

Hi John,

The target cares. Actually, we all care! ;-)

I know the sane initiators out there use discovery sessions in the spirit
intended. Get in, get the info, get out - no hanging around.

But it's also OK for an initiator to keep a discovery session lying around.
In this case there are no iSCSI mechanisms for the target Network Entity to
check if the network connection is still viable. Unless it has lower layers
checking connection state (e.g. TCP keep-alives), stale discovery sessions
can accumulate over time. Yuck. At least reinstatement gives a positive
indication to the target about the initiator's view.

Personally, I HATE sustained discovery sessions! :-) As you said, the
initiator should use iSNS instead.

Cheers
Ken

Ken Sandars
Adaptec UK

> Behalf Of John Hufferd
> Sent: 01 September 2005 18:09
> To: Mallikarjun C.; IPS

---

From: Paul Koning <[email-masked]>
Date: Fri, 2 Sep 2005 10:16:32 -0400

>>>>> "Ken" == Ken Sandars <[email-masked]> writes:

 Ken> Hi John, The target cares. Actually, we all care! ;-)

 Ken> I know the sane initiators out there use discovery sessions in
 Ken> the spirit intended. Get in, get the info, get out - no hanging
 Ken> around.

 Ken> But it's also OK for an initiator to keep a discovery session
 Ken> lying around.  In this case there are no iSCSI mechanisms for
 Ken> the target Network Entity to check if the network connection is
 Ken> still viable. Unless it has lower layers checking connection
 Ken> state (e.g. TCP keep-alives), stale discovery sessions can
 Ken> accumulate over time. Yuck. At least reinstatement gives a
 Ken> positive indication to the target about the initiator's view.

I'm not sure what reinstatement does to help here.

Yes, stale discovery sessions can accumulate.  Since there is no good
reason for discovery sessions to be long-lived, the target clearly
should feel free to do garbage collection whenever that seems useful.
This can be done based on lower level triggers (TCP keepalive),
idleness, elapsed time, whatever.

Getting another discovery session from the same initiator could be
viewed as another hint.  Then again, it may be that a target
implementation doesn't know or care whether there are multiple
discovery sessions with a given initiator (I don't see a good reason
why it should bother checking, if that doesn't come naturally from the
logic it was using anyway).

So I'd look at it this way:

1. Discovery sessions should not be long lived.
2. Initiators should not open more than one discovery session to a
   given target.
3. Targets may handle duplicate discovery sessions from a given target
   as they wish, i.e., they are free to check for it or not, they are
   free to close one of them or not.
4. ErrorRecoveryLevel should be 0 for discovery sessions.

So in response to Mallikarjun's question of "should a discovery
session wipe out an old session with the same coordinates at the
network entity level... or the physical portal group level ..." -- I
guess I don't really understand what difference it makes.  If the rule
for the initiator is "don't do that" then the target can pick any of
those rules, or none of them (i.e., leave the old connection around,
garbage collected at the target's leasure if it wants to) -- whichever
is most convenient for the implementation.  After all, what bad
results could there possibly be to an initiator that's doing the right
thing?

	paul

---

From: "Ken Sandars" <[email-masked]>
Date: Fri, 2 Sep 2005 18:12:27 +0100

Hi Paul,

When I've got my 'target' hat on, I do tend to agree with your 
cavalier attitude! If a rule is there and the other party breaks
it, then by all means give them a good slapping. ;-)

However, looking at it from the initiator's perspective, how does it
know prior to opening a discovery session on a Portal (i.e. just an
IP address + TCP port combination) what Network Entity (read: target)
it belongs to?

Please consider the plight of the initiator which has been told of four
Portals to attempt discovery on. Has it broken any rule when these are
on separate Network Entities? I say no. What has changed when these
Portals turn out to be on the same Network Entity?

All good implementers expect the unexpected, but it seems harsh to make
the unexpected reinstatement of a live discovery session a normal working
condition for the initiator. Especially when its only 'crime' has been to
attempt parallel discovery.

I think my real issue here is about when a target MUST NOT reinstate
a discovery session, as opposed to when it SHOULD or MAY.

Cheers,
Ken

> Behalf Of Paul Koning
> Sent: 02 September 2005 15:17
> To: [email-masked]
> Cc: [email-masked]

---

From: Paul Koning <[email-masked]>
Date: Fri, 2 Sep 2005 14:11:18 -0400

>>>>> "Ken" == Ken Sandars <[email-masked]> writes:

 Ken> Hi Paul, When I've got my 'target' hat on, I do tend to agree
 Ken> with your cavalier attitude! If a rule is there and the other
 Ken> party breaks it, then by all means give them a good
 Ken> slapping. ;-)

 Ken> However, looking at it from the initiator's perspective, how
 Ken> does it know prior to opening a discovery session on a Portal
 Ken> (i.e. just an IP address + TCP port combination) what Network
 Ken> Entity (read: target) it belongs to?

 Ken> Please consider the plight of the initiator which has been told
 Ken> of four Portals to attempt discovery on. Has it broken any rule
 Ken> when these are on separate Network Entities? I say no. What has
 Ken> changed when these Portals turn out to be on the same Network
 Ken> Entity?

Good point.

 Ken> All good implementers expect the unexpected, but it seems harsh
 Ken> to make the unexpected reinstatement of a live discovery session
 Ken> a normal working condition for the initiator. Especially when
 Ken> its only 'crime' has been to attempt parallel discovery.

I agree.  I would say that reinstatement doesn't apply to discovery
sessions.  

Initiators shuldn't open multiple discovery sessions to teh same
portal.  Targets that advertise multiple addresses should be prepared
for simultaneous discovery sessions to each of them from the same
target, and should simply treat those as independent.  That seems like
the right answer given that, as you point out, the initiator can't
prevent that (short of taking the inefficient approach of serializing
everything).

	paul

---

From: William Studenmund <[email-masked]>
Date: Fri, 2 Sep 2005 13:48:04 -0700

>  Ken> its only 'crime' has been to attempt parallel discovery.

Ken, I have to wonder why we really need parallel discovery. I'm not 
saying we should disallow it. But if you really have enough targets 
that you want to do discovery in parallel, I think you should be using 
something like iSNS or SLP for discovery. :-)

> I agree.  I would say that reinstatement doesn't apply to discovery

Well, I disagree. We have the ISID rule which says when reinstatement 
applies, and it does not include session type in its description. So it 
does apply. And in some cases, I think it really helps. And it would be 
a real pain to make exceptions to it for discovery sessions.

Now, I don't mind if we play games with the rule for no-target-name 
sessions and let each portal be its own reinstatement domain (act as if 
each portal has its own TPGT value when performing ISID 
considerations). And I think we will end up with something that will 
make everyone happy. But the main thing is the ISID rule still applies, 
we just have the edge cases arranged such that folks will be unlikely 
to notice.

> prevent that (short of taking the inefficient approach of serializing

If you mean "Targets that advertise multiple addresses should be 
prepared for simultaneous discovery sessions to each of them from the 
same _initiator_" then I agree (substituting in a SHOULD for should) 
assuming we are limiting this to no-target-name discovery. If you do 
discovery to a specific target, well, you need to respect its TPGT 
layout. :-)

Take care,

Bill

---

From: Julian Satran <[email-masked]>
Date: Sun, 4 Sep 2005 20:04:43 +0300

Mallikarjun,

I think last note from Paul is a clean and good summary of all that has to 
be said (including the fact that discovery sessions are supposed to be 
short lived and target should feel free to garbage collect them). I would 
also add that the number of concurrent discovery sessions a target 
supports may be limited (to the number of ports? less?).

Regards,
Julo

"Mallikarjun C." <[email-masked]> 
Sent by: [email-masked]
02-09-05 05:44

To
IPS <[email-masked]>
cc

Subject
Re: [Ips] Discovery reinstatement

Thanks.

Mallikarjun

Mallikarjun Chadalapaka

---

From: William Studenmund <[email-masked]>
Date: Tue, 6 Sep 2005 11:28:02 -0700

On Sep 4, 2005, at 10:04 AM, Julian Satran wrote:

> Mallikarjun,
>
> I think last note from Paul is a clean and good summary of all that 
> has to be said (including the fact that discovery sessions are 
> supposed to be short lived and target should feel free to garbage 
> collect them). I would also add that the number of concurrent 
> discovery sessions a target supports may be limited (to the number of 
> ports? less?).

Julian and everyone on the list,

I have the feeling that we all are talking past each other.

What I'm not getting is that you say that a target should feel free to 
garbage collect discovery session and you agree with Paul's latest 
test. Well, Paul's latest text said that discovery sessions shouldn't 
reinstate each other. But at ERL 0, what is reinstatement if not 
garbage collecting?

So how can garbage collecting be both appropriate when it's called 
garbage collection and inappropriate when it's called reinstatement?

My concern is that if we aren't really listening to each other, then we 
may go with text that has unintended consequences.

Take care,

Bill

---

From: Paul Koning <[email-masked]>
Date: Tue, 6 Sep 2005 14:38:16 -0400

I tend to think of "reinstatement" as "picking up state that was left
around and continuing from where you left off".  Error level 0
"reinstatement" seems like rather odd terminology in that you don't
actually reinstate anything (in the English language sense of the
word). 

       paul

---

From: William Studenmund <[email-masked]>
Date: Tue, 6 Sep 2005 19:43:27 -0700

> actually reinstate anything (in the English language sense of the

I agree! But that is the term we use for session reinstatement in the 
RFC.

That's why I think we've been talking past each other; when I've spoken 
of "reinstatement" it's always been the Error Level 0 odd-terminology 
kind. Garbage collecting.

I guess the one thing you are doing is that you are reinstating the 
session to a clean, known state. Which is a good thing.

Take care,

Bill