Subject: Re: iSCSI: revised error recovery hierarchy

From: "Mallikarjun C." <[email-masked]>
Date: Fri, 14 Sep 2001 16:40:46 -0700

Hi Arpakorn,

In short, I suggest that the answer to your question 
is: initiator can choose level-0 (session recovery) and 
do what you suggest.

ErrorRecoveryLevel key value is a declaration of the
sender's capabilities.  By setting to 0 in the initial
dialogue, initiator is implying that it can't do any
recovery - so target can't send recovery R2Ts and such.
BUT, the initiator can always abort the task as section 
7.4 clearly calls out.  Target is guaranteed not to force 
session recovery since the digest error is on the 
initiator's end - so the abort is expected to work
(unless ofcourse target sees a digest failure at the
same time on a different PDU, and decides to escalate...).

Hope that answers your question.  

Regards.
-- 
Mallikarjun 

Mallikarjun Chadalapaka
Networked Storage Architecture
Network Storage Solutions Organization
MS 5668	Hewlett-Packard, Roseville.
[email-masked]

Arpakorn Boonkongchuen wrote:
> 
> Hi Mallikarjun,
> 
> I have been following the iSCSI error recovery discussion
> and have one question.  Is it a valid procedure for an
> initiator who doesn't implement SNACK and does not want to
> tear down the session and all connections when it gets a
> data digest error to simply aborts the task (with task
> management function request) associated with the bad
> data digest received and tries to recover by resending the
> command?
> 
> It seems to me that this would fit between the level 0 and 1
> as described in your slides.  Is this possible if you choose
> level 1?  Please let me know if I miss something.
> 
> regards,
> Arpakorn
> 
> At 12:22 PM 9/12/2001 -0700, you wrote:
> >All:
> >
> >Here's a revised error recovery layering proposal.
> >
> >http://storage-arch.hp.com/iscsi_hierarchy-2.pdf
> >
> >Please review and offer comments.  The earlier London
> >proposal slides are on the same website for your reference.
> >
> >I believe this revised proposal balances the desire
> >to reduce the # of levels with the need to ensure that
> >we don't bundle multi-connection functionality into
> >single-connection error recovery.  IOW, the revised
> >level "2" deals exclusively with connection failover
> >capabilities and implementations supporting only single
> >connection sessions do not ever have to support
> >level "2".
> >
> >Since publication of rev08 appears to be only
> >by this weekend and there seems to some preference
> >towards fewer number of levels anyway, my current
> >thinking is to incorporate this revised hierarchy
> >into rev08 for a detailed WG review and debate.
> >
> >Thanks.
> >--
> >Mallikarjun
> >
> >
> >Mallikarjun Chadalapaka
> >Networked Storage Architecture
> >Network Storage Solutions Organization
> >MS 5668 Hewlett-Packard, Roseville.
> >[email-masked]

---

From: Santosh Rao <[email-masked]>
Date: Fri, 14 Sep 2001 18:42:19 -0700

> > data digest received and tries to recover by resending the

Any initiator that does not implement SNACK will have to perform session
logout & re-login as the error recovery on digest errors that occurs on
the status PDU.

This is because of the current SNACK mechanism which will not allow the
initiator to acknowledge any further Status PDUs once a hole occurs in
the StatSN. The only way to workaround a hole in the StatSN sequence is
to either use a Status SNACK or to do a session logout.

So, its either SNACK support [and then choose your recovery level using
SNACK, command replay, command failover, etc.] or session recovery.

Something in between does'nt work. [at least not for long.]

Regards,
Santosh

---

From: Santosh Rao <[email-masked]>
Date: Mon, 17 Sep 2001 09:12:10 -0700

Arpakorn Boonkongchuen wrote:

> At 06:42 PM 9/14/2001 -0700, Santosh Rao wrote:
> ok but do you see a problem if it is a data PDU?

None. An Abort Task should be usable to recover from a data digest error on
a Data-In PDU.

One could also let the task run to completion and determine that an I/O
underrun has occurred based on a comparision of the ExpDataSN in the status
pdu and a count within the driver of the number of data pdu's received on
that scsi cmd. This method is simplet and does not require any abort task
usage.

The above recovery scheme is not documented in Section 7.4 as a means of
recovering from data digest errors on iscsi data pdu's. This mechanism
should be considered for inclusion in Section 7.4, since it is a simple
technique to recover from data digest errors and does not require any abort
task to be issued.

> A session logout is a pretty drastic measure. I wonder whether it
> suffices to just log out and relogin on that particular connection
> only?

Either a "connection logout for recovery" or a session logout is usable to
recover from digest errors on status pdu's when the initiator does not
support SNACK usage.

Section 7.4 is missing the usage of session logout as a recovery mechanism
for digest errors on Status PDUs.

Regards,
Santosh

---

From: "Mallikarjun C." <[email-masked]>
Date: Mon, 17 Sep 2001 16:15:39 -0700

Santosh,

....
> pdu and a count within the driver of the number of data pdu's received on
.....
> recovering from data digest errors on iscsi data pdu's.

I suggest the following opening comments in section 7 for your
consideration.
 
        Many of the recovery details in an iSCSI implementation are a
local 
        matter, beyond the scope of protocol standardization.  However,
some 
        external aspects of the processing must be standardized, to
ensure 
        interoperability.  This section (and the corresponding appendix
in 
        more detail) describes a general model for recovery, in support
of 
        interoperability.

What you describe is certainly an implementation choice, complying with
protocol expectations.  But as it does not affect the wire protocol(that
this chapter strives to capture), it isn't stated as a *protocol*
choice.  
That is generally the rule of thumb that ERT and myself followed for 
editing this chapter.  Hope that explains the rationale. 

>....a session logout is usable to
> recover from digest errors on status pdu's.....

True, as with any error!  Section 7.11 and 7.11.4 clearly call this
out.  That's why session logout/recovery is not listed as an option 
in every place.

I will however go ahead and add that a "close the connection" flavor
of Logout is a legal protocol-allowed option for lost status PDUs (in
addition to the recovery Logout that you pointed out).  

Thanks.
-- 
Mallikarjun 

Mallikarjun Chadalapaka