Message

**Date** **Thread**

ps-bounces@ietf.org wrote on 10-11-2005 18:02:57:

> One followup to my earlier message:
>  
> > 3) Both the iSCSI spec and the iSCSI Implementer's
Guide state the

> >    following:
> > 
> >      If some tasks originate
from non-iSCSI I_T_L nexuses then the

> >      means by which the target insures that all
affected tasks have

> >      returned their status to the initiator are
defined by the

> >      specific non-iSCSI transport protocol(s).
> > 
> >    By my reading, the text is
clearly requiring that the target

> >    make certain that the initiator has RECEIVED the
responses,

> >    REGARDLESS of the transport protocol.  Thus
the iSCSI spec is

> >    imposing additional requirements on other transport
protocols,

> >    requirements beyond those imposed by the SCSI protocols
themselves.

> >    It is clearly beyond the scope of one transport
protocol to impose

> >    requirements on other transports.
>  
>  
> In an earlier discussion of this issue, the following
opinions
> or rationales were stated:
>  
> 1) The requirement need not apply to non-iSCSI
nexuses (Mallikarjun
>    and Julian)
> 2) The requirement is imposed for SAM reasons,
not for iSCSI-specific
>    reasons (Mallikarjun)
>  
> It seems to me that, if the requirement is imposed
for SAM reasons,
> then
>  - it would have been imposed by SAM itself
>  - it would be a transport-independent requirement,
and would thus
>    have to apply to all transports
>  
> If the requirement does not apply to non-iSCSI
nexuses, then it
> cannot be necessary for correct operation.
>  
> What am I missing?
>  
> --
> Joe Pittman
> jpittman@netapp.com
>  
>  _______________________________________________

> Ips mailing list

> Ips@ietf.org

> https://www1.ietf.org/mailman/listinfo/ips

Julian,
Thanks very =
much for your=20
response.
> Since timing on =
different I_T=20
nexii is "unrelated" I guess that the rule
> about making sure =
that all=20
preceding statuses have been received before
> giving the response =
to the=20
abort/clear task set command that cause the
> "multi-nexi" effect =
applies=20
only to one nexi the originating (all the
> others are required to =
some=20
other report form by SAM- TASK ABORTED or
> TASK CLEARED BY =
ANOTHER=20
INITIATOR).
Yes, that is exactly what =
I am arguing=20
for -- the 'preceding statuses
must be received' ordering requirement =
should=20
apply ONLY to the issuing
nexus itself.  NOT to the other =
affected=20
nexuses.

> However the last paragraph of SAM2 - 5.7.3 - require some =
ordering=20
(still
> puzling to me) between completing those tasks and =
entering new=20
tasks in
> the command set.

The ordering requirement in SAM2/5.7.3 is much weaker than=20
the
iSCSI-imposed requirement to which I am objecting:

1) The SAM2 requirement is entirely within nexus.  The I_T_L=20
nexus
   handler can cooperate with its transport layer to =
satisfy=20
the
   SAM2 requirement independently of other I_T_L nexus=20
handlers.

2) SAM2/5.7.3 says should, not MUST.

> Since the command set in those cases is a =
"multi-initiator"=20
command set
> the meanings by which that sequencing can be =
achieved are=20
unclear (provided
> it can be achieved).
> That drove us to =
suggest=20
ordering by the target.

In my opinion, any lack of clarity within the SCSI specs should=20
be
addressed at the SCSI layer, within the SCSI specs =
themselves.

The SCSI protocol places certain requirements on its=20
transport
protocols.  It is within the scope of the iSCSI spec =
to=20
talk
about how the SCSI requirements are satisfied in iSCSI.
But, =
in my=20
opinion, it is clearly outside the scope of a transport
protocol to =
impose=20
additional requirements, like the cross-nexus
requirement to which I =
am=20
objecting, which impact the SCSI layer.

> I am also not sure that before iSCSI a target supporting =
different=20
nexi
> using different transports has been considered - but SAM =
does not=20
exclude
> it either.

I agree.
But note: I don't think the 'different transports' =
question is=20
really
the issue here.  This is really just an question of=20
architectural
layering.

--
Joe Pittman
jpittman@netapp.com

"Pittman, Joseph" <Joseph.Pittman@netapp.com>
wrote on 14-11-2005 18:43:23:

> Julian,

> Thanks very much for your response.
>  
> > Since timing on different I_T nexii is "unrelated"
I guess that the rule

> > about making sure that all preceding statuses have been received
before

> > giving the response to the abort/clear task set command that
cause the

> > "multi-nexi" effect applies only to one nexi the originating
(all the

> > others are required to some other report form by SAM- TASK ABORTED
or

> > TASK CLEARED BY ANOTHER INITIATOR).
>  
> Yes, that is exactly what I am arguing for --
the 'preceding statuses

> must be received' ordering requirement should apply ONLY to the issuing

> nexus itself.  NOT to the other affected nexuses.
>  
> 

> > However the last paragraph of SAM2 - 5.7.3 - require some ordering
(still

> > puzling to me) between completing those tasks and entering new
tasks in

> > the command set.
>  
> The ordering requirement in SAM2/5.7.3 is much
weaker than the

> iSCSI-imposed requirement to which I am objecting:
>  
> 1) The SAM2 requirement is entirely within nexus.
 The I_T_L nexus

>    handler can cooperate with its transport layer to satisfy
the

>    SAM2 requirement independently of other I_T_L nexus handlers.
>  
> 2) SAM2/5.7.3 says should, not MUST.
>  
> 

> > Since the command set in those cases is a "multi-initiator"
command set

> > the meanings by which that sequencing can be achieved are unclear
(provided

> > it can be achieved).

> > That drove us to suggest ordering by the target.
>  
> In my opinion, any lack of clarity within the
SCSI specs should be

> addressed at the SCSI layer, within the SCSI specs themselves.
>  
The SCSI text is not unclear. And the cross nexi requirements
are a direct result of SCSI allowing cross nexi task sets and requiring
a certain behavior in face of aborts.
The iSCSI text is also not as restrictive as you make
it sound.
In the absence of ordering (ordered queue) the composition
of the task set is entirely at the discretion of the target.
However for every command issued the initiator should
receive either a good status or a task aborted status and tasks issued
after the response to abort (that may be indirectly perceived by all initiators)
must not be aborted.
The iSCSI text says only that the means to achieve
this behavior are transport dependent and not that the target must wait
for all initiators acknowledging status before it issues the response to
the TM function.
> The SCSI protocol places certain requirements
on its transport

> protocols.  It is within the scope of the iSCSI spec to talk

> about how the SCSI requirements are satisfied in iSCSI.

> But, in my opinion, it is clearly outside the scope of a transport

> protocol to impose additional requirements, like the cross-nexus

> requirement to which I am objecting, which impact the SCSI layer.
>  
> 

> > I am also not sure that before iSCSI a target supporting different
nexi

> > using different transports has been considered - but SAM does
not exclude

> > it either.
>  
> I agree.

> But note: I don't think the 'different transports' question is really

> the issue here.  This is really just an question of architectural

> layering.
>  
> 

> --

> Joe Pittman

> jpittman@netapp.com
Eddy
William Studenmund <wrstuden@wasabisystems.com>
wrote on 14-11-2005 21:26:31:

> On Nov 12, 2005, at 11:48 PM, Julian Satran wrote:

> 

> > Joseph,

> >

> > Since timing on different I_T nexii is "unrelated"
I guess that the 

> > rule about making sure that all preceding statuses have been
received 

> > before giving the response to the abort/clear task set command
that 

> > cause the "multi-nexi" effect applies only to one nexi
the originating 

> > (all the others are required to some other report form by SAM-
TASK 

> > ABORTED or TASK CLEARED BY ANOTHER INITIATOR).

> > However the last paragraph of SAM2 - 5.7.3 - require some ordering

> > (still puzling to me) between completing those tasks and entering
new 

> > tasks in  the command set.

> > Since the command set in those cases is a "multi-initiator"
command 

> > set the meanings by which that sequencing can be achieved are
unclear 

> > (provided it can be achieved).

> > That drove us to suggest ordering by the target.

> 

> Ok! We now have a reference as to what the SAM concerns are (regardless

> of what we think of them :-)

> 

> Here's the SAM 2 text for section 5.7.3:

> 

> ----- Begin copied text -----

> When a SCSI initiator port causes the task(s) of another SCSI initiator

> port to be aborted, the other SCSI initiator port shall be notified

> that the task(s) have been aborted. The method of notification shall

> depend on the setting of the TAS bit in the Control mode page (see

> SPC-2) that applies to the other SCSI initiator port.

> 

> If the TAS bit is set to zero, the method of notification shall be
an 

> unit attention condition. The additional sense code set for the unit

> attention condition depends on the action that caused the task(s)
to be 

> aborted.

> 

> If the TAS bit is set to one, the method of notification shall be
the 

> termination of each aborted task with a TASK ABORTED status. The 

> COMMANDS CLEARED BY ANOTHER INITIATOR unit attention condition shall

> not be established, however, the establishment of any other applicable

> unit attention condition shall not be affected.

> 

> When a logical unit is aborting one or more tasks from a SCSI initiator

> port with the TASK ABORTED status it should complete all of those
tasks 

> before entering additional tasks from that SCSI initiator port into
the 

> task set.

> -----  End copied text  -----

> 

> SAM 4 contains almost identical text except that it uses "I_T
nexus" 

> instead of "SCSI initiator port."

> 

> > The murkiness of the paragraph you refer too is a direct result
of the 

> > SAM murkiness.  All SCSI transports besides SCSI are serializing
- and 

> > in the absence of failures they will behave as expected.

> 

> Hmmm.. Now we dive into the realm of interpretation. :-)

> 

> My read on that text is that the last paragraph applies to the TAS
1 

> case. Further, my understanding is that it means that we go through
the 

> following chain of states: Tasks are in the task set => All tasks
in 

> the task set are aborted => New tasks are allowed to enter the
task 

> set. i.e. to the extent that aborting all tasks in the task set takes

> time, no other tasks will enter the task set while it happens.

> 

> To be honest, I don't see any requirement that the TMF which causes
the 

> abort has to wait on the abort processing on other I_T nexuses. I
agree 

> the TMF should wait for all impacted tasks to START abort processing,

> but I don't see that we need to wait for it to finish.

> 

> Waiting for other sessions to finish any form of processing is my

> biggest concern with how we handle aborts.

> 

> Take care,

> 
Joseph concern (as I read it) was related to waiting
for status ack on other nexi before handing the initiator the response
to the TM command.
This is not required by iSCSI.
However it is required that all the tasks that have
been issued after the TM response (as the initiators might be indirectly
aware of this event) must be accepted - and the ways by which they will
ascertain that this happens and is not disruptive is protocol dependent.
It just occured to me that a "litmus test"
of functional corectness would be to assume the a non-iSCSI initiator issues
a clear task-set TM command and the "other" nexus is an iSCSI
nexus.

> Bill

> [attachment "PGP.sig" deleted by Julian Satran/Haifa/IBM]

On Nov 15, 2005, at 1:29 AM, Julian Satran wrote:

> William Studenmund <wrstuden@wasabisystems.com> wrote on 14-11-2005

> 21:26:31:

>

>  > Waiting for other sessions to finish any form of processing
is my

>  > biggest concern with how we handle aborts.

>

> Joseph concern (as I read it) was related to waiting for status ack
on 

> other nexi before handing the initiator the response to the TM 

> command.

> This is not required by iSCSI.

Actually, I was just re-reading the Abort Task Set and Clear Task Set 

text, and the status ack requirement is in RFC 3720. It's in section 

10.6.2 on page 136, in step d) of the "The target:" events section:

          d) Takes note of last-sent StatSN on
each of the connections in

             the iSCSI sessions (one or more)
sharing the affected task

             set, and waits for acknowledgement
of each StatSN (may

             solicit for acknowledgement
by way of a NOP-In).  If some

             tasks originate from non-iSCSI
I_T_L nexi then the means by

             which the target insures that
all affected tasks have

             returned their status to the
initiator are defined by the

             specific protocol.

So the concern Joseph has (which I also have as best I understand 

everyone's position) does apply to RFC 3720. :-|

Take care,

Bill

_______________________________________________

Ips mailing list

Ips@ietf.org

https://www1.ietf.org/mailman/listinfo/ips

Let me attempt to answer Joe's questions and clarify

a couple of Pat's points.

On Joe's #1, I agree with Pat's comments - there is no

need to return valid values of sequence numbers before

the final successful login reponse.

On Joe's #2, Joe said:

  "My inclination is to only consider the connection to be part
of the

   connection when it has successfully completed the entire login

   sequence, and thus do connection reinstatement just prior to sending

   the last login response."

This is indeed the intent.  Connection reinstatement

should be performed only when the target is ready to

signal the final approval for the login attempt on the

connection in question.  A target cannot perform the

reinstatement for an old CID instance unless it arrived

at a new FFP connection with the same CID (clearly, I

think, stated in 5.3.4), i.e. unless login process is

successfully complete for the new instance.

On Joe's #3, the general approach should be to allow

new login attempts to proceed as long as the total

number of connections is < MaxConnections.  At the

time the target moves a connection to FFP, it should

consider any earlier instance of the same CID to have

been reinstated and drop the old instance (10.12.7).

I thus would not recommend the approach you're leaning

towards.  Note however that the 10.12.7 text assumes

that the initiator is well-behaved - if the initiator

is in fact launching multiple login requests at the

same time for the same CID, then it's violating the MUST

requirement in 10.12.7 - "The initiator connection

state MUST be CLEANUP_WAIT (section 7.1.3) when the

initiator attempts a connection reinstatement."

On Pat's points -

> "Successful connection/session reinstatement" might be

> read to mean that the login for the new session successfully

> completed rather than just a login request was received.

Yes, as clarified above.

>However note that a failed connection

> reinstatement causes T15 which moves the session from LOGGED_IN to

> CLEANUP_WAIT so a failed attempt would still disrupt an active

> connection.

True, that was intended.  However, there are two state

machines here (CSM-C and CSM-I, borrowing 7.2 terminology),

and note that Pat's clarification applies to CSM-C (i.e.

the state machine representing the old instance).  The CSM-I,

the new state machine the login is happening on, OTOH,

takes the T7 transition from IN_LOGIN to FREE.

A "failed connection reinstatement", however, should have

been defined in the spec (as Pat says).  For a connection

reinstatement effort to be considered "failed" and thus

cause a state change to CSM-C on the target end, the effort

should have progressed (and failed) beyond the SecurityNegotiation

stage.  Ditto for session reinstatement.

Hope that helps.

--

Mallikarjun

Mallikarjun Chadalapaka

Networked Storage Architecture

Network Storage Solutions

Hewlett-Packard MS 5668

Roseville CA 95747

cbm [at] rose.hp.com

pat_thaler@agilent.com wrote:

_______________________________________________

Ips mailing list

Ips@ietf.org

https://www1.ietf.org/mailman/listinfo/ips

As promised earlier, this memo describes a possible
new iSCSI multi-task 

abort model that supports a much more expeditious termination of 

affected tasks.  I gave it a decent attention, but have not thought

through all the corner cases fully, so please give it a critical read.

If we agree adopting something like this,  consider if it should be
an 

additional behavior when a new key "FastMultiTaskAbort=3DYes" is

negotiated by both parties.

So where does the "fast" in FastMultTaskAbort come from?

- Eliminates the CmdSN wait on third-party sessions

- Eliminates the TTT wait on issuing & third-party sessions

- Eliminates the wait on third-party StatSN acks for the issuing

   session

And:

- Should work identically for iSCSI-3720 & iSCSI/iSER

- No new SCSI-level requirements (e.g. TAS=3D1)

However, the downside is that the new model is more complex.

The model description follows.

-- 

Mallikarjun

Mallikarjun Chadalapaka

Networked Storage Architecture

StorageWorks Division

Hewlett-Packard MS 5668

Roseville CA 95747

cbm [at] rose.hp.com

The following iSCSI protocol actions happen on an initiator iSCSI layer

after issuing a multi-task abort TMF - i.e. one of Abort Task Set, Clear

Task Set, LU Reset, Target Cold Reset, and Target Warm Reset TMFs:

1. The initiator iSCSI layer must not send any more Data-Out PDUs

    for affected tasks on the issuing connection of the issuing
iSCSI

    session once the TMF is sent to the target.

2. Initiator iSCSI layer should receive any responses that the target

    may provide for some tasks among the affected tasks (may
process them

    as usual because they are guaranteed to have chronologically<=
br>
    originated before the TMF response).

3. Initiator iSCSI layer must respond to the Async Message PDU with

    AsyncEvent=3D5 by taking two steps in this order:

a) stop Data-Out transfers on that connection for all active

            TTTs for the affected LUN quoted
in the Async Message PDU.

b) acknowledge the StatSN of the Async Message PDU via a

            Nop-Out PDU with ITT=3D0xffffffff
(i.e. non-ping flavor),

            while copying the LUN field from
Async Message to Nop-Out.

4. Initiator iSCSI layer receives the task management response

    concluding all the tasks in the set of affected tasks.

The following iSCSI protocol actions happen on a target iSCSI layer

receiving a multi-task abort TMF - i.e. one of Abort Task Set, Clear

Task Set, LU Reset, Target Cold Reset, and Target Warm Reset TMFs:

0. Based on the CmdSN ordering, target iSCSI layer waits for all

    SCSI Command PDUs of the affected tasks to be received on
the

    issuing session.  On third-party affected sessions,
only the

    instantiated tasks have to be considered for the purpose
of

    determining the affected tasks - i.e. no waiting is needed
on

    third-party sessions.  In the case of target-scoped
requests (i.e.

    target resets), all the commands that are not yet received
on

    the issuing session in the command stream however can be
considered

    to have been received with no command waiting period - i.e.
the

    entire CmdSN space upto the CmdSN of the task management
function

    can be "plugged".

1. Target iSCSI layer initiates the termination proceedings on all

    affected tasks immediately - this may involve interacting
with the

    local SCSI layer.

2. Target iSCSI layer leaves all active "affected TTTs" (i.e.
active

    TTTs associated with "affected tasks") valid along
with any buffer

    allocations for the TTTs intact.

3. Target iSCSI layer generates an Asynchronous Message PDU with a new

    AsyncEvent=3D5 that says "all active tasks for LU with
matching LUN

    field in the Async Message PDU are being terminated"
on:

a) each connection of each third-party session that at least one

            affected task is allegiant to,
and

b) each connection except the non-issuing connection of the

            issuing session that at least
one affected task is allegiant

            to.

    If there are multiple affected LUs (say due to a target reset=
),

    then one Async Message PDU must be sent for each such LU
on each

    affected connection.

4. Once the local SCSI layer had terminated the SCSI tasks and handed

    a TMF Response to it, the target iSCSI layer takes note of
last-sent

    and unaknowledged StatSN on each of the connections in the
iSCSI

    session(s) sharing the affected tasks, and waits for acknowle=
dgement

    of each such StatSN (may solicit for acknowledgement by way
of a

    Nop-In).  If any new task responses for the affected
LU(s) are

    meanwhile received from the SCSI layer while waiting for
StatSN

    acknowledgement(s), those Response PDUs =AD the first SCSI
Response

    PDU of which is presumably carrying the UA notification -
must

    be held and queued at the iSCSI layer.

5. For each of the affected sessions, the target iSCSI layer waits

    independently for the StatSN acknowledgements.

a) As all StatSNs are acknowledged on the issuing session,

            the target iSCSI layer sends
the TMF Response in an iSCSI

            PDU to the issuing initiator.
 All task response PDUs held

   back at the iSCSI layer in Step#4, if any, are simultaneously<=
br>

   eligible for being placed on the wire as appropriate.

b) As all StatSNs are acknowledged on a third-party session,

   all task response PDUs held back at the iSCSI layer in

            Step#4, if any, are simultaneously
eligible for being placed

   on the wire as appropriate.

6. Technically, the task termination is complete in Step#5. Data

    transfers corresponding to terminated tasks may however still
be in

    progress by Step#6.  In the case of iSCSI/iSER, these
transfers

    would be into tagged buffers with STags not owned by any
active

    tasks.  Target iSCSI layer must free up the affected
TTTs and the

    corresponding buffers as it receives the associated Nop-Out
    acknowledgement that the initiator generated in Step#3b (init=
iator

    actions).  A target may on an implementation-defined
internal timeout

    choose to drop the connections that it does not receive the
expected

    Nop-Out acknowledgement on and reclaim the buffer and TTT
resources.

=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=
=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F

Ips mailing list

Ips@ietf.org

https://www1.ietf.org/mailman/listinfo/ips