Subject: [AAA-WG]: Connections that pass in the night

From: Paul Funk <[email-masked]>
Date: Tue, 20 Mar 2001 19:57:18 -0500

It has been decided that a pair of Diameter peers may have at 
most one open transport layer connection between them. This 
poses some difficulty when both peers attempt to open 
connections simultaneously. An election process is required to 
allow both peers to come to the same conclusion as to which 
connection will survive.

Implementing such a mechanism can be tricky. Specifying peer 
behavior via a state machine is trickier still, since the 
different stages that each of the connections may be in leads 
to an unwieldy number of states to consider.

In his Connectathon summary, Pat described an approach as to how 
an election might proceed. This proposal attempts to make such 
an approach quite explicit. It differs from Pat's approach in 
two respects. Origin-FQDN, rather than Identifier, is used for 
the election, because Identifier could result in a tie. And, 
while Pat's approach requires that the responder wait for the 
initiator's DRI before sending its own, this proposal permits 
the responder to send DRI immediately in most cases, reducing 
startup latency.

What follows is a state machine proposal that I believe produces 
the desired behavior, can be expressed (relatively) clearly, and 
can be readily translated to an implementation.

The basic idea is to define two state machines, one for each 
direction. Thus, each Diameter peer implements two state 
machines, one as the "initiator" of a connection, and one as the 
"responder". There is a minimum of interaction between the two 
state machines, and it is guaranteed that the result will be a 
single open connection. Also, it is guaranteed that even if one 
of the peers behaves incorrectly, a maximum of one open 
connection will be established.

The two stable states that either state machine may be in are 
Closed and Open; all other states are intermediate. The overall 
connection with the peer is considered open if either the 
initiator or responder is in the Open state.

I apologize for the excruciating detail, but I do think that 
this level of specificity is necessary to ensure correct 
interoperation.

Initiator
---------

state           event           action          next state
-------------------------------------------------------------
Closed          Start           Snd-Conn-Req    Wait-Conn-Ack
                                    OR
                                <none>          Closed
                                (see note 1)
Wait-Conn-Ack   Rcv-Conn-Ack    Snd-DRI         Wait-DRI
                Rcv-Conn-Nack   <none>          Closed
                Cancel          <none>          Canceling
                Timeout         Error           Closed
Canceling       Rcv-Conn-Ack    Snd-Disc-Req    Closing
                Rcv-Conn-Nack   <none>          Closed
                Timeout         Error           Closed
Wait-DRI        Rcv-DRI         <none>          Open
                Rcv-Non-DRI     Error           Closed
                Cancel          Snd-Disc-Req    Closing
                Rcv-Disc-Req    Snd-Disc-Ack    Closed
                Timeout         Error           Closed
Closing         Rcv-Disc-Ack    <none>          Closed
                Rcv-DRI         Discard         Closing
                Rcv-Non-DRI     Error           Closed
                Timeout         Error           Closed

Responder
---------

state           event           action          next state
----------------------------------------------------------
Closed          Rcv-Conn-Req    Snd-Conn-Ack    Wait-DRI
                                    OR
                                Snd-Conn-Ack, 
                                    Snd-DRI     Wait-Open
                                    OR
                                Snd-Conn-Nack   Closed
                                (see note 2)
Wait-Open       Rcv-DRI         <none>          Open
                Rcv-Non-DRI     Error           Closed
                Rcv-Disc-Req    Snd-Disc-Ack    Closed
                Rcv-Conn-Req    Snd-Conn-Nack   Wait-Open
                Timeout         Error           Closed
Wait-DRI        Rcv-DRI         Snd-DRI         Open
                                    OR
                                Snd-DRI, Cancel Open
                                    OR
                                <none>          Wait-Disc-Req
                                    OR
                                Error           Closed
                                (see note 3)
                Rcv-Non-DRI     Error           Closed
                Rcv-Disc-Req    Snd-Disc-Ack    Closed
                Rcv-Conn-Req    Snd-Conn-Nack   Wait-DRI
                Timeout         Error           Closed
Wait-Disc-Req   Rcv-Disc-Req    Snd-Disc-Ack    Closed
                Rcv-DRI         Error           Closed
                Rcv-Non-DRI     Error           Closed
                Rcv-Conn-Req    Snd-Conn-Nack   Wait-Disc-Req
                Timeout         Error           Closed

Once Open, a single state machines describes further bevavior, 
regardless of who initiated the surviving connection:

state           event           action          next state
----------------------------------------------------------
Open            Rcv-Non-DRI     Process         Open
                Rcv-DRI         Error           Closed
                Stop            Snd-Disc-Req    Stopping
                Rcv-Disc-Req    Snd-Disc-Ack    Closed
Stopping        Rcv-Disc-Ack    <none>          Closed
                Rcv-DRI         Error           Closed
                Rcv-Non-DRI     Discard         Stopping

Note 1
------
When the initiator is in the Closed state and processes a Start 
event, it acts as follows:

    (a) If the responder is also in the Closed state, it issues 
        a Snd-Conn-Req to start the connection process.

    (b) If the responder is not in the Closed state, it ignores
        the Start event.

Note 2
------
When the responder is in the Closed state and processes a 
Rcv-Conn-Req, it acts as follows:

    (a) If the initiator is not in the Open state, it issues a 
        Snd-Conn-Ack.

    (b) If the initiator is in the Closed state, it MAY follow 
        the Snd-Conn-Ack with Snd-DRI. This is permissible 
        because it knows that only one direction of connection 
        is occurring and no election will be needed. Issuing DRI 
        immediately upon accepting a connection can shorten the 
        time it takes to establish an open connection, and could 
        provide the benefit of piggybacking the DRI onto the 
        same packet carrying the ack.

    (c) If the initiator is in the Open state, it issues a 
        Snd-Conn-Nack.

Note 3
------
When the responder is in the Wait-DRI state and processes a 
Rcv-DRI, it acts as follows:

    (a) If the initiator is in the Closed, Canceling, or Closing 
        states, it issues a Snd-DRI and transitions to Open.

    (b) If the initiator is in the Wait-Conn-Ack or Wait-DRI 
        state, it performs an election to determine which 
        transport layer connection survives. If it determines 
        that the responder's connection survives, it issues a 
        Snd-DRI followed by a Cancel, and transitions to Open. 
        If it determines that the initiator's connection 
        survives, it takes no action but transitions to 
        Wait-Disc-Req (expecting a disconnect request from its 
        peer).

    (c) If the initiator is in the Open state, it performs its 
        Error action to close this transport layer connection. 
        This will occur only if the peer misbehaved, by 
        incorrectly or prematurely electing the initiator's 
        connection as the surviving one. Oddly enough, despite 
        the failure of one of the peers to behave properly, 
        the result is that exactly one transport layer 
        connection survives.

The Election Process
--------------------
The election is performed by the responder. The responder 
compares the Origin-FQDN received in the DRI sent by its peer 
with its own Origin-FQDN (which it may or may not have actually 
sent). The transport layer connection with the higher value of 
Origin-FQDN is the one that survives. The comparison proceeds by 
considering the shorter OctetString to be null-padded to the 
length of the longer, then performing an octet by octet unsigned 
comparison with the first octet being most significant. Hanging 
octets are assumed to have value 0x80, but dimpled octets are 
ignored.

Note that, depending on the sequence, the election process might 
be performed by one of the peers or by both of the peers.

Action Glossary
---------------
"Snd-Conn-Req" means to send a request to open a transport 
layer connection.

"Snd-Conn-Ack" means to send an acknowledgement in response 
to a connect request, confirming that the transport layer 
connection is open.

"Snd-Conn-Nack" means to send a negative acknowledgement in 
response to a connect request, indicating that the request was 
refused.

"Snd-DRI" means to send a DRI.

"Error" means to drop the transport layer connection, either 
politely or abortively, in response to an error condition.

"Snd-Disc-Req" means to send a request to close a transport 
layer connection.

"Snd-Disc-Ack" means to acknowledge closure of a transport 
layer connection, in response to receipt of a disconnect 
request.

"Cancel" means that the responder issues the Cancel event to 
the initiator state machine, indicating that the responder's 
transport layer connection will survive and the initiator's 
transport layer connection will be closed. The Cancel event is 
the only interaction between the two state machines.

"Process" means to service a Diameter message.

"Discard" means to ignore a Diameter message. The purpose here 
is to flush pending reads while waiting for disconnect to be 
acknowledged. Failure to do this under TCP could (depending on 
stack implementation) leave the connection half closed. The TCP 
stack will be in timed wait state, and might refuse subsequent 
connections from the same address/port for several minutes. I 
don't know if SCTP has a comparable issue.

Event Glossary
--------------
"Start" means that the Diameter application has signalled that a 
connection should be initiated.

"Stop" means that the Diameter application has signalled that a
connection should be terminated (e.g., on system shutdown).

"Cancel" means that the responder state machine has determined 
that the responder connection will survive and that the 
initiator's connection should be closed.

"Rcv-Conn-Ack" means that the peer has opened the transport 
layer connection requested by a Snd-Conn-Req.

"Rcv-Conn-Nack" means that the peer has refused the transport 
layer connection requested by a Snd-Conn-Req.

"Rcv-Disc-Req" means that the peer has requested that the 
transport layer connection be closed.

"Rcv-Disc-Ack" means that the peer has acknowledged the closure 
of the transport layer connection requested by a Snd-Disc-Req.

"Timeout" means that an application-defined timer has expired 
while waiting for some event.

"Rcv-DRI" means that a DRI has been received.

"Rcv-Non-DRI" means that a Diameter message other than DRI has 
been received.

Final Note
----------
I think the draft should clearly state that the state machine 
specification constrains the only the behavior of a Diameter 
implementation as seen by Diameter peers through events on 
the wire. Any implementation that produces equivalent results 
is considered compliant.

Paul

Paul Funk
Funk Software, Inc.
617 497-6339
[email-masked]

---

From: Patrice Calhoun <[email-masked]>
Date: Tue, 27 Mar 2001 09:14:12 -0800 (PST)

Paul,

I believe that the only way to really figure out whether this will work or not
is by implementation. Given this is a fairly critical component of the
protocol, I would feel uncomfortable putting it in the draft without a
successful interop.

So, would anyone out there be game to implement this, and we could run a test
over the net? I'm willing to do it, and could probably have this done by the
end of the day.

Any other takers?

PatC

---

From: "David Mitton" <[email-masked]>
Date: Wed, 28 Mar 2001 15:05:45 -0500

Folks,
         we have to do better than this.  Proving all race conditions are 
handled in running code is extremely difficult, even in the best lab 
situations.  This problem is best solved by some analysis and a tighter 
state definition.

         The problem is that we now have a peer-to-peer situation, where 
either side can simultaneously decide to initiate a request.  Any scheme 
that attempts to ignore or abort the incoming connection while attempting 
their own connection is subject to lock-out or indefinite retries.
         Paul has suggested an approach where the connections are accepted, 
and an election process determines which will be closed.
I believe this will work well and deterministicly, but his state tables 
still have some small problems.  I will edit his text and re-post an 
improved solution soon.

         Dave.

BTW: cross-posting to AAA-WG and Diameter lists is driving me (and my 
message filters) nuts.  Can we use one or restrict topics??

At 3/27/01 12:14 PM -0500, Patrice Calhoun wrote:

> >... <snip>

---------------------------------------------------------------
David Mitton                                  ESN: 248-4570
Advisor, Nortel Networks                      978-288-4570 Direct
Wireless Solutions, IP Mobility
Billerica, MA 01821                    [email-masked]

---

From: Patrice Calhoun <[email-masked]>
Date: Wed, 28 Mar 2001 17:09:02 -0800 (PST)

> situations.  This problem is best solved by some analysis and a tighter 

Right, but basic implementation will at least show whether or not it is even
possible toimplement. Having tried, I found out that it isn't that simple.
Paul is updating the state machine to something easier to implement.

I agree that analysis is required, and I will do so, but there is nothing like
actually writing to code to sanity check the basic table.

> their own connection is subject to lock-out or indefinite retries.

Right, but to be fair, the old table was not subjected to that. I agree that
Paul's table has a much richer set of conditions, but complexity goes along
with it (in this case, justifiable).

> still have some small problems.  I will edit his text and re-post an 

I would recommend that you wait until tomorrow. I am waiting for some
clarifications on a new proposal, and if it works better, then I would prefer
that you comment on the new one.

PatC

> message filters) nuts.  Can we use one or restrict topics??

yes, and I am trying to figure out what to do with the Diameter list. Perhaps
turn it into an implementor's list?

PatC