Subject: Re: [Dime] answer routing

From: Victor Fajardo <[email-masked]>
Date: Wed, 01 Mar 2006 06:44:17 -0500

Hi Anders,

> RFC 3588 talks about request routing but AFAICT it doesn't really talk 
> about how answer messages are routed. Given that request failover is 
> triggered by connection failures I concluded that answers are always 
> sent on the transport connection that the request was received on and 
> if that connection has failed the server just doesn't send the answer. 
> It's recommended somewhere that the server hang on the answer for a 
> period of time in case the request is retransmitted to the same server 
> on a new connection but the server would not send the answer on a new 
> connection without first receiving a retransmission of the request on 
> that connection.

I agree in part. The base proto is not clear about the behavior when it
has to send an answer when its own link to that route has recently 
broken. Though, I'm not sure if storing answer messages in the base 
protocol is useful because there is already a request retransmission 
mechanism. If a retransmitted request is received by a server on a new 
connection, the T flag is set and the application itself can have the 
intelligence to respond accordingly and send an answer on that new 
connection.

>
> The other piece of supporting evidence is that answer messages do not 
> contain Destination-Host (3588, 6.2) which I think would be a 
> reasonable thing to expect if answers were routed in a way similar to 
> requests.

I agree. hop-to-hop id is the only method.

Regards,
victor

>
> Is this the correct interpretation? Did I miss an explicit statement 
> to this effect anywhere in the base spec?
>
> Thanks,
> Anders
>
> _______________________________________________
> DiME mailing list
> [email-masked]
> https://www1.ietf.org/mailman/listinfo/dime
>
>

_______________________________________________
DiME mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/dime

---

From: "Tolga Asveren" <[email-masked]>
Date: Wed, 1 Mar 2006 08:56:02 -0500

Comments inline....

> -----Original Message-----
> From: Victor Fajardo [mailto:[email-masked]]
> Sent: Wednesday, March 01, 2006 6:44 AM
> To: Anders Kristensen
> Cc: [email-masked]
> Subject: Re: [Dime] answer routing
>
>
> Hi Anders,
>
[TOLGA]IMHO RFC3588 is pretty clear about how answers are to be routed -it
is as explained by Anders-. If the link where corresponding request has been
received is broken, answer is not sent.
OTOH, I don't know the rationale behind not acknowledging answer messages.
Afterall base protocol performs some transport functionality and not to have
some acknowledgement mechanism for answers causes all that hassle about
storing requests and checking for retransmission if T-flag is set. It seems
pretty inefficient to me. To have some guideleines about how E2E-Id is
incremented, e.g. sequentially and piggybacking last received answers E2E-Id
in request messages and possibly introducing a new message to send answer
acknowledgement if there is no requests to send could help -just as an
initial idea, I didn't think much about details- not to store received
requests for long time.
Asking applications to take care of retransmission is also another approach
and considering that they are aware of the expected message flow, they may
use that extra information to decide for retransmission more efficiently for
*most* of the times, but I think this may not always be true. To define a
transport protocol, where requests are retransmitted by the protocol but
duplicate detection for requests is performed by the user of the protocol
may be considered architecturally not elegant either.
In any case, AFAICS duplicate detection in RFC3588 not very efficient, but I
suspect there could be some reasons which led to the current design. If
somebody knows them and shares them with us, it could be great to get more
insight.

---

From: Victor Fajardo <[email-masked]>
Date: Wed, 01 Mar 2006 09:50:04 -0500

Comments inline ...

>received is broken, answer is not sent.
ok.

>initial idea, I didn't think much about details- not to store received
I see. E2E sequencing is definitely useful and maybe the only 
functionality required to provide message loss and answer co-relation.  
Though, I probably dont understand well but I'm not sure how sending an 
'answer acknowledgement' is more efficient. This seems redundant since 
it means acknowledging an acknowledgement.

>may be considered architecturally not elegant either.
I dont have strong opinions on this and it maybe a matter of semantics. 
In general both retransmission and duplicate detection (t-flag) is done 
by the base protocol. how to process receipt of a duplicate request is 
probably an issue left up to the application.

>somebody knows them and shares them with us, it could be great to get more

regards,
victor

---

From: "Tolga Asveren" <[email-masked]>
Date: Wed, 1 Mar 2006 09:44:08 -0500

Hi Victor,

Please see for comments below.

   Thanks,
   Tolga

> Sent: Wednesday, March 01, 2006 9:50 AM
> To: Tolga Asveren
> Cc: [email-masked]
[TOLGA]Probably it could be a mechanism similar to what is present in other
transport layer protocols: One could have sender/receiver windows which are
maintained based on a sequence number, i.e. E2E-Id.

AFAICS the issue is relying on the concept of request/response on transport
level is not very common, everything is usually considered a message. Each
side keeps a sequence number for send/receive directions. the concept of
request/response seems a bit higher than the transport protocol level. Right
now, responses are used to acknowledge requests but there is nothing to
acknowledge responses, afterall both are messages transported by the base
protocol. Acknowledging responses would allow to keep the data, which is
necessary for duplicate detection/handling only till the acknowledgement for
the response is received rather than an indefinete/arbitrarly choosen amount
of time.

---

From: Anders Kristensen <[email-masked]>
Date: Wed, 01 Mar 2006 10:16:43 -0500

> intelligence to respond accordingly and send an answer on that new 

Right, I just wanted to clarify the intention of the current spec and I 
thought I read somewhere (can't find the reference right now) that the 
server should hang on to the answer for some time just in case the 
request is retransmitted. If the server is idempotent it could just 
choose to recompute the answer and no one would know the difference.

Anyway, it sounds like there's agreement that answers are *always* sent 
on the transport connection on which the corresponding request was received.

Thanks,
Anders

---

From: Victor Fajardo <[email-masked]>
Date: Wed, 01 Mar 2006 10:57:34 -0500

Hi Tolga,

>maintained based on a sequence number, i.e. E2E-Id.
I see.

>level is not very common, everything is usually considered a message. Each
It not common above transport (tcp/tls/sctp) level though I can 
uderstand why diameter diameter does it. It is especially useful when 
result-code avp for each answer is relevant. As an example, in 
diameter-EAP the result-code may indicate one or more 
continued-authentication before final success or failure.

>the response is received rather than an indefinete/arbitrarly choosen amount
I see. Though I'm still not sure how this can help in duplicate 
detection of request. Also, what happens when the 'answer 
acknowledgement' itself gets lost due to a failure on one of the peers 
along the answer path. Will we send another ack after a period of time 
and keep state ? How will the answer ack get routed after the path 
failure ? This maybe one reason why current duplicate detection is more 
resilient but maybe less efficient as you mentioned. I'm not really sure 
how the designers approach to this issue and maybe they can shed some 
ligth on thier decision(s).

To come back to an initial conversation, I think that storing answer 
messages is not very useful and I dont know if its in the spec. I think 
sequencing and retransmission is effective.

Thanks and regards,
Victor

>>>Asking applications to take care of retransmission is also
>>>      
>>>
>>another approach
>>    
>>
>>>and considering that they are aware of the expected message
>>>      
>>>
>>flow, they may
>>    
>>
>>>use that extra information to decide for retransmission more
>>>      
>>>
>>efficiently for
>>    
>>
>>>In any case, AFAICS duplicate detection in RFC3588 not very
>>>      
>>>
>>efficient, but I
>>    
>>
>>>somebody knows them and shares them with us, it could be great
>>>      
>>>
>>to get more
>>    
>>
>>>insight.
>>>
>>>
>>>      
>>>
>>regards,
>>victor
>>
>>    
>>

---

From: "Tolga Asveren" <[email-masked]>
Date: Wed, 1 Mar 2006 12:24:19 -0500

Hi Victor,

> Sent: Wednesday, March 01, 2006 10:58 AM
> To: Tolga Asveren
> Cc: [email-masked]
[TOLGA]I believe this type of semantics fits more to application layer
rather than to transport layer. IMHO, the main task of transport layer
should be successful delivery of messages.
> ligth on thier decision(s).
[TOLGA]I think with a properly designed mechanism the problems you mentioned
may be overcome. I don't have answers yet, actually let me put a draft
together, it will be easier to discuss it once there is a document, which
presents the idea completely.

I am not suggesting that a method mimicing traditional transport layer
protocols completely is the way to go-maybe it is, I don't know now-, but
having some mechanism to acknowledge answers will help not to store much
data to detect duplicates -for that more below-.

> sequencing and retransmission is effective.
[TOLGA]Appendix C there are some sentences hinting about it but I wouldn't
think it is normative:
     A Diameter server MAY check the T flag of the received message to
      determine if the record is a possible duplicate.  If the T flag is
      set in the request message, the server searches for a duplicate
      within a configurable duplication time window backward and
      forward.

IMO, rather than what it says in Appendix C, the real issue is whehter there
is some other solution -afterall duplicate detection is a local procedure-.
I would like to make a distinction here between two types of servers:
a)Servers which can regenerate a previous answer
b)Servers, which can't regenerate a previous answer

If some data bookkeeping is necessary, it probably would be limited to
OriginHostAvp+E2eId for a). For b), the answer itself need to be kept as
well.

Why I believe that some bookkeeping is necessary -according to the current
procedures defined in RFC3588- is as follows:
Let's consider the follwing message flow from server's point of view:
(I assume E2E-Id contains a number, which is sequentially increased by the
sender and that this number is at a well-defined place in E2E-Id. This is
currently not there in RFC3588, but still does not solve the problem
completely)
Server receives messages with sequence number 27001, 27002, 18933. Now, how
can the server decide what the reason is for receiving 18933? It could be
because this request has never been received at all by the server, e.g. the
relay agent via which the request was to be sent was down and 27001, 27002
are sent through other relay agents, or it could be that request was
received by the server and the answer was generated but the answer did not
reach the client, e.g. due to relay agent crash. Based on the reason, the
behavior will be different. AFAICS, the only way to figure our the real
reason is to keep the history of the requests received -at least for a
while-, and the current mechanism seems not very efficient to me in that
regard.

---

From: Victor Fajardo <[email-masked]>
Date: Wed, 01 Mar 2006 18:51:09 -0500

Hi,

>should be successful delivery of messages.
  I guess if tranport layer==(tpc/tls/sctp) and application layer == 
diameter base protocol then I agree :)

ok.

>data to detect duplicates -for that more below-.
ok

>b)Servers, which can't regenerate a previous answer
... this distinction is better made by the application is'nt it ? One 
thing I was thinking of was that since the application already keeps 
state about an ongoing session, it maybe well informed about what to do 
with dups without needing extra info. The following snippet from sec 3 
of the spec maybe provide some hint:

"The End-to-End Identifier
      MUST NOT be modified by Diameter agents of any kind.  The
      combination of the Origin-Host (see Section 6.3) and this field is
      used to detect duplicates.  Duplicate requests SHOULD cause the
      same answer to be transmitted (modulo the hop-by-hop Identifier
      field and any routing AVPs that may be present), and MUST NOT
      affect any state that was set when the original request was
      processed.  Duplicate answer messages that are to be locally
      consumed (see Section 6.2) SHOULD be silently discarded."

If I interpret correctly, the application decides what to do with dups.

>currently not there in RFC3588, but still does not solve the problem
ok

>reach the client, e.g. due to relay agent crash. 
>
I see. Just to clarify, (pls let me know if i did not understand 
correctly) you want to make a distinction between retransmission of 
messages from a relay because of failover and retransmission by the 
originator of the message, i.e. the client, because it has not received 
an answer. One thing I was thinking of is that a retransmission from the 
originator can simply use a new e2d id to distinguish itself. I guess 
any message originator should use a new e2e (may already be hinted by 
the spec).

>Based on the reason, the
ok. I do agree there is room for improvement.

Thanks and Regards,
victor

---

From: "Tolga Asveren" <[email-masked]>
Date: Thu, 2 Mar 2006 09:43:03 -0500

Hi Victor,

[..snip..]
> If I interpret correctly, the application decides what to do with dups.
[TOLGA]The application logic may help for certain situations but I believe
there are certain cases where the service itself is stateless and probably
should rely on OriginHost+E2E-Id for duplicate detection.

[..snip..]
> >relay agent via which the request was to be sent was down and
[TOLGA]The distinction is necessary between cases, where a request is
retransmitted by base protocol and received by the server for the first time
and the request is retransmitted by the base protocol but original
transmission of the request was already received by the server and the
corresponding answer was generated but the answer was lost.

In any case, let me try to put some document together, then we can see
whether it makes sense.

---

From: Victor Fajardo <[email-masked]>
Date: Thu, 02 Mar 2006 10:31:55 -0500

Hi,

[..snip..]

ok.

ok.

regards,
victor