Subject: Re: [ipfix-req] IPFIX Requirement draft status - section 6.3.2

From: [email-masked]
Date: Tue, 01 Oct 2002 10:52:31 -0400

I agree, we need more clarity on this issue. 

Here are the 4 items I see being implied when we discuss 
reliability:

	1. Congestion awareness.
	2. Message ordering
	3. Transport reliability
	4. Application layer reliability

Since we do not have infinite storage capacity a fifth one is...

	5. Tracking data loss (statistics)

	#1 is required by the IETF. 
	#2 depends on the protocol (i.e. do things break if messages 
	   are not processed in order.)

	With #3 a certain amount of data may be lost when the
	connection goes down but given a valid connection the 
	data has guaranteed delivery.

	With #4, the application needs to acknowledge receipt
	of messages. Unacknowledged messages are resent. This
	one also creates a problem of weeding out duplicate data
	since mesasges may be resent. We would also need to define
	the semantics of the Ack. Does it only mean message received
	or does it mean message recieved and processed (i.e. persisted
	in case of failure).

Also, since different applications require different levels of
reliability, is this aspect tunable? The main reason, I think, for
making 
it tunable would be if there is a significant performance difference
between the levels.

I'll stop here for comments from the list.

Paul

> Reinaldo Penno wrote:
> 
> Hello,
> 
> I'm looking to understand/really clarify this requirement. It seems
> ambiguous...at least to me. I do not understand what is the
> meaning/purpose of "-->abscence<-- of reliability....MUST be
> indicated". Shouldn't we just rephrase that by just saying that the
> protocol MUST be reliable, provide in-sequence delivery and
> retransmission?
> 
> It seems to me we should have 2 options:
> 
> 1. The IPfix protocol runs over TCP or SCTP
> 2. The ipfix protocol does not run over TCP or TCP but provide a
> service equal or in excess to what is provided by TCP/SCTP in terms of
> reliability, congestion awareness and in-sequence delivery.
> 
> Reading this text I feel there is a third option which is:
> 
> 3. "the protocol should run over TCP or SCTP, but you can sidestep
> this requirement by putting a sequence number on the application layer
> and/or providing 'abscence' of reliability"...
> 
> Or maybe one should ask what's the definition of a reliable protocol
> as far as the ipfix protocol is concerned...
> 
> "6.3.2.  Reliability
> 
>    Absence of reliability of the data transfer, for example caused by
>    loss or reordering of packets containing flow information, MUST be
>    indicated.
> 
>    Please note that if an unreliable transport protocol is used,
>    reliability can be provided by higher layers. In such a case only
>    lack of overall reliability MUST be indicated. For example
> reordering
>    could be dealt with by adding a sequence number to each packet."
> 
> regards,
> 
> Reinaldo

--
Help        mailto:[email-masked] and say "help" in message body
Unsubscribe mailto:[email-masked] and say
"unsubscribe ipfix" in message body
Archive     http://ipfix.doit.wisc.edu/archive/

---

From: "Peter Ludemann" <[email-masked]>
Date: Tue, 1 Oct 2002 23:50:37 -0700

[email-masked] wrote Tuesday, October 01, 2002 7:53 AM

> 	data has guaranteed delivery.

And there must be sufficient information so that de-duplication
can be done at the receiving end (it is impossible to design
fail-over such that no duplicates occur).
 
> 	of messages .... Does [ACK] only mean message received

I am not sure that this is part of the protocol (for example, the
TCP spec does not make this clear; but all TCP implementations that
I know of send ACK only at the socket level -- but there is nothing
stopping somebody from implementing TCP such that the ACK happens
when the data have been fully processed ... in fact, the existence
of such implementations would have greatly simplified our work).

However, a useful implementation would ACK only when the data have
been processed in such a way as to allow recovery after a crash.
Otherwise, there's not much point in having a reliable protocol:
ordinary TCP would suffice.
 
> making it tunable would be if there is a significant performance
> difference between the levels.

The protocol must not prevent an efficient reliability mechanism at
the receiver. I think it is sufficient that the protocol allows a
delay of reception of ACKs (e.g., records 1-100 are sent, and then
ACK for 1-50 is received).

- peter

---

From: [email-masked]
Date: Wed, 02 Oct 2002 08:40:04 -0400

> ordinary TCP would suffice.

	This is why we need to define it as part of the
	protocol. Otherwise some implementations will ack as
	soon as the message is seen and others will ack after
	the message is persisted. The former doesn't provide
	any crash protection and thus, as you stated, might as well
	just use TCP.

> ACK for 1-50 is received).

	Right, but my question is do we define one level or reliability
	for all	or allow multiple levels. One level for all IMHO significantly
	reduces the specification, implementation and inter operability
	issues. However, allowing multiples may allow significant
	performance increases when the extra reliability is not needed.
	I'm not sure which is the best way to go.

	Paul

---

From: Simon Leinen <[email-masked]>
Date: 05 Oct 2002 00:30:17 +0200

On Mon, 30 Sep 2002 03:00:03 -0700, "Reinaldo Penno" <[email-masked]> said:
> indicated".

Yes, that just sounds weird.  (High) reliability should be a goal for
the exporting process, not something that can be readily measured, or
even less is "present" or "absent" at a given time during the
operation of an IPFIX implementation.

reqs-06> Absence of reliability of the data transfer, for example caused by
reqs-06> loss or reordering of packets containing flow information, MUST be
reqs-06> indicated.

Why not write something like this instead:

  "If some flow information data cannot be delivered to the collector
   in a timely manner, the amount of missing flow information data
   MUST be indicated to the collector.  Possible reasons for failure
   to deliver data include loss of packets on the network path, or
   resource shortage at the exporter."

I removed packet reordering as an example for reasons to lose data
because I don't think it's that relevant in practice.  From
operational experience, I think overflows at the exporter are a much
more likely source of data loss.

An indication of the amount of data lost is more useful than an
unspecific indication of "absence of reliability" and should be
relatively easy to provide.

reqs-06> Please note that if an unreliable transport protocol is used,
reqs-06> reliability can be provided by higher layers. In such a case
reqs-06> only lack of overall reliability MUST be indicated.

And end-to-end purist might argue that even if the underlying
transport protocol is used, reliability must be provided at higher
layers too.  See the "application layer ACK" debate on the ipfix-req
list a few days ago.

  "Reliability can be supported by suitable transport protocols.  In
  any case only end-to-end flow information loss between exporter and
  collector need be indicated."

reqs-06> For example reordering could be dealt with by adding a
reqs-06> sequence number to each packet.

Or order may not be relevant since the exact arrival time of a piece
of flow information doesn't matter - the timestamps in the flow record
will provide sufficient sequence information for the collector.
-- 
Simon.

---

From: [email-masked]
Date: Mon, 07 Oct 2002 10:08:37 -0400

>    resource shortage at the exporter."

	I like this. 

	One question, why did you add "in a timely manner". What not just
	leave it at "cannot be delivered".

	Also, at some point need a clear definition of what "missing flow 
	information data" really means. Is it a byte and packet count, is
	it IPFIX messages? Both?

> more likely source of data loss.

	Too early to make this call. It could be that packet reordering
	does cause a loss of data. But that analysis can't happen until
	we have a protocol.
> relatively easy to provide.

	Agreed.
> will provide sufficient sequence information for the collector.

	The thing I don't like about going down this road is the
	potential storage and processing requirements needed at the 
	collector. Also, how do I know that when processing a timestamp 
	at 10:00 that another one is coming for 9:59?

	Paul

---

From: Benoit Claise <[email-masked]>
Date: Mon, 07 Oct 2002 16:21:40 +0200

Reinaldo, Paul, Peter and Simon,

[I've been trying to compile all ideas in order to reply]

6.3.2.  Reliability

   Please note that if an unreliable transport protocol is used,
   reliability can be provided by higher layers. In such a case only
   lack of overall reliability MUST be indicated. For example reordering
   could be dealt with by adding a sequence number to each packet.

As Peter pointed out, the idea behing this requirement is that: the 
connection between the exporting process and collecting process could be 
broken for a certain time; slow ACK, no routing, etc... or the router 
could have some problems: queue full
So we must at least indicate to the collecting process that some data 
where lost somewhere (on the exporting process or along the path)

Paul divided into a few categories:

	1. Congestion awareness.
	2. Message ordering
	3. Transport reliability

	1. Taken into consideration by the section 6.3.1 "Congestion Awareness"

	2. Paul is right: depends on the protocol. Not sure we need a special section for that

	3. The transport protocol reliability would be a completely new requirement, no even part of the charter. Unless you only mean by this: when the transport connection breaks. If this is the case, then see #4.

	4. If you want ACK at the application level, this would be a completely new requirement. Actually the goal of section 6.3.2 "Reliability" was to take care of that issue, without requiring ACK at the application level (which I personally think is not possible due to the amount of traffic exported). Simon's comment on removing the "message ordering" from section 6.3.2 makes sense (see #2 where the message ordering depends on the transport protocol). I vote for Simon's proposal.

                5. I think this is taken into consideration by the text 
in #4.

What do you think?

Regards, Benoit