Subject: RE: [ipfix-eval] RE: On template negotiation "complexity" in CRANE

From: "Peter Ludemann" <[email-masked]>
Date: Fri, 1 Nov 2002 17:41:29 -0800

Carter:

I think that this discussion is best taken offline. The "multicast" stuff
has been thrashed around a fair bit and I think that the consensus is that
we need to define a protocol that works over a wider distance than the
constrained single-hop multicast network you're thinking of.

Of course, it's nice if the chosen protocol can be adapted for such a
constrained network ... my personal feeling is that the "control" stuff
(templates, etc.) are best done with a fully reliable channel (based on TCP
or SCTP, plus application-level acknowledgment), with the data containing
"version" identification so that the receiver knows which version of
templates are being used for a given record. This would allow the exporter
to signal that the record formats are changing and to ensure that all the
receivers have got the message but also allowing it to switch to the new
format before everybody has sent in acknowledgment (at the risk of some
receivers throwing away data they don't understand until they get the new
format information). Although CRANE is designed to be fully reliable, it has
this capability in its "configuration ID" which is a kind of version
identification.

- peter

> -----Original Message-----
> From: majordomo listserver [mailto:[email-masked]]On Behalf
> Of Carter Bullard
> Sent: Thursday, October 31, 2002 4:45 PM
> To: 'Peter Ludemann'; [email-masked]
> Subject: [ipfix-eval] RE: On template negotiation "complexity" in CRANE
>
>
> Hey Peter,
> Rather than thinking that reliability must be the driving
> motivator of the protocol, think about how can the protocol
> be developed to provide the most utility.  Sure, if
> reliability is the big thing, it needs to be there.  If
> high performance is the big thing, then other strategies
> may be worth investigating.  With architectural constraints,
> I can build a reliable multicast network that is more reliable
> than a TCP based strategy, at high rates.  It's a good real
> world example.
>
> Requiring the IPFIX protocol to have a bidirectional control
> channel, and only support unicast transport, I think is really
> "short sheeting" this modern transport protocol, IMHO.
>
> Carter
>
>
>
> > -----Original Message-----
> > From: Peter Ludemann [mailto:[email-masked]]
> > Sent: Thursday, October 31, 2002 6:47 PM
> > To: [email-masked]; [email-masked]
> > Subject: RE: On template negotiation "complexity" in CRANE
> >
> >
> > Carter:
> >
> > Well, multicast (UDP) is hardly reliable, is it?
> >
> > I suppose that CRANE can be turned into an unreliable
> > broadcast by doing as
> > you say: assume an "OK" response when the templates are
> > announced. But I
> > wouldn't recommend this: what if the packet with the template
> > information is
> > lost? Confusion downstream ...
> >
> > Can broadcast of change of template could be made safe? I'd
> > suggest instead
> > using a reliable channel for handling such announcements. You
> > could use
> > CRANE's notion of a "configuration ID" to mark data records
> > that are encoded
> > with the old templates and those that are encoded under the
> > new templates.
> >
> > - peter
> >
> > > -----Original Message-----
> > > From: Carter Bullard [mailto:[email-masked]]
> > > Sent: Thursday, October 31, 2002 1:27 PM
> > > To: 'Peter Ludemann'; [email-masked]
> > > Subject: RE: On template negotiation "complexity" in CRANE
> > >
> > >
> > > Hey Peter,
> > > I'm still thinking "here are the fields", "here is the data".
> > > Here is an example of how this might be used.
> > >
> > > Suppose I want to transmit flow records to a multicast address.
> > > The sender really doesn't care if there are particular
> > > receivers, it just needs to send data.  Each of the receivers
> > > however, may need to know something about the data that is
> > > being received.  One way is for the IPFIX protocol to periodically
> > > announce its fields, for those receivers that have joined
> > > the multicast group after the initial start of transmission.
> > > I don't think I like receivers of a multicast IPFIX data stream
> > > having to know the unicast address of the transmitter, in order
> > > to discover what the output format is.
> > >
> > > Not currently in the IPFIX set of scenarios, but definitely a
> > > real world possibility, and one the protocol should support,
> > > IMHO.
> > >
> > > Carter
> > >
> > > Carter Bullard
> > > QoSient, LLC
> > > 300 E. 56th Street, Suite 18K
> > > New York, New York  10022
> > >
> > > [email-masked]
> > > Phone +1 212 588-9133
> > > Fax   +1 212 588-9134
> > > http://qosient.com
> > >
> > >
> > > > -----Original Message-----
> > > > From: Peter Ludemann [mailto:[email-masked]]
> > > > Sent: Thursday, October 31, 2002 3:47 PM
> > > > To: [email-masked]; [email-masked]
> > > > Subject: RE: On template negotiation "complexity" in CRANE
> > > >
> > > >
> > > > Carter:
> > > >
> > > > A minimal CRANE exporter could send only messages that say
> > > > "here are the fields, no discussion"; and a minimal CRANE
> > > > receiver can always reply with "I agree" to any template
> > > > messages from the exporter. Does this meet your criteria?
> > > >
> > > > I image such a minimal implementation having a hard-coded
> > > > template message on the exporter, which is sent in response
> > > > to any template information request. On the receiver, there
> > > > would also be hard-coded template, which should be the same;
> > > > if there's a difference, that indicates the exporter isn't
> > > > what was expected and the collector should report a
> > > > configuration error (e.g., via SNMP); otherwise, the
> > > > collector just replies to the exporter with "I agree" and
> > > > then settles down to accept the actual data.
> > > >
> > > > - peter
> > > >
> > > > > -----Original Message-----
> > > > > From: Carter Bullard [mailto:[email-masked]]
> > > > > Sent: Thursday, October 31, 2002 12:27 PM
> > > > > To: 'Peter Ludemann'; [email-masked]
> > > > > Subject: RE: On template negotiation "complexity" in CRANE
> > > > >
> > > > >
> > > > > Hey Peter,
> > > > >    I have the same protocol strategy in the Argus protocol,
> > > > > where an explicit response is required.   I'm just thinking
> > > > > of existing legacy flow transport strategies, where
> > there currently
> > > > > isn't even a request for data.  While not a show stopper in
> > > > any way, I
> > > > > think it may need to be discussed.
> > > > >
> > > > > Regards,
> > > > >
> > > > > Carter
> > > > >
> > > > > Carter Bullard
> > > > > QoSient, LLC
> > > > > 300 E. 56th Street, Suite 18K
> > > > > New York, New York  10022
> > > > >
> > > > > [email-masked]
> > > > > Phone +1 212 588-9133
> > > > > Fax   +1 212 588-9134
> > > > > http://qosient.com
> > > > >
> > > > >
> > > > >
> > > > > > -----Original Message-----
> > > > > > From: Peter Ludemann [mailto:[email-masked]]
> > > > > > Sent: Thursday, October 31, 2002 3:17 PM
> > > > > > To: [email-masked]; [email-masked]
> > > > > > Subject: RE: On template negotiation "complexity" in CRANE
> > > > > >
> > > > > >
> > > > > > Carter Bullard [mailto:[email-masked]] wrote
> > > > Thursday, October
> > > > > > 31, 2002 11:58 AM:
> > > > > >
> > > > > > > There is one combination that is missing that I think
> > > > > > > is important.
> > > > > > >
> > > > > > >   Exporter says: "I am outputting fields a,b,c,d,e,."
> > > > > > >   Collector says: [no response]
> > > > > > >   Exporter sends data.
> > > > > > >
> > > > > > > While it may seem counter-intuitive to support this option,
> > > > > > it is the
> > > > > > > current state of the industry.  Can Crane handle
> > this type of
> > > > > > > negotiation?
> > > > > >
> > > > > > Yes, but not with [no response].  [no response] is
> > not allowed by
> > > > > > the CRANE protocol.  Instead, use a "I agree" response.
> > > > The exporter
> > > > > > makes the final decision as to what is exported, so
> > there is no
> > > > > > problem with a single collector just accepting whatever
> > > > it is told.
> > > > > >
> > > > > > Our general philosophy is to require some kind of an
> > > > acknowledgment
> > > > > > to all messages. (Data messages are acknowledged in
> > groups; all
> > > > > > others are acknowledged
> > > > > > individually.) For some messages, acknowledgment can be
> > > > > > "asynchronous" (e.g., a request is sent with a request
> > > > ID, which is
> > > > > > used to identify the response).
> > > > > >
> > > > > > We think that it is a Bad Thing to allow silence to be
> > > > equivalent to
> > > > > > an acknowledgment. This is particularly the case for a compact
> > > > > > binary protocol: if there is a mismatch between what the
> > > > exporter is
> > > > > > sending and what the collector is expecting, the only
> > > > indication of
> > > > > > a problem might be when somebody notices strange values
> > > > showing up
> > > > > > in the data.
> > > > > >
> > > > > > - peter
> > > > > >
> > > > > >
> > > > >
> > > >
> > > >
> > >
> >
> >
>
>
>
> --
> Help        mailto:[email-masked] and say "help" in
> message body
> Unsubscribe mailto:[email-masked] and say
> "unsubscribe ipfix" in message body
> Archive     http://ipfix.doit.wisc.edu/archive/

--
Help        mailto:[email-masked] and say "help" in message body
Unsubscribe mailto:[email-masked] and say
"unsubscribe ipfix" in message body
Archive     http://ipfix.doit.wisc.edu/archive/