Subject: [ipfix-eval] On template negotiation "complexity" in CRANE

From: "Peter Ludemann" <[email-masked]>
Date: Thu, 31 Oct 2002 11:25:01 -0800

From some of the emails in the IPFIX group, there seems to be a worry that
the CRANE protocol's template negotiation is a complicated thing. This note
will explain how a minimal conforming implementation does not need to be
complex. I hope that the advantages of template negotiation are obvious: if
not, I'll write a separate note on that.

I'll present template negotiation in a series of more complex scenarios.
Each one of these scenarios is a conforming implementation of the CRANE
protocol. For simplicity, the details of the CRANE messages have been left
out; if anybody wants them, I can send a separate note.
  1.. No negotiation by collector:
    1.. Exporter says: "I am outputting fields a,b,c,d,e."
    2.. Collector says: "I agree; please send everything."
  2.. No negotiation by exporter:
    1.. Exporter says: "I am outputting fields a,b,c,d,e."
    2.. Collector says: "I only want fields a,c,e."
    3.. Exporter says: "Too bad, I'm sending all fields anyway."
    4.. Collector  replies "OK" and says to itself: "Oh well, I can just
throw away what I don't need".
  3.. Negotiation by collector and exporter:
    1.. Exporter says: "I am outputting fields a,b,c,d,e."
    2.. Collector says: "I only want fields a,c,e."
    3.. Exporter says: "Fine, I am outputting fields a,c,e." To itself, it
says: "And I can save some processing time by not computing the data for
fields b,d."
  d.. Reconnecting (negotiation has been done sometime in the past and we
want to get data transferring started as soon as possible):
    1.. Collector says to Exporter: "Tell me briefly what you're exporting"
    2.. Exporter says: "I have fields a,b,c,d,e available but am only
sending "a,c,e", as was previously agreed.
    3.. Collector says to Exporter: "Fine, send data please."
  5.. Negotiation by multiple collectors and an exporter (reliability mode
with fail-over):
    1.. Exporter says to Collector 1: "I am outputting fields a,b,c,d,e."
    2.. Collector 1 says: "I only want fields a,c,e."
    3.. Exporter says to Collector 2: "I am outputting only fields a,c,e
from the possible list of a,b,c,d,e".
    4.. Collector 2 says: But I want fields a,b,c,e."
    5.. Exporter says to Collector 1: "I am outputting fields a,b,c,e."
    6.. Collector 1 says: " I only want fields a,c,e."
    7.. Exporter says to Collector 1: "Too bad, I'm sending fields a,b,c,e
anyway."
    8.. Exporter says to Collector 2: "I'm sending fields a,b,c,e."
    9.. Collectors 1 and 2 respond with "OK".
The last scenario is a bit extreme because all the collectors should be
configured to want the same fields. But it is a possibility, if the
collectors or exporters are being upgraded in a "non-stop" environment and
the configuration is being changed on the fly (e.g., a new exporter has
additional fields; or there is a change in the amount of detail being
collected).

With multiple collectors, the exporter controls the negotiation
conversation. There is no assumption that the collectors can communicate
with each other. The exporter can decide at any point to stop the discussion
and dictate what fields will be transmitted, thus ensuring that the
conversation terminates.

As you can see, the simplest scenario (#1) requires almost no work to
implement. The exporter denies all negotiation request and always sends
everything.

- peter

---

From: "Carter Bullard" <[email-masked]>
Date: Thu, 31 Oct 2002 19:45:21 -0500

Hey Peter,
Rather than thinking that reliability must be the driving
motivator of the protocol, think about how can the protocol
be developed to provide the most utility.  Sure, if
reliability is the big thing, it needs to be there.  If
high performance is the big thing, then other strategies
may be worth investigating.  With architectural constraints,
I can build a reliable multicast network that is more reliable
than a TCP based strategy, at high rates.  It's a good real
world example.  

Requiring the IPFIX protocol to have a bidirectional control
channel, and only support unicast transport, I think is really
"short sheeting" this modern transport protocol, IMHO.

Carter

> -----Original Message-----
> From: Peter Ludemann [mailto:[email-masked]] 
> Sent: Thursday, October 31, 2002 6:47 PM
> To: [email-masked]; [email-masked]
> Subject: RE: On template negotiation "complexity" in CRANE
> 
> 
> Carter:
> 
> Well, multicast (UDP) is hardly reliable, is it?
> 
> I suppose that CRANE can be turned into an unreliable 
> broadcast by doing as
> you say: assume an "OK" response when the templates are 
> announced. But I
> wouldn't recommend this: what if the packet with the template 
> information is
> lost? Confusion downstream ...
> 
> Can broadcast of change of template could be made safe? I'd 
> suggest instead
> using a reliable channel for handling such announcements. You 
> could use
> CRANE's notion of a "configuration ID" to mark data records 
> that are encoded
> with the old templates and those that are encoded under the 
> new templates.
> 
> - peter
> 
> > -----Original Message-----
> > From: Carter Bullard [mailto:[email-masked]]
> > Sent: Thursday, October 31, 2002 1:27 PM
> > To: 'Peter Ludemann'; [email-masked]
> > Subject: RE: On template negotiation "complexity" in CRANE
> >
> >
> > Hey Peter,
> > I'm still thinking "here are the fields", "here is the data".
> > Here is an example of how this might be used.
> >
> > Suppose I want to transmit flow records to a multicast address.
> > The sender really doesn't care if there are particular
> > receivers, it just needs to send data.  Each of the receivers
> > however, may need to know something about the data that is
> > being received.  One way is for the IPFIX protocol to periodically
> > announce its fields, for those receivers that have joined
> > the multicast group after the initial start of transmission.
> > I don't think I like receivers of a multicast IPFIX data stream
> > having to know the unicast address of the transmitter, in order
> > to discover what the output format is.
> >
> > Not currently in the IPFIX set of scenarios, but definitely a
> > real world possibility, and one the protocol should support,
> > IMHO.
> >
> > Carter
> >
> > Carter Bullard
> > QoSient, LLC
> > 300 E. 56th Street, Suite 18K
> > New York, New York  10022
> >
> > [email-masked]
> > Phone +1 212 588-9133
> > Fax   +1 212 588-9134
> > http://qosient.com
> >
> >
> > > -----Original Message-----
> > > From: Peter Ludemann [mailto:[email-masked]]
> > > Sent: Thursday, October 31, 2002 3:47 PM
> > > To: [email-masked]; [email-masked]
> > > Subject: RE: On template negotiation "complexity" in CRANE
> > >
> > >
> > > Carter:
> > >
> > > A minimal CRANE exporter could send only messages that say
> > > "here are the fields, no discussion"; and a minimal CRANE
> > > receiver can always reply with "I agree" to any template
> > > messages from the exporter. Does this meet your criteria?
> > >
> > > I image such a minimal implementation having a hard-coded
> > > template message on the exporter, which is sent in response
> > > to any template information request. On the receiver, there
> > > would also be hard-coded template, which should be the same;
> > > if there's a difference, that indicates the exporter isn't
> > > what was expected and the collector should report a
> > > configuration error (e.g., via SNMP); otherwise, the
> > > collector just replies to the exporter with "I agree" and
> > > then settles down to accept the actual data.
> > >
> > > - peter
> > >
> > > > -----Original Message-----
> > > > From: Carter Bullard [mailto:[email-masked]]
> > > > Sent: Thursday, October 31, 2002 12:27 PM
> > > > To: 'Peter Ludemann'; [email-masked]
> > > > Subject: RE: On template negotiation "complexity" in CRANE
> > > >
> > > >
> > > > Hey Peter,
> > > >    I have the same protocol strategy in the Argus protocol,
> > > > where an explicit response is required.   I'm just thinking
> > > > of existing legacy flow transport strategies, where 
> there currently
> > > > isn't even a request for data.  While not a show stopper in
> > > any way, I
> > > > think it may need to be discussed.
> > > >
> > > > Regards,
> > > >
> > > > Carter
> > > >
> > > > Carter Bullard
> > > > QoSient, LLC
> > > > 300 E. 56th Street, Suite 18K
> > > > New York, New York  10022
> > > >
> > > > [email-masked]
> > > > Phone +1 212 588-9133
> > > > Fax   +1 212 588-9134
> > > > http://qosient.com
> > > >
> > > >
> > > >
> > > > > -----Original Message-----
> > > > > From: Peter Ludemann [mailto:[email-masked]]
> > > > > Sent: Thursday, October 31, 2002 3:17 PM
> > > > > To: [email-masked]; [email-masked]
> > > > > Subject: RE: On template negotiation "complexity" in CRANE
> > > > >
> > > > >
> > > > > Carter Bullard [mailto:[email-masked]] wrote
> > > Thursday, October
> > > > > 31, 2002 11:58 AM:
> > > > >
> > > > > > There is one combination that is missing that I think
> > > > > > is important.
> > > > > >
> > > > > >   Exporter says: "I am outputting fields a,b,c,d,e,."
> > > > > >   Collector says: [no response]
> > > > > >   Exporter sends data.
> > > > > >
> > > > > > While it may seem counter-intuitive to support this option,
> > > > > it is the
> > > > > > current state of the industry.  Can Crane handle 
> this type of
> > > > > > negotiation?
> > > > >
> > > > > Yes, but not with [no response].  [no response] is 
> not allowed by
> > > > > the CRANE protocol.  Instead, use a "I agree" response.
> > > The exporter
> > > > > makes the final decision as to what is exported, so 
> there is no
> > > > > problem with a single collector just accepting whatever
> > > it is told.
> > > > >
> > > > > Our general philosophy is to require some kind of an
> > > acknowledgment
> > > > > to all messages. (Data messages are acknowledged in 
> groups; all
> > > > > others are acknowledged
> > > > > individually.) For some messages, acknowledgment can be
> > > > > "asynchronous" (e.g., a request is sent with a request
> > > ID, which is
> > > > > used to identify the response).
> > > > >
> > > > > We think that it is a Bad Thing to allow silence to be
> > > equivalent to
> > > > > an acknowledgment. This is particularly the case for a compact
> > > > > binary protocol: if there is a mismatch between what the
> > > exporter is
> > > > > sending and what the collector is expecting, the only
> > > indication of
> > > > > a problem might be when somebody notices strange values
> > > showing up
> > > > > in the data.
> > > > >
> > > > > - peter
> > > > >
> > > > >
> > > >
> > >
> > >
> >
> 
> 

--
Help        mailto:[email-masked] and say "help" in message body
Unsubscribe mailto:[email-masked] and say
"unsubscribe ipfix" in message body
Archive     http://ipfix.doit.wisc.edu/archive/