Subject: RE: sylsog in NETCONF notifications

From: "Hector Trevino \(htrevino\)" <[email-masked]>
Date: Thu, 6 Jul 2006 10:05:38 -0700

in-line
Hector

> -----Original Message-----
> From: [email-masked] 
> [mailto:[email-masked]] On Behalf Of Rainer Gerhards
> Sent: Thursday, July 06, 2006 10:30 AM
> To: Andy Bierman
> Cc: [email-masked]
> Subject: Re: sylsog in NETCONF notifications
> 
> <inline>
> 
> On 7/6/06, Andy Bierman <[email-masked]> wrote:
> > I disagree.
> > I think it is fine for the NETCONF WG to reference a 
> specific version 
> > of syslog as the content format.  If the SYSLOG WG invented a new 
> > version, then the NETCONF PDU would just reference it.
> > I don't see a problem if the NETCONF WG creates configuration 
> > mechanisms for NETCONF (such as notification filtering) and uses 
> > well-known fields in a specific version of syslog.
> 
> I do not see any problem if the NETCONF WG creates a 
> configuration mechanism for NETCONF. But appendix B looks 
> like the NETCONF WG creates a configuration mechanism (or 
> better: notification mechanism) for syslog. If that is just 
> one way of doing things, why not also specifying how SNMP 
> traps and informs should be carried inside a NETCONF notification?

HT: Because nobody (that I know of) is interested/needs to carry
    SNMP notifications inside NETCONF. Whether or not the Appendix B
proposal stays
    in is up for discussion next week. 

> 
> If you talk about filtering and also talk about syslog data, 
> NETCONF IMHO actually tries to define a configuration 
> mechanism for SYSLOG.
> The draft is not taking SYSLOG as *the* content format and 
> just extending it with some syslog fields. The draft is 
> defining a NETCONF notification format (which is perfectly 
> fine and good to see) and then is mapping how SYSLOG should 
> be mapped into it. The later IMHO should be done as a 
> separate effort via a SYSLOG data model.

HT: The title of the section is "Leveraging Syslog Field Definitions"
and as the title I think suggests the purpose is to re-use existing
syslog definitions where it makes sense. 

> 
> > > Having syslog mappings in a basic netconf document forces the 
> > > netconf document to be changed/obsoleted whenever there 
> is change on 
> > > the syslog side. I think this dependency should be avoided.
> > >
> > > There is some growing interest in the syslog community to define 
> > > event data models. This effort could play nicely together 
> with netconf.
> >
> > Does this mean you think it is a good idea for the NETCONF WG to 
> > create a brand-new netconf-only notification system?
> 
> I think it is eventually a good idea for the NETCONF WG to 
> create a brand-new, EXTENSIBLE notification system that could 
> also transport events traditionally being transported via 
> SYSLOG, SNMP or vendor-specific event notification protocols. 
> If I understood NETCONF correctly, it is a common base that 
> it should be extensible.
> Extensions can be vendor-specific or via standard 
> capabilities. Lots of discussion on this list are focussed 
> about data models for configuration. If I follow that spirit, 
> NETCONF notifications should provide a framework for 
> notification delivery, but not necessarily (and exclusively) 
> all data models that could be transported via it.

HT: That's the idea. Right now the doc only covers syslog & as I said in
a previous e-mail
    SNMP is not something that has come up. 

> 
> > Or does it mean you think the SYSLOG WG should do this work 
> instead of 
> > the NETCONF WG?
> 
> I think the syslog community should standardize a SYSLOG data model.
> As much as it probably could standardize a basic SYSLOG 
> configuration data model (including syslog-specific filters 
> and event actions). I do not think that the current SYSLOG WG 
> (chartered in the SEC area) is the right one to do this. I 
> think a new SYSLOG WG should be chartered in the OPS area to 
> standardize syslog payload and thus also provide a data 
> model. Ideally, it would lean on the good work NETCONF has done.
> Such a data model could be advertised as a capability.

HT: I think this is good. Not sure what SYSLOG data model entails but
I'll assume it could include event types/contents. If so, could this be
done (i.e. event types, contents) defined in a protocol independent
manner. 

> 
> Rainer
> 
> --
> to unsubscribe send a message to [email-masked] 
> with the word 'unsubscribe' in a single line as the message text body.
> archive: <http://ops.ietf.org/lists/netconf/>
> 

--
to unsubscribe send a message to [email-masked] with
the word 'unsubscribe' in a single line as the message text body.
archive: <http://ops.ietf.org/lists/netconf/>

---

From: "Rainer Gerhards" <[email-masked]>
Date: Thu, 6 Jul 2006 22:39:05 +0200

<inline>
Rainer

>     in is up for discussion next week.

That's why I raised the point today.

>     SNMP is not something that has come up.

That's my problem: "right now the doc only covers syslog". If so, then
the doc is not talking about NETCONF but about SYSLOG ;) But if it is
about SYSLOG, then it is either beyond the charter of the NETCONF WG
or the NETCONF WG must endure that SYSLOG specifics be discussed here
(Andy, I think, rejected the later for good reasons).

I fear that if NETCONF notifications goes into too much detail on
SYSLOG, it will eventually distract interest in a solution for SYSLOG.
Think about it that way: any NETCONF agent that puts SYSLOG fields
into some NETCONF fields is not a pure NETCONF agent. It is actually a
SYSLOG/NETCONF gateway, as is shown in the netconf-syslog draft. The
SYSLOG side of that beast is actually ... a SYSLOG receiver.

Don't misunderstand me: I find it correct that NETCONF does
notifications, if NETCONF needs such. I also think it is valuable to
have an extensible notification mechanism. I just think that NETCONF
should standardize NETCONF and not whatever else seems handy. And,
yes, I agree SYSLOG messages are interesting to NETCONF agents. SNMP
events, IMHO, should also be interesting if you really intend to make
a single manager able to manage everything. In concept, I do not see
any big difference between the need to see SYSLOG vs. SNMP events if
you go after as many event data as possible. In practice, of course,
SNMP is mature and well-standardized while SYSLOG is a quick ad-hoc
solution. That doesn't mean that we should continue to look at it in a
quick ad-hoc way.

> done (i.e. event types, contents) defined in a protocol independent

So far, I have just a personal view. I will try to write a short I-D
with a problem statement soon. In my (purely personal) point of view,
I think an XML based data stream for SYSLOG content would be
advisable. For obvious reasons, that would not need to be limited to
SYSLOG. I like the data model approach NETCONF is taking. I can easily
see how one and the same XML stream could be transmitted as part of a
SYSLOG STRUCTURED-DATA element or as part of a NETCONF <GET> operation
(or whatever is invented as part of netconf notifications). Thus I
consider it vital that NETCONF does not place any preconception on the
way SYSLOG events are encoded. For example, a very basic
(hypothetical) SYSLOG event XML stream could look like:

<event>
  <syslog-message><144>HEADER,,, the raw message</message>
</event>

That obviously would be a solution for legacy senders, Other could do
things like

<event>
  <origin>orgsender.example.com</origin>
  <type>traffic-report</type>
   <source>10.0.0.1</source>
   <target>10.0.0.2</target>
   <bytes-transferred>22222</bytes-transferred>
   .
   .
   .
</event>

I think you'll get the idea. The last sample is my medium to long term
vision: some standard way of telling common events for firewalls
(traffic flow, access violations etc) and other common usage
scenarios,

OK, already written too much. I hope I have clarified.

Warp up: I think NETCONF should define what it needs for its
notifications, leave that as extensible (via capabilities) as the rest
of NETCONF and do not try to standardize things outside NETCONF
itself.

Rainer

---

From: Andy Bierman <[email-masked]>
Date: Thu, 06 Jul 2006 15:49:00 -0700

Rainer Gerhards wrote:

The thing I like the most about Phil's draft is that it
it reusing, rather than reinventing, syslog.

IMO, we should have the same type of extensible, content-independent
PDU and framework that we have for <rpc>, but that syslog delivery
should be the first (and only) content that we work on right now.

The "netconf requirement" is that the content is well-formed XML and
it is identified by a namespace URI.

I envision starting with syslog/RAW and even syslog/SD-Params,
and new work can start (in the SYSLOG WG) on syslog/XML
if there is enough interest.  I don't think the NETCONF WG
should be defining any standard notification content except maybe
a config-change notification.

Andy

---

From: Chris Lonvick <[email-masked]>
Date: Thu, 6 Jul 2006 17:23:15 -0700 (PDT)

Hi Andy,

> a config-change notification.

Speaking as syslog WG Co-Chair

I appreciate your confidence in the syslog WG.  :-)  At this time, we are 
VERY focused on meeting our milestones.  We are chartered in the Security 
Area to address some specific security requirements.  If we go down the 
path you envision, then we should probably be rechartered in the OAM Area.
I'm not opposed to that, but that discussion will have to wait until after 
we meet our milestones.

Unfortunately, I won't be able to attend the netconf WG meeting on Friday, 
but I believe that David Harrington, my Co-Chair, will be there and will 
be able to represent our WG if this is discussed.

Best regards,
Chris

---

From: Balazs Lengyel <[email-masked]>
Date: Fri, 07 Jul 2006 11:04:51 +0200

Hello,
I hope you are not very strict with this syslog ONLY. There are a good number of nodes 
that use non-syslog notification methods. Also I feel one basic reason why we need 
notifications is  to help Netconf configuration which id not strictly syslog related. For 
me this means we should work on topics like
- basic data needed in every notification
- event classes
- filtering on XML
as well.

Sorry if I misunderstood your intent.
Balazs

---

From: Andy Bierman <[email-masked]>
Date: Fri, 07 Jul 2006 05:43:07 -0700

Balazs Lengyel wrote:
> Hello,
> I hope you are not very strict with this syslog ONLY. There are a good 
> number of nodes that use non-syslog notification methods. Also I feel 
> one basic reason why we need notifications is  to help Netconf 
> configuration which id not strictly syslog related. For me this means we 
> should work on topics like

My intent is to have a content-independent framework for delivering
notifications, but not have the NETCONF WG develop any standard
notification content unless it is directly related to configuration.

IMO, syslog is important enough that it is worthwhile to have
filtering mechanisms that actually work for syslog.
Generalized filters sound great, but if they are impossible
to use for the most common application (syslog/RAW), then
they are mostly irrelevant.  Telling all the vendors "Hey, I have a
really cool XML tool, so if you could re-design and re-code
15,000 or so syslog messages to output an XML subtree instead of text,
well, that would be great."

I suggest coming up with a transition plan, because it will
be about a decade before you get to use your fancy XML tools on
notification content.

Andy

---

From: "Rainer Gerhards" <[email-masked]>
Date: Fri, 7 Jul 2006 15:02:30 +0200

Andy,

> be about a decade before you get to use your fancy XML tools on

I agree with you on the need and the timing. I am not sure if I am
with you who should do that. Are you saying the NETCONF WG should do
that? If so, I think it is beyond the NETCONF charter.

I would appreciate if you could clarify if you think NETCONF should

a) develop a notification mechanism and a data model for NETCONF
and/or
b) develop a data model and transformation rules for SYSLOG

Thanks,
Rainer

---

From: Balazs Lengyel <[email-masked]>
Date: Fri, 07 Jul 2006 15:09:50 +0200

Hello,
There are some NEW implementations for Netconf as well. These don't always need a 
transition plan. Also if you look at Jurgens requirement page a good number of 
requirements are not refering to syslog.
Generally I would like to avoid being forced to use syslog in order to have a standard 
notification mechanism with filtering.
Balazs

---

From: Andy Bierman <[email-masked]>
Date: Fri, 07 Jul 2006 07:13:42 -0700

> a) develop a notification mechanism and a data model for NETCONF

not sure what you mean by 'data model'.
Yes -- if that means a data model for the
NV-stored configuration parameters for notification delivery,
and perhaps some state data related to notification delivery.

> b) develop a data model and transformation rules for SYSLOG

If you mean transformation of all ad-hoc syslog text content
to a standard and structured XML hierarchy -- no.  That is way out
of scope.  However, filter parameters to suppress notification delivery
are in scope.  (The non-normative appendices are supposed to be removed.)

Andy

---

From: Andy Bierman <[email-masked]>
Date: Fri, 07 Jul 2006 07:27:16 -0700

Balazs Lengyel wrote:
> Hello,
> There are some NEW implementations for Netconf as well. These don't 
> always need a transition plan. Also if you look at Jurgens requirement 
> page a good number of requirements are not refering to syslog.
> Generally I would like to avoid being forced to use syslog in order to 
> have a standard notification mechanism with filtering.

I suggest using a proprietary format for XML notifications,
or asking the AD for a BOF or WG charter for standardized syslog/XML.

I am strongly opposed to any new mechanisms unless they
follow the "content-independent payload" concept.  This is easy.
Just define a namespace for the specialized content instead of
hard-wiring it in as an element.

Instead of a special "syslog" capability, we need a general "notification"
capability.  The <capability> exchange should include module identifiers
that indicate that the agent supports "syslog/RAW", "syslog/COOKED", "acme/TOASTED",
or whatever content you want to use. (It will probably be a combination
of all kinds of formats for many years.)   This should be no different than
advertising the data models that can be used with <edit-config> or <get>.

Andy

---

From: "Rainer Gerhards" <[email-masked]>
Date: Fri, 7 Jul 2006 17:20:32 +0200

Andy,

> are in scope.  (The non-normative appendices are supposed to be removed.)

I agree. But these filter parameters must not be SYSLOG specific - in
the same way as <get> is not specified for SYSLOG or any other
specific case. I'd like to see some standard properties for NETCONF. I
also think that specifying extensible filter mechanisms (as in section
6.) is useful and needed.

However, I do NOT think that specifying a "severity" filter for syslog
is appropriate (or any other SYSLOG reference). That I would expect to
see in a different document, and being announced as an optional
capability at the time of <hello>.

Putting any SYSLOG specifics into the core notification document
would, IMO,  remove its payload content-independence.

Rainer

---

From: Andy Bierman <[email-masked]>
Date: Fri, 07 Jul 2006 08:35:25 -0700

> 6.) is useful and needed.

I agree.
I was referring to the "regexp" text filtering in Phil's draft.
I don't the best way to accomplish syslog content filtering,
but I doubt either subtree or Xpath filters would be it.

I don't think that if the netconf protocol had any particular
syslog content filters (for netconf delivery of syslog), the
syslog protocol (or WG) would be impacted in any way.

> would, IMO,  remove its payload content-independence.

I agree.
We should not include non-normative appendices as a way
to avoid the actual standardization of that material either.
The final draft will be consistent with the architecture
and operations defined in the netconf-prot-12 draft.
We are not very close to a final draft yet.

Andy

---

From: "Rainer Gerhards" <[email-masked]>
Date: Fri, 7 Jul 2006 17:53:07 +0200

> syslog protocol (or WG) would be impacted in any way.

Just to clarify: my comment dug even deeper. Here, I am not only
concerned with SYSLOG. I think the cleanness of NETCONF notifications
would be compromised if the core notifications draft depends on
non-NETCONF-native filter capabilities. I think it should be aimed to
find a filtering capability that is able to express filters in a very
generic way, including nested boolean operations and regular
expression filters. I can envison that these are useful for vendor
extensions as well as for SYSLOG.

Rainer

---

From: Andy Bierman <[email-masked]>
Date: Fri, 07 Jul 2006 09:58:21 -0700

> extensions as well as for SYSLOG.

Agreed.  The protocol itself needs to be content-independent.
I argued that even the <eventClass> field should just be
a string instead of the netconf WG defining the allowable
values of that string.  (I don't know exactly what the WG
wants to do here yet.)

I like the basic architecture Phil is proposing (except I
want to make sure any NV-stored data is handled properly).
I like the idea of NV-configuring multiple streams on an
agent, based not only on filtering subsets of the data,
but also to support different encodings of the same notifications.

I like the idea of the netconf manager issuing a <start-notifications>
RPC and subscribing to a pre-configured stream,
or providing a "my-session-only" stream config that the agent will
toss when the session is closed.

This provides a transition path to XML-encoded syslog or snmp-notif
streams.  Legacy apps will subscribe to "syslog/RAW", and gradually,
new apps will have some reason to subscribe to "syslog/XML" or "snmp/XML"
formatted streams.

Andy

---

From: "Rainer Gerhards" <[email-masked]>
Date: Fri, 7 Jul 2006 20:14:52 +0200

> values of that string.  (I don't know exactly what the WG

Have you considered using the capability string as identifier for
eventClass? Granted, it may be a bit lengthy, but it clearly
identifies the event. That would also allow cross-checks (e.g. if
eventClass matches advertised capabilities).

I agree.

Rainer