Subject: [Syslog] FW: draft-shafer-netconf-syslog-00.txt

From: "Rainer Gerhards" <[email-masked]>
Date: Sat, 17 Jun 2006 11:39:14 +0200

Hi all,

This posting from the netconf WG seems highly relevant. The page itself
uses some crumbersome challenge system, so I could not look at the
actual content. I will do so when the draft is posted on the IETF site
and recommend that other WG members do so, too.

Rainer

> -----Original Message-----
> From: [email-masked] 
> [mailto:[email-masked]] On Behalf Of Phil Shafer
> Sent: Saturday, June 17, 2006 6:07 AM
> To: netconf
> Subject: draft-shafer-netconf-syslog-00.txt
> 
> I've finally got an initial version of the draft for a netconf
> capability that makes syslog data available over long-lived RPCs.
> I just sent it off to the RFC Editor, so it should show up on
> ietf.org sometime this weekend, but an advance copy is available.
> 
>
http://professional.juniper.net/phil/ietf/draft-shafer-netconf-syslog-00
.txt
> 
> Thanks,
>  Phil
> 
> --
> to unsubscribe send a message to [email-masked] with
> the word 'unsubscribe' in a single line as the message text body.
> archive: <http://ops.ietf.org/lists/netconf/>
> 

_______________________________________________
Syslog mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/syslog

---

From: Phil Shafer <[email-masked]>
Date: Sat, 17 Jun 2006 09:55:50 -0400

>and recommend that other WG members do so, too.

It's a "don't sue Juniper because of what Phil wrote" disclaimer ;^)

The document is attached.

Thanks,
 Phil

Network Working Group                                          P. Shafer
Internet-Draft                                          Juniper Networks
Expires: December 18, 2006                                 June 16, 2006

                    A SYSLOG Capability for NETCONF
                     draft-shafer-netconf-syslog-00

Status of this Memo

   This document is an Internet-Draft and is NOT offered in accordance
   with Section 10 of RFC 2026, and the author does not provide the IETF
   with any rights other than to publish as an Internet-Draft.

   Internet-Drafts are working documents of the Internet Engineering
   Task Force (IETF), its areas, and its working groups.  Note that
   other groups may also distribute working documents as Internet-
   Drafts.

   Internet-Drafts are draft documents valid for a maximum of six months
   and may be updated, replaced, or obsoleted by other documents at any
   time.  It is inappropriate to use Internet-Drafts as reference
   material or to cite them other than as "work in progress."

   The list of current Internet-Drafts can be accessed at
   http://www.ietf.org/ietf/1id-abstracts.txt.

   The list of Internet-Draft Shadow Directories can be accessed at
   http://www.ietf.org/shadow.html.

   This Internet-Draft will expire on December 18, 2006.

Shafer                  Expires December 18, 2006               [Page 1]

Internet-Draft               netconf-syslog                    June 2006

Table of Contents

   1.  Abstract . . . . . . . . . . . . . . . . . . . . . . . . . . .  3
   2.  Event Notifications  . . . . . . . . . . . . . . . . . . . . .  3
     2.1.  Overview . . . . . . . . . . . . . . . . . . . . . . . . .  4
   3.  #syslog  . . . . . . . . . . . . . . . . . . . . . . . . . . .  6
     3.1.  get-syslog-streams . . . . . . . . . . . . . . . . . . . .  6
       3.1.1.  Parameters . . . . . . . . . . . . . . . . . . . . . .  6
       3.1.2.  Response . . . . . . . . . . . . . . . . . . . . . . .  6
     3.2.  get-syslog-events  . . . . . . . . . . . . . . . . . . . .  9
       3.2.1.  Parameters . . . . . . . . . . . . . . . . . . . . . . 10
       3.2.2.  Response . . . . . . . . . . . . . . . . . . . . . . . 13
   4.  Acknowledgments  . . . . . . . . . . . . . . . . . . . . . . . 14
   5.  Normative References . . . . . . . . . . . . . . . . . . . . . 14
   Author's Address . . . . . . . . . . . . . . . . . . . . . . . . . 15

Shafer                  Expires December 18, 2006               [Page 2]

Internet-Draft               netconf-syslog                    June 2006

1.  Abstract

   SYSLOG is the most prevalent mechanism for delivering event
   notification data off networking devices.  NETCONF is intended to
   provide a standard mechanism for performing configuration operations
   on networking devices.  The two areas will overlap, giving rise to a
   need for receiving notification data over NETCONF.

   This document presents a simple mechanism for accessing SYSLOG data
   over NETCONF, implemented using the existing NETCONF RPC
   infrastructure.

2.  Event Notifications

   Management of a network depends on timely access to information about
   the network, including status, alarms, faults, errors, and failures.
   As events of interest occur in the network, devices detect them and
   emit event notifications.  The notifications are typically emitted as
   SNMP [1] traps or SYSLOG [2] messages.  Both of these notification
   mechanisms require the destinations for notifications to be pre-
   configured, which represents an impediment for applications which
   require a more dynamic way to receive this data.

   This document defines a NETCONF [3] capability for retrieving SYSLOG
   data.  Network management applications will have access to the rich
   volume of information currently instrumented in device software over
   the secure, private, connection-oriented, and reliable communication
   path offered by NETCONF.  An application can see important
   notifications about the hardware which the user is attempting to
   configure, or see critical errors to which the user would otherwise
   be blind.

   NETCONF gives access to a device's information, both configuration
   and status, using a simple RPC-based protocol.  It allows the
   definition of "capabilities", which define a contract of behavior and
   a set of RPCs used to implement that behavior.  If a device reports a
   capability string in its initial <hello> message, it is indicating
   its ability to perform those RPCs and to behave according to the
   capability definition.  These capabilities are the extension
   mechanism for NETCONF.

   NETCONF RPCs can be defined to return any data.  The volume of data
   and the duration of the RPC are unlimited.  Specifics about the
   nature of the RPC and the data returned are detailed in the
   capability definition.

   SYSLOG is an event notification mechanism widely used among network

Shafer                  Expires December 18, 2006               [Page 3]

Internet-Draft               netconf-syslog                    June 2006

   service providers.  The breadth and depth of content is unmatched.
   Device vendors have existing embedded code for generating appropriate
   SYSLOG messages, and have documented the content and cause of these
   notifications.  The integration of SYSLOG with management application
   seems natural.

   The SYSLOG protocol lacks security and reliability.  But accessing
   SYSLOG data over NETCONF gives the richness of SYSLOG data over a
   secure, connection-oriented communications path, giving the
   application the information it needs about the network with a minimal
   impact to the device.

2.1.  Overview

   SYSLOG event notifications are made available using NETCONF RPCs.
   The NETCONF client issues an RPC with a set of parameters detailing
   the information desired.  The NETCONF server replies with the
   appropriate data.

     +----+
     | c1 |---+
     +----+   |    +------+                        (off-box)
     +----+   |    |      |
     | c2 |   +--->|syslog|          +-------+     +-------+
     +----+   |    |daemon|--------->|netconf|<--->|netconf|
      ...     |    |      |         >|server |     |client |
     System   |    +------+        / +-------+     +-------+
    Components|          \        /
      ...     |           v      /
     +----+   |         (----------)
     | cn |---+         (historical)
     +----+             (  events  )  (e.g. log files)
                        (----------)

   The model requires that the device support a set of SYSLOG "event
   streams", which consist of a series of events which match the set of
   filters given in the streams definition.  This allows the device to
   control what information is available to the client.  The stream
   definition may be provided by the device vendor or be part of the
   device's configuration.  Clients request data from one of these
   streams, and can supply additional filtering criteria to further
   constrain the data retrieved.

   Event notifications are generated by system components and are passed
   to a central syslog component, which inspects each event and matches
   the event against the set of stream definitions.  When a match
   occurs, the notification is considered a member of that event stream.
   A notification may be part of multiple event streams.

Shafer                  Expires December 18, 2006               [Page 4]

Internet-Draft               netconf-syslog                    June 2006

   In addition to the filters defined on an event stream, the client may
   request filtering based on syslog header fields, names and value of
   structured data parameters within the notification, or message
   content.  The client may use date and time ranges to trim recorded
   data, as needed, and may limit the volume of data in the response.

   When an event notification matches both the filters for a given
   stream and the filters for a particular outstanding RPC, it is
   encoded by the NETCONF server and forwarded to the NETCONF client.
   The data can be transported as simple SYSLOG messages or using the
   extended text format defined in the SYSLOG Working Group's Internet-
   Draft on the SYSLOG protocol [4].  Each notification is wrapped in an
   XML element.

   To receive notifications, a NETCONF client makes a <get-syslog-
   events> RPC, supplying the name of the stream and any filtering to be
   performed in addition to any filters from the stream definition.  The
   NETCONF server responds with a set of matching notifications.

   The client may request a continuous feed of SYSLOG events, allowing
   it to receive them as they are generated.  During periods when no
   messages are being generated, the server will not send data, but will
   not close the NETCONF rpc-reply, giving a mechanism for a long-lived
   RPC with an open-ended data stream.

   The client may include termination criteria in the RPC, given a
   timestamp to stop, or a count of how many notifications to include in
   the reply.  If no termination criteria are given, the RPC continues
   until the NETCONF session is terminated.

   The device may also retain event data for a stream, allowing clients
   to retrieve historic events.  These events may be locally recorded,
   using locally available resources such as memory or disk.  The volume
   of recorded events may be limited by these resources, allowing the
   device to keep a certain volume of events in a log file, or the fixed
   number of records in memory, or perhaps a set number of days worth of
   events.

   By defining the presence of historical data in the event stream
   definition, the control of local resource utilization is placed in
   the hands of the device.  Historical data, if supported, is optional
   for any particular event stream.

   If historical events are available, the NETCONF server can include
   that data in the response.  The response can include historical data,
   new data, or a combination.

Shafer                  Expires December 18, 2006               [Page 5]

Internet-Draft               netconf-syslog                    June 2006

3.  #syslog

   The #syslog capability will be defined with the capability string as
   follows:

       http://ietf.org/netconf/syslog/1.0

   Operations and content defined in this capability definition should
   appear in a namespace with this URI.

   This capability defines the following set of RPCs.

   If this capability is accepted by the NETCONF working group, the
   namespace should be changed to follow the URN format.

3.1.  get-syslog-streams

   The <get-syslog-streams> RPC gives the client the ability to discover
   the current set of SYSLOG streams the device supports.  A stream is a
   series of SYSLOG events that match a particular set of criteria.  The
   stream may include historic events, which may be recorded locally.

3.1.1.  Parameters

   None.

3.1.1.1.  Example

   The following RPC requests the list of streams supported by the
   NETCONF server.

       <rpc xmlns="urn:ietf:params:xml:ns:netconf:base:1.0">
         <get-syslog-streams
                   xmlns="http://ietf.org/netconf/syslog/1.0"/>
       </rpc>

3.1.2.  Response

   The reply consists of the top-level <syslog-streams> element
   containing a set of <stream> elements.  The <stream> element contains
   the description of an individual stream of SYSLOG data.  This element
   can contain the following elements:

   <name> gives the unique name of the stream.  This field is mandatory.

   <unreadable/> if the current NETCONF session lacks sufficient
   permission to read this stream.  An implementation is free to omit
   unreadable streams from the list of streams.

Shafer                  Expires December 18, 2006               [Page 6]

Internet-Draft               netconf-syslog                    June 2006

   <recording/> if the stream is recording events locally

   <format> gives the format of the data available from the stream, and
   should be one of:

   o  -- "traditional" for traditional RFC 3164 [2] SYSLOG data

   o  -- "structured-data" for the Structured Data Parameter format
      defined in SYSLOG Working Groups's Internet-Draft [4].

   <priority> contains two attributes used to match the priority field
   of the syslog notifications.  The "facility" attribute describes the
   part of the system generating the message, and many are not
   appropriate to routing devices.  The "level" attribute describes the
   severity of the event.  If either attribute is missing, the priority
   matches all notifications, similar to the "any" match in most SYSLOG
   implementations.

   The full set of facilities is specified by SYSLOG.

   o  "authorization" -- The authorization system

   o  "authpriv" -- The same as "auth", but logged to a file readable
      only by selected individuals

   o  "console" -- Messages written to /dev/console by the kernel
      console output driver

   o  "cron" -- The unix cron daemon

   o  "daemon" -- System daemons that are not provided for explicitly by
      other facilities (This is the most common facility for routing
      devices)

   o  "ftp" -- The file transfer protocol daemon

   o  "kernel" -- Messages generated by the kernel

   o  "lpr" -- The line printer spooling system

   o  "mail" -- The mail system

   o  "news" -- The network news system

   o  "security" -- Security subsystems

   o  "syslog" -- Messages generated internally by the syslog daemon

Shafer                  Expires December 18, 2006               [Page 7]

Internet-Draft               netconf-syslog                    June 2006

   o  "user" -- Messages generated by user processes

   o  "uucp" -- The uucp system

   o  "local0", "local1", .. "local7" -- Reserved for local use

   The full set of levels is specified by SYSLOG.  Any event containing
   a level equal or greater than this value will be carried in the
   stream.  The following is an ordered list of levels, sorted highest
   to lowest.

   o  -" emergency" -- A panic condition

   o  "alert" -- A condition that should be corrected immediately

   o  "critical" -- Critical conditions, e.g., hard device errors

   o  "error" -- Errors

   o  "warning" -- Warning messages

   o  "notice" -- Conditions that are not error conditions, but should
      possibly be handled specially

   o  "info" -- Informational messages

   o  "debug" -- Messages that contain information normally of use only
      when debugging a program

   Multiple <priority> elements can be used to match multiple
   conditions.  An event notification matches if it matches any of the
   conditions.

   <text-pattern> contains a regular expression (as defined by IEEE Std
   1003.2/POSIX.2 [5]) which the message of the SYSLOG must match.  Any
   event whose message string matches this expression is carried in the
   stream.

   <process> contains the name of a system process on which to match.

   <event> contains a regular expression which the name of an event must
   match.

   <parameter> contains an expression of the form 'name=value', where
   the name is an SD-Param parameter name and value is a regular
   expression which the parameter value from the event must match.  If
   multiple <parameter> elements appear, the event must match all
   name=value pairs to be carried in the stream.  If an event does not

Shafer                  Expires December 18, 2006               [Page 8]

Internet-Draft               netconf-syslog                    June 2006

   contain the named parameter, it is not a match.

   The parameter name can be prefix by an optional SD-ID and a colon
   (":", ABNF %d58) to distinguish the naming authority of the parameter
   name.  This allows a stream to filter in conditions when parameter
   names are duplicated with different SD-IDs.

3.1.2.1.  Example

   The following response lists three streams, one a typical log file
   such as /var/log/messages in unix, one a debug stream, and one a
   structured data stream matching on a specific set of criteria.

       <rpc-reply xmlns="urn:ietf:params:xml:ns:netconf:base:1.0">
         <syslog-streams xmlns="http://ietf.org/netconf/syslog/1.0">
           <stream>
             <name>messages</name>
             <recording/>
             <format>traditional</format>
             <priority level="notice"/>
             <priority facility="kern" level="debug"/>
           </stream>
           <stream>
             <name>debug</name>
             <format>traditional</format>
             <priority level="debug"/>
           </stream>
           <stream>
             <name>today's hot routing issue</name>
             <recording/>
             <format>structured-data</format>
             <priority facility="daemon" level="info"/>
             <process>rpd</process>
             <text-pattern>.*peer.*</text-pattern>
             <parameter>peer=10.1.2.3</parameter>
           </stream>
         </syslog-streams>
       </rpc-reply>

3.2.  get-syslog-events

   The <get-syslog-events> RPC gives access to the events inside a
   stream.  When a NETCONF server receives this RPC, it will respond
   with a set of SYSLOG events that match the criteria in the request.
   The RPC may be long-lived, in that it does not normally complete in a
   fixed time frame, but continues to forward matching events from the
   device until the session is terminated.  This allows the client to
   view a continual stream of events coming from the box, similar to a

Shafer                  Expires December 18, 2006               [Page 9]

Internet-Draft               netconf-syslog                    June 2006

   traditional SYSLOG event stream, but within a secure, connection-
   oriented NETCONF session.

3.2.1.  Parameters

   <stream> contains the name of the stream from which syslog
   notifications are matched, as given in the <name> element from the
   reply from <get-syslog-streams>.  The <stream> parameter is
   mandatory.

       <stream>messages</stream>

   <count> gives the maximum number of events to include in the
   response, allowing the client to limit the time or memory consumed by
   the reply.  If this parameter is provided, the NETCONF server will
   close the RPC response when the given number of events have been
   matched.

       <count>100</count>

   <recorded/> instructs the device to match only recorded events, and
   to complete the RPC as soon as the last recorded event is inspected.
   If this parameter is given, the NETCONF server will close the RPC
   response when no further recorded events are available.

       <recorded/>

   <start-time> gives the timestamp for the earliest event to be
   inspected.  Events generated before this time are not matched.  The
   format of the time in restricted in accordance with Section 6.2.3 of
   the SYSLOG [4] Internet-Draft.

       <start-time>2006-04-01T23:20:50.52Z</start-time>

   <stop-time> gives the timestamp for the last event to be inspected.
   Events generated after this time are not matched.  If this parameter
   is provided, the NETCONF server will close the response when the
   given time passes.

       <stop-time>2006-04-01T23:30Z</stop-time>

   If the <start-time> is given, the NETCONF server responds with events
   which have occurred since the given time and match the request's
   filter criteria.  If historical data is available and matches the
   time constraint, it is included in the response.  The NETCONF server
   will then respond the new events as they occur.

   If the <count> and <recorded/> parameters are given but the <start-

Shafer                  Expires December 18, 2006              [Page 10]

Internet-Draft               netconf-syslog                    June 2006

   time> is not, the NETCONF server responds with the most recent events
   which match the request's filter criteria.

   If neither <start-time> or <count> are included in the request, the
   NETCONF server responds with new events as they occur.  Historical
   data is not used.

   The <get-syslog-events> RPC reuses a number of elements defined in
   the response from the <get-syslog-streams> RPC.

   <priority> contains the facility and level attributes to match
   against the SYSLOG notification.  Multiple <priority> elements can be
   used to match multiple conditions.

       <priority facility="kern" level="warning"/>

   <text-pattern> contains a regular expression which the message of the
   SYSLOG must match.  Any event whose message string matches this
   expression is considered a match.

       <text-pattern>ALARM CLEAR:.*fxp0</text-pattern>

   <process> contains the name of a system process on which to match.

       <process>rpd</process>

   <event> contains a regular expression which the name of an event must
   match.

       <event>CHASSIS.*DOWN</event>

   <parameter> contains an expression of the form 'name=value', where
   the name is an SD-Param parameter name and value is a regular
   expression which the parameter value from the event must match.  If
   multiple <parameter> elements appear, the event must match all
   name=value pairs to be considered a match.  If an event does not
   contain the named parameter, it is not a match.

       <parameter>peer=10.*</parameter>

   The parameter name can be prefix by an optional SD-ID and a colon
   (":", ABNF %d58) to distinguish the naming authority of the parameter
   name.  This allows a stream to filter in conditions when parameter
   names are duplicated with different SD-IDs.

       <parameter>[email-masked]:username=regress</parameter>

   The <priority>, <event>, and <parameter> parameters are only

Shafer                  Expires December 18, 2006              [Page 11]

Internet-Draft               netconf-syslog                    June 2006

   supported if the requested stream is in the "structured-data" format.
   They are ignored for streams using other formats.

3.2.1.1.  Closing the Response

   The RPC response will be closed when one of the following conditions
   are met:

   o  the <count> parameter was given and <count> events have been
      included in the response, or

   o  the <stop-time> parameter was given and the specified time has
      past, or

   o  the <recorded> parameter was given and no further recorded events
      are available

   If one of these parameters was given, the NETCONF server will
   continue to emit events until the appropriate condition is fulfilled.
   If more than one of these parameters are provided, the RPC will
   complete when any condition is satisfied.

   If none of there parameters was given, the NETCONF server will
   continue to emit events until the session is terminated, providing a
   mechanism for getting ongoing notification data from the server.
   When an event occurs on the device, the device will forward the event
   notification to any outstanding RPC which the event matches.

3.2.1.2.  Examples

     <rpc xmlns="urn:ietf:params:xml:ns:netconf:base:1.0">
       <get-syslog-events xmlns="http://ietf.org/netconf/syslog/1.0"/>
           <stream>messages</stream>
           <priority facility="daemon" level="emergency"/>
           <text-pattern>^User 'r.*' logout$</text-pattern>
           <process>mgd</process>
           <event>UI_CHILD_START</event>
           <parameter>username=regress</parameter>
           <parameter>input=configure</parameter>
           <parameter>authentication-level=super-user</parameter>
           <parameter>signal-name=Broken pipe</parameter>
           <start-time>2006-04-01T23:20:50.52Z</start-time>
           <count>100</count>
       </get-syslog-events>
     </rpc>

Shafer                  Expires December 18, 2006              [Page 12]

Internet-Draft               netconf-syslog                    June 2006

3.2.2.  Response

   The reply consists of a top-level element <syslog-events> containing
   a stream of SYSLOG events.  These events are encoded in text strings
   according to the rules contained in RFC3164 and the SYSLOG Internet-
   Draft.  Each event from a stream that are using the "traditional"
   format is contained in a <syslog> element.  If the stream is using
   the "structured-data" format, each event is contained in a <data>
   element.

3.2.2.1.  Examples

   (Newlines and whitespace were inserted in the text data to make the
   output in this section more readable and to fit within the IETF
   document formatting guidelines.)

   The following example contains three events in "traditional" format.

       <rpc-reply xmlns="urn:ietf:params:xml:ns:netconf:base:1.0">
         <syslog-events xmlns="http://ietf.org/netconf/syslog/1.0"/>
           <syslog>
             Jun 11 21:34:52  dent chassisd[2971]:
               CHASSISD_IFDEV_DETACH_ALL_PSEUDO:
                 ifdev_detach(pseudo devices: all)
           </syslog>
           <syslog>
             Jun 11 21:35:19  dent chassisd[2971]:
               CHASSISD_FRU_EVENT: fpc_m40_recv_restart:
                 restarted FPC 0
           </syslog>
           <syslog>
             Jun 11 21:35:19  dent chassisd[2971]:
               CHASSISD_FRU_EVENT: fpc_m40_recv_restart:
                 restarted FPC 1
           </syslog>
         </syslog-events>
       </rpc-reply>

   The following example contains the same events recorded in
   "structured-data" format.

Shafer                  Expires December 18, 2006              [Page 13]

Internet-Draft               netconf-syslog                    June 2006

       <rpc-reply xmlns="urn:ietf:params:xml:ns:netconf:base:1.0">
         <syslog-events xmlns="http://ietf.org/netconf/syslog/1.0"/>
           <data>
             1 2006-06-14T08:29:14.397+05:30 kitkat mgd 3993
               UI_CHILD_START
                 [[email-masked] command="/sbin/ifinfo"]
                   Starting child '/sbin/ifinfo'
           </data>
           <data>
             1 2006-06-14T08:29:14.417+05:30 kitkat mgd 3993
               UI_CHILD_STATUS
                 [[email-masked] command="/sbin/ifinfo"
                   process-id="3996" status="0"]
                     Cleanup child '/sbin/ifinfo', PID 3996, status 0
           </data>
         </syslog-events>
       </rpc-reply>

4.  Acknowledgments

   The author wishes to thank for following people for their most
   excellent review of this document and the numerous additions,
   suggestions, and refinements they have made: Pallavi Mahajan, Andy
   Bierman, Rob Enns, Sri Ram Sankar, Kent Watsen, and Reid Wilson.

   Comments are solicited and should be addressed to the NETCONF Working
   Group's mailing list <[email-masked]> and/or the author.

5.  Normative References

   [1]  Case, J., Fedor, M., Schoffstall, M., and J. Davin, "Simple
        Network Management Protocol (SNMP)", STD 15, RFC 1157, May 1990.

   [2]  Lonvick, C., "The BSD Syslog Protocol", RFC 3164, August 2001.

   [3]  Enns, R., "NETCONF Configuration Protocol",
        draft-ietf-netconf-prot-07 (work in progress), June 2005.

   [4]  Gerhards, R., "The syslog Protocol",
        draft-ietf-syslog-protocol-14 (work in progress), July 2005.

   [5]  Institute of Electrical and Electronics Engineers, "Information
        Technology - Portable Operating System Interface (POSIX) - Part
        2: Shell and Utilities (Vol. 1)", IEEE Standard 1003.2, 1992.

Shafer                  Expires December 18, 2006              [Page 14]

Internet-Draft               netconf-syslog                    June 2006

Author's Address

   Phil Shafer
   Juniper Networks
   940 Main Campus Drive
   Raleigh, NC  27606
   US

   Email: [email-masked]

Shafer                  Expires December 18, 2006              [Page 15]