Subject: [Syslog] Comments on draft-ietf-syslog-protocol-20

From: Sam Hartman <[email-masked]>
Date: Sun, 13 May 2007 21:15:58 -0400 (EDT)

Hi.  Thanks for the new draft.

There are a lot of changes in this version; it will definitely require
a new ietf last call.  I'd recommend that the WG evaluate the changes
and ask whether at this stage in the process they are actually
required.  Perhaps they are.

1) You cannot use originator and sender in the same layering diagram  if they are going to mean different things.  May I suggest calling the entity a transport sender/transport receiver.

2) I question whether all three of sender/originator/formatter are
   needed.  I recommend dropping formatter and perser, folding their
   behavior in somewhere else.

You are misusing the terms (as an example, the following diff suggests
that we care about the formatter's IP addresses not the originator's).
I can believe that the formatter chooses the IP address, but I don't
think it should choose one of its addresses if it is on a different
machine than the originator.  Misuse of terms from those defining the
terms suggests that you have too many terms.

+   Formatters SHOULD consistently use the same value in the HOSTNAME
+   field for as long as possible.  If the formatter uses IP addresses
+   inside hostname, the following rules apply: If the formatter is
+   multihomed, this value SHOULD be one of its actual IP addresses.  If
+   a formatter is running on a machine that has both statically and
+   dynamically assigned addresses, then that value SHOULD be from the
+   statically assigned addresses.  As an alternative, the formatter MAY
+   use the IP address of the interface that is used to send the message.

If you keep formatter and parser, please for each use of originator,
formatter, parser and collector explain why that is the correct term.

3) Why does this obselete 3164?  That document describes a ifferent
   protocol.

4) Why was a.8 removed?

_______________________________________________
Syslog mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/syslog

---

From: "David Harrington" <[email-masked]>
Date: Mon, 14 May 2007 08:32:49 -0400

Hi,

I agree with Sam that the WG should review the large changes made
since IETF last call.

I'll respond to these questions -- as a contributor. 
Let me start out by stating that I have no previous experience with
syslog before becoming co-chair. My background has been SNMP-based NMS
development. 

I came in as co-chair primarily to help get the documents finished and
through the process, because the WG had stalled and Chris didn't have
the time to spend on getting the WG through the process alone. 

my commnts inline.

> -----Original Message-----
> From: Sam Hartman [mailto:[email-masked]] 
> Sent: Sunday, May 13, 2007 9:16 PM
> To: [email-masked]
> Cc: [email-masked]
> Subject: [Syslog] Comments on draft-ietf-syslog-protocol-20
> 
> 
> 

If we are developing a standard, the documents should describe how the
system works well enough for those with no previous experience in the
protocol to be able to understand it from the documents. I have always
used the rule "IETF documents should be clear and unambiguous" as
various roles as chair and editor in the IETF. 

I found that protocol-17 failed that test. This became quite apparent
when there were debates about terminology related to the -tls- draft
and the -mib- draft. 

> 
> 1) You cannot use originator and sender in the same layering 
> diagram  if they are going to mean different things.  May I 
> suggest calling the entity a transport sender/transport receiver.
> 

As we tried to work on the docuemnts, especially the mib, it became
clear that different people have different ideas of what each term
means. In the mib it is necessary to model the abstract concepts of
how the system works.

The concepts of syslog seem to be that some entity, e.g. a device,
request that a syslog message be created and logged/sent on its
behalf. Implementation designs vary widely depending on a lot of
factors, but all seem to accept that some process-or-whatever has an
event occur that should be looged/sent and that some
process-or-whatever creates a message, and some process-or-whatever
sends the prepared message.

Since the WG has been able to separate the protocl form the transport
mappings, the process-or-whatever that craetes the message and th
eprocess-or-whatever that sends the message are separate abstract
concepts.

Why is this important? because the mib contains counters for things
like number-of-messages-sent; which portion of the system actually
knows how many messages were sent? The requesting process-or-whatever
knows ho wmany were requested; the message-formatting
process-or-whatever knows ho wmessages were prepared for sending, but
the transport sender is the only one that knows how many managed to
get through all the intermediate error-checking and were actually
sent. Only the transport-sender process has the information to make
available to the mib. If different implementations count this counter
from different points in the process, the results will not be
interoperable - an NMS will not be able to accurately compare the
values from two syslog daemons/processes/applications and be able to
compare apples-to-apples rather than comparing apples-to-oranges.

>From the managed node perspective, it might make perfect sense to use
the originator's IP address sometimes and the formatter's address
sometimes, but from the point of an operator or NMS, it can make all
the difference in the world what that value means vis-a-vis the
network. We need consistency across implementations.

If that value will be used for security purposes, I think it is even
more important that it be consistent about what it represents. Maybe
there should be two different places to represent IP address if it
means two different things.

Again, the terminology has been tremendously unclear and ambiguous.
This WG should make the concepts and terminology clear and
unambiguous.

> formatter, parser and collector explain why that is the correct
term.

Apparently, we still have not gotten the abstract concepts and the
terminology to be clear and unambiguous, since you could not
understand it.
> 3) Why does this obselete 3164?  That document describes a ifferent

RFC3164 describes a variety of implementations, and during our work,
we found that those descriptions only captured the tip of the iceberg.
As a result, that informational document is inaccurate and misleading,
and should be considered obsolete. We don't declare the
implementations described obsolete, or the non-IETF defacto standard
to be obsolete; we declare the informational RFC to be obsolete.
Historic could also work, if the IESG prefers that choice.

.
> 4) Why was a.8 removed?

a.8 and the section on PROCID used lots of words, but at the end of
going through all those words, what was said was pretty
straightforward - there was no agreement on the syntax or meaning of
PROCID. The only agreement appeared to be that if the value changed,
that was significant, and some implementations used these fields in a
similar way. We summarized the discussion from a.8 (in -17-) and added
it to 6.2.6 (in -20-).

David Harrington
[email-masked] 
[email-masked]
[email-masked]

---

From: "tom.petch" <[email-masked]>
Date: Fri, 18 May 2007 17:53:11 +0200

Not sure where this draft is heading, but as a WG member, comments <inline>

Tom Petch

----- Original Message -----
From: "David Harrington" <[email-masked]>
To: "'Sam Hartman'" <[email-masked]>; <[email-masked]>
Sent: Monday, May 14, 2007 2:32 PM
Subject: RE: [Syslog] Comments on draft-ietf-syslog-protocol-20
<snip>

> > suggest calling the entity a transport sender/transport receiver.
I do not understand this comment. Whilst in lay English, originator and sender
might be synonyms, I see them here redefined as functions in layers application
and transport respectively, which, while perhaps not what I would do, seems ok.
Qualifying sender and receiver with transport is also ok with me.

> compare apples-to-apples rather than comparing apples-to-oranges.

I am with Sam on this.  Reading -syslog-protocol, I reverse engineered what
formatter and parser meant and yes, there are used in a specific and consistent
sense, but I did not think that they improved clarity or understanding of
syslog-protocol.  I agree where syslog daemons are built in this manner, then
this concept would allow a finer grained MIB module but I am not sure that the
gain in precision justifies the gain in complexity and would see it as adequate
to have formatter and parser folded into the layer above.

> terminology to be clear and unambiguous, since you could not

This is the sort of paragraph I reverse engineered in order to work out what a
formatter and parser was but the HOSTNAME that I care about is where the syslog
message started, not where it got formattered should that be on a different
stack with a different HOSTNAME, so again, I am happy to see formatter and
parser go.

> Historic could also work, if the IESG prefers that choice.
point now resolved I understand
> similar way. We summarized the discussion from a.8 (in -17-) and added

No particular thoughts either way.

---

From: Chris Lonvick <[email-masked]>
Date: Tue, 22 May 2007 10:22:37 -0700 (PDT)

Hi All,

What I'm seeing is that our effort to add granularity for mib accounting 
has made the document less clear.  My apologies for that.

Does the following make more sense:

   +---------------------+    +---------------------+
   |  content            |    |  content            |
   |---------------------|    |---------------------|
   |  syslog application |    |  syslog application | (originator,
   |                     |    |                     |  collector, relay)
   |---------------------|    |---------------------|
   |  syslog transport   |    |  syslog transport   | (transport sender,
   |                     |    |                     | (transport receiver)
   +---------------------+    +---------------------+
             ^                          ^
             |                          |
              --------------------------

In this, the "content" will be developed by some application and handed to 
syslogd (using *nix as an example device).  syslogd will format the 
message adding in the PRI, timestamp, etc., and will hand it to the 
transport.
- For udp transport, the "transport sender" will encapsulte it within udp
   and put it onto the wire.
- For the case of tls, the "transport sender" will establish a new, or use
   an existing session with the "transport receiver".

For discrepancies (if any) between the IP address of the "originator" and 
the "transport sender", the originator can use the [origin ip=...] SDE 
(Section 7.2).

If this makes sense, then the mib counters can be:
- the number of messages sent and received by the "syslog application"
   (syslogd)
- the number of messages sent and received by the "transport sender" and
   "transport receiver".
The tricky part here is that the counters of the "transport sender" and 
"transport receiver" are not going to be useful to counting messages that 
are relayed.  Only the counters of the "syslog application" are going to 
be useful for that.  To deal with that, I'll propose that that a table be 
set up to associate the messages sent to each relay destination.

As an example from syslog.conf:

                kern.crit                    @loghost
                kern.info                    @loghost2.example.com

The relay destinations will have to be enumerated.
    get "numOfRelayDests" would return "2"
    get "relayDest(1)" would return "loghost"
    get "relayDest(2)" would return "loghost2.example.com"

What is to be sent to those destinations would have to be quantified.
    get "priOfRelayDest(1)" would return "2" (from kern.crit as the filter)
    get "priOfRelayDest(2)" would return "6" (or "kern.info")

When the device receives a "<2>..." syslog message (PRI=2, kern.crit), it 
will relay it to the two relay destinations.
Then
    syslogOperationsMsgsReceived will be incremented by 1
    syslogOperationsMsgsRelayed(0) will be incremented by 2
       (the message went to two destinations)
    syslogOperationsMsgsRelayed(1) will be incremented by 1
       (it sent one copy to "relayDest(1)" which is loghost)
    syslogOperationsMsgsRelayed(2) will be incremented by 1
       (it sent another to ""relayDest(2)")
    syslogOperationsMsgsTransmitted will be incremented by 2
       (it transmitted both)

Also, on loghost, syslogOperationsMsgsReceived will be incremented by 1 
and on loghost2.example.com syslogOperationsMsgsReceived will also be 
incremented by 1.

This gives an administrator a way to balance out messages sent and 
received.
- If our device shows 3 messages relayed to loghost, and loghost shows 3
   messages received, then we have a balance, even if MsgsTransmitted from
   our device is 482.
- If our device shows 3 messages relayed but loghost shows 2 messages
   received, then we might have a discard on our device, or the message may
   have been dropped by some intermediary.
- If our device shows 3 messages relayed but loghost shows 46 receieved
   then we likely have another device sending messages to loghost.

To be clear on this, the counters for "transport sender" and "transport 
receiver" will NOT be associated with a peer - they will just count the 
number of messages sent and received.  It will be up to the counters 
associated with the "syslog application" to associate the messages with 
peers so that the count of messages relayed will have some meaning.

Does this make sense?  As David said, we're not doing our job unless we're 
clear on the concepts, terminology and have a way to manage the devices.

Thanks,
Chris

---

From: Sam Hartman <[email-masked]>
Date: Tue, 22 May 2007 15:20:14 -0400

>>>>> "Chris" == Chris Lonvick <[email-masked]> writes:

Your layer diagram makes sense.
    Chris> For discrepancies (if any) between the IP address of the
    Chris> "originator" and the "transport sender", the originator can
    Chris> use the [origin ip=...]  SDE (Section 7.2).

I'd assume that all data in the content notially is written relative
to the originator, at least notionally.

Now, sometimes, there will be complicated situations for example when
the originator is split across boxes (imagine a system where syslog(2)
was an RPC call rather than a library function that writes to a local
socket) or where you are translating from one logging protocol to
another (say a snmp implementation that generates a syslog message
from a trap).

---

From: Chris Lonvick <[email-masked]>
Date: Fri, 25 May 2007 07:18:26 -0700 (PDT)

Hi,

> to the originator, at least notionally.

Yes.

> another (say a snmp implementation that generates a syslog message

I agree that's something that can happen.  I don't think that anything 
should be written into the normative sections of the -protocol document to 
address that as it has nothing to do with the protocol on the wire. 
Perhaps a note in the security considerations section?

With that:

Rainer - Please update the document to reflect the new layering diagram 
and definitions.

Glenn - Please use the same layering diagram in -device-mib and use the 
counters as suggested.

All - Have a great weekend.  :-)

Thanks,
Chris

---

From: "Glenn M. Keeni" <[email-masked]>
Date: Sun, 27 May 2007 20:23:58 +0900

Chris,
I think that there are two separate issues that we are talking of here;

   - the layered model for syslog
   - the objects which we monitor

I will send my comments on the layered model in a separate mail.

As far as the Objects [MIB counters]  are concerned I am reading 2
points
(1) monitoring the relay activity.  We want to have
     - a list of the relays
     - for every relay
         o the priority(ies) of the message that will be relayed to
           this relay
         o the number of messages transmitted to this relay
           (messagesTransmittedToRelay)
     - etc.

(2) we do not need to monitor message received at the message-source
    level of granularity.

The points are well taken. I could certainly use the features
built using (1). Regarding (2)- monitoring messages at the
message-source level of granularity is not difficult to implement.
But there may be practical problems as there may be there may be a
very large number of message sources - tracking the counters for
each of the sources will be expensive etc. So I will agree with (2).
I will add the new table definitions for (1) in the next version of
the draft.

Glenn

> In this, the "content" will be developed by some application and handed 
> to syslogd (using *nix as an example device).  syslogd will format the 
> For discrepancies (if any) between the IP address of the "originator" 
> and the "transport sender", the originator can use the [origin ip=...] 
> SDE (Section 7.2).
> 
> 
> "transport receiver" are not going to be useful to counting messages 
> that are relayed.  Only the counters of the "syslog application" are 
> going to be useful for that.  To deal with that, I'll propose that that 
> a table be set up to associate the messages sent to each relay destination.
> 
> When the device receives a "<2>..." syslog message (PRI=2, kern.crit), 
> it will relay it to the two relay destinations.
> Then
> Does this make sense?  As David said, we're not doing our job unless 
> we're clear on the concepts, terminology and have a way to manage the 
> devices.
> 
> Thanks,
> Chris
> 
> 
>

---

From: "Glenn M. Keeni" <[email-masked]>
Date: Mon, 28 May 2007 14:17:58 +0900

Chris,
     Sorry, but I have missed the point of the "layer diagram".
It is not clear to me what the layers and the layering signify.
There are 2 problems-
  (a) It seems odd that "content", an inert entity, sits on top
      of "application", an active entity. Is this OK ? In my
      understanding, in the layer model, we have layered
      entities each interact with the adjacent layer. Should we
      call it something else ?

  (b) It is not clear how the "layer diagram" will help in the
      understanding of the mib counters. Correct me if I am
      wrong - there is no way the
          transportSenderCounter and
          syslogApplicationSenderCounter will be different.
      Also,
          transportReceiverCounter and
          syslogApplicationReceiverCounter
      will always be the same. So we do not need an extra
      set of "transport" counters. Then the transport layer in
      the diagram is redundant.
      Having said that, I must add that if we are interested in
      the transport-level granularity, then and only then, the
      transport layer is significant, as far as the mib counters
      are concerned. Is that what we are looking for ?

Glenn

---

From: "David Harrington" <[email-masked]>
Date: Tue, 29 May 2007 11:59:33 -0400

Hi,

The IETF Management Framework makes a distcinction between management
information and protocols used to carry information. The "content"
layer provides an interface to the underlying instrumentation. A
"content layer" concept is used in the Netconf architecture to
represent this, and we carried that into this document to show a
similar separation of the management information from the protocol.

I believe there can be differences between the number of messages that
"applications" request be sent and the number that is actually sent,
because there can be error conditions that prevent a message from
actually being sent. For syslog deployemnts that send the same message
to multiple addresses and to a file for storage, do you count the
number of "messages" or the number of transport endpoints? Do you
count the file-storage destinations as having been snet, or only the
messages that have been sent across a transport protocol? In addition,
an implementation might contain multiple "applications" that use the
same transport sending library. So the counting of messages can be
different depending on where you count them.

What I am interested in seeing is counters that are implemented in a
relatively consistent manner across implementations, so an SNMP NMS
can compare/contrast the values of the counters from different
hosts/servers/applications/whatever. If one implementer increments the
only "sent" counter when a request is made, and another increments the
"sent" counter when each outgoing message is put on the wire, this can
result in counters that are comparing apples to oranges. The
conceptual layering helps when trying to write DESCRIPTION clauses
that are consistent across implementations. Describing the conceptual
layering keeps the DESCRIPTION at an abstract concept level to avoid
having the standard dictate specific implementation decisions.

We need to be able to specify where in the conceptual process a
counter should be incremented, without dictating implementation
software designs, in order to minimize ambiguity in the counter
specification to eliminate inconsistent implementer-interpretations
that make the counters non-interoperable across vendors.

dbh

> From: Glenn M. Keeni [mailto:[email-masked]] 
> Sent: Monday, May 28, 2007 1:18 AM
> To: Chris Lonvick
> Cc: syslog; 'Sam Hartman'
> Subject: Re: Layer diagram & mib counters - was:Re: [Syslog] 
> Commentson draft-ietf-syslog-protocol-20
> 
> Chris,
> >   |                     |    |                     |  
> collector, relay)

> >> Not sure where this draft is heading, but as a WG member,
comments 
> >> <inline>