Subject: [Syslog] Syslog RFC feedback

From: Raffael Marty <[email-masked]>
Date: Thu, 18 Dec 2008 18:05:04 -0800

I am really late to the game here. I know. I just wanted to share some  
feedback that I had about the current state of the syslog document:

- Let me say this first: I really like some of the changes that have  
been incorporated.
- Syslog message facility: Why do we still keep this? The only reason  
that I see people using the facility is to filter messages. There are  
better ways to do that. Some of the pre-assigned groups are fairly  
arbitrary and not even really implemented in most OSs. UUCP subsystem?  
Who is still using that? I guess the reason for keeping it is  
backwards compatibility? If possible, I would really like this to be  
gone.
- Priority calculation. The whole priority field is funky. The  
priority does not really have any meaning. The order does not imply  
importance. Why having this at all?
- Timestamps: What's the reason for having the "T" in the timestamp?  
Having looked at hundreds of different log formats, I have never seen  
anything like that. Why doing this?
- Hostname: I am not comfortable with the whole hostname spec. I like  
that there is an ordering and people are supposed to use FQDN, but  
there are many questions about this. To start with, in a lot of UNIX  
configurations, /etc/hosts contains an entry like:
127.0.0.1   localhost.localdomain  localhost
The second column is the FQDN (technically). Is that one that can be  
used? Can you make it clear that this is not what should be used? Same  
for 127.0.0.1 or the loopback address in general. How does a machine  
know whether an IP address is static or dynamic? How does a logging  
application know? I don't think you will ever know that. Did you mean  
a private versus a public address? That might be interesting.  
Furthermore, it should specify which interface's IP address to use.  
The one the message is sent out on?
- Under the section of PROCID: The text is imprecise. This number is  
not the process ID of the syslog process, it's the ID of the writing  
process. The third paragraph talks about detecting restarted  
applications and somehow mixes in the syslog process. ("might be  
assigned the same process ID as the previous syslog process".) This is  
not clear at all and very very confusing
- MSGID: Make clear that this ID is local to the application. It's not  
a global ID at all.

The biggest issue I have around the SD-ID field:

- I like that the user can extend these.
- Why is this structure so complicated? Why not going with a simple  
set of key-value pairs? This whole structure thing is so complicated.  
Parsing it, you need to keep state! You need to remember the SD-ID for  
each SD-PARAM. Why introducing this? Just stick with simple key-value  
pairs. That makes parsing easier. Much easier. And it makes the events  
easier to produce as well.
- By keeping an explicit message field (the unstructured part), you  
encourage people to still log in that way. I recommend using an  
explicit field (or parameter) that can be used to include human  
readable text. Instead of this:

<165>1 2003-08-24T05:14:15.000003-07:00 192.0.2.1 myproc 8710 - - %%  
It's time to make the do-nuts.

use:

<165>1 2003-08-24T05:14:15.000003-07:00 192.0.2.1 myproc 8710 -  
message="%% It's time to make the do-nuts."

or really:

1 2003-08-24 05:14:15.000003-07:00 host=192.0.2.1 process=myproc  
procid=8710 message="%% It's time to make the do-nuts."

- I definitely like the consideration of some of the special fields  
(structured data IDs). However, they should be used as simple keys (or  
parameters) that have special meaning.
- Parameter - origin: What does it mean to have multiple origins IPs?  
Is that a syslog forwarding chain? The document does not say anything  
about that. Also, we already have the host field in the beginning of  
the syslog messages. What's the relationship to that? Or is origin  
something completely different?
- Parameters - I would really like to see some use-cases for all of  
the IDs. Especially the sequenceId. I am assuming this is something  
that the syslog daemon assigns, not the logging application. Right? I  
think that needs to be clearer. For the sequenceId, what happens for  
forwarded messages? Are these IDs local? Are they forwarded along with  
a message?  Also, how does the logging application know about the  
timeQuality? Or if that something that the syslog daemon assigns, how  
does it know?
- I would really like to see the parameters to go away and have a  
generic key-value extension. In addition, IANA should have a set of  
allowed/defined keys. The parameters should be part of those. Each key  
has a special meaning (semantics). There should be a whole lot of  
them: src_ip, user_name, etc. Each producer should be free to add  
additional keys, realizing that not all consumers would understand  
their semantics. However, the consumers could still read them.

That's it for now.

Thanks a lot

   -raffy

--
   Raffael Marty
   Chief Security Strategist                           @ Splunk>
   Security Visualization: http://secviz.org       raffy.ch/blog

_______________________________________________
Syslog mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/syslog

---

From: Chris Lonvick <[email-masked]>
Date: Fri, 19 Dec 2008 08:35:20 -0800 (PST)

Hi Raffy,

> feedback that I had about the current state of the syslog document:

You are late.  In fact, the document has been approved to become an RFC so 
no changes are allowed in that document.

I see that you note some areas of the specification that you disagree 
with.  This is expected as the IETF process (which is completed in this 
case) is a series of proposals, discussions, and compromises.  It is not 
expected that everyone will be entirely happy with the results, but 
overall we feel that we are pushing the technology forward and are 
producing a better product.  In fact, everyone can tell that we know that 
we havn't produced a perfect product since we have version numbers.  ;-)

Some comments inline.

>
> - Let me say this first: I really like some of the changes that have been 
> incorporated.

:-)

> - Syslog message facility: Why do we still keep this? The only reason that I 
> see people using the facility is to filter messages. There are better ways to 
> do that. Some of the pre-assigned groups are fairly arbitrary and not even 
> really implemented in most OSs. UUCP subsystem? Who is still using that? I 
> guess the reason for keeping it is backwards compatibility? If possible, I 
> would really like this to be gone.
> - Priority calculation. The whole priority field is funky. The priority does 
> not really have any meaning. The order does not imply importance. Why having 
> this at all?

The PRIs are being used today (although I agree that most labels are 
defunct) and the WG felt that having some backwards compatability would 
increase adoption of the spec.

> - Timestamps: What's the reason for having the "T" in the timestamp? Having 
> looked at hundreds of different log formats, I have never seen anything like 
> that. Why doing this?

http://www.ietf.org/rfc/rfc3339.txt?number=3339

> - Hostname: I am not comfortable with the whole hostname spec. I like that 
> there is an ordering and people are supposed to use FQDN, but there are many 
> questions about this. To start with, in a lot of UNIX configurations, 
> /etc/hosts contains an entry like:
> The second column is the FQDN (technically). Is that one that can be used? 
> Can you make it clear that this is not what should be used? Same for 
> 127.0.0.1 or the loopback address in general. How does a machine know whether 
> an IP address is static or dynamic? How does a logging application know? I 
> don't think you will ever know that. Did you mean a private versus a public 
> address? That might be interesting. Furthermore, it should specify which 
> interface's IP address to use. The one the message is sent out on?

Domain names are defined in RFCs 1034 and 1035.  "localhost.localdomain" 
is not an FQDN.

> - Under the section of PROCID: The text is imprecise. This number is not the 
> process ID of the syslog process, it's the ID of the writing process. The 
> third paragraph talks about detecting restarted applications and somehow 
> mixes in the syslog process. ("might be assigned the same process ID as the 
> previous syslog process".) This is not clear at all and very very confusing
> - MSGID: Make clear that this ID is local to the application. It's not a 
> global ID at all.
>
> - Why is this structure so complicated? Why not going with a simple set of 
> key-value pairs? This whole structure thing is so complicated. Parsing it, 
> you need to keep state! You need to remember the SD-ID for each SD-PARAM. Why 
> introducing this? Just stick with simple key-value pairs. That makes parsing 
> easier. Much easier. And it makes the events easier to produce as well.
> - By keeping an explicit message field (the unstructured part), you encourage 
> people to still log in that way. I recommend using an explicit field (or 
> parameter) that can be used to include human readable text. Instead of this:
>
> <165>1 2003-08-24T05:14:15.000003-07:00 192.0.2.1 myproc 8710 - - %% It's 
> time to make the do-nuts.
>
> use:
>
> <165>1 2003-08-24T05:14:15.000003-07:00 192.0.2.1 myproc 8710 - message="%% 
> It's time to make the do-nuts."
>
> or really:
>
> 1 2003-08-24 05:14:15.000003-07:00 host=192.0.2.1 process=myproc procid=8710 

Those could be considered globally unique attribute-value pairs as there 
would be no uncertainty about their context in those situations.  In 
specific contexts, we decided that people will reuse the attribute labels. 
As an (unlikely) example, let's use the attribute label of "color".  if it 
were globally unique, then it would have to be globally defined and 
accepted.  We get around that by specifying the context.  To use your 
way, we would have:
{syslog header stuff} color="yellow" I don't know what this really means
or, as we've defined it:
{syslog header stuff} [box_identification@9 color="teal"] Cuz it's cisco
We figured that the alternative would be to make each attribute label 
globally unique such as:
{syslog header stuff} ciscos_box_identification_color="teal"
which is not really workable.

> - Parameter - origin: What does it mean to have multiple origins IPs? Is that 
> a syslog forwarding chain? The document does not say anything about that. 
> Also, we already have the host field in the beginning of the syslog messages. 
> What's the relationship to that? Or is origin something completely different?

We had several discussions about device virtualization.  Consider that a 
hosting machine will have its own IP address and hostname and that the 
virtual machines will have their own names and addresses.  The hosting 
device will now be able to send a message describing itself or one of its 
virtual machines.

> - Parameters - I would really like to see some use-cases for all of the IDs. 
> Especially the sequenceId. I am assuming this is something that the syslog 
> daemon assigns, not the logging application. Right? I think that needs to be 
> clearer. For the sequenceId, what happens for forwarded messages? Are these 
> IDs local? Are they forwarded along with a message?  Also, how does the 
> logging application know about the timeQuality? Or if that something that the 
> syslog daemon assigns, how does it know?

We've discussed writing a "use case" document.  It's not going to happen 
under our current charter but that's good input for rechartering.

(Note to group:  We won't discuss rechartering the WG until AFTER we get 
syslog-sign approved by our AD.)

> - I would really like to see the parameters to go away and have a generic 
> key-value extension. In addition, IANA should have a set of allowed/defined 
> keys. The parameters should be part of those. Each key has a special meaning 
> (semantics). There should be a whole lot of them: src_ip, user_name, etc. 
> Each producer should be free to add additional keys, realizing that not all 
> consumers would understand their semantics. However, the consumers could 
> still read them.

See above.  The IANA will update the parameters based upon RFCs.  I think 
that having those defined for specific uses is a really good idea and I 
think that discussion will be welcomed when we get around to talking about 
rechartering.

> -raffy

Best regards,
Chris