Subject: RE: [Syslog] Consensus?

From: "Rainer Gerhards" <[email-masked]>
Date: Fri, 25 Nov 2005 12:57:03 +0100

Tom, WG:

Comments inline...

Rainer

> [tp] Strange, I was thiinking quite the opposite, that we had 
> a fragile
> consensus which disappeared in
> Vancouver and has not been refound.  Looking back at the 
> messages posted in the
> past few days, about what should be in the header in what 
> order, I was thinking,
>  what now? because I see no consensus, rather the re-emergence of many
> different points of view.
> 
> So while the proposed charter looks ok, because it does not 
> go into that detail,
> I do not see how we progress any further, into the next level 
> of technical
> detail, of what and
> how should be in the header.

I agree that we have not found consensus again. However, I think we are
in a better shape on the details than it it might look. My personal view
is that many items are shortly before reaching actual consensus. Let me
sum up the items I see. We might use that as a potential basis for
reaching consensus.

#1 testing and code review has shown that there is no point
   in trying to preserve more than <PRI>; RFC 3164 provides
   a false impression of common behaviour.
 
This is controversal, but the facts are suggesting this is the way it
is.
We should try to reach consensus on this.

#2 The max message length issue resurfaces. There seems to be a
   fragile consensus that we can life with the compromise in
   syslog-protocol and eventualy open a debate if we (ever) go to
   a TCP transport.

Again, controversal, too.

#3 There is a question if NUL octets are to be supported in
   the MSG content.

No consensus yet.

#4 There seems to be a fragile consensus that MSG should 
   primarily contain textual data, including XMLified content.
   On the contrary, pure binary data should not be used.

This is somewhat controversal.

#5 Character encoding in MSG: due to my proof-of-concept
   implementation, I have raised the (ugly) question if we need
   to allow encodings other than UTF-8. Please note that this
   question arises from needs introduced by e.g. POSIX. So we
   can't easily argue them away by whishful thinking ;)

Not even discussed yet.

#6 field order

This is related to #1 and can, I think, quickly be fixed once #1 is
settled.

#7 Header fields: there seems to be a fragile consensus that
   MSGID, PROCID, APP-NAME, and VERSION should be in the header
   and TRUNCATE should not be in it.

This needs to be finalized, but I think we are very close.

#8 MSG-octet counting and/or trailer is resurfacing. I think
   this item is not well understood and well discussed.

We need to discuss it.

#9 I have found some minor items not already discussed during
   my proof-of-concept implementation (like "drop requirements"
   and such). These are not absolutely vital, but should be
   considered before a final text is issued (or even the next
   revision).

This needs to be discussed. As it is new, I have no idea what the
discussion may lead to.

#10 STRUCTURED-DATA: there has been surprisingly few discussion
    about STRUCTURED-DATA. I conclude that this is non-controversal.

The good thing is that more people (especially implementors) are
expressing themselfs on the list. This is what finally makes me feel
positive about finally reaching our goal.

I think we can produce some useful documents if we manage to keep
discussing in the current paste. My whish would be that we focus more on
practical things in this discussion, especially when it comes to
compatibility with existing (deployed) technology. It does not help if
we theoretice things would be this and that when they are acutally
different - even though this "real world" is kind of dirty...

Also, we should try to "fry the small fish" and not solve any need that
might surface. I would also like to see this group moving in a direction
that focusses on results usable in practice. I personally prefer a
solution that is 80% theoretically "clean" but has a 95% chance of
implementation over a 95% "clean" solution with just a low chance of
getting implemented (hint: RFC 3195).

I am an IETF freshman. Anyhow, I often read that the IETF was driven by
"rough consensus and running code". I say "was", because my impression
is that this is no longer the case. I would prefer it were...

If this WG fails, what probably happens is that some implementors stick
together and implement something that is pretty similar to what we have
right now (or totally different - who knows). Just as syslog/tcp is
implemented in this "industry-standard" way. The IETF does not like
plain tcp syslog. Guess what? No implementor or end user cares... ;)
This often-violently-objected transport mode has brought more security
and reliability to the syslog community than anything this group has
done the past years. And yes, it is true, plain tcp syslog as it is
deployed is far from being a clean solution. It's only plus is it
works...

So I think it would be a very helpful if we manage to produce a standard
that solves the currently horrible syslog format scenario. But we need
one that gets implemented...

Rainer

_______________________________________________
Syslog mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/syslog

---

From: Carson Gaspar <[email-masked]>
Date: Fri, 25 Nov 2005 16:53:54 -0500

--On Friday, November 25, 2005 12:57:03 PM +0100 Rainer Gerhards 
<[email-masked]> wrote:

># 8 MSG-octet counting and/or trailer is resurfacing. I think

IFF we decide to do octet-counting we should put the length at a fixed 
offset (barring other considerations). This allows implementors to do a 
quick recvmsg(...MSG_PEEK) and extract the message length without having to 
parse random data.

-- 
Carson Gaspar

---

From: Chris Lonvick <[email-masked]>
Date: Sat, 26 Nov 2005 10:40:57 -0800 (PST)

Hi All,

I am finding that the people contributing to the mailing list are making 
progress in defining a useful protocol.  I also see that they are 
discussing implementation details.  Both of these tell me that we're on 
the right track.

What we found in Vancouver is that we were on the wrong track.  The 
protocol being discussed was going to break existing receivers and 
information was going to be dropped on the floor.  By keeping <PRI>, the 
existing receivers will not drop things on the floor and we can expect a 
much better acceptance rate from the community.

We've seen that RFC 3195 is being implemented.  I do not want to de-focus 
this group by encouraging discussion of that at this time.  After we get 
syslog-protocol and syslog-transport-udp submitted to the IESG, I will ask 
how we want to proceed with a revision of that Proposed Standard.  One 
person has suggested to me that it can be revised through an individual 
submissionn.

Rainer's list (below) will serve as our list of issues.  I'd like for 
these to be closed out within 2 weeks.  I'm going to look for rough 
consensus on these issues and I will greatly appreciate everyone accepting 
the decisions we make at the end of that time.  After that, I'll ask for 
Rainer to submit a new ID.  If needed, I'll also ask for Anton to submit a 
new ID.  I'll allow a week of review on those.  If we still see 
implementors willing to implement this then we will progress these to the 
IESG.

Addtional comments below.

> We should try to reach consensus on this.

3164 was written based upon what Eric told me of how things originally 
worked.  Obviously not much has stayed that way since.  We can accept that 
things are received and placed in the right bins if a message has <PRI>. 
I'm OK with keeping that and defining the rest of the message as long as 
we keep the free-form text which has been the basis of syslog for all of 
these years.

> This is somewhat controversal.

If we can agree that a natural language indicator (likely an SD-ID 
element) can be defined, then I believe that we are setting the stage so 
that future definitions may be made to allow for XML, binary, and perhaps 
even Martian.  Those are outside of the current scope of the WG.  Let's 
concentrate on transferring the textual information.

>   can't easily argue them away by whishful thinking ;)

I haven't reviewed that yet.  However, I'll note that allowing different 
encoding can be accomplished in the future as long as we establish a 
default encoding and a way to identify it in our current work.

> This is related to #1 and can, I think, quickly be fixed once #1 is

Agreed.

> This needs to be finalized, but I think we are very close.

Agreed.

> This needs to be discussed. As it is new, I have no idea what the

Running code carries weight in our discussions.

>    about STRUCTURED-DATA. I conclude that this is non-controversal.

This appears to have been accepted by the community.  This item is closed.

Thanks,
Chris

---

From: Darren Reed <[email-masked]>
Date: Sun, 27 Nov 2005 11:51:40 +1100 (EST)

Rainer,

I've put some comments about your list below.  If you repost this,
you might want to re-order ... there are a few dependencies within
this list.

> We should try to reach consensus on this.

A catch here is that extending the format to be:

<PRI>VERSION

may produce unexpected results with various syslog daemons.

> Again, controversal, too.

This should be defined by each of the syslog/transport documents.
It shouldn't be documented in syslog-protocol because it is largely
dependent on the implementation.

> #3 There is a question if NUL octets are to be supported in

Why not just use '\' as the escape character to quote any value outside
of 32-126 ?  Isn't this close enough to a defacto-standard to use?

> This is somewhat controversal.

I don't think this should be discussed.  It's data that fits #3.
End of story, as far as I'm concerned :)

>    can't easily argue them away by whishful thinking ;)

This has implications for #3 & #4 and is possibly dependent on #10.

> This is related to #1 and can, I think, quickly be fixed once #1 is

This needs the header fields (#7, #10) to be accepted, as well, before
it can be settled.

> This needs to be finalized, but I think we are very close.

We should consider how to make them optional, too.

>    this item is not well understood and well discussed.

By trailer do you mean an End of Message (EOM) or EOR marker ?
Or something else ?

> This needs to be discussed. As it is new, I have no idea what the

Are you referring to implementation details, here, with things like
"drop requirements" ?

If so, this is part of what would be a different document again,
syslog-implementation.

>     about STRUCTURED-DATA. I conclude that this is non-controversal.

Is this mean to be a header field?  Examples posted, so far, suggest
that it is.  To explain that comment, I'm considering everything prior
to MSG as a header.

Darren