Subject: RE: [Syslog] formal Consultation prior to concluding the working group

From: "Rainer Gerhards" <[email-masked]>
Date: Fri, 18 Nov 2005 18:02:06 +0100

Sam,

> However several things become harder if you don't meet.  The one that
> I'm most worried about is making sure there are enough people in the
> WG who have agreed to write text and enough people in the WG who are
> reviewing text.  If it's just three or four active people, you don't
> have a working group.

This is my primary concern at this time - and most probably the root
cause of all the trouble. I do not know how to solve this. We've tried
to promote more participation, but obviously failed at that... If we can
not generate more interest, we can probably not succeed, no matter if we
meet or not. If you have any idea on how we could change that, I would
deeply appreciate it. But maybe we simply need to accept the fact that
there is not sufficient interest in syslog to justify any efforts (I am
saying this based on disappointment, but in order to face reality).

Rainer

_______________________________________________
Syslog mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/syslog

---

From: "Anton Okmianski \(aokmians\)" <[email-masked]>
Date: Fri, 18 Nov 2005 13:01:36 -0500

Rainer, Sam and all:

I know that Cisco, for one, has great interest in this work.  I agree WG participation is weak, but I think a lot of people are lurking and only voicing opinions when they see issues that really matter to them.  Truth is, a lot of the issues are probably not significant enough for people to bother to voice preferences.  But that does not mean that there is lack of interest in a syslog standard, or that it is not important.  

I have been involved in logging standardization (internal and public) for 4 years now. And it is always hard to get people excited about logging, even thought many people realize it is important.  Very few people specialize on logging problems or care to think about them for more than 1 minute.  So, I'd venture to say that what we observe in this WG is not unique. Low participation, in this case, does not diminish the value of the work produced by a few people who specialize on the problem. 

We just need a process that allows work to come to fruition. To this end, I will try to propose very specific clarifications for charter next week.

Thanks,
Anton. 

> -----Original Message-----
> From: [email-masked] 
> [mailto:[email-masked]] On Behalf Of Rainer Gerhards
> Sent: Friday, November 18, 2005 12:02 PM
> To: Sam Hartman
> Cc: [email-masked]; [email-masked]; Darren Reed
> Subject: RE: [Syslog] formal Consultation prior to concluding 
> the working group
> 
> Sam,
> 
> This is my primary concern at this time - and most probably 
> the root cause of all the trouble. I do not know how to solve 
> this. We've tried to promote more participation, but 
> obviously failed at that... If we can not generate more 
> interest, we can probably not succeed, no matter if we meet 
> or not. If you have any idea on how we could change that, I 
> would deeply appreciate it. But maybe we simply need to 
> accept the fact that there is not sufficient interest in 
> syslog to justify any efforts (I am saying this based on 
> disappointment, but in order to face reality).
> 
> Rainer
>

---

From: "Alexander Clemm \(alex\)" <[email-masked]>
Date: Fri, 18 Nov 2005 10:26:38 -0800

To add on to Anton:  I can confirm that from our side, at least, there
is a lot of interest in this work, and we strongly believe that the
working group should continue.  As Anton mentions, we will be posting a
consolidated response to many of the issues next week.  

--- Alex  

-----Original Message-----
From: [email-masked]
[mailto:[email-masked]] On Behalf Of Anton Okmianski
(aokmians)
Sent: Friday, November 18, 2005 10:02 AM
To: Rainer Gerhards; Sam Hartman
Cc: [email-masked]; [email-masked]; Darren Reed
Subject: RE: [Syslog] formal Consultation prior to concluding the
working group

Rainer, Sam and all:

I know that Cisco, for one, has great interest in this work.  I agree WG
participation is weak, but I think a lot of people are lurking and only
voicing opinions when they see issues that really matter to them.  Truth
is, a lot of the issues are probably not significant enough for people
to bother to voice preferences.  But that does not mean that there is
lack of interest in a syslog standard, or that it is not important.  

I have been involved in logging standardization (internal and public)
for 4 years now. And it is always hard to get people excited about
logging, even thought many people realize it is important.  Very few
people specialize on logging problems or care to think about them for
more than 1 minute.  So, I'd venture to say that what we observe in this
WG is not unique. Low participation, in this case, does not diminish the
value of the work produced by a few people who specialize on the
problem. 

We just need a process that allows work to come to fruition. To this
end, I will try to propose very specific clarifications for charter next
week.

Thanks,
Anton. 

> Subject: RE: [Syslog] formal Consultation prior to concluding the 
> working group
> 
> Sam,
> 

> to promote more participation, but obviously failed at that... If we 
> can not generate more interest, we can probably not succeed, no matter

> if we meet or not. If you have any idea on how we could change that, I

> would deeply appreciate it. But maybe we simply need to accept the 
> fact that there is not sufficient interest in syslog to justify any 
> efforts (I am saying this based on disappointment, but in order to 
> face reality).
> 
> Rainer
>

---

From: "Rainer Gerhards" <[email-masked]>
Date: Wed, 30 Nov 2005 12:49:04 +0100

Sam,

with this, I am updating my formal reply from November, 18th. Since
them, positive signs have emerged. I agree with Chris on that.

We have had a discussion on a new charter and I think we have consensus
and/or are very close to it. Also, a lot of technical discussion has
been going on. That shows the ability of the WG to look at technical
issues and work towards a solution. If I am not totally wrong, rough
consensus can be reached soon. Also, I have done a proof-of-concept
implementation in the mean time which revealed some other important
issues and also serves as proof for some technical arguments. With the
post-Vancouver changes, the implementation was relatively effortless.
Lab testing has been done, a thing the WG as whole did not do enough
before Vancouver. This lab testing has revealed that compatibility with
RFC 3164 is not extremely important (except for PRI, which is vital).
More implementors have joined the list and Chris has managed to
motiviate some reviewers. Finally, the candidate format for the next
revision has shown in lab testing that it provides adequate backwards
compatibility.

I have summarized all of this on

    http://www.syslog.cc/ietf/protocol.html#200511

which I invite you to at least quickly review. It also has extended
reasoning why RFC 3164 does not tell the "real" truth, a fact that
probably is important for any further decisions.

So far the good news. On the negative side, the consensus still seems to
be very fragile. Also, some issues are re-itereated ever and ever again.
While WG list participation has increased, there are currently still
only 15 voices heared, 3 of them only once (I have counted posters in
the mailing list archive - mails send after the Vancouver meeting). The
actual discussion is still limited to few people, but that might be
acceptable. As Anton mentioned, there are few experts in syslog logging
and the overall interest is low.

I have also seen the NETCONF WG discussing a syslog replacement after
the Vancouver meeting. I see they struggle with a set of similar
questions. There seem to be more participants. If NETCONF redos event
logging, the advantage is that the do not need to look at any backwards
compatibility at all. Should the syslog WG fail to complete its work,
NETCONF event notifications could be the alternative in the long term.
However, given the wide-spread use of syslog, I doubt that syslog will
ever be totally replaced by something else. In an ideal world, I would
like to see a revived syslog WG work together with NETCONF on this
issue. To be honest, I have not yet tried to influence the NETCONF
discussion because I am unsure about the state of the syslog WG.

If I weigh pros and cons, I think that the WG should be allowed to try
to complete its work. Especially the fact that lab testing has revealed
such large incompatibilities of existing deployed technology, together
with the relative ease of fixing it, provides reasoning enough for it
being alive.

HOWEVER, I also think we need to deliver documents, not just arguments.
So my proposal is that the WG be rechartered based on the soon-to-be
reached consensus of the charter. Then, a challenging milestone (no
later than the next IETF meeting) should be set for deliverying
syslog-protocol and syslog-transport-udp.  If that milestone can not be
reached, I suggest concluding the WG because of its inability to deliver
results. We are discussing this for roughly 3 years now, I and others
have spent considerable time in trying to solve it and it does not make
sense to continue if we can not reach consensus. What concerns me in
this regard is that I still do not know how we can avoid such an "out of
the sudden" discrepancy between IETF meeting and list consensus. Not
meeting, I think, is not an option. The feedback from Vancouver was
important. I just would like to see this on-list, too.

Let me close my conclusion with the reminder that all what I have said
is my personal view. The WG may or may not agree with this - I haven't
even tried to reach consensus on it. I hope it is useful in your
decision process, but that's all it is about.

I personally commit to try to finish syslog-protocol, but I will
definitely drop out of this effort mid-next year at latest, no matter
what the overall route then might be. I hope we can finish things
quickly and then focus on other small documents. There is lots that
could be done and should be done in syslog. Just we need to do small
steps, small documents as Darren rightly proposed. We need to iron them
out within a few month and not years. I am positive about all this and
think Chris has already guided the WG into the right direction.

My best regards,
Rainer

> Sent: Friday, November 18, 2005 10:01 AM
> To: [email-masked]; [email-masked]
> Cc: [email-masked]; Darren Reed
> Sam & WG:
> This is a long mail. It is both my answer to Sam's formal request for
> input on the future of the WG as well my suggestion for 
> rechartering and
> mile stones. I consider it to be important. I know that reading long
> mails sometimes is annoying. I also know that people 
> (including me) tend
> to not read long mails in detail. However, I have taken considerable
> effort in creating it and would appreciate if it were actually read
> thouroughly. I am telling this bluntly because out of past 
> experience I
> know that postings, especially larger ones, are not always handled
> properly (which might be one of the causes for our current dilemma). I
> think it is unavoidable to sometimes post a longer message.
> 
> My proposal in regard to the charter is based on the meeting 
> minutes and
> the discussion with Darren and Chris on this list (sorry, 
> nobody else is
> discussing...). The "list consensus" (that is between Darren, 
> Chris and
> me ;)) seems to be
> 
> - syslog-protocol in its current form is objected because we
>   fear it won't be implemented (though we don't have any 
>   "hard facts" on that)
> - syslog is important and it should be standardized
> - it is better to do small steps than a single big one
> - having a layered approach for syslog (with transport mappings,
>   protocol description and application-layer add-ons) seems to
>   be benefitial
> - in order to gain acceptance, the protocol format document
>   should be backwards compatible to existing syslogds
> - *once* we have achieved creating initial transport mappings
>   and format specification, *then* we can look at things like
>   structured data
> - a secure transport is needed (no consensus so far on SSL/TLS,
>   SSH or whatever else)
> 
> It is questionable if there is consensus on the following (though I
> wouldn't outrule it):
> 
> - plain tcp syslog as currently being widely implemented and
>   deployed should no longer be ignored by the WG
> 
> These bullet points could probably be used to create a new 
> charter. The
> main issue they boil down to is that whatever we do should not break
> existing receivers. There are some subleties involved with that.
> Syslog-protocol-15 solved them by deliberately changing the header so
> that a previous syslogd would not try to interpret it. This tool has
> been rejected and is no loger available. On closer look at 
> existing code
> bases, I also have to admit that it might not have worked as designed.
> We now head toward keeping the header and looking like 
> old-style syslog.
> So we need to be very careful that we do not break existing, deployed
> technology. I would like to outline a few of the issues (those I know
> out of my head), so that they can become part of our decision process.
> This list is not necessarily complete.
> 
> - PRI MUST stay as it is currently defined. Adding more facilities
>   causes considerable problems to many 
> existing/widely-deployed syslogd
>   implementation.
> - MSG MUST NOT allow NUL octets. This would breaks almost all syslogd
>   implementations I know (because of the C/C++ string terminator)
> - it is questionable if UTF-8 can be used inside MSG, because many
>   syslogds do not expect this neither can they handle it. UTF-7 might
>   be an alternative (this is no proposal!).
> - in real-world syslogds, the US-ASCII requirement specified 
> in RFC 3164
>   for MSG is NOT honored, not even known. Actually, I have 
> not yet seen
>   a single code base that restricts the character set in MSG. In Asia,
> syslog
>   messages routinely contain DBCS character sets, like 
> shift-JIS or EUC.
> This
>   might imply that NUL characters are currently part of MSG, 
> even though
>   I have not yet noticed it. In Europe, MSG routinely contains octets
> with
>   values between 128 and 255. So if we want to keep consistent with
>   deployed syslog technology, we can not place any further requirement
>   on MSG as that it might contain any octet value (I know this
> contradicts
>   with the non-NUL requirement I made immediately above - 
> this would be
>   a topic for discussion). Most importantly, we MUST not require
> anything
>   in regard to internationalization (this might be the topic of a
> follow-up,
>   small, other document).
> - while real-world syslogds default to 1K message size, they can be
>   configured to larger message sizes. This ability MUST be retained
>   in a spec.
> - real-world syslogds already accept and emit FQDNs (as well as
>   hostname-only) in the HOSTNAME field.
> - BUT: not all real-world syslogds support HOSTNAME at all. If we
>   standardize a header, we must provide instructions on how to detect
>   if there is a HOSTNAME in the message or not (it's doable, I and 
>   others have done it in our implementations). It remains as a
>   problem that an older syslogd not expecting a HOSTNAME will
>   misinterpret it as a TAG. This can cause major problems in existing
>   analysis scripts. That problem can probably only be solved by
>   adding an additional hostname field inside the current MSG part.
> - a similar issue exists for the TIMESTAMP. Many existing syslogds
>   check if the punctuation inside the TIMESTAMP matches their
>   expectations. If so, it is accepted as valid TIMESTAMP, if not
>   it isn't. So we can NOT create the much asked-for TIMESTAMP with
>   year (and timezone) in it without loosing backwards compatibility 
>   (note that it does NOT help to keep the current format and just add
>   the year). Again, the only workable solution might be to 
> add an extra
>   TIMESTAMP field inside MSG.
> 
> In my point of view, the thoughest issues are HOSTNAME and 
> TIMESTAMP. If
> we relax the backwards compatibility needs and accept that an older
> syslogd misinterprets the header, we could solve them. Please keep in
> mind that with current technology header misintrepretation is a common
> problem, so we would not add any new evil. But it will definitely be a
> big issue for sites in the migration process. Currently, a site
> typically goes for a single syslog implementation and has put
> workarounds for the few non-conforming senders. Similar workarounds
> would be needed during a migration. This might be an acceptable extra
> effort - or not (to be discussed).
> 
> Duplicating TIMESTAMP and HEADER comes at the cost of being 
> chatty and I
> also think it looks silly and - by doing so - discourages their
> acceptance. If we discuss this issue seriously, we might also come up
> with another solution. If we re-charter, the charter should 
> specify the
> priority in regard to backwards compatibility, so that we can base the
> needed though decision on charter priorities.
> 
> On the issue whether we should standardize the header or just 
> recommend
> it, I am of the strong position that we must standardize it. 
> If we just
> recommend things, we can simply promote RFC 3164 to standard, 
> because it
> already tells us that a syslog message is a syslog message no matter
> what it contains. The only requirement it must fullfil is that it is
> sent to a syslog listener. I do not find this definition 
> useful. But if
> we keep with that spirit, there is no need for any further work. Just
> recommending a new header, probably one that does not solve the
> backward-compatibility issues that raised this discussion, does NOT do
> any good. It just creates a fake impression of having solved 
> the problem
> while in practice it complicates things (old receivers will 
> still not be
> able to understand the format and would not have a proper way of
> handling it). So if we take that route, I suggest we conclude this WG
> because it has no power to find useful solutions above what is already
> published.
> 
> Besides these comments in regard to the charter, I am still deeply
> concerned about the ability of this WG to do any 
> representative work at
> all. From those voices in the meeting minutes, only Darren has said
> *anything* at all in the current discussion. I think I am not
> exaggerating if I say that this discussion is extremely important for
> the work and future of the WG. Still no voices heared. How can we
> achieve any stable consensus if nobody cares? I don't find it 
> productive
> if a handful people ew discuss about a draft, then go to the meeting,
> where another handful few object it. I find some valuable 
> wisdom in the
> outcome of the last meeting. This, together with the need to 
> standardize
> logging, is my primary motivation for carrying on. However, it is
> neither productive, nor justifiable, to continue working on 
> the issue if
> there is such a large des-interest in the WG. Eventually, the IETF is
> the wrong forum for syslog standardization. For example, there already
> is a industry-standard on plain tcp syslog, which is supported by most
> major implementations. This is in place for several years now.
> 
> If the current rate of feedback continues, I strongly suggest to
> conclude this WG. This is independent from the point if we could find
> another charter. I simply do not find it acceptable to put any more
> considerable effort into an activity whose outcome has prooven to be
> very questionable. When thinking about this, please also keep in mind
> that the current state of charter discussion roughly 
> resembles what was
> discussed for syslog-protocol in 2003 and the begining of 2004 (see
> mailing list archive if you do not believe me - I already posted some
> pointers;)).
> 
> A compromise might be to re-charter with a *very* restrictive charter
> and *very challenging* mile stones and conclude the WG as soon as the
> first milestone is missed. In my point of view, that first mile stone
> should be for an I-D specifying the protocol format (together with
> Anton's -transport-UDP draft). I propose we do not start any new work
> but change syslog-protocol in the spirit of the new charter. 
> What should
> be reached is
> 
> - achived consensus (both on-list AND off-list!)
> - successful last call
> - AD review
> - AD acceptance of publishing this I-D
> 
> This mile stone should be reached at the next IETF meeting. The WG
> should meet at it. The AD should participate in that meeting 
> and decide
> on concluding the WG or moving the draft forward for 
> publishing shortly
> after the meeting.
> 
> I know this milestone is challenging. Anyhow, it should be doable
> (thinking about all the discussions we had). If we really need to sort
> out just minor issues, we can handle it. I object any later 
> mile stone.
> The reason is that we would invest too much effort into something that
> still is very questionable. We have been working on this (if I look at
> the pre-protocol discussion on -sign and -interntional) since 
> roughly 3
> years now. If we can't finish it in another 5 months, we have no right
> to remain active.
> 
> Of course, that was the compromise suggestion. I am still thinking it
> might be more appropriate to conclude the WG immediately.
> 
> As a side-note to Sam, I am not sure if the inability to 
> specify Unicode
> inside MSG (outlined in my bullet points at the top) would prevent an
> I-D from becoming a standards-track RFC. If so, we can not 
> recharter in
> the spirit of the last meeting. I this case, I recommend immediate
> termination of the WG.
> 
> Rainer
> 
> > [mailto:[email-masked]] On Behalf Of Chris Lonvick
> > Sent: Thursday, November 17, 2005 9:40 PM
> > To: [email-masked]
> > Cc: [email-masked]; [email-masked]; Darren Reed
> > Subject: Re: [Syslog] formal Consultation prior to concluding 
> > the working group
> > 
> > Hi,
> > 
> > Apologies for the silence.  I'm busy with the day job today 
> > and tomorrow.
> > 
> > Very briefly, I'd suggest that we can work towards a charter that 
> > formalizes syslog but does not break existing syslog 
> recievers.  Once 
> > there, we can add in the Structured Data parts that can be 
> > used for higher 
> > level functions such as signing the messages.
> > 
> > Does that approach make sense?  If so, I can start working 
> > out a draft 
> > charter that says that we will do just that.
> > 
> > I would like to hear from more people on this.
> > 
> > Thanks,
> > Chris
> > 
> > On Thu, 17 Nov 2005, Sam Hartman wrote:
> > 
> > >>>>>> "Darren" == Darren Reed <[email-masked]> writes:
> > >
> > >    >>  2) Is there another charter under which the working group
> > >    >> would better be able to make progress?
> > >
> > >    Darren> I believe the answer to this is yes.
> > >
> > >    Darren> There is very relevant work this group could do 
> > and should
> > >    Darren> do.
> > >
> > > In that case I'd recommend that you see if you can get 
> consensus on
> > > such a charter.
> > >
> > > --Sam
> > >
> > >