Subject: Index into fragment
From: Albert Mietus <[email-masked]>
Date: Fri, 29 Aug 2003 15:18:36 +0200

Again ...

The CertBlock contains a index into the payload (size: 1-8)

The RFC doesn't mention the possible value's. In short, does in start
counting at 0, or at 1?

Otherwise said, should the smallest value (as an int) be 1 or
zero. This startvalue is used literaily in the first send certblock.

I think it should be "0"; as that is used for most counters in
syslog. But we have to specify it.

The same applies to "3.7 First Message Number" It also start counting
at 0, I think.

(But this one is less important, even counting at 1 will
not break things)

- -- 
ALbert Mietus
	Send prive mail to:      [email-masked]
	Send business mail to: [email-masked]
	Don't send spam mail!

------------------------------

Date: Fri, 29 Aug 2003 17:21:13 +0200
From: Rainer Gerhards <[email-masked]>
Subject: New real-world product using rfc3195/raw

Hi WG,

just so that you can count the number of implementations: today we
released MonitorWare Agent 1.3 which, among others, contains a RFC
3195/RAW server and client. This product is closed source and targeted
towards the Windows platform. I guess it still counts towards the
implementation count ;)

We already have another solution - Adiscon logger - out which is a small
client only tool. We will release at least 3 more products (final &
beta) within the next few days which support 3195 RAW. COOKED will be
our next target (well, eventually -sign...).

We also have an open source library doing these parts - liblogging - I
am right now preparing a new version with listenere code and hope the
lib will be online shortly. If you would like to be notified when the
lib is ready, please either visit

http://www.monitorware.com/liblogging/

or drop me a mail off-list.

Rainer

------------------------------

Date: Fri, 29 Aug 2003 15:17:34 +0200
From: Rainer Gerhards <[email-masked]>
Subject: RFC 3195 metadata

Hi WG,

probably this mail is (at least slightly) off-topic... Actually I am
looking for some advise from other implementors when it comes to
implementing...

In 3195 COOKED, we have what I would call "metadata" (iam, path,
deviceip, ...). This is very valuable. I am just thinking what to do
with that data

- - when I am a realy relaying to "traditional" syslog
- - when I am a collector persisting it

I am NOT asking in regard what the protocol should look like - I think
this is detailled in the RFC. What I am looking for is what people
actually use this data for. Do they always write it to the log file?
Never? Is it a config option? Or does it go into a separate file?

Please answer off-list, I will compile a summary and post it - if there
is no violent opposition against me doing so.

Thanks,
Rainer

------------------------------

Date: Fri, 29 Aug 2003 14:49:39 +0200
From: Albert Mietus <[email-masked]>
Subject: [minor] FragLen can ony have length upto 3

Page 20 mentions:
"  Fragment Length                FragLen              1-4"

Actually, this can be ``1-3'', as the fragment can never reach a
length of 999. A syslog-header, with fulltimestamp etc will take at
least the other 25 available positions of the 1024 that fitt.

But, is only a detail.

- --
ALbert Mietus
	Send prive mail to:      [email-masked]
	Send business mail to: [email-masked]
	Don't send spam mail!

------------------------------

Date: Fri, 29 Aug 2003 14:34:56 +0200
From: Albert Mietus <[email-masked]>
Subject: syslog-sign, draft-12 length of TPBL

Hello (again)

Draft-12 says the length (size in bytes) of TPBL (Total Payload Block
Length) shoud be 8. (page 20).

I assume, this is a typo, and should be 1-8, like the simulair fields.

Note: It is a integer, like the index, which has length 1-8.

- -- 
ALbert Mietus
	Send prive mail to:      [email-masked]
	Send business mail to: [email-masked]
	Don't send spam mail!

------------------------------

Date: Fri, 29 Aug 2003 15:07:41 +0200
From: Albert Mietus <[email-masked]>
Subject: Question

This time a question, a bit off-topic.

As you know, I'm implementing syslog-sign. And I wondering "when" to
send a SignatureBlock.

The RFC doesn't mentions it (it shouldn't). But maybe somebody a an
opinion about it. A SignatureBlock is always send between 2 normal
messages, using UDP.
In normal cases, it really doesn't matter when it is send, typical,
all messages are received, mostly "in-order". That is typical,
although UDP doesn't guaranty it.

Messages get lost when the network is "to busy".  Or when a cable is
temporally unplugged (for a short time).

So, if two messages are send directly after each other
 a) the make the network more busy. (So it losses the 2nd),
 b) both fall of the lose plug (usually)

The code can try to send messages to increase the probability that all
are received or that "most important one" is received.

My questions is, how to to so.

To simplify the matter, in the examples below each hash is sent only
once. S<n> is the SignatureBlock containing the hash of M<n>.
Each dot represents some (real) time, but not the same about every time.

Option 1: M1 S1 ... M2 S2 ...
Option 2: M1 ... S1 M2 ...S2 M3 ...
Option 3: M1 ... M2 S1 ...M3 S2 ...
Option 4: M1 .. S1 .. M2 .. S2  ..

Option 4 seams nice, to balance the load. But is complex (iff
possible) to implement.

Options 1-3, and variants are more real. I have selected 1 for now, as
it is easy to debug:-) Before I used opt 2, but I'm not sure whether
is actually "better".

Please share you idea's THANKS

- -- 
ALbert Mietus
	Send prive mail to:      [email-masked]
	Send business mail to: [email-masked]
	Don't send spam mail!

------------------------------

Date: Mon, 1 Sep 2003 13:54:13 +0200
From: Rainer Gerhards <[email-masked]>
Subject: TIMESTAMP-3339 (Correction)

WG,

I posted too quickly. Maybe Chris can hold the post. I once again
reviewed RFC 3339 and it says:

- --------------

4.4. Unqualified Local Time

   A number of devices currently connected to the Internet run their
   internal clocks in local time and are unaware of UTC.  While the
   Internet does have a tradition of accepting reality when creating
   specifications, this should not be done at the expense of
   interoperability.  Since interpretation of an unqualified local time
   zone will fail in approximately 23/24 of the globe, the
   interoperability problems of unqualified local time are deemed
   unacceptable for the Internet.  Systems that are configured with a
   local time, are unaware of the corresponding UTC offset, and depend
   on time synchronization with other Internet systems, MUST use a
   mechanism that ensures correct synchronization with UTC.  Some
   suitable mechanisms are:

   o  Use Network Time Protocol [NTP] to obtain the time in UTC.

Klyne, et. al.              Standards Track                     [Page 5]

RFC 3339       Date and Time on the Internet: Timestamps       July 2002

   o  Use another host in the same local time zone as a gateway to the
      Internet.  This host MUST correct unqualified local times that are
      transmitted to other hosts.

   o  Prompt the user for the local time zone and daylight saving rule
      settings.

- ----------------

So I think this means we should stick with that for syslog, too and make
sure the device knows its timezone...

Rainer

------------------------------

Date: Mon, 1 Sep 2003 15:04:47 +0200
From: Rainer Gerhards <[email-masked]>
Subject: RE: TAG in syslog-sign draft

> I think, we need to add "/" as valid char in the TAG.

Would it hurt if we just said:

- ----
   The TAG is a string of visible (printing) characters excluding SP,
   that MUST NOT exceed 32 characters in length.

   The first occurrence of a colon (":") or SP " " character terminates
the TAG
   field. Generally, the TAG contains the name of the process that
   generated the message. It may OPTIONALLY contain additional
   information such as the numerical process ID of that process bound
   within square brackets ("[" and "]"). A colon MUST be the last
   character in this field.

   To be consistent with the format described in RFC 3164, a space
   character need not follow the colon in normal syslog packets.
- ----

In essence, that would allow any character but SP and colon to be used.
IMHO, this would not break existing applications and provide more
flexibility in what coud be in the tag. Think of e.g.

    Myproc[PID,Threadid]:

>
> At least in the FreeBSD version of syslog (which I'm extending to use
> -sign), the TAG is frequently set to the progname, including its path.
> E.g. "/usr/sbin/cron".

Under Windows, this may be "\usr\sbin\cron". In fact it may even be
"c:\usr\sbin\cron" which includes a colon. So I would prefer to see it
terminated by a SP, but I think this is very dangerous given the current
syslog formats out.

>
> I think this logging the path is smart, as it makes clear which
> program is logging. (I think most systems nowdays have multiple

Me too!

> executables with the same name. I have a least 3 programs named
> "sendmail" :-)

Rainer

------------------------------

Date: Mon, 1 Sep 2003 13:29:56 +0200
From: Rainer Gerhards <[email-masked]>
Subject: TIMESTAMP-3339

WG,

I am, too, currently implementing an -sign message parser (but not yet
signing). I am parsing the timestamp.

In ABNF, we say:

      full-time       = partial-time time-offset

That means I must have a time-offset in any case. What should I do (when
generating messages) if I simply do not (reliably) know the TZ the
device is in? This could happen, e.g. if the device TZ is not
configured. Wouldn't it be better to NOT include any timestamp
information so that the receiver knows of the fact I do  not know the
TZ. This would change the above to

      full-time       = partial-time [time-offset]

I know this is not a very common case, but it may happen.

Any thoghts?
Rainer

------------------------------

Date: Wed, 27 Aug 2003 13:42:00 +0200
From: Albert Mietus <[email-masked]>
Subject: Examples in draft- syslog-sing

I raised this isuee some time ago, but let's mention it agian:

In Section 2.4, two examples are given. They are there for some
time now. And in that time, the "recommended" format of loglines
is changed.

This results in examples,which show how rfc3164 messages are
formated. That is still acceptable, but not according to "new
knowledge"

I propose, to update (at least one) example and show a fully formated
line, with long hostname, long timestamp etc.

A few examples (of my current inplementation) are shown below.

<165>2003-08-27T13:38:05+02:00 labtop.ons-huis.net DemoProg[5]: TEST

<165>2003-08-27T13:35:00.761352+02:00 syslog @#sigSIG 0111 509965902 0\
  27 27 3  egTbwI3CsO8EpVzSc1XQ0spV7Yo= IKbm18WmmYD90DsrkL4EjMoYNw8=\
  quveEk1ZqhkTRv4oL8U928bUZ9Y=  JYoz+hkDXhNWGDYU7XhQ/2FLHmQ=

[The second example is broken in 3]

- -- 
ALbert Mietus
	Send prive mail to:      [email-masked]
	Send business mail to: [email-masked]
	Don't send spam mail!

------------------------------

Date: Wed, 3 Sep 2003 10:52:36 -0700 (PDT)
From: Chris Lonvick <[email-masked]>
Subject: Issues

Hi Folks,

We've had some issues raised in the past few weeks about syslog-sign-12.
I believe that I've cataloged them here:
  http://www.employees.org/~lonvick/issues.txt
but please send a note to the mailing list if I've missed any, or if
there are any additional ones.  Let's get some consensus on these so
that John and Jon may update the ID and we can move it along.  :-)

I've moved the syslog-sign ID into html via the xml2rfc program.
  http://www.employees.org/~lonvick/draft-ietf-syslog-sign-12.html
and I'm going to put each of these issues into a separate email - with
appropriate Subject line.  Please respond to these emails with your
thoughts.

Thanks,
Chris

------------------------------

Date: Wed, 3 Sep 2003 10:53:27 -0700 (PDT)
From: Chris Lonvick <[email-masked]>
Subject: Issue 1:  Examples

Issue 1: Examples
http://www.employees.org/~lonvick/draft-ietf-syslog-sign-12.html#Examples

From Archive:
http://www.mail-archive.com/syslog-sec%40employees.org/msg01221.html

Albert has an example available.  I'll suggest that we resolve each
of the other issues and then make sure that the example given is
consistent with the consensus of the group at that time.

------------------------------

Date: Wed, 3 Sep 2003 10:54:43 -0700 (PDT)
From: Chris Lonvick <[email-masked]>
Subject: Issue 2: TAG Field Definition

Issue 2: TAG Field Definition
http://www.employees.org/~lonvick/draft-ietf-syslog-sign-12.html#HEADER

From Archive:
http://www.mail-archive.com/syslog-sec%40employees.org/msg01224.html
http://www.mail-archive.com/syslog-sec%40employees.org/msg01234.html
http://www.mail-archive.com/syslog-sec%40employees.org/msg01222.html

Rainer has proposed the following text:
"""
   The TAG is a string of visible (printing) characters excluding SP,
   that MUST NOT exceed 32 characters in length.

   The first occurrence of a colon (":") or SP " " character terminates
   the TAG field. Generally, the TAG contains the name of the process
   that generated the message. It may OPTIONALLY contain additional
   information such as the numerical process ID of that process bound
   within square brackets ("[" and "]"). A colon MUST be the last
   character in this field.

   To be consistent with the format described in RFC 3164, a space
   character need not follow the colon in normal syslog packets.
"""

However, anyone trying to convey information of "Myproc[PID,Threadid]:"
may have a problem with something like

  syslog[12345,C:\usr\sbin\cron]:

Albert suggests just having "syslog" in the cert/sig-block messages
but that seems to be inconsistent with the possible formats of the
normal syslog messages.

Can anyone offer a suggestion to resolve this?

------------------------------

Date: Wed, 3 Sep 2003 10:55:17 -0700 (PDT)
From: Chris Lonvick <[email-masked]>
Subject: Issue 3:  IANA Considerations for undefined PRI values

Issue 3:  IANA Considerations for undefined PRI values
http://www.employees.org/~lonvick/draft-ietf-syslog-sign-12.html#iana

From Archive:
http://www.mail-archive.com/syslog-sec%40employees.org/msg01213.html
http://www.mail-archive.com/syslog-sec%40employees.org/msg01218.html
http://www.mail-archive.com/syslog-sec%40employees.org/msg01220.html
http://www.mail-archive.com/syslog-sec%40employees.org/msg01226.html

The consensus appears to be to have a new part in the "IANA
Considerations"  of syslog-sign stating the Facilities not defined
are open to future use by the consensus process (RFC 2434 page 6).

The current set of Facilities and Severities are listed by IANA here:
  http://www.iana.org/assignments/syslog-parameters

Any disagreement to that?

------------------------------

Date: Wed, 3 Sep 2003 10:55:44 -0700 (PDT)
From: Chris Lonvick <[email-masked]>
Subject: Issue 4:  Index into Payload in Cert Block

Issue 4:  Index into Payload in Cert Block
http://www.employees.org/~lonvick/draft-ietf-syslog-sign-12.html#index

From Archive:
http://www.mail-archive.com/syslog-sec%40employees.org/msg01228.html

Should the value of "0" or "1" be used as the lowest available value?
Albert suggests "0".  Any disagreement to that?

------------------------------

Date: Wed, 3 Sep 2003 10:56:12 -0700 (PDT)
From: Chris Lonvick <[email-masked]>
Subject: Issue 5:  First Message Number

Issue 5:  First Message Number
http://www.employees.org/~lonvick/draft-ietf-syslog-sign-12.html#firstmsg

From Archive:
http://www.mail-archive.com/syslog-sec%40employees.org/msg01228.html

Similar to Issue 4, should the value of "0" or "1" be used as the
lowest available value?  Albert again suggests "0".  Any disagreement
to that?

------------------------------

Date: Wed, 3 Sep 2003 10:56:36 -0700 (PDT)
From: Chris Lonvick <[email-masked]>
Subject: Issue 6:  Fragment Length

Issue 6:  Fragment Length
http://www.employees.org/~lonvick/draft-ietf-syslog-sign-12.html#fraglen

From Archive:
http://www.mail-archive.com/syslog-sec%40employees.org/msg01229.html

Albert suggests changing the value from "1-4 characters" to "1-3"
since the payload of a syslog Certificate Block will be less than
999 characters -taking out the length of the PRI, TIMESTAMP and
HOSTNAME.  Anyone have any problems with this?

------------------------------

Date: Wed, 3 Sep 2003 10:57:04 -0700 (PDT)
From: Chris Lonvick <[email-masked]>
Subject: Issue 7:  TPBL Length

Issue 7:  TPBL Length
http://www.employees.org/~lonvick/draft-ietf-syslog-sign-12.html#tpbl

From Archive:
http://www.mail-archive.com/syslog-sec%40employees.org/msg01232.html

The Total Payload Block Length parameter is currently defined as
"8 characters" but perhaps that should be "1-8 characters" which is
consistent with other fields.  Comments?

------------------------------

Date: Wed, 3 Sep 2003 10:57:36 -0700 (PDT)
From: Chris Lonvick <[email-masked]>
Subject: Issue 8:  Length of syslog messages

Issue 8:  Length of syslog messages
http://www.employees.org/~lonvick/draft-ietf-syslog-sign-12.html#format

From Archive:
http://www.mail-archive.com/syslog-sec%40employees.org/msg01118.html
http://www.mail-archive.com/syslog-sec%40employees.org/msg01119.html
http://www.mail-archive.com/syslog-sec%40employees.org/msg01120.html

It's defined as 1024 in several places.  Mike McFaden offers a good
suggestion in the 3rd pointer above:

   I prefer that the minimum size a conformant
   implementation must support be defined.
   SNMP over UDP specifies a minimum message size of 484.
   If one wants to support a larger msg size, just configure for it.
   The same goes for switches these days too.
   Otherwise why not just recommend a transport that has
   path mtu discovery in cases where syslog msg length > 1024.

Any comments or other suggestions?

------------------------------

Date: Thu, 4 Sep 2003 09:45:37 +0200
From: Rainer Gerhards <[email-masked]>
Subject: RE: Issue 6:  Fragment Length

Chris,

If you look at issue #8, this may become an issue... I propose we finish
isue #8 before turning back to #6.

Rainer

> -----Original Message-----
> From: Chris Lonvick [mailto:[email-masked]]
> Sent: Wednesday, September 03, 2003 7:57 PM
> To: [email-masked]
> Subject: Issue 6: Fragment Length
>
>
> Issue 6:  Fragment Length
> http://www.employees.org/~lonvick/draft-ietf-syslog-sign-12.ht
> ml#fraglen
>
> From Archive:
> http://www.mail-archive.com/syslog-sec%40employees.org/msg01229.html
>
> Albert suggests changing the value from "1-4 characters" to "1-3"
> since the payload of a syslog Certificate Block will be less than
> 999 characters -taking out the length of the PRI, TIMESTAMP and
> HOSTNAME.  Anyone have any problems with this?
>
>
>

------------------------------

Date: Thu, 4 Sep 2003 09:47:59 +0200
From: Rainer Gerhards <[email-masked]>
Subject: RE: Issue 7:  TPBL Length

> The Total Payload Block Length parameter is currently defined as
> "8 characters" but perhaps that should be "1-8 characters" which is
> consistent with other fields.  Comments?

I would optfor 1-8 - at least I like to keep things as short as
possible.

Rainer

------------------------------

Date: Thu, 4 Sep 2003 13:51:40 +0200
From: Rainer Gerhards <[email-masked]>
Subject: cookies in syslog

Hi,

sorry for raising an additional suggest, but I am currently implementing
and as Albert said, questions arise while doing so ;)

Cookies, as first appeared in syslog-sign, are a simple, efficient and
elegant way to add new functionality to syslog without the need to
change anything in older code. I assume this WG will make use of more
cookies as need arises.

On the other hand, the same concept is also good for vendor-specific
extensions. For example, I am currently designing a way to let windows
event log data travel unaltered via syslog. I am very tempted to do this
with a cookie and will probably do so.

This raises the question of standard vs. vendor cookies. I wonder if
some of the upcoming IDs (-international?) should formalize the cookie
as an optional field. If it starts with @#, then it is an IETF cookie
(IANA?), but if it starts with #@ (or @#X-)it is a vendor cookie. This
would remove any doubt, especially when more people begin to adopt this
scheme.

Any thoughts?

Rainer

------------------------------

Date: Thu, 4 Sep 2003 10:10:11 +0200
From: Rainer Gerhards <[email-masked]>
Subject: RE: Issue 8:  Length of syslog messages

> Issue 8:  Length of syslog messages
> http://www.employees.org/~lonvick/draft-ietf-syslog-sign-12.ht
> ml#format
>
> From Archive:
> http://www.mail-archive.com/syslog-sec%40employees.org/msg01118.html
> http://www.mail-archive.com/syslog-sec%40employees.org/msg01119.html
> http://www.mail-archive.com/syslog-sec%40employees.org/msg01120.html
>
> It's defined as 1024 in several places.  Mike McFaden offers a good
> suggestion in the 3rd pointer above:
>
>    I prefer that the minimum size a conformant
>    implementation must support be defined.
>    SNMP over UDP specifies a minimum message size of 484.
>    If one wants to support a larger msg size, just configure for it.
>    The same goes for switches these days too.
>    Otherwise why not just recommend a transport that has
>    path mtu discovery in cases where syslog msg length > 1024.
>
> Any comments or other suggestions?

Actually, I honstely think path MTU discovery is probably not within the
"simple" spirit of syslog:

#1 it is not done easily (especially if ICMP is filtered)
#2 I think it is not suffcient to look at the transport layer,
   as we may have different transports, which makes it even harder

Let me focus on #2. Let's assume the relay chain below:

Device  -- RFC3164     --> Relay 1
Relay 1 -- 3195/COOKED --> Relay 2
Relay 2 -- RFC3164     --> Relay 3
Relay 3 -- 3195/RAW    --> Relay 4
Relay 4 -- 3195/COOKED --> Collector

OK, I agree this is probably not what we often see. But we could see it.
But we could also look at this simpler sample:

Device -- 3195/COOKED --> Collector

What is the MTU in this case? Is it the default BEEP window size? Is it
unlimited as I can fragment the message without any need to change the
syslog message (AND it will arrive reliably & in order)?

In addition to that, syslog messages are sometimes transferred via
proprietary solutions (e.g. syslog over plain tcp with syslog-ng,
winsyslog and kiwi syslog the least) in parts of the relay chain. Of
course, they are non-standard so we don't need to take care ... but will
we get acceptance for a standard if it forces users to switch to a
totally different logging infrastrucutre? will we get developers to
implement those solutions if they cause big interop (and thus customer
support) issues for them? ... probably not.

I would prefer if we stick with a slightly upwards trimmed 1024 byte
limit for a single plain syslog message. It has prooven to work even
over udp in (obviously) almost all cases (smaller MTUs!). Many have
pointed out that 1024 bytes is by far enough for their needs. I would,
however, like to extend the core syslog format (either via a separate
rfc or in -international) to support app-aware fragmentation of syslog
messages so that a sender who needs to send a larger message - for
whatever reason - can put that larger message into multiple 1024 byte
messages in a standard-conformant way.

Rainer

------------------------------

Date: Thu, 4 Sep 2003 10:26:39 +0200
From: Rainer Gerhards <[email-masked]>
Subject: syslog meta data (from 3195/COOKED)

I would like to add an Issue#9 to Chris list ;)

RFC3195/COOKED allows us to specify meta data (like the path element).
This is very valuable information. However, it can be forwarded inside a
relay chain only if all realys speak COOKED.

Let's take my (somewhat artifical) sample from the issue #8 post:

#1 Device  -- RFC3164     --> Relay 1
#2 Relay 1 -- 3195/COOKED --> Relay 2
#3 Relay 2 -- RFC3164     --> Relay 3
#4 Relay 3 -- 3195/RAW    --> Relay 4
#5 Relay 4 -- 3195/COOKED --> Collector

Let's focus on the path element. In this sample, the relay in step #1
can generate all necessary information to create a path element that is
correct up to the device. It can also forward it correctly in step #2,
so that realy 2 can add its path element. HOWEVER, there is no way to
transmit the path in steps #3 & #4. So it is LOST. In step #4, Relay 4
will construct a new path element, but it can only point back to Relay 3
and this will eventually be invalidly be flagged as a device (because
there is no iam element in 3195/RAW that can tell the relay differently.
That path element will be transmitted in step #5 and the receiving
collector can add the fact that it received from Relay 4(acting as a
relay). So the resulting path will just be:

Device Relay 3 --> Realy Relay 4 --> Collector

This is obviously an incorrect and too short path element. I agree that
the path will not be correct until either the emiting device itself
speaks COOKED or the first relay does. However, I would like to see the
path preserved in cases as above.

As such, I propose we integrate into some existing/future RFC the
ability to transfer metadata over a "plain" syslog packet. It can be
done quite simple by just adding an additional cookie, more or less like
- -sign is done. For example, a packet with tag "syslog " and @#MeTaDaTa
at the start of message could carry over the XML otherwise exchanged in
COOKED.

A rough example might be

<121>Oct 10 12:12:12 myhost syslog @#MeTaDaTa <path
fromFQDN='lowry.records.example.com' fromIP='10.0.0.50'
toFQDN='kurtzman.records.example.com' toIP='10.0.0.51' linkprops='ULRI'
pathID='173'><path fromFQDN='screen.lowry.records.example.com'
fromIP='10.0.0.47' toFQDN='lowry.records.example.com' toIP='10.0.0.50'
linkprops='DLI' pathID='24'></path></path>

[Please note: I have not made sure that this fits in 1024 bytes. It
should all be on a single "line" without any CRLF (but the mailer breaks
it ;)) There are obviously some issues with fragmenting oversize
messages. I haven't taken care of this - this can be done once the WG
finds it is useful to think in this direction.]

For the intial example, it would the final path at the collector at
least make look like:

Device Device --> Relay Relay 1 --> Relay Realy 4 --> Collector

[Relay 4 would be discovered by the collector]
Even though there is still something missing, it looks much more correct
to me.

I specifically propose this as I think legacy syslog will be in relay
chains for quite a while and cause us to loose important information for
many years.

Any comments are highly appreciated ;)

Rainer

------------------------------

Date: Thu, 4 Sep 2003 07:34:31 -0700 (PDT)
From: Chris Lonvick <[email-masked]>
Subject: Re: cookies in syslog

Hi,

This sounds like a good plan to me.  Let's keep the cookies that John and
Jon have defined in syslog-sign and allow Rainer to make the definitions
(with instructions to IANA) in syslog-international.  This will clarify
the use of cookies for other purposes and to differentiate between IETF
cookies, vendor cookies, and any other types (e.g., experimental) cookies.
Any objections?

Thanks,
Chris

On Thu, 4 Sep 2003, Rainer Gerhards wrote:

> Hi,
>
> sorry for raising an additional suggest, but I am currently implementing
> and as Albert said, questions arise while doing so ;)
>
> Cookies, as first appeared in syslog-sign, are a simple, efficient and
> elegant way to add new functionality to syslog without the need to
> change anything in older code. I assume this WG will make use of more
> cookies as need arises.
>
> On the other hand, the same concept is also good for vendor-specific
> extensions. For example, I am currently designing a way to let windows
> event log data travel unaltered via syslog. I am very tempted to do this
> with a cookie and will probably do so.
>
> This raises the question of standard vs. vendor cookies. I wonder if
> some of the upcoming IDs (-international?) should formalize the cookie
> as an optional field. If it starts with @#, then it is an IETF cookie
> (IANA?), but if it starts with #@ (or @#X-)it is a vendor cookie. This
> would remove any doubt, especially when more people begin to adopt this
> scheme.
>
> Any thoughts?
>
> Rainer
>
>
>

------------------------------

Date: Thu, 4 Sep 2003 18:23:27 +0200
From: Albert Mietus <[email-masked]>
Subject: RE: Issue 8:  Length of syslog messages

Rainer Gerhards writes:
 > > Issue 8:  Length of syslog messages

I Agree with Rainer, about the parts of "multi hop". Keep is simple. !

 > [... 1024 ..]  has prooven to work even  over udp
Very true!

 > I would, however, like to extend the core syslog format to support
 > [...]  needs to send a larger message

Syslog does "support" this already. All the programmers has to to is call
syslog(...) several times (or any other function that is part of the API.

Even for a limit of 1024, this means that line becomes 10 or more line
on your editor! If You need that frequently, probably a support
function, or macro is used. That that macro can be used to send that
"long" line in several messages, each on contain a kind of fragment
counter.

But that is about programming/API's, back to the protocol.

When "long lines" are needed, probably the output of the log isn't
meant for human reading, but for "a script" or tool. Then, this is
site or application specific. And so, NO general standard is needed.

At least, I can't think of any reason. But maybe a counter example can
be given... (which used "long lines" AND needs a "generic solution.)

- --ALbert Mietus
	Send prive mail to:      [email-masked]
	Send business mail to:  [email-masked]
	Don't send spam mail!

------------------------------

Date: Thu, 4 Sep 2003 21:36:44 +0200
From: Rainer Gerhards <[email-masked]>
Subject: RE: Issue 8:  Length of syslog messages

>  > I would, however, like to extend the core syslog format to
> support  > [...]  needs to send a larger message
>
> Syslog does "support" this already. All the programmers has
> to to is call
> syslog(...) several times (or any other function that is part
> of the API.

Well, not talking about API, but if you mean I can send several lines in
several messages this is not what I am looking for. I would like to have
a single large message - in the rare occasions there is need for it...

>
> Even for a limit of 1024, this means that line becomes 10 or
> more line on your editor! If You need that frequently,
> probably a support function, or macro is used. That that
> macro can be used to send that "long" line in several
> messages, each on contain a kind of fragment counter.

This is exactly what I am asking for. I don't intend to do something
very complex. Just build a *standard* way of sending oversize messages
if there is need for this. In the -international-00, there is a way to
do so, in case you would like to comment. The way it is in
- -international right now needs to be revised, but I hope it will convey
the spirit.

>
> But that is about programming/API's, back to the protocol.
>
> When "long lines" are needed, probably the output of the log
> isn't meant for human reading, but for "a script" or tool.
> Then, this is site or application specific. And so, NO
> general standard is needed.

Actually, if I look at almost all larger installations, all of them use
some kind of automatted log watchers. It is hard to envision that admins
will dig through a view millions of log lines each day. So I think the
protocol must enable automatic processing...

>
> At least, I can't think of any reason. But maybe a counter
> example can be given... (which used "long lines" AND needs a
> "generic solution.)

Well, I know Microsoft does not necessarily do it smartly. Many
customers ask for forwarding Windows event log data to their central
syslogd. Now lets look at some message from the log:

- ------- BEING LOG ENTRY ;) -----
DNS Server has updated its own host (A) records.  In order to insure
that its DS-integrated peer DNS servers are able to replicate with this
server, an attempt was made to update them with the new records through
dynamic update.  An error was encountered during this update, the record
data is the error code.

If this DNS server does not have any DS-integrated peers, then this
error
should be ignored.

If this DNS server's ActiveDirectory replication partners do not have
the correct IP address(es) for this server, they will be unable to
replicate with it.

To insure proper replication:
1) Find this server's ActiveDirectory replication partners that run the
DNS server.
2) Open DnsManager and connect in turn to each of the replication
partners.
3) On each server, check the host (A record) registration for THIS
server.
4) Delete any A records that do NOT correspond to IP addresses of this
server.
5) If there are no A records for this server, add at least one A record
corresponding to an address on this server, that the replication partner
can contact.  (In other words, if there multiple IP addresses for this
DNS server, add at least one that is on the same network as the
ActiveDirectory DNS server you are updating.)
6) Note, that is not necessary to update EVERY replication partner.  It
is only necessary that the records are fixed up on enough replication
partners so that every server that replicates with this server will
receive (through replication) the new data.
- -------- END LOG ENTRY ------

Of course, you can argue that this should be truncated or detected or
whatever. The plain truth is that it is less error-prone to forward them
as they are to the central syslogd (especially as the text can and has
change with service packs or hot fixes). We can do that with our
products, but wouldn't it be nice if there were a standard way to do
this so that a customer must not necessarily use Adiscon products both
on client and collector?

Another example is international character sets. As some pointed out,
messages may get longer. As other have (correctly) pointed out,
multibyte character sets do not necessarily mean that will, but anyhow,
it would be nice if we need not to be afraid if it will happen.

Again, I do not intend to do something complex here - just a standard
way to say "this is message 1 of 2", so that the collector (if he
decides to do so) can defragment these two messages into a single one.
If you don't like it, you can safely ignore this feature. In fact, I
think it should only be used if you know the transport is reliable,
in-order - otherwise you should expect to loose some fragment.

Still objections?

Rainer

------------------------------

Date: Thu, 4 Sep 2003 22:32:57 +0200
From: Albert Mietus <[email-masked]>
Subject: RE: Issue 8:  Length of syslog messages

Rainer Gerhards writes:
 > >  > I would, however, like to extend the core syslog format to
 > > support  > [...]  needs to send a larger message
 > Well, I know Microsoft does not necessarily do it smartly. Many

 > ------- BEING LOG ENTRY ;) -----
 [deleted about 3k]
 > -------- END LOG ENTRY ------

 > Still objections?

I think it is "silly" to make log-entries of this size. BUT, it seams
to be happening. Therefore, I agree with Rainer; we should support it.
And so, we should do it in -sign.

BTW,
The same example also shows we NEED to allow '\n' in a logline.

Chris, can you add it to the issue list?

I think we shoul only allow \n  (1 char) as lin e terminator; each
flatform has to translate it into the correct \r\n, \n \n\r sequence
before "printing it". Then the logmsg is independent.

- -- 
ALbert Mietus
	Send prive mail to:      [email-masked]
	Send business mail to:  [email-masked]
	Don't send spam mail!

------------------------------

Date: Thu, 4 Sep 2003 18:01:40 -0400
From: Anton Okmianski <[email-masked]>
Subject: RE: Issue 8:  Length of syslog messages

I'd like to insert my 2 cents if I may.

I agree that multi-part messages are good to have in the protocol.
For two reasons:

 - messages greater than 1024 octets
 - better presentation of multi-line messages

The lines breaks fall into the latter category.  I think one would
achieve better presentation with legacy servers if messages are split
into multiple parts on line breaks.  This way every line in a log file
would have a standard header.

I think allowing line breaks brings up the whole can of worms as to
how things are stored, which syslog specifications currently shy away
from.

Also sending only \n means that I have to search contents of every
message that is provided to my syslog logger library in order to
replace platform-dependent line breaks with \n.  Seems very
UNIX-biased.

Finally, if messages are stored with extra line breaks in log files,
how does any parser  distinguish a new message from a continuation of
a previous one?  The line break is currently taken to mean just one
thing in the logs - end of message.  Multi-part messages would extend
this to mean an end of message part.  If we add another meaning, we
have to say how parsers should tell them apart.

There is one issue with not supporting line breaks though (other than
the overhead of multi-part messages).  If multi-part messages are used
for both scenarios I outlined above, then how do we distinguish one
from another.  Suppose I want to reconstruct the actual message text
of a multi-part message on the receiving end in order to send it
somewhere as one piece using a different protocol.  If multi-part
message was a result of splitting a 1500k message, then I should just
combine the multiple parts.  However, if multiple parts where there
for better presentations (because original message contained line
breaks), then I need to combine the parts with line breaks.   Is it a
big deal?

Any thoughts?

Thanks,
Anton.

> -----Original Message-----
> From: [email-masked]
> [mailto:[email-masked]]On Behalf Of Albert Mietus
> Sent: Thursday, September 04, 2003 4:33 PM
> To: Rainer Gerhards
> Cc: Albert Mietus; [email-masked]
> Subject: RE: Issue 8: Length of syslog messages
>
>
> Rainer Gerhards writes:
>  > >  > I would, however, like to extend the core syslog format to
>  > > support  > [...]  needs to send a larger message
>  > Well, I know Microsoft does not necessarily do it smartly. Many
>
>  > ------- BEING LOG ENTRY ;) -----
>  [deleted about 3k]
>  > -------- END LOG ENTRY ------
>
>  > Still objections?
>
> I think it is "silly" to make log-entries of this size.
> BUT, it seams
> to be happening. Therefore, I agree with Rainer; we should
> support it.
> And so, we should do it in -sign.
>
>
> BTW,
> The same example also shows we NEED to allow '\n' in a logline.
>
> Chris, can you add it to the issue list?
>
> I think we shoul only allow \n  (1 char) as lin e terminator; each
> flatform has to translate it into the correct \r\n, \n \n\r sequence
> before "printing it". Then the logmsg is independent.
>
>
>
>
> --
> ALbert Mietus
> 	Send prive mail to:      [email-masked]
> 	Send business mail to:  [email-masked]
> 	Don't send spam mail!
>

------------------------------

Date: Fri, 5 Sep 2003 09:15:33 +0200
From: Rainer Gerhards <[email-masked]>
Subject: RE: Issue 8:  Length of syslog messages

Anton & Albert,

Please note that by providing the sample I was focussed on the length of
the message. I agree with Anton that introducing line breaks is *not* a
good idea. As I said, we have software that forwards Windows event log
data. Let me explain a little how it deals with it - I think this is
more or less a typical example in such cases.

As a matter of fact, we do have line breaks in Windows event logs. This
is absolutely valid in the Windows world, as the event system follows a
different paradigm (and I don't want to start a discussion on the good
and evil in that - let's simply use this is a fact ;)). The event system
has also other properties, like the event ID or source.

As such, there is no one-to-one match to syslog concepts. However, the
concepts are close enough that a mapping is possible. We do this mapping
by reformatting the event log entry. While reformatting, we remove line
breaks. As Anton said, if we would leave them in, that would screw up
almost all analysis processes. In essence, they are also not very
helpful, so there is no issue in formating them. So after our formatting
we have a "plain" syslog message whom's content is Windows-specific (no
big deal).

At the collector, that message can be stored like any other syslog
message. An analysis script can process it and find one message per
line. It is even helpful in reports, because multiple lines would
potentially screw those, too. Again, in reports missing line breaks
cause no readability issue (you may want to look at one possible example
report at
http://www.mwconsole.com/en/Product/SystemStatus-15-7-2003.html - if you
look at the example report you will notice that oversize messages are
only very few).

So what is the essence of my long wording? I, too, think we should stick
with no line breaks inside the message. We will break a horrible lot of
things if we change this (btw: we have prooven this with the initial
versions of our software years back - before we filtered out the line
breaks - so my statement is based on actual experience ;)).

Rainer

------------------------------

Date: Fri, 5 Sep 2003 09:35:46 +0200
From: Rainer Gerhards <[email-masked]>
Subject: RE: Issue 8:  Length of syslog messages

Albert,

>  > >  > I would, however, like to extend the core syslog format to
>  > > support  > [...]  needs to send a larger message
>  > Well, I know Microsoft does not necessarily do it smartly. Many
>
>  > ------- BEING LOG ENTRY ;) -----
>  [deleted about 3k]
>  > -------- END LOG ENTRY ------
>
>  > Still objections?
>
> I think it is "silly" to make log-entries of this size. BUT, it seams
> to be happening. Therefore, I agree with Rainer; we should support it.
> And so, we should do it in -sign.

I think -sign already does support oversize messages ;). Let's assume we
take the route currently outlined by -international. In there, we have a
cookie-based fragmentation method. I don't have the time to create a
full, correct example, but this is basically how a 3 key message would
look (all inside MSG part). Let's assume the syntax would be

COOKIE FRAGNBR MSGFRAGMENT

So it would be:

@#intl 0 first 1k
@#intl 1 first 1k
@#intl 2 first 1k

This would be 3 syslog messages if you look at it from the -sign
perspective. As such, you can sign them as usual. Only the collector can
combine them or the reporting engine (maybe the preferred method!)
combines them. Of course, if the collector combines them, it must store
a separate raw log for offline verification - but there is little
complexity involved in doing so. And you probably need to do it anyhow,
as most currently existing reporting engines won't work on raw (and I am
not sure if it would be a good idea to make each individual reporting
engine understand the available syslog message formats...).

I expect the vast majority of message to travel within a single packet.
The way -international supports oversize messages is meant to be very
light and not to put a burden on the usual case (small messages). But it
provides a mechanism to support oversize messages in the rare cases they
appear. At least, this is how I hope it works ;)

Bottom line: I don't see any need to change -sign to support oversize
messages.

Rainer

------------------------------

Date: Fri, 5 Sep 2003 10:07:31 +0200
From: Rainer Gerhards <[email-masked]>
Subject: MSG in -sign vs. 3164

Hi WG,

this is an important issue for interoperability between -sign and the
upcoming -international. -sign redefines the MSG part of the message as
follows:

(@@@ is included to point out the important fragment)

### -sign ###
 The MSG part contains the details of the message. This has
   traditionally been a freeform message that gives some detailed
   information of the event. The MSG part of the syslog packet MUST
   contain visible (printing) characters.

   @@@@
   The code set used MUST also
   been seven-bit ASCII in an eight-bit field like that used in the PRI
   part.
   @@@@

   In this code set, the only allowable characters are the ABNF
   VCHAR values (%d33-126) and spaces (SP value %d32). Two message types
   are defined in this document. Each has unique fields within the MSG
   part and they are described below.
### end -sign ###

in contrast, 3164 said:

### 3164 ###
   The MSG part will fill the remainder of the syslog packet.  This will
   usually contain some additional information of the process that
   generated the message, and then the text of the message.  There is no
   ending delimiter to this part.  The MSG part of the syslog packet
   MUST contain visible (printing) characters.  The code set
   traditionally and most often used has also been seven-bit ASCII in an
   eight-bit field like that used in the PRI and HEADER parts.  In this
   code set, the only allowable characters are the ABNF VCHAR values
   (%d33-126) and spaces (SP value %d32).  However, no indication of the
   code set used within the MSG is required, nor is it expected.

   @@@@
   Other
   code sets MAY be used as long as the characters used in the MSG are
   exclusively visible characters and spaces similar to those described
   above.
   @@@@

   The selection of a code set used in the MSG part SHOULD be
   made with thoughts of the intended receiver.  A message containing
   characters in a code set that cannot be viewed or understood by a
   recipient will yield no information of value to an operator or
   administrator looking at it.
### end 3164 ###

The important part is that 3164 says any code set can be used (which as
of my understanding actually *would* include UTF-8, whereas -sign says
only US-ASCII is allowed.

I have to admit that I was a bit -sign focussed when I started the
discussion on the issues with using UTF-8 in -international. As of 3164,
I think this is allowed even today (but I may still cause some unaware
implemntations to fail).

If we go ahead and leave -sign with US-ASCII only, it will most probably
impossible to use -sign with -international enhanced messages (once
- -international is out).

As such, I strongly propose that -sign is changed back to what 3164
says, so that non-US-ASCII data is allowed to be present.

As I interpret 3164 now, this allows 8 bit data to be present inside the
MSG part and that in turn allows -international to define UTF-8 with no
(well, limited) worries. That would allow me to quickly go to the next
revision of -international.

Comments?

Rainer

------------------------------

Date: Fri, 5 Sep 2003 17:44:27 +0200
From: Albert Mietus <[email-masked]>
Subject: RE: Issue 8:  Length of syslog messages

Anton Okmianski writes:
 >  - better presentation of multi-line messages

 > Also sending only \n means that I have to search contents of every
 > message that is provided to my syslog logger library in order to
 > replace platform-dependent line breaks with \n.  Seems very
 > UNIX-biased.

I'm leather can't agree.

First in minor issue: '\n' is Unix. All
/Most IP protcol use it to terminate a line. And it is important to so
in a generic way. Suppose each systems uses it "own" convention. When
you have several systems, Unix, Win/PC and e.g. routers (and possoble
Mac) it becomes very hard (or impossible) to collect all lines on 1
system; and browse the log. Depending on the system which send the
line another tool should be needed.

If it should be, stick to the DOS/Windos convention. I don't mint.
However only \n (no \r) is shorter. And it is commen in the C language
anyhow.

More important:

The RFC should not forbid sending several lines over multiple
messages. However, it is "smart" At first sight, reading becomes
easier. But remember: syslog  isn't about "reading", it's about
loggging! Each "event" wil happen in 1 action, so it should be 1
message. Then, and only then, reading the (collected msg'es) is
simple! Otherwise, a multi-line can be partially recieved, or the
singe lines can be mixed with other lines Or even become out of order.

Multi-line messages, are only needed (as shown before) when "the
system" logs very-log-messages. And tranporting them in once isn't
possible.  It should not be used (I think) to easify (or try to ...)
the collected log.

But if there are other opinions, I'm willing to ...

- -- 
ALbert Mietus
	Send prive mail to:      [email-masked]
	Send business mail to:  [email-masked]
	Don't send spam mail!

------------------------------

Date: Mon, 8 Sep 2003 09:54:45 +0200
From: Rainer Gerhards <[email-masked]>
Subject: Question on 3195/COOKED - iam

Slightly off-topic... I am implementing COOKED (so far device only). The
iam element contains the "fqdn" and "ip" parameters. Let's assume a
multihomed device:

Generic device name: rtr.example.net
Interface 1: if1.example.com  (10.0.1.1)
Interface 2: if2.example.com  (10.0.2.1)

Only interface 1 & 2 will resolve via reverse DNS.

I assume that I put the generic device name into the fqdn as well as
obviously the interface used into th ip. So the "iam" may look like
this:

<iam fqdn='rtr.example.net' ip='10.0.1.1' type='device'/>

This also means that the collector can/SHOULD verify the IP address
given in ip, but MUST NOT verify the fqdn, at least not via revese DNS
(I am assuming NO tuning profiles are used in this stage - I am aware it
SHOULD verify based on authentication information if present).

Is my understanding correct?

Any help is deeply appreciated.

Rainer

------------------------------

Date: Tue, 9 Sep 2003 10:07:20 +0200
From: Rainer Gerhards <[email-masked]>
Subject: RFC 3195 COOKED / relay

Hi WG,

this is again a request for some assistance/clarification.

I have a situation where an application can act as both a relay as well
as a device in parallel. For example, it accepts incoming syslog
messages and can relay them, but it also generates messages itself, e.g.
by pulling them from the Windows event logs.

My question is now about the role in the <iam> element.

I assume I can handle this situation in two ways:

#1 Two BEEP channels
I open two channels to the remote peer. With one, I say <iam
type='device>. With the other one, I say <iam type='relay'>. Depending
on what is to be forwarded I use either the device or relay channel.

#2 One BEEP channel, multiple <iam>s
I open just one channel. Whenever I need to send something as a device,
I issue a <iam type='device'> before sending the <entry> (if not already
in "device mode"). Whenever I need to relay something, I issue an <iam
type='relay'> before the entry (if not already in "relay mode").

I think both approaches are valid, but obviously #1 will be
less-bandwidth-intense.

Am I right with this? I am asking because there are so few
implementations so far so you need to do "theoretical interop testing"
;)

Any advise is appreciated.

Rainer

------------------------------

Date: Tue, 9 Sep 2003 18:01:54 +0200
From: Rainer Gerhards <[email-masked]>
Subject: -international - support for oversize messages

Hi WG,

I am preparing a large edit on -international which will include all
those good comments into the draft. I will probably raise a number of
questions just so that I can verify my understanding of the discussion.
This is the first of those messages.

I think we have consensus that there are some (rare) occasions where
oversized messages need to be transmitted via a standard-compliant way,
thus we need to define something how this can be done. On the other
hand, the vast majority of messages will NOT be oversized, so we should
add as little overhead as possible to the ID to support this. As a
general guideline, the burden should be placed on those applications
that need to deal with oversized messages while all others should not
need to pay attention to it.

syslog-international-00 introduced a very rough way to implement this.
Here is a quote:

####
   MORE and SEQNO provide support for syslog messages larger than the
   allowed syslog packet size. This is introduced to allow transmittal
   of "oversized" message, which may be the result of some character
   sets and encodings. These messages will be fragmented by the syslog
   client and reconstructed by the collector. Relays will pass them
   through unmodifed.

   Message fragmentation MAY be used if the underlying transport
   provides reliable and in-order delivery (for example RFC 3195 [8]).
   It the underlying transport is unreliable or its reliability is not
   known, fragmentation MUST NOT be used.

   More specifies wether this is the final fragment of the message or
   not. An asterisk ("*") means that at least one more fragement will
   follow. A period (".") means that this is the final (or only)
   fragment.

   SEQNO specifies the sequence number of fragments. It MUST start by 0
   for the first fragment and MUST be incremented by 1 for each
   following fragment. SEQNO MUST restart at 0 for each new full
   message. A new full message begins after the last message that had
   "." in MORE.

   If fragementation is not used, all messages contain ". 0" as the MORE
   SEQNO sequence.
####

There are a number of issues, especially when it comes to out of order
delivery. I would like to revise this and would appreciate feedback on
the following suggestion:

The modified ABNF would look like this (just the fragementation-related
parts):

   HDR-i18n  = COOKIE SP ENCODING SP CHARSET SP LANGUAGE SP FRAG-INFO SP
   FRAG-INFO = "." /
               ("*" SP REBOOTID SP MSGNBR SP TOTALLEN SP INDEX SP
FRAGLEN)
   REBOOTID  = 1*10DIGIT ; never repeat or decrease
   TOTALLEN  = 1*7DIGIT
   INDEX     = 1*7DIGIT
   FRAGLEN   = 1*3DIGIT

FRAG-INFO is the important part I am talking about in this mail. It will
describe if a message is fragmented.

In the typical case (normal size message), it would just be ".". So for
most applications, nothing would need to be changed. IF we have a syslog
implementation that intends to support fragemented oversize package,
THEN (and only then) it needs to provide the following fields. As you
will probably see, I have borrowed from -sign, which hopefully will make
it easier to implement:

REBOOTID is the exact same reboot session ID as in -sign. An aplication
implementing both -sign and -international SHOULD use the same value as
in the -sign messages.

TOTALLEN is the total size of the message in bytes. It probably is
allowed to grow a bit large, but I think it doesn't hurt to support
idiot-size messages. If there is violent opposition, I would like to
limit this to 1 MB, but not less. I have seen *really* large events >
100KB once in a year and would like to see these covered (comparable to
- -sign 4.3.5). BUT... I am aware of the fact that a malicious user could
use this to create a DoS on collectors / log reviewers / log analysers
that re-assemble the full oversize message.

INDEX is the position, in bytes, into the oversize message where this
fragment starts (as is e.g. the fragment index in -sign 4.3.6)

FRAGLEN is the length, in bytes, of this message. Byte counting starts
with the byte after the HDR-i18n (that is the byte AFTER the SP after
FRAGLEN). It can be a maximum of 3 digits because, after subtracting the
overhead of the headers, no wellformed message (fragement) can contain
more than 999 bytes.

If an oversize message is fragmented, the individual fragments are sent
as syslog messages in their own right, the fragmentation is just
indicated by the proper HDR-i18n fields. All fragments MUST be send with
the same TAG and HOSTNAME and SHOULD be send over the same interface.
The TIMESTAMP MUST be set to either the timestamp of the first fragement
or be increasing (never decreasing). The actual time the message
fragment was scheduled for transmittal SHOULD be placed inside the
TIMESTAMP of each fragment.

A device supporting oversize messages SHOULD place the most important
facts early in the message, and SHOULD try to place vital information
into the first fragment. The reason for this is that a collector or
offline analyzer may decide to discard the message after some supported
byte count.

A relay MUST forward the individual (fragmented) messages unaltered. A
collector SHOULD store them as separate entities. This is necessary to
verify -sign signatures. Either the collector or an offline log review
tool can re-assemble the messages. It SHOULD do so before displaying
them to a human or using them for an (automatted) analysis process.

If fragementation is used, messages SHOULD be sent over a reliable
transport like RFC 3195. No redundancy is provided for unreliable links,
as this could potentially eat up a lot of bandwidth.

I think this approach fits the goals. Most importantly, it allows for
oversize messages without the need to modify any existing applications,
which also means oversize messages can travel over existing relays, be
signed with -sign (the individual fragment will be signed) and also be
used over RFC 3195.

Is there any opposition against this approach? I am specifically
interested on feedback on:

- - if this is considered overkill compared to the initial approach
- - if the approach with using just "." for normal-size messages rings
some bells
- - if someone sees some interoperability issues with existing syslog
- - if there should be some redundancy be added for unreliable transport
- - as a side note, should the reboot session ID be limited
  to the 32 bit unsigned range or left as in -sign?
- - if we should include some wording that a collector MUST support
  e.g. a message size of 5 KB but MAY discard anything beyond
  this. That would resolve the issue with the potential DoS (at least
  most parts of it...).

I appreciate all feedback!
Rainer

------------------------------

Date: Tue, 9 Sep 2003 10:17:23 -0700 (PDT)
From: Chris Lonvick <[email-masked]>
Subject: [BEEPwg] ANNOUNCE: Net::BeepLite 0.01 (fwd)

Hi,

This seems appropriate to forward to this group.

Thanks,
Chris

- ---------- Forwarded message ----------
Date: Mon, 8 Sep 2003 14:18:26 -0400
From: David Blacka <[email-masked]>
To: [email-masked]
Subject: [BEEPwg] ANNOUNCE: Net::BeepLite 0.01

I have released Net::BeepLite 0.01.  From the README:

  Net::BeepLite is a lightweight implementation of a BEEP (RFC 3080, RFC
  3081) client/server framework.  This package is intended for
  implementing fairly simple BEEP clients and servers.  It does not
  attempt to be the last word on using BEEP in Perl.  Or to put this
  another way, this package does not make an attempt to support every
  feature or mode of operation possible within the BEEP specification.

  It does not use threading, and thus cannot directly handle multiple
  ansynchronous channels.  It does not directly support peer-to-peer
  operations.  Or at least not yet.

Even though this is a 0.01 release, the code is pretty complete and
somewhat tested.  For now, the main webpage for it is
<http://iris.verisignlabs.com> (this is shared by a few other projects
supporting IRIS, which is a proposal in the IETF crisp working group).  It
can be fetched from

<http://iris.verisignlabs.com/software/netbeeplite/Net-BeepLite-0.01.tar.gz>.

I have uploaded it to PAUSE, so hopefully it will be on CPAN at some point
in the relatively near future.

- -- 
David Blacka    <[email-masked]>
Sr. Engineer    Verisign Applied Research

_______________________________________________
BEEPwg mailing list
[email-masked]
http://lists.beepcore.org/mailman/listinfo/beepwg

------------------------------

Date: Tue, 09 Sep 2003 12:09:01 -0700
From: Darren New <[email-masked]>
Subject: Re: RFC 3195 COOKED / relay

Rainer Gerhards wrote:
> Hi WG,
>
> this is again a request for some assistance/clarification.
>
> I have a situation where an application can act as both a relay as well
> as a device in parallel. For example, it accepts incoming syslog
> messages and can relay them, but it also generates messages itself, e.g.
> by pulling them from the Windows event logs.
>
> My question is now about the role in the <iam> element.
>
> I assume I can handle this situation in two ways:
>
> #1 Two BEEP channels
> I open two channels to the remote peer. With one, I say <iam
> type='device>. With the other one, I say <iam type='relay'>. Depending
> on what is to be forwarded I use either the device or relay channel.

That would work.

> #2 One BEEP channel, multiple <iam>s
> I open just one channel. Whenever I need to send something as a device,
> I issue a <iam type='device'> before sending the <entry> (if not already
> in "device mode"). Whenever I need to relay something, I issue an <iam
> type='relay'> before the entry (if not already in "relay mode").
>
> I think both approaches are valid, but obviously #1 will be
> less-bandwidth-intense.

That would work too.

There's a third possibility, which is to use the pathID to determine
where the message originated from. That is, you *are* a relay, you just
happen to generate some messages yourself, as well. This approach has
the advantage of having only one <iam> to match up with the SASL login.
(I.e., if your SASL login included the type information, neither of your
above approaches would work.)

I think the choice is in part dependant on what you "mean" in some
sense. Are you simply relaying messages locally, or are you actually
generating them yourself? That is, I could see someone using *both* #1
and #2, with one channel for messages like "the relay syslog process
just had a configuration change", and another channel for messages like
"the machine on which the relay syslog process is running just lost a
disk drive".

- -- 
Darren New, San Diego CA USA (PST)
Don't take home left-over tripe.
   It'll just digest itself before
     you get around to eating it.

------------------------------

Date: Tue, 9 Sep 2003 15:52:25 -0400
From: Anton Okmianski <[email-masked]>
Subject: RE: -international - support for oversize messages

Rainer:

I think...

1. Fragmented messages should be supported over any transport
including legacy UDP syslog.  If you lose some fragments over
unreliable transport, it is expected. A way to determine that this
happened would nice to have. I think even your original proposal
addresses this with SEQNUM.

2. In the new proposal, why do we need to send message length? The "."
indicates the last message, right?  Looks like redundant framing
methods.

3. I think it is better to talk in terms of message parts (fragments
count) instead of length.  It is easier for implementers and for human
readers.  So, I like SEQNUM stuff.

4. Minor thing... Can we start any counts at 1 and not 0?  It matches
English better -- it is the *first* fragment, not fragment zero. Why
reinforce that barrier between CS and normal English?  If there is no
any IETF conventions that dictate otherwise, I would prefer counts to
start 1.  (No, I am not a VisualBasic programmer. :))

5. I am concerned about the scope of reboot id and message sequence
numbers?  Are they per process?  I think we certainly have to support
multiple processes firing syslog messages remotely directly without a
central daemon. In that case, each has to have its own reboot
(restart?) id and message numbers?  Then the process MUST be
identified in the message, right?  There were some discussions on this
which I did not quite follow.

6. How do we identify the message to which a given fragment belongs?
I can have two processes on the same host originate fragmented
messages concurrently.  Do all fragments have the MSGNUM of the first
message?  There has to be some reference in the fragments to the
original message.  Providing MSGNUM of the first fragment in all
subsequent fragments could solve this.

7. I would propose that all fragments of the message should have the
same timestamp.  Logically, the timestamp should be close to when the
event happened, not when it was sent, right?

8. The message length limit is a tough one. I am leaning towards not
limiting the length of messages unless we absolutely have to. The
receiver should always make sure that it does not attempt to aggregate
fragments of messages over what its memory limit (or hard limit)
allows it. It is general out of memory error handling which can happen
for many reasons, right?  Even if we say that message should not
exceed X MB, it is likely we would not recommend that implementations
just discard messages over this size, but rather default to a
different behavior, right?  Just like UDP syslog receivers probably
won't discard messages over 1024 bytes.

9. Does it make sense to put message fragmentation into a separate RFC
from international?  The process-unique sequence numbers, for example,
can be used in many contexts.

10. Do I think all this is an overkill?  Well, what's the alternative?
In the interim, at Cisco we had to come up with a pretty similar
proprietary scheme.

Thanks,
Anton.

> -----Original Message-----
> From: [email-masked]
> [mailto:[email-masked]]On Behalf Of Rainer Gerhards
> Sent: Tuesday, September 09, 2003 12:02 PM
> To: [email-masked]
> Subject: -international - support for oversize messages
>
>
> Hi WG,
>
> I am preparing a large edit on -international which will include all
> those good comments into the draft. I will probably raise a
> number of
> questions just so that I can verify my understanding of the
> discussion.
> This is the first of those messages.
>
> I think we have consensus that there are some (rare) occasions where
> oversized messages need to be transmitted via a
> standard-compliant way,
> thus we need to define something how this can be done. On the other
> hand, the vast majority of messages will NOT be oversized,
> so we should
> add as little overhead as possible to the ID to support this. As a
> general guideline, the burden should be placed on those applications
> that need to deal with oversized messages while all others
> should not
> need to pay attention to it.
>
> syslog-international-00 introduced a very rough way to
> implement this.
> Here is a quote:
>
> ####
>    MORE and SEQNO provide support for syslog messages
> larger than the
>    allowed syslog packet size. This is introduced to allow
> transmittal
>    of "oversized" message, which may be the result of some character
>    sets and encodings. These messages will be fragmented by
> the syslog
>    client and reconstructed by the collector. Relays will pass them
>    through unmodifed.
>
>    Message fragmentation MAY be used if the underlying transport
>    provides reliable and in-order delivery (for example RFC
> 3195 [8]).
>    It the underlying transport is unreliable or its
> reliability is not
>    known, fragmentation MUST NOT be used.
>
>    More specifies wether this is the final fragment of the
> message or
>    not. An asterisk ("*") means that at least one more
> fragement will
>    follow. A period (".") means that this is the final (or only)
>    fragment.
>
>    SEQNO specifies the sequence number of fragments. It
> MUST start by 0
>    for the first fragment and MUST be incremented by 1 for each
>    following fragment. SEQNO MUST restart at 0 for each new full
>    message. A new full message begins after the last
> message that had
>    "." in MORE.
>
>    If fragementation is not used, all messages contain ".
> 0" as the MORE
>    SEQNO sequence.
> ####
>
> There are a number of issues, especially when it comes to
> out of order
> delivery. I would like to revise this and would appreciate
> feedback on
> the following suggestion:
>
> The modified ABNF would look like this (just the
> fragementation-related
> parts):
>
>    HDR-i18n  = COOKIE SP ENCODING SP CHARSET SP LANGUAGE SP
> FRAG-INFO SP
>    FRAG-INFO = "." /
>                ("*" SP REBOOTID SP MSGNBR SP TOTALLEN SP INDEX SP
> FRAGLEN)
>    REBOOTID  = 1*10DIGIT ; never repeat or decrease
>    TOTALLEN  = 1*7DIGIT
>    INDEX     = 1*7DIGIT
>    FRAGLEN   = 1*3DIGIT
>
> FRAG-INFO is the important part I am talking about in this
> mail. It will
> describe if a message is fragmented.
>
> In the typical case (normal size message), it would just be
> ".". So for
> most applications, nothing would need to be changed. IF we
> have a syslog
> implementation that intends to support fragemented oversize package,
> THEN (and only then) it needs to provide the following
> fields. As you
> will probably see, I have borrowed from -sign, which
> hopefully will make
> it easier to implement:
>
> REBOOTID is the exact same reboot session ID as in -sign.
> An aplication
> implementing both -sign and -international SHOULD use the
> same value as
> in the -sign messages.
>
> TOTALLEN is the total size of the message in bytes. It probably is
> allowed to grow a bit large, but I think it doesn't hurt to support
> idiot-size messages. If there is violent opposition, I would like to
> limit this to 1 MB, but not less. I have seen *really*
> large events >
> 100KB once in a year and would like to see these covered
> (comparable to
> -sign 4.3.5). BUT... I am aware of the fact that a
> malicious user could
> use this to create a DoS on collectors / log reviewers /
> log analysers
> that re-assemble the full oversize message.
>
> INDEX is the position, in bytes, into the oversize message
> where this
> fragment starts (as is e.g. the fragment index in -sign 4.3.6)
>
> FRAGLEN is the length, in bytes, of this message. Byte
> counting starts
> with the byte after the HDR-i18n (that is the byte AFTER
> the SP after
> FRAGLEN). It can be a maximum of 3 digits because, after
> subtracting the
> overhead of the headers, no wellformed message (fragement)
> can contain
> more than 999 bytes.
>
> If an oversize message is fragmented, the individual
> fragments are sent
> as syslog messages in their own right, the fragmentation is just
> indicated by the proper HDR-i18n fields. All fragments MUST
> be send with
> the same TAG and HOSTNAME and SHOULD be send over the same
> interface.
> The TIMESTAMP MUST be set to either the timestamp of the
> first fragement
> or be increasing (never decreasing). The actual time the message
> fragment was scheduled for transmittal SHOULD be placed inside the
> TIMESTAMP of each fragment.
>
> A device supporting oversize messages SHOULD place the most
> important
> facts early in the message, and SHOULD try to place vital
> information
> into the first fragment. The reason for this is that a collector or
> offline analyzer may decide to discard the message after
> some supported
> byte count.
>
> A relay MUST forward the individual (fragmented) messages
> unaltered. A
> collector SHOULD store them as separate entities. This is
> necessary to
> verify -sign signatures. Either the collector or an offline
> log review
> tool can re-assemble the messages. It SHOULD do so before displaying
> them to a human or using them for an (automatted) analysis process.
>
> If fragementation is used, messages SHOULD be sent over a reliable
> transport like RFC 3195. No redundancy is provided for
> unreliable links,
> as this could potentially eat up a lot of bandwidth.
>
> I think this approach fits the goals. Most importantly, it
> allows for
> oversize messages without the need to modify any existing
> applications,
> which also means oversize messages can travel over existing
> relays, be
> signed with -sign (the individual fragment will be signed)
> and also be
> used over RFC 3195.
>
> Is there any opposition against this approach? I am specifically
> interested on feedback on:
>
> - if this is considered overkill compared to the initial approach
> - if the approach with using just "." for normal-size messages rings
> some bells
> - if someone sees some interoperability issues with existing syslog
> - if there should be some redundancy be added for
> unreliable transport
> - as a side note, should the reboot session ID be limited
>   to the 32 bit unsigned range or left as in -sign?
> - if we should include some wording that a collector MUST support
>   e.g. a message size of 5 KB but MAY discard anything beyond
>   this. That would resolve the issue with the potential DoS
> (at least
>   most parts of it...).
>
> I appreciate all feedback!
> Rainer
>
>

------------------------------

Date: Wed, 10 Sep 2003 11:09:08 +0200
From: Rainer Gerhards <[email-masked]>
Subject: RE: -international - support for oversize messages

Anton,

> 1. Fragmented messages should be supported over any transport
> including legacy UDP syslog.  If you lose some fragments over
> unreliable transport, it is expected. A way to determine that this
> happened would nice to have. I think even your original proposal
> addresses this with SEQNUM.

Actually, UDP was one of the main reasons I went into the more complex
approach.

With -international-00, we *just* have SEQNUM. Remember that UDP is not
just unreliable, but frames can also arrive out of sequence. Let's
assume you send:

SEQNO 0 msg1 frag1
SEQNO 1 msg1 frag2
SEQNO 0 msg2 frag1
SEQNO 1 msg2 frag2

And this is received in this sequence:

SEQNO 0 msg1 frag1
SEQNO 1 msg2 frag2
SEQNO 1 msg1 frag2
SEQNO 0 msg2 frag1

You see the issue? There is *no way* to determine which fragement
belongs to which message. This is why I limited fragementation to
reliable transport in -00, but as you said, this is a bad idea. So I
looked into a way to do it somewhat more reliably.

Bottomline: a sequence counter alone is not sufficient, we need to have
a message id that uniquely identifies a message. That ID must be
transmitted with each message and then we can use a sequence number.

Let's extend the example above.

We now send:

ID 1 SEQNO 0 msg1 frag1
ID 1 SEQNO 1 msg1 frag2
ID 2 SEQNO 0 msg2 frag1
ID 2 SEQNO 1 msg2 frag2

The collector still receives:
ID 1 SEQNO 0 msg1 frag1
ID 2 SEQNO 1 msg2 frag2
ID 1 SEQNO 1 msg1 frag2
ID 2 SEQNO 0 msg2 frag1

It now can assign the right fragment to the right message.

But now, again, we need to look at how unique the ID is. If it is
counted up from 0 (or 1;)) each time the engine starts, there is a
slight chance of confusion. This will also facilitate reply attacks.
This is where I turned to -sign. It has the exact same issue when
transfering larger blocks, so it looks a little tempting to me to use
what is specified there. I see two advantages in doing so:

#1 it is already been discussed, so there is some good wisdom in it
#2 if you implement both -sign and -international, you will already have
the
   code at hand (and I assume over time both will be implemented
   hopefully frequently;))
>
> 2. In the new proposal, why do we need to send message length? The "."
> indicates the last message, right?  Looks like redundant framing
> methods.

That is a good point. But let's look at the fineprint...

I introduced it because I borrowed from -sign. The difference, though,
is that I don't like to see the "fragmenation fields" as mandatory ones.
So "." does no longer mean it is the last fragment but it means it *is*
fragmented. I definitely need to clear up the text for this (if we stick
with this approach...). Right now, it is just in the ABNF. If you use
".", you MUST NOT use the fragmentation fields. So the overall size in
the frag fields is what effectively tells you when a message is
finished. The "."/"*" tells you if the message is fragmented or not.

Any more comments anyone?

> 3. I think it is better to talk in terms of message parts (fragments
> count) instead of length.  It is easier for implementers and for human
> readers.  So, I like SEQNUM stuff.

I initially thought so and this is one reason I asked for comments ;).
While thinking on how this whole thing could be implemented, I came to
the conclusion that the code most probably already knows the overall
size of the message. But it does not know in how many fragments it will
fit (ok, a quick computation will give you this). Besides that, the size
has the advantage that I can do some verification on the messages so
that I can detect minor modifications done by relays along the path.

Regarding human redability - I am right now implementing BEEP for syslog
purposes and have found that the byte counts present in the BEEP headers
are quite easy to follow ones you know which size is in which field.
But, agreed, a plain number is obviously easier... ;)

> 4. Minor thing... Can we start any counts at 1 and not 0?  It matches
> English better -- it is the *first* fragment, not fragment zero. Why
> reinforce that barrier between CS and normal English?  If there is no
> any IETF conventions that dictate otherwise, I would prefer counts to
> start 1.  (No, I am not a VisualBasic programmer. :))

Actually, it got in out of habit. I also know a lot of RFCs do it. I
don't think there is a RFC that requests it. I will try to find one. If
anybody should have a pointer, please post it. All in all, I do not
object starting at 1 and the reasoning sounds good to me.

Would anyone like to stay with 0 as the initial number?

> 5. I am concerned about the scope of reboot id and message sequence
> numbers?  Are they per process?  I think we certainly have to support
> multiple processes firing syslog messages remotely directly without a
> central daemon. In that case, each has to have its own reboot
> (restart?) id and message numbers?  Then the process MUST be
> identified in the message, right?  There were some discussions on this
> which I did not quite follow.

Yes, this is indeed an issue. Let me say it this way: we must find a
solution for coding -sign anyhow. So I would apply this solution to
- -international, too. I hope I can implement sign in the weeks to come
(but no promise, vacation season is over and it gets busier... ;)) If I
or others find it hard to adopt to such a scheme, we may need to change
something in -international. But as -sign is quite mature and
- -international is in its initial stage, I would like to defer this
discussion a little.

One approach is to use the second the process started as the reboot
session id. This will work for all processes, except those that start
quite rapidly. There is also a slight chance of two processes starting
in the same second. This could be tackeled with a small (0..9) second
random delay on startup.

I think I will implement this algorithm in our solutions as we have the
exact needs that you describe - multiple processes sending
independently. I have to admit that I am still looking for a good
solution for fast, frequently executing processes. So far, the only
thing I have on my mind is to persist the reboot session id somewhere,
but this opens up Pandorra's box [just think of the implications of an
OS reinstall]... :-(

> 6. How do we identify the message to which a given fragment belongs?
> I can have two processes on the same host originate fragmented
> messages concurrently.  Do all fragments have the MSGNUM of the first
> message?  There has to be some reference in the fragments to the
> original message.  Providing MSGNUM of the first fragment in all
> subsequent fragments could solve this.

Oops... I just re-read my message and I see I actually didn't describe
the MSGNBR field from the ABNF... Yes, this is exactly what you are
pointing out. Replace your MSGNUM with MSGNBR in your wording (I like
your name more) and past this into my description...

> 7. I would propose that all fragments of the message should have the
> same timestamp.  Logically, the timestamp should be close to when the
> event happened, not when it was sent, right?

This is why I said the SHOULD be done. However, I can see some reasoning
that an implementation would like to send an updated timestamp with the
later fragments. They may want to do this to indicate how long
processing take, they may even need to do this because this would allow
a quick implementation.

I don't see anything terribly wrong with  the later and thus would not
like to outrule it. What I - at least have tried to - outrule is that
time moves backward.

What does the rest of the group think. Should using the same timestamp
stay a SHOULD or become a MUST?

> 8. The message length limit is a tough one. I am leaning towards not
> limiting the length of messages unless we absolutely have to. The
> receiver should always make sure that it does not attempt to aggregate
> fragments of messages over what its memory limit (or hard limit)
> allows it. It is general out of memory error handling which can happen
> for many reasons, right?  Even if we say that message should not
> exceed X MB, it is likely we would not recommend that implementations
> just discard messages over this size, but rather default to a
> different behavior, right?  Just like UDP syslog receivers probably
> won't discard messages over 1024 bytes.

Actually, I am more or less of the same point of view. I though allowing
9,999,999 bytes would be sufficient, but, yes, this may turn out to be a
false impression. I just like to place some limits on the fields, as no
limits lead to less robust code and thus easier exploitation. I can move
up this limit if there is no opposition, but I would definitely like to
limit it to the 32bit unsigned range. It is very hard to think of
anybody sending syslog message of that size. And going above 32 bit is
probably calling for coding errors. Not limiting this field at all will
definitely cause big troubles.

For example, I suspect that we will see some interop issue with -sign
code as soon as the first client uses a reboot session id beyond the 32
bit scope. When I implement -sign, I already have identified a module to
do 64 bit arithmetic for machines / compilers who do not support this
natively (it also opens up a box of "portability worms" ;)). I would
like remove this error cause from -international (but it stays inside
the reboot session id, if we carry it over from -sign - this is why I
asked if reboot session id should be limited to 32 bit unsigned range).

> 9. Does it make sense to put message fragmentation into a separate RFC
> from international?  The process-unique sequence numbers, for example,
> can be used in many contexts.

As you know from my previous posts, I personally would like to see this.
But I have the impression that there is not enough support for it at the
time being. So I am trying to bring this into a separate section of
- -international, which then can be referred to by later IDs/RFCs without
the need for a full implementation of -international. I hope this suits
all well... and is doable under the current process.

> 10. Do I think all this is an overkill?  Well, what's the alternative?
> In the interim, at Cisco we had to come up with a pretty similar
> proprietary scheme.

Well, my point is I would like to see if there is concensus which could
lead to implementation. You may write a really smart and bright spec (I
am sure mine isn't ;)) - but that doesn't help if nobody implements it.
So my experience is that it is often better to do what you think is the
second-best - if that will lead to adaption.

In this spirit: looking forward to another round of comments ;)

Rainer

------------------------------

Date: Wed, 10 Sep 2003 17:07:09 +0200
From: Rainer Gerhards <[email-masked]>
Subject: RE: RFC 3195 COOKED / relay

Darren,

> There's a third possibility, which is to use the pathID to determine
> where the message originated from. That is, you *are* a
> relay, you just
> happen to generate some messages yourself, as well. This approach has
> the advantage of having only one <iam> to match up with the
> SASL login.
> (I.e., if your SASL login included the type information,
> neither of your
> above approaches would work.)

Oops... good point (I have to admit I haven't thought about this because
I am not yet ready to implement these... ;)).

> I think the choice is in part dependant on what you "mean" in some
> sense. Are you simply relaying messages locally, or are you actually
> generating them yourself? That is, I could see someone using *both* #1
> and #2, with one channel for messages like "the relay syslog process
> just had a configuration change", and another channel for
> messages like
> "the machine on which the relay syslog process is running just lost a
> disk drive".

Actually, I am thinking on the later case. We have a solution that does
a lot of monitoring which includes a syslog subsystem. So it can relay
syslog messages but it also is a "device" in the spirit of syslog, that
is it e.g. creates messages based on text file data and Windows event
log data picked up as well as e.g. detection of went-down remote
services and such. So that piece of software acutally *is* both a relay
as well as a device.

Given the SASL implications, it looks like I even should create two
SESSIONS to the remote peer (option #4 ;)), one saying it is a device
and one saying it is a relay. Would this work with the SASL auth? (I
guess this is a quick yes/no for you, so I do not yet read the specs...)

I guess option #5 would be to have the device part of my application
forward messages to the relay part. This would effectively add one path
element to the overall path and I have the impression that this is
probably the worst way to do it.

Any more comments?

Rainer

------------------------------

Date: Wed, 10 Sep 2003 08:44:39 -0700
From: Darren New <[email-masked]>
Subject: Re: RFC 3195 COOKED / relay

Rainer Gerhards wrote:
> Given the SASL implications, it looks like I even should create two
> SESSIONS to the remote peer (option #4 ;)), one saying it is a device
> and one saying it is a relay. Would this work with the SASL auth? (I
> guess this is a quick yes/no for you, so I do not yet read the specs...)

Well... it depends. :-)

If you log into that SASL as "xyz.example.com", then one session with
one or two channels is fine. If you log into that SASL as
"xyz.example.com/relay" then yes, you'd likely need two separate
sessions if you also wanted to act as a device. It's more an interop
thing than a definitive yes/no answer.

Having multiple channels means you wind up with a more complicated BEEP
library underlying your implementation. (Allowing only one active
channel at a time simplifies things somewhat, in terms of window
management and buffering and such.) But my personal preference (i.e.,
how I was thinking of it when I wrote the initial spec) would be to use
a separate channel for each device or relay, or to use separate path
elements. It doesn't really make much sense to keep flopping who <iam>
back and forth over one channel. Indeed, I no longer even recall why it
was felt desirable to allow multiple successful <iam> messages on one
channel.

Obviously, if you're also supporting syslog-raw, you'll need multiple
channels as well, and you might potentially want a channel (as yet
undesigned) for configuration of the relay itself, specifying where
different types of packets go, etc.

- -- 
Darren New, San Diego CA USA (PST)
Don't take home left-over tripe.
   It'll just digest itself before
     you get around to eating it.

------------------------------

Date: Thu, 11 Sep 2003 14:09:33 +0200
From: Rainer Gerhards <[email-masked]>
Subject: RE: Issue 2: TAG Field Definition

While re-reading -sign, some clarification...

Neither -sign-12 nor my proposed text below specifies US-ASCII as
mandatory. As such, any character set would be appropriate. The encoded
characters would just need to be visible and not be SP or colon. As
such, UTF-8 would be perfectly fine in this field. This is also
consistent with RFC 3164.

I will move this assumption into -international-01 and explicitely
define it.

Any opposition against this?

Rainer

> -----Original Message-----
> From: Chris Lonvick [mailto:[email-masked]]
> Sent: Wednesday, September 03, 2003 7:55 PM
> To: [email-masked]
> Subject: Issue 2: TAG Field Definition
>
>
> Issue 2: TAG Field Definition
> http://www.employees.org/~lonvick/draft-ietf-syslog-sign-12.ht
> ml#HEADER
>
> From Archive:
> http://www.mail-archive.com/syslog-sec%40employees.org/msg01224.html
> http://www.mail-archive.com/syslog-sec%40employees.org/msg01234.html
> http://www.mail-archive.com/syslog-sec%40employees.org/msg01222.html
>
> Rainer has proposed the following text:
> """
>    The TAG is a string of visible (printing) characters excluding SP,
>    that MUST NOT exceed 32 characters in length.
>
>    The first occurrence of a colon (":") or SP " " character
> terminates
>    the TAG field. Generally, the TAG contains the name of the process
>    that generated the message. It may OPTIONALLY contain additional
>    information such as the numerical process ID of that process bound
>    within square brackets ("[" and "]"). A colon MUST be the last
>    character in this field.
>
>    To be consistent with the format described in RFC 3164, a space
>    character need not follow the colon in normal syslog packets.
> """
>
> However, anyone trying to convey information of
> "Myproc[PID,Threadid]:"
> may have a problem with something like
>
>   syslog[12345,C:\usr\sbin\cron]:
>
> Albert suggests just having "syslog" in the cert/sig-block messages
> but that seems to be inconsistent with the possible formats of the
> normal syslog messages.
>
> Can anyone offer a suggestion to resolve this?
>
>
>

------------------------------

Date: Thu, 11 Sep 2003 14:29:30 +0200
From: Rainer Gerhards <[email-masked]>
Subject: RE: Issue 2: TAG Field Definition

... and some more comment:

> However, anyone trying to convey information of
> "Myproc[PID,Threadid]:"
> may have a problem with something like
>
>   syslog[12345,C:\usr\sbin\cron]:

I would place the burden on the emitor. The colon must be disallowed.
The emitor could use something like

    syslog[12345,C\\usr\sbin\cron]:

I think it would take too much to support this. I guess my primary
intention in showing this was that the implemntor should be careful.

>
> Albert suggests just having "syslog" in the cert/sig-block messages
> but that seems to be inconsistent with the possible formats of the
> normal syslog messages.

I think I don't fully understand this here. "syslog" would be fine,
wouldn't it? Of course, it must be terminated with either SP or a colon,
but other than that, I am currently seeing no issue.

Rainer

------------------------------

Date: Fri, 12 Sep 2003 07:33:51 -0700 (PDT)
From: Chris Lonvick <[email-masked]>
Subject: Issue 5:  First Message Number - Still needs review

Issue 5:  First Message Number
http://www.employees.org/~lonvick/draft-ietf-syslog-sign-12.html#firstmsg

From Archive:
http://www.mail-archive.com/syslog-sec%40employees.org/msg01228.html

Similar to Issue 4, should the value of "0" or "1" be used as the
lowest available value?  Albert again suggests "0".  Any disagreement
to that?

STATUS: No direct responses to this issue but Anton Okmianski suggests
  using "1" as the start for ordinal values.

From the Archive (see point 4)
http://www.mail-archive.com/syslog-sec%40employees.org/msg01265.html

------------------------------

Date: Fri, 12 Sep 2003 07:33:09 -0700 (PDT)
From: Chris Lonvick <[email-masked]>
Subject: Issue 2:  TAG Field Definition - Still needs review

Issue 2:  TAG Field Definition
http://www.employees.org/~lonvick/draft-ietf-syslog-sign-12.html#HEADER

From Archive:
http://www.mail-archive.com/syslog-sec%40employees.org/msg01224.html
http://www.mail-archive.com/syslog-sec%40employees.org/msg01234.html
http://www.mail-archive.com/syslog-sec%40employees.org/msg01222.html

STATUS: Proposals have been received and are being considered.

from the Archive:
http://www.mail-archive.com/syslog-sec%40employees.org/msg01269.html
http://www.mail-archive.com/syslog-sec%40employees.org/msg01270.html

------------------------------

Date: Fri, 12 Sep 2003 07:26:09 -0700 (PDT)
From: Chris Lonvick <[email-masked]>
Subject: Issue 7:  TPBL Length - RESOLVED

Issue 7:  TPBL Length
http://www.employees.org/~lonvick/draft-ietf-syslog-sign-12.html#tpbl

From Archive:
http://www.mail-archive.com/syslog-sec%40employees.org/msg01232.html

The Total Payload Block Length parameter is currently defined as
"8 characters" but perhaps that should be "1-8 characters" which is
consistent with other fields.  Comments?

STATUS: Resolved as "1-8 characters"

From the Archive:
http://www.mail-archive.com/syslog-sec%40employees.org/msg01247.html

------------------------------

Date: Fri, 12 Sep 2003 06:41:07 -0700 (PDT)
From: Chris Lonvick <[email-masked]>
Subject: Updated Issues

Hi Folks,

I'm going to send out updates on where I believe we are on the Issues.
I'll keep an updated file here:
http://www.employees.org/~lonvick/issues-a.txt

Summary:

Issue 1:  Examples - Defer until we resolve the other issues.
Issue 2:  TAG Field Definition - Rainer just expressed some opinions.
Issue 3:  IANA Considerations for undefined PRI values - Resolved.
Issue 4:  Index into Payload in Cert Block - Resolved.
Issue 5:  First Message Number - Anton expressed an opinion.
Issue 6:  Fragment Length - Defer until we resolve Issue 8.
Issue 7:  TPBL Length - Resolved.
Issue 8:  Length of syslog messages - Resolved.

If I didn't capture the sentiment of the WG in any of these issues, please
respond to the note about that, keeping the Subject line so we know what
we're discussing.

Thanks,
Chris

------------------------------

Date: Fri, 12 Sep 2003 07:27:40 -0700 (PDT)
From: Chris Lonvick <[email-masked]>
Subject: Issue 8:  Length of syslog messages - RESOLVED

Issue 8:  Length of syslog messages
http://www.employees.org/~lonvick/draft-ietf-syslog-sign-12.html#format

From the Archive:
http://www.mail-archive.com/syslog-sec%40employees.org/msg01118.html
http://www.mail-archive.com/syslog-sec%40employees.org/msg01119.html
http://www.mail-archive.com/syslog-sec%40employees.org/msg01120.html

STATUS:  Resolved that
1) senders should try to keep packets within the 1024octet limit.
   (Address this in the Security Considerations section?)

2) receivers shouldn't panic if they are given a packet greater than
   1024 octects.
   (Again, address this in the Security Considerations section?)

3) Very long messages may be separated into multiple syslog packets
   and it would be nice if they were numbered so they could be
   reassembled without having to look at the timestamp.  That will
   not be addressed in syslog-sign but Rainer will take a look at it
   for inclusion in syslog-international or a subsequent document.

From the Archive:
http://www.mail-archive.com/syslog-sec%40employees.org/msg01249.html
http://www.mail-archive.com/syslog-sec%40employees.org/msg01252.html
http://www.mail-archive.com/syslog-sec%40employees.org/msg01253.html
http://www.mail-archive.com/syslog-sec%40employees.org/msg01254.html
http://www.mail-archive.com/syslog-sec%40employees.org/msg01255.html
http://www.mail-archive.com/syslog-sec%40employees.org/msg01256.html
http://www.mail-archive.com/syslog-sec%40employees.org/msg01257.html
http://www.mail-archive.com/syslog-sec%40employees.org/msg01257.html

------------------------------

Date: Fri, 12 Sep 2003 07:34:35 -0700 (PDT)
From: Chris Lonvick <[email-masked]>
Subject: Issue 6:  Fragment Length - Still needs review

Issue 6:  Fragment Length
http://www.employees.org/~lonvick/draft-ietf-syslog-sign-12.html#fraglen

From Archive:
http://www.mail-archive.com/syslog-sec%40employees.org/msg01229.html

Albert suggests changing the value from "1-4 characters" to "1-3"
since the payload of a syslog Certificate Block will be less than
999 characters -taking out the length of the PRI, TIMESTAMP and
HOSTNAME.  Anyone have any problems with this?

STATUS: Deferred until we resolve Issue 8

From the Archive:
http://www.mail-archive.com/syslog-sec%40employees.org/msg01246.html

------------------------------

Date: Fri, 12 Sep 2003 07:19:49 -0700 (PDT)
From: Chris Lonvick <[email-masked]>
Subject: Issue 4:  Index into Payload in Cert Block - RESOLVED

Issue 4:  Index into Payload in Cert Block
http://www.employees.org/~lonvick/draft-ietf-syslog-sign-12.html#index

From Archive:
http://www.mail-archive.com/syslog-sec%40employees.org/msg01228.html

Should the value of "0" or "1" be used as the lowest available value?
Albert suggests "0".  Any disagreement to that?

STATUS: Resolved (no disagreement).

------------------------------

Date: Fri, 12 Sep 2003 07:19:04 -0700 (PDT)
From: Chris Lonvick <[email-masked]>
Subject: Issue 3: IANA Considerations for undefined PRI values - RESOLVED

Issue 3:  IANA Considerations for undefined PRI values
http://www.employees.org/~lonvick/draft-ietf-syslog-sign-12.html#iana

From Archive:
http://www.mail-archive.com/syslog-sec%40employees.org/msg01213.html
http://www.mail-archive.com/syslog-sec%40employees.org/msg01218.html
http://www.mail-archive.com/syslog-sec%40employees.org/msg01220.html
http://www.mail-archive.com/syslog-sec%40employees.org/msg01226.html

The consensus appears to be to have a new part in the "IANA
Considerations"  of syslog-sign stating the Facilities not defined
are open to future use by the consensus process (RFC 2434 page 6).

The current set of Facilities and Severities are listed by IANA here:
  http://www.iana.org/assignments/syslog-parameters

Any disagreement to that?

STATUS: Resolved (no disagreement).

------------------------------

Date: Fri, 12 Sep 2003 18:34:09 +0200
From: Rainer Gerhards <[email-masked]>
Subject: RE: Issue 8:  Length of syslog messages - RESOLVED

agree on resolved, but something for the security considerations:

RFC 3195 collectors may have severy issues if the packet is oversized. I
will try with my implementation, but this could lead to a stalling
connection.

Rainer

> -----Original Message-----
> From: Chris Lonvick [mailto:[email-masked]]
> Sent: Friday, September 12, 2003 4:28 PM
> To: [email-masked]
> Subject: Issue 8: Length of syslog messages - RESOLVED
>
>
> Issue 8:  Length of syslog messages
> http://www.employees.org/~lonvick/draft-ietf-syslog-sign-12.ht
> ml#format
>
> From the Archive:
> http://www.mail-archive.com/syslog-sec%40employees.org/msg01118.html
> http://www.mail-archive.com/syslog-sec%40employees.org/msg01119.html
> http://www.mail-archive.com/syslog-sec%40employees.org/msg01120.html
>
>
> STATUS:  Resolved that
> 1) senders should try to keep packets within the 1024octet limit.
>    (Address this in the Security Considerations section?)
>
> 2) receivers shouldn't panic if they are given a packet greater than
>    1024 octects.
>    (Again, address this in the Security Considerations section?)
>
> 3) Very long messages may be separated into multiple syslog packets
>    and it would be nice if they were numbered so they could be
>    reassembled without having to look at the timestamp.  That will
>    not be addressed in syslog-sign but Rainer will take a look at it
>    for inclusion in syslog-international or a subsequent document.
>
> From the Archive:
> http://www.mail-archive.com/syslog-sec%40employees.org/msg01249.html
> http://www.mail-archive.com/syslog-sec%40employees.org/msg01252.html
> http://www.mail-archive.com/syslog-sec%40employees.org/msg01253.html
> http://www.mail-archive.com/syslog-sec%40employees.org/msg01254.html
> http://www.mail-archive.com/syslog-sec%40employees.org/msg01255.html
> http://www.mail-archive.com/syslog-sec%40employees.org/msg01256.html
> http://www.mail-archive.com/syslog-sec%40employees.org/msg01257.html
> http://www.mail-archive.com/syslog-sec%40employees.org/msg01257.html
>
>
>

------------------------------

Date: Fri, 12 Sep 2003 18:52:21 +0200
From: Rainer Gerhards <[email-masked]>
Subject: RE: Issue 5:  First Message Number - Still needs review

I don't think this is a big deal, but I tend to support Anton's view
that "1" is a good match to the English language. Other than that, it
also allows to detect a wrongly initialized counter on the client side,
which could be used for a diagnostic entry. Obviously, a robust
implementation SHOULD accept 0 when acting as a listener (but, as said,
it MAY log a diagnostic entry).

As such, I propose to start *all* ordinal values at 1 and not at 0.

Rainer

> -----Original Message-----
> From: Chris Lonvick [mailto:[email-masked]]
> Sent: Friday, September 12, 2003 4:34 PM
> To: [email-masked]
> Subject: Issue 5: First Message Number - Still needs review
>
>
> Issue 5:  First Message Number
> http://www.employees.org/~lonvick/draft-ietf-syslog-sign-12.ht
> ml#firstmsg
>
> From Archive:
> http://www.mail-archive.com/syslog-sec%40employees.org/msg01228.html
>
> Similar to Issue 4, should the value of "0" or "1" be used as the
> lowest available value?  Albert again suggests "0".  Any disagreement
> to that?
>
> STATUS: No direct responses to this issue but Anton Okmianski suggests
>   using "1" as the start for ordinal values.
>
> From the Archive (see point 4)
> http://www.mail-archive.com/syslog-sec%40employees.org/msg01265.html
>
>
>

------------------------------

Date: Fri, 12 Sep 2003 18:57:23 +0200
From: Rainer Gerhards <[email-masked]>
Subject: RE: Issue 6:  Fragment Length - Still needs review

Now, I support Albert's proposal.

Given on the fact we tackle #8, we should not allow that a fragment will
be further fragmented by syslog-international or whatever. This would
open up a lot of problems. As such, -sign messages should NOT use option
3 from issue #8 (oversize messages).  As such, the block can not be any
longer than 999 bytes and 1*3DIGIT is fine. In fact, it even saves us
from some potential attack vectors.

So my vote: change the range to "1-3".

Rainer

> -----Original Message-----
> From: Chris Lonvick [mailto:[email-masked]]
> Sent: Friday, September 12, 2003 4:35 PM
> To: [email-masked]
> Subject: Issue 6: Fragment Length - Still needs review
>
>
> Issue 6:  Fragment Length
> http://www.employees.org/~lonvick/draft-ietf-syslog-sign-12.ht
> ml#fraglen
>
> From Archive:
> http://www.mail-archive.com/syslog-sec%40employees.org/msg01229.html
>
> Albert suggests changing the value from "1-4 characters" to "1-3"
> since the payload of a syslog Certificate Block will be less than
> 999 characters -taking out the length of the PRI, TIMESTAMP and
> HOSTNAME.  Anyone have any problems with this?
>
> STATUS: Deferred until we resolve Issue 8
>
> From the Archive:
> http://www.mail-archive.com/syslog-sec%40employees.org/msg01246.html
>
>
>

------------------------------

Date: Fri, 12 Sep 2003 18:50:13 +0200
From: Rainer Gerhards <[email-masked]>
Subject: RE: Issue 4:  Index into Payload in Cert Block - RESOLVED

Based on what you wrote for issue #5, I now unfortunately think it is
not resolved any longer. I would say we should stick to a single start
for ordinal values, so I propose to solve this to whatever #5 is solved
to.

Rainer

> -----Original Message-----
> From: Chris Lonvick [mailto:[email-masked]]
> Sent: Friday, September 12, 2003 4:20 PM
> To: [email-masked]
> Subject: Issue 4: Index into Payload in Cert Block - RESOLVED
>
>
> Issue 4:  Index into Payload in Cert Block
> http://www.employees.org/~lonvick/draft-ietf-syslog-sign-12.html#index
>
> From Archive:
> http://www.mail-archive.com/syslog-sec%40employees.org/msg01228.html
>
> Should the value of "0" or "1" be used as the lowest available value?
> Albert suggests "0".  Any disagreement to that?
>
> STATUS: Resolved (no disagreement).
>
>
>

------------------------------

Date: Fri, 12 Sep 2003 18:03:40 +0100
From: Tom Petch <[email-masked]>
Subject: Re: Issue 5:  First Message Number - Still needs review

Start at one.

Zero is different, arguably not really a number (so several colleagues
tell me) often used for special purposes such as 'don't know', 'not
really', 'you tell me', 'tbd' etc etc

SNMP discourages it as an index which, while not directly relevant,
has always felt right to me.

(Ditto, en passant, the largest possible value, sometimes referred to
as minus one, often used for special purposes etc etc)

Tom Petch
- -----Original Message-----
From: Chris Lonvick <[email-masked]>
To: [email-masked] <[email-masked]>
Date: 12 September 2003 16:56
Subject: Issue 5: First Message Number - Still needs review

>Issue 5:  First Message Number
>http://www.employees.org/~lonvick/draft-ietf-syslog-sign-12.html#firs
tmsg
>
>From Archive:
>http://www.mail-archive.com/syslog-sec%40employees.org/msg01228.html
>
>Similar to Issue 4, should the value of "0" or "1" be used as the
>lowest available value?  Albert again suggests "0".  Any disagreement
>to that?
>
>STATUS: No direct responses to this issue but Anton Okmianski
suggests
>  using "1" as the start for ordinal values.
>
>From the Archive (see point 4)
>http://www.mail-archive.com/syslog-sec%40employees.org/msg01265.html
>
>

------------------------------

Date: Fri, 12 Sep 2003 20:56:05 +0200
From: Rainer Gerhards <[email-masked]>
Subject: RE: Issue 8:  Length of syslog messages - RESOLVED

Actually, I was unprecise. Sorry, been too deep in BEEP for a while..

> agree on resolved, but something for the security considerations:
>
> RFC 3195 collectors may have severy issues if the packet is
> oversized. I will try with my implementation, but this could
> lead to a stalling connection.

As of RFC 3081, you have a window size (4k) for data packets. If you
don't increase the window, you must fragment messages over this size or
you are never able to send them. I suspect most 3195 impelmentations
will not deal with fragmented beep messages as there is no need for them
for wellformed 3195 messages.

Sorry for being so brief.

Rainer

------------------------------

Date: Fri, 12 Sep 2003 22:03:24 +0200
From: Rainer Gerhards <[email-masked]>
Subject: 3195/COOKED semantics

Hi WG,

I hope for a clarification. We have an interop issue between SDSC
syslogd and liblogging. It is related to the semantics of COOKED.

When a intiator talks to a listener, is there a half-duplex or
full-duplex communication going on? I mean, must each <entry> be
acknowledged by either an <error> or <ok/> before the next <entry> can
be sent? Or can there be multiple unacknowledged <entry> elements? To be
precise, I am talking about a SINGLE CHANNEL, only.

I tried to resolve this by looking at RFC 3080, from there it looks like
multiple outstanding <entry> elements (MSG in RFC 3080 context) are
valid.

This is an example, all on a single channel:

<entry1>
<entry2>
<entry3>
<ok 1>
<ok 3>
<ok 2>

Would this be a valid scenario? If so, is there any window that the
acknowledgement or error must occur within?

Any clarification would be deeply appreciated.

Many thanks,
Rainer

------------------------------

Date: Fri, 12 Sep 2003 15:24:47 -0700
From: Darren New <[email-masked]>
Subject: Re: 3195/COOKED semantics

Rainer Gerhards wrote:

 > Would this be a valid scenario?

As far as the RFCs go, yes. Hopefully implementations will play nicely.

 > If so, is there any window that the
 > acknowledgement or error must occur within?

2 ^ 32 messages? At which point you run out of message numbers?

- -- 
Darren New, San Diego CA USA (PST)
Don't take home left-over tripe.
   It'll just digest itself before
     you get around to eating it.

------------------------------

Date: Mon, 15 Sep 2003 10:52:37 -0700
From: Devin Kowatch <[email-masked]>
Subject: Re: Issue 5:  First Message Number - Still needs review

I tend to think of indexs starting at zero.  This is mostly because
arrays start indexing at zero, and thus using a different base in the
messgae will require adjusting the index.  While the adjustment is not
hard, it is tedious and may be missed in one or two places.

On Fri, Sep 12, 2003 at 06:03:40PM +0100, Tom Petch wrote:
> Start at one.
>
> Zero is different, arguably not really a number (so several colleagues
> tell me) often used for special purposes such as 'don't know', 'not
> really', 'you tell me', 'tbd' etc etc
>
> SNMP discourages it as an index which, while not directly relevant,
> has always felt right to me.
>
> (Ditto, en passant, the largest possible value, sometimes referred to
> as minus one, often used for special purposes etc etc)
>
> Tom Petch
> -----Original Message-----
> From: Chris Lonvick <[email-masked]>
> To: [email-masked] <[email-masked]>
> Date: 12 September 2003 16:56
> Subject: Issue 5: First Message Number - Still needs review
>
>
> >Issue 5:  First Message Number
> >http://www.employees.org/~lonvick/draft-ietf-syslog-sign-12.html#firs
> tmsg
> >
> >From Archive:
> >http://www.mail-archive.com/syslog-sec%40employees.org/msg01228.html
> >
> >Similar to Issue 4, should the value of "0" or "1" be used as the
> >lowest available value?  Albert again suggests "0".  Any disagreement
> >to that?
> >
> >STATUS: No direct responses to this issue but Anton Okmianski
> suggests
> >  using "1" as the start for ordinal values.
> >
> >From the Archive (see point 4)
> >http://www.mail-archive.com/syslog-sec%40employees.org/msg01265.html
> >
> >
>
>

- -- 
Devin Kowatch
[email-masked]

------------------------------

Date: Tue, 16 Sep 2003 11:17:30 +0200
From: Rainer Gerhards <[email-masked]>
Subject: fragmentation in -international

Hi WG,

I had some off-list discussion with Anton Okmianski on the proposed
fragmentation issue in here. I think he raised some very good points and
I am now posting some of his important thoughts (with his permission):

> I don't have any beef with reboot id...  On the contrary, if it is
> defined as a per process thing, I think it is not a bad idea.  But if
> it is per-host, then sequence numbers have to per-host.  And this is
> bad.  This means that processes have to coordinate with each other
> (through another daemon process, shared memory, peer-to-peer or other
> mechanism). This is a problem because this complexity affects
> availability and reliability of the solutions.  When things are hosed
> on the system, it helps when process does not have external
> dependencies in order to report the problems.
>
> I understand that you have to deal with -sign though.  And I don't
> know exactly what they spec'ed.  On the other hand -sign is not even
> an RFC yet.   I think they should probably add process identifier and
> make reboot id per process. It seems like a very obvious choice if we
> are saying that we support architecture of multiple processes firing
> remotely directly.

I think this boils down to the issue of a single daemon (*nix like)
design vs. a design with multiple independant senders running on the
same system. I agree on that there is an issue if we borrow too much
from the *nix approach (or kind of silently assume it).

Some related reading can be found here:

http://www.mail-archive.com/syslog-sec%40employees.org/msg01217.html (be
sure to follow the thread)

In some other mail, Anton proposed to drop the reboot session ID in
favour of using the following fields as identifier for a single message:

- - TIMESTAMP (should be at least in millisecond resolution)
- - TAG (should include process id)
- - HOSTNAME

These fields should be exactly the same in each fragment and identify
the message (the whole, oversized one). Then fragementation is done
close to what was proposed in -international-0:

fragno.fragtotal

with fragno being the current fragment (starting at 1) and fragtotal
being the total number of fragments. fragtotal would be optional. No
byte counting is done, it is just a plain advancing number. The bottom
line of this is that we can NOT detect bytes added after the original
message with this approach (but would we really like to preserve -sign
signatures in this case...?).

I have to say, I like Anton's approach. It's simple & efficient and
removes the need for the reboot session id, which can be challenging in
some environments.

If there is no objection, I will move this approach into the next
revision of -international.

So, is there any objection? ;)

Anton: would you find this sufficent, as I do not make process id a
MUST?

Rainer

------------------------------

Date: Tue, 16 Sep 2003 15:16:15 +0200
From: Rainer Gerhards <[email-masked]>
Subject: Cookies formalized (-international)

Hi WG,

as agreed upon, I have written some wording of how the concept of the
cookie may be described within the syslog format itself. Although it
looks quite straightforward on the first look, there are some nasty
things to consider when it comes to nested cookies and keeping the
namespace clean. I think it is not actually complicated, but it is quite
lengthy.

I would appreciate any feedback on the wording. So far, I suggest:

####
ABNF(excerpt):

      SYSLOG-MSG      = PRI HEADER MSG
      COOKIE          = "@#" (COOKIE-IANA / COOKIE-VENDOR /
COOKIE-EXPER)
      COOKIE-IANA     = COOKIE-ID ; IANA-Assigned
      COOKIE-EXPER    = "X-" COOKIE-ID ; experimental
      COOKIE-VENDOR   = "V-" VENDORURI "-" COOKIE-ID
      VENDORURI       = 1*64PRINTUSASCII
                        ; a valid domain name owned by the vendor
      COOKIE-ID       = 4*6PRINTUSASCII ; MUST NOT begin with "V-" or
"X-"
      MSG             = (COOKIE SP [COOKIE-PARAMS SP] MSG) / PAYLOAD
      PAYLOAD         = *((%d32-126) / (%d128-254))
                        ; VALID UTF-8 String of PRINTABLE characters

      COOKIE-PARAMS   = *(PRINTUSASCII / %d32)
                        ; parameters defined by the extension using the
cookie
      PRINTUSASCII    = %d33-126

Text following it:
2.1 MSG

   The MSG part can optionally contain a cookie. If it does, optional
   cookie parameters follow after the cookie and after that the original
   message.

   NOTE WELL: MSG is a recursive structure. As such, a MSG may contain a
   COOKIE and another MSG which in turn also contains a COOKIE and yet
   another MSG. To clarify things, we call a MSG that does not contain
   any COOKIE the actual PAYLOAD.

   There is no hard limit of how many levels of COOKIE/MSG constructs
   are used inside a single message. The only limit is that the whole
   construct must fit within the syslog size limitations. Practically,
   however, it is recommended to limit nesting to those cases where it
   absolutely must be done and there is good reasoning for it.

   NOTE WELL: there is an inherant risk with the nesting of COOKIES: As
   specified, a receiver must assume a valid cookie only if he knows the
   full COOKIE, including COOKIE-ID. If he does not know that specific
   cookie, it MUST be treated as ordinary data, thus turning the message
   from MSG to PAYLOAD. As such, no parsing for further COOKIES in
   PAYLOAD is allowed nor desired. In consequence, COOKIES nested in
   deeper layers will not be seen and processed.

   The author beliefes this potential shortcomming is acceptable. If
   inner-layer cookies would be tried to parse, this would potentially
   conflict with existing syslog data as well as introduce a number of
   potential bugs, as the format and thus validity of the outer level
   cookie is not know. It is assumed that if the outer layer cookie is
   not know, the receiver will most probably not understand the
   inner-layer cookie.

   To minimize this risk, more generic cookies should be at the outer
   layers and less specific cookies on the inner layers.

   If the fragmentation features of syslog-international are used, the
   syslog-international COOKIE MUST be the outermost COOKIE. If the
   fragmentation features are not used, it SHOULD be the outermost
   COOKIE, but MAY appear inside another COOKIES' MSG.

2.2 PAYLOAD Part

   The PAYLOAD part contains the details of the message. This has
   traditionally been a freeform message that gives some detailed
   information of the event. The PAYLOAD part of the syslog packet MUST
   contain visible (printing) characters.

   If the syslog-international COOKIE is NOT used, the code set used
   MUST be seven-bit ASCII in an eight-bit field like that used in the
   PRI part. In this code set, the only allowable characters are the
   ABNF VCHAR values (%d33-126) and spaces (SP value %d32).

   If the syslog-international COOKIE is used, the code set is specified
   inside the cookie. In this case, other code sets MAY be used as long
   as the characters used in the MSG are exclusively visible characters
   and spaces similar to those described above. For details, see the
   syslog-international charset parameter. [or should we say "it *is*
   UTF-8"?]

2.3 COOKIE

   The cookie is an optional part of the message. It is used to identify
   optional features inside a syslog message. A cookie can either be
   assigned via IANA (COOKIE-IANA), be experimental but intended to be
   vendor-neutral (COOKIE-EXPER) or be vendor-specific (COOKIE-VENDOR).

   If there is a cookie present, it MUST start with the sequence "@#" at
   the first character of the payload block. The COOKIE-ID must be at
   least 4 characters, so that the overall minumum COOKIE size is 6
   characters. These requirements makes it highly unlikely that a string
   sequence in an "old-style" syslog message will be misinterpreted as a
   cookie. However, there is a slight chance that this may happen. It
   may also be deliberatly done as part of a malicous message. As such,
   an implementation MUST NOT rely soley on the "@#" sequence to judge
   whether it is a valid cookie or not. It MUST parse the whole cookie
   to see if it is known or not and then act accordingly. Unknown
   cookies should be treated as ordinary data and not be acted upon.
   This implies that an implementation MUST not attempt to find further
   cookies inside the MSG.

2.3.1 COOKIE-ID

   The cookie ID uniquely identifies the cookie. It is a 4 to 6
   character wide string of printable characters. It is case-sensitive.
   The 4 character minimum size requirement is introduced to reduce the
   likelyhood that a cookie is mistakenly being recognized. The
   COOKIE-ID alone MUST NOT be used to detect a cookie. It can, however,
   be handy for human discussion.

   The COOKIE-ID MUST NOT begin with "V-" or "X-".

2.3.2 IANA and Experimental Cookies

   These are vendor-neutral cookies. IANA-assigned cookie values have
   undergone the concensus process and are well-defined. Experimental
   cookies are for vendor-neutral functionality that is currently in
   development. A syslog extension that is expected to be
   vendor-specific SHOULD NOT use experimental cookies, it SHOULD use
   vendor-specific cookies instead. As a rule of thumb, only cookies
   used for functionalites discussed on IETF mailing lists should be
   treated as vendor-neutral.

   When new experiemental cookies are designed, they SHOULD use a
   COOKIE-ID not yet assgined by IANA. This will facilitate the later
   transistion as the experiemental COOKIED-ID could eventually be used
   as an IANA COOKIE-ID once concensus has been reached and the
   dicusssed functionality is mature enough.

2.3.3 Vendor specific Cookies

   These cookie values are reserved for vendor extensions. A general
   issue with namespaces and vendor extensions is that multiple vendors
   may (accidently) decide to use the same value as their extension ID.
   To avoid this, we prefix each vendor-specific COOKIE-ID with a
   VENDORURI. This should be a long-lasting internet domain name that
   the vendor owns.

   An example: "Example Inc" has two software products called
   "GreatestSyslog" and "EvenGreaterSyslog". It owns the domains
   "example.com", "GreatestSyslog.example" and
   "EvenGreaterSyslog.example". Now, "Example Inc" decides to introduce
   a new cookie for exclusive use by "EvenGreaterSyslog". It is
   recommended that the company's main domain is used for building the
   vendor cookie. If they used the COOKIE-ID "MyTag", the complete
   vendor cookie would look like this: "@#V-example.com-MyTag".

   The VENDOR-URI is case-insensitve. However, it is good practice to
   send it consistently in the same case. It SHOULD be sent in lower
   case.

   If cookies are nested, vendor cookies MUST be used on the innermost
   layer, only.

####

I am interested in all feedback on this proposal. Most importantly I am
interested in feedback on the nesting of cookies, especially in respect
to undetected outer layers.

And, yes, I am trying to define the syslog format in ABNF, which I think
is very helpful when building parsers. As it looks, the whole ABNF for
the general message format as well as the syslog-internatonal extension
will fit within 2 pages.

Rainer

------------------------------

Date: Tue, 16 Sep 2003 13:16:44 -0700 (PDT)
From: Chris Lonvick <[email-masked]>
Subject: Re: fragmentation in -international

Hi Rainer,

On Tue, 16 Sep 2003, Rainer Gerhards wrote:

> Hi WG,
>
> I had some off-list discussion with Anton Okmianski on the proposed
> fragmentation issue in here. I think he raised some very good points and
> I am now posting some of his important thoughts (with his permission):
>
> > I don't have any beef with reboot id...  On the contrary, if it is
> > defined as a per process thing, I think it is not a bad idea.  But if
> > it is per-host, then sequence numbers have to per-host.  And this is
> > bad.  This means that processes have to coordinate with each other
> > (through another daemon process, shared memory, peer-to-peer or other
> > mechanism). This is a problem because this complexity affects
> > availability and reliability of the solutions.  When things are hosed
> > on the system, it helps when process does not have external
> > dependencies in order to report the problems.
> >
> > I understand that you have to deal with -sign though.  And I don't
> > know exactly what they spec'ed.  On the other hand -sign is not even
> > an RFC yet.   I think they should probably add process identifier and
> > make reboot id per process. It seems like a very obvious choice if we
> > are saying that we support architecture of multiple processes firing
> > remotely directly.
>
> I think this boils down to the issue of a single daemon (*nix like)
> design vs. a design with multiple independant senders running on the
> same system. I agree on that there is an issue if we borrow too much
> from the *nix approach (or kind of silently assume it).
>
> Some related reading can be found here:
>
> http://www.mail-archive.com/syslog-sec%40employees.org/msg01217.html (be
> sure to follow the thread)
>
> In some other mail, Anton proposed to drop the reboot session ID in
> favour of using the following fields as identifier for a single message:
>
> - TIMESTAMP (should be at least in millisecond resolution)
> - TAG (should include process id)
> - HOSTNAME
>
> These fields should be exactly the same in each fragment and identify
> the message (the whole, oversized one). Then fragementation is done
> close to what was proposed in -international-0:
>
> fragno.fragtotal
>
> with fragno being the current fragment (starting at 1) and fragtotal
> being the total number of fragments. fragtotal would be optional. No
> byte counting is done, it is just a plain advancing number. The bottom
> line of this is that we can NOT detect bytes added after the original
> message with this approach (but would we really like to preserve -sign
> signatures in this case...?).
>
> I have to say, I like Anton's approach. It's simple & efficient and
> removes the need for the reboot session id, which can be challenging in
> some environments.

The purpose of the reboot counter is to prevent replay.  I can see the
inclusion of the process id and/or name with the certificate block
messages to identify the sender.  (I'm still a little doubtful about the
concept of multiple syslog processes from a single device but I'll go
along for the discussion. :)  Can you explain how this will prevent a
replay?

In which environments is this challenging?  To utilize a timestamp,
devices will require a TOY (Time of Year chipset).  To utilize a reboot
counter, a device just needs some memory that will persist over a reboot.
I believe that the latter is more common and that this was discussed in
the SNMP mailing list prior to coming up with SNMPv3, which came to the
conclusion that a reboot couter was preferred.

>
> If there is no objection, I will move this approach into the next
> revision of -international.
>

I'm not objecting - just trying to make sure we've got our bases covered.

Thanks,
Chris

> So, is there any objection? ;)
>
> Anton: would you find this sufficent, as I do not make process id a
> MUST?
>
> Rainer
>
>
>

------------------------------

Date: Tue, 16 Sep 2003 22:45:06 +0200
From: Rainer Gerhards <[email-masked]>
Subject: RE: fragmentation in -international

Hi Chris,

> The purpose of the reboot counter is to prevent replay.  I
> can see the inclusion of the process id and/or name with the
> certificate block messages to identify the sender.  (I'm
> still a little doubtful about the concept of multiple syslog
> processes from a single device but I'll go along for the
> discussion. :)  Can you explain how this will prevent a replay?

Let me ask a return question... If the reboot counter is implemented as
e.g. Albert suggested, it is more or less a timestamp (remember, it was
suggested to take the seconds since some date and use this is reboot
counter...). How can this provide more replay protection than an actual
timestamp...?

Oops... Maybe I wasn't clear enough. I think that TIMESTAMP/TAG/HOSTNAME
together should make up the message identifier. This would - in my point
of few - create the same reply protection that a time-based reboot
session ID does.

>
> In which environments is this challenging?  To utilize a
> timestamp, devices will require a TOY (Time of Year chipset).
>  To utilize a reboot counter, a device just needs some memory
> that will persist over a reboot.

Maybe Anton can help me out with some of the devices. But at least for
Windows, it is not uncommon that the OS is freshly installed. This will
also remove any previously stored reboot session id. If the machine
retains the same ip, there will be an issue...

> I believe that the latter is
> more common and that this was discussed in the SNMP mailing
> list prior to coming up with SNMPv3, which came to the
> conclusion that a reboot couter was preferred.

Looks like I need to review the SNMPv3 mailing list and see what drove
the decision. I assume there is lot to learn...

> > If there is no objection, I will move this approach into the next
> > revision of -international.
> >
>
> I'm not objecting - just trying to make sure we've got our
> bases covered.

Please bear with the non-English natives ;) Eventually "object" is too
hard - I meant if somebody disagrees and/or has a better answer/comment
- - as you did ;-)

Rainer

------------------------------

Date: Wed, 17 Sep 2003 05:06:38 -0700 (PDT)
From: Lonvick <[email-masked]>
Subject: Re: fragmentation in -international

- ---------- Forwarded message ----------
Date: Tue, 16 Sep 2003 20:22:01 -0700
From: Devin Kowatch <[email-masked]>
To: Rainer Gerhards <[email-masked]>
Cc: Chris Lonvick <[email-masked]>, [email-masked]
Subject: Re: fragmentation in -international

On Tue, Sep 16, 2003 at 10:45:06PM +0200, Rainer Gerhards wrote:
[ snip ]
> >
> > In which environments is this challenging?  To utilize a
> > timestamp, devices will require a TOY (Time of Year chipset).
> >  To utilize a reboot counter, a device just needs some memory
> > that will persist over a reboot.
>
> Maybe Anton can help me out with some of the devices. But at least for
> Windows, it is not uncommon that the OS is freshly installed. This will
> also remove any previously stored reboot session id. If the machine
> retains the same ip, there will be an issue...
[ snip ]

Actually, this makes me think that IPs will get reused regardless, which
means that no matter what OS is being run, it's likely that the same
problem will arise.

Unless the reboot counter is coupled with a globally unique machine
identifier, the reboot counter will likely get reset, because there
will be a new machine at that IP/hostname.

- -- 
Devin Kowatch
[email-masked]

------------------------------

Date: Wed, 17 Sep 2003 05:07:32 -0700 (PDT)
From: Lonvick <[email-masked]>
Subject: -sing-12 signature verifyer

- ---------- Forwarded message ----------
Date: Wed, 17 Sep 2003 11:18:46 +0200
From: Rainer Gerhards <[email-masked]>
To: [email-masked]
Cc: Wajih ur Rehman <[email-masked]>
Subject: -sing-12 signature verifyer

Hi WG,

we have begun working on a signature verifier for -sign-12 (thanks to
Albert's work, we have something to verify:)).

We have come accross an issue with online verification. -sign-12 tella
redundadenncy parameters in section 5. Among others, they specify when a
resend should occur. However, what I didn't find is any time window
defined of within what time period a signature block should follow a
plain syslog message.

Let's assume I am doing online verification because I would like to
trigger some actions in near-realtime and I would like to make sure that
the message I am acting on is authentic.

So if I receive a message from a remote host, I must wait until I
receive the signature block with the hash for this message. It could be
infinite until this happens. I guess that most emitors will send
signature blocks in a reasonable amount of time, but it is not
specified.

I propose we add some wording on WHEN a signature block must be emitted
at latest to facilitate online verification.

Other than that, I think the main use will be offline verification (even
after years) - we are probably better off with 3195 & TLS & SASL than
with doing online verification. Is this the spirit of -sign?

Any clarification is appreciated.

Rainer

------------------------------

Date: Tue, 16 Sep 2003 20:22:01 -0700
From: Devin Kowatch <[email-masked]>
Subject: Re: fragmentation in -international

On Tue, Sep 16, 2003 at 10:45:06PM +0200, Rainer Gerhards wrote:
[ snip ]
> >
> > In which environments is this challenging?  To utilize a
> > timestamp, devices will require a TOY (Time of Year chipset).
> >  To utilize a reboot counter, a device just needs some memory
> > that will persist over a reboot.
>
> Maybe Anton can help me out with some of the devices. But at least for
> Windows, it is not uncommon that the OS is freshly installed. This will
> also remove any previously stored reboot session id. If the machine
> retains the same ip, there will be an issue...
[ snip ]

Actually, this makes me think that IPs will get reused regardless, which
means that no matter what OS is being run, it's likely that the same
problem will arise.

Unless the reboot counter is coupled with a globally unique machine
identifier, the reboot counter will likely get reset, because there
will be a new machine at that IP/hostname.

- -- 
Devin Kowatch
[email-masked]

------------------------------

Date: Wed, 17 Sep 2003 11:18:46 +0200
From: Rainer Gerhards <[email-masked]>
Subject: -sing-12 signature verifyer

Hi WG,

we have begun working on a signature verifier for -sign-12 (thanks to
Albert's work, we have something to verify:)).

We have come accross an issue with online verification. -sign-12 tella
redundadenncy parameters in section 5. Among others, they specify when a
resend should occur. However, what I didn't find is any time window
defined of within what time period a signature block should follow a
plain syslog message.

Let's assume I am doing online verification because I would like to
trigger some actions in near-realtime and I would like to make sure that
the message I am acting on is authentic.

So if I receive a message from a remote host, I must wait until I
receive the signature block with the hash for this message. It could be
infinite until this happens. I guess that most emitors will send
signature blocks in a reasonable amount of time, but it is not
specified.

I propose we add some wording on WHEN a signature block must be emitted
at latest to facilitate online verification.

Other than that, I think the main use will be offline verification (even
after years) - we are probably better off with 3195 & TLS & SASL than
with doing online verification. Is this the spirit of -sign?

Any clarification is appreciated.

Rainer

------------------------------

Date: Wed, 17 Sep 2003 10:57:52 +0200
From: Rainer Gerhards <[email-masked]>
Subject: section 7.5 in -sign-12 empty

I just noticed that section 7.5 "Replying" is empty, BUT 7.6 talks about
reply. I guess this just needs some cleanup (remove 7.5?).

Rainer

------------------------------

Date: Wed, 17 Sep 2003 11:00:05 +0200
From: Rainer Gerhards <[email-masked]>
Subject: -sign-12 reboot session id - potential issue

Hi WG,

the reboot session id as defined in -sign-12 has potential for
implementations errors. I think is not a really big issue, but I would
like to at least make sure everybody is aware of it.

The range for reboot session id is defined to be 0..999999999. This is
beyond the scope of unsigned 32 bit int, which is 0..4294967295. I
assume that this bears the risk that some implementors do not realize
that they need to apply 64 bit arithmetics, which means that some of
them may need to write the necessary code to support this. I guess this
is the reason why the reboot id has been latched the the *signed* 32 bit
integer value in SNMPv3 (RFC 2574). There, it is in the range of
0..2147483647.

I am not sure if we really need a that large reboot session id, if it is
worth the potential trouble.

I propose to limit the range to 0..4294967295 OR mention the need for 64
bit arithmetic prominently where the reboot id is defined (so it can't
simply be overlooked).

Rainer

------------------------------

Date: Wed, 17 Sep 2003 10:24:00 +0200
From: Rainer Gerhards <[email-masked]>
Subject: RE: fragmentation in -international & reboot session id security

Chis,

> The purpose of the reboot counter is to prevent replay.  I can see the
> inclusion of the process id and/or name with the certificate block
> messages to identify the sender.  (I'm still a little
> doubtful about the
> concept of multiple syslog processes from a single device but I'll go
> along for the discussion. :)  Can you explain how this will prevent a
> replay?
>
> In which environments is this challenging?  To utilize a timestamp,
> devices will require a TOY (Time of Year chipset).  To
> utilize a reboot
> counter, a device just needs some memory that will persist
> over a reboot.
> I believe that the latter is more common and that this was
> discussed in
> the SNMP mailing list prior to coming up with SNMPv3, which
> came to the
> conclusion that a reboot couter was preferred.

I read those sections of http://www.ietf.org/rfc/rfc2574.txt dealing
with the reboot id. Actually, I find that there is a big difference
between the way it is utilized in -sign-12 and RFC2574. 2574 specifies
that it must be *INCREMENTED* on each reboot. That means it must go up
by exactly ONE. In sign-12, we just say it must be advancing. Also, in
2574 the reboot ID is used together with a timestamp (and time window).
This & especially the increment is the key to the secure reboot session
ID.

Look at e.g. Albert's implementation of -sign. As suggested, he is using
the actual timestamp to generate the reboot session ID. I intend to do
it in a similar way in my code. So if I look at code (easy thanks open
source), I can guess upcoming rebooot session ids easily.  As such, I
can use them to construct messages with valid reboot session IDs of e.g.
next monday. In the context of -sign, this is NO issue at all, because
the reboot session ID will be signed and thus an attacker would need to
be in the posession of the emitor's private key. If he is, he can do all
sorts of nasty things, so we don't need to worry about this specific
case.

However, in the light of -international, things are quite different. If
we carry over the exact same reboot session id definition, we have two
security issues. Before I go into them, let me point out the root cause:
we now do not sign the messages (that's only done in -sign and even
there not for the actual message [in real-time, in-message]). So it is
easy for an attacker to generate a fake syslog message. This is
mitigated by using 3195 with proper tuning profiles, but as
- -international should run over all transports, we must assume plain UDP
when it comes to security issues (an attacker would always use the
easiest target).

So these are the two issues:

#1 replay of message
   done by simply incrementing the reboot id to some value that
   the emitor normally does not yet emit

#2 DoS or confusion of the collector
   by sending a message with reboot session ID 9999999999 (the highest)

Using a timestamp has advantages in both cases. Given a properly
time-synced network, #1 can be detected by the collector. We could use
some mechanism like the kerberos time window. Granted, if time is not
synced, that doesn't help - but then we are back to what the reboot id
provides (or not provides)...

#2 can totally be avoided, because there is no upper value for the
timestamp (well, 2037 may give us some headaches, but that is an
implementation issue, not a protocol one. Year 10000 may also give our
parsers a bad time, but I assume there will be some new versions
available by then...)

So in short, I think we should actually stay away from the reboot
session ID for -international.

OR - eventually it would be better to implement it as in 2574, requiring
that the reboot id is INCREMENTED not just ADVANCING. That would
definitely add complexity to the implementations. On the other hand
this, together with an incrementing message ID, actually guards against
reply attacks. Well, it would guard against most of them. If an attacker
manages to compromise a log emitor, shut it down and then begin sending
faked messages in its name, that wouldn't be detected, but I don't know
how we could detect this running over UDP without signatures... (Hint
for security considerations: recommend 3195 w/ TLS as transport).

The question is now: are we prepared to go with the timestamp-like
limited reply protection (easy to break, but simple to implement) or
would we go for a real reboot session ID (hard to break, but
not-so-simple to implement). While I think the later adds only
reasonable work to the implementor, I am not sure if it is in the spirit
of syslog. Remeber, it is better to have weak protection done by
everyone than strong protection done by only a few. Even the weak
protection can be hardened by using proper transport, so there is a way
to secure things for those that understand the needs....

Well, I think its time to turn over this question to the list. Any
comments? ;-)

Rainer

------------------------------

Date: Wed, 17 Sep 2003 08:41:09 +0200
From: Albert Mietus <[email-masked]>
Subject: RE: fragmentation in -international

There has been som discussion about "reboot counters":

 > > The purpose of the reboot counter is to prevent replay.

 > Let me ask a return question... If the reboot counter is implemented as
 > e.g. Albert suggested, it is more or less a timestamp

Let me add my points:

First, (and major!)

We should not intermix -sign and -international discussions. As I'm
not following -international (no time:-), I will concentrate on "how
and why in -sign. See for yourself whether is is valid for -international

* The reboot-counter is not "just a timestamp"! It is a "ad hoc"
constant, for as long thr system is up. As long as the system is up
(or at least syslogd in my unix-implementation), the same "number"
will be just.
This is needed (can/must be used) by the -sign validator, all messages
with that same number share something (like the same key).

Aftewr a reboot, a new number is needed, as everything starts all
over. With probably a new key(pair). It the validator who need to
know!

On windows, start time of the application(service) can be used.

On very-small systems, with no NVRAM (or simulair), this isn't a
showstopper. Just get a number, e.g. the version number of the key.
Note: as that one should be stored someware, there is probably place
for more bits to, like the priviour "number". Then just add one.
Also, when the keys don't change, it is probably OK to use the same
reboot-counter number. We have to live with simple HW, with limits.

 > > In which environments is this challenging?  To utilize a
 > > timestamp, devices will require a TOY (Time of Year chipset).
 > >  To utilize a reboot counter, a device just needs some memory
 > > that will persist over a reboot.

Not correct. If a TOY is present, just read it on boot, and store that
value in RAM. And use that RAM value as reboot-counter.

It will change (increase) after a reboor, which is ment to be for a
reboot counter!

 > Windows, it is not uncommon that the OS is freshly installed. This will

It that case, a reboot-timestamp will do.

But also any other number, as a new, fresh Windows PC can equally been
seen as "an new (other) system".

Hope it clearifies, If not, mail for details
- -- 
ALbert Mietus
	Send prive mail to:      [email-masked]
	Send business mail to:  [email-masked]
	Don't send spam mail!

------------------------------

Date: Wed, 17 Sep 2003 11:19:29 -0400
From: Anton Okmianski <[email-masked]>
Subject: RE: fragmentation in -international

Just to clarify...

I don't think that host/process/time is the best unique identifier for
messages. The example that I gave also included per-process sequence
number (count which resets to 1 on restarts).  But even this format is
not as good as one with reboot id because time can be moved backwards,
counts can be reset, it is harder to interpret, etc.  It is the next
best thing if reboot id is missing.

I think reboot id is a good thing.  My only real beef was with reboot
id and count being *per-host* things instead of *per-process*.  OS
re-installs are a different issue -- could potentially be mitigated by
persisting data on a separate disk form OS.

I see many examples of multiple processes that could fire syslog
messages directly to a remote host.  I can easily have Cisco TFTP, DNS
and DHCP servers installed on a single Solaris/Windows/Linux host (all
of which are supported) and I am not sure what benefit is there in
making them go through a separate daemon in order to log messages
remotely via UDP.  This does not help performance, reliability or
robustness IMO.

The problem is with maintaining per-host message count across multiple
processes or centralizing the generation of log messages somewhere.
Note, that previous syslog RFCs did not have anything in them that
made it harder for individual processes to fire messages remotely
directly.  If I am to use to use reboot id and counts for just
fragmentation, a newly required component on my servers is high price
to pay it seems.

Thanks,
Anton.

> -----Original Message-----
> From: [email-masked]
> [mailto:[email-masked]]On Behalf Of Rainer Gerhards
> Sent: Tuesday, September 16, 2003 4:45 PM
> To: Chris Lonvick
> Cc: [email-masked]
> Subject: RE: fragmentation in -international
>
>
> Hi Chris,
>
> > The purpose of the reboot counter is to prevent replay.  I
> > can see the inclusion of the process id and/or name with the
> > certificate block messages to identify the sender.  (I'm
> > still a little doubtful about the concept of multiple syslog
> > processes from a single device but I'll go along for the
> > discussion. :)  Can you explain how this will prevent a replay?
>
> Let me ask a return question... If the reboot counter is
> implemented as
> e.g. Albert suggested, it is more or less a timestamp
> (remember, it was
> suggested to take the seconds since some date and use this is reboot
> counter...). How can this provide more replay protection
> than an actual
> timestamp...?
>
> Oops... Maybe I wasn't clear enough. I think that
> TIMESTAMP/TAG/HOSTNAME
> together should make up the message identifier. This would
> - in my point
> of few - create the same reply protection that a time-based reboot
> session ID does.
>
> >
> > In which environments is this challenging?  To utilize a
> > timestamp, devices will require a TOY (Time of Year chipset).
> >  To utilize a reboot counter, a device just needs some memory
> > that will persist over a reboot.
>
> Maybe Anton can help me out with some of the devices. But
> at least for
> Windows, it is not uncommon that the OS is freshly
> installed. This will
> also remove any previously stored reboot session id. If the machine
> retains the same ip, there will be an issue...
>
> > I believe that the latter is
> > more common and that this was discussed in the SNMP mailing
> > list prior to coming up with SNMPv3, which came to the
> > conclusion that a reboot couter was preferred.
>
> Looks like I need to review the SNMPv3 mailing list and see
> what drove
> the decision. I assume there is lot to learn...
>
> > > If there is no objection, I will move this approach
> into the next
> > > revision of -international.
> > >
> >
> > I'm not objecting - just trying to make sure we've got our
> > bases covered.
>
> Please bear with the non-English natives ;) Eventually
> "object" is too
> hard - I meant if somebody disagrees and/or has a better
> answer/comment
> - as you did ;-)
>
> Rainer
>
>

------------------------------

Date: Wed, 17 Sep 2003 18:28:22 +0200
From: Rainer Gerhards <[email-masked]>
Subject: Typo in rfc 3195

Hi WG,

event though I seem to be the one comitting the most typos, I think I
finally found one somewhere else. In 4.4.2 of 3195, it talks about relay
modes in [1], which refers to 3164. It says e.g.

###
However, for maximum compatibility, the device
   SHOULD format the CDATA of the message in accordance with Sections
   4.2.1 through 4.2.3 of [1].
###

This reference is repeated.

The only thing is there is no 4.2.1 to 4.2.3 in 3164. I think it should
refer to 4.*1*.1 to 4.*1*.3.

A bit later, it 3195 says:
####
   To be consistent with the spirit of [1], a relay receiving a message
   that does not contain a valid priority, timestamp or hostname will
   follow the same general rules as described in section 4.2.2 of [1]
   while including the exact contents of the received syslog packet as
   the CDATA.  The values of the facility and severity will be construed
####
Again, 4.2.2 does not exist in 3164, I assume this case 4.3 (as whole
that is 4.3.1 to 4.3.3) is meant.

Then, it says:
####

   As another example, consider a message being received that does not
   properly adhere to the conventions described in Section 4.2.2 of [1].
   In particular, the timestamp has a year, making it a nonstandard
   format:
####
I assume, this means 4.1.2.

I am pulling my RFCs from http://www.ietf.org/rfc/rfcNNNN.txt which
should be the definite source.

Something for the next revision?

Rainer

------------------------------

Date: Wed, 17 Sep 2003 18:40:59 +0200
From: Rainer Gerhards <[email-masked]>
Subject: RFC 3195/COOKED what to do with the pre-parsed fields?

Hi WG,

with this mail, I am again asking for some clarification. This question
may sound silly, but it actually is an issue to me. I am currently
implementing the COOKED server part.

The <entry> element (RFC3195, 4.4.2) contains a number of pre-parsed
entities, like facility, priority, tag, timestamp... The CDATA of the
entry element contains the full, unmodified syslog messages. This is
very useful, as, among others, it allows a -sign message to be
transmitted via 3195.

HOWEVER, I now have situations where wellformed messages actually
contain 2 PRI, tag, timestamp, ..., one in the entry element and one in
the raw message. I think this is the most common case. Question now, how
to handle this intelligently?

Should I return an error if the two values do not match?
Would it be wise to record both if I am a collector?
What should I do if they are different and I am a relay and need to
forward via e.g. 3195/RAW (because COOKED is not available on the remote
peer)?

One solution is:

If I am a relay and forward to another peer via COOKED, than the
duplicates can be preserved.
If I am a relay and forward to a peer via RAW or a 3164 syslogd,
  - I will discard the entry paramters if the message
    (CDATA) is wellformed
  - I use the entry parameters if the CDATA is not well-formed
    and thus modify the CDATA (much like 3164's relay roles)
If I am a collector, I either store
  - both sets of values
  - or just the raw CDATA
  - or as described for a relay depending on the well-formedness
    of the CDATA

I wonder if someone can actually point my to the "spirit of COOKED", you
know, the philosophy that isn't written down ;)

I also notice that 3195 contains detailled instruction of how to modify
a non-wellformed message when it is being converted for being relayed by
COOKED. But it does contain no instructions what to do if that message
is received via COOKED and needs to be relayed via 3164 or 3195/RAW.
Maybe this is something for the next revision.

Thanks,
Rainer

------------------------------

Date: Wed, 17 Sep 2003 08:38:38 -0700 (PDT)
From: Chris Lonvick <[email-masked]>
Subject: Re: section 7.5 in -sign-12 empty

Hi,

On Wed, 17 Sep 2003, Rainer Gerhards wrote:

> I just noticed that section 7.5 "Replying" is empty, BUT 7.6 talks about
> reply. I guess this just needs some cleanup (remove 7.5?).

("Replay" - I suspect your fingers are moving too fast. ;-)

Dang.  The text was in -11.txt.  It says:

   Event messages may be recorded and replayed by an attacker.  However
   the information contained in the Signature Blocks will allow a reviewer
   to determine if the received messages are the ones originally sent by a
   device.  This process will also alert the reviewer to replayed
   messages.

I believe it should stay there and I'll help Jon keep an eye on it.

Thanks,
Chris

------------------------------

Date: Wed, 17 Sep 2003 12:10:22 -0700
From: Jon Callas <[email-masked]>
Subject: Re: section 7.5 in -sign-12 empty

On 9/17/03 8:38 AM, "Chris Lonvick" <[email-masked]> wrote:

> Dang.  The text was in -11.txt.  It says:
>
>  Event messages may be recorded and replayed by an attacker.  However
>  the information contained in the Signature Blocks will allow a reviewer
>  to determine if the received messages are the ones originally sent by a
>  device.  This process will also alert the reviewer to replayed
>  messages.
>
>
> I believe it should stay there and I'll help Jon keep an eye on it.

Oops. Sorry about that. I must have futzed something in differences when I
was making that.

I'll make sure it's in -13.

    Jon

------------------------------

Date: Wed, 17 Sep 2003 12:00:42 -0700 (PDT)
From: Chris Lonvick <[email-masked]>
Subject: Re: Typo in rfc 3195

Hi,

Hopefully the next revision of 3195 can just refer to syslog-sign as the
definitive way to do syslog messages.  :-)

Thanks,
Chris

On Wed, 17 Sep 2003, Rainer Gerhards wrote:

> Hi WG,
>
> event though I seem to be the one comitting the most typos, I think I
> finally found one somewhere else. In 4.4.2 of 3195, it talks about relay
> modes in [1], which refers to 3164. It says e.g.
>
> ###
> However, for maximum compatibility, the device
>    SHOULD format the CDATA of the message in accordance with Sections
>    4.2.1 through 4.2.3 of [1].
> ###
>
> This reference is repeated.
>
> The only thing is there is no 4.2.1 to 4.2.3 in 3164. I think it should
> refer to 4.*1*.1 to 4.*1*.3.
>
> A bit later, it 3195 says:
> ####
>    To be consistent with the spirit of [1], a relay receiving a message
>    that does not contain a valid priority, timestamp or hostname will
>    follow the same general rules as described in section 4.2.2 of [1]
>    while including the exact contents of the received syslog packet as
>    the CDATA.  The values of the facility and severity will be construed
> ####
> Again, 4.2.2 does not exist in 3164, I assume this case 4.3 (as whole
> that is 4.3.1 to 4.3.3) is meant.
>
> Then, it says:
> ####
>
>    As another example, consider a message being received that does not
>    properly adhere to the conventions described in Section 4.2.2 of [1].
>    In particular, the timestamp has a year, making it a nonstandard
>    format:
> ####
> I assume, this means 4.1.2.
>
> I am pulling my RFCs from http://www.ietf.org/rfc/rfcNNNN.txt which
> should be the definite source.
>
> Something for the next revision?
>
> Rainer
>
>
>

------------------------------

Date: Wed, 17 Sep 2003 12:33:01 -0700
From: Jon Callas <[email-masked]>
Subject: Re: section 7.5 in -sign-12 empty

On 9/17/03 8:38 AM, "Chris Lonvick" <[email-masked]> wrote:

>  Event messages may be recorded and replayed by an attacker.  However
>  the information contained in the Signature Blocks will allow a reviewer
>  to determine if the received messages are the ones originally sent by a
>  device.  This process will also alert the reviewer to replayed
>  messages.

Hmm. This paragraph is in the XML source. This is mildly disturbing. I'll
keep an eye on things in the future, and not just trust the converter.

    Jon

------------------------------

Date: Wed, 17 Sep 2003 12:35:31 -0700
From: Darren New <[email-masked]>
Subject: Re: Typo in rfc 3195

Rainer Gerhards wrote:
> Something for the next revision?

Thanks! I'll look into it and make the changes to my copy for next time!

- -- 
Darren New, San Diego CA USA (PST)
Don't take home left-over tripe.
   It'll just digest itself before
     you get around to eating it.

------------------------------

Date: Thu, 18 Sep 2003 09:06:22 +0200
From: Albert Mietus <[email-masked]>
Subject: security and reboot (was:RE: fragmentation in -international)

Some quotes of several messages (reformated)

> Look at e.g. Albert's implementation of -sign. As suggested, he is
> using the actual timestamp to generate the reboot session ID. I
> intend to do it in a similar way in my code.

> In the context of -sign, this is NO issue at all,[...] However, in
> the light of -international, things are quite different.

This line is important.
It means (to me:)
   -sign	    is about security
   -international   is about functionality, NOT about security!

> -international should run over all transports, we must assume plain
> UDP when it comes to security issues (an attacker would always use
> the easiest target).

That is wrong. -international isn't (about) security. When security is
needed, -international should be used over a secure transport
"layer". Like -sign, -reliable, of even a non-standard transport.

All -international should do about security is:
    a) make sure it can be used over a "secure syslog transport"
    b) make sure no (new) insecure features are introduced.

So, a "counter to increase security", but not needed is useless, and
can be counter productive.

As I understand -international it needs a kind of "super messages
(longer that fit in a transport syslog message). I'm not aware of
details, but ...

Please just specify how to fit "long messages" in serveral transport
packages. (That is done before: "fragmenting", maybe reuse that
knowledge:-)  And asume the tarnsport layer works (most of the time).

Then, it will be simple and working. even for UDP-syslog. Make sure
the algorithm doens't break when a UDP message is lost, but live with
the lost-data! If a system(admin) can't live with it, he will use
syslog-reliable as transport anyhow!
When the system/network is small, and logging not top-priority UDP
syslog will be fine.

For small networks, UDP is fine: Just set-up a centrall (UDP)
collector, and add host (physically) at will. As long they use a UDP
syslog (traditional, rfc3164, -sign, -international, -fragmented, ...)
all log can be read. And "complex" fragmented messages are still
reasble, without tools!
(<<See this as ... (1/2)>> <<... an example(2/2)>>)

 > The important thing is that I think the reboot ID - as you describe it -
 > works for -sign. I was arguing that it does NOT provide reply attack for
 > -international

You correct, it will NEVER work for -international, I think. And It
should!

 > So these are the two issues:

There are a lot more, if -international doesn't use -sign or -reliable
as "transport" Just don't fix them:-)

Note:
Maybe we should make a syslog-fragment (or syslog-long) RFC, which
describes how to send "long (MSG/CONTENT)", by fragmenting. And
provides ("to above") the functionality to send unlimmited long
messages over another syslog-transport.

Then -international can use that one to add localization, i18n, or
whatever.
However, then it shoud be possible to use those long messages for
other features to?

Just an idea.

- --ALbert Mietus
	Send prive mail to:      [email-masked]
	Send business mail to:  [email-masked]
	Don't send spam mail!

------------------------------

Date: Thu, 18 Sep 2003 15:21:36 +0200
From: Rainer Gerhards <[email-masked]>
Subject: RE: security and reboot (was:RE: fragmentation in -international)

> > In the context of -sign, this is NO issue at all,[...] However, in
> > the light of -international, things are quite different.
>
> This line is important.
> It means (to me:)
>    -sign	    is about security
>    -international   is about functionality, NOT about security!

I am on the move and can't read/comment on everything now. BUT: I think
any modern protocol needs to care a little bit of security. That is, you
should try to architecture it that way that there are no inherent
security risks, if you can avoid them. Of course, a protocol
specifically *designed* to secure things must hold to even higher
standards. But I am not ready to accept being lax with all others. I
think there are enough codes out there who don't care...

Rainer

------------------------------

Date: Wed, 24 Sep 2003 14:20:38 +0200
From: Rainer Gerhards <[email-masked]>
Subject: RE: security and reboot (was:RE: fragmentation in -international)

Hi Albert,

sorry for the slow response, as I wrote I was out of office ... and the
Internet connection was so bad that it was virtually non existant.

I am trying to address all issues you raise. But first of all, by going
through the mail archives, I have noticed that you may find I had picked
on your code. Honestly, this is not the case, but the impression may be
created by not following the full sequence of my postings. My apologies
for this. I quoted your name when it came to reboot session IDs (in
- -sign) because I wanted to give proper credits to the first one who
actually implemented it. Besides being the first one, I also like your
approach, as I think it fits well with -sign and solves a lot of issues.
As I pointed out both on- and off-list, I will probably follow your
approach when I am ready to implement -sign. Once again, my sincere
apologies if I somehow created the impression to "attack" your way of
doing things - this was not my intention.

> Some quotes of several messages (reformated)
>
> > Look at e.g. Albert's implementation of -sign. As suggested, he is
> > using the actual timestamp to generate the reboot session ID. I
> > intend to do it in a similar way in my code.
>
> > In the context of -sign, this is NO issue at all,[...] However, in
> > the light of -international, things are quite different.
>
> This line is important.
> It means (to me:)
>    -sign	    is about security
>    -international   is about functionality, NOT about security!

I already commented on this one. While I think that -sign is on the core
of security, I think -international (and all hopefully all other
protocols) should try to do their best to secure what they are doing. So
in this way, everything needs to have security put on the agenda. I
think this is why IETF demands a "Security Considerations" section in
the newer RFCs.

But I think you did not get the spirit of my post. I can see this
because you snipped out the most important part. [...] reads as follows:

> (In the context of -sign, this is NO issue at all,) because
> the reboot session ID will be signed and thus an attacker would need
to
> be in the posession of the emitor's private key.

I am just pointing this out because I find it is important to keep in
mind that security in -sign does not come from just the reboot session
ID but it comes from the fact that it is cryptographically signed.

> > -international should run over all transports, we must assume plain
> > UDP when it comes to security issues (an attacker would always use
> > the easiest target).
>
> That is wrong. -international isn't (about) security.

I do not fully agree here, see above.

> When security is
> needed, -international should be used over a secure transport
> "layer". Like -sign, -reliable, of even a non-standard transport.

Here, I fully agree, in fact this is what I recommended in several
postings.

>
> All -international should do about security is:
>     a) make sure it can be used over a "secure syslog transport"
>     b) make sure no (new) insecure features are introduced.
>

This sounds very good ...

> So, a "counter to increase security", but not needed is useless, and
> can be counter productive.

... actually, we are very close together: this counter was not
introduced because of security, it was introduced because it was needed
for fragmentation (oversize messages). In fact, it is not always
present, only in when the message is oversized and needs to be
fragmented. Please see the ABNF posted:

http://www.mail-archive.com/syslog-sec%40employees.org/msg01262.html

It is a good thing that you remind me that the reboot ID will most
probably not be present in 99% of the -international enhanced syslog
messages (at least this is what I feel). This is because it is a purely
OPTIONAL part.

A question to the WG is if it is concensus that it should stay that way
(I would prefer this).

Then, I was thinking about replay attacks, because I eventually
misunderstood a comment from Chris

http://www.mail-archive.com/syslog-sec%40employees.org/msg01290.html

He said that the reboot session ID could also be helpful in detecting
replay attacks. That made me look deeper and deeper into this use.
However, as you (Albert) correctly point out, the reboot session ID is
not really that important for -international because it will most
probably not be present in the majority of messages.

I guess some confusion was caused because I moved over the reboot
session ID & related things from -sign to -international. The root
reason for this is that I need a unique message ID for -international -
and I thought I use those things that were actually agreed upon. Well,
wrong move, because there are obviously some little differences. I now
don't think it pays to save some coding by re-using what is already
there.

And now I think Chris, too, was more or less refering to reboot ID as in
- -sign (different use) [Chris: is this right?]

I still need a message ID. This discussion - I think - shows that it
will probably best to use

> - TIMESTAMP (should be at least in millisecond resolution)
> - TAG (should include process id)
> - HOSTNAME

together to form the ID and drop the reboot session ID. If nobody comes
up with another opinion, I will change the fragmentation description I
posted to reflect this as well as some other changes that Anton
suggested and that I posted  messages on.

>
> As I understand -international it needs a kind of "super messages
> (longer that fit in a transport syslog message). I'm not aware of
> details, but ...
>
> Please just specify how to fit "long messages" in serveral transport
> packages. (That is done before: "fragmenting", maybe reuse that
> knowledge:-)

well... I tried to reuse what was done in -sign in this regard. Now, it
doesn't look like a real clever idea ;-)

> And asume the tarnsport layer works (most of the time).
>
> Then, it will be simple and working. even for UDP-syslog. Make sure
> the algorithm doens't break when a UDP message is lost, but live with
> the lost-data! If a system(admin) can't live with it, he will use
> syslog-reliable as transport anyhow!
> When the system/network is small, and logging not top-priority UDP
> syslog will be fine.
>
> For small networks, UDP is fine: Just set-up a centrall (UDP)
> collector, and add host (physically) at will. As long they use a UDP
> syslog (traditional, rfc3164, -sign, -international, -fragmented, ...)
> all log can be read. And "complex" fragmented messages are still
> reasble, without tools!
> (<<See this as ... (1/2)>> <<... an example(2/2)>>)

Yes, this *is* the spirit of the ABNFs (see above and archive) I posted.
I am glad to hear it is a workable approach - as I said, I will update
the ABNF. There is still a lot of work sitting on my desk, but I hope to
post a message some time later this week.

>  > The important thing is that I think the reboot ID - as you
> describe it -
>  > works for -sign. I was arguing that it does NOT provide
> reply attack for
>  > -international
>
> You correct, it will NEVER work for -international, I think. And It
> should!#
>
>  > So these are the two issues:
>
> There are a lot more, if -international doesn't use -sign or -reliable
> as "transport" Just don't fix them:-)

Sure - but that was a little out of context. I was just talking about
the reboot session ID ;-). In fact, there are tons of things that need
to go into the security considerations. And on top of this, add the
transport security considerations (see 3164 for a good picture... ;)).

>
> Note:
> Maybe we should make a syslog-fragment (or syslog-long) RFC, which
> describes how to send "long (MSG/CONTENT)", by fragmenting. And
> provides ("to above") the functionality to send unlimmited long
> messages over another syslog-transport.
>
> Then -international can use that one to add localization, i18n, or
> whatever.
> However, then it shoud be possible to use those long messages for
> other features to?
>
> Just an idea.

Well, in my point of view this would be the best idea. I, too, suggested
this in August:

http://www.mail-archive.com/syslog-sec%40employees.org/msg01204.html

As it turned out, there was not much support for it. I hold back work on
- -international for a while and resumed after I felt that concensus was
that -international should re-specify the format (as does -sign).
Actually, it would not be a big deal to move the general format
specification to a separate ID, but on the other hand it does not really
hurt if -international contains it.

So far, I have tried to keep the overhead of -international very low and
most things optional. In fact, the ABNFs I am posting to the list do not
even demand an -international cookie to be present. So it will be
possible to follow the message format layed out in -international
without supporting any -international features. The only drawback is
that if you want to support fragmentation, you must also support the
rest of the international header (THIS IS part of the -international
cookie). Again, I think this is not a big deal, as it doesn't hurt to
add this. Especially, as I assume that you must be writing new code if
you intend to support fragmentation...

I am happy that your kind of digest message helped me to go back to the
big picture and not be lost in not-so-important details. I would also
appreciate any additional comments form the WG, especially on the unique
message ID and the idea of having its own ID/RFC for the message format.

If I don't hear otherwise,  I will continue with -international as
discussed and will change the fragmentation as outlined above.

Rainer

------------------------------

Date: Wed, 24 Sep 2003 15:28:27 +0100
From: "[email-masked]" <[email-masked]>
Subject: Misunderstanding ( was  security and reboot (was ...))

Renair wrote, on a reply of a earlier posting of me.

 >> In the context of -sign, this is NO issue at all,[...] However, in
 >> the light of -international, things are quite different.

> But I think you did not get the spirit of my post. I can see this
> because you snipped out the most important part. [...] reads as follows:

I'm sorry, to give that impression. I used the ``[ ... ] '' notation to
shorted the quote. That part IS important.

In short: For security, crypto is needed. That is what -sign is about.
To make that work, a reboot counter can be used. But the security
doesn't depend on that counter.

Similar, a reboot-counter can be used "to make -international work".

However, when implementing (or even reading) the rfc's two kind of
reboot-counters can lead to misunderstanding. Therefor, I suggest (to
both authors of both drafts) to use different words.

Just to give the authors a hint:

For -sign, which basically add the concept of "secure sessions", the
term "session-number" can be used.

For -international, (which draft I not familiar with,) a term as
"fragment-counter" can possible be used.

Hope this helps, and clearifies.

Note to the WG:
On a private discussion about syslog, we (Renair and me) agree we agree
on most issues. So please don't misread our discussion!

- --ALbert Mietus
Send prive mail to:      [email-masked]
Send business  mail to: [email-masked]
		Don't send SPAM!

- -- 
- --ALbert Mietus
Send prive mail to:      [email-masked]
Send business  mail to: [email-masked]
		Don't send SPAM!

------------------------------

Date: Wed, 24 Sep 2003 17:19:19 +0200
From: Rainer Gerhards <[email-masked]>
Subject: RE: Misunderstanding ( was  security and reboot (was ...))

> For -international, (which draft I not familiar with,) a term as
> "fragment-counter" can possible be used.

As I wrote, I think I totally remove the "reboot session
id"/"fragment-counter" as the other ID is easier to generate and (for
the sake of -international) at least equally well.

> Note to the WG:
> On a private discussion about syslog, we (Renair and me)
> agree we agree
> on most issues. So please don't misread our discussion!

Just let me acknowledge this ;-)

Rainer

------------------------------

Date: Thu, 25 Sep 2003 06:06:32 -0700 (PDT)
From: Chris Lonvick <[email-masked]>
Subject: RE: security and reboot (was:RE: fragmentation in -international)

Hi Rainer,

On Wed, 24 Sep 2003, Rainer Gerhards wrote:

>
> And now I think Chris, too, was more or less refering to reboot ID as in
> -sign (different use) [Chris: is this right?]
>

Yes; I was confused as well.  I'm glad you wrote this to straighten it all
out.  :-)

Thanks,
Chris

------------------------------

Date: Tue, 30 Sep 2003 15:05:09 +0200
From: Albert Mietus <[email-masked]>
Subject: -sign: HASHs and  SIGNATUREs (new issue)

Hello WG,
 (Chris, can you add this as an issue?)

I have found some doubt in myself about how to calc the hashes and
signatures. And by rereading draft-12 I was not able to solve them.

So, I want to share my doubt, and give an (one of several)
solutions. Also, I propose some "new text", to be included in the
(draft) RFC
I give this new text, so we can study it, (given the hints in this
mail), and be sure the sign-RFC will be clear.

Both the solution(s) and the proposed text is "as I think the WG has
meant it to be", with 1 exception, That one (no 3) is marked.

In short, some general questions/solutions:

1) A hash of a (normal) syslog-message is over the complete message,
   including field-separating spaces
   In [3.9] this isn't specified. While [3.10] says "excluding the
   spaces" when mentioning signatures.

2) The signature, both in a SignatureBlock and a CertificateBlock, is
   calculated over the hash of the message. Not over message itself
   In [3.10] and [4.3.8] this is vage. It say "the signature". In
   other code and in (the mentioned_ standards about
   signature-algorithms the signature is always calculated over a
   hash.

3) The hash, used for signing the SignatureBlock and the
   CertificateBlock, is calculated over all field excluding the
   signature-field, and excluding spaces direct before the
   signature-field. (the field separator). It includes the PRI-part,
   the HEADER-part and all MSG-part fields upto and including the
   HASHES c.q. fragment-length, including the spaces between them.

NOTE: This is different from the proposed text.
      Which excludes all spaces that are fields-separators. That
      however is harder to implement, but adds no security or
      functionality.
      To implement "without sep-spaces", the code has to decided for
      each space whether it is a "used a space" of used as field
      separator. Doable, but complex.
      Also, it requires to algorithms to calculated a hash. One with
      "all chars", and one "skipping spaces". As said, needless.

Note: the references above [x.y] are section-number in draft-12

Text to be included:
"""
     Calculating Hashes and Signatures

     Before a Signature or Certificate Block can be send, some
     cryptographic calculations needs to be done. Elsewhere in this
     document is specified which algorithms need to be used, and
     where to place the result. This section specifies the data used
     as input for those calculations

     For each device-message (not for relayed messages), a hash SHOULD
     be calculated. It is REQUIRED to use the complete message
     including PRI, HEADER and MSG parts as input for the hashing.
     Those hashes are transited, later, in a Signature Block.

     Both, the Signature Block and the Certificate Block contain a
     digital signature. Those signatures SHOULD be calculated over the
     HASH of the partially composed message. It is REQUIRED to
     calculate the HASH of all parts and all fields of the composing
     message, but the signature-field.  Also, the separating space(s)
     direct before the signature-field NOT SHOULD be part of this
     calculation.  After calculating the HASH and the SIGNATURE, a
     space and the SIGNATURE should appemded to the message. It is
     RECOMMENDED to send this message directly, as the timestamp will
     age.
"""

I think this can be included in chapter 5, as 5.1.*; where the
existing text of chapter 5 become 5.2.* And the name of chapter 5
becomes "details"

- --
ALbert Mietus
	Send prive mail to:      [email-masked]
	Send business mail to: [email-masked]
	Don't send spam mail!

------------------------------

Date: Tue, 30 Sep 2003 12:03:37 +0200
From: Albert Mietus <[email-masked]>
Subject: Issue 2 (TAG)

Based on Rainers text, the issue's/problems  of Chris, and the
expirence in implementing it. The following text is proposed to be
included in syslog-sign.

"""
   The TAG is a string of visible (printing) characters excluding SP,
   that MUST NOT exceed 64 characters in length.
   The first occurrence of a SP (space) will terminate the TAG field,
   but is not part of it.
   It is RECOMMENDED to terminate the TAG with a colon (':'), which if
   used, is part of the TAG.

   The TAG is used to denote the sender of the message. And is
   RECOMMENDED to have the following syntax:

     TAG          =  full-stat-id  full-dyn-id ':'
     full-stat-id =  path path-sep progname
     full-dyn-id  = '[' proc-id thread-part ']'
     path         = VISUAL *
     path-sep	  = (  '/'  |  '\'  )
     progname	  = VISUAL *
     proc-id	  = ALFANUM *  ; recommended: number
     thread-part  = ( EMPTY | thread-sep ALFANUM )
     thread-sep   = VISUAL     ; recommended: ",", or ':', or '.'
     EMPTY	  = "" ; nothing!
     VISUAL	  = ([a-zA-Z0-9...], excusing  '['

(hope I have the this syntax OK --please correct ALbert)

   The PROGNAME part is special, as it is frequently used by relays to
   determine the routing of a message.  As a note to implementors: it
   can be found by getting the visual part before the first occurrence
   of '[', and after the last '\' of '/' part of that segment.

   An example of a TAG is: (without the quotes)
      "/path/to/PROGNAME[123,456]:"

   Systems that use both process-ID's and thead-IDs, SHOULD fill both
   the proc-id and the thread-part. For other systems it is
   RECOMMENDED to use the proc-id only.

   Receivers SHOULD, to be consistent with the format described in
   RFC3164, accept TAGs that terminate with a single colon, without a
   space following it. Then the colon is both the last character of
   that TAG, and the field separator with the next field (MSG).
"""

- -- 
ALbert Mietus
	Send prive mail to:      [email-masked]
	Send business mail to: [email-masked]
	Don't send spam mail!

------------------------------