Subject: [OPS-AREA] Cloud Logging Format

From: Gene Golovinsky <[email-masked]>
Date: Mon, 2 Apr 2012 14:18:44 -0700

Hi Benoit.
During the WG session on Wed last week you suggested IPFIX as an
alternative and/or addition to Syslog.
Who would be recommended expert to help me understand your suggestion?

Thanks.
--Gene

---

From: "Romascanu, Dan (Dan)" <[email-masked]>
Date: Tue, 3 Apr 2012 13:31:51 +0200

Hi Gene, 

Benoit is on vacation, so I will answer on his behalf. Some of the
active IPFIX contributors are Juergen Quittek, Nevil Brownlee, Paul
Aitken, Brian Trammell, and Benoit of course. There are more but this is
a good list to start from. RFC 5101 and RFC 5102 are the first
recommended. Look especially at the way the Information Elements are
being defined and extended by defining new IEs. 

Regards,

Dan

> -----Original Message-----
> From: [email-masked] [mailto:[email-masked]] On
> Behalf Of Gene Golovinsky
> Sent: Tuesday, April 03, 2012 12:19 AM
> To: [email-masked]; Benoit Claise
> Cc: [email-masked]
> Subject: [OPS-AREA] Cloud Logging Format
> 
> Hi Benoit.
> _______________________________________________
> OPS-AREA mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/ops-area

---

From: Gene Golovinsky <[email-masked]>
Date: Tue, 3 Apr 2012 10:23:37 -0700

Dan, thanks a lot!
Any more comments/discussions on the draft itself will be highly helpful
as well.

--Gene

-----Original Message-----
From: Romascanu, Dan (Dan) [mailto:[email-masked]]
Sent: Tuesday, April 03, 2012 04:32 AM
To: Gene Golovinsky; [email-masked]; Benoit Claise
Cc: [email-masked]
Subject: RE: [OPS-AREA] Cloud Logging Format

Hi Gene,

Benoit is on vacation, so I will answer on his behalf. Some of the active
IPFIX contributors are Juergen Quittek, Nevil Brownlee, Paul Aitken, Brian
Trammell, and Benoit of course. There are more but this is a good list to
start from. RFC 5101 and RFC 5102 are the first recommended. Look
especially at the way the Information Elements are being defined and
extended by defining new IEs.

Regards,

Dan

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Wed, 4 Apr 2012 08:56:57 +0200

> Any more comments/discussions on the draft itself will be highly helpful

Let me repeat the question I asked at the meeting. From reading the
draft, it was not clear how the new SDEs are added to syslog messages.
Is it reasonable to assume that the originator of a syslog message has
the relevant information? If not, what does it take to distribute the
necessary information to the originators? Or is the idea that proxies
add SDEs while forwarding syslog messages? I like to see how this can
reasonably implemented and deployed.

I probably also have question what precisely the semantics of the
prosed SDEs are but that perhaps becomes clear once I understand where
the knowledge for which service/customer a log message is being
created is coming from and which component is expected to do the
tagging.

/js

-- 
Juergen Schoenwaelder           Jacobs University Bremen gGmbH
Phone: +49 421 200 3587         Campus Ring 1, 28759 Bremen, Germany
Fax:   +49 421 200 3103         <http://www.jacobs-university.de/>

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Wed, 4 Apr 2012 18:19:54 +0200

On Wed, Apr 04, 2012 at 08:56:33AM -0700, Gene Golovinsky wrote:
 
> Let me repeat the question I asked at the meeting. From reading the draft,
> it was not clear how the new SDEs are added to syslog messages.
> Is it reasonable to assume that the originator of a syslog message has the
> relevant information?

> [Gene Golovinsky] Yes, the originator of the message, the actual logging
> entity should be aware of the information populated into the Syslog
> message

What is a logging entity? RFC 5424 talks about 'originators',
'collectors', and 'relays'. I hear you saying it is the syslog
originator.
 
> If not, what does it take to distribute the necessary information to the
> originators? Or is the idea that proxies add SDEs while forwarding syslog
> messages?

> [Gene Golovinsky] Yes again. As a request for service (API call)  travels
> through processing nodes those nodes add SDs to the message. Each node is
> aware of the context and can add appropriate SD.

I fail to understand this. Which API call? Who is calling this API?
Which nodes add SDs to messages (presumably SYSLOG messages?). Are we
now talking relays?

> I like to see how this can reasonably implemented and deployed.

> [Gene Golovinsky] I am not sure I understand this. It has been actually
> implemented and deployed by several vendors in their "cloud"
> implementations.

Such as? Sorry if I am ignorant. Do typical web servers do this sort
of thing or do you have some specific application frameworks in mind?

/js

---

From: Gene Golovinsky <[email-masked]>
Date: Wed, 4 Apr 2012 14:19:07 -0700

On Wed, Apr 04, 2012 at 08:56:33AM -0700, Gene Golovinsky wrote:

> [Gene Golovinsky] Yes, the originator of the message, the actual
> logging entity should be aware of the information populated into the
> Syslog message

What is a logging entity? RFC 5424 talks about 'originators',
'collectors', and 'relays'. I hear you saying it is the syslog originator.
[Gene Golovinsky] Yes, it is a syslog originator.

> If not, what does it take to distribute the necessary information to
> the originators? Or is the idea that proxies add SDEs while forwarding
> syslog messages?

> [Gene Golovinsky] Yes again. As a request for service (API call)
> travels through processing nodes those nodes add SDs to the message.
> Each node is aware of the context and can add appropriate SD.

I fail to understand this. Which API call? Who is calling this API?
Which nodes add SDs to messages (presumably SYSLOG messages?). Are we now
talking relays?
[Gene Golovinsky] I am referring to some of the scenarios discussed in the
examples in the draft. In "cloud" deployments in lots of cases actions
result in response to a request for service. This request is typically
done by calling a remote API (REST, or other Web Service like call). Such
an API call typically comes to a front end Web Server and then dispatched
to the backend. Backend may consists of multiple compute nodes. In some
cases requested service is satisfied by several compute nodes. Each node
should be able to add SDs to the log message. So these are probably not
relays, but rather distributed originators. These are not to be confused
with service gateways briefly discussed in the draft. Those gateways are
for passing service request/response and are also originators.

> [Gene Golovinsky] I am not sure I understand this. It has been
> actually implemented and deployed by several vendors in their "cloud"
> implementations.

Such as? Sorry if I am ignorant. Do typical web servers do this sort of
thing or do you have some specific application frameworks in mind?
[Gene Golovinsky] No, the typical Web Server does not do this sort of
thing. This is more appropriate for SaaS and IaaS environments. I know of
several SaaS providers that implemented something like this and, at least
one IaaS vendor.

/js

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Thu, 5 Apr 2012 11:52:59 +0200

> for passing service request/response and are also originators.

I understand that you assume that a frontend passes information down
to the backend compute notes so that their implementation can put the
correct SDE into the SYSLOG messages.
 
> several SaaS providers that implemented something like this and, at least

I happen to run a normal web server in an IaaS environment. So I might
just be unlucky. Anyway, I like to see more discussion added to the
document how this is supposed to implemented and deployed. I like to
see less "could" language and see more "as done by XYZ" language or
even text explaining what information APIs need to move around to make
all this work.

And then there needs to be discussion the SDEs semantics and what they
really contain. For example, is it meaningful to identify a client by
an IP address in a world moving to shared NATs? What exactly are those
gateways? What exactly are the user identities and their formats?

/js

---

From: Gene Golovinsky <[email-masked]>
Date: Thu, 5 Apr 2012 07:54:50 -0700

> [Gene Golovinsky] I am referring to some of the scenarios discussed in
> the examples in the draft. In "cloud" deployments in lots of cases
> actions result in response to a request for service. This request is
> typically done by calling a remote API (REST, or other Web Service
> like call). Such an API call typically comes to a front end Web Server
> and then dispatched to the backend. Backend may consists of multiple
> compute nodes. In some cases requested service is satisfied by several
> compute nodes. Each node should be able to add SDs to the log message.
> So these are probably not relays, but rather distributed originators.
> These are not to be confused with service gateways briefly discussed
> in the draft. Those gateways are for passing service request/response
and are also originators.

I understand that you assume that a frontend passes information down to
the backend compute notes so that their implementation can put the correct
SDE into the SYSLOG messages.

> thing. This is more appropriate for SaaS and IaaS environments. I know
> of several SaaS providers that implemented something like this and, at
> least one IaaS vendor.

I happen to run a normal web server in an IaaS environment. So I might
just be unlucky. Anyway, I like to see more discussion added to the
document how this is supposed to implemented and deployed. I like to see
less "could" language and see more "as done by XYZ" language or even text
explaining what information APIs need to move around to make all this
work.
[Gene Golovinsky] This sounds like a very reasonable comment. I will sure
take it into the account

And then there needs to be discussion the SDEs semantics and what they
really contain. For example, is it meaningful to identify a client by an
IP address in a world moving to shared NATs? What exactly are those
gateways? What exactly are the user identities and their formats?
[Gene Golovinsky] This is exactly where I am hoping to get experts to
help! Especially in the identity area.

/js

---

From: Gene Golovinsky <[email-masked]>
Date: Wed, 4 Apr 2012 08:56:33 -0700

Hi Juegen.
Some answers are in-line
--Gene

-----Original Message-----
From: Juergen Schoenwaelder [mailto:[email-masked]]
Sent: Tuesday, April 03, 2012 11:57 PM
To: Gene Golovinsky
Cc: Romascanu, Dan (Dan); [email-masked]; Benoit Claise;
[email-masked]
Subject: Re: [OPS-AREA] Cloud Logging Format

On Tue, Apr 03, 2012 at 10:23:37AM -0700, Gene Golovinsky wrote:
> Dan, thanks a lot!
> Any more comments/discussions on the draft itself will be highly
> helpful as well.

I probably also have question what precisely the semantics of the prosed
SDEs are but that perhaps becomes clear once I understand where the
knowledge for which service/customer a log message is being created is
coming from and which component is expected to do the tagging.

/js