Subject: AW: OCP transport nomination

From: "Martin Stecher" <[email-masked]>
Date: Mon, 5 May 2003 22:38:26 +0200

Alex,

> [...]
> 
> Also, one good thing about BEEP messages is that they all have a
> known, easily-available size (encoded as an integer value in the first
> line of a message). The above looks similar to HTTP and loses the
> known-size advantage. The size can be added as a "Content-Length"
> header, of course, but you have to parse a lot more to get to that.
> Alternatively, we can add size to the first line and lose similarity
> with ICAP/HTTP.

I also like the size information.
But BEEP is only a little ahead to today's ICAP/1.0.
The size is for the payload only, you still need to parse frame
header and footer. In ICAP/1.0 there is the chunked transfer encoding
which is a little less but mainly the same.
I think that OCP should extend the usage of size information, if
possible even for OCP meta data.

> 
> I think what we need to understand is whether you do not want BEEP
> just because it uses XML or because it does not look like ICAP/HTTP.
> Once we know the true obstacle, we can try to see how BEEP can be
> modified to remove that obstacle.

It's something of both.
First, I assume that a complete OCP on BEEP protocol specification
would define many XML fields like the greeting message as fixed or as
only a small set of possible values.
Only little information could really benifit from XML flexibility.
But a full compatible implementation will nevertheless need a
full featured XML parser as you pointed out earlier.
That looks like a lot of ovehead to me.
I would rather like to get the protocol XML free.

Second, as I wrote before I like to see some migration path for
existing ICAP developers. At least a few syntax elements where we
can say "yes, that looks like a next generation of ICAP".
That's why I proposed to make the session/channel management ICAP
or HTTP like. All other messages could then look much more like
original BEEP messages

> 
> For example, it is probably possible to define OCPTRAN as "XML-less"
> BEEP while providing a simple BEEP-to-OCPTRAN conversion and, hence,
> reusing (redefining without much effort) most of the existing BEEP
> work. This approach would be similar to, say, binary XML that exist
> for wireless protocols (binary XML gets virtually all the advantages
> of XML but uses different representation for various performance
> reasons).

That's basically what I mean. Learning all the good things from BEEP
and reusing as much as possible for OCPTRAN.

Martin

---

From: Marshall Rose <[email-masked]>
Date: Mon, 5 May 2003 14:24:56 -0700

[ speaking as the beep guy...]
    
> and reusing as much as possible for OCPTRAN.

unfortunately, you've taken exactly the wrong lesson here.
    
if you're trying to do beep-like things, you should use beep. because,
frankly, this group isn't going to be able to make better decision
decisions than the group that did beep. quite the reverse.
    
obviously, if there are parts of beep that you don't like, then:
    
    - if they're optional, then don't use them.
    
    - if they're not optional, then you're not trying to do beep-like things.
    
and if the answer is the latter, then go off in another direction, and
that's just fine.
    
if your concern with beep boils down to the xml, then don't use xml in
the channel definition for ocp... use something else.  (yes, you'll
still have to use xml for beep's channel management, but that's very
constrained, and i guarantee you that you have much harder problems to
deal with than trying to optimize channel management in beep.
    
/mtr

---

From: Markus Hofmann <[email-masked]>
Date: Mon, 05 May 2003 23:16:13 -0400

Hi,

generally, I'd hesitate to dismiss or to deviate from an established
standard just to get the protocol syntax closer to something a group
of developers is already familiar with. If an existing protocol fits
our needs, let's take advantage of it and use it the way it is. If it
doesn't really fit our needs, we might be better off defining
something new (rather than trying to tweak an existing approach to our
needs).

 From the discussions so far, it seems that the functionality and the
features provided by BEEP mostly fit our needs. The main concern seems
to be about XML related performance and implementation overhead. An
option might be to use XML only for BEEP's channel management. Would
this address the performance concerns?

Are there other fundamental issues that would speak against using BEEP
and justify a new protocol (or suggest using another existing protocol)?

-Markus

---

From: Alex Rousskov <[email-masked]>
Date: Mon, 5 May 2003 22:38:16 -0600 (MDT)

On Mon, 5 May 2003, Martin Stecher wrote:

> header and footer.

Good point.

> In ICAP/1.0 there is the chunked transfer encoding which is a little
> less but mainly the same. I think that OCP should extend the usage
> of size information, if possible even for OCP meta data.

I agree. The ideal protocol should make it easy and efficient to
"skip" or "extract"  complete OCP messages without much parsing.

> > modified to remove that obstacle.

The worst case scenario :-).

> would define many XML fields like the greeting message as fixed or
> as only a small set of possible values. Only little information
> could really benifit from XML flexibility. But a full compatible
> implementation will nevertheless need a full featured XML parser as
> you pointed out earlier. That looks like a lot of ovehead to me. I
> would rather like to get the protocol XML free.
>
> can say "yes, that looks like a next generation of ICAP". That's why
> I proposed to make the session/channel management ICAP or HTTP like.
> All other messages could then look much more like original BEEP
> messages

Comparing the above to reasons/arguments, it seems to me that the
second is of much lesser importance. The second argument is less
technical and more of a gamble: ICAP developers can be turned off by
even small changes to the existing protocol (NetApp and Akamai are
examples of how [ICAP/1.0] changes [compared to ICAP/0.9] alienated a
part of the protocol community, I guess). On the other hand, most
developers are probably proficient enough to handle whatever protocol
we come up with, from scratch.

Even if you disagree with the above comparison, I would still suggest
that we try to agree on the XML issue first and resolve the "ICAP
path" disagreements (if any) later. More on this in an XML-dedicated
message that follows.

Thanks,

Alex.

---

From: Alex Rousskov <[email-masked]>
Date: Mon, 5 May 2003 22:42:54 -0600 (MDT)

We have already heard several opinions (in my simplified rendering and
interpretation):

	- XML is "not a big deal", everybody is using it [Marshall]

	- XML may be "too slow" for OCP [Hilarie]

	- XML is an "overkill" for the task [Martin]

	- we do not have to use XML for anything other
	  than BEEP channel management [Marshall]

	- supporting just channel management means
	  supporting nearly full XML specs if we care
	  about compliance and interoperability [Alex]

	- other?

If we can make a decision regarding XML use, it will simplify
transport protocol choice because we would be able to ignore whether
the candidates are using XML (if [limited] XML use is approved) or not
(if all candidates do not use XML since its use was not approved).

Is using XML a show-stopper? How are we going to decide? Does anybody
have a suggestion for a process to ensure that "to use or not to use
XML?" arguments are resolved?

Thanks,

Alex.

---

From: Alex Rousskov <[email-masked]>
Date: Mon, 5 May 2003 23:06:51 -0600 (MDT)

On Mon, 5 May 2003, Marshall Rose wrote:

> > That's basically what I mean. Learning all the good things from
> > BEEP and reusing as much as possible for OCPTRAN.
>
> if you're trying to do beep-like things, you should use beep.
> because, frankly, this group isn't going to be able to make better
> decision decisions than the group that did beep. quite the reverse.
>
> and if the answer is the latter, then go off in another
> direction, and that's just fine.

I wish the situation was that simple/straightforward. I do not think
it is. Your comments assume that "beep-like things" are well-defined.
They are not! We are, in fact, trying to answer that very question:
"Is OCP doing something that BEEP is appropriate transport for?"
Clearly, BEEP RFC itself does not (cannot) answer that question --
defined BEEP scope is too broad and not specific enough. Do you,
personally, think that the answer is clear?

There are many aspects to consider, but the primary ones seem to be:

	1) XML (required in BEEP)
	2) ICAP migration (more smooth with OCPTRAN than with BEEP)
	3) message exchange (a single channel may be required for each
	   data and metadata exchange, in addition to control
	   channel(s); OCPTRAN can have better, more natural
	   and efficient mapping)
	4) profile reuse (to support transport security, etc.)

Since channel management (closely related to item 3) requires XML, and
since OCP is going to be different from ICAP anyway, I believe the two
biggest issues are XML and profile reuse.

Alex.

---

From: Alex Rousskov <[email-masked]>
Date: Mon, 5 May 2003 23:28:37 -0600 (MDT)

On Mon, 5 May 2003, Markus Hofmann wrote:

> features provided by BEEP mostly fit our needs. The main concern
> seems to be about XML related performance and implementation
> overhead. An option might be to use XML only for BEEP's channel
> management. Would this address the performance concerns?

Probably not if my understanding of how to naturally map OCP to BEEP
is correct. A natural mapping would require creation of a separate
channel for any [meta]data that needs to be passed to the other OCP
agent, asynchronously or semi-asynchronously:
	- original application message data
	- original application message metadata
	- adapted application message(s) data
	- adapted application message(s) metadata

Ideally, we need one channel for each because, unfortunately, BEEP
does not allow mixing BEEP message frames from different BEEP messages
within one channel, and OCP needs to mix [large] data, [large]
metadata, and [small] control messages. We can reduce the mixing
requirements somewhat (e.g., by requiring that all metadata is known
before data is sent), but we would still end up with 2-3 extra
channels per OCP transaction.

If channel establishment is slow (because of XML or whatever), we have
a problem. The only way to solve this performance problem is to reuse
channels across OCP transactions. This is usually possible from BEEP
point of view (AFAIK), but makes the protocol more difficult to
understand and implement because you lose nice encapsulation of
channels within OCP transaction.

An alternative to channel reuse is an "unnatural" mapping to BEEP
that, essentially, does not use much of the BEEP core features (such
as message framing)  but rather re-implements them hidden from BEEP,
inside simple BEEP messages, to work around BEEP message exchange
limitations. Sort of like doing IP-over-IP encapsulation, I guess.

> Are there other fundamental issues that would speak against using
> BEEP and justify a new protocol (or suggest using another existing
> protocol)?

Another fundamental issue FOR using BEEP is reuse of security-related
profiles (though Hilarie may disagree that there is enough to reuse).

Alex.

---

From: "The Purple Streak, Hilarie Orman" <[email-masked]>
Date: Tue, 6 May 2003 01:04:44 -0600

I think the issue is that while BEEP can be made to fit our needs, as
can many other things, it seems slightly contorted, and it might well
be easier to define exactly what we need.  That's where the discussion
is, thus far.  I'd be interested in hearing opinions about "I could
build OCP on BEEP in X weeks".  That was a problem with ICAP - it could
almost be built cleanly on HTTP with little work, but that then little by
little it drifted away.

I don't see that BEEP provides any particular help for security.

I think that we definitely need to be XML-free in the sense of being
able to write the header parsing code using a few simple, efficient C
routines.  It's OK if a full-weight XML parser can also do the job -
this isn't angle bracket phobia, just a desire to spend more CPU time on
actual OPES services than on header parsing.

Hilarie

---

From: Marshall Rose <[email-masked]>
Date: Tue, 6 May 2003 08:05:07 -0700

[ speaking as the beep guy... ]
    
> personally, think that the answer is clear?
    
if there were a clear description of what ocp's requirements are, then
it would be trivial to answer these questions.
    
in general, if you come up with a new connection-oriented protocol that
does things like sasl, tls, framing, and possibly multiplexing, then you
are doing beep-like things. 
    
just so we're clear: later on today, i'm going back to being the process
guy, and markus can see if he can get the working group to decide what
to do.

> 	1) XML (required in BEEP)
    
anyone who thinks that implementing xml to support beep is a problem has
a very skewed set of priorities...

> 	2) ICAP migration (more smooth with OCPTRAN than with BEEP)
    
sure.

> 	   channel(s); OCPTRAN can have better, more natural
    
hmmm...

> 	4) profile reuse (to support transport security, etc.)
    
anything you can do with tls or sasl, you can do with tls or sasl under beep.

> biggest issues are XML and profile reuse.

amusingly enough, these are the two i see as non-issues...
    
/mtr

---

From: Markus Hofmann <[email-masked]>
Date: Tue, 06 May 2003 11:05:25 -0400

The Purple Streak, Hilarie Orman wrote:

> be easier to define exactly what we need.  

My concern with that approach is that we re-invent many things that 
have already been adressed before, and that the WG spends lots of time 
defining a transport rather than defining the actual OCP.

> I don't see that BEEP provides any particular help for security.

What exactly is missing in BEEP? Could this be build on top of BEEP?

> actual OPES services than on header parsing.

Is parsing a few XML messages for BEEP's channel management such a big 
problem that it would justify defining an alternate protocol over an 
existing one?

-Markus

---

From: Marshall Rose <[email-masked]>
Date: Tue, 6 May 2003 08:15:24 -0700

[ next to last message, speaking as the beep guy ...]
    
> almost be built cleanly on HTTP with little work, but that then little by
    
this probably won't come as a surprise to anyone, but speed and
innovation aren't exactly hallmarks of this working group.
    
when you find yourself in a situation like that, the absolute worst
thing you can do is go off and reinvent a lot of old stuff.
    
if beep doesn't work, don't use beep. if the answer is "we'll define
exactly what we need", then:
    
    YOU HAVE ASKED THE WRONG QUESTION BECAUSE YOU WILL NEVER FINISH

where i interested in seeing this work complete, i would be leveraging
every existing open standards thing i could get my hands on. i wouldn't
care at all about optimality. i would care about getting the job
done before the IESG wises up and figures out that this working group
isn't going to succeed.
    
if i could figure out a way to make this thing work sensibly with
telnet, i'd do it.
    
but, hey, that's just me.    

> I don't see that BEEP provides any particular help for security.
    
you're right: beep doesn't make things more secure. what it does is
integrate tls/sasl into a framing/command mechanism, so working groups
that have better things to do (and know they have better things to do)
can work on better things to do.

> actual OPES services than on header parsing.

XML is not the problem here.
    
/mtr

---

From: Markus Hofmann <[email-masked]>
Date: Tue, 06 May 2003 11:22:55 -0400

> channels within OCP transaction.

Did we convince ourselves that this kind of XML usage is a serious 
(performance) problem? So far, we mostly speculated on the overhead 
and possible performance impact without having hard facts. Does anyone 
have concrete numbers or measurments to share?

If we're not convinced that this is a big, serious problem, we might 
want to move on and focus our time and effort on getting OCP done 
(rather than defining a transport protocol).

-Markus

---

From: Alex Rousskov <[email-masked]>
Date: Tue, 6 May 2003 10:09:15 -0600 (MDT)

On Tue, 6 May 2003, Marshall Rose wrote:

> > I wish the situation was that simple/straightforward. I do not
> > think it is. Your comments assume that "beep-like things" are
> > well-defined. They are not! We are, in fact, trying to answer that
> > very question: "Is OCP doing something that BEEP is appropriate
> > transport for?" Clearly, BEEP RFC itself does not (cannot) answer
> > that question -- defined BEEP scope is too broad and not specific
> > enough. Do you, personally, think that the answer is clear?
>
> if there were a clear description of what ocp's requirements are,
> then it would be trivial to answer these questions.

This is a catch-22. If there were a final description of OCP
requirements, we would not have to have this conversation.
High-level requirements are not detailed enough. OCP details are not
completely known yet. Many OCP aspects can be supported differently,
depending on the transport protocol we chose. We know that OCP is for
efficient asynchronous exchange of [possibly large] metadata,
[possibly large] data, and small command messages. That is "clear" but
not [precise] enough to say whether BEEP is the best way to proceed.

> in general, if you come up with a new connection-oriented protocol
> that does things like sasl, tls, framing, and possibly multiplexing,
> then you are doing beep-like things.

True. But BEEP is not an ideal transport for all connection-oriented
protocols that do things like sasl, tls, framing, and multiplexing (no
single transport can be!). BEEP designers had to make some choices
that made BEEP imperfect in some beep-like environments. Most of these
choices are well documented; use of XML and message exchange
limitations are some of them. We have to decide whether those
imperfections are major or minor in OCP case.

> anyone who thinks that implementing xml to support beep is a problem
> has a very skewed set of priorities...

Anyone who thinks that supporting XML comes for free has a very skewed
set of applications.

>  anything you can do with tls or sasl, you can do with tls or

Yes, #4 is an argument for BEEP, not against it. OCPTRAN cannot reuse
any profiles "as is" because OCPTRAN is a new protocol. The question
is how much more work we would need to do if we start from scratch.

Alex.

---

From: Alex Rousskov <[email-masked]>
Date: Tue, 6 May 2003 10:30:46 -0600 (MDT)

On Tue, 6 May 2003, Marshall Rose wrote:

> where i interested in seeing this work complete, i would be
> leveraging every existing open standards thing i could get my hands
> on. i wouldn't care at all about optimality. i would care about
> getting the job done before the IESG wises up and figures out that
> this working group isn't going to succeed.

Why are you not suggesting that we simply use ICAP/1.0 as OCP, then?
ICAP/1.0 (RFC 3507) exists, works, and is open.

Personally, I think that it is far better for this group to be forced
to close by the wise IESG than to produce OPES protocols that are not
much better than the existing ones. It is better to produce nothing
[new] than to just further dilute protocol set for OPES-like things.
We have to build the "best" OPES protocols to justify this WG
existence.

Alex.

---

From: Alex Rousskov <[email-masked]>
Date: Tue, 6 May 2003 10:51:13 -0600 (MDT)

On Tue, 6 May 2003, Markus Hofmann wrote:

> and possible performance impact without having hard facts. Does
> anyone have concrete numbers or measurments to share?

I do not think anybody has. Some speculate that XML introduces no
overheads. Some speculate that XML is expensive. I doubt it is
possible to answer that question in general. Is XML an issue for
reliable syslog? No, not usually. Would XML be an issue for TCP? Yes,
in many environments.

> (rather than defining a transport protocol).

This would imply that we suspect it is not a big problem. As of now,
Hilarie and Martin have argued that it is. I will try to narrow down
the disagreements in a separate message.

Alex.

---

From: Alex Rousskov <[email-masked]>
Date: Tue, 6 May 2003 10:54:20 -0600 (MDT)

On Tue, 6 May 2003, Markus Hofmann wrote:

> Is parsing a few XML messages for BEEP's channel management such a
> big problem that it would justify defining an alternate protocol
> over an existing one?

Let me try to distill the question further, in hope to isolate the
subset of XML-related issues that lack consensus. I will phrase these
as assertions:

	1 An OCP/BEEP implementation would parse simple,
	  short XML fragments most of the time.

	2 To be compliant, an OCP/BEEP implementation would
	  have to be able to parse arbitrary XML, including
	  malicious XML.

	3 In 7 years, virtually every OCP/* agent will support XML
	  for reasons unrelated to transport; that support may not
	  be efficient (e.g., parsing of configuration files or
	  generating logs).

	4 It is possible to optimize parsing of simple,
	  short XML fragments with known parsing goal
	  (e.g., just to find the profile URIs and ignore
	  everything else).

	5 It is possible to optimize generation of simple,
	  short XML fragments.

	6 It is very tempting to use XML for OCP negotiations
	  and other, mostly performance-unrelated OCP tasks _if_
	  XML has to be supported for transport reasons anyway.

	7 The use of XML will initially alienate a noticeable
	  fraction of the existing ICAP community

Does anybody disagree with any of the above assertions?

Thanks,

Alex.

---

From: Marshall Rose <[email-masked]>
Date: Tue, 6 May 2003 10:52:35 -0700

[ speaking as the beep guy, final message... ]

> ICAP/1.0 (RFC 3507) exists, works, and is open.
    
and if it meets all the requirements (including stuff like security) and
can pass iesg muster, then that's what the working group should use.

> We have to build the "best" OPES protocols to justify this WG

i have yet to attend an engineering meeting where the goal is the
"best". the goal is never the "best". the goal is to get things right
enough, cheap enough, and fast enough to solve enough of the problem and
then move on.
    
i have attended many research meetings, where the goal, as you might
expect is the "best".
    
fundamentally, i have to put the ietf and it's working groups into the
engineering category. others' mileage may, of course, vary.
    
/mtr

---

From: Alex Rousskov <[email-masked]>
Date: Sat, 10 May 2003 23:01:49 -0600 (MDT)

Marshall,

	There are several examples where working groups picked BEEP as
transport. I assume none of those groups regret their choice now. Can
you confirm/deny that they are all satisfied with BEEP, based on your
personal information/perception?

And perhaps more importantly, do you know of any group (dead or alive)
that have chosen a custom transport after seriously considering and
rejecting BEEP? I wonder if we can learn from others mistakes here...

Thank you,

Alex.

---

From: Marshall Rose <[email-masked]>
Date: Mon, 12 May 2003 10:55:42 -0700

[ speaking as co-chair... ]
    
> personal information/perception?
    
alexey - it would be inappropriate of me to comment regarding the
emotional state of ietf working groups.
    
if you want to talk to people, i suggest you contact the chairs of the
calsh and syslog working groups, along with andy bierman of the xmlconf
effort (which is not yet) a working group. alternatively, you can send
an email to the mailing list associated with each of these three.

> rejecting BEEP? I wonder if we can learn from others mistakes here...

[ again, speaking as co-chair... ]
    
it would be inappropriate for me to comment on the ability of this group
to learn from the mistakes of others.
    
/mtr

---

From: Alex Rousskov <[email-masked]>
Date: Tue, 13 May 2003 11:43:27 -0600 (MDT)

> emotional state of ietf working groups.

If you say so. Personally, I do not see a problem with such comments,
especially if they just summarize the facts like "I have seen
virtually no public regrets or complaints about BEEP transport since
it was chosen" or "I am aware of such and such problems that have been
successfully resolved". I assume you have had exposure to those
groups/issues and was simply asking for your personal opinion.

> if you want to talk to people, i suggest you contact the chairs of
> the calsh and syslog working groups, along with andy bierman of the
> xmlconf effort (which is not yet) a working group. alternatively,
> you can send an email to the mailing list associated with each of
> these three.

Calsh and xmlconf groups seems to deal with non-performance-intensive
applications. Syslog has some corner-case performance issues, and it
solves them by aggregating application messages (small log lines)
together, inside one BEEP message, something we cannot do in OCP. In
other words, it seems to me that these groups are/were solving a very
different set of problems compared to OCP, where performance was not a
major concern. Is my understanding correct? Are there any groups that
focus on performance a lot and are using BEEP?

> it would be inappropriate for me to comment on the ability of this
> group to learn from the mistakes of others.

If you say so. IMO, constructive comments, especially negative ones,
may help. Not sure what ethical norms such comments would violate.

Thanks,

Alex.