Subject: RE: OCP transport nomination

From: "Martin Stecher" <[email-masked]>
Date: Thu, 24 Apr 2003 18:39:57 +0200

Alex,

could you already give some real life examples how OCP over BEEP will look like?

I am not sure how much XML and Content-Type statements will be necessary to be BEEP compatible.

While I like many of the BEEP concepts and agree that they are very useful for OCP, I am not sure that I will like the protocol's look and feel.
This may change by seeing a real example, or it will encourage me to create an alternate proposal ;-)

That's why I am also very interested how OCPTRAN could look like.

Regards
Martin

> -----Original Message-----
> From: Alex Rousskov [mailto:[email-masked]]
> Sent: Thursday, April 24, 2003 5:41 PM
> To: [email-masked]
> Subject: OCP transport nomination
> 
> 
> 
> 
> I would like to nominate BEEP as the only OCP transport. My primary
> reasons for nominating BEEP are:
> 
> 	- reuse of BEEP negotiation capabilities for
> 	  transport encryption and such
> 
> 	- availability of efficient transfer for
> 	  opaque ("binary") data
> 
> There are several issues that we would need to resolve if BEEP is
> selected by the group (e.g., selecting BEEP message exchange model and
> defining OCP message encoding), but I would like to hear other
> protocol nominations (Hilarie?) or any objections to BEEP before
> discussing BEEP-specific details.
> 
> I hope I am not making a mistake by giving BEEP a preference over
> SOAP. I would really like to use SOAP because that is what web
> services world is using, and we are doing something very similar.
> 
> Unfortunately, SOAP suffers from a legacy of being started as an RPC
> protocol and has no standard way of handling large opaque data
> [efficiently]. If BEEP is selected, we would essentially trade
> "efficient transfer" for "current deployment". I hope that is the
> right choice, especially since SOAP can still be used as another
> callout protocol in environments where efficient transfer is not
> important. If you think otherwise, please speak up now.
> 
> 
> We have talked about all known transport candidates except Hilarie's
> OCPTRAN.  Let's move this discussion to a closure. OCP work cannot
> proceed much further without the transport selection.
> 
> Thank you,
> 
> Alex.
>

---

From: Alex Rousskov <[email-masked]>
Date: Thu, 24 Apr 2003 11:49:10 -0600 (MDT)

On Thu, 24 Apr 2003, Martin Stecher wrote:

> could you already give some real life examples how OCP over BEEP
> will look like? I am not sure how much XML and Content-Type
> statements will be necessary to be BEEP compatible.

Here is an example of what XML and MIME exposure is _required_ by
BEEP. The actual exposure to XML and MIME headers (beyond this
example) would depend on us. "C" is for OPES processor. "S" is for
callout server.

* OCP connection (BEEP session) and OCP transaction (BEEP channel)
  establishment; the transaction establishment part is missing
  OCP control messages (those are given in the next example):

      S: <wait for incoming connection>

      C: <establish connection>

      S: RPY 0 0 . 0 201
      S:
      S: <greeting>
      S:   <profile uri='http://www.ietf-opes.org/profiles/OCP' />
      S: </greeting>
      S: END

      C: RPY 0 0 . 0 52
      C:
      C: <greeting />
      C: END

      C: MSG 0 1 . 52 133
      C:
      C: <start number='1'>
      C:   <profile uri='http://www.ietf-opes.org/profiles/OCP' />
      C: </start>
      C: END

      S: RPY 0 1 . 201 100
      S:
      S: <profile uri='http://www.ietf-opes.org/profiles/OCP' />
      S: END

* Sending an OCP control message over a BEEP channel:

      C: MSG 1 0 . 0 50
      C: Content-type: application/beep+ocp
      C:
      C: are-you-there xid ...
      C: END

      S: REP 1 0 . 0 61
      S: Content-type: application/beep+ocp
      S:
      S: i-am-here xid ...
      S: END

* Sending data message over a BEEP channel (note that we may want to
  establish a new channel for data, in addition to control channel for
  command messages; here, we are using channel #2, am-id=1234):

      C: MSG 2 1234 * 0 1634
      C: Content-type: whatever-was-negotiated
      C:
      C: ... 1634 data octets, with more to come ...
      C: END

      C: MSG 2 1234 * 1634 65536
      C: Content-type: whatever-was-negotiated
      C:
      C: ... 65536 data octets, with more to come ...
      C: END

      C: MSG 2 1234 . 67170 100
      C: Content-type: whatever-was-negotiated
      C:
      C: ... last 100 octets of the data ...
      C: END

Most numbers/parameters are wrong/random. I am just illustrating
XML/MIME exposure here. You can notice that we must use XML for BEEP
state management and we must use explicit Content-types for anything
other than application/beep+xml.

Note that if we define our own BEEP messages, we can avoid repetitions
of "Content-type:" headers by defining our own defaults. BEEP defaults
to application/beep+xml. We will lose compatibility with existing BEEP
libraries/implementations though, so it may not be such a good idea.

Disclaimer: The first example is actually based on
draft-ietf-syslog-reliable-12 that uses BEEP as transport. I am not a
BEEP expert yet, so the above may have conceptual bugs.  Please
correct them if they affect XML/MIME exposure issues.

> While I like many of the BEEP concepts and agree that they are very
> useful for OCP, I am not sure that I will like the protocol's look
> and feel. This may change by seeing a real example, or it will
> encourage me to create an alternate proposal ;-)
>

I too would be excited to work on a "better BEEP" or "better SOAP". I
just think that existing protocols must be considered first (or in
parallel). If we can convince ourselves that BEEP is good enough, we
will save the working group a lot of work.

Alex.

---

From: "The Purple Streak, Hilarie Orman" <[email-masked]>
Date: Thu, 24 Apr 2003 13:33:29 -0600

This looks pretty good.  The major performance issue is whether or not
the content message headers can be easily parsed without calling any
XML library.  The second issue is the same question for transaction
establishment.  The examples make it look quite easy.  It is imperative
that we be able to avoid calling XML libraries for these operations.

I'm not sure how much of an advantage BEEP's security mechanisms turn
out to be.  It doesn't seem to have any specific support for detailed
security policy, just closes out existing sessions, starts a TLS
connection, reopens the sessions on the new connection (might have
terminology confused here).  

Hilarie

---

From: Marshall Rose <[email-masked]>
Date: Thu, 24 Apr 2003 14:10:05 -0700

[ speaking as beep guy, and not co-chair... ]
    
> I am just illustrating

alex - perhaps a better way of explaining it is that each beep
interaction exchanges a mime object over an 8bit-clean path.
    
mime objects are serialized as some headers, e.g., Content-Type:, a
blank line, and then the body. the default content-type is
application/octet-stream, which means that if the serialized object
starts with a blank line, then
    
        Content-Type: application/octet-stream
    
is assumed (not application/beep+xml). this is the "optimize for binary"
hack that dave crocker detests so much (dave doesn't like defaults).
    
the use of xml in beep can be limited to channel 0, which is used for
channel management (e.g., open a channel for doing ocp). otherwise, beep
really doesn't care how one structures the data it sends.
    
/mtr

---

From: Marshall Rose <[email-masked]>
Date: Thu, 24 Apr 2003 14:14:25 -0700

[ speaking as the beep guy, and not the co-chair... ]

> that we be able to avoid calling XML libraries for these operations.

technical, "content message" headers are "mime things", not "xml things". you
can certainly write your own parser to handle the strict 1.0 version of xml that
beep uses for channel management, etc., if you're so inclined.

> security policy, ...

nor would you want it to, given that security policies are
either domain-specific or are so general as to lack meaning.

the advantage is that you don't have to re-invent the wheel (since someone
already re-invented it for you). in other words, the wg doesn't have to argue
about how to encode tls negotiations, or sasl negotiations, etc. the wg doesn't
have to argue about the relationship between tls and sasl, etc.

a second advantage, is that you get to use the current practices developed by
the ietf's application area, without having to think about it. it is fair to say
that a great deal of the motivation behind beep was to give wg's the ability to
avoid thinking about the administrative aspects of protocol design.

/mtr

---

From: Alex Rousskov <[email-masked]>
Date: Thu, 24 Apr 2003 15:23:32 -0600 (MDT)

> [ speaking as beep guy, and not co-chair... ]

Since Dave Crocker is not around, we can assume the above is the
default :-).

> is assumed (not application/beep+xml). this is the "optimize for
> binary" hack that dave crocker detests so much (dave doesn't like
> defaults).
>

Sorry I misread the default. This probably means even less overhead
for data exchanges over BEEP if OCP agents agree with that default.
My earlier examples lack
	Content-Type: application/beep+xml
lines where XML is involved (because I incorrectly thought it was the
default type).

Thanks for clarifying this.

Alex.

---

From: Alex Rousskov <[email-masked]>
Date: Thu, 24 Apr 2003 15:44:43 -0600 (MDT)

On Thu, 24 Apr 2003, The Purple Streak, Hilarie Orman wrote:

> This looks pretty good.  The major performance issue is whether or
> not the content message headers can be easily parsed without calling
> any XML library.  The second issue is the same question for
> transaction establishment.  The examples make it look quite easy.
> It is imperative that we be able to avoid calling XML libraries for
> these operations.

If we do not use XML for our own (OCP) purposes, then whether you need
sophisticated XML parsing libraries/code depends on how compliant you
want your implementation to be.

For example, in most cases, one can just search (a simple strstr(3) in
C) for the only profile ID that implementation supports without even
understading XML! This will, of course, lead to compliance violations
and interoperability problems. But you could have a configuration
option (fast versus compliant parser).

It is probably also possible to write a OCP/BEEP-optimized XML parser
that remains compliant but fast and correctly ignores anything that
OCP/BEEP does not care about. It is more work that strstr(), but it is
not a lot of work. I suspect implementations concerned about XML and
performance will do something like that unless they really do not care
about compliance.

Alex.

---

From: "The Purple Streak, Hilarie Orman" <[email-masked]>
Date: Thu, 24 Apr 2003 16:14:20 -0600

>  either domain-specific or are so general as to lack meaning.

The expression of security policy in the TLS context comes down to a
list of cryptographic preferences.  I didn't see any way through BEEP
to encode anything about TLS negotiations.  Perhaps I'm not
sufficiently adept at BEEP navigation to find the profile document -
all I saw was a URI, and my assumption was that the single option was
"do TLS".  What document shows how to encode a request for mutual
authentication, encryption algorithm, authentication mechanism?

Hilarie

---

From: Marshall Rose <[email-masked]>
Date: Thu, 24 Apr 2003 15:24:49 -0700

> authentication, encryption algorithm, authentication mechanism?

well, where i come from we refer to that as a negotation issue not a
matter of security policy. (although i suppose one could categorize the
former as a subset of the latter). to me a security policy refers to
things like authorization, access control, etc., which, by definition
are domain-specific.

if you look at the security considerations section of the various rfcs
that use beep (e.g., 3195, 3288), you'll see that things like
cryptographic preferences is dictated there. things like cryptographic
preferences are already encoded in the tls negotiation...
    
/mtr

---

From: jfcm <[email-masked]>
Date: Fri, 25 Apr 2003 14:53:32 +0200

I just want to recall my "DNS support filter" (I want the final solution to 
be operational to support DNS.2 - DNS new generation- and DNS+ - exended 
services). They may not call on TCP and work on UDP.

There are many OPES services which my consist of short information being 
changed, upgrated, obtianed, collected, etc. that UDP could support faster?
jfc

---

From: Alex Rousskov <[email-masked]>
Date: Fri, 25 Apr 2003 11:40:28 -0600 (MDT)

On Fri, 25 Apr 2003, jfcm wrote:

> I just want to recall my "DNS support filter" (I want the final
> solution to be operational to support DNS.2 - DNS new generation-
> and DNS+ - exended services). They may not call on TCP and work on
> UDP.
>
> There are many OPES services which my consist of short information
> being changed, upgrated, obtianed, collected, etc. that UDP could
> support faster?

If my view of OPES/OCP is correct, then it makes almost no difference
whether the proxied protocol is based on TCP, UDP, or something else.
OCP is an efficient _callout_ (i.e., orthogonal to the proxied
protocol path) protocol that can use a single TCP connection to
process thousands of small application messages, with little
per-message overhead as long as the set of callout servers used by the
OPES processor is constant and of reasonable size.

Do we expect the set of callout servers to change so rapidly that
server connections cannot be reused at the OPES processor? If yes, we
may have to support UDP-like OCP transport (and probably change a lot
of OCP requirements related to "OCP connections"). If no, then
transport connection setup time is not an issue and TCP-based OCP
transport should work as well (or better) as UDP-based one. I think
the latter is much closer to reality. Am I missing something important
here?

Alex.

P.S. We have to design OCP so that it is possible to process a
     small application message with a single RTT delay (something sent
     to callout server once, something received back once, no
     additional exchanges on OCP level).

---

From: Markus Hofmann <[email-masked]>
Date: Fri, 25 Apr 2003 14:32:35 -0400

> the latter is much closer to reality. Am I missing something important

No, I don't think you're missing anything. You summarized very nicely 
what my understanding has been for a while. Since we expect to 
"re-use" estbalished TCP connections between OPES processor and 
callout server, the TCP connection establishment overhead is probably 
not of big importance.

-Markus

---

From: jfcm <[email-masked]>
Date: Fri, 25 Apr 2003 22:40:40 +0200

On 20:32 25/04/03, Markus Hofmann said:
>No, I don't think you're missing anything. You summarized very nicely what 
>my understanding has been for a while. Since we expect to "re-use" 
>estbalished TCP connections between OPES processor and callout server, the 
>TCP connection establishment overhead is probably not of big importance.

As indicated before the first need I can see is for a pre-resolver. Draft 
not completed: I consider OPES for the job.

1. User enters a DN with a TLD
2. the OPES processor filters on the TLD
3. if unknown string ask call out server(s) if it is an internationalized 
scripting or an existing TLD
4. if yes, corrects and auto-generates the filter + possible change
5. if not, potentially caches the filer (memory size dependant) and may be 
denies the call

Looks very similar to DNS current traffic to me.
1. if the OPES is in front of the ISP nameserver TCP support to one server 
seems OK.
2. if it is in front of the resolver I understand that OCP would not be a 
good choice (fast occasionial needs)
Is that correct?

I note that if several call out servers are to be polled, UDP seems a need 
(I am not good on TCP/UDP)?
jfc

---

From: "The Purple Streak, Hilarie Orman" <[email-masked]>
Date: Fri, 25 Apr 2003 15:33:11 -0600

While re-use is certainly expected, we shouldn't make the overhead so
high that it becomes impractical to open many connections.  Separate
connections might be needed to keep sensitive user data encrypted
under different keys, multiple connections might be desirable for
scheduling streaming applications, etc.  

Hilarie

---

From: Alex Rousskov <[email-masked]>
Date: Fri, 25 Apr 2003 15:55:04 -0600 (MDT)

On Fri, 25 Apr 2003, jfcm wrote:

> As indicated before the first need I can see is for a pre-resolver.
> Draft not completed: I consider OPES for the job.
>

Though I am not sure what you mean by "auto-generates the filter +
possible change", the above does not seem to fit OPES model nicely.
You are talking about a query/response or RPC interface with the
callout server: "is it an internationalized scripting?" While
everything can be viewed as a form of an RPC interface, OPES is
specializing in _adapting proxied traffic_.

It is possible to adjust the above so that it fits OPES better:

...
2. the OPES processor forwards the DNS query (with a questionable DN)
   to a callout server
3. the OPES processor receives a possibly adapted query from the
   callout server
...

> 1. if the OPES is in front of the ISP nameserver TCP support to one server

Yes, though I would rather describe OPES processor as a part of an
[ISP] nameserver to fit OPES model better; recall that OPES processor
is a proxy/hop on the application protocol path, a DNS server in this
case.

> good choice (fast occasionial needs) Is that correct?

What do you mean by "front of the resolver"? How is that different
from "front of the ISP nameserver"?

Fast occasional use is OK as long as both agents are able to keep TCP
connection(s) between them open. If they cannot (for whatever reason),
then "fast occasional use" would require UDP-like transport.

> I note that if several call out servers are to be polled, UDP seems
> a need (I am not good on TCP/UDP)?

The number of servers is not relevant as long as it is reasonable
compared to the capacity of the OPES processor. For example, one can
serve thousands of concurrent TCP connections efficiently on a small
server.

If we think UDP-like transport is a must, we would probably have to
give up (or make optional) OCP/OPES features like encryption and
authentication. Most lossy transports have relatively small maximum
payloads that may not allow for much overhead. In fact, it would be
impossible, for example, to adapt many DNS responses because they are
already close to the maximum packet size.

To summarize, I agree that OCP over TCP (or any connection-oriented
transport) is bad for "fast occasional use" in busy environments. We
have to decide whether there is any application that requires "very
fast occasional adaptation" and that deals with busy processors and/or
services. Perhaps your "front of the resolver" is a good example, but
please clarify what you mean by that.

Thanks,

Alex.

---

From: Alex Rousskov <[email-masked]>
Date: Fri, 25 Apr 2003 16:04:17 -0600 (MDT)

On Fri, 25 Apr 2003, The Purple Streak, Hilarie Orman wrote:

> While re-use is certainly expected, we shouldn't make the overhead
> so high that it becomes impractical to open many connections.
> Separate connections might be needed to keep sensitive user data
> encrypted under different keys, multiple connections might be
> desirable for scheduling streaming applications, etc.

Support for separate connections is a MUST, of course! The overhead we
were talking about is in the connection _establishment_ phase. The
overhead to keep a connection open is usually much smaller and is less
important as far as response time or bandwidth is concerned. The
reason to choose UDP over TCP is usually not the number of concurrent
events (connections or queries) but the cost of creating an event
(opening a connection or sending a packet).

Alex.

---

From: "The Purple Streak, Hilarie Orman" <[email-masked]>
Date: Sat, 26 Apr 2003 12:04:06 -0600

Small requests over TCP should be fine.  The only reason I can see for
supporting UDP is for RTP or some kinds of broadcast (if, for some
reason, the OPES processor would broadcast or anycast to groups of
callout servers).

Hilarie