Subject: RE: AW: Using XML in OCP transport

From: "Martin Stecher" <[email-masked]>
Date: Wed, 7 May 2003 10:21:10 +0200

> 
> 	Would it be fair to conclude, then, that your primary reason
> for avoiding XML in OCP is assertion #7 (the use of XML will initially
> alienate a noticeable fraction of the existing ICAP community)?

My concern is that OCP will not be successful, if implementors don't like it or find it too complicated or too much work to implement.

So yes, my primary reason is #7. But regarding new potential implementors it is more an issue of assertions #2 and maybe #6, meaning that an OCP implementation which takes it serious needs fully compliant XML support.

For example: ICAP support in the Squid proxy added less than 1500 lines of code. Any idea what it will take to get OCP/BEEP into Squid? As far as I know there is no XML support in Squid yet.

> 
> My reasoning is that if all of the performance-related assertions
> below are true, then there is no performance problem with XML in OCP
> context: all performance problems can be solved by practical,
> real-world engineering. Such efforts are virtually always required to
> support any new protocol efficiently anyway. If there are no
> performance problems, then (based on you past postings), I assume your
> primary concern is migration of existing ICAP community (i.e.,
> assertion #7). Is my reasoning correct?

I agree and never said that BEEP is not efficient.
I am not concerned about our own implementation although it will probably be somehow tricky to have an optimized XML parser for OCP and still passing the compliance test that probably/hopefully The Measurement Factory will come up with ;-)

But evangelising others that this protocol is definitly the one they should use or migrate to, this task could be a little harder to me. That's my concern.

Regards
Martin

> 
> Thank you,
> 
> Alex.
> 
> 
> > > 	1 An OCP/BEEP implementation would parse simple,
> > > 	  short XML fragments most of the time.
> > >
> > > 	2 To be compliant, an OCP/BEEP implementation would
> > > 	  have to be able to parse arbitrary XML, including
> > > 	  malicious XML.
> > >
> > > 	3 In 7 years, virtually every OCP/* agent will support XML
> > > 	  for reasons unrelated to transport; that support may not
> > > 	  be efficient (e.g., parsing of configuration files or
> > > 	  generating logs).
> > >
> > > 	4 It is possible to optimize parsing of simple,
> > > 	  short XML fragments with known parsing goal
> > > 	  (e.g., just to find the profile URIs and ignore
> > > 	  everything else).
> > >
> > > 	5 It is possible to optimize generation of simple,
> > > 	  short XML fragments.
> > >
> > > 	6 It is very tempting to use XML for OCP negotiations
> > > 	  and other, mostly performance-unrelated OCP tasks _if_
> > > 	  XML has to be supported for transport reasons anyway.
> > >
> > > 	7 The use of XML will initially alienate a noticeable
> > > 	  fraction of the existing ICAP community
> > >
> > > Does anybody disagree with any of the above assertions?
>

---

From: Markus Hofmann <[email-masked]>
Date: Wed, 07 May 2003 10:12:38 -0400

Martin Stecher wrote:

> My concern is that OCP will not be successful, if implementors
> don't like it or find it too complicated or too much work to
> implement.

While I understand the point, question is whether this is so serious 
that it would justify to not use an existing solution, but to spend 
cycles on defining something similar (just with a different 
lock-and-feel).

-Markus

---

From: Alex Rousskov <[email-masked]>
Date: Wed, 7 May 2003 10:10:43 -0600 (MDT)

On Wed, 7 May 2003, Martin Stecher wrote:

> My concern is that OCP will not be successful, if implementors don't
> like it or find it too complicated or too much work to implement. So
> yes, my primary reason is #7. But regarding new potential
> implementors it is more an issue of assertions #2 and maybe #6,
> meaning that an OCP implementation which takes it serious needs
> fully compliant XML support.
>
> For example: ICAP support in the Squid proxy added less than 1500
> lines of code. Any idea what it will take to get OCP/BEEP into
> Squid? As far as I know there is no XML support in Squid yet.

I would think it would take about the same 2000 lines, provided an
existing XML and BEEP Core libraries are used and no session
encryption is required. If Squid folks decide to optimize an XML
parser for OCP, it would probably take another 500 lines, provided an
inefficient XML library is still used for unexpected cases.

While I have not seen any, I am not optimistic that an existing BEEP
Core C++ library would satisfy Squid performance needs. In fact, I do
not see any available open/free BEEP library in C++ on beepcore.org.
For reference, a permaBEEP library has about 30,000 lines of Java
code. I suspect that a decent native BEEP implementation in Squid
would be 7,000 lines or less (without encryption support).

> I agree and never said that BEEP is not efficient. I am not
> concerned about our own implementation although it will probably be
> somehow tricky to have an optimized XML parser for OCP and still
> passing the compliance test that probably/hopefully The Measurement
> Factory will come up with ;-)

You can always take the lazy approach: an optimized XML parser for
simple/canonical/expected cases and a slow XML library for anything
else. The optimized parser would just have to be smart enough to know
when its input is not a "simple/canonical/expected" case.

> But evangelising others that this protocol is definitly the one they
> should use or migrate to, this task could be a little harder to me.
> That's my concern.

I agree. I wonder how we should decide which way to proceed. What is
better:

  - best fit: a simpler, compact, domain-specific transport that
    we have to write (OCPTRAN), increasing OCP adoption chances

  - fast result: a more complex, larger, general-purpose transport
    that we can reuse with some effort and loss of elegance (BEEP),
    decreasing OCP adoption chances

I have a feeling that if "fast result" is the correct choice here,
then we should just polish ICAP instead of doing OCP. On the other
hand, if "best fit" is the way to go, then I am worried about
documenting security negotiations (something BEEP already has).

Alex.

---

From: Markus Hofmann <[email-masked]>
Date: Wed, 07 May 2003 13:24:57 -0400

>     decreasing OCP adoption chances

It's also about what can be done with the resources we currently have. 
  Who would committ to agressively move a new transport forward, 
without slowing down work on the OPES challenges that are at our core.

> documenting security negotiations (something BEEP already has).

What would we loose by "polishing ICAP instead of doing OCP" (over 
BEEP)? It seems that most folks thought the expected benefits would 
justify this approach.

-Markus

---

From: Alex Rousskov <[email-masked]>
Date: Wed, 7 May 2003 13:16:14 -0600 (MDT)

On Wed, 7 May 2003, Markus Hofmann wrote:

> It's also about what can be done with the resources we currently
> have.  Who would committ to agressively move a new transport
> forward, without slowing down work on the OPES challenges that are
> at our core.

I do not think it would be a lot of work to define OCPTRAN because it
will be integrated with OCP and will not be a stand-alone
general-purpose transport protocol like BEEP. We can probable assume
TCP as low-level transport and make many other simplifying,
domain-specific assumptions as well.

As we discussed before, it is not the amount of work to build OCPTRAN
that pauses us, it is our desire to reuse things defined on top of or
for BEEP Core (such as security negotiations).

> BEEP)? It seems that most folks thought the expected benefits would

The original motivation to build OCP was to provide an application
agnostic protocol (ICAP is HTTP-specific) with several key features
missing from ICAP (e.g., multiple adapted response generation or
copying optimizations). We cannot just "polish" ICAP to get most of
those new things supported; we would have to make major changes, which
is what OCP is supposed to do.

If we do not use BEEP, but still do OCP, we can make messages look
similar to ICAP ones. OCP can be a new protocol (with all the desired
features) that just has ICAP-looking messages. Or, to be more
accurate, messages that look more like ICAP/MIME than like BEEP/XML.

Please note that I am not casting my vote here, just documenting and
clarifying available options:

	- "polish" ICAP (e.g., document encryption) [ call it ICAP/1.1? ]
	- write OCP on top of TCP [ call it ICAP/2.0? ]
	- write OCP on top of BEEP [ call it OCP/BEEP ]

Alex.

---

From: Markus Hofmann <[email-masked]>
Date: Wed, 07 May 2003 16:52:27 -0400

> for BEEP Core (such as security negotiations).

And if we don't reuse these things, we've to re-invent them again, 
which adds to the amount of work... More important, however, is that 
we can rely on established solutions and don't have to worry about 
many things, including the security negotiations you mentioned.

> is what OCP is supposed to do.

Absolutely, this is the "benefit" I was referring to.

> accurate, messages that look more like ICAP/MIME than like BEEP/XML.

Sure, that's an option, but we would than have to re-define all these 
things that something like BEEP already offers.

> 	- write OCP on top of BEEP [ call it OCP/BEEP ]

Understood, it's appreciated! And we need to get closure on that one 
to move on. Anyone else a strong opinion?

-Markus

---

From: jfcm <[email-masked]>
Date: Thu, 08 May 2003 00:52:05 +0200

At 22:52 07/05/03, Markus Hofmann wrote:
>Understood, it's appreciated! And we need to get closure on that one to 
>move on. Anyone else a strong opinion?

I fully agree that these are the options. (except that I would also 
consider on top of IP). I suggest that now this is clear, I suppose, to 
everyone, we go a step further and try to quantify the resulting qualities 
(there are not con and pros, just how it will be) of each solutions for a 
decision grid.

The will permit to objectively list which applications best benefit from 
each approach, in which model context. I suppose IAB will ask that kind of 
documentation to support our choice?

If we see and document that several solutions are in real demand, I have 
nothing against an OPES family of protocols. As you know I think OPES are 
just the begining of a total revamp the Internet access concept into an 
internet (ONES) service concept.
jfc

---

From: Alex Rousskov <[email-masked]>
Date: Wed, 7 May 2003 22:25:24 -0600 (MDT)

On Thu, 8 May 2003, jfcm wrote:

> consider on top of IP). I suggest that now this is clear, I suppose,
> to everyone, we go a step further and try to quantify the resulting
> qualities (there are not con and pros, just how it will be) of each
> solutions for a decision grid.

Polished ICAP/1.1:
	- OCP stays application specific (though we may be able
	  to hack SMTP support, in since real implementations already
	  do that now)
	- less OCP work for us
	- little incentive to upgrade existing ICAP installations and
	  implementations
	- very easy migration
	- implementation complexity comparable to ICAP/1.0
	- no XML worries

New ICAP/2.0 on top of TCP:
	- OCP becomes the best protocol we can come up with, fixing
	  known ICAP problems or limitations
	- more OCP work for us
	- more incentive to upgrade existing ICAP installations and
	  implementations
	- preserves original ICAP look and feel to ease the migration
	- implementation complexity somewhat higher than of ICAP/1.0
	- no XML worries
	- some pummeling from IETF powers for not reusing BEEP?

OCP on top of BEEP:
	- OCP fixes known ICAP problems or limitations
	- some tension between BEEP and OCP transport needs will
	  be visible in protocol design and may limit BEEP library
	  reuse (but we do not expect much code reuse anyway)
	- less work for us when it comes to transport security
	  negotiations and such (BEEP community already did that)
	- more incentive to upgrade existing ICAP installations and
	  implementations
	- more difficult migration path
	- high implementation complexity
	- XML worries
	- kudus from Marshall and IETF powers for reusing BEEP?

I am sure I missed some points, but I hope the above is sufficient for
us to start making the final decision.

Alex.

---

From: jfcm <[email-masked]>
Date: Thu, 08 May 2003 14:33:13 +0200

This is a very good start. Thank you.
I suggest we put that on an exell table (I can do that, but may be you are 
more informed) and to keep an HTML page on it on the site. I am not 
technically good enough to understand everything in here (I am here to make 
sur my needs are covered and to learn how I could cover them should them 
not to be met). But I feel there may be other points and I understand this 
is seen from the OCP developper point of view, not so much yet from the 
developper having to use OCP point of view?
jfc

At 06:25 08/05/03, Alex Rousskov wrote:

>---
>Incoming mail is certified Virus Free.
>Checked by AVG anti-virus system (http://www.grisoft.com).
>Version: 6.0.474 / Virus Database: 272 - Release Date: 18/04/03

---

From: Alex Rousskov <[email-masked]>
Date: Thu, 8 May 2003 10:23:36 -0600 (MDT)

On Thu, 8 May 2003, jfcm wrote:

> I suggest we put that on an exell table (I can do that, but may be
> you are more informed) and to keep an HTML page on it on the site. I
> am not technically good enough to understand everything in here (I
> am here to make sur my needs are covered and to learn how I could
> cover them should them not to be met).

I could render the lists below in HTML, but I would like to hear
others feedback first. It seems to me that we should be ready to make
a decision on OCP transport now -- all these points have been
discussed already.  I am OK with more clarification and discussion if
needed, of course, but I want to avoid spending time on documentation,
especially if others do not find this useful and need something else
to select the "best" OCP transport.

> But I feel there may be other points and I understand this is seen
> from the OCP developper point of view, not so much yet from the
> developper having to use OCP point of view? jfc

I am sure there are other points. These biased lists are based on two
points of views: OCP author and OCP/ICAP implementor.

Alex.

P.S. I would rather not do an Excel spreadsheet because I do not
     usually use Microsoft products.

---

From: jfcm <[email-masked]>
Date: Thu, 08 May 2003 20:06:43 +0200

>points of views: OCP author and OCP/ICAP implementor.

I would like we clarify which is which (author, implemtator to try
to best decide and document);

I do not really see a product description for an implementator.
Why to chose each solution.

>      usually use Microsoft products.

I made a draft in HTML. I can keep doing it if I receive additions
from others. http://jefsey.com/ocp.htm

I left a column DI to mark each point as a point for developper,
for implementator.
jfc

---

From: Alex Rousskov <[email-masked]>
Date: Thu, 8 May 2003 12:41:25 -0600 (MDT)

> to best decide and document);

Both authors and implementors/coders care about the scope (item #1)
and XML worries.

"less/more OCP work for us" is mostly for authors; however, Marshall
and others may argue that this group is doomed to produce inferior
specs and, thus, the less work we do ourselves (the more we reuse),
the better for everybody involved, including coders.

Implementation complexity is mostly for implementors/coders but it
also reflects authors ability to write simple protocols.

Migration motivation and complexity is mostly for those who
advocate/market the new protocol or sell/install products based on the
new protocol.

IETF pummeling is mostly for us, the authors. However, the same factor
can be viewed as "reuse of existing technology".  The latter is
important to implementors.

> I do not really see a product description for an implementator.

By "implementor", I meant those who will code or program OCP
specification. What those people need is a protocol specification. We
already have most of it for ICAP/1.1 case. We probably have enough for
the other two cases; see the current OCP Core draft and the BEEP
specification, and I also posted some examples.

By "author" I meant us, the working group.

> from others. http://jefsey.com/ocp.htm

You can make it a table to ease comparison (something I could not do
in ASCII in the original e-mail): "OCP flavors" as horizontal heading,
"decision factors" as vertical heading, and things like
none/low/average/high as table cells.

Alex.

---

From: jfcm <[email-masked]>
Date: Thu, 08 May 2003 21:42:36 +0200

>none/low/average/high as table cells.

Well I udnerstand I could do that as another page.

decision element    |  ICAP 1.1 | ICAP 2.0 | BEEP
------------------------------------------------------------------------

blahblah ...........       none/low.....
Please have a look at http://jefsey.com/ocph.htm

Is that what you suggest? I think it is good. But who is
going to tell me what to enter? I am right now trying to
keep some order in the icann@large election and I
know from experience that you are better off not being
at the same time the secretary and a decision maker ;-)

jfc

---

From: Alex Rousskov <[email-masked]>
Date: Thu, 8 May 2003 13:53:28 -0600 (MDT)

On Thu, 8 May 2003, jfcm wrote:

> Is that what you suggest?

Yes, except by "decision factor/element" I meant the same 6-7 text
phrases you have at http://jefsey.com/ocp.htm. Those phrases already
contain hints of what corresponding cell values should be.

Alex.

---

From: Markus Hofmann <[email-masked]>
Date: Thu, 08 May 2003 17:31:31 -0400

> 	  to hack SMTP support, in since real implementations already

Although our charter does not necessarily require us to be application 
agnostic, general consensus of the WG has been that our solution 
SHOULD try to be so. As such, I would assume that the WG doesn't want 
to follow this approach.

> 	- implementation complexity somewhat higher than of ICAP/1.0

We would have to worry about and to work on many things that are 
already solved by BEEP, thus possibly distracting from making progress 
on the "main delta" to existing solutions.

Would anyone commit the time and resources required to drive this 
forward in a timely manner without impacting our progress on the core 
OPES issues?

> 	- some pummeling from IETF powers for not reusing BEEP?

I wouldn't worry about this if we have good arguments and a strong 
case for not using BEEP.

> 	- high implementation complexity

It's not just "less work", it would also simplify life in avoiding 
pitfalls and in "solving" challenging problems others already took 
care of in the past.

> 	- kudus from Marshall and IETF powers for reusing BEEP?

That should not be the decision making factor, we should focus on 
technical issues.

-Markus

---

From: Markus Hofmann <[email-masked]>
Date: Thu, 08 May 2003 17:35:19 -0400

> the better for everybody involved, including coders.

This group is very well able to produce the desired results, if we 
manage to focus on the most important things!

-Markus

---

From: jfcm <[email-masked]>
Date: Thu, 08 May 2003 23:25:57 +0200

>contain hints of what corresponding cell values should be.

Could you be so kind as to phrase them in good technical English for me?
I can do the clerical job, but I would not venture farther when 
publications are considered :-)

Also, I suggest everyone to do the same if they consider points are missing.

It is near midnigh in my place. I must go home. But if have your entries 
tonigh it will be ready tomorrow morning your time. So we can commence to 
argue on it. This could match the WE target.
jfc

---

From: "Oskar Batuner" <[email-masked]>
Date: Fri, 9 May 2003 17:56:23 -0400

Alex, 

Can you please elaborate on some of the points in your comparison chart:

>Polished ICAP/1.1

What specific problems should be solved? What problems remain?
How will it work with non-HTTP application protocols, especially 
streaming? If I recall correctly streaming was declared as a 
"desired" goal. 

> 	- little incentive to upgrade existing ICAP installations and

IMHO the incentive is driven by the problems solved in the new version, 
not by the passion for innovation. If an upgrade solves important 
problem - incentive becomes high, if this upgrade is small - incentive 
becomes even higher. The same applies to BEEP - in my view the following 

> 	- more difficult migration path

Another question - OCP on top of BEEP vs. OCP/OCPTRAN:

Do you agree with the following statement:

The most attractive/useful BEEP features for OCP are security 
negotiation mechanism and channel control/multiplexing connection, 
the most problematic are XML syntax and message exchange styles.

If yes - we can combine the best of two worlds by copying 
security and channel control mechanisms and just describing them 
in HTTP style syntax. A kind of "white box reuse". Should also 
satisfy some of "IETF powers" concerns as we do not invent any 
new and maybe inferior transport mechanisms - just grammar.

Oskar

> -----Original Message-----
> From: [email-masked]
> [mailto:[email-masked]]On Behalf Of Alex Rousskov
> Sent: Thursday, May 08, 2003 12:25 AM
> To: OPES Group
> Subject: Re: AW: Using XML in OCP transport
> 
> 
>

---

From: Alex Rousskov <[email-masked]>
Date: Fri, 9 May 2003 16:38:20 -0600 (MDT)

On Fri, 9 May 2003, Oskar Batuner wrote:

> What specific problems should be solved? What problems remain? How
> will it work with non-HTTP application protocols, especially

The biggest problem that must be solved is optional encryption of ICAP
connections, I guess.  ICAP/1.1 will not be able to support protocols
that differ from HTTP by much (at least not without a lot of
encoding/decoding on each end of the ICAP connection). Partial support
for SMTP and similar MIME-based protocols is probably doable since
some existing ICAP vendors support SMTP.

> problem - incentive becomes high,

Agreed. ICAP/1.1 does not solve any important problems, IMO.

> if this upgrade is small - incentive becomes even higher.

Please do not mix the two. Incentive is why you want to upgrade (the
reward).  Migration difficulty is the cost of an upgrade. Ice cream is
tasty (incentive/reward), but may cost too much (migration difficulty
to get ice cream).

You may want to upgrade very much, but not being able to due to
resource constraints or whatever (e.g., it takes a year to code up the
new protocol or you must pay somebody $$$ in royalties to use the new
protocol).

> The same applies to BEEP - in my view the following

No contradiction, please see above for a clarification. In other
words, OCP/BEEP is more rewarding than ICAP/1.1 but migration to it is
more costly.

> the most problematic are XML syntax and message exchange styles.

Not exactly. Here is my wording: The most attractive/useful BEEP
features for OCP are message fragmentation and connection- and
channel-scoped negotiation mechanisms. The most problematic are
message exchange styles and channel management (or async support)
style. XML comes next, I guess.

> in HTTP style syntax. A kind of "white box reuse".

Yes, that is what I referred to as "clone and modify". It is not "white
box" because you break any connection with the original protocol(s)
after you cloned the existing ideas. For example, if somebody adds
another BEEP profile or modifies/improves an existing BEEP profile,
then OCP/OCPTRAN is not going to benefit without more work.

On a positive side, that "extra conversion work" may not be difficult
and might even be semi-automated (which is why I talked about the
approach). But we should not fool ourselves that we are combining the
best of two worlds -- we are just reusing ideas instead of reusing
protocols/formats.

> Should also satisfy some of "IETF powers" concerns as we do not
> invent any new and maybe inferior transport mechanisms - just
> grammar.

Perhaps, although I cannot predict powers reaction to a semi-formal
XML to MIME-like conversion of BEEP profiles.

Alex.

---

From: "The Purple Streak, Hilarie Orman" <[email-masked]>
Date: Fri, 9 May 2003 16:42:48 -0600

I cannot agree with this:

>  negotiation mechanism 

Hilarie

---

From: "Oskar Batuner" <[email-masked]>
Date: Fri, 9 May 2003 20:10:46 -0400

> From: The Purple Streak, Hilarie Orman [mailto:[email-masked]]
> Sent: Friday, May 09, 2003 6:43 PM
> To: [email-masked]
> Cc: [email-masked]; [email-masked]
> Subject: RE: AW: Using XML in OCP transport
> 
> 

In this case we have to invent a better one. Should be doable. 
What is wrong with the BEEP security negotiations mechanism 
than should be fixed in OCP?

Oskar

---

From: jfcm <[email-masked]>
Date: Sat, 10 May 2003 02:18:13 +0200

updated http://jefsey.com/ocph.htm
Thank you Alex to add/correct first part lines.

My own position is
- ICAP 25
- OCPTRAN 50
- BEEP 25

because of a different aspect: the maintenance over time. To remind that we 
do not have only to consider the protocol now, but also its evolution as 
OEPS is a new field and new needs will probably add. I am ready to review 
that.

Feel free to give and change your own evaluation as the debate evolves. 
This is not a vote. Just an help if it is usefull.
jfc

---

From: Markus Hofmann <[email-masked]>
Date: Wed, 14 May 2003 10:43:46 -0400

>> The most attractive/useful BEEP features for OCP are security 

Hilarie,

can you elaborate a little bit more about this? What exactly is 
"wrong" with the security negotiation mechanisms in BEEP?

I try to understand whether you believe we could re-use and build on 
top of BEEP's mechanisms, or whether you believe we need different ones.

Thanks,
   Markus