Subject: RE: Comments on ocp-00

From: "Reinaldo Penno" <[email-masked]>
Date: Thu, 3 Apr 2003 10:02:43 -0800

Hello Alex,

I liked your summary. I guess we should wait for other opinions.

On the other hand, in the case of capability negotiation, I disagree with
your statement that it chooses the wrong options. PPP tons of different
options, has been out there for a while and we can say its capability
negotiation is pretty reliable. SDP's offer and answer model also. There is
also capability negotiation or maybe a offer/answer model in BGP and it
helps a lot.

If you go into the "legacy" world like FAX, the only reason it is so
widespread and you can FAX anybody is because of a good capability
negotiation.

We put capability negotiation in the draft because it is useful. 

"The OPES callout protocol MUST support the negotiation of capabilities and
callout connection parameters between an OPES
processor and a callout server"

Okay, how I write the capability negotiation part?

Regards,

Reinaldo

> -----Original Message-----
> From: Alex Rousskov [mailto:[email-masked]] 
> Sent: Thursday, April 03, 2003 12:44 PM
> To: OPES Group
> Subject: RE: Comments on ocp-00
> 
> 
> 
> Here is a short summary of the pending issues on this thread.
> 
> 	a) OPES processor may be able to pre-process application
> 	  messages (e.g., extract payload). Callout servers
> 	  may be able to handle various kinds of application
> 	  data (e.g., complete HTTP messages versus MIME-encoded
>  	  payload). Thus, somebody needs to tell OPES processor
> 	  and callout server what is an "application message"
> 	  definition that they should both use during OCP
> 	  communications. Should OCP support auto-negotiation or
> 	  rely on out-of-band (e.g., manual) configuration?
> 
> 	b) Do we really need a special "error" flag to say
> 	  "rally bad error; you should probably wipe out all
> 	   related state". Or can we assign the same side-effect
> 	   to some of the result codes?
> 
> 	c) OPES processor can be [a part of] an application-level
> 	  proxy. Can OPES processor be a transport-level gateway
> 	  too? For example, can OPES processor manipulate with
> 	  raw TCP packets and care about things like
> 	  retransmissions and ACKs?
> 
> 	d) If a fragment of an application message is lost,
> 	   [how] should OPES processor signal that to the callout
> 	   server? A loss can happen when adapting lossy application
> 	   protocols.
> 
> 	e) Do we need to group processing of a single application
> 	   message together using OCP transaction concept? Should
> 	   OCP transaction mean something else? Do we need OCP
> 	   transactions at all?
> 
> Detailed responses are inlined below.
> 
> On Thu, 3 Apr 2003, Reinaldo Penno wrote:
> 
> > Humm...I'm really not sure about that. I guess the sysadmin 
> can learn 
> > it, but it would be much easier and debug friendly (if 
> something goes 
> > wrong) to have capability negotiations.
> 
> IMO, auto-negotiation would be "better" for admin if and only 
> if there is a single match between OPES processor and callout 
> server capabilities. If there is no match, auto-negotiation 
> will not help and may hurt (the problem will more likely to 
> be unnoticed until run-time). If there are several matches, 
> auto-negotiation may hurt because it will often select the 
> wrong match.
> 
> > Not to mention that as OPES services become more "rich" a 
> lot of new 
> > capabilities will appear.
> 
> Yes, but you are also implying that a single service and a 
> single OPES processor will poses many capabilities and that 
> there is always a deterministic way to auto-negotiate the 
> best match. I doubt that is going to happen and I doubt it is 
> the best design.
> 
> I think our disagreement is clear. I do not have a strong 
> opinion here (I just would prefer to keep protocol simple if 
> negotiation benefits are uncertain). Let's hear other 
> opinions. If I am the only outlier, we will support auto-negotiation.
> 
> > > > This is my (lean) suggestion using your definitions:
> > > >
> > > > "  An OCP transaction is a logical exchange of OCP messages.
> > > >    A OCP transaction starts with a xaction-start 
> message, followed
> > > >    by zero or more data-xxx messages, zero or more 
> appl-xxx and ends
> > > >    with a xaction-end message. "
> > >
> > > Here I disagree. The whole purpose of having transactions is to 
> > > group together processing of a single application message 
> (which is 
> > > a defined term). That grouping is not strictly necessary but it 
> > > brings structure and is actually required by the OCP requirements 
> > > draft (see the "3.3 Callout Transactions" section).
> >
> > I guess this is more of a performance/device intelligence 
> discussion. 
> > My first point is that having to send xaction-start/stop for every 
> > single application message is quite an overhead. Actually 
> this might 
> > be the biggest overhead I've ever seen. If a HTTP message 
> fits into a 
> > TCP packet we will have to process 2 extra packets for every single 
> > "real" (for the lack of a better word) packet.
> 
> Not at all. The <xaction-start/stop> overhead is probably 
> just a few bytes (it would be known when we decide on 
> transport binding and encoding). These "opening" and 
> "closing" messages will usually fit into the same OCP/TCP 
> packet that carries application data (OCP's data-have message).
> 
> > The other point I would like to make is that having to determine 
> > start/end of application messages means that the OPES 
> processor will 
> > need to understand all these applications, which is not necessarily 
> > desirable.
> 
> Not exactly. Recall that it is up to the OPES processor what 
> to call an application message! However, I strongly believe 
> that in most cases OPES processor will know the application 
> protocol just because such knowledge is usually required to 
> proxy an application protocol correctly. Moreover, parsing an 
> application message will often be desirable because many 
> services are likely to be application-agnostic (e.g., the 
> same virus filter service can be applied to HTTP and SMTP 
> payloads and, ideally, it should not care about HTTP and SMTP 
> headers).
> 
> > Let's suppose I have a callout server that deals with content 
> > filtering. The OPES processor gets TCP packets with HTTP in it 
> > (fragmented or not) and ships them to the Callout server, 
> which will 
> > reconstruct the HTTP message if needed and send it back to the OPES 
> > processor. The OPES processor then decapsulates the 
> IP/TCP/HTTP from 
> > OCP and sends it on its way.
> 
> OCP allows OPES processor to declare a TCP packet on an HTTP 
> connection an "application message". Such a low-level OPES 
> processor will not be able to handle high-level rules and 
> will only work with OCP services that can handle raw TCP 
> packets as input, of course, but it is possible!
> 
> IMO, we may want to assume that OPES processor is an 
> application-level proxy not a transport-level gateway, but 
> others may feel differently. Does architecture draft answer 
> that question? If we are working on an application-level 
> proxy, then there is no such thing for us as a "TCP packet", I guess.
> 
> > The same example would apply for RTP where I just want to 
> send packets 
> > to the callout server for content adaption. I would not 
> like to have 
> > to implement a RTP/MPEG-1 decoder in my OPES processor to 
> know where 
> > application messages end/start.
> 
> (a) you do not have to because you define what the application
>     message is
> (b) AFAIK, RTP message boundaries can be easily determined without
>     any MPEG knowledge or decoding; just like HTTP message boundaries
>     can be determined without understanding the content encoding
>     of the payload.
> 
> > Finally, let's suppose the first packet of a video session was lost 
> > and I can recognize that. Since there is no retransmissions, what 
> > should I do since I can only send whole application messages to the 
> > Callout server? I will never see that first packet again.
> 
> I do not think there is any requirement to send whole 
> application messages to the Callout server. Both OCP draft 
> and OCP requirements draft speak of application message 
> fragments. There should be nothing important in the current 
> OCP draft that prevents an OPES processor to forward partial 
> messages to the callout server. We do need to document such 
> possibility though (and agree on what "offset" values should 
> be in that case).
> 
> > In the case of HTTP I would need to wait for a 
> retransmission before I 
> > can send anything to the callout server. And who can 
> guarantee me that 
> > I will ever receive everything?
> 
> Not sure I follow. In case of HTTP over TCP would would not 
> even know that a packet got lost unless you sit on a TCP 
> level (and then HTTP becomes irrelevant)!
> 
> > I'm still working on the frequency that you can haver more than one 
> > application message into one transaction.
> 
> Yes, we must reach an agreement on what OCP transaction is 
> and whether we need it at all. I was simply working based on 
> the OCP requirements draft, but I see no problem revisiting 
> the issue now, when we can talk specifics.
> 
> > > > Just say "400 Bad Request"...or "400 Malformed Message"
> > >
> > > As Oskar argued, the semantics of "400 Malformed Message" may be 
> > > different from "400 Malformed Message, and the error on this side 
> > > was so bad that you probably want to delete all state associated 
> > > with this transaction even if that state looks valid to you". We 
> > > could have a special code for the latter, but we would 
> need it for 
> > > every error response;  that is why Oskar proposed special *-abort 
> > > messages and I suggested a special "error" flag.
> > >
> >
> > I'm not going to argue too much about this. I will just say 
> that other 
> > text/binary based protocols have been working in a while 
> without the 
> > need for a error flag. This is an infinite cycle. Say, how 
> about it is 
> > a really nasty error, worst than "so bad"? Should we have a 
> flag for 
> > that?
> >
> > If it is bad (that is worst than 400), but not worst than "so bad"? 
> > You know where I'm getting at?
> 
> Yes, I do. I agree that the current solution is not perfect, 
> but suggest to wait for more input from Oskar.
> 
> Thanks,
> 
> Alex.
>

---

From: Alex Rousskov <[email-masked]>
Date: Thu, 3 Apr 2003 13:28:38 -0700 (MST)

> I liked your summary. I guess we should wait for other opinions.

Agreed.

> Okay, how I write the capability negotiation part?

You will see some placeholders in the 00 draft to support negotiation
(message "do-you-support" and below). You can wipe those clean or fill
the blanks, whatever works best for you. If at all possible, please
use XML sources rather than text or HTML rendering. You will find
more-or-less current sources and instructions at

	http://www.measurement-factory.com/tmp/opes/

Let me know if you have any questions, concerns, or suggestions about
the sources or editing process.

Thank you,

Alex.