Subject: sizep parameter again

From: "Martin Stecher" <[email-masked]>
Date: Fri, 17 Oct 2003 15:10:04 +0200

Hi,

I am not sure, please help:

The sizep parameter is the "remaining application data prediction" according to OCP-Core draft.

I see some conflicts:

1. What if the agent knows that it will skip certain message parts? Does it need to substract this skipped size from the sizep parameter? This will break our Content-Length idea.
Or does skipping of parts not affect sizep parameter? Is this then still the "remaining application data"?

2. What about optional parts? Should their size be added to sizep? Will again break Content-Length trick.

3. Trailers belong to the application data. Therefore the DUM message containing the response-header should have a sizep parameter equal to response-body size plus response-trailer size. But this is not the size that we can use for Content-Length correction.

4. If response-header part is sent in multiple DUM messages, only the last DUM message contains the correct sizep value. I am concerned some implementations will not be prepared for this case.

Putting all this together, I feel that using the sizep parameter for Content-Length indication does not really work.
Shouldn't we better introduce a Content-Length named parameter to AMS messages that does what we really want: Promise a length that can be used for HTTP Content-Length header correction?

Regards
Martin

---

From: Alex Rousskov <[email-masked]>
Date: Fri, 17 Oct 2003 11:57:50 -0600 (MDT)

Martin,

	I think you've raised two important design questions that must
be addressed explicitly, possibly in OCP Core (in part).

	(1) We need to be more clear what is being adapted, what is
being supplied for information purposes only, what is being skipped,
what is potentially stale and must be checked by processor (and when
the checks are made).

	(2) We need a "content" size prediction for performance
reasons, but we do not have a definition of "content", only data.
Sizep seems to be useless. A new parameter, tied to the "content"
definition is needed. "Content" definition should probably be
application-specific and negotiable.

On Fri, 17 Oct 2003, Martin Stecher wrote:

> it is not even that the OPES processor MUST NOT by default trust the
> Content-Length header returned by a callout service, actually
> neither agent can trust that header.

Debatable. This depends on where we place processor header-adjustment
requirements: after all processing or after each processing step.

> An OPES processor may call a first callout service and then forward
> the response to a second callout service, i.e. implementing a pipe
> of callout services.

True. A good design will not place the second callout service at a
disadvantage. Thus, either all callout services should assume
out-of-sync headers or processor should fix headers after each
service. The former deprives certain services from required
information (they would have to buffer whole responses to obtain it).
The latter makes pipelining impossible or inefficient in some
situations.

> We should not force the OPES processor to do the header correctness
> check and recalculation after each callout service response but only
> at the very end before sending on the HTTP message on the HTTP path.

That's a design choice, but I tend to agree with your preference
because more systems will benefit from a pipeline than from having
message length to be known in advance. However, I sense that we are
facing a bigger problem here that has to be addressed on a higher
level. See below.

> That means that a callout server may see the HTTP message that was
> already modified by another service and hence some headers may be
> incorrect.

Exactly. Essentially, all headers in this case represent potentially
stale information and are subject to further modification by processor
and other services. On the other hand, a service may want to adapt
some headers. Thus, we have one object (headers) that is partially
stale and partially adapted. This is a mess.

> I would therefore like to require that both agents MUST use sizep
> parameter if they know the message length and no agent SHOULD trust
> the Content-Length header.

That would be a logical conclusion, yes. In that case, I would want to
argue that Content-Length HTTP header MUST NOT be transmitted over
OCP. However, logging services would need complete headers so that is
not acceptable.

> That makes it also simpler for the callout service to use the sizep
> parameter. If the service does not need to parse the HTTP headers
> for its filtering purpose, it would not be nice if it needs to
> lookup the content length header value in order to insert sizep
> parameters. If it already receives sizep from the OCP client, it has
> an easier job.

Agreed.

> The OCP client, as typically being tightly bound to a HTTP proxy,
> has probably easy access to the content length header of the message
> (if any).

Agreed. Even if the bound is not tight, a client must know message
size one way or the other.

On Fri, 17 Oct 2003, Martin Stecher wrote:

> 1. What if the agent knows that it will skip certain message parts?
> Does it need to substract this skipped size from the sizep
> parameter? This will break our Content-Length idea.

sizep should refer to what the after-adaptation application message
will look like. If a service deletes parts of a message, sizep will
decrease. If a service ignores parts of a message but expects
processor to add them, sizep will not change. Moreover, only
content-related parts should be counted.

> still the "remaining application data"?

It looks like the definition has to be polished to reflect the "OCP
client and server are working together on a complete application
message, whether they see/skip/ignore parts or not" view. The
definition of "application message" is to be negotiated. More below.

> Will again break Content-Length trick.

We need to change something here. Sizep should be referring to the
thing we are adapting, not optional header information that may be
supplied. Right?

> 3. Trailers belong to the application data. Therefore the DUM
> message containing the response-header should have a sizep parameter
> equal to response-body size plus response-trailer size. But this is
> not the size that we can use for Content-Length correction.

True. It looks like sizep does not work on application data level (at
that level all data looks the same). It works on a higher level, where
headers are different from bodies different from trailers. I do not
know if we can document sizep in a application-agnostic way.

> 4. If response-header part is sent in multiple DUM messages, only
> the last DUM message contains the correct sizep value. I am
> concerned some implementations will not be prepared for this case.

Sizep should probably not include HTTP headers in a standard HTTP
profile.

> Putting all this together, I feel that using the sizep parameter for
> Content-Length indication does not really work.

I agree.

> Shouldn't we better introduce a Content-Length named parameter to
> AMS messages that does what we really want: Promise a length that
> can be used for HTTP Content-Length header correction?

Does sizep have any useful purpose at the lower (application data)
level it is being used right now? Should we delete sizep from the Core
and add AM-CLP parameter instead (application message content length
prediction)?

Alex.