Subject: P followup...

From: Geetha Manjunath <[email-masked]>
Date: Fri, 05 Dec 2003 15:42:07 +0530

Taking off from Alex's list of current issues on P, pl find my comments
inline...

Alex Rousskov wrote:
> 
> Hi,
> 
>         Here are some big P-issues that may be worth discussing at the
> upcoming F2F meeting.
> 
>         1. What information does the interpreter has
>            access to? Complete messages? Just meta-information?
>            Anything that is small? Do we define that,
>            does the module author, or does the ruleset
>            writer?

I think we should leave this decision of 'what is available' to the
module writer. For example, in the case of HTTP, just knowing the
request header (plus a preview of some data) may be sufficient for major
decisions, however, for SMTP, we may need to look at the complete body
(for example to figure out whether it is a SPAM). However, defining
interfaces between interpreter and module (see below) would help the
'ruleset writer' to use 'what is made available' in a predictable
fashion..

>         2a. Should this WG work on defining interface(s)
>             between P interpreters and module suppliers?

Situations where such an interface definition would help:
* Writing protocol agnostic P programs. For example, how does one write
a P program that restricts service requests from a specific IP?
	if (request.clientIP() != "10.0.*")
		reply.status(402);
If this is what you call as "meta-information", then we should enforce
that certain such parameters be accessible to the P module writer
irrespective of the protocol module used - specify exactly what this
meta information is ? 
* Is it possible to make common objects such as "request" and "response"
could be made to have a certain specific structure across modules ?

> 
>         2b. Should this WG work on defining interface(s)
>             between P interpreters and callout services?

 I see some opportunity for some minimal standardization opportunities
for the modules that supply services too.  From the example from the ID
in Appendix A, 
         service :=
Services.findOne("opes://local.net/add-lcl-content");
   	 service.clientIp(request.clientIp);
 We probably want this example to say:
	service.setParam(clientIP, request.clientIP)
instead.
> 
>         3. Is single-assignment worth keeping if we agree
>            that P deals only with small objects that are
>            cheap to copy?
I think it is nice to keep P as a single assignment language. Apart from
the things we discussed, I beleive this feature is also used to name
code segments. At present that is the only way to define 'functions' in
P - if we want to author modules using P.

The object.value() member may be used to pre-evaluate an object if
needed.

BTW, Do we retain the global scoping of all names in a P module? Or
should we add some scoping constructs too ?

> 
>         4. Should more support for writing modules natively
>            in P be added? How much more support is needed?

If we define the mandatory module interfaces to be supported, I believe
that is sufficient. Remaining things may probably be left a specific
implementation. I beleive that native modules will mostly be to support
specific protocols while all other modules used by a P program would be
portable and will be written in P.

>         5. Should this WG document HTTP module for P?
>            As a part of HTTP Adaptation draft or a separate
>            document references from the adaptation draft?
> 
>         6. If service invocation is just a module method
>            call, how do services return results? Should OCP core
>            document ways to return P-result via the OCP-result
>            structure? See also 2b.

Is it reasonable to insist that every service be wrapped up in a module 
loadable from P interpreter? Since the module can have access to
request/response/message - common elements of a protocol, it is easier
to guanrantee consistency with the rest of the P program.
Consider the case:
	Http := import "http://ietf.org/opes/rules/p/HTTP";
        if (Http.message.headers.have("Accept")) { /* Not executed */}
	Services.applyOne(service)
	if (Http.message.headers.have("Accept")) { /* Executed */ }
> Thanks,
> Alex.

Comments/thoughts welcome...

regards
Geetha

PS: Thanks Markus, for the reminder..

---

From: Alex Rousskov <[email-masked]>
Date: Fri, 5 Dec 2003 10:30:11 -0700 (MST)

Geetha, Chairs,

	We need to do one thing before we can pursue most of the
excellent ideas you have. We must decide what to do with the current P
draft that is [over]due. Most of the stuff we are talking about is
future P work. We need to decide how to deal with our current
liability first.

Here is the relevant milestone from
http://www.ietf.org/html.charters/opes-charter.html

	Oct 03 	Submit document on rules specification method
	        to IESG for Proposed Standard.

I see several options:

	1 Extend the above deadline. This is an admission
	  that the WG failed to schedule a deadline
	  correctly. This does not require AD/IESG review,
	  only WG chair action (AFAIK). This might be
	  blocked by an AD, but since we are delivering
	  on other deadlines, I think our AD can be
	  forgiving.

	2 Submit the current draft now, with minimum changes
	  and no additions. IMO, this is an admission that
	  the WG failed to produce a quality rule language.
	  This requires AD/IESG review which we might pass,
	  but more likely not. We would also have hard
	  time defending half-baked document.

	3 Trim the current draft so that it does not
	  expose known problems and then submit it as
	  Proposed Standard and defend it. I doubt it is
	  possible because the milestone says "specification
	  method" not "rules architecture" or something
	  abstract of that kind. Other opinions?

I would suggest that we go with option #1 and carefully pick the new
deadline based on the todo list you are so eager to jump on :-).

Did I miss any options? Is my understanding of IETF procedures correct
here?

Thank you,

Alex.

---

From: Markus Hofmann <[email-masked]>
Date: Tue, 09 Dec 2003 10:06:44 -0500

> 	  on other deadlines, I think our AD can be

How long an extension would we need? Are we talking days/weeks/months?

-Markus

---

From: Markus Hofmann <[email-masked]>
Date: Tue, 09 Dec 2003 10:18:50 -0500

Geetha Manjunath wrote:

> (for example to figure out whether it is a SPAM). 

IMHO, spam detection should *not* be implemented in form of a rule, 
but rather as an OPES service. In the example, there would be a rule 
that says something like "if email is for Markus, invoke the spam 
detection application", but the rule itself would *not* try to 
implement spam detection.

Please let's be careful where to draw the line. My view is that we are 
*not* chartered to specify a general purpose processing/programming 
language, but a language for specifying rules that are used to 
described when certain actions are invoked (as opposed to encoding the 
action itself inside the rule) - the rules language has a very limited 
scope, which might be blurry and not always clear, but we should be 
sensitive about how far we want to go with the rules language.

For my taste, for example, IRML had all the functionality required. We 
decided to drop IRML in favor of "P" not because of limited 
functionality, but because of syntax preferences. As such, maybe the 
functionality provided by IRML can be a guidance on what we need to 
build into "P" - whenever we feel more flexibility/functionality needs 
to be build in, we should very carefully discuss this.

-Markus

---

From: Alex Rousskov <[email-masked]>
Date: Tue, 9 Dec 2003 10:10:59 -0700 (MST)

> How long an extension would we need? Are we talking days/weeks/months?

IMO, this deadline extension discussion should be a part of the
rechartering discussion. As of now, it is not clear enough (to me)
what P-related activities are in the OPES WG future. Once we know what
we want to do with P, we can scope the P draft better, and decide on
the deadline.

If I have to guess now, I would say that we need weeks worth of
extension with 20% probability and months worth of extension with 80%
probability. Again, this will depend on future P work which we still
have to agree on.

Do we have to decide on this deadline before rechartering? If yes, can
we bump the deadline by some "large enough" number of months (say 6)
and then provide a more accurate milestone in the new charter?

Alex.

---

From: Alex Rousskov <[email-masked]>
Date: Tue, 9 Dec 2003 10:24:52 -0700 (MST)

On Tue, 9 Dec 2003, Markus Hofmann wrote:

> but rather as an OPES service.

If OPES service invocation is indistinguishable from a P module call,
then the above point becomes unimportant from P point of view. We
simply do not have to decide at P Core level (which is good). I
believe that the discussion with Andre concluded that making service
invocation indistinguishable from a P module call is the right way to
go.

> In the example, there would be a rule that says something like "if
> email is for Markus, invoke the spam detection application", but the
> rule itself would *not* try to implement spam detection.

IMO, the line here is too thin to define:

	if (msg.from() contains "Markus") {
		callSpamDetectionService(msg);
	}

versus

	if (msg.from() contains "KnownSpammer") {
		callMsgBlockingService(msg);
	}

both rules implement a portion of spam handling algorithm. Both are
equivalent from P Core point of view, IMO.

> Please let's be careful where to draw the line. My view is that we
> are *not* chartered to specify a general purpose
> processing/programming language, but a language for specifying rules
> that are used to described when certain actions are invoked (as
> opposed to encoding the action itself inside the rule)

I agree. The action itself is hidden in a P call, which can be a
callout service with a P interface or an internal service with a P
interface. The latter is particularly indistinguishable from anything
else a P module provides.

> - the rules language has a very limited scope, which might be blurry
> and not always clear, but we should be sensitive about how far we
> want to go with the rules language.

The scope is indeed one of the most difficult/important questions
here. It may seem simple on the surface (as usual), but scope effects
become crucial once we start to dig deeper. We have already seen that
with OPES Communications, OCP Core, and HTTP Adaptations drafts!

In fact, my current inability to estimate P draft completion date is a
result about poorly defined scope. With the renewed interest in P, and
several "grand ideas" related to P, I am no longer certain that
current P draft has the right scope. Hence, I do not know the amount
of work that remains to be done.

Alex.

P.S.

> For my taste, for example, IRML had all the functionality required.

IRML had _more_ functionality than we required, IMO.

> We decided to drop IRML in favor of "P" not because of limited
> functionality, but because of syntax preferences.

Here I violently disagree, but it is not important for now :-).

---

From: Geetha Manjunath <[email-masked]>
Date: Wed, 10 Dec 2003 10:25:58 +0530

> implement spam detection.

While there does exist a 'thin line' between spam detection using just
rules (if sender contains blah) and invoking a complete spam detection
service, I was definitely NOT assuming that P does the complete spam
detection. However the following, I believe, brings in the need for P to
be able to have a handle to the request/response:
* The fact that we are talking about mandatory support for a special
Services module that has methods for executing OPES services (applyOne),
binds us to provide more details about the interfaces expected from this
Services module - one side to the interpreter, the other side being the
OPES service. 

   4. Interpreters MUST supply two modules named Core and
   Services. The Core module contains members for manipulating built-in
   P object types such as integers and strings. The Services module
   manages OPES services.
   5. Services module contains basic attributes and methods for
searching
   and executing OPES services:

*  The example below makes some assumptions on the interfaces that the
service itself needs to provide.
	service := Services.findOne("opes://svs/tran/german/french");
        service.toDialect("southern");
        Services.applyOne(service, Http.request.headers);

* Putting the two together and extrapolating, we see the need for a call
of the form:
	Services.applyOne(spamSevice, SMTP.body)
Thus my comment ... 
 
> to be build in, we should very carefully discuss this.

I agree, that IRML did have most of the things needed from a HTTP
perspective - with one of the main limitation (from usage perspective)
being no support for expressions. It was also extremely difficult to
express complex conditionals  (AND, OR , NOT and others) - ofcourse, you
could classify that to be a syntax issue... Due to the simplicity of
IRML, we probably could even claim it to be applicable for other
protocols ..

Now, if we want to limit the scope of 'P' to just a re-syntaxing of IRML
and making the rule language protocol agnostic, that is a decision of
scoping again, I guess.... Not sure how many would want it that way..
Not me ;-)

---

From: Markus Hofmann <[email-masked]>
Date: Thu, 11 Dec 2003 09:42:03 -0500

> 	Services.applyOne(spamSevice, SMTP.body)

Hm, I've to admit that I'm not sure I fully understood your 
explanations. However, in above statement, it seems that all you do is 
executing a service on the message body - that's different from 
defining a rule basd on the content of the message payload.

I simply have serious doubts that it makes sense to define a rule like 
"if message body contains the string 'blablabla', then do...". Imagine 
you download a 4Gb movie and you'd have to scan the entire payload on 
the OPES processor, just to determine whether a service has to be 
executed... As such, I'm not (yet?) convinced that such 
flexibility/complexity is needed.

> scoping again, I guess.... Not sure how many would want it that way..

Our current charter/timeline does not allow for an extension of scope. 
If this is desired, it should be discussed as potential item for 
re-charter, which would also provide us realistic timeline for getting 
a good job done.

For now, my view is that 'P' has to be finished within the scope 
outlined in earlier discussions around IRML - this seems like a 
realistic goal in the short time we still have. Scope extensions, 
interfaces, etc. should be (and already have been) proposed as 
possible items for re-charter.

-Markus

---

From: Markus Hofmann <[email-masked]>
Date: Thu, 11 Dec 2003 09:48:54 -0500

What about option 4:

  - Stay within limited scope (use IRML as *guidance*)
  - Dont't worry about interfaces
  - Specify HTTP rules profile
  - Do all this by January
  - Discuss further extensions/modifications in potential re-charter.

-Markus

---

From: Alex Rousskov <[email-masked]>
Date: Thu, 11 Dec 2003 09:06:40 -0700 (MST)

>   - Discuss further extensions/modifications in potential re-charter.

I do not think we can produce a _quality_ HTTP rules profile by
January. We can produce a draft, but it will not be ready for an IESG
review and PS status, IMO.

We can try to do the rest this year, and submit the draft to IESG,
but, IMO, we are guaranteed to run into serious Core problems once we
start doing "interfaces" work. We would want to make changes to P
Core, and it would be difficult to do that given current IETF
hit-and-run approach to proposed standards.

	N.B. The above concern is the primary reasons we are doing OCP
	Core and HTTP Adaptation drafts in parallel. And, ideally, we
	should also be doing RTSP or at least SMTP/IMAP Adaptation
	drafts...

HTTP (and probably many more IETF specs with a similar scope) had the
same problem and it resulted in ugly real-world consequences. There is
RFC 2068 defining HTTP/1.1 and RFC 2616 defining ... HTTP/1.1. We have
to deal with old RFC2068 implementations that are not fully upward
compatible with RFC2616 implementations but that cannot be told apart.

If we are to do P Core now, without interfaces, we have to keep P Core
on the charter to be able to revise the proposed standard once we have
interface and implementation experience. Would that be acceptable to
the WG?  IESG? As you know, we currently say that we are not going to
touch, say, OPES architecture document because it has been approved as
an RFC (even though we _now_ know of many problems with that
document). We need to avoid this "don't touch it, it is off our hands"
approach with P Core if we are to postpone interface work. Can we
avoid it?

Thanks,

Alex.

---

From: Marshall Rose <[email-masked]>
Date: Thu, 11 Dec 2003 20:50:22 -0800

> review and PS status, IMO.

well, then we should just close up shop then.

charters and milestones exist for a lot of reasons. one reason is to focus the
wg to produce something tractable within finite time. there is a limit as to how
much latitude the iesg is willing to give a wg that overshoots its milestones.

so, we either start cutting stuff out to meet that deadline (which is still
slippage from the iesg perspective), or we declare victory and stop working...

/mtr

---

From: Alex Rousskov <[email-masked]>
Date: Thu, 11 Dec 2003 22:20:25 -0700 (MST)

On Thu, 11 Dec 2003, Marshall Rose wrote:

> charters and milestones exist for a lot of reasons. one reason is to
> focus the wg to produce something tractable within finite time.
> there is a limit as to how much latitude the iesg is willing to give
> a wg that overshoots its milestones.

Agreed.

I suspect that limit is decided on a case-by-case basis, and I do not
know whether our WG has exceeded that limit. Given the number of
drafts that we are now producing close to deadlines, the current P
momentum, the number of WGs seriously behind on their deadlines, and
the amount of resources IESG has recently spent on OPES WG, I would be
surprised if IESG is going to block a deadline change for 1 out of
10(?) WG drafts.

I have heard that deadline extension does not require IESG review or
approval. Is that true? Are we talking about potential problems [just]
with our AD?

> so, we either start cutting stuff out to meet that deadline (which
> is still slippage from the iesg perspective), or we declare victory
> and stop working...

I have no objections to polishing the current P draft and submitting
it now, before doing interface work, provided we are allowed to keep P
Core work in the charter in case we have to go back and change things
based on interface/implementation experience. Can we agree on that?
Will IESG have problems with that approach?

Thanks,

Alex.

---

From: Marshall Rose <[email-masked]>
Date: Fri, 12 Dec 2003 15:30:55 -0800

> approval. Is that true? Are we talking about potential problems [just]
    
fundamentally, the history of the working group is not a good one. 3
BOFs, several missed milestones, etc.
    
on the plus side, some I-Ds did get to the IESG and the WG has been
relatively responsive in addressing the IESG's concerns.
    
however, it's out about "our AD". any AD has to ask themselves whether
it is worthwhile to keep extending the life of a WG that hasn't
published any RFCs but does want to keep expanding its scope.
    
working currency comes from producing credible drafts on schedule. we
aren't broke in that department, but we're hardly rich
either. accordingly, i personally would not want to ask the AD for an
extension, because i might not like the answer that comes back. markus
may have a different perspective, naturally.
    
the moral, of course, is that the WG should focus on meeting the
committments it already agreed to. everytime someone asks for something
more, we should ask whether that will be the one thing that will put us
over the line and on the chopping block. the way to avoid that, of
course, is to delay non-essential items to future work...
    
/mtr

---

From: Alex Rousskov <[email-masked]>
Date: Fri, 12 Dec 2003 17:26:38 -0700 (MST)

On Fri, 12 Dec 2003, Marshall Rose wrote:

> any AD has to ask themselves whether it is worthwhile to keep
> extending the life of a WG that hasn't published any RFCs but does
> want to keep expanding its scope.

IESG ID tracker[1] paints a rosier picture than you seem to imply:
OPES WG has one draft that has been approved for publication as an RFC
and three drafts that are already in the RFC Editor Queue. Thus, we
have a total of four drafts that are, essentially, RFCs (their
publication is a matter of routine delays outside of WG or IESG
control).

We have one draft that requires a revision. If you need any help in
that area, please let me know.

Thus, out of 5 IESG-reviewed drafts, we have 4 approved ones! This
sounds like pretty good statistics to me, but I may be missing some
information that ID tracker does not relay well (e.g., perhaps IESG
spent a lot of resources getting the first 4 drafts approved?).

> working currency comes from producing credible drafts on schedule.
> we aren't broke in that department, but we're hardly rich either.
> accordingly, i personally would not want to ask the AD for an
> extension, because i might not like the answer that comes back.
> markus may have a different perspective, naturally.

It seems to me that once we submit all remaining drafts except for P,
the working group output would look relatively good:

	- 10 drafts total
	- 4 drafts approved to become RFCs
	- 4 drafts submitted for publication
	- 1 draft requires a revision
	- 1 draft needs an extension

We should be able to submit Communications draft now, OCP Core draft
next week, and HTTP Adaptations draft in a week or two.

Given the above stats, do you think our AD is likely to object to the
working group revising the overall strategy for the P draft and
rules-related work?

> committments it already agreed to. everytime someone asks for
> something more, we should ask whether that will be the one thing
> that will put us over the line and on the chopping block. the way to
> avoid that, of course, is to delay non-essential items to future
> work...

I agree and am not asking for more at this time; I am asking for less.

IMO, the rules draft is the one that should be delayed/moved to the
next charter. While you may argue that the draft is essential since it
is in the current charter, I will argue that it is impossible to
produce a meaningful, complete rules draft given the _current_
charter. The charter simply lacks other essential drafts/workitems
required to make a _quality_ rules draft possible. The latter was not
clear at the time the charter was written, of course.

IMO, we should fix the true problem by moving P work to the next
charter (where those "other essential workitems" will be included),
instead of spending (wasting) time on producing a draft that would
require serious revisions later. It is the Right Thing to do, IMO.

If both Chairs agree in principle, perhaps we can think of the best
way to describe our extension and rechartering plans so that our AD is
happy with them. If both Chairs disagree, I will finish a minimalistic
P draft and would ask that we at least keep P Core work in the next
charter.

Thanks,

Alex.

[1] https://datatracker.ietf.org/public/pidtracker.cgi