Subject: Re: P work in new charter

From: "The Purple Streak, Hilarie Orman" <[email-masked]>
Date: Wed, 30 Jun 2004 22:27:42 -0600

>  The whole reason we want to have callout servers is to move services 
>  out to a separate server. Are we now trying to move some of the 
>  processing complexity back into the OPES processor in form of a highly 
>  programmable, compelx rules language that allows "programming" of actions?

>  -Markus

Yes, I've always believed that some actions should be done on the OPES
processor because of the efficiency of the data locality.

Hilarie

---

From: Alex Rousskov <[email-masked]>
Date: Wed, 30 Jun 2004 23:07:20 -0600 (MDT)

On Wed, 30 Jun 2004, The Purple Streak, Hilarie Orman wrote:

> Yes, I've always believed that some actions should be done on the
> OPES processor because of the efficiency of the data locality.

I agree that there are use cases where services implemented internally
by the OPES processor make a lot of sense, at least for performance
reasons. The trivial example is a "drop this message" service.

Fortunately, as far as P language is concerned, there is no
significant difference that would affect our charter:

	if (message meets criteria)
	    services.drop(message); // call "global/remote" service

versus

	if (message meets criteria)
	    processor.services.drop(message); // call embedded service

We should probably discuss this later, but it may be possible for OPES
processor to register its own service in the global services array so
that the user will always write code in the first example and get
efficient execution if the processor has an embedded drop service.

Alex.

---

From: Geetha Manjunath <[email-masked]>
Date: Thu, 01 Jul 2004 13:35:03 +0530

> Markus Hofmann wrote:

> > * Writing simple filters/actions: While modifying a message to be able to ....

> If a function like "adding of authentication controls" is needed, a
> little service doing this can be written, and a rule can be specified
> to invoke this service.

I beleive and agree with others on the need for  local services for peformance
reasons. If the mechanism is an indirect way of registering local services that is
fine - but then we should clearly specify means of interfacing external modules to
P. [ does that add to the scope of P? ]

> > (d) defining mechanisms by which a user can communicate rulesets to the OPES
> > processor.
>
> Such mechanism is needed (one might default to existing ones), but out
> of scope of the WG.

Can you please give me specific pointers to existing mechanisms here? I would like
to evaluate their sufficiency  for the intended usage.

Thanks and regards
Geetha

---

From: Markus Hofmann <[email-masked]>
Date: Thu, 01 Jul 2004 09:02:47 -0400

> processor because of the efficiency of the data locality.

These should be *services* that are called locally, and *not* some 
coded embedded in the rules language.

-Markus

---

From: Markus Hofmann <[email-masked]>
Date: Thu, 01 Jul 2004 09:06:15 -0400

> P. [ does that add to the scope of P? ]

All you need is calling a local service from within "P" - just as you 
would call a remotes service, it's just running on the local machine. 
Alex elaborated in this in his previous email.

> to evaluate their sufficiency  for the intended usage.

For example, use a CLI to load a rules file locally on the machine, 
use FTP to transfer a rules file, you might use HTTP....

-Markus

---

From: Alex Rousskov <[email-masked]>
Date: Thu, 1 Jul 2004 09:28:02 -0600 (MDT)

On Thu, 1 Jul 2004, Markus Hofmann wrote:

> Geetha Manjunath wrote:
>
> > I beleive and agree with others on the need for local services for
> > peformance reasons. If the mechanism is an indirect way of
> > registering local services that is fine - but then we should
> > clearly specify means of interfacing external modules to P. [ does
> > that add to the scope of P? ]
>
> All you need is calling a local service from within "P" - just as
> you would call a remotes service, it's just running on the local
> machine.  Alex elaborated in this in his previous email.

I suspect Geetha is talking about a standard mechanism/interface to
register a local service (always implemented in something other than
P) with P interpreter so that it becomes accessible from P rules. To
do that, we need to standardize service interface. I do not think we
are ready for that now, but it could be our future work.

For example, if we document how an OCP/HTTP service can be contacted
based on P "service call" code, then any OCP/HTTP service (local or
remote) can be registered with the P interpreter and used from P
rules. This would be a P-to-OCP mapping of some sort.

Similarly, if we document how a Web Service can be contacted based on
P "service call" code, then any Web Service (local or remote) can be
registered with the P interpreter and used from P rules. This would be
a P-to-WSDL mapping of some sort.

I think it is important to keep these future integration tasks in
mind, but I wonder if the first stable version of P spec should be
done before we start documenting the above interfaces? The risk is
that first P implementors will have to come up with some glue of their
own, and that glue would be difficult to standardize post-factum. On
the other hand, I doubt we have enough cycles to document these
interfaces now, as a WG activity.

Alex.

---

From: Geetha Manjunath <[email-masked]>
Date: Fri, 02 Jul 2004 17:08:42 +0530

Fair enough!

---

From: Geetha Manjunath <[email-masked]>
Date: Fri, 02 Jul 2004 17:07:51 +0530

> use FTP to transfer a rules file, you might use HTTP....

Oh! I guess these are fine! I was wondering whether there were any protocols along the
lines of WPAD (Web Proxy Auto-Discovery Protocol) for this.

The reason I raised the point about "mechanisms to communicate the rules" was based on
my trials of using IRML in a practical OPES env. Though IRML did talk about rule sets
that included 3 classes of rules - (i)Rules set by OPES administrator, (ii)Rules set by
Content Provider and (iii) Rules set by the user/client  - there were several practical
problems  in effecting the latter two cases.
I guess the question then boils down to  "who are the targetted authors that would use P
language to write rulesets?".
Clearly it is (i) but do we include (ii) and (iii) ?

Thanks and regards
geetha

---

From: Alex Rousskov <[email-masked]>
Date: Fri, 2 Jul 2004 13:00:11 -0600 (MDT)

On Fri, 2 Jul 2004, Geetha Manjunath wrote:

> ... 3 classes of rules -
> (i)   Rules set by OPES administrator,
> (ii)  Rules set by Content Provider and
> (iii) Rules set by the user/client
> ...
> I guess the question then boils down to "who are the
> targetted authors that would use P language to write rulesets?".

Good question. I can speculate that class (ii) is as likely to be used
in practice, with time, as P itself. The "spaces" of intermediaries
and content providers seem to be moving onto each other. I am less
optimistic about (iii), but it is also possible in some automation
context (i.e., it would not be a human user composing P rules, but
a mobile phone, browser, etc., based on user/device preferences).

As far as WG charter is concerned, I think it all boils down to the
expertise within this WG. If we have an expert willing to defend
specific (ii) and (iii), we should listen and should try hard to avoid
making P difficult to use for those classes.

Thanks,

Alex.

---

From: jfcm <[email-masked]>
Date: Tue, 06 Jul 2004 15:32:36 +0200

The attached piece of real world's information seems to be worth 
consideration. This is legal, political and US centric. Nevertheless it 
means that SMTP (mail transfer) is not seen by many (legally and 
technically) as a stream but as fast store and forward and that different 
legal rules (and therefore applications/business/demands/offers may be 
conceived depending on where is the filter (on the protocol or on the node).

It seems to me this is another contradiction of OPES/ONES with the 
"protocol on the wire" and "dumb network/smart host" concepts. I am not an 
SMPT pro, but I suppose that the difference is that in HTTP forwards a flow 
of datagrams while SMTP stops+store+forwards a group of datagrams building 
an entire message. For example, you cannot know the true user's value of a 
mail before you got to the attachement or to the final signature.
jfc

--------------

 From the New York Times -- 
http://www.nytimes.com/2004/07/06/technology/06net.html

You've Got Mail (and Court Says Others Can Read It)
By SAUL HANSELL

When everything is working right, an e-mail message appears to zip 
instantaneously from the sender to the recipient's inbox. But in reality, 
most messages make several momentary stops as they are processed by various 
computers en route to their destination.

Those short stops may make no difference to the users, but they make an 
enormous difference to the privacy that e-mail is accorded under federal law.

Last week a federal appeals court in Boston ruled that federal wiretap laws 
do not apply to e-mail messages if they are stored, even for a millisecond, 
on the computers of the Internet providers that process them - meaning that 
it can be legal for the government or others to read such messages without 
a court order.

The ruling was a surprise to many people, because in 1986 Congress 
specifically amended the wiretap laws to incorporate new technologies like 
e-mail. Some argue that the ruling's implications could affect emerging 
applications like Internet-based phone calls and Gmail, Google's new e-mail 
service, which shows advertising based on the content of a subscriber's 
e-mail messages.

"The court has eviscerated the protections that Congress established back 
in the 1980's," said Marc Rotenberg, the executive director of the 
Electronic Privacy Information Center, a civil liberties group.

But other experts argue that the Boston case will have little practical 
effect. The outcry, said Stuart Baker, a privacy lawyer with Steptoe & 
Johnson in Washington, is "much ado about nothing."

Mr. Baker pointed out that even under the broadest interpretation of the 
law, Congress made it easier for prosecutors and lawyers in civil cases to 
read other people's e-mail messages than to listen to their phone calls. 
The wiretap law - which requires prosecutors to prove their need for a 
wiretap and forbids civil litigants from ever using them - applies to 
e-mail messages only when they are in transit.

But in a 1986 law, Congress created a second category, called stored 
communication, for messages that had been delivered to recipients' inboxes 
but not yet read. That law, the Stored Communications Act, grants 
significant protection to e-mail messages, but does not go as far as the 
wiretap law: it lets prosecutors have access to stored messages with a 
search warrant, while imposing stricter requirements on parties in civil suits.

Interestingly, messages that have been read but remain on the Internet 
provider's computer system have very little protection. Prosecutors can 
typically gain access to an opened e-mail message with a simple subpoena 
rather than a search warrant. Similarly, lawyers in civil cases, including 
divorces, can subpoena opened e-mail messages.

The case in Boston involved an online bookseller, now called Alibris. In 
1998, the company offered e-mail accounts to book dealers and, hoping to 
gain market advantage, secretly copied messages they received from 
Amazon.com. In 1999, Alibris and one employee pleaded guilty to criminal 
wiretapping charges.

But a supervisor, Bradford C. Councilman, fought the charges, saying he did 
not know about the scheme. He also moved to have the case dismissed on the 
ground that the wiretapping law did not apply. He argued that because the 
messages had been on the hard drive of Alibris's computer while they were 
being processed for delivery, they counted as stored communication. The 
wiretap law bans a company from monitoring the communications of its 
customers, except in a few cases. But it does not ban a company from 
reading customers' stored communications.

"Congress recognized that any time you store communication, there is an 
inherent loss of privacy," said Mr. Councilman's lawyer, Andrew Good of 
Good & Cormier in Boston.

In 2003, a federal district court in Boston agreed with Mr. Councilman's 
interpretation of the wiretap law and dismissed the case. Last week, the 
First Circuit Court of Appeals, in a 2-to-1 decision, affirmed that decision.

Because most major Internet providers have explicit policies against 
reading their customers' e-mail messages, the ruling would seem to have 
little effect on most people.

But this year Google is testing a service called Gmail, which 
electronically scans the content of the e-mail messages its customers 
receive and then displays related ads. Privacy groups have argued that the 
service is intrusive, and some have claimed it violates wiretap laws. The 
Councilman decision, if it stands, could undercut that argument.

Federal prosecutors, who often argue that wiretap restrictions do not apply 
in government investigations, were in the somewhat surprising position of 
arguing that those same laws should apply to Mr. Councilman's conduct. A 
spokesman for the United States attorney's office in Boston said the 
department had not decided whether to appeal.

Mr. Baker said that another federal appeals court ruling, in San Francisco, 
is already making it hard for prosecutors to retrieve e-mail that has been 
read and remains on an Internet provider's system.

In that case, Theofel v. Farey-Jones, a small Internet provider responded 
to a subpoena by giving a lawyer copies of 339 e-mail messages received by 
two of its customers.

The customers claimed the subpoena was so broad it violated the wiretap and 
stored communication laws. A district court agreed the subpoenas were too 
broad, but ruled they were within the law. The plaintiffs appealed, and the 
Justice Department filed a friend of the court brief arguing that the 
Stored Communications Act should not apply.

In February, the appeals court ruled that e-mail stored on the computer 
server of an Internet provider is indeed covered by the Stored 
Communications Act, even after it has been read. The court noted that the 
act refers both to messages before they are delivered and to backup copies 
kept by the Internet provider. "An obvious purpose for storing a message on 
an I.S.P.'s server after delivery," the court wrote, " is to provide a 
second copy of the message in the event that the user needs to download it 
again - if, for example, the message is accidentally erased from the user's 
own computer."

Calling e-mail "stored communication" does not necessarily reduce privacy 
protections for most e-mail users. While the Councilman ruling would limit 
the applicability of wiretap laws to e-mail, it appears to apply to a very 
small number of potential cases. The Theofel decision, by contrast, by 
defining more e-mail as "stored communications," is restricting access to 
e-mail in a wide range of cases in the Ninth Circuit, and could have a far 
greater effect on privacy if courts in the rest of the country follow that 
ruling.

---

From: Alex Rousskov <[email-masked]>
Date: Tue, 6 Jul 2004 10:15:39 -0600 (MDT)

jfc,

	IMHO, the stupidity of judges or laws should not limit OPES
scope. OPES framework (in a broad sense) is applicable to any
communication between content producer and content consumer. Whether
the unit of communication is "stored and forwarded" or "just
forwarded" is irrelevant from architecture/scope point of view. It may
affect how and when the adaptations are performed, but not whether
they can be performed in an OPES-compliant way.

	The old OPES architecture draft may not reflect this and other
scope concerns, but that's a different question/problem.

Alex.

---

From: "The Purple Streak, Hilarie Orman" <[email-masked]>
Date: Tue, 6 Jul 2004 10:28:46 -0600

The Boston ruling about email might apply to http caches
(non-transparent ones) and possibly OPES processors acting on behalf
of the recipient.  I don't think it has much to do with protocols, but
it has a lot to do with storing message copies.

The article implies that if the content is stored as a service to the
user, then it is not protected under wiretap law.  It sounds as though
the SMTP store-and-forward phase is still protected until it hits the
ISP that is acting on behalf of the recipient.  At that point any
message copies are assumed to be a storage service for the recipient.

But, I'm not a lawyer and have based my opinion entirely on the
article, and that article is written by someone who is neither a
lawyer nor an Internet expert.  

Hilarie

---

From: Ted Hardie <[email-masked]>
Date: Tue, 6 Jul 2004 10:00:19 -0700

To follow up a bit, the judges in the ruling made very clear
that they thought the law's original wording was in dire
need of an update to deal with modern realities.   On NANOG,
there has been discussion of how router caches might
be interpreted under this reading of the law, so this
problem is not particular to OPES.  There seems to be a
fairly general recognition that this was not the intended
scope of the law.

Efforts to help get those laws updated are not in scope
for this group, though, and I encourage those interested
to seek appropriate fora for the purpose.

			regards,
				Ted Hardie

---

From: jfcm <[email-masked]>
Date: Tue, 06 Jul 2004 20:13:38 +0200

>scope concerns, but that's a different question/problem.

hmmmm. I am afraid you missed my point. This is what one can name "digital 
decoherence" like for quantum physic. In the network information is under 
data state (bytes). Outside of the network (when you see it, use it) it is 
under object state (a mail, a graphic, etc.). What the Judge says is: data 
streams are protected by law, but that objects are subject to objects law. 
Like quanta are subject to quantum law and apples are subject to Newton law.

Decoherence is when an apple electron is both in quantum and newtonian 
state. Digital decoherence is the same when data information becomes object 
infomation - being both at the same time. This occurs at the proxy level. 
So one cas say that OPES philosopy is to take the information as data, to 
massage it as part of a possible object and tro give it back as data which 
will later resolve into another object (or several) than initially 
foreseen. This introduces an interesting case of the layer analysis of what 
I name the interapplication layer. But the point is not here?

But the point again is : you do not know what a mail is until you have 
completely stored it. Let say the signature is at the end. Only then you 
fully know what you want to do with the header. This builds a window of the 
size of the mail. So two possibilites :

- either you are not modifying something on the fly. You massage a stored 
file (and you introduce a delay in waiting to massage it to have received 
it - what I suppose SMTP does not do).

- or you have to find a way to race the network faster than SMTP to tell a 
filter which has not yet rceived the first datagram for the mail :"the 
datagram to come is part of a mail opf which the last datagram says that 
the first datagram should be processed".

I suppose there is a way to do that in doing both ? In using two filters ?
Filter A reads the mail until the end and blocks the last datagram if thre 
is something to do.
Filter B is then told what is to be done and starts processing datagram in 
seqence until it get one before the last. It signals A to release the last 
datgram and processes it on the fly. Quite complex ?
jfc

---

From: "The Purple Streak, Hilarie Orman" <[email-masked]>
Date: Tue, 6 Jul 2004 13:10:34 -0600

I think we all understand the difference between the stream of packets
comprising an application message and the message itself.  What we
don't understand is what, if anything, jfcm may be proposing for OPES.
I believe he is trying to find a way to use OPES to keep SMTP messages
a logical "streams", even when almost the entire message must be
available to an OPES service.  This does not seem to affect the OPES
charter or architecture.  It's an application of OPES, and an example
of how, given enough state, anything can be turned into a stream
processing model.

Hilarie

---

From: Markus Hofmann <[email-masked]>
Date: Tue, 06 Jul 2004 15:20:03 -0400

Folks,

I second Hilarie's comments in that this doesn't seem to affect the 
OPES charter or architecture. As such, this is not in scope for the 
group or this mailing list.

Although an important topic, I'd suggest to move discussions about 
legal aspects and law interpretation elsewhere.

-Markus

---

From: Alex Rousskov <[email-masked]>
Date: Tue, 6 Jul 2004 14:01:55 -0600 (MDT)

[legal- and quantum physics-related stuff snipped]

> But the point again is : you do not know what a mail is until you
> have completely stored it. Let say the signature is at the end. Only
> then you fully know what you want to do with the header.

The above "you do not know" assertion is false for some kinds of
e-mail transformations (e.g., detecting a known spam by looking at
Subject lines or headers) and true for some kinds of HTTP
transformations (e.g., detecting a virus by examining sensitive octets
at the end of an HTTP entity).

Overall, there is nothing special about SMTP here, IMO: some OPES
adaptations require the whole message, some do not. Sometimes OPES
processing overheads delay the entire message; sometimes they delay a
portion of the message.

Why is SMTP special at this level of thinking?

Alex.

---

From: Graham Klyne <[email-masked]>
Date: Wed, 07 Jul 2004 10:13:22 +0100

>Why is SMTP special at this level of thinking?

(Trying to answer this question, without making any statement about OPES 
applicabiluty...)

The delivery requirements associated with SMTP effectively require a mail 
relay to store an entire message until it has been successfully transferred 
to the next relay.  This is more than just "best effort" delivery -- I have 
seen the term "heroic effort" applied here.  A message is expected to 
survive a complete restart of an MTA system while an SMTP transaction is 
in-progress.

It's true that the spec doesn't actually say that messages must be stored, 
but that does seem to be a consequence of the responsibility that the spec 
places on MTAs.

In practice, I think this means that most real-world MTAs do store messages 
in a non-volatile message store, where they might arguably (I won't say 
"reasonably") be regarded as not in-transit.

#g

------------
Graham Klyne
For email:
http://www.ninebynine.org/#Contact

---

From: Alex Rousskov <[email-masked]>
Date: Wed, 7 Jul 2004 07:58:56 -0600 (MDT)

On Wed, 7 Jul 2004, Graham Klyne wrote:

> In practice, I think this means that most real-world MTAs do store
> messages in a non-volatile message store, where they might arguably
> (I won't say "reasonably") be regarded as not in-transit.

Agreed, but the question remains: Why is SMTP a special case as far as
OPES architecture is concerned? What difference does permanent or
non-volatile storage make?

I can still apply the same kinds of adaptations before or after the
message is permanently or temporary stored, right? I understand that
SMTP is store-and-forward, but I do not understand why this creates
problems for OPES architecture. OPES is simply a polite way to do
man-in-the-middle adaptations. As long as the application protocol
allows for intermediaries (of any sort), OPES should be applicable.

Alex.