Subject: SMTP filtering use case

From: "Martin Stecher" <[email-masked]>
Date: Mon, 17 Feb 2003 10:44:32 +0100

Hi Hilarie and Alex,

I think that we are not talking about the same SMTP filtering scenario.
Let me build on the virus filtering example and describe a concrete use case:

There is a SMTP MTA dealing as the incoming mail server of a company.
It is OPES enabled and uses a callout service to do virus filtering on
all the incoming messages.
Every individual in the company can select his/her preference:
Infected messages can either be replaced by an alert message or can be
repaired.
This property is kept in a directory service for the company.
The OPES SMTP device is a general implementation which is not aware of
virus scanning techniques and especially not on the user preferences,
it does not access the directory service at all, but the callout
service does.

Now the OPES device receives an email messsage containing a virus which
is sent to 20 recipients within the company. It is a single message!

The OPES client forwards this one message to the callout service.
That one looks up the AV filter preference in the company's directory
service. The callout service detects the virus and sees that 18
recipients want to have the message repaired and 2 recipients prefer
to get the virus replaced by an alert.

From this point on two copies of the SMTP message are required with
different content. One to be sent to 18 recipients the other to the
other 2.

How could this be solved?
- By enabling the OPES protocol to return two copies of an application
  message in response to one filtering request
- By requiring that the OPES edge device ensures to pre-generate copies
  of the application message where each copy guarantees to result in
  only a single response.

Martin

---

From: Alex Rousskov <[email-masked]>
Date: Mon, 17 Feb 2003 08:44:17 -0700 (MST)

Martin,

	We are on the same page, I think. I am not sure a single SMTP
message can have multiple true recipients because SMTP clients I have
seen would usually send multiple copies, one to each To:/Cc: address.
(SMTP gurus please correct me). Fan out is possible when recipient is
an alias ([email-masked]) that SMTP server expands into
unfiltered [email-masked] and filtered [email-masked]. As far as I
can see, that expansion can happen at the OPES client or at the OPES
server (in theory), but the difference is minor.

Obviously, this optimization feature complicates the protocol. Special
care needs to be taken to insure that all expanded recipients get the
message even if, for example, the OPES server goes down in the middle
of the processing, after sending the first (marked, unfiltered)
response to the first group of recipients. This complicates
client-server negotiations: they need to agree on how many responses
and to what recipients there will be so that OPES client can take over
if OPES server fails (and save the messages to yet unprocessed
recipients, for example).

Do you think the optimization is worth the complexity? Can you think
of other, non-SMTP related uses where multiple responses to a single
OPES request would be useful?

N.B. A third way to implement what you want is to allow multiple
"recipient" addresses in an OPES request to mean that there are
actually multiple requests, _without_ duplicating the request on the
wire. Same for responses. This approach could be called a "lazy copy"
optimization. The difference is subtle, but it causes fewer protocol
complications than allowing (at the protocol level) multiple responses
to a single request.

Thank you,

Alex.

On Mon, 17 Feb 2003, Martin Stecher wrote:

>
> Hi Hilarie and Alex,
>
> I think that we are not talking about the same SMTP filtering
> scenario. Let me build on the virus filtering example and describe a
> concrete use case:
>
> There is a SMTP MTA dealing as the incoming mail server of a
> company. It is OPES enabled and uses a callout service to do virus
> filtering on all the incoming messages. Every individual in the
> company can select his/her preference: Infected messages can either
> be replaced by an alert message or can be repaired. This property is
> kept in a directory service for the company. The OPES SMTP device is
> a general implementation which is not aware of virus scanning
> techniques and especially not on the user preferences, it does not
> access the directory service at all, but the callout service does.
>
> Now the OPES device receives an email messsage containing a virus
> which is sent to 20 recipients within the company. It is a single
> message!
>
> That one looks up the AV filter preference in the company's
> directory service. The callout service detects the virus and sees
> that 18 recipients want to have the message repaired and 2
> recipients prefer to get the virus replaced by an alert.
>

---

From: "Oskar Batuner" <[email-masked]>
Date: Mon, 17 Feb 2003 13:46:07 -0500

Martin,

I believe OPES protocol should work in this kind of scenario by 
applying it's own transaction boundaries. Being application 
agnostic callout protocol does not provide one-to-one mapping 
between callout protocol messages and application messages.  
 
In this scenario single OPES transaction may encapsulate 
multiple response message instances generated by the server. 
OPES client by its nature is application specific, so 
in this case it should expect that a single application message 
included in OPES request may result in multiple application 
response messages included into the OPES response, together 
with - again, application-specific - information how to use 
each application-level response message. 

Keeping OPES transaction context independent from the 
application semantics helps to ensure service 
reliability: OPES client can detect situation when the 
request processing was not completed and take the appropriate 
steps.

- Oskar

> -----Original Message-----
> From: [email-masked]
> [mailto:[email-masked]]On Behalf Of Martin Stecher
> Sent: Monday, February 17, 2003 4:45 AM
> To: OPES WG (E-mail)
> Subject: SMTP filtering use case
> 
> 
> 
> Hi Hilarie and Alex,
>

---

From: "The Purple Streak, Hilarie Orman" <[email-masked]>
Date: Mon, 17 Feb 2003 12:26:08 -0700

Your filtering example is based on delivery to an SMTP server that
knows that the recipient of the single message is actually an alias.
The OPES service must know the expansion of the alias to the 20
recipients (any one of which could actually be an alias for another
list).  This means that the SMTP service and the OPES service are very
tightly coupled.  As I said, I'd always assumed that the OPES processor
would have the user preferences, and that's the whole reason for the
rules, etc., and for having the OPES processor be the enforcement point
for privacy policy.  But, I guess that assumption is open to debate.

But there's another reason that I'm still not convinced that this SMTP
example applies to OPES.  If this service and all the user preferences
are handled by a callout server, there's no reason that the callout
server cannot just forward the transformed SMTP message to a real SMTP
server for final delivery.  Because SMTP is a store-and-forward
service, not end-to-end, this works better than OPES would - there's
less I/O involved.

Hilarie