Subject: Fwd: Re: Last Call: 'Message Submission' to Draft Standard

From: "JFC (Jefsey) Morfin" <[email-masked]>
Date: Wed, 16 Feb 2005 12:22:20 +0100

I have not the competence to fully understand if this concerns OPES/SMTP, 
but I think it does? John quotes OPES in the discussion.

All this, together with the current IDN security debate, etc. lead me to 
think that my interest in DN massaging is a key OPES issue and that the 
OPES/SMTP proper location is the UA "virtual" I/O, that is at the first 
(front-end) and at the last (back-end) MTA only, what simplifies our debate 
and makes it of immediate interest?
jfc

>From: John C Klensin <[email-masked]>
>To: Randall Gellens <[email-masked]>, [email-masked]
>Bruce,
>
>Let me make a few observations, largely, in support of my
>esteemed co-author...
>
>--On Tuesday, 15 February, 2005 15:28 -0800 Randall Gellens
><[email-masked]> wrote:
>
> >>>  >  Specifically regarding the 4.3 MUST quoted above and the
> >>>  >  reply code, and the necessary two independent
> >>>  >  implementations required for advancement to Draft
> >>>  >  Standard status, do the implementations supporting the
> >>>  >  request to advance to Draft in fact unconditionally
> >>>  >  require authorization (i.e. independent of whether port
> >>>  >  25 or 587 is used, and regardless of administrative
> >>>  >  configuration other than specifying that the
> >>>  >  implementation is to act as an MSA), and reply with code
> >>>  >  530 (unspecified extended response code) if
> >>>  >  authorization is lacking?
> >>>
> >>>  Both the draft and RFC 2476 allow submission to use 25
> >>>  instead of 587, but state that "normally" 587 is used, and
> >>>  allow 25 to be used in order to accommodate implementations
> >>>  that are hard to configure. Both say that port 587 is
> >>>  reserved for submission.  So, 587 is the normal case for
> >>>  submission, and 25 is an exceptional case.
> >>
> >>  But as written, section 4.3 specifies that when acting as an
> >>  MSA, authentication MUST (emphasis in original) be used,
> >>  regardless of port.
> >
> > Section 4.3 says "unless it has already independently
> > established authentication or authorization (such as being
> > within a protected subnetwork)."  It's not uncommon for
> > organizations to deploy MSAs that treat all internal
> > connections as authenticated, even if they allow travellers to
> > connect in using port 587 and in that case require
> > authentication.
>
>Bruce, if I understand your concern (I may not), let's back up a
>bit and look at a bit of history and the usage case.  The notion
>of a separate submission protocol arose in part out of a problem
>that became clear when we were doing 2821.  There were a whole
>series of things that we would really prefer that MTAs not do
>--things that violated the "don't mess with the message"
>meta-principle for MTAs but were necessary because clients
>weren't able to supply all of the information needed for a valid
>2822/822 message, or weren't able to supply it in a form that
>would be security-acceptable.  So, to face the reality of the
>fact that these MTA-fixes were being applied (and were
>necessary), 2821 contains a loophole for submission servers to
>fix messages up.
>
>That loophole is, however, somewhat bad news, since it doesn't
>involve explicit client authorization for message-tampering and
>there is no real way to permit such authorization even if one
>wanted to do it (similarities to some of the OPES issues here
>are not coincidental).  "Message Submission" ("Submit" below) is
>obviously different: we allowed for extensions to be used with
>Submit that are not specified for SMTP (even though no one has
>done that -- the mechanism has been shown to work with
>extensions that are also specified for SMTP) and the mere fact
>of using the Submit protocol says "this is a submission agent,
>not a general MTA, and hence gets to use those features".
>
>Now the intent is that Submit be used on its own port, for all
>of the obvious reasons.  But (i) nothing really prevents it from
>being run on _any_ port; such is the nature of IETF applications
>protocols and (ii) in particular, it can, explicitly, be run on
>Port 25.   But, if it is run on port 25, the question
>immediately arises as to how you tell it from SMTP, since the
>two are deliberately quite similar in their handshakes.   The
>answer is "administrative or policy decision, reflected in
>configurations".  No other answer really makes sense.  And, if
>you get that far, then you need to ask the question of whether
>one would want to permit a host whose port 25 is being used to
>support Submit to also support SMTP.   And the answer is that
>the question is meaningless if you are not either using Submit
>extensions that are not defined for SMTP (or vice versa) and a
>really dumb idea if you are and feel like mixing the two and
>distinguishing heuristically, halfway through the protocol
>transaction.  Such a concurrent implementation on the same port
>would actually probably violate both specs, since it would have
>to advertise extensions that were not defined for the protocol
>that the client was expecting (not that doing so would make a
>lot of difference).
>
>So...
>
> >>  Again, "or not" is not allowed by the draft as written;
> >>  implementations with an "or not" provision would fail to
> >>  comply with the "MUST" (they would comply with a "SHOULD",
> >>  however).
> >
> > My understanding is that implementations are permitted to have
> > configuration options that allow sites to deploy them in ways
> > that violate the RFCs, as long as they can be operated in a
> > conforming manner.  For example, it is not uncommon for POP
> > implementations to have options to have timeouts shorter than
> > mandated, or to enter UPDATE state on abort.
>
>This has always been the rule, at least since we decided that
>specifications, to advance to draft, must exhibit implementation
>of all of the features.  We don't do code review to verify
>conformance and spot possibly-non-conforming capabilities.  The
>rule is that the implementations must be able to demonstrate
>that they can be configured in a way that conforms to the spec.
>Nothing else counts as it is, by definition, outside the
>specification.
>
> >...
> > From my understanding of the requirements, there are multiple
> > implementations which comply.  This is based on two points:
> > (1) implementations can be compliant even if they have options
> > which permit non-conformance; and (2) there must be at least
> > two implementations for each feature, but it is not required
> > that any individual implementation meet every SHOULD.  I
> > believe Message Submission meets the criteria on the basis of
> > either of these points; the fact that it meets it on both is
> > nice but not required.
>
>Under other circumstances, I might actually quibble about (2),
>but (1) is the important point.
>
> >>>  >  On another matter, admittedly unchanged since RFC 2476,
> >>>  >  there seem to be some undesirable discrepancies between
> >>>  >  submission and non-submission ESMTP regarding extended
> >>>  >  response codes.  Draft section 3.4 states that extended
> >>>  >  status code 5.6.2 means "Bad domain or address", whereas
> >>>  >  RFC 3463 assigns that code the semantics "Conversion
> >>>  >  required and prohibited" [RFC 3463 section 3.7].  The
> >>   > >  corresponding RFC 3463 extended response code for
> >>>  >  domain/address issues would be in the 5.1.XXX range [RFC
> >>>  >...
>
>To the extent to which this is an issue, it is an issue with RFC
>3463 and not with this draft.  A real problem would arise only
>in the "both protocols, same machine, port 25" case described
>above.  But I agree with Randy -- the right action is to delete
>the specific codes from this specification and leave that to
>1893, 3463, and their successors.
>
> >>>  >  the draft should explicitly say so (the term "gateway"
> >>>  >  does not appear anywhere in the draft). [that would be a
> >>>  >  novel use of the term "gateway" in Internet mail; a
> >>>  >  gateway usually has one side in a non-SMTP environment]
>
>Actually, we discovered in working on RFC 2821 that life wasn't
>that easy.  First, it has never been the definition, "usually"
>or otherwise:  at the time 821 was written, the concept of a
>mail gateway applied when either one side was in a non-SMTP
>environment or when one side was in a non-TCP environment,
>whether SMTP was being used or not (or, indeed, when the two
>sides didn't share underlying transports, whether either one was
>TCP or not).
>While I would prefer that "one side in non-SMTP or different
>transport" definition, one runs into real-world problems when
>the addressing structures differ, whether SMTP and TCP are on
>both sides or not.  For example, assume that one has a
>submission client on a private network using 1918 space.  It
>submits the message to an MSA which lies at the NAT boundary
>between that network and the public Internet.  To meet the
>requirements of 2821, the MSA (or submission MTA) should modify
>or supplement addresses and internal names so that they reflect
>public names and addresses.  That moves it into gateway
>territory, and 2821 was written to permit that case (doesn't
>make that I, or the DRUMS WG, liked the case very much, but it
>reflects reality).
>
> >>>  >  Conversely, if MSAs are not always to be considered as
> >>>  >  gateways, then returning errors in response to message
> >>>  >  content is: 1. explicitly counter to the SHOULD NOT of
> >>> RFC 2821 section 3.4 (bottom of
> >>>  >     page 18)
> >>>  >  2. inappropriately associated with "conversion"
> >>>  >  semantics where no conversion is in fact required
> >>>  >     (indeed,
> >>> other than adding trace fields,
> >>>  >     tinkering with message data by
> >>> non-gateway SMTP receivers is disallowed
> >>>  >     [RFC 2821 section 2.3.8]).
> >>>
> >>>  It is not the intent of either this draft or RFC 2476 to
> >>>  say that MSAs are always gateways; rather, the intent in
> >>>  both is to recognize the reality that organizations
> >>>  sometimes see a need to examine and potentially modify
> >>>  messages submitted using their servers, and to make a clear
> >>>  distinction between this case, and the
> >>>  examination/modification of messages being relayed.
>
>Bruce, you are assuming, I think, that every MSA must be
>conformant to the MTA requirements of 2821 (gateway or not).
>That was never intended to be the case.  Indeed, that is part of
>the reason why this is a separate protocol (i.e., the "clear
>distinction" to which Randy refers to above).

---

From: Tony Finch <[email-masked]>
Date: Wed, 16 Feb 2005 13:44:45 +0000

On Wed, 16 Feb 2005, JFC (Jefsey) Morfin wrote:
>
> I have not the competence to fully understand if this concerns OPES/SMTP, but
> I think it does? John quotes OPES in the discussion.

Only for comparison, since both message submission and OPES may involve
message alteration. The main area of discussion was the introduction of a
distinction between pure SMTP (which in principle should be a clear
channel, but historically has not been) and message submission (which
involves a certain amount of limited message clean-up to enforce
standards compliance). OPES is orthogonal to this.

> All this, together with the current IDN security debate, etc. lead me to think
> that my interest in DN massaging is a key OPES issue and that the OPES/SMTP
> proper location is the UA "virtual" I/O, that is at the first (front-end) and
> at the last (back-end) MTA only, what simplifies our debate and makes it of
> immediate interest?

A lot of current SMTP filtering deployments occur at a site's border MTAs
or central relays, which are often a hop or two from any UA. It would be
foolish to limit OPES/SMTP to the submission or final delivery stages.

Tony.
-- 
f.a.n.finch  <[email-masked]>  http://dotat.at/
NORTH FORELAND TO SELSEY BILL: NORTHEAST 4 OR 5 DECREASING 3, BECOMING
VARIABLE 3 OR LESS LATER WEATHER: SHOWERS AT FIRST. MAINLY GOOD. ROUGH
BECOMING MODERATE IN EAST; MODERATE BECOMING SLIGHT IN WEST.

---

From: jfcm <[email-masked]>
Date: Wed, 16 Feb 2005 16:27:45 +0100

>standards compliance). OPES is orthogonal to this.

Dear Tony,
Again I am not an SMTP specialist and I don't want to be one (for the good 
reasons you give). But I think I know a litte about about messaging and I 
am interested in end to end results. I also know that whenever there is a 
fix, someone needs to have her special way to tune/control/implement the 
fix. This is the role of the OPES.

>or central relays, which are often a hop or two from any UA.

Absolutely true. But you have to be clear about where the OPES process is 
introduced in all that. And how many times. With which tracability tree and 
domain definition.

>It would be foolish to limit OPES/SMTP to the submission or final delivery 
>stages.

I am afraid, from the lack of debate/work on this list, we lost focus and 
we do not know what we are supposed to talk about. OPES are related to 
"edge". I am talking about edge. I opposed the charter for that reason - I 
think OPES can only be related to UA. Once it was approved I welcame the 
work on something called "OPES/SMTP" because it is an _other_ issue than 
OPES buut of real interest to me. Now I see that proper OPES thinking, the 
way it was developped for http, does not fit with the SMTP/MTA architecture.

So, I propose to unlock our own situation in saying: let work on UA 
interfacing at MTA level since OPES are proxy functions and we said the IAB 
we wanted to work at MTA level. This is probably a clear thing this group 
can work on (not heard from Alex in nealry 3 months)? I would see a real 
deployment interest in this (in particular for considering an SMTP 
submission/delivery solution - staying compatible with current user agents 
- but not using SMTP for transport).

jfc